Co.Lab

Lunch-time Entertainment: Robot Orchestra

This is cool - ping pong balls fire at a xylophone and and then roll back down into the firing mechanism.

Bot-nets and Spam

According to this cnet news report on sources of spam:

European spam networks have pumped out more unsolicited e-mail than those in the U.S. for the third month in a row...

Wider broadband penetration in Europe has lead to increases in gangs running bot-nets - networks of virus compromised computers. These bot-nets are used to send out spam from the compromised machines. One the largest of these bot-nets was created by the storm worm. It is in response to these bot-nets that some ISPs (Internet Service Providers) have shut down port 25, the standard port used for smtp. Which effectively cuts out part of their support for the Internet protocols (TCP/IP) they are supposed to provide.

Also on cnet, Russia has overtaken China as the largest malware distributor. Malware can includes virus' to create bot-nets and other malicious software, such as spyware designed to steal passwords or sensitive information.

Our spam protection system, Tonsho, effectively filters spam by authenticating that senders are real human users, not zombie machines sending out junk. Visit Tonsho.com to get as free trial.

Relocating a jsp application

This post is about allowing your jsp web application to be installed in different folders.  Lets say the web application is called myapp and the domain is www.acmewebapps.com. We could have different locations for the application:

1. The default folder: www.acmewebapps.com/myapp/index.jsp
2. The root folder: www.acmewebnapps.com/index.jsp
3. Custom folder: www.acmewebapps.com/myapp2/index.jsp

We can move the location of the web application by changing the "Context" entry in the tomcat servers.xml:

1. Default: <Context docBase="myapp" path="myapp" reloadable="true"="org.eclipse.jst.j2ee.server:myapp" />
2. Root: <Context docBase="myapp" path="" reloadable="true"="org.eclipse.jst.j2ee.server:myapp" />
3. Custom: <Context docBase="myapp" path="myapp2" reloadable="true"="org.eclipse.jst.j2ee.server:myapp" />

The trouble is that changing the location of the application changes the references to pages within the application.  Probably the links in your application use relative references, but these can be tricky to get just right.

On the client side we might have different types of link like:

• Css resource: <link type="text/css" href="includes/style.css">
• Script link: <script type="text/javascript" src="includes/util.js">
• Regular hyperlink: <a href="about.jsp>About</a>

We could also have server side includes like:

• <jsp:include page="/includes/header.jsp">

The server side links are easy, just include a leading slash and tomcat will look for the page in the context root folder of the web application.

For client side includes add "<%=request.getContextPath()%>/" and then use the relative path in your web app. This will allow your web app to be redeployed to any folder.

Why this way? For client side links the full path is created by the web browser. If the relative url starts with a slash then it is added to the domain of the referring page.  If the relative url doesn't start with a slash then the it is added to the path of the referring page. Lets assume that we want to link to a style sheet with url:

www.acmewebapps.com/myapp/includes/style.css

For example consider these links from the page www.acmewebapps.com/myapp/index.jsp:

• href="/includes/style.css" gives www.acmewebapps.com/includes/style.css Incorrect, missing web application folder
• href="includes/style.css" gives www.acmewebapps.com/myapp/includes/style.css Correct

The first style without a leading slash works, since the context root of the web application is kept. But note that the second style, with the leading slash only works if the web application root is in the root of the domain - in this example the myapp folder is missing.

However, the first style also has drawbacks if referencing a page from a different subfolder, consider links from a page in the admin subfolder www.acmewebapps.com/admin/config.jsp:

• href="includes/style.css" gives www.acmewebapps.com/myapp/admin/includes/style.css Incorrect, extra folder in path
• href="/includes/style.css" still gives www.acmewebapps.com/includes/style.css Incorrect, missing web application folder
• href="../includes/style.css" gives www.acmewebapps.com/myapp/includes/style.css Correct

This would work, but we have to always remember to add the correct number of ".." entries to the path.  Which means that we can't use server side include of standard 'boiler-plate' includes, such as menu.jsp, because the paths will change depending on where the server side include is used.

The way to do client side includes in a way that works anywhere is to explicitly refer to the web application context root in your server side generation of the include, as follows:

<%=request.getContextPath()%>

So that the full include for the style.css example looks like:

href="<%=request.getContextPath()%>/includes/style.css"

To recap - for server side includes, just start with a slash and use the relative path in your web app.  For client side includes add "<%=request.getContextPath()%>/" and then use the relative path in your web app. This will allow your web app to be redeployed to any folder.

Outbound mail, SMTP Port 25, Blocked

It seems that some ISPs (Internet Service Providers) such as Wanadoo and AOL, are blocking port 25 traffic to all but their own smtp servers, in an attempt to combat spam.  The idea is that viruses on infected customers computers connect to open smtp servers on port 25 and mail out junk.  This seems like a rather clunky way for an ISP to try to deal with spam, they are effectively cutting out part of their support for TCP/IP - the connection they are supposed to provide.

Spam mail is huge problem, with current spam filters struggling to avoid marking legitimate email as junk - known as false positives in the industry. This means that real email is constantly ending up in peoples junk folders. So at least the ISPs are trying to help.

Our heavyweight mail service Tonsho, which allows sending of large email attachments and special spam filtering, is affected by this port blocking. We need users to be able to connect to our smtp server. Therefore we have also made the Tonsho smtp server available on port 26.

Our spam protection system is based on authenticating users, and only accepting mail from authenticated users.  In the first instance we distinguish between automated emailers and genuine humans by means of a captcha.

We maintain a list of allowed contacts.  If a user who is not in your allowed contacts list send you an emails, they are automatically sent a challenge mail which links to a captcha. Once users have solved the captcha and been added to your allowed contacts, mails come straight through. Meaning they only have to authenticate once.

If you want to try Tonsho for yourself, for free, you can sign up here.

Announcing Tonsho.com

On Monday 4th February we will officially launch the beta of Tonsho - an email service that solves two of the biggest problems with email: sending large file attachments and spam.

With Tonsho you can send large email attachments as normal using our outgoing smtp mail server. The server strips off your attachment, puts it on a web sever and sends your recipient a link.

Our Spam filtering works by authenticating senders.  When someone not in your contacts tries to reach you they are sent a challenge email. This contains a link to a captcha - one of those squiggly things. Once solved the sender is always allowed to send to you; meaning you don't miss important mails from your customers and friends.

Naturally I'm excited about launching Tonsho. The file sending stems from an idea I had when discussing sending files with Dan Maxwell in the lake district. Creating a working prototype was fairly straight forward. Deciding to go ahead with the idea was the most difficult part. Creating a full blown service was very involved - things like how to describe and present the service, creating help pages and getting the user options correct all take a lot of time and effort.

At first we'll quietly announce the service so that we can be fast to respond to questions and fix any teething problems. Then start to really market it once we're confident its really robust.

Please do sign up for a free beta account here!

Productizing a software program

There's a big difference between a program that runs and a software product. In his book The Mythical Man Month Frederick Brooks claims that to go from a working program to a 'programming systems product' (something with APIs which is fully generalized, tested and documented) takes 9 times as much effort.  Brooks is discussing experiences of developing whole operating systems, my experience with medium sized systems is similar, it takes multiples of the time one would allow to just write code that works, to get it to code that is really ready for use in general situations.  There are three basic things needed to go from a working program, developed on perhaps one developers machine and perhaps even delivered to handful of customers, to a supportable, maintainable product.

Version Control Repository

This is to manage the code from multiple developers and manage the process of iteratively testing and refining the code.  CVS is open source and widely used or Microsoft source safe is a popular proprietary choice. My preference is Subversion which is essentially CVS with folder versioning.

Bug Tracking

The bugs should to recorded and always assigned to one person who is responsible for either fixing the bug, or assigning responsibility to someone else to fix.  Once the bug is resolved, responsiblity returns to the reporter to check it is resolved correctly and close the bug.  A web system like the open source Mantis is good for this.

Automated Build

Its no good compiling the code on the development machine and then delivering it.  The build process needs to check out the code from the CVS, with a given tag, and perform all the compilation from there.  There should be no manual input to this process at all. If this process occurs on the development machine you should ideally install a virtual machine, to ensure that the build is cleanly depending on just the CVS copy. You must also make sure that the build id (the CVS tag) is placed in the package so that when it is deployed on the customers machine it is identifiable.

The idea of eliminating any manual parts to the process is to make sure that a human error can't damage the build - although this may take some initial investment of effort, this will the long in the long term also save time.  This also means that if a customer reports a problem the version that they are using can also be reproduced from CVS.

More

Of course there's more, but I think these are the big three that I couldn't work without. The Joel Test lists 12 points and doesn't include anything on testing.

Big Tech Companies

Microsoft is trying to but Yahoo! See Search Engine Land article or Fuller View .

I was interested to see how big yahoo is is relative to other tech giants.  I looked up the big players that came to mind on Yahoo Finance, these are the market capitalizations (all in billions).

• Microsoft: MSFT 303.41B
• Google: GOOG 176.54B
• Apple: AAPL 118B
• IBM: IBM/NYSE  148B
• Ebay: EBAY 36.3B
• Amazon: AMZN 32B
• Yahoo: YHOO 25.63B
• Sun: JAVA 13B

All of these are traded on the NASDAQ with the exception of IBM, which is on the NYSE - I guess that's because its more old school.  But I don't know the ins-and-outs of these different stock exchanges.