To automate or not to automate, that is the question

Test tools can be a valuable resource in software testing – but they are not a substitute for testers

The key fits beautifully and turns like a dream. But did you check if it locks the door?
(Photo: Alan Cleaver, Flickr)

There is a lot of talk and excitement over the use of test tools in software testing. A whole chapter of the ISEB foundation syllabus is devoted to it. Test tools range from small-scale open source applications to comprehensive commercial packages. A favourite selling point is how test tools eliminate human error. Some vendors promise you savings beyond your wildest dreams (one company even promises benefits within an hour of use). But a good tester needs an eye for what can go wrong in software, and software they’re using for testing is no exception. So: are these tools any good?

Obviously there are cases when you’d have to use a test tool (such testing a website to see if 10,000 people can log on at once, unless you happen to have 10,000 people at your disposal), and cases where you’d never use one (such as tester user-friendliness for IT novices). But there is a vast range of test tools out there covering every kind of activity you can imagine, so it would be impossible to cover them all in one blog entry. Instead, I’ll concentrate on Selenium IDE, which I’ve been using over the last few weeks. It’s an open-source extension to Firefox which allows you to automatically test websites; you can either automatically record yourself clicking through all the links and entering data into forms for replay later, or manually program the test yourself.

The great patent fight

Software patents are a menace to IT development. Instead of protecting innovation, they are being used to stifle it.

Names: Joseph-Michael and Jacques-Etienne Montgolfier

Invention: First manned hot air balloon

Patent infringed: Taking a wig to an altitude over 2,000 ft

Okay, I have relented: in spite of my disdain for updating your Facebook status every five minutes, I’m going to get a smartphone. I’ve therefore been looking for a suitable handset and my current preference is for a Samsung. I don’t have any strong preferences between brands – to me, a handset is a handset – but I do want to show my support for Samsung in their patent battle with Apple.

Since most of you won’t know what I’m talking about, it works as follows: Samsung has been banned from selling its Android tablet in Germany following legal action from Apple over patents it holds. Similar action in Holland has stopped the sales of three Samsung Android phones. However, in turn Apple is being sued by HTC for infringing patents that the latter company brought from Google. Meanwhile, Microsoft claims that Android phone violate its patents and consequently HTC pays royalties to Microsoft, whilst non-compliant Motorola is being sued in the US courts. But Microsoft have been successfully sued by Canadian firm i4i who claimed Word violates their patents. I could go on, but you get the idea.

How to spot a black swan

New research suggests one in six IT projects run three times over budget. Keeping expectations realistic might avoid this.

"Well, maybe it collided with a tin of paint"
(Photo: Jon Smith photography, Flickr)

A study that came out last week was about IT projects breaking their budgets (see this and this). According to the research, in a sample of 1,471 large-scale IT projects, they ran on average 27% over budget, but the headline-grabber was then observation that one in six projects go three times over budget. The researchers have named these projects “black swans”, and blames managers for failing to account for low-probability high-cost risks in big IT projects. To the more cynical IT professionals, this is nothing unexpected. It’s not hard for a software tester to witness at least one project like this – failing that, you don’t have to look far for the latest story about the notorious NHS IT system.

What was interesting, however, was the reference to the Black Swan theory. This phrase was originally coined by Lebanese-American essayist Nassim Nicholas Taleb. There’s a whole book about this, but the basic idea was that there was a time when it was believed all swans were white. No-one had ever seen a swan in any other colour, so no-one gave serious thought to this possibility. Then Dutch explorer William de Vlamingh went to Australia and discovered that some swans are black, fundamentally changing how people saw swans. And in hindsight, it was nonsensical to assume swans could never be that colour just because you hadn’t seen one before. Taleb used this analogy for all sorts of events: he suggested, amongst other things, the attack on the World Trade Center and the Credit Crunch could be considered “black swan events” – both unexpected at the time, both easy to rationalise now.