The big bang theory

If you look beyond the political point-scoring over the latest debacle on Universal Credit, the real lesson is that the "big bang" approach to IT projects rarely pays off.

A completely inaccuarate depiction of the Big Bang.
Also not a good approach to most software projects.

Well, I hate to say I told you so, but ... I told you so. Just under a year ago, I idly speculated that the next big story about an IT cock-up might be the upcoming Universal Credits system. I won't go over the whole thing in detail, but it boiled down to two concerns: firstly, I was sceptical over whether the intended launch of October 2013 was realistic; and secondly, I know from my experience of ID cards that there is a culture in the civil service of making promises that cannot be delivered. And what do I find yesterday? Oh dear, oh dear, oh dear.

Now, before I jump on any bandwagons, it's helpful to put this in a bit of context. Firstly, the National Audit Office is notorious for nit-picking (as is the Public Accounts Committee), and their supposedly damning reports are often little more than minor points blown out of proportion by the press. Secondly, benefit reform is a hugely controversial issue and a lot of criticism (and defence) of this IT project will be down to ideological stance on benefits rather than whether the product does the job. (For the record, I think the principles of Universal Credit - that work should always pay and simplification of a bloated complex system - are a good idea, but there's valid points over using the reform as a smokescreen for cuts.) Nevertheless, it looks like there's more to this one than political hype. The October launch is now just six pilot sites, which is a common Civil Service method of back-pedalling in a way they can claim they "met" the deadline.

So what's gone wrong? There is a good summary of reported mistakes on BBC news, and the thing that struck me the most about this is how similar these mistakes are to the mistakes made with ID cards. Comparing what's happening now to what happened with ID cards, I can tell you the following:

Politics versus Plan B

There is no more important place to get IT projects right than central government. Unfortunately, internal politics encourages the opposite.

This is a visual metaphor, with little or no relevance to the actual article.

Who’d be a prime minister? On one day it’s all “We Love You Dave/Gordon/Tony”, then the moment you’re under 35% in the opinion polls it’s a catalogue of everything your government’s doing wrong. This year it’s been the granny tax, pastygate, the petrol non-strike, G4S and the West Coast rail franchise to name a few. All that’s missing is a good old IT shambles. After all, the last government kept us busy with the lost child benefit discs and the ill-fated NHS system. Well, for all you restless journalists itching for a story, I recommend you keep an eye on the upcoming Universal Credit benefit system.

In case you’re not following UK politics on an hourly basis, Universal credit is a plan to merge a number of key benefits such as jobseekers’ allowance and tax credits into a single system. Benefits is a controversial issue right now, but this is a software testing blog, and the issue of interest is that this is all dependent on a new IT system being developed. Now, before I go any further, I must stress I don’t know anything about how this project is going. For all I know, it could be all going swimmingly. But what if it isn’t? There doesn’t seem to be any kind of Plan B ready if the project goes behind schedule. And if this happens, it won’t be the first, because I worked on the last IT project where that happened.

Which IT project, I hear you ask? Well, please don’t be too harsh, it wasn’t my idea, they made me do it, but – I did software testing for ID cards. Yes, those ID cards. Remember them?

So what went wrong at Natwest?

A lot of questions need to be asked over RBS’s computer problems – but if we want to stop this happening again, we need to listen to the answers.

An easy answer. But not a useful one.

So there we have it. For anyone who questions the value of software testing, here is a prime example of what happens when you let a bug slip through. I know we’ve already moved on to another banking scandal, but in case you’ve forgotten: many Natwest customers failed to get paid owing to a botched system upgrade. This has led to all sorts of consequences, and the obvious question of how this could be allowed to happen.

Except that when people ask this question, I fear most of them have already decided on the answer, which is that RBS is a bank and therefore Big and Evil and responsible for everything bad in the world from Rabies to Satan to Geordie Shore. That answer might make people feel better but does little to stop this happening again. In practice, what went wrong is likely to have little to do with the credit crunch or banking practices and a lot to do with boring old fact that any bank – no matter how responsibly they borrow and lend – runs on a highly business-critical IT system where any fault can be disastrous.

The dreaded feature creep

Even in the best managed projects, feature creep is difficult to avoid. Here are my tips for how to reduce the risk.

Apologies for another quantum mechanics in-joke. But this explains a lot.

Right, I’ve been told off for starting too many blog entries with “I’m afraid this is going to be another moan”, so this time I’m going to try to be a bit more positive. My last post had a go a web designers often over-charge for websites, and people who actually pay them that much. This contained an observation that this can apply to IT procurement more widely, with an example of the notorious contracts for £3,500 per computer in some government departments. Having thought about this, it was a harsh generalisation.

Where government IT projects overrun costs, it’s rarely because a company charged a fortune upfront. It’s usually because the initial costs are cheap but the contractor charges extra for things like including additional features, or installing new hardware. In some cases this gets out of control, like ridiculous call-out fees for something as simple as changing a mouse, and that is a key driver to the argument that IT companies rip off Whitehall. But the IT companies do have a good counter-argument. They often say that if government departments ask them to do a simple task, and then keep changing their mind in mid-project, it really does cost that much to keep making all the changes. I have come across both scenarios in my time.

But if we forget these two extremes and assume both client and contractor are genuinely motivated to work together and keep costs down, the fact remains that controlling costs is an absolute bugger. It is very difficult to get every detail of a working IT system right when the system currently only exists in paper plans. The mistake that must be avoided at all costs is “feature creep”, where more and more changes are requested to software in development, until costs rocket, the original design is no longer fit for purpose, and if you’re the NHS – well, we know what happened there. But there’s nothing new about feature creep, so why is does this mistake keep being made?

Give penguins a chance

Would switching to open source software save public money? I don’t know, but we should at least try to find out.

The Windows logo versus the Linux mascot. A little-known but very bloody feud.

I know software testing is a very absorbing activity, but in between bouts of testing you might have noticed there’s a bit of a financial crisis going on. As tax rises, benefit cuts and axing public services don’t go down that well with the public, the government is keen to find less painful ways of saving money. This, in part, was the idea behind the Spending Challenge letters that went out to all public sector workers shortly after the 2010 election asking for ideas to save money. The ideas ranged from the pragmatic to the ridiculous, but one suggestion that caught my eye was to switch proprietary software for free open-source alternatives. This is not an unthinkable as you might expect; the Lib Dem manifesto said they’d look into this, and George Osborne himself is said to be interested.

I’ll be open and upfront here: I use Linux, LibreOffice (effectively the successor to OpenOffice) and other free open-source products wherever possible. It’s partly I don’t want to pay for software when free stuff does the job, and partly because I have problems with the way Microsoft uses its dominant position to make life difficult for people who use competitors’ products. But I don’t believe in imposing my views on other people, and I’ll help out with any IT problems whatever software they’re using. (Indeed, a software tester who doesn’t is a short-lived one.) I wouldn't push savings too much with a charity (Microsoft usually heavily discounts software for them). I’d also be hesitant to encourage a small business to switch to open-source when everyone they work with expects them to do all things Microsoft. The public sector does not have that problem – they mostly communicate with each other, and they’re big and ugly enough to insist anyone else works with their software if they wish – but any move away from Microsoft or any other proprietary software must save the public money, and not just be done to prove a point.

Don’t be afraid to upgrade

Upgrading software in the workplace requires caution – but some companies make this far more complicated than it needs to be.

No, you’re not having a strange dream, Microsoft really is celebrating the demise of a flagship product. Continuing the tradition of celebrating milestones in web browser development with cakes, Microsoft’s latest cake marks the “death” of Internet Explorer 6 – or, more accurately, the decline in US IE6 usage to 1%. Microsoft have make a huge effort to get people off Internet Explorer 6 (obviously, they’d rather you went to Internet Explorer 7, 8 or 9 than Firefox, Chrome or Safari, but an effort nonetheless) through hasty development, advertising campaigns, and now even silent updates to upgrade remaining computers. And with Microsoft themselves admitting IE6 has had its day and even the die-hard open sources fans accepting that IE7 onwards is a big improvement, you’d think everyone would be happy.

If, however, you’re reading this blog from a UK government building, you may think you’re accessing news from a parallel universe. The UK public sector is inexplicably at odds with the rest of the world. IE6, like most early browsers, has a sluggish Java engine that runs at snail’s pace on modern Java-Rich pages. Most public web pages have now dropped support for IE6. And yet when the China hacking scandal exposed hugely embarrassing security flaws in IE6, and the French and German governments warned everyone off IE6 (and , for a while, later versions), the Cabinet Office insisted there was nothing to worry about.  To be fair, web browser security isn’t the be-all-and-end-all for government buildings – their strongest defence will always be the safeguards within the Government Secure Internet – but the web browser is the last line of defence in a compromised network, and it’s a reckless to rely on a web browser written before widespread broadband adoption and the security threats it brought along.

Security should be everyone’s responsibility

There are two main enemies to security: convenience, and inconvenience. Better public education of the risks would make things safer.

"But I only wanted to check my Facebook."
(Photo: 48states, Wikipedia)

Security testing is a very specialised branch of software testing. Unlike most branches of software testing, where you’re simply trying to iron out things that go wrong by mistake, in security testing you’re fighting people trying to make things go wrong on purpose. It requires a lot of responsibility on the part of the testers and a lot of trust on the part of the clients – indeed, there are suspicions this gets abused – and consequently, many software testers won’t put themselves forward for security testing. Nevertheless, most testers will highlight security concerns as and when they notice them, and therefore take an interest in whichever high-profile security breach is in the news this week. Which brings me nicely on to of Hackgate.

Now, in case you lost track of the plot somewhere around episode 4,605 of the Leveson Inquiry, one of the latest developments is a claim that hacking extended to e-mails. At the moment, unlike phone hacking, this has not yet been proven or admitted to. But, quite frankly, it would come as no surprise if this turns out to be true. Like voicemails, the security surrounding personal e-mails has been notoriously lax, and practically an open invitation for hackers to pry into private matters.

How to spot a black swan

New research suggests one in six IT projects run three times over budget. Keeping expectations realistic might avoid this.

"Well, maybe it collided with a tin of paint"
(Photo: Jon Smith photography, Flickr)

A study that came out last week was about IT projects breaking their budgets (see this and this). According to the research, in a sample of 1,471 large-scale IT projects, they ran on average 27% over budget, but the headline-grabber was then observation that one in six projects go three times over budget. The researchers have named these projects “black swans”, and blames managers for failing to account for low-probability high-cost risks in big IT projects. To the more cynical IT professionals, this is nothing unexpected. It’s not hard for a software tester to witness at least one project like this – failing that, you don’t have to look far for the latest story about the notorious NHS IT system.

What was interesting, however, was the reference to the Black Swan theory. This phrase was originally coined by Lebanese-American essayist Nassim Nicholas Taleb. There’s a whole book about this, but the basic idea was that there was a time when it was believed all swans were white. No-one had ever seen a swan in any other colour, so no-one gave serious thought to this possibility. Then Dutch explorer William de Vlamingh went to Australia and discovered that some swans are black, fundamentally changing how people saw swans. And in hindsight, it was nonsensical to assume swans could never be that colour just because you hadn’t seen one before. Taleb used this analogy for all sorts of events: he suggested, amongst other things, the attack on the World Trade Center and the Credit Crunch could be considered “black swan events” – both unexpected at the time, both easy to rationalise now.