Re: Unit testing is not (generally) useful

Tuesday, February 10th, 2009

There is a post up on Bill Moorier’s blog about the usefulness of Unit Testing. In it, he writes the following, which largely sums up his post:

The metric I, and others I know, have used to judge unit testing is: does it find bugs? The answer has been no. For the majority of the code I write, it certainly doesn’t find enough bugs to make it worth the investment of time.

I think we’re missing a couple of big advantages to Unit Testing here. Unit Testing catches very few bugs when writing a new program, this is true. Most developers test their code manually in much the same way as they would with unit tests. But this is not the true strength of Unit Testing.

First of all, Unit Testing forces one to write their programs so that they are testable. That usually means: well thought out APIs, consistent interfaces and more modular code.

Second, Unit Tests serve as documentation. They are prime examples of how code should be called, what side-effects one can expect and how various components work together. That’s assuming you write decent Unit Tests, naturally.

Finally, Unit Tests are invaluable when doing regression testing, which is where they truly shine. They are not for catching bugs in new code, they are for catching bugs in old code. Suppose you have a library which hasn’t changed in four years. Now you find a small problem or inaccuracy in one of the routines, so you decide to fix it. You test the new code, and it works. But when you run the Unit Tests, you suddenly find that on the other side of your application twenty tests fail where they passed with flying colors before you made the change. You just discovered a regression bug.

I have the feeling that Bill Moorier is talking about Unit Testing in the context of Test Driven development. I’m not much of a fan of Test Driven development, even though most of my points still hold true. However the Test Driven development proponents, much like the Extreme Programming proponents, somehow make it out to be a magically superior way of writing software. This I have to disagree with. I see no reason at all why Test Driven development and Extreme Programming in general would deliver better software than traditional software developing methodologies and, especially, common sense. There is no silver bullet. Bad software, in my view, is more often caused by bad or lazy programmers, time limits and seriously flawed requirements than the use of the wrong methodology.