The real-world value of high test coverage

I’d like to share a real-world example of where having good test coverage has paid off for me personally. First, a little background.

I’ve explained unit testing to many different clients, and on many different projects. Some of these have been TDD or even BDD projects, and others have been more, shall we say, traditional. Regardless of the methodology, all these approaches should leave behind some fairly comprehensive tests when you’re all done, and to me, that’s one of their major benefits. We’ve had discussions on my teams before about the value of high test coverage, with some developers taking the stance that coverage numbers are meaningless. This is partially true. 100% test coverage still doesn’t mean that your code actually works the way you intended, only that it works to the satisfaction of the test suite. Of course, if you’re a T/BDD developer, then presumably you wrote your tests with the end goal clearly in mind, and the fact that the tests are passing means that you’ve achieved that goal. If this is the case, then I suppose you can be more certain than most that your code is actually correct.

Another benefit of high code coverage that doesn’t get nearly as much press as TDD’s “emergent design” benefits, is catching regression bugs. Regardless of how they were created, your tests become documentation of how your code behaved at a certain point in time. The higher your coverage, the better your odds are of noticing if you inadvertently change something down the line. In addition, your tests can detect the ripple effects that changes in component “A” might have on component “B”. Both of these benefits were demonstrated quite clearly for me this last week.

A while back, I was part of a team developing a line-of-business application for a client. We had successfully delivered the first phase of the project, and were busily working on an additional round of phase two features when suddenly, and without much warning, the client’s priorities shifted, and phase two was put on indefinite hold. At the time we had completed a pretty decent amount of the phase two features, but without all the features, the completed ones weren’t going to do much good. It was a kind of all-or-nothing release. Our in-process work was shelved, and we all moved on to other assignments.

Now, nearly two years later, the client is ready to pick up where they left off, and we’ve been called back to finish phase two. The trouble is that things are not as we left them. There’s been a lot of internal development during the interim, including some not-insignificant architectural changes. In addition, these changes were not made on the phase two branch where we had previously been working, but on another branch off of phase one. My task for the last week has been to try to pull forward as many of the finished phase two features as possible.

Fortunately, we left behind some pretty decent test coverage for both phase one and two. As I have spent the last week pulling feature after feature back from the abandoned phase two branch, I have done so with a high degree of confidence because I know the tests will alert me if I break anything in the process. Each time I pull a feature forward, and all the existing tests continue to pass, I am more convinced that we did the right thing implementing the tests that we did. On top of that, as I pull the phase two tests forward, I can see that the old features still work under the new architecture. I’ve been able to salvage months of past work in a fraction of the time because I can merge the changes in with confidence that the features are working as intended, and not breaking anything else in the process.

This isn’t to say that all the tests have passed on the first try. Code that was affected by architecture differences between the two branches will still give me an initial failure which I have to hunt down and fix, but it’s taking a lot less time than it would without the tests. The tests are dutifully drawing my attention to anything that behaves differently than it did the first time around, and for that I am truly grateful.

This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s