The state of TDD

The state of test-driven development across our industry is terrible. This may seem a rather brazen statement, given pretty much everybody and their dog is writing tests somewhere vaguely around the same point as they write code, and you don't have to scroll far down this blog to find an article in which I bemoan the way it's become so much the accepted way to develop that nobody even thinks about it any more.

And therein lies the rub. Nobody's thinking about what they're doing with this. The result is that every day thousands, millions of developers are sitting down in front of Visual Studio or IntelliJ or Webstorm and writing thousands of lines of interfaces, wiring and brittle, close-coupled tests in place of what ought to be 10 lines of code and some basic exception handling. We're letting tiny 2 and 3 point stories hog entire sprints against some notional goal of getting testability just right. We're not even appalled by this; we're patting ourselves on the back for "doing it right" and getting accolades from our manager or architect because they can tick the "Doing TDD" box on their objectives.

Oh, and we're completely missing the point into the bargain. These days, the only time a proper unit test gets written is by accident. The truth?

Nobody's Unit Testing Anything Any More

We've forgotten how to write an actual unit test, and by extension how to write a unit.

It's probably no bad thing to introduce some definition of what exactly a unit is at this point, and here's mine: a unit of code is something that takes an input, validates it, transforms it in some way and provides an output or effect. Could be a class, a bunch of classes, or even a single function - doesn't matter. A good unit should be trivially easy to split out into a module or microservice: it does something useful, it's independent enough that it doesn't need to spend all its time chatting to other units, and the context of what it does is well-bounded.

It therefore follows that if we need to inject the entire world into our unit, it's failing the second part of that. Because what you're implicitly saying by injecting a ton of dependencies is that your unit isn't an independent thing with a well-bounded context. In order to do its job, it needs to co-operate with a whole bunch of other things, which (if you follow the same pattern) need to co-operate with more things, which need... if we tried to turn these demi-units into microservices we'd sink in network latency issues with all the chat that's going on.

A unit shouldn't need to take a dozen parameters to instantiate itself. You should only be injecting the things where you expect to change the underlying implementation: stuff like data persistence or rules engines. The bits where your project has two or more things that implement the same interface.

This is where the TDD-as-it's-done-today crowd will accuse me of heresy. "But how can you mock it if it's not passed in?"

Which is how we've got to writing class tests rather than unit tests. Let's go back to that definition of unit. Validate, transform, output. I don't need to be able to mock out the validation code or the guts of the transformation. It's part of the unit! All you need to be testing here is, "do I get the result I expect when I send this data?"

The temptation, of course, is to say, "well I might use this validation code elsewhere, I ought to drag it out and test it separately"... but now you have a situation where you might decide to change the validation to support a new component, alter your tests to support that new behaviour... and oh, your application blew up and you didn't spot it until the integration test. Potentially you didn't spot it at all. If each unit had its validation behaviour tested appropriately, however, you'd be fine. Plus you'd be free to move that code around as you please without having to touch the tests at all.

It's worth remembering that when you start passing an interface around in a dependency injection world, you're moving some of the responsibility for how your application behaves to an IoC container. That's not a good thing, as issues with IoC wiring are subtle, hard to debug and too often are covered at best by integration testing, and sometimes not at all. Plus your container wiring starts to turn into an omnipotent monster with tentacles reaching into every part of your application.

That's not the big problem with class tests, though. The big problem with writing class tests rather than proper unit tests is the sheer amount of design damage you're going to inflict on your application to support such fine-grained testing, and the enormous cognitive load you're going to inflict on anyone unfortunate enough to maintain it in future. See, the thing about a unit is it's a nice, coherent thing. If I've got well-bounded units, I expect to have to mock only my persistence layer and external connections, and many units won't need anything mocked at all. That in turn means my application only has these things injected where they actually matter and I do need to be passing interfaces around to keep things loosely coupled. Outside of this, I can keep the rest of the code coherent, it's obvious to anybody reading where the boundaries are, and they don't have to go hunting through a load of wiring code or doing runtime debugging to find out where an interface is actually implemented.

If I'm writing class tests, I don't get to do that, because I need to take classes outside of their unit and test them separately. Which means I now need to mock internal implementation details of my unit wherever those cross class boundaries rather than just mocking around the context boundaries. Given the tendency for the single responsibility principle to be misunderstood as creating dozens of tiny horizontal slices, that's a lot of boundaries with a lot of interfaces being passed around at each instantiation. So I now have this enormous suite of tests which assume my implementation always has a BLL, and a DAL, and a WTFL, and I can get this situation where all of my tests are green but the application blows up in my face because the container doesn't know how to instantiate an IValidationEngine or passes in the wrong implementation for ILogicEvaluator or whatever.

Worse, while I can claim my application is loosely coupled to itself it has an extremely tight coupling to its test suite. As a result, it's very hard for me to refactor without having to change the tests; this is very bad as I shouldn't be touching those tests (and potentially introducing errors) during my refactoring. So not only have I compromised my application in the name of class testing, I've even compromised my notion of TDD as a red-green-refactor cycle - I end up doing either a red-green-reddish-greeny-sort-of-refactoring-ish cycle or a red-green-run-away cycle and neither is good for the long term health of my code.

Class tests suck, okay?

In summary, the case against class tests:

• They don't test units as independent pieces of functionality
• They miss failing code through over-reliance on mocks
• They push code into areas of the application which inherently have poor coverage in the class-test scenario
• They force proliferation of unnecessary interfaces
• They turn IoC containers into god objects
• They tightly couple an application to its test code
• They outright prevent non-trivial refactoring

In one line? Class-by-class testing outright breaks TDD by compromising both the quality of your tests and your ability to refactor.

How do we fix this?

Usually at some point in this discussion I can expect the response, "well yes, but that's the only way to tick off all these well-meaning checklists about how development should be done, and anyway this is the way we've always done it, and anyway Matt you just like parading around with an attitude that's somewhere between grumpy old man and cowboy coder, so leave our tests alone and go rant about how all Javascript modules should be less than 100 lines long or whatever else it is you do around here". (Disclaimer: some editorialisation and embellishment may be occurring here. But not much.)

Which is pretty much my whole problem with the way we do unit testing - or as I've pointed out, the way we write all of these icky class tests to reconcile our understanding of "must write tests" with our understanding of "must have single responsibility" and "must have defined layers" and "must use an IoC container" and "in this house we obey the laws of the visitor pattern". At no point do we step back and think, "so what is the unit here? How am I going to test it?" You know, those trifling details that TDD was supposed to make us think about rather than hacking together spaghetti and introducing a dozen bugs every time we refactor something.

So this is how to fix it. Think about what you're doing.

That means working out what exactly the unit is, and figuring out where the bounds of its context are. It means working out a sensible approach for testing it, and carefully weighing up what compromises you're taking on board to facilitate your test strategy. Hardest of all, it means knowing when to break with "the way we've always done it" or "what's in the textbook" to do something that is objectively better for your needs today than what went before. Learn to think of practices as guidelines to steer you on to the right path, rather than checklists which must be obeyed at all times.

And write some damn unit tests.

_{Original image by Waifer X CC BY 2.0. Image has been modified.}