This post is part of an ongoing series about Unit Testing.

Camps

There are a number of different approaches to developer testing. Since many of us seem to like binary choices, these approaches have often been divided into opposing “camps”. Here are a few of them:

  • TDD vs BDD
  • xUnit vs spec-style
  • Statist vs Mockist
  • Detroit-style vs London-style
  • Abstraction vs Explicitness
  • etc.

Note that there is a lot of overlap between many of these pairs of camps.

If you’ve been reading this blog for long, you’ve probably noticed that I tend to take a middle road when there are dichotomies like these. I look at the pluses and minuses of both sides and try to synthesize the best parts of each approach into something workable.

What I Believe About Testing

My approach to testing has evolved over the years. I read a lot and learn new ideas. At my last job, I worked in the same code base for more than a decade and had to live with the consequences of the choices my colleagues and I had made earlier. When I feel pain from that, I adjust my practice to avoid that pain in the future.

My testing approach was originally influenced heavily by the original TDD proponents like Kent Beck, Ron Jeffries, and Uncle Bob Martin. Even when I thought I understood TDD well, I experienced several epiphanies when I read Kent Beck’s Test Driven Development By Example. More recently, I’ve been heavily influenced by Steve Freeman and Nat Pryce’s Growing Object-Oriented Software Guided By Tests (GOOS) and Sandi Metz’ Practical Object-Oriented Design in Ruby (POODR).

With that, here are some things I believe about writing good developer tests. I will expand on many of these points in future posts.

TDD vs BDD

I’ve largely been a TDD person. I didn’t follow BDD closely when it first came on the scene, and the little bit I read about it didn’t seem to me to be different enough from TDD to convince me to switch.

My tests now are probably closer to the BDD style, but I really tend to focus on testing the responsibilities of the object. I don’t test each public method of the object separately; instead, I test the responsibilities of the object using one or more of the public methods. I will go into more detail on this subtle distinction in future posts.

xUnit vs Spec-Style

I started out using the xUnit frameworks. As I’ve moved into the web development world, many of the projects I’ve worked on use RSpec and/or Jasmine so I’ve learned them.

I’m firmly in the middle on this one; I’m reasonably comfortable using either tool, and I tend to write similar tests either way. I like a lot of things about Minitest, but I also like some of the features of RSpec.

I find the spec-style tests a bit too verbose sometimes, but that may have more to do with the way other people use the tools; fixing that is part of the reason I’m writing this series :-). I find nested contexts quite nice, but they can be overused.

In general, I feel like I need to be comfortable with both frameworks, and I think it’s possible to write good tests with either one.

Statist vs Mockist (aka Detroit-style vs London-style)

This is probably the place where my testing style has evolved the most over the past few years. I used to be strongly in the statist/Detroit-style camp. I thought the need for mock objects was a design smell. I’d seen some test suites that used mocks, and I found them hard to read and understand.

After reading GOOS and POODR, my opinions changed a lot. I began to understand how to use mocks properly. POODR addressed several of the concerns I had about mock-based testing.

I still write both state-based and mock-based tests as appropriate to the situation. I think mocks can be overused, but I also think they can be used to great effect in many situations.

A couple of years ago, there was a thread about this on the extreme-programming mailing list (yes, it still exists). I haven’t been able to find the original post in order to credit the author, but someone made approximately the following point:

If you are focused on the objects in your system, you will tend to write state-based tests. If you are focused on the messages in your system, you will tend to write mock-based tests.

This may not be entirely true, but I feel like it helps me understand why I’ve transitioned from state-based testing to mock-based testing. Being a Smalltalker, the idea of focusing on messages really appealed to me, but I hadn’t been doing that. After reading POODR, I really began to focus on messages more than objects and my testing style changed accordingly.

That said, I’m also fascinated by Arlo Belshee’s ideas on mock-free testing, but I haven’t had a good chance to really try out his ideas. At my last job, there was one part of the system that I really wanted to re-implement using both Arlo’s approach and a mock-based approach to see how it turned out each way, but I never had the opportunity before leaving.

Abstraction vs Explictness

There are people who argue that you should treat test code just like production code. You should apply all of the same duplication-removal and refactoring techniques, extracting classes and methods, etc. Others argue that doing this results in tests that are too hard to understand, and that you should instead tolerate duplication in order to make your tests more expressive and understandable in isolation.

Once again, I fall squarely in the middle on this. I really dislike duplication, but I also really like expressive and understandable tests.

My current thinking is that I want to eliminate noise from my tests; I want to communicate only the important details and hide away anything irrelevant. I think of it as introducing a higher level language that I use in my tests. I will illustrate this with examples in future posts.

Keep the Tests Running Fast

When you’re in the middle of the TDD cycle, you want fast feedback. Slow tests interrupt the flow of development, causing you to lose your train of thought and get distracted. The slower the tests, the less likely you are to run them, and the less helpful they’ll be.

Test Independence Is Critical

I strongly believe that each test should stand alone. No test should depend on any other test having been run first. This allows the tests to run in any order and allows me to run a single test at a time if I need.

If there is some very expensive setup that is used by several tests, I might allow that, but I try to avoid it if I can. If the project I’m on uses Rails fixtures for example, I’d rather pay the price for their creation only once per test run.

Test Things Only Once

Each behavior of the system should only be tested one time. Many test suites are too slow as it is, so re-testing the same behavior multiple times just makes the problem worse without adding any more confidence in the system.

POODR gives some great advice about how to accomplish this goal.

I’ll talk about this more in future posts.

Watch Tests Fail

It’s very important to watch each test fail for the right reason before watching it pass. If you’re truly doing TDD, this will happen automatically since the TDD algorithm is:

  • Write a failing test
  • Write the code to make it pass
  • Refactor
  • Repeat

However, sometimes you need to add more tests to existing code. In that case, a good adaptation to the TDD algorithm is:

  • Comment out the code you’re adding tests to
  • Write a failing test
  • Uncomment just enough code to make the test pass
  • Refactor
  • Repeat

Failure Messages Are Important

GOOS talks about adding another step to the TDD algorithm above:

  • Write a failing test
  • Make the diagnostics clear
  • Write the code to make it pass
  • Refactor
  • Repeat

I hadn’t really thought about this before reading GOOS, but it makes a lot of sense. If I’m only focused on the TDD cycle, I don’t care about the error messages and diagnostics, because I know exactly why the test failed.

However, if this test ever fails in the future, I want the failure message to give me a clue about what’s wrong and how to fix it. Many failure messages don’t do that, so I have to spend a lot of time figuring everything out again.

Don’t Be Scared To Delete Tests

If there are tests that aren’t pulling their weight, don’t be scared to delete them. Make sure the other tests give you enough confidence in your code.

A corollary to this is to be willing to write throw-away tests.

If you’re working on a complicated feature or algorithm, you might write a bunch of low-level tests of private behavior in order to get things working. But once everything’s working, you might be able to cover the behavior with some higher-level tests. At that point, you can delete the low-level tests.

When working with legacy code, you might have to break all the rules and write some really ugly tests just to build a safety net for yourself. Once you have that, you can safely refactor the code into a more testable form. Then you can write better tests and delete the ugly characterization tests.

Wrap-Up

I’ve now given you a high-level overview of my approach to testing. Next post, we’ll start getting into the details and see some concrete examples.