Getting Testy: Outside-In

This post is part of an ongoing series about Unit Testing.

Introduction

In the previous post, we wrote learning tests to figure out how to work with Ruby’s built-in OptionParser class to handle command-line arguments.

Now it’s time to start implementing our application.

We’re building a simple command-line application that can do two things:

1. Convert an amount of money from one currency to another.

2. Show a list of all currencies supported by the application.

We may add a web front-end to this later or completely replace the command-line interface with a web interface.

Testing the CLI

Earlier, we wrote some acceptance tests that form the outermost layer of our test suite. Now we’re going to peel back the onion and work our way into the core of the application. We’re working from the outside of the application in towards the center. This is known as an outside-in approach.

The next layer of the application is the command-line interface (CLI). Because we want our tests to run as fast as possible, we’d prefer not to test the CLI by running it in a separate process and capturing the output; our Cucumber step definitions will likely do that. Instead, we want to run the tests in the same process as the CLI.

The way to do that is to make the executable a very thin wrapper over an object that we can test directly. In a fit of originality, we’ll call that object CLI.

We’ll use RSpec this time.

Simple First Test

Let’s start with a simple test just to get everything hooked up. We’ll start by checking that calling our CLI with no arguments results in a usage message being printed.

module Currencyfx
  RSpec.describe CLI do
    subject(:cli) { described_class.new }
 
    context "with no arguments" do
      let(:arguments) { [] }
 
      it "displays usage information" do
        expect { cli.run(arguments) }.to output(/Usage/).to_stdout
      end
    end
  end
end

This is a pretty basic test, but the main thing to note is that we’re testing only externally-visible behavior. We’re not testing anything internal. Most notably, we’re not directly testing the OptionParser that this code uses internally. We learned how to use that class when we wrote our learning tests, but now it’s just an internal implementation detail that we want to hide away. The parts of it that we care about will be tested through the public interface of the CLI object. If we later want to use something else to handle the command-line, we can swap out our existing implementation with no changes to the specs.

We’re also not testing the exact format of the output. As you may recall from the learning tests, the usage output was one of the most brittle parts of the option parser, so we don’t want to couple our tests to that too heavily.

There are a few other minor things to note:

• I’ve embedded the outer describe block in the same module that contains the code I’m testing; that way, I can refer to any classes in that module without having to prefix them with the module name.

• I’ve introduced a let for the arguments we’re going to pass to the CLI object. I talked about this in an earlier post. I like to have the let definitions in each context show exactly what’s special about the context.

• We’re using RSpec’s output matcher to capture and test the output.

Currency Conversion

Let’s move on to a more involved test now. What happens when our CLI is asked to convert an amount from one currency to another?

What should that look like on the command-line? This is the main use case of our system, so let’s make that as easy as possible and just take three positional arguments: the amount, the source currency, and the target currency. A typical invocation might look like currencyfx 100 USD EUR.

Let’s start with what we’d like to see:

context "when exchanging currency" do
  let(:arguments) { [100, "USD", "EUR"] }
 
  it "displays the converted amount and currency" do
    # Somehow inject an exchange rate
    expect { run_cli }.to output(/91.87 EUR/).to_stdout
  end
end
 
def run_cli
  cli.run(arguments)
end

Notice that I’ve now extracted a helper method, run_cli, to actually run the application. This removes noise and duplication from the tests, allowing me to focus more on the important parts.

In its current state, this test obviously won’t pass. Where does this magic 91.87 come from? This is where we need to allow the specs to drive our design.

Following the approach that Sandi Metz advocates in her book, let’s think about the messages we need.

We really only need a single message, maybe named convert, that takes the amount and the two currencies and returns the converted amount.

Now that we’ve identified the message, we need to think about what object we should send the message to. We don’t have any other objects besides our CLI so we probably need a new one. Let’s call it an Exchange.

We don’t really know how Exchange will work; we just know what message we want to send it and what kind of response we need back. That’s enough information to finish our current test if we use a test double.

We need to identify what kind of message we’re talking about. If we follow the command-query separation approach, there are only two choices:

• Queries return a result, but do not change the observable state of the system (no side effects).

• Commands change the state of the system, but do not return a value.

In our case, the convert method returns a value and has no observable side-effects, making it a query message. According to Sandi Metz’ approach, we don’t need to test outgoing query messages; however, we may need to stub them in order to test the behavior of our object.

That’s certainly true in our case; we need to stub out the convert method in order to get back our magic 91.87 value. Let’s see what that looks like:

context "when exchanging currency" do
  let(:arguments) { [100, "USD", "EUR"] }
 
  it "displays the converted amount and currency" do
    allow(exchange).to receive(:convert).with(100, "USD", "EUR") { 91.87 }
    expect { run_cli }.to output(/91.87 EUR/).to_stdout
  end
end

In RSpec, stubs are defined using allow. I’ve chosen to be explicit about the arguments I’m expecting to be sent to the stub as well.

Now we have to define what exchange is and how the CLI gets to know about it. That happens up at the top of our spec; we’ll use a pattern called dependency injection to tell the CLI about the exchange.

RSpec.describe CLI do
  subject(:cli) { described_class.new(exchange: exchange) }
  let(:exchange) { instance_double(Exchange) }
 
  # ...
end

In order to define a test double in RSpec, you can use the double method or one of its siblings. In this case, I’m using the instance_double method and passing in the Exchange class. instance_double creates a verifying double which only allows us to mock and/or stub methods that are implemented by the class we’re doubling. This helps us avoid the problem of our real interface getting out of sync with our tests. I use verifying doubles whenever I can.

We’re injecting the exchange using a keyword parameter. In order to avoid extra pain for our clients, we can make it an optional keyword with the default value being an instance of the real exchange class.

module Currencyfx
  class CLI
    def initialize(exchange: Exchange.new)
      @options = Options.new
      @exchange = exchange
    end
  end
end

In order to get past the verifying double, we also have to add a skeleton version of the Exchange class.

module Currencyfx
  class Exchange
    def convert(amount, source, target)
      fail "Test-drive this!"
    end
  end
end

For now, we don’t need to think about the implementation of this class at all; I’ll often use a raise or fail that will remind me to come back and implement this method. Even better is to use some kind of marker method to make it easy to search for any work we still have left to do.

For now, we can stay focused on the CLI class. Once it’s done, we can start fleshing out Exchange using TDD.

Currency List

We can use the same approach to test-drive the currency list feature of the CLI. Once again, we need to decide what the command-line interface should look like. Let’s just provide a simple flag option; when it is present, we’ll print out a list: currencyfx --list.

We also need to decide what messages we need to send for this feature.

We need to ask some object for the list of supported currencies, maybe by sending a currency_list message. We’ll expect back some kind of currency list; it might be an Array or other Enumerable, or it might be a custom CurrencyList object. We can decide that later. Once again, we look at the objects we have: CLI and Exchange. Should either of these handle the currency_list method? I think it makes sense for the Exchange object to handle it; it needs to know about currencies in order to convert them, so it would make sense for it to know about what currencies are supported.

Once we get back the currency list, we need to print it out. That seems like it might be a CLI responsibility. It certainly doesn’t belong to the Exchange; that class shouldn’t be responsible for the format of the output.

As we start to think about the CLI tests we’ll need to write for the printing, though, they seem kind of involved. And testing the different cases by changing the stubbed return value of the currency_list method seems tedious. It seems like maybe printing is a separate responsibility that belongs somewhere else.

Let’s implement a message named format that takes a currency list and returns a string containing the formatted output. None of our existing objects seem to be the right place for this behavior, so let’s introduce a new class named ListFormatter.

Once again, we can use a verifying test double, dependency injection, and a skeleton implementation to write the spec.

module Currencyfx
  RSpec.describe CLI do
    subject(:cli) { described_class.new(exchange: exchange, list_formatter: formatter) }
    let(:exchange) { instance_double(Exchange) }
    let(:formatter) { instance_double(ListFormatter) }
 
    context "when asked for a currency list" do
      let(:arguments) { %w(--list) }
      let(:currency_list) { double("currency list") }
 
      before do
        allow(exchange).to receive(:currency_list) { currency_list }
        allow(formatter).to receive(:format).with(currency_list) { "FORMATTED LIST" }
      end
 
      it "displays the currency list" do
        expect { run_cli }.to output("FORMATTED LIST\n").to_stdout
      end
    end
  end
end

Both messages are query messages again, so we just need to stub them out.

We’re using a non-verifying double for the currency list; we don’t care what it actually is here, as long as the same object that comes back from the currency_list method gets passed along to the formatter. By using a simple double, we can decouple this test from that decision, because it really doesn’t matter here. We don’t want this test to change or break when we later decide what a currency list is.

Next Steps

There are more specs we’d want to write for the CLI: what happens when errors occur? Do we exit with correct error codes? What happens if both the --list and the conversion parameters are supplied? I’ll leave those as an exercise for now, but we may talk about them again later.

Once we’ve finished testing the CLI, we can go ahead and delete the learning tests we wrote for the OptionParser. We learned what we needed to learn and captured the important parts in our CLI tests.

We also have to test-drive the implementation of the Exchange and ListFormatter classes. We’ll talk about the Exchange in the next post. The ListFormatter is a simple object with a single format method that will probably be a pure function; we’ve already talked about testing those, so I won’t explore that further here.

Conclusion

I hope this gives you a good idea about how an outside-in testing process might look.

Not everyone likes this style of testing. Test double-based testing can be done very badly, resulting in brittle tests that are painful to write and expensive to maintain.

Avdi Grimm has been doing a series of Ruby Tapas screencasts on “Mocking Smells”. In this series, he outlines many of the bad ways of doing mock-based testing and how to do it right. Ruby Tapas is generally awesome and worth subscribing to, but this series in particular is pure gold. Highly recommended.

Avoiding Smells

I find that I can avoid most of the mocking smells by doing the things we’ve talked about here:

• Test only the externally-visible behavior of the object; don’t couple the tests to internal implementation details.

• If I feel like I need to test internal details, there’s probably an object that is trying to get out. I need to pay attention to that.

• If I feel like I have to write a lot of tests using an inconvenient API, that’s also a sign of a design problem. Either the API is wrong, or there’s another object that wants out.

• Once I identify the need for a collaborator, I focus on the messages first. I want to find a nice high-level message that communicates what I want done, but not how to do it.

• After I identify the message, I think about what object needs to respond to that message. I try not to constrain my thinking to the objects I already have. I’m not scared to introduce new objects if it improves my design.

Benefits

By following the outside-in approach the way I’ve outlined it here, I gain a number of benefits:

• I can focus on one object at a time. I can test a single object’s responsibilities in isolation without worrying about the details of other objects. As the need for other objects arise, I write a quick skeleton of them, but I don’t need to think about them until I get to them.

• Once I move down to the next layer, the skeleton classes I’ve built act as a natural to-do list. I know exactly what the public API of these objects needs to be, so I know what I need to test.

• It’s much easier to write tests for my objects. I don’t need to do a bunch of complicated setup for the collaborators, and I can easily test weird corner cases and error conditions using mocked or stubbed methods on the collaborators.

• I don’t spend a lot of time building up lower-level objects that I end up not needing. When I develop inside-out (or bottom-up), I have some ideas about what I’ll need at the higher levels, but those often turn out to be wrong.

• My objects have nice, high-level APIs that specify “what”, not “how”. That makes them very understandable and easy to re-use.

• I end up with a lot flexibility. For example, if we decided we wanted to support a number of different formats for the currency list feature, we could turn ListFormatter into an interface (or role) and have a number of different implementations. Whatever formatter we like could be plugged into the CLI object using the optional keyword parameter we introduced.

• I maintain ease-of-use for the common case. The injected constructor parameters are all optional with sensible defaults; the normal user of the CLI object does not need to supply any additional parameters.

• I end up with a lot of small, single-purpose objects that are easy to understand and easy to wire together in different ways. The Single Responsibility Principle emerges naturally.

If you’re not used to testing this way, I recommend trying it out on a small side project to see what you think.