Getting Testy: Steps

This post is part of an ongoing series about Unit Testing.

Introduction

We’re building a simple command-line application that can do two things:

1. Convert an amount of money from one currency to another.

2. Show a list of all currencies supported by the application.

In the last post we finished test-driving our application from the outside in. We did all of our unit testing in isolation using test doubles. We were pretty careful to make sure that interfaces lined up properly by using instance_double and friends. Now it’s time to see if the application works properly when everything is hooked up.

We started this project by writing some Cucumber specs. Let’s get those running to see if our application is working correctly. As a reminder, here are the acceptance tests we ended up with:

Feature: Currency Exchange
  Allow the exchange of amounts of money from one
  currency to another.
 
  Scenario: basic currency exchange
    Given the exchange rate for 1 USD is 0.91 EUR
    When I convert 100 from USD to EUR
    Then I should get 91.00 EUR
 
Feature: Currency List
  Show the list of supported currencies in alphabetical order
  by currency symbol.
 
  Scenario: currency list
    Given the following currencies exist:
      | symbol | description |
      | USD    | United States Dollars |
      | CAD    | Canadian Dollars |
      | EUR    | European Union Euros |
    When I ask for a currency list
    Then I should see currencies and descriptions in this order:
      | symbol | description |
      | CAD    | Canadian Dollars |
      | EUR    | European Union Euros |
      | USD    | United States Dollars |

Cucumber Step Definitions

We need to implement the step definitions for these tests.

The Given statements in both specs involve injecting currency data into the system. We used VCR in our unit tests to isolate them from the real external service. That seems like a good approach here as well for the same reasons. We don’t want our Cucumber specs to fail due to service outages or constantly updating data.

The first decision we need to make is how we’re going to run our application.

We want these specs to be as end-to-end as possible, which suggests that we should shell out to our main executable file and capture the output using %x{bin/currencyfx} or similar. However, VCR won’t work this way - it doesn’t capture HTTP interactions that come from a forked process like this.

There are some ways we could work around this issue, but let’s not go there unless we later decide we need to.

The fallback position is to use our CLI object like we did with the unit tests. This will run the application in-process allowing VCR to do its thing, but still be very close to the outermost layer of our application.

Running the Application

Let’s start with the When steps. That’s where we’ll run the application, and also where we’ll get the VCR cassettes in place.

The first When step is for performing conversion.

When(/^I convert ([\d.]+) from (\w{3}) to (\w{3})$/) do |amount, source, target|
  VCR.use_cassette("open_exchange_rates/rates") do
    @output = run_application(amount, source, target)
  end
end

I’m not that fussy about the regular expressions I use in my Cucumber steps; I’d rather something simple that generally works than the perfect regex that no one can understand.

We’re assigning the returned output to an instance variable so that our Then steps can examine the output and make assertions about it. It’s not a good idea to overuse instance variables in Cucumber steps, but we need some way of capturing the output for the Then steps to check.

Rather than immediately figuring out how I’m going to run the application and capture its output, I’m writing this step as if there were already a run_application method that does exactly that.

This is a technique known as test first, by intention. The technique is described very nicely by Ron Jeffries, Ann Anderson, and Chet Hendrickson in their book, Extreme Programming Installed.

We can now write the run_application method that we wish already existed.

def run_application(*args)
  capture_output { Currencyfx::CLI.run(args) }
end

Once again, we can write the code assuming that there’s a handy capture_output method available for our use. We’re using our CLI object to do the work and using a splat to collect all of the arguments into an array to pass along, which is what will happen when the real application uses ARGV.

Finally, we have to write capture_output.

def capture_output
  original_stdout = $stdout
  $stdout = StringIO.new
  yield
  $stdout.string
ensure
  $stdout = original_stdout
end

This technique of capturing output isn’t really the point of this post, but it’s a reasonably common pattern.

We can write the When step for the currency list feature using the same helper methods.

When(/^I ask for a currency list$/) do
  VCR.use_cassette("open_exchange_rates/currencies") do
    @output = run_application("--list")
  end
end

If we comment out the pending markers in our other step definitions, we can run these specs and see where we’re at.

Integration Errors

We immediately get an error: NoMethodError: undefined method '/' for "":String coming from our OpenExchangeRates class. We’ve written all of our code to expect a number for amount, but our Cucumber step is passing in a string.

This is why it’s a good idea to have a few end-to-end tests to make sure everything is hooked up correctly.

We need to figure out where to fix this. Then we can write a lower-level unit test and fix the bug. We have a few layers we can choose from:

• The Cucumber step
• The CLI object
• The Exchange object
• The OpenExchangeRates object

When we run the program by hand from the command line, the amount is going to be a string just like it is here. So the Cucumber step is doing the right thing and has caught a real bug that would affect the application in normal use.

It’s generally best to do all input sanitizing and normalizing at the system boundaries. That way, the rest of the code can be written confidently with the assumption that it will be called with the correct arguments. Avdi Grimm’s excellent Confident Ruby book (highly recommended) has a great section title called “Guard the borders, not the hinterlands”. By that guideline, the CLI object is the right place for this work. Let’s look at our CLI specs again:

context "when exchanging currency" do
  let(:arguments) { [100, "USD", "EUR"] }
 
  it "displays the converted amount and currency" do
    allow(exchange).to receive(:convert).with(100, "USD", "EUR") { 91.87 }
    expect { run_cli }.to output(/91.87 EUR/).to_stdout
  end
end

This spec is written to pass in a number, not a string. Let’s change that and see if we can get the unit test to fail.

context "when exchanging currency" do
  let(:arguments) { %w[100 USD EUR] }
 
  it "displays the converted amount and currency" do
    allow(exchange).to receive(:convert).with(100, "USD", "EUR") { 91.87 }
    expect { run_cli }.to output(/91.87 EUR/).to_stdout
  end
end

Here, we’ve updated arguments to take all strings using Ruby’s word array (%w) syntax. Note that we don’t update the allow statement; we’re saying that the CLI will take a string but will still pass a number to the exchange. That’s exactly the behavior we want.

This spec now fails because exchange is receiving a message with unexpected arguments. We update the code to convert the amount to a Float and try again. The test passes and we can jump back out to the Cucumber level.

Checking the Output

Let’s flesh out our Then steps. We’ve captured the program output into the @output variable and we can use RSpec expectations to check it.

Then(/^I should get ([\d.]+) (\w{3})$/) do |amount, currency|
  expect(@output).to match /#{amount} #{currency.upcase}/
end

We don’t care about the exact format of the output; we just want to be sure that it gives us the correct converted amount and the destination currency. We can use the match matcher and construct a regex out of the provided amount and currency.

Running the spec gives an error because our expected amount doesn’t match. This is because we’re not yet injecting the exchange rate into the VCR cassette, so we’re just getting back whatever the exchange rate in the cassette happens to be.

Let’s write our other Then step and then deal with the injection part.

This step is a bit trickier, because we need to take a table of expected symbols and descriptions and match that against the output. The step also says that order matters, so we can’t just do a bunch of separate assertions.

Here’s one way to do it.

Then(/^I should see currencies and descriptions in this order:$/) do |table|
  expected_output = table.raw[1..-1].map { |row| row.join("\\s+\\|\\s+") }.join("\\s+\\|.*\\|\\s+")
  expect(@output).to match(/#{expected_output}/m)
end

I’m not particularly pleased with this code, but I don’t have a better option yet. We’re converting the table into a multi-line regular expression that is then matched against the output.

To build the regex, we skip the header row of the table with table.raw[1..-1]. For each row, we expect the symbol to appear, followed by any amount of whitespace, followed by a vertical bar (|), followed by more whitespace, followed by the description. We then join the rows by expecting more whitespace and another vertical bar to finish off the current row, any number of intervening rows, and then a vertical bar and whitespace to start off the next expected row.

This spec also fails, but this time because our injected currency descriptions don’t match the ones returned by the external service.

It’s time to figure out the injection issue.

Injecting Data

In order to keep these specs robust and repeatable, we need to be able to inject fake data to be returned by the external service. As with the specs we wrote for the OpenExchangeRates API, there is a bit of danger here because we’re not using the external service directly. However, it would be difficult to write a meaningful fast, robust spec that used the real external service.

Once again, we’ll take advantage of VCR’s ability to use ERB templates in the response.

Starting with the currency conversion test, we’ll edit our saved cassette to replace the exchange rate for Euros with an ERB variable, and then use the erb option to inject the exchange rate.

Given(/^the exchange rate for 1 USD is ([\d.]+) ([A-Z]{3})$/) do |dest_amount, dest_currency|
  @cassette_options = { erb: { dest_currency.downcase.to_sym => dest_amount } }
end
 
When(/^I convert ([\d.]+) from (\w{3}) to (\w{3})$/) do |amount, source, target|
  VCR.use_cassette("open_exchange_rates/rates", @cassette_options) do
    @output = run_application(amount, source, target)
  end
end

In the Given step, we create a Hash of options to be passed into the VCR cassette. In the When step, we use those options to perform the ERB substitution.

Running this spec results in a new failure, because our output came back as 91.0 EUR, not the 91.00 EUR that we were expecting. Yet another bug caught by our Cucumber specs.

We’ll fix that in a moment; first, let’s finish fleshing out our steps.

In order to inject the currency list, we have to do more table conversion work and this time, it’s all going to have to be done in the ERB template.

First, let’s look at the steps.

Given(/^the following currencies exist:?$/) do |table|
  currencies = table.raw[1..-1]
  @cassette_options = { erb: { currencies: currencies } }
end
 
When(/^I ask for a currency list$/) do
  VCR.use_cassette("open_exchange_rates/currencies", @cassette_options) do
    @output = run_application("--list")
  end
end

Again, we’re stripping off the header row of the table. We then inject the raw table into the ERB template using the @cassette_options instance variable.

I haven’t been showing the captured VCR responses, but this time I will so you can see what the ERB template code looks like.

body:
  encoding: UTF-8
  string: |-
    {
      <% currencies.each_with_index do |(symbol, description), index| %>
        "<%= symbol %>": "<%= description %>"<%= index == currencies.size - 1 ? "" : "," %>
      <% end %>
    }

This is mostly simple ERB code, but the logic to place or not place the trailing comma is a bit ugly. We’ll live with it for now. currencies is a Hash, and we’re using Ruby’s destructuring feature to split each entry into its key and value (symbol and description).

With this code in place, the currency list spec is now passing correctly. Let’s go tackle our failing conversion spec now.

Fixing the Formatting Bug

The formatting bug isn’t really an integration bug; we just caught it because we used a nice round number in our spec.

What happened is that we missed a couple of cases when we wrote the unit tests for the CLI object. Let’s go correct that oversight now.

The obvious case we missed is when there are no cents in the converted amount. But this also makes us think of the case where there are fractional cents in the converted amount. We need to handle both cases.

After writing tests for the new cases and doing a bit of refactoring, we end up with this spec.

context "when exchanging currency" do
  let(:arguments) { %w[100 USD EUR] }
  let(:converted) { 91.87 }
 
  before do
    allow(exchange).to receive(:convert).with(100, "USD", "EUR") { converted }
  end
 
  it "displays the converted amount and currency" do
    expect { run_cli }.to output(/91.87 EUR/).to_stdout
  end
 
  context "when the converted amount has fractional cents" do
    let(:converted) { 91.86598 }
 
    it "rounds the converted amount to the nearest cent" do
      expect { run_cli }.to output(/91.87 EUR/).to_stdout
    end
  end
 
  context "when converted amount has no fractional amount" do
    let(:converted) { 91 }
 
    it "still displays two decimal places" do
      expect { run_cli }.to output(/91.00 EUR/).to_stdout
    end
  end
end

Notice how I’ve extracted a let for the converted amount, allowing each context to override that to show exactly what’s different about that context with a minimum amount of noise and duplication.

I could have wrapped the first it block in its own context, but I couldn’t think of a good when statement for it, so I left it alone.

Both of these new specs fail. After a few simple changes in CLI, we get them both passing. We then pop back out to the Cucumber specs and they are all passing as well.

We now have a functional command-line application that implements both of our features. Time to celebrate a bit!

Changes

OK, celebration over. Changes are coming.

With this currency exchange example, I’ve demonstrated the approach I prefer to take when test-driving my applications. I’ve explained much of my reasoning along the way, but let’s perform a couple of thought experiments regarding potential changes to this application.

New External Service

If we need to switch to a new external service in order to get our exchange rate data, we’d need to:

1. Test-drive a new subclass of API for the new service. This subclass needs to implement the two methods defined by the abstract API base class.

2. Modify the CLI’s default API to point at the new API.

3. Capture a set of VCR cassettes for the new API to use in the Cucumber specs. The specs themselves won’t change, but we’ll have to modify the step definitions to use the new cassettes. Depending on how different the new cassettes are from the current ones, we might have to tweak the ERB substitutions a bit.

That’s it.

The executable doesn’t change.

The Cucumber specs don’t change; the step definitions get a bit of tweaking, but probably not much.

The CLI object only changes to point at the new API subclass.

Exchange and the API base class don’t change at all.

We add new code and specs for the new API subclass.

We add new VCR cassettes for the new API.

None of the existing unit tests change.

New Front End

If we need to replace our command-line front end with a web-based front end, we’d need to:

1. Test-drive the web front end, using a test double of the Exchange class. These tests would be similar in spirit to those for the CLI object.

2. Modify the executable file to launch the server instead of running the command-line application.

3. Modify the Cucumber step files to access the web front-end instead of running the CLI. The When and Then steps are what would likely need to change.

Again, that’s it.

The executable changes to start a server instead of running a command-line.

The Cucumber specs don’t change; the step definitions do.

We add new code and specs for the web front-end.

Exchange, the API class, and its subclasses don’t change.

The VCR cassettes don’t change.

None of the existing unit tests change.

Overall, the code is pretty resilient to change. More importantly, the tests are very resilient to change. We have to add specs for new classes, but we are not changing any of our unit tests or Cucumber specs to support these new features. That gives us a lot of confidence that the changes we make don’t break anything.

Conclusion

This concludes the currency exchange example.

Throughout the example, I’ve been showing the development of the two features in parallel. In a real development workflow, I’d have built the entire currency conversion feature outside-in first, then I’d have come back and built the entire currency list feature outside-in.

I developed them in parallel during this series just so I could illustrate the various layers with a couple of examples.

I’ve made the code available on GitHub. Feel free to play with it. In order to run the actual application, you’ll have to get your own API key for the Open Exchange Rates API. See the README file for more details.

I have a few more testing topics to cover in the upcoming posts.