When we started building React/Redux applications at work, we had some ramp-up time on our first project. We put some effort into getting our infrastructure set up as we prototyped various features of the application. We then extracted the infrastructure into our own boilerplate project. As we improve things, we continue to update the boilerplate, so it always reflects our best ideas about how we want to structure our applications.

Along with other tools, we adopted MochaJs for testing and Webpack for bundling our projects.

The Problem

We started by following the most common advice, which was to use Mocha’s command-line interface (CLI) for running the tests from the command-line and Karma for running in the browser. This worked OK at first, but as we starting building up applications, we ran into some issues. Here are a few of them:

  • We were unable to test code that used Webpack loaders that Mocha didn’t recognize. We found workarounds for some of these, but not all.

  • mocha --watch is very CPU intensive and is possibly going to be deprecated.

  • We were using Karma’s webpack preprocessor so when we ran our tests in the browser, they were being compiled differently than they were when running with the Mocha CLI.

Looking for Answers

We’d finally had enough, so we started to look for better solutions. I spent a bunch of time searching for answers, and ran across a few helpful resources:

First, there’s Webpack’s testing documentation. It is very terse, but contains some enticing nuggets. There wasn’t enough information there to really run with, though.

Those documents reference a mocha-loader that looked promising. The docs for that package are even more sparse than the Webpack testing docs.

Next, I found a StackOverflow answer by Jim Skerritt that provided some additional pieces of the puzzle.

Finally, I stumbled across a new project, mocha-webpack, that looked like it might be a win.

The Solution

After much experimentation, we came up with a solution that works for us.

The results can be found in the pull request we opened on our boilerplate project.

I’ll walk through the details here.

Goals

Here are the main goals we had:

  • Be able to run tests from the command line or from the browser.

  • The code in both environments should be compiled the same way as much as possible.

  • Support a watch option that doesn’t thrash the CPU.

  • Allow our tests to work with Webpack loaders, including the ability to test the results of the loaders. For example, we might want to assert things about our CSS Modules-based styles, or about images or other assets.

Command Line

For running from the command-line, we used the new mocha-webpack package that we found. It’s still a young package, but works well for us.

To use mocha-webpack, we:

  • Renamed our mocha.opts file to mocha-webpack.opts and moved it to the root of our project. Unlike mocha, mocha-webpack doesn’t (yet) support a --opts flag to specify the file location. A co-worker of mine will be submitting a pull request to rectify that.

  • Add a --webpack-config option to mocha-webpack.opts to point at a new test-only Webpack configuration file.

The new Webpack configuration is quite simple:

Webpack Test Configuration
const config = require('./base')

 
config.target = 'node'

 
module.exports = config

We simply use our base Webpack configuration, change the target to node, and re-export that. Our tests use the exact same Webpack loaders that our code does.

In our package.json file we define a couple of scripts:

package.json scripts
{

  "scripts": {

    "test": "mocha-webpack",

    "test:watch": "npm run test -- --watch"

  }

}

With this configuration, npm test runs our tests once, and npm run test:watch runs them once and then watches for any changes. Watch mode is interesting. It seems to use Webpack’s watch mode which doesn’t thrash the CPU like Mocha’s does. But also, it rebuilds only what has been affected by a change. Thus, it only runs the tests that recursively depend on the code that was changed. This is a really cool feature that I’ll talk more about below.

Note that mocha-webpack writes output files in a .tmp directory; you’ll want to make sure you add that to your .gitignore file or equivalent for your version control system.

Browser

How about running the tests in the browser? Sometimes it’s nice to do that when you want to debug something.

We chose to run a second webpack-dev-server on a different port. With hot-reloading turned on, we get a similar workflow to that of npm run test:watch above.

To make this work, we needed yet another Webpack configuration file. We use mocha-loader in that file:

Webpack Browser Test Configuration
const path = require('path')

const config = require('./base')

 
config.devServer = {

  host: 'localhost',

  port: '8081'

}

 
const index = path.resolve(__dirname, '../client/__tests__/index.js')

 
config.entry = {

  test: [`mocha!${index}`]

}

 
config.output.publicPath = 'http://localhost:8081/'

 
module.exports = config

Once again, we import our base Webpack configuration and modify it a bit. We need to configure something other than the default port for the dev server, and we need to create a new test-only entry point. Make sure that your base Webpack configuration uses an output filename something like [name].js.

Note that, while mocha-webpack can take a glob expression for finding the test files, webpack itself can’t. We had to create an index.js for our tests. I found a recipe online and used that:

Test index.js
import './specHelper.js' // if needed

 
const context = require.context('..', true, /.+-spec\.js$/)

 
context.keys().forEach(context)

 
module.exports = context

We name our test files with a -spec.js on the end, so that’s what we’re looking for. You can change the regex above to match your naming convention.

We add one more script to our package.json:

One more package.json script
{

  "scripts": {

    "test:debug": "webpack-dev-server --config webpack/testDebug.js --hot --inline"

  }

}

With that, we can run npm run test:debug and open a browser tab on http://localhost:8081/test. All of our tests will run using Mocha’s browser UI. With hot loading, every time we change a file, the tests will automatically reload and run again.

As with npm test:watch above, Webpack’s very nature means that only the tests that were affected by the change are run. To run all of the tests, just refresh the browser tab.

Big Win

This testing workflow has been a big win for us. We no longer have to work around Webpack loaders that aren’t available in our testing environment. npm test:watch no longer keeps our laptop fans running all day. We have the ability to easily run and debug our tests in the browser whenever we need.

I often hear people asking, “How can I run only the tests that are affected by the code I just changed?” The reason most people ask this is because their test suites are too slow and they don’t want to wait for the entire suite to run after every change.

The best answer to this question is to keep your test suite fast enough that you don’t notice. This is easier said than done, but just because it might be difficult doesn’t mean it isn’t a goal worth aiming at. So this is always my first advice when I hear this question.

However, because of the way Webpack works, the testing workflow I’ve described here is the second best answer to that question that I’ve ever seen.

In most testing frameworks, you can run the tests that correspond to the file or class you just changed pretty easily. It’s much harder to detect other related tests that might need to run. There’s always a risk of forgetting to run some important test that then breaks your build.

But because Webpack traces dependencies from the entry point all the way down, it automatically knows what tests depend on the code you just changed. That’s the code that it needs to reload via --watch or via hot-loading, and when it reloads, any reloaded tests are run automatically.

Gotchas

Along the way, we ran into a couple of gotchas that you might need to watch for

CommonsChunkPlugin

Some of our projets are using Webpack’s CommonsChunkPlugin. That plugin doesn’t work in the command-line environment, so we only include it in our production Webpack configuration, not in development or test configurations.

Style Loader

Webpack’s style-loader also doesn’t work in the command-line environment. The recommended work-around is to use the null-loader, but that takes away the ability to test styles, especially when using CSS modules.

There is an open pull request that makes it possible to use the style-loader in the command-line environment; we’ve been using that fix in our code, and it works great for us. Hopefully, it will get merged soon.

Acknowledgements

Alex Crawford, one of my co-workers at Zeal, worked with me to come up with this solution.

Update

Since I originally wrote this post, the situation has improved. See this followup post for more information.