When we started building React/Redux applications at work, we had some ramp-up time on our first project. We put some effort into getting our infrastructure set up as we prototyped various features of the application. We then extracted the infrastructure into our own boilerplate project. As we improve things, we continue to update the boilerplate, so it always reflects our best ideas about how we want to structure our applications.
We started by following the most common advice, which was to use Mocha’s command-line interface (CLI) for running the tests from the command-line and Karma for running in the browser. This worked OK at first, but as we starting building up applications, we ran into some issues. Here are a few of them:
We were unable to test code that used Webpack loaders that Mocha didn’t recognize. We found workarounds for some of these, but not all.
mocha --watchis very CPU intensive and is possibly going to be deprecated.
We were using Karma’s webpack preprocessor so when we ran our tests in the browser, they were being compiled differently than they were when running with the Mocha CLI.
Looking for Answers
We’d finally had enough, so we started to look for better solutions. I spent a bunch of time searching for answers, and ran across a few helpful resources:
First, there’s Webpack’s testing documentation. It is very terse, but contains some enticing nuggets. There wasn’t enough information there to really run with, though.
Those documents reference a mocha-loader that looked promising. The docs for that package are even more sparse than the Webpack testing docs.
Next, I found a StackOverflow answer by Jim Skerritt that provided some additional pieces of the puzzle.
Finally, I stumbled across a new project, mocha-webpack, that looked like it might be a win.
After much experimentation, we came up with a solution that works for us.
The results can be found in the pull request we opened on our boilerplate project.
I’ll walk through the details here.
Here are the main goals we had:
Be able to run tests from the command line or from the browser.
The code in both environments should be compiled the same way as much as possible.
watchoption that doesn’t thrash the CPU.
Allow our tests to work with Webpack loaders, including the ability to test the results of the loaders. For example, we might want to assert things about our CSS Modules-based styles, or about images or other assets.
For running from the command-line, we used the new mocha-webpack package that we found. It’s still a young package, but works well for us.
To use mocha-webpack, we:
mocha-webpack.optsand moved it to the root of our project. Unlike mocha, mocha-webpack doesn’t (yet) support a
--optsflag to specify the file location. A co-worker of mine will be submitting a pull request to rectify that.
mocha-webpack.optsto point at a new test-only Webpack configuration file.
The new Webpack configuration is quite simple:
We simply use our base Webpack configuration, change the target to
node, and re-export that. Our tests use the exact same Webpack loaders that our code does.
package.json file we define a couple of scripts:
With this configuration,
npm test runs our tests once, and
npm run test:watch runs them once and then watches for any changes. Watch mode is interesting. It seems to use Webpack’s watch mode which doesn’t thrash the CPU like Mocha’s does. But also, it rebuilds only what has been affected by a change. Thus, it only runs the tests that recursively depend on the code that was changed. This is a really cool feature that I’ll talk more about below.
Note that mocha-webpack writes output files in a
.tmp directory; you’ll want to make sure you add that to your
.gitignore file or equivalent for your version control system.
How about running the tests in the browser? Sometimes it’s nice to do that when you want to debug something.
We chose to run a second webpack-dev-server on a different port. With hot-reloading turned on, we get a similar workflow to that of
npm run test:watch above.
To make this work, we needed yet another Webpack configuration file. We use mocha-loader in that file:
Once again, we import our base Webpack configuration and modify it a bit. We need to configure something other than the default port for the dev server, and we need to create a new test-only entry point. Make sure that your base Webpack configuration uses an output filename something like
Note that, while mocha-webpack can take a glob expression for finding the test files, webpack itself can’t. We had to create an
index.js for our tests. I found a recipe online and used that:
We name our test files with a
-spec.js on the end, so that’s what we’re looking for. You can change the regex above to match your naming convention.
We add one more script to our
With that, we can run
npm run test:debug and open a browser tab on
http://localhost:8081/test. All of our tests will run using Mocha’s browser UI. With hot loading, every time we change a file, the tests will automatically reload and run again.
npm test:watch above, Webpack’s very nature means that only the tests that were affected by the change are run. To run all of the tests, just refresh the browser tab.
This testing workflow has been a big win for us. We no longer have to work around Webpack loaders that aren’t available in our testing environment.
npm test:watch no longer keeps our laptop fans running all day. We have the ability to easily run and debug our tests in the browser whenever we need.
I often hear people asking, “How can I run only the tests that are affected by the code I just changed?” The reason most people ask this is because their test suites are too slow and they don’t want to wait for the entire suite to run after every change.
The best answer to this question is to keep your test suite fast enough that you don’t notice. This is easier said than done, but just because it might be difficult doesn’t mean it isn’t a goal worth aiming at. So this is always my first advice when I hear this question.
However, because of the way Webpack works, the testing workflow I’ve described here is the second best answer to that question that I’ve ever seen.
In most testing frameworks, you can run the tests that correspond to the file or class you just changed pretty easily. It’s much harder to detect other related tests that might need to run. There’s always a risk of forgetting to run some important test that then breaks your build.
But because Webpack traces dependencies from the entry point all the way down, it automatically knows what tests depend on the code you just changed. That’s the code that it needs to reload via
--watch or via hot-loading, and when it reloads, any reloaded tests are run automatically.
Along the way, we ran into a couple of gotchas that you might need to watch for
Some of our projets are using Webpack’s CommonsChunkPlugin. That plugin doesn’t work in the command-line environment, so we only include it in our production Webpack configuration, not in development or test configurations.
Webpack’s style-loader also doesn’t work in the command-line environment. The recommended work-around is to use the null-loader, but that takes away the ability to test styles, especially when using CSS modules.
There is an open pull request that makes it possible to use the style-loader in the command-line environment; we’ve been using that fix in our code, and it works great for us. Hopefully, it will get merged soon.
Since I originally wrote this post, the situation has improved. See this followup post for more information.