Snapshot Testing has been getting a fair bit of attention recently with some new tool support that makes it easy to use. But is that a good thing?
This has brought increased attention to a feature that was introduced in version 14, Snapshot Testing.
Totally digging this snapshot thing from Jest! Woot! pic.twitter.com/iPqj7QhL9a— @kentcdodds
What is Snapshot Testing?
Snapshot Testing in Jest was designed to make it easy to test React components, but it can be used for more than that. In recent days, I’ve seen tweets talking about using it to test Redux reducers and API calls among others.
The basic idea is that you can write a test like this (example copied from the Jest v14 announcement blog post):
The first time the test is run, Jest will capture a JSON representation of the rendered component into a snapshot file. During future runs, the test will compare the latest JSON with the snapshot file. If it is the same, everything is fine. If not, the test fails. You then need to inspect the differences and decide whether the new output is correct. If it is, you update the saved snapshot. If it isn’t, you fix your code.
Updating snapshots is easy. You either run
jest -u from the command line, or in watch mode, press the
Why Is This Cool?
With a simple API and easy-to-update snapshots, it’s much faster to write tests for things that can be notoriously hard to test. Rather than figuring out how to pick apart a somewhat complex structure, you can just capture a snapshot instead.
Jest manages the snapshot files for you. You don’t have to worry about naming files, opening them, running diffs, etc. As long as you remember to check them into your version control system, Jest handles everything for you. This is a huge win.
Is It New?
This kind of testing has been around for a while, and is more commonly known as Golden Master Testing.
There are other libraries that support this style of testing, notably Katrina Owen’s wonderful Approvals gem for Ruby.
Another name for this kind of testing is the anti-pattern known as Guru Checks Output.
Anti-Pattern? That’s Bad Isn’t It?
I wrote about Golden Master Testing as an anti-pattern in my Getting Testy series.
In particular, I noted:
The biggest problem with manual verification is that it requires a human in the loop. This slows down the process, and is also potentially error-prone. What if the only human available is not as experienced at looking at the output and misses something critical?
What if we need to rush a hot-fix into production? That’s the time when any human involved is likely to be the most stressed and rushed. That’s not when we’re likely to do our best at looking carefully at changed output.
The ability to quickly update a snapshot when it changes makes for a nicer, faster workflow. But it also makes it easy to accept new snapshots that have problems.
Indeed Christoph Pojer, one of the main contributors to Jest, recently warned of this danger.
Another potential issue with snapshot testing is that it can make for fragile tests. For example, a test might be concerned with only a few details of the captured output, but the test will fail when any detail of the captured output changes.
Finally, when writing Snapshot Tests that are intended to stick around, you need to be careful to write really solid test descriptions, because the body of the test no longer communicates anything about what’s important about the test case.
So You’re Saying I Shouldn’t Use It
I’m not saying that you should never use Snapshot Testing. I’m certainly not saying that it isn’t a worthwhile feature to have in Jest.
If it helps you write valuable tests that you wouldn’t write otherwise, I’m all for it.
There are contexts in which Golden Master Testing is the best tool for the job. The most notable example is when testing legacy code.
With legacy code, especially when writing characterization tests, you care more about whether the output has changed and less about whether it’s correct.
In those contexts, having good tools to help with the process is essential, and Jest’s Snapshot Testing feature is a great tool that is really well-executed.
What I am saying is that you should be aware of the potential dangers and downsides of Golden Master Testing and choose wisely.
It is definitely worth experimenting with Jest’s Snapshot Testing feature. See what it’s good at and what it’s not so good at.
Pay attention to whether future you can understand the tests when you come back to them. Notice when you get test failures for unimportant details. See if you or your teammates accidentally update snapshots that shouldn’t have been updated.
But also pay attention to what benefits you gain from it. Can you now test something you didn’t know how to test before? Does your development cycle speed up, both short-term and long-term? Most importantly, is your software better at delighting your users?
As with any development tool or approach, keep your eyes open and think about the consequences, both good and bad, of your decisions.