Automated Regression Testing for Pattern Libraries

The other week we had a personal development day at Rareloop and we each spent a Friday day working on something we’d wanted to experiment with. I decided to take a look at automated front end regression testing and to see if we could get something working with our Pattern Library software Primer.

If you’re not familiar with what ‘Automated Regression Testing’ is, it’s really not as scary as it sounds. Think of a time you’ve made a small change to a CSS file, done some testing, pushed to the live website and then later happened upon some un-wanted consequence elsewhere on the site, or at a viewport you hadn’t managed to test. Now imagine, you had some automated process to catch these mishaps for you before you go live – thats what Automated Regression Testing aims to do (there’s more info on the subject over at CSS Tricks).

To me, a Pattern Library like Primer feels like the perfect place to run these tests, not only do you have all the full templates but you also have access to all the individual components too.

There are quite a few options available for testing front end codebases but I decided to have a play with Backstop.js as it had a well documented setup process and lots of flexibility with regards the viewport sizes to test against. After quite a bit of experimenting, Primer Backstop was born. The package includes a simple plugin for the Primer CLI that wraps around Backstop and makes it easy to create reference images and then test against them. The wrapper also knows about the elements/components/templates in Primer and automatically generates the required Backstop rules for them.

I can’t take any credit for the actual comparison or reporting during the test, thats all Backstop, but its great to have a drop in tool that works well with Primer. It’s still a little rough around the edges and hasn’t been battle tested on a live project yet but the code is on GitHub. Give it a go and let me know if you have ideas for ways to improve it.