Digital Marketing & Communications

We've seen 1s and 0s you wouldn't believe

Visual regression testing

📥  Communication, Design, Style, content and design, Tools

Our new site consists of 15 different layout templates. Each one of these is further broken down into numerous different design patterns for consistently displaying content. The rules that govern the presentation of these patterns (or elements, if you are familiar with atomic design) are generated from a combination of the Zurb Foundation framework and our own 'Origins' framework - all in all over 2000 rules spanning almost 6500 lines of css.

With this level of complexity it is extremely hard work to track the effect of any changes, almost certainly there is an unexpected knock-on effect of changing, re-specifying, or removing a rule.

Up until now, we have relied on our in-depth knowledge of the site to know where we expect changes to appear, and we use Browserstack to quickly check a representative sample set of our layout templates.

However, this requires a fair whack of time to run, and also needs a person to sit and look at each snapshot that's generated. And they need to know what to look for.

None of that is optimal, we needed a way to automate this process. Enter visual regression testing.

We followed this online guide by Kevin Lamping for setting up a prototype for a visual regression testing framework for Origins and the site templates.

Our prototype is in a repository here: https://github.bath.ac.uk/digital/visual-regression-testing/ (you will need a University of Bath login to view this). It contains a README with instructions on how to get it up and running.

Essentially, what it does is use some clever existing technology (WebDriverIO, WebDriverCSS, graphicsMagick, Selenium standalone and Node) to make a brower load a specific page, take a screenshot of a defined element, and then compare it to a baseline image. If there is any visual changes, it will then create a 'diff' file showing the change - and also alert us by throwing an error.

Snapshot of the website header

First we create a baseline image.

Snapshot of an updated website header

Then, when a change happens, another image is generated and compared against the first.

A diff file of changes to the header

If there are any visual changes, a third file is generated showing these changes.

Currently we are manually running these tests, but ultimately we will integrate into our continuous delivery framework so that the test run automatically whenever a new build of the css is pushed to our staging server.

Pretty neat.

Leave a Reply

  • (will not be published)