When we think of regression testing, we tend to think about functionality and how new code affects the way previously working elements behave. Will a new integration cause something to break? Will everything still work the same after updating backend code?
But just because you’re looking at functionality doesn’t mean you should let visual design testing fall through the cracks during regressions.
In fact, visual regression testing is necessary to make sure that style issues don’t pop up and your web application continues to look just as good as it works.
What is Visual Regression Testing?
Imagine trying to use a web application on your phone and not being able to find certain elements because of an overlapping image or text. Visual testing isn’t just important for making sure that your application looks good but also ensuring that users are able to navigate it without running into style issues that functional testing might pass as technically correct.
But while we might know the need for visual testing, what’s visual regression testing?
While visual testing compares screenshots of a web page in its current state side-by-side with different browser versions of the same page, visual regression testing is used to look at historical differences.
As we know regression testing is used to check that new code has disrupted the functionality of a previous version, visual regression testing allows to do the same with a visual testing tool to make sure that a web page still renders as expected across different browsers after changes to the code.
Many organizations, such as America’s Test Kitchen, use visual regression testing to archive versions of a certain page to record how it changes from iteration to iteration.
With software teams that are shifting left and moving towards more continuous integration and continuous testing, visual regression testing is important to make sure that new changes don’t cause chaos to your layout. This is important to consider as the application evolves from version to version as well as across browsers.
Especially with mobile testing where smaller screen sizes can throw off responsive design and browser rendering, visual regression testing helps spot major differences before your users do.
How Can You Automate Visual Testing?
Tools like CrossBrowserTesting use a screenshot comparison engine to take automated screenshots of the same page across multiple browsers and devices. From here, you can choose a baseline browser, then compare highlighted layout differences side-by-side.
However, this process is not completely automated. Since pages are not passing or failing like in a Selenium test, it’s up to you to decide what differences are acceptable and which aren’t. For example, an image may be off by a few pixels in one browser but not the other. This is probably fine unless it’s throwing off other elements.
The same goes for visual regression testing. CrossBrowserTesting can automatically spin up the recent and previous versions of a web page, but you have to analyze where there are bugs or inconsistencies.
Tools and Tips for Automated Visual Regression Testing
There are a few ways to do visual regression testing that are worth considering.
- PhantomCSS – PhantomCSS is basically a combination of PhantomJS or SlimerJS, CasperJS, and ResembleJS. This tutorial on CSS Tricks gives a good look at how to use PhantomCSS and Grunt for visual regression testing. However, it’s worth noting this requires installation and JavaScript knowledge.
- Wraith – You can also use Wraith, which works by crawling two websites, taking screenshots, and comparing them. Like PhantomCSS, it also does take installation and scripting to use, but Wraith is more useful for testing sites with dynamic content in comparison. You can see this tutorial using Wraith for visual regression testing.
- Selenium – Selenium isn’t traditionally known as a visual testing tool, but that doesn’t mean you there’s no way to take screenshots. Our Selenium 101 tutorial shows you how to take screenshots during automated tests. This way you can have, automated, functional, visual, and browser testing all-in-one.
- CrossBrowserTesting and Applitools – For visual regression testing that requires no installation, setup, or coding knowledge, our integration with Applitools is your best bet. You can run visual comparisons of the same web page on the same browser or mobile device from a previous test session and get alerted of regression errors upon deployment or new releases.
Making Your Website Visually and Functionally Flawless
You don’t have to be shallow to care about how your application looks.
A working web application is important, but to ensure it’s also visually appealing takes its own type of testing.
Visual regression testing allows you to pinpoint every pixel at the UI level so that HTML, CSS, and JavaScript differences don’t affect your user experience.