Visual testing: obstacle that every tester had so far..

It’s a common obstacle that every tester has, to check the UI manually. The tester has to have a keen eye, in order to point out the tiny alignment changes at the front-end elements. And it’s a test case that misses the test automation scope as it is impossible to do with regular test automation aspects. In order to address this problem, visual testing came up with Image Processing!

What is Visual Testing?

Visual testing is how you ensure that your app appears to the user as you intended!

In today’s world, in the world of HTML, web developer create pages that appear on a mix of browsers and operating systems. Because HTML and CSS are standards, frontend developers want to feel comfortable with a ‘write once, run anywhere’ approach to their software. Which also translates to “Let QA sort out the implementation issues.” QA is still stuck checking each possible output combination for visual bugs -definition by Applitools

Hope now you have an idea what visual testing is.. right? Then..

Why do Visual Testing?

OMG..! This is in production! How did they miss it? They would probably have run test automation and all the elements and texts are there in the DOM, so the test has passed. But visually, can you let this pass?

The tests are passing because traditional assertions would never catch this. So if you are not doing visual testing, you are missing bugs.. expensive and obvious ones. That’s why we must move on with Visual Testing.

Setting up Visual Testing on Cypress

Cypress Image Snapshot Plugin

Installation

npm install --save-dev cypress-image-snapshot

Then add the following line to the index.js file in plugins folder within cypress folder

const {
addMatchImageSnapshotPlugin,
} = require('cypress-image-snapshot/plugin');

module.exports = (on, config) => {
addMatchImageSnapshotPlugin(on, config);
};

Then add the following lines to the commands.js file in support folder within cypress folder

import { addMatchImageSnapshotCommand } from 'cypress-image-snapshot/command';

addMatchImageSnapshotCommand();

Syntax of the above command is as follows:

addMatchImageSnapshotCommand();
addMatchImageSnapshotCommand(commandName);
addMatchImageSnapshotCommand(options);
addMatchImageSnapshotCommand(commandName, options);

Now you can use the plugin as follows:

cy.matchImageSnapshot();

If you do not like command term you can change it too! As the syntax shows, open the commands.js file and change the last line you added as follows..

addMatchImageSnapshotCommand('validateUI');

Now you can use it as follows:

cy.validateUI();

Cool right? Okay, apart from that you can send options to the command. Using the options you can change behavior of the image comparison such as image comparison threshold.

Options

customSnapshotsDir : Path to the directory that snapshot images will be written to, defaults to <rootDir>/cypress/snapshots

customDiffDir : Path to the directory that diff images will be written to, defaults to a sibling __diff_output__ directory alongside each snapshot

The local options in cy.matchImageSnapshot() will overwrite the default options set in addMatchImageSnapshot

Following are the default options used in commands.js

addMatchImageSnapshotCommand({
failureThreshold: 0.03, // threshold for entire image
failureThresholdType: 'percent', // percent of image or number of pixels
customDiffConfig: { threshold: 0.1 }, // threshold for each pixel
capture: 'viewport', // capture viewport in screenshot
});

Usage

One!

cy.matchImageSnapshot()

This will take a screenshot of the page and the screenshot name will be the test title. If there is already a snapshot available with the name, the plugin will match the images and create a diff image if there are any differences.

Two!

cy.matchImageSnapshot(‘login’)

This will take a screenshot of the page and the screenshot name will be the name passed in. The rest of the process is as same as above.

Three!

cy.matchImageSnapshot(options)

Here the plugin will behave according to the options object passed in

Four!

cy.get(‘elementLocator’).matchImageSnapshot()

Okay.. this is what I wanted in my tests! This is take a screenshot of the particular web element you passed in. The screenshot name will be the test title. If there is already a snapshot available with the name, the plugin will match the images and create a diff image if there are any differences.

jeez.. you’ve learned a lot! The last one was much useful personally for me in my test automation scripts. Finally apart from the options learned so far, there are two more important options.

Updating Snapshots

Run Cypress with --env updateSnapshots=true in order to update the base image files for all of your tests

Preventing test case failures due to visual testing

Run Cypress with --env failOnSnapshotDiff=false in order to prevent test failures when an image diff does not pass. This will neglect the image diff but it will store all the diffs as usual.

Phew..! Have a breath now! That’s all you have to keep in mind!

See ya on another story..

Associate Lead QE @ Nagarro