Automate Accessibility Testing with Selenium Web Driver (Java)

Accessibility testing using Axe Core with Selenium Web driver on Java!

UX means a LOT! Get rid of the headache of UI A11Y testing via automation with Selenium!

ACCESSIBILITY TESTING, in short A11Y Testing(because it has 11 letters between a & y :)) is a separate section that holds a big responsibility.

What is A11Y Testing?

Ask ten accessibility professionals what accessibility testing is, and, you’ll get ten different answers. Some will emphasize usability. Some will tell you it’s a code quality problem. Others may say it’s all about compliance and check lists. I still find it hard to answer this question.

No, it’s not because we don’t have standards. We have several standards that different nations have come up with. So Accessibility testing is a type of software testing performed to make sure an app complies with recommendations of the so called standards. Segment 508 and WCAG rules are the main standards being used in the world.

Section 508 is the availability standard characterized by the US government, to ensure that all US government sites can be gotten to by people with disabilities.

Web Content Accessibility Guidelines or WCAG characterize the norms for openness for people, associations, and governments around the world. WCAG 2.0 has been acknowledged as an International Standards Organization (ISO) standard, and numerous nations have embraced WCAG 2.0 as their lawful standard for web openness. (WCAG 2.1 available by now)

Why we need A11Y Testing?

As a rule, applications that are used by a wide range of people, including people with disabilities are made to be adapted. Apart from the fact that accessibility allows for the expansion of the number of users of an application, many companies think it is a great marketing move. Accessibility gains companies more trust and in return, more reaches from the outside world.

Accessibility is, usability. I think, accessibility often helps not only people with disabilities, but also any other people under certain circumstances. For example, the one could watch a video with subtitles when it’s noisy and there are no headphones at a hand. Or, alternatively, when there is no opportunity to read something, you could just listen it through headphones. Applications are often adapted for users with hearing and visual impairments, and, less often, for people who are not able to use a keyboard or any other manual input device.

How to test A11Y?

As mentioned before, testers test the apps for the compliance with the selected standards. There are lots of rules in each standard to be checked and often the testers fight on Internet chat channels over whether an element should have been coded as a button or a link! Haha.. This is serious!

There are lots of tools to do the a11y testing now. Each tool gives you the capability to pick a standard or several standard to run a test on the app. And it will generate a report as well. So how about doing this manually every time the app changes.. C’mon! Seriously.. Over and over again! Then what about the reports?

Yes, it’s a pain in the head! B’coz you have to be keen on watching for a11y issues manually. That’s why the Automated A11Y testing has come to the lime light. Thank GOD!

Automated A11Y Testing with Selenium

Okay.. For the automation of a11y testing we will be using Axe Core engine, which is developed by Deque Labs. As told Axe is an accessibility testing engine for websites and other HTML-based user interfaces. It’s fast, secure, lightweight, and was built to seamlessly integrate with any existing test environment so you can automate accessibility testing alongside your regular functional testing.

Automated accessibility testing is a huge time-saver, it doesn’t require special expertise, and it allows teams to focus expert resources on the accessibility issues that really need them. Unfortunately, most accessibility tools are meant to be run on sites and applications that have reached the end of the development process and often don’t give clear or consistent results, causing frustration and delays just when you thought your product was ready to ship.

Axe was built to reflect how web development actually works. It works with all modern browsers, tools, and testing environments a dev team might use. With axe, accessibility testing can be performed as part of your unit testing, integration testing, browser testing, and any other functional testing your team already performs on a day-to-day basis. Building accessibility testing into the early development process saves time, resources, and all kinds of frustration.

Setting up Selenium Test Project for Axe

First we need to download a file from the Axe Selenium Java git repository. Head over here and download the file “axe.min.js”. Then place the downloaded file either anywhere within the project or keep it in “src/main/test/resources” folder.

Now open the POM file and add the following dependency:

<dependency>
<groupId>com.deque</groupId>
<artifactId>axe-selenium</artifactId>
<version>3.0</version>
</dependency>

Now we have done setting up Axe to our test project. So before writing our first accessibility test, let’s go through some information about this plugin.

A11Y Standards on Axe engine

Cypress-axe or the axe-core engine supports following accessibility standards:

WCAG 2.1

WCAG 2.0

Section 508

Apart from that it also have it’s own standards as best-practices, experimental and cat. More information on those can be referred from here.

Axe Commands

Axe have the following commands to do the a11y testing:

This will inject the axe-core runtime into the page under test. You must run this after a call to driver.get() and before you run the AXE.Builder().analyze() command. When using this command you need to pass two parameter, they are the driver and the axe.min.js file.

The current driver and the path to the downloaded script should be passed into the Builder() commad as parameters here. Then this will analyze and check for the accessibility issues on the current page and return a JSONObject including the issues found. You can pass different parameters to the analyze command to fulfill you different needs.

Let’s write our first selenium-java a11y automation script

Are you ready to roll out..? Okay.. let’s go on..

In your test script file define the path to the downloaded “axe.min.js” file like shown in the first line of code below.

private static final URL scriptUrl = I.class.getResource("/axe.min.js"); //path to the axe.min.js file

Then I’m going to write the test as follows:

@Test
public void testOne(){
driver.get("https://www.amazon.com");
AXE.inject(driver, scriptUrl);
JSONObject responseJSON = new AXE.Builder(driver, scriptUrl).analyze();
JSONArray violations = responseJSON.getJSONArray("violations");
if (violations.length() == 0)
{
Assert.assertTrue(true, "No violations found");
}
else {
AXE.writeResults("path & name of the file you want to save the report"), responseJSON);
Assert.assertTrue(false, AXE.report(violations));
}
}

Now run the test and check the path you gave to save the report. You will that a JSON file is created with the given name. If you open it, it will list down:

all the issues found

where the issues are found

how to solve the issue

which accessibility standard is violated

and so on..

There are lot of options that we can pass into Builder().analyze() command which will behave in different ways and give different outputs. Check this file for different usages of the command: HERE!

That’s all! Try it & let me know in the comments section if you have any issues! Happy testing!!!

Senior Quality Engineer @ Wiley