Automated Accessibility Testing using AXE 4 with Selenium + Java

Accessibility testing using new Axe 4.0 with Selenium Web driver on Java!

UX means a LOT! Get rid of the headache of UI A11Y testing via automation with Selenium!

ACCESSIBILITY TESTING, in short A11Y Testing(because it has 11 letters between a & y :)) is a separate section that holds a big responsibility.

What is A11Y Testing?

Ask ten accessibility professionals what accessibility testing is, and, you’ll get ten different answers. Some will emphasize usability. Some will tell you it’s a code quality problem. Others may say it’s all about compliance and check lists. I still find it hard to answer this question.

No, it’s not because we don’t have standards. We have several standards that different nations have come up with. So Accessibility testing is a type of software testing performed to make sure an app complies with recommendations of the so called standards. Segment 508 and WCAG rules are the main standards being used in the world.

Section 508 is the availability standard characterized by the US government, to ensure that all US government sites can be gotten to by differently abled people.

Web Content Accessibility Guidelines or WCAG characterize the norms for openness for people, associations, and governments around the world. WCAG 2.0 has been acknowledged as an International Standards Organization (ISO) standard, and numerous nations have embraced WCAG 2.0 as their lawful standard for web openness. (WCAG 2.1 available by now)

Why we need A11Y Testing?

As a rule, applications that are used by a wide range of people, including differently abled people are made to be adapted. Apart from the fact that accessibility allows for the expansion of the number of users of an application, many companies think it is a great marketing move. Accessibility gains companies more trust and in return, more reaches from the outside world.

Accessibility is, usability. I think, accessibility often helps not only differently abled people, but also any other people under certain circumstances. For example, the one could watch a video with subtitles when it’s noisy and there are no headphones at a hand. Or, alternatively, when there is no opportunity to read something, you could just listen it through headphones. Applications are often adapted for users with hearing and visual impairments, and, less often, for people who are not able to use a keyboard or any other manual input device.

How to test A11Y?

As mentioned before, testers test the apps for the compliance with the selected standards. There are lots of rules in each standard to be checked and often the testers fight on Internet chat channels over whether an element should have been coded as a button or a link! Haha.. This is serious!

There are lots of tools to do the a11y testing now. Each tool gives you the capability to pick a standard or several standard to run a test on the app. And it will generate a report as well. So how about doing this manually every time the app changes.. C’mon! Seriously.. Over and over again! Then what about the reports?

Yes, it’s a pain in the head! B’coz you have to be keen on watching for a11y issues manually. That’s why the Automated A11Y testing has come to the lime light. Thank GOD!

Automated A11Y Testing with AXE 4.0

For the automation of a11y testing we will be using Axe Core engine, which is developed by Deque Labs. As told Axe is an accessibility testing engine for websites and other HTML-based user interfaces.

Axe have come a long way since its beginning and the latest version of Axe is 4.1.2 as of 23rd Feb 2021. Deque Labs did major changes when moving forward from Axe 3.0 to 4.0 to ease the use of their library. One major change is, for versions older than 4.0 needed to store the Java Script file used to inject to the page separately as a resource file within the java project, but with 4.0 onwards they have included the Java Script file within the library which has ease out lot of hassle.

Automated accessibility testing is a huge time-saver, it doesn’t require special expertise, and it allows teams to focus expert resources on the accessibility issues that really need them. Unfortunately, most accessibility tools are meant to be run on sites and applications that have reached the end of the development process and often don’t give clear or consistent results, causing frustration and delays just when you thought your product was ready to ship. Axe works with all modern browsers, tools, and testing environments which might be used by a dev team.

Setting up Selenium Test Project for Axe

Assume you have created a maven project and added the required dependencies for Selenium test automation. Now open the pom.xml file of the maven project and add the following dependency:

<dependency>
<groupId>com.deque.html.axe-core</groupId>
<artifactId>selenium</artifactId>
<version>3.1-SNAPSHOT</version>
<scope>test</scope>
</dependency>

That’s it! Now you are good to do some scripting. Before that I’ll walk you through some things you might need to know before.

A11Y Standards on Axe engine

Cypress-axe or the axe-core engine supports following accessibility standards:

WCAG 2.1

WCAG 2.0

Section 508

Apart from that AXE also has it’s own standards defined like best-practices, experimental and CAT. More information on those can be referred from here.

Axe Commands

Axe have the following commands to do the a11y testing:

AXE.injectAxe

This will inject the axe-core runtime into the page under test. You must run this after a call to driver.get() and before you run the AXE.Builder().analyze() command. When using this command you need to pass two parameter, they are the driver and the axe.min.js file.

AXE.Builder().analyze()

The current driver and the path to the downloaded script should be passed into the Builder() command as parameters here. Then this will analyze and check for the accessibility issues on the current page and return a JSON object including the issues found. You can pass different parameters to the analyze command to fulfill your different needs.

Let’s write our first selenium-java a11y automation script

Are you ready to roll out..? Okay.. let’s go on..

I writing my first test to navigate to the amazon.com and check the accessibility issues in their page. The code goes as follows:

@Test
public void testOne(){
driver.get("https://www.amazon.com");
AxeBuilder builder = new AxeBuilder();
Results results = builder.analyze(webDriver);
List<Rule> violations = results.getViolations();
if (violations.size() == 0)
{
Assert.assertTrue(true, "No violations found");
}
else
{
JsonParser jsonParser = new JsonParser();
Gson gson = new GsonBuilder().setPrettyPrinting().create();
AxeReporter.writeResultsToJsonFile(reportFile, results);
JsonElement jsonElement = jsonParser.parse(new FileReader(reportFile + ".json"));
String prettyJson = gson.toJson(jsonElement);
AxeReporter.writeResultsToTextFile(reportFile, prettyJson);
Assert.assertEquals(violations.size(), 0, violations.size() + " violations found");
}
}

This code will automatically inject the JS file and check for the accessibility issues using ALL possible standards on the page. If it finds any issues it will log all the issues to a JSON file and also a text file it the given directory. If you open it, it will list down:

all the issues found

where the issues are found

how to solve the issue

which accessibility standard is violated

and so on..

There are lot of options that we can pass into Builder().analyze() command which will behave in different ways and give different outputs. Check this file for different usages of the command: HERE!

Testing with Specific Accessibility Standards

Okay.. Let’s say you want to do the check with only WCAG 2.0 Level AA standard.

First add this line of code as a global variable and replace with the accessibility standard you want test. If needed you can specify a specific guideline of a standard as well like wcag241 here.

private static List<String> tags = Arrays.asList("wcag2aa");

And now you can use builder.withTags() to set it up as follows:

@Test
public void testOne(){
driver.get("https://www.amazon.com");
AxeBuilder builder = new AxeBuilder();
builder.withTags(tags);
Results results = builder.analyze(webDriver);
List<Rule> violations = results.getViolations();
if (violations.size() == 0)
{
Assert.assertTrue(true, "No violations found");
}
else
{
JsonParser jsonParser = new JsonParser();
Gson gson = new GsonBuilder().setPrettyPrinting().create();
AxeReporter.writeResultsToJsonFile(reportFile, results);
JsonElement jsonElement = jsonParser.parse(new FileReader(reportFile + ".json"));
String prettyJson = gson.toJson(jsonElement);
AxeReporter.writeResultsToTextFile(reportFile, prettyJson);
Assert.assertEquals(violations.size(), 0, violations.size() + " violations found");
}
}

Have a kick start with this framework code I have developed using Java Selenium with AXE 4.1.2 >

thecharmingweirdo/axeaccessbilitytestautomation: This repository contains a Selenium Java Test Automation framework integrated with AXE library for accessibility test automation (github.com)

Have a kick start with this framework code I have developed using Java Selenium with AXE 4.1.2 : GitHub Repository

That’s all! Try it & let me know in the comments section if you have any issues! Happy a11y testing!!!

Senior Quality Engineer @ Wiley

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store