Continuous Integration with Azure build pipeline for Cypress automation testing!

Hello every one, by now I hope all who you are reading this know about Cypress test automation and how cool it is! Yet you might be thinking how can I integrate this automation on Azure DevOps. Oh! I forgot.. or else you might be thinking why the hell do I need Continuous Integration and Continuous Deployment? So.. first things first!

Why do I need Continuous Integration & Continuous Deployment?

Usually, developers write, compile and test code locally and then manually upload the package to the server. However, once the project grows bigger and more developers start to take part in it, problems begin to appear:

  • Deployment preparation is getting more time-consuming as more time is needed for preparation.
  • Software iteration is getting longer with every release, which results in less feedback from the client.
  • Testing is more difficult as every team member has their own development environment.
  • Team members need to communicate more often, increasing the risk of communication failures.

So how can I overcome this mess? The problems above can be successfully addressed by implementing a build server and deployment automation to your workflow.

So when you do Continuous Integration and Continuous Deployment:

  • The software is built and tested in one consistent environment, avoiding dependencies on local dev boxes.
  • You save time on setting up builds, installing required software, etc.
  • Having one centralized build server provides easy access to code metrics, making it easier to properly cover your code with tests.
  • Automatic deployments allow you to release software and receive feedback more often with the whole process simplified to minimum.

Thanks “Buddy!” for the above points. So now you know why we should implement this. Then comes the question HOW? or you might say I tried few time to do this with Cypress but I failed! Don’t worry.. I show you the way!

Setting up JUnit reporter on Cypress project

As per what I have come across, Cypress needs to be configured with the “mocha-junit-reporter” to integrate Cypress on Azure DevOps properly. That is because, Azure DevOps supports only JUnit, NUint, VSTest, XUnit & CTest test result formats.

Let’s setup “mocha-junit-reporter” on our cypress project first. To do so, you need to install two dependencies to your project. Head over to the “package.json” file of your cypress project and add the following under dev dependencies.

"cypress-multi-reporters": "^1.2.4",
"mocha-junit-reporter": "^1.23.3"

Now run the following commands to install the packages:

npm install

Just one more thing to be added to the “package.json” file:

"scripts": {
"scripts": "cypress run",
"test": "npm run scripts"

Okay! Now open the “cypress.json” file and add the following lines:

"reporter": "mocha-junit-reporter",
"reporterOptions": {
"mochaFile": "cypress/reports/junit/test-results.[hash].xml",
"testsuitesTitle": false

In the above snippet, “mochaFile” property directs where your test result files are stored. You might be wondering why it has “.[hash].xml”. Yes keep it as it is. Because when cypress runs multiple test scripts it creates the test result file with the same name. So if you miss-out the “hash” part, it will keep replacing the file after each test script run and at the end you end up with only the test results of the last test script execution in your test suite. By adding the “hash” part it will generate a random hash value and assign it to each test result xml file name, so that the files will not get replaced. So keep in mind not to skip the “.[hash]” part.

Creating a Pipeline for Cypress test execution on Azure DevOps

At this point, I assume you have a Microsoft Azure DevOps account, you have created an agent on Azure DevOps and you have created & pushed your cypress test project to a Azure DevOps git repository (this process is a separate story, if you need help, mention on comments).

So let’s create a pipeline.. Head on to Azure DevOps Pipelines screen.

On the pipelines screen, click on “New Pipeline” button at the top right corner.

Select your option, I’m selecting the first option as I’m using Azure Repos Git. On the next step select the repository where your Cypress test project is pushed in. Then, you will be directed to the Configure pipeline screen like below. Pick Node.js!

Watch the following video before proceeding! After the video I’ll explain what’s happening..

Let me explain what happens now.. First I removed the “npm build” from the first script as we do not need that part and I rename the task name as appropriate. Then you need to add another npm script task to the configuration. To add a npm script click on “Show assistant” button and search “npm” and click add as shown. On the npm configurations, select “custom” and write the command as “run test”.

Then you need to publish test results, to do so you have to add the task “Publish Test Results”. Search from assistant and add it. On configurations:

  • Select the result format as “JUnit
  • Edit and set Test results files to “*.xml
  • For the search folder, give the path to the JUnit test reports folder of your cypress project
  • Don’t forget to tick the option “Merge test results”
  • Give a name to the Test run title

Then click add! One more thing, in the previous npm script we wrote add the following line as shown on video.

continueOnError: true

It’s Done! Save the pipeline & give it a RUN! GOOD LUCK!

Associate Lead QE @ Nagarro