The Ultimate Guide: Test Management for Atlassian Jira

It is not unheard of to manage your test cases in Jira. Issues are a great container to hold test specifications and steps within it.

Traditionally, if you wanted to do test management using Jira, you would have to create new custom fields with test steps and a separate field for requirements. This often feels clumsy and makes people drift away from Jira for test management and just report on test results in Confluence. There are, however, multiple Jira add-ons for making product managers’ and testers’ life easier.

In our experience, the four big outstanders within this sector are Xray by Xpand IT, Zephyr, TM4J by Adaptavist and SynapseRT by Go2Group.

Every plugin has its positives and negatives. We and our customers have had the most success with Xray and that’s what we will focus on today.

What is Xray Test Management for Jira?

Xray is “a complete test management tool for Jira”. This means that a project manager can define Robot Framework test cases that the test developers should start writing. Xray uses the underlying Jira Issues system which means all the usual Jira functionalities like custom fields, comments, JQL and Jira REST API work as they would with everything else in Jira.

Xray issue types

When Xray is installed in Jira, five new issue types are created:

The issue types can be used in Jira projects where those have been added to the issue type scheme. To define which projects within Jira are to be used to store requirements, Jira project admin can enable “Requirement Coverage” in the project configuration view. Requirement issue types are defined in Xray Issue Type Mapping configuration.

Precondition issue type
Defines what must be done before test steps can be executed

Test Plan issue type
Defines what tests should be executed for what version

Test Set issue type
A group of tests, for example all tests related to some application functionality. One test can belong to multiple test sets.

Test issue type
A test case and it includes test steps, actions and expected results from those

Test execution issue type
Represents a single execution of the test. Test executions can be created directly via Jenkins integration

How to use Xray in Jira

Step 1: Installing the Xray plugin

Install the plugin by following these instructions: Xray installation. Without a license, the Xray panels won’t appear in Jira UI so remember to follow the license instructions as well!

The basic usage goes as follows:

The existing tests are structured within a Jira project where Xray is enabled:

Xray also offers reporting tools for tracking requirement coverage, test executions and historical data. Test plan, test execution, test run and traceability reports can also be exported to CSV.

For a deep dive on how to use Xray in Jira, check out this seminar by Xpand IT >

Here are the topics covered:

How do I integrate Xray with Jenkins and Robot Framework?

Installing Xray connector to Jenkins

Integration with Jenkins Documentation Page

Using Jenkins, Xray and Robot Framework together

5. After you have run the build and robot tests, you should go back to Jira and check out the User Story you created. Under that user story, you should find the “test” issue you created, and next to that issue you should see the automatically created issues. Each issue is for one keyword and you can find all the keys in a single robot file under a “test execution”

Would you like to learn more about test automation in software development?

My colleagues at Eficode have written multiple great blog posts on testing in Software world and why it is crucial, here’s for example a blog on (Acceptance) Test-Driven Development, and about test automation maturity self-assessment.

The biggest benefit of having your test cases in Jira is the fact that you can have requirements, bugs, and stories all in one system. With the help of Jira’s powerful search functionality, you will most likely find why and how something was done. By using Jira to also manage your helpdesk, the bug reporting, fixing, and testing will also remain within a single application.

Xpand IT has been working on Xray meticulously and pushing out new features consistently.

We’ll be following Xray closely and keeping an eye out for even more integrations for automated testing and other cost-saving improvements!

The leading DevOps company in Europe, driving the DevOps movement and building the future of software development together across 🇫🇮🇸🇪🇩🇰🇳🇴🇳🇱🇩🇪🇵🇱