Designing IA for AI - Information Architecture Conference 2024
Xray for Jira - Overview
1. Xray for JIRA Overview
Bruno Conde
Lead Product Developer
Manage Automated and Manual Tests inside JIRA
2. Why JIRA?
“JIRA is the ideal environment for Test Management”
• Project Management
• Defect Tracking (bugs)
• Collaboration / Comments
• Customizable and adaptable
• Advanced Workflow capabilities
• Searching
• Work Assignment
• Tracking changes
• Relating issues using links
• Importing issues from other systems or
formats
Tests
Defects
Requirements
3. ... so what is missing?
• Test related artifacts (tests, executions, plans)
• Visibility of testing results and progress in Agile boards
• Requirement Coverage visibility
• Advanced reporting for generating documents and drill-down
on data
4. Testing with Xray
Specify
• Test
• Pre-Condition
Organize
• Test Set
• Test Repository (v3.0)
Plan
• Test Plan
Execute
• Test Execution
• Sub-Test Execution
• Test Run
6. • Composed by a list of steps
• Each Step is defined by:
• Action/Step
• Data
• Expected Result
• Input Attachments
• Support for JIRA Wiki markup
• Importing steps from
• Excel (using copy&paste)
• CSV
• JSON
• Other Test issues
Specifying “Manual Tests”
7. • Native support for Cucumber
• BDD – Specify tests in a business-readable
domain-specific language
• Test definition serves as documentation
• Automated test steps may be implemented in
Ruby, Java, .Net, etc
• Export specification and execute it during
Continuous Integration
Specifying “Automated Cucumber Tests”
8. Abstracts initial steps that must be ensured before starting the test.
• Manual or Automated (mapped to background clause in case of Cucumber)
• Reusable; may be linked to multiple Tests
Complement specifications with “Pre-Conditions”
9. Xray for JIRA
Organizing your Test Cases
Organize Test Cases using Test Sets or create a hierarchy
10. Flat, with “Test Sets”
Organizing Tests Cases
A issue that represents a list of tests organized
by some logical way.
For the “hierarchy/folder mindset”…
Hierarchical (v3.0)
11. Xray for JIRA
Plan which Test Cases will be included in your next
“release/cycle” and analyze the testing progress
Creating a Test Plan
12. A “Test Plan” has the following goals:
• Specify which Test Cases will be
included in your testing goal (e.g.
the next release)
• Consolidating the latest execution
results for the Tests
Planning with “Test Plans”
14. A task for executing a group of tests on a given version and environment of the
system.
• Contains a list of tests and their results (i.e. test runs)
• Can be labeled, prioritized, assigned to testers and contain components, etc.
• May be created manually or during Continuous Integration
Executing with “Test Executions”
15. • Set Execution Result
• Create Defect issues
• Attach Evidences
• Add comments
• Check the Execution Activity Log
At the execution screen of a Test Run…
17. Supported formats
• Xray JSON
• Cucumber
• Behave
• JUnit
• Nunit
• Robot framework
• Calabash
• … more to come
Integrating with automated test frameworks
18. Xray REST API and CI tools
https://confluence.xpand-addons.com/display/XRAY/REST+API
19. Xray for JIRA
Where do I create Test cases? And what about Executions?
Project Organization Scenarios
20. A single project to manage your Requirements and Defects, Test
related issues and also have all your Test Executions.
All for One and One for All
21. A dedicated project for Xray issue types.
Don't mix my Requirements and Defects with Tests
22. Ok, great… but how can I know the
status of requirements?
Xray for JIRA
23. Requirements with testing information
With Xray:
• Real-time information about the
coverage status of the
requirement for a given version
• Real-time information about the
associated test results, for a
given version
• A ”requirement” may be any
issue type you may have; test
coverage can be made in any
issue type
25. • Built-in reports
• JIRA gadgets in JIRA dashboards
• Agile boards
• Through other top-notch JIRA add-ons
Multiple ways of reporting
26. Provides a quick way of evaluating the current (coverage) status of your project's
requirements, so you will be able to see if they are ready to release or not.
These reports are provided both as native reports as well as Gadgets.
Requirement Coverage Reports
27. Shows the testing traceability, from requirements to defects.
This helps you analyze the requirements and related Tests, Test Runs and
defects, for quickly identifying uncovered or incomplete/faulty requirements.
Traceability Report
28. Xray gives you visibility of your testing results and its progress.
Enhanced Agile Boards with Xray
29. Use Xporter to generate fully customizable Word, Excel or PDF files with your testing
reports to give to your management, colleagues or to your customers
(e.g. Test specification, Test Execution progress and respective results, Test Plan details).
Reports as deliverable assets
32. w w w. x p a n d - a d d o n s . c o m
Thank you!
bruno.conde@xpand-it.com
Editor's Notes
JIRA is used for project management and is also defect tracker tool. There are two areas that are deeply related with QA assurance.
As we all know, JIRA is a great collaboration tool and this is also very important for testing (as an example, please consider for instance the comunication between testing teams and development teams)
One of the main goals of Xray, was always to relate tests with requirements and defects because now they all live in the same platform.
In order to give you a brief explanation of Xray entities we have categorize then into these 4 testing phases
Pros
Self-contained
Easy to understand and manage
Promotes team collaboration
Cons
You may not want to have Bugs, Test Executions in the same project for security concerns
This separation allows to manage permissions more effectively, so maybe only a set of testers has permission to write tests and others to execute.If you already have a JIRA project for managing requirements and defects and don't want to have any tests nor executions in this project, you can create a separate companion project just for testing purposes.
Pros
This separation allows to manage permissions more effectively, so maybe only a set of testers has permission to write tests and others to execute
Keeps your existing requirements and defects project as-is, completely separated from testing related tasks
Isolates Test specification and organization
Keeps sensible data related with bugs and test planning/execution in a different project
Cons
Requires manually synchronizing versions from the requirements project to the other ones
More projects to manage
May promote team separation