Learn how to leverage risk based testing with JIRA to prioritize the testing of features and functions based on risk of failure, function of importance and likelihood or impact of failure.

1. September 25, 2015
Introduction to QASymphony
for [INSERT COMPANY
NAME]
#AgileTransformation
Agile Transformation: People,
Process and Tools to Make
Your Transformation Successful
Using JIRA for Risk
Based Testing
#RiskBasedTesting

2. #JIRATesting @RyanYackAttack
Test Case Management | Exploratory Testing | Test Analytics | BDD and more...
ABOUT QASYMPHONY
The #1 Enterprise Testing Platform for

3. #AgileTransformation
OUR PRESENTER
Ryan Yackel
Senior Product Engineer
QASymphony
@ryanyackattack
#RiskBasedTesting
• Serves as the “customer voice” for all
QASymphony products.
• Comes to QASymphony from Macy’s,
where he managed testing on large
enterprise initiatives.
• Certified scrum master from Scrum
Alliance and holds a Bachelor of Arts
degree from Covenant College.
• Has a newborn baby, and doesn’t sleep a
ton :-)

4. The Test Everything Mentality
Traditional vs Risk Based Testing
How can JIRA be used to track risk
Use Case of JIRA + qTest Products
Recommended Add-ons to try out
WHY ARE WE HERE?
4 #RiskBasedTesting

5. #AgileTransformation
SOFTWARE TESTING IN A RISKY
BUSINESS
5 #RiskBasedTesting

6. #AgileTransformation
PRESSURE IS ON TEST TEAMS
6 #RiskBasedTesting
What’s the plan to
test X new
requirements?
What’s the plan to
test this one
requirement, that
affects 50
applications?
I guess I could
create X tests,
multiplied by the
100
configurations
I guess I could
create 1,000
tests, multiplied
by the 50
configurations

7. #AgileTransformation
TIME IS OF THE ESSENCE
7 #RiskBasedTesting

8. #AgileTransformation
TEST EVERYTHING
88 #RiskBasedTesting

9. #AgileTransformation
WHAT DO I TEST?
99 #RiskBasedTesting
Do we need to test
everything?....If not, what
can I cut from my test
plans?
When should my testing
be complete?
How do I get team
convinced that we are
limited in test?
Where am I going to
get all my resources for
this?

10. #AgileTransformation
TRADITIONAL vs RISK BASED TESTING
1010 #RiskBasedTesting
TRADITIONALRISKBASED
TESTING

11. September 25, 2015
Introduction to QASymphony
for [INSERT COMPANY
NAME]
#AgileTransformation
Agile Transformation: People,
Process and Tools to Make
Your Transformation Successful
11
RISK BASED
TESTING (RBT)
#RiskBasedTesting

12. #AgileTransformation
WHAT IS RISK?
12 #RiskBasedTesting
Risk = Damage *
Frequency
Risk = Impact *
Probability of Failure
A Risk is a problem or situation that has not happened yet and it may never
happen in future as well. It is a possible issue or problem that may occur.
DEFINITIONS
Michael Scott poorly shoots a three pointer (Impact) * Everytime he gets
the ball (Frequency) = EXTREMELY HIGH Risk

13. #AgileTransformation
TWO TYPES OF RISK ANALYSIS
1313
Statistical Risk Analysis
a) Weighted Metrics
b) Color Coding
c) P1 - P4 Areas
Heuristic Risk Analysis
a) Consider more possibilities
and interesting aspects of
the problem
b) Prioritize a List of Risks
DistributedPopular
NewComplex
Down Stream
Strategic
#RiskBasedTesting
(Daniele Gagliardi – eclipsecon Europe 2015)

14. #AgileTransformation
STATISTICAL RISK ANALYSIS
1414 #RiskBasedTesting
(Sticky Minds Risk-Based Testing: Test Only What Matters)
Marginal impact—1 Moderate impact—
2
Medium impact—3 High impact—4 Critical impact—5
A cosmetic error. The progress was
disrupted with
manageable
extensions to the
short-term schedule
and cost; customers
were not affected
The progress was
disrupted with large
extensions to project
schedule and cost;
customers were not
directly affected
An application or
transaction caused a
significant disruption
to production
The problem caused
all major and critical
functioning of the
application to fail.
This resulted in a
large loss of
revenue.
Marginal—1 Moderate—2 Medium—3 High—4 Critical—5
Low complex code
written on low
complicated system
architectural design
Moderate complex
code written on
moderate
complicated system
architectural design
Medium complex
codes written on
medium complicated
system architectural
design
There is a large
amount of complex
code written on a
highly complicated
system architectural
design
This suggests
extreme complex
code that has been
written on top of on
extremely
complicated
architectural design
IMPACT
PROBABILITY OF FAILURE

15. #AgileTransformation
STATISTICAL RISK ANALYSIS
1515 #RiskBasedTesting
- P1 – Critical Impact: Must be tested;
ideal candidate for automation
(candidate for automated smoke
testing)
- P2 – High Impact: Should be tested
- P3 – Medium Impact: Can be tested if
budget and schedule permits
- P4 – Low Impact: May not be tested;
no impact on application and no need
of automation
(Sticky Minds Risk-Based Testing: Test Only What Matters)

16. #AgileTransformation
HEURISTIC RISK ANALYSIS: INSIDE OUT
1616 #RiskBasedTesting
- Vulnerabilities: What
weaknesses or possible failures
are there in this component?
- Threats: What inputs or
situations could there be that
might exploit a vulnerability and
trigger a failure in this
component?
- Victims: Who or what would be
impacted by potential failures
and how bad would that be?
Grab a developer and ask “What can go wrong here? and here? and …” :-)
(James Bach Heuristic Risk-Based Testing)

17. #AgileTransformation
HEURISTIC RISK ANALYSIS: OUTSIDE IN
1717 #RiskBasedTesting
(James Bach Heuristic Risk-Based Testing)
E
D
F
A
B
C
Complex: anything disproportionately large,
intricate, or convoluted.
New: anything that has no history in the
product.
Changed: anything that has been tampered
with or "improved".
Upstream Dependency: anything whose
failure will cause cascading failure in the rest
of the system.
Downstream Dependency: anything that is
especially sensitive to failures in the rest of
the system.
Critical: anything whose failure could cause
substantial damage.
Risk Lists

18. #AgileTransformation
HEURISTIC RISK ANALYSIS: IMPACT
1818 #RiskBasedTesting
Component Risk Matrix
Component Risk Risk Heuristics
Advanced Shipping Normal Complex, popular
GUI Higher New, complex, third-
party, critical
Manifest Lower Legacy, changed,
Point of Sale Lower Legacy, no change
Purchasing Higher Upstream, Critical
Receiving Normal Recent Failure
Shipping Normal Downstream

19. #AgileTransformation
PLAN OF ATTACK
1919 #RiskBasedTesting
RISK IDENTIFICATION
RISK ANALYSIS
TEST PLANNING
TEST EXECUTION
TEST RESULT
P1 P2 P3 P4 P5
Exploratory Testing

20. #AgileTransformation
THE GOAL OF RBT
2020 #RiskBasedTesting
GOAL
Assurance of Quality
CostofFailure
CostofTesting
(Sticky Minds Risk-Based Testing: Test Only What Matters)
• Not Everything
Needs to be Tested
• Finding Balance the
cost of testing and
Failures with Quality
Assurance
• Taking Calculative
Risk and Test

21. September 25, 2015
Introduction to QASymphony
for [INSERT COMPANY
NAME]
#AgileTransformation
Agile Transformation: People,
Process and Tools to Make
Your Transformation Successful
21
USING JIRA TO
MANAGE RISK
#RiskBasedTesting

22. #AgileTransformation
FLEXIBLE AND CONFIGURABLE
2222 #RiskBasedTesting
SOFTWARE
DEVELOPMENT
ISSUE TRACKING
RISK MITIGATION
SOFTWARE
DEVELOPMENT
MARKETING GO TO
MARKET
SOFTWARE
DEVELOPMENT
MARKETING GO TO
MARKET
PERSONAL TASK
MANAGEMENT
HR ON BOARDING

23. #AgileTransformation
CONFIGURING JIRA ISSUES
23 #RiskBasedTesting
Option 1: Create New “Risk” Issue

24. #AgileTransformation
CONFIGURING JIRA ISSUES
24 #RiskBasedTesting
Create Custom Statistical Fields
a) Probability of Failure
b) Impact
c) Risk Analysis
Custom Heuristic RISK Fields
a) Risk Heuristics
b) Heuristic Analysis
Option 2: Custom Risk Fields

25. #AgileTransformation
RISK WITH JIRA ISSUES
25 #RiskBasedTesting
Issue Search Analysis
Statistical HeuristicCommon

26. #AgileTransformation
RISK WITH JIRA ISSUES
26 #RiskBasedTesting
Board Analysis

27. #AgileTransformation
CATAGORIZE RISK FOR TEST
27 #RiskBasedTesting
Organize Risks for Test

28. #AgileTransformation
PLAN AGAINST RISKS
28 #RiskBasedTesting
Map High Priority Tests w/ High Risk

29. #AgileTransformation
EXECUTE YOUR AGAINST RISKS
29 #RiskBasedTesting
Coordinate Line of Priority for Execution

30. September 25, 2015
Introduction to QASymphony
for [INSERT COMPANY
NAME]
#AgileTransformation
Agile Transformation: People,
Process and Tools to Make
Your Transformation Successful
30
JIRA
ADD-ONS
#RiskBasedTesting

31. #AgileTransformation
RISK REGISTER
31 #RiskBasedTesting

32. #AgileTransformation
VISUALIZER FOR JIRA
32 #RiskBasedTesting

33. #AgileTransformation
SCRIPT RUNNER FOR JIRA
33 #RiskBasedTesting
JIRA Misc Custom Fields ScriptRunner for JIRA

34. September 25, 2015
Introduction to QASymphony
for [INSERT COMPANY
NAME]
#AgileTransformation
Agile Transformation: People,
Process and Tools to Make
Your Transformation Successful
34
WRAPPING
IT UP
#RiskBasedTesting

35. #AgileTransformation
PROS AND CONS
35 #RiskBasedTesting
Management might think you are lazy
Miscalculate or mis-appropriate weight to a
certain risk
Coordinating risk identifiers is a challenge
The impossible could happen
High severity defects are flushed out
Low values tests are kicked to curb
Critical areas are tested first
Confidence is boosted
Deadline anxiety is removed

36. #AgileTransformation
RESOURCES
36 #RiskBasedTesting
JIRA Add-Ons
• Risk Register - https://marketplace.atlassian.com/plugins/com.projectbalm.riskregister.riskregister-jira/server/overview
• Visualizer for JIRA - https://marketplace.atlassian.com/plugins/vfjc/cloud/overview
• ScriptRunner - https://marketplace.atlassian.com/plugins/com.onresolve.jira.groovy.groovyrunner/cloud/overview
• JIRA MISC Custom Fields - https://marketplace.atlassian.com/plugins/com.innovalog.jmcf.jira-misc-custom-
fields/server/overview
References
• Sticky Minds Risk-Based Testing: Test Only What Matters
• Xoriant – An Approach for Risk Based Testing
• James Bach - Heuristic Risk-Based Testing
• Daniele Gagliardi – eclipsecon Europe 2015

37. Thank You
#AgileTransformation
Thank you
#RiskBasedTesting