Software Testing meetup within "Open Technology Week" by Sigma Software, Odesa 2017.

1. RISK BASED TESTING,
WFT?
KATERINA MESHKOVA
ODESSA, SEPTEMBER 7TH 2017

2. ABOUT ME
KATERINA MESHKOVA
Sigma Software fan QA team lead /
business analyst
Sure, ISTQB certified

3. AGENDA
Introduction
Risk based testing overview
Risk based strategies
Product & project risks overview

4. INTRODUCTION: DEFINING
PURPOSES
The purpose of testing could vary
depending on different projects.
However, the most important are
common for all types of projects.

5. INTRODUCTION: WHAT IS RISK?
RISK
A FUTURE
EVENT THAT
May not
happen
May occur –
what is the
effect on
Project scope
Project Schedule
Project Cost
Project Quality

6. INTRODUCTION: POSSIBLE RISKS
Risks
Business
risks
Project
risks
Product
risks
Process
risks
Product/quality – a risk
directly related to test object.
The possibility that the product
or system might fail to deliver
quality attributes
Project risk – a risk related to
management and control of the
project, e.g. lack of staffing,
strict deadlines, changing
requirements, etc.

7. RBT - WHAT IS IT ABOUT?
“ … an approach to reduce the level
of product risks and inform
stakeholders of their status, starting
in the initial stages of a project. It
involves the identification of product
risks and the use of risk levels to
guide the test process.”
RBT also means managing project risks, which are possible events or
outcomes that endanger product quality and successful completion of the
project.

8. DO WE REALLY NEED RISK BASED
TESTING?
AGILE PROJECTS
• Specification issues
TECHNICAL REQUIREMENTS DO NOT
REFLECT BUSINESS VALUE
• Projects with high risks factors

9. HOW TO START RISK BASED
TESTING?
Define responsible person
Business, Development, Test
representatives
Track success and
fail
Be proud of your
fail
Benefits
Review approach

10. LIGHTWEIGHT RISK BASED
STRATEGIES
ADVANTAGES:
1. Easy to understand and
use.
2. Minimum efforts for risk
analysis and documentation.
DRAWBACKS:
1. Could miss some important
points due to limited
functions.
2. In some cases is not
applicable for safety-critical
systems.
PRODUCT
RISK
MANAGEMENT
• Provide risk analysis
visualization on diagram
• PRISMA
[vanVeenendaal12
PRAGMATIC
RISK
ANALYSIS
• Based on RPN calculation
• PRAM [Black09]
SYSTEMATIC
SOFTWARE
TESTING
• Requires Specification
• SST [Craig02]

11. PRISMA IN REAL LIFE
ADVANTAGES:
1. Takes into account more
inputs.
2. Visual representation.
DRAWBACKS:
1. Has limitations in
comparison with heavy-
weight technics

12. HEAVY-WEIGHT RISK BASED
STRATEGIES
ADVANTAGES:
1. The most suitable for
safety-critical systems.
DRAWBACKS:
1. Requires staff training.
2. Requires a lot of efforts to
support process including
documentation.
COST OF
EXPOSURE
• Based on likelihood and
cost of failure
HAZARD
ANALYSIS
• Based on extending
analytical process to
identify the hazards that
underlie risks
FMEA
• Failure mode and effect
analysis
• Based on potential causes
and their likely effects

13. FAILURE MODE AND EFFECT
ANALYSIS
Risk Identification Risk Evaluation Risk Control
Descrip
tion
Failure Mode Effect Cause Impact
(C, M,
m, b)
Detect
ability
(H, M,
L)
Initial
Risk
Risk Mitigation Final
Risk
(H, M,
L)
Risk Verification Final
Accep
tance
(H, M,
L)
Incoming
water
Not applicable,
since the incoming
water is controlled
by city regulation
laws
No
Samp
ling
Unknown,
Since no
sampling
C H H From the midpoint to the
final stages of the
purification process,
there are quality
indicators in place such
as total organic carbon
(TOC), conductivity, and
ultraviolet (UV) data
measurements.
M Verified during PQ
and routine QC
sampling phase (post-
PQ phase, the
moment the system is
accepted or handed
over to operations
within the operational
phase).
L
Filtered
incoming
water –
cartridge
filter
product
water
Equipment not
reducing
impurities per
design
No
samp
ling
Unknown,
Since no
sampling
M H L From the midpoint to the
final stages of the
purification process,
there are quality
indicators in place such
as TOC, conductivity, and
UV data measurements.
L Verified during
routine QC sampling
phase.
L

14. 5 4 3 2 1
Factor Failure mode Failure Mode Failure Mode Effect
Spilling
Salty broth
Rust of hose
connector
Losing shape
of hose
connector
Coming off of
hose
Gas blow
Very Far
Very close Very close Very closeVery close
FAILURE MODE AND EFFECT
ANALYSIS

15. STRATEGY CHOICE
Resources
• Time
• People
• Cost
Project
• Critical-
safety
• E-Commerce
Skills
• Existing
• Possible
Risk
• Types
• Mitigation
strategies

16. RISK MANAGEMENT PROCESS

17. PRODUCT / QUALITY RISKS
EXAMPLES
ID Categories
(ISO 9126)
Specific quality risk Likelihood Impact
Q1.00 Functionality Failures that blocks specific features
Q1.01 New “Sign in” feature will not work for specific
accounts
Medium High
Q2.00 Usability Failures in usability compliance
Q3.00 Reliability Failures in capability of software to maintain its level
of performance under stated conditions for a stated
period of time
Q4.00 Maintainability Failures in maintainability compliance
Q4.01 Migration from 4.5 to 4.6.1 .NET Framework will
cause application failures
High High

18. PROJECT RISKS EXAMPLES
ID Categories Specific project risk Likelihood Impact
P1.00 Planning and
control
Issues and gaps in planning and control that
decrease project productivity
P1.01 Unrealistic schedules and budgeting; artificial
deadlines defined by the customer
Medium High
P2.00 Requirements/
scope
Issues related to requirements quality and
change management procedure
P2.01 Lack of clear shared information about
application real business purposes
Medium High
P3.00 Technology Issues related to usage of new technologies, 3rd
party components and legacy code
P4.00 Team Challenges in people and project management
that could lead to set of negative consequences

19. CONCLUSION
MAIN ADVANTAGES PROVIDED
BY RBT:
• Efficient effort allocation
• Allow to achieve better quality
• Metrics for good enough quality
• Priority-order bug discovery, optimizing
bug-fix procedure
• Expectations management
KEY POINTS FOR SUCCESS:
• Start risk analysis early in the lifecycle
• Include wright people in risk identification
and analysis

20. Q&A
CONTACTS:
Skype: katerina_meshkova
E-mail: kateryna.meshkova@sigma.software