The document outlines a half-day tutorial on root cause analysis (RCA) for software testers, presented by Alon Linetzki of Best-Testing. It details his extensive experience in testing and quality assurance, the structure of the course, and the importance of RCA in identifying and resolving underlying issues in software defects. Additionally, it includes a variety of methods, case studies, and guidelines for effectively implementing RCA in testing processes.

1. TP
Half-day Tutorials
5/6/2014 1:00:00 PM
Root Cause Analysis for
Software Testers
Presented by:
Alon Linetzki
Best-Testing
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

2. Alon Linetzki
Best-Testing
Alon Linetzki, founder and managing director of Best-Testing, has been a coach and consultant in
development, testing, and quality assurance for twenty-eight years. Alon helps organizations
enhance the test engineer’s professional and personal skills, training test managers in optimization
and test process improvement, and optimizing their test operations. His main areas of expertise are
in agile testing and transitioning to agile, exploratory testing, test process Improvement, risk-based
testing, and test automation. Alon is a member of the ISTQB® authors and review team for the
ISTQB® Agile Tester Add-on certification, cofounder of ISTQB® in Israel, leader of the ISTQB®
Partner Program, and founder of the SIGiST Israel conference.

3. Root Cause Analysis in Testing

4.  Partners in this course are:
Course Partners
March 2014
2

5. March 20143
 The course materials have been prepared by:
Best – Testing,
 Copyright: © Best – Testing, 2008
 All materials contained in this Handbook were prepared by Best-Testing. All rights of this material are
reserved solely for Best-Testing. The material is intended for personal, noncommercial use. All
materials published in this material are protected by copyright, and owned or controlled by Best-
Testing, or the party credited as the provider of the content. You may not modify, publish, transmit,
participate in the transfer of sale of, reproduce, create new works from, distribute, perform, store on
any magnetic device or any other device, display, on in any way exploit, any of the content in whole
or in part. You may not alter or remove any trademark, copyright or other notice from copies of the
content. You may no use the material in this material for the purpose of training of any kind, internal
or for customers, without beforehand a written approval of Best-Testing.
 The Use of this material
 The material in this material is designed to assist the student during the course. It does not include all
of the information that will be referred to during the course and should not be regarded as a
replacement for reference manuals or other instructions.
Copyrights

6. March 20144
 Limit of responsibility
 Best-Testing invests significant effort in updating this material, however, Best-Testing is not responsible of
any errors or material which may not meet specific requirements. The user alone is responsible for
decisions based on the information contained in this material.
 Protected Trademark
 In this material, protected trademarks appear that are under copyright. All rights to the trademarks in this
material are reserved to the authors.
Best–Testing wishes you success in the course!
Copyrights

7. Introduction
March 2014
5
• No bullets…should be fun…
Chapter 1

8. 6
Alon Linetzki
 30 years in IT, 20+ years as a test engineer and a test manager
 Certified Scrum Master, Scrum Alliance, 2008
 ISQTB® Partner Program leader, ISTQB® Agile Tester
Certification Leader
 Specializes in Agile testing test strategy & optimization
test process improvement  test management  test design
 risk based testing  test automation  Building Smart
Teams
 A degree in ‘testing’ – B.Sc. statistics and criminology,
 International Speaker worldwide, since 1995
 Co-founder of the Israeli Testing Certification Board
(www.itcb.org.il, 2004)
 Founder of the SIGiST Israel (testing forum) in Israel
(www.sigist.org.il, 2000)

9. We shall cover…
Introduction
 Presenting participants and trainer
 Workshop expectations
What is Root Cause Analysis?
How to use RCA for Defect analysis?
7
March 2014

10. We shall cover…
How to Use RCA for analyzing critical problems?
 Cause Effect graphing, amended
 Case Study #0
 Summary
How to use RCA for process improvement?
 Case Study #1
 Case Study #2
Retrospective
 Workshop retrospective
8
March 2014
(*) Bonus topic – only if there is enough time left…

11.  Professional Background
 Years experience in testing/development/both/other
 Responsibilities in current role
 Have you implemented RCA? Can you share?
 Workshop expectations?
Introduce participants…
9
March 2014

12.  Interactive sessions
 Discussions, skepticism, curiosity
 Examples, demonstration
 Thinking, communicating
 Keeping an open mind for ideas and concepts
 Contribute to the class team
 Learn…discuss…review…
 Give quick feedback on everything you may think of
 Time boxed sessions…breaks…exercises…discussions
Introducing workshop dynamics
10
March 2014

13.  There are FAR more slides than we can
cover…
 Say if you have something to say… do not
wait till the end of the workshop!
Introducing workshop dynamics
11
March 2014

14.  Start : 10:15
 Lunch: 11:45 – 12:45
 Finish line: 14:15
But… we may have more
breaks if we feel like it… 
Logistics…
March 2014
12

15. What Root Cause Analysis?
March 2014
13
Chapter 2

16. Root Cause Analysis definition
(My interpretation)
From wiki:
 Root cause analysis (RCA) is a class of problem solving
methods aimed at identifying the root causes of problems
or events.
 The practice of RCA is predicated on the belief that
problems are best solved by attempting to correct or
eliminate root causes, as opposed to merely addressing
the immediately obvious symptoms.
 By directing corrective measures at root causes, it is
hoped that the likelihood of problem recurrence will be
minimized.
14

17. Root Cause Analysis definition
From Bill Willson’s website:
 Root cause analysis (RCA) is a methodology for
finding and correcting the most important reasons
for performance problems.
 It differs from troubleshooting and problem-solving
in that these disciplines typically seek solutions to
specific difficulties, whereas RCA is directed at
underlying issues
15

18. Root Cause Analysis - definition
Root cause analysis (RCA) is a methodology for finding
and correcting the most important reasons for
performance problems.
It differs from troubleshooting and problem-solving in
that these disciplines typically seek solutions to
specific difficulties, whereas RCA is directed at
underlying issues.
16

19. Root Cause Analysis definition -
summary
 As a business process improvement tool, RCA seeks
out unnecessary constraints as well as inadequate
controls.
 In safety and risk management, it looks for both
unrecognized hazards and broken or missing barriers.
 It helps target CAPA (corrective action and preventive
action) efforts at the points of most leverage.
 RCA is an essential ingredient in pointing
organizational change efforts in the right direction.
 Finally, it is probably the only way to find the core
issues contributing to your toughest problems.
17

20. Root Cause Analysis
(My interpretation)
A problem solving method/process designed to search
for the root causes of a problem using a predefined
structural thinking process, identifying the underlying
issues, with the expectation that dealing with these
issues will dramatically reduce the likelihood of the
problem to occur.
The process involves data collection, cause charting, root
cause identification and recommendation generation
and implementation
18

21. “Cows falling on a road from a mountain”
– is it a problem or a symptom?
 Should we eliminate all cows on
that area?
 Should we dig-out the mountain?
 Should we rotate the sign?
 Should we divert the road
elsewhere?
It seems that sometimes
eliminating the causes is not an
easy task, and finding the problems
is even harder!
19

22. Root causes are underlying causes
 The investigator’s goal should be to identify
specific underlying causes
 The more specific the investigator can be about
why an event occurred, the easier it will be to
arrive at recommendations that will prevent
recurrence.
Practical RCA – definition guidelines
March 2014
20

23. Root causes are those that can reasonably be
identified
 Occurrence investigations must be cost beneficial
 It is not practical to keep valuable manpower
occupied indefinitely searching for the root
causes of occurrences…
 Structured RCA helps analysts get the most out of
the time they have invested in the investigation.
Practical RCA – definition guidelines
March 2014
21

24. Root causes are those over which management has control
 Analysts should avoid using general cause classifications such
as “operator error”, “equipment failure” or “external factor”
 Such causes are not specific enough to allow management to
make effective changes
 Management needs to know exactly why a failure occurred
before action can be taken to prevent recurrence
 We must also identify a root cause that management can
influence:
 Identifying “severe weather” as the root cause of parts not
being delivered on time to customers is not appropriate
 Severe weather is not controlled by management.
Practical RCA – definition guidelines
March 2014
22

25. Root causes are those for which effective recommendations can
be generated (if cannot be done, keep on searching!)
 Recommendations should directly address the root causes
identified during the investigation
 If the analysts arrive at vague recommendations such as,
“Improve adherence to written policies and procedures,” then
they probably have not found a basic and specific enough
cause and need to expend more effort in the analysis process.
Practical RCA – definition guidelines
March 2014
23

26.  Root causes are underlying causes
 Root causes are those that can reasonably be
identified
 Root causes are those over which
management has control
 Root causes are those for which effective
recommendations can be generated (if
cannot be done, keep on searching!)
Practical RCA – definition guidelines
March 2014
24

27. Time for discussion…25
March 2014

28.  Root Cause Analysis for Beginners - by James J. Rooney and Lee N. Vanden Heuvel
References – what is Root Cause Analysis?
26
March 2014

29. How to use RCA in Defect Analysis?
March 2014
27

30.  Escaping defects occur all the time
 We try to contain them, from going to production,
but much efforts should be done in preventing
them going from Design to Code
 A set of questions may help out leading us to Data
Collection on those Defects for performing the
RCA – next page…
Defects RCA introduction
March 2014
28

31. Defects RCA guided questions
Data Collection
March 2014
29
Question Who
asks?
Describe circumstances and consequence of the
Problem?
Test Team
Where was the problem introduced? Dev Team
Describe the root cause of the problem Dev Team
What reviews were held for the phase? Dev Team
How and why did the problem escape testing? Test Team
What could be done to prevent this type of error in the
future?
Both
What could be done to find this type of problem in the
future?
Both
John Ruberto – Root Cause for Problems Escape

32.  Defect Analysis is the process of analyzing a defect
to determine its root cause.
 Defect Prevention is the process of addressing
root causes of defects to prevent their future
occurrence.
RCA – few definitions
March 2014
30

33.  Defect Source – Finding defects in the stage they
were introduced and as early in the lifecycle as
possible – Eliminating escaping defects
RCA – few definitions
March 2014
31

34. Defects Root Cause Analysis
32

35. Canceled Defects Root Cause Analysis
 Cancelled defects are not real defects of the
system-under-test
 They can be the result of: environment problem
(non product), out of scope, test design or test
execution problem, un-reproducible defect, not
understanding the specifications, etc.
33

36. Canceled Defects Root Cause Analysis
34

37. Code Inspection - Example
March 2014
35
 Background:
 180 people on the project
 3,700 modules interconnecting with each other
 Complex infrastructure
 25 testers
 Using Test Director/Quality Center as reporting
platform
 Strategic customer

38. Code Inspection - Example
March 2014
36

39. Code Inspection - Example
March 2014
37
 Analysis:
 Main concern – Complience with design (10%
Medium+)
 Investigated RCA:
 A few specs were written by same designer
 A few modules were written by 3 programmers
 Initiated training for designer and programmers

40. Code Inspection - Example
March 2014
38
Background:
Another project,
450 people
working, 55 testers,
4,300 modules,
strategic customer
Focus on:
 Performance
 Maintainability
 Proper algorithm
usage

41. Code Inspection - Example
March 2014
39
 Analysis:
 Investigated RCA:
 Performance defects were on resource usage, memory
leaks
 Implementing improper algorithm was mainly due to
specification documents written unclear + implementation
complexity chosen by a few programmers
 Initiated training for architects and programmers

42.  Adding defect template fields:
 Injected, Detected, Removed
 Version [build] + Date per field
 [Updating the defect lifecycle process (status)]
 Changing field CRUD permissions – Dev, Test,
Managers
 Defining reports
 Updating the Defect Management process
 Training the teams
 Implementing action plan…
Implementation guidelines –
defects RCA
March 2014
40

43. RCA process defined
March 2014
41

44. Time for discussion…42
March 2014

45. John Ruberto – Root Cause for Problems Escape,
http://blog.ruberto.com/2008/04/another-post/
References
March 2014
43

46. How to Use RCA for analyzing critical
problems?
March 2014
44

47. Challenges the current method could
not solve – using 5whys
45

48.  ‘5ys’ or ‘5 whys’ technique, and the cause-effect diagram.
 Presenting a problem,
 Asking “why?” it happens, finding the effect that caused it
(1 effect),
 Presenting the effect on the diagram,
 Asking “why?” it happens… [back to previous step, unless
we ask it for 5 times already]
 Done.
Presenting the 5whys Technique
46

49. ‘5ys’ or ‘5 whys’ technique, and the cause-effect
diagram.
Presenting the RCA Technique
47
Cause
Cause Cause Cause
Cause
Problem
Why
#1
Why
#2
Why
#3
Why
#4
Why
#5
Thinking path…

50. ‘5ys’ or ‘5 whys’ technique, and the cause-effect diagram.
1. There is the assumption that a single cause, at each level
of "why", is sufficient to explain the effect in question.
2. What if one of the ‘Why’ is answered wrongly? Maybe
our answer is possible, but what if the actual cause is
something else entirely?
3. When we have found the problem, and draw the route,
how ‘strong’ is this solution? Maybe we should prefer one
over the other?
Challenges: what the method can
not solve48

51. Enhancing the method – case study
49

52.  Short structured interview with rep’s of management,
development, release, system, testing, product teams.
 Step 1: Draw a cause-effect diagram & exercise the 5whys
 Step 2: Investigate the arrows/causes for:
 What evidence you have that the cause exist? (relevancy)
 What evidence you have that the cause leads to the effect?
(strength)
 Is anything else needed together with the cause for the effect to
occur? (strength)
 Is there evidence that the cause is contributing to the problem
I’m looking at? How much it contributes? (Impact:
direct/indirect)
Enhancing the Method:
Example project50

53. Enhancing the Method:
Example project51
Type Question Cause-Effect
Relevancy What evidence you have that the cause exist? H/M/L
Strength
(S or W)
What evidence you have that the cause leads to
the effect?
H/M/L
Strength
(S or W)
Is anything else needed, together with the
cause, for the effect to occur?
Yes/No
Impact
(D or I)
Is there a evidence that the cause is contributing
to the problem I’m looking at?
Yes / No
Impact
(D or I)
How much this cause is contributing to a
possible resolution?
Direct /
Indirect
Mark

54. Enhancing the Method:
Example project52
Type Question Cause-Effect
Relevancy What evidence you have that the cause exist? High (3)
Strength
(S or W)
What evidence you have that the cause leads to
the effect?
Medium (2)
Strength
(S or W)
Is anything else needed, together with the
cause, for the effect to occur?
Yes (1)
Impact
(D or I)
Is there a evidence that the cause is contributing
to the problem I’m looking at?
Yes (1)
Impact How much this cause is contributing to a
possible resolution?
Direct (2)
Mark 9
 You should mark each arrow using this table.

55.  Step 3: Identify the routes leading to the
problem/s,
 Step 4: Identify the strength and direction (impact)
they have (calculating the mark for each arrow),
 Step 5: Choose the best route to focus on,
 [Improve it, and go to the next one].
Enhancing the Method:
Example project53

56. Case Study - implementation
54

57.  Background:
 company was using a very advanced technology, and
a complex product line,
 Complex product, uses mechanics, electronics,
hardware, software, devices, cooling device, has water
resistant, has heating resistant, accurate up to
1:1,000,000 cm,
 In the last 0.5 year, 50% of released machines
returned from the floor (clients) for fixing,
Example project – Hi-Tech Company
55

58.  SQA manager was at a course I gave, and liked
one of the tools,
 He thought automation can solve many of his
problems, because:
 A lot more tests running,
 Identifying more defects before the clients do,
 Less products coming back,
 Clients are happy!
Example project – Hi-Tech Company
56

59.  I investigated their automation
needs,
 Followed the steps of the
enhanced method,
 Found out their problems might
be elsewhere…
Example project – Hi-Tech Company
57
Lets see the drawing board from
that meeting…

60. 1st drawing – RCA meeting
58
Our way of thinking1
2

61. The RCA meeting (company exec’s and directors):
 At first, the belief was that the primary problem
was:
Partial Test Planning (less tests are executed)
Example project – Hi-Tech Company
59
Lets see an illustration diagram …

62. 1st drawing – RCA meeting
60
Many clients
ask for
different Sw
of the
product
Many
versions
open in
parallel
Complexity of
version control
management is
very high
Defining req’
not good
enough by
client
Spec Lvl 0
No specs
in lvl 1
Spec Lvl
1 not
complete
or does
not fit
Spec Lvl
2 not
written
Good
definition of
Spec Lvl 0
Spec Lvl
1 fits
Spec Lvl
2 fit
Spec Lvl 2
does not
fit/complete
Code
written
with low
match to
client req’
Only
Partial Test
planning
and not full
coverage
Partial test
case planning
and coverage
Partial test
execution
and low
coverage
Our way of thinking1 2

63. 1st drawing – RCA meeting
61
Many clients
ask for
different Sw
of the
product
Many
versions
open in
parallel
Complexity of
version control
management is
very high
Defining req’
not good
enough by
client
Spec Lvl 0
No specs
in lvl 1
Spec Lvl
1 not
complete
or does
not fit
Spec Lvl
2 not
written
Good
definition of
Spec Lvl 0
Spec Lvl
1 fits
Spec Lvl
2 fit
Spec Lvl 2
does not
fit/complete
Code
written
with low
match to
client req’
Only
Partial Test
planning
and not full
coverage
Partial test
case planning
and coverage
Partial test
execution
and low
coverage
Our way of thinking1 2

64. 1st drawing – RCA meeting
62
Many clients
ask for
different Sw
of the
product
Many
versions
open in
parallel
Complexity of
version control
management is
very high
Defining req’
not good
enough by
client
Spec Lvl 0
No specs
in lvl 1
Spec Lvl
1 not
complete
or does
not fit
Spec Lvl
2 not
written
Good
definition of
Spec Lvl 0
Spec Lvl
1 fits
Spec Lvl
2 fit
Spec Lvl 2
does not
fit/complete
Code
written
with low
match to
client req’
Only
Partial Test
planning
and not full
coverage
Partial test
case planning
and coverage
Partial test
execution
and low
coverage
Our way of thinking
1
2

65. 2nd drawing – RCA meeting
63
Many clients
ask for
different Sw
of the
product
Many
versions
open in
parallel
Complexity of
version control
management is
very high
Defining req’
not good
enough by
client
Spec Lvl 0
No specs
in lvl 1
Spec Lvl
1 not
complete
or does
not fit
Spec Lvl
2 not
written
Good
definition of
Spec Lvl 0
Spec Lvl
1 fits
Spec Lvl
2 fit
Spec Lvl 2
does not
fit/complete
Code
written
with low
match to
client req’
Only
Partial Test
planning
and not full
coverage
Partial test
case planning
and coverage
Partial test
execution
and low
coverage
Our way of thinking1 2

66.  After a while, we shifted the focus and agreed that
the real problem was actually:
Poor Product Quality
 Because that was the reason the clients returned
their product.
 And we started RCA from there.
 After a while, we started to see the light – real
problems started to crystallize, problems that
involved people and processes
Example project – Hi-Tech Company
64

67. 3rd drawing – RCA meeting
65

68. Our way of thinking
3rd drawing – RCA meeting
66
Many clients
ask for
different Sw
of the
product
Many
versions
open in
parallel
SCM - Complexity
of version control
management is very
high
Defining req’ not
good enough by
client
Spec Lvl 0
No
specs in
lvl 1
Spec Lvl 1
not complete
or does not
fit
Spec Lvl
2 not
written
Good
definition
of Spec
Lvl 0
Spec Lvl
1 fits
Spec Lvl
2 fit
Spec Lvl 2
does not
fit/complete
Code written
with low
match to
client req’
Only
Partial Test
planning
and not full
coverage
Partial test
case
planning and
coverage
Partial test
execution
and low
coverage
1 2
Tight
schedule
projectPrioritization
and
compromise
on scope to
clients
Low
Quality
Product
Req’
managemen
t not good
enough
Lack of methods
and techniques
in testing
Low lvl of test
identification

69.  We then defined the relevancy, strength and
impact of each arrow (cause),
 And calculated the grades for the arrows (which
are not seen here),
Example project – Hi-Tech Company
67
Back to the board…

70. 4th drawing – RCA meeting
68

71. 5th drawing – RCA meeting
69
Many
clients ask
for
different
Sw of the
product
Many
versions
open in
parallel
SCM - Complexity of version control
management is very high
Defining
req’ not
good
enough by
client
Spec Lvl 0
No specs in
lvl 1
Spec Lvl 1
not
complete
or does not
fit
Spec Lvl 2
not written
Good
definition
of Spec Lvl
0
Spec Lvl 1
fits
Spec Lvl 2
fit
Spec Lvl 2
does not
fit/complete
Code
written
with low
match to
client req’
Only
Partial Test
planning
and not
full
coverage
Partial test
case
planning
and
coverage
Partial test
execution
and low
coverage
Our way of thinking1
2
Tight
schedule
project
Prioritization
and
compromise on
scope to clients
Low
Quality
Product
Req’
management
not good
enough
Lack of methods and
techniques in testing
Low lvl of test
identification
S/D
W/D
W/I
S/I

72.  We went back to double check the RCA of the
routes leading to the primary problem, marking
the arrows with their grades (from the table,
remember?)
 We ended up circling the main causes, that have
initiated the strongest routes that are directly
impacting our problem,
Example project – Hi-Tech Company
70

73. 6th drawing – RCA meeting
71

74. Last drawing – RCA meeting
72

75. Many
clients ask
for
different
Sw of the
product
Many
versions
open in
parallel
SCM - Complexity of version control
management is very high
Defining
req’ not
good
enough by
client
Spec Lvl 0
No specs in
lvl 1
Spec Lvl 1
not
complete
or does not
fit
Spec Lvl 2
not written
Good
definition
of Spec Lvl
0
Spec Lvl 1
fits
Spec Lvl 2
fit
Spec Lvl 2
does not
fit/complete
Code
written
with low
match to
client req’
Only
Partial Test
planning
and not
full
coverage
Partial test
case
planning
and
coverage
Partial test
execution
and low
coverage
Our way of thinking1
2
Tight
schedule
project
Prioritization
and
compromise on
scope to clients
Low
Quality
Product
Req’
manageme
nt not
good
enoughLack of methods and
techniques in testing
Low lvl of test
identification
S/D
W/D
W/I
S/I
Last drawing – RCA meeting
73

76. Many
clients ask
for
different
Sw of the
product
Many
versions
open in
parallel
SCM - Complexity of version control
management is very high
Defining
req’ not
good
enough by
client
Spec Lvl 0
No specs in
lvl 1
Spec Lvl 1
not
complete
or does not
fit
Spec Lvl 2
not written
Good
definition
of Spec Lvl
0
Spec Lvl 1
fits
Spec Lvl 2
fit
Spec Lvl 2
does not
fit/complete
Code
written
with low
match to
client req’
Only
Partial Test
planning
and not
full
coverage
Partial test
case
planning
and
coverage
Partial test
execution
and low
coverage
Our way of thinking1
2
Tight
schedule
project
Prioritization
and
compromise on
scope to clients
Low
Quality
Product
Req’
manageme
nt not
good
enoughLack of methods and
techniques in testing
Low lvl of test
identification
S/D
W/D
W/I
S/I
74
4/6
Last drawing – RCA meeting
Lets see the routes…
3/3

77.  5 major Root Topics were Identified, explained and prioritized:
1. Produce requirements from client definitions
2. Requirements management
3. Either ‘No Spec Level 1’, or ‘Spec level 1 not matching
requirements’
4. Lack of methods and techniques in testing for development
and testing teams
5. Allot of clients define slightly different requirement for the
SW – allot of specials
 We defined a pragmatic corrective actions plan, with priority items.
Example project – Hi-Tech Company
75

78.  Major Areas of Concern identified and prioritized:
1. Requirements Management
2. Configuration Management
3. Design Documentation and Flow
4. Testing Methodologies, techniques and tools
 Not discussed:
- Release Management
- Risk Management + Risk Based Testing
- Requirements Definition
- Project Management
- Professional Development
Example project – Hi-Tech
Company76
Organization
Language!

79. Differentiating problems from
symptoms
77

80.  If we follow the questioning and grading
process, than problems shall be the ‘leafs’
of the cause-effect diagram, with:
 A positive answer to the questions:
 Relevancy – ‘yes’
 Highest strength mark (direct/Indirect),
and
 Direct impact on the undesired
outcome (initial hypothesis)
 Highest grade per route guideline,
 Translate into company language (areas
of concern)
 Enables an Immediate vs. long term
action plan
Differentiating Problems from
Symptoms78
Just what our office needs..

81. Solving problems rather than
covering symptoms
79

82.  Cause-effect & RCA may eliminate the problems of the model
when answering the questions and marking them on the diagram
for the best route(s),
 (but) We must deal with symptoms some time, in the short term,
 We should estimate effort needed (or money required) for each
route, and integrate that into our decision (i.e. 60/40 weight)
 The questions (answers) make sure that our analysis is with
minimal deviations, and that the route we take is a ‘strongest’ one,
 It is a constructive process that leads to Understanding, Clarity,
Focus and the right Priority setting
Summary
80

83.  Further enhancing the mode, we must think of the following:
 What about the junctions points (inbound and outbound):
direct impact of routes with those? Indirect? Impact on speed of
performance (bottle-necks)?
 What is the ROI of this method within context?
 Can we validate a route? Can we tie it to be a successful
problem eliminator?
 How much the method is context dependant?
 Can we hook it to Test Process Improvement methods or other
Key Performance/Area Indicators?
 Other?
Food for Thought…
81

84. Time for discussion…82
March 2014

85. RCA and Process Improvement?
March 2014
83
Introduction
Case Study #1
Case Study #2

86.  Process improvement should use data and RCA
outputs
 CI example… - process was changed, training was
done to relevant people
 Defects example… - lessons learned, training was done
to relevant testers, discussions with development
made it clear which information is required more on
the defect template
RCA and Process Improvement?
March 2014
84

87. Analyzing test environments
utilization downtime
March 2014
85

88.  Analysis:
 Small, yet impacting, breaks in work, caused huge
impact on the release (big team of testers)
 Breaks were 15-45 min each
 Areas of impact were: performance, network, DB
Analyzing test environments
utilization downtime
March 2014
86

89.  Improved process:
 Defined measurements for monitoring downtime
of test environments
 Identify trend of areas of impact on infrastructure
 Used it weekly to establish Status of Downtime, to
decrease technical impacts (reached -25% waste
on next release).
Analyzing test environments
utilization downtime
March 2014
87

90. Analyzing development effort
with defects found
March 2014
88
15
26
11
120
65
43
32
187
12
38
210
294
588
2184
504
210
126
1344
336 378
80%
99%
21%
62%
145%
230%
286%
156%
40%
113%
0%
50%
100%
150%
200%
250%
300%
1
50
SC MAF FBF CSM E-Care E-Support Rater AR Reports BL Interface
Investment vs Defects
Defects Dev Effort Def % vs Dev Eff %
~28 years development.
~150 people in development
team.

91.  Analysis:
 Areas of defects clustering mainly in: Rater, E-
Support, E-Care, AR
 Relative % of defects are much higher than % of
development effort (days)
 Investigated UT and Staging (integration test)
phases, found missing testing types done, UT &
INT regression testing, performance testing,
interface testing
 Further considerations: Complexity, New/Legacy
code, # people touching code
Analyzing development effort
with defects found
March 2014
89

92.  Improved process:
 Defined Unit test and Integration test process and
strategy guidelines and work instructions
 Trained programmers in UT and INT techniques and
new processes
 Aligned effort estimation and duration with project
management and product management
 Defined coverage criteria for UT and INT
 Initiated test automation project on UT and INT levels
Analyzing development effort
with defects found
March 2014
90

93. Time for discussion…91
March 2014

94.  Root Cause Analysis techniques can help us in
finding the underlying problems (root causes), and
get to deal with real problems
 Structured RCA methods support productive
thinking, identifying problems and identify more
symptoms
 Cause Effect Graphing (amended), Defects RCA
 RCA and process improvement
 Summary
Retrospective
March 2014
92

95. “It is not the strongest of the species that
survives, nor the most intelligent but the one
that is most responsive to change”
Charles Darwin
A changing world…
93

96. Or perhaps . . .
94
. . . the one who had anticipated all possible
requirements !

97. Root Cause Analysis in Testing