EuroSTAR Software Testing Conference 2012 presentation on Agile Testing Isn’t Risking It by Bram Bronneborg. See more at: http://conference.eurostarsoftwaretesting.com/past-presentations/

2. Agile Testing isn’t
Risking IT!
Wednesday 7 november 2012
Eurostar 2012

3. B.W.F.P.M. BRONNEBERG
TEST MANAGER / CONSULTANT
REQUIREMENT & QUALITY MANAGEMENT
© All rights reserved
BRAM.BRONNEBERG@CGI.COM
WWW.BRAMBRONNEBERG.NL
@BRAMBRONNEBERG
+31 6 55 11 35 10

4. Introduction
Agile
Testing
isn’t
Risking IT!
What is it all
about?
Which
problems
did I face?
How to do it
right?
Q & A
© All rights reserved 4

5. Introduction
Agile
Testing
isn’t
Risking IT!
What is it all
about?
Which
problems did
I face?
How to do it
right?
Q & A
© All rights reserved 5

6. Agile in a nutshell
Product Owner Team Scrum Master
© All rights reserved
6
Product
Backlog
Scrum
24 u
Sprint
Backlog
Sprint
2-4 wk
Demo
Planning Meeting
Retrospective

7. Risk & Requirement Based Testing in a nutshell
Risk without requirement:
• Add requirement (a fault found
early)
• Remove Risk (no worries about
nothing)
Requirement without Risk
• Add product risk (improves
coverage of test)
• Remove Requirement (only
make what is needed)
Requirements
Product Risks
© All rights reserved 7

8. Product Risk Analysis in a nutshell
“A process or method to analyse the risks to the product, with the
intention to utilize the testing effort in the most valuable way.”
Requirement Product Risk
Risk class: High
Risk class: Medium
Risk class: Low
© All rights reserved
“The likelihood that a product fails
in relation to the conceivable
damage when a product fails in
production.”
8

9. Introduction
Agile
Testing
isn’t
Risking IT!
What is it all
about?
Which
problems
did I face?
How to do it
right?
Q & A
© All rights reserved 9

10. Merging or splitting features invalidate the PRA.
Risk analysis were not translated to the Backlog which makes it
difficult to utilise a risk based approach.
High
Medium
Low
© All rights reserved
10

11. PRA isn’t updated
Risk analysis were not updated every iteration, which could mean that
they will be out dated within a sprint.
© All rights reserved
11

12. PRA are done with tunnel vision
Risk analysis were only done from the requirements which meant that
some product risks were missed.
Analyse
Requirements
Order
Requirements
© All rights reserved
Determine
Risk
Specify
Test Cases
Analyse
Requirements
Order
Requirements
Analyse Risk
Analyse Risk
Specify
Test Cases
Agile Testing isn’t Risking IT! 12

13. Risk class isn’t used in planning meeting
Risk analysis were done, but the outcome wasn’t taken into account in
the planning meetings.
Team Velocity
=
7 Story Points
Hoog = n * 4
Middel = n * 2
Laag = n * 1
© All rights reserved

14. Testing is prioritized based on product risks
Risk analysis were done, but the risks where prioritised using
MoSCoW, which doesn’t automatically fit in a Agile environment.
Sprint 1 Sprint 2 Sprint 3
© All rights reserved
14

15. Introduction
Agile
Testing
isn’t
Risking IT!
What is it all
about?
Which
problems did
I face?
How to do it
right?
Q & A
© All rights reserved 15

16. Gather Risk
Items
•Gather & Order
Requirements
•Gather Product
Risks
© All rights reserved
Plot Risk Items
•Map Requirements
& Product Risks
Calculate Risk
Items
•Determine Impact
& Likelihood
Factors
• Score & Calculate
Risk Items/Factors
Strategy to
tactic
• Risk class tactic
• Sprint tactic
Gather Risk Items
16

17. Gather & Order Requirements
© All rights reserved
REQ
04
REQ
02
REQ
03
REQ
07
REQ
01
REQ
05
REQ
06
REQ
08 REQ
09 REQ
10
Rank #
1
2
3
4
5
6
7
8
9
10
17

18. Gather Product Risks
© All rights reserved
PR
02
PR
05
PR
01
PR
04
PR
03
PR
08
PR
07
PR
06
PR
09
18

19. Gather Risk
Items
•Gather & Order
Requirements
•Gather Product
Risks
© All rights reserved
Plot Risk
Items
•Map
Requirements &
Product Risks
Calculate Risk
Items
•Determine Impact
& Likelihood
Factors
• Score & Calculate
Risk Items/Factors
Strategy to
tactic
• Risk class tactic
• Sprint tactic
Plot Risk Items
19

20. Map Requirements & Product Risks 1/3
Requirements
Product Risks
REQ
01
REQ
04
PR
02
PR
05
PR
01
PR
04
PR
03
REQ
PR
08 08
PR
PR
07
PR
06
REQ
02
REQ
03
REQ
07
REQ
05
REQ
06
09
REQ
09 REQ
10
© All rights reserved 20

21. Map Requirements & Product Risks 2/3
Requirements
Product Risks
REQ
01
REQ
04
PR
02
PR
05
PR
01
PR
04
PR
03
REQ
PR
08 08
PR
PR
07
PR
06
REQ
02
REQ
03
REQ
07
REQ
05
REQ
06
09
REQ
09 REQ
10
© All rights reserved 21

22. Map Requirements & Product Risks 3/3
© All rights reserved
22
Product Risk
Requirement
PR
01
PR
02
PR
03
PR
04
PR
05
PR
06
PR
07
PR
08
PR
09
REQ 01 ● ● ● ●
REQ 02 ● ● ●
REQ 03 ●
REQ 04 ● ●
REQ 05 ● ●
REQ 06 ● ●
REQ 07 ● ●
REQ 08 ● ●
REQ 09 ● ●
REQ 10 ● ● ●

23. Gather Risk
Items
•Gather & Order
Requirements
•Gather Product
Risks
© All rights reserved
Plot Risk Items
•Map Requirements
& Product Risks
Calculate
Risk Items
•Determine Impact
& Likelihood
Factors
• Score & Calculate
Risk Items/Factors
Strategy to
tactic
• Risk class tactic
• Sprint tactic
Calculate Risk Items
23

24. Determine Impact & Likelihood Factors
Impact factors e.g.
• Frequency of Use
• Visibility
• Cost of failure
• …
Likelihood Factors e.g.
• Complexity
• Re-use
• Interfaces
• Dependencies
• Size
• …
퐼푚푝푎푐푡 = 퐹푟푒푞푢푒푛푐푦 표푓 푈푠푒 ∗ 푉푖푠푖푏푖푙푖푡푦 ∗ 퐶표푠푡 표푓 푓푎푖푙푢푟푒 ∗ …
퐿푖푘푒푙푖ℎ표표푑 = 퐶표푚푝푙푒푥푖푡푦 ∗ 푅푒 − 푢푠푒 ∗ 퐼푛푡푒푟푓푎푐푒푠 ∗ …
© All rights reserved 24

25. Score & Calculate Risk Item/Factors
© All rights reserved
25
LIKELIHOOD REQUIR
EMENT
RISK
CLASS
IMPACT
Factor
A
Factor
B
Factor
C
TOTAL TOTAL Factor
A
Factor
B
Factor
C
High High High High REQ 01 Medium Low Low Low Low
Low Medium Medium Low REQ 02 Medium Medium High High Low
Medium Medium Low Medium REQ 03 Medium Medium Medium Low Low
High High Medium High REQ 04 High Medium High Medium High
High Low Low Medium REQ 05 Medium Medium Low Medium Low
Medium Low Low Low REQ 06 Low Low Low Low Medium
Medium Medium Medium Medium REQ 07 High High Medium High High
Low Medium High Low REQ 08 Low Low Medium Low Low
Low Low Low Low REQ 09 Low Low Medium Low Low
Medium Medium Medium Medium REQ 10 Medium Medium Low Medium Medium

26. Gather Risk
Items
•Gather & Order
Requirements
•Gather Product
Risks
© All rights reserved
Plot Risk Items
•Map Requirements
& Product Risks
Calculate Risk
Items
•Determine Impact
& Likelihood
Factors
• Score & Calculate
Risk Items/Factors
Strategy to
tactic
•Risk class tactic
•Sprint tactic
Strategy to tactic
26

27. © All rights reserved
High Medium Low
Check documentation Inspection Walkthrough Peer review
UT test case review Test lead Tester -
ST test case review Test lead Tester -
Include test case in
Sprint +
Release -
regression set
release
Automate test case CodedUI Record &
Playback
-
Test execution on
browsers
Top 3 Top 2 Top 1
Risk class tactic
High = test effort * 4 Medium = test effort * 2 Low = test effort * 1
27

28. Sprint Tactic
Rank #
1
2
3
4
5
6
7
8
9
10
© All rights reserved
Sprint 1
28
Product Risk
Requirement
PR
01
PR
02
PR
06
REQ 03 ●
REQ 07 ● ●

29. Sprint Tactic
© All rights reserved
Sprint 2
Rank #
1
2
3
4
5
6
7
8
9
10
29
Product Risk
Requirement
PR
01
PR
04
PR
07
PR
08
PR
09
REQ 01 ● ● ● ●
REQ 05 ● ●
REQ 08 ● ●

30. Scrum bord
S P R I N T 2
To Do Doing Done
REQ 8
REQ 1
REQ 5
© All rights reserved 30

31. Insight into Product Risk & Requirement
coverage
© All rights reserved
31
Product Risk
Requirement
PR
01
PR
02
PR
03
PR
04
PR
05
PR
06
PR
07
PR
08
PR
09
REQ 01 2 2 2 2 
REQ 02 ● ● ● ?
REQ 03 1 
REQ 04 ● ● ?
REQ 05 2 2 
REQ 06 ● ● ?
REQ 07 1 1 
REQ 08 2 2 
REQ 09 3 3 
REQ 10 ● ● ● ?
? ? ?  ? ? ?  

32. Introduction
Agile
Testing
isn’t
Risking IT!
What is it all
about?
Which
problems did
I face?
How to do it
right?
Q & A
© All rights reserved 32

33. © All rights reserved
B.W.F.P.M. BRONNEBERG
TEST MANAGER / CONSULTANT
REQUIREMENT & QUALITY MANAGEMENT
BRAM.BRONNEBERG@CGI.COM
WWW.BRAMBRONNEBERG.NL
@BRAMBRONNEBERG
+31 6 55 11 35 10
33