The document discusses experiences with the Test Maturity Model integration (TMMi). It provides an overview of a TMMi assessment that the author conducted for 20 organizations in various industries in the Netherlands. The assessment involved quick scans of five TMMi level 2 process areas: Test Policy and Strategy, Test Planning, Test Monitoring and Control, Test Design and Execution, and Test Environment. The author analyzes the results, finding that organizations scored highest in Test Design and Execution (60%) and lowest in Test Policy and Strategy (30%). The document also discusses why test maturity models like TMMi are relevant for testers globally, including in countries like Romania that engage in significant offshore testing.

1. Experiences with TMMi:
facts and figures
Jan Jaap Cannegieter
Vice President SYSQA B.V.

2. ©SYSQAAlmere
2
Agenda
0
10
20
30
40
50
60
70
80
90
100

3. What is
A maturity model
and
an assessment model

4. ©SYSQAAlmere
4
Test Maturity Model integration

5. ©SYSQAAlmere
5
TMMi levels
1. Initial
2. Managed
Test Policy and Strategy
Test Planning
Test Monitoring and Control
Test Design and Execution
Test Environment
3. Defined
Test Organization
Test Training Program
Test Life Cycle and Integration
Non-Functional Testing
Peer Reviews
4. Management
and Measurement
Test Measurement
Software Quality Evaluation
Advanced Peer Reviews
5. Optimization
Defect Prevention
Test Process Optimization
Quality Control

6. ©SYSQAAlmere
6

7. ©SYSQAAlmere
9
Assessment type: Quick Scan
Test
manager 1
Tester 1
QA
manager
Test
environment
administrator
Test
manager 2
Tester 2
Project
manager
You
8.30 Opening & TMMi
9.00 Rating PA’s level 2
12.00 Lunch
13.00 Rating PA’s
14.00 Conclusions
14.30 Determine improvement actions
16.00 Making action plan
16.30 Evaluation
(20.30 On the couch watching TV)

8. ©SYSQAAlmere
10
The tool

9. ©SYSQAAlmere
11
Time
Start
I I I
QS QS QS QS QS
F
QS
Assessments and TMMi implementations
Level 2

10. ©SYSQAAlmere
12
Assessment types compared
0 100
Formal Informal Quick scan
Accuracy
Costs
100% Use at
SYSQA

11. ©SYSQAAlmere
13
TMMi today
http://www.tmmi.org/

12. The maturity of testing in
The Netherlands

13. ©SYSQAAlmere
15
TMMi levels
1. Initial
2. Managed
Test Policy and Strategy
Test Planning
Test Monitoring and Control
Test Design and Execution
Test Environment
3. Defined
Test Organization
Test Training Program
Test Life Cycle and Integration
Non-Functional Testing
Peer Reviews
4. Management
and Measurement
Test Measurement
Software Quality Evaluation
Advanced Peer Reviews
5. Optimization
Defect Prevention
Test Process Optimization
Quality Control

14. ©SYSQAAlmere
16
How I got my data
• Goals
• 20 organizations
• Telecommunication: 2
• Government: 7
• Financial services: 9
• Industry: 2
• Approach assessments: quick scans
• Analysis

15. ©SYSQAAlmere
17
Test policy and strategy - goals
• Establish a test policy
• Establish a test strategy
• Establish test performance indicators

16. ©SYSQAAlmere
18
Test policy and strategy: A: 30% V: 14%
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Test policy and strategy
Organizations
Percentageofspecificgoalsmet

17. ©SYSQAAlmere
19
Test planning - goals
• Perform a product risk assessment
• Establish a test approach
• Establish test estimates
• Develop a test plan
• Obtain commitment to the test plan

18. ©SYSQAAlmere
20
Test planning: A: 41%, V: 13%
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Test planning
Organizations
Percentageofspecificgoalsmet

19. ©SYSQAAlmere
21
Test monitoring and control - goals
• Monitor test progress against plan
• Monitor product quality against plan and
expectations
• Manage corrective action to closure

20. ©SYSQAAlmere
22
Test monitoring and control: A: 47%, V: 15%
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Test monitoring and control
Organizations
Percentageofspecificgoalsmet

21. ©SYSQAAlmere
23
Test design and execution - goals
• Perform test analysis and design using test design
techniques
• Perform test implementation
• Perform test execution
• Manage test incidents to closure

22. ©SYSQAAlmere
24
Test design and execution: A: 60%, V: 8%
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Test design and execution
Organizations
Percentageofspecificgoalsmet

23. ©SYSQAAlmere
25
Test environment - goals
• Develop test environment requirements
• Perform test environment implementation
• Manage and control test environments

24. ©SYSQAAlmere
26
Test environment: A: 59%, V: 9%
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Test environment
Organizations
Percentageofspecificgoalsmet

25. ©SYSQAAlmere
27
Overview
0
10
20
30
40
50
60
70
80
90
100
Per process area

26. ©SYSQAAlmere
28
My conclusion
60%: Test design and execution
30%: Test policy and strategy
41%: Test planning
47%: Test monitoring and control
59%: Test environment
Inside
Outside

27. What about Romanian testers;
should they care about TMMi?

28. ©SYSQAAlmere
30
Reasons to care about TMMi

29. ©SYSQAAlmere
31
Outsourcing and the Allen curve
‘The distance between engineers’ offices affected the frequency of their communication’
– Prof. Thomas Allen MIT

30. ©SYSQAAlmere
32
8 meters

31. ©SYSQAAlmere
33
16 meters

32. ©SYSQAAlmere
34
80 kilometers

33. ©SYSQAAlmere
35
1600 kilometers

34. ©SYSQAAlmere
36

35. ©SYSQAAlmere
37
Email for questions, the documents I
showed you or additional information:
jcannegieter@sysqa.nl