Tester, get out of your cave!Jan Jaap Cannegieter –Vice President SYSQA B.V. jcannegieter@sysqa.nl@jjcannegieter
2    © SYSQA Almere
Why are we here?                   © SYSQA Almere                   3
Agenda•   TMMi in under 10 minutes•   Data from 20 TMMi workshops•   Conclusions•   How to get out of your cave           ...
TMMi – essentials•   Test maturity framework•   Benchmark model for assessments•   5 levels•   Based on:    • TMM    • Evo...
TMMi levels                                                          Defect Prevention                                 5. ...
How I got my data• Goals • Learn • Presales • Assignments• 20 organizations •   Telecommunication: 2 •   Government: 7 •  ...
ResultsFasten your seatbelts
Test policy and strategy - goals• Establish a test policy• Establish a test strategy• Establish test performance indicator...
Test policy and strategy: A: 30% V: 14%                                                                Test policy and str...
Test planning - goals•   Perform a product risk assessment•   Establish a test approach•   Establish test estimates•   Dev...
Test planning: A: 41%, V: 13%                                                                        Test planning   Perce...
Test monitoring and control - goals• Monitor test progress against plan• Monitor product quality against plan and  expecta...
Test monitoring and control: A: 47%, V: 15%                                                            Test monitoring and...
Test design and execution - goals• Perform test analysis and design using test design  techniques• Perform test implementa...
Test design and execution: A: 60%, V: 8%                                                                    Test design an...
Test environment - goals• Develop test environment requirements• Perform test environment implementation• Manage and contr...
Test environment: A: 59%, V: 9%                                                                    Test environment   Perc...
Overview            Per process area      100       90       80       70       60       50       40       30       20     ...
My conclusion  Outside   30%: Test policy and strategy            41%: Test planning            47%: Test monitoring and c...
We are in a cave!                    © SYSQA Almere                    23
© SYSQA Almere24
In case you disagree…                        © SYSQA Almere                        25
Best practices ‘outside of your cave’• Perform a product risk assessment and establish a test  approach with relevant stak...
Stakeholder managementThe basics                         27
Stakeholder                                                              © SYSQA Almere    A person or group of persons wh...
Stakeholder management• Stakeholder identification• Stakeholder analysis• Stakeholder involvement                         ...
Stakeholder aspects•   Organizational level•   Power•   Business interest•   Mindset•   Ambition•   Behavioral style•   St...
Type of stakeholders                       Power                           Sleeping               Dangerous              D...
Stakeholdercard – an example                               © SYSQA Almere                               32
Please think this overI’ll turn on some inspiring music
© SYSQA Almere34
© SYSQA Almere35
Stuart Reid’s test competence model fortesters                      Test skills        Soft skills                 IT skil...
My test competence model for testconsultants & test managers                             Test skills         Political    ...
© SYSQA Almere38
More interesting graphs
TMMi level by percentage     84%                              L1                              L2                          ...
CMMI vs. Non-CMMI8070605040                              CMMI30                              no CMMI2010                  ...
TMMi score by Industry70605040                                                      Government30                          ...
Formeel vs. Informeel                   14%                         Formeel                         Informeel             ...
Email for the stakeholder card:jcannegieter@sysqa.nl                                  © SYSQA Almere                      ...
© SYSQA Almere46
Tester get out of your cave SIGIST 2012 London
Upcoming SlideShare
Loading in …5
×

Tester get out of your cave SIGIST 2012 London

1,225 views

Published on

Website SIGIST:
TMMi, the successor of TMM, was first published in 2008. Since then, a lot of organisations have used it to assess or improve their test processes. Jan Jaap Cannegieter used TMMi in Telecom, public services, finance and utility. Based on over 25 assessments conclusions about the level of testing in The Netherlands can be drawn:

- Which areas of testing are well developed?
- Which areas of testing are not well developed?
- What conclusions can be drawn based on these data?

These questions will be answered in the presentation. The main conclusion will be that those areas that can easily be influenced by testers are quite well developed; test process improvement is something done by testers in their cave! The areas where other stakeholders need to participate are not that well developed. So to improve testing we need to involve other stakeholders better. The way we should act to involve the relevant stakeholders will be explained and analyzed.

Jan Jaap Cannegieter is a leading test, QA, CMMI and requirements expert in the Netherlands. He has 20 years of experience in ICT, starting with software testing and quality assurance in ICT-projects. In the last few years he has completed several TMMi test process improvement, CMMI, SPI and requirements projects. In addition to his assignments, Jan Jaap is Vice President of SYSQA B.V., a company of 180 employees specializing in requirements, software testing, quality assurance and software process improvement. Jan Jaap is the writer of several articles and books, including The little
TMMi. Jan Jaap was a member of the TMMi development team and is TMMi accredited lead assessor.

Published in: Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,225
On SlideShare
0
From Embeds
0
Number of Embeds
100
Actions
Shares
0
Downloads
11
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Tester get out of your cave SIGIST 2012 London

  1. 1. Tester, get out of your cave!Jan Jaap Cannegieter –Vice President SYSQA B.V. jcannegieter@sysqa.nl@jjcannegieter
  2. 2. 2 © SYSQA Almere
  3. 3. Why are we here? © SYSQA Almere 3
  4. 4. Agenda• TMMi in under 10 minutes• Data from 20 TMMi workshops• Conclusions• How to get out of your cave © SYSQA Almere 4
  5. 5. TMMi – essentials• Test maturity framework• Benchmark model for assessments• 5 levels• Based on: • TMM • Evolutionary Testmodel (Gelperin / Hetzel) • CMMI for Development• In line with ISTQB terminology• Developed by the TMMi-foundation• Public domain © SYSQA Almere• www.tmmifoundation.org 6
  6. 6. TMMi levels Defect Prevention 5. Optimization Test Process Optimization Quality Control Test Measurement 4. Management Software Quality Evaluation and Measurement Advanced Peer Reviews Test Organization Test Training Program 3. Defined Test Life Cycle and Integration Non-Functional Testing Peer Reviews Test Policy and Strategy Test Planning 2. Managed Test Monitoring and Control Test Design and Execution © SYSQA Almere Test Environment 1. Initial 7
  7. 7. How I got my data• Goals • Learn • Presales • Assignments• 20 organizations • Telecommunication: 2 • Government: 7 • Financial services: 9 • Industry: 2 © SYSQA Almere• Analysis 8
  8. 8. ResultsFasten your seatbelts
  9. 9. Test policy and strategy - goals• Establish a test policy• Establish a test strategy• Establish test performance indicators © SYSQA Almere 11
  10. 10. Test policy and strategy: A: 30% V: 14% Test policy and strategy 100 Percentage of specific goals met 90 80 70 60 50 40 30 20 © SYSQA Almere 10 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Organizations 12
  11. 11. Test planning - goals• Perform a product risk assessment• Establish a test approach• Establish test estimates• Develop a test plan• Obtain commitment to the test plan © SYSQA Almere 13
  12. 12. Test planning: A: 41%, V: 13% Test planning Percentage of specific goals met 100 90 80 70 60 50 40 30 20 © SYSQA Almere 10 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Organizations 14
  13. 13. Test monitoring and control - goals• Monitor test progress against plan• Monitor product quality against plan and expectations• Manage corrective action to closure © SYSQA Almere 15
  14. 14. Test monitoring and control: A: 47%, V: 15% Test monitoring and control Percentage of specific goals met 100 90 80 70 60 50 40 30 20 © SYSQA Almere 10 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Organizations 16
  15. 15. Test design and execution - goals• Perform test analysis and design using test design techniques• Perform test implementation• Perform test execution• Manage test incidents to closure © SYSQA Almere 17
  16. 16. Test design and execution: A: 60%, V: 8% Test design and execution 100 Percentage of specific goals met 90 80 70 60 50 40 30 20 © SYSQA Almere 10 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Organizations 18
  17. 17. Test environment - goals• Develop test environment requirements• Perform test environment implementation• Manage and control test environments © SYSQA Almere 19
  18. 18. Test environment: A: 59%, V: 9% Test environment Percentage of specific goals met 100 90 80 70 60 50 40 30 20 © SYSQA Almere 10 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 Organizations 20
  19. 19. Overview Per process area 100 90 80 70 60 50 40 30 20 10 0 © SYSQA Almere 21
  20. 20. My conclusion Outside 30%: Test policy and strategy 41%: Test planning 47%: Test monitoring and control 59%: Test environment © SYSQA Almere Inside 60%: Test design and execution 22
  21. 21. We are in a cave! © SYSQA Almere 23
  22. 22. © SYSQA Almere24
  23. 23. In case you disagree… © SYSQA Almere 25
  24. 24. Best practices ‘outside of your cave’• Perform a product risk assessment and establish a test approach with relevant stakeholders• Obtain commitment for your plan from relevant stakeholders• Plan stakeholder involvement in your project• Monitor stakeholder involvement in your project• Make release advice with people outside of testing• Establish and maintain a test strategy with relevant management• Define, measure and report test performance indicators to relevant stakeholders © SYSQA Almere• Establish and maintain a test policy with higher management (business and IT) 26
  25. 25. Stakeholder managementThe basics 27
  26. 26. Stakeholder © SYSQA Almere A person or group of persons who will be involved in or effected by your test project or improvement project 28
  27. 27. Stakeholder management• Stakeholder identification• Stakeholder analysis• Stakeholder involvement © SYSQA Almere 29
  28. 28. Stakeholder aspects• Organizational level• Power• Business interest• Mindset• Ambition• Behavioral style• Stakeholder type © SYSQA Almere 30
  29. 29. Type of stakeholders Power Sleeping Dangerous Dominant DefinitiveUrgency Demanding Discretionary Legitimacy © SYSQA Almere Dependent Mitchell, Agle, Wood, 1997 31
  30. 30. Stakeholdercard – an example © SYSQA Almere 32
  31. 31. Please think this overI’ll turn on some inspiring music
  32. 32. © SYSQA Almere34
  33. 33. © SYSQA Almere35
  34. 34. Stuart Reid’s test competence model fortesters Test skills Soft skills IT skills © SYSQA Almere Domain knowledge 36
  35. 35. My test competence model for testconsultants & test managers Test skills Political IT skills skills © SYSQA Almere Domain Soft skills knowledge 37
  36. 36. © SYSQA Almere38
  37. 37. More interesting graphs
  38. 38. TMMi level by percentage 84% L1 L2 L3 L4 L5 © SYSQA Almere 10% 6% TMMi Level 40
  39. 39. CMMI vs. Non-CMMI8070605040 CMMI30 no CMMI2010 © SYSQA Almere0 TPG TP TMC TT TO 41
  40. 40. TMMi score by Industry70605040 Government30 Finance Industry2010 © SYSQA Almere 0 TPS TP TMC TDE TE Note: Telecom weg gelaten slechts 2 data punten 42
  41. 41. Formeel vs. Informeel 14% Formeel Informeel © SYSQA Almere 86% 43
  42. 42. Email for the stakeholder card:jcannegieter@sysqa.nl © SYSQA Almere 45
  43. 43. © SYSQA Almere46

×