Tester get out of your cave SIGIST 2012 London
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Tester get out of your cave SIGIST 2012 London

on

  • 1,186 views

Website SIGIST: ...

Website SIGIST:
TMMi, the successor of TMM, was first published in 2008. Since then, a lot of organisations have used it to assess or improve their test processes. Jan Jaap Cannegieter used TMMi in Telecom, public services, finance and utility. Based on over 25 assessments conclusions about the level of testing in The Netherlands can be drawn:

- Which areas of testing are well developed?
- Which areas of testing are not well developed?
- What conclusions can be drawn based on these data?

These questions will be answered in the presentation. The main conclusion will be that those areas that can easily be influenced by testers are quite well developed; test process improvement is something done by testers in their cave! The areas where other stakeholders need to participate are not that well developed. So to improve testing we need to involve other stakeholders better. The way we should act to involve the relevant stakeholders will be explained and analyzed.

Jan Jaap Cannegieter is a leading test, QA, CMMI and requirements expert in the Netherlands. He has 20 years of experience in ICT, starting with software testing and quality assurance in ICT-projects. In the last few years he has completed several TMMi test process improvement, CMMI, SPI and requirements projects. In addition to his assignments, Jan Jaap is Vice President of SYSQA B.V., a company of 180 employees specializing in requirements, software testing, quality assurance and software process improvement. Jan Jaap is the writer of several articles and books, including The little
TMMi. Jan Jaap was a member of the TMMi development team and is TMMi accredited lead assessor.

Statistics

Views

Total Views
1,186
Views on SlideShare
1,101
Embed Views
85

Actions

Likes
0
Downloads
9
Comments
0

3 Embeds 85

http://www.sysqa.com 41
http://www.sysqa.nl 38
http://www.twylah.com 6

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Tester get out of your cave SIGIST 2012 London Presentation Transcript

  • 1. Tester, get out of your cave!Jan Jaap Cannegieter –Vice President SYSQA B.V. jcannegieter@sysqa.nl@jjcannegieter
  • 2. 2 © SYSQA Almere
  • 3. Why are we here? © SYSQA Almere 3
  • 4. Agenda• TMMi in under 10 minutes• Data from 20 TMMi workshops• Conclusions• How to get out of your cave © SYSQA Almere 4
  • 5. TMMi – essentials• Test maturity framework• Benchmark model for assessments• 5 levels• Based on: • TMM • Evolutionary Testmodel (Gelperin / Hetzel) • CMMI for Development• In line with ISTQB terminology• Developed by the TMMi-foundation• Public domain © SYSQA Almere• www.tmmifoundation.org 6
  • 6. TMMi levels Defect Prevention 5. Optimization Test Process Optimization Quality Control Test Measurement 4. Management Software Quality Evaluation and Measurement Advanced Peer Reviews Test Organization Test Training Program 3. Defined Test Life Cycle and Integration Non-Functional Testing Peer Reviews Test Policy and Strategy Test Planning 2. Managed Test Monitoring and Control Test Design and Execution © SYSQA Almere Test Environment 1. Initial 7
  • 7. How I got my data• Goals • Learn • Presales • Assignments• 20 organizations • Telecommunication: 2 • Government: 7 • Financial services: 9 • Industry: 2 © SYSQA Almere• Analysis 8
  • 8. ResultsFasten your seatbelts
  • 9. Test policy and strategy - goals• Establish a test policy• Establish a test strategy• Establish test performance indicators © SYSQA Almere 11
  • 10. Test policy and strategy: A: 30% V: 14% Test policy and strategy 100 Percentage of specific goals met 90 80 70 60 50 40 30 20 © SYSQA Almere 10 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Organizations 12
  • 11. Test planning - goals• Perform a product risk assessment• Establish a test approach• Establish test estimates• Develop a test plan• Obtain commitment to the test plan © SYSQA Almere 13
  • 12. Test planning: A: 41%, V: 13% Test planning Percentage of specific goals met 100 90 80 70 60 50 40 30 20 © SYSQA Almere 10 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Organizations 14
  • 13. Test monitoring and control - goals• Monitor test progress against plan• Monitor product quality against plan and expectations• Manage corrective action to closure © SYSQA Almere 15
  • 14. Test monitoring and control: A: 47%, V: 15% Test monitoring and control Percentage of specific goals met 100 90 80 70 60 50 40 30 20 © SYSQA Almere 10 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Organizations 16
  • 15. Test design and execution - goals• Perform test analysis and design using test design techniques• Perform test implementation• Perform test execution• Manage test incidents to closure © SYSQA Almere 17
  • 16. Test design and execution: A: 60%, V: 8% Test design and execution 100 Percentage of specific goals met 90 80 70 60 50 40 30 20 © SYSQA Almere 10 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Organizations 18
  • 17. Test environment - goals• Develop test environment requirements• Perform test environment implementation• Manage and control test environments © SYSQA Almere 19
  • 18. Test environment: A: 59%, V: 9% Test environment Percentage of specific goals met 100 90 80 70 60 50 40 30 20 © SYSQA Almere 10 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 Organizations 20
  • 19. Overview Per process area 100 90 80 70 60 50 40 30 20 10 0 © SYSQA Almere 21
  • 20. My conclusion Outside 30%: Test policy and strategy 41%: Test planning 47%: Test monitoring and control 59%: Test environment © SYSQA Almere Inside 60%: Test design and execution 22
  • 21. We are in a cave! © SYSQA Almere 23
  • 22. © SYSQA Almere24
  • 23. In case you disagree… © SYSQA Almere 25
  • 24. Best practices ‘outside of your cave’• Perform a product risk assessment and establish a test approach with relevant stakeholders• Obtain commitment for your plan from relevant stakeholders• Plan stakeholder involvement in your project• Monitor stakeholder involvement in your project• Make release advice with people outside of testing• Establish and maintain a test strategy with relevant management• Define, measure and report test performance indicators to relevant stakeholders © SYSQA Almere• Establish and maintain a test policy with higher management (business and IT) 26
  • 25. Stakeholder managementThe basics 27
  • 26. Stakeholder © SYSQA Almere A person or group of persons who will be involved in or effected by your test project or improvement project 28
  • 27. Stakeholder management• Stakeholder identification• Stakeholder analysis• Stakeholder involvement © SYSQA Almere 29
  • 28. Stakeholder aspects• Organizational level• Power• Business interest• Mindset• Ambition• Behavioral style• Stakeholder type © SYSQA Almere 30
  • 29. Type of stakeholders Power Sleeping Dangerous Dominant DefinitiveUrgency Demanding Discretionary Legitimacy © SYSQA Almere Dependent Mitchell, Agle, Wood, 1997 31
  • 30. Stakeholdercard – an example © SYSQA Almere 32
  • 31. Please think this overI’ll turn on some inspiring music
  • 32. © SYSQA Almere34
  • 33. © SYSQA Almere35
  • 34. Stuart Reid’s test competence model fortesters Test skills Soft skills IT skills © SYSQA Almere Domain knowledge 36
  • 35. My test competence model for testconsultants & test managers Test skills Political IT skills skills © SYSQA Almere Domain Soft skills knowledge 37
  • 36. © SYSQA Almere38
  • 37. More interesting graphs
  • 38. TMMi level by percentage 84% L1 L2 L3 L4 L5 © SYSQA Almere 10% 6% TMMi Level 40
  • 39. CMMI vs. Non-CMMI8070605040 CMMI30 no CMMI2010 © SYSQA Almere0 TPG TP TMC TT TO 41
  • 40. TMMi score by Industry70605040 Government30 Finance Industry2010 © SYSQA Almere 0 TPS TP TMC TDE TE Note: Telecom weg gelaten slechts 2 data punten 42
  • 41. Formeel vs. Informeel 14% Formeel Informeel © SYSQA Almere 86% 43
  • 42. Email for the stakeholder card:jcannegieter@sysqa.nl © SYSQA Almere 45
  • 43. © SYSQA Almere46