Test Process Improvement with TPI NEXT - what the model does not tell you but you should know
•Download as PPTX, PDF•
11 likes•16,955 views
The document discusses the Test Process Improvement (TPI) NEXT model. It describes how the model divides testing into 16 key areas and 4 maturity levels, and provides 157 checkpoints and improvement suggestions. The results of an assessment using the model are presented visually in a testing maturity matrix. The document emphasizes that properly defining and implementing improvements are critical steps, and discusses challenges organizations may face in the improvement process and how to address them.
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Test Process Improvement with TPI NEXT - what the model does not tell you but you should know
Similar to Test Process Improvement with TPI NEXT - what the model does not tell you but you should know (20)
More from SQALab
More from SQALab (20)
Recently uploaded
Recently uploaded (20)
Test Process Improvement with TPI NEXT - what the model does not tell you but you should know
- 1. Lvov, Ukraine Nov 7, 2013 Test Process Improvement with TPI Next What the model does NOT tell you BUT you should know Tapani Aaltio Sogeti Finland Oy
- 2. Using a Maturity Model as a Reference Helps in Structuring and Communicating the Improvement Actions CMM(i) TMM(i) …How can you tell if you do the right things (right)… Tapani Aaltio: What the TPI Model does not tell you – but you should know SQA Days Lvov, Nov 7, 2013
- 3. Improving the Test Process is a Continuous Change Towards Better Maturity Create Awareness Determine Goal, Scope and Approach Assess Current Situation Define Improvements Evaluate and Redirect Make a Plan of Action Implement Actions Tapani Aaltio: What the TPI Model does not tell you – but you should know SQA Days Lvov, Nov 7, 2013
- 4. The TPI NEXT Model Consists of 16 Key Areas, Four Maturity Levels and 157 Checkpoints Test maturity matrix Key areas Maturity levels Checkpoints Tapani Aaltio: What the TPI Model does not tell you – but you should know SQA Days Lvov, Nov 7, 2013
- 5. In the TPI NEXT Model, the Field of Testing is Divided into 16 Key Areas in Three Groups 1 Stakeholder commitment 2 Degree of involvement Stakeholder Relations 3 Test Strategy 4 Test Organization 5 Communication 6 Reporting 7 Test Process Management Test Management 8 Estimating and Planning 9 Metrics 10 Defect Management 11 Testware Management 12 Methodology Practice 13 Tester Professionalism Test Profession 14 Test Case Design 15 Test Tools 16 Test Tapani Aaltio: What the TPI Model does not tell you – but you should know Environments SQA Days Lvov, Nov 7, 2013
- 6. The TPI Model has Four Maturity Levels, with Requirements for Each Key Area on Each Maturity Level • Adhoc Activities Initial • Doing the right Things Controlled • Doing it the right way Efficient • Continuously adapting Optimizing Description of Key Area ”Test Strategy” on different maturity levels The test strategy enables the distribution of the test efforts and resources among the test levels and test activities. The test strategy is used to achieve adequate balance between analyzed product risks, test coverage and available efforts and resources, involving all stakeholders. The method of the test strategy is maintained well to ensure easy and valid application. Tapani Aaltio: What the TPI Model does not tell you – but you should know SQA Days Lvov, Nov 7, 2013
- 7. The TPI Model has Four Maturity Levels, with 2-4 Checkpoints for Each Key Area on Each Maturity Level • Adhoc Activities Initial • Doing the right Things Controlled • Doing it the right way Efficient • Continuously adapting Optimizing Examples of check points of Key Area ”Test Strategy” on different maturity levels The principal stakeholder agrees with the documented test strategy. The test strategy is based on product risk analysis. All relevant stakeholders agree with the documented test strategy. The overlap and gaps in test coverage between test levels or test types are well considered. The process of creating the test strategy is evaluated regularly. The test strategy is evaluated against metrics on indicents that occur in production. Tapani Aaltio: What the TPI Model does not tell you – but you should know SQA Days Lvov, Nov 7, 2013
- 8. The Results of an Assessment are Presented Visually in a Testing Maturity Matrix 16 Key Areas • Doing the right Things • Doing it the right way • Continuously adapting Controlled Efficient Optimizing Total 157 Check Points 63% of checkpoints met 12% of checkpoints met Tapani Aaltio: What the TPI Model does not tell you – but you should know SQA Days Lvov, Nov 7, 2013
- 9. To Understand the Whole and the Details, Use Multiple Channels to Gather Information Interview the right people with different points of view Investigate documents to find evidence “Hang around in the lobby” Tapani Aaltio: What the TPI Model does not tell you – but you should know SQA Days Lvov, Nov 7, 2013
- 10. Defining the Improvements is a Crucial Step – And it is Fully Supported by the TPI NEXT Model Create Awareness Determine Goal, Scope and Approach Assess Current Situation Define Improvements Evaluate and Redirect Make a Plan of Action Implement Actions Tapani Aaltio: What the TPI Model does not tell you – but you should know SQA Days Lvov, Nov 7, 2013
- 11. The TPI Model Offers Concrete Improvement Suggestions for Each Key Area on Each Maturity Level • Adhoc Activities Initial • Doing the right Things Controlled • Doing it the right way Efficient • Continuously adapting Improvement Suggestions of Key Area ”Test Strategy” on different maturity levels Include various stakeholders in Product Risk Analysis and Test Strategy. Investigate possible overlaps between different test levels. Consider combining test levels, if possible. Define the role of Test and Evaluation Coordinator, to co-ordinate and monitor all test and evaluation activities. Optimizing Tapani Aaltio: What the TPI Model does not tell you – but you should know SQA Days Lvov, Nov 7, 2013
- 12. The Clusters (A, B, C, … ) Help in Finding the Balanced Improvement Paths to a Higher Maturity Level Prioritize Key Areas to 16 Key Areas create Business Driven• Doing the right Improvement Paths Things Controlled • Doing it the right way • Continuously adapting Efficient Optimizing Tapani Aaltio: What the TPI Model does not tell you – but you should know SQA Days Lvov, Nov 7, 2013
- 13. The Complete TPI® NEXT Model Test Maturity Matrix Key areas Clusters Maturity levels Checkpoints Improvement suggestions Enablers e.g. Requirements Management and Project management Tapani Aaltio: What the TPI Model does not tell you – but you should know SQA Days Lvov, Nov 7, 2013
- 14. Making a Realistic Plan of Action Requires Detailed Planning – Don’t be Greedy and Try to Do Too Much! Create Awareness Determine Goal, Scope and Approach Assess Current Situation Define Improvements Evaluate and Redirect Make a Plan of Action Implement Actions Tapani Aaltio: What the TPI Model does not tell you – but you should know SQA Days Lvov, Nov 7, 2013
- 15. To Sell your Ideas to Management, Remember Use Both Facts and Feelings Tapani Aaltio: What the TPI Model does not tell you – but you should know SQA Days Lvov, Nov 7, 2013
- 16. If the Planned Improvement Actions cannot be Implemented, the Whole Investment has Gone to Waste Create Awareness The fun part Determine Goal, Scope and Approach The tough part Assess Current Situation Define Improvements Evaluate and Redirect Make a Plan of Action Implement Actions Tapani Aaltio: What the TPI Model does not tell you – but you should know SQA Days Lvov, Nov 7, 2013
- 17. Typical Impediments for Improvement – as Seen by Customers Resource availability for the Improvement Project Resource competencies in testing and quality assurance Cultural change – not easy to implement Improving feels like “an additional effort” No clear benefits seen Unclear scope and focus No owner for the improvement Tapani Aaltio: What the TPI Model does not tell you – but you should know SQA Days Lvov, Nov 7, 2013
- 18. The Organization must be Ready to Accept Declining Productivity in the Early Phases of Implementation Productivity Work systematically to reach a higher level Goal Current level Don’t give up here! Time Tapani Aaltio: What the TPI Model does not tell you – but you should know SQA Days Lvov, Nov 7, 2013
- 19. In a Successful Improvement Project, Assessing the Current Situation Represents only 10-30% of Total Work Example Improvement Project Work Estimates # Task Sogeti (days) Customer (days) % of total 1 Assessing the Current Situation 20 5 12,5% 2 Organizing and training the Customer Test Improvement Team 20 20 20,0% 3 Define General Test Agreements and Test Processes 40 10 25,0% 4 Support for Implementation of the new process in projects 60 25 42,5% Total 140 60 100,0% 19 Tapani Aaltio: What the TPI Model does not tell you – but you should know SQA Days Lvov, Nov 7, 2013
- 20. TPI NEXT Provides Tangible Profit to Business Improved quality (60%) Prevented damage by finding a majority of critical & major defects in testing phase Shorten time for testing (24%) Reduction in test duration due to a uniform controlled test strategy and risk based testing approach Reduce cost (35%) Less test cases & higher coverage = less test effort Shorter fix cycle Deadline? Implement Test Fix Test Fix Test Etc Why not do it right the first time – and minimize the fixing cycle? Tapani Aaltio: What the TPI Model does not tell you – but you should know SQA Days Lvov, Nov 7, 2013
- 21. Quality Gates Help in Catching Defects in the Same Phase Where They are Injected – No Faults Forward! Project without quality gates 167 Defects per Test Level Waste, customer dissatisfaction 109 74 54 47 34 19 21 ST Project with quality gates 6 5 FAT SIT E2E 29 UAT Aftercare Tapani Aaltio: What the TPI Model does not tell you – but you should know SQA Days Lvov, Nov 7, 2013
- 22. ROI calculation for No Faults Forward! without Quality Gates vs. with Quality Gates Baseline # defects (without quality gates) Pilot # defects (with quality gates) 276 63 Reduction # defects 213 Test cost per defect ≈€ 1.500,00 Rework cost per defect ≈€ 2.000,00 Total cost per defect ≈€ 3.500,00 # Defects * cost per defect Investment QA-consultant Cost reduction € 745.500,00 € 30.400,00 € 715.100,00 Tapani Aaltio: What the TPI Model does not tell you – but you should know SQA Days Lvov, Nov 7, 2013
- 23. Summary TPI NEXT is an easy-to-use, practice based assessment model for your test process The model provides balanced, business-driven improvement paths to reach a higher level of maturity The implementation of the improvements is hard work. The following ideas might help you achieve good results Engage the organisation from day one Use facts and feelings to sell your improvements to key people and management Find an executive sponsor – and keep him/her engaged Proceed in small steps Demonstrate added value early Don’t give up before you have reached a higher level of productivity For more information on TPI NEXT, navigate to http://www.tmap.net/en/tpi-next Tapani Aaltio: What the TPI Model does not tell you – but you should know SQA Days Lvov, Nov 7, 2013
- 24. Information on TPI NEXT is Easily Available in Different Formats Download Android App at Google Play Check out online resources at http://www.tmap.net/en/tpi-next, Buy the book at including the Excel-based Test Maturity Matrix Tool http://www.utn.nl/ and read it! Pass the EXIN TPI Foundation Certificate exam Tapani Aaltio: What the TPI Model does not tell you – but you should know SQA Days Lvov, Nov 7, 2013
- 25. Now it’s your Turn – Questions, Answers and Discussion... Thank you for your attention, this was “Test Process Improvement with TPI Next – What the model Does NOT tell you BUT you should know” Tapani Aaltio Sogeti Finland tapani.aaltio@sogeti.com Lvov, Ukraine Nov 7, 2013 Tapani Aaltio: What the TPI Model does not tell you – but you should know SQA Days Lvov, Nov 7, 2013
- 26. Back-up Slides Tapani Aaltio: What the TPI Model does not tell you – but you should know SQA Days Lvov, Nov 7, 2013
- 27. TPI Next Can be Applied together with CMMi, Using CMMi-driven Clusters Download white paper on TPI Next and CMMi at http://www.tmap.net/en/tpi-next/publications Tapani Aaltio: What the TPI Model does not tell you – but you should know SQA Days Lvov, Nov 7, 2013
- 28. The Structured Test Process According to TMap® Specification Preparation Creating test specifications, Defining central starting points, Specifying test object intake Collection of the test basis, Testability assessment Execution Control Management, Monitoring, Reporting, Adjusting Ctrl Plan Prep Spec Exec Intake of test object, Preparing the starting points, Executing the (re)tests, Checking and assessing the test results Comp Completion Infra Planning (main tasks) Analysing the product risk, Determining the test strategy, Estimating the effort, Defining test products, organisation and infrastructure Evaluating the test process, Preserving testware Infrastructure management Specifying the infrastructure, Realising the infrastructure, Specifying the infrastructure intake, Intake of the infrastructure, Maintaining the infrastructure, Preserving the infrastructure the TPI Model does not tell you – but you should know Tapani Aaltio: What SQA Days Lvov, Nov 7, 2013
Editor's Notes
- Many companies lose money or even their reputation because of bad quality IT-systems. When quality is bad, it sounds logical to start improving testing practices. The TPI NEXT –model is a great tool for assessing where you are and for creating a coherent roadmap for improvement.However, the improvement actions fail too often. Sometimes the plans are not realistic, the right stakeholders are not involved, or management doesn’t really support the initiatives. These – and other - potential impediments should be taken into account already from day one in the TPI assessment. In this talk, I will shortly present the TPI NEXT -model and discuss my practical experiences on how to succeed in improving - not only the test process, but also the whole software development life cycle. The topics discussed include both success stories and LFMF’s (=learn from my failure). In short, I will tell you the things you should know, but can't find in the TPI NEXT book.1. Short presentation of the structure and main points of the TPI NEXT model, including its connections to other processes in the SDLC2. Assessment of the strengths and weaknesses of the TPI NEXT Model, based on several assessments with different customers3. Tips on how to conduct a TPI assessment in such a way that when the assessment is done, everything is set up for a fruitful improvement project.
- Create awarenessMostly awareness is there, because of quality problems in prod, long test & fix cycles, TODODetermine goal, scope and approachThe scope is normally a project or two, it’s important to have a limited scopeTwo different packages: full TPI or Quick ScanBe sure to agreewith the stakeholders – and understand -what the purpose of the assessment is - or is it just a nice to knowmeasurementAssess current situationFirst thing to do: choose and engage the right people to work with you – these will be your change agents later!
- Degree ofinvolvement: howtightlytesting is coupledwith the wholeSwdevelopmentlifecycle
- TPI is not mathematics – it is not b&wTODO example of ”notmathematics”
- DefineImprovementsFocused improvement suggestions and stepwise improvement (in short cycles) are two strengths of the model!Unfortunately, itdoesnottellwhatyourimprovementvelocitycouldbe - howmanycheckpoints / monthDon’toverdoit, evenRomewasn’tbuilt in a dayShow the customeryour vision of the targetsituationOn the other hand, you sometimes need “revolution and evolution”!Not all areas are equally important? If you have the wrong test cases, you are lost! BUT to define the right cases, you need other skills, e.g. business knowledge
- Make a plan of actionthe book covers this on only 2 pages!Considerbothquickwins and longertermgoalsH2 sellyourideas to the organization - Rememberthatdecisionsare made based on bothfacts and feelings.
- Youcannotsell the best test process to everybody with a detaileddescription of the process itself!Create a vision of an easy life of somethingnice.
- Evaluate and redirectThis belongs to the fun part!this makes continuous improvement possible, but is covered only on 1 page!Implement actionsthis is the hardest part, but the book only covers this on 1 page!people should have an understanding of the whole, “the big picture” to understand the meaning of “improvements” – training neededcreating the awareness, training, coaching, support neededCommunication using all possible channels neededthis cannot be accomplished without a permanent test / quality role in place to monitor progress of the implementationThis role should not do it him/herself, but motivate and make sure somebody in the project doesButevenwithallthis in place, youaregoing to meetchangeresistance…
- Rememberyourultimategoal: clientsatisfaction - but at whatpoint in time?
- Revolution and evolution!Sohowdogetupthatcurve?
- Whereas, the support forimplementationcouldeasilybealmosthalf of the effort!Peoplefeelthattheyhave to domore…. … and ifyoudoallthis, whatdoyouget? – is itpossible to a systemwhich is good, cheap and fast at the sametime?
- Whynotfix the defects in the phasewheretheyareinjected?
- Transition:… and ifwecalculate the savings in money…
- CreatedbySogeti, butavailable for everybody in differentformats.No licensefees, just by the bookordownload the application!