This document provides an overview of several test process improvement frameworks: - The Test Maturity Model (TMM) uses five staged levels to measure test process maturity and suit regulatory environments. - Test Process Improvement (TPI) allows for asynchronous improvements across four process cornerstones and twenty processes at four levels. - Critical Testing Processes (CTP) focuses on continuously improving critical, high-impact testing processes. - The Systematic Test and Evaluation Process (STEP) assesses planning, implementation, and measurement of testing through qualitative and quantitative metrics.

1. OVERVIEW OF TEST
PROCESS IMPROVEMENT
FRAMEWORKS
Nikita Knysh, Ciklum, February 21, 2012

2. Speaker
 Nikita Knysh, 31 y.o.
 E-comonic, Ciklum
 11 years in IT, 5 years in software test
 ISTQB Certified Test Manager

3. Agenda
 The way of improvement
 Why use process frameworks
 TMM
 TPI
 CTP
 STEP

4. The Way of Improvement

5. Why Use Process Frameworks
 Get aware of industry practices
 See where we are, measure process maturity
 Compare to others
 Suggest improvements, set goals
 Evaluate improvement

6. Test Maturity Model (TMM)
 Complements CMM
 Establish basics first, then grow
 Staged
 Need to achieve previous level to claim for next
 Once a level is achieved, no further improvement
needed in it
 Binary
 You either fulfill or not. A stage cannot be 50%
fulfilled

7. Test Maturity Model (TMM)
 The five maturity levels
 Initial
 Defined
 Integrated
 Managed and measured
 Optimized
 Documentation-intensive.
 Suitable for contractual test, in regulated
industries, where auditable process is required.

8. Test Process Improvement
(TPI)
 Provides finer-grained measurement than
TMM
 The four process cornerstones
 Lifecycle
 Organization
 Infrastructure and tools
 Techniques
 20 Processes, 4 levels of maturity
 Asynchronous improvements in prescribed
order

9. Test Process Improvement
(TPI) #2

10. Test Process Improvement
(TPI) #3
 Allows to establish the process and start add
value quickly, then improve steadily and
incrementally
 Suitable for startups or introducing QA function
to existing companies

11. Critical Testing Processes
(CTP)
 Be good at critical processes! A critical
process:
 Repeated frequently, affects team efficiency
 Involves large number of people, affects cohesion
and cooperation
 Visible to peers and superiors
 Linked to project success
 Identifies attributes of a good process and
opportunities for improvement
 Provides metrics and qualitative evaluations
for each process, allows tailoring

12. Critical Testing Processes
(CTP) #2
 Testing
 Establishing context
 Quality risk analysis
 Test estimation
 Test planning
 Test team
development
 Test system
development
 Test release
management
 Test execution
 Bug reporting
 Results reporting
 Change management All evaluations are done against business
needs
 May be tailored for wide range of the orgs

13. Systematic Test and Evaluation Process
(STEP)
 Non-prescriptive in terms of improvements
 Assesses three major stages of test:
 Planning
 Acquisition (test analysis, design and
implementation)
 Measurement (test execution and reporting)
 Qualitative factors considered:
 Defined test process utilization
 Customer satisfaction

14. Systematic Test and Evaluation Process
(STEP) #2
 Quantitative metrics:
 Test status over time
 Test requirements or risk coverage
 Defect trends, including detection, severity, and
clustering
 Defect density
 Defect removal effectiveness
 Defect detection %
 Defect introduction, detection, and removal phases
 Cost of testing in terms of time, effort, and money
 Assumes ’test then code’ approach, close
collaboration of dev and test, defects are
prevented or detected early or systematically
analyzed.

15. Some Other Frameworks
 Test Organization Maturity (TOM)
 Test Improvement Model (TIM)
 Software Quality Rank (SQR)
 TMap

16. Thank you!
 Questions