1. OVERVIEW OF TEST
PROCESS IMPROVEMENT
FRAMEWORKS
Nikita Knysh, Ciklum, February 21, 2012

2. Speaker
 Nikita Knysh, 31 y.o.
 E-comonic, Ciklum
 11 years in IT, 5 years in software test
 ISTQB Certified Test Manager

3. Agenda
 The way of improvement
 Why use process frameworks
 TMM
 TPI
 CTP
 STEP

4. The Way of Improvement

5. Why Use Process Frameworks
 Get aware of industry practices
 Measure process maturity
 Compare to others
 Suggest improvements, set goals
 Evaluate improvement

6. Test Maturity Model (TMM)
 Complements CMM
 5 maturity levels, a number of processes in
each
 Establish basic processes first, then grow
 Staged
 Need to achieve previous level to claim for next
 Once a level is achieved, no further improvement
needed in its processes
 Binary
 You either fulfill or not. A process cannot be 50%
fulfilled

7. Test Maturity Model (TMM)
 The five maturity levels
 Initial
 Defined
 Integrated
 Managed and measured
 Optimized
 Heavy, documentation-intensive.
 Suitable for contractual test, in regulated
industries, where auditable process is required.

8. Test Process Improvement
(TPI)
 The four process cornerstones
 Lifecycle
 Organization
 Infrastructure and tools
 Techniques
 20 Processes, 4 levels of maturity

9. Test Process Improvement
(TPI) #2

10. Test Process Improvement
(TPI) #3
 Allows to establish the process and start add
value quickly, then improve steadily and
incrementally
 Suitable for startups or introducing QA function
to existing companies

11. Critical Testing Processes
(CTP)
 Be good at critical processes! A critical
process:
 Repeated frequently, affects team efficiency
 Involves large number of people, affects cohesion
and cooperation
 Visible to peers and superiors
 Linked to project success
 Identifies attributes of a good process and
opportunities for improvement
 Provides metrics and qualitative evaluations
for each process, allows tailoring

12. Critical Testing Processes
(CTP) #2
 Testing  Test system
 Establishing context development
 Quality risk analysis  Test release
 Test estimation management
 Test execution
 Test planning
 Bug reporting
 Test team
development  Results reporting
  Change management
All evaluations are done against business
needs
 May be tailored for wide range of the orgs

13. Systematic Test and Evaluation Process
(STEP)
 Assesses three major stages of test:
 Planning
 Acquisition
(test analysis, design and
implementation)
 Measurement (test execution and reporting)
 Qualitative factors considered:
 Defined
test process utilization
 Customer satisfaction

14. Systematic Test and Evaluation Process
(STEP) #2
 Quantitative metrics:
 Test status over time
 Test requirements or risk coverage
 Defect trends, including detection, severity, and
clustering
 Defect density
 Defect removal effectiveness
 Defect detection %
 Defect introduction, detection, and removal phases
 Cost of testing in terms of time, effort, and money
 Assumes ’test then code’ approach, close
collaboration of dev and test, defects are
prevented or detected early or systematically
analyzed.

15. Some Other Frameworks
 Test Organization Maturity (TOM)
 Test Improvement Model (TIM)
 Software Quality Rank (SQR)
 TMap

16. Thank you!
 Questions