5. Why Use Process Frameworks
Get aware of industry practices
See where we are, measure process maturity
Compare to others
Suggest improvements, set goals
Evaluate improvement
6. Test Maturity Model (TMM)
Complements CMM
Establish basics first, then grow
Staged
Need to achieve previous level to claim for next
Once a level is achieved, no further improvement
needed in it
Binary
You either fulfill or not. A stage cannot be 50%
fulfilled
7. Test Maturity Model (TMM)
The five maturity levels
Initial
Defined
Integrated
Managed and measured
Optimized
Documentation-intensive.
Suitable for contractual test, in regulated
industries, where auditable process is required.
8. Test Process Improvement
(TPI)
Provides finer-grained measurement than
TMM
The four process cornerstones
Lifecycle
Organization
Infrastructure and tools
Techniques
20 Processes, 4 levels of maturity
Asynchronous improvements in prescribed
order
10. Test Process Improvement
(TPI) #3
Allows to establish the process and start add
value quickly, then improve steadily and
incrementally
Suitable for startups or introducing QA function
to existing companies
11. Critical Testing Processes
(CTP)
Be good at critical processes! A critical
process:
Repeated frequently, affects team efficiency
Involves large number of people, affects cohesion
and cooperation
Visible to peers and superiors
Linked to project success
Identifies attributes of a good process and
opportunities for improvement
Provides metrics and qualitative evaluations
for each process, allows tailoring
12. Critical Testing Processes
(CTP) #2
Testing
Establishing context
Quality risk analysis
Test estimation
Test planning
Test team
development
Test system
development
Test release
management
Test execution
Bug reporting
Results reporting
Change management All evaluations are done against business
needs
May be tailored for wide range of the orgs
13. Systematic Test and Evaluation Process
(STEP)
Non-prescriptive in terms of improvements
Assesses three major stages of test:
Planning
Acquisition (test analysis, design and
implementation)
Measurement (test execution and reporting)
Qualitative factors considered:
Defined test process utilization
Customer satisfaction
14. Systematic Test and Evaluation Process
(STEP) #2
Quantitative metrics:
Test status over time
Test requirements or risk coverage
Defect trends, including detection, severity, and
clustering
Defect density
Defect removal effectiveness
Defect detection %
Defect introduction, detection, and removal phases
Cost of testing in terms of time, effort, and money
Assumes ’test then code’ approach, close
collaboration of dev and test, defects are
prevented or detected early or systematically
analyzed.
15. Some Other Frameworks
Test Organization Maturity (TOM)
Test Improvement Model (TIM)
Software Quality Rank (SQR)
TMap
- A framework is a set of practices and measurements
Establishing context: study documents, talking to project stakeholders, fitting in organization and its operation
Test system development: identify specific tests to address critical quality risks, designing and implementing test cases, data, scripts, environments, documenting tests and test coverage
Test release management: delivery of a known set of components for testing, hand-off prom the project team to test team, balance need for quick feedback against need for testing progress (each release should be better than the previous)
Change management: balancing risks and costs of change / bugfix with benefits and opportunities, select the right changes in the right order, balance schedule/budget/features/quality.