2. Four Schools of Software Testing
3. Analytical School: Core Beliefs
• Software is a logical artifact
• Testing is a branch of CS/Mathematics
• Objective, rigorous, comprehensive
• Testing techniques must have a logico-mathematical form
• “one right answer”
• Testing is technical
• Which techniques should we use?
sees testing as rigorous and technical with many proponents in academia
4. Factory School: Core Beliefs
• Software development is a project
• Testing is a measure of progress
• Testing must be managed
• Predictable, repeatable, planned
• Testing must be cost-effective
• Low-skilled workers require direction
• Testing is following rules
• How can we measure whether we’re making progress? When will we be done?
sees testing as a way to measure progress with emphasis on cost and repeatable
5. Quality Assurance School: Core Beliefs
• Software quality requires discipline
• Testing determines whether development processes are being
• Testers may need to police developers to follow the rules
• Testers have to protect users from bad software
• Are we following a good process?
emphasizes process, policing developers and acting as the
7. The Seven Basic Principles of the Context-
1. The value of any practice depends on its context.
2. There are good practices in context, but there are no best practices.
3. People, working together, are the most important part of any project’s context.
4. Projects unfold over time in ways that are often not predictable.
5. The product is a solution. If the problem isn’t solved, the product doesn’t work.
6. Good software testing is a challenging intellectual process.
7. Only through judgment and skill, exercised cooperatively throughout the entire project, are
we able to do the right things at the right times to effectively test our products.
What tests would be most valuable right now?
emphasizes people, seeking bugs that stakeholders care about
12. Differences Between Perspectives
Analytical School Factory School Quality Assurance School Context-Driven School
Not a form of
Outside the testing skill
set. Let someone else do
Do it if requested by
Hard to prove
Definitely do it.
Usability bugs are bugs.
Testing Without Specs Impossible
Some kind of spec is
Force developers to
follow the process
Do what you can
to be useful
Ask questions if
Dig up “hidden”
to hire, train and manage
are based on practice, not
13. What is testing?
Analytical School Factory School Quality
A branch of
A branch of
A branch of
Part of customer
◦ Lessons Learned in Software Testing: A Context-Driven Approach
- Cem Kaner, James Bach & Bret Pettichord
◦ Schools of software testing, Cem Kaner http://kaner.com/?p=15
Origin of this Analysis: “Lessons
Learned in Software Testing”
aka “Structural” testing
Dozens of code-coverage metrics have been designed and compared
Provides an objective “measure” of testing
Precise and detailed specifications are a prerequisite for testing
Testers verify that the software behavior conforms to its specification
High-reliability industry (e.g. Telecom)
Boris Beizer, Paul Jorgensen, Robert V. Binder, John Musa
Make sure that every requirement has been tested
Require clear boundaries between testing and other
activities (start/stop criteria)
" Resist changing plans (complicates progress tracking)
" Software testing assembly line (V-model)
" Accept management assumptions about testing
" Encourage standards, “best practices,” and certification
Rex Black, Dorothy Graham
The Gatekeeper - The software isn’t ready until QA says it’s ready
Prefer “Quality Assurance” over “Testing”
Testing is a stepping stone to “process improvement”
May alienate developers
Organizations under stress
Context-driven testers choose their testing objectives, techniques, and deliverables (including test documentation) by looking first to the details of the specific situation, including the desires of the stakeholders who commissioned the testing. The essence of context-driven testing is project-appropriate application of skill and judgment. The Context-Driven School of testing places this approach to testing within a humanistic social and ethical framework.
Ultimately, context-driven testing is about doing the best we can with what we get. Rather than trying to apply “best practices,” we accept that very different practices (even different definitions of common testing terms) will work best under different circumstances.
Context-Driven School: Core Beliefs
Software is created by people.
People set the context. Testing finds bugs.
A bug is anything that could bug a stakeholder.
Testing provides information to the project
Testing is a skilled, mental activity
Testing is multidisciplinary
Key Question: What tests would be most valuable right now?
Expect changes. Adapt testing plans based on test results
Unchallenged assumptions are dangerous.
Effectiveness of test strategies can only be determined with field research
Focus on skill over practice
Commercial, Market-driven Software
Cem Kaner, Brian Marick, James Bach
" Used for test-driven development
" Developers must provide automation frameworks
" Slow to appreciate value of exploratory testing
" IT Consulting
" ASP Development
" Pattern Workshops