Four Schools of
MORE ABOUT CONTEXT DRIVEN SCHOOL
Four Schools of Software Testing
Analytical School: Core Beliefs
• Software is a logical artifact
• Testing is a branch of CS/Mathematics
• Objective, rigorous, comprehensive
• Testing techniques must have a logico-mathematical form
• “one right answer”
• Testing is technical
• Which techniques should we use?
sees testing as rigorous and technical with many proponents in academia
Factory School: Core Beliefs
• Software development is a project
• Testing is a measure of progress
• Testing must be managed
• Predictable, repeatable, planned
• Testing must be cost-effective
• Low-skilled workers require direction
• Testing is following rules
• How can we measure whether we’re making progress? When will we be done?
sees testing as a way to measure progress with emphasis on cost and repeatable
Quality Assurance School: Core Beliefs
• Software quality requires discipline
• Testing determines whether development processes are being
• Testers may need to police developers to follow the rules
• Testers have to protect users from bad software
• Are we following a good process?
emphasizes process, policing developers and acting as the
“CONTEXT-DRIVEN HAS TESTERS
LEARNING AS THEY TEST”
The Seven Basic Principles of the Context-
1. The value of any practice depends on its context.
2. There are good practices in context, but there are no best practices.
3. People, working together, are the most important part of any project’s context.
4. Projects unfold over time in ways that are often not predictable.
5. The product is a solution. If the problem isn’t solved, the product doesn’t work.
6. Good software testing is a challenging intellectual process.
7. Only through judgment and skill, exercised cooperatively throughout the entire project, are
we able to do the right things at the right times to effectively test our products.
What tests would be most valuable right now?
emphasizes people, seeking bugs that stakeholders care about
◦ Concurrent test design and test execution
◦ Rapid learning
- develops team understanding of risks
- develops tester’s ability to design tests for
A fifth testing
uses testing to prove
that development is
Agile School: Core Beliefs
• Software is an ongoing conversation
• Testing tells us that a development story is complete
• Tests must be automated
• Is the story done?
Differences Between Perspectives
Analytical School Factory School Quality Assurance School Context-Driven School
Not a form of
Outside the testing skill
set. Let someone else do
Do it if requested by
Hard to prove
Definitely do it.
Usability bugs are bugs.
Testing Without Specs Impossible
Some kind of spec is
Force developers to
follow the process
Do what you can
to be useful
Ask questions if
Dig up “hidden”
to hire, train and manage
are based on practice, not
What is testing?
Analytical School Factory School Quality
A branch of
A branch of
A branch of
Part of customer
Where are you
Where is your client?
◦What happens when people of
different schools work together?
◦Can we cross-fertilize between
◦Do I have to pick a school?
◦ Lessons Learned in Software Testing: A Context-Driven Approach
- Cem Kaner, James Bach & Bret Pettichord
◦ Schools of software testing, Cem Kaner http://kaner.com/?p=15
Origin of this Analysis: “Lessons
Learned in Software Testing”
Thank you all for your
Code Coverage aka “Structural” testing Dozens of code-coverage metrics have been designed and compared Provides an objective “measure” of testing
Implications Precise and detailed specifications are a prerequisite for testing Testers verify that the software behavior conforms to its specification
Most prevalent Academia High-reliability industry (e.g. Telecom)
Authors Boris Beizer, Paul Jorgensen, Robert V. Binder, John Musa
Requirements Traceability Make sure that every requirement has been tested
Implications Require clear boundaries between testing and other activities (start/stop criteria) " Resist changing plans (complicates progress tracking) " Software testing assembly line (V-model) " Accept management assumptions about testing " Encourage standards, “best practices,” and certification
Most Prevalent IT projects Government projects
Authors Rex Black, Dorothy Graham
Ex The Gatekeeper - The software isn’t ready until QA says it’s ready
Implications Prefer “Quality Assurance” over “Testing” Testing is a stepping stone to “process improvement” May alienate developers
Most Prevalent Large bureaucracies Organizations under stress
Authors Alka Jarvis
Context-driven testers choose their testing objectives, techniques, and deliverables (including test documentation) by looking first to the details of the specific situation, including the desires of the stakeholders who commissioned the testing. The essence of context-driven testing is project-appropriate application of skill and judgment. The Context-Driven School of testing places this approach to testing within a humanistic social and ethical framework.
Ultimately, context-driven testing is about doing the best we can with what we get. Rather than trying to apply “best practices,” we accept that very different practices (even different definitions of common testing terms) will work best under different circumstances.
Context-Driven School: Core Beliefs Software is created by people. People set the context. Testing finds bugs. A bug is anything that could bug a stakeholder. Testing provides information to the project Testing is a skilled, mental activity Testing is multidisciplinary Key Question: What tests would be most valuable right now?
Context-Driven School Implications Expect changes. Adapt testing plans based on test results Unchallenged assumptions are dangerous. Pragmatism Effectiveness of test strategies can only be determined with field research Focus on skill over practice
Most Prominent Commercial, Market-driven Software
Authors Cem Kaner, Brian Marick, James Bach
Unit Tests " Used for test-driven development
Implications " Developers must provide automation frameworks " Slow to appreciate value of exploratory testing