This document summarizes trends in software testing that could diminish its effectiveness and enjoyment. It notes an increasing focus on verification over validation, precise measurement over subjective judgement, and short-term metrics over long-term quality. This narrowing scope risks making testers isolated and limiting their creativity, motivation and ability to consider the full context of a project. The document advocates a holistic and subjective approach that considers people and intangible factors, not just short-term quantifiable results. Subjectivity and considering the whole system, not just parts, are presented as useful for testing.

1. Testing is an Island
A Software Testing Dystopia
Rikard Edgren
TIBCO Spotfire Division
EuroSTAR 2008-11-12, Haag

2. My problem
• Software Testing is a lot of fun
• I see trends that make it less fun
• Don’t destroy the good things about
software testing!

3. Testing is an Island
• Introduction film...

4. Software Complexity
• Software is complex; and the usage of
software is even more complex
• There is not one solution; no silver bullets
• We need to look in many different ways

5. 3D Software Testing
...there are so many angles; we need to feel what is most important

6. Trends
• “all testing is verification testing”
• Status/certification/standards/specialization
• You can’t control what you can’t measure
• We only think about results; not about people
• Management rules

7. Project Thinking
• SMART projects are easier to manage
• The future over-usage will give
– Testing is very limited in scope
– All testing activities are measurable
– Only sometimes making a better product
Keyword Negative side-effect
Specific we become narrow-minded
Measurable can’t use fluffy, good things
Attainable can’t aim really high; no holistic stuff
Realistic no objections, but I’d prefer Important
Time-bound gives short-term thinking

8. Verification & Validation
• Validation will mean looking at high-level; and
verification at low-level
• That validation means that the product actually
does good things is forgotten
• The combinations and grey areas in between will
be ignored

9. Product Investigator
• simultaneous verification & validation
• many testers do this today, and it helps
the products
– but can’t be done by automation
– and is very difficult to measure

10. too much focus on... ...and we lose on...
precision serendipity
objectivity subjectivity
specialists generalists
metrics judgement
presentation result
standardization creativity
short-term long-term
independence organic
uniformity diversity
verification validation
speed depth
ease complexity

11. Non-measurable people stuff
• joy
• participation
• optimism
• creativity
• learning
• trust
• tolerance
• motivation

12. ”no man is an island”
(John Donne)

13. The Future Tester
• Precise
• Alone
• Quiet

14. Holistic-Subjective Approach
• Holistic
– time
– product
– whole company
– all quality aspects
• Subjective
– people
– quality
– no metrics

15. Summary
• subjectivity is something useful
• the seemingly sound ideas have risks
• people and results go hand in hand

16. Questions
• ???
• I have a paper with details and references
redgren@tibco.com