Download to read offline
Uploaded byMaaret Pyhäjärvi
BTDConf: Breaking Illusions: Testing is Your Most Valuable Asset
1. Testing provides valuable feedback that breaks illusions about code quality and helps improve software. Testing focuses on both checking code systematically and exploring it openly. 2. The context of testing matters - testing in production has different risks than testing in development. Skilled testers explore products and businesses rather than just checking facts. 3. Testers help dispel illusions about code by finding bugs, not just defects, and empirical evidence is more useful than speculation alone. The role of testing is changing as development processes become more automated.
More Related Content
Similar to BTDConf: Breaking Illusions: Testing is Your Most Valuable Asset
BTDConf: Breaking Illusions: Testing is Your Most Valuable Asset
- 1. Breaking Illusions: Testingis your most valuable asset! Maaret Pyhäjärvi Email: <maaret@iki.fi> | Twitter: maaretp Maaret Pyhäjärvi Nimeä | Attribution (Finland) http://creativecommons.org/licenses/by/1.0/fi/ http://creativecommons.org/licenses/by/1.0/fi/deed.en
- 2. Testing is aboutFeedback Checking • Machines can do with algorithmic decision rules • Focuses on specific facts • Cmp. “compiling”, “generating” Exploring • Skilled humans can do • Open-ended search for information • Cmp. “programming” Testing not testers – but becoming a skilled tester requires time and focus
- 3. Context Matters –Expectations Differ • Testing in Production? – Rolling releases: limit the extent of impact – Product lock-in: limit the options • Testing in the delivery chain? – Development teams skilled in exploring (as well as checking) – Internal releases – Beta releases – Releases • Available skills? – Intentional vs. accidental production feedback
- 4. Testers don’t breakyour code, they break your illusions about the code. -- adapted from James Bach
- 5. Dispelling Illusions: Empirical Evidenceover Speculation VALU E ILLUSIONS
- 6. The World HasAlready Changed COMMODITY TESTERS • Manual checkers • Tests are an artifact SKILLED TESTERS • Explorers of products and businesses • Testing is a performance • DEVELOPERS IN TEST • All development is automating, testing is just a domain of development
- 7. Exploration Skills Source: Adaptedfrom James Bach, Jon Bach, Michael Bolton. Exploratory Testing Dynamics. v.2.2. 2009 Self- managemen t Developing ideas Examining product Done To Do Issues Coverage All sources available Best use of time – effective and efficient work Making models Tool support – creative solutions Risk-based testing – scientific approach Keeping one’s eyes open
- 9. Things Can LookDifferent from Different Perspectives
- 10. Culture of Feedbackand Experiments • Illusions are meant to be broken • Teams over individuals – Diverse teams (personalities, skills) deliver better results – Social dissonance gets the team talking and voices out uncertainties to address on time • Experiment: try something different
- 11. Summary 1. TESTING isFEEDBACK and LEARNING. EMPIRICAL supports learning over theoretical. FAST is better than slower. Not outsourcing the cost of feedback could be better. 2. CONTEXT matters: sometimes the “best testing” happens live – as long as you’re covering the checking part; testing != testers 3. BUGS not defects: anything this might bug a user, there’s many types of illusions we have 4. COMMODITY testers and SKILLED testers are two completely different breeds 5. SKILL to break DIVERSE illusions – skill in team, not in individual 6. DIVERSITY brings good results: people who choose to love testing tend to be different from those who choose to love checking and development
Editor's Notes
- #2 This presentation is about FEEDBACK. I don’t care if you have specialized testers to help you work out the multidimensional feedback or not. Feedback is still valuable, and for all contexts, asking for all the feedback from the end users / product management just isn’t the thing to do… I come from an organization where (some) developers strongly still believe that it would be better to hear about bugs from the end users so that we would only hear about “real bugs” and that regardless of me showing I do understand what they complain on, it would still be different (there would be less of it…)
- #3 Point of this slide: TESTING is FEEDBACK and LEARNING. EMPIRICAL supports learning over theoretical. FAST is better than slower. Not outsourcing the cost of feedback could be better. MENTION: Testing looks into more than functionality. Other quality criteria and stakeholder views are particular relevance. Emergent information, you do not know what you are looking for For more info, see http://www.satisfice.com/blog/archives/856 Checking the process of making evaluations by applying algorithmic decision rules to specific observations of a product. Testing the process of evaluating a product by learning about it through exploration and experimentation, which includes to some degree: questioning, study, modeling, observation, inference, etc.
- #4 Point of this slide: CONTEXT matters: sometimes the “best testing” happens live – as long as you’re covering the checking part; testing != testers Example: Security vulnerabilities Vulnerabilities are considered real problems Still only the black market pays real money for reporting them Functional bugs / missing value reporting is actual effort too, and wastes time for people Something we as end users can do: reclamation I tend to work with PRODUCT BUSINESS and CUSTOMER ORGANIZATIONS, not delivering bespoke software. Inability to produce quality vs. productivity service – different stakeholder considerations, incl. “design with a change in mind’ I belong to context-driven school of testing – I know things can be different! History of having things tested in production – wasting a lot of time. Not every team has the most skilled developers but they can still have diverse teams delivering excellent results
- #5 Point of this slide: Testing helps with EFFECTIVENESS, as a productivity service. Your users might not really like reporting your bugs for you, as it wastes their time. It’s co-creation. The cake example. You can’t separate the ideas, ideas feeding the ideas and implementation.
- #6 Illusion stories: “I tested it, it works” “reporting engine” “no time for refactoring” / “releasing as scheduled over done” “Need document grouping feature over others” --- Ray Arell (Intel) giving apple example: showing you’re living with an illlusion. Tester mentality!! ---- Point of this slide: BUGS not defects: anything this might bug a user, there’s many types of illusions we have Finding really expensive bugs: missing value, wrong features Illusion: releasing as scheduled (sprint) vs. releasing when done (kanban) EMPIRICAL evidence trumps speculation. Every Single Time. – testobsessed
- #7 Point of this slide: COMMODITY testers and SKILLED testers are two completely different breeds The spider web idea. Your ideas of testers may be outdated. You get what you ask for, and if you ask for test cases, you get commodity approaches.
- #8 Point of this slide: SKILL to break DIVERSE illusions – skill in team, not in individual
- #9 Exploratory is a MINDSET / APPROACH, not an activity. Testing is PERFORMANCE, not ARTIFACT. Testing is testing even if context is agile.
- #10 Point of this slide: DIVERSITY brings good results: people who choose to love testing tend to be different from those who choose to love checking and development Not every team has the most skilled developers but they can still have diverse teams delivering excellent results
- #11 Creating culture of feedback and experiments with diversity of personalities and skills in the teams