SlideShare a Scribd company logo
1 of 11
Download to read offline
Breaking Illusions: Testing is
your most valuable asset!
Maaret Pyhäjärvi
Email: <maaret@iki.fi> | Twitter: maaretp
Maaret Pyhäjärvi
Nimeä | Attribution (Finland)
http://creativecommons.org/licenses/by/1.0/fi/
http://creativecommons.org/licenses/by/1.0/fi/deed.en
Testing is about Feedback
Checking
• Machines can do with
algorithmic decision
rules
• Focuses on specific
facts
• Cmp. “compiling”,
“generating”
Exploring
• Skilled humans can do
• Open-ended search for
information
• Cmp. “programming”
Testing not testers – but
becoming a skilled tester
requires time and focus
Context Matters – Expectations
Differ
• Testing in Production?
– Rolling releases: limit the extent of impact
– Product lock-in: limit the options
• Testing in the delivery chain?
– Development teams skilled in exploring (as
well as checking)
– Internal releases – Beta releases – Releases
• Available skills?
– Intentional vs. accidental production feedback
Testers don’t break your
code, they break your
illusions about the
code.
-- adapted from James Bach
Dispelling Illusions: Empirical
Evidence over Speculation
VALU
E
ILLUSIONS
The World Has Already
Changed
COMMODITY
TESTERS
• Manual checkers
• Tests are an artifact
SKILLED TESTERS
• Explorers of products
and businesses
• Testing is a performance
• DEVELOPERS IN TEST
• All development is
automating, testing is
just a domain of
development
Exploration Skills
Source: Adapted from James Bach, Jon Bach, Michael Bolton. Exploratory Testing
Dynamics. v.2.2. 2009
Self-
managemen
t
Developing
ideas
Examining
product
Done
To Do
Issues
Coverage
All sources available
Best use of time – effective and efficient work
Making models
Tool support – creative solutions
Risk-based testing – scientific approach
Keeping one’s eyes open
Things Can Look Different from
Different Perspectives
Culture of Feedback and
Experiments
• Illusions are meant to be broken
• Teams over individuals
– Diverse teams (personalities, skills) deliver
better results
– Social dissonance gets the team talking and
voices out uncertainties to address on time
• Experiment: try something different
Summary
1. TESTING is FEEDBACK and LEARNING. EMPIRICAL
supports learning over theoretical. FAST is better than
slower. Not outsourcing the cost of feedback could be better.
2. CONTEXT matters: sometimes the “best testing” happens
live – as long as you’re covering the checking part; testing !=
testers
3. BUGS not defects: anything this might bug a user, there’s
many types of illusions we have
4. COMMODITY testers and SKILLED testers are two
completely different breeds
5. SKILL to break DIVERSE illusions – skill in team, not in
individual
6. DIVERSITY brings good results: people who choose to love
testing tend to be different from those who choose to love
checking and development

More Related Content

What's hot

Exploratory testing
Exploratory testingExploratory testing
Exploratory testingHuib Schoots
 
Reducing Test Case Bloat2 1
Reducing Test Case Bloat2 1Reducing Test Case Bloat2 1
Reducing Test Case Bloat2 1Lanette Creamer
 
Introduction to Exploratory Testing
Introduction to Exploratory TestingIntroduction to Exploratory Testing
Introduction to Exploratory TestingCodrin Pruteanu
 
Open source bridge testing antipatterns presentation
Open source bridge testing antipatterns presentationOpen source bridge testing antipatterns presentation
Open source bridge testing antipatterns presentationmmrobins
 
Exploratory Testing in an Agile Context
Exploratory Testing in an Agile ContextExploratory Testing in an Agile Context
Exploratory Testing in an Agile ContextElisabeth Hendrickson
 
All you need know about testing
All you need know about testingAll you need know about testing
All you need know about testingJorge Barroso
 
Expecting the Unexpected: Preparing for Successful User Research Sessions (Do...
Expecting the Unexpected: Preparing for Successful User Research Sessions (Do...Expecting the Unexpected: Preparing for Successful User Research Sessions (Do...
Expecting the Unexpected: Preparing for Successful User Research Sessions (Do...Fiona Tranquada
 
Graham Thomas - The Testers Toolbox - EuroSTAR 2010
Graham Thomas - The Testers Toolbox - EuroSTAR 2010Graham Thomas - The Testers Toolbox - EuroSTAR 2010
Graham Thomas - The Testers Toolbox - EuroSTAR 2010TEST Huddle
 
Ajay Balamnrugadas - Weekend Testing, Skilled Software Testing Unleashed - Eu...
Ajay Balamnrugadas - Weekend Testing, Skilled Software Testing Unleashed - Eu...Ajay Balamnrugadas - Weekend Testing, Skilled Software Testing Unleashed - Eu...
Ajay Balamnrugadas - Weekend Testing, Skilled Software Testing Unleashed - Eu...TEST Huddle
 
Pragmatic programmer
Pragmatic programmerPragmatic programmer
Pragmatic programmerMaulik Shah
 
Lets Talk Toolbox Talks: How to Effectively Reinforce Safe Work Practices
Lets Talk Toolbox Talks: How to Effectively Reinforce Safe Work PracticesLets Talk Toolbox Talks: How to Effectively Reinforce Safe Work Practices
Lets Talk Toolbox Talks: How to Effectively Reinforce Safe Work PracticesTriumvirate Environmental
 
Shrini Kulkarni - Software Metrics - So Simple, Yet So Dangerous
Shrini Kulkarni -  Software Metrics - So Simple, Yet So Dangerous Shrini Kulkarni -  Software Metrics - So Simple, Yet So Dangerous
Shrini Kulkarni - Software Metrics - So Simple, Yet So Dangerous TEST Huddle
 
Engineering Anti-Patterns @JQueryTO 2014
Engineering Anti-Patterns @JQueryTO 2014Engineering Anti-Patterns @JQueryTO 2014
Engineering Anti-Patterns @JQueryTO 2014Nahim Nasser
 
6 myths of Software Testing (As I have seen during my testing journey)
6 myths of Software Testing (As I have seen during my testing journey)6 myths of Software Testing (As I have seen during my testing journey)
6 myths of Software Testing (As I have seen during my testing journey)Brian Osman
 
How to think as a technical tester
How to think as a technical testerHow to think as a technical tester
How to think as a technical testerBassam Al-Khatib
 
TestWorksConf: Experience exploratory testing
TestWorksConf: Experience exploratory testingTestWorksConf: Experience exploratory testing
TestWorksConf: Experience exploratory testingMaaret Pyhäjärvi
 
What is What is Professional Testing?
What is What is Professional Testing?What is What is Professional Testing?
What is What is Professional Testing?James Thomas
 

What's hot (20)

Make product not war
Make product not warMake product not war
Make product not war
 
Exploratory testing
Exploratory testingExploratory testing
Exploratory testing
 
Exploratory Testing in Practice
Exploratory Testing in PracticeExploratory Testing in Practice
Exploratory Testing in Practice
 
Reducing Test Case Bloat2 1
Reducing Test Case Bloat2 1Reducing Test Case Bloat2 1
Reducing Test Case Bloat2 1
 
Introduction to Exploratory Testing
Introduction to Exploratory TestingIntroduction to Exploratory Testing
Introduction to Exploratory Testing
 
Open source bridge testing antipatterns presentation
Open source bridge testing antipatterns presentationOpen source bridge testing antipatterns presentation
Open source bridge testing antipatterns presentation
 
Exploratory Testing in an Agile Context
Exploratory Testing in an Agile ContextExploratory Testing in an Agile Context
Exploratory Testing in an Agile Context
 
All you need know about testing
All you need know about testingAll you need know about testing
All you need know about testing
 
Expecting the Unexpected: Preparing for Successful User Research Sessions (Do...
Expecting the Unexpected: Preparing for Successful User Research Sessions (Do...Expecting the Unexpected: Preparing for Successful User Research Sessions (Do...
Expecting the Unexpected: Preparing for Successful User Research Sessions (Do...
 
Graham Thomas - The Testers Toolbox - EuroSTAR 2010
Graham Thomas - The Testers Toolbox - EuroSTAR 2010Graham Thomas - The Testers Toolbox - EuroSTAR 2010
Graham Thomas - The Testers Toolbox - EuroSTAR 2010
 
Ajay Balamnrugadas - Weekend Testing, Skilled Software Testing Unleashed - Eu...
Ajay Balamnrugadas - Weekend Testing, Skilled Software Testing Unleashed - Eu...Ajay Balamnrugadas - Weekend Testing, Skilled Software Testing Unleashed - Eu...
Ajay Balamnrugadas - Weekend Testing, Skilled Software Testing Unleashed - Eu...
 
Pragmatic programmer
Pragmatic programmerPragmatic programmer
Pragmatic programmer
 
Lets Talk Toolbox Talks: How to Effectively Reinforce Safe Work Practices
Lets Talk Toolbox Talks: How to Effectively Reinforce Safe Work PracticesLets Talk Toolbox Talks: How to Effectively Reinforce Safe Work Practices
Lets Talk Toolbox Talks: How to Effectively Reinforce Safe Work Practices
 
Shrini Kulkarni - Software Metrics - So Simple, Yet So Dangerous
Shrini Kulkarni -  Software Metrics - So Simple, Yet So Dangerous Shrini Kulkarni -  Software Metrics - So Simple, Yet So Dangerous
Shrini Kulkarni - Software Metrics - So Simple, Yet So Dangerous
 
Engineering Anti-Patterns @JQueryTO 2014
Engineering Anti-Patterns @JQueryTO 2014Engineering Anti-Patterns @JQueryTO 2014
Engineering Anti-Patterns @JQueryTO 2014
 
Intro to TDD
Intro to TDDIntro to TDD
Intro to TDD
 
6 myths of Software Testing (As I have seen during my testing journey)
6 myths of Software Testing (As I have seen during my testing journey)6 myths of Software Testing (As I have seen during my testing journey)
6 myths of Software Testing (As I have seen during my testing journey)
 
How to think as a technical tester
How to think as a technical testerHow to think as a technical tester
How to think as a technical tester
 
TestWorksConf: Experience exploratory testing
TestWorksConf: Experience exploratory testingTestWorksConf: Experience exploratory testing
TestWorksConf: Experience exploratory testing
 
What is What is Professional Testing?
What is What is Professional Testing?What is What is Professional Testing?
What is What is Professional Testing?
 

Similar to BTDConf: Breaking Illusions: Testing is Your Most Valuable Asset

Intro to Product Discovery
Intro to Product DiscoveryIntro to Product Discovery
Intro to Product DiscoveryMatthew Godfrey
 
Software testing _mod_9
Software testing _mod_9Software testing _mod_9
Software testing _mod_9hellosashi
 
Learn Learning + Prototype Testing
Learn Learning + Prototype TestingLearn Learning + Prototype Testing
Learn Learning + Prototype TestingDave Hora
 
A Rapid Introduction to Rapid Software Testing
A Rapid Introduction to Rapid Software TestingA Rapid Introduction to Rapid Software Testing
A Rapid Introduction to Rapid Software TestingTechWell
 
A Rapid Introduction to Rapid Software Testing
A Rapid Introduction to Rapid Software TestingA Rapid Introduction to Rapid Software Testing
A Rapid Introduction to Rapid Software TestingTechWell
 
Michael Bolton - Two Futures of Software Testing
Michael Bolton - Two Futures of Software TestingMichael Bolton - Two Futures of Software Testing
Michael Bolton - Two Futures of Software TestingTEST Huddle
 
Testing the unknown: the art and science of working with hypothesis
Testing the unknown: the art and science of working with hypothesisTesting the unknown: the art and science of working with hypothesis
Testing the unknown: the art and science of working with hypothesisArdita Karaj
 
Exploratory Testing Explained and Experienced
Exploratory Testing Explained and ExperiencedExploratory Testing Explained and Experienced
Exploratory Testing Explained and ExperiencedMaaret Pyhäjärvi
 
A New Model For Testing
A New Model For TestingA New Model For Testing
A New Model For TestingTEST Huddle
 
1. Concept Development By Alejandro
1. Concept Development By Alejandro1. Concept Development By Alejandro
1. Concept Development By AlejandroUurdmandakh Gongor
 
How to get what you really want from Testing' with Michael Bolton
How to get what you really want from Testing' with Michael BoltonHow to get what you really want from Testing' with Michael Bolton
How to get what you really want from Testing' with Michael BoltonTEST Huddle
 
A Rapid Introduction to Rapid Software Testing
A Rapid Introduction to Rapid Software TestingA Rapid Introduction to Rapid Software Testing
A Rapid Introduction to Rapid Software TestingTechWell
 
Hypothesize your Way into Better Product Development
Hypothesize your Way into Better Product DevelopmentHypothesize your Way into Better Product Development
Hypothesize your Way into Better Product DevelopmentElias Nema
 
Arch factory - Agile Design: Best Practices
Arch factory - Agile Design: Best PracticesArch factory - Agile Design: Best Practices
Arch factory - Agile Design: Best PracticesIgor Moochnick
 
Product Discovery Stories: when and how to use a discovery sprint to validate...
Product Discovery Stories: when and how to use a discovery sprint to validate...Product Discovery Stories: when and how to use a discovery sprint to validate...
Product Discovery Stories: when and how to use a discovery sprint to validate...Cprime
 
Getting Started with User Research
Getting Started with User ResearchGetting Started with User Research
Getting Started with User ResearchDiane Loviglio
 

Similar to BTDConf: Breaking Illusions: Testing is Your Most Valuable Asset (20)

Intro to Product Discovery
Intro to Product DiscoveryIntro to Product Discovery
Intro to Product Discovery
 
Software testing _mod_9
Software testing _mod_9Software testing _mod_9
Software testing _mod_9
 
How to evaluate a tester
How to evaluate a testerHow to evaluate a tester
How to evaluate a tester
 
Learn Learning + Prototype Testing
Learn Learning + Prototype TestingLearn Learning + Prototype Testing
Learn Learning + Prototype Testing
 
Lean Startup 301
Lean Startup 301Lean Startup 301
Lean Startup 301
 
A Rapid Introduction to Rapid Software Testing
A Rapid Introduction to Rapid Software TestingA Rapid Introduction to Rapid Software Testing
A Rapid Introduction to Rapid Software Testing
 
A Rapid Introduction to Rapid Software Testing
A Rapid Introduction to Rapid Software TestingA Rapid Introduction to Rapid Software Testing
A Rapid Introduction to Rapid Software Testing
 
Core define and_win_cmd_line gr
Core define and_win_cmd_line grCore define and_win_cmd_line gr
Core define and_win_cmd_line gr
 
Michael Bolton - Two Futures of Software Testing
Michael Bolton - Two Futures of Software TestingMichael Bolton - Two Futures of Software Testing
Michael Bolton - Two Futures of Software Testing
 
Testing the unknown: the art and science of working with hypothesis
Testing the unknown: the art and science of working with hypothesisTesting the unknown: the art and science of working with hypothesis
Testing the unknown: the art and science of working with hypothesis
 
Exploratory Testing Explained and Experienced
Exploratory Testing Explained and ExperiencedExploratory Testing Explained and Experienced
Exploratory Testing Explained and Experienced
 
New model
New modelNew model
New model
 
A New Model For Testing
A New Model For TestingA New Model For Testing
A New Model For Testing
 
1. Concept Development By Alejandro
1. Concept Development By Alejandro1. Concept Development By Alejandro
1. Concept Development By Alejandro
 
How to get what you really want from Testing' with Michael Bolton
How to get what you really want from Testing' with Michael BoltonHow to get what you really want from Testing' with Michael Bolton
How to get what you really want from Testing' with Michael Bolton
 
A Rapid Introduction to Rapid Software Testing
A Rapid Introduction to Rapid Software TestingA Rapid Introduction to Rapid Software Testing
A Rapid Introduction to Rapid Software Testing
 
Hypothesize your Way into Better Product Development
Hypothesize your Way into Better Product DevelopmentHypothesize your Way into Better Product Development
Hypothesize your Way into Better Product Development
 
Arch factory - Agile Design: Best Practices
Arch factory - Agile Design: Best PracticesArch factory - Agile Design: Best Practices
Arch factory - Agile Design: Best Practices
 
Product Discovery Stories: when and how to use a discovery sprint to validate...
Product Discovery Stories: when and how to use a discovery sprint to validate...Product Discovery Stories: when and how to use a discovery sprint to validate...
Product Discovery Stories: when and how to use a discovery sprint to validate...
 
Getting Started with User Research
Getting Started with User ResearchGetting Started with User Research
Getting Started with User Research
 

More from Maaret Pyhäjärvi

ATAGTR '20 Keynote: Building status
ATAGTR '20 Keynote: Building statusATAGTR '20 Keynote: Building status
ATAGTR '20 Keynote: Building statusMaaret Pyhäjärvi
 
MoT Cork: Exploring Realities of Testing - From Theory to Practice
MoT Cork: Exploring Realities of Testing - From Theory to PracticeMoT Cork: Exploring Realities of Testing - From Theory to Practice
MoT Cork: Exploring Realities of Testing - From Theory to PracticeMaaret Pyhäjärvi
 
Testing Voices '21 keynote: Testing Becoming Harder To BeValuable
Testing Voices '21 keynote: Testing Becoming Harder To BeValuableTesting Voices '21 keynote: Testing Becoming Harder To BeValuable
Testing Voices '21 keynote: Testing Becoming Harder To BeValuableMaaret Pyhäjärvi
 
TestJSConf: Go Find What We May Have Missed
TestJSConf: Go Find What We May Have MissedTestJSConf: Go Find What We May Have Missed
TestJSConf: Go Find What We May Have MissedMaaret Pyhäjärvi
 
SauceCon: It's More Complicated than THAT
SauceCon: It's More Complicated than THATSauceCon: It's More Complicated than THAT
SauceCon: It's More Complicated than THATMaaret Pyhäjärvi
 
TSQA: Something In The Way We Test
TSQA: Something In The Way We TestTSQA: Something In The Way We Test
TSQA: Something In The Way We TestMaaret Pyhäjärvi
 
ATAGTR: Targeting Errors Of Omission
ATAGTR: Targeting Errors Of OmissionATAGTR: Targeting Errors Of Omission
ATAGTR: Targeting Errors Of OmissionMaaret Pyhäjärvi
 
Pivotal Moments, Pivotal Experiences
Pivotal Moments, Pivotal ExperiencesPivotal Moments, Pivotal Experiences
Pivotal Moments, Pivotal ExperiencesMaaret Pyhäjärvi
 
Keynote at Sanae Beer.EX: The Automationist's Gambit
Keynote at Sanae Beer.EX: The Automationist's GambitKeynote at Sanae Beer.EX: The Automationist's Gambit
Keynote at Sanae Beer.EX: The Automationist's GambitMaaret Pyhäjärvi
 
Testing Assembly: From Observations to Insights on Test Specialist Careers
Testing Assembly: From Observations to Insights on Test Specialist CareersTesting Assembly: From Observations to Insights on Test Specialist Careers
Testing Assembly: From Observations to Insights on Test Specialist CareersMaaret Pyhäjärvi
 
Let's Do a Thing and Call it Foo
Let's Do a Thing and Call it FooLet's Do a Thing and Call it Foo
Let's Do a Thing and Call it FooMaaret Pyhäjärvi
 
Selenium Conf: Patterns to Whole Team Test Automation Transformation
Selenium Conf: Patterns to Whole Team Test Automation TransformationSelenium Conf: Patterns to Whole Team Test Automation Transformation
Selenium Conf: Patterns to Whole Team Test Automation TransformationMaaret Pyhäjärvi
 
AxiomSummit 2022: A Practitioner's Guide to Contemporary Exploratory Testing
AxiomSummit 2022: A Practitioner's Guide to Contemporary Exploratory TestingAxiomSummit 2022: A Practitioner's Guide to Contemporary Exploratory Testing
AxiomSummit 2022: A Practitioner's Guide to Contemporary Exploratory TestingMaaret Pyhäjärvi
 
MoT Sfax: Good Exploratory Testing Tactics for Taking Over Testing
MoT Sfax: Good Exploratory Testing Tactics for Taking Over TestingMoT Sfax: Good Exploratory Testing Tactics for Taking Over Testing
MoT Sfax: Good Exploratory Testing Tactics for Taking Over TestingMaaret Pyhäjärvi
 
Geekle QA: Automationist's Gambit
Geekle QA: Automationist's GambitGeekle QA: Automationist's Gambit
Geekle QA: Automationist's GambitMaaret Pyhäjärvi
 
Programming Assignments in Tester Interviews
Programming Assignments in Tester InterviewsProgramming Assignments in Tester Interviews
Programming Assignments in Tester InterviewsMaaret Pyhäjärvi
 
ATAGTR Keynote: I have a rock in my shoe - Navigating Improvement
ATAGTR Keynote: I have a rock in my shoe - Navigating ImprovementATAGTR Keynote: I have a rock in my shoe - Navigating Improvement
ATAGTR Keynote: I have a rock in my shoe - Navigating ImprovementMaaret Pyhäjärvi
 
MoT Sarajevo: How to get better at Exploratory testing
MoT Sarajevo: How to get better at Exploratory testingMoT Sarajevo: How to get better at Exploratory testing
MoT Sarajevo: How to get better at Exploratory testingMaaret Pyhäjärvi
 
HUSTEF '21 Keynote: Hands Off Exploratory Testing - Managing at Scale
HUSTEF '21 Keynote: Hands Off Exploratory Testing - Managing at ScaleHUSTEF '21 Keynote: Hands Off Exploratory Testing - Managing at Scale
HUSTEF '21 Keynote: Hands Off Exploratory Testing - Managing at ScaleMaaret Pyhäjärvi
 
EuroSTAR '21 Keynote: Automationist's Gambit
EuroSTAR '21 Keynote: Automationist's GambitEuroSTAR '21 Keynote: Automationist's Gambit
EuroSTAR '21 Keynote: Automationist's GambitMaaret Pyhäjärvi
 

More from Maaret Pyhäjärvi (20)

ATAGTR '20 Keynote: Building status
ATAGTR '20 Keynote: Building statusATAGTR '20 Keynote: Building status
ATAGTR '20 Keynote: Building status
 
MoT Cork: Exploring Realities of Testing - From Theory to Practice
MoT Cork: Exploring Realities of Testing - From Theory to PracticeMoT Cork: Exploring Realities of Testing - From Theory to Practice
MoT Cork: Exploring Realities of Testing - From Theory to Practice
 
Testing Voices '21 keynote: Testing Becoming Harder To BeValuable
Testing Voices '21 keynote: Testing Becoming Harder To BeValuableTesting Voices '21 keynote: Testing Becoming Harder To BeValuable
Testing Voices '21 keynote: Testing Becoming Harder To BeValuable
 
TestJSConf: Go Find What We May Have Missed
TestJSConf: Go Find What We May Have MissedTestJSConf: Go Find What We May Have Missed
TestJSConf: Go Find What We May Have Missed
 
SauceCon: It's More Complicated than THAT
SauceCon: It's More Complicated than THATSauceCon: It's More Complicated than THAT
SauceCon: It's More Complicated than THAT
 
TSQA: Something In The Way We Test
TSQA: Something In The Way We TestTSQA: Something In The Way We Test
TSQA: Something In The Way We Test
 
ATAGTR: Targeting Errors Of Omission
ATAGTR: Targeting Errors Of OmissionATAGTR: Targeting Errors Of Omission
ATAGTR: Targeting Errors Of Omission
 
Pivotal Moments, Pivotal Experiences
Pivotal Moments, Pivotal ExperiencesPivotal Moments, Pivotal Experiences
Pivotal Moments, Pivotal Experiences
 
Keynote at Sanae Beer.EX: The Automationist's Gambit
Keynote at Sanae Beer.EX: The Automationist's GambitKeynote at Sanae Beer.EX: The Automationist's Gambit
Keynote at Sanae Beer.EX: The Automationist's Gambit
 
Testing Assembly: From Observations to Insights on Test Specialist Careers
Testing Assembly: From Observations to Insights on Test Specialist CareersTesting Assembly: From Observations to Insights on Test Specialist Careers
Testing Assembly: From Observations to Insights on Test Specialist Careers
 
Let's Do a Thing and Call it Foo
Let's Do a Thing and Call it FooLet's Do a Thing and Call it Foo
Let's Do a Thing and Call it Foo
 
Selenium Conf: Patterns to Whole Team Test Automation Transformation
Selenium Conf: Patterns to Whole Team Test Automation TransformationSelenium Conf: Patterns to Whole Team Test Automation Transformation
Selenium Conf: Patterns to Whole Team Test Automation Transformation
 
AxiomSummit 2022: A Practitioner's Guide to Contemporary Exploratory Testing
AxiomSummit 2022: A Practitioner's Guide to Contemporary Exploratory TestingAxiomSummit 2022: A Practitioner's Guide to Contemporary Exploratory Testing
AxiomSummit 2022: A Practitioner's Guide to Contemporary Exploratory Testing
 
MoT Sfax: Good Exploratory Testing Tactics for Taking Over Testing
MoT Sfax: Good Exploratory Testing Tactics for Taking Over TestingMoT Sfax: Good Exploratory Testing Tactics for Taking Over Testing
MoT Sfax: Good Exploratory Testing Tactics for Taking Over Testing
 
Geekle QA: Automationist's Gambit
Geekle QA: Automationist's GambitGeekle QA: Automationist's Gambit
Geekle QA: Automationist's Gambit
 
Programming Assignments in Tester Interviews
Programming Assignments in Tester InterviewsProgramming Assignments in Tester Interviews
Programming Assignments in Tester Interviews
 
ATAGTR Keynote: I have a rock in my shoe - Navigating Improvement
ATAGTR Keynote: I have a rock in my shoe - Navigating ImprovementATAGTR Keynote: I have a rock in my shoe - Navigating Improvement
ATAGTR Keynote: I have a rock in my shoe - Navigating Improvement
 
MoT Sarajevo: How to get better at Exploratory testing
MoT Sarajevo: How to get better at Exploratory testingMoT Sarajevo: How to get better at Exploratory testing
MoT Sarajevo: How to get better at Exploratory testing
 
HUSTEF '21 Keynote: Hands Off Exploratory Testing - Managing at Scale
HUSTEF '21 Keynote: Hands Off Exploratory Testing - Managing at ScaleHUSTEF '21 Keynote: Hands Off Exploratory Testing - Managing at Scale
HUSTEF '21 Keynote: Hands Off Exploratory Testing - Managing at Scale
 
EuroSTAR '21 Keynote: Automationist's Gambit
EuroSTAR '21 Keynote: Automationist's GambitEuroSTAR '21 Keynote: Automationist's Gambit
EuroSTAR '21 Keynote: Automationist's Gambit
 

Recently uploaded

The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...
The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...
The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...kalichargn70th171
 
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdfAndrey Devyatkin
 
eSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration toolseSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration toolsosttopstonverter
 
Introduction to Firebase Workshop Slides
Introduction to Firebase Workshop SlidesIntroduction to Firebase Workshop Slides
Introduction to Firebase Workshop Slidesvaideheekore1
 
Ronisha Informatics Private Limited Catalogue
Ronisha Informatics Private Limited CatalogueRonisha Informatics Private Limited Catalogue
Ronisha Informatics Private Limited Catalogueitservices996
 
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4jGraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4jNeo4j
 
Tech Tuesday Slides - Introduction to Project Management with OnePlan's Work ...
Tech Tuesday Slides - Introduction to Project Management with OnePlan's Work ...Tech Tuesday Slides - Introduction to Project Management with OnePlan's Work ...
Tech Tuesday Slides - Introduction to Project Management with OnePlan's Work ...OnePlan Solutions
 
Keeping your build tool updated in a multi repository world
Keeping your build tool updated in a multi repository worldKeeping your build tool updated in a multi repository world
Keeping your build tool updated in a multi repository worldRoberto Pérez Alcolea
 
[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf
[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf
[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdfSteve Caron
 
VictoriaMetrics Q1 Meet Up '24 - Community & News Update
VictoriaMetrics Q1 Meet Up '24 - Community & News UpdateVictoriaMetrics Q1 Meet Up '24 - Community & News Update
VictoriaMetrics Q1 Meet Up '24 - Community & News UpdateVictoriaMetrics
 
Strategies for using alternative queries to mitigate zero results
Strategies for using alternative queries to mitigate zero resultsStrategies for using alternative queries to mitigate zero results
Strategies for using alternative queries to mitigate zero resultsJean Silva
 
What’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 UpdatesWhat’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 UpdatesVictoriaMetrics
 
Understanding Plagiarism: Causes, Consequences and Prevention.pptx
Understanding Plagiarism: Causes, Consequences and Prevention.pptxUnderstanding Plagiarism: Causes, Consequences and Prevention.pptx
Understanding Plagiarism: Causes, Consequences and Prevention.pptxSasikiranMarri
 
Mastering Project Planning with Microsoft Project 2016.pptx
Mastering Project Planning with Microsoft Project 2016.pptxMastering Project Planning with Microsoft Project 2016.pptx
Mastering Project Planning with Microsoft Project 2016.pptxAS Design & AST.
 
OpenChain AI Study Group - Europe and Asia Recap - 2024-04-11 - Full Recording
OpenChain AI Study Group - Europe and Asia Recap - 2024-04-11 - Full RecordingOpenChain AI Study Group - Europe and Asia Recap - 2024-04-11 - Full Recording
OpenChain AI Study Group - Europe and Asia Recap - 2024-04-11 - Full RecordingShane Coughlan
 
Effort Estimation Techniques used in Software Projects
Effort Estimation Techniques used in Software ProjectsEffort Estimation Techniques used in Software Projects
Effort Estimation Techniques used in Software ProjectsDEEPRAJ PATHAK
 
SAM Training Session - How to use EXCEL ?
SAM Training Session - How to use EXCEL ?SAM Training Session - How to use EXCEL ?
SAM Training Session - How to use EXCEL ?Alexandre Beguel
 
Pros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdf
Pros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdfPros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdf
Pros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdfkalichargn70th171
 
Osi security architecture in network.pptx
Osi security architecture in network.pptxOsi security architecture in network.pptx
Osi security architecture in network.pptxVinzoCenzo
 
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdf
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdfEnhancing Supply Chain Visibility with Cargo Cloud Solutions.pdf
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdfRTS corp
 

Recently uploaded (20)

The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...
The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...
The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...
 
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
 
eSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration toolseSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration tools
 
Introduction to Firebase Workshop Slides
Introduction to Firebase Workshop SlidesIntroduction to Firebase Workshop Slides
Introduction to Firebase Workshop Slides
 
Ronisha Informatics Private Limited Catalogue
Ronisha Informatics Private Limited CatalogueRonisha Informatics Private Limited Catalogue
Ronisha Informatics Private Limited Catalogue
 
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4jGraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
 
Tech Tuesday Slides - Introduction to Project Management with OnePlan's Work ...
Tech Tuesday Slides - Introduction to Project Management with OnePlan's Work ...Tech Tuesday Slides - Introduction to Project Management with OnePlan's Work ...
Tech Tuesday Slides - Introduction to Project Management with OnePlan's Work ...
 
Keeping your build tool updated in a multi repository world
Keeping your build tool updated in a multi repository worldKeeping your build tool updated in a multi repository world
Keeping your build tool updated in a multi repository world
 
[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf
[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf
[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf
 
VictoriaMetrics Q1 Meet Up '24 - Community & News Update
VictoriaMetrics Q1 Meet Up '24 - Community & News UpdateVictoriaMetrics Q1 Meet Up '24 - Community & News Update
VictoriaMetrics Q1 Meet Up '24 - Community & News Update
 
Strategies for using alternative queries to mitigate zero results
Strategies for using alternative queries to mitigate zero resultsStrategies for using alternative queries to mitigate zero results
Strategies for using alternative queries to mitigate zero results
 
What’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 UpdatesWhat’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 Updates
 
Understanding Plagiarism: Causes, Consequences and Prevention.pptx
Understanding Plagiarism: Causes, Consequences and Prevention.pptxUnderstanding Plagiarism: Causes, Consequences and Prevention.pptx
Understanding Plagiarism: Causes, Consequences and Prevention.pptx
 
Mastering Project Planning with Microsoft Project 2016.pptx
Mastering Project Planning with Microsoft Project 2016.pptxMastering Project Planning with Microsoft Project 2016.pptx
Mastering Project Planning with Microsoft Project 2016.pptx
 
OpenChain AI Study Group - Europe and Asia Recap - 2024-04-11 - Full Recording
OpenChain AI Study Group - Europe and Asia Recap - 2024-04-11 - Full RecordingOpenChain AI Study Group - Europe and Asia Recap - 2024-04-11 - Full Recording
OpenChain AI Study Group - Europe and Asia Recap - 2024-04-11 - Full Recording
 
Effort Estimation Techniques used in Software Projects
Effort Estimation Techniques used in Software ProjectsEffort Estimation Techniques used in Software Projects
Effort Estimation Techniques used in Software Projects
 
SAM Training Session - How to use EXCEL ?
SAM Training Session - How to use EXCEL ?SAM Training Session - How to use EXCEL ?
SAM Training Session - How to use EXCEL ?
 
Pros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdf
Pros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdfPros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdf
Pros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdf
 
Osi security architecture in network.pptx
Osi security architecture in network.pptxOsi security architecture in network.pptx
Osi security architecture in network.pptx
 
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdf
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdfEnhancing Supply Chain Visibility with Cargo Cloud Solutions.pdf
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdf
 

BTDConf: Breaking Illusions: Testing is Your Most Valuable Asset

  • 1. Breaking Illusions: Testing is your most valuable asset! Maaret Pyhäjärvi Email: <maaret@iki.fi> | Twitter: maaretp Maaret Pyhäjärvi Nimeä | Attribution (Finland) http://creativecommons.org/licenses/by/1.0/fi/ http://creativecommons.org/licenses/by/1.0/fi/deed.en
  • 2. Testing is about Feedback Checking • Machines can do with algorithmic decision rules • Focuses on specific facts • Cmp. “compiling”, “generating” Exploring • Skilled humans can do • Open-ended search for information • Cmp. “programming” Testing not testers – but becoming a skilled tester requires time and focus
  • 3. Context Matters – Expectations Differ • Testing in Production? – Rolling releases: limit the extent of impact – Product lock-in: limit the options • Testing in the delivery chain? – Development teams skilled in exploring (as well as checking) – Internal releases – Beta releases – Releases • Available skills? – Intentional vs. accidental production feedback
  • 4. Testers don’t break your code, they break your illusions about the code. -- adapted from James Bach
  • 5. Dispelling Illusions: Empirical Evidence over Speculation VALU E ILLUSIONS
  • 6. The World Has Already Changed COMMODITY TESTERS • Manual checkers • Tests are an artifact SKILLED TESTERS • Explorers of products and businesses • Testing is a performance • DEVELOPERS IN TEST • All development is automating, testing is just a domain of development
  • 7. Exploration Skills Source: Adapted from James Bach, Jon Bach, Michael Bolton. Exploratory Testing Dynamics. v.2.2. 2009 Self- managemen t Developing ideas Examining product Done To Do Issues Coverage All sources available Best use of time – effective and efficient work Making models Tool support – creative solutions Risk-based testing – scientific approach Keeping one’s eyes open
  • 8.
  • 9. Things Can Look Different from Different Perspectives
  • 10. Culture of Feedback and Experiments • Illusions are meant to be broken • Teams over individuals – Diverse teams (personalities, skills) deliver better results – Social dissonance gets the team talking and voices out uncertainties to address on time • Experiment: try something different
  • 11. Summary 1. TESTING is FEEDBACK and LEARNING. EMPIRICAL supports learning over theoretical. FAST is better than slower. Not outsourcing the cost of feedback could be better. 2. CONTEXT matters: sometimes the “best testing” happens live – as long as you’re covering the checking part; testing != testers 3. BUGS not defects: anything this might bug a user, there’s many types of illusions we have 4. COMMODITY testers and SKILLED testers are two completely different breeds 5. SKILL to break DIVERSE illusions – skill in team, not in individual 6. DIVERSITY brings good results: people who choose to love testing tend to be different from those who choose to love checking and development

Editor's Notes

  1. This presentation is about FEEDBACK. I don’t care if you have specialized testers to help you work out the multidimensional feedback or not. Feedback is still valuable, and for all contexts, asking for all the feedback from the end users / product management just isn’t the thing to do… I come from an organization where (some) developers strongly still believe that it would be better to hear about bugs from the end users so that we would only hear about “real bugs” and that regardless of me showing I do understand what they complain on, it would still be different (there would be less of it…)
  2. Point of this slide: TESTING is FEEDBACK and LEARNING. EMPIRICAL supports learning over theoretical. FAST is better than slower. Not outsourcing the cost of feedback could be better. MENTION: Testing looks into more than functionality. Other quality criteria and stakeholder views are particular relevance. Emergent information, you do not know what you are looking for For more info, see http://www.satisfice.com/blog/archives/856 Checking the process of making evaluations by applying algorithmic decision rules to specific observations of a product. Testing the process of evaluating a product by learning about it through exploration and experimentation, which includes to some degree: questioning, study, modeling, observation, inference, etc.
  3. Point of this slide: CONTEXT matters: sometimes the “best testing” happens live – as long as you’re covering the checking part; testing != testers Example: Security vulnerabilities Vulnerabilities are considered real problems Still only the black market pays real money for reporting them Functional bugs / missing value reporting is actual effort too, and wastes time for people Something we as end users can do: reclamation I tend to work with PRODUCT BUSINESS and CUSTOMER ORGANIZATIONS, not delivering bespoke software. Inability to produce quality vs. productivity service – different stakeholder considerations, incl. “design with a change in mind’ I belong to context-driven school of testing – I know things can be different! History of having things tested in production – wasting a lot of time. Not every team has the most skilled developers but they can still have diverse teams delivering excellent results
  4. Point of this slide: Testing helps with EFFECTIVENESS, as a productivity service. Your users might not really like reporting your bugs for you, as it wastes their time. It’s co-creation. The cake example. You can’t separate the ideas, ideas feeding the ideas and implementation.
  5. Illusion stories: “I tested it, it works” “reporting engine” “no time for refactoring” / “releasing as scheduled over done” “Need document grouping feature over others” --- Ray Arell (Intel) giving apple example: showing you’re living with an illlusion. Tester mentality!! ---- Point of this slide: BUGS not defects: anything this might bug a user, there’s many types of illusions we have Finding really expensive bugs: missing value, wrong features Illusion: releasing as scheduled (sprint) vs. releasing when done (kanban) EMPIRICAL evidence trumps speculation. Every Single Time. – testobsessed
  6. Point of this slide: COMMODITY testers and SKILLED testers are two completely different breeds The spider web idea. Your ideas of testers may be outdated. You get what you ask for, and if you ask for test cases, you get commodity approaches.
  7. Point of this slide: SKILL to break DIVERSE illusions – skill in team, not in individual
  8. Exploratory is a MINDSET / APPROACH, not an activity. Testing is PERFORMANCE, not ARTIFACT. Testing is testing even if context is agile.
  9. Point of this slide: DIVERSITY brings good results: people who choose to love testing tend to be different from those who choose to love checking and development Not every team has the most skilled developers but they can still have diverse teams delivering excellent results
  10. Creating culture of feedback and experiments with diversity of personalities and skills in the teams