This document discusses exploratory testing and provides definitions and concepts related to testing. It defines testing, quality, bugs, and why we test. It then discusses exploratory testing and how it is about continuous learning. Elements of exploratory testing discussed include test design, careful observation, critical thinking, diverse ideas, and rich resources. Specific techniques like session-based testing, heuristics, oracles, and fractals are also covered. The importance of context and skills like communication, courage, and self-esteem for testers are emphasized.

1. Exploratory Testing

2. First some definitions.

3. Testing
• “Gathering information with the
intention of informing a decision.” –
Jerry Weinberg
• “Testing is an empirical, technical
investigation of a product, done on behalf
of stakeholders, with the intention of
revealing quality-related information of the
kind that they seek.” – Cem Kaner
• “Questioning a product in order to
evaluate it.” – James Bach

4. Quality
• “Conformance with requirements.” – Philip
Crosby
• “Quality is value to some person.” – Jerry
Weinberg

5. • “Successful testers avoid a fascination with
functional correctness, and focus on ways in
which people might obtain value from a
program—or have their value unfulfilled or
threatened.” – Michael Bolton
http://www.developsense.com/blog/2011/02/why-do-some-testers-find-the-critical-problems/

6. Bug
• “Doesn’t match specifications of written
requirements.”
• “Failure to meet reasonable expectations of a
user.” – G. Myers
• “A bug is something that bugs somebody.” –
James Bach
• “Anything that causes an unnecessary or
unreasonable reduction of the quality of a
software product.” – BBST Bug Advocacy

7. Why are we testing?

8. http://en.wikipedia.org/wiki/File:Columbus_letter_Spanish_text.jpg

10. Exploratory Testing is about learning
• “If you are not using the learning garnered
from test design and test execution in a
continuous and rapid loop to optimize the
quality of the work, you are not doing
exploratory testing.” – Michael Bolton

11. Session-Based Testing
http://www.bettertesting.co.uk/content/wp-content/uploads/2011/03/Session-Based-Testing-in-Practice.jpg

12. Some of the Elements of Exploratory
Testing
• Test Design
• Careful Observation
• Critical Thinking
• Diverse Ideas
• Rich Resources
http://www.satisfice.com/articles/et-article.pdf

13. Test Design
• Function Testing
• Domain Testing
• Specification-based Testing
• Risk-based Testing
• …
• Heuristic Test Strategy Model
(http://www.satisfice.com/tools/htsm.pdf)

14. Few Words About Context-Driven
Testing
• The value of any practice depends on its context.
• There are good practices in context, but there are no best practices.
• People, working together, are the most important part of any project’s
context.
• Projects unfold over time in ways that are often not predictable.
• The product is a solution. If the problem isn’t solved, the product doesn’t
work.
• Good software testing is a challenging intellectual process.
• Only through judgment and skill, exercised cooperatively throughout the
entire project, are we able to do the right things at the right times to
effectively test our products.
http://context-driven-testing.com/

15. Careful Observation

16. Inattentional Blindness

17. The video shows that it’s possible to miss something right in front of your
eyes. The point of the video is to show that it’s possible to miss something
that’s right in front of your eyes.
IB makes you confront the illusion of attention.
That’s it.
Knowing about inattentional blindness doesn’t make you better at noticing
things, and there’s zero correlation between the observational skills of those
who see the gorilla vs. those who don’t. While I agree that it’s critical for
testers to know that inattentional blindness exists, it’s a small nugget of
information in a pretty big pool of stuff that actually helps testers.
Alan Page - http://angryweasel.com/blog/?p=594

21. We need to vary our actions
• Need different perspective / angle
– Personas
– Tours (http://michaeldkelly.com/blog/2005/9/20/touring-heuristic.html or
http://msdn.microsoft.com/en-us/library/jj620911.aspx#bkmk_tours)
– Pairing

22. Critical Thinking

23. Huh? Really? So?

26. http://www.nasa.gov/centers/ivv/pdf/174321main_Annual_Report_06_Final.pdf

30. http://lesswrong.com/lw/hab/rocket_science_and_big_money_a_cautionary_tale_of/

31. Diverse Ideas
• Heuristics
– You Are Not Done Yet
http://www.thebraidytester.com/downloads/YouAreNotDoneYet.pdf
– Test Heuristics Cheat Sheet http://testobsessed.com/wp-
content/uploads/2011/04/testheuristicscheatsheetv1.pdf
– The Little Black Book On Test Design
http://www.thetesteye.com/papers/TheLittleBlackBookOnTestDesign.pdf

32. Oracles
“A way to recognize a problem.” – Michael Bolton & James Bach
a) A feeling like confusion or annoyance.
b) A desirable consistency between related things.
c) A person whose opinion matters.
d) An opinion held by a person who matters.
e) A disagreement among people who matter.
f) A reference document with useful information.
g) A known good example output.
h) A known bad example output.
i) A process or tool by which output is checked.
j) A process or tool that helps a tester identify patterns.
http://www.developsense.com/blog/2013/01/oracles-the-brainstream-media/

33. • People!
– “The diversity of tester temperaments and
backgrounds on a team can also be harnessed by
savvy exploratory testers through the process of
group brainstorming to produce better test ideas.”
– James Bach
“The roof don’t leak because the holes in each layer
don’t line up with each other.” – George Dinwiddie
on Twitter

34. Rich Resources
• Tools
• Information sources (notes, books, blogs)
• Test data
• Friends

35. Tester Freedom Scale
Jon Bach / Quardev, Inc., Seattle, WA 98105

36. Fractals
Adam Knight (http://www.a-sisyphean-task.com/2013/01/fractal-exploratory-
testing.html#.UX7WTitgYZ8)

37. Which are the most important traits a tester should have
(or should develop) in order to succeed today?
(http://qablog.practitest.com/2012/11/five-testing-
question-with-jerry-weinberg/)
• Courage
• Communication skill
• Self-esteem.
Jerry Weinberg