No test script survives contact with the software. That’s where scripted tests fail. Scripts rely heavily on assumptions, inhibit investigative work, and cost too much. Automating tests won’t cut it either; it may be efficient, but still won’t dive wide and deep where the problems lie. This is where exploratory testing adds the most value; however it is still largely, albeit incorrectly, perceived as an undisciplined, ineffective test technique. In this talk, I discussed why exploratory testing works better than scripted tests, what critical gap it addresses, and how to do it well.

1. Tests are Nothing
Testing is Everything
The Dark Side of Test Scripts
Ian Pestelos

3. ian@averageup.ph
@averageupph
/in/ianpestelos
—————
#testing

4. meet…
ME :)

5. photo credit: http://memecrunch.com/meme/7UQP/scratch-head/image.jpg

6. Why are we trying to
control the future?

7. photo credits: https://www.facebook.com/biglaanglakad

8. photo credit: http://forevertwentysomethings.com/wp-content/uploads/2014/07/change_sign1.jpg

9. Reliance on Certainty
PROBLEM 1

11. meet…
RALPH

12. 27 Bugs Found
2 Hours Testing
12 Bugs Found
4 Hours “Testing”

13. Bugs are Missed
PROBLEM 2

14. meet…
MARI

15. photo credit: http://www.copyblogger.com/common-blogging-mistakes/

16. Focus Away from Value
PROBLEM 3

18. "Test scripting is a practice
that many of us still think
is the best one.
Or is the only one."

19. The Law of the Instrument
photo credit: http://www.180360720.no/wp-content/uploads/2010/04/hammer-nail1-420x315.jpg

20. We should let testers…
DIE
Discover. Investigate. Explore.

21. — is not a tool or methodology
— is not just for agile
— is a complementary technique
Exploratory Testing

22. …simultaneously learning about the system
while designing and executing tests,
using feedback from the last test
to inform the next."

23. photo credit: http://less.works/img/test_automation/xet_scripted_difference.png.pagespeed.ic.X0uMS-HZ2g.png

24. “The Freedom Scale” - Jon Bach

25. “The Learning Scale” - Ian Pestelos

26. EXPLORATORY = AD-HOC TESTING ?
ad hoc testing is performed
without a plan of action and
any actions taken are not typically documented

27. Your mission, should you choose to accept it,
is to PENETRATE the highly-secured archive
inside THE KREMLIN and retrieve COBALT’S FILE
BEFORE he can destroy it.

28. explore <Target>
using <Resources>
discover <Information>

29. What’s a Good Test Charter?
It offers DIRECTION
Without Over-specification

30. What’s a Good Test Charter?
Too
Specific:
It’s
a
different
(and
weird)
way
of
expressing
individual
tests.

31. What’s a Good Test Charter?
Too
Broad:
Not
providing
enough
focus;
you
won’t
know
how
to
tell
when
you’re
done

32. A “Special” Test Charter?
Reconnaissance:
It
is
a
special
charter
to
survey
or
understand
something
you
don’t
know
how
to
test
yet
photo credit: http://offload.goarmy.com/special-forces/primary-missions/special-reconnaissance/jcr:content/contentpar/header.png

33. A “Special” Test Charter?
“explore
<TARGET>
using
my
brain
and
my
senses
to
understand
how
it
works”
photo credit: http://offload.goarmy.com/special-forces/primary-missions/special-reconnaissance/jcr:content/contentpar/header.png

37. DEBRIEFING

38. Debriefing catches “gaps”
photo credit: http://www.sorryformarketing.com/blog/weird-things-business-people-do-on-twitter

39. Debriefing promotes learning

40. Debriefing rattles people

41. Case Studies

42. Case Studies

43. We NEED testers to…
DIE
Discover. Investigate. Explore.
Everyday

44. where to
begin?

45. Quick Start Guide
Start SMALL

46. Quick Start Guide
Find your “VISIONARIES”

47. Quick Start Guide
Promote
GENERALISING SPECIALISTS

48. Quick Start Guide
Try DEBRIEFING… ASAP

49. Quick Start Guide
Do periodic RETROSPECTIVES

50. Tests are Nothing
Testing is Everything
?

51. Focus on
PRODUCT quality
(not the Artefacts)

52. Agility & Flexibility
to Changes & New Info
(not assuming certainty)

53. Maximum opportunities
for Bug Hunting
(not blind checking)

54. Exploratory Testing
Do More TESTS
Find More PROBLEMS
Get More LEARNING

55. Quality, in the end, is in the product,
not in paper documentation.
Paper documentation allows
lies and illusions of quality,
working software doesn’t.
— Gunther Verheyen