When people say they do test automation I say they do Automated checking and yes there is a lot of automation used in testing.

1. @EddyBruin

3. There is test automation!
@EddyBruin
Nor is there program automation

4. Who of you agrees?

5. Ariane 5
Conversion caused overflow

6. XXXXXXX = Lucky nr 7 + Racer
XXXXXXX = No license plate

7. Why scripted testing has it’s limitations
System under test
Intended inputs
Monitored
output
Program state
System state
Configurations
(3rd party) libraries
Processes from other systems
System resources
Program state with unexpected outputs
System state
Impact on system resources
Impact on connected systems

8. Call this “Checking” not Testing

16. The 5 orders of ignorance
0OI = Lack of Ignorance
I (probably) know something. I have the answer and can explain it.
1OI = Lack of Knowledge
I don't know something, but I have a well formed question.
2OI = Lack of Awareness
I don't know that I don't know something. I have no idea what the right question is.
3OI = Lack of Process
I don't know a suitably way to find out I don't know that I don't know something.
I have no clue how to find the right idea's to search for the right question.
4OI = Meta-ignorance
I don't know about the Five Orders of Ignorance.

17. How much do we already know?

18. Testing is…

19. Testing is in your head…mostly…

20. Tested = +
Checked + Explored
Exploring

21. “Exploratory testing is simultaneous
learning, test design, and test execution.”
James Bach, 1995
Test techniques improve result
Immediate investigation gives most information
Get to know the system
Definition of Exploratory testing

22. Monkey testing

23. • Freestyle exploratory
• Testing tours
• Bug hunting
• Session based testing
• And more…
The many flavors of exploratory testing

24. • The landmark tour
• The FedEx tour
• The couch potato tour
• The supermodel tour
• The clubbing tour
Testing Tours

26. Jury
Bug boardTimebox
Programmer
Prices

27. Session based testing

28. Ok,but wait!
Real companies don’t do this right ?

29. Ok,but wait! I’m a programmer!
So…what about
Checked
Exploring

30. What is BDD to you in practice?

31. BDD definition
(by of one of the founders)
“Using examples in conversation
to illustrate behaviour.”
Liz Keogh

32. Liz Keogh

34. Problem & solution space

35. “A tester is somebody who knows
that things can be different.”
testing is about critical
thinking and imagination
Jerry Weinberg

36. …solving problems through an indirect
and creative approach, using reasoning that is
not immediately obvious and involving ideas
that may not be obtainable by using only
traditional step-by-step logic.
Lateral thinking is…

37. Testing is in your head…mostly…

39. Never
Always
Design good checks & Generate test ideas

40. Ok,but wait!
There is still a lot of Test Automation?!
,but there is Automation in Testing

41. Set up test data
Crawlers
Scripts
Visual reviewing
Data manipulation
Provisioning
... tools that do automation to aid testing

42. There is test automation!

43. Who of you agrees more than 40 minutes ago?

44. 1. Checking = a part of testing (which can often be automated)
2. BDD = mainly about having a conversation understanding system behaviour
3. Automated checking =! Test automation
4. You need creativity, critical and lateral thinking skills for testing
5. Assume that you are wrong when you think you know something (5 orders)
There is test automation!

45. “Learning comes from
experiencing the unexpected”
“Ignorance is the single greatest
impediment to throughput”
Dan North - Introducing deliberate discovery

46. Hey, I just met you
And this was crazy
So here’s my twitter
So tweet me maybe?!
@eddybruin
Credits for some slides go to: Cem Kaner, James Bach, Ilari Henrik, Elisabeth Hendrickson
Or me: eddy@loopforward.nl
Useful books on testing & checking