Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

There is NO test automation


Published on

When people say they do test automation, I say they do automated checking. Testing is a creative job. Checking is only a small part of that.

Published in: Software
  • D0WNL0AD FULL ▶ ▶ ▶ ▶ ◀ ◀ ◀ ◀
    Are you sure you want to  Yes  No
    Your message goes here
  • D0WNL0AD FULL ▶ ▶ ▶ ▶ ◀ ◀ ◀ ◀
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

There is NO test automation

  1. 1. @EddyBruin
  2. 2. There is test automation! @EddyBruin Nor is there program automation
  3. 3. Who of you agrees?
  4. 4. Ariane 5 Conversion caused overflow
  5. 5. XXXXXXX = Lucky nr 7 + Racer XXXXXXX = No license plate
  6. 6. Why scripted testing has it’s limitations System under test Intended inputs Monitored output Program state System state Configurations (3rd party) libraries Processes from other systems System resources Program state with unexpected outputs System state Impact on system resources Impact on connected systems
  7. 7. Call this “Checking” not Testing
  8. 8. The 5 orders of ignorance 0OI = Lack of Ignorance I (probably) know something. I have the answer and can explain it. 1OI = Lack of Knowledge I don't know something, but I have a well formed question. 2OI = Lack of Awareness I don't know that I don't know something. I have no idea what the right question is. 3OI = Lack of Process I don't know a suitably way to find out I don't know that I don't know something. I have no clue how to find the right idea's to search for the right question. 4OI = Meta-ignorance I don't know about the Five Orders of Ignorance.
  9. 9. How much do we already know?
  10. 10. Testing is…
  11. 11. Testing is in your head…mostly…
  12. 12. Tested = + Checked + Explored Exploring
  13. 13. “Exploratory testing is simultaneous learning, test design, and test execution.” James Bach, 1995 Test techniques improve result Immediate investigation gives most information Get to know the system Definition of Exploratory testing
  14. 14. Monkey testing
  15. 15. • Freestyle exploratory • Testing tours • Bug hunting • Session based testing • And more… The many flavors of exploratory testing
  16. 16. • The landmark tour • The FedEx tour • The couch potato tour • The supermodel tour • The clubbing tour Testing Tours
  17. 17. Jury Bug boardTimebox Programmer Prices
  18. 18. Session based testing
  19. 19. Ok,but wait! Real companies don’t do this right ?
  20. 20. Ok,but wait! I’m a programmer! So…what about Checked Exploring
  21. 21. What is BDD to you in practice?
  22. 22. BDD definition (by of one of the founders) “Using examples in conversation to illustrate behaviour.” Liz Keogh
  23. 23. Liz Keogh
  24. 24. Problem & solution space
  25. 25. “A tester is somebody who knows that things can be different.” testing is about critical thinking and imagination Jerry Weinberg
  26. 26. …solving problems through an indirect and creative approach, using reasoning that is not immediately obvious and involving ideas that may not be obtainable by using only traditional step-by-step logic. Lateral thinking is…
  27. 27. Testing is in your head…mostly…
  28. 28. Never Always Design good checks & Generate test ideas
  29. 29. Ok,but wait! There is still a lot of Test Automation?! ,but there is Automation in Testing
  30. 30. Set up test data Crawlers Scripts Visual reviewing Data manipulation Provisioning ... tools that do automation to aid testing
  31. 31. There is test automation!
  32. 32. Who of you agrees more than 40 minutes ago?
  33. 33. 1. Checking = a part of testing (which can often be automated) 2. BDD = mainly about having a conversation understanding system behaviour 3. Automated checking =! Test automation 4. You need creativity, critical and lateral thinking skills for testing 5. Assume that you are wrong when you think you know something (5 orders) There is test automation!
  34. 34. “Learning comes from experiencing the unexpected” “Ignorance is the single greatest impediment to throughput” Dan North - Introducing deliberate discovery
  35. 35. Hey, I just met you And this was crazy So here’s my twitter So tweet me maybe?! @eddybruin Credits for some slides go to: Cem Kaner, James Bach, Ilari Henrik, Elisabeth Hendrickson Or me: Useful books on testing & checking