Exploratory Testing in Practice

3,837 views

Published on

What is this thing called "Exploratory Testing"? How do you do it? Why is it important?

Published in: Technology, Education

Exploratory Testing in Practice

  1. 1. Exploratory Testing in Practice Elisabeth Hendrickson Blog: testobsessed.com Twitter: @testobsessedSlides Last updated June 22, 2012
  2. 2. Two Sides of Testing Tested = Checked + Explored
  3. 3. That’s Nice… …But we Believe in ourcomprehensive, repeatable regression tests.
  4. 4. Incidental Correctness Given a series of 1’s and 0’s, Compute the corresponding decimal number Input Expected Result 000 0 0 111 7 7 101 5 5x 011 3 6
  5. 5. Missed RequirementsHey, Where’s the Motor?!? motor for this You neverremote control said anything car? about a motor!
  6. 6. Real World Users Happen You assume they’re reasonable people who will do reasonable things. They aren’t and they won’t.
  7. 7. So What Is Exploratory Testing?a systematic approach fordiscovering risksusing rigorousanalysis techniquescoupled withtesting heuristics.
  8. 8. OK…But What Do You Actually Do?Simultaneously……learning about the Target…While designing tests…And executing Them Immediatelyusing feedback from the last LittleExperiment to inform the next (Term coined by Cem Kaner. Definition refined byJames Bach. Others have contributed to ET including James Lyndsay, Jonathan Kohl, Michael Bolton.)
  9. 9. ExperimentSo it handled all Next, let’s try the variations interrupting theon disconnected transaction networks we here. could think of…
  10. 10. Discover Things You Can Vary
  11. 11. Use State Models Idle Time to No connect NetworkSystemUpdated Check Network No Updates Network Avail Update Get System Updates Content Downloaded 11
  12. 12. Use Data Models Customer VendorHas 0 or More Orders Has 0 or More Has 1 Or More Products
  13. 13. Use Deployment Diagrams Server FarmLocal Agent AWS
  14. 14. Use Heuristics See the Test Heuristics Cheatsheet for more http://bit.ly/testcheatsheetGoldilocks: too big, too small, just right Counts: 0, 1, ManySelection: Some, None, All Position: Beginning, Middle, EndViolate Domain-Specific Rules:Age -1 IP address 999.999.999.999Interrupt States Delete Dependent Data
  15. 15. In Flight Feedback at #AU2H Matt Barcomb (Coding)Jon Bach(Exploring)
  16. 16. Result: Feedback on Process AND Product We’ve automated I’ll add things the regression like that to thetest when we fixed acceptance the bug. criteria. We need to think about that kind of thing in future designs too.
  17. 17. Pitfall: Going Off in the Weeds it doesn’t work on15 year old laptopsrunning Windows 3.1! Uh… Why Would We Care?
  18. 18. Focus with ChartersExplore (Area/feature)With (Resources,conditions, Dependency,etc.)to discover (information)
  19. 19. Structure with Time Boxed Sessions Charters SessionsExplore login in combination Sessionwith bookmarks & history todiscover information about Sessioninteractions. SessionExplore the site with variousbrowsers and settings to Sessiondiscover risks related tounsupported configurations. SessionExplore error handling usingdata type attacks to find Sessionproblems related to datavalidation.
  20. 20. Explore Early, Explore Often
  21. 21. Exploratory Testing on Agile Projects Within an Iteration Automated Automated Exploratory Acceptance Unit Tests Testing TestsDefine “done”, Drive design, Discover represent represent executable executable risksrequirements specifications EARLY

×