Testing Lessons from the Rolling Stones


Published on

Session at Let's Test 2012 Conference, Runö, Stockholm, 7-9 May 2012

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Testing Lessons from the Rolling Stones

  1. 1. Testing Lessons From the Rolling StonesSimon Morley Let’s Test 2012 @YorkyAbroad
  2. 2. What’s the most significant part of testing? I claim, it is: Communication Communication Communication
  3. 3. But, not just any type of communication..... CongruentConway Heuristic Latin defn Weinberg
  4. 4. Q:How can the rolling stoneshelp me do better testing? And get here Not here
  5. 5. Playing with fire
  6. 6. Playing with fireContext-free reportingContext-free requests
  7. 7. Context-free reporting gets in “it works” the way of good testing!“it doesn’t work” “we ran lots of tests” “we found lots of bugs”
  8. 8. Context-free reporting gets in “it works” the way of good testing!“it doesn’t work” “we ran lots of tests” “we found lots of bugs” Context-free “just test...” requests can get in the way of good testers. “the feature cannot be used...”
  9. 9. Case Example report c.2009 Test Case Stats Trouble Reports Pass 1020 100,00% Prio A 0 Fail 0 0% Prio B 0 Not Started 0 0% Prio C 0Only one problem/concern observed: “We have to cope with >30% regression failures with each run… that means 3-4 re-runs at least before we are having “stable” results… ”
  10. 10. Traps visible1. Test case counting self-herding2. Bug report counting3. Silent evidence confirmatory testing
  11. 11. Countering such traps #1Stop talking only in numbers Get qualitative Use numbers as a footnote - not as a headline Ask for qualitative information
  12. 12. Countering such traps #2Silent evidence - report to the whole context E What’s excluded? N What was not considered N What was not covered
  13. 13. Nervous Breakdown
  14. 14. “Why did you miss that bug?” “Why is testing taking so long?” “we are delayed due to the X testing activity... again” Taylorism in Testing Symptoms of one-sided communication or a need for test brand-building
  15. 15. Traps visibleLimited Framing Self-herdingLack of Context awarenessLack of Systems Thinking (Linear Trap)
  16. 16. Avoiding “Nervous Breakdowns”Frame + context awareness poor testing? missed tests? poor assignments? poor communication? communication raise education awareness brand building
  17. 17. Avoiding “Nervous Breakdowns”Linear traps Beware of Silent Evidence Communication flow Testing Story + Product Story?Self-herding Awareness Ask yourself control (why) questions
  18. 18. Wants vs Needs
  19. 19. Wants vs Needs Sur vey ExpectationsExpertitis Quality gates
  20. 20. Getting closer to needsTesting Transparency What’s involved Framing Root causes vs Symptoms Feynman’s “Why” Conway Heuristic
  21. 21. Summing up
  22. 22. Traps to congruent communicationSelf-herding Silent evidence Context oblivious Quality gates
  23. 23. Getting to congruent communicationContext-free questions Context-specific questionsContext-specific reporting Reporting the whole context
  24. 24. Congruent -> Satisfaction? Test brand building Making silent evidence visible Context-specific Trap awareness Trap evangelist
  25. 25. Reading & Study ReferencesFICL: Russo & Schoemaker Taylorism in Testing Systems Thinking: Weinberg, Silent Evidence in Testing Larman Self-Herding: Ariely Linear TrapsSilent Evidence: Taleb Brand buildingCognitive Ease: Kahneman FramingContext-free questions: Gause & Weinberg Framing: Weinberg, Tversky & Why questions: Youtube - Feynman Kahneman, Russo & Schoemaker (magnets)
  26. 26. Thanks!Let’s talk & test: @YorkyAbroad testers-headache.blogspot.com
  27. 27. Image creditshttp://www.flickr.com/photos/vipez/5537785589/http://www.flickr.com/photos/khiltscher/3277538820http://www.flickr.com/photos/khiltscher/5121877064/http://www.flickr.com/photos/khiltscher/4691902082/http://www.flickr.com/photos/linneberg/4481309196/http://www.flickr.com/photos/hoyvinmayvin/2369945892/http://www.flickr.com/photos/zachklein/54389823/
  28. 28. Conway HeuristicTo communicate well about testing one must beable to misunderstand the communication(report, result, interpretation, explanation) in asmany ways as the other participants in thecommunication process.