Your SlideShare is downloading. ×
Testing Lessons from the Rolling Stones
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Testing Lessons from the Rolling Stones


Published on

Session at Let's Test 2012 Conference, Runö, Stockholm, 7-9 May 2012

Session at Let's Test 2012 Conference, Runö, Stockholm, 7-9 May 2012

Published in: Technology, Business

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Testing Lessons From the Rolling StonesSimon Morley Let’s Test 2012 @YorkyAbroad
  • 2. What’s the most significant part of testing? I claim, it is: Communication Communication Communication
  • 3. But, not just any type of communication..... CongruentConway Heuristic Latin defn Weinberg
  • 4. Q:How can the rolling stoneshelp me do better testing? And get here Not here
  • 5. Playing with fire
  • 6. Playing with fireContext-free reportingContext-free requests
  • 7. Context-free reporting gets in “it works” the way of good testing!“it doesn’t work” “we ran lots of tests” “we found lots of bugs”
  • 8. Context-free reporting gets in “it works” the way of good testing!“it doesn’t work” “we ran lots of tests” “we found lots of bugs” Context-free “just test...” requests can get in the way of good testers. “the feature cannot be used...”
  • 9. Case Example report c.2009 Test Case Stats Trouble Reports Pass 1020 100,00% Prio A 0 Fail 0 0% Prio B 0 Not Started 0 0% Prio C 0Only one problem/concern observed: “We have to cope with >30% regression failures with each run… that means 3-4 re-runs at least before we are having “stable” results… ”
  • 10. Traps visible1. Test case counting self-herding2. Bug report counting3. Silent evidence confirmatory testing
  • 11. Countering such traps #1Stop talking only in numbers Get qualitative Use numbers as a footnote - not as a headline Ask for qualitative information
  • 12. Countering such traps #2Silent evidence - report to the whole context E What’s excluded? N What was not considered N What was not covered
  • 13. Nervous Breakdown
  • 14. “Why did you miss that bug?” “Why is testing taking so long?” “we are delayed due to the X testing activity... again” Taylorism in Testing Symptoms of one-sided communication or a need for test brand-building
  • 15. Traps visibleLimited Framing Self-herdingLack of Context awarenessLack of Systems Thinking (Linear Trap)
  • 16. Avoiding “Nervous Breakdowns”Frame + context awareness poor testing? missed tests? poor assignments? poor communication? communication raise education awareness brand building
  • 17. Avoiding “Nervous Breakdowns”Linear traps Beware of Silent Evidence Communication flow Testing Story + Product Story?Self-herding Awareness Ask yourself control (why) questions
  • 18. Wants vs Needs
  • 19. Wants vs Needs Sur vey ExpectationsExpertitis Quality gates
  • 20. Getting closer to needsTesting Transparency What’s involved Framing Root causes vs Symptoms Feynman’s “Why” Conway Heuristic
  • 21. Summing up
  • 22. Traps to congruent communicationSelf-herding Silent evidence Context oblivious Quality gates
  • 23. Getting to congruent communicationContext-free questions Context-specific questionsContext-specific reporting Reporting the whole context
  • 24. Congruent -> Satisfaction? Test brand building Making silent evidence visible Context-specific Trap awareness Trap evangelist
  • 25. Reading & Study ReferencesFICL: Russo & Schoemaker Taylorism in Testing Systems Thinking: Weinberg, Silent Evidence in Testing Larman Self-Herding: Ariely Linear TrapsSilent Evidence: Taleb Brand buildingCognitive Ease: Kahneman FramingContext-free questions: Gause & Weinberg Framing: Weinberg, Tversky & Why questions: Youtube - Feynman Kahneman, Russo & Schoemaker (magnets)
  • 26. Thanks!Let’s talk & test: @YorkyAbroad
  • 27. Image credits
  • 28. Conway HeuristicTo communicate well about testing one must beable to misunderstand the communication(report, result, interpretation, explanation) in asmany ways as the other participants in thecommunication process.