Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

QA Fest 2017. Ilari Henrik Aegerter. Complexity Thinking, Cynefin & Why Your Testing Metrics All Suck Big Time

254 views

Published on

From your own experience it might not come as a surprise that most of today’s testing is unhelpful, filled with unnecessary paper work and folkloric activities. For some reason testing work often does not seem to be very helpful in projects. That is definitely a problem. If you are a tester, your manager might ask you for metrics that don’t make sense to you. And since you are a smart person, you have probably once in a while gamed the system. All that is certainly damaging to the industry. What can you do? This session brings you insight into Complexity Thinking with Dave Snowden’s Cynefin model and ties that to your job as a software tester. It offers you a way to look at software testing from a complexity thinking standpoint of view and gives you tools to argue your case if you are exposed to dysfunctional project settings. In addition to that, we will have some fun with idiotic metrics and to lighten up the serious topic we’ll engage in hilariously entertaining real life examples of bad metrics. To round it up, we’ll propose more meaningful alternatives.

Published in: Education
  • Be the first to comment

  • Be the first to like this

QA Fest 2017. Ilari Henrik Aegerter. Complexity Thinking, Cynefin & Why Your Testing Metrics All Suck Big Time

  1. 1. Complexity Thinking, Cynefin & Why Your Testing Metrics All Suck Big Time
  2. 2. Complexity Thinking, Cynefin & Why All Your Testing Metrics All Sucks Big Time
  3. 3. Who Am I? Who – Problem – The Model –Application – Metrics – Action – End
  4. 4. llari Henrik Aegerter Managing Director – House of Test VP of Marketing – Association for Software Testing Beer Brewer @ilarihenrik www.houseoftest.ch Who – Problem – The Model –Application – Metrics – Action – End
  5. 5. Disclaimer: Keep in mind that I am a *%§!$ Consultant Question everything I say! Who – Problem – The Model –Application – Metrics – Action – End
  6. 6. What is the problem? Who – Problem – The Model –Application – Metrics – Action – End
  7. 7. We are applying methods in software testing we wouldn’t in comparable other situations Who – Problem – The Model –Application – Metrics – Action – End
  8. 8. Who – Problem – The Model –Application – Metrics – Action – End
  9. 9. The Cynefin Model Who – Problem – The Model –Application – Metrics – Action – End
  10. 10. Confusion between Complicated vs. Complex Complicated: Degree to which something is difficult to understand Complex: Degree to which Cause and Effect are deterministic Who – Problem – The Model –Application – Metrics – Action – End
  11. 11. Who – Problem – The Model –Application – Metrics – Action – End
  12. 12. Ordered Linear Complex Non-Linear Who – Problem – The Model –Application – Metrics – Action – End
  13. 13. How to apply Cynefin in Testing Who – Problem – The Model –Application – Metrics – Action – End
  14. 14. When confronted with a testing task, ask yourself how to act Who – Problem – The Model –Application – Metrics – Action – End
  15. 15. Obvious Sense - Categorize – Respond World Expample: Traffic Light Testing Example: Checking GUI Elements Who – Problem – The Model –Application – Metrics – Action – End
  16. 16. Complicated Sense - Analyze – Respond World Expample: Building a Space Ship Testing Example: Analysis of non-trivial calculations Who – Problem – The Model –Application – Metrics – Action – End
  17. 17. Complex Probe - Sense – Respond World Expample: A roundabout Testing Example: Almost everything in testing, Unknown unknowns, Exploratory Testing Who – Problem – The Model –Application – Metrics – Action – End
  18. 18. Safe to Fail Experiments Who – Problem – The Model –Application – Metrics – Action – End
  19. 19. Chaos Act - Sense – Respond World Expample: Kids Testing Example: A project in disarray Who – Problem – The Model –Application – Metrics – Action – End
  20. 20. Some words about metrics Who – Problem – The Model –Application – Metrics – Action – End
  21. 21. Cem Kaner on Metrics and Measurements http://kaner.com/pdfs/PracticalApproachToSoftwareMetrics.pdf Who – Problem – The Model –Application – Metrics – Action – End
  22. 22. A General Problem with Metrics Question is about Quality - Measurements are Quantitative Hence your measurement is a surrogate one Who – Problem – The Model –Application – Metrics – Action – End
  23. 23. Measurement Distortions and Goal Replacements Who – Problem – The Model –Application – Metrics – Action – End
  24. 24. Who – Problem – The Model –Application – Metrics – Action – End
  25. 25. Examples of useless Metrics - Number of Test Cases - Number of Bugs found - Degree (in %) of Automation Example: Percentage of manual TC ”transformed” into Automation → coverage is always according to a model, not the software itself - ““““Test Progress” i.e. “number of “test cases” “executed”””” Who – Problem – The Model –Application – Metrics – Action – End
  26. 26. Who – Problem – The Model –Application – Metrics – Action – End
  27. 27. Who – Problem – The Model –Application – Metrics – Action – End
  28. 28. Who – Problem – The Model –Application – Metrics – Action – End
  29. 29. Who – Problem – The Model –Application – Metrics – Action – End
  30. 30. Here’s a useful metric: Number of managers fired for asking their employees to follow idiotic metrics Who – Problem – The Model –Application – Metrics – Action – End
  31. 31. Here’s a useful metric: Number of managers fired for asking their employees to follow idiotic metrics, per month Who – Problem – The Model –Application – Metrics – Action – End
  32. 32. Here’s a useful metric: Number of managers fired for asking their employees to follow idiotic metrics, per month, per tester Who – Problem – The Model –Application – Metrics – Action – End
  33. 33. Call to Action Who – Problem – The Model –Application – Metrics – Action – End
  34. 34. Study Cynefin to get a better understanding of complexity Who – Problem – The Model –Application – Metrics – Action – End
  35. 35. Keep in mind that most of testing belongs in the complex area Who – Problem – The Model –Application – Metrics – Action – End
  36. 36. Push back on invalid demands Who – Problem – The Model –Application – Metrics – Action – End
  37. 37. Good Luck on Your Journey! Who – Problem – The Model –Application – Metrics – Action – End
  38. 38. And Now Your Questions Thanks! @ilarihenrik www.houseoftest.ch ilari.aegerter@houseoftest.ch Who – Problem – The Model –Application – Metrics – Action – End

×