1. Complexity Thinking,
Cynefin & Why Your
Testing Metrics All Suck
Big Time

2. Complexity Thinking, Cynefin &
Why All Your Testing
Metrics All
Sucks Big Time

3. Who Am I?
Who – Problem – The Model –Application – Metrics – Action – End

4. llari Henrik Aegerter
Managing Director – House of Test
VP of Marketing – Association for Software Testing
Beer Brewer
@ilarihenrik
www.houseoftest.ch
Who – Problem – The Model –Application – Metrics – Action – End

5. Disclaimer:
Keep in mind that I am a
*%§!$ Consultant
Question everything I say!
Who – Problem – The Model –Application – Metrics – Action – End

6. What is the problem?
Who – Problem – The Model –Application – Metrics – Action – End

7. We are applying methods in
software testing we wouldn’t in
comparable other situations
Who – Problem – The Model –Application – Metrics – Action – End

8. Who – Problem – The Model –Application – Metrics – Action – End

9. The Cynefin Model
Who – Problem – The Model –Application – Metrics – Action – End

10. Confusion between
Complicated vs. Complex
Complicated:
Degree to which something is difficult to
understand
Complex:
Degree to which Cause and Effect
are deterministic
Who – Problem – The Model –Application – Metrics – Action – End

11. Who – Problem – The Model –Application – Metrics – Action – End

12. Ordered
Linear
Complex
Non-Linear
Who – Problem – The Model –Application – Metrics – Action – End

13. How to apply Cynefin in
Testing
Who – Problem – The Model –Application – Metrics – Action – End

14. When confronted with a
testing task, ask yourself
how to act
Who – Problem – The Model –Application – Metrics – Action – End

15. Obvious
Sense - Categorize – Respond
World Expample:
Traffic Light
Testing Example:
Checking GUI Elements
Who – Problem – The Model –Application – Metrics – Action – End

16. Complicated
Sense - Analyze – Respond
World Expample:
Building a Space Ship
Testing Example:
Analysis of non-trivial
calculations
Who – Problem – The Model –Application – Metrics – Action – End

17. Complex
Probe - Sense – Respond
World Expample:
A roundabout
Testing Example:
Almost everything in testing,
Unknown unknowns, Exploratory
Testing
Who – Problem – The Model –Application – Metrics – Action – End

18. Safe to Fail Experiments
Who – Problem – The Model –Application – Metrics – Action – End

19. Chaos
Act - Sense – Respond
World Expample:
Kids
Testing Example:
A project in disarray
Who – Problem – The Model –Application – Metrics – Action – End

20. Some words about metrics
Who – Problem – The Model –Application – Metrics – Action – End

21. Cem Kaner on Metrics and
Measurements
http://kaner.com/pdfs/PracticalApproachToSoftwareMetrics.pdf
Who – Problem – The Model –Application – Metrics – Action – End

22. A General Problem with Metrics
Question is about Quality -
Measurements are Quantitative
Hence your measurement is a
surrogate one
Who – Problem – The Model –Application – Metrics – Action – End

23. Measurement Distortions
and
Goal Replacements
Who – Problem – The Model –Application – Metrics – Action – End

24. Who – Problem – The Model –Application – Metrics – Action – End

25. Examples of useless Metrics
- Number of Test Cases
- Number of Bugs found
- Degree (in %) of Automation
Example: Percentage of manual TC
”transformed” into Automation
→ coverage is always according to a model,
not the software itself
- ““““Test Progress” i.e. “number of “test
cases” “executed””””
Who – Problem – The Model –Application – Metrics – Action – End

26. Who – Problem – The Model –Application – Metrics – Action – End

27. Who – Problem – The Model –Application – Metrics – Action – End

28. Who – Problem – The Model –Application – Metrics – Action – End

29. Who – Problem – The Model –Application – Metrics – Action – End

30. Here’s a useful metric:
Number of managers fired for
asking their employees to
follow idiotic metrics
Who – Problem – The Model –Application – Metrics – Action – End

31. Here’s a useful metric:
Number of managers fired for
asking their employees to
follow idiotic metrics, per
month
Who – Problem – The Model –Application – Metrics – Action – End

32. Here’s a useful metric:
Number of managers fired for
asking their employees to
follow idiotic metrics, per
month, per tester
Who – Problem – The Model –Application – Metrics – Action – End

33. Call to Action
Who – Problem – The Model –Application – Metrics – Action – End

34. Study Cynefin to get a
better understanding of
complexity
Who – Problem – The Model –Application – Metrics – Action – End

35. Keep in mind that most of
testing belongs in the
complex area
Who – Problem – The Model –Application – Metrics – Action – End

36. Push back on invalid
demands
Who – Problem – The Model –Application – Metrics – Action – End

37. Good Luck on Your
Journey!
Who – Problem – The Model –Application – Metrics – Action – End

38. And Now Your Questions
Thanks!
@ilarihenrik
www.houseoftest.ch
ilari.aegerter@houseoftest.ch
Who – Problem – The Model –Application – Metrics – Action – End