One of the key features of agile is transparency which is usually done via large visible charts of team metrics in and around the team space. As agile is adopted by large organizations there is usually an effort to standardize metrics and develop organization specific metrics. However these new metrics can drive the wrong behavior or measure the wrong thing. This session identifies common agile metrics and demonstrates how they should and shouldn’t be used. It will also cover how to “coach-up” to ensure that metrics are consumed correctly by upper management. Finally we will also discuss how to identify bad metrics with a few common sense questions.

1. Bad agile metrics and
how to find them

2. Agenda
• Introduction
• Level set on what metrics are
• Why we use metrics
• How bad metrics can impact your team
• Bad agile metrics
• What good metrics look like
• Takeaways
• Q&A

3. Introduction
Faye Thompson
Agile Coach/Scrum Master
Cardinal Solutions
Charles Husemann
Agile Coach/Scrum Master
Cardinal Solutions

4. #MusicCityTech
#MusicCityAgile
Also we’ve updated our privacy policy….

5. What is a metric?

6. Metric:
A standard of measurement

7. Agile Metric:
Empirical measurement of a
dimension of a teams performance

8. Who are metrics for?
• The development team
• Management
• Project stakeholders
• Anyone walking by the team space

9. Why are metrics important?

10. Why metrics are important
• Give you the ability to Measure - Inspect - Adapt
• Tell the “story” of your team
• Promote transparency and visibility
• Tracks sprint/release progress
• Creates accountability
• Helps to achieve and optimize predictability

11. The power of metrics
• Sharpen team focus
• Provide measurable goals
• Promote understanding of goals
• Motivational tool
• See trends in work (good and bad)
• Drives team behavior (good and bad)

12. Why sniffing out bad metrics is
important

13. The impact of the misused metrics
• Disruptive to team
• Counterproductive to intent
• Set unmanageable/unrealistic expectations
• Discourages cooperation
• Team worries about visibility of metrics
• Encourages the wrong behavior

14. Two psychological effects to keep in mind
when talking about metrics
• Hawthorne Effect
• Streetlight Effect

15. Hawthorne Effect
The alteration of behavior by the subjects of a study due to their awareness of
being observed.

17. How else might this show up in
your team?

18. Streetlight Effect
A type of observational bias that occurs when people are searching for something
and look only where it is easiest

20. How would you fix this on your
team?

21. Bad agile metric smells
With great power comes great responsibility

22. Internal Team Competitions

23. Leaderboards

24. Competition within the team
• Impacts work selection and estimation
• Complexity of work
• Smaller stories
• Sandbag estimates and work
• Discourages cooperation by focusing on the individual rather than the
team

25. Team vs Team

26. Team comparison

27. Team size and composition could be different

28. Sprint length

29. Point sizes differ by team

30. Definition of done

31. Metrics out of context

32. Metrics without context
2016 Dark Side Challenge 2017 Dark Side Challenge

33. 0%
20%
40%
60%
80%
100%
120%
140%
Sprint
31
Sprint
32
Sprint
33
Sprint
34
Sprint
35
Sprint
36
Sprint
37
Sprint
38
Sprint
39
Sprint
40
Sprint
41
Sprint
42
Sprint
43
Sprint
44
Sprint
45
Sprint
46
Sprint
47
Sprint
48
2017 Sprint Completion Performance
Actual Performance Target Performance

34. Metrics live within a context
• Internal blockers
• Team member(s) out sick
• Holidays
• Influx of bugs
• Unplanned work
• External blockers
• BU availability
• Technical/Infrastructure issues
• Weather

35. Incentivizes the wrong thing

37. Incentivizes the wrong thing
• Avoid creating “Heroes”
• Goal is a sustainable and predictable pace – not to see who can get
the most points done in a sprint
• Is this measuring a desired behavior?

38. Other bad signs bad metrics
• Measure bad smells/non-agile things
• Lack sufficient data to see a trend
• Attempts to measure abstract concept

39. Good metrics

40. What do good metrics look like?
• Clear intent
• Not gameable
• Show progress over time
• Identify potential issues and areas for improvement
• Are understood by the entire team, not just the Scrum Master
• Smallest metrics that can possibly work
• A good metric for one team may not be applicable to another team

41. Examples
Are these good or bad?

42. Sprint Burndown Chart
65 65
62 62 62
58
58
58
58
16
0 0
3 3 3
11
11
11 11
0
10
20
30
40
50
60
70
80
StoryPoints
Transformers Sprint 3 Burndown
Remaining Total Story Points Completed

43. Velocity Chart
107
246
126
173
193
318
250
291
176
227
151
133
104
139
155
100
265
355
116
0
50
100
150
200
250
300
350
400
Team Awesome Velocity (Story and Defect Points)

44. Points per developer per sprint

45. Backlog Health
0
2
4
6
8
10
12
Sprint 31 Sprint 32 Sprint 33 Sprint 34 Sprint 35 Sprint 36 Sprint 37 Sprint 38 Sprint 39 Sprint 40 Sprint 41 Sprint 42 Sprint 43 Sprint 44 Sprint 45 Sprint 46 Sprint 47 Sprint 48
Product Backlog Health

46. Defects found per tester
0
2
4
6
8
10
12
14
16
18
20
Sprint 10 Sprint 11 Sprint 12 Sprint 13 Sprint 14 Sprint 15
Wally Sue Diana

47. WIP vs. Velocity
0
20
40
60
80
100
120
140
Sprint 12 Sprint 13 Sprint 14 Sprint 15 Sprint 16 Sprint 17 Sprint 18 Sprint 19 Sprint 20 Sprint 21
Work in Process Points per Week

48. Takeaways:
• Bad Metrics:
• Pit team members against each other
• Pit teams against each other
• Be obscure
• Measure bad practices
• Encourage bad behavior
• Good Metrics:
• Are clear and concise
• Encourage team behavior and cooperation
• Help the team identify areas for improvement and track progress towards goals
• Tell part of the story of your team
• Act as a conversation starter

49. Open Discussion

50. Faye Thompson
fthompson@cardinalsolutions.com
@agilefaye
Charles Husemann
chusemann@cardinalsolutions.com
@chusemann
Cardinal Solutions Group
Agile Advisory Services - CMH
Thank
You

51. Contact information
Faye Thompson - fthompson@cardinalsolutions.com
@agilefaye
Chuck Husemann – chusemann@cardinalsolutions.com
@chusemann