Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Metrics for Agile Teams Forget Velocity: 42 Other Things to Ponder

115 views

Published on

Velocity is one of the most commonly used - and abused - development metrics. Teams (and their stakeholders) often focus on “improving velocity” without either a proper consideration for root causes that impact velocity or a holistic view of a team’s outcomes.

Join Andy Cleff in a lively discussion that explores how we can remove perverse incentives and instead provide healthier ways for teams to gain meaningful insights into the outcomes of their experiments.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Metrics for Agile Teams Forget Velocity: 42 Other Things to Ponder

  1. 1. Metrics for Agile Teams Forget Velocity, 42 Other Things to Consider! Presented by Andy Cleff Sr. Agile Coach and Trainer
  2. 2. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Overview 12 rules for measurement 5 Categories & 42+ metrics How to choose? Workshop 2 andycleff.com/metrics
  3. 3. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Poll: Who’s here? What is your primary role? (pick one) • Product Owner • Dev-Team • Scrum Master • Agile Coach • Management • Other 3 POLL
  4. 4. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Poll Results: Who’s here? 4 POLL
  5. 5. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Andy.Cleff@ agilevelocity.com AgileVelocity.com Sr. Agile Coach and Trainer Andy Cleff 5
  6. 6. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com 12 rules for measurement 1: Measure for a purpose 2: Shrink the unknown 3. Seek to improve 4: Delight all stakeholders 5: Distrust all numbers 6: Set imprecise targets 7: Own your metrics 8: Don’t connect metrics to rewards 9: Promote values and transparency 10: Visualize and humanize 11: Measure early and often 12: Try something else change and innovation practices MANAGEMENT 3.0 12 Rules for Measurement When selecting metrics, ask: Rule 1: Measure for a purpose You must always understand why you are measuring. The metric is not a goal in itself. Never forget that it’s just a means to an end. It all starts with why. Rule 2: Shrink the unknown A metric is just a surrogate for what you really want to know. Don’t jump to conclusions. Always try to reduce the size of what is still unknown. Rule 3. Seek to improve Don’t only measure things that will make you look good. There is plenty of data around, but you must focus on what enables you to do better work. Rule 4: Delight all stakeholders Your work depends on others, and others depend on you. Never optimize for just one stakeholder. Instead, measure your work from multiple perspectives. Rule 5: Distrust all numbers attitude towards any reported numbers. Rule 6: Set imprecise targets When people have targets, they have an inclination to focus on the targets instead of the real purpose. Avoid this tendency by keeping your targets vague. Rule 7: Own your metrics Everyone is responsible for their own work, and metrics help us improve that work. Therefore, everyone should be responsible for their own metrics. Rule 8: Don’t connect metrics to rewards Rewards often kill intrinsic motivation and lead to dysfunctional behaviors in organizations. Don’t incentivize people to do work they should like doing. Rule 9: Promote values and transparency To prevent gaming, be transparent about values, intentions, and the metrics everyone is using. Rule 10: Visualize and humanize Numbers tend to dehumanize everything. Replace digits with colors and pictures, and keep the measurements close to where the actual work is done. Rule 11: Measure early and often Most people don’t measure often enough. Measure sooner and faster to prevent risks and problems from growing too big for you to handle. Rule 12: Try something else It’s rarely a good idea to do the same things over and over. The environment changes all the time. The same should apply to how and what you measure. Why “this metric?” – Why does it matter? What insights might we gain from it? What is expected to change? What is expected variability, consistency – are we looking for trends or absolute values? Working to improve one thing may temporarily reduce promote knowledge sharing, collaboration with other teams and trust with our sponsors? Is this metric a leading or lagging indicator? 6 Shout out to Jurgen Appelo & Management 3.0 Big thanks to Ralph van Rosmalen for getting the ball rolling
  7. 7. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Rule 1- Measure for a purpose You must always understand what you are measuring. The metric is not a goal in itself. Never forget that it’s just a means to an end. It all starts with why. 7
  8. 8. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Measure for outcomes, not outputs Employee Engagement Continuous Improvement Predictability Customer Satisfaction Market Responsiveness Productivity Speed Quality Innovation 8
  9. 9. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Reasons why we do measure To know revenues and drive resource & people allocation accordingly To make decisions that are not based on gut feelings alone Monitor alignment with mission / vision / goal Observe quality of product / process Judge customer happiness / employee satisfaction 9
  10. 10. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Reasons why we don’t measure Measurements might be used as weapons Some metrics are not useful or actionable Implementing measures would cost too much time / effort Some things might just be immeasurable 10
  11. 11. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Rule 2 - Shrink the unknown A metric is just a surrogate for what you really want to know. Don’t jump to conclusions. Always try to reduce the size of what is still unknown. 11
  12. 12. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Cynefin Framework 12
  13. 13. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Ordered Domains Cynefin Framework 13 OBVIOUS CLEAR
  14. 14. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Here Things Get Interesting… Cynefin Framework 14 APORETIC or CONFUSED
  15. 15. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Rule 3 - Seek to improve Don’t only measure things that will make you look and feel good. There is plenty of data around, but you must focus on what enables you to do better work. 15
  16. 16. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Actionable metrics “A good metric changes the way you behave. This is by far the most important criterion for a metric: what will you do differently based on changes in the metric?” Lean Analytics, Alistair Croll and Benjamin Yoskovitz 16
  17. 17. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Vanity metrics “When we rely on vanity metrics, a funny thing happens. When the numbers go up, I've personally witnessed everyone in the company naturally attributing that rise to whatever they were working on at the time. That's not too bad, except for this correlate: when the numbers go down, we invariably blame someone else” Eric Ries 17
  18. 18. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Rule 4 - Delight all stakeholders Your work depends on others, and others depend on you. Never optimize for just one stakeholder. Instead, measure your work from multiple perspectives. 18
  19. 19. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com It is impossible to please everyone, but you would like to know who is pleased at certain moments and who is not. 19 What do your partners care about?
  20. 20. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Rule 5 - Distrust all numbers Observers usually influence their own metrics, and they suffer from all kinds of biases. Have a healthy, skeptical attitude towards any reported numbers. 20
  21. 21. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com 21 Cognitive biases
  22. 22. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Story Hawthorne Works, Chicago, ca. 1925. 22
  23. 23. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Story 23
  24. 24. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Story 24
  25. 25. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Story 25 The Hawthorne, or Observer, Effect
  26. 26. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Rule 6 - Set imprecise targets When people have targets, they have an inclination to focus on the targets instead of the real purpose. Avoid this tendency by keeping your targets vague. 26
  27. 27. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com "When a measure becomes a target, it ceases to be a good measure." Goodhart's law 27
  28. 28. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Rule 7 - Own your metrics Everyone is responsible for their own work, and metrics help us improve that work. Therefore, everyone should be responsible for their own metrics. 28
  29. 29. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Important considerations How many metrics should a team “own”? Which ones? How long should they use the ones selected? 29
  30. 30. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Anti-Patterns People know/suspect they are being watched and judged (Hawthorn) Striving for ever increasing values instead of striving for consistency and stability (Goodhart) 30
  31. 31. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com More Anti-Patterns Correlation is not necessarily causation (Milton Friedman’s Thermostat) 31
  32. 32. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com More Anti-Patterns Comparing metrics across teams that are very different 32
  33. 33. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Poll: Scale of your ecosystem Number of Teams (pick one) • Less than 5 • Between 6 and 9 • More than 10 33 POLL
  34. 34. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Poll Results: Scale of your ecosystem 34 POLL
  35. 35. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Rule 8 - Don’t connect metrics to rewards Rewards often kill intrinsic motivation and lead to dysfunctional behaviors in organizations. Don’t incentivize people to do work they should like doing. 35
  36. 36. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com DILBERT © 1999 Scott Adams. Used By permission of ANDREWS MCMEEL SYNDICATION. All rights reserved. 36 Anti-patterns
  37. 37. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Rule 9 - Promote values and transparency Human beings are smart and able to game any system. To prevent gaming, be transparent about values, intentions, and the metrics everyone is using. 37
  38. 38. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Values, intention, purpose Do we get paid a sustainable value for what we do? Are we great at what we do in the eyes of our customers? Do our employees / teammates love what we do and the way we do it? Will what we do make the world a better place for our grandchildren? 38
  39. 39. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com 39 Values, intention, purpose
  40. 40. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Rule 10 - Visualize and humanize Numbers tend to dehumanize everything. Replace digits with colors and pictures, and keep the measurements close to where the actual work is done. 40
  41. 41. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Story 41
  42. 42. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Story 42
  43. 43. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Story 43
  44. 44. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Story 44
  45. 45. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Story 45
  46. 46. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Rule 11 - Measure early and often Most people don’t measure often enough. Measure sooner and faster to prevent risks and problems from growing too big for you to handle. 46
  47. 47. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com 47 “The only way to win is to learn faster than anyone else” Eric Ries “What you want to do as a company is maximize the number of experiments you can do per unit of time.” Jeff Bezos On winning…
  48. 48. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Rule 12 - Try something else It’s rarely a good idea to do the same things over and over. The environment changes all the time. The same should apply to how and what you measure. 48
  49. 49. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Limited lifespan of all metrics That which is measured will improve, at a cost. When a measure becomes a target, it ceases to be a good measure. (Goodhart) Correlation is not causation, but it sure is a hint. (Friedman) Use multiple viewpoints - technical as well as human - to get a holistic perspective 49
  50. 50. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com 12 rules for measurement 1: Measure for a purpose 2: Shrink the unknown 3. Seek to improve 4: Delight all stakeholders 5: Distrust all numbers 6: Set imprecise targets 7: Own your metrics 8: Don’t connect metrics to rewards 9: Promote values and transparency 10: Visualize and humanize 11: Measure early and often 12: Try something else 50
  51. 51. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Discussion / Q & A Is anything confusing so far? What are you doing already that's working? What's not working for you? 51
  52. 52. Optional ✦ When you return from the break, ask yourself the following question: “What is the most important concept I’ve learned so far?” Break Over In: Sponge Activity
  53. 53. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Toolbox of metrics 53
  54. 54. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com 54 Business Outcomes
  55. 55. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Poll: Your biggest “why” What business outcome is the most important now? (pick one) • Employee Engagement • Continuous Improvement • Customer Satisfaction • Predictability • Market Responsiveness • Productivity • Speed • Quality • Innovation 55 POLL
  56. 56. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Poll Results: Your biggest “why” 56 POLL
  57. 57. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Five categories of metrics Process Health Release Product Development Technical / Code People / Teams 57 Shout out to: Jason Tice @TheAgileFactor
  58. 58. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Process health metrics Assess day-to-day delivery team activities and evaluate process changes: • Cycle time • Cumulative flow diagrams • Control charts • Number of experiments performed • Number of improvements made to process over time • Flow efficiency • “Concept to cash” delivery lead time • Batch size • Successful iteration completion • Escaped defect resolution time • Time thievery 58
  59. 59. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Shine a light on the time thieves 1. Too much work in progress (WIP) 2. External dependencies 3. Unplanned work 4. Conflicting priorities 5. Neglected work 59
  60. 60. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Cumulative Flow Diagram (CFD) 60
  61. 61. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Cumulative Flow Diagram (CFD) 61 kanbanize.com Arrivals Departures WIP
  62. 62. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Cumulative Flow Diagram (CFD) 62 kanbanize.com
  63. 63. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Cumulative Flow Diagram (CFD) 63 Cycle Time kanbanize.com
  64. 64. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Release metrics Focus on identifying impediments to continuous delivery: • Deployment frequency • Release success rate • Failed build frequency • Mean time to restore (MTTR) • Release time • Cost per release • Escaped defects • Release net promoter score • Release adoption / install rate 64
  65. 65. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Net promoter score (NPS) 65 surveymonkey.com
  66. 66. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Product development metrics Help measure alignment of product features to user needs: • Customer value delivered • Risk burn down • Value stream mapping • Sales velocity • Product net promoter score (NPS) • User analytics / DAU / MAU • Backlog health index • Number of validated business-level hypotheses developed • Number of times a week you talk to an actual customer • Ratio of implemented to non- implemented customer-driven changes • HEART • AARRR 66
  67. 67. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com DAU/MAU 67 geckoboard.com
  68. 68. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com HEART Framework 68 Goal Signals Metrics Happiness For users to feel like the site is unique A satisfaction rating from a survey Satisfaction Perceived ease of use Net promoter score Engagement For users to keep discovering more content The amount to time people spend viewing from analytics Number of videos watched per user per week Number of uploads Number of shares Adoption For users to use the site or app The number of signup per days, from analytics Upgrades to latest version New subscriptions Purchased by new users Retention For users to continue to use the site or app The number of return customers from customer data Number of active users Renewal rate Repeat purchase Task Success For users to accomplish their goal The number of successful completes from a study Search result success Time to upload Profile creation completion Google Ventures
  69. 69. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com AARRR (Pirate Metrics) • Acquisition • Activation • Retention • Referral • Revenue 69
  70. 70. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Technical & code metrics Help determine quality of implementation and architecture: • Automated test coverage • Number of tests written before coding • Unit / regression test coverage • Build time • Defect density • Code churn • Code ownership • Code complexity • Coding standards adherence • Crash rate / time to restore service • Build breaks / change fail rate • Technical drag • Ratio of fixing work vs feature work 70
  71. 71. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Test automation dashboards 71 getzephyr.com
  72. 72. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com People & teams Reveal issues that impact a team’s sustainable pace and level of engagement: • Team health and well-being • Team / manager / org eNPS • % of time w/o interruptions (fire fights) • Trust between leadership and team • Learning log • Overtime hours • New employee setup time • Team tenure / manager half-life • Mean time to reorg • Whole team contribution • Transparency (access to data, customers, shared learning) • Employee silence vs voice • Number of experiments / changes 72
  73. 73. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Why team health and well being matters A strong constitution is necessary for teams to overcome adversity, maintain resilience and nurture a culture rich in collaboration and innovation. 73
  74. 74. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Resilience Dictionary: The capacity to adapt to stress or loss healthfully • Positivity • Focus • Flexibility • Organization • Proactivity 74 Shout out to: Daryl Conner
  75. 75. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Visualizing team health: andycleff.com/teamhealth Gallup Q12 Mood App + Health Monitor TeamMood eNPS Comparative Agility Survey Crisp Happiness Index Team Barometer Spotify Team Health Check Happiness Wall / Window Lean Agile Intelligence Team Morale Metric H.I.P. Survey OfficeVibe 15Five 7Geese 75
  76. 76. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Team mood 76 teammood.com Team members will receive a daily email and/or slack prompt, asking for their mood of the day. One click and done!
  77. 77. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Team mood 77 teammood.com
  78. 78. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Measuring Team Culture 78 Forsgren PhD, Nicole. Accelerate . IT Revolution Press. Kindle Edition. On my team….
  79. 79. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Measuring Trust and Ownership 79
  80. 80. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Measuring Trust and Ownership 80
  81. 81. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Measuring Trust and Ownership 81
  82. 82. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Measuring Trust and Ownership 82
  83. 83. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Discussion / Q & A Is anything confusing so far? What are you doing already that's working? What's not working for you? 83
  84. 84. Optional ✦ When you return from the break, think about what you’ve learned so far and make a list of the names of colleagues with whom you could share some or all of the information you’ve picked up so far. Break Over In: Sponge Activity
  85. 85. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Care and usage As George Box said, “All models are wrong, but some are useful.” 85
  86. 86. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Integral lenses I / Personal: This is about “Me” – my psychological, inner world; how I see things, what I experience, my mindset. 86 We / Relational: A shared vision and interpersonal dynamics among peers; how we interact, what we value together, the team micro-culture. Its / Environmental: Things external to the team systems, including other value streams, handoffs, rules, corporate culture, and policies. It / Practices: Refers to the team structure, methods, metrics, processes, decision making patterns, leadership and team practices.
  87. 87. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Who a metrics model is for There are two groups of stakeholders: • The individuals & teams (I + We) • The people & systems supporting the teams (It + Its) I Personal Subjective IT Practices Objective WE Relational Intersubjective ITS Environment Interobjective Individual / Singular Collective / Multiples Interior / Invisible Exterior / Visible 87
  88. 88. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com From the team perspective • Builds up awareness • Reveals working and what’s not • Expands perspectives • Provides a balanced view 88
  89. 89. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com From the supporting role perspective • Provides high level summary • Reveals patterns across teams • Shows opportunities to change the system 89
  90. 90. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com What this is not • A competition • A comparison • A weapon 90
  91. 91. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com What this is not • An incentivisation model 91
  92. 92. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com What this is not • Done 92
  93. 93. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com How to choose? 93
  94. 94. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Review the options…. Why “this metric?” – Why does it matter? To whom does it matter? What insights might we gain from it? What is expected to change? Are we looking for variability, consistency, trends, values, direction? 94
  95. 95. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Review the options…. How might a metric be gamed, misused (or abused)? What are some trade offs (collateral damage) as well as the costs of getting “better data”? 95
  96. 96. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Review the options…. How often would we like to “take a data point”? How long will we run the experiment? How when we know when we’re “done” with this metric? 96
  97. 97. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Review the options…. How will we make this measurement transparent? Is this metric a leading or lagging indicator? 97
  98. 98. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com If you’re really in a hurry to get started… 1 = Choose from People & Teams 2 = People & Teams + track average age of items in process 3 = People & Teams + any two from Little’s Law (CT = WIP / TH) and then you’ll get one bonus for free! 5 = People & Teams + Little’s Law + Customer NPS + Time Since Last Release 98
  99. 99. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Summary so far… • Use predictability to delight • Abandon old concepts • Measure many things (just not too many) • Learn rapidly • More geek joy 99
  100. 100. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Discussion / Q & A Is anything confusing so far? What are you doing already that's working? What's not working for you? 100
  101. 101. Optional ✦ When you’ve returned from the break, and before you sit down, take a couple of really deep breaths. Maybe stretch your arms overhead too. ✦ Then add a comment about what you like best about the course so far. Break Over In: Sponge Activity Print out worksheets from: andycleff.com/metrics
  102. 102. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Putting Theory into Practice 102 andycleff.com/metrics
  103. 103. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com 1. Explore what’s going well along with your main pain points in the context of your desired business outcome 2. What items are a priority to experiment with? 3. Brainstorm what metrics you might try. Come up with some good ones, and some bad ones too… 4. Work thru the “when selecting metrics” questions… Workshop: Overview 103 andycleff.com/metrics
  104. 104. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Workshop: Paint your canvas… 1. 2. What items are a priority to experiment with? 3. Brainstorm what metrics you might try. Come up with some good ones, and some bad ones too… 4. Work thru the “when selecting metrics” questions… 104 1. Explore what’s going well along with your main pain points in the context of your desired business outcome
  105. 105. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Example: What’s going well / main pain points 105 + Team does lunch and learns ! Too much “must fix now” ! Sprints over capacity ! No ability to A/B test ideas ! Infrequent releases
  106. 106. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Workshop: Paint your canvas… 1. 2. What items are a priority to experiment with? 3. Brainstorm what metrics you might try. Come up with some good ones, and some bad ones too… 4. Work thru the “when selecting metrics” questions… 106 1. Explore what’s going well along with your main pain points in the context of your desired business outcome
  107. 107. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Workshop: Paint your canvas… 1. Explore what’s going well along with your main pain points in the context of your desired business outcome 2. What items are a priority to experiment with? 3. Brainstorm what metrics you might try. Come up with some good ones, and some bad ones too… 4. Work thru the “when selecting metrics” questions… 107
  108. 108. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com 108 Example: Priority to experiment with 0% slack time 20% slack time 6 wks 2x
  109. 109. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Workshop: Paint your canvas… 1. Explore what’s going well along with your main pain points in the context of your desired business outcome 2. What items are a priority to experiment with? 3. Brainstorm what metrics you might try. Come up with some good ones, and some bad ones too… 4. Work thru the “when selecting metrics” questions… 109
  110. 110. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Workshop: Paint your canvas… 1. Explore what’s going well along with your main pain points in the context of your desired business outcome 2. What items are a priority to experiment with? 3. Brainstorm what metrics you might try. Come up with some good ones, and some bad ones too… 4. Work thru the “when selecting metrics” questions… 110
  111. 111. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com 111 Example: What metrics you might try Customer NPS DAU/MAU eNPS / Team Mood Story points per sprint DAU (alone)
  112. 112. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Workshop: Paint your canvas… 1. Explore what’s going well along with your main pain points in the context of your desired business outcome 2. What items are a priority to experiment with? 3. Brainstorm what metrics you might try. Come up with some good ones, and some bad ones too… 4. Work thru the “when selecting metrics” questions… 112
  113. 113. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Workshop: Paint your canvas… 1. Explore what’s going well along with your main pain points in the context of your desired business outcome 2. What items are a priority to experiment with? 3. Brainstorm what metrics you might try. Come up with some good ones, and some bad ones too… 4. Work thru the “when selecting metrics” questions… 113
  114. 114. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Workshop: Paint your canvas… Assumptions and corresponding tests: • We believe that [persona type] has [trouble/need/desire] doing [problem/opportunity]. • We will know we have succeeded when [qualitative outcome] and [quantitative outcome]. This will improve [kpi / measurable signal]. 114
  115. 115. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com 115 Example: Assumptions and corresponding tests We believe that our active users has a desire for more frequent releases that include innovative social sharing features. We know we have succeeded when: • DAU/MAU reaches 50% within 3 months… • Dev-Team mood on average is above 7/10… • Velocity decreases 20%…
  116. 116. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Workshop: Paint your canvas… Assumptions and corresponding tests: • We believe that [persona type] has [trouble/need/desire] doing [problem/opportunity]. • We will know we have succeeded when [qualitative outcome] and [quantitative outcome]. This will improve [kpi / measurable signal]. 116
  117. 117. Sharing •What are the things you’ll measure? •How will you learn rapidly? •Will this move you towards predictability? •How will you visualize things? •How will your information help you make better decisions? 117Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
  118. 118. If you get stuck… •How much? As in, how many “widgets” are you producing each week? •How fast? How long does it take to create that item from the second the conversation starts to when it’s finished? •How good? What’s the quality? •How predictable? If you tell a customer to expect an item within a certain number of days, how frequently do you meet that deadline? •How happy? Is your team happy and is your culture healthy? 118Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
  119. 119. If you get stuck… 119Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Business Outcome Example Measures Employee Engagement • eNPS • Bad turnover • Employee referrals • Offer acceptance  • Feedback surveys • Team learning logs Continuous Improvement • Value stream flow efficiency • Reduction in recurring impediments • Cumulative flow
  120. 120. 120Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Business Outcome Example Measures Innovation • Market Share • Number of validated business-level hypotheses developed and tested • Slack time Customer Satisfaction • Number of times a week team members talk to an actual customer • NPS  • Retention (DAU/MAU) • Referrals
  121. 121. 121Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Business Outcome Example Measures Market Responsiveness • AARRR (Acquisition, Activation, Retention, Referral, Revenue) • HEART, (Happiness, engagement, adoption, retention, task success) Predictability • % complete of Sprint plan • Velocity variability • Say-Do ratio  • Unplanned work items Speed • Cycle time  • Lead time • Deployment frequency • Meantime to restore (MTTR)
  122. 122. 122Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Business Outcome Example Measures Quality • Defects in production • Change fail percentage • Automated test coverage • Ratio of fixing work vs feature work Productivity • Value delivered • ROI • KPIs • Thieves of time
  123. 123. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com Let’s keep the conversation going… If you need to work on your foundational practices... If you need to re-focus... If you feel stuck mid-transformation... If you need ongoing support during your transformation… Please contact us, we offer: • Private Agile Training • Agile Assessments • Agility Tune-ups • Agile Transformation Services 123

×