Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Kanban Metrics in practice for leading Continuous Improvement


Published on

Why should I bother collecting metrics? How can they help me? My CFD is pretty and colourful, but what is it actually trying to tell me?

CFD, control chart, lead time distribution, percentiles...Metrics can be daunting to start with but if you know how to interpret them they can drive continuous improvement and forecast the future and take your Kanban system to the next level! It’s much easier than you think, no need for complex maths or expensive software.

At Sky Network Services a few teams are using Kanban and metrics. In this talk I’ll share our experience: what metrics we use, how we use each one of them, what little data we collect to get a whole lot of value, what pitfalls we encountered.


Published in: Software

Kanban Metrics in practice for leading Continuous Improvement

  1. 1. for leading Continuous Improvement Kanban Metrics in practice @BattistonMattia
  2. 2. About me ● from Verona, Italy ● software dev & continuous improvement ● Kanban, Lean, Agile “helper” ● Sky Network Services Mattia Battiston @BattistonMattia Ciao!
  4. 4. Kan...what? a little knowledge of Kanban helps (limiting WIP, lead time, value vs waste, queues, batches, etc.)
  5. 5. Why do we need metrics? #1: drive continuous improvement #2: forecast the future
  6. 6. But I thought metrics were bad.... Typical problems: gaming dysfunctions
  7. 7. Good vs Bad metrics ● look at improving the whole system ● reward/punish individuals “95% performance is attributable to the system, 5% to the people” W. Edwards Deming ● feedback about state of reality ● used as target ● leading (let you change behaviour) ● lagging (tell you about the past) ● all metrics must improve ● local optimisations
  8. 8. Our system Iteration-Based On-demand Direct
  9. 9. How do we collect the data?
  10. 10. The Spreadsheet Inputs: story details; start time and duration of each state Public version: For you to copy, reuse, get inspired, etc.
  11. 11. All the maths you need ● Min, Max Normal: data is distributed around a central value e.g. height of UK population Skewed: data has a long tail on one side (positive or negative) e.g. income of UK population (positive skew) Lead time of stories follows skewed distribution ● Average (mean) avg(1,2,2,2,3,14) = (1+2+2+2+3+14)/6 = 4 ● Median: separates the high half from the low half. Less impacted by outliers median(1,2,2,2,3,14) = 2 ● Mode: value that occurs more frequently mode(1,2,2,2,3,14) = 2 ● Standard Deviation: measures the amount of dispersion from the average. When high, values are spread over a large range. stdev(1,2,2,2,3,14) = 4.5; stdev(1,2,2,2,3,5) = 1.2; ● Percentile: percentage of elements that fall within a range 50% perc(1,2,2,3,7,8,14) = 3; 80% perc(1,2,2,3,7,8,14) = 7.8; ● Normal Distribution vs Skewed Distribution:
  12. 12. Cumulative Flow Diagram Description: Each day shows how many stories are in each state n.stories days
  13. 13. Cumulative Flow Diagram Ideal CFD: thin lines growing in parallel at a steady rate -> good flow!
  14. 14. Cumulative Flow Diagram ● Objective: retrospect (but needs a good facilitator) CFD used for Retrospective ● Objective: demonstrate effectiveness of changes changed WIP limit in DEV from 3 to 2
  15. 15. Cumulative Flow Diagram ● Objective: decide what you should work on today ● Objective: forecasting: rough info about lead time, wip, delivery date (although they’re easier to use when tracked separately) WIP Lead Time
  16. 16. CFD Patterns (taken from CFD article by Pawel Brodzinski) growing lines: indicate large WIP + context switching. action: use WIP limits stairs: indicates large batches and timeboxes action: move towards flow (lower WIP, more releases, cross-functional people) flat lines: nothing’s moving on the board action: investigate blockers, focus on finishing, split in smaller stories single flat line: testing bottleneck action: investigate blockers, pair with testers, automate more typical timeboxed iterationdropping lines: items going back action: improve policies
  17. 17. metrics for Delivery Time
  18. 18. Control Chart Description: For each story it shows how long it took. Displays Upper and Lower control limits; when a story falls out of limits something went wrong and you should talk about it. stories leadtime(days)
  19. 19. Cycle/Lead Time stats + History Description: Stats to get to know your cycle time and lead time. They let you predict “how long is the next story likely to take?”. Visualize trends of improvement
  20. 20. Lead Time distribution lead time (days) n.storiesthattookthatlong Description: For each lead time bucket (#days), how many stories have taken that long. Useful to show as a percentage to know probability. 50% 80%
  21. 21. Story Health Description: Indicates if the story is in good health or if we should worry about it. Based on lead time distribution 50-80% >90%80-90%0-50% 0-4 gg 5-7 gg 8-10 gg >10 gg
  22. 22. Cycle Time vs Release Prep. Time stories days Description: For each story shows how long it spent in the iteration and in release preparation (Context specific). Used to discuss cost vs value of release testing.
  23. 23. metrics for Predictability
  24. 24. Iteration Throughput iteration no.storiescompleted Description: Number of stories that get done for each iteration
  25. 25. Rolling Wave Forecasting Description: visualise in the backlog the likelihood of stories getting done in the next 2 weeks
  26. 26. Arrivals Rate Description: how often a story is started, aka pulled into our system (arrival). This is how often you can change your mind about what to do next
  27. 27. Points vs Lead Time leadtime(days) story points Description: Shows low correlation between estimated points and actual lead time
  28. 28. Disney Stations Description: Like queueing at Disneyland. “How long in here? How long from here?”
  29. 29. Task Time Description: Shows how long tasks usually take (context specific). Gives you an idea of how long a story will take based on n. of tasks
  30. 30. metrics for Quality
  31. 31. Bugs percentage Description: Percentage of bugs over stories. Also expressed as “1 bug every X stories”
  32. 32. metrics for Continuous Improvement
  33. 33. Flow Efficiency Description: Shows how long stories have spent in queues - nobody was working on them. Shows how much you could improve if you removed waiting time.
  34. 34. Time in status timespentinstate(days) story Description: for each story visualise how long it spent in each status (absolute and percentage). Shows trends of where stories spend more time
  35. 35. Retrospective
  36. 36. Resources Books Metrics ● Data driven coaching - Troy Magennis ● Seven Deadly Sins of Agile Measurement - Larry Maccherone ● The Impact of Lean and Agile Quantified - Larry Maccherone ● Kanban at Scale: A Siemens Success Story - Bennet Vallet ● FocusedObjectives@Github - Troy Magennis ● Visual feedback brings key Agile principles to life - Bazil Arden ● How visualisation improves Psychological Safety - Bazil Arden Forecasting ● Cycle Time Analytics - Troy Magennis ● Top Ten Data and Forecasting Tips - Troy Magennis ● Forecasting Your Oranges - Dan Brown ● Using Maths to work out Potentially Deliverable Scope - Bazil Arden ● Forecasting Cards - Alexei Zheglov Story Points ● Story Points and Velocity: The Good Bits - Pawel Brodzinski ● No correlation between estimated size and actual time taken - Ian Carroll Lead Time ● Analyzing the Lead Time Distribution Chart - Alexei Zheglov ● Inside a Lead Time Distribution - Alexei Zheglov ● Lead Time: what we know about it, how we use it - Alexei Zheglov ● The Economic Impact of Software Development Process Choice - Troy Magennis More ● Flow Efficiency - Julia Wester ● Cumulative Flow Diagram - Pawel Brodzinski
  37. 37. Thank you! @BattistonMattia really, really appreciated! Help me improve