Metrics As A Learn And Change Agent


Published on

my presentation at AgileIndia 2012

Published in: Technology, Business
1 Comment
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Metrics As A Learn And Change Agent

  1. metrics as alearn and change agent Gaetano Mazzanti @mgaewsj Gama-Tech
  2. as an agile coach I am often asked todefine/introduce KPIs “why?”
  3. Case 1 “uhm, ehm, weneed to measure code & fixif and how muchwe are improving” no process chaos
  4. *typicallyfor performance appraisals Case 2 “we need to measure traditional if and how much we are improving;rigid process top-down company policies enforce using KPIs*”
  5. unfortunately things arenot so simple and linear
  6. systems: some definitions system any network that has coherence, it may be fuzzy, it may have purpose agent anything which acts within the system: individual, group, idea, etc.ordered complex chaotic
  7. complex system lightly constrains agents agents modify system by their interaction with it and each other, they co-evolveorderedsystem constrains agents chaotic agents underconstrained & independent of each otherordered complex chaotic
  8. no metrics code & fix no process chaosordered complex chaotic
  9. metrics for a linear,deterministic world no metrics traditional code & fix rigid process top-down no process chaos ordered complex chaotic
  10. ??? agilemetrics for a linear,deterministic world no metrics traditional code & fix rigid process top-down no process chaos ordered complex chaotic
  11. complex systems exhibit one or more properties (behavior) not obvious from the properties of the individual interconnected parts they are composed of
  12. product development is complex“self-organizing, non-linear, feedback systems are inherently unpredictable, they are not controllable“ D.Meadows
  13. so what are wesupposed to do?
  14. we canwatch, learn and work with the system
  15. metricslearn & change
  16. how?
  17. loop!
  18. PDCA Loop planact do check Shewhart & Deming
  19. OODA Loopobserve orient act decide J.Boyd
  20. LeanStartup Loop Ideas  Learn   Build   Data   Product   are we building the right product? Measure   E.Ries
  21. when loopingminimize total time through the looplearn fastfail fast! “biggest failure is failure to learn” (M.Poppendieck)
  22. single loop learning lead to actions results how which shape future efficiency doing things right incremental change
  23. single loop drawbacks uncertainty emerging information unexpected impediments delays plan defects more detailed planning rigid change management overburden (waste)
  24. double loop learning Chris Argyris guide values, actions results assumptions how whystepoutsidethesystem! lead to new/improvedeffectiveness efficiencydoing the right things doing things rightquestion assumptions incremental changeradical change
  25. timebound metrics learn, change, move on define metric* set expiration date goal ok or values, expiration actions results assumptions date passed?*actionable, accessible, auditable, time-bound
  26. do not focus on the metric itself, it is just a mean to understand/learn and changemeasures have value only if they inform decisions or motivate action
  27. be carefulhow do you know that the results you are seeing are related to the changes you have made?how do you know you are learning correctly from those changes?
  28. what’s going on?openbugs time
  29. “bugs show lack of quality not itspresence” Gojko Adzicremoving what you don’t want (i.e. bugs)does not imply getting what you do want
  30. some hintsmeasure what scares youmeasure outcome not much value created vs how many stories donedefects trending down vs how many unit tests created
  31. but stillorganizations want to measure individuals=> people game metrics⇒ “if you do X I will give you Y”reduces motivation1)  no autonomy: feeling controlled by who gives rewards2)  being payed for doing something: may imply it’s not worth doing for its own sake
  32. Hawthorne Effect
  33. how to avoid this?no bonus, appraisals, etc.shared goals (let the team find how to reach them)visualize feedback as informational and not controllinggive unexpected rewardsif you really have, let the team do individual appraisals
  34. metricsquadrants
  35. inward & outwardlooking metrics inward outward looking loooking feedback Business & R&D Other Stakeholders boundary objects
  36. boundary objects R&D business metricboundary object [sociology]: something that helpsdifferent communities exchange ideas and information.could mean different things to different peoplebut allows coordination and alignment
  37. metrics quadrants Business outward looking & feedback ProductProcess inward looking Team Maturity
  38. metrics quadrants Business boundary objects ProductProcess Team Maturity
  39. metrics quadrants Business boundary objects ProductProcess agile fragile Team Maturity
  40. metrics quadrants Business ProductProcess Team Maturity
  41. metrics quadrants Business Lead  Time   Revenues   Cycle  Time   Quality  of  Service  (SLA)   ROI   Customer  SaHsfacHon   Throughput     Business  Value   ProductProcess Bugs?   WIP   Cadence   CI  Failures   Code  Quality   Rework   Technical  Debt   Impediments   Test  Coverage   RetrospecHves   Morale   Team Maturity
  42. metrics quadrants Business what!? no Lead  Time   Revenues  velocity? Cycle  Time   Quality  of  Service  (SLA)   ROI   Customer  SaHsfacHon   Throughput     Business  Value   Product Process Bugs?   WIP   Cadence   CI  Failures   Code  Quality   Rework   Technical  Debt   Impediments   Test  Coverage   RetrospecHves   Morale   Team Maturity
  43. fragility code quality reduce technical debtlack of advanced engineering practices(i.e. TDD, CI) => rework
  44. code quality evolution
  45. code quality evolution
  46. agilitybeing agile is not the goal,it’s a meanif you are really interested there areplenty of agility tests on the Internet:Nokia TestScrum Open Assessment - ScrumAllianceAgile Maturity ModelAgile Evaluation FrameworkComparative Agility Assessmentetc.
  47. impediments, retrospectives, reviews# of questions answered ?# of questions asked# action items addressed# action items assigned (at previous meetings)# of WTFs WTF!? WTF!?
  48. metricsqueues
  49. queues are badincrease reducecycle time quality risk motivationvariability overhead stop starting start finishing
  50. cumulative flow diagram arrivals queue sizecumulative (WIP) quantity time in queue departures (cycle time) (throughput) time source: Donald Reinertsen
  51. cumulative flow diagram WIP is a leading indicator cycle timecumulative WIP quantity time
  52. cumulative flow diagram large batches large queuescumulative quantity time
  53. cumulative flow diagram small batches small queuescumulative quantity time
  54. Kanban board if you can’t see it you can’t manage itbacklog to do in progress done 2 3 WIP cycle time = throughput cycle time
  55. no WIP limit -> queue!backlog   to do   ready   in progress   done   2   3  
  56. slack (%) optimize flow absorb variation
  57. flow related metricsactive WIP - buffered WIPtasks that are really in progress – taskwaiting to be handed-off (#,%,% of timespent)process efficiencyactive time / cycle timetechnical debt WIP / standard WIP# of projects a person works in parallel(should be 1!!!)
  58. visualizing tasks dynamicsbacklog to do in progress done 2 41   2   3   4   days inactive task
  59. cumulative flow diagram 35   not so helpful? 30   25   backlog Backlog   20  # user stories to do To  Do   In  Progress   15   in progress cycle time Done   WIP 10   throughput done 5   0   time
  60. single column dynamics In  Progress   6   5   4   3   WIP   2   1   0   4   1   2   3   4   5   6   7   8   9   10   11   12   13   14   15   16   17   18   19   20   3   2   1   in   0   1   2   3   4   5   6   7   8   9   10   11   12   13   14   15   16   17   18   19   out  -­‐1  -­‐2  -­‐3  -­‐4  
  61. Kanban board dynamics
  62. control charts source: Samuli Heljo
  63. metricseasy but powerful 42
  64. Happiness Indexleading or lagging? feedback board Mon Tue Wed Thu FriTom J K J L JAnne J J J J JPaul L J K J JJoe J J J J KEva J J J L J niko-niko calendar
  65. Pizza IndexPizza = Overtime => not good Steve Denning
  66. how long since? you talked to a customer last useful retrospective you learned something at work your boss last freaked out last critical bug6weeks 2 days 3 days 1 52 week days
  67. and don’t forgetbus factor# of key developers that need to be hit by abus to kill a project
  68. “for every true one “per una verathousands are fake” mille sono finte” F. De André
  69. Gaetano Mazzanti Gama-Tech