Your SlideShare is downloading. ×
Metrics As A Learn And Change Agent
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Metrics As A Learn And Change Agent


Published on

my presentation at AgileIndia 2012

my presentation at AgileIndia 2012

Published in: Technology, Business

1 Comment
No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. metrics as alearn and change agent Gaetano Mazzanti @mgaewsj Gama-Tech
  • 2. as an agile coach I am often asked todefine/introduce KPIs “why?”
  • 3. Case 1 “uhm, ehm, weneed to measure code & fixif and how muchwe are improving” no process chaos
  • 4. *typicallyfor performance appraisals Case 2 “we need to measure traditional if and how much we are improving;rigid process top-down company policies enforce using KPIs*”
  • 5. unfortunately things arenot so simple and linear
  • 6. systems: some definitions system any network that has coherence, it may be fuzzy, it may have purpose agent anything which acts within the system: individual, group, idea, etc.ordered complex chaotic
  • 7. complex system lightly constrains agents agents modify system by their interaction with it and each other, they co-evolveorderedsystem constrains agents chaotic agents underconstrained & independent of each otherordered complex chaotic
  • 8. no metrics code & fix no process chaosordered complex chaotic
  • 9. metrics for a linear,deterministic world no metrics traditional code & fix rigid process top-down no process chaos ordered complex chaotic
  • 10. ??? agilemetrics for a linear,deterministic world no metrics traditional code & fix rigid process top-down no process chaos ordered complex chaotic
  • 11. complex systems exhibit one or more properties (behavior) not obvious from the properties of the individual interconnected parts they are composed of
  • 12. product development is complex“self-organizing, non-linear, feedback systems are inherently unpredictable, they are not controllable“ D.Meadows
  • 13. so what are wesupposed to do?
  • 14. we canwatch, learn and work with the system
  • 15. metricslearn & change
  • 16. how?
  • 17. loop!
  • 18. PDCA Loop planact do check Shewhart & Deming
  • 19. OODA Loopobserve orient act decide J.Boyd
  • 20. LeanStartup Loop Ideas  Learn   Build   Data   Product   are we building the right product? Measure   E.Ries
  • 21. when loopingminimize total time through the looplearn fastfail fast! “biggest failure is failure to learn” (M.Poppendieck)
  • 22. single loop learning lead to actions results how which shape future efficiency doing things right incremental change
  • 23. single loop drawbacks uncertainty emerging information unexpected impediments delays plan defects more detailed planning rigid change management overburden (waste)
  • 24. double loop learning Chris Argyris guide values, actions results assumptions how whystepoutsidethesystem! lead to new/improvedeffectiveness efficiencydoing the right things doing things rightquestion assumptions incremental changeradical change
  • 25. timebound metrics learn, change, move on define metric* set expiration date goal ok or values, expiration actions results assumptions date passed?*actionable, accessible, auditable, time-bound
  • 26. do not focus on the metric itself, it is just a mean to understand/learn and changemeasures have value only if they inform decisions or motivate action
  • 27. be carefulhow do you know that the results you are seeing are related to the changes you have made?how do you know you are learning correctly from those changes?
  • 28. what’s going on?openbugs time
  • 29. “bugs show lack of quality not itspresence” Gojko Adzicremoving what you don’t want (i.e. bugs)does not imply getting what you do want
  • 30. some hintsmeasure what scares youmeasure outcome not much value created vs how many stories donedefects trending down vs how many unit tests created
  • 31. but stillorganizations want to measure individuals=> people game metrics⇒ “if you do X I will give you Y”reduces motivation1)  no autonomy: feeling controlled by who gives rewards2)  being payed for doing something: may imply it’s not worth doing for its own sake
  • 32. Hawthorne Effect
  • 33. how to avoid this?no bonus, appraisals, etc.shared goals (let the team find how to reach them)visualize feedback as informational and not controllinggive unexpected rewardsif you really have, let the team do individual appraisals
  • 34. metricsquadrants
  • 35. inward & outwardlooking metrics inward outward looking loooking feedback Business & R&D Other Stakeholders boundary objects
  • 36. boundary objects R&D business metricboundary object [sociology]: something that helpsdifferent communities exchange ideas and information.could mean different things to different peoplebut allows coordination and alignment
  • 37. metrics quadrants Business outward looking & feedback ProductProcess inward looking Team Maturity
  • 38. metrics quadrants Business boundary objects ProductProcess Team Maturity
  • 39. metrics quadrants Business boundary objects ProductProcess agile fragile Team Maturity
  • 40. metrics quadrants Business ProductProcess Team Maturity
  • 41. metrics quadrants Business Lead  Time   Revenues   Cycle  Time   Quality  of  Service  (SLA)   ROI   Customer  SaHsfacHon   Throughput     Business  Value   ProductProcess Bugs?   WIP   Cadence   CI  Failures   Code  Quality   Rework   Technical  Debt   Impediments   Test  Coverage   RetrospecHves   Morale   Team Maturity
  • 42. metrics quadrants Business what!? no Lead  Time   Revenues  velocity? Cycle  Time   Quality  of  Service  (SLA)   ROI   Customer  SaHsfacHon   Throughput     Business  Value   Product Process Bugs?   WIP   Cadence   CI  Failures   Code  Quality   Rework   Technical  Debt   Impediments   Test  Coverage   RetrospecHves   Morale   Team Maturity
  • 43. fragility code quality reduce technical debtlack of advanced engineering practices(i.e. TDD, CI) => rework
  • 44. code quality evolution
  • 45. code quality evolution
  • 46. agilitybeing agile is not the goal,it’s a meanif you are really interested there areplenty of agility tests on the Internet:Nokia TestScrum Open Assessment - ScrumAllianceAgile Maturity ModelAgile Evaluation FrameworkComparative Agility Assessmentetc.
  • 47. impediments, retrospectives, reviews# of questions answered ?# of questions asked# action items addressed# action items assigned (at previous meetings)# of WTFs WTF!? WTF!?
  • 48. metricsqueues
  • 49. queues are badincrease reducecycle time quality risk motivationvariability overhead stop starting start finishing
  • 50. cumulative flow diagram arrivals queue sizecumulative (WIP) quantity time in queue departures (cycle time) (throughput) time source: Donald Reinertsen
  • 51. cumulative flow diagram WIP is a leading indicator cycle timecumulative WIP quantity time
  • 52. cumulative flow diagram large batches large queuescumulative quantity time
  • 53. cumulative flow diagram small batches small queuescumulative quantity time
  • 54. Kanban board if you can’t see it you can’t manage itbacklog to do in progress done 2 3 WIP cycle time = throughput cycle time
  • 55. no WIP limit -> queue!backlog   to do   ready   in progress   done   2   3  
  • 56. slack (%) optimize flow absorb variation
  • 57. flow related metricsactive WIP - buffered WIPtasks that are really in progress – taskwaiting to be handed-off (#,%,% of timespent)process efficiencyactive time / cycle timetechnical debt WIP / standard WIP# of projects a person works in parallel(should be 1!!!)
  • 58. visualizing tasks dynamicsbacklog to do in progress done 2 41   2   3   4   days inactive task
  • 59. cumulative flow diagram 35   not so helpful? 30   25   backlog Backlog   20  # user stories to do To  Do   In  Progress   15   in progress cycle time Done   WIP 10   throughput done 5   0   time
  • 60. single column dynamics In  Progress   6   5   4   3   WIP   2   1   0   4   1   2   3   4   5   6   7   8   9   10   11   12   13   14   15   16   17   18   19   20   3   2   1   in   0   1   2   3   4   5   6   7   8   9   10   11   12   13   14   15   16   17   18   19   out  -­‐1  -­‐2  -­‐3  -­‐4  
  • 61. Kanban board dynamics
  • 62. control charts source: Samuli Heljo
  • 63. metricseasy but powerful 42
  • 64. Happiness Indexleading or lagging? feedback board Mon Tue Wed Thu FriTom J K J L JAnne J J J J JPaul L J K J JJoe J J J J KEva J J J L J niko-niko calendar
  • 65. Pizza IndexPizza = Overtime => not good Steve Denning
  • 66. how long since? you talked to a customer last useful retrospective you learned something at work your boss last freaked out last critical bug6weeks 2 days 3 days 1 52 week days
  • 67. and don’t forgetbus factor# of key developers that need to be hit by abus to kill a project
  • 68. “for every true one “per una verathousands are fake” mille sono finte” F. De André
  • 69. Gaetano Mazzanti Gama-Tech