metrics            as alearn and change           agent    Gaetano Mazzanti            @mgaewsj             Gama-Tech
as an agile coach I am often asked todefine/introduce KPIs      “why?”
Case 1  “uhm, ehm, weneed to measure      code & fixif and how muchwe are improving”   no process                       ch...
*typicallyfor                          performance                          appraisals                      Case 2        ...
unfortunately things arenot so simple and linear
systems: some definitions                      system          any network that has coherence,    it may be fuzzy, it may ...
complex     system lightly constrains agents    agents modify system by their interaction     with it and each other, they...
no metrics                     code & fix                    no process                       chaosordered   complex      ...
metrics for a linear,deterministic world                                       no metrics      traditional                ...
???                           agilemetrics for a linear,deterministic world                                             no...
complex systems       exhibit one or more     properties (behavior)      not obvious from the             properties of   ...
product development is        complex“self-organizing, non-linear,    feedback systems are  inherently unpredictable, they...
so what are wesupposed to do?
we canwatch, learn and work   with the system
metricslearn & change
how?
loop!
PDCA Loop         planact               do        check                       Shewhart & Deming
OODA Loopobserve   orient  act     decide                   J.Boyd
LeanStartup Loop                Ideas	  Learn	                     Build	   Data	                     Product	     are we ...
when loopingminimize total time through the looplearn fastfail fast! “biggest failure is failure to learn” (M.Poppendieck)
single loop learning                 lead to       actions             results         how         which shape future     ...
single loop drawbacks      uncertainty  emerging information unexpected impediments                   delays   plan       ...
double loop learning                                              Chris Argyris                  guide      values,       ...
timebound metrics      learn, change, move on               define metric*               set expiration date              ...
do not focus on the metric itself, it  is just a mean to understand/learn  and changemeasures have value only if they  inf...
be carefulhow do you know that the results you are seeing are related to the changes you have made?how do you know you are...
what’s going on?openbugs            time
“bugs show lack of quality not itspresence” Gojko Adzicremoving what you don’t want (i.e. bugs)does not imply getting what...
some hintsmeasure what scares youmeasure outcome not outputi.e.how much value created vs how many stories donedefects tren...
but stillorganizations want to measure individuals=> people game metrics⇒ “if you do X I will give you Y”reduces motivatio...
Hawthorne Effect
how to avoid this?no bonus, appraisals, etc.shared goals (let the team find how to reach them)visualize feedback as inform...
metricsquadrants
inward & outwardlooking metrics inward               outward looking              loooking           feedback             ...
boundary objects           R&D                          business                       metricboundary object [sociology]: ...
metrics quadrants                 Business                outward                looking                   &              ...
metrics quadrants                 Business                       boundary                        objects                  ...
metrics quadrants                     Business                          boundary                           objects        ...
metrics quadrants                 Business                               ProductProcess               Team Maturity
metrics quadrants                                     Business                  Lead	  Time	                              ...
metrics quadrants                                         Business what!?   no                 Lead	  Time	               ...
fragility      code quality              reduce               technical debtlack of advanced engineering practices(i.e. TD...
code quality evolution
code quality evolution
agilitybeing agile is not the goal,it’s a meanif you are really interested there areplenty of agility tests on the Interne...
impediments, retrospectives, reviews# of questions answered                                ?# of questions asked# action i...
metricsqueues
queues are badincrease              reducecycle time           quality   risk             motivationvariability overhead  ...
cumulative flow diagram             arrivals                             queue sizecumulative                     (WIP) qu...
cumulative flow diagram      WIP is a leading indicator                  cycle timecumulative      WIP quantity           ...
cumulative flow diagram     large batches large queuescumulative quantity                time
cumulative flow diagram     small batches small queuescumulative quantity                time
Kanban board                                      if you can’t see it                                      you can’t manag...
no WIP limit -> queue!backlog	     to do	     ready	     in progress	     done	                  2	                       ...
slack (%)        optimize flow       absorb variation
flow related metricsactive WIP - buffered WIPtasks that are really in progress – taskwaiting to be handed-off (#,%,% of ti...
visualizing tasks dynamicsbacklog                         to do   in progress         done                                ...
cumulative flow diagram                 35	                                                        not so helpful?        ...
single column dynamics                                                              In	  Progress	       6	       5	      ...
Kanban board dynamics
control charts          source: Samuli Heljo
metricseasy but powerful         42
Happiness Indexleading or lagging?                                        feedback board       Mon   Tue      Wed   Thu   ...
Pizza IndexPizza = Overtime => not good        Steve Denning
how long since? you talked to a customer last useful retrospective you learned something at work your boss last freaked ou...
and don’t forgetbus factor# of key developers that need to be hit by abus to kill a project
“for every true one   “per una verathousands are fake”   mille sono finte”                      F. De André
Gaetano Mazzanti  Gama-Tech     @mgaewsjinfo@gama-tech.net
Metrics As A Learn And Change Agent
Upcoming SlideShare
Loading in …5
×

Metrics As A Learn And Change Agent

47,849 views

Published on

my presentation at AgileIndia 2012

Published in: Technology, Business
1 Comment
25 Likes
Statistics
Notes
No Downloads
Views
Total views
47,849
On SlideShare
0
From Embeds
0
Number of Embeds
162
Actions
Shares
0
Downloads
249
Comments
1
Likes
25
Embeds 0
No embeds

No notes for slide

Metrics As A Learn And Change Agent

  1. metrics as alearn and change agent Gaetano Mazzanti @mgaewsj Gama-Tech
  2. as an agile coach I am often asked todefine/introduce KPIs “why?”
  3. Case 1 “uhm, ehm, weneed to measure code & fixif and how muchwe are improving” no process chaos
  4. *typicallyfor performance appraisals Case 2 “we need to measure traditional if and how much we are improving;rigid process top-down company policies enforce using KPIs*”
  5. unfortunately things arenot so simple and linear
  6. systems: some definitions system any network that has coherence, it may be fuzzy, it may have purpose agent anything which acts within the system: individual, group, idea, etc.ordered complex chaotic
  7. complex system lightly constrains agents agents modify system by their interaction with it and each other, they co-evolveorderedsystem constrains agents chaotic agents underconstrained & independent of each otherordered complex chaotic
  8. no metrics code & fix no process chaosordered complex chaotic
  9. metrics for a linear,deterministic world no metrics traditional code & fix rigid process top-down no process chaos ordered complex chaotic
  10. ??? agilemetrics for a linear,deterministic world no metrics traditional code & fix rigid process top-down no process chaos ordered complex chaotic
  11. complex systems exhibit one or more properties (behavior) not obvious from the properties of the individual interconnected parts they are composed of
  12. product development is complex“self-organizing, non-linear, feedback systems are inherently unpredictable, they are not controllable“ D.Meadows
  13. so what are wesupposed to do?
  14. we canwatch, learn and work with the system
  15. metricslearn & change
  16. how?
  17. loop!
  18. PDCA Loop planact do check Shewhart & Deming
  19. OODA Loopobserve orient act decide J.Boyd
  20. LeanStartup Loop Ideas  Learn   Build   Data   Product   are we building the right product? Measure   E.Ries
  21. when loopingminimize total time through the looplearn fastfail fast! “biggest failure is failure to learn” (M.Poppendieck)
  22. single loop learning lead to actions results how which shape future efficiency doing things right incremental change
  23. single loop drawbacks uncertainty emerging information unexpected impediments delays plan defects more detailed planning rigid change management overburden (waste)
  24. double loop learning Chris Argyris guide values, actions results assumptions how whystepoutsidethesystem! lead to new/improvedeffectiveness efficiencydoing the right things doing things rightquestion assumptions incremental changeradical change
  25. timebound metrics learn, change, move on define metric* set expiration date goal ok or values, expiration actions results assumptions date passed?*actionable, accessible, auditable, time-bound
  26. do not focus on the metric itself, it is just a mean to understand/learn and changemeasures have value only if they inform decisions or motivate action
  27. be carefulhow do you know that the results you are seeing are related to the changes you have made?how do you know you are learning correctly from those changes?
  28. what’s going on?openbugs time
  29. “bugs show lack of quality not itspresence” Gojko Adzicremoving what you don’t want (i.e. bugs)does not imply getting what you do want
  30. some hintsmeasure what scares youmeasure outcome not outputi.e.how much value created vs how many stories donedefects trending down vs how many unit tests created
  31. but stillorganizations want to measure individuals=> people game metrics⇒ “if you do X I will give you Y”reduces motivation1)  no autonomy: feeling controlled by who gives rewards2)  being payed for doing something: may imply it’s not worth doing for its own sake
  32. Hawthorne Effect
  33. how to avoid this?no bonus, appraisals, etc.shared goals (let the team find how to reach them)visualize feedback as informational and not controllinggive unexpected rewardsif you really have, let the team do individual appraisals
  34. metricsquadrants
  35. inward & outwardlooking metrics inward outward looking loooking feedback Business & R&D Other Stakeholders boundary objects
  36. boundary objects R&D business metricboundary object [sociology]: something that helpsdifferent communities exchange ideas and information.could mean different things to different peoplebut allows coordination and alignment
  37. metrics quadrants Business outward looking & feedback ProductProcess inward looking Team Maturity
  38. metrics quadrants Business boundary objects ProductProcess Team Maturity
  39. metrics quadrants Business boundary objects ProductProcess agile fragile Team Maturity
  40. metrics quadrants Business ProductProcess Team Maturity
  41. metrics quadrants Business Lead  Time   Revenues   Cycle  Time   Quality  of  Service  (SLA)   ROI   Customer  SaHsfacHon   Throughput     Business  Value   ProductProcess Bugs?   WIP   Cadence   CI  Failures   Code  Quality   Rework   Technical  Debt   Impediments   Test  Coverage   RetrospecHves   Morale   Team Maturity
  42. metrics quadrants Business what!? no Lead  Time   Revenues  velocity? Cycle  Time   Quality  of  Service  (SLA)   ROI   Customer  SaHsfacHon   Throughput     Business  Value   Product Process Bugs?   WIP   Cadence   CI  Failures   Code  Quality   Rework   Technical  Debt   Impediments   Test  Coverage   RetrospecHves   Morale   Team Maturity
  43. fragility code quality reduce technical debtlack of advanced engineering practices(i.e. TDD, CI) => rework
  44. code quality evolution
  45. code quality evolution
  46. agilitybeing agile is not the goal,it’s a meanif you are really interested there areplenty of agility tests on the Internet:Nokia TestScrum Open Assessment - ScrumAllianceAgile Maturity ModelAgile Evaluation FrameworkComparative Agility Assessmentetc.
  47. impediments, retrospectives, reviews# of questions answered ?# of questions asked# action items addressed# action items assigned (at previous meetings)# of WTFs WTF!? WTF!?
  48. metricsqueues
  49. queues are badincrease reducecycle time quality risk motivationvariability overhead stop starting start finishing
  50. cumulative flow diagram arrivals queue sizecumulative (WIP) quantity time in queue departures (cycle time) (throughput) time source: Donald Reinertsen
  51. cumulative flow diagram WIP is a leading indicator cycle timecumulative WIP quantity time
  52. cumulative flow diagram large batches large queuescumulative quantity time
  53. cumulative flow diagram small batches small queuescumulative quantity time
  54. Kanban board if you can’t see it you can’t manage itbacklog to do in progress done 2 3 WIP cycle time = throughput cycle time
  55. no WIP limit -> queue!backlog   to do   ready   in progress   done   2   3  
  56. slack (%) optimize flow absorb variation
  57. flow related metricsactive WIP - buffered WIPtasks that are really in progress – taskwaiting to be handed-off (#,%,% of timespent)process efficiencyactive time / cycle timetechnical debt WIP / standard WIP# of projects a person works in parallel(should be 1!!!)
  58. visualizing tasks dynamicsbacklog to do in progress done 2 41   2   3   4   days inactive task
  59. cumulative flow diagram 35   not so helpful? 30   25   backlog Backlog   20  # user stories to do To  Do   In  Progress   15   in progress cycle time Done   WIP 10   throughput done 5   0   time
  60. single column dynamics In  Progress   6   5   4   3   WIP   2   1   0   4   1   2   3   4   5   6   7   8   9   10   11   12   13   14   15   16   17   18   19   20   3   2   1   in   0   1   2   3   4   5   6   7   8   9   10   11   12   13   14   15   16   17   18   19   out  -­‐1  -­‐2  -­‐3  -­‐4  
  61. Kanban board dynamics
  62. control charts source: Samuli Heljo
  63. metricseasy but powerful 42
  64. Happiness Indexleading or lagging? feedback board Mon Tue Wed Thu FriTom J K J L JAnne J J J J JPaul L J K J JJoe J J J J KEva J J J L J niko-niko calendar
  65. Pizza IndexPizza = Overtime => not good Steve Denning
  66. how long since? you talked to a customer last useful retrospective you learned something at work your boss last freaked out last critical bug6weeks 2 days 3 days 1 52 week days
  67. and don’t forgetbus factor# of key developers that need to be hit by abus to kill a project
  68. “for every true one “per una verathousands are fake” mille sono finte” F. De André
  69. Gaetano Mazzanti Gama-Tech @mgaewsjinfo@gama-tech.net

×