Is Your Metrics Programme Out of Control
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Is Your Metrics Programme Out of Control

  • 225 views
Uploaded on

 

More in: Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
225
On Slideshare
217
From Embeds
8
Number of Embeds
2

Actions

Shares
Downloads
0
Comments
0
Likes
0

Embeds 8

http://www.linkedin.com 4
https://www.linkedin.com 4

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Ally  Gill  MBCS  CITP,  UKSMA  Conference  -­‐  Nov  8    2012  IS  YOUR  METRICS  PROGRAMME  OUT  OF  CONTROL?  
  • 2. “It is really just asbad technique to makea measurement moreaccurately than isnecessary as it is tomake it not accuratelyenough”Arthur  David  Ritchie  -­‐  ScienKfic  Method:  An  Inquiry  into  the  Character  and  Validity  of  Natural  Law  (1923)   2
  • 3. “Just because you canmeasure it, it doesn’tmean you should”Disgruntled  employees  the  world  over...   3
  • 4. OBJECTIVES  q Why  do  Metrics  Programmes  go  out  of  control?  q How  can  we  recognise  the  symptoms?  q What  can  we  do  to  cure  them?  q Can  we  prevent  the  problems  in  the  first  place?   4  
  • 5. IN  THE  BEGINNING   5
  • 6. IN  THE  BEGINNING  q An  execuKve  launches  an  Improvement  IniKaKve  using  the  next   Silver  Bullet  e.g.  CMMI  q A   guru  is  assigned  to  manage  the  measurement  programme  q CMMI  becomes  the  holy  grail  for  measurement  q Demands  for  data  go  viral  because  of  a  command  and  control   mentality  q Chaos  ensues   6   6  
  • 7. AND  THE  RESULT  OFTEN  IS...   7   7  
  • 8. THE  GURUS  HAVE  TAKEN  OVER  THE  ASYLUM   8
  • 9. THE  GURUS  HAVE  TAKEN  OVER  THE  ASYLUM  q  Management  fails  to  set  objecKves  against  which  data  should   be  collected  and  accept  what  the  metrics  gurus  tell  them  they   should  be  doing  q  Everything  and  anything  that  can  be  measured  is  measured,   regardless  of  how  useful  the  measures  are  q  Measurement  becomes  a  self  fulfilling  business  in  its  own  right   9  
  • 10. THE  GURUS  HAVE  TAKEN  OVER  THE  ASYLUM  q  Management  loses  sight  of  the  extent  of  the  demands  and  costs   of  the  data  collecKon  programme  q  Measurement  gurus  hide  behind  the  CMMI  but  misinterpret  it  q  New  measures  are  constantly  added  but  none  are  reKred  q  Data  is  collected  long  acer  anyone  remembers  why  it  was   collected  in  the  first  place   10  
  • 11. THE  GURUS  HAVE  TAKEN  OVER  THE  ASYLUM  q  Senior  Management  must  retain  control  over  the  programme   and  regularly  review  the  data  being  collected  against  the   business  objecKves  q  Measures  must  be  constantly  reviewed  in  respect  of  their  value   and  outdated  measures  reKred  q  Management  acKons  and  decisions  based  on  measures  must  be   taken  and  be  made  transparent  to  the  organisaKon     11  
  • 12. DEATH  BY  A  THOUSAND  DATA  REQUESTS   12
  • 13. DEATH  BY  A  THOUSAND  DATA  REQUESTS  q  Incessant  demands  are  made  to  record  and  report  data,  ocen   the  same  data  in  different  formats,  through  different  tools  and   to  different  people  q  Data  providers  report  the  data  they  expect  the  requestors  to   want  to  see   13  
  • 14. DEATH  BY  A  THOUSAND  DATA  REQUESTS  q  Project  managers  and  staff  find  increasingly  ingenious  ways  to   create   the  right  numbers  rather  than  focusing  on  delivery  q  So  much  data  is  being  collected,  consolidated  and  reported  that   no-­‐one  can  actually  find  Kme  to  analyse  the  informaKon  and   take  appropriate  acKons    q  Metrics  are  seen  as  a  unnecessary  evil  and  the  organisaKon   loses  any  value  from  its  metrics  programme   14  
  • 15. DEATH  BY  A  THOUSAND  DATA  REQUESTS  q  Automate  data  collecKon  methods  as  much  as  possible    q  Eliminate  redundancies  between  reporKng  systems  q  Review  the  cost  of  data  collecKon  systems  on  a  regular  basis   and  monitor  the  number  of  teams  collecKng  data  q  Listen  to  the  data  providers  and  act  accordingly   15  
  • 16. METRICS  FOR  METRICS  SAKE  
  • 17. METRICS  FOR  METRICS  SAKE  q  Management  make  decisions  based  on  percepKon  rather  than   facts    q  Different  programme/projects  reviews  focus  on  a  single  set  of   numbers  -­‐  usually  financial  and  schedule  based  -­‐  regardless  of   the  real  focus  of  the  review  board   17  
  • 18. METRICS  FOR  METRICS  SAKE  q  The  cost  of  collecKng,  storing  and  analysing  the  data  outweighs   any  financial  benefit  or  value  from  using  the  data  to  improve   the  business  q  The  feedback  loop  is  broken  and  those  dealing  with  the  metrics   data  fail  to  report  back  to  those  who  provide  it  q  The  governance  system  may  have  lost  it s  way  and  needs  to  be   brought  back  into  line   18  
  • 19. METRICS  FOR  METRICS  SAKE  q  If  you  don t  have  the  capacity  to  perform  proper  analysis  of  the   data  you  collect  -­‐  stop  collecKng  it  q  If  you  can t  explain  how  and  why  specific  measures  are   recorded  -­‐  then  stop  recording  them  q  Just  because  you ve  always  measured  something,  it  doesn t   mean  you  will  always  have  to  q  If  trends  are  stable  and  outliers  are  few  and  far  between  you   may  assume  that  a  process  is  stable  and  focus  on  something   else  q  Review  the  governance  process   19  
  • 20. GARBAGE  IN,  GARBAGE  OUT   20
  • 21. GARBAGE  IN,  GARBAGE  OUT    q  Data  providers  report  the  data  they  expect  the  requestors  to   want  to  see  q  Metrics  data  is  misinterpreted  differently  by  different  groups  of   people  q  Heads  of  departments  enter  into  compeKKon  with  each  other   and  set  arbitrary  targets  to  be  met   21  
  • 22. GARBAGE  IN,  GARBAGE  OUT    q  There  are  mad  flurries  of  PM/PMO  acKvity  when  monthly  status   reports  are  due  q  Data  from  last  month  looks  remarkably  similar  to  the  data  from   the  previous  6  months  q  The  numbers  are  always   spot-­‐on  q  Departments  always  show  small  period  on  period  improvement   in  their  reporKng   22  
  • 23. GARBAGE  IN,  GARBAGE  OUT    q  Check  different  types  of  data  -­‐  not  just  financial  and  schedule  -­‐   to  ensure  that  all  the  numbers  match  the  prevailing   environmental  condiKons  q  Stop  using  data  to  compare  unlike  systems  q  Use  data  to  improve  the  system  as  a  whole  not  just  the  units   within  the  system   23  
  • 24. KEEP  IN  CONTROL   24
  • 25. KEEP  IN  CONTROL  q  Management  need  to  manage  their  metrics  programme    before   it  starts  to  take  control  q  Only  collect  and  analyse  measures  which  will  add  value  to  the   business  q  Discard  metrics  which  no  longer  add  value  q  Don t  hide  behind  CMMI  to  suit  a  hidden  agenda  q  Don t  use  the  numbers  to  beat  up  your  people  q  Do  use  numbers  to  make  the  system  work  bejer   25  
  • 26. KEEP  IN  CONTROL  q  Watch  out  for  dysfuncKonal  behaviours  and  take  correcKve   acKon  as  early  as  possible  -­‐  start  with  yourself!  q  Data  must  be  treated  holisKcally  not  in  isolaKon  of  other   environmental  factors  q  Listen  to  staff  -­‐  if  people  are  complaining  about  the  volume  of   data  requests  they  may  well  have  a  valid  issue  q  Validate  metrics  by  other  management  techniques  like   management  by  walking  around     26  
  • 27. START  OFF  ON  THE  RIGHT  FOOT   27
  • 28. START  OFF  ON  THE  RIGHT  FOOT   q  MANAGEMENT    q  Align  your  metrics  to  your  business  goals  and  objecKves  q  Review  your  measurement  programme  on  a  regular  basis  q  Ensure  data  collected  in  used  to  improve  the  business   28  
  • 29. START  OFF  ON  THE  RIGHT  FOOT   q  METRICS  GURUS    q  Measures  are  not  the  core  business  they  are  enablers  to  bejer   business  q  Consider  all  stakeholder  viewpoints,  and  don t  assume  that   management  understand  what  you  do  any  more  than  anyone   else  q  If  staff  are  struggling  to  meet  your  data  demands,  there s   probably  a  very  good  reason  -­‐  find  out  what  it  is  and  fix  it  q  Don t  hide  behind  CMMI  and  insist  it s  a   CMMI  compliance   thing   29  
  • 30. START  OFF  ON  THE  RIGHT  FOOT   q  PROJECT  MANAGERS  &  TEAMS    q  Push  back  when  duplicate  or  redundant  data  demands  come   your  way  q  Don t  make  up  the  numbers  -­‐  find  a  bejer  way  to  deal  with  the   problem  q  If  you  don t  get  visibility  of  how  the  data  is  being  used  to   improve  your  daily  work  find  out  why   30  
  • 31. “...management bynumerical goal is anattempt to managewithout knowledge ofwhat to do, and infact is usuallymanagement by fear.”Chapter  2,  Page  76   31
  • 32. ? 32  
  • 33. CONTACT DETAILS" " email – ally@allygill.co.uk" web - www.allygill.co.uk" twitter - @allygill"Ally  Gill  MBCS  CITP,  UKSMA  Conference  -­‐  Nov  8    2012  IS  YOUR  METRICS  PROGRAMME  OUT  OF  CONTROL?   33