C prime webinar-ppt-validating agile


Published on

cPrime's latest Agile Meetup discussion will center around methods for how to monitor and validate the performance of agile.

Many firms that have been doing agile can not determine how or if it has had an impact on the company. Agile expert, Jeff Howey will discuss ways to evaluate agile performance. Join our webinar to learn how to identify the benefits of agile and uncover the differences between companies that exhibit "agile-like" behavior and highly functioning teams.

Published in: Business, Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

C prime webinar-ppt-validating agile

  1. 1. Agile  Performance  Metrics   Presenter:  Jeff  Howey  –  Agile  Coach,  CSM   4100  E.  Third  Ave,  Suite  205,  Foster  City,  CA  94404    |    650-­‐931-­‐1651    |    www.cprime.com  The  leader  in  training  and  consulGng  for  project  management  and  agile  development  
  2. 2. ©  2012,  cPrime.  All  Rights  Reserved  
  3. 3. !   If  you  have  a  quesGon,  please  type  it  in  the   “QuesGons”  Chat  Window.    Our  team  will  collect   quesGons  and  include  responses  to  many  of   them  when  we  post  the  material  online.  !   We  will  post  a  copy  of  the  presentaGon,  a   recording  of  the  webinar  and  follow-­‐up   informaGon  on  www.cprime.com     !   We  will  send  email  with  links     !   Feel  free  to  share!     ©  2012,  cPrime.  All  Rights  Reserved  
  4. 4. !   What  is  the  current  adopGon  of  Agile  processes   in  your  organizaGon?   !   PracGcing  Agile  processes  on  most/all  projects   !   PiloGng  Agile  on  select  projects   !   Considering  Agile  for  future  projects   !   Not  considering  Agile  in  the  foreseeable  future   ©  2012,  cPrime.  All  Rights  Reserved  
  5. 5. And  What  Metrics  Will  Demonstrate  Impact?   ©  2012,  cPrime.  All  Rights  Reserved  
  6. 6. !   Don’t  we  value  working  so/ware  over   comprehensive  documenta6on?  !   Why  do  I  need  tools  and  processes  if  our  focus  is   on  individuals  and  interac6ons?  !   Is  management  measuring  my  performance   based  on  how  many  story  points  I  complete   when  my  goal  is  to  increase  customer   sa6sfac6on?  !   Do  my  stakeholders  really  care  about  numbers   as  long  as  I  am  responsive  to  my  customer’s   requests?   ©  2012,  cPrime.  All  Rights  Reserved   -­‐  Excerpts  from  the  Agile  Manifesto  
  7. 7. !   The  OrganizaGon  as  a  whole  needs  to   understand  how  and  why  Agile  is  making  a   difference  in  project  delivery  !   The  Agile  Team  needs  insight  into  areas  they  can   improve  !   Agile  sGll  values  processes,  tools  and   documentaGon!  !   No  more  “fake  it,  ‘Gl  you  make  it”  jusGficaGons   !   You  need  to  “move  it,  ‘Gl  you  prove  it”   !   And  keep  proving  it   ©  2012,  cPrime.  All  Rights  Reserved  
  8. 8. 2010 Project Success 21% Fail 37% Success 42% Standish Chaos Reports Challenge Project Success 60   % 50,000 Projects 50   40  Challenged  •   late  (100%  median)   30  •   over  budget  (50%  median)   20  •   lacking  funcGonality  •   low  quality   10   0   1994   1996   1998   2000   2004   2006   2009   2010   %  Success   16   27   26   28   29   35   32   37   %  Challenged   53   33   46   49   53   46   44   42   %  Failed   31   40   28   23   18   19   24   21  Data from 50,000 projectsCopyright © by The Standish Group International, Inc. ©  2012,  cPrime.  All  Rights  Reserved  
  9. 9. !   How  would  you  rate  your  team’s  history  of   delivering  projects?   !   More  successes  than  failures   !   More  failures  than  successes   !   About  the  same  as  Standish  reports   !   I’d  rather  not  say   ©  2012,  cPrime.  All  Rights  Reserved  
  10. 10. Feature  Usage  by  Frequency   How  much   did  this   cost???   16%   Always     45%   64%   Ohen   19%  13%   SomeGmes   Rarely   7%   Never   64% of product features are rarely or never used 2006  –  The  Standish  Group   ©  2012,  cPrime.  All  Rights  Reserved  
  11. 11. •  The  average  Fortune  500  sohware  project  costs  $3.2  million  and   takes  1,290  (resource)  months  to  complete.  Yet  only  37%  of   respondents  answered  "yes"  when  asked  if  their  projects  met   users  needs   •    2008  –  Voke,  Inc.   ©  2012,  cPrime.  All  Rights  Reserved  
  12. 12. !   “In  2002,  agile  projects  made  up  less  than  2%  of  overall   projects  and  less  than  5%  of  new  applicaGon   development  projects.  Today,  agile  projects  account  for   almost  9%  of  all  projects  and  29%  of  new  applicaGon   development  projects...     The  increase  in  project  success  rates  can   directly  @e  back  to  projects  resolved  through   the  agile  process.”   2011  Chaos  Manifesto     Copyright © by The Standish Group International, Inc   ©  2012,  cPrime.  All  Rights  Reserved  
  13. 13. !   Predictability   !   Delivery  Date   !   Delivery  Content  !   Reliability   !   Of  EsGmates   !   Of  Requirements  !   Quality   !   IniGal  Quality   !   Cost  of  Defect  ResoluGon  !   Customer  SaGsfacGon   ©  2012,  cPrime.  All  Rights  Reserved  
  14. 14. !   Understand  History   !   What  is  your  personal  best/worst/average  in  each   measure?  !   Set  a  Baseline  Target   !   At  least  aim  to  improve  1-­‐3  measures  in  the  next   3-­‐6  months  without  hurGng  any  others  metrics   !   Avoid  serng  aim  to  improve  everything   immediately  !   Set  Future  Targets   !   ConGnue  improvement  of  key  measures   !   Target  to  improve  some  others   ©  2012,  cPrime.  All  Rights  Reserved  
  15. 15. ©  2012,  cPrime.  All  Rights  Reserved  
  16. 16. !   How  ohen  do  you  make  major  releases?   !   Monthly  or  more  frequently   !   Quarterly   !   Every  6-­‐12  months   !   Once  a  year  or  less  frequently   ©  2012,  cPrime.  All  Rights  Reserved  
  17. 17. !   Somewhat  related  to  Reliability  of  EsGmates  but   disGnctly  different   !   Target  dates  should  be  met  without  “heroics”  of  the  team  !   Baseline  from  History   !   What  %  of  projects  deliver  within  a  range  +/-­‐  the  iniGal   date?   !   By  how  much  are  projects  delivered  late  when  they  are  not   within  a  given  range?  !   Set  Goals!   !   Somewhere  between  “best”  and  “average”  history   !   Definitely  beter  than  “worst”   ©  2012,  cPrime.  All  Rights  Reserved  
  18. 18. Project  ID   Delivery  Plan  Date   Actual  Delivery   Delta  from  Plan   Total  Days  in   Date   Project   1   3/31/2009   4/30/2009   +30  days   180   2   12/31/2009   12/31/2009   0  days   180   3   3/31/2010   7/31/2010   +210  days   480   4   3/31/2010   5/31/2010   +60  days   60   5   6/30/2011   7/15/2011   +15  days   180   6   6/30/2011   12/15/2011   +165  days   480   7   1/31/2012   4/30/2012  (est)   +90  days   90*  Average  Variance  of  Release  Date  from  Plan    !   Mean:    35%  late  !   Median:    33%  late   What  is  your  reality?   ©  2012,  cPrime.  All  Rights  Reserved  
  19. 19. !   First,  understand  WHY  target  dates  have  been   missed  in  the  past   !   What  can  Agile  address?    Uncertainty  or  Changing   Scope  in  Requirements,  unrealisGc  dates  for  full   scope  compleGon,  etc.   !   What  can  Agile  not  solve?  Corporate  culture,  ritual-­‐ intensive  release  processes,  etc.  !   Brainstorm  soluGon  opGons  !   PrioriGze  opGons  to  pursue  !   Inspect,  Adapt  and  maintain  Transparency  while   implemenGng  soluGon  alternaGves   ©  2012,  cPrime.  All  Rights  Reserved  
  20. 20. !   Define  a  Target  %  of  Releases  that  make  the   date   !   Adjust  scope  based  on  priority  to  deliver   funcGonality  that  is  good  enough  ON  the  planned   date  !   Define  a  beta-­‐type  release  strategy  where   feasible   !   Especially  in  organizaGons  with  long/complex   release  procedures  for  producGon  availability   !   IdenGfy  where  can  you  release  a  beta  version  and   receive  valuable  feedback   ©  2012,  cPrime.  All  Rights  Reserved  
  21. 21. ©  2012,  cPrime.  All  Rights  Reserved  
  22. 22. !   Comparison  of   !   IniGal  Requirements  vs.  Delivered  Requirements:   What  %  of  funcGonality  was  removed  from  scope?   !   Were  undelivered  requirements  rescheduled  for  later   release?   !   Were  features  given  up  out  of  frustraGon?  !   Not  always  bad  to  deliver  less  than  planned   !   Were  features  not  needed?       !   Were  they  simplified  -­‐  but  sGll  “good  enough”  for  users?   !   If  it  was  actually  a  “savings”  or  diverted  focus  to   higher  prioriGes,  it  could  be  a  benefit   ©  2012,  cPrime.  All  Rights  Reserved  
  23. 23. !   How  many  implemented  stories  were  rejected   by  the  Product  Owner  at  a  Sprint  Review?  !   What  were  the  Drivers?   !   incomplete  requirements?  (Product  Owner  has   opportuniGes  to  improve)     !   incomplete  implementaGon?  (Agile  Team  has   opportuniGes  to  improve)   ©  2012,  cPrime.  All  Rights  Reserved  
  24. 24. !   Agile  Teams  -­‐  Set  goals  around  the  number  of   stories  that  are  rejected  from  the  Product   Owner  once  implemented     !   Is  the  right  number  for  you  80%?  95%?   ©  2012,  cPrime.  All  Rights  Reserved  
  25. 25. ©  2012,  cPrime.  All  Rights  Reserved  
  26. 26. !   Look  at  Story  Points  first   !   Aher  implementaGon,  perhaps  during   RetrospecGves,  replay  Planning  Poker®     !   “Knowing  what  you  know  now,  what  would  be  the   esGmate  for  this  story”   !   Measure  all  stories  (big  and  small)  to  keep  fair  track  items   !   Compare  and  Average  variances     !   Break  down  by  Small/Medium/Large   !   Is  the  team  beter  at  esGmaGng  in  one  area  vs.  others?   !   Adjust  the  esGmaGng  process  as  needed   ©  2012,  cPrime.  All  Rights  Reserved  
  27. 27. !   As  the  team  refines  their  EsGmaGng  Process,   conGnue  to  review  the  Product  Backlog  and  re-­‐ esGmate  upcoming  User  &  Technical  Stories   !   Review  the  “ Top  1/3”  of  the  Product  Backlog  if  Gme   allows  for  the  first  Release  or  first  5-­‐7  Sprints  of  a   new  project  or  new  team  composiGon   !   Reset  and  baseline  Story  Points,  conGnue  to  track   progress  toward  beter  esGmaGon  !   Keep  this  grooming  process  short!   !   PerfecGon  is  expensive  and  unatainable   !   Set  limits  (1  hour  per  Sprint  during  Story  Review   MeeGngs,  perhaps?)   ©  2012,  cPrime.  All  Rights  Reserved  
  28. 28. !   At  the  Sprint  Backlog  (hours)  level  vs.  the   Product  Backlog  level  (Story  Points)   !   IdenGfy  any  recurring  procedures  that  may  be   candidates  for  project  standards  and  assign  hours   esGmates  to  include  in  future  Sprints   !   Compare  Actual:EsGmated  hours  for  tasks  and   understand  variances   !   They  are  expected,  so  idenGfy  paterns  and  don’t  dwell   for  more  than  5  minutes  on  specific  reasons   !   For  instance,  “programming  stored  procedures  are   always  underesGmated”  or  “Joe  is  new  to  Java  so  takes   25%  more  Gme  than  Sally  does  at  coding  right  now”   !   Adjust  esGmaGng  process  to  account  for  paterns   ©  2012,  cPrime.  All  Rights  Reserved  
  29. 29. ©  2012,  cPrime.  All  Rights  Reserved  
  30. 30. !   What  %  of  Stories  are  prioriGzed  for  change  to   the  iniGal  implementaGon  in  the  following  3   Sprints?  Why?   !   Focus  criGcal  atenGon  on  Stories  that  are   prioriGzed  for  change  in  the  Sprint  immediately   following  iniGal  implementaGon  !   Product  Owners  –  This  really  is  one  of  your   biggest  measures   !   Does  this  indicate  you  need  to  do  more  research  or   refinement  of  Stories  prior  to  Sprint  Planning?   ©  2012,  cPrime.  All  Rights  Reserved  
  31. 31. ©  2012,  cPrime.  All  Rights  Reserved  
  32. 32. !   Just  kidding  –  0  defects  is  expensive  and  likely   indicates  shortcomings  in  the  tesGng  process!  !   IdenGfy  historical  number  of  defects  idenGfied   post-­‐release  by  priority  (P1,  P2,  etc.)  !   Set  goals  by  Release  to  drasGcally  reduce  the   number  of  defects  idenGfied  post-­‐release  !   Set  goals  and  standards  to  minimize  or   eliminate  high-­‐priority  defects  to  consider  a   Sprint  “done”     !   For  example,  the  Agile  Team  sets  a  standard  that   they  cannot  exit  a  Sprint  with  outstanding  P1’s   ©  2012,  cPrime.  All  Rights  Reserved  
  33. 33. ©  2012,  cPrime.  All  Rights  Reserved  
  34. 34. !   Track  at  an  Aggregate  Level  and  begin  to   IdenGfy  where  defects  are  being  resolved  or   (ideally)  prevented   !   Agile  Teams  &  Product  Owners  are  all  on  the  hook!   ©  2012,  cPrime.  All  Rights  Reserved  
  35. 35. ©  2012,  cPrime.  All  Rights  Reserved  
  36. 36. !   UlGmately  this  is  the  only  metric  that  used  to   compare  Agile  Teams     !   Story  Points  for  Blue  Team  <>  Story  Points  for  Green  Team   !   Velocity  for  Blue  Team  <>  Velocity  for  Green  Team   !   Defect  Rate  for  Blue  Team  <>  Defect  Rate  for  Green  Team  !   IdenGfy  your  customers  –  Internal  &  External  !   Track  what  is  important  to  Your  Customer!   !   Time  to  Market   !   Length  of  Visit  per  page   !   Click-­‐through  Rate  !   Ideally  track  and  monitor  per  Release   ©  2012,  cPrime.  All  Rights  Reserved  
  37. 37. !   Use  a  4-­‐point  Scale  vs.  5  or  10   !   Forces  a  “3”  or  “4”  RaGng  and  avoids  “Average”   !   4  keeps  it  a  simple  comparison  vs.  10   !   Words  like  “Awful,  Could  Be  Beter,  It  Works,  Great”   are  more  powerful  than  “Poor,  Fair,  Good,   Excellent”  !   Be  sure  to  capture  comments!   !   Allow  input  on  every  quesGon,  not  just  a  generic   feedback  text  entry  secGon   !   AcGvely  Solicit!    “ Tell  us  what  could  be  beter”  is   more  powerful  than  “Comments”     ©  2012,  cPrime.  All  Rights  Reserved  
  38. 38. !   Record  feedback  and  scores  for  future  reference  !   During  RetrospecGve  and  Story  Review   meeGngs,  review  the  feedback  and  keep  it  fresh  !   Adapt  funcGonality,  schedule  or  whatever   elements  are  important  to  the  Customer  and   given  as  feedback  !   Incorporate  “What  could  we  do  beter?”   feedback  into  features  or  processes   ©  2012,  cPrime.  All  Rights  Reserved  
  39. 39. ©  2012,  cPrime.  All  Rights  Reserved  
  40. 40. !   Defects  per  developer   !   Legacy  performance  evaluaGon  criteria   !   NOT  valid  with  Agile:  self-­‐organizing  teams  should   allow  individuals  to  learn  new  things  –  the  “test-­‐n-­‐ learn”  process  anGcipates  defects  !   Lines  of  Code/Story  Points  per  developer  or   team   !   My  5-­‐point  Story  is  NOT  your  5-­‐point  Story  (this  is   true  at  the  individual  and  team  level)  !   What  other  metrics  do  you  think  are  missing?       !   Compare  to  the  Agile  Manifesto  and  Principles  to   validate  their  use   ©  2012,  cPrime.  All  Rights  Reserved  
  41. 41. !   Once  goals  are  set  for  any  set  of  metrics,  these   steps  are  CRITICAL   !   ConGnue  to  measure  against  goals   !   Review  as  an  Agile  Team   !   Share  with  Product  Owners,  Management,   Customers  !   ConGnuously  Refine  goals    !   Do  not  tackle  everything  at  once!   !   PrioriGze  what  is  most  important  to  the  Agile  Team   and  the  Customer  to  address  first   !   Take  on  new  challenges  when  goals  are  met   ©  2012,  cPrime.  All  Rights  Reserved  
  42. 42. ©  2012,  cPrime.  All  Rights  Reserved  
  43. 43. !   How  much  money  is  6ed  up  in  projects  that  are   approved  but  delayed  or  “on  deck”  !   Can  we  prove  that  releasing  more  frequently   based  on  prioriGzed  features  helps  the   organizaGon  more  wisely  invest  in  Products  &   Projects?  !   This  is  tricky  to  measure  and  even  more  tricky  to   track,  but  it  is  CRITICAL   ©  2012,  cPrime.  All  Rights  Reserved  
  44. 44. !   Do  you  have  projects  that  are  approved,  but  are   overdue,  delayed  or  sirng  "on  deck"  without   resources  to  work  them?   !   Yes,  and  maybe  Agile  can  help  us  manage  our   investments  beter   !   Yes,  but  I  don’t  think  Agile  would  help   !   No,  we  only  allocate  budget  for  projects  when   resources  are  ready  to  go   !   None  of  the  above  quite  fit   ©  2012,  cPrime.  All  Rights  Reserved  
  45. 45. !   As  organizaGons  implement  Agile,  it  is  criGcal  to   track  the  $  invested  in  projects  and  the  speed  at   which  the  investments  are  realized  through   Product  Releases  !   Supports  buy-­‐in  and  evangelizaGon  of  Agile   benefits  where  Agile  makes  sense   ©  2012,  cPrime.  All  Rights  Reserved  
  46. 46. !   How  much  money  has  been  spent  on  the  64%  of   features  that  are  rarely  or  never  used?  Can  Agile   put  that  to  beter  use  in  the  future?  !   Do  you  track  usage  of  features?   !   Through  automaGon  in  the  applicaGon   !   Via  User  Survey  !   Product  Owners  need  this  informaGon  to   prioriGze  work.    IT/Project  Management  needs   this  to  understand  impact  on  Por•olio   ©  2012,  cPrime.  All  Rights  Reserved  
  47. 47. Create  Your  Own,  but  Here  are  Some  Ideas   ©  2012,  cPrime.  All  Rights  Reserved  
  48. 48. Metric   DescripIon   Target  *  Sprint  Predictability   FracGon  of  Sprints  in  which  all  planned  Stories   >  90%   are  completed  Deliverable   Number  of  Stories  rejected  by  Product  Owner   <  1  RejecGon  Rate   at  end  of  Sprint  Requirement   Number  of  Stories  rejected  in  Sprint  Planning   <  1  RejecGon  Rate   MeeGng  EsGmaGon   Accuracy  of  Sprint  Backlog  esGmate   ±20%  of  hours  in  Reliability   80%  of  Sprints  Tracking  Reliability   No.  of  Tasks  with  incorrect  Status  per  day   <  5%  Defect  Impact   Effort  /  Sprint  for  P0,  P1  ProducGon  bugs  (i.e.,   <  20%   more  effort  devoted  to  value  delivery)  Quality   P0,  P1  ProducGon  bugs,  per  Release   3X  reducGon  Cost  of  Defects   Cost  to  correct  ProducGon  bugs,  per  Release   3X  reducGon   *  Example  Goal:  “ The  Team  aims  to  achieve  these  aOer    3  Releases”   ©  2012,  cPrime.  All  Rights  Reserved  
  49. 49. Metric   DescripIon  Time  to  Market   Time  from  request  to  implementaGon  for  highest-­‐value   features  Rate  of  Value  Delivery   Value  delivered  per  quarter  /  release  Adaptability   How  well  /  frequently  do  we  change  course  successfully,   when  needed?  Predictability   Extent  to  which  we  deliver  planned  features  on  schedule   (excepGng  change  requests)  Customer  SaGsfacGon   How  well  are  we  meeGng  customer  needs?  Customer  InteracGon   How  well  do  we  involve  customers  in  defining  needs  and   gerng  feedback?  Internal  CollaboraGon   How  well  are  we  meeGng  internal  needs  of  various   departments?  (Support,  Sales,  etc.)   These  are  more  tricky    to  track  but  very  useful   ©  2012,  cPrime.  All  Rights  Reserved  
  50. 50. !   How  did  we  do?   !   I  learned  quite  a  few  things  that  I  can  use   !   A  good  reminder  of  the  basics  and  a  few  new  ideas   !   I’m  not  sure  how  I  would  apply  most  of  what  I   heard   !   Waste  of  Gme   ©  2012,  cPrime.  All  Rights  Reserved