Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Agile Metrics and Quality
Metrics -> ImprovementsUse metrics to •   Improve quality •   Shorten product cycle time •   Decrease costs •   Decrease r...
Control Limits and VariancesWe can have limits for• WIP• Defects• Team size• Velocity• Time-boxes• Others?We can analyze v...
Focus on Trends• Trends matter more than values• Trends and spikes are only the symptoms• Trends are leading indicators   ...
Improve quality ◦ Quality is measured by passed tests and   customer acceptance ◦ Quality is defined by Definition of Done...
Defect Injection and     ResolutionWhat is the trend of total Active Defects?                                 Image from M...
Analyzing and PreventingPareto Chart           Defects                              Fishbone / 5 Why Technique
Escaped Defects• Used to measure the quality of delivered  code Calculation:                             9  ◦ For all rel...
Defect Cycle Time• Rapid resolution of defects is important• Average bug-fix time can also be tracked for  different prior...
Defect Spill-over• Resolution of defects for a sprint story within the sprint is  important• Definition of Done can includ...
Test Automation Critical factor for regression testing Coverage should be as high as possible
Shorten product cycle time   Cumulative Flow Diagrams    ◦ Lead Time    ◦ Cycle Time
Features   CFD – An Elaboration                 Time
What does this CFD say?                                          Not started                                          Star...
Identifying bottlenecks with a             CFD  1   2   3   4   5   6    7 8   8   10 11 12 13                          Da...
Decrease costs• Earned Value Management• Wastage
Earned Value AnalysisPV – Planned ValueEV - Earned ValueAC - Actual costBAC     - Budget @CompletionEAC     - Estimate @Co...
EVM - InterpretationPV   EV       AC   Schedule            Cost$1   $1       $1   On schedule         On cost$2   $2      ...
EVM - Progress EV                PV                    AC           CV      SV                Ref: Earned Value and Agile ...
Wastage          Any time not dedicated to working on the           highest priority story / task on the sprint          ...
Decrease risks• WIP Limits• Small Iterations• More predictability
Increase Predictability /                   Reliability•   Velocity•   Lead Time•   Cycle Time
Metrics to Avoid•   Velocity comparisons•   Productivity e.g. Story Points per day•   Effort Estimation accuracy•   Number...
Questions?ContactAnu.khendry@gmail.com
Upcoming SlideShare
Loading in …5
×

Agile metrics and quality

32,839 views

Published on

Agile metrics and quality

  1. 1. Agile Metrics and Quality
  2. 2. Metrics -> ImprovementsUse metrics to • Improve quality • Shorten product cycle time • Decrease costs • Decrease risks • Increase predictability and reliability
  3. 3. Control Limits and VariancesWe can have limits for• WIP• Defects• Team size• Velocity• Time-boxes• Others?We can analyze variances to understand the special causes of variation
  4. 4. Focus on Trends• Trends matter more than values• Trends and spikes are only the symptoms• Trends are leading indicators Images from MountainGoat Software
  5. 5. Improve quality ◦ Quality is measured by passed tests and customer acceptance ◦ Quality is defined by Definition of Done and Acceptance Criteria ◦ Defect trends to use in Agile projects  Number of open defects  Escaped defects  Defect Cycle Time  Defect spill-over ◦ Test Automation is critical!
  6. 6. Defect Injection and ResolutionWhat is the trend of total Active Defects? Image from MountainGoat Software
  7. 7. Analyzing and PreventingPareto Chart Defects Fishbone / 5 Why Technique
  8. 8. Escaped Defects• Used to measure the quality of delivered code Calculation: 9 ◦ For all releases, find 8 7 all defects related to 6 the release found 5 after the release date 4 3 ◦ Add up all the defects 2 1 ◦ Can be captured per 0 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec day / week / month / … or per sprint / Escaped Defects over Time release
  9. 9. Defect Cycle Time• Rapid resolution of defects is important• Average bug-fix time can also be tracked for different priority bugs Defect Cycle Time 20 18 16 Desired Threshold 14 12 Hours 10 8 6 4 2 0 1 2 3 4 5 6 7 8 9 10 Sprint No.
  10. 10. Defect Spill-over• Resolution of defects for a sprint story within the sprint is important• Definition of Done can include a criteria like “No P1 defects”• We can also tracked whether spilled-over defects are being closed in the next sprint Defect Spill Over 9 8 7 No of defects 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 10 Sprint No
  11. 11. Test Automation Critical factor for regression testing Coverage should be as high as possible
  12. 12. Shorten product cycle time Cumulative Flow Diagrams ◦ Lead Time ◦ Cycle Time
  13. 13. Features CFD – An Elaboration Time
  14. 14. What does this CFD say? Not started Started Completed Too much Work-In-Progress! Image from MountainGoat Software
  15. 15. Identifying bottlenecks with a CFD 1 2 3 4 5 6 7 8 8 10 11 12 13 Day Where is the bottleneck? In DB Procs – After the widening Analysis phase Image from http://www.soliantconsulting.com
  16. 16. Decrease costs• Earned Value Management• Wastage
  17. 17. Earned Value AnalysisPV – Planned ValueEV - Earned ValueAC - Actual costBAC - Budget @CompletionEAC - Estimate @CompletionETC - Estimate toCompletionCost Variance (CV) = EV – ACVAC- Variance @ Completion (SV) Schedule Variance = EV – PV Cost Performance Index (CPI) = EV / AC Schedule Performance Index (SPI) = EV / PV Estimate at Completion (EAC) = BAC / CPI or = AC + BAC – EV Estimate To Complete (ETC) = EAC – AC Variance at Completion (VAC) = BAC – EAC
  18. 18. EVM - InterpretationPV EV AC Schedule Cost$1 $1 $1 On schedule On cost$2 $2 $1 On schedule Under cost$1 $1 $2 On schedule Over cost$1 $2 $2 Ahead of schedule On cost$1 $2 $3 Ahead of schedule Over cost$1 $2 $1 Ahead of schedule Under cost$3 $2 $1 Behind schedule Under cost$2 $1 $3 Behind schedule Over cost$2 $1 $1 Behind schedule On cost
  19. 19. EVM - Progress EV PV AC CV SV Ref: Earned Value and Agile Reporting Anthony Cabri, Mike Griffiths
  20. 20. Wastage  Any time not dedicated to working on the highest priority story / task on the sprint backlog  Should be tracked over time and by cause Wastage (in hrs) Wastage (in hrs)60 160 14050 120 10040 80 6030 40 2020 010 0 1 2 3 4 5 6 7 8 9 10 11 12
  21. 21. Decrease risks• WIP Limits• Small Iterations• More predictability
  22. 22. Increase Predictability / Reliability• Velocity• Lead Time• Cycle Time
  23. 23. Metrics to Avoid• Velocity comparisons• Productivity e.g. Story Points per day• Effort Estimation accuracy• Number of stories completed• LOC/Function Points• Code quality
  24. 24. Questions?ContactAnu.khendry@gmail.com

×