SlideShare a Scribd company logo

Agile metrics and quality

Anu Khendry,ACC,SAFe Aglst, PMI-ACP,CSPO,PMP
Anu Khendry,ACC,SAFe Aglst, PMI-ACP,CSPO,PMP
Anu Khendry,ACC,SAFe Aglst, PMI-ACP,CSPO,PMPCertified Coach, Consultant and Trainer: Agile, Project Management and Software Process Improvement at Independent

Agile metrics and quality

1 of 24
Download to read offline
Agile Metrics and Quality
Metrics -> Improvements

Use metrics to
 •   Improve quality
 •   Shorten product cycle time
 •   Decrease costs
 •   Decrease risks
 •   Increase predictability and reliability
Control Limits and Variances
We can have limits for
• WIP
• Defects
• Team size
• Velocity
• Time-boxes
• Others?
We can analyze variances to understand
  the special causes of variation
Focus on Trends
• Trends matter more than values
• Trends and spikes are only the symptoms
• Trends are leading indicators




                          Images from MountainGoat Software
Improve quality
 ◦ Quality is measured by passed tests and
   customer acceptance
 ◦ Quality is defined by Definition of Done
   and Acceptance Criteria
 ◦ Defect trends to use in Agile projects
     Number of open defects
     Escaped defects
     Defect Cycle Time
     Defect spill-over
 ◦ Test Automation is critical!
Defect Injection and
     Resolution




What is the trend of total Active Defects?
                                 Image from MountainGoat Software

Recommended

Top 10 Agile Metrics
Top 10 Agile MetricsTop 10 Agile Metrics
Top 10 Agile MetricsXBOSoft
 
Lean Agile Metrics And KPIs
Lean Agile Metrics And KPIsLean Agile Metrics And KPIs
Lean Agile Metrics And KPIsYuval Yeret
 
Impediments: Silent killer of agile teams
Impediments: Silent killer of agile teamsImpediments: Silent killer of agile teams
Impediments: Silent killer of agile teamsPooja Wandile
 
An introduction to agile estimation and release planning
An introduction to agile estimation and release planningAn introduction to agile estimation and release planning
An introduction to agile estimation and release planningJames Whitehead
 

More Related Content

What's hot

Agile Performance Metrics
Agile Performance MetricsAgile Performance Metrics
Agile Performance MetricsACM
 
Introduction to Agile software testing
Introduction to Agile software testingIntroduction to Agile software testing
Introduction to Agile software testingKMS Technology
 
SAFe SCRUMxp Overview
SAFe SCRUMxp OverviewSAFe SCRUMxp Overview
SAFe SCRUMxp OverviewRob Betcher
 
Scrum of Scrums Patterns Library
Scrum of Scrums Patterns LibraryScrum of Scrums Patterns Library
Scrum of Scrums Patterns LibraryDavid Hanson
 
Agile Estimating & Planning
Agile Estimating & PlanningAgile Estimating & Planning
Agile Estimating & PlanningAgileDad
 
Agile Metrics...That Matter
Agile Metrics...That MatterAgile Metrics...That Matter
Agile Metrics...That MatterErik Weber
 
DevOps Monitoring and Alerting
DevOps Monitoring and AlertingDevOps Monitoring and Alerting
DevOps Monitoring and AlertingKhairul Zebua
 
The Next Wave of Reliability Engineering
The Next Wave of Reliability EngineeringThe Next Wave of Reliability Engineering
The Next Wave of Reliability EngineeringMichael Kehoe
 
The Five Phases of Agile Maturity (Part 2): Phase 3 and 4
The Five Phases of Agile Maturity (Part 2): Phase 3 and 4The Five Phases of Agile Maturity (Part 2): Phase 3 and 4
The Five Phases of Agile Maturity (Part 2): Phase 3 and 4Cprime
 
Site Reliability Engineering (SRE) - Tech Talk by Keet Sugathadasa
Site Reliability Engineering (SRE) - Tech Talk by Keet SugathadasaSite Reliability Engineering (SRE) - Tech Talk by Keet Sugathadasa
Site Reliability Engineering (SRE) - Tech Talk by Keet SugathadasaKeet Sugathadasa
 
Henrik Kniberg: Lean from the Trenches keynote @ AgileEE
Henrik Kniberg: Lean from the Trenches keynote @ AgileEEHenrik Kniberg: Lean from the Trenches keynote @ AgileEE
Henrik Kniberg: Lean from the Trenches keynote @ AgileEEAgileee
 
Product Backlog - Refinement and Prioritization Techniques
Product Backlog - Refinement and Prioritization TechniquesProduct Backlog - Refinement and Prioritization Techniques
Product Backlog - Refinement and Prioritization TechniquesVikash Karuna
 
What is Agile Testing? Edureka
What is Agile Testing? EdurekaWhat is Agile Testing? Edureka
What is Agile Testing? EdurekaEdureka!
 

What's hot (20)

Agile Performance Metrics
Agile Performance MetricsAgile Performance Metrics
Agile Performance Metrics
 
Agile Metrics
Agile MetricsAgile Metrics
Agile Metrics
 
Scrum Process
Scrum ProcessScrum Process
Scrum Process
 
User Story Sizing using Agile Relative Estimation
User Story Sizing using Agile Relative EstimationUser Story Sizing using Agile Relative Estimation
User Story Sizing using Agile Relative Estimation
 
Introduction to Agile software testing
Introduction to Agile software testingIntroduction to Agile software testing
Introduction to Agile software testing
 
Agile Story Writing
Agile Story WritingAgile Story Writing
Agile Story Writing
 
SAFe SCRUMxp Overview
SAFe SCRUMxp OverviewSAFe SCRUMxp Overview
SAFe SCRUMxp Overview
 
Scrum of Scrums Patterns Library
Scrum of Scrums Patterns LibraryScrum of Scrums Patterns Library
Scrum of Scrums Patterns Library
 
Agile Estimating & Planning
Agile Estimating & PlanningAgile Estimating & Planning
Agile Estimating & Planning
 
Agile Metrics...That Matter
Agile Metrics...That MatterAgile Metrics...That Matter
Agile Metrics...That Matter
 
Agile Methodology
Agile MethodologyAgile Methodology
Agile Methodology
 
DevOps Monitoring and Alerting
DevOps Monitoring and AlertingDevOps Monitoring and Alerting
DevOps Monitoring and Alerting
 
Scrumban (r)Evolution
Scrumban (r)EvolutionScrumban (r)Evolution
Scrumban (r)Evolution
 
The Next Wave of Reliability Engineering
The Next Wave of Reliability EngineeringThe Next Wave of Reliability Engineering
The Next Wave of Reliability Engineering
 
The Five Phases of Agile Maturity (Part 2): Phase 3 and 4
The Five Phases of Agile Maturity (Part 2): Phase 3 and 4The Five Phases of Agile Maturity (Part 2): Phase 3 and 4
The Five Phases of Agile Maturity (Part 2): Phase 3 and 4
 
Agile Scrum Estimation
Agile   Scrum EstimationAgile   Scrum Estimation
Agile Scrum Estimation
 
Site Reliability Engineering (SRE) - Tech Talk by Keet Sugathadasa
Site Reliability Engineering (SRE) - Tech Talk by Keet SugathadasaSite Reliability Engineering (SRE) - Tech Talk by Keet Sugathadasa
Site Reliability Engineering (SRE) - Tech Talk by Keet Sugathadasa
 
Henrik Kniberg: Lean from the Trenches keynote @ AgileEE
Henrik Kniberg: Lean from the Trenches keynote @ AgileEEHenrik Kniberg: Lean from the Trenches keynote @ AgileEE
Henrik Kniberg: Lean from the Trenches keynote @ AgileEE
 
Product Backlog - Refinement and Prioritization Techniques
Product Backlog - Refinement and Prioritization TechniquesProduct Backlog - Refinement and Prioritization Techniques
Product Backlog - Refinement and Prioritization Techniques
 
What is Agile Testing? Edureka
What is Agile Testing? EdurekaWhat is Agile Testing? Edureka
What is Agile Testing? Edureka
 

Similar to Agile metrics and quality

Webinar 2nd edition of the secrets of agile software development from the exe...
Webinar 2nd edition of the secrets of agile software development from the exe...Webinar 2nd edition of the secrets of agile software development from the exe...
Webinar 2nd edition of the secrets of agile software development from the exe...Belatrix Software Factory
 
Kanban i praksis hos maersk line
Kanban i praksis hos maersk line Kanban i praksis hos maersk line
Kanban i praksis hos maersk line BestBrains
 
Kanban i praksis hos maersk line
Kanban i praksis hos maersk lineKanban i praksis hos maersk line
Kanban i praksis hos maersk lineBestBrains
 
Meet or Exceed Schedule Expectations through Intelligent Acceleration
Meet or Exceed Schedule Expectations through Intelligent AccelerationMeet or Exceed Schedule Expectations through Intelligent Acceleration
Meet or Exceed Schedule Expectations through Intelligent AccelerationAcumen
 
Supply chain design and operation
Supply chain design and operationSupply chain design and operation
Supply chain design and operationAngelainBay
 
Preparing for Enterprise Continuous Delivery - 5 Critical Steps
Preparing for Enterprise Continuous Delivery - 5 Critical StepsPreparing for Enterprise Continuous Delivery - 5 Critical Steps
Preparing for Enterprise Continuous Delivery - 5 Critical StepsXebiaLabs
 
When do you need it by? Business Agility Metrics
When do you need it by? Business Agility MetricsWhen do you need it by? Business Agility Metrics
When do you need it by? Business Agility MetricsMartin Aziz
 
Modeling, simulation & data mining: Answering Tough Executive Questions (Agil...
Modeling, simulation & data mining: Answering Tough Executive Questions (Agil...Modeling, simulation & data mining: Answering Tough Executive Questions (Agil...
Modeling, simulation & data mining: Answering Tough Executive Questions (Agil...Troy Magennis
 
Modeling, simulation & data mining agile 2012 (magennis & maccherone)
Modeling, simulation & data mining   agile 2012 (magennis & maccherone)Modeling, simulation & data mining   agile 2012 (magennis & maccherone)
Modeling, simulation & data mining agile 2012 (magennis & maccherone)drewz lin
 
Agile Metrics - Modeling, Simulation, and Data Mining
Agile Metrics - Modeling, Simulation, and Data MiningAgile Metrics - Modeling, Simulation, and Data Mining
Agile Metrics - Modeling, Simulation, and Data MiningRally Software
 
Nacada team samples 2012 (steve)
Nacada team samples 2012 (steve)Nacada team samples 2012 (steve)
Nacada team samples 2012 (steve)Laura Pasquini
 
7 (+/- 2) Steps to Agility
7 (+/- 2) Steps to Agility7 (+/- 2) Steps to Agility
7 (+/- 2) Steps to AgilityTim Gifford
 
From Waterfall to Weekly Releases: A Case Study in using Evo and Kanban (2004...
From Waterfall to Weekly Releases: A Case Study in using Evo and Kanban (2004...From Waterfall to Weekly Releases: A Case Study in using Evo and Kanban (2004...
From Waterfall to Weekly Releases: A Case Study in using Evo and Kanban (2004...Tathagat Varma
 
Introduction of Kanban for large scale maintenance at mobile.international Gm...
Introduction of Kanban for large scale maintenance at mobile.international Gm...Introduction of Kanban for large scale maintenance at mobile.international Gm...
Introduction of Kanban for large scale maintenance at mobile.international Gm...Markus Andrezak
 
Taking the Bull by the Horns with Reliability Centered Maintenance - Trino Pe...
Taking the Bull by the Horns with Reliability Centered Maintenance - Trino Pe...Taking the Bull by the Horns with Reliability Centered Maintenance - Trino Pe...
Taking the Bull by the Horns with Reliability Centered Maintenance - Trino Pe...marcus evans Network
 
BPStudy #45 自転車 de ダイエット
BPStudy #45 自転車 de ダイエットBPStudy #45 自転車 de ダイエット
BPStudy #45 自転車 de ダイエットMitsukuni Sato
 
Automate your way to agility
Automate your way to agilityAutomate your way to agility
Automate your way to agilityYuval Yeret
 
Why use Gitlab
Why use GitlabWhy use Gitlab
Why use Gitlababenyeung1
 
Why and How to Run Your Own Gitlab Runners as Your Company Grows
Why and How to Run Your Own Gitlab Runners as Your Company GrowsWhy and How to Run Your Own Gitlab Runners as Your Company Grows
Why and How to Run Your Own Gitlab Runners as Your Company GrowsNGINX, Inc.
 
How to live with agile - Aware in BugDay Bangkok 2012
How to live with agile - Aware in BugDay Bangkok 2012How to live with agile - Aware in BugDay Bangkok 2012
How to live with agile - Aware in BugDay Bangkok 2012Prathan Dansakulcharoenkit
 

Similar to Agile metrics and quality (20)

Webinar 2nd edition of the secrets of agile software development from the exe...
Webinar 2nd edition of the secrets of agile software development from the exe...Webinar 2nd edition of the secrets of agile software development from the exe...
Webinar 2nd edition of the secrets of agile software development from the exe...
 
Kanban i praksis hos maersk line
Kanban i praksis hos maersk line Kanban i praksis hos maersk line
Kanban i praksis hos maersk line
 
Kanban i praksis hos maersk line
Kanban i praksis hos maersk lineKanban i praksis hos maersk line
Kanban i praksis hos maersk line
 
Meet or Exceed Schedule Expectations through Intelligent Acceleration
Meet or Exceed Schedule Expectations through Intelligent AccelerationMeet or Exceed Schedule Expectations through Intelligent Acceleration
Meet or Exceed Schedule Expectations through Intelligent Acceleration
 
Supply chain design and operation
Supply chain design and operationSupply chain design and operation
Supply chain design and operation
 
Preparing for Enterprise Continuous Delivery - 5 Critical Steps
Preparing for Enterprise Continuous Delivery - 5 Critical StepsPreparing for Enterprise Continuous Delivery - 5 Critical Steps
Preparing for Enterprise Continuous Delivery - 5 Critical Steps
 
When do you need it by? Business Agility Metrics
When do you need it by? Business Agility MetricsWhen do you need it by? Business Agility Metrics
When do you need it by? Business Agility Metrics
 
Modeling, simulation & data mining: Answering Tough Executive Questions (Agil...
Modeling, simulation & data mining: Answering Tough Executive Questions (Agil...Modeling, simulation & data mining: Answering Tough Executive Questions (Agil...
Modeling, simulation & data mining: Answering Tough Executive Questions (Agil...
 
Modeling, simulation & data mining agile 2012 (magennis & maccherone)
Modeling, simulation & data mining   agile 2012 (magennis & maccherone)Modeling, simulation & data mining   agile 2012 (magennis & maccherone)
Modeling, simulation & data mining agile 2012 (magennis & maccherone)
 
Agile Metrics - Modeling, Simulation, and Data Mining
Agile Metrics - Modeling, Simulation, and Data MiningAgile Metrics - Modeling, Simulation, and Data Mining
Agile Metrics - Modeling, Simulation, and Data Mining
 
Nacada team samples 2012 (steve)
Nacada team samples 2012 (steve)Nacada team samples 2012 (steve)
Nacada team samples 2012 (steve)
 
7 (+/- 2) Steps to Agility
7 (+/- 2) Steps to Agility7 (+/- 2) Steps to Agility
7 (+/- 2) Steps to Agility
 
From Waterfall to Weekly Releases: A Case Study in using Evo and Kanban (2004...
From Waterfall to Weekly Releases: A Case Study in using Evo and Kanban (2004...From Waterfall to Weekly Releases: A Case Study in using Evo and Kanban (2004...
From Waterfall to Weekly Releases: A Case Study in using Evo and Kanban (2004...
 
Introduction of Kanban for large scale maintenance at mobile.international Gm...
Introduction of Kanban for large scale maintenance at mobile.international Gm...Introduction of Kanban for large scale maintenance at mobile.international Gm...
Introduction of Kanban for large scale maintenance at mobile.international Gm...
 
Taking the Bull by the Horns with Reliability Centered Maintenance - Trino Pe...
Taking the Bull by the Horns with Reliability Centered Maintenance - Trino Pe...Taking the Bull by the Horns with Reliability Centered Maintenance - Trino Pe...
Taking the Bull by the Horns with Reliability Centered Maintenance - Trino Pe...
 
BPStudy #45 自転車 de ダイエット
BPStudy #45 自転車 de ダイエットBPStudy #45 自転車 de ダイエット
BPStudy #45 自転車 de ダイエット
 
Automate your way to agility
Automate your way to agilityAutomate your way to agility
Automate your way to agility
 
Why use Gitlab
Why use GitlabWhy use Gitlab
Why use Gitlab
 
Why and How to Run Your Own Gitlab Runners as Your Company Grows
Why and How to Run Your Own Gitlab Runners as Your Company GrowsWhy and How to Run Your Own Gitlab Runners as Your Company Grows
Why and How to Run Your Own Gitlab Runners as Your Company Grows
 
How to live with agile - Aware in BugDay Bangkok 2012
How to live with agile - Aware in BugDay Bangkok 2012How to live with agile - Aware in BugDay Bangkok 2012
How to live with agile - Aware in BugDay Bangkok 2012
 

Agile metrics and quality

  • 2. Metrics -> Improvements Use metrics to • Improve quality • Shorten product cycle time • Decrease costs • Decrease risks • Increase predictability and reliability
  • 3. Control Limits and Variances We can have limits for • WIP • Defects • Team size • Velocity • Time-boxes • Others? We can analyze variances to understand the special causes of variation
  • 4. Focus on Trends • Trends matter more than values • Trends and spikes are only the symptoms • Trends are leading indicators Images from MountainGoat Software
  • 5. Improve quality ◦ Quality is measured by passed tests and customer acceptance ◦ Quality is defined by Definition of Done and Acceptance Criteria ◦ Defect trends to use in Agile projects  Number of open defects  Escaped defects  Defect Cycle Time  Defect spill-over ◦ Test Automation is critical!
  • 6. Defect Injection and Resolution What is the trend of total Active Defects? Image from MountainGoat Software
  • 7. Analyzing and Preventing Pareto Chart Defects Fishbone / 5 Why Technique
  • 8. Escaped Defects • Used to measure the quality of delivered code  Calculation: 9 ◦ For all releases, find 8 7 all defects related to 6 the release found 5 after the release date 4 3 ◦ Add up all the defects 2 1 ◦ Can be captured per 0 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec day / week / month / … or per sprint / Escaped Defects over Time release
  • 9. Defect Cycle Time • Rapid resolution of defects is important • Average bug-fix time can also be tracked for different priority bugs Defect Cycle Time 20 18 16 Desired Threshold 14 12 Hours 10 8 6 4 2 0 1 2 3 4 5 6 7 8 9 10 Sprint No.
  • 10. Defect Spill-over • Resolution of defects for a sprint story within the sprint is important • Definition of Done can include a criteria like “No P1 defects” • We can also tracked whether spilled-over defects are being closed in the next sprint Defect Spill Over 9 8 7 No of defects 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 10 Sprint No
  • 11. Test Automation  Critical factor for regression testing  Coverage should be as high as possible
  • 12. Shorten product cycle time  Cumulative Flow Diagrams ◦ Lead Time ◦ Cycle Time
  • 13. Features CFD – An Elaboration Time
  • 14. What does this CFD say? Not started Started Completed Too much Work-In-Progress! Image from MountainGoat Software
  • 15. Identifying bottlenecks with a CFD 1 2 3 4 5 6 7 8 8 10 11 12 13 Day Where is the bottleneck? In DB Procs – After the widening Analysis phase Image from http://www.soliantconsulting.com
  • 16. Decrease costs • Earned Value Management • Wastage
  • 17. Earned Value Analysis PV – Planned Value EV - Earned Value AC - Actual cost BAC - Budget @ Completion EAC - Estimate @ Completion ETC - Estimate to CompletionCost Variance (CV) = EV – AC VAC- Variance @ Completion (SV) Schedule Variance = EV – PV Cost Performance Index (CPI) = EV / AC Schedule Performance Index (SPI) = EV / PV Estimate at Completion (EAC) = BAC / CPI or = AC + BAC – EV Estimate To Complete (ETC) = EAC – AC Variance at Completion (VAC) = BAC – EAC
  • 18. EVM - Interpretation PV EV AC Schedule Cost $1 $1 $1 On schedule On cost $2 $2 $1 On schedule Under cost $1 $1 $2 On schedule Over cost $1 $2 $2 Ahead of schedule On cost $1 $2 $3 Ahead of schedule Over cost $1 $2 $1 Ahead of schedule Under cost $3 $2 $1 Behind schedule Under cost $2 $1 $3 Behind schedule Over cost $2 $1 $1 Behind schedule On cost
  • 19. EVM - Progress EV PV AC CV SV Ref: Earned Value and Agile Reporting Anthony Cabri, Mike Griffiths
  • 20. Wastage  Any time not dedicated to working on the highest priority story / task on the sprint backlog  Should be tracked over time and by cause Wastage (in hrs) Wastage (in hrs) 60 160 140 50 120 100 40 80 60 30 40 20 20 0 10 0 1 2 3 4 5 6 7 8 9 10 11 12
  • 21. Decrease risks • WIP Limits • Small Iterations • More predictability
  • 22. Increase Predictability / Reliability • Velocity • Lead Time • Cycle Time
  • 23. Metrics to Avoid • Velocity comparisons • Productivity e.g. Story Points per day • Effort Estimation accuracy • Number of stories completed • LOC/Function Points • Code quality