Your SlideShare is downloading. ×
  • Like
Measuring Agility: Top 5 Metrics And Myths
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Measuring Agility: Top 5 Metrics And Myths

  • 12,521 views
Published

Pete presented a webinar on Measuring Agility for VersionOne. This presentation looks at four fundamental elements of measuring agility: predictability, value, productivity and quality. In the …

Pete presented a webinar on Measuring Agility for VersionOne. This presentation looks at four fundamental elements of measuring agility: predictability, value, productivity and quality. In the presentation, Pete evaluates ten effective measurements and some measurement myths.

Published in Business , Lifestyle , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
12,521
On SlideShare
0
From Embeds
0
Number of Embeds
4

Actions

Shares
Downloads
838
Comments
0
Likes
19

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Measuring Agility Top 5 Metrics and Myths Pete Behrens Agile Organization & Process Coach © 2009 Trail Ridge Consulting, LLC pete@trailridgeconsulting.com 303.819.1809
  • 2. Pete Behrens Agile Organization & Process Coach  Certified Scrum Trainer  Certified Scrum Coach  Guide enterprise organizations in transitioning to an agile organization implementing agile methods  Services for agile assessment, alignment, training and coaching Previous Experience  Led development of the requirements management solution RequisitePro – a core product in the IBM Rational product line – using the Rational Unified Process (RUP)  Consulted with EDS leading development of large data warehouse solutions using Rapid Application Development (RAD) © 2009 Trail Ridge Consulting, LLC 2
  • 3. Measurement Dimensions Predictability Value Quality Productivity © 2009 Trail Ridge Consulting, LLC 3
  • 4. How are projects measured today? Failed 23% Median Overrun Succeeded 28% Cost 50% Challenged 49% Schedule 100% • On Time • On Budget The average project • With all initially planned features Costs 50% more and takes twice as long as planned Source: Chaos Report, Standish Group, 2001 © 2009 Trail Ridge Consulting, LLC 4
  • 5. Traditional project visibility is often too late Surprise ! n la tP s Features Te al t& ctu en t A pm s Te lo & ve t en De m e lop D ev Analysis & Requirements Time © 2009 Trail Ridge Consulting, LLC 5
  • 6. Agile seeks transparency from the outset of the project Surprise ! n la Features stP Te i ty ibil t& is al l eV en ctu i m A Ag st p lo Te ve & De e nt m e lop v Analysis & Requirements De Sprint 1 Sprint 2 Sprint 3 Sprint 4 Time © 2009 Trail Ridge Consulting, LLC 6
  • 7. Predictability Metric - Velocity 25 20 Average = 19 15 10 Teams will tend toward a 5 consistent velocity after a few sprints if the team and domain stay consistent 0 Sprint 1 Sprint 2 Sprint 3 Sprint 4 Sprint 5 Sprint 6 Sprint 7 Sprint 8 © 2009 Trail Ridge Consulting, LLC 7
  • 8. Sprints drive predictability Traditional Project Predictable Uncertain Unpredictable Agile Project with Timeboxed Iterations Sprint Sprint Sprint Sprint Sprint Sprint Sprint Sprint Project Timeline Definition: Sprint = Iteration = Timebox © 2009 Trail Ridge Consulting, LLC 8
  • 9. Velocity - Advanced Burn Down  Measures team velocity (work complete per sprint)  Measures scope change over time  Guides release-level decision making © 2009 Trail Ridge Consulting, LLC 9
  • 10. Predictability Metric - On Time Delivery Last non-agile release Since March 2007 every Salesforce.com agile release has been deployed on-time (down to the exact minute) Source: Scrum Gathering 2008 - Salesforce.com Keynote Address © 2009 Trail Ridge Consulting, LLC 10
  • 11. Agile is value-driven Predictive Process Adaptive Process (Waterfall) (Agile) Constraints Requirements Cost Schedule Plan Value/Vision Driven Driven Estimates Cost Schedule Features The plan creates The vision creates cost/schedule estimates feature estimates © 2009 Trail Ridge Consulting, LLC 11
  • 12. What is valued? Actual use of requested features in predictive projects Always 7% Never Often 45% 13% Results: • 64% Rarely or never used Sometimes 16% • 20% Frequently used Rarely 19% Source: Standish Group study presented at XP2002 by Jim Johnson © 2009 Trail Ridge Consulting, LLC 12
  • 13. The Value of Time Traditional Single Release Delivered 1-5 Agile Incremental Release Delivered Delivered Delivered Delivered Delivered 1 2 3 4 5 Time Value Gap Value © 2009 Trail Ridge Consulting, LLC 13
  • 14. Value Metric - Feature Delivery  94% feature request increase from 2006 - 2007  38% increase in feature request delivered per developer Source: Scrum Gathering 2008 - Salesforce.com Keynote Address © 2009 Trail Ridge Consulting, LLC 14
  • 15. Value Metric - Customer Survey  Ask your customers!  Set a baseline and measure quarterly  Qualitative & Quantitative  Questions cover  Responsiveness  Quality of features  Support provided  Delivery timeliness  Feature value © 2009 Trail Ridge Consulting, LLC  ... 15
  • 16. Quality Metric - Running Tested Features (RTF)  Measures the number of automated unit and functional tests for a team/product over time  Measures quality as a leading indicator  Measures productivity with respect to complexity better than other measures © 2009 Trail Ridge Consulting, LLC Source: http://www.xprogramming.com/xpmag/jatRtsMetric.htm 16
  • 17. RTF Example Israeli Air Force in 2005  Increased confidence in team and management  Enabled accurate and effective decision making  Motivated writing tests  Motivated writing smaller tests - more adaptable  # of tests generally reflected http://www.cs.huji.ac.il/~davidt/papers/Agile_Metrics_AgileUnited05.pdf complexity better than other methods (e.g. SLOC, Function Points, etc.) © 2009 Trail Ridge Consulting, LLC 17
  • 18. Quality Metric - Issue / Defect Costs  Measure the # of product issues and defects multiplied by the cost of addressing them  Measures quality as a lagging indicator  Measures support cost impact of quality © 2009 Trail Ridge Consulting, LLC 18
  • 19. Issue/Defect Cost @ IBM 1,056 1,056 792 2008 Expected Cost per Actual Defect = $16,000 528 Ticket = $500 Savings = $2.6M 264 168 67 Economics of Agile Development 32 Sue McKinney, IBM - 2008 Agile2008 Conference Case Study 0 Defects Tickets © 2009 Trail Ridge Consulting, LLC 19
  • 20. Myth - Metrics drive team performance Metrics are not inherently good or bad It is the use of the metric that drives team dysfunction © 2009 Trail Ridge Consulting, LLC 20
  • 21. Myth - Velocity measures productivity (or value)  Story points are relative  Cannot compare velocity across teams  All teams, products, environments, constraints, and dependencies are different  Some stories are more valued than others © 2009 Trail Ridge Consulting, LLC 21
  • 22. Myth - 100% Committed vs. Actual drives estimation accuracy 40 78% 64% 91% 100% 120% 30 32 Predictability 28 Story Points 25 24 20 22 20 20 20 20 Committed 18 Actual 10 Productivity 0 Sprint 1 Sprint 2 Sprint 3 Sprint 4 Sprint 5 © 2009 Trail Ridge Consulting, LLC 22
  • 23. Commitment vs. Actual Completed Remaining Share commitment vs. actual as a fact to drive discussions: 1. Why didn’t we get it done? 2. What are we doing about it? 3. What are the impact to the release goals? © 2009 Trail Ridge Consulting, LLC 23
  • 24. Commitment vs. Actual 150.0 Completed Remaining 112.5 Share commitment vs. actual as a fact to drive discussions: Points 75.0 1. Why didn’t we get it done? 37.5 2. What are we doing about it? 3. What are the impact to the 0 Jan Feb Mar Apr May Jun Jul Aug Sep release goals? © 2009 Trail Ridge Consulting, LLC 23
  • 25. Myth - Higher velocity is always a good thing  Technical debt is bad  Technical debt is any “not- quite-right” code not fixed (e.g. Bugs, refactors, workarounds, etc.)  Pushing too hard on new product value and velocity tends to increase technical debt  Measure and limit technical debt accumulation © 2009 Trail Ridge Consulting, LLC 24
  • 26. Myth - Sprints “Fail” Maximum information is generated when the probability of failure is 50% - not when hypothesis are always correct. It is necessary to have a reasonable failure rate in order to generate a reasonable amount of new information. - Reinertsen, Managing the Design Factory © 2009 Trail Ridge Consulting, LLC 25
  • 27. Treat Sprints as Practice  Sprints allow teams to practice the skill of delivering high-quality software on time  Preventing failure in sprints limits team learning, growth, discipline, empowerment and productivity © 2009 Trail Ridge Consulting, LLC 26
  • 28. Top 5 (or 6) Agile Metrics Predictability Value 2. On time delivery 3. Customer 1. Velocity Surveys 4. # Features or Value Delivered 6. Issue/Defect Cost 5. Running Tested Features Quality Productivity © 2009 Trail Ridge Consulting, LLC 27
  • 29. Top 5 (or 6) Agile Metric Myths 1. Metrics drive team performance 2. Velocity measures productivity 3. Achieving 100% commitment to actual increases estimation accuracy 4. Increasing velocity is always a good thing 5. Sprints “Fail” 6. An Agile tool will make you agile © 2009 Trail Ridge Consulting, LLC 28
  • 30. V1 provides metrics and an agile framework - You guide agility © 2009 Trail Ridge Consulting, LLC 29
  • 31. Extending the Metrics 7. Velocity --> Investment, $/sprint, $/story pt 8. Features Delivered --> Earned Value 9. Customer Surveys --> Employee Surveys 10.Quality/Productivity --> Technical Debt © 2009 Trail Ridge Consulting, LLC 30