Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Impact of Agile Quantified - Late 2014 Edition

2,744 views

Published on

Did you know that dedicating team members to a single team doubles productivity? Do you know what levers will cut your time to market in half? Did you know that teams that most aggressively control their simultaneous work in process (WiP) have as much as a 75% reduction in released defects? How much of an impact do the simple things like holding regular retrospectives have on performance?

Groundbreaking research on the performance impact of software engineering practices derived from tens of thousands of teams across a wide variety of industries and contexts!

This session answers the questions your organization is asking.

Published in: Data & Analytics
  • Be the first to comment

Impact of Agile Quantified - Late 2014 Edition

  1. 1. The Impact Agile of Quantified @LMaccherone | Larry@Maccherone.com @Tasktop |
  2. 2. Lean-Agile city. This place runs on folklore, intuition, and anecdotes. If you want to know the truth about this town, stick with me. I’ll give you a tour you’ll never forget. But if you don’t want your beliefs challenged with facts, you’d better beat it, kid. I don’t want to upset you. @LMaccherone | Larry@Maccherone.com @Tasktop |
  3. 3. My sidekick down there? That’s Larry Maccherone. He’s worked in this town his entire professional life. @LMaccherone | Larry@Maccherone.com @Tasktop |
  4. 4. Credits Many thanks to my friends at Rally Software and their customers whose non-attributable data is the source for much of this research. Much of this material was previously published when I was Director of Analytics and Research at Rally Software, and much of that is derived from my PhD work at Carnegie Mellon, while other aspects come from collaboration with the Software Engineering Institute. @LMaccherone | Larry@Maccherone.com @Tasktop |
  5. 5. I’m going to give you the tools to find the real-world numbers that can help you make the economic case to get the resources you need and get your people to commit to change. Really. - @LMaccherone | Larry@Maccherone.com @Tasktop |
  6. 6. The Seven Deadly Sins of Agile Measurement 1 Manipulating Others 2 Unbalanced Metrics 3 Quantitative Idolatry 4 Overpriced Metrics 5 Lazy Metrics 6 Bad Analysis 7 Linear Forecasting @LMaccherone | Larry@Maccherone.com @Tasktop |
  7. 7. Sin #1 Manipulating Others Using metrics as a lever to drive someone else’s behavior @LMaccherone | Larry@Maccherone.com @Tasktop |
  8. 8. Heavenly Virtue #1 Self Improvement Using metrics to reflect on your own performance @LMaccherone | Larry@Maccherone.com @Tasktop |
  9. 9. Linear Sin #7 Forecasting Forecasting without discussing probability and risk @LMaccherone | Larry@Maccherone.com @Tasktop |
  10. 10. Heavenly Virtue #7 Probability Tools Using the proper tools to predict the likelihood of results @LMaccherone | Larry@Maccherone.com @Tasktop | (Not likely)
  11. 11. Monte Carlo Simulation @LMaccherone | Larry@Maccherone.com @Tasktop |
  12. 12. CAUTION: Correlation does not necessarily mean causation @LMaccherone | Larry@Maccherone.com @Tasktop |
  13. 13. CAUTION: There are no best practices Only good practices in context @LMaccherone | Larry@Maccherone.com @Tasktop |
  14. 14. The investigation continues with ... Iteration length @LMaccherone | Larry@Maccherone.com @Tasktop |
  15. 15. Crowd wisdom or shared delusion? Iteration length Teams using 1 week 6.2% 2 weeks 59.1% 3 weeks 23.4% 4 weeks 9.8% 5+ weeks 1.5% @LMaccherone | Larry@Maccherone.com @Tasktop |
  16. 16. @LMaccherone | Larry@Maccherone.com @Tasktop |
  17. 17. @LMaccherone | Larry@Maccherone.com @Tasktop |
  18. 18. SDPI current dimensions Productivity (Throughput) Predictability (Stability of Throughput) Responsiveness (Time in Process) Quality (Defect Density) @LMaccherone | Larry@Maccherone.com @Tasktop |
  19. 19. Future SDPI dimensions Customer/ Stakeholder Satisfaction (Late 2014) Build-the- Right- Thing metric (2015) Employee Engagement/ Satisfaction (Late 2014) Code Quality from Static Analysis (2015) @LMaccherone | Larry@Maccherone.com @Tasktop |
  20. 20. Raw metrics → Percentiles = Index @LMaccherone | Larry@Maccherone.com @Tasktop |
  21. 21. The investigation continues with ... Iteration length @LMaccherone | Larry@Maccherone.com @Tasktop |
  22. 22. @LMaccherone | Larry@Maccherone.com @Tasktop |
  23. 23. @LMaccherone | Larry@Maccherone.com @Tasktop |
  24. 24. @LMaccherone | Larry@Maccherone.com @Tasktop |
  25. 25. @LMaccherone | Larry@Maccherone.com @Tasktop |
  26. 26. Facts Discovered: ● Teams using two-week iterations have the best balanced performance ● Longer iterations correlate with higher Quality ● Shorter iterations correlate with higher Productivity and Responsiveness ● However, some teams are acting like “tough guys” by pretending to operate at one-week iterations when they can’t back it up @LMaccherone | Larry@Maccherone.com @Tasktop |
  27. 27. The investigation continues with ... Ratio of testers to developers @LMaccherone | Larry@Maccherone.com @Tasktop |
  28. 28. @LMaccherone | Larry@Maccherone.com @Tasktop |
  29. 29. @LMaccherone | Larry@Maccherone.com @Tasktop |
  30. 30. @LMaccherone | Larry@Maccherone.com @Tasktop |
  31. 31. Facts Discovered: ● More testers lead to better Quality ● But they also generally lead to worse Productivity and Responsiveness ● Interestingly, teams that self-identify as having no testers have: o The best Productivity o Almost as good Quality o But much wider variation in Quality @LMaccherone | Larry@Maccherone.com @Tasktop |
  32. 32. The investigation continues with ... Retrospectives @LMaccherone | Larry@Maccherone.com @Tasktop |
  33. 33. @LMaccherone | Larry@Maccherone.com @Tasktop |
  34. 34. @LMaccherone | Larry@Maccherone.com @Tasktop |
  35. 35. @LMaccherone | Larry@Maccherone.com @Tasktop |
  36. 36. @LMaccherone | Larry@Maccherone.com @Tasktop |
  37. 37. The investigation continues with ... Motive @LMaccherone | Larry@Maccherone.com @Tasktop |
  38. 38. @LMaccherone | Larry@Maccherone.com @Tasktop |
  39. 39. @LMaccherone | Larry@Maccherone.com @Tasktop |
  40. 40. Evidence Found: ● Motive has a small but statistically significant impact on performance ● Extrinsic motivation does not have a negative impact on performance ● Executive support is critical for success with Agile ● Teamwork is not the dominant factor; talent, skills, and experience are ● Those motivated by quality perform best @LMaccherone | Larry@Maccherone.com @Tasktop |
  41. 41. The investigation continues with ... Co-location @LMaccherone | Larry@Maccherone.com @Tasktop |
  42. 42. @LMaccherone | Larry@Maccherone.com @Tasktop |
  43. 43. Evidence Found: ● Teams distributed within the same time zone have up to 25% better productivity. ● Is distraction a problem? @LMaccherone | Larry@Maccherone.com @Tasktop |
  44. 44. One year earlier ... rallydev.com/agilemetrics @LMaccherone | Larry@Maccherone.com @Tasktop |
  45. 45. Teams with low WiP have up to: ● 4x better Quality ● 2x faster Time to market ● But 34% worse productivity Stable teams result in up to: ● 60% better Productivity ● 40% better Predictability Dedicated teams: Teams made up of people who only work on that one team have double the Productivity Smaller teams have better Productivity Larger teams have better Quality @LMaccherone | Larry@Maccherone.com @Tasktop |
  46. 46. @LMaccherone | Larry@Maccherone.com @Tasktop | Future research
  47. 47.  Tasktop Data – Provides a single aggregated stream of Data from connected tools: • ALM, Build, SCM, Support, Sales, etc. – Operates in real time. – Maintains complete history. – Normalizes data models. Resolves users and projects. – Supports analytics and reporting tools. – Does not directly provide visualization or analysis.  Get insights like the ones in this talk for your teams
  48. 48. A fact without a theory is like a ship without a sail, is like a boat without a rudder, is like a kite without a tail. A fact without a figure is a tragic final act. But one thing worse in this universe is a theory without a fact. ~ George Schultz @LMaccherone | Larry@Maccherone.com @Tasktop | ©2014 Larry Maccherone
  49. 49. Tasktop booth • Your questions • Extracting insights from your data Panel discussion • 4:30pm Grand Ballroom • Dave West – Tasktop Chief Product Officer • Ken Pugh • Mike Cottmeyer • Steve Davis • Jeff Estill @LMaccherone | Larry@Maccherone.com @Tasktop |
  50. 50. Additional slides @LMaccherone | Larry@Maccherone.com #RallyON14 @Tasktop |
  51. 51. The investigation continues with ... SDPI dimensions @LMaccherone | Larry@Maccherone.com @Tasktop |
  52. 52. Productivity = Throughput Throughput is simply the count of User Stories completed in a given time period. Productivity (by default) is the percentile scoring of the raw Throughput metric for User Stories normalized by team size. @LMaccherone | Larry@Maccherone.com @Tasktop | ©2014 Larry Maccherone
  53. 53. Predictability = Stability of Throughput Predictability measures how consistent you are at producing the same amount of work each month as measured by the Coefficient of Variation (CoV) of Throughput. Predictability (by default) is the percentile scoring of the raw CoV of Throughput. @LMaccherone | Larry@Maccherone.com @Tasktop | ©2014 Larry Maccherone
  54. 54. Responsiveness = Time in Process TiP shows how long it takes to get one work item through your system. It's the work days that a User Story spends in development and testing. Similar to lead time or cycle time. Responsiveness (by default) is the percentile scoring of the raw Time In Process (TiP) metric for User Stories. @LMaccherone | Larry@Maccherone.com @Tasktop | ©2014 Larry Maccherone
  55. 55. Quality = Defect Density Defect Density is a representation of the number of defects found in your code. It's the count of defects found in a given time period, normalized by team size. Quality (by default) is the percentile scoring of the raw defect density metrics for both defects found in test as well as those found in production. @LMaccherone | Larry@Maccherone.com @Tasktop | ©2014 Larry Maccherone
  56. 56. The investigation continues with ... Team time together @LMaccherone | Larry@Maccherone.com @Tasktop |
  57. 57. @LMaccherone | Larry@Maccherone.com @Tasktop |
  58. 58. @LMaccherone | Larry@Maccherone.com @Tasktop |
  59. 59. @LMaccherone | Larry@Maccherone.com @Tasktop |
  60. 60. The investigation continues with ... Controlling WiP @LMaccherone | Larry@Maccherone.com @Tasktop |
  61. 61. @LMaccherone | Larry@Maccherone.com @Tasktop |
  62. 62. @LMaccherone | Larry@Maccherone.com @Tasktop |
  63. 63. @LMaccherone | Larry@Maccherone.com @Tasktop |
  64. 64. @LMaccherone | Larry@Maccherone.com @Tasktop |
  65. 65. Facts Discovered: Teams that most aggressively control WiP: ● Have ½ the Time in Process (TiP) ● Have ¼ as many defects ● But have 34% lower productivity @LMaccherone | Larry@Maccherone.com @Tasktop |
  66. 66. Recommendations: ● If your WiP is high, reduce it ● If your WiP is already low, consider your economic drivers ○ If Productivity drives your bottom line, don’t push WiP too low ○ If time to market or quality drives your bottom line, push WiP as low as it will go @LMaccherone | Larry@Maccherone.com @Tasktop |
  67. 67. The investigation continues with ... Estimating process @LMaccherone | Larry@Maccherone.com @Tasktop |
  68. 68. Estimating process Process Type Teams Using No Estimates 3% Full Scrum 79% Lightweight Scrum 10% Hourly-Oriented 8% @LMaccherone | Larry@Maccherone.com @Tasktop |
  69. 69. @LMaccherone | Larry@Maccherone.com @Tasktop |
  70. 70. @LMaccherone | Larry@Maccherone.com @Tasktop |
  71. 71. @LMaccherone | Larry@Maccherone.com @Tasktop |
  72. 72. @LMaccherone | Larry@Maccherone.com @Tasktop |
  73. 73. @LMaccherone | Larry@Maccherone.com @Tasktop |
  74. 74. @LMaccherone | Larry@Maccherone.com @Tasktop |
  75. 75. Facts Discovered: ● Teams doing Full Scrum have 250% better Quality than teams doing no estimating ● Lightweight Scrum performs better overall, with better Productivity, Predictability, and Responsiveness @LMaccherone | Larry@Maccherone.com @Tasktop |
  76. 76. Recommendations: ● Experienced teams may get best results from Lightweight Scrum ● If new to Agile or focused strongly on Quality, choose Full Scrum @LMaccherone | Larry@Maccherone.com @Tasktop |
  77. 77. The investigation continues with ... Team stability & Dedication to one team @LMaccherone | Larry@Maccherone.com @Tasktop |
  78. 78. @LMaccherone | Larry@Maccherone.com @Tasktop |
  79. 79. @LMaccherone | Larry@Maccherone.com @Tasktop |
  80. 80. @LMaccherone | Larry@Maccherone.com @Tasktop |
  81. 81. @LMaccherone | Larry@Maccherone.com @Tasktop |
  82. 82. @LMaccherone | Larry@Maccherone.com @Tasktop |
  83. 83. @LMaccherone | Larry@Maccherone.com @Tasktop |
  84. 84. @LMaccherone | Larry@Maccherone.com @Tasktop |
  85. 85. @LMaccherone | Larry@Maccherone.com @Tasktop |
  86. 86. Another Fact Discovered: One out of four team members changes every three months! @LMaccherone | Larry@Maccherone.com @Tasktop |
  87. 87. @LMaccherone | Larry@Maccherone.com @Tasktop |
  88. 88. @LMaccherone | Larry@Maccherone.com @Tasktop |
  89. 89. @LMaccherone | Larry@Maccherone.com @Tasktop |
  90. 90. @LMaccherone | Larry@Maccherone.com @Tasktop |
  91. 91. @LMaccherone | Larry@Maccherone.com @Tasktop |
  92. 92. Facts Discovered: Stable teams result in up to: ● 60% better Productivity ● 40% better Predictability Another Fact Discovered: One out of four team members changes every three months! @LMaccherone | Larry@Maccherone.com @Tasktop |
  93. 93. Recommendations: ● Dedicate people to a single team ● Keep teams intact and stable @LMaccherone | Larry@Maccherone.com @Tasktop |
  94. 94. The investigation continues with ... Team size @LMaccherone | Larry@Maccherone.com @Tasktop |
  95. 95. Balance your team’s Performance Agile recommends that the ideal team size is 7± 2. How ideal is that when we actually look at the data? @LMaccherone | Larry@Maccherone.com @Tasktop |
  96. 96. @LMaccherone | Larry@Maccherone.com @Tasktop |
  97. 97. @LMaccherone | Larry@Maccherone.com @Tasktop |
  98. 98. @LMaccherone | Larry@Maccherone.com @Tasktop |
  99. 99. @LMaccherone | Larry@Maccherone.com @Tasktop |
  100. 100. @LMaccherone | Larry@Maccherone.com @Tasktop |
  101. 101. Facts Discovered: Small teams (of 1-3) people have: ● 17% lower Quality ● But 17% more Productivity than teams of the recommended size. @LMaccherone | Larry@Maccherone.com @Tasktop |
  102. 102. Recommendations: ● Set up team size of 7±2 people for the most balanced performance ● If you are doing well with larger teams, there’s no evidence that you need to change @LMaccherone | Larry@Maccherone.com @Tasktop |
  103. 103. The investigation continues with ... Geography @LMaccherone | Larry@Maccherone.com @Tasktop |
  104. 104. @LMaccherone | Larry@Maccherone.com @Tasktop |
  105. 105. Israel-based teams ● Find more defects overall ● But find fewer in production ● Theory: May correlate with high use of static analysis tools India-based teams ● Find more defects overall ● Released and unreleased ● Theory: May correlate with high use of static analysis tools ● Theory: Could be recording bias @LMaccherone | Larry@Maccherone.com @Tasktop |
  106. 106. Facts Discovered: ● Differences are slight but statistically significant. ● Australia has the best overall performance. ● India the worst. However, there could be a reporting bias for defects. ● Israel seems to catch the most defects before production. Heavy use of static analysis? @LMaccherone | Larry@Maccherone.com @Tasktop |
  107. 107. The investigation continues with ... Geography: US and Europe @LMaccherone | Larry@Maccherone.com @Tasktop |
  108. 108. @LMaccherone | Larry@Maccherone.com @Tasktop |
  109. 109. @LMaccherone | Larry@Maccherone.com @Tasktop |

×