The Impact of Agile Quantified - Agile Australia 2014

2,480
-1

Published on

Larry Maccherone, Director of Analytics for Rally Software, presents: The Impact of Agile Quantified. The tools to find the real-world numbers that can help you make the economic case to get the resources you need and get your people to commit to change. Learn more: http://www.rallydev.com/insights

Published in: Technology, Business

The Impact of Agile Quantified - Agile Australia 2014

  1. 1. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware
  2. 2. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Lean-Agile city. This place runs on folklore, intuition, and anecdotes. If you want to know the truth about this town, stick with me. I’ll give you a tour you’ll never forget. But if you don’t want your beliefs challenged with facts, you’d better beat it, kid. I don’t want to upset you.
  3. 3. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware My sidekick down there? That’s Larry Maccherone. He’s worked in this town his entire professional life.
  4. 4. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware I’m going to give you the tools to find the real-world numbers that can help you make the economic case to get the resources you need and get your people to commit to change. Really. See? It says so right here: -
  5. 5. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware
  6. 6. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware The Seven Deadly Sins of Agile Measurement To avoid creating a monster, stay away from ... 11 Manipulating Others 22 Unbalanced Metrics 33 Quantitative Idolatry 4 Overpriced Metrics 5 Lazy Metrics 6 Bad Analysis 7 Linear Forecasting
  7. 7. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Manipulating Others Sin #1 Using metrics as a lever to drive someone else’s behavior
  8. 8. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Self Improvement Heavenly Virtue #1 Using metrics to reflect on your own performance
  9. 9. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Linear Forecasting Sin #7 Forecasting without discussing probability and risk
  10. 10. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Probability Tools Heavenly Virtue #7 Using the proper tools to predict the likelihood of results (Not likely)
  11. 11. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Monte Carlo Simulation
  12. 12. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Correlation does not necessarily mean causation CAUTION:
  13. 13. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware CAUTION: Only good practices in context There are no best practices
  14. 14. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware
  15. 15. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware The investigation continues with ... Iteration length
  16. 16. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Crowd wisdom or shared delusion? Iteration length Teams using 1 week 6.2% 2 weeks 59.1% 3 weeks 23.4% 4 weeks 9.8% 5+ weeks 1.5%
  17. 17. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware
  18. 18. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware
  19. 19. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware SDPI current dimensions Productivity (Throughput) Predictability (Stability of Throughput) Responsiveness (Time in Process) Quality (Defect Density)
  20. 20. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Future SDPI dimensions Customer/ Stakeholder Satisfaction (Late 2014) Build-the- Right- Thing metric (2015) Employee Engagement/ Satisfaction (Late 2014)
  21. 21. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Raw metrics → Percentiles = Index
  22. 22. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware The investigation continues with ... Iteration length
  23. 23. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Overall Performance Relationship of Iteration Length to Overall Performance index score
  24. 24. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Productivity Relationship of Iteration Length to Productivity index score
  25. 25. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Quality Relationship of Iteration Length to Quality index score
  26. 26. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Responsiveness Relationship of Iteration Length to Responsiveness index score
  27. 27. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Responsiveness Relationship of Iteration Length Transition to Responsiveness index score
  28. 28. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Overall Performance Relationship of Iteration Length Transition to SDPI Overall Performance
  29. 29. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Facts Discovered: ● Teams using two-week iterations have the best balanced performance ● Longer iterations correlate with higher Quality ● Shorter iterations correlate with higher Productivity and Responsiveness ● However, some teams are acting like “tough guys” by pretending to operate at one- week iterations when they can’t back it up
  30. 30. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware The investigation continues with ... Survey-based research
  31. 31. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Teams represented in survey
  32. 32. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Where in the world?
  33. 33. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Where in the U.S.?
  34. 34. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Where in the Europe?
  35. 35. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware The investigation continues with ... Ratio of testers to developers
  36. 36. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Testers per Developer The numbers
  37. 37. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Productivity Relationship of Testers per Developer to Productivity index score
  38. 38. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Quality Relationship of Testers per Developer to Quality index score
  39. 39. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Responsiveness Relationship of Testers per Developer to Responsiveness index score
  40. 40. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Predictability Relationship of Testers per Developer to Predictability index score
  41. 41. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Facts Discovered: ● More testers lead to better Quality ● But they also generally lead to worse Productivity and Responsiveness ● Interestingly, teams that self- identify as having no testers have: ○ the best Productivity ○ almost as good Quality ○ but much wider variation in Quality
  42. 42. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware The investigation continues with ... Motive
  43. 43. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware
  44. 44. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware
  45. 45. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Evidence Found: ● Motive has a small but statistically significant impact on performance ● Extrinsic motivation does not have a negative impact on performance ● Executive support is critical for success with Agile. ● Teamwork is not the dominant factor; talent, skills, and experience are ● Those motivated by quality perform best
  46. 46. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware The investigation continues with ... Co-location
  47. 47. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware
  48. 48. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Evidence Found: ● Teams distributed within the same time zone have up to 25% better productivity ● Is distraction a problem?
  49. 49. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware The investigation continues with ... Geography
  50. 50. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware
  51. 51. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Israel-based teams ● Find more defects overall ● But find fewer in production ● Theory: May correlate with high use of static analysis tools ● Theory: Could be recording bias
  52. 52. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Facts Discovered: ● Differences are slight but statistically significant ● Australia has the best overall performance ● India the worst. However, there could be a reporting bias for defects ● Israel seems to catch the most defects before production. Heavy use of static analysis? ● Recording bias worries?
  53. 53. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware One year earlier ... rallydev.com/agilemetrics
  54. 54. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Stable teams result in up to: ● 60% better Productivity ● 40% better Predictability ● 60% better Responsiveness Dedicated teams: Teams made up of people who only work on that one team have double the Productivity Smaller teams have better Productivity Larger teams have better Quality Teams with low WiP have up to: ● 4x better Quality ● 2x faster time-to-market ● But 34% worse productivity
  55. 55. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware ● Demo of Rally Insights ○ Implements SDPI ● Roadmap ○ Self-assessment and tracking (surveys) ■ Two more dimensions ● Customer/Stakeholder Satisfaction ● Employee Engagement/Satisfaction ■ Probe your environment with customized surveys (maturity, practices compliance, etc.) ○ Recommendation Engine ■ What are the top five things we should improve next? What’s next?
  56. 56. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware More research Over 55 variables under study
  57. 57. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Start right now! ● Bring your completed card to the Rally booth for a free copy of the best-selling book, “Agile Business” ● Pick up a free poster ● Insights demo at 3:40pm ● Talk with me about “Metrics that Matter” at the 1:45pm Open Space ● Questions? Visit me and other Rally staff members at the Rally Booth. We’re in the bright red T-shirts.
  58. 58. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware A fact without a theory is like a ship without a sail, is like a boat without a rudder, is like a kite without a tail. A fact without a figure is a tragic final act. But one thing worse in this universe is a theory without a fact. ~ George Schultz
  59. 59. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware Replace Folklore with Facts Swap Anecdotes with Evidence Upgrade Intuition to Insights
  60. 60. ©2014 Rally Software@LMaccherone | LMaccherone@rallydev.com | #AgileAus@RallySoftware

×