Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How to measure the outcome of agile transformation

3,721 views

Published on

This presentation covers details on how we can measure that Agile Transformation is providing the intended outcome or not. I presents a research & survey which tries to understand how different people measure value of Agile Transformation

Published in: Leadership & Management
  • Be the first to comment

How to measure the outcome of agile transformation

  1. 1. How to measure the outcome of Agile transformation? Rahul Sudame CSP, SAFe SPC, PMI-ACP
  2. 2. Agenda  Lets play some game!  We need to have clear goals  Transformation Roadmap  Agile Readiness Assessment Model  Current State Assessment Model  ‘Agile’ Metrics  Agile maturity Model  Survey on ROI for Agile  Case Study
  3. 3. Lets brainstorm!
  4. 4. Agile Innovation Games for Chapter Strategies Is Agile delivering value to you?  Innovation Game: Teams to discuss if Agile is delivering value to you?  Are you measuring the outcome of Agile Transformation in your organization?  Is Agile implementation providing the desired impact?
  5. 5. Defining goal of going Agile
  6. 6. Moving to Agile is not easy!  Everyone wants to go Agile!  But Agile does not come for free, it needs: ◦ Breaking the bones (waterfall structure) of the organization ◦ Changing mindset ◦ Investment & Commitment  So, how to ensure returns of this huge investment? 23 35 41 46 52 0 20 40 60 No Mgmt support Non-willingness Alignment Difficult to Transition No Clear Metrics Response % for challenges in achieving its goals with Scrum Ref: The 2015 State of Scrum Report - Scrum Alliance
  7. 7. We need to have clear goals  Clear identification of Customer & Business benefits  Envisioning the goal, based on type of the organization  Identification of investment needs  Clear identification of ROI parameters Business Case of Agile Transformation Business Case Customer Benefits Business Benefits Project Benefits Investment Needs ROI Parameters
  8. 8. Building Transformation Roadmap  Agile Transformation roadmap in alignment with organizational environment  Pilot to try Agile methodology and for the groundwork  Inspect & Adapt at all the milestones  Build the right environment iteratively Agile Transformation Roadmap
  9. 9. Agile Assessment Models
  10. 10. Agile Readiness Assessment Model  Are we ready for Agile?  Identification of Agile readiness using objective assessment  Team’s interest / enthusiasm to be factored in  Best suited teams to initiate pilot  Gap analysis can highlight organization’s preparedness Agile Readiness Questionnaire Question Guidelines Points Team size • 5 – Team size of 7 to 9 3 – Team size less than 7 2 – Team with 10 to 15 members • 1 – More than 15 members 2 Team Composition • 5 - Completely cross functional 3 - Cross functional with some dependency on external teams 0 - Not Cross functional 2 Business Value Stream Coverage • 5 - Team covers the entire business value stream 1- Less than 50% of the value stream 0 - Less than 10% of the business value stream 1
  11. 11. Focus on values, not just practices  Coach the team to implement Agile ‘principles’ and not just the ‘practices’  Build an environment of continuous improvement  Don’t get trapped in ‘Trough of disillusionment’  Deliver value periodically Sprint Cycle Define Roles Backlog Grooming Agile Estimation Release Planning Sprint Rituals Activities in Sprint Cycles
  12. 12. Agile Current State Assessment  Conduct periodic assessment of team’s Agile practices with involvement of the entire team  Assess the effectiveness of different aspects such as Release Planning, Sprint Planning, Product Ownership  Identify the Gaps and define a plan of bridging those gaps Current State Assessment Model
  13. 13. ‘Agile’ Metrics  Identify the metrics that provide ‘real’ status of progress  Define a mechanism to aggregate required data periodically  Do NOT enforce waterfall metrics on Agile teams   Don’t get trapped in ‘Perfect’ burn down chart  • Earned Business Value / ROI • Release Cycle • Cost of Delay (COD) • Lead / Cycle Time Business Metrics • Velocity / Burn down • Backlog Size • User Stories Done per Iteration • User Stories carried forward to next iteration Process Metrics • Defects count per story / Defect density • Acceptance tests per story • Code Quality & code coverage metrics • Test Automation metrics Quality Metrics
  14. 14. Adaptability meets Predictability  Consistent implementation of Agile and usage of tools can provide release predictability  Dynamic data extrapolation can help in analyzing impact of scope changes  Real-time visibility to all the stakeholders Release Predictability Chart
  15. 15. Agile Maturity Model  Maturity level of a team can be assessed based on different parameters  Organizational benchmarks can be provided as guidelines  Agile Maturity Model itself should be lean & aligned with Agile philosophy Level 1 Level 2 Level 3 Level 4 Level 5 Process Group Rating Applicability Score Maturity Index Release Mgmt Effectiveness 6 7 86% 1.79 Sprint Mgmt Effectiveness 7.5 9 83% Product Backlog Mgmt 7 7 100% Frequent Product Delivery 7 8 88% 0.88Quality Assurance Effectiveness 5.5 6 92% Retro & Improvements 1 3 33% 0.33 Customer focus 5 5 100% 1.00 Maturity Index 4.12 Agile Maturity Model
  16. 16. Reduce Lead Time  Optimizing Lead time & Cycle Time would help in reducing ‘waste’ in the system  ‘Cost of Delay’ identification would help in delivery of higher value items sooner  Focus on business value would help in reducing delays at all levels Deployment 1 month delay for deployment QA 3 weeks delay for signoff Development 2 weeks delay for integration Design 8 days delay for signoff Requirements 2 weeks delay for clarifications Delays at various phases
  17. 17. Best Agile Team Award  Healthy competition amongst the teams can be encouraged  Effectiveness of teams on various parameters related with Agile implementations can be considered  Team’s approach, Scrum Master / PO’s effectiveness can also be factored in Team Retro Effectiveness Scrum Master Effectiveness Points Team 1 12 24 73 Team 2 18 27 65 Team 3 14 11 62 Team 4 8 14 62 Team 5 10 13 61 Team 6 9 11 46 Assessment for Best Team Award
  18. 18. Are we making progress?  Agile Project Progress can be reviewed periodically  Quantitative as well as Qualitative measures to be presented  Organization-wide changes across different value- streams to be reviewed Project Progress Report Quantitative Benefits • Delivery cycle reduction • Revenue realization ($$$) Qualitative Benefits • Value delivered to customer • Team efficiency • Customer/Team satisfaction Benefits Not Realized • Value delivered to customer Project Progress Status Repot
  19. 19. Survey on Outcome of Agile
  20. 20. Survey on Outcome of Agile Independent Survey Findings – Q2 2015 Highlights parameters used for different people to measure outcome of Agile  In Progress: ◦ Details on survey participants ◦ Details on organization, industries, demographics represented ◦ Survey methodology
  21. 21. Survey on Outcome of Agile Independent Survey Findings – Q2 2015 Highlights parameters used for different people to measure outcome of Agile
  22. 22. Survey Findings  Delivery cycle time - 22%  Delivering business value - 19%  Working Software / Customer Satisfaction – 14%  Not measuring ROI – 11%  Reduction in defects, waste & risks – 8% Independent Survey Findings – Q2 2015 Highlights parameters used for different people to measure outcome of Agile
  23. 23. Survey Findings: Parameters  Parameters for measuring outcome of Agile: ◦ Delivery Cycle Time / WIP Reduction ◦ Achievement of business goals/value ◦ Working Software / Customer Satisfaction ◦ Reduction in defects, waste & risks ◦ Improvement in Business/Engineering processes ◦ On time delivery ◦ Delivery within the budget ◦ Break-even point - bang for your buck ◦ Team satisfaction Independent Survey Findings – Q2 2015 Highlights parameters used for different people to measure outcome of Agile
  24. 24. Survey Findings: Feedback Independent Survey Findings – Q2 2015 Highlights parameters used for different people to measure outcome of Start from what you expect... what value did you expect the transformation to bring? If I could choose just one measure I'd choose Cycle Time - time (in days) Lean up front, big picture planning, determine business goals, and, especially, how those goals interrelate. Its not worth to measure the ROI of Agile. Reducing WiP can have up to a 4x impact on Quality, half your Time- to-Market, and improve your Predictability
  25. 25. Case Study
  26. 26. Case Study  Product development organization applied Agile enterprise-wide for product development cycles  Outcome: ◦ Release Cycle reduced from 1.5 years to 3 months ◦ First release was not appreciated by customers due to low user experience – Agile helped in Failing Faster! ◦ Manual regression testing efforts (for sanity suite): 5 days ◦ Automated regression testing efforts (for sanity suite): 4 hours  90% QA effort reduction ◦ Conversion of a short-term customer contract into a long-term engagement ◦ Time taken for matured environment - 1 year
  27. 27. Summary  Agile Transformation is a journey and involves continuous improvement  For effectively presenting ROI, build a baseline and set of parameters/metrics for before and after Agile state ◦ E.g. Pre & Post Agile Delivery Cycles / Durations ◦ Achievement of business / customer goals  Identify and present both quantitative and qualitative benefits to the stakeholders periodically ◦ E.g. Customer Satisfaction ◦ Reduction in production hot fixes, change requests  Retrospect Agile Maturity at all levels of product development and focus on building effectiveness iteratively  E.g. Target: Agile maturity Level 3 or Project Maturity Index > 3.5
  28. 28. Thank You ! - Rahul Sudame rahulsudame@yahoo.com

×