Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Product Development Metrics: More Harm Than Good?

57 views

Published on

This is a presentation and workshop given to the Silicon Valley Engineering Leadership Group, held in Palo Alto, CA. First it shows how metrics related to compensation can really drive bad behavior. Then the presentation turns to product development metrics that can be used in the context of program management consulting, to improve effectiveness. More is available at https://tcgen.com/product-development-metrics, where there are papers on metrics as well as downloadable tools for improving the product development process.
Metrics svelg aug16_26

Published in: Business
  • Be the first to comment

  • Be the first to like this

Product Development Metrics: More Harm Than Good?

  1. 1. 1 1 SILICON VALLEY SOFTWARE LEADERSHIP COMMUNITY METRICS: MORE HARM THAN GOOD? John Carter August 2018
  2. 2. “In order to get results, you need to measure them”
  3. 3. WRONG!
  4. 4. $100B Cost of Bad Metrics driven by Bad Management
  5. 5. RED, YELLOW, GREEN – REALLY?
  6. 6. 8 88
  7. 7. 9 99
  8. 8. 10 1010
  9. 9. “You miss 100 percent of the shots you never take.”
  10. 10. 15 1515 SUCCESS WITH METRICS: APPLE • Pinpoint a small number of improvement levers • Organizational changes teased apart and divided into incremental improvements, sequenced and prioritized • For each lever, a target metric to seed changes within the organization • Only move on to the next transformation when metrics indicate that an improvement has taken hold
  11. 11. 16 1616 METRICS CHECKLIST Criteria Good Metric Traceable to results (not measure results)?  Hard to game (objectively measured)?  Target curve that varies with time?  Not related to compensation?  Culturally aligned?  Easy to measure?  Rapidly change? 
  12. 12. 17 1717 PREDICTIVE METRICS TREE Goal Key Drivers Initiatives Predictive Metrics
  13. 13. 18 1818 HALF LIFE TABLE • Complexity drives the time to achieve an objective • THIS time is PREDICTABLE and should be used to GUIDE PROGRESS • Based on empirical study of more that 50 improvement programs • The rate of improvement is related to organizational complexity • In the absence of a definitive plan this heuristic is much better than a traffic light
  14. 14. 19 1919 HOW TO USE METRICS TO MANAGE CHANGE • Reported to Steering Committee bi-weekly • Action items were generated based on metrics • Once goal maintained over time • Up the ante / raise the bar • Find a replacement metric to replace Time (Months) 0 % 20 % 40 % 60 % 80 % 100 % 12/15 1/15 2/15 3/18 4/18 5/2 Percent Using Target Actual gap Measure existence Measure coverage Measure quality • Only one metric at a time • Inch-wide, Mile-deep
  15. 15. 20 2020 WORKSHOP With the scenario you provide, construct an improvement approach; suggest a predictive metric and a target curve. Discuss with your table – PICK A TABLE LEADER! • Decide on an improvement goal • List initiatives: pick one that eliminates root causes • Create a candidate PREDICTIVE metric • Describe your target curve: dependent on complexity
  16. 16. 21 2121 DISCUSSION • Sharing workshop results? • Questions, comments?
  17. 17. “We are making better decisions, faster”
  18. 18. 23 2323
  19. 19. 24 24 TCGen Inc. Menlo Park CA, 94025 jcarter@tcgen.com (650) 733-5310
  20. 20. WORKSHOP With the scenario the table agrees to, construct an initiative that reverses root cause, and suggest a predictive metric and describe a target curve, inspired by half-life; finally check you work against the checklist Discuss with your table • Define your improvement goal: Describe an improvement objective • • Discuss initiatives that reverse root causes and pick one • Create a candidate PREDICTIVE metric: That measures progress • • Describe target curve: Dependent on complexity • Check your work by reviewing CHECKLIST – How did you do? Write answer here Write answer here Write answer here FIRST - PICK A LEADER FOR YOUR TABLE! jcarter@tcgen.com Silicon Valley Engineering Leadership Community
  21. 21. METRICS CHECKLIST & HALF LIFE TABLE (TIME TO 50% IMPROVEMENT) Criteria Check if Yes Traceable to results (but not measure results)? o Hard to game (objectively measured)? o Target curve that varies with time? o Not related to compensation? o Culturally aligned? o Easy to measure? o Rapidly change? o Typically Half-Lives are 1 – 6 Months for most transformations that don’t involve partners or large new IT systems. Make your best guess, unless you have a program plan that defines it. Good metrics satisfy most to all of these criteria. The most important are: Traceable to results, Having clear targets that vary over time, avoid compensation, supported by culture, and rapidly change. jcarter@tcgen.com Silicon Valley Engineering Leadership Community

×