Metrics for the lean agile pm


Published on

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Metrics for the lean agile pm

  1. 1. Metrics for the LEAN/Agile Product Manager<br />Yuval Yeret<br />
  2. 2. Main things we want to pay attention to<br />Performance of the Production floor – covered elsewhere (Simple KPIs Slides)<br />Performance of the Product Management group: <br />Business Value<br />Wastes related to PM<br />Technical Debt<br />
  3. 3. Business value<br />We care about outcome – features delivered, adopted, used, paid for<br />How can we measure this? <br />Manage a kanban at the high level features level, that tracks when features are adopted, and upon first paying customer. <br />Then see how much WIP of features not yet adopted we have, LEAd/Cycle time to adoption, features that we dropped on the way. <br />
  4. 4. Debt<br />A lot of time debt is taken due to PM decision<br />We want to track how much debt we have, and take action to minimize it. <br />E.g. we need to release CustomerFeatureXnow, so we don’t “automate tests”/”code it correctly”, so every work on ModuleY which is used in FeatureXis slower, until this is fixed<br />
  5. 5. Tracking debt in kanban<br />Have debt card type that is created when debt is taken on<br />Track amount of debt versus overall WIP/Backlog<br />See whether stable, improving, worsening trend<br />Decide on policy for dealing with debt – WIP Limit, etc. <br />Track the cycle time and WIP for debt cards to see whether they get the SLA they deserve<br />
  6. 6. Wastes related to PM<br />Waiting for PM<br />PM related Churn / Context switching / Expediting<br />Sunk Costs<br />Rework due to late feedback by PM<br />
  7. 7. Waiting for PM<br />Look at the CFD, observe the size of the PM-related queues over time. <br />Especially Pending PM Review which is in the middle of DEV/Test<br />And Ready-MMFs as well as DEV Ready in some cases which depend on PM approval<br />Advanced – in the cycle time performance report, focus on PM areas<br />When looking at exceptions to Cycle time, participate in the root cause analysis, and see if interaction with PM was part of the long cycle time. <br />
  8. 8. PM related Churn / Context switching / Expediting<br />Add Expedited class of service<br />Can be used by PM to override priorities in DEV WIP – just to top of queue, don’t override current WIP<br />Add emergency class of service<br />Can also override current WIP<br />Assumption – <br />This is value trumps flow. We give up efficiency when we use these COSs<br />
  9. 9. Measuring the effect of value over flow COS<br />Look at cycle times for different kinds of classes of service<br />Look at distribution of different COS in the WIP<br />
  10. 10. Look at amount of changes in scope<br />Replace – need visualization that shows scope changes in content<br />Add – can simply look at total scope for a “Release” and observe whether its growing<br />
  11. 11. Case Study – Typical release behavior <br />Added Scope<br />Growth in Feature Cost / Dark Matter<br />
  12. 12. Dark Matter – Is where we thought a feature costs X<br />But then, during breakdown, analysis, creating iteration stories, we understand it actually costs X+D<br />Then, PM decides whether to scope to fit down to X again, or D is worth it. <br />Worthwhile tracking our behaviour on this, and learning from it. <br />What is the right D number/percent? Good question!<br />Can be observed in the CFD for a release. <br />
  13. 13. Sunk Costs<br />Add a LANE that collects features/stories that are “ON HOLD” – the Recycle Bin in the archive area<br />The amount of work done on them is the sunk cost<br />Amount of work hard to measure, so use alternative:<br />CYCLE TIME – look at cycle time for end lane being the recycle bin<br />
  14. 14. Rework due to late feedback by PM<br />Will appear as high cycle times<br />Will appear as moving back cards on the board (need to find way to measure)<br />Can use special Class of service / card type to identify these kinds of stories better for measurement/tracking purposes<br />
  15. 15. Workload compared to DEV<br />See how much workload is in PM compared to DEV<br />Look for trends and major changes that can indicate:<br />Bottleneck in PM<br />Idle and slack capacity – expect to see PM seeing clients/customers at those times<br />
  16. 16. Release OVERHEAD<br />“How often do we release? What does it cost us?”<br />The Usual Suspects<br />PM wants to release more often. He wants the ready feature to be out there as soon as possible. <br />R&D usually wants to limit the amount of releases, as they cost a lot, and R&D doesn’t like to do hardening<br />
  17. 17. How often do we release<br />On kanban, simply add a card type of “Release” and flow it thru the board to signify releases. <br />The size should be the hardening cost planned for this release. <br />based on SPC and other charts, you can understand:<br />Plan versus actual on hardening costs/times/dates<br />Frequency of actual releases<br />Ratio of hardening work compared to feature work (see next slide for a view on this)<br />
  18. 18. Release Overhead<br />This metric shows how much effort is spent on releasing versus developing. <br />The aim is to reduce the overhead of each release, such that the organization can increase the frequency of releases to meet business expectations. <br />
  19. 19. Reducing the release overhead<br />Two things we need to do:<br />Reduce the overhead of each release<br />Make sure our release frequency makes sense<br />
  20. 20. Reducing the release overhead of each release<br />Invest in reducing legacy hardening debt<br />As the PM you’ll be asked to invest. <br />Ask for a plan that associates investment of X days with Y days of reduction in hardening cost. <br />Decide what is your investment horizon<br />Based on the horizon, the X/Y ratio, and the current frequency of releases, make your decision<br />Typical areas of investment - Continuous Integration, Automation of EVERYTHING (Including platform matrix, performance, any thing that is currently in the hardening plan)<br />Avoid hardening debt while developing new features<br />Build quality in – don’t let defects wait for the end<br />Consider different types of releases – e.g. Majors, Feature/Service Packs, Patch Bundles<br />And associate the relevant risk-driven hardening work for each<br />
  21. 21. Does our release frequency make sense?<br />First step – have the visibility<br />How many releases<br />What kinds of releases – scheduled major “Trains”, emergency fixes “Ambulance?”, “special delivery/Taxis”<br />Second step – lay out what the business actually needs and is willing to pay for<br />Are those aligned?<br />A lot of the times, you will see “Trains” and “Taxis”. Think of investing in a “Subway” <br />a predictable frequent release mechanism <br />