Measuring Product Management/Ownership Effectiveness in a Lean/Agile world
Metrics for the LEAN/Agile Product Manager Yuval Yeret
Main things we want to pay attentionto • Performance of the Production floor – covered elsewhere (Simple KPIs Slides) • Performance of the Product Management group: – Business Value – Wastes related to PM – Technical Debt
Business value • We care about outcome – features delivered, adopted, used, paid for • How can we measure this? • Manage a kanban at the high level features level, that tracks when features are adopted, and upon first paying customer. • Then see how much WIP of features not yet adopted we have, LEAd/Cycle time to adoption, features that we dropped on the way.
Debt • A lot of time debt is taken due to PM decision • We want to track how much debt we have, and take action to minimize it. • E.g. we need to release ASN1 now, so we don’t “automate tests”/”code it correctly”, so every work on ASN1 is slower, until this is fixed
Tracking debt in kanban • Have debt card type that is created when debt is taken on • Track amount of debt versus overall WIP/Backlog • See whether stable, improving, worsening trend • Decide on policy for dealing with debt – WIP Limit, etc. • Track the cycle time and WIP for debt cards to see whether they get the SLA they deserve
Wastes related to PM • Waiting for PM • PM related Churn / Context switching / Expediting • Sunk Costs • Rework due to late feedback by PM
Waiting for PM • Look at the CFD, observe the size of the PM-related queues over time. – Especially Pending PM Review which is in the middle of DEV/Test – And Ready-MMFs as well as DEV Ready in some cases which depend on PM approval – Advanced – in the cycle time performance report, focus on PM areas • When looking at exceptions to Cycle time, participate in the root cause analysis, and see if interaction with PM was part of the long cycle time.
PM related Churn / Context switching /Expediting • Add Expedited class of service – Can be used by PM to override priorities in DEV WIP – just to top of queue, don’t override current WIP • Add emergency class of service – Can also override current WIP • Assumption – – This is value trumps flow. We give up efficiency when we use
Measuring the effect of value overflow COS • Look at cycle times for different kinds of classes of service • Look at distribution of different COS in the WIP
Look at amount of changes in scope • Replace – need visualization that shows scope changes in content • Add – can simply look at total scope for a “Release” and observe whether its growing
Case Study – Typical release behavior 250 Added Growth in Scope Feature Cost / Dark 200 Matter Total Required Work 150 Actual Done Planned Scope Planned Done Original + Dark Matter 100 Linear (Total Required Work) Linear (Original + Dark Matter) 50 0 1 2 3 4 5 6 7 8 9 10
• Dark Matter – Is where we thought a feature costs X• But then, during breakdown, analysis, creating iteration stories, we understand it actually costs X+D• Then, PM decides whether to scope to fit down to X again, or D is worth it.• Worthwhile tracking our behaviour on this, and learning from it.• What is the right D number/percent? Good question!• Can be observed in the CFD for a release.
Sunk Costs • Add a LANE that collects features/stories that are “ON HOLD” – the Recycle Bin in the archive area • The amount of work done on them is the sunk cost • Amount of work hard to measure, so use alternative: – CYCLE TIME – look at cycle time for end lane being the recycle bin
Rework due to late feedback by PM • Will appear as high cycle times • Will appear as moving back cards on the board (need to find way to measure) • Can use special Class of service / card type to identify these kinds of stories better for measurement/tracking purposes
Workload compared to DEV • See how much workload is in PM compared to DEV • Look for trends and major changes that can indicate: – Bottleneck in PM – Idle and slack capacity – expect to see PM seeing clients/customers at those times
A particular slide catching your eye?
Clipping is a handy way to collect important slides you want to go back to later.