Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Evidence based decision-making - lean product development


Published on

Presentation mostly based on Don Reinertsen's book The Principles of Product Development: Flow, Second Generation Lean Product Development. It was presented at the MN Agile Experience Group meeting at the University of St. Thomas on Jan. 17, 2017.

Published in: Software
  • ...My Scandalous Secret to Crushing The Odds So Effortlessly... ◆◆◆
    Are you sure you want to  Yes  No
    Your message goes here
  • Winning the Lottery is Based on This [7 Time Winner Tells All] ➤➤
    Are you sure you want to  Yes  No
    Your message goes here

Evidence based decision-making - lean product development

  1. 1. Evidence-based Decision-making for Lean Product Development presented by Kevin Burns @ MN AEG Jan. 17, 2017, @kevinbburns
  2. 2. Kevin Burns Coach Org Change Agent, @kevinbburns 2
  3. 3. My work history and experience, @kevinbburns 3
  4. 4., @kevinbburns 4
  5. 5. Peace Corp Recruitment and Public Affairs Story of how we used technology to improve Peace Corp recruitment. Switch from USPS to email Switch from manual data entry to wild-card search in gopher email system and screen scraping results, an early for of ETL. Conduct direct email campaigns when spam still meant ‘meat in a can’, @kevinbburns 5
  6. 6., @kevinbburns 6
  7. 7. Open Discussion Who’s measuring value, outcomes, and/or impacts today? How are you measuring them? If you’re not measuring them, why not? Who’s measuring cost? How are you measuring cost?, @kevinbburns 7
  8. 8. The 1st Agile Principle Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. (should we change valuable to beneficial impact?) How do we define value (impact) and how do we measure it? Not all Projects (or Features) are created equal., @kevinbburns 8
  9. 9. Is value determined by delivery on time, on budget, and on scope? Is the scope delighting the customer? Are they using everything we delivered?, @kevinbburns 9
  10. 10. In a survey of 4 products, 65% of the features were rarely or never used. How much money could have been saved if we never built them? In the Waterfall project world, we have to ask for everything we can think of because capital will end at the end of the project. Instead we should be asking what has the most value in terms of the business outcome and/or impact and how are we going to measure it., @kevinbburns 10
  11. 11., @kevinbburns 11
  12. 12., @kevinbburns 12
  13. 13. Marty Cagan Quotes • Customers don’t know what they want. It’s very hard to envision the solution you want without actually seeing it. • At least 2/3 of our ideas are never going to work. The other 1/3 will take 3-4 iterations to get right. • The role of the product manager is to discover a product that is valuable, usable, and feasible. Product, design, and engineering work together to arrive at optimal solution., @kevinbburns 13
  14. 14., @kevinbburns 14
  15. 15. What we measure is changing Business Customer PO, SM, BL Software Engineering AD, DD, DA User UX, BA, QA, SME Business Valuable Design Usable Technically Feasible INNOVATIVE SOLUTION, @kevinbburns 15
  16. 16. Lean Startup • Are we asking what are Minimum Viable (Valuable) Product and how do we know when we’ve delivered it? • Use a scientific method to measure, learn and pivot or preserver. • Use meaningful quantitative objective measure to evaluate impact. • Can you use A/B testing?, @kevinbburns 16
  17. 17. MVP Innovation User UX, BA, QA, SME Business Valuable Design Usable Software Engineering AD, DD, DA Business Customer PO, SM, BL Use scientific method (measurable) to learn and discovery your Minimum Viable (Valuable) Product (MVP) Technically Feasible MVP innovations emerge from Conversations, @kevinbburns 17
  18. 18. Value and/or Impact driven culture • Are we measuring the Cost vs Benefit at all levels of our work items? • Portfolio • Program • Project • Feature/Capability • Story/Requirement • Tasks/Test • Are we measuring the Impact our features have on our customers? • The act of sizing helps us define done and value • Use story telling and test statements create understanding of value and DoD, @kevinbburns 18
  19. 19. Don Reinertsen’s Seven Big Ideas 2nd Generation Lean Product Development 1. Understand your economics 2. Manage your queues 3. Exploit variability 4. Enable smaller batches 5. Control WIP and start rates 6. Prioritize based on economics 7. Accelerate feedback, @kevinbburns 19
  20. 20. 1. Understand Your Economics • In product development all difficult decisions involve multiple variables. • Making decisions that affect multiple variables requires quantification. • Doing quantification, isn’t as hard as you might think., @kevinbburns 20
  21. 21. A Typical Question Should we operate our testing process at 80% utilization with a 2-week queue, or 90% utilization with a 4 week queue?, @kevinbburns 21
  22. 22. Making Economic Decisions • Waste • Cycle time • Variability • Efficiency • Revenue • Unit Cost • Value-Added Proxy Variable Space Transformations Life Cycle Profits Economic Space, @kevinbburns 22
  23. 23. Modeling Process Model Expense Overrun Model Cost Overrun Model Value Shortfall Model Schedule Delay Model Risk Change Create Baseline Model Determine Total Profit Impact of Missing a MOP Calculate Sensitivity Factors, @kevinbburns 23
  24. 24. Guidelines for decision-making • Set strategic goals/guidelines for decision-making so low-level tactical decision can be decentralized while still being aligned within strategic goals/objectives. • While good intentioned, centralized decision-making is often slow and suboptimal because it lacks the context of all the variable at play at tackle level, @kevinbburns 24
  25. 25. 2. Manage Your Queues • Many product developers assume higher utilization leads to faster development. • They neither measure nor manage the invisible queues in their process. • Consequently, they underestimate the true cost of overloading their processes. • Such overloads severely hurt all aspects of development performance., @kevinbburns 25
  26. 26. Effect of Capacity Utilization 10 20 30 40 50 60 70 80 90 100 05101520 QueueSize % Capacity Utilization, @kevinbburns 26
  27. 27. Managing Queues Cost of Excess Delay Total Cost Dollars Excess Product Development Resources Minimize Total Cost to Maximize Profits, @kevinbburns 27
  28. 28. Why Queues Matter • Queues Create… • Longer cycle time • Lower Quality • More variability • Increased risk • More overhead • Less motivation Managing queues is the key to improving product development economics, @kevinbburns 28
  29. 29. 3. Exploit Variability • In manufacturing it is always desirable to reduce variability • In product development eliminating variability eliminates innovation • We must understand the specific conditions that make variability valuable and manage our process to create these conditions • We need development process that function in the presence of variability, @kevinbburns 29
  30. 30. Asymmetric Payoff and Option Pricing Expected Price Payoff vs Price Expected Payoff ExpectedPayoff Price PricePrice Probability Payoff Strike Price Strike Price, @kevinbburns 30
  31. 31. Higher Variability Raises the PayoffExpectedPayoff Price Strike Price Payoff SD = 15 Payoff SD = 5 Option Price = 2, Strike Price = 50, Mean Price – 50, Standard Deviation = 5 and 15, @kevinbburns 31
  32. 32. 4. Enable Smaller Batches • When work products are invisible, batch sizes are invisible • When batch sizes are invisible, product developers pay little attention to them • Many companies institutionalize large batch sizes • Batch size reduction is attractive because it is fast, easy, cheap, granular, leveraged, and reversible • It is a great starting point for LPD Batch Size Queues Cycle Time X 0.5 X 0.5 X 0.5, @kevinbburns 32
  33. 33. Drawing Review Process 200 10 Weeks 20 1 Week Unreviewed Drawings Large Batch Small Batch, @kevinbburns 33
  34. 34. Benefits of Small Batch Testing Higher ValidityFewer Open Bugs Faster Cycle Time Early Feedback Less Debug Complexity More Efficient Debugging More Uptime Smaller Change Fewer Status Reports Less Requirement Changes Faster Learning Lower Cost Changes Cheaper Debugging Cheaper Testing Less Non-Value-Added Better Code Cheaper Correction Better Economics, @kevinbburns 34
  35. 35. Setting Batch Size Transaction Cost Cost Items per Batch Economic Batch Size 1 2 3 4 5 6 7 8 9 10 05101520 Total Cost, @kevinbburns 35
  36. 36. 5. Control WIP and Start Rates • Many developers incorrectly assume that the sooner they start work, the sooner they will finish it • They are constantly tempted to start too much work • This dilutes resources and causes long transit time through their processes • A long transit time hurts efficiency, quality and responsiveness, @kevinbburns 36
  37. 37. Little’s Formula • By constraining WIP in development processes we can control cycle time • This approach, which is known as Lean Kanban, is currently growing rapidly in software development MeanResponseTime = MeanNumberInSystem / MeanThroughput, @kevinbburns 37
  38. 38. Control Number of Active Projects 1 2 3 4 1 2 3 4 COD Savings of Project 1 and 2 Late Start Advantages for Project 3 and 4 Time to Deliver Time to Deliver Time to Deliver, @kevinbburns 38
  39. 39. Avoid Long Planning Horizons • The further out you plan, the less likely your forecast will be accurate • Don’t do detailed analysis on things beyond a quarter • Market conditions change everyday, this can change requirements • Changing requirements cause churn (waste), @kevinbburns 39
  40. 40. Visual WIP Control Boards Ready Queue Coding Ready to Test Testing Done WIP constraints = 10 13 14 15 16 11 10 9 8 7 6 5 4 3 2 112 ?, @kevinbburns 40
  41. 41. 6. Sequence Work Correctly • The sequence in which work is processed is called the queuing discipline • By changing the queuing discipline we can reduce the cost of a queue without decreasing the size of the queue • Since manufacturing has homogeneous flows it always uses FIFO (First-In-First-Out) • For the non-homogeneous flows of product development other approaches have better economics, @kevinbburns 41
  42. 42. Use FIFO for Homogeneous Flow First-In First-Out Cost of Delay 1 2 3 A B Time Cost Delay Cost Last-In First-Out Cost of Delay 1 2 3 A B Time Cost Project Duration Cost of Delay 1 3 3 2 3 3 3 3 3, @kevinbburns 42
  43. 43. Weighted Shortest Job First (WSJF) for Non-homogenous flow High Weight First Cost of Delay 1 2 3 A B Time Cost Delay Cost Low Weight First Cost of Delay A B Time Cost Project Duration Cost of Delay Weight = COD/Duration 1 1 10 10 2 3 3 1 3 10 1 0.1 1 2 3 160 7 96 % Reduction in COD, @kevinbburns 43
  44. 44. 7. Create Faster Feedback • When queues and batch sizes are large feedback is slow • Slow feedback hurts quality, efficiency, and cycle time • Feedback speed has enormous economic leverage in product development, but it is rarely explicitly managed, @kevinbburns 44
  45. 45. The Front-Loaded Lottery • A lottery ticket pays $3000 to winning three digit number • You can pick the number in two ways: • Pay $3 to select all three digits at once • Pay $1 for the first digit, find out if it is correct, then choose if you wish to pay $1 for the second digit, and then choose if you wish to pay $1 for the third digit., @kevinbburns 45
  46. 46. Value of Feedback 100% Spend $1 Savings = $0.90 Savings = $0.99 10% 1% 0 $1 $2 $3 Probability of Occurrence Cumulative, @kevinbburns 46
  47. 47. ? @kevinbburns 612-396-7724, @kevinbburns 47
  48. 48. References • Don Reinertsen • • =1789&creative=9325&creativeASIN=1935401009&linkCode=as2&tag=reinertas soci-20&linkId=U56AHKXXQVN4VZFY •, @kevinbburns 48