Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How do you know you are delivering value?

163 views

Published on

Most organizations are measuring costs but few are measuring the impact their software changes are having on users and their organizations bottom-line. This talk brings together concepts from different authors to discuss how to measure software delivery value, impact and outcomes.

Published in: Software
  • Be the first to comment

  • Be the first to like this

How do you know you are delivering value?

  1. 1. How do we know we’re delivering value or having the intended impact? Presented by Kevin Burns 1-27-2017 kburns@sagesw.com, @kevinbburns
  2. 2. Open Discussion Who’s measuring value, outcomes, and/or impacts today? How are you measuring them? If you’re not measuring them, why not? Who’s measuring cost? How are you measuring cost? kburns@sagesw.com, @kevinbburns 2
  3. 3. The 1st Agile Principle Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. (should we change valuable to beneficial impact?) How do we define value (impact) and how do we measure it? Not all Projects (or Features) are created equal. kburns@sagesw.com, @kevinbburns 3
  4. 4. Is value determined by delivery on time, on budget, and on scope? Is the scope delighting the customer? Are they using everything we delivered? kburns@sagesw.com, @kevinbburns 4
  5. 5. In a survey of 4 products, 65% of the features were rarely or never used. How much money could have been saved if we never built them? In the Waterfall project world, we have to ask for everything we can think of because capital will end at the end of the project. Instead we should be asking what has the most value in terms of the business outcome and/or impact and how are we going to measure it. kburns@sagesw.com, @kevinbburns 5
  6. 6. kburns@sagesw.com, @kevinbburns 6
  7. 7. kburns@sagesw.com, @kevinbburns 7
  8. 8. kburns@sagesw.com, @kevinbburns 8
  9. 9. Assumptions Challenged • 3 things we wish were true • Customer knows what they want • Developers know how to build it • Nothing will change along the way • 3 things we have to live with • Impact isn’t known until software is used in production • Developers discover how to build it • Many things change along the way kburns@sagesw.com, @kevinbburns 9
  10. 10. Marty Cagan Quotes • Customers don’t know what they want. It’s very hard to envision the solution you want without actually seeing it. • At least 2/3 of our ideas are never going to work. The other 1/3 will take 3-4 iterations to get right. • The role of the product manager is to discover a product that is valuable, usable, and feasible. Product, design, and engineering work together to arrive at optimal solution. kburns@sagesw.com, @kevinbburns 10
  11. 11. kburns@sagesw.com, @kevinbburns 11
  12. 12. What we measure is changing Business Customer PO, SM, BL Software Engineering AD, DD, DA User UX, BA, QA, SME Business Valuable Design Usable Technically Feasible INNOVATIVE SOLUTION kburns@sagesw.com, @kevinbburns 12
  13. 13. Lean Startup • Are we asking what are Minimum Viable (Valuable) Product and how do we know when we’ve delivered it? • Use a scientific method to measure, learn and pivot or preserver. • Use meaningful quantitative objective measure to evaluate impact. • Can you use A/B testing? kburns@sagesw.com, @kevinbburns 13
  14. 14. MVP Innovation User UX, BA, QA, SME Business Valuable Design Usable Software Engineering AD, DD, DA Business Customer PO, SM, BL Use scientific method (measurable) to learn and discovery your Minimum Viable (Valuable) Product (MVP) Technically Feasible MVP innovations emerge from Conversations kburns@sagesw.com, @kevinbburns 14
  15. 15. Impact-Drive Development kburns@sagesw.com, @kevinbburns 15
  16. 16. kburns@sagesw.com, @kevinbburns 16
  17. 17. Value and/or Impact driven culture • Are we measuring the Cost vs Benefit at all levels of our work items? • Portfolio • Program • Project • Feature/Capability • Story/Requirement • Tasks/Test • Are we measuring the Impact our features have on our customers? • The act of sizing helps us define done and what the really valuable work is • Use story telling and test statements create understanding of value and DoD kburns@sagesw.com, @kevinbburns 17
  18. 18. Multi-month Monthly 2-weeks Leadership T-Shirt Sizing X-S 1 Sprint S <1 month M 1-3 months L 3-9 months X-L >9 months Team Planning-Poker Fibonacci Sizing (1,2,3,5,8,13,20,40,100) Team task hours to capacity (2,4,6) Solution Decomposition Sizing Pattern Sizing our Cost The act of sizing helps us understand what’s valuable to deliver. Scope doesn’t grow, our understanding does. kburns@sagesw.com, @kevinbburns 18
  19. 19. Deliver 100% of 10% of Project • Can we incrementally deliver value and test it’s impact? • Can we create incremental release plans to deliver 100% of 10% of project? • What constraints do we have in working this way? • Can we overcome or work within these constraints and still deliver incrementally? • What/who is preventing this approach? kburns@sagesw.com, @kevinbburns 19
  20. 20. Create Faster Feedback • When queues and batch sizes are large feedback is slow • Slow feedback hurts quality, efficiency, and cycle time • Feedback speed has enormous economic leverage in product development, but it is rarely explicitly managed kburns@sagesw.com, @kevinbburns 20
  21. 21. The Front-Loaded Lottery • A lottery ticket pays $3000 to winning three digit number • You can pick the number in two ways: • Pay $3 to select all three digits at once • Pay $1 for the first digit, find out if it is correct, then choose if you wish to pay $1 for the second digit, and then choose if you wish to pay $1 for the third digit. kburns@sagesw.com, @kevinbburns 21
  22. 22. Value of Feedback 100% Spend $1 Savings = $0.90 Savings = $0.99 10% 1% 0 $1 $2 $3 Probability of Occurrence Cumulative Investmentkburns@sagesw.com, @kevinbburns 22
  23. 23. Sequence Work Correctly (Cost of Delay) • The sequence in which work is processed is called the queuing discipline • By changing the queuing discipline we can reduce the cost of a queue without decreasing the size of the queue • Since manufacturing has homogeneous flows it always uses FIFO (First-In-First-Out) • For the non-homogeneous flows of product development other approaches have better economics kburns@sagesw.com, @kevinbburns 23
  24. 24. Use FIFO for Homogeneous Flow First-In First-Out Cost of Delay 1 2 3 A B Time Cost Delay Cost Last-In First-Out Cost of Delay 1 2 3 A B Time Cost Project Duration Cost of Delay 1 3 3 2 3 3 3 3 3 kburns@sagesw.com, @kevinbburns 24
  25. 25. Weighted Shortest Job First (WSJF) for Non-homogenous flow High Weight First Cost of Delay 1 2 3 A B Time Cost Delay Cost Low Weight First Cost of Delay A B Time Cost Project Duration Cost of Delay Weight = COD/Duration 1 1 10 10 2 3 3 1 3 10 1 0.1 1 2 3 160 7 96 % Reduction in COD kburns@sagesw.com, @kevinbburns 25
  26. 26. Paul Ellarby example 1. Create Method for measuring Value 2. Understand the value and cost of each portfolio down to the feature level 3. Allocate Value Points across feature/capabilities 4. Track Value vs Cost for each iteration https://www.thoughtworks.com/insights/blog/how-do-you-measure-value kburns@sagesw.com, @kevinbburns 26
  27. 27. How to measure anything – Douglas Hubbard http://www.howtomeasureanything.com/ kburns@sagesw.com, @kevinbburns 27
  28. 28. Measurement Basics • A measurement is an observation that quantitatively reduces uncertainty. Measurements might not yield precise, certain judgments, but they do reduce your uncertainty. • A good object of measurement is something that is clearly defined and it’s observable. • Uncertainty is the lack of certainty: the true outcome/state/value is not known. • Risk is a state of uncertainty in which some of the possibilities involve a loss. • Much pessimism about measurement comes from a lack of experience making measurements. Hubbard, who is far more experienced with measurement than his readers, says: • Your problem is not as unique as you think. • You have more data than you think. • You need less data than you think. • An adequate amount of new data is more accessible than you think. kburns@sagesw.com, @kevinbburns 28
  29. 29. Apply Information Economics for Decision-making 1. Define a decision problem and the relevant variables. 2. Determine what you know. 3. Pick a variable, and compute the value of additional information for that variable. 4. Apply the relevant measurement instrument(s) to the high- information-value variable. 5. Make a decision and act on it. kburns@sagesw.com, @kevinbburns 29
  30. 30. Selecting a measurement method To figure out which category of measurement methods are appropriate for a particular case, we must ask several questions: 1. Decomposition: Which parts of the thing are we uncertain about? 2. Secondary research: How has the thing (or its parts) been measured by others? 3. Observation: How do the identified observables lend themselves to measurement? 4. Measure just enough: How much do we need to measure it? 5. Consider the error: How might our observations be misleading? kburns@sagesw.com, @kevinbburns 30
  31. 31. Tools http://www.howtomeasureanything.com/ kburns@sagesw.com, @kevinbburns 31
  32. 32. Questions & Next Steps • How many of us know what business/user outcomes and impacts we’re trying to achieve on our projects? • Do you have metrics in place to evaluation our progress/success outcomes and impacts? • Who want’s help creating some objective measures? • Where do we go from here? kburns@sagesw.com, @kevinbburns 32

×