Recommended
More Related Content
What's hot
What's hot (7)
Similar to Evidence driven development - a lean methodology to product development
Similar to Evidence driven development - a lean methodology to product development (20)
Recently uploaded
Recently uploaded (20)
Evidence driven development - a lean methodology to product development
- 1. Joshuah Vincent @joshuahvincent joshuahv.com Evidence Driven DesignA lean product development methodology
- 2. Why Evidence Driven Design (EDD)
- 3. Using EDD helps ensure we have conversations around value to build features that have greatest relative impact. All the stuff we could build
- 4. High level Product/Feature Description Product Description Sheet Product Description Score Roadmap
- 5. Make decisions with data Use data where you have it and identify your assumptions Measure the return on investment for what you are building Run experiments to turn assumptions into foundations EDD VISION
- 6. Product Description Overview of the product idea Score Calculating ROI of a product Roadmap Activities needed to build and ship the product EDD Process Overview
- 8. Product Description Sheet Customer Who is this for? For whom are we creating value?
- 9. Product Description Problem Whats the top problem (or 3) facing this customer segment?
- 10. Product Description Value Proposition How are we uniquely solving their need? Compelling message that states why we are worth buying
- 11. Product Description Solution Top ways we will solve the problems
- 12. Product Description Key Business Success Metrics (KPIs): How do we measure success?
- 13. Product Description Early adopter: How Who is the potential early user of the solution?
- 14. Product Description Alternatives How are users solving this today?
- 15. Product Description Channels How are going to acquire customers? Drive awareness? Keep users?
- 16. Product Description Key Stakeholders Whose buy in do we need? Who are key influences to stakeholders?
- 17. Product Description Key Resources/Partners Critical internal and external resources need to deliver soln.
- 18. Product Description Revenue/Business value What’s the value to the business for delivering the product?
- 19. Product Description Cost structure Resources/expenses for building and maintaining product.
- 20. Defining Product/Feature in Terms of ROI EDD Score EDD Product Description: Overview of the product idea. Similar to an exec summary. Score Spreadsheet for calculating ROI of a product EDD Roadmap: Includes all activities needed to build and ship the product.
- 21. Relative indicator of ROI for a feature/product over the long term in $/effort. Relies on assumptions so not perfect or precise. Used as metric to compare initiatives side by side. Way to prioritize by generating a value that can be used to rank features. What is an EDD Score?
- 22. 6 Metrics to calc. EDD Score Impact Per User Customer Acquisition Cost Churn Rate Cost to Serve Months to ReleaseCost to Build
- 23. Start Small Calculating the EDD score doesn’t have to take a lot of time, invest only enough to make the next decision. Do the Very Simple exercise If there is enough yield there, do the Simple and/or Moderate exercises Do the Detailed exercise to fully comprehend the project compared to other initiatives
- 24. Very Simple - 5 min
- 25. Simple - 10 minCritical Numbers Impact per user (monthly) $13.00 Customer Acquisition Cost $0.00 Staff needed 7 Months of development before release 6 Churn rate (monthly) 2.0% Cost to serve (monthly) $2.00 Customers in Year 3 300 EDD Score 34 7 critical numbers Estimate when no data is available
- 26. Moderate - 30 min+ Consider the customer growth Apply an attach rate for each year to the user numbers Will attach change over time? Will critical numbers change over time? Critical Numbers Impact per user $13.00 Customer Acquisition Cost $0.00 Churn rate (monthly) 2.00% Cost to serve $4.00 Cost to build $697,200 Months of development before release 6 Customers Customers in Year 1 9 Customers in Year 2 60 Customers in Year 3 300 Customers in Year 4 1,200 Customers in Year 5 4,800 Customers in Year 6 14,400 Customers in Year 7 43,200 Customers in Year 8 129,600 Customers in Year 9 129,600 Customers in Year 10 129,600 EDD Score 34
- 27. Name of Product Score Product A 533 Product B 483 Feature A 160 Feature B 150 Product D 118 Product E 37 Compare Scores
- 28. Most of the value is in those conversations
- 29. Can you trust the EDD Scores?
- 30. Turning Your Gut Feel into Data EDD ROADMAP Product Description Overview of the product idea. Similar to an exec summary. Score Spreadsheet for calculating ROI of a product Roadmap Includes all activities needed to build and ship the product.
- 31. With the EDD score sheet and product description complete, build your case. Create experiments for each metric Scope each experiment Do the ones with the best yield first Product Experiment Roadmap Gaining confidence in the numbers you have placed in your EDD Score
- 32. Validating your assumptions Hypothesis Experiment Results Conclusions Target your metrics Document & discuss your findings Confirm/change your assumptions. Publish findings. Create and run experiments
- 33. Start with your gut, decide with data Hypothesis Experiment Results Users will value and pay for a bookkeeper on demand. Ask 50 customers, bookkeepers and partners if they would be willing to pay for this service. 27 of 50 respondents said they would be willing to pay for online support from a bookkeeper for up to 60 minutes. Conclusions Now that we know a realistic attach rate, we can better calculate the ROI on this effort.
- 34. Anatomy of an experiment What is <our cost to build, revenue per user, etc.> We will <describe experiment> Then we will see if <Hypothesis> We believe <Predict Result> This means … and we can be more / less confident in our critical question Critical Question Experiment Observe Hypothesize Result
- 35. Experiment Definition 1. What we will do to conduct the experiment. 2. The data that we will gather and how we will analyze it. 3. What we expect the data to show. 4. How we will interpret the data if it is what we expect. "We will....." "Then we will see if…” "we believe...." "and this means...." + "...so we can be more confident <critical question>
- 36. 1. We will research competitors Ys pricing offering for feature X 2. Then we will see if our planned pricing is less expensive. 3. We believe our planned pricing is less than theirs... 4. and this means that customers will pay $1 per for transaction so we can be more confident in our monthly revenue per user critical number. Experiment Definition 4 key sections
- 37. Certainty Hours Lag Total Cost Cost of Certainty Pricing/costing experiments Units 1 - 10 Hours Days Hours * $100 + Days * $1,000 Total Cost / Certainty
- 38. Execute in order of Cost of Certainty
- 39. Idea Designed Working Waiting on Data Doc. Findings Debate Findings Publish Findings Experiment Process
- 40. Tracking Experiments LHlhjkTeam Experiment Board ` ` `
- 41. Document, Debate, Share Results - What was the data you generated? Experiment Quality - Did you gain any useful data? Budget - Did it take you as long as you expected? Confidence Increase - Did your confidence go up? How much? Share the results on company intranet if interesting or insightful Did the model change?