Business Model Innovation by Experimentation

1,292 views

Published on

How to maximize learning and minimize risk

All new products start as a series of unvalidated assumptions. The most critical assumptions are usually implicit and relate to the purpose of the product and the value it is intended to deliver. The more key assumptions involved, the greater the risk. It is enough to have 7 key assumptions about which you are 90% certain for the combined odds of success to be below 50%.

Contrary to popular belief, when we know very little about a situation, it only takes a small amount of new data to realise significant insights.

Unfortunately, people often underestimate the value of information and misunderstand risk. As Product Owners we are often afraid to test our assumptions. We routinely pile on additional risk without a second thought.

Risk management is the bread and butter of the finance and insurance industries. Isn't it time we evolved?

In this fast paced and practical session we will explore answers to the following questions:
- What is risk and how do we quantify and manage it?
- How do we assess the value of information?
- How can experimentation reduce risk and where does it fit in the product development cycle?
- What makes a good experiment?
- How to run experiments in a cost effective manner?
- What are good metrics?

Published in: Business, Economy & Finance
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,292
On SlideShare
0
From Embeds
0
Number of Embeds
127
Actions
Shares
0
Downloads
20
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Business Model Innovation by Experimentation

  1. 1. Business Model Innovation by Experimentation Yoav Aviram Energized Work @yobo © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  2. 2. IT Projects 60% don’t meet schedule, budget or quality goals - IBM 50% failed to achieve what they set out to achieve - KPMG 17% go so badly that they can threaten the very existence of the company - Calleam Consulting © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  3. 3. Innovative IT Projects (startups) 95% of technology startups fail - Allmand Law 93% of Angel investments never achieve expected ROI - University of Washington 80% of VC investments never achieve expected ROI - National VC Association © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  4. 4. WHY? © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  5. 5. Too late Poor quality Missing functionality Software, the Tip of the Iceberg © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  6. 6. An iterative and incremental approach to development, where requirements and solutions evolve through collaboration between selforganizing, cross-functional teams - Wikipedia Agile © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  7. 7. ...the Product Iceberg Who is the customer? How do we acquire new users? How do we convert users to paying customers? How should the product look and behave? How is our product used? What problems does the product attempt to solve? What features provide the greatest value? © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  8. 8. A combination of business-hypothesis-driven experimentation, iterative product releases, and "validated learning" - The Lean Startup, Eric Ries Lean Startup © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  9. 9. ...the Business Model Iceberg Who are our customers? Who is the competition? How do we reach our customers? What are the revenue streams? What is the investment needed? What is the operational cost? What is our unique selling point? What team members to recruit? What are our business goals? © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  10. 10. A plan for the successful operation of a business, identifying sources of revenue, the intended customer base, products, and details of financing - Oxford Dictionary Business Model © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  11. 11. Business Model Innovation © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM - Business Model Generation, Alexander Osterwalder
  12. 12. There is a poorly met need a reachable market faces There is a solution people are happy to pay for There is a way to package and deliver the solution in a cost effective manner Product-Market Fit © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  13. 13. Think About Risk © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  14. 14. The lack of complete certainty, that is, the existence of more than one possibility. The "true" outcome/state/result/value is not known. - How to Measure Anything, Douglas Hubbard Uncertainty © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  15. 15. A set of probabilities assigned to a set of possibilities Quantifying Uncertainty “There is a 60% chance this market will double in five years” © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  16. 16. A state of uncertainty where some of the possibilities involve a loss, catastrophe, or other undesirable outcome. - How to Measure Anything, Douglas Hubbard Risk © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  17. 17. A set of possibilities each with quantified probabilities and quantified losses Quantifying Risk “There is a 40% chance the proposed oil well will be dry, with a loss of $12 million in exploratory drilling costs” © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  18. 18. The Problem with Risk Technology is used to solve complex problems Our mind is wired to oversimplify complexity (we are not very good at understanding probabilities) We are solution oriented We are optimistic (overconfident) by nature People (grossly) underestimate risk & uncertainty © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  19. 19. Combination of two or more related risks Compound Risk Compound probabilities are (very) counter intuitive © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  20. 20. A contestant on a game show is given the choice of three doors. Behind one door is a new car, behind the others, goats. After a contestant picks a door, the host, who knows what's behind all the doors, opens an unchosen door, which reveals a goat. He then asks the contestant, "Do you want to switch doors?" Monty Hall © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  21. 21. Should the contestant switch doors? © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  22. 22. Yes! The odds of winning are 2 out of 3 if you switch © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  23. 23. This answer is so counter intuitive, most people get it wrong It’s why they made a game show off of it: the trick works every time Compound Probabilities The lesson: We need to give serious consideration to risk We must not rely on our intuition when considering risk & uncertainty © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  24. 24. Risk Management 101 Avoid by eliminating the situation or activity that presents it Transfer through insurance or through other types of contracts Reduce by hedging your bets or reducing uncertainty Retain: because some risks are worth assuming © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  25. 25. Risk Reduction © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  26. 26. Spread your bets on a portfolio of investments instead of one Decide on an investment strategy Split the investment into small incremental bets: abort bad products quickly and ramp up investment in the good ones The Investor Mindset © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  27. 27. Uncertainty Reduction © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  28. 28. Value of Information Information reduces uncertainty Reduced uncertainty improves decisions Improved decisions have observable consequences with measurable value - Information Theory (1948) © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  29. 29. Sources of Information Existing within the company Other’s research Experiments Live product © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  30. 30. A scientific procedure undertaken to make a discovery, test a hypothesis, or demonstrate a known fact - Wikipedia Experiment © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  31. 31. Experimental Discovery © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  32. 32. Prioritising Assumptions Uncertainty around making a decision is high (little existing information) The value of the opportunity, or the cost of a mistake, is high Experimentation is least expensive in terms of cost and time © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  33. 33. Attempt to falsify a hypothesis Are objective, measurable and repeatable Are controlled (variables tested in isolation) Are cost effective Influence a decision Designing Good Experiments Experiments aren't free It's an investment decision Information may be cheaper elsewhere © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  34. 34. A measurement is an observation that results in information (reduction of uncertainty) about a quantity Measurement © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  35. 35. The soft stuff Subjective, messy and hard to quantify Customer surveys, focus groups, interviews But … provide insights about Perceived Value Qualitative Measurements © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  36. 36. Good Qualitative Questions Avoid leading phrases such as "do you agree that..." Make customers part with money or sign up, rather than asking them whether they would Use follow up questions to get to the ‘why?’ © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  37. 37. Start by decide on the right Metric (what to measure) Then set lines in the sand: what success and failure look like Always validate results with qualitative data Quantitative Measurements © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  38. 38. Good Metric Influences a decision Comparative Rates and Ratios are easier to compare and act upon Measure what’s important to your customers © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  39. 39. Putting it all Together The key to the success of IT Projects is better risk management Information reduces uncertainty & risk Experiments are the most effective way to obtain new information Experiments have the biggest impact in a high value, high uncertainty and little data situations Hedge your bets by making small incremental investments Challenge the distinction between business people and techies © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  40. 40. The Lean Startup by Eric Ries Business Model Generation by Alexander Osterwalder How To Measure Anything by Douglas W. Hubbard Lean Analytics by Alistair Croll and Benjamin Yoskovitz Reading List © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM
  41. 41. Questions? Yoav Aviram Energized Work yoav@energizedwork.com @yobo © 2013 ENERGIZED WORK LIMITED - ALL RIGHTS RESERVED - WWW.ENERGIZEDWORK.COM

×