Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

In-Market Experiments: The Science of Placing Small Bets Fast

Corporate ventures face extreme pressure to succeed early and often. From the outset, it’s important to confirm that there is a real pain the venture is solving for and that it makes sense for your organization to solve it. How do you test for a pain point? In-market experimentation.

  • Login to see the comments

In-Market Experiments: The Science of Placing Small Bets Fast

  1. 1. i IN-MARKET EXPERIMENTS: THE SCIENCE OF PLACING SMALL BETS FAST January 2013
  2. 2. Meeting Logistics ›› 35 minute presentation ›› 25 minutes for Q&A ›› Submit questions through QA widget at anytime—some answered during webinar and some post-webinar ›› Slides and replay of event available after webinar ›› Download our first webinar, Design Thinking for the Business Case: http://peerinsight.com/downloads/dt4bcwebinar.pdf ABOUT TODAY’S WEBINAR
  3. 3. 1 Tim Ogilvie is co-founder of Peer Insight. He is keen on helping large organizations create new experiences, services, and business models. Tim is co-author of Designing for Growth but spends most of his time practicing in the field, not writing or teaching. Tim is passionate about entrepreneurship and placing small bets fast. Jessica Dugan is a Senior Design Consultant at Peer Insight with a passion for people and their stories. Drawing on her background in design research and strategy, Jessica uses this passion to collaborate in the creation of innovative services that improve the quality of the life. She holds a Masters of Design and MBA from the Institute of Design in Chicago. TODAY’S PRESENTERS
  4. 4. 2 HOW DO LEAPS OF FAITH OCCUR? Progressive: Rate Comparison “Ticker” (2001) HP: Snapfish 1-hour print (2006) The Hartford: FleetAhead (2009) AARP: Life Reimagined (2011) Will side-by-side quotes drive away customers we want to win? If we refer our customer to a nearby retail photo finisher, do we lose her for life? Will our fleet customers trade privacy for lower insurance premiums? If we offer a different kind of service, can we expand our customer base?
  5. 5. 3 Asked to describe design, Tim Brennan of Apple’s Creative Services drew the following picture: DESIGN THINKING IS OFTEN TREATED AS A MYSTERY ? $
  6. 6. 4 WE SEE DESIGN THINKING AS A PROBLEM-SOLVING PROCESS What is? What if? What wows? What works?? $
  7. 7. 5 OUR BUSINESS CASE METHOD FOLLOWS FIVE STEPS *Adapted from Rita McGrath, “Discovery-driven Planning” 1. Formulate a novel value proposition 2. Define future success as an income statement* 3. Spell out the assumptions ›› What would have to be true? 4. Test the make-or-break assumptions ›› Thought experiments ›› In-market experiments 5. Based on test results, refine the model, refine the value proposition, or re-assess the investment potential of this opportunity
  8. 8. 6 1. Formulate a novel value proposition 2. Define future success as an income statement* 3. Spell out the assumptions ›› What would have to be true? 4. 5. Based on test results, refine the model, refine the value proposition, or re-assess the investment potential of this opportunity 4. Test the make-or-break assumptions ›› Thought experiments ›› In-market experiments OUR BUSINESS CASE METHOD FOLLOWS FIVE STEPS *Adapted from Rita McGrath, “Discovery-driven Planning”
  9. 9. 7 Thought Experiments Hypothetical experiments that use logic and existing data. ›› Envisioned experiences, require imagination ›› Set in the future ›› 2D and 3D In-Market Experiments Experiments conducted in the marketplace with actual partners and customers. ›› Live experiences, meant to feel real (even if some elements are faked) ›› Set in the present ›› 4D, persist over time (even in brief) THERE ARE TWO WAYS TO TEST ASSUMPTIONS
  10. 10. 8 Progressive: Rate Comparison “Ticker” (2001) HP: Snapfish 1-hour print (2006) The Hartford: FleetAhead (2009) AARP: Life Reimagined (2011) REMEMBER THESE LEAPS OF FAITH? Will side-by-side quotes drive away customers we want to win? If we refer our customer to a nearby retail photo finisher, do we lose her for life? Will our fleet customers trade privacy for lower insurance premiums? If we offer a different kind of service, can we expand our customer base?
  11. 11. 9 tantalizing possibility A B PARALYSIS what lies beneath chasm of unintended consequences potential reward POUNCE A B PARALYSIS POUNCE ANOTHER INNOVATOR’S DILEMMA
  12. 12. 10 SIX SINS OF IN-MARKET EXPERIMENTS 1. Hi-Fidelity Believing it has to look and feel real/polished 2. Quantifying Thinking it must yield quantitatively significant (and repeatable) results 3. Going Solo Using only your firm’s resources and IP to stand up the experiment— even if it takes longer 4. Business Model Next Focusing on the user experience and technical solution now and worrying about defensibility and scalability later 5. IP Protection Keeping all the ideas concealed until they are patent-protected 6. 100% Belief Committing to a specific vision of the solution and looking only for signs of success
  13. 13. 11 tantalizing possibility A B PARALYSIS what lies beneath chasm of unintended consequences potential reward POUNCE A B PARALYSIS POUNCE TWO POSSIBLE RESULTS – BOTH BAD 67%Unexplored opportunities 33%Large bets placed slowly (too-big-to-fail)
  14. 14. 12 tantalizing possibility PROBE what lies beneath chasm of unintended consequences potential reward B1 B2 A B PARALYSIS POUNCE THE REMEDY? IN-MARKET EXPERIMENTS
  15. 15. 13 SIN #1: HI-FIDELITY Hi-Fidelity Quantifying Going Solo Business Model Next IP Protection 100% Belief Low-Fidelity Fake-it-before-you-make-it instead of try this
  16. 16. 14 INSTEAD OF HI-FIDELITY TRY LOW-FIDELITY Key Assumption Customers who see a slightly lower price from a competitor will still favor Progressive. PROGRESSIVE 484 AM. FAMILY 562 000 STATE FARM 525 ALLSTATE 644 000 000 000
  17. 17. 15 SIN #2: QUANTIFYING Hi-Fidelity Quantifying Going Solo Business Model Next IP Protection 100% Belief Build One (then 10, then 100) instead of try this
  18. 18. 16 INSTEAD OF QUANTIFYING TRY BUILD ONE Key Assumption Customers who try the kiosk for 2-hour service will return to Snapfish.
  19. 19. 17 INSTEAD OF QUANTIFYING TRY BUILD ONE Tool: Alpha-Beta Planning Table Number  of  Installations Number  of  Participants Dollar  Investment Build  Time Test  Time Fulfillment  Partners Testing  What? Is  it  Scalable? Is  there  a  Hard  Stop? What  is  the  Feature  Set? Is  the  Technology  viable? No none Key  assumptions No Yes Hypothesized  Features 0 $500 5  days 30  days 3  weeks Beta  Feature  Set  + Same ... ... IN-­‐MARKET  EXPERIMENTS Yes  (back-­‐end  platform+year  1  biz  model) Yes Same  as  Beta,  plus No Full  Scale No Specific,  Limited  Features Yes No Key  assumptions Small  only 5  months 9  weeks Branded,  if  possible 150  days  (+option  to  extend) Yes Initial  Feature  Set   Full  Initial  Feature  Set No COMMERCIAL  LAUNCH 30X 30X $5M+ DETAILS ALPHA 2  sites $750k-­‐$1.5M 150-­‐200 THOUGHT  EXPERIMENTS 15-­‐20 $500k 10  sites BETA 0  sites
  20. 20. 18 SIN #3: GOING SOLO Hi-Fidelity Quantifying Going Solo Business Model Next IP Protection 100% Belief Other People’s Money (and IP) instead of try this
  21. 21. 19 INSTEAD OF GOING SOLO TRY OTHER PEOPLE’S MONEY Key Assumption Fleet customers will incur the inconvenience of telematics to get a lower premium.
  22. 22. 20 INSTEAD OF GOING SOLO TRY OTHER PEOPLE’S MONEY Key Assumption Customers will engage in peer- to-peer mentoring in an environment hosted by AARP and its partners.
  23. 23. 21 SIN #4: BUSINESS MODEL NEXT Hi-Fidelity Quantifying Going Solo Business Model Next IP Protection 100% Belief Control Points Nowinstead of try this
  24. 24. 22 UBIQUITOUS CONSTRAINED Free, available to all Commodity Many providers exist Several providers exist Possible to observe + copy Branded + expensive to copy Producible by only a few firms Protected by contractual secrecy Protected by patent Protected by concealment Brand Device Data transmission Display/cues to drivers Display/cues to manager Historical data Customized interventions Re-pricing algorithms Revenue sharing model Architecture control INSTEAD OF BUSINESS MODEL NEXT TRY CONTROL POINTS NOW Tool: Control Point Grid ? ? ?
  25. 25. 23 OTHER COMMON SINS Hi-Fidelity Quantifying Going Solo Business Model Next IP Protection 100% Belief IP Connection Services IP table; 3-ring template instead of try this Innovation Accounting Split tests Reverse income statement instead of try this
  26. 26. 24 RECAP: SIX SINS OF IN-MARKET EXPERIMENTS 1. Hi-Fidelity 2. Quantifying 3. Going Solo 4. Business Model Next 5. IP Protection 6. 100% Belief
  27. 27. 25 BUT THEN THE WORLD LOOKS LIKE THIS tantalizing possibility A B PARALYSIS what lies beneath chasm of unintended consequences potential reward POUNCE A B PARALYSIS POUNCE
  28. 28. 26 INSTEAD: USE IN-MARKET EXPERIMENTS the science of placing small bets fast Hi-Fidelity Quantifying Going Solo Business Model Next IP Protection 100% Belief instead of Low-Fidelity Build One Other People’s Money Control Points Now IP Connection Innovation Accounting try this using these tools! Key assumptions list Alpha-Beta table Control point grid Services IP table 3-ring template Split tests Reverse income statement
  29. 29. 27 IN-MARKET EXPERIMENTS IN PERSPECTIVE IN-MARKET EXPERIMENTS TECHNICAL BUILD + QUANTITATIVE RESEARCH WHAT HOW “building the right thing” “building things right”
  30. 30. TIM OGILVIE togilvie@peerinsight.com 703 314 3123 JESSICA DUGAN jdugan@peerinsight.com 312 532 8729 CHECK OUT OUR BLOG peerinsight.com/musings FOLLOW US ON TWITTER @peerinsight Thank you!

×