Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Software Design Class (Session 5): Finding the Points of Light- Actionable Experimentation

955 views

Published on

These slides are for the software design class I teach (currently at UVA Darden).

The materials page is here: bit.ly/sw-class .

Published in: Technology
  • Be the first to comment

Software Design Class (Session 5): Finding the Points of Light- Actionable Experimentation

  1. 1. © 2015 COWAN+ SOFTWARE DESIGN: SESSION 5 FINDING THE POINTS OF LIGHT- 
 ACTIONABLE EXPERIMENTATION Alex Cowan
  2. 2. © 2015 COWAN+ THE VENTURE DESIGN PROCESS SCALE? PIVOT? PRODUCT & PROMOTION USER STORIES & PROTOTYPES CUSTOMER DISCOVERY & EXPERIMENTS VALUE PROPOSITIONS & ASSUMPTIONS SHOW M E … ? WHATIF? Is the problem relevant? Is the proposition better vs alternatives? Do we understand this person? What makes them tick? Did the implementation deliver on the story? Was the implemented story relevant to the proposition? How did the customer /user react? WHO?PERSONAS WHAT? PROBLEM SCENARIOS & ALTERNATIVES
  3. 3. © 2015 COWAN+ WHAT WE COVERED LAST TIME VALUE PROPOSITIONS & ASSUMPTIONS
  4. 4. © 2015 COWAN+ HYPOTHESIS-DRIVEN INNOVATION VIA ‘LEAN STARTUP’ Do I have real evidence from my buyer that this is compelling? 6.a YES results disprove hypothesis 01 IDEA! 02 HYPOTHESIS 03 EXPERIMENTAL DESIGN 04 EXPERIMENTATION 05 REVISE? 6.b NO we appear to have a valid hypothesis What is our ‘value hypothesis’ and what are its key assumptions? 6.a YES results disprove hypothesis 01 IDEA! 02 HYPOTHESIS 03 EXPERIMENTAL DESIGN 04 EXPERIMENTATION 05 REVISE? 6.b NO we appear to have a valid hypothesis How do I definitely prove or disprove the assumptions with a minimum of time and effort? 6.a YES results disprove hypothesis 01 IDEA! 02 HYPOTHESIS 03 EXPERIMENTAL DESIGN 04 EXPERIMENTATION 05 REVISE? 6.b NO we appear to have a valid hypothesis 6.a YES results disprove hypothesis 01 IDEA! 02 HYPOTHESIS 03 EXPERIMENTAL DESIGN 04 EXPERIMENTATION 05 REVISE? 6.b NO we appear to have a valid hypothesis Am I reacting or am I focused on validating my pivotal assumptions? ‘Pivot or persevere?’ VALUE HYPOTHESISsource: adapted from ‘The Lean Startup’
  5. 5. © 2015 COWAN+ WORKING YOUR PRODUCT HYPOTHESIS … and they have a certain PROBLEMS(S) … where they’re currently using certain ALTERNATIVE(S) … and I have a VALUE PROPOSITION that’s better enough than the alternatives to cause the persona to act (purchase, use, etc.). A certain VALUE HYPOTHESIS QUESTIONS How much better than the best alternative is your product? How obvious is that to the customer? 
 TOOLS Structured Assumptions Structured Experiments (including MVP)
  6. 6. © 2015 COWAN+ WHERE YOU SHOULD BE NOW VALUE PROPOSITIONS & ASSUMPTIONS Do we understand this person? What makes them tick? WHO?PERSONAS WHAT? PROBLEM SCENARIOS & ALTERNATIVES – Personas, including Think- See-Feel-Do (Converged) – Problem Scenarios – Alternatives – Value Propositions (Converged) Do we understand this person? What makes them tick? WHO?PERSONAS WHAT? PROBLEM SCENARIOS & ALTERNATIVES – Interview Guide – 2 (or more) Interviews – Product Hypothesis (Converged) – Value Hypothesis – Assumptions for Value Hypothesis
  7. 7. © 2015 COWAN+ WHAT WE’LL COVER TODAY CUSTOMER DISCOVERY & EXPERIMENTS VALUE PROPOSITIONS & ASSUMPTIONS WHATIF?
  8. 8. © 2015 COWAN+ WHAT WE’LL WORK ON TODAY Finding the Right PROBLEM Finding the Right SOLUTION time alternatives divergence convergence divergence convergence source: adapted from ‘The Design of Everyday Things’ Personas Problems Alternatives Field Discovery Value Hypothesis & Assumptions Product Hypothesis Experiments on Motivation
  9. 9. © 2015 COWAN+ UNBUNDLING THE VALUE HYPOTHESIS INTO ASSUMPTIONS MVP Archetype Example Assumptions (Enable Quiz) Wizard of Oz If we get HR managers to a landing page with a demo, 10% will sign up for our email product announcements. Concierge If we create position-specific quizzes for HR managers, they’ll use them ~100% of the time and, after two positions, be willing to pay. Sales If we offer the service at [x] price with [y] supplemental assistance, companies that hire a lot of engineers will pay [z]. If we [do something] for [persona], they will [respond in a certain way]
  10. 10. © 2015 COWAN+ UNBUNDLING THE VALUE HYPOTHESIS INTO ASSUMPTIONS MVP Archetype Example Assumptions (Enable Quiz) Wizard of Oz If we get HR managers to a landing page with a demo, 10% will sign up for our email product announcements. Concierge If we create position-specific quizzes for HR managers, they’ll use them ~100% of the time and, after two positions, be willing to pay. Sales If we offer the service at [x] price with [y] supplemental assistance, companies that hire a lot of engineers will pay [z]. If we [do something] for [persona], they will [respond in a certain way]
  11. 11. © 2015 COWAN+ YOUR VENTURE, THE EXPERIMENT Component Notes What assumption will this test? How? What is/are the pivotal metric(s)? What is the threshold for true (validated) vs. false (invalidated)? What will you do next if the result is true? False? How much time, money will it take to set up? Roughly, what will it take for each individual test? Subsequent tests? Roughly, how long will it take for each test to run and produce definitive, actionable results?
  12. 12. © 2015 COWAN+ EXAMPLE: ENABLE QUIZ CONCIERGE MVP Component Notes What assumption will this test? How? If we create position-specific quizzes for HR managers, they’ll use them ~100% of the time and, after two positions, be willing to pay. If the HR managers use the quiz, they’ll send through <1/2 as many candidates. If we offer the service at [x] price with [y] supplemental assistance, companies that hire a lot of engineers will pay [z]. What is/are the pivotal metric(s)? What is the threshold for true (validated) vs. false (invalidated)? [quizzes administered] / [candidates interviewed] comparison on candidates screened out (historical) ability to sell What will you do next if the result is true? False? True: Build a simple1.0 1 & 2 True: Look at customer creation hyp., value hypothesis False: Consider value hyp., problem hyp. How much time, money will it take to set up? see bit.ly/exp-test Roughly, what will it take for each individual test? Subsequent tests? see bit.ly/exp-test Roughly, how long will it take for each test to run and produce definitive, actionable results? see bit.ly/exp-test
  13. 13. © 2015 COWAN+ EXAMPLE: ENABLE QUIZ ADWORDS MVP Component Notes What assumption will this test? How? If Enable Quiz offers its quizzing solution in the area of [x], customers will subscribe. What is/are the pivotal metric(s)? What is the threshold for true (validated) vs. false (invalidated)? absolute click-through-rate; we’d like to see >=2% relative to CTR (on this basis we’ll select our top 10 topics) What will you do next if the result is true? False? True (along with previous): Build a simple1.0 with selected topics. False (all CTR’s <2%): Revise customer creation plan & test an alternate topic assessment strategy How much time, money will it take to set up? see bit.ly/exp-test Roughly, what will it take for each individual test? Subsequent tests? see bit.ly/exp-test Roughly, how long will it take for each test to run and produce definitive, actionable results? see bit.ly/exp-test
  14. 14. © 2015 COWAN+ EXAMPLE: ENABLE QUIZ CONCIERGE MVP Component Notes What assumption will this test? How? If we create position-specific quizzes for HR managers, they’ll use them ~100% of the time and, after two positions, be willing to pay. If the HR managers use the quiz, they’ll send through <1/2 as many candidates. If we offer the service at [x] price with [y] supplemental assistance, companies that hire a lot of engineers will pay [z]. What is/are the pivotal metric(s)? What is the threshold for true (validated) vs. false (invalidated)? [quizzes administered] / [candidates interviewed] comparison on candidates screened out (historical) ability to sell What will you do next if the result is true? False? True: Build a simple1.0 1 & 2 True: Look at customer creation hyp., value hypothesis False: Consider value hyp., problem hyp. How much time, money will it take to set up? see bit.ly/exp-test Roughly, what will it take for each individual test? Subsequent tests? see bit.ly/exp-test Roughly, how long will it take for each test to run and produce definitive, actionable results? see bit.ly/exp-test Draft a set of experiments to verify your assumptions
  15. 15. © 2015 COWAN+ ASSIGNMENTS 1. Do/Individual: Design (an) Experiment(s) for Your Assumption(s) (30-60 minutes)
  16. 16. © 2015 COWAN+ bit.ly/vdesignExperimentLearn Hypothesize @cowanSF acowan@alexandercowan.com bit.ly/cdhandbook bit.ly/sw-class FINI Check out 
 Venture Design Twitter Get in touch! Use Customer Discovery Handbook View this deck

×