Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Qual vs Quant: Using Better Data to Build Better Products

2,213 views

Published on

How to combine qualitative and quantitative data to improve your product development process.

From the Lean Product Meetup on 4/14 in Palo Alto.

Published in: Technology

Qual vs Quant: Using Better Data to Build Better Products

  1. 1. Qual vs Quant Using Better Data to Build Better Products Laura Klein laura@usersknow.com @lauraklein
  2. 2. Who’s Tired Of Hearing About Data?* *be honest
  3. 3. A/B Testing
  4. 4. A/B Testing Funnel Analysis
  5. 5. A/B Testing Cohorts Funnel Analysis
  6. 6. A/B Testing Cohorts Funnel Analysis Blah
  7. 7. A/B Testing Cohorts Funnel Analysis Blah Blah
  8. 8. A/B Testing Cohorts Funnel Analysis Blah Blah Blah
  9. 9. Sorry. I’m Going To Talk About Data Some More.
  10. 10. Better Data == Better Products
  11. 11. Well…Sometimes
  12. 12. What Data Can’t Do Replace Design Replace Listening to Users Tell Us What We Should Be Building
  13. 13. What Data Can Do Inform Design Inform Research Tell Us What We Should Be Investigating
  14. 14. How About An Example?
  15. 15. A Funnel
  16. 16. A Checkout Funnel Create Account Select Plan Payment Info Address Confirm Thanks & Up Sell
  17. 17. A Leaky Checkout Funnel 100% Create Account Select Plan Payment Info Address Confirm Thanks & Up Sell -20% -10% -50% -33% -25% == ???
  18. 18. What That Means to You 100 Buyers Create Account Select Plan Payment Info Address Confirm Thanks & Up Sell -20 Buyers -8 Buyers -36 Buyers -12 Buyers -6 Buyers == 18 Buyers
  19. 19. 100 To 18???? How Do We Fix This?
  20. 20. Identifying a Friction Point 100 Buyers Create Account Select Plan Payment Info Address Confirm Thanks & Up Sell -20 Buyers -8 Buyers -36 Buyers -12 Buyers -6 Buyers == 18 Buyers
  21. 21. Stop It
  22. 22. Quantitative Data Qualitative Data Understanding the Friction Point
  23. 23. Quantitative Data ! WHAT Qualitative Data ! WHY
  24. 24. How Can You Use Them Together?
  25. 25. Step 1: Identify the Biggest Problem
  26. 26. Step 1: Identify the Biggest Problem Use Funnel Metrics
  27. 27. The Biggest Problem 100 Buyers Create Account Select Plan Payment Info Address Confirm Thanks & Up Sell -20 Buyers -8 Buyers -36 Buyers -12 Buyers -6 Buyers == 18 Buyers
  28. 28. Step 1: Identify the Biggest Problem Use Funnel Metrics Quant
  29. 29. Step 1: Identify the Biggest Problem Use Funnel Metrics Step 2: Understand Why
  30. 30. Step 1: Identify the Biggest Problem Use Funnel Metrics Step 2: Understand Why Observational Usability
  31. 31. Step 1: Identify the Biggest Problem Use Funnel Metrics Step 2: Understand Why Observational Usability Qual
  32. 32. Problem Possibilities Create Account Select Plan Payment Info Address Confirm Thanks & Up Sell -36 People • Didn’t have payment info handy • Confused by specific question • Looked for promo code
  33. 33. Step 1: Identify the Biggest Problem Use Funnel Metrics Step 2: Understand Why Observational Usability Step 3: Propose Solutions
  34. 34. Step 1: Identify the Biggest Problem Use Funnel Metrics Step 2: Understand Why Observational Usability Step 3: Propose Solutions Create Solution Hypotheses
  35. 35. Solution Hypothesis ! Problem People looked for a promo code.
  36. 36. Solution Hypothesis ! Possible Solution Remove promo code entirely.
  37. 37. Solution Hypothesis ! What Do We Think Would Happen? An increase in conversion.
  38. 38. Solution Hypothesis ! What Might Go Wrong? A decrease in acquisition or revenue.
  39. 39. Step 4: Learn & Iterate* *The last step is always learn & iterate! Quant & Qual
  40. 40. Learn… ! How Would We Know We Were Right? A/B test of the checkout flow with and without the promo code.
  41. 41. What Really Happened? Create Account Select Plan Payment Info Address Confirm Thanks & Up Sell
  42. 42. Why Did It Happen? Qual!
  43. 43. …And Iterate ! Another Possible Solution Create custom landing pages rather than promo codes.
  44. 44. Step 1: Identify the Biggest Problem Use Funnel Metrics Step 2: Understand Why Observational Usability Step 3: Propose Solutions Create Solution Hypotheses Step 4: Learn & Iterate!
  45. 45. Or…
  46. 46. Step 1: Identify the Biggest Problem
  47. 47. Step 1: Identify the Biggest Problem Interview Current Users Qual
  48. 48. Step 1: Identify the Biggest Problem Interview Current Users Step 2: Understand What
  49. 49. Step 1: Identify the Biggest Problem Interview Current Users Step 2: Understand What Study Relevant Metrics Quant
  50. 50. But Why?
  51. 51. Step 1: Identify the Biggest Problem Interview Current Users Step 2: Understand What Study Relevant Metrics Step 3: Propose Solutions Create Solution Hypotheses
  52. 52. Step 1: Identify the Biggest Problem Look at Your Sales Pipeline Step 2: Understand Why Interview People Who Said No Step 3: Propose Solutions Create Solution Hypotheses
  53. 53. Step 4: Learn & Iterate* *The last step is always learn & iterate!
  54. 54. Once Again… Quantitative Data ! WHAT Qualitative Data ! WHY
  55. 55. Better Data == Better Products
  56. 56. Questions? Laura Klein @lauraklein laura@usersknow.com
  57. 57. The Velocity Conference
 Santa Clara, May 27-29 Join us for 3 days of keynotes, sessions and training and learn to optimize your business end-to-end with speakers like Jez Humble, Gary Gruver, Charity Majors & more. Use discount code LKLEAN20 during registration for 20% off All-access, 3-day & 2-day passes http://velocityconf.com/ velocity2015 Bringing a team? Contact velocityreg@oreilly.com for up to 30% off.

×