Designing To Learn: Creating Effective MVP Experiments - Lean UX NYC 2014

23,334 views

Published on

Designing MVP workshop for Lean UX NYC 2014.

Published in: Design, Technology, Business

Designing To Learn: Creating Effective MVP Experiments - Lean UX NYC 2014

  1. 1. Designing to Learn Creating Effective MVP Experiments Melissa Perri @lissijean
  2. 2. Consultant Lean Product Manager UX Designer Teacher Twitter addict (@lissijean) Blogger (melissaperri.com) Hi! I’m Melissa.
  3. 3. @lissijean • What is an MVP (and why should we care)? • Setting Up MVP Experiments • Types of MVPs and How to Choose • Iterating on MVPs • Agile + MVP: How does it work together? Agenda
  4. 4. @lissijean Why MVP?
  5. 5. @lissijean “Match your taste with the exceptional finds of industry insiders.”
  6. 6. @lissijean
  7. 7. @lissijean
  8. 8. My way is faster. This isn’t what I needed. I don’t know how to use it. The information I want to see isn’t on here. @lissijean
  9. 9. @lissijean
  10. 10. @lissijean
  11. 11. @lissijean Validate Before Building
  12. 12. @lissijean Build Measure Learn
  13. 13. @lissijean Iterate your way to Product/Market Fit your users
  14. 14. @lissijean MVPMinimum Viable Product
  15. 15. @lissijean MVPSmallest amount of work you can do to test your idea.
  16. 16. @lissijean
  17. 17. @lissijean
  18. 18. @lissijean
  19. 19. @lissijean
  20. 20. @lissijean LEARN
  21. 21. @lissijean MVP = Minimum Feature Set Agile Methodology MVPs
  22. 22. @lissijean Focus on problem not features.
  23. 23. @lissijean MYTH
  24. 24. @lissijean Crappy products Crappy products everywhere
  25. 25. @lissijean Good MVPs offer a great experience to the customer.
  26. 26. @lissijean
  27. 27. @lissijean MVP = Experiment
  28. 28. @lissijean Experiment Process Set Up ! Design ! Run ! Feedback ! Iterate
  29. 29. @lissijean
  30. 30. @lissijean Setting up MVP Experiments PART I
  31. 31. @lissijean 1. Define Customer & Problem 2. Investigate assumptions 3. Design a test 4. Measure customer behavior 5. Evaluate success 6. Pivot or persevere Process
  32. 32. @lissijean
  33. 33. @lissijean Define Define
  34. 34. @lissijean Customers want twitter-like messages in their feed.
  35. 35. @lissijean Customers want twitter-like messages in their feed. Not a problem.
  36. 36. @lissijean Real Problem Customers don’t think the curators are authentic.
  37. 37. @lissijean Investigate
  38. 38. @lissijean Assumptions Shoppers care that curators are authentic. Shoppers will engage more with real messages on site. Shoppers think tweets are authentic.
  39. 39. @lissijean Riskiest Assumption Shoppers care that curators are authentic. Shoppers will engage more with curator messages on site. Shoppers think tweets are authentic.
  40. 40. @lissijean Design a Test
  41. 41. @lissijean Setting up a test 1. What do we want to learn? 2. How do we measure that? 3. What’s the least we can build to learn?
  42. 42. @lissijean Will shoppers engage more with curator messages? What do we want to learn?
  43. 43. @lissijean Engagement Clicks, purchases, added to cart How do we measure that?
  44. 44. @lissijean What is the least we can build to learn? Wizard of Oz Looks real to the customer, completely manual on the backend.
  45. 45. @lissijean
  46. 46. @lissijean
  47. 47. @lissijean Measure
  48. 48. @lissijean How do we measure engagement?
  49. 49. @lissijean Clicks on product. Purchases. Add to cart.
  50. 50. @lissijean What do we consider successful?
  51. 51. @lissijean Minimum Success Criteria What is the minimum number we need to hit to consider this validated?
  52. 52. @lissijean Time Salary# Peoplex x Cost to build completely
  53. 53. @lissijean Minimum Success Criteria Improvement needed to justify cost to build.
  54. 54. @lissijean 25% increase in engagement in A/B test.
  55. 55. @lissijean
  56. 56. @lissijean Evaluate
  57. 57. @lissijean
  58. 58. @lissijean 0% change over other products.
  59. 59. @lissijean INVALIDATED
  60. 60. @lissijean But why?
  61. 61. @lissijean Why didn’t users engage? What do we want to learn?
  62. 62. @lissijean User Testing Sessions watch them use it. How do we find that out?
  63. 63. @lissijean Emails.
  64. 64. @lissijean Pivot
  65. 65. @lissijean strategic change in direction based on learning. Pivot
  66. 66. @lissijean Riskiest Assumption Shoppers will engage more with curator messages.
  67. 67. @lissijean Will shoppers engage more with curator messages through email? What do we want to learn?
  68. 68. @lissijean Engagement Opens, Click throughs, purchases. How do we measure that?
  69. 69. @lissijean What is the least we can build to learn?
  70. 70. @lissijean How do we determine success? 30% increase in engagement over other emails.
  71. 71. @lissijean Success! 2x Open Rate 3x Purchases
  72. 72. @lissijean Now, we automate. Now, we invest.
  73. 73. @lissijean workshop one: Setting up MVP Experiments
  74. 74. @lissijean
  75. 75. @lissijean We’ve recently received a flurry of complaints from Macy’s shoppers.
  76. 76. @lissijean “Every time I come into Macy’s, I spend HOURS I don’t have and can never seem to find what I’m looking for. I know exactly what type of item I want, but options are all over the place. It’s daunting.”
  77. 77. @lissijean “I needed a blue tie to go with my suit for a friend’s wedding, so I went to find it but there were too many options. Where do I even start? I don’t even know how to choose.”
  78. 78. @lissijean Who is the customer?
  79. 79. @lissijean “OpenSky shoppers who visit the site at least once a month, but rarely buy.”
  80. 80. @lissijean What is their problem?
  81. 81. @lissijean “Do not think the curators are authentic.”
  82. 82. @lissijean 10 minutes In your teams, write down your customer and their problem for Macy’s.
  83. 83. @lissijean Shoppers who are purchasing something specific out of necessity, not browsing. Customer Hypothesis
  84. 84. @lissijean Cannot find what they need efficiently. Problem Hypothesis
  85. 85. @lissijean What are we assuming about the customer and the problem?
  86. 86. @lissijean Riskiest Assumption What are we the most uncertain about?
  87. 87. @lissijean 10 minutes In your teams, write down your assumptions and pick the riskiest one.
  88. 88. @lissijean Assumptions Shoppers are overwhelmed by the amount of options. Shoppers do not want to browse. Shoppers want help finding items.
  89. 89. @lissijean Assumptions Shoppers are overwhelmed by the amount of options. Shoppers do not want to browse. Shoppers want help finding items.
  90. 90. @lissijean Now, we design a test for that riskiest assumption.
  91. 91. @lissijean Types of MVPs PART II
  92. 92. @lissijean WHAT SHOULD YOU ALWAYS DO FIRST?
  93. 93. @lissijean Interviews Get out of the building and talk to people.
  94. 94. @lissijean • First thing you should do. • Talk to at least 10 people. • Verify the problem and the customer hypothesis. • Verify as many assumptions as you can. When to do Interviews
  95. 95. @lissijean Concierge Delivering a service manually. Delivering a service manually. Concierge
  96. 96. @lissijean
  97. 97. @lissijean
  98. 98. @lissijean The Good The Bad • Very close to the customer, easy to learn. • Can adjust easily based on feedback. • Very easy and cheap to get started. • Not scalable. • Time consuming to run. Concierge
  99. 99. @lissijean • If you are testing a service. • Trying to determine what customers will respond to. • Testing and refining with a small number of people. When to use Concierge.
  100. 100. @lissijean Wizard of Oz Looks real, but manual on the back end.
  101. 101. @lissijean
  102. 102. @lissijean Subscription
  103. 103. @lissijean
  104. 104. @lissijean
  105. 105. @lissijean The Good The Bad • Looks exactly real to customers, so very accurate response to if you would build it. • Need to code to get started. • Can’t tell you why customers don’t use it. Wizard of Oz
  106. 106. @lissijean • Running a test with a large audience. • Trying to sell a feature or service before it is built, but pretend it is already there. • If you are concerned about sacrificing brand. When to use Wizard of Oz.
  107. 107. @lissijean Landing Page A pitch of your product, to gauge idea reception.
  108. 108. @lissijean
  109. 109. @lissijean
  110. 110. @lissijean The Good The Bad • Just a pitch, don't need to refine the whole idea • Can launch in a matter of minutes • MOST PEOPLE FORGET THEY NEED TO DIRECT PEOPLE HERE THROUGH MARKETING. Landing Page
  111. 111. @lissijean • Looking for initial customers to test. • Trying to gauge how much interest there is in the idea. • Great way to pitch to a large audience. When to use Landing Page.
  112. 112. @lissijean Videos Demo what your product can do through hacking it together for a video.
  113. 113. @lissijean
  114. 114. @lissijean The Good The Bad • Can demonstrate complicated things. • Show a prototype before producing them all. • More complicated to put together. • Needs to be a certain amount done before you can make the video. Videos
  115. 115. @lissijean • If your product does something very easily. • When your product sounds complicated. • When people need to see it to believe it. When to use Videos.
  116. 116. @lissijean What if I’m not solving a dire need?
  117. 117. @lissijean workshop two: Design your test.
  118. 118. @lissijean You have already interviewed your customer, they have that problem.
  119. 119. @lissijean Now how are you going to solve that problem?
  120. 120. @lissijean Choose an MVP strategy and plan out a test. 30 minutes • What you are trying to learn? • Explain every little detail of the test. • What are you measuring? • Minimum success criteria.
  121. 121. @lissijean Resources Access to 2 Macy’s stores Access to Macy’s online page ! 3 Store Interns $30,000 salary each 10 Sales Associates In Store $40,000 salary each 5 Developers $100,000 salary each 1 Designer $100,000 salary Yourself $100,000 salary
  122. 122. @lissijean Every time you start with a minimum feature set, a kitten dies.
  123. 123. @lissijean Responding to Constraints & Feedback PART III
  124. 124. @lissijean The goal of an MVP is to learn.
  125. 125. @lissijean Customer Feedback
  126. 126. @lissijean
  127. 127. @lissijean
  128. 128. @lissijean Constraints
  129. 129. @lissijean Restricts medium. Restricts resources.
  130. 130. @lissijean workshop three: Alter your test based on constraints.
  131. 131. @lissijean You learned that people only have this problem online. How do you make an online MVP?
  132. 132. @lissijean You learned that people only have this problem in store. How do you make an in person MVP?
  133. 133. @lissijean How would you change your MVP experiment based on these new constraints? 5 minutes
  134. 134. @lissijean MVPs in Agile Teams PART IV
  135. 135. @lissijean Gather Requirements Spec features Design Interfaces Developm ent QA Process Release Waterfall
  136. 136. @lissijean Gather Requirements Spec features Design Interfaces Developm ent QA Process Release Product Manager UX Designer Developer
  137. 137. @lissijean Agile
  138. 138. @lissijean Individuals and Interactions over Processes and Tools
  139. 139. @lissijean Agile Sprints Short, goal oriented sprints formed around a problem running Build, Measure, and Learn.
  140. 140. @lissijean Build Measure Learn
  141. 141. @lissijean Product Manager UX Designer Developer Small, cross-functional teams
  142. 142. @lissijean
  143. 143. @lissijean monday tuesday wednesday thursday friday Reflect & Define Specify Build & Refine Build & Refine Customer Feedback Sprint Structure
  144. 144. @lissijean Problem Exploration MVP Experiments Feature Development
  145. 145. @lissijean MVPs are only a test.
  146. 146. @lissijean “My solution is too obvious, if I just build it people will use it.”
  147. 147. @lissijean Customers will only use your features if it fills a need. Customers will only use your features if it fills a need.
  148. 148. @lissijean “Lean is just for startups, my company is too grown up.”
  149. 149. @lissijean Lean applies to all companies.
  150. 150. @lissijean Melissa Perri ! Want an MVP Guide? melissa@produxlabs.com ! produxlabs.com melissaperri.com ! ! @lissijean

×