Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Designing To Learn: Creating Effective MVP Experiments - Lean UX NYC 2014

32,859 views

Published on

Designing MVP workshop for Lean UX NYC 2014.

Published in: Design, Technology, Business

Designing To Learn: Creating Effective MVP Experiments - Lean UX NYC 2014

  1. 1. Designing to Learn Creating Effective MVP Experiments Melissa Perri @lissijean
  2. 2. Consultant Lean Product Manager UX Designer Teacher Twitter addict (@lissijean) Blogger (melissaperri.com) Hi! I’m Melissa.
  3. 3. @lissijean • What is an MVP (and why should we care)? • Setting Up MVP Experiments • Types of MVPs and How to Choose • Iterating on MVPs • Agile + MVP: How does it work together? Agenda
  4. 4. @lissijean Why MVP?
  5. 5. @lissijean “Match your taste with the exceptional finds of industry insiders.”
  6. 6. @lissijean
  7. 7. @lissijean
  8. 8. My way is faster. This isn’t what I needed. I don’t know how to use it. The information I want to see isn’t on here. @lissijean
  9. 9. @lissijean
  10. 10. @lissijean
  11. 11. @lissijean Validate Before Building
  12. 12. @lissijean Build Measure Learn
  13. 13. @lissijean Iterate your way to Product/Market Fit your users
  14. 14. @lissijean MVPMinimum Viable Product
  15. 15. @lissijean MVPSmallest amount of work you can do to test your idea.
  16. 16. @lissijean
  17. 17. @lissijean
  18. 18. @lissijean
  19. 19. @lissijean
  20. 20. @lissijean LEARN
  21. 21. @lissijean MVP = Minimum Feature Set Agile Methodology MVPs
  22. 22. @lissijean Focus on problem not features.
  23. 23. @lissijean MYTH
  24. 24. @lissijean Crappy products Crappy products everywhere
  25. 25. @lissijean Good MVPs offer a great experience to the customer.
  26. 26. @lissijean
  27. 27. @lissijean MVP = Experiment
  28. 28. @lissijean Experiment Process Set Up ! Design ! Run ! Feedback ! Iterate
  29. 29. @lissijean
  30. 30. @lissijean Setting up MVP Experiments PART I
  31. 31. @lissijean 1. Define Customer & Problem 2. Investigate assumptions 3. Design a test 4. Measure customer behavior 5. Evaluate success 6. Pivot or persevere Process
  32. 32. @lissijean
  33. 33. @lissijean Define Define
  34. 34. @lissijean Customers want twitter-like messages in their feed.
  35. 35. @lissijean Customers want twitter-like messages in their feed. Not a problem.
  36. 36. @lissijean Real Problem Customers don’t think the curators are authentic.
  37. 37. @lissijean Investigate
  38. 38. @lissijean Assumptions Shoppers care that curators are authentic. Shoppers will engage more with real messages on site. Shoppers think tweets are authentic.
  39. 39. @lissijean Riskiest Assumption Shoppers care that curators are authentic. Shoppers will engage more with curator messages on site. Shoppers think tweets are authentic.
  40. 40. @lissijean Design a Test
  41. 41. @lissijean Setting up a test 1. What do we want to learn? 2. How do we measure that? 3. What’s the least we can build to learn?
  42. 42. @lissijean Will shoppers engage more with curator messages? What do we want to learn?
  43. 43. @lissijean Engagement Clicks, purchases, added to cart How do we measure that?
  44. 44. @lissijean What is the least we can build to learn? Wizard of Oz Looks real to the customer, completely manual on the backend.
  45. 45. @lissijean
  46. 46. @lissijean
  47. 47. @lissijean Measure
  48. 48. @lissijean How do we measure engagement?
  49. 49. @lissijean Clicks on product. Purchases. Add to cart.
  50. 50. @lissijean What do we consider successful?
  51. 51. @lissijean Minimum Success Criteria What is the minimum number we need to hit to consider this validated?
  52. 52. @lissijean Time Salary# Peoplex x Cost to build completely
  53. 53. @lissijean Minimum Success Criteria Improvement needed to justify cost to build.
  54. 54. @lissijean 25% increase in engagement in A/B test.
  55. 55. @lissijean
  56. 56. @lissijean Evaluate
  57. 57. @lissijean
  58. 58. @lissijean 0% change over other products.
  59. 59. @lissijean INVALIDATED
  60. 60. @lissijean But why?
  61. 61. @lissijean Why didn’t users engage? What do we want to learn?
  62. 62. @lissijean User Testing Sessions watch them use it. How do we find that out?
  63. 63. @lissijean Emails.
  64. 64. @lissijean Pivot
  65. 65. @lissijean strategic change in direction based on learning. Pivot
  66. 66. @lissijean Riskiest Assumption Shoppers will engage more with curator messages.
  67. 67. @lissijean Will shoppers engage more with curator messages through email? What do we want to learn?
  68. 68. @lissijean Engagement Opens, Click throughs, purchases. How do we measure that?
  69. 69. @lissijean What is the least we can build to learn?
  70. 70. @lissijean How do we determine success? 30% increase in engagement over other emails.
  71. 71. @lissijean Success! 2x Open Rate 3x Purchases
  72. 72. @lissijean Now, we automate. Now, we invest.
  73. 73. @lissijean workshop one: Setting up MVP Experiments
  74. 74. @lissijean
  75. 75. @lissijean We’ve recently received a flurry of complaints from Macy’s shoppers.
  76. 76. @lissijean “Every time I come into Macy’s, I spend HOURS I don’t have and can never seem to find what I’m looking for. I know exactly what type of item I want, but options are all over the place. It’s daunting.”
  77. 77. @lissijean “I needed a blue tie to go with my suit for a friend’s wedding, so I went to find it but there were too many options. Where do I even start? I don’t even know how to choose.”
  78. 78. @lissijean Who is the customer?
  79. 79. @lissijean “OpenSky shoppers who visit the site at least once a month, but rarely buy.”
  80. 80. @lissijean What is their problem?
  81. 81. @lissijean “Do not think the curators are authentic.”
  82. 82. @lissijean 10 minutes In your teams, write down your customer and their problem for Macy’s.
  83. 83. @lissijean Shoppers who are purchasing something specific out of necessity, not browsing. Customer Hypothesis
  84. 84. @lissijean Cannot find what they need efficiently. Problem Hypothesis
  85. 85. @lissijean What are we assuming about the customer and the problem?
  86. 86. @lissijean Riskiest Assumption What are we the most uncertain about?
  87. 87. @lissijean 10 minutes In your teams, write down your assumptions and pick the riskiest one.
  88. 88. @lissijean Assumptions Shoppers are overwhelmed by the amount of options. Shoppers do not want to browse. Shoppers want help finding items.
  89. 89. @lissijean Assumptions Shoppers are overwhelmed by the amount of options. Shoppers do not want to browse. Shoppers want help finding items.
  90. 90. @lissijean Now, we design a test for that riskiest assumption.
  91. 91. @lissijean Types of MVPs PART II
  92. 92. @lissijean WHAT SHOULD YOU ALWAYS DO FIRST?
  93. 93. @lissijean Interviews Get out of the building and talk to people.
  94. 94. @lissijean • First thing you should do. • Talk to at least 10 people. • Verify the problem and the customer hypothesis. • Verify as many assumptions as you can. When to do Interviews
  95. 95. @lissijean Concierge Delivering a service manually. Delivering a service manually. Concierge
  96. 96. @lissijean
  97. 97. @lissijean
  98. 98. @lissijean The Good The Bad • Very close to the customer, easy to learn. • Can adjust easily based on feedback. • Very easy and cheap to get started. • Not scalable. • Time consuming to run. Concierge
  99. 99. @lissijean • If you are testing a service. • Trying to determine what customers will respond to. • Testing and refining with a small number of people. When to use Concierge.
  100. 100. @lissijean Wizard of Oz Looks real, but manual on the back end.
  101. 101. @lissijean
  102. 102. @lissijean Subscription
  103. 103. @lissijean
  104. 104. @lissijean
  105. 105. @lissijean The Good The Bad • Looks exactly real to customers, so very accurate response to if you would build it. • Need to code to get started. • Can’t tell you why customers don’t use it. Wizard of Oz
  106. 106. @lissijean • Running a test with a large audience. • Trying to sell a feature or service before it is built, but pretend it is already there. • If you are concerned about sacrificing brand. When to use Wizard of Oz.
  107. 107. @lissijean Landing Page A pitch of your product, to gauge idea reception.
  108. 108. @lissijean
  109. 109. @lissijean
  110. 110. @lissijean The Good The Bad • Just a pitch, don't need to refine the whole idea • Can launch in a matter of minutes • MOST PEOPLE FORGET THEY NEED TO DIRECT PEOPLE HERE THROUGH MARKETING. Landing Page
  111. 111. @lissijean • Looking for initial customers to test. • Trying to gauge how much interest there is in the idea. • Great way to pitch to a large audience. When to use Landing Page.
  112. 112. @lissijean Videos Demo what your product can do through hacking it together for a video.
  113. 113. @lissijean
  114. 114. @lissijean The Good The Bad • Can demonstrate complicated things. • Show a prototype before producing them all. • More complicated to put together. • Needs to be a certain amount done before you can make the video. Videos
  115. 115. @lissijean • If your product does something very easily. • When your product sounds complicated. • When people need to see it to believe it. When to use Videos.
  116. 116. @lissijean What if I’m not solving a dire need?
  117. 117. @lissijean workshop two: Design your test.
  118. 118. @lissijean You have already interviewed your customer, they have that problem.
  119. 119. @lissijean Now how are you going to solve that problem?
  120. 120. @lissijean Choose an MVP strategy and plan out a test. 30 minutes • What you are trying to learn? • Explain every little detail of the test. • What are you measuring? • Minimum success criteria.
  121. 121. @lissijean Resources Access to 2 Macy’s stores Access to Macy’s online page ! 3 Store Interns $30,000 salary each 10 Sales Associates In Store $40,000 salary each 5 Developers $100,000 salary each 1 Designer $100,000 salary Yourself $100,000 salary
  122. 122. @lissijean Every time you start with a minimum feature set, a kitten dies.
  123. 123. @lissijean Responding to Constraints & Feedback PART III
  124. 124. @lissijean The goal of an MVP is to learn.
  125. 125. @lissijean Customer Feedback
  126. 126. @lissijean
  127. 127. @lissijean
  128. 128. @lissijean Constraints
  129. 129. @lissijean Restricts medium. Restricts resources.
  130. 130. @lissijean workshop three: Alter your test based on constraints.
  131. 131. @lissijean You learned that people only have this problem online. How do you make an online MVP?
  132. 132. @lissijean You learned that people only have this problem in store. How do you make an in person MVP?
  133. 133. @lissijean How would you change your MVP experiment based on these new constraints? 5 minutes
  134. 134. @lissijean MVPs in Agile Teams PART IV
  135. 135. @lissijean Gather Requirements Spec features Design Interfaces Developm ent QA Process Release Waterfall
  136. 136. @lissijean Gather Requirements Spec features Design Interfaces Developm ent QA Process Release Product Manager UX Designer Developer
  137. 137. @lissijean Agile
  138. 138. @lissijean Individuals and Interactions over Processes and Tools
  139. 139. @lissijean Agile Sprints Short, goal oriented sprints formed around a problem running Build, Measure, and Learn.
  140. 140. @lissijean Build Measure Learn
  141. 141. @lissijean Product Manager UX Designer Developer Small, cross-functional teams
  142. 142. @lissijean
  143. 143. @lissijean monday tuesday wednesday thursday friday Reflect & Define Specify Build & Refine Build & Refine Customer Feedback Sprint Structure
  144. 144. @lissijean Problem Exploration MVP Experiments Feature Development
  145. 145. @lissijean MVPs are only a test.
  146. 146. @lissijean “My solution is too obvious, if I just build it people will use it.”
  147. 147. @lissijean Customers will only use your features if it fills a need. Customers will only use your features if it fills a need.
  148. 148. @lissijean “Lean is just for startups, my company is too grown up.”
  149. 149. @lissijean Lean applies to all companies.
  150. 150. @lissijean Melissa Perri ! Want an MVP Guide? melissa@produxlabs.com ! produxlabs.com melissaperri.com ! ! @lissijean

×