Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Experimenting Your Way to MVP

576 views

Published on

Slides from my talk on lean experimentation at Denver Startup Week 2016.

Published in: Technology

Experimenting Your Way to MVP

  1. 1. Experimenting Your Way to MVP Catherine Shyu PM @ FullContact
  2. 2. About Product Manager at FullContact Previously PM at SendGrid, Generalist at BandPage Bay Area native @cthrin
  3. 3. We all want to build this
  4. 4. Nobody wants to spend years building this
  5. 5. Our job is this Address the right target market, with the right problem, with the right solution and marketing, at the right time… Easy right?
  6. 6. So how do we make sure we’re working on the right things?
  7. 7. Shorten the Lean Startup cycle Catch core problems before investing too much time/money Validate your riskiest assumptions through experiments Image from Google Ventures
  8. 8. What are experiments? We run experiments to test hypotheses that... 1. Can be validated / invalidated 2. Address areas of risk in the product
  9. 9. No brainer, right? So why aren’t more companies doing it?
  10. 10. Top 5 Reasons for not Experimenting 1. I can’t justify taking time away from building the product. 2. I don’t have approval from management to do this. 3. We have a deeply complex product. There’s no way to simulate things on it. 4. Our product is too niche. I can’t find users to talk to. 5. I don’t want to damage our relationship with potential customers.
  11. 11. I can’t justify taking time away from building the product. Excuse 1. Time $ $$ $$$ $$$$ Cost of Change Increases with Product Complexity
  12. 12. Example: Customer Development Goal: Product/market fit Assumption: [X] industry is in our target market Methods: Customer development interviews with 5 people from each industry Most powerful questions asked: “What are the top three challenges you face in your industry?” “How do you currently solve [x] problem?”Results ● Findings helped me reorder our product roadmap ● Saved us development time by cutting down v1 scope Excuse 1.
  13. 13. I don’t have approval from management to do this. Excuse 2.
  14. 14. Have a plan & explain it in cost/benefits 1. Make a list of big assumptions and costs associated with them if you get it wrong a. Have a few experiments prepared to validate/invalidate b. Lean towards experiments that are quick and easy to run but get you big insights 2. Are there easy tests you can run to mitigate big expensive risks? 3. Frame as “early risk mitigation” Results ● If you can validate/invalidate your most important assumption, it will 1) inform your product direction and 2) pave the way for more experiments. Excuse 2.
  15. 15. We have a deeply complex product. There’s no way to simulate things on it. Niche industries, Hardware, AI, VR, etc. Excuse 3.
  16. 16. Powerpoint is your friend Take shortcuts however you can Invision: string together high-fidelity mockups to simulate interactivity Powerpoint: when you need to mock something up very quickly 3D Printing: when you are testing the physical UX of a product Results ● Ability to go through rounds of changes before engineering starts work Excuse 3.
  17. 17. Our product is too niche. I can’t find users to talk to. Excuse 4.
  18. 18. Use the power of LinkedIn stalking LinkedIn has robust advanced search features (job title, company, industry) Craigslist ads for local in-person interviews Leverage existing customers Results ● Once I got one person to talk to me, they would often introduce me to their peers in the industry. Always ask for intros to more people. Excuse 4.
  19. 19. I don’t want to damage our relationship with potential customers. Excuse 5.
  20. 20. Frame the interviews Pick customers who will be sympathetic Use framing techniques to prime the customer for giving feedback “I appreciate your time…” “This is still early stages but we wanted to get your opinion at the start…” “Since you are a long-time customer, I wanted to get your feedback on a change I’m considering…” Make them feel special Results ● Many of our private beta testers are still avid users of the app and respond with feedback when asked. They also have free access to the app. Excuse 5.
  21. 21. Lean Experiments: Basic Principles Goal is to test your riskiest assumptions early in the game Experiments run for 3-5 days Multiple experiments can run at once 5 people = pattern Don’t worry, it doesn’t have to be a huge ordeal
  22. 22. Team exercise: biggest risks in your product Risk Severity (1-5, 5 highest) Hypothesis 1 Experiment 1 People won’t like my Dilbert comics 5, obviously People hate the comics but are too nice to say so. Distribute post-talk anonymous survey
  23. 23. What’d We Do at FullContact? Team of 5: two Sales, one PM, one PMM, one Customer Success, one Engineer Trello board for tracking experiments Weekly check-ins to share learnings
  24. 24. Our prototype when we started. Now: FullContact for Teams Beta - onwards to v1!
  25. 25. Results De-scoped the feature set into two releases, by customer segment Validated the need to invest in building a core set of features Able to test out different pricing on potential customers Discovered the problem we are solving is on a top 3 list of problems for multiple industries
  26. 26. Always Be Experimenting! 👋 Thanks for listening @cthrin

×