Main takeaways:
- Continuous discovery is the only way to do product
- Recruiting users will be hard, build relationships, or contact strangers in context
- Product development can be a struggle, agree on a recipe as a team, stick to it, and iterate
16. I built the wrong thing.
Was I solving a problem?
vs.
Was it just a cool idea?
17. Ideas fail when they don’t solve
problems.
“Cool to combine IG, YT, FB all into one app.”
18. How do I build the right thing?
1.Find some users to talk to.
2.Identify their problems.
3.Generate ideas for specific problems.
4.Test with users, and iterate.
20. 1. Goal: 2x Engagement
2. Talk to users who are not engaging
3. Find their problems/blockers
4. Hypothesize solutions for those problems/blockers
5. Validate hypotheses with experiment
21. Dual-track Agile
1. Discovery Sprint (PM, Eng Lead, Design)
a. Find problems, test solutions
1. Delivery Sprint (Entire team)
a. Ship solutions
28. How do I build the right thing?
1.Find some users to talk to.
2.Identify their problems.
3.Generate ideas for specific problems.
4.Test with users, and iterate.
29. Find users, schedule time
Existing users/customers
Reddit, Facebook, etc
Validately.com ($$)
UserTesting.com ($$$$)
Mechanical Turk ($)
Co-workers
30. How do I build the right thing?
1.Find some users to talk to.
2.Identify their problems.
3.Generate ideas for specific problems.
4.Test with users, and iterate.
32. Goal: Find pain points of beginner music makers
Generative Research: Find pain points
33. Goal: Find pain points of beginner music makers
Methodology: User interviews
Generative Research: Find pain points
34. Goal: Find pain points of beginner music makers
Methodology: User interviews
User Segment: 20 Beginner laptop musicians
Generative Research: Find pain points
35. Goal: Find pain points of beginner music makers
Methodology: User interviews
User Segment: 20 Beginner laptop musicians
Hypothesis: n/a
Generative Research: Find pain points
36. Interview Questions
● Tell me about yourself & the music you make.
● What motivates you?
● What are some goals you have?
● Examples of how you’re reaching your goals today?
● What’s preventing you from achieving those goals?
Generative Research: Find pain points
39. Findings
● Can’t get the “sound in my head” into the software
● Don’t know how to build an audience
● No good way to get “feedback” on my music
Generative Research: Find pain points
40. How do I build the right thing?
1.Find some users to talk to.
2.Identify their problems.
3.Generate ideas for specific problems.
4.Test with users, and iterate.
43. Goal: Find out if our “feedback stream” solved the
user problem; find usability issues
Evaluative Research: Test the solution
44. Goal: Find out if our “feedback stream” solved the
user problem; find usability issues
Methodology: Moderated usability test
Evaluative Research: Test the solution
45. Goal: Find out if our “feedback stream” solved the
user problem; find usability issues
Methodology: Moderated usability test
User Segment: 5 users on metapop
Evaluative Research: Test the solution
46. Goal: Find out if our “feedback stream” solved the
user problem; find usability issues
Methodology: Moderated usability test
User Segment: 5 users on metapop
Hypothesis: Users able to figure out UX
Evaluative Research: Test the solution
47. Findings
● Watched users struggle & understood why
○ Fixed the UI
● Next time would also do “value test”
This was only qualitative (tiny scale).
Evaluative Research: Test the solution
48. How do I build the right thing?
1.Find some users to talk to.
2.Identify their problems.
3.Generate ideas for specific problems.
4.Test with users, and iterate.
50. Goal: After launch, check if our “feedback stream”
was engaged with, and affected retention
Evaluative Research: Validate at scale
51. Goal: After launch, check if our “feedback stream”
was engaged with, and affected retention
Methodology: Analytics after 7, 14, 30 days
Evaluative Research: Validate at scale
52. Goal: After launch, check if our “feedback stream”
was engaged with, and affected retention
Methodology: Analytics after 7, 14, 30 days
User Segment: Feedback requesters vs. Not
Evaluative Research: Validate at scale
53. Goal: After launch, check if our “feedback stream”
was engaged with, and affected retention
Methodology: Analytics after 7, 14, 30 days
User Segment: Feedback requesters vs. Not
Hypothesis: At least 40% of users who upload will
request feedback. Retention will increase by 10%.
Evaluative Research: Validate at scale
54. Findings
● 25% of users requested feedback
● Cohort of users increased retention by >10%
Could now try to increase % of requests.
Evaluative Research: Validate at scale
55. 3 Discovery Sprints
● Generative Research to find some user problems
● Evaluative Research to test value & usability of solution
● Evaluative Research to quantitatively verify success
58. Goal: Find out if users are interested in Music
Industry news/tutorials
Evaluative Research: Check engagement w/ funnel
59. Goal: Find out if users are interested in Music
Industry news/tutorials
Methodology: Funnel for land, mouse-over & click
Evaluative Research: Check engagement w/ funnel
60. Goal: Find out if users are interested in Music
Industry news/tutorials
Methodology: Funnel for land, mouse-over & click
User Segment: All logged-in users
Evaluative Research: Check engagement w/ funnel
61. Goal: Find out if users are interested in Music
Industry news/tutorials
Methodology: Funnel for land, mouse-over & click
User Segment: All logged-in users
Hypothesis: At least 50% of users will hover. At
least 40% will click.
Evaluative Research: Check engagement w/ funnel
62. We built a little module with news links.
Evaluative Research: Check engagement w/ funnel
63. Findings
● <10% hovered over the module
● <5% clicked on a link (expected >40%)
● Could try another experiment
This was an idea in search of a problem.
Evaluative Research: Check engagement w/ funnel
67. Goal: Find out main concerns of parents
Generative Research: Find the main problems
68. Goal: Find out main concerns of parents
Methodology: User interviews via Mechanical Turk
Generative Research: Find the main problems
69. Goal: Find out main concerns of parents
Methodology: User interviews via Mechanical Turk
User Segment: Parents with kids in K-6
Generative Research: Find the main problems
70. Goal: Find out main concerns of parents
Methodology: User interviews via Mechanical Turk
User Segment: Parents with kids in K-6
Hypothesis: n/a
Generative Research: Find the main problems
71. Approach
● Burner phone number w/ Google Voice
● Create Calendly cal with 30 min slots
● Create a $4 task on MTurk
● Wait for calendar events
Generative Research: Find the main problems
72. Approach
● Burner phone number w/ Google Voice
● Create Calendly cal with 30 min slots
● Create a HIT (segmented) on MTurk with
instructions
● Wait for calendar events
Generative Research: Find the main problems
75. 1.Find some users to talk to.
2.Identify their problems.
3.Generate ideas for specific problems.
4.Test with users, and iterate.
Succeed as a PM by identifying
the right thing to build.
76. www.productschool.com
Part-time Product Management, Coding, Data Analytics, Digital
Marketing, UX Design and Product Leadership courses in San
Francisco, Silicon Valley, New York, Santa Monica, Los Angeles,
Austin, Boston, Boulder, Chicago, Denver, Orange County,
Seattle, Bellevue, Washington DC, Toronto, London and Online