Main takeaways: - Continuous discovery is the only way to do product - Recruiting users will be hard, build relationships, or contact strangers in context - Product development can be a struggle, agree on a recipe as a team, stick to it, and iterate

1. www.productschool.com
How to Succeed as a PM by Native
Instruments fmr Dir of Product

2. FREE INVITE
Join 35,000+ Product Managers on

3. COURSES
Product Management
Learn the skills you need to land a product manager job

4. COURSES
Coding for Managers
Build a website and gain the technical knowledge to lead software engineers

5. COURSES
Data Analytics for Managers
Learn the skills to understand web analytics, SQL and machine learning concepts

6. COURSES
Digital Marketing for Managers
Learn how to acquire more users and convert them into clients

7. COURSES
UX Design for Managers
Gain a deeper understanding of your users and deliver an exceptional end-to-end
experience

8. COURSES
Product Leadership for Managers
For experienced Product Managers looking to gain strategic skills needed for top leadership
roles

9. COURSES
Corporate Training
Level up your team’s product management skills

10. Michael Mukhin
TONIGHT’S SPEAKER

11. ex-Software Engineer
Co-Founder/CTO @ metapop
Transition to PM @ NI

12. PM Theory
5 Examples
Ask questions any time.
Slides will be posted.
Talk Outline

13. PM Scenarios I found myself in

14. Tons of ideas.
What should I
build?

15. Idea shipped.
Why isn’t anybody using it?

16. I built the wrong thing.
Was I solving a problem?
vs.
Was it just a cool idea?

17. Ideas fail when they don’t solve
problems.
“Cool to combine IG, YT, FB all into one app.”

18. How do I build the right thing?
1.Find some users to talk to.
2.Identify their problems.
3.Generate ideas for specific problems.
4.Test with users, and iterate.

19. Solutions/IdeasTalk to users
Workflow

20. 1. Goal: 2x Engagement
2. Talk to users who are not engaging
3. Find their problems/blockers
4. Hypothesize solutions for those problems/blockers
5. Validate hypotheses with experiment

21. Dual-track Agile
1. Discovery Sprint (PM, Eng Lead, Design)
a. Find problems, test solutions
1. Delivery Sprint (Entire team)
a. Ship solutions

22. Generative Research: look for problems.
Evaluative Research: test the solution.
Discovery Sprints

23. Evaluative
● Usability Tests
● A/B Tests
● Cohort Analysis
● Funnel Analysis
● Landing Pages
● Selling
● 5-Second Tests
● Wizard of Oz
Generative
● User Interviews
● Observational Research
● Contextual Research
● Funnel Analysis
● Usability Tests
● 5-Second Tests
● Concierge Tests

24. Examples from Metapop

25. What is metapop?

26. Competitions

27. Typical User

28. How do I build the right thing?
1.Find some users to talk to.
2.Identify their problems.
3.Generate ideas for specific problems.
4.Test with users, and iterate.

29. Find users, schedule time
Existing users/customers
Reddit, Facebook, etc
Validately.com ($$)
UserTesting.com ($$$$)
Mechanical Turk ($)
Co-workers

30. How do I build the right thing?
1.Find some users to talk to.
2.Identify their problems.
3.Generate ideas for specific problems.
4.Test with users, and iterate.

31. Generative Research
User Interviews

32. Goal: Find pain points of beginner music makers
Generative Research: Find pain points

33. Goal: Find pain points of beginner music makers
Methodology: User interviews
Generative Research: Find pain points

34. Goal: Find pain points of beginner music makers
Methodology: User interviews
User Segment: 20 Beginner laptop musicians
Generative Research: Find pain points

35. Goal: Find pain points of beginner music makers
Methodology: User interviews
User Segment: 20 Beginner laptop musicians
Hypothesis: n/a
Generative Research: Find pain points

36. Interview Questions
● Tell me about yourself & the music you make.
● What motivates you?
● What are some goals you have?
● Examples of how you’re reaching your goals today?
● What’s preventing you from achieving those goals?
Generative Research: Find pain points

37. Tiger Astronaut
Generative Research: Find pain points

38. Rainbow Chart of Problems
Generative Research: Find pain points

39. Findings
● Can’t get the “sound in my head” into the software
● Don’t know how to build an audience
● No good way to get “feedback” on my music
Generative Research: Find pain points

40. How do I build the right thing?
1.Find some users to talk to.
2.Identify their problems.
3.Generate ideas for specific problems.
4.Test with users, and iterate.

41. Evaluative Research
Usability Tests

42. “Feedback” Solution
Tracks wanting feedback
Request Feedback
Evaluative Research: Test the solution

43. Goal: Find out if our “feedback stream” solved the
user problem; find usability issues
Evaluative Research: Test the solution

44. Goal: Find out if our “feedback stream” solved the
user problem; find usability issues
Methodology: Moderated usability test
Evaluative Research: Test the solution

45. Goal: Find out if our “feedback stream” solved the
user problem; find usability issues
Methodology: Moderated usability test
User Segment: 5 users on metapop
Evaluative Research: Test the solution

46. Goal: Find out if our “feedback stream” solved the
user problem; find usability issues
Methodology: Moderated usability test
User Segment: 5 users on metapop
Hypothesis: Users able to figure out UX
Evaluative Research: Test the solution

47. Findings
● Watched users struggle & understood why
○ Fixed the UI
● Next time would also do “value test”
This was only qualitative (tiny scale).
Evaluative Research: Test the solution

48. How do I build the right thing?
1.Find some users to talk to.
2.Identify their problems.
3.Generate ideas for specific problems.
4.Test with users, and iterate.

49. Evaluative Research
Cohort Analysis

50. Goal: After launch, check if our “feedback stream”
was engaged with, and affected retention
Evaluative Research: Validate at scale

51. Goal: After launch, check if our “feedback stream”
was engaged with, and affected retention
Methodology: Analytics after 7, 14, 30 days
Evaluative Research: Validate at scale

52. Goal: After launch, check if our “feedback stream”
was engaged with, and affected retention
Methodology: Analytics after 7, 14, 30 days
User Segment: Feedback requesters vs. Not
Evaluative Research: Validate at scale

53. Goal: After launch, check if our “feedback stream”
was engaged with, and affected retention
Methodology: Analytics after 7, 14, 30 days
User Segment: Feedback requesters vs. Not
Hypothesis: At least 40% of users who upload will
request feedback. Retention will increase by 10%.
Evaluative Research: Validate at scale

54. Findings
● 25% of users requested feedback
● Cohort of users increased retention by >10%
Could now try to increase % of requests.
Evaluative Research: Validate at scale

55. 3 Discovery Sprints
● Generative Research to find some user problems
● Evaluative Research to test value & usability of solution
● Evaluative Research to quantitatively verify success

56. User outcome
Delighted to see we read their mind.
Solved a problem.

57. Evaluative Research
Funnel Analysis
(We were wrong)

58. Goal: Find out if users are interested in Music
Industry news/tutorials
Evaluative Research: Check engagement w/ funnel

59. Goal: Find out if users are interested in Music
Industry news/tutorials
Methodology: Funnel for land, mouse-over & click
Evaluative Research: Check engagement w/ funnel

60. Goal: Find out if users are interested in Music
Industry news/tutorials
Methodology: Funnel for land, mouse-over & click
User Segment: All logged-in users
Evaluative Research: Check engagement w/ funnel

61. Goal: Find out if users are interested in Music
Industry news/tutorials
Methodology: Funnel for land, mouse-over & click
User Segment: All logged-in users
Hypothesis: At least 50% of users will hover. At
least 40% will click.
Evaluative Research: Check engagement w/ funnel

62. We built a little module with news links.
Evaluative Research: Check engagement w/ funnel

63. Findings
● <10% hovered over the module
● <5% clicked on a link (expected >40%)
● Could try another experiment
This was an idea in search of a problem.
Evaluative Research: Check engagement w/ funnel

64. Generative Research
User Interviews
Mechanical Turk

65. Mechanical Turk

66. Mechanical Turk

67. Goal: Find out main concerns of parents
Generative Research: Find the main problems

68. Goal: Find out main concerns of parents
Methodology: User interviews via Mechanical Turk
Generative Research: Find the main problems

69. Goal: Find out main concerns of parents
Methodology: User interviews via Mechanical Turk
User Segment: Parents with kids in K-6
Generative Research: Find the main problems

70. Goal: Find out main concerns of parents
Methodology: User interviews via Mechanical Turk
User Segment: Parents with kids in K-6
Hypothesis: n/a
Generative Research: Find the main problems

71. Approach
● Burner phone number w/ Google Voice
● Create Calendly cal with 30 min slots
● Create a $4 task on MTurk
● Wait for calendar events
Generative Research: Find the main problems

72. Approach
● Burner phone number w/ Google Voice
● Create Calendly cal with 30 min slots
● Create a HIT (segmented) on MTurk with
instructions
● Wait for calendar events
Generative Research: Find the main problems

73. Findings
● Bullying
● Social acceptance
● Important curriculum
Generative Research: Find the main problems

74. Develop a skill to make products
for any user.

75. 1.Find some users to talk to.
2.Identify their problems.
3.Generate ideas for specific problems.
4.Test with users, and iterate.
Succeed as a PM by identifying
the right thing to build.

76. www.productschool.com
Part-time Product Management, Coding, Data Analytics, Digital
Marketing, UX Design and Product Leadership courses in San
Francisco, Silicon Valley, New York, Santa Monica, Los Angeles,
Austin, Boston, Boulder, Chicago, Denver, Orange County,
Seattle, Bellevue, Washington DC, Toronto, London and Online