Main takeaways: - Continuous discovery is the only way to do product - Recruiting users will be hard, build relationships, or contact strangers in context - Product development can be a struggle, agree on a recipe as a team, stick to it, and iterate

1. www.productschool.com
How to Succeed as a PM by fmr Native
Instrument Dir of Product

2. Join 35,000+Product
Managers on
Free Resources
Discover great job
opportunities
Job Portal
prdct.school/PSJobPortalprdct.school/events-slack

3. C O U R S E S
Product
Management
Learn the skills you need to land
a Product Manager job

4. C O U R S E S
Coding
for Managers
Build a website and gain the
technical knowledge to lead
software engineers

5. C O U R S E S
Data Analytics
for Managers
Learn the skills to understand web
analytics, SQL and machine learning
concepts

6. C O U R S E S
Learn how to acquire more users
and convert them into clients
Digital Marketing
for Managers

7. C O U R S E S
UX Design
for Managers
Gain a deeper understanding of your
users and deliver an exceptional end-to-
end experience

8. C O U R S E S
For experienced Product Managers
looking to gain strategic skills needed
for top leadership roles
Product
Leadership

9. Michael Mukhin
T O N I G H T ’ S S P E A K E RS

10. HOW TO SUCCEED AS A PM
Build the right thing.
Michael Mukhin

11. ex-Software Engineer
Co-Founder/CTO @ metapop
Transition to PM @ NI
Now PM at Rollbar

12. ● Theory
● Practice (5 Examples)
Talk Outline

13. Scenarios as a PM

14. Scenario 1
● Tons of ideas
● What should I build?

15. Scenario 2
● Idea shipped
● Why isn’t anybody
using it?

16. We built the wrong thing.
Were we solving a problem?
vs.
Was it just a cool idea?

17. Ideas fail when they don’t solve
problems.
“Cool to combine IG, YT, FB all into one app.”

18. How do I build the right thing?

19. How do I build the right thing?
1.Find some users to talk to.
2.Identify their problems.
3.Generate ideas for specific problems.
4.Test with users, and iterate.

20. SolutionsTalk to users
At a Company
Validated
solution?

21. 1. Goal: Increase retention from 20% to 40%
2. Talk to users who are not retaining & find their problems/blockers
3. Hypothesize solutions for those problems/blockers
4. Create experiments to test our solutions
Validated
solution?

22. In Practice: Dual-track agile
1. Discovery Sprint (PM, Designer, Engineer)
a. Find problems, test solutions, iterate
1. Delivery Sprint (Entire team)
a. Ship solutions, iterate

23. Generative Research: look for problems.
Evaluative Research: test our solutions.
Discovery Sprints

24. Evaluative
● Usability Tests
● A/B Tests
● Cohort Analysis
● Funnel Analysis
● Landing Pages
● Selling
● 5-Second Tests
● Wizard of Oz
Generative
● User Interviews
● Observational Research
● Contextual Research
● Funnel Analysis
● Usability Tests
● 5-Second Tests
● Concierge Tests

25. Examples from Metapop

26. What is metapop?

27. Problems we solved
● Problem: Musicians looking for feedback
Outcome: Improve their music
● Problem: Musicians looking for collaborators
Outcome: Make new music & friends

28. Typical User

29. How do I build the right thing?
1.Find some users to talk to.
2.Identify their problems.
3.Generate ideas for specific problems.
4.Test with users, and iterate.

30. Recruit users
● Your current users
● Reddit, Facebook, etc
● Co-workers
● ($$) Validately.com
● ($$$$) UserTesting.com
● ($) Mechanical Turk

31. How do I build the right thing?
1.Find some users to talk to.
2.Identify their problems.
3.Generate ideas for specific problems.
4.Test with users, and iterate.

32. #1 Generative Research
User Interviews

33. Goal: Find pain points of beginner music makers
Generative Research: Find pain points

34. Goal: Find pain points of beginner music makers
Methodology: User interviews
Generative Research: Find pain points

35. Goal: Find pain points of beginner music makers
Methodology: User interviews
User Segment: 20 Beginner laptop musicians
Generative Research: Find pain points

36. Goal: Find pain points of beginner music makers
Methodology: User interviews
User Segment: 20 Beginner laptop musicians
Hypothesis: n/a
Generative Research: Find pain points

37. Interview Questions
● Tell me about yourself & the music you make.
● What motivates you?
● What are some goals you have?
● Examples of how you’re reaching your goals today?
● What’s preventing you from achieving those goals?
Generative Research: Find pain points

38. Tiger Astronaut
Generative Research: Find pain points

39. Rainbow Chart of Problems
Generative Research: Find pain points

40. Findings
● Can’t get the “sound in my head” into the software
● Don’t know how to build an audience
● No good way to get “feedback” on my music
Generative Research: Find pain points

41. How do I build the right thing?
1.Find some users to talk to.
2.Identify their problems.
3.Generate ideas for specific problems.
4.Test with users, and iterate.

42. #2 Evaluative Research
Usability Tests

43. “Feedback” Solution
Tracks wanting feedback
Request Feedback
Evaluative Research: Test the solution

44. Goal: Find out if our “feedback stream” solved the
user problem; find usability issues
Evaluative Research: Test the solution

45. Goal: Find out if our “feedback stream” solved the
user problem; find usability issues
Methodology: Moderated usability test
Evaluative Research: Test the solution

46. Goal: Find out if our “feedback stream” solved the
user problem; find usability issues
Methodology: Moderated usability test
User Segment: 5 users on metapop
Evaluative Research: Test the solution

47. Goal: Find out if our “feedback stream” solved the
user problem; find usability issues
Methodology: Moderated usability test
User Segment: 5 users on metapop
Hypothesis: Users able to figure out UX
Evaluative Research: Test the solution

48. Findings
● Watched users struggle & understood why
○ Fixed the UI
● Next time would also do “value test”
This was only qualitative (tiny scale).
Evaluative Research: Test the solution

49. How do I build the right thing?
1.Find some users to talk to.
2.Identify their problems.
3.Generate ideas for specific problems.
4.Test with users, and iterate.

50. #3 Evaluative Research
Cohort Analysis

51. Goal: After launch, check if our “feedback stream”
was engaged with, and affected retention
Evaluative Research: Validate at scale

52. Goal: After launch, check if our “feedback stream”
was engaged with, and affected retention
Methodology: Analytics after 7, 14, 30 days
Evaluative Research: Validate at scale

53. Goal: After launch, check if our “feedback stream”
was engaged with, and affected retention
Methodology: Analytics after 7, 14, 30 days
User Segment: Feedback requesters vs. Not
Evaluative Research: Validate at scale

54. Goal: After launch, check if our “feedback stream”
was engaged with, and affected retention
Methodology: Analytics after 7, 14, 30 days
User Segment: Feedback requesters vs. Not
Hypothesis: At least 40% of users who upload will
request feedback. Retention will increase by 10%.
Evaluative Research: Validate at scale

55. Findings
● 25% of users requested feedback
● Cohort of users increased retention by >10%
Could now try to increase % of requests.
Evaluative Research: Validate at scale

56. 3 Discovery Sprints
● Generative Research to find some user problems
● Evaluative Research to test value & usability of solution
● Evaluative Research to quantitatively verify success

57. User outcome
Delighted to see we read their mind.
Solved a problem.

58. #4 Evaluative Research
Funnel Analysis
(We were wrong)

59. Goal: Find out if users are interested in Music
Industry news/tutorials
Evaluative Research: Check engagement w/ funnel

60. Goal: Find out if users are interested in Music
Industry news/tutorials
Methodology: Funnel for land, mouse-over & click
Evaluative Research: Check engagement w/ funnel

61. Goal: Find out if users are interested in Music
Industry news/tutorials
Methodology: Funnel for land, mouse-over & click
User Segment: All logged-in users
Evaluative Research: Check engagement w/ funnel

62. Goal: Find out if users are interested in Music
Industry news/tutorials
Methodology: Funnel for land, mouse-over & click
User Segment: All logged-in users
Hypothesis: At least 50% of users will hover. At
least 40% will click.
Evaluative Research: Check engagement w/ funnel

63. We built a little module with news links.
Evaluative Research: Check engagement w/ funnel

64. Findings
● <10% hovered over the module
● <5% clicked on a link (expected >40%)
● Could try another experiment
This was an idea in search of a problem.
Evaluative Research: Check engagement w/ funnel

65. #5 Generative Research
User Interviews
Mechanical Turk

66. Mechanical Turk

67. Mechanical Turk

68. Goal: Find out main concerns of parents
Generative Research: Find the main problems

69. Goal: Find out main concerns of parents
Methodology: User interviews via Mechanical Turk
Generative Research: Find the main problems

70. Goal: Find out main concerns of parents
Methodology: User interviews via Mechanical Turk
User Segment: Parents with kids in K-6
Generative Research: Find the main problems

71. Goal: Find out main concerns of parents
Methodology: User interviews via Mechanical Turk
User Segment: Parents with kids in K-6
Hypothesis: n/a
Generative Research: Find the main problems

72. Approach
● Burner phone number w/ Google Voice
● Create Calendly cal with 30 min slots
● Create a $4 task on MTurk
● Wait for calendar events
Generative Research: Find the main problems

73. Approach
● Burner phone number w/ Google Voice
● Create Calendly cal with 30 min slots
● Create a HIT (segmented) on MTurk with
instructions
● Wait for calendar events
Generative Research: Find the main problems

74. Findings
● Bullying
● Social acceptance
● Important curriculum
Generative Research: Find the main problems

75. 1.Find some users to talk to.
2.Identify their problems.
3.Generate ideas for specific problems.
4.Test with users, and iterate.
Succeed as a PM by
choosing the right thing to build.

76. www.productschool.com
Part-time Product Management, Coding, Data Analytics, Digital
Marketing, UX Design, Product Leadership courses and
Corporate Training