Product Management Event at #ProductCon Los Angeles with Mike Iampietro, Director of Product Management at Mixpanel

1. www.productschool.com
Did It Work? Measuring the Impact of
Product Launches by Mixpanel Dir of PM

2. Join 40,000+
Product Managers on
Free Resources
Discover great job
opportunities
Job Portal
prdct.school/PSJobPortalprdct.school/events-slack

3. CERTIFICATES
Your Product Management Certificate Path
Product Leadership
Certificate™
Full Stack Product
Management Certificate™
Product Management
Certificate™
20 HOURS40 HOURS40 HOURS

4. Corporate
Training
Level up your team’s Product
Management skills

5. Mike Iampietro
S P E A K E R

6. Did it work?
Measuring the impact of your product launch
Mike Iampietro | Director of Product @ Mixpanel
November 12, 2019
6

7. 7
● Attracted to tech because it empowers people
to do things that were beyond their reach
● First job – computer music products
● Consumer digital media at Avid and Adobe
● Love solving problems for customers
● Became PM leader to solve more problems
for more customers
● Reinventing reading at Amazon Kindle
● Mixpanel!
Who am I?

8. 8
So many questions…
Answering questions
is crucial for PMs:
1. What happened?
2. Why did it happen?
3. Did it work?
?
?
?
?
?
?
?
?
?
?
?

9. 9
Measure key metrics:
● Conversion
● Engagement
● Retention
● etc.
What happened?

10. 10
Segment to find causes:
● Different factors
(device, geo, date, etc.)
● Groups of users
(new users, power users, etc.)
Why did it happen?

11. 11
How to move our metrics?
● What feature can we ship?
● What UX can we improve?
● What marketing can we do?
What should we DO?

12. 12
Did it work?
Did it work?

13. 13
Did it work?
Did it work?

14. 14
New feature launched
Did it
work?
Oct 1 Oct 3 Oct 5 Oct 7 Oct 9 Oct 11 Oct 13 Oct 15

15. 15
Did it
work?
So what caused the
upward trend
New feature launched
Oct 1 Oct 3 Oct 5 Oct 7 Oct 9 Oct 11 Oct 13 Oct 15
● New feature launched
● Shipped new app
● Shipped new visualizer
● Launched new campaign

16. 16
Happy customers Unhappy customers
A/B testing?

17. 1717
Did it
work?
Used new feature
Didn’t use new feature
Oct 1 Oct 3 Oct 5 Oct 7 Oct 9 Oct 11 Oct 13 Oct 15

18. 1818
Did it
work?
Used new feature
Didn’t use new feature

19. Used new feature
1919
Did it
work?
SUCCESS!
Didn’t use new feature

20. 20
Did it
work?

21. 21
But what about…
● Confounding factors?
● Self-selection?
● More segmentation!
● Compare users based on how
likely they were to do your goal
Impact with Propensity
Matching
Propensity Matched

22. 22
Did It Work?
It’s the hardest question to answer
but...
● Segment customers by used/not
used
● Segment by days since first use
● Segment by propensity to do goal
FTW!

23. Thank you

24. www.productschool.com
Part-time Product Management Training Courses
and
Corporate Training