This document discusses the scientific method for product development and management. It argues that product analytics alone is useless and that the tool is not the action. Instead, companies should use analytics for exploration by making hypotheses and testing them incrementally through controlled experiments. This allows companies to learn and make data-driven decisions at each step of development. The document advocates an approach of focused activity, constant learning, and experimentation over quick decisions. It provides examples from Netflix and the disk drive industry of how incremental improvement through experimentation leads to success over quick releases without refinement.

1. www.productschool.com
Product Analytics is Useless
by Heap CEO

2. Join 55,000+
Product Managers on
Free Resources
Discover great job
opportunities
Job Portal
prdct.school/PSJobPortalprdct.school/events-slack

3. CERTIFICATES
Your Product Management Certificate Path
Product Executive
Certificate™
Product Leader
Certificate™
Product Manager
Certificate™
20 HOURS40 HOURS40 HOURS

4. Corporate
Training
Level up your team’s Product
Management skills

5. Product Analytics
is Useless
PRODUCTANALYTICS01

6. January the 1st...
PRODUCTANALYTICS02

7. January the 1st...
PRODUCTANALYTICS03

8. January 2nd
PRODUCTANALYTICS04

9. January 3rd
PRODUCTANALYTICS05

10. The Tool is Not the Action
06

11. The Tool is Not the Action
PRODUCTANALYTICS07

12. The Tool is Not the Action
PRODUCTANALYTICS08

13. The Tool is Not the Action
PRODUCTANALYTICS09

14. The Tool is Not the Action
PRODUCTANALYTICS010

15. The Scientific
Method for
Product
11

16. Focused activity Constant learning ExperimentationFocused activity Constant learning Experimental mindset
Make hypotheses, not decisions
PRODUCTANALYTICS12

17. PRODUCTANALYTICS13
Make hypotheses, not decisions
Hypothesis
“Machine-ranked timeline”
Supporting belief:
“Users don’t read everything in
their feed”
Supporting belief:
“There’s content they’d find
interesting that doesn’t show
up now”

18. Focused activity Constant learning ExperimentationFocused activity Constant learning Experimental mindset
Make hypotheses, not decisions
PRODUCTANALYTICS14

19. Analytics for Exploration
15

20. Analytics for Exploration
PRODUCTANALYTICS16
“There is a disproportionate payoff from the unknown,
since you typically have little to lose and plenty to gain.”
Nicholas Nasim Taleb, The Black Swan

21. PRODUCTANALYTICS17
Documentation Exploration
Analytics for Exploration

22. PRODUCTANALYTICS18
Documentation Exploration
Analytics for Exploration

23. Iterate Incrementally
PRODUCTANALYTICS19

24. Iterate
Incrementally
20

25. Iterate Incrementally
PRODUCTANALYTICS21
“Victory in the disk-drive industry appears to have gone to the
manufacturers that were good at incremental improvements,
whether or not they were the first to market the disruptive new format.
Companies that were quick to release a new product but not
skilled at tinkering have tended to flame out.”

26. PRODUCTANALYTICS22
“We use controlled experiments to learn and make decisions at every
step of product development, from design to algorithms. They are
equally important in shaping the user experience.”
The Scientific Method for Product

27. PRODUCTANALYTICS23
“Every product change Netflix considers goes through a rigorous A/B
testing process before becoming the default user experience.”
The Scientific Method for Product

28. The Scientific Method for Product
Three components (of many)
Iterate
incrementally
Make hypotheses,
not decisions
Analytics for
exploration
PRODUCTANALYTICS24

29. The Scientific Method for Product
Three components (of many)
Iterate
incrementally
Make hypotheses,
not decisions
Analytics for
exploration
PRODUCTANALYTICS25

30. A Smarter Approach
to Product
PRODUCTANALYTICS

31. www.productschool.com
Part-time Product Management Training Courses
and
Corporate Training