This document discusses key metrics and analysis that are important for data scientists. It emphasizes that daily active users, retention rates, time spent, and revenue are classic good metrics to analyze. Good analysis should describe user needs and inform decisions. Complexity should be avoided as an anti-goal. Measuring impact and having observable consequences are important. Assumptions, small data, talking to users, and making predictions can help overcome objections around data and analysis. Confidence intervals, testing intuitions with data, and the value of information are also discussed.

1. Thinking Like a
Data Scientist
Danny Hernandez
@Hernandez_Danny

2. A Good Primary Metric
Is a compass
Why compasses > maps
Bottom of the hierarchy of data needs

3. Classic Bad Metric
Pageviews :/
understand why?
🙌…
1/10

4. Headline Temptations
Engadget: Pokémon Go'
hits 100 million downloads
Mentioned: It’s also earning $10 million
in daily revenue.
Not Mentioned: 30M daily active users

5. Inspiration
Emily at the Age of 9 takes down
“therapeutic touch” at the science fair.
The I told you so Envelope

6. Classics
Daily active users
Birth and Rolling Retention
Time Spent
Revenue
🙌…

7. Good Analysis
Describes the needs of users:
Number of users, kind of
user, impact per user
Inform decisions

8. Complexity
is an Anti Goal

9. Good Analysis
Costing us ~12% of
our traffic
Provides guidance
for years
A problem well
stated is a problem
half solved.

10. Everything is Measurable
You only care about things that have
observable consequences
Projects justified by “core values”
should have measurable impact

11. Classic Objections
Too expensive
You can prove anything with stats
We won’t be able to get anything
“statistically significant”

12. 4 Useful Assumptions
It’s been measured before
You have more data than you think
You need less data than you think
Useful new observations are more
accessible than you think

13. Small Data
If I know literally nothing. What’s the
chance a single selected sample is the
majority of the population? 🙌…

14. Talking to Users
My prediction…
Which channels would you click on and why?
They represent 25% of viewership on Twitch

15. Build the Future by
Predicting It
Wants to influence product? 🙌
Frustrated with how decisions are made? 🙌
Everything is a bet; cutting edge basic…
What I mean when I say I’ll do something.
A highly aligned team: signup funnel example

16. A script
Colleague: I’ll get this done this quarter
Leader: How surprised would you be if it wasn’t done
by the end of the quarter?
Colleague: Actually not that surprised. Project X is my
top priority and projects like Y have taken a full
month in the past.
Leader: So it’s unlikely to be done this quarter. When
are you 80% sure it’ll be done by?
Colleague: I feel 80% confident I can deliver it by the
end of next quarter.
Leader: That sounds right. Let me know if that
changes.

17. 80% Confidence Intervals
Train like a Neural Net
Google Speed Results 0.4s to 0.9s (2006) 🙌

18. Is that really
the next step?
The pain means it’s working
Turn your intuition into data
The Value of Information

19. Double It

20. More Predictions 🙌
What percent of Twitch Staff recommend this training.
Germany’s organ donation rate (opt out)
Austria’s Organ Donation Rate (opt in)
What income put you in the global 1% (US is 340K)

21. Thank You
Would recommend? 🙌
Medium: @dannyhernandez
Twitter/Whale: @hernandez_danny

22. Questions