Startup Analytics
Topi Järvinen
You? Instincts
rule
Data-
aware
drivenData-
Looks great!
0
1000
2000
3000
1 2 3 4 5
Active users
0
2000
4000
6000
8000
1 2 3 4 5
Sign-ups (cum)
Or not?
0%
20%
40%
60%
80%
100%
2 3 4 5
Active users (existing)
0%
20%
40%
60%
80%
100%
1 2 3 4 5
Active users by cohort
months after they registered
Disaster or good business?
500
1000
1500
2000
1 2 3 4 5
New monthly sign-ups
• In this fictional case, good flow of
new sign-ups keeps the general
metrics looking good
• These numbers could mean a
disaster for some businesses (for
ex. monthly subscription)
• They can also be OK for certain
revenue models (for ex. one-time
fee + upsell), but in that case
you’d need to look at the
acquisition cost
Vanity Metrics
They make you feel
good, but they don’t
offer clear guidance for
what to do.
-Eric Ries
The right way to use analytics: case Twitter
Analytics: Twitter learned that
users that follow 5-10 others,
are more likely to come back.
Solution: re-engineered the
whole site to get users to
follow when they sign up
Confidential7
What does your
success look like?
Where’s the value?
Measure it and the
preceeding steps
Use spreadsheet models
to understand the logic
Useful for finding what to validate,
not for proving the success
Quick’n’Dirty guide to choosing metrics
Not very useful
by itself
Are we creating value
for our customers?
Are our users
willing to pay for it
1. Anything
happening?
2. What works,
what doesn’t
3. Is it a
business?
Sign-ups,
downloads,
visitors etc.
Conversions, active
users, cart size, viral
co-efficient etc.
Revenue,
Customer lifetime
value etc.
Forexample
Read this
http://leananalyticsbook.com
Croll & Yoskovitz
Good metrics
are actionable and
give focus
A good metric is:
Comparative Understandable Ratio or a rate Changes behavior
Helps you to
understand how
different areas
are progressing
(time, groups of
competitors or
users etc.)
If your team
doesn’t
understand the
metric, they
won’t be able to
act on it.
Ratios and rates
are comparative
and usually
easier act on.
Metric are for
learning what is
working and
how to do
better.
Pick
One Metric That Matters
relevant for your business and stage you’re at
Lean Analytics
One Metric That Matters
It answers the most
important question
you have
It forces you to
draw a line in
the sand
It focuses the
entire company
It inspires a
culture of
experimentation
Where to focus?
Think about your
company stage
Lean Analytics by Croll & Yoskovitz
Empathy
Stickiness
Virality
Revenue
Scale
Focus on things important to the company stage
Empathy Problem and solution validation: discover real needs you can solve.
Stickiness Engage with customers in a meaningful, valuable way.
Virality Grow adoption through virality (inherent, artificial word-of-mouth).
Revenue Convince users to pay with optimal pricing
Scale
Growing through customer acquisition, channel relationships,
finding efficiencies, and participating in a market ecosystem.
Questions for different stages and business types
Business type
E-commerce
Two-sided
marketplace
Software
as a Service
Free
mobile app Media
User-
generated
content
The really big question
Empathy
Will they buy
enough for enough
money from you?
Will it solve a pain
they’ll pay for?
Will they engage with
content in a
repeatable manner
Interviews, qualitative results, quantitative scoring, surveys
StageMetrics
Questions for different stages and business types
Will it grow?
Stickiness Will they find you
and tell others?
Will they sign up,
stick around, and tell
others?
Can you grow traffic
to a level that can be
profitably monetized?Virality
Business type
Stickiness
Loyalty,
conversion
Inventory,
listings
Engagement,
churn
Downloads,
churn, virality
Content,
spam
Traffic, visits,
returns
Virality
CAC, shares,
reactivation
SEM, sharing
Inherent
virality, CAC
WoM, app
ratings, CAC
Invites,
sharing
Content
virality, SEM
Stage
E-commerce
Two-sided
marketplace
Software
as a Service
Free
mobile app Media
User-
generated
content
Metrics
Questions for different stages and business types
Primary source of money
Revenue
Transactions Active users Ad revenue
Scale
Business type
Revenue
Transaction,
CLV
Transactions,
commission
Upselling,
CAC, CLV
CLV,
ARPDAU
Ads,
donations
CPE, affiliate
%, eyeballs
Scale
Affiliates,
white-label
Other
verticals
API, magic #,
mktplace
Spinoffs,
publishers
Analytics,
user data
Syndication,
licenses
Stage
E-commerce
Two-sided
marketplace
Software
as a Service
Free
mobile app Media
User-
generated
content
Metrics
Know your baseline
ie. don’t make stupid
decisions based on data
Some interesting benchmarks
• Growth: 5-7% per week (Y Combinator)
• User engagement/day: 10% (see Fred Wilson’s 30/10/10 rule)
• Monthly churn: 2% (=22% annual churn)
• Mailing list effectiveness: 20-30% open rate, 5% click-through rate
• Freemium: 2% of your users will actually sign up for the full offering
• ARPDAU: $0.01-$0.05 for puzzle, caretaking and simulation games.
• Lots of benchmarks, for example, in Lean Analytics book
• Find yours!
Use funnels
to understand and drive
your customers’ actions
Startup Metrics for Pirates by Dave McClure
A Acquisition (how do users find you?)
A Activation (do they have a great first experience?)
R Retention (do they return?)
R Revenue (how do you make money?)
R Referral (do users tell others?)
Use cohorts
to find out what’s
really happening
Cohort
group of people who share something in common
For example,
• Conversion rate of people that signed up in week 3
• Retention rate of users after five months
• Cart size of people that came via an email campaign
• Retention rate of men
• Cancelation rate for customer after one week vs. one month
• Users based on engagement level (for ex. how many time they use the
service, how many service features they use etc.)
Growth hacking
New marketing
Mindset of data, creativity, and
curiosity allows a growth hacker to
accomplish the feat of growing a user
base into the millions
-Aaron Ginn
Growth hackers asks “How do I get
customers for my product?” and answers
with A/B tests, landing pages, viral factor,
email deliverability, and Open Graph.
Growth hackers make sure virality is
embedded at the core of a product.
- Andrew Chen
Airbnb made it really easy to post
the listing on craigslist.com
Accessed instantly millions of
potential customers
Note. Craigslist had no public API
Confidential34
Mailbox application
download wait list showed
users how many others were
in front of them
In six weeks, million users
signed up
Confidential35
Dropbox made a referral
program with incentives
100k to 4M users in under
two years
Confidential36
Hotmail put a tagline in each message sent
through the service.
After five weeks they had 2M users
Confidential37
--------
Get your free email at Hotmail.com
Linkedin gave users an
option to create public
profiles.
Went from 2M users to
200M
Confidential38
Facebook, for example,
enabled users to put FB
widgets on other sites
Comments, activity,
recommendations, feed, etc.
Confidential39
Twitter learned that users that
follow 5-10 others, are more
likely to come back.
They re-engineered the whole
site to get users to follow
when they sign up
Confidential40
Youtube has made it really
easy to share Youtube
videos and embed them on
other sites
Confidential41
Growth hacking
This is what you do
Pick a metric
to change
Find
correlation
Test
causality
Optimize
the causal
factor
Does either of the
correlated variables
really have an impact
on the other?
Two variables that seem
to be associated with
each other
Metric that is relevant
for your company’s
stage and model
Use the causality to
make improvements.
And always remember
correlation doesn’t necessarily mean causality!
Source: www.tylervigen.com
Contact
Topi Järvinen
+358 40 754 3131
topi.jarvinen@nestholma.com
http://www.nestholma.com

Startup analytics

  • 1.
  • 2.
  • 3.
    Looks great! 0 1000 2000 3000 1 23 4 5 Active users 0 2000 4000 6000 8000 1 2 3 4 5 Sign-ups (cum)
  • 4.
    Or not? 0% 20% 40% 60% 80% 100% 2 34 5 Active users (existing) 0% 20% 40% 60% 80% 100% 1 2 3 4 5 Active users by cohort months after they registered
  • 5.
    Disaster or goodbusiness? 500 1000 1500 2000 1 2 3 4 5 New monthly sign-ups • In this fictional case, good flow of new sign-ups keeps the general metrics looking good • These numbers could mean a disaster for some businesses (for ex. monthly subscription) • They can also be OK for certain revenue models (for ex. one-time fee + upsell), but in that case you’d need to look at the acquisition cost
  • 6.
    Vanity Metrics They makeyou feel good, but they don’t offer clear guidance for what to do. -Eric Ries
  • 7.
    The right wayto use analytics: case Twitter Analytics: Twitter learned that users that follow 5-10 others, are more likely to come back. Solution: re-engineered the whole site to get users to follow when they sign up Confidential7
  • 8.
  • 9.
    Where’s the value? Measureit and the preceeding steps
  • 10.
    Use spreadsheet models tounderstand the logic Useful for finding what to validate, not for proving the success
  • 11.
    Quick’n’Dirty guide tochoosing metrics Not very useful by itself Are we creating value for our customers? Are our users willing to pay for it 1. Anything happening? 2. What works, what doesn’t 3. Is it a business? Sign-ups, downloads, visitors etc. Conversions, active users, cart size, viral co-efficient etc. Revenue, Customer lifetime value etc. Forexample
  • 12.
  • 13.
  • 14.
    A good metricis: Comparative Understandable Ratio or a rate Changes behavior Helps you to understand how different areas are progressing (time, groups of competitors or users etc.) If your team doesn’t understand the metric, they won’t be able to act on it. Ratios and rates are comparative and usually easier act on. Metric are for learning what is working and how to do better.
  • 15.
    Pick One Metric ThatMatters relevant for your business and stage you’re at Lean Analytics
  • 16.
    One Metric ThatMatters It answers the most important question you have It forces you to draw a line in the sand It focuses the entire company It inspires a culture of experimentation
  • 17.
    Where to focus? Thinkabout your company stage
  • 18.
    Lean Analytics byCroll & Yoskovitz Empathy Stickiness Virality Revenue Scale
  • 19.
    Focus on thingsimportant to the company stage Empathy Problem and solution validation: discover real needs you can solve. Stickiness Engage with customers in a meaningful, valuable way. Virality Grow adoption through virality (inherent, artificial word-of-mouth). Revenue Convince users to pay with optimal pricing Scale Growing through customer acquisition, channel relationships, finding efficiencies, and participating in a market ecosystem.
  • 20.
    Questions for differentstages and business types Business type E-commerce Two-sided marketplace Software as a Service Free mobile app Media User- generated content The really big question Empathy Will they buy enough for enough money from you? Will it solve a pain they’ll pay for? Will they engage with content in a repeatable manner Interviews, qualitative results, quantitative scoring, surveys StageMetrics
  • 21.
    Questions for differentstages and business types Will it grow? Stickiness Will they find you and tell others? Will they sign up, stick around, and tell others? Can you grow traffic to a level that can be profitably monetized?Virality Business type Stickiness Loyalty, conversion Inventory, listings Engagement, churn Downloads, churn, virality Content, spam Traffic, visits, returns Virality CAC, shares, reactivation SEM, sharing Inherent virality, CAC WoM, app ratings, CAC Invites, sharing Content virality, SEM Stage E-commerce Two-sided marketplace Software as a Service Free mobile app Media User- generated content Metrics
  • 22.
    Questions for differentstages and business types Primary source of money Revenue Transactions Active users Ad revenue Scale Business type Revenue Transaction, CLV Transactions, commission Upselling, CAC, CLV CLV, ARPDAU Ads, donations CPE, affiliate %, eyeballs Scale Affiliates, white-label Other verticals API, magic #, mktplace Spinoffs, publishers Analytics, user data Syndication, licenses Stage E-commerce Two-sided marketplace Software as a Service Free mobile app Media User- generated content Metrics
  • 23.
    Know your baseline ie.don’t make stupid decisions based on data
  • 24.
    Some interesting benchmarks •Growth: 5-7% per week (Y Combinator) • User engagement/day: 10% (see Fred Wilson’s 30/10/10 rule) • Monthly churn: 2% (=22% annual churn) • Mailing list effectiveness: 20-30% open rate, 5% click-through rate • Freemium: 2% of your users will actually sign up for the full offering • ARPDAU: $0.01-$0.05 for puzzle, caretaking and simulation games. • Lots of benchmarks, for example, in Lean Analytics book • Find yours!
  • 25.
    Use funnels to understandand drive your customers’ actions
  • 26.
    Startup Metrics forPirates by Dave McClure A Acquisition (how do users find you?) A Activation (do they have a great first experience?) R Retention (do they return?) R Revenue (how do you make money?) R Referral (do users tell others?)
  • 27.
    Use cohorts to findout what’s really happening
  • 28.
    Cohort group of peoplewho share something in common For example, • Conversion rate of people that signed up in week 3 • Retention rate of users after five months • Cart size of people that came via an email campaign • Retention rate of men • Cancelation rate for customer after one week vs. one month • Users based on engagement level (for ex. how many time they use the service, how many service features they use etc.)
  • 29.
  • 30.
    Mindset of data,creativity, and curiosity allows a growth hacker to accomplish the feat of growing a user base into the millions -Aaron Ginn
  • 31.
    Growth hackers asks“How do I get customers for my product?” and answers with A/B tests, landing pages, viral factor, email deliverability, and Open Graph. Growth hackers make sure virality is embedded at the core of a product. - Andrew Chen
  • 32.
    Airbnb made itreally easy to post the listing on craigslist.com Accessed instantly millions of potential customers Note. Craigslist had no public API Confidential34
  • 33.
    Mailbox application download waitlist showed users how many others were in front of them In six weeks, million users signed up Confidential35
  • 34.
    Dropbox made areferral program with incentives 100k to 4M users in under two years Confidential36
  • 35.
    Hotmail put atagline in each message sent through the service. After five weeks they had 2M users Confidential37 -------- Get your free email at Hotmail.com
  • 36.
    Linkedin gave usersan option to create public profiles. Went from 2M users to 200M Confidential38
  • 37.
    Facebook, for example, enabledusers to put FB widgets on other sites Comments, activity, recommendations, feed, etc. Confidential39
  • 38.
    Twitter learned thatusers that follow 5-10 others, are more likely to come back. They re-engineered the whole site to get users to follow when they sign up Confidential40
  • 39.
    Youtube has madeit really easy to share Youtube videos and embed them on other sites Confidential41
  • 40.
    Growth hacking This iswhat you do Pick a metric to change Find correlation Test causality Optimize the causal factor Does either of the correlated variables really have an impact on the other? Two variables that seem to be associated with each other Metric that is relevant for your company’s stage and model Use the causality to make improvements.
  • 41.
    And always remember correlationdoesn’t necessarily mean causality! Source: www.tylervigen.com
  • 42.
    Contact Topi Järvinen +358 40754 3131 topi.jarvinen@nestholma.com http://www.nestholma.com