0
Startup Metrics, a love story.
Everything you need to know about
Startup Product Metrics.
#iCatapult Workshop - 2013-08-12...
@andreasklinger
“Startup Founder”
“Product Guy”

@andreasklinger
@andreasklinger
“Startup Founder”
“Product Guy”

What we will cover
- Early Stage Metrics (Pre Product Market Fit).
- Crea...
The Biggest Risk of a Startup?

#emminvest – @andreasklinger
The Biggest Risk of a Startup?
Time/Money? No You will learn a lot + huge network gain.

#emminvest – @andreasklinger
The Biggest Risk of a Startup?

Building ALMOST the right thing.
Seeing the successful new competitor who does what you do...
“Startups don’t fail because they lack a product;
they fail because they lack customers and a
profitable business model” St...
To miss your opportunity. By focusing on the wrong thing.

Market
Customer
Job/Problem
Willing to pay

Your
Solution

@and...
Or by looking at the wrong customer.

Market
Other
Market

Customer
Job/Problem
Willing to pay

competitor

competitor
com...
Startup Founders.

@andreasklinger
Some startups have
ideas for a new product.

Looking for customers
to buy (or at least use) it.

Customers don’t buy.

“ea...
traction

Product/Market Fit

time

With early stage
I do not mean “X Years”
I mean before product/market fit.
@andreasklin...
Product/market fit
Being in a good market
with a product that can satisfy
that market.
~ Marc Andreessen

@andreasklinger
Product/market fit
Being in a good market
with a product that can satisfy
that market.
~ Marc Andreessen
= People want your...
traction

Product/Market Fit

time

@andreasklinger
traction

Product/Market Fit

time
Discovery

Validation

Efficiency

Scale

Steve Blank - Customer Development

@andreaskl...
traction

Product/Market Fit

time
Empathy

Stickness

Virality

Revenue

Scale

Ben Yoskovitz, Alistair Croll - Lean Anal...
traction

Product/Market Fit

time
Empathy

Stickness

Virality

Revenue

Scale

Find a product
the market wants.

@andrea...
traction

Product/Market Fit

time
Empathy

Stickness

Find a product
the market wants.

Virality

Revenue

Scale

Optimis...
traction

Product/Market Fit

time
Empathy

Stickness

Virality

Find a product
the market wants.
People in search
for new...
traction

Product/Market Fit

time
Empathy

Stickness

Product & Customer
Development

Virality

Revenue

Scale

Scale Mar...
traction

Product/Market Fit

time
Empathy

Stickness

Virality

Revenue

Scale

Startups have phases
but they overlap.
@a...
traction

Product/Market Fit

time
Empathy

Stickness

Virality

Revenue

Scale

83% of all startups are in here.

@andrea...
traction

Product/Market Fit

time
Empathy

Stickness

Virality

83% of all startups are in here.

Revenue

Scale

Most st...
Most of the techniques/pattern
we find when we look to learn about
metrics come out from Online Marketing

@andreasklinger
A/B Testing, Funnels, Referral Optimization, etc etc
They are about optimizing, not innovating

@andreasklinger
What are we testing here?

@andreasklinger
What are we testing here?
Value Proposition, Communication.
Maybe Channels, Target Group.
Not the product implementation.
...
We need product insights.

@andreasklinger
Good news: We don’t need many people
It’s too early for optimizations.
Challenge #1: Get them to stay.
@andreasklinger
“...interesting...”
What does this mean for my product?
Are we even on the right track?
eg. What is a good Time on Site?
Maybe users spend time reading
your support pages just because
they are super confused.
Google Analytics is meant for
Referral Optimization.
Where does traffic come from?
Is this traffic of value?
What does it mean anyway?

@andreasklinger
What does it mean anyway?
“We have 50k registered users!”
Do they still use the service? Are they the right people?
“We ha...
Some graphs always go up.
Vanity. Use it for the Press.
Not for your product.

@andreasklinger
It’s easy to improve conversions
(of the wrong people)

Source: http://www.codinghorror.com/blog/2009/07/how-not-to-advert...
It’s easy to improve conversions
(of the wrong people)

This is a real example.
Thank the Internet.

Source: http://www.co...
The same is true for funnels.
It’s easy to optimize by pushing the wrong
people forward.

Disclaimer: this is not a real e...
The same is true for funnels.
It’s easy to optimize by pushing the wrong
people forward.

Disclaimer: this is not a real e...
“Small Data” something
small
Early Stage Product Metrics suck:
- We have the wrong product suck.
why early stage metrics
-...
Early stage product metrics get easily
affected by external traffic.
One of our main goals is to minimize that effect.
@and...
#1

Values: 0, 0, 0, 100
What’s the average?

@andreasklinger
#1

Values: 0, 0, 0, 100
What’s the average? 25
What’s the median?

@andreasklinger
#1

Values: 0, 0, 0, 100
What’s the average? 25
What’s the median? 0

Eg. “Average Interactions”
can be Bollocks

#scb13 –...
#1

Values: 0, 0, 0, 100
What’s the average? 25
What’s the median? 0

Eg. “Average Interactions”
can be Bollocks

#scb13 –...
#2

Values: 0, 5, 10
What’s the average?

@andreasklinger
#2

Values: 0, 5, 10
What’s the average? 5
What’s the standard
deviation? 4,08

=> “Average” hides information.
#scb13 – @...
#3

“This is not statistical
significant.”
I don’t care.
We are anyway playing dart in a
dark room.
Let’s try to get it as ...
Correlation / Causality

#scb13 – @andreasklinger
Source: Lean Analytics
Correlation / Causality

Source:@andreasklinger
Lean Analytics
How can we use metrics?

@andreasklinger
How can we use metrics?
To Explore (Examples)

To Report (Examples)

Investigate an
assumption.

Measure Progress.
(Accoun...
Hits

Page Views

Visits

Visitors

People

Views

Clicks

Conversions

Engagement

What do i measure?
TL;DR: In case of d...
What are good KPIs?

Linked to assumptions of your product (validation/falsify)

Framework: AARRR

#scb13 – @andreasklinge...
What are good KPIs?

Rate or Ratio (0.X or %)
Framework: AARRR

#scb13 – @andreasklinger
What are good KPIs?

Framework: AARRR
Comparable (To your history (or a/b). Forget the market)

#scb13 – @andreasklinger
What are good KPIs?

Framework: AARRR

Explainable (If you don’t get it it means nothing)
#scb13 – @andreasklinger
What are good KPIs?

Linked to assumptions of your product (validation/falsify)
Rate or Ratio (0.X or %)
Framework: AARRR
...
“Industry Standards”

Framework: AARRR

Use industry averages as reality check.
Not as benchmark.
- Usually very hard to g...
Let’s start.

#scb13 – @andreasklinger
Segment Users into Cohorts

Cohorts = Groups of people that share attributes.

@andreasklinger
Segment Users into Cohorts

Like it

People

23%

@andreasklinger
Segment Users into Cohorts

Like it
People 0-25

3%

People 26-50

4%

People 51-75

65%
@andreasklinger
Segment Users into Cohorts

Average Spending
Jan

€5

Feb

€4.5

Mar

€5

Apr

€4.25

May

€4.5

...

...

Averages can hi...
Segment Users into Cohorts
Month Lifecycle

Registration Month

1

2

3

4

5

Jan

€5

€3

€2

€1

€0.5

Feb

€6

€4

€2
...
Apply a framework: AARRR

@andreasklinger
Acquisition
Visit / Signup / etc

Activation
Use of core feature

Retention
Come + use again

Referral
Invite + Signup

Re...
Example: Blossom.io
Kanban Project Mangement Tool

(people who) registered an account
Acquisition

created a card
Activati...
Example: Photoapp

(people who) registered an account
Acquisition

created first photo
Activation

opened app twice in peri...
Example: Photoapp
ARRR

acquisition

activation

retention

referral

revenue

registration

first photo

twice a month

sh...
Example: Photoapp
Cohorts based on registration week AARRR
WK

acquisition

activation

Photoapp registration first photo

...
Acquisition
Visit / Signup / etc

Activation
Use of core feature

Which Metrics to focus on?

Retention
Come + use again

...
Acquisition
Visit / Signup / etc

Activation
Use of core feature

Short Answer:

Focus on Retention

Retention
Come + use ...
Because
Retention = f(user_happiness)
Because
Retention = f(user_happiness)

Not only “visited again”.
But “did core activity X again”
Acquisition
Visit / Signup / etc

Activation
Use of core feature

Long answer - It depends on two things:
Retention

Come ...
#1 Phase
Acquisition
Visit / Signup / etc

Activation
Use of core feature

Retention
Come + use again

Referral
Invite + S...
#2 Engine of Growth
Acquisition
Visit / Signup / etc

Paid
Activation

Make more money on a customer
than you spend, to bu...
#2 Type of Company (linked to Engine of Growth)
Acquisition
Visit / Signup / etc

Saas
Build a better product

Activation
...
Acquisition
Visit / Signup / etc

Activation
Use of core feature

Short Answer:

Focus on Retention

Retention
Come + use ...
Retention Matrix
Stickness over lifetime.
Source: www.usercycle.com
Acquisition
Visit / Signup / etc

Activation
Use of core feature

Short Answer:

Focus on Retention

Eg. Did core-action X...
Dig deeper: Crashpadder’s Happiness Index
e.g. Weighted sum over core activities by hosts.
Cohorts by cities and time.
= H...
Marketplaces
/ 2-sided models
Marketplaces
/ 2-sided models
Marketplaces
/ 2-sided models
Amazon Approach:
1) Create loads of inventory
for longtail search
2) Create demand
3) Have s...
How to solve the
Chicken and Egg
Problem TL;DR:
You need very few
but very good and
happy chickens
to get a lot of eggs.

...
Example Mobile App: Pusher2000
Trainer2peer pressure sport app (prelaunch “beta”).
Rev channel: Trainers pay monthly fee.
...
Groupwork
- What stage are you in?
Empathy

Stickness

Framework: AARRR

Virality

Revenue

Scale

#scb13 – @andreasklinge...
Groupwork
- Define your AARRR Metrics!
Acquisition, Activation, Retention, Referral, Revenue
Framework: (e.g. User who xxxx...
Groupwork
- What’s your current core metric.

Framework: AARRR

#scb13 – @andreasklinger
Show & Tell.

@andreasklinger
The dirty side of metrics.

@andreasklinger
Metrics need to hurt

@andreasklinger
Metrics need to hurt

A Startup should always only focus on
one core metric to optimize.
Your dashboard should only show
m...
Metrics need to hurt

Have two dashboards (example bufferApp):
A actionable board to work on your product
(eg. % people bu...
Metrics need to hurt

If you are not ashamed about the KPIs in
your product dashboard than something is
wrong.
Either you ...
Metrics need to hurt

Example: Garmz/LOOKK
Great Numbers:
90% activation (activation = vote)
But they only voted for frien...
Let’s talk cash.

@andreasklinger
Let’s talk cash: Life Time Value (LTV)

Life Time = 1/Churn
Eg. 1 / 0.08 = 12.5 months
Average Revenue Per User (ARPU)
= R...
Let’s talk cash: Customer Acquisition Costs

CAC = Total Acquisition Costs this month /
New Customers this month
Goal:
CAC...
Monitor Revenue

MRR
Monthly Recurring
Revenue
Revenue Churn &
Qty Churn
Source: www.usercycle.com

@andreasklinger
But how to make more money?

@andreasklinger
Big wins are often cheap.

source: thomas fuchs - www.metricsftw.com
Viral Distribution

@andreasklinger
Viral Distribution

3 kinds of virality
Product Inherent
A function of use. Eg. Dropbox sharing
Product Artificial
Added to...
Viral Distribution

Invitation Rate = Invited people / inviters
eg. 1250 users invited 2000 people = 1.6

Acceptance Rate ...
Viral Distribution

Important #1:
Unless your Growth Engine is Viral,
don’t focus early on Viral Factors.
Focus on retenti...
Churn is not what you think it is

#scb13 – @andreasklinger
@andreasklinger
Churn is not what you think it is

“5% Churnrate per month”
One of the highest churn reasons is expired
credit cards.
Sad ...
Churn is not what you think it is

People don’t
wake up and
suddenly want to
unsubscribe your
service...
“Oh.. it’s May 5t...
Churn is not what you think it is

You loose them in the
first 3 weeks.
And then at some point just
remind them to unsubscr...
Churn is not what you think it is

#scb13 – Lean Analytics
Source: @andreasklinger
Checkout Intercom.io
Segment + message customers = Awesome

#scb13 – @andreasklinger
User activation.
Some users are happy (power users)
Some come never again (churned users)
What differs them? It’s their ac...
Sub-funnels
“What did the people do
that signed up,
before they signed up,
compare to those who didn’t?”

Source: Usercycl...
Example Twitter
How often did activated users
use twitter in the first month:

7 times
What did they do?
Follow 20 people, ...
Example Twitter
Example Twitter:
How did they get more people
to follow 30people within
7visits in the first 30 days?
Ran a...
Data is noisy.

@andreasklinger
Data is noisy.

source: thomas fuchs - www.metricsftw.com
Data is noisy.

source: thomas fuchs - www.metricsftw.com
Data is noisy.

source: thomas fuchs - www.metricsftw.com
Data is noisy.

source: thomas fuchs - www.metricsftw.com
Data is noisy.

source: thomas fuchs - www.metricsftw.com
Dataschmutz
A layer of dirt obfuscating
your useable data.

Usually “wrong intent”.
Usually our fault.

(~ sample noise we...
Dataschmutz

A layer of dirt obfuscating
your useable data.

@andreasklinger
Dataschmutz
A layer of dirt obfuscating
your useable data.

e.g. Traffic Spikes of wrong
customer segment.
(wrong intent)

...
How to minimize the impact of Dataschmutz
Base your KPIs on wavebreakers.
WK

visitors

acquisition activation

retention
...
Dataschmutz

MySugr
is praised as
“beautiful app”
example.…
=> Downloads
=> Problem:
Not all are diabetic
They focus on pe...
Dataschmutz
Competitions create artificial incentive

Competition Created
“Dataschmutz”
Competitions (before P/M Fit)

are ...
Lean Analytics

@andreasklinger
Lean Analytics

@andreasklinger
Lean Analytics
Lean Analytics

Hypothesis: We believe that introducing a newsfeed
will increase interaction between users.
Track: Engagem...
Lean Analytics

Important #1:
Don’t forget to timebox experiments.
Important #2:
Worry less about statistical significance ...
AARRR misses something

@andreasklinger
CUSTOMER INTENT

Acquisition

Activation

Retention

Referral

Revenue

FULFILMENT OF CUSTOMER INTENT
(c) Dave McClure
CUSTOMER INTENT (JOB)

Acquisition
Customer
Interviews

Activation

Retention

Customer
Interviews
& Metrics

Referral

Re...
That’s you using metrics
“Look ma, the light started blinking”

@andreasklinger
Metrics are horrible way to understand customer intent

Customer Intent = His “Job to be done”

Products are bought becaus...
Metrics are horrible way to understand customer intent

Great Way: Customer Interviews

But: We bias our people,
when we a...
Source: Lukas Fittl - http://speakerdeck.com/lfittl
Groupwork.
- Draw your customer journey
- Pick a point where there is likely a problem
- Formulate assumption
Framework: A...
Show & Tell.

@andreasklinger
Summary

@andreasklinger
Summary
- Use Metrics for Product and Customer Development.
- Use Cohorts.
- Use AARRR.
- Figure Customer Intent through n...
Read on
Startup metrics for Pirates by Dave McClure
http://www.slideshare.net/dmc500hats/startup-metrics-for-pirates-long-...
Thank you
Liked it?
Tweet it.

@andreasklinger
Slides: http://slideshare.net/andreasklinger

All pictures: http://flickr.co...
20130812 icatapult-130812090501-phpapp02
20130812 icatapult-130812090501-phpapp02
Upcoming SlideShare
Loading in...5
×

20130812 icatapult-130812090501-phpapp02

2,457

Published on

Published in: Business, Technology

Transcript of "20130812 icatapult-130812090501-phpapp02"

  1. 1. Startup Metrics, a love story. Everything you need to know about Startup Product Metrics. #iCatapult Workshop - 2013-08-12 Slideshare Exclusive: The full Powerdeck. ;) @andreasklinger
  2. 2. @andreasklinger “Startup Founder” “Product Guy” @andreasklinger
  3. 3. @andreasklinger “Startup Founder” “Product Guy” What we will cover - Early Stage Metrics (Pre Product Market Fit). - Create your dashboard & customer journey map. - Wrong assumptions on Metrics - Lean Analytics & Advanced Topics @andreasklinger
  4. 4. The Biggest Risk of a Startup? #emminvest – @andreasklinger
  5. 5. The Biggest Risk of a Startup? Time/Money? No You will learn a lot + huge network gain. #emminvest – @andreasklinger
  6. 6. The Biggest Risk of a Startup? Building ALMOST the right thing. Seeing the successful new competitor who does what you do, but has one little detail different that you never focused on. #emminvest – @andreasklinger
  7. 7. “Startups don’t fail because they lack a product; they fail because they lack customers and a profitable business model” Steve Blank And it might have been where you didn’t look. @andreasklinger
  8. 8. To miss your opportunity. By focusing on the wrong thing. Market Customer Job/Problem Willing to pay Your Solution @andreasklinger
  9. 9. Or by looking at the wrong customer. Market Other Market Customer Job/Problem Willing to pay competitor competitor competitor mer usto blem C Pro Job/ g to pay Willin competitor Your Solution @andreasklinger
  10. 10. Startup Founders. @andreasklinger
  11. 11. Some startups have ideas for a new product. Looking for customers to buy (or at least use) it. Customers don’t buy. “early stage” @andreasklinger
  12. 12. traction Product/Market Fit time With early stage I do not mean “X Years” I mean before product/market fit. @andreasklinger
  13. 13. Product/market fit Being in a good market with a product that can satisfy that market. ~ Marc Andreessen @andreasklinger
  14. 14. Product/market fit Being in a good market with a product that can satisfy that market. ~ Marc Andreessen = People want your stuff. @andreasklinger
  15. 15. traction Product/Market Fit time @andreasklinger
  16. 16. traction Product/Market Fit time Discovery Validation Efficiency Scale Steve Blank - Customer Development @andreasklinger
  17. 17. traction Product/Market Fit time Empathy Stickness Virality Revenue Scale Ben Yoskovitz, Alistair Croll - Lean Analytics @andreasklinger
  18. 18. traction Product/Market Fit time Empathy Stickness Virality Revenue Scale Find a product the market wants. @andreasklinger
  19. 19. traction Product/Market Fit time Empathy Stickness Find a product the market wants. Virality Revenue Scale Optimise the product for the market. @andreasklinger
  20. 20. traction Product/Market Fit time Empathy Stickness Virality Find a product the market wants. People in search for new product start here. Revenue Scale Optimise the product for the market. Most clones start here. @andreasklinger
  21. 21. traction Product/Market Fit time Empathy Stickness Product & Customer Development Virality Revenue Scale Scale Marketing & Operations @andreasklinger
  22. 22. traction Product/Market Fit time Empathy Stickness Virality Revenue Scale Startups have phases but they overlap. @andreasklinger
  23. 23. traction Product/Market Fit time Empathy Stickness Virality Revenue Scale 83% of all startups are in here. @andreasklinger
  24. 24. traction Product/Market Fit time Empathy Stickness Virality 83% of all startups are in here. Revenue Scale Most stuff we learn about web analytics is meant for this part @andreasklinger
  25. 25. Most of the techniques/pattern we find when we look to learn about metrics come out from Online Marketing @andreasklinger
  26. 26. A/B Testing, Funnels, Referral Optimization, etc etc They are about optimizing, not innovating @andreasklinger
  27. 27. What are we testing here? @andreasklinger
  28. 28. What are we testing here? Value Proposition, Communication. Maybe Channels, Target Group. Not the product implementation. @andreasklinger
  29. 29. We need product insights. @andreasklinger
  30. 30. Good news: We don’t need many people It’s too early for optimizations. Challenge #1: Get them to stay. @andreasklinger
  31. 31. “...interesting...”
  32. 32. What does this mean for my product? Are we even on the right track?
  33. 33. eg. What is a good Time on Site? Maybe users spend time reading your support pages just because they are super confused.
  34. 34. Google Analytics is meant for Referral Optimization. Where does traffic come from? Is this traffic of value?
  35. 35. What does it mean anyway? @andreasklinger
  36. 36. What does it mean anyway? “We have 50k registered users!” Do they still use the service? Are they the right people? “We have 5000 newsletter signups!” Do they react? Are they potential customers? “We have 500k app downloads!” Do they still use the app? @andreasklinger
  37. 37. Some graphs always go up. Vanity. Use it for the Press. Not for your product. @andreasklinger
  38. 38. It’s easy to improve conversions (of the wrong people) Source: http://www.codinghorror.com/blog/2009/07/how-not-to-advertise-on-the-internet.html @andreasklinger
  39. 39. It’s easy to improve conversions (of the wrong people) This is a real example. Thank the Internet. Source: http://www.codinghorror.com/blog/2009/07/how-not-to-advertise-on-the-internet.html @andreasklinger
  40. 40. The same is true for funnels. It’s easy to optimize by pushing the wrong people forward. Disclaimer: this is not a real example. btw newrelic is awesome ;) @andreasklinger
  41. 41. The same is true for funnels. It’s easy to optimize by pushing the wrong people forward. Disclaimer: this is not a real example. btw newrelic is awesome ;) @andreasklinger
  42. 42. “Small Data” something small Early Stage Product Metrics suck: - We have the wrong product suck. why early stage metrics - With the wrong communication - Attacting the wrong targetgroup - Who provide us too few datapoints Most likely @andreasklinger
  43. 43. Early stage product metrics get easily affected by external traffic. One of our main goals is to minimize that effect. @andreasklinger
  44. 44. #1 Values: 0, 0, 0, 100 What’s the average? @andreasklinger
  45. 45. #1 Values: 0, 0, 0, 100 What’s the average? 25 What’s the median? @andreasklinger
  46. 46. #1 Values: 0, 0, 0, 100 What’s the average? 25 What’s the median? 0 Eg. “Average Interactions” can be Bollocks #scb13 – @andreasklinger
  47. 47. #1 Values: 0, 0, 0, 100 What’s the average? 25 What’s the median? 0 Eg. “Average Interactions” can be Bollocks #scb13 – @andreasklinger
  48. 48. #2 Values: 0, 5, 10 What’s the average? @andreasklinger
  49. 49. #2 Values: 0, 5, 10 What’s the average? 5 What’s the standard deviation? 4,08 => “Average” hides information. #scb13 – @andreasklinger
  50. 50. #3 “This is not statistical significant.” I don’t care. We are anyway playing dart in a dark room. Let’s try to get it as good as possible and use our intuation for the rest. @andreasklinger
  51. 51. Correlation / Causality #scb13 – @andreasklinger Source: Lean Analytics
  52. 52. Correlation / Causality Source:@andreasklinger Lean Analytics
  53. 53. How can we use metrics? @andreasklinger
  54. 54. How can we use metrics? To Explore (Examples) To Report (Examples) Investigate an assumption. Measure Progress. (Accounting) Look for causalities. Measure Feature Impact. Validate customer feedback. See customer happiness/ health. Validate internal opinions. @andreasklinger
  55. 55. Hits Page Views Visits Visitors People Views Clicks Conversions Engagement What do i measure? TL;DR: In case of doubt, people. @andreasklinger
  56. 56. What are good KPIs? Linked to assumptions of your product (validation/falsify) Framework: AARRR #scb13 – @andreasklinger
  57. 57. What are good KPIs? Rate or Ratio (0.X or %) Framework: AARRR #scb13 – @andreasklinger
  58. 58. What are good KPIs? Framework: AARRR Comparable (To your history (or a/b). Forget the market) #scb13 – @andreasklinger
  59. 59. What are good KPIs? Framework: AARRR Explainable (If you don’t get it it means nothing) #scb13 – @andreasklinger
  60. 60. What are good KPIs? Linked to assumptions of your product (validation/falsify) Rate or Ratio (0.X or %) Framework: AARRR Comparable (To your history (or a/b). Forget the market) Explainable (If you don’t get it it means nothing) #scb13 – @andreasklinger
  61. 61. “Industry Standards” Framework: AARRR Use industry averages as reality check. Not as benchmark. - Usually very hard to get. - Everyone defines stuff different. - You might end up with another business model anyway. - Compare yourself vs your history data. #scb13 – @andreasklinger
  62. 62. Let’s start. #scb13 – @andreasklinger
  63. 63. Segment Users into Cohorts Cohorts = Groups of people that share attributes. @andreasklinger
  64. 64. Segment Users into Cohorts Like it People 23% @andreasklinger
  65. 65. Segment Users into Cohorts Like it People 0-25 3% People 26-50 4% People 51-75 65% @andreasklinger
  66. 66. Segment Users into Cohorts Average Spending Jan €5 Feb €4.5 Mar €5 Apr €4.25 May €4.5 ... ... Averages can hide patterns. @andreasklinger
  67. 67. Segment Users into Cohorts Month Lifecycle Registration Month 1 2 3 4 5 Jan €5 €3 €2 €1 €0.5 Feb €6 €4 €2 €1 Mar €7 €6 €5 Apr €8 €7 May €9 ... ... Insight: Users spend less over time in average. We still don’t know if they spend less, or if less people spend at all. @andreasklinger
  68. 68. Apply a framework: AARRR @andreasklinger
  69. 69. Acquisition Visit / Signup / etc Activation Use of core feature Retention Come + use again Referral Invite + Signup Revenue $$$ Earned (c) Dave McClure
  70. 70. Example: Blossom.io Kanban Project Mangement Tool (people who) registered an account Acquisition created a card Activation moved a card Retention invited team members Referral have upgraded Revenue
  71. 71. Example: Photoapp (people who) registered an account Acquisition created first photo Activation opened app twice in perioid Retention shared photo to fb Referral ??? (exit?) Revenue
  72. 72. Example: Photoapp ARRR acquisition activation retention referral revenue registration first photo twice a month share … 8750 65% 23% 9%
  73. 73. Example: Photoapp Cohorts based on registration week AARRR WK acquisition activation Photoapp registration first photo retention referral revenue twice a month share … 1 400 62,5% 25% 10% 2 575 65% 23% 9% 3 350 64% 26% 4% … … … … …
  74. 74. Acquisition Visit / Signup / etc Activation Use of core feature Which Metrics to focus on? Retention Come + use again Referral Invite + Signup Revenue $$$ Earned (c) Dave McClure
  75. 75. Acquisition Visit / Signup / etc Activation Use of core feature Short Answer: Focus on Retention Retention Come + use again Referral Invite + Signup Revenue $$$ Earned (c) Dave McClure
  76. 76. Because Retention = f(user_happiness)
  77. 77. Because Retention = f(user_happiness) Not only “visited again”. But “did core activity X again”
  78. 78. Acquisition Visit / Signup / etc Activation Use of core feature Long answer - It depends on two things: Retention Come + use again Phase of company Referral Invite + Signup Type of Product Revenue Engine of Growth $$$ Earned (c) Dave McClure
  79. 79. #1 Phase Acquisition Visit / Signup / etc Activation Use of core feature Retention Come + use again Referral Invite + Signup Revenue $$$ Earned Source: Lean Analytics Book - highly recommend
  80. 80. #2 Engine of Growth Acquisition Visit / Signup / etc Paid Activation Make more money on a customer than you spend, to buy new ones. Eg. Saas Use of core feature Viral Retention A Users brings more than one new user. Eg. typical interactive ad-campaigns Come + use again Referral Sticky Invite + Signup Keep your userbase to improve your quality. Eg. Communities Revenue $$$ Earned @andreasklinger
  81. 81. #2 Type of Company (linked to Engine of Growth) Acquisition Visit / Signup / etc Saas Build a better product Activation Use of core feature Social Network/Community “(subjective) Critical Mass” Marketplace Get the right sellers and many more... Retention Come + use again Referral Invite + Signup Revenue $$$ Earned @andreasklinger
  82. 82. Acquisition Visit / Signup / etc Activation Use of core feature Short Answer: Focus on Retention Retention Come + use again Referral Invite + Signup Revenue $$$ Earned (c) Dave McClure
  83. 83. Retention Matrix Stickness over lifetime. Source: www.usercycle.com
  84. 84. Acquisition Visit / Signup / etc Activation Use of core feature Short Answer: Focus on Retention Eg. Did core-action X times in period. Retention Come + use again Referral Invite + Signup Revenue $$$ Earned (c) Dave McClure
  85. 85. Dig deeper: Crashpadder’s Happiness Index e.g. Weighted sum over core activities by hosts. Cohorts by cities and time. = Health/Happiness Dashboard
  86. 86. Marketplaces / 2-sided models
  87. 87. Marketplaces / 2-sided models
  88. 88. Marketplaces / 2-sided models Amazon Approach: 1) Create loads of inventory for longtail search 2) Create demand 3) Have shitloads of money. 4) Wait and win. Boticca (aka Startup) Approach: 1) Focus on one niche Targetgroup 2) Create “enough” inventory with very few sellers 3) Create demand 4) Create more demand 5) Add a few more sellers. Repeat. Pray.
  89. 89. How to solve the Chicken and Egg Problem TL;DR: You need very few but very good and happy chickens to get a lot of eggs. Btw: If the chickens don’t come by themselves, buy them. Uber paid 30$/h to the first drivers. Ignore the “minor chickens”, they come anyway.
  90. 90. Example Mobile App: Pusher2000 Trainer2peer pressure sport app (prelaunch “beta”). Rev channel: Trainers pay monthly fee. Two sided => Segment AARRR for both sides (trainer/user) Marketplace => Value = Transactions / Supplier Social Software => DAU/MAU to see if activated users stay active Chicken/Egg => You need a few very happy chickens for loads of eggs. Activated User:AARRR two training sessions Framework: More than Week/Week retention to see if public launch makes sense Optimize retention: Interviews with Users that left Measure Trainer Happiness Index Pushups / User / Week to see if the core assumption (People will do more pushups) is valid #scb13 – @andreasklinger
  91. 91. Groupwork - What stage are you in? Empathy Stickness Framework: AARRR Virality Revenue Scale #scb13 – @andreasklinger
  92. 92. Groupwork - Define your AARRR Metrics! Acquisition, Activation, Retention, Referral, Revenue Framework: (e.g. User who xxxx) One KPI each. AARRR #scb13 – @andreasklinger
  93. 93. Groupwork - What’s your current core metric. Framework: AARRR #scb13 – @andreasklinger
  94. 94. Show & Tell. @andreasklinger
  95. 95. The dirty side of metrics. @andreasklinger
  96. 96. Metrics need to hurt @andreasklinger
  97. 97. Metrics need to hurt A Startup should always only focus on one core metric to optimize. Your dashboard should only show metrics you are ashamed of. @andreasklinger
  98. 98. Metrics need to hurt Have two dashboards (example bufferApp): A actionable board to work on your product (eg. % people buffering tweets in week 2) A vanity board one to show to investors and tell to the press (eg. total amount of tweets buffered) @andreasklinger
  99. 99. Metrics need to hurt If you are not ashamed about the KPIs in your product dashboard than something is wrong. Either you focus on the wrong KPIs. OR you do not drill deep enough. @andreasklinger
  100. 100. Metrics need to hurt Example: Garmz/LOOKK Great Numbers: 90% activation (activation = vote) But they only voted for friends instead of actually using the platform. We drilled (not far) deeper: Activation = Vote for 2 different designers. Boom. Pain. Metrics need to hurt. @andreasklinger
  101. 101. Let’s talk cash. @andreasklinger
  102. 102. Let’s talk cash: Life Time Value (LTV) Life Time = 1/Churn Eg. 1 / 0.08 = 12.5 months Average Revenue Per User (ARPU) = Revenue / Amount of Users Eg. 20.000€ / 1500 = 13.33€ LTV = ARPU * LifeTime = 13.33€ * 12.5 = 166.6666 @andreasklinger
  103. 103. Let’s talk cash: Customer Acquisition Costs CAC = Total Acquisition Costs this month / New Customers this month Goal: CAC < LTV even better: 2-3xCAC < LTV @andreasklinger
  104. 104. Monitor Revenue MRR Monthly Recurring Revenue Revenue Churn & Qty Churn Source: www.usercycle.com @andreasklinger
  105. 105. But how to make more money? @andreasklinger
  106. 106. Big wins are often cheap. source: thomas fuchs - www.metricsftw.com
  107. 107. Viral Distribution @andreasklinger
  108. 108. Viral Distribution 3 kinds of virality Product Inherent A function of use. Eg. Dropbox sharing Product Artificial Added to support behaviour. E.g. Reward systems Word of Mouth Function of customer satisfaction. Source: Lean Analytics
  109. 109. Viral Distribution Invitation Rate = Invited people / inviters eg. 1250 users invited 2000 people = 1.6 Acceptance Rate = Invited Signed-up / Invited total eg. out of the 2000 people 580 signed up = 0.29 Viral Coefficient = Invitation Rate * Acceptance Rate eg. 1.6 * 0.29 = 0.464 (every customer will bring half a customer additionally)
  110. 110. Viral Distribution Important #1: Unless your Growth Engine is Viral, don’t focus early on Viral Factors. Focus on retention/stickyness. Important #2: Viral Cycle Time (time between recv invite and sending invite) is extremely important. eg. K = 2 Cycle Time: 2 days => 20 days = 20.470 users Cycle Time: 1 day => 20 days = 20mio users Source: http://www.forentrepreneurs.com/lessons-learnt-viral-marketing/
  111. 111. Churn is not what you think it is #scb13 – @andreasklinger @andreasklinger
  112. 112. Churn is not what you think it is “5% Churnrate per month” One of the highest churn reasons is expired credit cards. Sad Fact: People forget about you. #scb13 – @andreasklinger @andreasklinger
  113. 113. Churn is not what you think it is People don’t wake up and suddenly want to unsubscribe your service... “Oh.. it’s May 5th.. Better churn from that random online startup i found 2 months ago.” #scb13 – @andreasklinger @andreasklinger
  114. 114. Churn is not what you think it is You loose them in the first 3 weeks. And then at some point just remind them to unsubscribe. That’s when we measure it (too late). #scb13 – @andreasklinger Source: Intercom.io
  115. 115. Churn is not what you think it is #scb13 – Lean Analytics Source: @andreasklinger
  116. 116. Checkout Intercom.io Segment + message customers = Awesome #scb13 – @andreasklinger
  117. 117. User activation. Some users are happy (power users) Some come never again (churned users) What differs them? It’s their activities in their first 30 days. @andreasklinger
  118. 118. Sub-funnels “What did the people do that signed up, before they signed up, compare to those who didn’t?” Source: Usercycle
  119. 119. Example Twitter How often did activated users use twitter in the first month: 7 times What did they do? Follow 20 people, followed back by 10 Churn: If they don’t keep them 7 times in the first 30 days. They will lose them forever. It doesn’t matter when a user remembers to unsubscribe #scb13 – @andreasklinger
  120. 120. Example Twitter Example Twitter: How did they get more people to follow 30people within 7visits in the first 30 days? Ran assumptions, created features and ran experiments! Watch: http://www.youtube.com/watch?v=L2snRPbhsF0 #scb13 – @andreasklinger
  121. 121. Data is noisy. @andreasklinger
  122. 122. Data is noisy. source: thomas fuchs - www.metricsftw.com
  123. 123. Data is noisy. source: thomas fuchs - www.metricsftw.com
  124. 124. Data is noisy. source: thomas fuchs - www.metricsftw.com
  125. 125. Data is noisy. source: thomas fuchs - www.metricsftw.com
  126. 126. Data is noisy. source: thomas fuchs - www.metricsftw.com
  127. 127. Dataschmutz A layer of dirt obfuscating your useable data. Usually “wrong intent”. Usually our fault. (~ sample noise we created ourselves) @andreasklinger
  128. 128. Dataschmutz A layer of dirt obfuscating your useable data. @andreasklinger
  129. 129. Dataschmutz A layer of dirt obfuscating your useable data. e.g. Traffic Spikes of wrong customer segment. (wrong intent) More “wrong” people come. Lower conversion rate on the landing page. @andreasklinger
  130. 130. How to minimize the impact of Dataschmutz Base your KPIs on wavebreakers. WK visitors acquisition activation retention referral revenue twice a month share … Birchbox visit registration first photo 1 6000 66% / 4000 62,5% 25% 10% 2 25000 35% / 8750 65% 23% 9% 3 5000 70% / 3500 64% 26% 4%
  131. 131. Dataschmutz MySugr is praised as “beautiful app” example.… => Downloads => Problem: Not all are diabetic They focus on people who activated. @andreasklinger
  132. 132. Dataschmutz Competitions create artificial incentive Competition Created “Dataschmutz” Competitions (before P/M Fit) are nothing but Teflon Marketing “Would you use my app and might win 1.000.000 USD?” * Users had huge extra incentive. * Marketing People leave. People come.can hurt your numbers. * While we decided on how to They leave dirt in your database. relaunch we had dirty numbers. @andreasklinger
  133. 133. Lean Analytics @andreasklinger
  134. 134. Lean Analytics @andreasklinger
  135. 135. Lean Analytics
  136. 136. Lean Analytics Hypothesis: We believe that introducing a newsfeed will increase interaction between users. Track: Engagement between users (comments/likes) Falsifiable Hypothesis: People who visited the newsfeed will give a 30% more comments and likes, than people who didn’t.
  137. 137. Lean Analytics Important #1: Don’t forget to timebox experiments. Important #2: Worry less about statistical significance (while early stage). Just use experiments to doublecheck your entrepreneurial intuition.
  138. 138. AARRR misses something @andreasklinger
  139. 139. CUSTOMER INTENT Acquisition Activation Retention Referral Revenue FULFILMENT OF CUSTOMER INTENT (c) Dave McClure
  140. 140. CUSTOMER INTENT (JOB) Acquisition Customer Interviews Activation Retention Customer Interviews & Metrics Referral Revenue FULFILMENT OF CUSTOMER INTENT (c) Dave McClure
  141. 141. That’s you using metrics “Look ma, the light started blinking” @andreasklinger
  142. 142. Metrics are horrible way to understand customer intent Customer Intent = His “Job to be done” Products are bought because they solve a “job to be done”. Learn about Jobs to be done Framework Watch: http://bit.ly/cc-jtbd @andreasklinger
  143. 143. Metrics are horrible way to understand customer intent Great Way: Customer Interviews But: We bias our people, when we ask them. Even if we try not to. Reason: we believe our own bullshit. Watch: www.hackertalks.io
  144. 144. Source: Lukas Fittl - http://speakerdeck.com/lfittl
  145. 145. Groupwork. - Draw your customer journey - Pick a point where there is likely a problem - Formulate assumption Framework: AARRR - Formulate Success Criteria #scb13 – @andreasklinger
  146. 146. Show & Tell. @andreasklinger
  147. 147. Summary @andreasklinger
  148. 148. Summary - Use Metrics for Product and Customer Development. - Use Cohorts. - Use AARRR. - Figure Customer Intent through non-biasing interviews. - Understand your type of product and it’s core drivers - Find KPIs that mean something to your specific product. - Avoid Telfonmarketing (eg Campaigns pre-product). - Filter Dataschmutz - Metrics need to hurt - Focus on the first 30 days of customer activation. - Connect Product Hypotheses to Metrics. - Don’t hide behind numbers. TL;DR: Use metrics to validate/doublecheck. Use those insights when designing for/speaking to your customers.
  149. 149. Read on Startup metrics for Pirates by Dave McClure http://www.slideshare.net/dmc500hats/startup-metrics-for-pirates-long-version Actionable Metrics by Ash Mauyra http://www.ashmaurya.com/2010/07/3-rules-to-actionable-metrics/ Data Science Secrets by DJ Patil - LeWeb London 2012 http://www.youtube.com/watch?v=L2snRPbhsF0 Twitter sign up process http://www.lukew.com/ff/entry.asp?1128 Lean startup metrics - @stueccles http://www.slideshare.net/stueccles/lean-startup-metrics Cohorts in Google Analytics - @serenestudios http://danhilltech.tumblr.com/post/12509218078/startups-hacking-a-cohort-analysis-with-google Rob Fitzpatrick’s Collection of best Custdev Videos - @robfitz http://www.hackertalks.io Lean Analytics Book http://leananalyticsbook.com/introducing-lean-analytics/ Actionable Metrics - @lfittl http://www.slideshare.net/lfittl/actionable-metrics-lean-startup-meetup-berlin App Engagement Matrix - Flurry http://blog.flurry.com/bid/90743/App-Engagement-The-Matrix-Reloaded My Blog http://www.klinger.io
  150. 150. Thank you Liked it? Tweet it. @andreasklinger Slides: http://slideshare.net/andreasklinger All pictures: http://flickr.com/commons @andreasklinger
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×