Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Lean Analytics
Melbourne Business School
October 14, 2015
@acroll
Percent of businesses that fail
Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Year 7 Year 8 Year 9 Year 10
71%69%66%63%60%55%5...
Still operating after 4 years
Finance Insurance and Real Estate
Education and Health
Agriculture
Services
Wholesale
Mining...
Don’t sell what you can make. Make what you can sell.
Kevin Costner is a lousy entrepreneur.
The core of Lean
is iteration.
Unfortunately,
we’re all liars.
Everyone’s idea is
the best right?
People love
this part!
(but that’s not always
a good thing)
This is where
things fall a...
Most startups don’t know what they’ll
be when they grow up.
Hotmail

was a
database
company
Flickr

was going to
be an MMO...
Analytics can help.
Analytics is the measurement of
movement towards your business
goals.
In a startup, the purpose of analytics is
to iterate to product/market fit
before the money runs out.
We’re bad at data.
What are the chances
the other is a boy?
BB BG
GB GG
2 of 3 (66%) are boys.
GB GG BG
A good metric is:
Understandable
If you’re busy
explaining the
data, you won’t
be busy acting
on it.
Comparative
Compariso...
The
simplest
rule
bad

metric.
If a metric won’t change how
you behave, it’s a
h"p://www.flickr.com/photos/circasassy/78581...
Metrics help you know yourself.
Acquisition
Hybrid
Loyalty
70%

of retailers
20%

of retailers
10%

of retailers
You are
j...
MayAprMarFeb
Slicing and dicing data
Jan
0
5,000
Activeusers
Cohort:
Comparison of
similar groups
along a timeline.
(this ...
Which of these two companies
is doing better?
  January February March April May
Rev/customer $5.00 $4.50 $4.33 $4.25 $4.50
Is this company
growing or stagnating?
Cohor...
Cohort 1 2 3 4 5
January $5 $3 $2 $1 $0.5
February $6 $4 $2 $1  
March $7 $6 $5    
April $8 $7      
May $9        
Avera...
The Lean Analytics framework.
Eric’s three engines of growth
Virality
Make people
invite friends.
How many they
tell, how fast they
tell them.
Price
Spe...
Dave’s Pirate Metrics
AARRR
Acquisition
How do your users become aware of you?
SEO, SEM, widgets, email, PR, campaigns, bl...
Stage
EMPATHY
I’ve found a real, poorly-met need that a
reachable market faces.
STICKINESS
I’ve figured out how to solve th...
Six business model archetypes.
E-commerce SaaS Media
Mobile

app
User-gen

content
2-sided

market
The business you’re in
(Which means eye
charts like these.)
Customer Acquisition Cost
paid direct search wom
inherent
virality
VISITOR
Freemium/t...
Model + Stage = One Metric That Matters.
One Metric

That Matters.
The business you’re in
E-Com SaaS Mobile 2-Sided Media ...
Really? Just one?
Yes, one.
In a startup, focus is hard to achieve.
Having only one metric
addresses this problem.
www.theeastsiderla.com
Metrics are like squeeze toys.
http://www.flickr.com/photos/connortarter/4791605202/
Empathy
Stickiness
Virality
Revenue
Scale
E-
commerce
SaaS Media
Mobile

app
User-gen

content
2-sided

market
Interviews;...
Better: bit.ly/BigLeanTable
Drawing some lines in the sand.
A company loses a quarter of its
customers every year.
Is this good or bad?
Not knowing what normal is
makes you do stupid things.
Baseline:
5-7% growth a week
“A good growth rate during YC
is 5-7% a week,” he says. “If
you can hit 10% a week you're
doi...
It’s oxygen
You need customers to keep learning
It’s a substitute for solvency
PhotobyPaulMilleronFlickr.https://www.flickr...
Baseline:
10% visitor engagement/day
Fred Wilson’s social ratios
30% of users/month use web or mobile app
10% of users/day...
Baseline:
2-5% monthly churn
• The best SaaS get 1.5% - 3% a month. They have multiple Ph.D’s
on the job.
• Get below a 5%...
Baseline:
Calculating customer lifetime
25%

monthly churn
100/25=4

The average
customer lasts
4 months
5%

monthly churn...
Baseline:
CAC under 1/3 of CLV
• CLV is wrong. CAC Is probably wrong, too.
• Time kills all plans: It’ll take a long time ...
Who is worth more?
Today
A
Lifetime:
$200
Roberto Medri, Etsy
B
Lifetime:
$200
Visits
The Lean Analytics cycle
Draw a new line
Pivot or

give up
Try again
Success!
Did we move the
needle?
Measure
the results
Make changes
in productio...
Do AirBnB hosts
get more business
if their property is
professionally
photographed?
Gut instinct (hypothesis)
Professional photography helps AirBnB’s business
Candidate solution (MVP)
20 field photographers ...
5,000 shoots per month
by February 2012
Hang on a second.
Gut instinct (hypothesis)
Professional photography helps AirBnB’s business
SRSLY?
Draw a new line
Pivot or

give up
Try again
Success!
Did we move the
needle?
Measure
the results
Make changes
in productio...
“Gee, those
houses that do
well look really
nice.”
Maybe it’s the
camera.
“Computer: What
do all the
highly rented
houses ...
Landing page design A/B testing
Cohort analysis General analytics
URL shortening
Funnel analytics
Influencer Marketing
Publ...
When you’re a startup
your goal is to find a sustainable,
repeatable business model.
When you’re a big company
your goal is...
In a startup, the purpose of analytics is
to iterate to product/market fit
before the money runs out.
In a big company,
analytics replaces opinion with fact.
Arbitron and radio data
Times a song in “heavy
rotation” is played daily
2007 2012
266
Conclusions
“The most important figures that one
needs for management are unknown
or unknowable, but successful
management must neverth...
Pic by Twodolla on Flickr. http://www.flickr.com/photos/twodolla/3168857844
ARCHIMEDES
HAD TAKEN
BATHS BEFORE.
Once, a leader convinced others
in the absence of data.
Now, a leader knows
what questions to ask.
Alistair Croll
acroll@gmail.com
@acroll
Ben Yoskovitz
byosko@gmail.com
@byosko
Upcoming SlideShare
Loading in …5
×

Melbourne Business School - mba talk october 14 - croll - 40m - lean analytics

13,385 views

Published on

Brief overview of Lean Analytics for the Melbourne Business Schools Master of Business Analytics programme

Published in: Business
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Get access to 16,000 woodworking plans, Download 50 FREE Plans... ◆◆◆ http://ishbv.com/tedsplans/pdf
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Melbourne Business School - mba talk october 14 - croll - 40m - lean analytics

  1. 1. Lean Analytics Melbourne Business School October 14, 2015 @acroll
  2. 2. Percent of businesses that fail Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Year 7 Year 8 Year 9 Year 10 71%69%66%63%60%55%50%44%36%25% http://www.statisticbrain.com/startup-failure-by-industry/
  3. 3. Still operating after 4 years Finance Insurance and Real Estate Education and Health Agriculture Services Wholesale Mining Manufacturing Construction Retail Transportation Communication and Utilities Information 37% 45% 47% 47% 49% 51% 54% 55% 56% 56% 58% http://www.statisticbrain.com/startup-failure-by-industry/
  4. 4. Don’t sell what you can make. Make what you can sell. Kevin Costner is a lousy entrepreneur.
  5. 5. The core of Lean is iteration.
  6. 6. Unfortunately, we’re all liars.
  7. 7. Everyone’s idea is the best right? People love this part! (but that’s not always a good thing) This is where things fall apart. No data, no learning.
  8. 8. Most startups don’t know what they’ll be when they grow up. Hotmail
 was a database company Flickr
 was going to be an MMO Twitter
 was a podcasting company Autodesk
 made desktop automation Paypal
 first built for Palmpilots Freshbooks
 was invoicing for a web design firm Wikipedia
 was to be written by experts only Mitel
 was a lawnmower company
  9. 9. Analytics can help.
  10. 10. Analytics is the measurement of movement towards your business goals.
  11. 11. In a startup, the purpose of analytics is to iterate to product/market fit before the money runs out.
  12. 12. We’re bad at data.
  13. 13. What are the chances the other is a boy?
  14. 14. BB BG GB GG
  15. 15. 2 of 3 (66%) are boys. GB GG BG
  16. 16. A good metric is: Understandable If you’re busy explaining the data, you won’t be busy acting on it. Comparative Comparison is context. A ratio or rate The only way to measure change and roll up the tension between two metrics (MPH) Behavior
 changing What will you do differently based on the results you collect?
  17. 17. The simplest rule bad
 metric. If a metric won’t change how you behave, it’s a h"p://www.flickr.com/photos/circasassy/7858155676/
  18. 18. Metrics help you know yourself. Acquisition Hybrid Loyalty 70%
 of retailers 20%
 of retailers 10%
 of retailers You are just like Customers that buy >1x in 90d Once 2-2.5
 per year >2.5
 per year Your customers will buy from you Then you are in this mode 1-15% 15-30% >30% Low acquisition cost, high checkout Increasing return rates, market share Loyalty, selection, inventory size Focus on (Thanks to Kevin Hillstrom for this.)
  19. 19. MayAprMarFeb Slicing and dicing data Jan 0 5,000 Activeusers Cohort: Comparison of similar groups along a timeline. (this is the April cohort) A/B test: Changing one thing (i.e. color) and measuring the result (i.e. revenue.) Multivariate
 analysis Changing several things at once to see which correlates with a result. ☀ ☁ ☀ ☁ Segment: Cross-sectional comparison of all people divided by some attribute (age, gender, etc.) ☀ ☁
  20. 20. Which of these two companies is doing better?
  21. 21.   January February March April May Rev/customer $5.00 $4.50 $4.33 $4.25 $4.50 Is this company growing or stagnating? Cohort 1 2 3 4 5 January $5 $3 $2 $1 $0.5 February $6 $4 $2 $1 March $7 $6 $5 April   $8 $7 May       $9 How about this one?
  22. 22. Cohort 1 2 3 4 5 January $5 $3 $2 $1 $0.5 February $6 $4 $2 $1   March $7 $6 $5     April $8 $7       May $9         Averages $7 $5 $3 $1 $0.5 Look at the same data in cohorts
  23. 23. The Lean Analytics framework.
  24. 24. Eric’s three engines of growth Virality Make people invite friends. How many they tell, how fast they tell them. Price Spend money to get customers. Customers are worth more than they cost. Stickiness Keep people coming back. Approach Get customers faster than you lose them. Math that matters
  25. 25. Dave’s Pirate Metrics AARRR Acquisition How do your users become aware of you? SEO, SEM, widgets, email, PR, campaigns, blogs ... Activation Do drive-by visitors subscribe, use, etc? Features, design, tone, compensation, affirmation ... Retention Does a one-time user become engaged? Notifications, alerts, reminders, emails, updates... Revenue Do you make money from user activity? Transactions, clicks, subscriptions, DLC, analytics... Referral Do users promote your product? Email, widgets, campaigns, likes, RTs, affiliates...
  26. 26. Stage EMPATHY I’ve found a real, poorly-met need that a reachable market faces. STICKINESS I’ve figured out how to solve the problem in a way they will keep using and pay for. VIRALITY I’ve found ways to get them to tell their friends, either intrinsically or through incentives. REVENUE The users and features fuel growth organically and artificially. SCALE I’ve found a sustainable, scalable business with the right margins in a healthy ecosystem. Gate Thefivestages
  27. 27. Six business model archetypes. E-commerce SaaS Media Mobile
 app User-gen
 content 2-sided
 market The business you’re in
  28. 28. (Which means eye charts like these.) Customer Acquisition Cost paid direct search wom inherent virality VISITOR Freemium/trial offer Enrollment User Disengaged User Cancel Freemium churn Engaged User Free user disengagement Reactivate Cancel Trial abandonment rate Invite Others Paying Customer Reactivation
 rate Paid conversion FORMER USERS User Lifetime Value Reactivate FORMER CUSTOMERS Customer Lifetime Value Viral coefficient Viral rate Resolution Support data Account Cancelled Billing Info Exp. Paid Churn Rate Tiering Capacity Limit Upselling rate Upselling Disengaged DissatisfiedTrial Over
  29. 29. Model + Stage = One Metric That Matters. One Metric
 That Matters. The business you’re in E-Com SaaS Mobile 2-Sided Media UCG Empathy Stickiness Virality Revenue Scale Thestageyou’reat
  30. 30. Really? Just one?
  31. 31. Yes, one.
  32. 32. In a startup, focus is hard to achieve.
  33. 33. Having only one metric addresses this problem.
  34. 34. www.theeastsiderla.com
  35. 35. Metrics are like squeeze toys. http://www.flickr.com/photos/connortarter/4791605202/
  36. 36. Empathy Stickiness Virality Revenue Scale E- commerce SaaS Media Mobile
 app User-gen
 content 2-sided
 market Interviews; qualitative results; quantitative scoring; surveys Loyalty, conversion CAC, shares, reactivation Transaction, CLV Affiliates, white-label Engagement, churn Inherent virality, CAC Upselling, CAC, CLV API, magic #, mktplace Content, spam Invites, sharing Ads, donations Analytics, user data Inventory, listings SEM, sharing Transactions, commission Other verticals (Money from transactions) Downloads, churn, virality WoM, app ratings, CAC CLV, ARPDAU Spinoffs, publishers (Money from active users) Traffic, visits, returns Content virality, SEM CPE, affiliate %, eyeballs Syndication, licenses (Money from ad clicks)
  37. 37. Better: bit.ly/BigLeanTable
  38. 38. Drawing some lines in the sand.
  39. 39. A company loses a quarter of its customers every year. Is this good or bad?
  40. 40. Not knowing what normal is makes you do stupid things.
  41. 41. Baseline: 5-7% growth a week “A good growth rate during YC is 5-7% a week,” he says. “If you can hit 10% a week you're doing exceptionally well. If you can only manage 1%, it's a sign you haven't yet figured out what you're doing.” At revenue stage, measure growth in revenue. Before that, measure growth in active users. Paul Graham, Y Combinator • Are there enough people who really care enough to sustain a 5% growth rate? • Don’t strive for a 5% growth at the expense of really understanding your customers and building a meaningful solution • Once you’re a pre-revenue startup at or near product/market fit, you should have 5% growth of active users each week • Once you’re generating revenues, they should grow at 5% a week
  42. 42. It’s oxygen You need customers to keep learning It’s a substitute for solvency PhotobyPaulMilleronFlickr.https://www.flickr.com/photos/94674772@N03/8788576498
  43. 43. Baseline: 10% visitor engagement/day Fred Wilson’s social ratios 30% of users/month use web or mobile app 10% of users/day use web or mobile app 1% of users/day use it concurrently
  44. 44. Baseline: 2-5% monthly churn • The best SaaS get 1.5% - 3% a month. They have multiple Ph.D’s on the job. • Get below a 5% monthly churn rate before you know you’ve got a business that’s ready to grow (Mark MacLeod) and around 2% before you really step on the gas (David Skok) • Last-ditch appeals and reactivation can have a big impact. Facebook’s “don’t leave” reduces attrition by 7%.
  45. 45. Baseline: Calculating customer lifetime 25%
 monthly churn 100/25=4
 The average customer lasts 4 months 5%
 monthly churn 100/5=20
 The average customer lasts 20 months 2%
 monthly churn 100/2=50
 The average customer lasts 50 months
  46. 46. Baseline: CAC under 1/3 of CLV • CLV is wrong. CAC Is probably wrong, too. • Time kills all plans: It’ll take a long time to find out whether your churn and revenue projections are right • Cashflow: You’re basically “loaning” the customer money between acquisition and CLV. • It keeps you honest: Limiting yourself to a CAC of only a third of your CLV will forces you to verify costs sooner. Lifetime of 20 mo. $30/mo. per customer $600 CLV $200 CAC Now segment those users! 1/3 spend
  47. 47. Who is worth more? Today A Lifetime: $200 Roberto Medri, Etsy B Lifetime: $200 Visits
  48. 48. The Lean Analytics cycle
  49. 49. Draw a new line Pivot or
 give up Try again Success! Did we move the needle? Measure the results Make changes in production Design a test Hypothesis With data:
 find a commonality Without data: make a good guess Find a potential improvement Draw a linePick a KPI
  50. 50. Do AirBnB hosts get more business if their property is professionally photographed?
  51. 51. Gut instinct (hypothesis) Professional photography helps AirBnB’s business Candidate solution (MVP) 20 field photographers posing as employees Measure the results Compare photographed listings to a control group Make a decision Launch photography as a new feature for all hosts
  52. 52. 5,000 shoots per month by February 2012
  53. 53. Hang on a second.
  54. 54. Gut instinct (hypothesis) Professional photography helps AirBnB’s business SRSLY?
  55. 55. Draw a new line Pivot or
 give up Try again Success! Did we move the needle? Measure the results Make changes in production Design a test Hypothesis With data:
 find a commonality Without data: make a good guess Find a potential improvement Draw a linePick a KPI
  56. 56. “Gee, those houses that do well look really nice.” Maybe it’s the camera. “Computer: What do all the highly rented houses have in common?” Camera model. With data:
 find a commonality Without data: make a good guess
  57. 57. Landing page design A/B testing Cohort analysis General analytics URL shortening Funnel analytics Influencer Marketing Publisher analytics SaaS analytics Gaming analytics User interaction Customer satisfaction KPI dashboardsUser segmentation User analytics Spying on users
  58. 58. When you’re a startup your goal is to find a sustainable, repeatable business model. When you’re a big company your goal is to perpetuate one.
  59. 59. In a startup, the purpose of analytics is to iterate to product/market fit before the money runs out.
  60. 60. In a big company, analytics replaces opinion with fact.
  61. 61. Arbitron and radio data
  62. 62. Times a song in “heavy rotation” is played daily 2007 2012 266
  63. 63. Conclusions
  64. 64. “The most important figures that one needs for management are unknown or unknowable, but successful management must nevertheless take account of them.” Lloyd S. Nelson
  65. 65. Pic by Twodolla on Flickr. http://www.flickr.com/photos/twodolla/3168857844
  66. 66. ARCHIMEDES HAD TAKEN BATHS BEFORE.
  67. 67. Once, a leader convinced others in the absence of data.
  68. 68. Now, a leader knows what questions to ask.
  69. 69. Alistair Croll acroll@gmail.com @acroll Ben Yoskovitz byosko@gmail.com @byosko

×