Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The Art of Metrics

708 views

Published on

Notes on commonly missed aspects of designing an analytics strategy.
The first part of the presentation deals with well known lean-startup principles that many new practitioners seem (from my personal experience) to consistently overlook.
The second and third part deal with Churn and Customer Acquisition Cost, two important but not nice-looking metrics that startups seem to avoid or oversimplify because... they both "hurt".
Presentation delivered at Balkan Venture Forum 2015 in Zagreb.

Published in: Business
  • Be the first to comment

  • Be the first to like this

The Art of Metrics

  1. 1. George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015 The Art of Metrics notes about commonly missed aspects of designing an analytics strategy
  2. 2. George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015 george@ useful.io we build software products using Meteor.js
  3. 3. George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015 “Analytics” does not refer to a single business function or purpose. It’s a multi-purpose toolkit for decision making and monitoring. operations ensure your business operates smoothly troubleshooting / optimizing why things happen? how can I improve? predicting the future / investor pitch will my start-up grow to…?
  4. 4. http://www.growthverse.com
  5. 5. George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015 • Agenda: • Part A: things people forget over and over • Part B & C: things people avoid over and over
  6. 6. George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015 Part A: Tips on picking good metrics
  7. 7. George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015 Good metrics depend on your growth stage Stage EMPATHY I’ve found a real, poorly-met need that a reachable market faces. STICKINESS I’ve figured out how to solve the problem in a way they will keep using and pay for. VIRALITY I’ve found ways to get them to tell their friends, either intrinsically or through incentives. REVENUE The users and features fuel growth organically and artificially. SCALE I’ve found a sustainable, scalable business with the right margins in a healthy ecosystem. Gate Thefivestages Source: Lean Analytics (by A.Croll & B. Yoskovitz) highly recommended reading
  8. 8. George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015 Good metrics depend on what you do Empathy Stickiness Virality Revenue Scale E- commerce SaaS Media Mobile
 app User-gen
 content 2-sided
 market Interviews; qualitative results; quantitative scoring; surveys Loyalty, conversion CAC, shares, reactivation Transaction, CLV Affiliates, white-label Engagement, churn Inherent virality, CAC Upselling, CAC, CLV API, magic #, mktplace Content, spam Invites, sharing Ads, donations Analytics, user data Inventory, listings SEM, sharing Transactions, commission Other verticals (Money from transactions) Downloads, churn, virality WoM, app ratings, CAC CLV, ARPDAU Spinoffs, publishers (Money from active users) Traffic, visits, returns Content virality, SEM CPE, affiliate %, eyeballs Syndication, licenses (Money from ad clicks) A more information-rich version is available at: bit.ly/BigLeanTable Source: Lean Analytics (by A.Croll & B. Yoskovitz) highly recommended reading
  9. 9. George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015 Good metrics are actionable, auditable, accessible • Actionable: need to result to an action • Auditable: can be replicated, accuracy can be verified • Accessible: can be measured, everyone understands what they represent, everyone has access source: Eric Rees’ (The Lean Startup)
  10. 10. George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015 Good metrics are “mindful” • Qualitative versus quantitative metrics discover a story (*) vs prove/confirm a story • Vanity versus actionable metrics feel good vs changing your behavior by helping you pick a course of action. • Exploratory versus reporting metrics speculative insights vs day-to-day operations health status. • Leading versus lagging metrics predict the future vs explain the past. • Correlated versus causal metrics two metrics change together(**) vs one metric causes another metric to change.
  11. 11. George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015 Good metrics: are rates 0 50 100 150 200 April May June July August Active users (x100) 0 50 100 150 200 April May June July August 0 500 1000 1500 2000 April May June July August
  12. 12. George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015 Good metrics: are not only about growth Eric Rees’ (The Lean Startup) “Sustainable growth is a requirement for any business. […] New customers come from the actions of past customers” The three growth engines: Virality Rate of growing Stickiness Rate of churn (compound) Price Rate of expenditure
  13. 13. George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015 Part B: Churn rate
  14. 14. George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015 Churn • understand how good you are at retaining customers • identify what actions result in a higher retention rate
  15. 15. George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015 Churn • users who abandoned (over a specific time frame) • there’s more than one types of churn
  16. 16. George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015 –source: Localitics: http://info.localytics.com/blog/how-to-keep-your-app-users-with-the-3x3-rule “58% of app users churn in the first 30 days after they download your app, on average. And, over the first three months, 75% of app users will churn.”
  17. 17. George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015 Churn: an oversimplified formula https://www.shopify.com/technology/4018382-defining-churn-rate-no-really-this-actually-requires-an-entire-blog-post have you spotted the flaw? (number of churns during period) (number of customers at beginning of period)
  18. 18. George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015 Churn in practice… https://www.shopify.com/technology/4018382-defining-churn-rate-no-really-this-actually-requires-an-entire-blog-post http://christophjanz.blogspot.de/2012/05/know-your-user-cohorts.html
  19. 19. George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015 Predicting Churn compounding
 depends on user replacement 
 5% a month could be between… 
 46% - 80%… per year!
  20. 20. George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015 Measuring Churn • New customers will have a higher churn rate than mature customers • Measuring Churn will very likely force you to do more fine-grained segmentation. • Second reminder: make sure you factor-in user acquisition… by modeling / calculating separately http://sixteenventures.com/saas-churn-rate
  21. 21. George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015 “Churn will grow because your product market fit is just now improving” [as a result] “Churn plagues early stage companies with volatile product market fit, so assume the worst!” source: Nikos Moraitakis: https://medium.com/@moraitakis/early-stage-saas-financial-planning-1f9e848f63ef#.7084b188y
  22. 22. George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015 –source: Bessemer Venture Partners via http://sixteenventures.com/saas-churn-rate “[…] an acceptable [SaaS] churn rate is in the 5 – 7% range ANNUALLY, ”
  23. 23. George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015 –source: Fred Wilson (http://avc.com/2011/07/301010/) “[Typically] 30% of the registered users/ downloads will use the service each month 10% of the registered users/downloads will use the service each day the max number of concurrent users of a real- time service will be 10% of the number of daily users[…]” There is no single truth
  24. 24. George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015 Part C: Customer Acquisition Costs (CAC)
  25. 25. George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015 –Anonymous “Keeping a customer is [generally] cheaper than getting a new one.”
  26. 26. George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015 Customer Acquisition Costs (CAC) • The actual cost of growth (almost…) • Crash course (*): • Average Revenue Per User (ARPU): Revenues / Users • Lifetime Value (LTV): ARPU * Lifetime • Customer Acquisition Cost (CAC): 
 Total monthly acq costs / new monthly customers
  27. 27. George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015 –Bill Clark, Micro Ventures (http://microventures.com/education/metrics-every-startup-should-track) “you have to take every dollar you spend on reaching potential customers—from marketing to public relations and beyond—and divide it by the number of customers you actually acquire in a given period”
  28. 28. George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015 CAC: making or breaking? Are We Bleeding? CAC > LTV Are We Happy(*)? 3 x CAC < LTV
  29. 29. George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015 • CAC(s) are not uniform; put effort into creating a detailed model • CAC will most likely grow* (but beware channel capacity and growth rate ceilings)
  30. 30. George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015 Breaking down CAC source: Rob Moffat https://medium.com/venture-capital-growth-hacking/understanding-customer-acquisition-costs-74aec7538b4d
  31. 31. George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015 CAC considerations • Focus on CPA (acquisition) not CPV (visit); conversion rate varies dramatically per channel • Use it as a driver to ask questions like: “What can I do to grow volume of acquisitions through my free channels?” “What is likely to happen to CPA in my paid channels as I grow spend? What limits will I hit?” “Are there other marketing channels I should be looking at?” “Start layering in paid acquisition, starting with the cheapest channels” source: https://medium.com/venture-capital-growth-hacking/understanding-customer-acquisition-costs-74aec7538b4d
  32. 32. George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015 Recommended further reading (topics not covered in this presentation) • On financial planning: “https://medium.com/@moraitakis/early-stage-saas- financial-planning-1f9e848f63ef#.7084b188y" 
 — by Nikos Moraitakis • On market sizing: “http://www.bothsidesofthetable.com/2009/06/16/sorry-guys-its- the-size-of-the-wave-not-the-motion-of-the-ocean/“ 
 — by Mark Suster • On growth: “http://www.paulgraham.com/growth.html” 
 — by Paul Graham • On the tech implementation of analytics: “https://segment.com/academy/“ & “http://info.localytics.com/blog"
  33. 33. George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015 Front-load your analytics strategy. It will pay off.
  34. 34. George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015 Thank you george@useful.io george@voulgaris.net @gevou George Voulgaris, Ph.D. - The Art of Metrics Zagreb, Dec 4th, 2015

×