Lean Analytics @ MicroConf


Published on

Presentation on Lean Analytics at MicroConf 2013. Understanding what metrics are the most value, when, for your type of business.

* What makes a good metric?
* Types of metrics (qualitative vs. quantitative, vanity vs. actionable, etc.)
* Lean Analytics framework

Shared a number of case studies: Airbnb, Buffer, ClearFit, OffceDrop and others.

Published in: Business, Technology
  • I am involved so much as I have always like to share all he good and even nice things that I have used, developed

    faith in and let my readers get something special.

    you can download best internet filter software from there!
    Are you sure you want to  Yes  No
    Your message goes here
  • terimakasih !! :)
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Lean Analytics @ MicroConf

  1. 1. (or flail around blindly...it’s up to you)Measure What MattersBen Yoskovitz@byosko
  2. 2. I am aproduct guyentrepreneurbloggerangel investorhttp://instigatorblog.com
  3. 3. What I’ve learnedIt’s easy to get zombified.Recruitment sucks.Startup accelerators are fun.Eventually you get it right.TBD.
  4. 4. I’m not an analytics expert.
  5. 5. Or a data scientist.
  6. 6. http://leananalyticsbook.comBut I wrote a book onanalytics anyway.
  7. 7. Why did I write it?h"p://www.flickr.com/photos/horiavarlan/4290549806/sizes/l/in/photostream/
  8. 8. We’re all liars.
  9. 9. The basics of Lean StartupEveryone’s idea isthe best right?People lovethis part!(but that’s not always agood thing)This is where thingsfall apart.No data, nolearning.
  10. 10. What I Hope You Get From Ith"p://www.flickr.com/photos/romtomtom/4382603005/sizes/o/in/photostream/
  11. 11. Follow the Lean model, and itbecomes increasingly hard to lie,especially to yourself.The importance of intellectual honesty
  12. 12. Using your gut properlyInstincts are experiments.Data is proof.
  13. 13. Better decision making abilitiesEveryone has data, the key isfiguring out what pieces willimprove your learning anddecision making.
  14. 14. FocusDon’t chase shiny objects. Youmight succeed without focus, butit’ll be by accident.
  15. 15. Measure what matters1 What makes a good metric?2 Types of metrics3 Analytical superpowers4 Lean Analytics framework5 The One Metric That Matters6 Lean Analytics Cycle
  16. 16. What Makes a Good Metric?h"p://www.flickr.com/photos/artnoose/2263480871/sizes/l/in/photostream/
  17. 17. Analytics is the measurement ofmovement towards yourbusiness goals.What is analytics?
  18. 18. comparativewww.naturalhealth365.com  &  www.orangeclaire.com
  19. 19. understandablewww.speakingpracEcally.com
  20. 20. ratio or rateh"p://www.flickr.com/photos/pyth0ns/4816846174/
  21. 21. changes your behaviorh"p://www.flickr.com/photos/68001867@N00/426536440/sizes/z/in/photostream/
  22. 22. If it won’t changehow you behave,it’s abadmetric.If a metric won’t change howyou behave, it’s ah"p://www.flickr.com/photos/circasassy/7858155676/
  23. 23. h"p://www.flickr.com/photos/maImaIla/3822631755/Types of Metrics
  24. 24. Warm and fuzzy. Cold and hard.Unstructured,anecdotal,revealing, hard toaggregate.Numbers and stats;hard facts but lessinsight.Qualitativevs.Quantitative
  25. 25. Discover qualitatively.Prove quantitatively.Quantitative vs. qualitative data
  26. 26. Do hosts withprofessionalphotography getmore business?Airbnb experiments...
  27. 27. Professional photography helps Airbnb’s businessGut instinctConcierge MVP20 photographers in the fieldTest resultsTwo to three times more bookings!Back to the beginningUse additional data to keep experimenting
  28. 28. 5,000 shoots / month in Feb. 2012
  29. 29. Makes you feelgood but doesn’tchange how you’llact.Helps you pick adirection andchange yourbehavior.“Up and to the right.” These are good.Vanity Actionablevs.
  30. 30. HitsA metric from the early, foolish days of the Web.Count people instead.Page viewsMarginally better than hits. Unless you’re displayingad inventory, count people.VisitsIs this one person visiting a hundred times, or are ahundred people visiting once? Fail.UsersThis tells you nothing about what they did, why theystuck around, or if they left.Followers/friends/likesCount actions instead. Find out how many followerswill do your bidding.LoginsBut what are they actually doing when they login?Logins don’t tell you about actions and value.Vanity metrics are bad!
  31. 31. Speculative, tries tofind unexpected orinteresting insights.Predictable, keepsyou abreast ofnormal, day-to-dayoperations.The cool stuff. The necessary stuff.Exploratory Reportingvs.
  32. 32. Pivoting from friends to moms•Started as Circle of Friends•Leveraged Facebook early•Grew to 10M usersBut engagement sucked!
  33. 33. Moms are crazy!(in a good way)Engagement solved!• Messages to one another were on average 50% longer.• 115% more likely to attach a picture to a post they wrote.• 110% more likely to engage in a threaded (i.e. deep) conversation.• Friends, once invited, were 50% more likely to become engaged users.• 180% more likely to click on Facebook news feed items.• 60% more likely to accept invitations to the app.
  34. 34. Historical metricthat shows youhow you’re doing:reports the news.Number today thatshows a metrictomorrow: makesthe news.Try and get here.Start here.Lagging Leadingvs.
  35. 35. h"p://www.flickr.com/photos/bloke_with_camera/401812833/sizes/o/in/photostream/Analytical Superpowers(or what the heck is growth hacking?)
  36. 36. 110100100010000Ice cream consumption DrowningsJan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec
  37. 37. SummerIce creamconsumptionDrowningCorrelatedCausalCausalTwo variables thatchange in similar ways,perhaps becausethey’re linked tosomething else.CorrelatedAn independent factorthat directly impacts adependent one.Causalvs.
  38. 38. Correlation lets youpredict the futureCausality lets youchange the future“I will have 420 engaged usersand 75 paying customers nextmonth.”“If I can make more first-timevisitors stay on for 17 minutes Iwill increase sales in 90 days.”Find correlation Test causalityOptimize thecausal factorCausality is a superpower, because it letsyou change the future.
  39. 39. Lean Analytics Frameworkh"p://www.flickr.com/photos/ikhlasulamal/2331176652/
  40. 40. Your Business + StageWhat businessare you in?What stageare you at?•E-Commerce•SaaS•Free Mobile App•2-Sided Marketplace•Media•User-Generated Content•Empathy•Stickiness•Virality•Revenue•Scale
  41. 41. business models
  42. 42. The SaaS CustomerLifecycleCustomer Acquisition Costpaid direct search wominherentviralityVISITORFreemium/trial offerEnrollmentUserDisengaged UserCancelFreemiumchurnEngaged UserFree userdisengagementReactivateCancelTrial abandonment rateInvite OthersPaying CustomerReactivation ratePaidconversionFORMER USERSUser Lifetime ValueReactivateFORMER CUSTOMERSCustomer Lifetime ValueViral coefficientViral rateResolutionSupport dataAccount Cancelled Billing Info Exp.Paid Churn RateTieringCapacity LimitUpsellingrate UpsellingDisengaged DissatisfiedTrial Over
  43. 43. •Stage: Revenue / Scale•Model: SaaS (Paid)•Recruitment marketing andassessment software•Switched business models frommonthly subscription to pay perjob postingDoes recurring revenuework for everyone?
  44. 44. 10xrevenue increaseoff of 3x in salesvolume“People don’t do subscriptions for haircuts, hamburgers,and hiring. You have to understand your customer, whothey are, how and why they buy, and how they valueyour product or service.” - Ben Baldwin, co-founderLots of money!
  45. 45. h"p://www.flickr.com/photos/86791111@N00/3126955746/don’t just followthe leader
  46. 46. lean analytics stages
  47. 47. EMPATHYSTICKINESSGROWTHRATEVIRALITYREVENUESCALELean AnalyticsStagesI’ve found a real, poorly-met need thata reachable market faces.I’ve figured out how to solve the problem ina way they will adopt and pay for.I’ve built the right product/features/functionality that keeps users around.The users and features fuel growthorganically and artificially.I’ve found a sustainable, scalable businesswith the right margins in a healthyecosystem.“Gates” needed tomove forward
  48. 48. •Stage: Scale•Model: SaaS•Popular social sharing application•Focused primarily on customeracquisition•Charged from day oneFrom Stickiness to Scale(through Revenue)
  49. 49. 20%60%20%2%of visitors created an account(acquisition / Empathy)of sign-ups returned in the 1st month(engagement / Stickiness)of sign-ups were active after 6 months(engagement / Stickiness)convert from free to paid(Virality & Revenue)Buffer charges early to provepeople want the problem solved
  50. 50. skip steps at yourown risk
  51. 51. One MetricThat Matters.How It All Comes TogetherThe business you’re inE-Com SaaS Mobile 2-Sided Media UCGEmpathyStickinessViralityRevenueScaleThestageyou’reat
  52. 52. Choose only one metric anddraw a line in the sand.Putting the pieces together...
  53. 53. •Stage: Revenue•Model: SaaS (Freemium)•Paper and digital collaboration•180,000 users•Paid churn is their One MetricThat Matters (OMTM)Building a revenue engine
  54. 54. • Target < 4% paid churn (hitting 2% latelyon a monthly basis)•Anything over 5% means they don’t havea business that will generate positivemargin returns: the bucket is too leakyThe OMTM: Paid Churn
  55. 55. • Can we acquire more valuable customers?•What product features can increase engagement?• Can we improve customer support?•Was a marketing campaign successful?•Were customer complaints lowered?•Was a product upgrade valuable?If Paid Churn: Why & Next Steps:Paid Churn = “business health” indicator•Are the new customers not the right segment?• Did a marketing campaign fail?• Did a product upgrade fail somehow?• Is customer support falling apart?
  56. 56. Some interesting benchmarksGrowth5% / week (revenue or activeusers)Churn2% / monthEngaged visitors30% monthly users10% daily usersTime on site17 minutesPage load time< 5 secondsCLV:CAC3:1Mobile file size< 50MBFree to paid2% of free users
  57. 57. Lean Analytics cycleh"p://www.flickr.com/photos/jrodmanjr/4728457415/
  58. 58. Identify a key business problem,pick the OMTM, draw a line in thesand, and get started.Time to experiment
  59. 59. Draw a new linePivot orgive upTry againSuccess!Did we move theneedle?Measure theresultsMake changes inproductionDesign a testHypothesisWith data:find acommonalityWithout data:make a goodguessFind a potentialimprovementDraw a linePick a KPIThe Lean Analytics Cycle
  60. 60. Thank you.byosko@gmail.com@byoskoORDER!follow me.instigatorblog.comleananalyticsbook.comsubscribe.email me.