0
Metrics forEarly-Stage Startups                       #scb13 – @andreasklinger
@andreasklinger“Startup Founder”“Product Guy”                    #scb13 – @andreasklinger
@andreasklinger“Startup Founder”“Product Guy”What we will cover- Why early stage metrics are different.- Applicable method...
The Main Problem with Metrics in Early Stage:- Product not ready or even wrong.- Little to no useable data.- Data points c...
Startup Founders.        #scb13 – @andreasklinger
Some startups haveideas for a new product.Looking for customersto buy (or at least use) it.Customers don’t buy.“early stag...
Product/Market Fittraction                                                  time                   With early stage       ...
Product/market fitBeing in a good marketwith a product that can satisfythat market.~ Marc Andreessen                       ...
Product/market fitBeing in a good marketwith a product that can satisfythat market.~ Marc Andreessen= People want your stuf...
Product/Market Fittraction                                       time                                #scb13 – @andreasklin...
Product/Market Fittraction                                                                   time           Discovery     ...
Product/Market Fittraction                                                                 time           Discovery       ...
Product/Market Fittraction                                                                     time           Discovery   ...
Product/Market Fit   traction                                                                          time              D...
Product/Market Fittraction                                                                   time           Discovery     ...
Product/Market Fittraction                                                            time           Discovery   Validatio...
Product/Market Fittraction                                                                    time              Discovery ...
Product/Market Fittraction                                                                         time              Disco...
Startups drown innon actionable datapoints.
What does this mean for my product?Are we on the right track?Meant for channel (referral)optimization.
Use of Metrics in Early Stage                                #scb13 – @andreasklinger
Use of Metrics in Early StageFocus on People- Not Hits, Pageviews, Visits, EventsValidation of customer feedback- saying v...
Segment Users into CohortsCohorts = Groups of people that share attributes.                                               ...
Segment Users into Cohorts                             #scb13 – @andreasklinger
Apply a framework: AARRR                           #scb13 – @andreasklinger
AcquisitionVisit / Signup / etc  ActivationUse of core feature  RetentionCome + use again   Referral  Invite + Signup   Re...
Example: PhotoappCohorts based on registration week   WK      acquisition   activation   retention   referral   revenue   ...
Acquisition                             Visit / Signup / etc                               Activation                     ...
Acquisition                     Visit / Signup / etc                       Activation                     Use of core feat...
Acquisition                             Visit / Signup / etc                               Activation                     ...
Long answer - It depends on two things:                             Acquisition                            Visit / Signup ...
Acquisition                     Visit / Signup / etc                       Activation                     Use of core feat...
BecauseRetention = f(user_happiness)
Because           Retention = f(user_happiness)Crashpadder’s Happiness Indexe.g. Weighted sum over core activities by host...
AARRR misses something                                  AcquisitionAnd Happiness is not everything                        ...
CUSTOMER INTENT          Acquisition          Activation          Retention           Referral           RevenueFULFILMENT...
Metrics are horrible way to understand customer intent                                                         (c) Dave Mc...
Metrics are horrible way to understand customer intent    Customer Intent = His “Job to be done”                          ...
Startups are obsessed by their solutionAnd ignore the customers job/problem                                   Market      ...
Metrics are horrible way to understand customer intent                                                         (c) Dave Mc...
Metrics are horrible way to understand customer intent       Great Way: Customer Interviews                               ...
Metrics are horrible way to understand customer intent           OK Way: Smoke Tests                                      ...
CUSTOMER INTENT (JOB)                       AcquisitionCustomerInterviews                       Activation                ...
Dig deeper - Good product centric KPIs:Framework: AARRR                                          #scb13 – @andreasklinger
Dig deeper - Good product centric KPIs:Linked to assumptions of your product (validation/falsify)Rate or Ratio (0.X or %)F...
“Industry Standards”Framework: AARRR       Use industry averages as reality check.                       Not as benchmark....
Example Mobile App: Pusher2000Trainer2peer pressure sport app (prelaunch “beta”).Rev channel: Trainers pay monthly fee.Two...
Dig Deeper - Dataschmutz        A layer of dirt obfuscating        your useable data.        Usually “wrong intent”.      ...
Dataschmutz    A layer of dirt obfuscating    your useable data.e.g. Traffic Spikes of wrongcustomer segment.(have wrong in...
Dataschmutz                                Exam              MySugr              is praised as              “beautiful app...
How to minimize the impact of Dataschmutz Base your KPIs on wavebreakers.  WK       visitors   acquisition activation    r...
DataschmutzCompetitions create artificial incentive   Competition Created   “Dataschmutz” Competitions (before P/M Fit)    ...
Dig Deeper - Metrics need to hurt                                    #scb13 – @andreasklinger
Dig Deeper - Metrics need to hurtIf you are not ashamed about the KPIs inyour dashboard than something is wrong.Either you...
Dig Deeper - Metrics need to hurtExample: Garmz/LOOKKGreat Numbers:90% activation (activation = vote)But they only voted f...
User activation.Some users are happy (power users)Some come never again.What differs them? It’s their activities in their ...
Example TwitterHow often did activated usersuse twitter in the first month:7 timesWhat did they do?Follow 20 people, follow...
Example TwitterExample Twitter:How did they get more peopleto follow 30people within7visits in the first 30 days?Ran assump...
Checkout Intercom.ioCustomer segmenting and messaging done right.                                       #scb13 – @andreask...
Summary          #scb13 – @andreasklinger
Summary- Use Metrics for Product and Customer Development.- Use Cohorts.- Use AARRR.- Figure Customer Intent through non-b...
Read onStartup metrics for Pirates by Dave McClurehttp://www.slideshare.net/dmc500hats/startup-metrics-for-pirates-long-ve...
Thank you@andreasklinger #SCB13Slides: http://slideshare.net/andreasklingerAll pictures: http://flickr.com/commons         ...
Upcoming SlideShare
Loading in...5
×

Metrics for early stage startups

33,799

Published on

How to use metrics in a startup that is yet before it's product market fit.

Published in: Business
0 Comments
145 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
33,799
On Slideshare
0
From Embeds
0
Number of Embeds
24
Actions
Shares
0
Downloads
670
Comments
0
Likes
145
Embeds 0
No embeds

No notes for slide

Transcript of "Metrics for early stage startups"

  1. 1. Metrics forEarly-Stage Startups #scb13 – @andreasklinger
  2. 2. @andreasklinger“Startup Founder”“Product Guy” #scb13 – @andreasklinger
  3. 3. @andreasklinger“Startup Founder”“Product Guy”What we will cover- Why early stage metrics are different.- Applicable methods & Lessons Learned. (this is an excerpt of 2h workshop - but with prettier slides ;) ) #scb13 – @andreasklinger
  4. 4. The Main Problem with Metrics in Early Stage:- Product not ready or even wrong.- Little to no useable data.- Data points contradict each other.- External Traffic can easily mess up our insights.- What is actionable?- Are we on the “right” track? #scb13 – @andreasklinger
  5. 5. Startup Founders. #scb13 – @andreasklinger
  6. 6. Some startups haveideas for a new product.Looking for customersto buy (or at least use) it.Customers don’t buy.“early stage” #scb13 – @andreasklinger
  7. 7. Product/Market Fittraction time With early stage I do not mean “X Years” I mean before product/market fit. #scb13 – @andreasklinger
  8. 8. Product/market fitBeing in a good marketwith a product that can satisfythat market.~ Marc Andreessen #scb13 – @andreasklinger
  9. 9. Product/market fitBeing in a good marketwith a product that can satisfythat market.~ Marc Andreessen= People want your stuff. #scb13 – @andreasklinger
  10. 10. Product/Market Fittraction time #scb13 – @andreasklinger
  11. 11. Product/Market Fittraction time Discovery Validation Efficiency Scale Steve Blank - Customer Development #scb13 – @andreasklinger
  12. 12. Product/Market Fittraction time Discovery Validation Efficiency Scale Find a product the market wants. #scb13 – @andreasklinger
  13. 13. Product/Market Fittraction time Discovery Validation Efficiency Scale Find a product Optimise the product the market wants. for the market. #scb13 – @andreasklinger
  14. 14. Product/Market Fit traction time Discovery Validation Efficiency Scale Find a product Optimise the product the market wants. for the market.People in search Most clonesfor new product start here. start here. #scb13 – @andreasklinger
  15. 15. Product/Market Fittraction time Discovery Validation Efficiency Scale Product & Customer Scale Marketing Development & Operations #scb13 – @andreasklinger
  16. 16. Product/Market Fittraction time Discovery Validation Efficiency Scale Startups have phases but they overlap. #scb13 – @andreasklinger
  17. 17. Product/Market Fittraction time Discovery Validation Efficiency Scale 83% of all startups are in here. #scb13 – @andreasklinger
  18. 18. Product/Market Fittraction time Discovery Validation Efficiency Scale 83% of all startups are in here. Most stuff we learn about web analytics is meant for this part #scb13 – @andreasklinger
  19. 19. Startups drown innon actionable datapoints.
  20. 20. What does this mean for my product?Are we on the right track?Meant for channel (referral)optimization.
  21. 21. Use of Metrics in Early Stage #scb13 – @andreasklinger
  22. 22. Use of Metrics in Early StageFocus on People- Not Hits, Pageviews, Visits, EventsValidation of customer feedback- saying vs doing- eg. did they really use the app?- does the app do what they need it to?Validation of internal opinions- believing vs knowing- eg. “Our users need/are/do/try…”Doublecheck + Falsify #scb13 – @andreasklinger
  23. 23. Segment Users into CohortsCohorts = Groups of people that share attributes. #scb13 – @andreasklinger
  24. 24. Segment Users into Cohorts #scb13 – @andreasklinger
  25. 25. Apply a framework: AARRR #scb13 – @andreasklinger
  26. 26. AcquisitionVisit / Signup / etc ActivationUse of core feature RetentionCome + use again Referral Invite + Signup Revenue $$$ Earned (c) Dave McClure
  27. 27. Example: PhotoappCohorts based on registration week WK acquisition activation retention referral revenue twice aPhotoapp registration first photo share … month 1 400 62,5% 25% 10% 2 875 65% 23% 9% 3 350 64% 26% 4% … … … … …
  28. 28. Acquisition Visit / Signup / etc Activation Use of core featureWhich Metrics to focus on? Retention Come + use again Referral Invite + Signup Revenue $$$ Earned (c) Dave McClure
  29. 29. Acquisition Visit / Signup / etc Activation Use of core feature Short Answer: RetentionFocus on Retention Come + use again Referral Invite + Signup Revenue $$$ Earned (c) Dave McClure
  30. 30. Acquisition Visit / Signup / etc Activation Use of core featureLong answer - It depends on two things: Retention Come + use againPhase of company Referral Invite + SignupType of Product (esp. Engine of Growth) Revenue $$$ Earned (c) Dave McClure
  31. 31. Long answer - It depends on two things: Acquisition Visit / Signup / etc Activation Use of core feature Retention Come + use again Referral Invite + Signup Revenue $$$ Earned Source: Lean Analytics Book - highly recommend
  32. 32. Acquisition Visit / Signup / etc Activation Use of core feature Short Answer: RetentionFocus on Retention Come + use again Referral Invite + Signup Revenue $$$ Earned (c) Dave McClure
  33. 33. BecauseRetention = f(user_happiness)
  34. 34. Because Retention = f(user_happiness)Crashpadder’s Happiness Indexe.g. Weighted sum over core activities by hosts.Cohorts by cities and time.= Health/Happiness Dashboard
  35. 35. AARRR misses something AcquisitionAnd Happiness is not everything Activation Retention Referral Revenue (c) Dave McClure
  36. 36. CUSTOMER INTENT Acquisition Activation Retention Referral RevenueFULFILMENT OF CUSTOMER INTENT (c) Dave McClure
  37. 37. Metrics are horrible way to understand customer intent (c) Dave McClure
  38. 38. Metrics are horrible way to understand customer intent Customer Intent = His “Job to be done” Products are bought because they solve a “job to be done”. Learn about Jobs to be done Framework Watch: http://bit.ly/cc-jtbd (c) Dave McClure
  39. 39. Startups are obsessed by their solutionAnd ignore the customers job/problem Market Job/ Problem Our Solution #scb13 – @andreasklinger
  40. 40. Metrics are horrible way to understand customer intent (c) Dave McClure
  41. 41. Metrics are horrible way to understand customer intent Great Way: Customer Interviews But: We bias our people, when we ask them. Even if we try not to. Reason: we believe our own bullshit. Watch: www.hackertalks.io (c) Dave McClure
  42. 42. Metrics are horrible way to understand customer intent OK Way: Smoke Tests If interviews suggest a new feature but you are Download Mobile Client unsure about critical mass (e.g. due to sample bias). Create Smoke Tests measure Click Conversion/ Signups Not for verification but falsification (c) Dave McClure
  43. 43. CUSTOMER INTENT (JOB) AcquisitionCustomerInterviews Activation Retention ReferralCustomerInterviews& Metrics Revenue FULFILMENT OF CUSTOMER INTENT (c) Dave McClure
  44. 44. Dig deeper - Good product centric KPIs:Framework: AARRR #scb13 – @andreasklinger
  45. 45. Dig deeper - Good product centric KPIs:Linked to assumptions of your product (validation/falsify)Rate or Ratio (0.X or %)Framework: AARRRComparable (To your history (or a/b). Forget the market)Explainable (If you don’t get it it means nothing) #scb13 – @andreasklinger
  46. 46. “Industry Standards”Framework: AARRR Use industry averages as reality check. Not as benchmark. - Usually very hard to get. - Everyone defines stuff different. - You might end up with another business model anyway. - Compare yourself vs your history data. #scb13 – @andreasklinger
  47. 47. Example Mobile App: Pusher2000Trainer2peer pressure sport app (prelaunch “beta”).Rev channel: Trainers pay monthly fee.Two sided => Segment AARRR for both sides (trainer/user)Marketplace => Value = Transactions / SupplierSocial Software => DAU/MAU to see if activated users stay activeChicken/Egg => You need a few very happy chickens for loads of eggs.Week/Week retention to see if public launch makes senseFramework: AARRROptimize retention: Interviews with Users that leftMeasure Trainer Happiness ScoreActivated User: More than two training sessionsPushups / User / Week to see if the core assumption (People will domore pushups) is valid #scb13 – @andreasklinger
  48. 48. Dig Deeper - Dataschmutz A layer of dirt obfuscating your useable data. Usually “wrong intent”. Usually our fault. (~ sample noise we created ourselves) #scb13 – @andreasklinger
  49. 49. Dataschmutz A layer of dirt obfuscating your useable data.e.g. Traffic Spikes of wrongcustomer segment.(have wrong intent) #scb13 – @andreasklinger
  50. 50. Dataschmutz Exam MySugr is praised as “beautiful app” example.… => Downloads => Problem: Not all are diabetic They focus on people who activated.
  51. 51. How to minimize the impact of Dataschmutz Base your KPIs on wavebreakers. WK visitors acquisition activation retention referral revenue twice aBirchbox visit registration first photo share … month 1 6000 66% / 4000 62,5% 25% 10% 2 25000 35% / 8750 65% 23% 9% 3 5000 70% / 3500 64% 26% 4%
  52. 52. DataschmutzCompetitions create artificial incentive Competition Created “Dataschmutz” Competitions (before P/M Fit) “Would you use my app and might are nothing but Teflon Marketing win 1.000.000 USD?” * Users had huge extra incentive. People come.can hurt your numbers. * Marketing People leave. * While we decided on how to relaunch we had dirty numbers. #scb13 – @andreasklinger
  53. 53. Dig Deeper - Metrics need to hurt #scb13 – @andreasklinger
  54. 54. Dig Deeper - Metrics need to hurtIf you are not ashamed about the KPIs inyour dashboard than something is wrong.Either you do not drill deep enough.Or you focus on the wrong KPIs. #scb13 – @andreasklinger
  55. 55. Dig Deeper - Metrics need to hurtExample: Garmz/LOOKKGreat Numbers:90% activation (activation = vote)But they only voted for friendsinstead of actually using the platform.We drilled (not far) deeper:Activation = Vote for 2 different designers. Boom. Pain. #scb13 – @andreasklinger
  56. 56. User activation.Some users are happy (power users)Some come never again.What differs them? It’s their activities in their first 30 days.How we think about Churn is wrong. #scb13 – @andreasklinger
  57. 57. Example TwitterHow often did activated usersuse twitter in the first month:7 timesWhat did they do?Follow 20 people, followedback by 10Churn:If they don’t keep them 7 timesin the first 30 days.They will lose them forever.It doesn’t matter when a userremembers to unsubscribe #scb13 – @andreasklinger
  58. 58. Example TwitterExample Twitter:How did they get more peopleto follow 30people within7visits in the first 30 days?Ran assumptions, createdfeatures and ran experiments!Watch: http://www.youtube.com/watch?v=L2snRPbhsF0 #scb13 – @andreasklinger
  59. 59. Checkout Intercom.ioCustomer segmenting and messaging done right. #scb13 – @andreasklinger
  60. 60. Summary #scb13 – @andreasklinger
  61. 61. Summary- Use Metrics for Product and Customer Development.- Use Cohorts.- Use AARRR.- Figure Customer Intent through non-biasing interviews.- Understand your type of product and it’s core drivers- Find KPIs that mean something to your specific product.- Avoid Telfonmarketing (eg Campaigns pre-product).- Filter Dataschmutz- Metrics need to hurt- Focus on the first 30 days of customer activation.TL;DR: Use metrics to validate/doublecheck.Use those insights when designing for/speaking to your customers. #scb13 – @andreasklinger
  62. 62. Read onStartup metrics for Pirates by Dave McClurehttp://www.slideshare.net/dmc500hats/startup-metrics-for-pirates-long-versionActionable Metrics by Ash Mauyrahttp://www.ashmaurya.com/2010/07/3-rules-to-actionable-metrics/Data Science Secrets by DJ Patil - LeWeb London 2012http://www.youtube.com/watch?v=L2snRPbhsF0Twitter sign up processhttp://www.lukew.com/ff/entry.asp?1128Lean startup metrics - @stueccleshttp://www.slideshare.net/stueccles/lean-startup-metricsCohorts in Google Analytics - @serenestudioshttp://danhilltech.tumblr.com/post/12509218078/startups-hacking-a-cohort-analysis-with-googleRob Fitzpatrick’s Collection of best Custdev Videos - @robfitzhttp://www.hackertalks.ioLean Analytics Bookhttp://leananalyticsbook.com/introducing-lean-analytics/Actionable Metrics - @lfittlhttp://www.slideshare.net/lfittl/actionable-metrics-lean-startup-meetup-berlinApp Engagement Matrix - Flurryhttp://blog.flurry.com/bid/90743/App-Engagement-The-Matrix-ReloadedMy Bloghttp://www.klinger.io #scb13 – @andreasklinger
  63. 63. Thank you@andreasklinger #SCB13Slides: http://slideshare.net/andreasklingerAll pictures: http://flickr.com/commons #scb13 – @andreasklinger
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×