Your SlideShare is downloading. ×
(Lean) Startup Metrics & Analytics      Nicola Junior Vitto - @njvitto @blomming  September 26, 2012 - Better Software - F...
Entrepreneurship...Entrepreneurs recognize they are going to have to follow their own internal compass and embrace      th...
Get out of the building (by Steve Blank)    Your business assumptions can be wrong     Go out and speak with your customers
Vanity Metrics
Vanity Metrics - TimelinesThere will always be something up and to the right
Vanity Metrics - Google Analytics                  Forget it!
Vanity Metrics - Google Analytics ...and this could become like a drug...killing you!
Vanity Metrics - TechCrunch & Co           “mDialog raises 5 Million          from Blackberry partners”          “GroupMe ...
Lean Metrics
What’s a startup?A startup is first of all an experiment, it is a humaninstitution designed to deliver a new product orserv...
What’s a Lean Startup?     Lean startup is a rigorous process           for iterating from Plan A            to a plan tha...
What Metrics Matter?    Example: Facebook on early days           150k users, little revenue          Many superior compet...
What Metrics Matter?    Example: Facebook on early days 75% of users visit one or more times per day   Within one month of...
What Metrics Matter?        Good Metrics (Three A’s)              Are Actionable              Can be Audited              ...
Actionable Metric     An actionable metric is one that ties      specific and repeatable actions             to observed re...
A/B Testing
A/B Testing - New Blomming home                  VS            Define your Goals
A/B Testing - Define your goals              A/B Testing Goals          One single important Goal:% Users that sign-up for ...
Multivariate Testing
Multivariate Testing at Google   “Yes, it’s true that a team at Google couldn’t decide between two blues, so they’re testi...
Multivariate Testing at Google
Multivariate Testing at Google
Multivariate Testing at Google      Google                     Bing
Multivariate Testing at Google      “That blue was worth at least $80M”       (by Paul Ray - Product Manager at Bing)
Funnels
David Cancel’s Funnel                 The Problem:         98% of people who visit your site,              leave never to ...
David Cancel’s Funnel                        2%  “The average conversion rate in the United States of   America wheter you...
David Cancel’s Funnel
David Cancel’s Funnel                    The Solution:          To find, engage and convert more       of your website visi...
Online Marketing   is shifting
Online Marketing Paradigm Shift                                  by David Cancel
Online Marketing Paradigm Shift                                  by David Cancel
Online Marketing Paradigm Shift                                  by David Cancel
Online Marketing Paradigm Shift  Internet Marketing 3.0 (different goal)                                    by David Cancel
Lifecycle Marketing                      by David Cancel
Optimize
Optimize for learning                 Optimize for learning     by creating a data-driven marketing framework
Optimize for learning          Analytics & A/B Testing are also             not exactly the answer
Optimize for learning         Use quantitative & qualitative data           to test *your* assumptions
Optimize for learning      The point of testing is to prove or disprove                assumptions quickly
Optimize for learning Pay attention to “overtesting” using Multivariate testing           because it generally takes too l...
The Local Maxima Problem
Frameworks
AARRR
AARRR Dave McClure’s Startup Metrics for Pirates is a good     framework for thinking about macro metrics
AARRR - Model
AARRR - Conversions Example
AARRR - Different orders (example)      1. Activation, Retention (happiness)            2. Revenue (sustainability)       ...
Acquisition
Tracking Marketing Channels                 Google Adwords                    Banners                  Social Media       ...
Tracking Marketing Channels        Use unique urls (tracking parameters)       on every url you create/give-out/pay for   ...
Measuring Cost of Visits               Cost per click (site visits)        Cost of campaign / Traffic generated    (Social ...
Measuring Cost of Acquisition         Cost of one visit / conversion rate =              Cost of an Acquisition      Acqui...
Marketing Channels                Per channel track:                    Scale                    Cost                  Con...
Acquisition    Cost of Acquisition < Customer Lifetime Profit    Then scale acquisition at a similar or lower rate       A/...
Acquisition including Referral If it costs €3 to acquire a customer through advertising                 But a customer bro...
Activation
Activation             Sure, they signed-up!!!
Activation             Sure, they signed-up!!!             BUT did the like it???
Activation at Facebook (i) which data points predict whether a user will stay?(ii) if they stay, which data points predict...
Activation at Facebook (i) which data points predict whether a user will stay?        having more than one session as a ne...
Activation at Facebook        “At the time, I heard many people criticize Facebook’s early investors, claiming that Facebo...
Activation Metrics    Activation metrics often drive other metrics            in Referral, Retention, Revenue
Retention
Returning Users                Tracking unregistered users that return often (useful for content sites with advertising)  ...
Cohort Analysis
Twitter Cohorts (example)                      by Fred Wilson’s blog   (more on: http://www.avc.com/a_vc/2009/10/the-cohor...
Twitter Cohorts (example)                      by Fred Wilson’s blog   (more on: http://www.avc.com/a_vc/2009/10/the-cohor...
Referral
Referral Mechanism             Social Media Sharing         Invite a friend type mechanism                 App Reviews    ...
Viral Coefficient         The average number of customers            that each customer refers
Viral Coefficient - Example              Each company customer         invites 5 other customers via email            20% c...
Viral Coefficient - ExampleViral Coefficient = Referrals x click through x conversion                   5 x 0.2 x 0.5 = 0.5
Viral Coefficient          A viral co-efficient greater than 1            means that every customer   gets more than one oth...
Viral Coefficient          A viral co-efficient greater than 1            means that every customer   gets more than one oth...
Viral Coefficient           Measure automatically for all           your built in referral mechanism         Give customers...
Viral Cycle Time     Viral Cycle Time = the average time taken for           a referral to turn into a customer           ...
Net Promoter Score  Net Promoter Score: Number of people who would         recommend your product to a friend             ...
Net Promoter Score “How likely is it you would recommend my product to a                     friend or colleague?” 0    1 ...
Net Promoter Score            Survey a small percentage       of your customers by email weekly           Record who you s...
Net Promoter Score          NPS is not a direct measurement           that translates to key metrics   Can act as a leadin...
Revenue
Making Money  Key for any business is to find out how much profit             they make for every customer         and scali...
Lifetime Customer Value             How much money you make          for every customer you acquire          For some this...
Cancellation Rate    Cancellation rate = is the percentage number of customers who cancel in any given month compared to  ...
Lifetime Customer Value (LTV)     LTV = monthly revenue x n° months in lifetime          N° months = 1 / cancellation rate...
Lifetime Customer Value (LTV)                     For example:  If €20 a month and cancellation rate is 10% a month       ...
Lifetime Customer Value (LTV)              Often not that simple.         Short term cancellation rates are           much...
Lifetime Customer Value (LTV)       r = short-term cancellation rate (e.g. 0,15)       p = long-term cancellation rate (e....
Some Tools
Event based tracking
Event based tracking
Event based tracking
Customer lifecycle  management
Customer lifecycle management
Customer lifecycle management
Tips
OODA Loop
Innovation Accounting  “To improve entrepreneurial outcomes, and tohold entrepreneurs accountable, we need to focus on the...
Innovation Accounting            Concentrate on macro metrics Everything you do should attempt to change a metric         ...
One Page Dashboard
Customer Development: pivoting
(Lean) Product DevelopmentProblem:unknown                              Solution:                             unknown
Scalable (lean) startup
SCRUM        Do you know SCRUM?
Kanban Board     I know: this isn’t an example of an exciting and well done Sprint :)
(Lean) Product Management             Well, trying to connect the dots:    what we learnt so far is how the Product Owner ...
Learning Boards - Lean Stack    We will talk about the Lean Stack of Ash Maurya     or other possible “lean mapping” frame...
Focus focus focus
Credits           Stuart Eccles - Lean Startup Metrics          Joshua Porter - Metrics-Driven Design          David Cance...
The Lean Startup
Running Lean
Business Models
An alternative?                  Lifestyle business
Thanks!          Blomming.com          Company.Blomming.com          Magazine.Blomming.com            Help.Blomming.com   ...
Upcoming SlideShare
Loading in...5
×

Lean Startup Metrics & Analytics

80,809

Published on

Published in: Business
11 Comments
438 Likes
Statistics
Notes
No Downloads
Views
Total Views
80,809
On Slideshare
0
From Embeds
0
Number of Embeds
34
Actions
Shares
0
Downloads
0
Comments
11
Likes
438
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Transcript of "Lean Startup Metrics & Analytics"

    1. 1. (Lean) Startup Metrics & Analytics Nicola Junior Vitto - @njvitto @blomming September 26, 2012 - Better Software - Firenze, Italy
    2. 2. Entrepreneurship...Entrepreneurs recognize they are going to have to follow their own internal compass and embrace the uncertainty as part of the journey. - Steve Blank - more on: http://steveblank.com/2009/06/29/agile-opportunism-entrepreneurial-dna/
    3. 3. Get out of the building (by Steve Blank) Your business assumptions can be wrong Go out and speak with your customers
    4. 4. Vanity Metrics
    5. 5. Vanity Metrics - TimelinesThere will always be something up and to the right
    6. 6. Vanity Metrics - Google Analytics Forget it!
    7. 7. Vanity Metrics - Google Analytics ...and this could become like a drug...killing you!
    8. 8. Vanity Metrics - TechCrunch & Co “mDialog raises 5 Million from Blackberry partners” “GroupMe is now sending one Million texts every day” “Google Plus hits 10 Million users in 2 weeks”
    9. 9. Lean Metrics
    10. 10. What’s a startup?A startup is first of all an experiment, it is a humaninstitution designed to deliver a new product orservice under conditions of extreme uncertaintyAt its heart, a startup is a catalyst that transformsideas into product. (From Lessons Learned blog By Eric Ries)
    11. 11. What’s a Lean Startup? Lean startup is a rigorous process for iterating from Plan A to a plan that works. (by Ash Maurya)
    12. 12. What Metrics Matter? Example: Facebook on early days 150k users, little revenue Many superior competitors What would indicate their future success?
    13. 13. What Metrics Matter? Example: Facebook on early days 75% of users visit one or more times per day Within one month of launching on a new campus, can acquire 90% of the students
    14. 14. What Metrics Matter? Good Metrics (Three A’s) Are Actionable Can be Audited Are Accessible
    15. 15. Actionable Metric An actionable metric is one that ties specific and repeatable actions to observed results (by Ash Maurya)
    16. 16. A/B Testing
    17. 17. A/B Testing - New Blomming home VS Define your Goals
    18. 18. A/B Testing - Define your goals A/B Testing Goals One single important Goal:% Users that sign-up for the service (as a seller) Secondary Goal: % Buyers that buy something after landing on the homepage
    19. 19. Multivariate Testing
    20. 20. Multivariate Testing at Google “Yes, it’s true that a team at Google couldn’t decide between two blues, so they’re testing 41 shades between each blue to see whic oneperforms better. I has a recent debate over whetera border should be 3, 4 or 5 pixels wide, and was asked to prove my case. I can’t operate in an environment like that.” (by Doug Bowman on Design at Google)
    21. 21. Multivariate Testing at Google
    22. 22. Multivariate Testing at Google
    23. 23. Multivariate Testing at Google Google Bing
    24. 24. Multivariate Testing at Google “That blue was worth at least $80M” (by Paul Ray - Product Manager at Bing)
    25. 25. Funnels
    26. 26. David Cancel’s Funnel The Problem: 98% of people who visit your site, leave never to return
    27. 27. David Cancel’s Funnel 2% “The average conversion rate in the United States of America wheter you’re selling elephants or iPods” (by Avinash Kaushik - Analytics Evangelist, Google)
    28. 28. David Cancel’s Funnel
    29. 29. David Cancel’s Funnel The Solution: To find, engage and convert more of your website visitors into customers
    30. 30. Online Marketing is shifting
    31. 31. Online Marketing Paradigm Shift by David Cancel
    32. 32. Online Marketing Paradigm Shift by David Cancel
    33. 33. Online Marketing Paradigm Shift by David Cancel
    34. 34. Online Marketing Paradigm Shift Internet Marketing 3.0 (different goal) by David Cancel
    35. 35. Lifecycle Marketing by David Cancel
    36. 36. Optimize
    37. 37. Optimize for learning Optimize for learning by creating a data-driven marketing framework
    38. 38. Optimize for learning Analytics & A/B Testing are also not exactly the answer
    39. 39. Optimize for learning Use quantitative & qualitative data to test *your* assumptions
    40. 40. Optimize for learning The point of testing is to prove or disprove assumptions quickly
    41. 41. Optimize for learning Pay attention to “overtesting” using Multivariate testing because it generally takes too long
    42. 42. The Local Maxima Problem
    43. 43. Frameworks
    44. 44. AARRR
    45. 45. AARRR Dave McClure’s Startup Metrics for Pirates is a good framework for thinking about macro metrics
    46. 46. AARRR - Model
    47. 47. AARRR - Conversions Example
    48. 48. AARRR - Different orders (example) 1. Activation, Retention (happiness) 2. Revenue (sustainability) 3. Acquisition, Referral (virality) (this is the Ash Maurya or 37signals order: “Charge from day one”)
    49. 49. Acquisition
    50. 50. Tracking Marketing Channels Google Adwords Banners Social Media Partnerships PR Blogging ...anything that links to your site!
    51. 51. Tracking Marketing Channels Use unique urls (tracking parameters) on every url you create/give-out/pay for Use bit.ly Use HTTP referrers to group PR
    52. 52. Measuring Cost of Visits Cost per click (site visits) Cost of campaign / Traffic generated (Social media) Cost of effort / Traffic generated
    53. 53. Measuring Cost of Acquisition Cost of one visit / conversion rate = Cost of an Acquisition Acquisition could be paid for acquisition of free sign-up ...unfortunately is not so simple
    54. 54. Marketing Channels Per channel track: Scale Cost Conversion Optimise and scale channels with best Cost of Acquisition (CoA) and potential to Scale
    55. 55. Acquisition Cost of Acquisition < Customer Lifetime Profit Then scale acquisition at a similar or lower rate A/B test acquisition channels to optimise
    56. 56. Acquisition including Referral If it costs €3 to acquire a customer through advertising But a customer brought in 2 other customers through referral The COA of that channel is €1
    57. 57. Activation
    58. 58. Activation Sure, they signed-up!!!
    59. 59. Activation Sure, they signed-up!!! BUT did the like it???
    60. 60. Activation at Facebook (i) which data points predict whether a user will stay?(ii) if they stay, which data points predict how active they’ll be after three months?
    61. 61. Activation at Facebook (i) which data points predict whether a user will stay? having more than one session as a new user and entering basic profile information.(ii) if they stay, which data points predict how active they’ll be after three months?how often a user was reached out by others, frequency of third party application use, and how available a user was on the site.
    62. 62. Activation at Facebook “At the time, I heard many people criticize Facebook’s early investors, claiming that Facebook had ‘no business model’ and only modest revenues relative to the valuation offered by its investors...”“...Facebook was different, because it employed a different Engine of Growth. It paid nothing for customer acquisition, and its high engagement meant that it was accumulating massive amount of customer attention every day.” (Eric Ries - The Lean Startup)
    63. 63. Activation Metrics Activation metrics often drive other metrics in Referral, Retention, Revenue
    64. 64. Retention
    65. 65. Returning Users Tracking unregistered users that return often (useful for content sites with advertising) Tracking registered users that return and take some sort of action.
    66. 66. Cohort Analysis
    67. 67. Twitter Cohorts (example) by Fred Wilson’s blog (more on: http://www.avc.com/a_vc/2009/10/the-cohort-analysis.html)
    68. 68. Twitter Cohorts (example) by Fred Wilson’s blog (more on: http://www.avc.com/a_vc/2009/10/the-cohort-analysis.html)
    69. 69. Referral
    70. 70. Referral Mechanism Social Media Sharing Invite a friend type mechanism App Reviews Word of Mouth
    71. 71. Viral Coefficient The average number of customers that each customer refers
    72. 72. Viral Coefficient - Example Each company customer invites 5 other customers via email 20% click on that email link 50% of those users convert into customers
    73. 73. Viral Coefficient - ExampleViral Coefficient = Referrals x click through x conversion 5 x 0.2 x 0.5 = 0.5
    74. 74. Viral Coefficient A viral co-efficient greater than 1 means that every customer gets more than one other customer on average
    75. 75. Viral Coefficient A viral co-efficient greater than 1 means that every customer gets more than one other customer on average Therefore your product will grow virally
    76. 76. Viral Coefficient Measure automatically for all your built in referral mechanism Give customers unique invite URLs and track conversions
    77. 77. Viral Cycle Time Viral Cycle Time = the average time taken for a referral to turn into a customer With the viral coefficient and viral cycle time you can determine your viral growth
    78. 78. Net Promoter Score Net Promoter Score: Number of people who would recommend your product to a friend A single question survey
    79. 79. Net Promoter Score “How likely is it you would recommend my product to a friend or colleague?” 0 1 2 3 4 5 6 7 8 9 10 NPS = % Promoters - % Detractors
    80. 80. Net Promoter Score Survey a small percentage of your customers by email weekly Record who you sent to so you don’t repeat the same people every week
    81. 81. Net Promoter Score NPS is not a direct measurement that translates to key metrics Can act as a leading indicator of other metrics
    82. 82. Revenue
    83. 83. Making Money Key for any business is to find out how much profit they make for every customer and scaling the number of customers
    84. 84. Lifetime Customer Value How much money you make for every customer you acquire For some this a single transaction For subscription businesses it depends on retention
    85. 85. Cancellation Rate Cancellation rate = is the percentage number of customers who cancel in any given month compared to total (paying) customers [Cancellation rate] = [product utility] + [service quality] + [acceptable price]
    86. 86. Lifetime Customer Value (LTV) LTV = monthly revenue x n° months in lifetime N° months = 1 / cancellation rate LTV = monthly revenue / cancellation rate
    87. 87. Lifetime Customer Value (LTV) For example: If €20 a month and cancellation rate is 10% a month LTV = €200
    88. 88. Lifetime Customer Value (LTV) Often not that simple. Short term cancellation rates are much higher than long term Use Cohort Analysis to determine
    89. 89. Lifetime Customer Value (LTV) r = short-term cancellation rate (e.g. 0,15) p = long-term cancellation rate (e.g. 0.03) s = number of months in the “short-term” age group (e.g. 3) (1-r)^s x (s + 1/p) = expected months
    90. 90. Some Tools
    91. 91. Event based tracking
    92. 92. Event based tracking
    93. 93. Event based tracking
    94. 94. Customer lifecycle management
    95. 95. Customer lifecycle management
    96. 96. Customer lifecycle management
    97. 97. Tips
    98. 98. OODA Loop
    99. 99. Innovation Accounting “To improve entrepreneurial outcomes, and tohold entrepreneurs accountable, we need to focus on the boring stuff: how to measure progress, how to setup milestones, how to prioritize work. This requires a new kind of accounting, specific to startups” (Eric Ries - The Lean Startup)
    100. 100. Innovation Accounting Concentrate on macro metrics Everything you do should attempt to change a metric Try to use as few tools as possible
    101. 101. One Page Dashboard
    102. 102. Customer Development: pivoting
    103. 103. (Lean) Product DevelopmentProblem:unknown Solution: unknown
    104. 104. Scalable (lean) startup
    105. 105. SCRUM Do you know SCRUM?
    106. 106. Kanban Board I know: this isn’t an example of an exciting and well done Sprint :)
    107. 107. (Lean) Product Management Well, trying to connect the dots: what we learnt so far is how the Product Owner can define and prioritize stories at every Sprint in a Startup that wants to be called, you know, Lean! :)
    108. 108. Learning Boards - Lean Stack We will talk about the Lean Stack of Ash Maurya or other possible “lean mapping” frameworks in a future presentation
    109. 109. Focus focus focus
    110. 110. Credits Stuart Eccles - Lean Startup Metrics Joshua Porter - Metrics-Driven Design David Cancel - Data-Driven Marketing Lean Startup Machine - How to measure the metrics that determine real progress(and the others specified directly in the slides when picking some parts of books or blogs)
    111. 111. The Lean Startup
    112. 112. Running Lean
    113. 113. Business Models
    114. 114. An alternative? Lifestyle business
    115. 115. Thanks! Blomming.com Company.Blomming.com Magazine.Blomming.com Help.Blomming.com Twitter.com/Blomming Facebook.com/Blomming nicola@blomming.com On FB & Twitter as: njvitto

    ×