Lean Startup Metrics & Analytics

Nicola Junior Vitto
Nicola Junior VittoHead of Product at ShopFully International Group (DoveConviene) | Co-Founder at Blomming
(Lean) Startup Metrics & Analytics

      Nicola Junior Vitto - @njvitto @blomming
  September 26, 2012 - Better Software - Firenze, Italy
Entrepreneurship



...Entrepreneurs recognize they are going to have
 to follow their own internal compass and embrace
      the uncertainty as part of the journey.

                              - Steve Blank -
 more on: http://steveblank.com/2009/06/29/agile-opportunism-entrepreneurial-dna/
Get out of the building (by Steve Blank)




    Your business assumptions can be wrong
     Go out and speak with your customers
Vanity Metrics
Vanity Metrics - Timelines




There will always be something up and to the right
Vanity Metrics - Google Analytics




                  Forget it!
Vanity Metrics - Google Analytics




 ...and this could become like a drug...killing you!
Vanity Metrics - TechCrunch & Co


           “mDialog raises 5 Million
          from Blackberry partners”

          “GroupMe is now sending
          one Million texts every day”

         “Google Plus hits 10 Million
             users in 2 weeks”
Lean Metrics
What’s a startup?


A startup is first of all an experiment, it is a human
institution designed to deliver a new product or
service under conditions of extreme uncertainty

At its heart, a startup is a catalyst that transforms
ideas into product.

       (From Lessons Learned blog By Eric Ries)
What’s a Lean Startup?



     Lean startup is a rigorous process
           for iterating from Plan A
            to a plan that works.

               (by Ash Maurya)
What Metrics Matter?


    Example: Facebook on early days

           150k users, little revenue

          Many superior competitors


   What would indicate their future success?
What Metrics Matter?


    Example: Facebook on early days

 75% of users visit one or more times per day

   Within one month of launching on a new
  campus, can acquire 90% of the students
What Metrics Matter?

        Good Metrics (Three A’s)

              Are Actionable

              Can be Audited

              Are Accessible
Actionable Metric



     An actionable metric is one that ties
      specific and repeatable actions
             to observed results

                (by Ash Maurya)
A/B Testing
A/B Testing - New Blomming home




                  VS




            Define your Goals
A/B Testing - Define your goals


              A/B Testing Goals

          One single important Goal:
% Users that sign-up for the service (as a seller)

                 Secondary Goal:
         % Buyers that buy something
         after landing on the homepage
Multivariate Testing
Multivariate Testing at Google


   “Yes, it’s true that a team at Google couldn’t
 decide between two blues, so they’re testing 41
    shades between each blue to see whic one
performs better. I has a recent debate over wheter
a border should be 3, 4 or 5 pixels wide, and was
               asked to prove my case.
 I can’t operate in an environment like that.”

       (by Doug Bowman on Design at Google)
Multivariate Testing at Google
Multivariate Testing at Google
Multivariate Testing at Google




      Google                     Bing
Multivariate Testing at Google




      “That blue was worth at least $80M”

       (by Paul Ray - Product Manager at Bing)
Funnels
David Cancel’s Funnel




                 The Problem:
         98% of people who visit your site,
              leave never to return
David Cancel’s Funnel



                        2%
  “The average conversion rate in the United States of
   America wheter you’re selling elephants or iPods”

   (by Avinash Kaushik - Analytics Evangelist, Google)
David Cancel’s Funnel
David Cancel’s Funnel




                    The Solution:
          To find, engage and convert more
       of your website visitors into customers
Online Marketing
   is shifting
Online Marketing Paradigm Shift




                                  by David Cancel
Online Marketing Paradigm Shift




                                  by David Cancel
Online Marketing Paradigm Shift




                                  by David Cancel
Online Marketing Paradigm Shift




  Internet Marketing 3.0 (different goal)
                                    by David Cancel
Lifecycle Marketing




                      by David Cancel
Optimize
Optimize for learning




                 Optimize for learning
     by creating a data-driven marketing framework
Optimize for learning




          Analytics & A/B Testing are also
             not exactly the answer
Optimize for learning




         Use quantitative & qualitative data
           to test *your* assumptions
Optimize for learning




      The point of testing is to prove or disprove
                assumptions quickly
Optimize for learning




 Pay attention to “overtesting” using Multivariate testing
           because it generally takes too long
The Local Maxima Problem
Frameworks
AARRR
AARRR

 Dave McClure’s Startup Metrics for Pirates is a good
     framework for thinking about macro metrics
AARRR - Model
AARRR - Conversions Example
AARRR - Different orders (example)


      1. Activation, Retention (happiness)

            2. Revenue (sustainability)

         3. Acquisition, Referral (virality)

      (this is the Ash Maurya or 37signals order:
                 “Charge from day one”)
Acquisition
Tracking Marketing Channels


                 Google Adwords
                    Banners
                  Social Media
                  Partnerships
                       PR
                    Blogging

        ...anything that links to your site!
Tracking Marketing Channels


        Use unique urls (tracking parameters)
       on every url you create/give-out/pay for

                     Use bit.ly

         Use HTTP referrers to group PR
Measuring Cost of Visits



               Cost per click (site visits)

        Cost of campaign / Traffic generated

    (Social media) Cost of effort / Traffic generated
Measuring Cost of Acquisition

         Cost of one visit / conversion rate =
              Cost of an Acquisition

      Acquisition could be paid for acquisition
                 of free sign-up


          ...unfortunately is not so simple
Marketing Channels

                Per channel track:

                    Scale
                    Cost
                  Conversion


           Optimise and scale channels
       with best Cost of Acquisition (CoA)
             and potential to Scale
Acquisition



    Cost of Acquisition < Customer Lifetime Profit

    Then scale acquisition at a similar or lower rate

       A/B test acquisition channels to optimise
Acquisition including Referral


 If it costs €3 to acquire a customer through advertising

                 But a customer brought
         in 2 other customers through referral

             The COA of that channel is €1
Activation
Activation



             Sure, they signed-up!!!
Activation



             Sure, they signed-up!!!


             BUT did the like it???
Activation at Facebook



 (i) which data points predict whether a user will stay?


(ii) if they stay, which data points predict how active they’ll
                    be after three months?
Activation at Facebook

 (i) which data points predict whether a user will stay?
        having more than one session as a new user
           and entering basic profile information.


(ii) if they stay, which data points predict how active they’ll
                    be after three months?
how often a user was reached out by others, frequency of
  third party application use, and how available a user was
                           on the site.
Activation at Facebook

        “At the time, I heard many people criticize
 Facebook’s early investors, claiming that Facebook had
   ‘no business model’ and only modest revenues
    relative to the valuation offered by its investors...”

“...Facebook was different, because it employed a different
     Engine of Growth. It paid nothing for customer
     acquisition, and its high engagement meant that
    it was accumulating massive amount of customer
                   attention every day.”

              (Eric Ries - The Lean Startup)
Activation Metrics




    Activation metrics often drive other metrics
            in Referral, Retention, Revenue
Retention
Returning Users



                Tracking unregistered users
 that return often (useful for content sites with advertising)

                Tracking registered users
         that return and take some sort of action.
Cohort Analysis
Twitter Cohorts (example)




                      by Fred Wilson’s blog
   (more on: http://www.avc.com/a_vc/2009/10/the-cohort-analysis.html)
Twitter Cohorts (example)




                      by Fred Wilson’s blog
   (more on: http://www.avc.com/a_vc/2009/10/the-cohort-analysis.html)
Referral
Referral Mechanism


             Social Media Sharing

         Invite a friend type mechanism

                 App Reviews

                Word of Mouth
Viral Coefficient




         The average number of customers
            that each customer refers
Viral Coefficient - Example


              Each company customer
         invites 5 other customers via email

            20% click on that email link

     50% of those users convert into customers
Viral Coefficient - Example




Viral Coefficient = Referrals x click through x conversion

                   5 x 0.2 x 0.5 = 0.5
Viral Coefficient


          A viral co-efficient greater than 1
            means that every customer
   gets more than one other customer on average
Viral Coefficient


          A viral co-efficient greater than 1
            means that every customer
   gets more than one other customer on average



       Therefore your product will grow virally
Viral Coefficient


           Measure automatically for all
           your built in referral mechanism



         Give customers unique invite URLs
                and track conversions
Viral Cycle Time



     Viral Cycle Time = the average time taken for
           a referral to turn into a customer


              With the viral coefficient
                and viral cycle time
         you can determine your viral growth
Net Promoter Score



  Net Promoter Score: Number of people who would
         recommend your product to a friend



              A single question survey
Net Promoter Score


 “How likely is it you would recommend my product to a
                     friend or colleague?”


 0    1     2   3     4    5    6    7    8     9   10




          NPS = % Promoters - % Detractors
Net Promoter Score


            Survey a small percentage
       of your customers by email weekly


           Record who you sent to so
       you don’t repeat the same people
                   every week
Net Promoter Score



          NPS is not a direct measurement
           that translates to key metrics



   Can act as a leading indicator of other metrics
Revenue
Making Money




  Key for any business is to find out how much profit
             they make for every customer
         and scaling the number of customers
Lifetime Customer Value


             How much money you make
          for every customer you acquire

          For some this a single transaction

 For subscription businesses it depends on retention
Cancellation Rate


    Cancellation rate = is the percentage number of
 customers who cancel in any given month compared to
                total (paying) customers


                     [Cancellation rate] =
   [product utility] + [service quality] + [acceptable price]
Lifetime Customer Value (LTV)



     LTV = monthly revenue x n° months in lifetime

          N° months = 1 / cancellation rate

       LTV = monthly revenue / cancellation rate
Lifetime Customer Value (LTV)



                     For example:

  If €20 a month and cancellation rate is 10% a month

                     LTV = €200
Lifetime Customer Value (LTV)


              Often not that simple.
         Short term cancellation rates are
           much higher than long term



        Use Cohort Analysis to determine
Lifetime Customer Value (LTV)


       r = short-term cancellation rate (e.g. 0,15)
       p = long-term cancellation rate (e.g. 0.03)
  s = number of months in the “short-term” age group
                        (e.g. 3)


         (1-r)^s x (s + 1/p) = expected months
Some Tools
Event based tracking
Event based tracking
Event based tracking
Customer lifecycle
  management
Customer lifecycle management
Customer lifecycle management
Tips
OODA Loop
Innovation Accounting


  “To improve entrepreneurial outcomes, and to
hold entrepreneurs accountable, we need to focus
 on the boring stuff: how to measure progress,
  how to setup milestones, how to prioritize
 work. This requires a new kind of accounting,
              specific to startups”

            (Eric Ries - The Lean Startup)
Innovation Accounting



            Concentrate on macro metrics

 Everything you do should attempt to change a metric

          Try to use as few tools as possible
One Page Dashboard
Customer Development: pivoting
(Lean) Product Development



Problem:
unknown




                              Solution:
                             unknown
Scalable (lean) startup
SCRUM




        Do you know SCRUM?
Kanban Board




     I know: this isn’t an example of an exciting and well done Sprint :)
(Lean) Product Management



             Well, trying to connect the dots:
    what we learnt so far is how the Product Owner
     can define and prioritize stories at every Sprint
 in a Startup that wants to be called, you know, Lean! :)
Learning Boards - Lean Stack




    We will talk about the Lean Stack of Ash Maurya
     or other possible “lean mapping” frameworks
                  in a future presentation
Focus focus focus
Credits

           Stuart Eccles - Lean Startup Metrics

          Joshua Porter - Metrics-Driven Design

          David Cancel - Data-Driven Marketing

               Lean Startup Machine -
 How to measure the metrics that determine real progress

(and the others specified directly in the slides when picking
              some parts of books or blogs)
The Lean Startup
Running Lean
Business Models
An alternative?




                  Lifestyle business
Thanks!

          Blomming.com
          Company.Blomming.com
          Magazine.Blomming.com
            Help.Blomming.com
           Twitter.com/Blomming
          Facebook.com/Blomming

           nicola@blomming.com
          On FB & Twitter as: njvitto
1 of 115

Recommended

Influencer marketing by
Influencer marketingInfluencer marketing
Influencer marketingBLAQUE VOCATIONAL TRAINING CENTRE
1.1K views3 slides
The marketing agency blueprint the handbook for building hybrid pr, seo, cont... by
The marketing agency blueprint the handbook for building hybrid pr, seo, cont...The marketing agency blueprint the handbook for building hybrid pr, seo, cont...
The marketing agency blueprint the handbook for building hybrid pr, seo, cont...MarketingForum
123 views255 slides
Influencer Marketing by
Influencer Marketing Influencer Marketing
Influencer Marketing Rebecca Herson
8.8K views13 slides
Social Media Marketing by
Social Media MarketingSocial Media Marketing
Social Media MarketingTeaching Excellence
1.7K views48 slides
Customer journey maps by
Customer journey mapsCustomer journey maps
Customer journey mapsRAMIALHATAB
921 views27 slides
What is an insight? by
What is an insight?What is an insight?
What is an insight?Umar Ghumman
147.8K views41 slides

More Related Content

What's hot

Startup Metrics for Pirates: AARRR! (Startonomics SF 2008) by
Startup Metrics for Pirates: AARRR! (Startonomics SF 2008)Startup Metrics for Pirates: AARRR! (Startonomics SF 2008)
Startup Metrics for Pirates: AARRR! (Startonomics SF 2008)Dealmaker Media
21.2K views58 slides
關鍵字廣告簡報 by
關鍵字廣告簡報關鍵字廣告簡報
關鍵字廣告簡報偉盛世科技 (網路電子商務)
2.5K views40 slides
What is consumer insight in advertising? by
What is consumer insight in advertising?What is consumer insight in advertising?
What is consumer insight in advertising?Aleksas Drozdovskis
104.7K views145 slides
How to develop Insights by
How to develop InsightsHow to develop Insights
How to develop InsightsTC Miles
10.2K views15 slides
Introduction to Google Analytics by
Introduction to Google AnalyticsIntroduction to Google Analytics
Introduction to Google AnalyticsArjun Parekh
16.1K views34 slides
What Works in B2B Marketing. Difference Between B2B and B2C Marketing. by
What Works in B2B Marketing. Difference Between B2B and B2C Marketing. What Works in B2B Marketing. Difference Between B2B and B2C Marketing.
What Works in B2B Marketing. Difference Between B2B and B2C Marketing. IndiaMART.com
2.3K views7 slides

What's hot(20)

Startup Metrics for Pirates: AARRR! (Startonomics SF 2008) by Dealmaker Media
Startup Metrics for Pirates: AARRR! (Startonomics SF 2008)Startup Metrics for Pirates: AARRR! (Startonomics SF 2008)
Startup Metrics for Pirates: AARRR! (Startonomics SF 2008)
Dealmaker Media21.2K views
What is consumer insight in advertising? by Aleksas Drozdovskis
What is consumer insight in advertising?What is consumer insight in advertising?
What is consumer insight in advertising?
Aleksas Drozdovskis104.7K views
How to develop Insights by TC Miles
How to develop InsightsHow to develop Insights
How to develop Insights
TC Miles10.2K views
Introduction to Google Analytics by Arjun Parekh
Introduction to Google AnalyticsIntroduction to Google Analytics
Introduction to Google Analytics
Arjun Parekh16.1K views
What Works in B2B Marketing. Difference Between B2B and B2C Marketing. by IndiaMART.com
What Works in B2B Marketing. Difference Between B2B and B2C Marketing. What Works in B2B Marketing. Difference Between B2B and B2C Marketing.
What Works in B2B Marketing. Difference Between B2B and B2C Marketing.
IndiaMART.com2.3K views
How to Build a Content Marketing Strategy by NewsCred
How to Build a Content Marketing StrategyHow to Build a Content Marketing Strategy
How to Build a Content Marketing Strategy
NewsCred68K views
B2B vs. B2C: 10 Marketing Experts Have Their Say (SlideShare) by Babcock Jenkins
B2B vs. B2C: 10 Marketing Experts Have Their Say (SlideShare)B2B vs. B2C: 10 Marketing Experts Have Their Say (SlideShare)
B2B vs. B2C: 10 Marketing Experts Have Their Say (SlideShare)
Babcock Jenkins10.9K views
Mastering GA4: How To Use The New Google Analytics Like A Pro by Search Engine Journal
 Mastering GA4: How To Use The New Google Analytics Like A Pro Mastering GA4: How To Use The New Google Analytics Like A Pro
Mastering GA4: How To Use The New Google Analytics Like A Pro
Product Discovery At Google by John Gibbon
Product Discovery At GoogleProduct Discovery At Google
Product Discovery At Google
John Gibbon2.5K views
Guide to Customer Journey Mapping by Jake Truemper
Guide to Customer Journey MappingGuide to Customer Journey Mapping
Guide to Customer Journey Mapping
Jake Truemper2.4K views
Influencer Marketing: A new way forward! by KUNALGOPALANI1
Influencer Marketing: A new way forward!Influencer Marketing: A new way forward!
Influencer Marketing: A new way forward!
KUNALGOPALANI193 views
A Hunter's Guide To BS-Free Insights by Tom Morton
A Hunter's Guide To BS-Free InsightsA Hunter's Guide To BS-Free Insights
A Hunter's Guide To BS-Free Insights
Tom Morton14.7K views
The Cheating Strategist’s guide to Mary Meeker’s Digital Trends 2015 by Michael Goldstein
The Cheating Strategist’s guide to Mary Meeker’s Digital Trends 2015The Cheating Strategist’s guide to Mary Meeker’s Digital Trends 2015
The Cheating Strategist’s guide to Mary Meeker’s Digital Trends 2015
Michael Goldstein27.5K views
B2B Digital Marketing Roadmap by Moses Gomes
B2B Digital Marketing RoadmapB2B Digital Marketing Roadmap
B2B Digital Marketing Roadmap
Moses Gomes4.9K views

Similar to Lean Startup Metrics & Analytics

Growth Hacking - 10 Key Checklist by
Growth Hacking - 10 Key Checklist Growth Hacking - 10 Key Checklist
Growth Hacking - 10 Key Checklist Bryan Ferguson
36.8K views23 slides
Multi-Channel Attribution - Where do Leads Come From? by
Multi-Channel Attribution - Where do Leads Come From?Multi-Channel Attribution - Where do Leads Come From?
Multi-Channel Attribution - Where do Leads Come From?Joshua Tree Internet Media, LLC
3K views37 slides
Applying the science of measurement to the art of advertising - 20 january 2... by
Applying the science of measurement to the art of advertising  - 20 january 2...Applying the science of measurement to the art of advertising  - 20 january 2...
Applying the science of measurement to the art of advertising - 20 january 2...Ron Jacobs
241 views35 slides
Hotspot ALPHA Camp_Setting Course with Metrics by
Hotspot ALPHA Camp_Setting Course with MetricsHotspot ALPHA Camp_Setting Course with Metrics
Hotspot ALPHA Camp_Setting Course with MetricsALPHA Camp
283 views77 slides
Startup analytics by
Startup analyticsStartup analytics
Startup analyticsTopi Järvinen
94 views42 slides
Social Media Business Intelligence (SNHU Bootcamp Presentation) by
Social Media Business Intelligence (SNHU Bootcamp Presentation)Social Media Business Intelligence (SNHU Bootcamp Presentation)
Social Media Business Intelligence (SNHU Bootcamp Presentation)wedu, Inc
529 views63 slides

Similar to Lean Startup Metrics & Analytics(20)

Growth Hacking - 10 Key Checklist by Bryan Ferguson
Growth Hacking - 10 Key Checklist Growth Hacking - 10 Key Checklist
Growth Hacking - 10 Key Checklist
Bryan Ferguson36.8K views
Applying the science of measurement to the art of advertising - 20 january 2... by Ron Jacobs
Applying the science of measurement to the art of advertising  - 20 january 2...Applying the science of measurement to the art of advertising  - 20 january 2...
Applying the science of measurement to the art of advertising - 20 january 2...
Ron Jacobs241 views
Hotspot ALPHA Camp_Setting Course with Metrics by ALPHA Camp
Hotspot ALPHA Camp_Setting Course with MetricsHotspot ALPHA Camp_Setting Course with Metrics
Hotspot ALPHA Camp_Setting Course with Metrics
ALPHA Camp283 views
Social Media Business Intelligence (SNHU Bootcamp Presentation) by wedu, Inc
Social Media Business Intelligence (SNHU Bootcamp Presentation)Social Media Business Intelligence (SNHU Bootcamp Presentation)
Social Media Business Intelligence (SNHU Bootcamp Presentation)
wedu, Inc529 views
The Future of Analytics: Multichannel Attribution by WP Engine UK
The Future of Analytics: Multichannel Attribution The Future of Analytics: Multichannel Attribution
The Future of Analytics: Multichannel Attribution
WP Engine UK413 views
The Beginner’s Guide to Growth Hacking by Frederik Hermann
The Beginner’s Guide to Growth HackingThe Beginner’s Guide to Growth Hacking
The Beginner’s Guide to Growth Hacking
Frederik Hermann38.6K views
The Freemium Model and A/B Testing by Webs
The Freemium Model and A/B TestingThe Freemium Model and A/B Testing
The Freemium Model and A/B Testing
Webs3.2K views
Funnels Workshop Web Summit 2014 @geckoboard @GA by Sofia Quintero
Funnels Workshop Web Summit 2014 @geckoboard @GAFunnels Workshop Web Summit 2014 @geckoboard @GA
Funnels Workshop Web Summit 2014 @geckoboard @GA
Sofia Quintero5.9K views
Social Media Conversion, SFU Bus450, Fall 2015.pptx by Jeff Salzsauler
Social Media Conversion, SFU Bus450, Fall 2015.pptxSocial Media Conversion, SFU Bus450, Fall 2015.pptx
Social Media Conversion, SFU Bus450, Fall 2015.pptx
Jeff Salzsauler407 views
Community Marketing: Using Customer & Peer Endorsement to Lift Conversions, G... by Percussion Software
Community Marketing: Using Customer & Peer Endorsement to Lift Conversions, G...Community Marketing: Using Customer & Peer Endorsement to Lift Conversions, G...
Community Marketing: Using Customer & Peer Endorsement to Lift Conversions, G...
Percussion Software1.9K views
How to measure Social Media ROI by Scott K. Wilder
How to measure Social Media ROIHow to measure Social Media ROI
How to measure Social Media ROI
Scott K. Wilder1.9K views
Conversion Whitepaper by WSI Ensenada
Conversion WhitepaperConversion Whitepaper
Conversion Whitepaper
WSI Ensenada413 views
Lean Launchpad Tucson - Customer Relationships by Aaron Eden
Lean Launchpad Tucson - Customer RelationshipsLean Launchpad Tucson - Customer Relationships
Lean Launchpad Tucson - Customer Relationships
Aaron Eden1K views
How To Convert Digital Engagement Into Leads and Sales - Bridget Randolph by FeverBee Limited
How To Convert Digital Engagement Into Leads and Sales - Bridget RandolphHow To Convert Digital Engagement Into Leads and Sales - Bridget Randolph
How To Convert Digital Engagement Into Leads and Sales - Bridget Randolph
FeverBee Limited934 views
Feverbee SPRINT 2016 - Convert Engagement Into Leads and Sales - Bridget Rand... by Bridget Randolph
Feverbee SPRINT 2016 - Convert Engagement Into Leads and Sales - Bridget Rand...Feverbee SPRINT 2016 - Convert Engagement Into Leads and Sales - Bridget Rand...
Feverbee SPRINT 2016 - Convert Engagement Into Leads and Sales - Bridget Rand...
Bridget Randolph1.3K views

More from Nicola Junior Vitto

The new era of bots, multi-channel messaging solutions, dynamic routing and a... by
The new era of bots, multi-channel messaging solutions, dynamic routing and a...The new era of bots, multi-channel messaging solutions, dynamic routing and a...
The new era of bots, multi-channel messaging solutions, dynamic routing and a...Nicola Junior Vitto
5.4K views26 slides
SCRUM & Lean Startup in Enterprises by
SCRUM & Lean Startup in EnterprisesSCRUM & Lean Startup in Enterprises
SCRUM & Lean Startup in EnterprisesNicola Junior Vitto
8.4K views64 slides
Building Blomming: A startup founder's experience by
Building Blomming: A startup founder's experienceBuilding Blomming: A startup founder's experience
Building Blomming: A startup founder's experienceNicola Junior Vitto
7.4K views74 slides
Lean Startup - the Italian way by
Lean Startup - the Italian wayLean Startup - the Italian way
Lean Startup - the Italian wayNicola Junior Vitto
21.7K views82 slides
Social Commerce & the new ways for E-commerce by
Social Commerce & the new ways for E-commerceSocial Commerce & the new ways for E-commerce
Social Commerce & the new ways for E-commerceNicola Junior Vitto
9.2K views50 slides
Blomming Lean Startup @ Better Software 2011 by
Blomming Lean Startup @ Better Software 2011Blomming Lean Startup @ Better Software 2011
Blomming Lean Startup @ Better Software 2011Nicola Junior Vitto
2.2K views122 slides

More from Nicola Junior Vitto(8)

Recently uploaded

Top 10 Web Development Companies in California by
Top 10 Web Development Companies in CaliforniaTop 10 Web Development Companies in California
Top 10 Web Development Companies in CaliforniaTopCSSGallery
24 views27 slides
Episode 258 Snippets: Rob Gevertz of First Five Yards by
Episode 258 Snippets: Rob Gevertz of First Five YardsEpisode 258 Snippets: Rob Gevertz of First Five Yards
Episode 258 Snippets: Rob Gevertz of First Five YardsNeil Horowitz
40 views58 slides
TNR Gold Investor Presentation - Building The Green Energy Metals Royalty and... by
TNR Gold Investor Presentation - Building The Green Energy Metals Royalty and...TNR Gold Investor Presentation - Building The Green Energy Metals Royalty and...
TNR Gold Investor Presentation - Building The Green Energy Metals Royalty and...Kirill Klip
72 views175 slides
Tanishq by
Tanishq Tanishq
Tanishq supiriyakithuva
11 views7 slides
RESIDENTIAL CARPENTRY SERVICES – COLOR RENOVATION INC by
RESIDENTIAL CARPENTRY SERVICES – COLOR RENOVATION INCRESIDENTIAL CARPENTRY SERVICES – COLOR RENOVATION INC
RESIDENTIAL CARPENTRY SERVICES – COLOR RENOVATION INCcolorrenovation
18 views3 slides
PROGRAMME.pdf by
PROGRAMME.pdfPROGRAMME.pdf
PROGRAMME.pdfHiNedHaJar
65 views13 slides

Recently uploaded(20)

Top 10 Web Development Companies in California by TopCSSGallery
Top 10 Web Development Companies in CaliforniaTop 10 Web Development Companies in California
Top 10 Web Development Companies in California
TopCSSGallery24 views
Episode 258 Snippets: Rob Gevertz of First Five Yards by Neil Horowitz
Episode 258 Snippets: Rob Gevertz of First Five YardsEpisode 258 Snippets: Rob Gevertz of First Five Yards
Episode 258 Snippets: Rob Gevertz of First Five Yards
Neil Horowitz40 views
TNR Gold Investor Presentation - Building The Green Energy Metals Royalty and... by Kirill Klip
TNR Gold Investor Presentation - Building The Green Energy Metals Royalty and...TNR Gold Investor Presentation - Building The Green Energy Metals Royalty and...
TNR Gold Investor Presentation - Building The Green Energy Metals Royalty and...
Kirill Klip72 views
RESIDENTIAL CARPENTRY SERVICES – COLOR RENOVATION INC by colorrenovation
RESIDENTIAL CARPENTRY SERVICES – COLOR RENOVATION INCRESIDENTIAL CARPENTRY SERVICES – COLOR RENOVATION INC
RESIDENTIAL CARPENTRY SERVICES – COLOR RENOVATION INC
colorrenovation18 views
Skilled Landscape Contractor by EmmanuelRyker
Skilled Landscape ContractorSkilled Landscape Contractor
Skilled Landscape Contractor
EmmanuelRyker19 views
bookmyshow-1.pptx by 125071035
bookmyshow-1.pptxbookmyshow-1.pptx
bookmyshow-1.pptx
1250710358 views
How to get your business featured on Forbes - Business Show 23 by Quibble
How to get your business featured on Forbes - Business Show 23How to get your business featured on Forbes - Business Show 23
How to get your business featured on Forbes - Business Show 23
Quibble26 views
Strategies for Responsible and Efficient Waste Disposal by AlfredoRaylan
Strategies for Responsible and Efficient Waste DisposalStrategies for Responsible and Efficient Waste Disposal
Strategies for Responsible and Efficient Waste Disposal
AlfredoRaylan67 views
NewBase 23 November 2023 Energy News issue - 1676 by Khaled Al Awadi_compre... by Khaled Al Awadi
NewBase  23 November 2023  Energy News issue - 1676 by Khaled Al Awadi_compre...NewBase  23 November 2023  Energy News issue - 1676 by Khaled Al Awadi_compre...
NewBase 23 November 2023 Energy News issue - 1676 by Khaled Al Awadi_compre...
Khaled Al Awadi17 views
Coomes Consulting Business Profile by Chris Coomes
Coomes Consulting Business ProfileCoomes Consulting Business Profile
Coomes Consulting Business Profile
Chris Coomes38 views
Pitch Deck Teardown: Scalestack's $1M AI sales tech Seed deck by HajeJanKamps
Pitch Deck Teardown: Scalestack's $1M AI sales tech Seed deckPitch Deck Teardown: Scalestack's $1M AI sales tech Seed deck
Pitch Deck Teardown: Scalestack's $1M AI sales tech Seed deck
HajeJanKamps24 views

Lean Startup Metrics & Analytics

  • 1. (Lean) Startup Metrics & Analytics Nicola Junior Vitto - @njvitto @blomming September 26, 2012 - Better Software - Firenze, Italy
  • 2. Entrepreneurship ...Entrepreneurs recognize they are going to have to follow their own internal compass and embrace the uncertainty as part of the journey. - Steve Blank - more on: http://steveblank.com/2009/06/29/agile-opportunism-entrepreneurial-dna/
  • 3. Get out of the building (by Steve Blank) Your business assumptions can be wrong Go out and speak with your customers
  • 5. Vanity Metrics - Timelines There will always be something up and to the right
  • 6. Vanity Metrics - Google Analytics Forget it!
  • 7. Vanity Metrics - Google Analytics ...and this could become like a drug...killing you!
  • 8. Vanity Metrics - TechCrunch & Co “mDialog raises 5 Million from Blackberry partners” “GroupMe is now sending one Million texts every day” “Google Plus hits 10 Million users in 2 weeks”
  • 10. What’s a startup? A startup is first of all an experiment, it is a human institution designed to deliver a new product or service under conditions of extreme uncertainty At its heart, a startup is a catalyst that transforms ideas into product. (From Lessons Learned blog By Eric Ries)
  • 11. What’s a Lean Startup? Lean startup is a rigorous process for iterating from Plan A to a plan that works. (by Ash Maurya)
  • 12. What Metrics Matter? Example: Facebook on early days 150k users, little revenue Many superior competitors What would indicate their future success?
  • 13. What Metrics Matter? Example: Facebook on early days 75% of users visit one or more times per day Within one month of launching on a new campus, can acquire 90% of the students
  • 14. What Metrics Matter? Good Metrics (Three A’s) Are Actionable Can be Audited Are Accessible
  • 15. Actionable Metric An actionable metric is one that ties specific and repeatable actions to observed results (by Ash Maurya)
  • 17. A/B Testing - New Blomming home VS Define your Goals
  • 18. A/B Testing - Define your goals A/B Testing Goals One single important Goal: % Users that sign-up for the service (as a seller) Secondary Goal: % Buyers that buy something after landing on the homepage
  • 20. Multivariate Testing at Google “Yes, it’s true that a team at Google couldn’t decide between two blues, so they’re testing 41 shades between each blue to see whic one performs better. I has a recent debate over wheter a border should be 3, 4 or 5 pixels wide, and was asked to prove my case. I can’t operate in an environment like that.” (by Doug Bowman on Design at Google)
  • 23. Multivariate Testing at Google Google Bing
  • 24. Multivariate Testing at Google “That blue was worth at least $80M” (by Paul Ray - Product Manager at Bing)
  • 26. David Cancel’s Funnel The Problem: 98% of people who visit your site, leave never to return
  • 27. David Cancel’s Funnel 2% “The average conversion rate in the United States of America wheter you’re selling elephants or iPods” (by Avinash Kaushik - Analytics Evangelist, Google)
  • 29. David Cancel’s Funnel The Solution: To find, engage and convert more of your website visitors into customers
  • 30. Online Marketing is shifting
  • 31. Online Marketing Paradigm Shift by David Cancel
  • 32. Online Marketing Paradigm Shift by David Cancel
  • 33. Online Marketing Paradigm Shift by David Cancel
  • 34. Online Marketing Paradigm Shift Internet Marketing 3.0 (different goal) by David Cancel
  • 35. Lifecycle Marketing by David Cancel
  • 37. Optimize for learning Optimize for learning by creating a data-driven marketing framework
  • 38. Optimize for learning Analytics & A/B Testing are also not exactly the answer
  • 39. Optimize for learning Use quantitative & qualitative data to test *your* assumptions
  • 40. Optimize for learning The point of testing is to prove or disprove assumptions quickly
  • 41. Optimize for learning Pay attention to “overtesting” using Multivariate testing because it generally takes too long
  • 42. The Local Maxima Problem
  • 44. AARRR
  • 45. AARRR Dave McClure’s Startup Metrics for Pirates is a good framework for thinking about macro metrics
  • 48. AARRR - Different orders (example) 1. Activation, Retention (happiness) 2. Revenue (sustainability) 3. Acquisition, Referral (virality) (this is the Ash Maurya or 37signals order: “Charge from day one”)
  • 50. Tracking Marketing Channels Google Adwords Banners Social Media Partnerships PR Blogging ...anything that links to your site!
  • 51. Tracking Marketing Channels Use unique urls (tracking parameters) on every url you create/give-out/pay for Use bit.ly Use HTTP referrers to group PR
  • 52. Measuring Cost of Visits Cost per click (site visits) Cost of campaign / Traffic generated (Social media) Cost of effort / Traffic generated
  • 53. Measuring Cost of Acquisition Cost of one visit / conversion rate = Cost of an Acquisition Acquisition could be paid for acquisition of free sign-up ...unfortunately is not so simple
  • 54. Marketing Channels Per channel track: Scale Cost Conversion Optimise and scale channels with best Cost of Acquisition (CoA) and potential to Scale
  • 55. Acquisition Cost of Acquisition < Customer Lifetime Profit Then scale acquisition at a similar or lower rate A/B test acquisition channels to optimise
  • 56. Acquisition including Referral If it costs €3 to acquire a customer through advertising But a customer brought in 2 other customers through referral The COA of that channel is €1
  • 58. Activation Sure, they signed-up!!!
  • 59. Activation Sure, they signed-up!!! BUT did the like it???
  • 60. Activation at Facebook (i) which data points predict whether a user will stay? (ii) if they stay, which data points predict how active they’ll be after three months?
  • 61. Activation at Facebook (i) which data points predict whether a user will stay? having more than one session as a new user and entering basic profile information. (ii) if they stay, which data points predict how active they’ll be after three months? how often a user was reached out by others, frequency of third party application use, and how available a user was on the site.
  • 62. Activation at Facebook “At the time, I heard many people criticize Facebook’s early investors, claiming that Facebook had ‘no business model’ and only modest revenues relative to the valuation offered by its investors...” “...Facebook was different, because it employed a different Engine of Growth. It paid nothing for customer acquisition, and its high engagement meant that it was accumulating massive amount of customer attention every day.” (Eric Ries - The Lean Startup)
  • 63. Activation Metrics Activation metrics often drive other metrics in Referral, Retention, Revenue
  • 65. Returning Users Tracking unregistered users that return often (useful for content sites with advertising) Tracking registered users that return and take some sort of action.
  • 67. Twitter Cohorts (example) by Fred Wilson’s blog (more on: http://www.avc.com/a_vc/2009/10/the-cohort-analysis.html)
  • 68. Twitter Cohorts (example) by Fred Wilson’s blog (more on: http://www.avc.com/a_vc/2009/10/the-cohort-analysis.html)
  • 70. Referral Mechanism Social Media Sharing Invite a friend type mechanism App Reviews Word of Mouth
  • 71. Viral Coefficient The average number of customers that each customer refers
  • 72. Viral Coefficient - Example Each company customer invites 5 other customers via email 20% click on that email link 50% of those users convert into customers
  • 73. Viral Coefficient - Example Viral Coefficient = Referrals x click through x conversion 5 x 0.2 x 0.5 = 0.5
  • 74. Viral Coefficient A viral co-efficient greater than 1 means that every customer gets more than one other customer on average
  • 75. Viral Coefficient A viral co-efficient greater than 1 means that every customer gets more than one other customer on average Therefore your product will grow virally
  • 76. Viral Coefficient Measure automatically for all your built in referral mechanism Give customers unique invite URLs and track conversions
  • 77. Viral Cycle Time Viral Cycle Time = the average time taken for a referral to turn into a customer With the viral coefficient and viral cycle time you can determine your viral growth
  • 78. Net Promoter Score Net Promoter Score: Number of people who would recommend your product to a friend A single question survey
  • 79. Net Promoter Score “How likely is it you would recommend my product to a friend or colleague?” 0 1 2 3 4 5 6 7 8 9 10 NPS = % Promoters - % Detractors
  • 80. Net Promoter Score Survey a small percentage of your customers by email weekly Record who you sent to so you don’t repeat the same people every week
  • 81. Net Promoter Score NPS is not a direct measurement that translates to key metrics Can act as a leading indicator of other metrics
  • 83. Making Money Key for any business is to find out how much profit they make for every customer and scaling the number of customers
  • 84. Lifetime Customer Value How much money you make for every customer you acquire For some this a single transaction For subscription businesses it depends on retention
  • 85. Cancellation Rate Cancellation rate = is the percentage number of customers who cancel in any given month compared to total (paying) customers [Cancellation rate] = [product utility] + [service quality] + [acceptable price]
  • 86. Lifetime Customer Value (LTV) LTV = monthly revenue x n° months in lifetime N° months = 1 / cancellation rate LTV = monthly revenue / cancellation rate
  • 87. Lifetime Customer Value (LTV) For example: If €20 a month and cancellation rate is 10% a month LTV = €200
  • 88. Lifetime Customer Value (LTV) Often not that simple. Short term cancellation rates are much higher than long term Use Cohort Analysis to determine
  • 89. Lifetime Customer Value (LTV) r = short-term cancellation rate (e.g. 0,15) p = long-term cancellation rate (e.g. 0.03) s = number of months in the “short-term” age group (e.g. 3) (1-r)^s x (s + 1/p) = expected months
  • 94. Customer lifecycle management
  • 97. Tips
  • 99. Innovation Accounting “To improve entrepreneurial outcomes, and to hold entrepreneurs accountable, we need to focus on the boring stuff: how to measure progress, how to setup milestones, how to prioritize work. This requires a new kind of accounting, specific to startups” (Eric Ries - The Lean Startup)
  • 100. Innovation Accounting Concentrate on macro metrics Everything you do should attempt to change a metric Try to use as few tools as possible
  • 105. SCRUM Do you know SCRUM?
  • 106. Kanban Board I know: this isn’t an example of an exciting and well done Sprint :)
  • 107. (Lean) Product Management Well, trying to connect the dots: what we learnt so far is how the Product Owner can define and prioritize stories at every Sprint in a Startup that wants to be called, you know, Lean! :)
  • 108. Learning Boards - Lean Stack We will talk about the Lean Stack of Ash Maurya or other possible “lean mapping” frameworks in a future presentation
  • 110. Credits Stuart Eccles - Lean Startup Metrics Joshua Porter - Metrics-Driven Design David Cancel - Data-Driven Marketing Lean Startup Machine - How to measure the metrics that determine real progress (and the others specified directly in the slides when picking some parts of books or blogs)
  • 114. An alternative? Lifestyle business
  • 115. Thanks! Blomming.com Company.Blomming.com Magazine.Blomming.com Help.Blomming.com Twitter.com/Blomming Facebook.com/Blomming nicola@blomming.com On FB & Twitter as: njvitto

Editor's Notes

  1. \n
  2. \n
  3. \n
  4. \n
  5. \n
  6. \n
  7. \n
  8. \n
  9. \n
  10. \n
  11. \n
  12. \n
  13. \n
  14. \n
  15. \n
  16. \n
  17. \n
  18. \n
  19. \n
  20. \n
  21. \n
  22. \n
  23. \n
  24. \n
  25. \n
  26. \n
  27. \n
  28. \n
  29. \n
  30. \n
  31. \n
  32. \n
  33. \n
  34. \n
  35. \n
  36. \n
  37. \n
  38. \n
  39. \n
  40. \n
  41. \n
  42. \n
  43. \n
  44. \n
  45. \n
  46. \n
  47. \n
  48. \n
  49. \n
  50. \n
  51. \n
  52. \n
  53. \n
  54. \n
  55. \n
  56. \n
  57. \n
  58. \n
  59. \n
  60. \n
  61. \n
  62. \n
  63. \n
  64. \n
  65. \n
  66. \n
  67. \n
  68. \n
  69. \n
  70. \n
  71. \n
  72. \n
  73. \n
  74. \n
  75. \n
  76. \n
  77. \n
  78. \n
  79. \n
  80. \n
  81. \n
  82. \n
  83. \n
  84. \n
  85. \n
  86. \n
  87. \n
  88. \n
  89. \n
  90. \n
  91. \n
  92. \n
  93. \n
  94. \n
  95. \n
  96. \n
  97. \n
  98. \n
  99. \n
  100. \n
  101. \n
  102. \n
  103. \n
  104. \n
  105. \n
  106. \n
  107. \n
  108. \n
  109. \n
  110. \n
  111. \n
  112. \n
  113. \n
  114. \n
  115. \n