Lean analytics for startupsWednesday, December 8, 2010
@acroll         (if you have questions)Wednesday, December 8, 2010
Wednesday, December 8, 2010
Wednesday, December 8, 2010
Lean   Not leanWednesday, December 8, 2010
Ries, Mclure, and Blank         are often misquoted.Wednesday, December 8, 2010
They never said “fail faster.”Wednesday, December 8, 2010
Instead:         “Learn and adapt.”Wednesday, December 8, 2010
Waterfall, agile, and lean         Three approaches for three situations.Wednesday, December 8, 2010
Waterfall approach         You know the problem and the solution.Wednesday, December 8, 2010
Known ways to                        Known set of         satisfy them                        requirements                ...
Agile methodologies         Know the problem, find the solutionWednesday, December 8, 2010
Unclear how                                      Unknown set                                                           Kno...
Most startups         Don’t even know what problem they solve.Wednesday, December 8, 2010
Possible viable                                                          offering                                IVOT     ...
All startups are lean at first.         Which has some interesting consequences.Wednesday, December 8, 2010
Knowing         Getting              what’s not         features right       working,         matters less.        quickly...
“We should”          “What if”         matters less.        matters more.Wednesday, December 8, 2010
Your                 Parallel         roadmap              experiments         matters less.        matter more.Wednesday,...
Real-world         User groups          adoption &         and surveys          usage         matter less.         analysi...
Business             Business         plans matter         models matter         less.                more.Wednesday, Dece...
http://www.flickr.com/photos/tinfoilraccoon/197640807/Wednesday, December 8, 2010
Revenue              (   Lemonade   )   x Units sold + tips               Cost           ( Lemon + cups + sugar ) x Units ...
Wednesday, December 8, 2010
(maybe not that fun a game)                              http://www.flickr.com/photos/ethanhein/3207831470/#/Wednesday, Dec...
Clouds and analytics         It’s never been a better time to start a company.Wednesday, December 8, 2010
Clouds make                 Analytics  experimenting               make  easy, avoid                 experimenting  upfron...
I love to talk         about clouds         (but not today)                              http://www.flickr.com/photos/ecast...
I’m going to focus on the         other part         What to watch on your website, if you’re a startup.Wednesday, Decembe...
So what could we watch?Wednesday, December 8, 2010
“Hard” data             Analytics                 Usability     Performability           (what did they           (how did...
Analytics is the     measurement of movement     towards your business     goals.                              http://www....
ATTENTION                             ENGAGEMENT     CONVERSION                                NEW                        ...
Visits                       Shopping cart                 Payment options                              ConversionsWednesd...
KPIs                                             Bounce                                             rate                  ...
http://www.flickr.com/photos/mrmoorey/160654236Wednesday, December 8, 2010
http://www.flickr.com/photos/duncan/1252272164/Wednesday, December 8, 2010
http://www.flickr.com/photos/intherough/3573333256/Wednesday, December 8, 2010
http://www.flickr.com/photos/jetheriot/648950773/Wednesday, December 8, 2010
Wednesday, December 8, 2010
Wednesday, December 8, 2010
Wednesday, December 8, 2010
Wednesday, December 8, 2010
Wednesday, December 8, 2010
Wednesday, December 8, 2010
Wednesday, December 8, 2010
Wednesday, December 8, 2010
Wednesday, December 8, 2010
Wednesday, December 8, 2010
Impact of page load time on           average daily searches per user        0%  -0.15%  -0.30%  -0.45%  -0.60%   50ms pre...
Wednesday, December 8, 2010
Impact of additional delay on                      business metrics        0%  -1.25%  -2.50%  -3.75%  -5.00%             ...
Shopzilla’s makeoverWednesday, December 8, 2010
5-12%Wednesday, December 8, 2010
Wednesday, December 8, 2010
Traffic levels                              9,000     Total number of visits                              6,750            ...
Bounce rate                            20                            15      Visits that bounced                          ...
Average time on site                              31     Time on site (minutes)                              23           ...
Pages per visit                          16     Average pages seen                          12                           8...
Conversion rate                                        20                                             and order value     ...
Community management              Hugging              Bunnies              Retweets              Friends!Wednesday, Decem...
Unpaid search                  Community     Email          Banner ad                                         mentions   c...
But there’s a danger:         vanity metrics.Wednesday, December 8, 2010
Wednesday, December 8, 2010
So what should you watch?         The lean analytics shortlist.Wednesday, December 8, 2010
If you remember only one thing...         In a startup, the purpose of         analytics is to iterate to a         produc...
Viral coefficient          Message reach       Amplification    Infrastructure health     Performability                    ...
Viral coefficient         On average, for each message they hear, how         many people does someone tell?Wednesday, Dece...
------------------------------------------------------       Get your free private email at http://www.hotmail.com       -...
!Wednesday, December 8, 2010
v ≠ 1, pt  = δp0 (1 –                                vt+1)       / (1 – v) + p0                               http://rober...
How well are messages         being amplified?Wednesday, December 8, 2010                                 2
Downstream reach                                                          (aggregate followers)                          5...
Seed (starting        community)                                                    Repurposing              Reach        ...
Viral message spread              Reach       (impressions)                Emphasis on                         Visits     ...
Megablogger proponents      Seed (starting                                    Emphasis on        community)               ...
A call to action              Reach       (impressions)                         Visits      Shopping cart                 ...
Goal attainment: funnel      Seed ratio:                  20        35,000:1          Followers                   700,000s...
How performable are we?         Health of the infrastructure (uptime, latency)Wednesday, December 8, 2010                 ...
5,000                                                                              $10,000                                ...
What do visitors think of us?         Both on the site (VoC) and in the world         (community monitoring)Wednesday, Dec...
Wednesday, December 8, 2010
Wednesday, December 8, 2010
How engaged are users?         Time since last visit, as a histogram, not an         average, compared to baselines.Wednes...
Days since last visit                  1200                                                         25000Number of users  ...
Core goals driving the         business         Pick the 3-4 tasks you’ve set for users. Always         know what these ar...
(Remember the model?)         Revenue              (   Lemonade   )   x Units sold + tips               Cost           ( L...
Extended funnel         abandonment         Not just on the site: Email open rateWednesday, December 8, 2010              ...
Wednesday, December 8, 2010
In the last build, what         moved us towards & away         from goals?         Requires knowing the goals and the key...
Cost per visitor         Operations costs per site visitorWednesday, December 8, 2010                                     ...
(this had better be greater         than the revenues and         conversion rate.)Wednesday, December 8, 2010
Peak-to-average ratio         How much busier are you at busy times?         What premium do you pay for that surplus?    ...
http://community.microsoftadvertising.comWednesday, December 8, 2010
Wednesday, December 8, 2010
Minimum sustainable burnWednesday, December 8, 2010                                12
http://www.flickr.com/photos/dougww/833127501/Wednesday, December 8, 2010
Viral coefficient          Message reach       Amplification    Infrastructure health     Performability       Market sentim...
Thanks!         alistair@bitcurrent.com         @acroll         www.yearonelabs.com         www.rpmmtl.com         www.wat...
Upcoming SlideShare
Loading in...5
×

Lean analytics for startups - Leweb2010

4,508

Published on

20-minute speed-run presentation on what metrics and web analytics information startups need to collect. Focuses on companies with a lean methodology, and the kinds of data that will actually help them achieve product/market fit before the money runs out.

Published in: Technology

Lean analytics for startups - Leweb2010

  1. 1. Lean analytics for startupsWednesday, December 8, 2010
  2. 2. @acroll (if you have questions)Wednesday, December 8, 2010
  3. 3. Wednesday, December 8, 2010
  4. 4. Wednesday, December 8, 2010
  5. 5. Lean Not leanWednesday, December 8, 2010
  6. 6. Ries, Mclure, and Blank are often misquoted.Wednesday, December 8, 2010
  7. 7. They never said “fail faster.”Wednesday, December 8, 2010
  8. 8. Instead: “Learn and adapt.”Wednesday, December 8, 2010
  9. 9. Waterfall, agile, and lean Three approaches for three situations.Wednesday, December 8, 2010
  10. 10. Waterfall approach You know the problem and the solution.Wednesday, December 8, 2010
  11. 11. Known ways to Known set of satisfy them requirements Spec Build Test LaunchWednesday, December 8, 2010
  12. 12. Agile methodologies Know the problem, find the solutionWednesday, December 8, 2010
  13. 13. Unclear how Unknown set Known set of to satisfy them of requirements requirements Problem statement Build Test Viable? Launch Sprints AdjustWednesday, December 8, 2010
  14. 14. Most startups Don’t even know what problem they solve.Wednesday, December 8, 2010
  15. 15. Possible viable offering IVOT You are P Trial startup here Possible Possible Possible viable Trial startup problem Trial startup viable offering space offering Trial startup Possible viable offeringWednesday, December 8, 2010
  16. 16. All startups are lean at first. Which has some interesting consequences.Wednesday, December 8, 2010
  17. 17. Knowing Getting what’s not features right working, matters less. quickly, matters more.Wednesday, December 8, 2010
  18. 18. “We should” “What if” matters less. matters more.Wednesday, December 8, 2010
  19. 19. Your Parallel roadmap experiments matters less. matter more.Wednesday, December 8, 2010
  20. 20. Real-world User groups adoption & and surveys usage matter less. analysis matters more.Wednesday, December 8, 2010
  21. 21. Business Business plans matter models matter less. more.Wednesday, December 8, 2010
  22. 22. http://www.flickr.com/photos/tinfoilraccoon/197640807/Wednesday, December 8, 2010
  23. 23. Revenue ( Lemonade ) x Units sold + tips Cost ( Lemon + cups + sugar ) x Units made ( Markers + cardboard ) x Signs made ( Staffing ) x Hours worked Profit!Wednesday, December 8, 2010
  24. 24. Wednesday, December 8, 2010
  25. 25. (maybe not that fun a game) http://www.flickr.com/photos/ethanhein/3207831470/#/Wednesday, December 8, 2010
  26. 26. Clouds and analytics It’s never been a better time to start a company.Wednesday, December 8, 2010
  27. 27. Clouds make Analytics experimenting make easy, avoid experimenting upfront smart & fast. dilution.Wednesday, December 8, 2010
  28. 28. I love to talk about clouds (but not today) http://www.flickr.com/photos/ecastro/3053916892/Wednesday, December 8, 2010
  29. 29. I’m going to focus on the other part What to watch on your website, if you’re a startup.Wednesday, December 8, 2010
  30. 30. So what could we watch?Wednesday, December 8, 2010
  31. 31. “Hard” data Analytics Usability Performability (what did they (how did they (could they do do on the interact with what they site?) it?) wanted to?) Complete Web Monitoring Community VoC Competition (what were (what were (what are they they saying?) their up to?) motivations?) “Soft” dataWednesday, December 8, 2010
  32. 32. Analytics is the measurement of movement towards your business goals. http://www.flickr.com/photos/itsgreg/446061432/Wednesday, December 8, 2010
  33. 33. ATTENTION ENGAGEMENT CONVERSION NEW VISITORS SEARCHES GROWTH CONVERSION PAGES TIME RATE TWEETS NUMBER OF VISITS PER ON x MENTIONS VISIT SITE GOAL ADS SEEN LOSS VALUE BOUNCE RATEWednesday, December 8, 2010
  34. 34. Visits Shopping cart Payment options ConversionsWednesday, December 8, 2010
  35. 35. KPIs Bounce rate Conversion Visits rate Shopping cart Payment options ConversionsWednesday, December 8, 2010
  36. 36. http://www.flickr.com/photos/mrmoorey/160654236Wednesday, December 8, 2010
  37. 37. http://www.flickr.com/photos/duncan/1252272164/Wednesday, December 8, 2010
  38. 38. http://www.flickr.com/photos/intherough/3573333256/Wednesday, December 8, 2010
  39. 39. http://www.flickr.com/photos/jetheriot/648950773/Wednesday, December 8, 2010
  40. 40. Wednesday, December 8, 2010
  41. 41. Wednesday, December 8, 2010
  42. 42. Wednesday, December 8, 2010
  43. 43. Wednesday, December 8, 2010
  44. 44. Wednesday, December 8, 2010
  45. 45. Wednesday, December 8, 2010
  46. 46. Wednesday, December 8, 2010
  47. 47. Wednesday, December 8, 2010
  48. 48. Wednesday, December 8, 2010
  49. 49. Wednesday, December 8, 2010
  50. 50. Impact of page load time on average daily searches per user 0% -0.15% -0.30% -0.45% -0.60% 50ms pre-header 100ms pre-header 200ms post-header 200ms post-ad 400ms post-headerWednesday, December 8, 2010
  51. 51. Wednesday, December 8, 2010
  52. 52. Impact of additional delay on business metrics 0% -1.25% -2.50% -3.75% -5.00% 50 200 500 1000 2000 Queries/visitor Query refinement Revenue/visitor Any clicks SatisfactionWednesday, December 8, 2010
  53. 53. Shopzilla’s makeoverWednesday, December 8, 2010
  54. 54. 5-12%Wednesday, December 8, 2010
  55. 55. Wednesday, December 8, 2010
  56. 56. Traffic levels 9,000 Total number of visits 6,750 4,500 8,505 2,250 4,740 0 Optimized Unoptimized Visitor experienceWednesday, December 8, 2010
  57. 57. Bounce rate 20 15 Visits that bounced 10 13.38% 14.35% 5 0 Optimized Unoptimized Visitor experienceWednesday, December 8, 2010
  58. 58. Average time on site 31 Time on site (minutes) 23 16 30.17 23.83 8 0 Optimized Unoptimized Visitor experienceWednesday, December 8, 2010
  59. 59. Pages per visit 16 Average pages seen 12 8 15.64 11.04 4 0 Optimized Unoptimized Visitor experienceWednesday, December 8, 2010
  60. 60. Conversion rate 20 and order value Difference due to optimization 15 10 16.07 5 5.51 0 Conversion rate Order valueWednesday, December 8, 2010
  61. 61. Community management Hugging Bunnies Retweets Friends!Wednesday, December 8, 2010
  62. 62. Unpaid search Community Email Banner ad mentions campaign ? • Google PageRank • Open rate • Cost of ads • Sessions-to-clicks • Opt-out rate (CPM) ratio • Clickthrough rate Visits Shopping cart Payment options ConversionsWednesday, December 8, 2010
  63. 63. But there’s a danger: vanity metrics.Wednesday, December 8, 2010
  64. 64. Wednesday, December 8, 2010
  65. 65. So what should you watch? The lean analytics shortlist.Wednesday, December 8, 2010
  66. 66. If you remember only one thing... In a startup, the purpose of analytics is to iterate to a product/market fit before the money runs out.Wednesday, December 8, 2010
  67. 67. Viral coefficient Message reach Amplification Infrastructure health Performability Customer research Market sentiment Community sentiment User engagement Time since last visit Core goals Business model Extended funnel abandonment Movement towards goals Cost per engaged visitor Sustainability Peak to average ratio Minimum sustainable burnWednesday, December 8, 2010
  68. 68. Viral coefficient On average, for each message they hear, how many people does someone tell?Wednesday, December 8, 2010 1
  69. 69. ------------------------------------------------------ Get your free private email at http://www.hotmail.com ------------------------------------------------------Wednesday, December 8, 2010
  70. 70. !Wednesday, December 8, 2010
  71. 71. v ≠ 1, pt  = δp0 (1 – vt+1) / (1 – v) + p0  http://robert.zubek.net/blog/2008/01/30/viral-coefficient-calculation/Wednesday, December 8, 2010
  72. 72. How well are messages being amplified?Wednesday, December 8, 2010 2
  73. 73. Downstream reach (aggregate followers) 50% 75% 100% 50% =4 2 3 1 =6 Amplification chance (how relevant the content 25% 50% 50% is + what the recipient thinks of the sender) .5 2 1 = 3.5 14.5Wednesday, December 8, 2010
  74. 74. Seed (starting community) Repurposing Reach (spread to other (impressions) communities) Visits Shopping cart Amplification (virality andPayment options message spread) ConversionsWednesday, December 8, 2010
  75. 75. Viral message spread Reach (impressions) Emphasis on Visits getting viral ratio above 1 Shopping cart (Retweeting, Fan, EmailPayment options forward, Reddit upvote, other Conversions loops)Wednesday, December 8, 2010
  76. 76. Megablogger proponents Seed (starting Emphasis on community) convincing highly- Reach followed, highly (impressions) acted-upon seed Visits group to spread the word. Shopping cartPayment options ConversionsWednesday, December 8, 2010
  77. 77. A call to action Reach (impressions) Visits Shopping cart Emphasis on maximizingPayment options impression-to- click ratio within Conversions the communityWednesday, December 8, 2010
  78. 78. Goal attainment: funnel Seed ratio: 20 35,000:1 Followers 700,000s 2 Repurposing 150 Amplification: Visitors: 2,000 RT 2.9% 1,642 0.23% 7 x $100 Revenues: Conversions: 1.95% 32 10 x $20 Average: $39.54 Median: $20 15 x $7 Total: $1,005Wednesday, December 8, 2010
  79. 79. How performable are we? Health of the infrastructure (uptime, latency)Wednesday, December 8, 2010 3
  80. 80. 5,000 $10,000 Revenue (total sales) Unique page views Conversions >4s or error 2-4s 0 $0 <2s 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16Wednesday, December 8, 2010
  81. 81. What do visitors think of us? Both on the site (VoC) and in the world (community monitoring)Wednesday, December 8, 2010 4&5
  82. 82. Wednesday, December 8, 2010
  83. 83. Wednesday, December 8, 2010
  84. 84. How engaged are users? Time since last visit, as a histogram, not an average, compared to baselines.Wednesday, December 8, 2010 6
  85. 85. Days since last visit 1200 25000Number of users January February 1000 20000 800 15000 600 10000 400 5000 200 0 0 1 2 3 4 5 6 7 8 9 Disengaged Days since last engagement (>10 days)Wednesday, December 8, 2010
  86. 86. Core goals driving the business Pick the 3-4 tasks you’ve set for users. Always know what these are and why they correlate to business growth.Wednesday, December 8, 2010 7
  87. 87. (Remember the model?) Revenue ( Lemonade ) x Units sold + tips Cost ( Lemon + cups + sugar ) x Units made ( Markers + cardboard ) x Signs made ( Staffing ) x Hours worked Profit!Wednesday, December 8, 2010
  88. 88. Extended funnel abandonment Not just on the site: Email open rateWednesday, December 8, 2010 8
  89. 89. Wednesday, December 8, 2010
  90. 90. In the last build, what moved us towards & away from goals? Requires knowing the goals and the key features addedWednesday, December 8, 2010 9
  91. 91. Cost per visitor Operations costs per site visitorWednesday, December 8, 2010 10
  92. 92. (this had better be greater than the revenues and conversion rate.)Wednesday, December 8, 2010
  93. 93. Peak-to-average ratio How much busier are you at busy times? What premium do you pay for that surplus? Can you decommission well?Wednesday, December 8, 2010 11
  94. 94. http://community.microsoftadvertising.comWednesday, December 8, 2010
  95. 95. Wednesday, December 8, 2010
  96. 96. Minimum sustainable burnWednesday, December 8, 2010 12
  97. 97. http://www.flickr.com/photos/dougww/833127501/Wednesday, December 8, 2010
  98. 98. Viral coefficient Message reach Amplification Infrastructure health Performability Market sentiment Customer research Community sentiment 12 User engagement Time since last visit things Core goals to Business model watch Extended funnel abandonment Movement towards goals Cost per engaged visitor Sustainability Peak to average ratio Minimum sustainable burnWednesday, December 8, 2010
  99. 99. Thanks! alistair@bitcurrent.com @acroll www.yearonelabs.com www.rpmmtl.com www.watchingwebsites.com www.bitcurrent.com www.human20.comWednesday, December 8, 2010
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×