Understanding Lean Analytics (and how analytics helps businesses win)

  • 8,695 views
Uploaded on

This is the workshop on Lean Analytics from the Web Analytics Congress (#wac13) in Amsterdam. It covers the basics of Lean Analytics + Lean Startup. It goes into details on specific business models …

This is the workshop on Lean Analytics from the Web Analytics Congress (#wac13) in Amsterdam. It covers the basics of Lean Analytics + Lean Startup. It goes into details on specific business models such as media and e-commerce and includes many case studies from the Lean Analytics book.

More in: Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
8,695
On Slideshare
0
From Embeds
0
Number of Embeds
26

Actions

Shares
Downloads
0
Comments
0
Likes
57

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Understanding Lean Analytics and how analytics helps business win Ben Yoskovitz @byosko byosko@gmail.com #leananalytics http://leananalyticsbook.comMonday, March 18, 13
  • 2. More wins than losses GoInstant 1st startup Year One Labs Started blogging Standout Jobs Big pivot 1996 1998 2001 2006 2007 2010 2011 The “I got too comfy” years Failed $0Monday, March 18, 13
  • 3. I BELIEVE IN GUTS + DATAMonday, March 18, 13
  • 4. What we’ll cover today • Lean Startup • Key aspects of metrics • Lean Analytics framework • The One Metric That Matters • The Lean Analytics Cycle • Exploring business models • Analytics in marketing • Trends in analyticsMonday, March 18, 13
  • 5. LEAN STARTUPMonday, March 18, 13
  • 6. Build Measure Learn Eric Ries http://theleanstartup.comMonday, March 18, 13
  • 7. THREE ENGINES OF GROWTH Sticky Virality Paid http://www.flickr.com/photos/fftang/4941889792/sizes/l/in/photostream/Monday, March 18, 13
  • 8. The Lean Canvas (leancanvas.com)Monday, March 18, 13
  • 9. BASICS OF ANALYTICSMonday, March 18, 13
  • 10. Analytics is the measurement of movement towards your business goals.Monday, March 18, 13
  • 11. Things you can explain to others about metrics * Qualitative vs. Quantitative * Exploratory vs. Reporting * Vanity vs. Actionable * Leading vs. LaggingMonday, March 18, 13
  • 12. Qualitative Quantitative Unstructured, Numbers and stats; anecdotal, revealing, hard facts but less hard to aggregate. insight. Warm and fuzzy. Cold and hard.Monday, March 18, 13
  • 13. DISCOVER QUALITATIVELY AND PROVE QUANTITATIVELYMonday, March 18, 13
  • 14. Exploratory Reporting Speculative, trying to Predictable, keeping find unexpected or you abreast of interesting insights. normal, managerial operations. http://www.flickr.com/photos/50755773@N06/5415295449/ http://www.flickr.com/photos/elwillo/4737933662/Monday, March 18, 13
  • 15. Case study: From friends to moms • Started as Circle of Friends • Grew to 10M users BUT ENGAGEMENT SUCKEDMonday, March 18, 13
  • 16. Case study: Moms are crazy (in a good way!) • Messages to one another were on average 50% longer. • 115% more likely to attach a picture to a post they wrote. • 110% more likely to engage in a threaded (i.e. deep) conversation. • Friends, once invited, were 50% more likely to become engaged users. • 180% more likely to click on Facebook news feed items. • 60% more likely to accept invitations to the app. ENGAGEMENT WAS GREATMonday, March 18, 13
  • 17. Vanity Actionable Picks a direction. Makes you feel good, but doesn’t change how you’ll act.Monday, March 18, 13
  • 18. VANITY METRICS ARE BAD A metric from the early, foolish days of the Web. Hits Count people instead. Marginally better than hits. Unless you’re displaying Page views ad inventory, count people. Is this one person visiting a hundred times, or are a Visits hundred people visiting once? Fail. This tells you nothing about what they did, why they Unique visitors stuck around, or if they left. Followers/ Count actions instead. Find out how many followers friends/likes will do your bidding.Monday, March 18, 13
  • 19. http://www.flickr.com/photos/circasassy/7858155676/ If it won’t change how you behave, it’s a bad metric. Monday, March 18, 13
  • 20. LEADING LAGGING Number today Historical metric that shows metric that shows how tomorrow-makes you’re doing- the news reports the news http://www.flickr.com/photos/vegaseddie/3310041214/sizes/l/in/photostream/Monday, March 18, 13
  • 21. What mode of e-commerce are you? How many of your customers Then you are in this Your customers will You are just Focus on buy a second time mode buy from you like in 90 days? Low CAC, 1-15% Acquisition Once 70% high of retailers checkout 15-30% Hybrid 2-2.5 20% Increasing per year of retailers returns Loyalty, >30% Loyalty >2.5 10% inventory per year of retailers expansion (Thanks to Kevin Hillstrom for this.)Monday, March 18, 13
  • 22. ANALYTICS SUPERPOWERS (or what the heck is growth hacking?) http://www.flickr.com/photos/bloke_with_camera/401812833/sizes/o/in/photostream/Monday, March 18, 13
  • 23. 10000 1000 100 10 1 Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec Ice cream consumption DrowningsMonday, March 18, 13
  • 24. 10000 1000 100 10 1 Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec Ice cream consumption DrowningsMonday, March 18, 13
  • 25. 10000 1000 100 10 1 Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec Ice cream consumption DrowningsMonday, March 18, 13
  • 26. Correlated Causal Two variables that An independent change in similar ways, factor that directly perhaps because they impacts a dependent are linked to something one. else. Summer al Ca us us Ca al Correlated Ice cream Drowning consumptionMonday, March 18, 13
  • 27. Causality is a superpower, because it lets you change the future. Correlation lets you Causality lets you predict the future change the future “I will have 420 engaged “If I can make more first- users and 75 paying time visitors stay on for customers next month.” 17 minutes I will increase sales in 90 days.” Optimize the Find correlation Test causality causal factorMonday, March 18, 13
  • 28. LEAN ANALYTICS FRAMEWORKMonday, March 18, 13
  • 29. simplify.http://www.flickr.com/photos/josefeliciano/3849557951/sizes/l/in/photostream/Monday, March 18, 13
  • 30. YOUR BASIC THE STAGE OF BUSINESS YOUR STARTUP MODEL How you make $$ Lifecycle • E-commerce • Empathy • SaaS • Stickiness • Free mobile app • Virality • Media site • Revenue • Collaborative content site • Scale • Two-sided marketplaceMonday, March 18, 13
  • 31. LEAN ANALYTICS “GATE” NEEDED TO STAGES MOVE FORWARD I’ve found a real, poorly-met need EMPATHY that a reachable market faces. I’ve figured out how to solve the problem in a way they will adopt STICKINESS and pay for. I’ve built the right product/features/ GROWTH RATE functionality that keeps users VIRALITY around. The users and features fuel growth REVENUE organically and artificially. I’ve found a sustainable, scalable business with the right margins in a SCALE healthy ecosystem.Monday, March 18, 13
  • 32. Case study: Buffer goes from Stickiness to Scale (through Revenue) • Stage: Scale • Model: SaaS (consumer) • Popular social sharing application. • Focused primarily on customer acquisition • Charged from day oneMonday, March 18, 13
  • 33. Buffer charges early to prove people want the problem solved 20% of visitors create an account (acquisition / Empathy) of sign-ups become active 64% (start of Stickiness) of sign-ups return in the 1st month 60% (engagement / Stickiness) of sign-ups are active after 6 months 20% (engagement / Stickiness) convert from free to paid 2% (Revenue)Monday, March 18, 13
  • 34. How it all comes together The business you’re in E- 2-sided Mobile User-gen SaaS Media commerce market app content Empathy The stage you’re at Stickiness Virality Revenue ScaleMonday, March 18, 13
  • 35. How it all comes together The business you’re in E- 2-sided Mobile User-gen SaaS Media commerce market app content Empathy The stage you’re at One Metric Stickiness Virality Revenue That Matters. ScaleMonday, March 18, 13
  • 36. Case study: SEOmoz reduces the KPIs it tracks • Stage: Scale • Model: SaaS • SEO toolkit (product suite) • Reduced KPIs to focus on Net AddsMonday, March 18, 13
  • 37. Net Adds = “health of the business” indicator If Net Adds: Why & Next Steps:Monday, March 18, 13
  • 38. Net Adds = “health of the business” indicator If Net Adds: Why & Next Steps: • Was a marketing campaign successful? • Was churn lowered? • Were customer complaints lowered? • Was a product upgrade valuable?Monday, March 18, 13
  • 39. Net Adds = “health of the business” indicator If Net Adds: Why & Next Steps: • Was a marketing campaign successful? • Was churn lowered? • Were customer complaints lowered? • Was a product upgrade valuable? • How can we acquire more valuable customers? • Can we increase site conversion? • How can we lower churn? • What product features can increase engagement?Monday, March 18, 13
  • 40. Net Adds = “health of the business” indicator If Net Adds: Why & Next Steps: • Was a marketing campaign successful? • Was churn lowered? • Were customer complaints lowered? • Was a product upgrade valuable? • How can we acquire more valuable customers? • Can we increase site conversion? • How can we lower churn? • What product features can increase engagement? • Are the new customers not the right segment? • Did a marketing campaign fail? • Are too many customers churning? • Did a product upgrade fail to impress or cause issues?Monday, March 18, 13
  • 41. DRAW A LINE IN THE SANDMonday, March 18, 13
  • 42. HighScore House defines an “active user”Monday, March 18, 13
  • 43. What’s your OMTM? E- 2-sided Mobile User-gen SaaS Media commerce market app content Empathy Interviews; qualitative results; quantitative scoring; surveys Loyalty, Inventory, Engagement, Downloads, Content, Traffic, visits, Stickiness conversion listings churn churn, virality spam returns CAC, shares, Inherent WoM, app Invites, Content Virality reactivation SEM, sharing virality, CAC ratings, CAC sharing virality, SEM (Money from transactions) (Money from active users) (Money from ad clicks) Transaction, Transactions, Upselling, CLV, Ads, CPE, affiliate Revenue CLV commission CAC, CLV ARPDAU donations %, eyeballs Affiliates, Other API, magic Spinoffs, Analytics, Syndication, Scale white-label verticals #, mktplace publishers user data licensesMonday, March 18, 13
  • 44. METRICS ARE LIKE SQUEEZE TOYSMonday, March 18, 13
  • 45. YOUR GOAL IS TO MAKE FASTER, MORE INTELLECTUALLY HONEST DECISIONS AND EMPOWER YOUR ORGANIZATION TO DO THE SAMEMonday, March 18, 13
  • 46. What these have in common: The Lean Analytics CycleMonday, March 18, 13
  • 47. What these have in common: The Lean Analytics Cycle Pick a KPIMonday, March 18, 13
  • 48. What these have in common: The Lean Analytics Cycle Pick a KPI Draw a line in the sandMonday, March 18, 13
  • 49. What these have in common: The Lean Analytics Cycle Pick a KPI Draw a line in the sand Find a potential improvementMonday, March 18, 13
  • 50. What these have in common: The Lean Analytics Cycle Pick a KPI Draw a line in the sand Find a potential improvement Without data: make a good guessMonday, March 18, 13
  • 51. What these have in common: The Lean Analytics Cycle Pick a KPI Draw a line in the sand Find a potential improvement Without With data: data: make a find a good guess commonalityMonday, March 18, 13
  • 52. What these have in common: The Lean Analytics Cycle Pick a KPI Draw a line in the sand Find a potential improvement Without With data: data: make a find a good guess commonality HypothesisMonday, March 18, 13
  • 53. What these have in common: The Lean Analytics Cycle Pick a KPI Draw a line in the sand Find a potential improvement Without With data: data: make a find a good guess commonality Hypothesis Make changes in productionMonday, March 18, 13
  • 54. What these have in common: The Lean Analytics Cycle Pick a KPI Draw a line in the sand Find a potential improvement Without With data: data: make a find a good guess commonality Design a test Hypothesis Make changes in productionMonday, March 18, 13
  • 55. What these have in common: The Lean Analytics Cycle Pick a KPI Draw a line in the sand Find a potential improvement Without With data: data: make a find a good guess commonality Design a test Measure the results Hypothesis Make changes in productionMonday, March 18, 13
  • 56. What these have in common: The Lean Analytics Cycle Pick a KPI Draw a line in the sand Find a potential improvement Did we move the needle? Without With data: data: make a find a good guess commonality Design a test Measure the results Hypothesis Make changes in productionMonday, March 18, 13
  • 57. What these have in common: The Lean Analytics Cycle Success! Pick a KPI Draw a line in the sand Find a potential improvement Did we move the needle? Without With data: data: make a find a good guess commonality Design a test Measure the results Hypothesis Make changes in productionMonday, March 18, 13
  • 58. What these have in common: The Lean Analytics Cycle Success! Pick a KPI Draw a line in the sand Pivot or give up Find a potential improvement Did we move the needle? Without With data: data: make a find a good guess commonality Design a test Measure the results Hypothesis Make changes in productionMonday, March 18, 13
  • 59. What these have in common: The Lean Analytics Cycle Success! Pick a KPI Draw a line in the sand Pivot or give up Draw a new line Find a potential improvement Did we move the needle? Without With data: data: make a find a good guess commonality Design a test Measure the results Hypothesis Make changes in productionMonday, March 18, 13
  • 60. What these have in common: The Lean Analytics Cycle Success! Pick a KPI Draw a line in the sand Pivot or give up Draw a new line Find a potential Try again improvement Did we move the needle? Without With data: data: make a find a good guess commonality Design a test Measure the results Hypothesis Make changes in productionMonday, March 18, 13
  • 61. EXPLORING BUSINESS MODELSMonday, March 18, 13
  • 62. MEDIA COMPANIES • Audience & churn (how many people visit the site, and how loyal they are) • Ad inventory (the number of impressions that can be monetized) • Ad rates (sometimes measured in cost-per-engagement-- essentially how much a site can make from those impressions based on the content it covers and the people who visit) • Click-through rates (how many of the impressions actually turn into money) • The content/advertising balance (the balance of ad inventory ad rates and content that maximizes overall performanceMonday, March 18, 13
  • 63. Monday, March 18, 13
  • 64. E-COMMERCE COMPANIES • Conversion rate (the # of visitors who buy something) • Purchases / year (the # of purchases made by each customer per year) • Average shopping cart size (the amount of money spent / purchase) • Cost of customer acquisition (the money spent to get someone to buy something) • Revenue / customer (the lifetime value of each customer) • Top keywords driving traffic to the site (those terms that people are looking for, and associate with you--a clue to adjacent products or markets) • Top search terms (both those that lead to revenue, and those that don’t have any results) • Effectiveness of recommendation engines (how likely a visitor is to add a recommended product to their cart) • Virality (word of mouth, and sharing per visitor) • Mailing list effectiveness (click-through rates and ability to make buyers return and buy)Monday, March 18, 13
  • 65. Case study: WineExpress increases revenues • Stage: Revenue • Model: E-commerce • Exclusive wine shop partner of the Wine Enthusiast catalog and website • “Wine of the day” page is highly trafficked, needed optimizationMonday, March 18, 13
  • 66. AMonday, March 18, 13
  • 67. BMonday, March 18, 13
  • 68. Case study: Before and after 41% increase in revenue per visitorMonday, March 18, 13
  • 69. Don’t forget the real world Shipping time, stock availability, logistics, ratings, and other factors have a real impact on most e- commerce companies. Shipping time, stock availability, DON’T logistics, ratings, and other factors FORGET THE have a real impact on most e- REAL WORLD commerce companies.Monday, March 18, 13
  • 70. Monday, March 18, 13
  • 71. Outside In: The Power of Putting Customers at the Center of Your Business http://www.amazon.com/Outside-Putting-Customers-Center-Business/dp/0547913982Monday, March 18, 13
  • 72. ANALYTICS IN MARKETINGMonday, March 18, 13
  • 73. HOW IT USUALLY HAPPENS • Some CxO catches on to the latest thing from talking to someone in social media, or tries out the latest cool app (Vine, Pinterest) • That person rushes into the office, exclaiming, “We need to do a Pinterest campaign now!” • The marketing team scrambles to come up with a campaign • They use that campaign to back into a metric • They run the campaign and report the results (maybe!) ...which of course means no line in the sand, no discipline, and attempts to move metrics that may not be core to the business in the first place.Monday, March 18, 13
  • 74. HOW IT SHOULD BE DONE • There’s a known business objective / problem to solve • There’s a key metric for it (OMTM) • There’s a goal (line in the sand) • We come up with a campaign or effort to move the needle (hypothesis) • We decide the medium that we want to try that on (campaign details) • We execute • We measure and adjust (learn & iterate)Monday, March 18, 13
  • 75. Monday, March 18, 13
  • 76. TRENDS IN ANALYTICSMonday, March 18, 13
  • 77. From: To: Aggregate Individual Segment Cohort Generic Vertical Silos Aggregation Daily Realtime Reports Exceptions Pages Events Funnels Influences Desktop Mobile Accounting PredictiveMonday, March 18, 13
  • 78. Aggregate to individual Aggregate to individual e.g. KISSmetricsMonday, March 18, 13
  • 79. Segment to cohort Segment to cohort e.g. MixpanelMonday, March 18, 13
  • 80. Generic to vertical Generic to vertical SaaS Mobile Publishing e.g. Totango, Flurry, Parse.lyMonday, March 18, 13
  • 81. Silos to aggregationMonday, March 18, 13
  • 82. Daily to realtime Daily to realtime Remember: right time is better than real timeMonday, March 18, 13
  • 83. Reports to exceptions Reports to exceptionsMonday, March 18, 13
  • 84. Pages to events Pages to events e.g. ChartbeatMonday, March 18, 13
  • 85. Funnels to influences Funnels to influencesMonday, March 18, 13
  • 86. Desktop to mobile e.g. Keen.ioMonday, March 18, 13
  • 87. Reporting to predictive e.g. SynapsifyMonday, March 18, 13
  • 88. Once, a leader convinced others in the absence of data.Monday, March 18, 13
  • 89. Now, a leader knows what questions to ask.Monday, March 18, 13
  • 90. Thank you. follow me. email me. @byosko byosko@gmail.com instigatorblog.com subscribe. ORDER! leananalyticsbook.comMonday, March 18, 13