Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Lean Analytics: How to find & test innovative Growth Hacks using Analytics

2,450 views

Published on

As a tribute to excellent book by work of Alistair Croll and Ben Yoskovitz, this my interpretation of Lean Analytics from my experience of consulting with start-ups.

Digital Analytics is at the heart of Growth Hacking mentality. It justifies, that marketing hacks and product changes are gaining traction, or need to be abandoned, and allows start-ups to interrelate quickly using learnings from conversion data and customer feedback. As real-time data usage becomes more widespread, this process will only accelerate.

During the workshop I will introduce participants to:
* Brief introduction to Lean Analytics Thinking, such as MVP, Iterations & Agility.
* Why an obsession over performance of one metric is important
* Explain difference between website vs product innovating and testing?
* Look as some examples of successful (and unsuccessful) Analytics Hacks.
* Developing a super analytics hack for your business.
* Process for testing and refining your hacks.

Published in: Data & Analytics
  • Today, I want to share with you my own "unfair advantage" ... An honest crack at an insider's edge that's so effective it's nothing less than performance enhancing for your own bottom line profits! ●●● http://t.cn/A6hPRSfx
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT (2019 Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download Full EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download Full doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book THIS can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer THIS is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBooks .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story THIS Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money THIS the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths THIS Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Get Paid To Waste Time On YouTube! ■■■ http://ishbv.com/socialpaid/pdf
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • The Odds of The Lotto Are Changing With These Tricks! ◆◆◆ http://t.cn/Airf5UFH
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • D0WNL0AD FULL ▶ ▶ ▶ ▶ http://1url.pw/73BU0 ◀ ◀ ◀ ◀
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Lean Analytics: How to find & test innovative Growth Hacks using Analytics

  1. 1. Lean Analytics Workout DAAHub Phil Pearce April 2014 Fitness Consultant linkedin.com/in/philpearce
  2. 2. Harder, Better, Faster, Stronger Leaner!
  3. 3. Who have we got in the room? 1. Entrepreneur – Started company 2. Corporation – Work in big company 3. Agency – Help startups
  4. 4. Have you read any of these?
  5. 5. Quick Quiz https://www.youtube.com/watch?v=usdeiJP7xh0 and http://prezi.com/lw-arulaenh4/copy-of-lean-startup-buzz-words/
  6. 6. START START GH PvT PMF LS Cdev FINISH FINISH FINISH IyBITwC TwCIyBi P2 E4 MVP BML VM
  7. 7. Web Analytics Exchange mentor750 GA questions answered Tracking protection group About Me Phil Pearce Analytics Consultant linkedin.com/in/philpearce
  8. 8. I`m not an entrepreneur Apart from this one
  9. 9. AdWords But... I have done alot of agency consulting & I worked for some innovative startups Sold for €16m Pivoted Changed business model IPO in ~1yrs Funded by Gwyneth Paltrow Sold for €37m Crazy growth & IPO plans IPO soon Metrics Plan Massive Revenue understanding own sites digital value to understand investments Grew Taxi booking Revenue by €10m in 2yrs
  10. 10. Intrapreneur & Technical marketer 1. Build PPC reporting platform MS access 2. Enabled KW level ROI bidding in 2007. 3. Managed £600K pm Adwords account & out- performed market leader. 4. Built end-to-end affiliate tracking system. 5. Reverse engineered Adwords Algo. 6. Built mathematical ClickFraud detection tool for mobile 7. Built free version of SpeedPPC 8. Building “4clicks” SaaS for Magneto (KPIs, dataLayer, Dashboards, Remarketing -> all auto- enabled)
  11. 11. GH GH GH GH GH GH GH GH GH GH ...and closet growth hacker FINISH GH
  12. 12. ... I have author-ed a book on Amazon
  13. 13. Agenda • Start: 9:30am-12:30am • Introduce Lean Analytics terminology – (e.g. MVP, Iterations, Agility) • Explain why obsessing over the performance of one key metric is vital • Describe the difference between website and product innovating and testing? • Look at some examples of successful (and unsuccessful) analytics hacks • Develop a super analytics hack for your business • Define a process for testing and refining your hacks
  14. 14. If you build it … they will come.
  15. 15. They will come ... if you build it
  16. 16. Because… Fast Feedback = Build what customers want Favourite Food
  17. 17. Most startups don’t know what their customers will consume (or what they are good at making) Hotmail was a database company Flickr was going to be an Video Game platform Twitter was a podcasting company Autodesk made desktop automation Paypal first built for Palmpilots Freshbooks was invoicing for a web design firm Wikipedia was to be written by experts only Mitel was a lawnmower company
  18. 18. Fast iterations/sprints using… Build > Measure > Learn BML Build • (products) Measure • (data) Learn • (ideas)
  19. 19. Fast iterations/sprints using… Build > Measure > Learn (repeat) BML Build • (products) Measure • (data) Learn • (ideas)
  20. 20. Even the book uses lean principles… 1. 5th edition in 8months (new edition every built 1.5months!) 2. “We liked to hear from you” feedback section in front & online blog comments encouraged. 3. Learnings have spawned start-up conferences
  21. 21. Build > Measure > Learn Measure
  22. 22. Problem: We lie to ourselves “We” are amazing!
  23. 23. Reality check…
  24. 24. …Analytics to the rescue
  25. 25. Analytics is the measurement of movement towards your business goals.
  26. 26. In a startup, the purpose of analytics is to iterate to product/market fit before the money runs out.
  27. 27. I have twocoins. Atleast one of them is heads.
  28. 28. What is the % probability that the other is tails?
  29. 29. Guess…
  30. 30. Tails Tails Heads Tails Tails Heads Heads Heads
  31. 31. Heads Tails Tails Heads Heads Heads 2 of 3 (66%) are tails.
  32. 32. Some fundamentals.
  33. 33. A good metric is: Understandable If you’re busy explaining the data, you won’t be busy acting on it. Comparative Comparison is context. Aratio or rate The only way to measure change and roll up the tension between two metrics (Miles Per Hour) Behavior changing If you’re busy explaining the data, you won’t be busy acting on it.
  34. 34. simplest rule Not a good metric. If metric won’t change how you behave, it’s…
  35. 35. Metrics help you know yourself. You are just like Customers that buy >1x in 90d Your customers will buy from you Then you are in this mode Acquisition 70% of retailers Once1-15% Low acquisition cost, high checkout Hybrid 20% of retailers 2-2.5 per year 15-30% Increasing return rates, market share Focus on Loyalty 10% of retailers >2.5 per year >30% Loyalty, selection, inventory size (Thanks to Kevin Hillstrom for this.)
  36. 36. Qualitative Unstructured, anecdotal, revealing, hard to aggregate, often too positive & reassuring. Warm and fuzzy. Quantitative Numbersand stats. Hard facts, less insight, easier to analyze; often sour and disappointing. Cold and hard.
  37. 37. Exploratory Speculative. Tries to find unexpected or interesting insights. Source of unfair advantages. Cool. Reporting Predictable. Keeps you abreast of the normal, day-to-day operations. Can be managed by exception. Necessary.
  38. 38. Rumsfeld on Analytics (Or rather, Avinash Kaushik channeling Rumsfeld) Things we know don’t know we know Are facts which may be wrong and should be checked against data. we don’t know Are questions we can answer by reporting, which we should baseline & automate. we know Are intuition which we should quantify and teach to improve effectiveness, efficiency. we don’t know Are exploration which is where unfair advantage and interesting epiphanies live.
  39. 39. May A/B test: Changing one thing (i.e. color) and measuring the result (i.e. revenue.) AprMar 0 Jan Feb Segment: Cross-sectional comparison of all people divided by some attribute (age, gender, etc.) Slicing and dicing data 5,000 Active users Cohort: Comparison of similar groups along a timeline. (this is the April cohort) Multivariate analysis Changing several things at once to see which correlates with a result. ☀ ☁ ☀ ☁ ☀ ☁
  40. 40. Which of these two companies is doing better?
  41. 41. Is this company growing or stagnating? Which of these two companies has the best Revenue/Customer? January February March April May Rev/customer $5.00 $ 4.50 $4.33 $4.25 $4.50 Cohort January February March April May Averages Cohort group5 € 5.00 € 6.00 € 7.00 € 8.00 € 9.00 € 7.00 group4 € 3.00 € 4.00 € 6.00 € 7.00 € 5.00 group3 € 2.00 € 2.00 € 5.00 € 3.00 group2 € 1.00 € 1.00 € 1.00 group1 € 0.50 € 0.50
  42. 42. Lagging Historical. Shows you how you’re doing; reports the news. Example: sales. Explaining the past. Leading Forward-looking. Number today that predicts tomorrow; reports the news. Example: pipeline. Predicting the future.
  43. 43. • AFacebook user reaching 7 friends within 10 days of signing up (Chamath Palihapitiya) • If someone comes back to Zynga a day after signing up for a game, they’ll probably become an engaged, paying user (Nabeel Hyatt) • ADropbox user who puts at least one file in one folder on one device (ChenLi Wang) • Twitter user following a certain number of people, and a certain percentage of those people following the user back (Josh Elman) • ALinkedIn user getting to X connections in Y days (Elliot Schmukler) Some examples (From the 2012 Growth Hacking conference. http://growthhackersconference.com/)
  44. 44. Which means it’s time to talk about correlation.
  45. 45. Number of Analysts ChessStarTrek correlated Liked Maths causal Number of Analysts Correlated vs Causal P2 E4
  46. 46. Correlated vs Causal But it is not the cause! Strong Correlation
  47. 47. Correlated Two variables that are related (but may be dependent on something else.) Ice cream & drowning. Causal An independent variable that directly impacts a dependent one. Summertime & drowning.
  48. 48. A leading, causal metric is a superpower. h" p ://www.flickr.com/photos/bloke_with_camera/401812833/sizes/o/in/photostream/
  49. 49. Growth hacking, demystified. Find correlation Test causality Optimize the causal factor Pick a metric to change
  50. 50. Why is Nigerian spam so badly written?
  51. 51. Aunshul Rege of Rutgers University, USA in 2009 Experienced scammers expect a “strike rate” of 1 or 2 replies per 1,000 messages emailed; they expect to land 2 or 3 “Mugu” (fools) each week. One scammer boasted “When you get a reply it’s 70% sure you’ll get the money” “By sending an email that repels all but the most gullible,” says [Microsoft Researcher Corman] Herley, “the scammer gets the most promising marks to self-select, and tilts the true to false positive ratio in his favor.” 1000 emails 1-2 responses 1 fool and their money, parted. Bad language (0.1% conversion) Gullible (70% conversion) 1000 emails 100 responses 1 fool and their money, parted. Good language (10% conversion) Not-gullible (.07% conversion) This would be horribly inefficient since humans are involved.
  52. 52. Turns out the word “Nigeria” is the best way to identify promising prospects.
  53. 53. Nigerian spammers really understand their target market. They see past vanity metrics.
  54. 54. The Lean Analytics framework.
  55. 55. Eric’s three engines of growth Virality Make people invite friends. How many they tell, how fast they tell them. Price Spend money to get customers. Customers are worth more than they cost. Stickiness Keep people coming back. Approach Get customers faster than you lose them. Math that matters
  56. 56. @agatestudio Lean Analytics Stages Empathy • I’ve found a real, poorly-met need & reachable market faces Stickiness • I’ve figured out how to solve the problem, in a way they will adore and pay for! Virality • I’ve built the right product/features/functionality that keeps users around. Revenue • The users and features fuel growth organically and artificially. Scale • I’ve found a sustainable, scalable business with right margin in a healthy ecosystem.
  57. 57. 1. Ecommerce 2. Two sided marketplace 3. SaaS 4. Mobile app 5. Media/Publishing 6. User generate content Six business model types
  58. 58. Model + Stage = One Metric That Matters. One Metric That Matters. The business you’re in E-Com 2-Sided SaaS Mobile Media UCG Empathy Stickiness Virality Revenue Scale Thestageyou’reat
  59. 59. Really? Just one?
  60. 60. Yes, one!
  61. 61. Because… In a startup`s “focus" is hard to achieve.
  62. 62. Having only one metric resolves this problem.
  63. 63. www.theeastsiderla.com Prevents distraction
  64. 64. Metrics are like squeeze toys. http://www.flickr.com/photos/connortarter/4791605202/
  65. 65. Revenue stage: CompareAndSave.com (2-sided marketplace) • Focus on one metric of CTR • Reduced CPC • Increased RPC (Effected of reverse economies of scale & tiered cpa volumes) • Marketplace: Consumers + Banks Technically a “comparison engine”
  66. 66. Empathy Stickiness Virality Revenue Scale E- commerce SaaS Media Mobile app User-gen content 2-sided market Loyalty, conversion CAC, shares, reactivation Transaction, CLV Affiliates, white-label Engagement, churn Inherent virality, CAC Upselling, CAC, CLV API, magic #, mktplace Content, spam Invites, sharing Ads, donations Analytics, user data Inventory, listings SEM, sharing Transactions, commission Other verticals (Money from transactions) Downloads, churn, virality WoM, app ratings, CAC CLV, ARPDAU Spinoffs, publishers (Money from active users) Traffic, visits, returns Content virality, SEM CPE, affiliate %, eyeballs Syndication, licenses (Money from ad clicks)
  67. 67. Workshop Task: 1. Select business type (E-Com, 2-Sided, SaaS, Mobile, Media, UCG) 2. Determine Stage (Empathy, Stickiness, Virality, Revenue, Scale) 3. Pick one metric 4. Set line in the sand (benchmark) Useful sheet bit.ly/BigLeanTable
  68. 68. Other measurement models bit.ly/kpishake
  69. 69. What other metrics do you want to know about?
  70. 70. Drawing some lines in the sand.
  71. 71. A company loses a quarter of its customers every year. Is this good or bad?
  72. 72. Baseline: 10% visitor engagement/day 30% of users/month use web or mobile app 10% of users/day use web or mobile app 1% of users/day use it concurrently
  73. 73. Baseline: 2-5% monthly churn • The best SaaS get 1.5% - 3% a month. They have multiple Ph.D’s on the job. • Get below a 5% monthly churn rate before you know you’ve got a business that’s ready to grow (Mark MacLeod) and around 2% before you really step on the gas (David Skok) • Last-ditch appeals and reactivation can have a big impact. Facebook’s “don’t leave” reduces attrition by 7%.
  74. 74. Who is worth more? Lifetime: $200 Lifetime: $200 Today A Roberto Medri, Etsy B Visits
  75. 75. The Lean Analytics cycle
  76. 76. Did we move the needle? Make changes in production Hypothesis Design a test Make changes in production Measure the results Success! Pivot or give up Pick a KPI Find a potential improvement Draw a line With data: find a commonality Without data: make a good guess Draw a new line Repeat test Did we move the needle?
  77. 77. Do AirBnB hosts get more business if their property is professionally photographed?
  78. 78. Gut instinct (hypothesis) Professional photography helps AirBnB’s business Candidate solution (MVP) 20 field photographers posing as employees Measure the results Compare photographed listings to a control group Make a decision Launch photography as a new feature for all hosts
  79. 79. 5,000 shoots per month by February 2012
  80. 80. Draw a new line Pivot or give up Find a potential improvement Try again Success! Did we move the needle? Measure the results Make changes in production Design a test Hypothesis With data: find a commonality Without data: make a good guess Draw a linePick a KPI
  81. 81. “G ee, tho se ho u se s that d o w e ll l o o k rea l l y n ic e.” Ma ybe i t ’s the ca m er a . “C o m puter : What d o a l l the h ig hl y r en te d ho u se s hav e i n co m m o n ?” C a m er a m o d el . With data: find a commonality Without data: make a good guess
  82. 82. Some non-tech examples.
  83. 83. I lied. Everyone is a tech company.
  84. 84. http://www.flickr.com/photos/puuikibeach/4789015423 Cost of attention: way up. http://www.flickr.com/photos/elcapitanbsc/3936927326 Cost of experiments:down.
  85. 85. Let’s pick on restaurants for a while.
  86. 86. A line in the sand Labor costs Gross revenue 30% 20% Just right Understaffed? = 24% Too costly?
  87. 87. A leading indicator http://www.flickr.com/photos/avlxyz/4889656453http://www.flickr.com/photos/mysticcountry/3567440970 50 reservations at 5PM 250 covers that night (Varies by restaurant. McDonalds ≠ Fat Duck.)
  88. 88. http://www.flickr.com/photos/southbeachcars/6892880699 Restaurant MVP
  89. 89. Is tip amount a leading indicator of long- term revenue?
  90. 90. Why does every table get the same menu?
  91. 91. Is purple ink better? http://tippingresearch.com/uploads/managing_tips.pdf
  92. 92. Growth hacking (is a word you should hate but will hear a lot about.)
  93. 93. Growth hacking, demystified. Find correlation Test causality Optimize the causal factor Pick a metric to change
  94. 94. Guerrilla marketing Data- driven learning Subversiveness GROWTH HACKING
  95. 95. • A Facebook user reaching 7 friends within 10 days of signing up (Chamath Palihapitiya) • If someone comes back to Zynga a day after signing up for a game, they’ll probably become an engaged, paying user (Nabeel Hyatt) • A Dropbox user who puts at least one file in one folder on one device (ChenLi Wang) • Twitter user following a certain number of people, and a certain percentage of those people following the user back (Josh Elman) • A LinkedIn user getting to Xconnections in Y days (Elliot Schmukler) (These are also great segments to analyze.) Leading indicators: Growth Hacks Read more examples: http://www.slideshare.net/mattangriffel/growth-hacking
  96. 96. • Growth hacking is simply what marketing should have been doing, but it fell in love with Don Draper and opinions along the way • Optimize a factor you think is correlated with growth The growth hack
  97. 97. Growth Hacking examples • Hotmail – P.S. I love you • Drobbox – Refer a friend • Facebook – Exclusive network appeal • Twitter – follow celebrities Read more examples: http://www.slideshare.net/mattangriffel/growth-hacking
  98. 98. AirBnB and Craigslist
  99. 99. What is PPC Growth? Adwords Marketing Conversion Data Sales Growth Classic model 1. Create new marketing campaign. 2. If CPA data is within target threshold = then growth achieved 3. Then Increase marketing. Repeat above steps, until threshold reached. Repeat until CPA threshold reached
  100. 100. And… needs to be Scalable, Repeatable and Sustainable. Growth Hacking for PPC SaaS Adwords Marketing Product Data Growth Hacking Note: “Product” could be value-proposition or incentives
  101. 101. SaaS hack
  102. 102. Have you Heard of “Battleships”?” Growth Hacking a PPC tool…
  103. 103. Growth Hacking a PPC tool… Download CTR battleships http://bit.ly/battleshipsctr
  104. 104. And… needs to be Scalable, Repeatable and Sustainable. Growth Hacking a PPC tool… Adwords Marketing (capture/test initial lead) Product (Variation on QS grader) Adwords API Data Growth Hacking Same product but different UI/Landing page - value proposition tweaked and made fun / viral… e.g every one likes to show off :)
  105. 105. Growth Hacking a PPC tool… Winner must tweet winning strategy #ctrbattleships Or take a photo them holding the Prize! #ctrbattleships Download CTR battleships http://bit.ly/battleshipsctr
  106. 106. Some tools and traps
  107. 107. Traction graphs Your business model The stage you’re at Yourone metric ... change often if you’re doing it right. So how do you track that over time?
  108. 108. Traction graphs Jan Signup sper day Feb Mar Conversio nrate Apr Chur nrate May Jun Viral coefficient This axis changes for each metric
  109. 109. Traction graphs Jan Signups per day Feb Mar Conversion rate Apr Churn rate May Jun Viral coefficient 0%
  110. 110. Use vanity to get to meaningfulmetrics • Your goal is to produce outcomes • If the outcomes require action, and vanity motivates actors, use it! • But show how the vanity metric is a leading indicator of the real one! x Web traffic Revenue Activation Cart Size Conversion rate VM
  111. 111. “The most important figures that one needs for management are unknown or unknowable, but successful management must nevertheless take account of them” Lloyd S. Nelson
  112. 112. Pic by Twodolla on Flickr. http://www.flickr.com/photos/twodolla/3168857844
  113. 113. ARCHIMEDES HAD TAKEN BATHS BEFORE
  114. 114. Once, a leader leader/king convinced others in the absence of data.
  115. 115. Now, a leader/king knows what questions to ask.
  116. 116. Alistair Croll & Ben Yoskovitz Special thanks to… Please buy their book if you want more info!
  117. 117. Copy of these slides bit.ly/leananalytics2
  118. 118. Appendix: Useful videos
  119. 119. 15min Lean Start-up Video https://www.youtube.com/watch?v=zOX1vC7_n6s
  120. 120. 60min Lean Analytics https://www.youtube.com/watch?v=-CB4w_OtrKw
  121. 121. 50min podcast www.twistimage.com/podcast/mp3/SPOS_351_-_Alistair_Croll.mp3

×