The Freemium Model and A/B Testing


Published on

Learn about the Freemium Model and A/B Testing to drive revenue. Presentation from HostingCon 2011 given by Haroon Mokhtarzada, CEO & Co-founder to

Published in: Business, Technology, Design
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Template change hopefully makes this more attractive
  • Template change hopefully makes this more attractive
  • The Freemium Model and A/B Testing

    1. 1. The Freemium Model & A/B Testing<br />HostingCon 2011<br />By Haroon Mokhtarzada<br />CEO & Co-Founder,<br />
    2. 2. Introduction<br />2<br />Haroon Mokhtarzada<br />Co-Founder and CEO <br />Entrepreneur from the age of 10. Co-founded Webs in 2001 with brothers Zeki and Idris while studying Economics at the University of Maryland. Obtained J.D. from Harvard Law School in 2005 and thereafter joined Webs full-time.<br />Founding Vision<br />“Build a free website creation tool so easy that Mom can use it”<br />Funding<br />Raised $12.1 million from Novak Biddle, Columbia Capital and prominent angels in August 2006.<br />
    3. 3. About Webs<br />3<br />Webs is a leading provider of online “front door” solutions for micro businesses across web, mobile and social platforms.<br />40 million websites created since inception<br />Started as a tool for anyone to easily create a website for free and evolved to focus on online solutions for micro businesses<br />Recently acquired and (Freemium Facebook page creation tool)<br />We run an average of 3 different A/B tests at any given time<br />
    4. 4. Freemium – A Success Story<br />4<br />As a disruptive business model, freemium has been the driving force behind many successful startups and $billions in value creation<br />
    5. 5. Simple Fundamentals<br />5<br />Freemium is a business model that works well in scenarios where the marginal cost of users can be driven towards 0 over time or through economies of scale<br />ARPU<br />$ Per Active User<br />Variable Cost<br />Time (or # users)<br />
    6. 6. What Is a Freemium Product/Service?<br />Freemium is NOT…<br />Free trial or sample – user forced to pay after period of time<br />Shareware – user forced to pay to realize actual benefit of product<br />6<br />
    7. 7. What Is a Freemium Product/Service?<br />Freemium IS…<br />A free, usable product/service<br />That attracts users through its price tag and free user base<br />And monetizes with optional paidpremium features(typically subscription based)<br />7<br />
    8. 8. Freemium is merely one option:<br /> Popular web business models:<br />Sell a product to customers (premium)<br />Upsell a product to your users (freemium)<br />Sell other people’s products to your users (ads)<br />Sell your users’ data (ex: addthis)<br />8<br />Thanks!<br />
    9. 9. Why we “chose” Freemium 10 yrs ago<br />Broke - No marketing budget<br />Clueless - Uncertainty about subscription/pricing model<br />Frugal- Cost per user was low<br />9<br />
    10. 10. Challenges for Freemium Players<br />Low Conversion - People who may have been willing to pay normally do not pay because the free product is sufficient<br />Low ARPU – makes it difficult to accelerate growth via traditional marketing methods<br />Infrastructure– millions of free users can create non-trivial and costly infrastructure problems<br />Customer Support – free users often still demand support<br />10<br />
    11. 11. Benefits of the Freemium Model<br />Viral – Rapid adoption, opens new, affordable, channels for getting users<br />Disruptive – Compete with even the most entrenched premium competitors by giving it away for free<br />Superior UI/UX – Offer a better product through rapid testing / customer development<br />11<br />
    12. 12. The Freemium Offering on Webs<br />12<br />
    13. 13. Leveraging our Free Users<br />13<br />Direct Marketing from Free Users<br />Links on bottom of sites<br />Branding - Logo on bottom of sites<br />SEO value from websites back to<br />URL has in it<br />Indirect Marketing from Free Users<br />Word of mouth<br />Size / Credibility<br />
    14. 14. How Premium Businesses Can Leverage Freemium<br />14<br />Offer a limited version for free (but be careful NOT to cannibalize)<br />Create new, related free or freemium product, and use as marketing funnel for your paid product<br />Partner with a freemium product, offer their premium product to your audience<br />
    15. 15. 15<br />Part II: Testing… Testing… 1 2 3<br />HostingCon 2011<br />
    16. 16. The Freemium Conversion Funnel<br />16<br />“What gets measured gets improved”<br />
    17. 17. Conversion Timeline<br />17<br />Understanding When Your Users Convert Is Key<br />Our likelihood of conversion decreases over time<br />Why? Users are publishing a new website and want all features at time of launch<br />Implications: Focus on upselling EARLY in customer lifecycle<br />Other models:<br />User more likely to convert as time goes on:<br />Dropbox (space fills up)<br />Mailchimp (email list grows)<br />Implications: Deemphasize upsell up front, and focus on getting as much usage as possible<br />
    18. 18. Simple A/B Testing<br />Define some goal event<br />Create test treatments<br />Test - randomly split visitors into test & control groups; measure success to goal<br />Good for conversion events with immediacy<br />Ex: Homepage, signup page, shopping cart<br />18<br />
    19. 19. Simple A/B Testing<br />What to test?<br />Calls-to-action (buttons)<br />Copy, Color, Size, Placement, Hover states<br />Forms/Fields<br />Descriptors, Size, Length/number, etc.<br />Text<br />Headlines, Value Props, Links, Amount (long/short)<br />Imagery<br />Segment results by source, use landing pages<br />* Bonus Points<br />19<br />
    20. 20.<br />20<br />Great site for testing inspiration<br />
    21. 21. Case Study<br />21<br />Control<br />Test<br />Shortened Signup Flow<br />Tested shorter 1-page flow against 2-page flow with more fields<br />Shortened version created 10,000 more signups over 3 weeks<br />Going 100% translates to over 25,000 more signups/month!<br />
    22. 22. A/B Test Reporting (GWO)<br />Confidence increases over time<br />Low variation is good<br />Google Website Optimizer is good because:<br />Provides visitors a consistent experience<br />Measures & reports confidence / significance<br />Offers multivariate (combination) testing<br />22<br />
    23. 23. Interpreting Results<br />When A/B Testing is Not Enough<br />Freemium Problem: Signup != $$$<br />A/B signup test only told us which produced more USERS, but does NOT tell us which produced more $$<br />Example: Asking more questions in signup will reduce signups but may increase user success rate and make more $$<br />23<br />
    24. 24. Database Driven Cohort Analysis<br />Solution:<br />A system that permanently fixes a user to a test group, allowing one to run results analysis on any variable that is also attached to that user (i.e. $ spent)<br />Offers the ability to compare users who received different treatments through your entire funnel over time<br />24<br />
    25. 25. Cohort Funnel Analysis Example<br />A/B testing would have shown Test2 as the winner<br />Test 1 makes more money. Tradeoff?<br />25<br />* Dummy data<br />
    26. 26. How to Set Up Cohort Testing<br />Thanks Eric Ries!<br />Display<br />Treatment<br />Yes<br />In Cohort?<br />User Enters<br />Assign to Cohort<br />Coin Toss<br />No<br />Quick & Dirty Shortcut: Mod on userid(ex: show even users one treatment and odd users another)<br />26<br />
    27. 27. Visualizing Results<br />27<br />Day to day reporting on metrics like Premium $ can often be lumpy. <br />Looking at a cumulative view helps visualize more smoothly<br />Diverging lines means one is superior <br />Parallel lines apart means one got lucky;)<br />
    28. 28. Segmentation & Cohort Analysis<br />Advantages<br />Integrated front- and back-end testing<br />Complete ownership of user data<br />Ability to track impact to user long after user-facing experience has gone away<br />Provide users with consistent test experience that integrates your back-end - Great for pricing & promotional tests<br />Ability to “slice” data by internal user variables<br />28<br />
    29. 29. Advanced Segmentation<br />29<br />Total/Aggregate results can hide the real story<br />Not all users are the same<br />Segmenting data and “slicing” can produce surprising gains<br />Treatments can be applied to different segments accordingly<br />
    30. 30. Visualizing Advanced Segmentation<br />30<br />Cumulative Charts<br />Different reports for each segment<br />Can use end results to find interesting variable combinations, then check for significance<br />
    31. 31. Why A/B Testing is Crucial<br />31<br />Lots of Leverage – since your free customers are often your largest referring source, improving your funnel can have compounding effects on both virality and conversion.<br />Users constantly surprise – its impossible to guess what users will do. That’s why all great companies use testing.<br />Validate ideas and time/energy spent on product efforts<br />Better Serve Your Users – build a better product and improve your intuition about your customers<br />
    32. 32. Summary<br />32<br />The freemium model is here to stay<br />When implemented properly, freemium is a disruptive business model allowing rapid growth in new or existing markets<br />Improving conversion is crucial to success, and requires rapid iteration and A/B testing<br />Rapid iteration and testing cannot be done without large influx of free users, favoring those who have achieved scale<br />Successful freemium products are always-improving, and tend to have the superior metrics & UX/UI (out of sheer necessity!)<br />Premium players who do not want to disrupt legacy business can can partner with freemium players to leverage their assets<br />
    33. 33. 33<br />Q&A<br />Haroon Mokhtarzada, CEO & Co-Founder of Webs<br />@haroon,<br />