The document discusses the freemium business model and the importance of A/B testing for improving conversion rates. It provides an overview of how Webs uses a freemium model to attract millions of free users and then employs A/B testing and cohort analysis to optimize the user experience and increase monetization through paid upgrades. Key points include how testing small changes like buttons, copy, forms can have major impacts on signups and revenue, and how segmenting users allows testing effects on different groups to be analyzed over time. The document advocates that continuous testing is crucial for freemium companies to rapidly iterate and better serve their large free user bases.

1. The Freemium Model & A/B TestingHostingCon 2011By Haroon MokhtarzadaCEO & Co-Founder, Webs.com

2. Introduction2Haroon MokhtarzadaCo-Founder and CEO Entrepreneur from the age of 10. Co-founded Webs in 2001 with brothers Zeki and Idris while studying Economics at the University of Maryland. Obtained J.D. from Harvard Law School in 2005 and thereafter joined Webs full-time.Founding Vision“Build a free website creation tool so easy that Mom can use it”FundingRaised $12.1 million from Novak Biddle, Columbia Capital and prominent angels in August 2006.

3. About Webs3Webs is a leading provider of online “front door” solutions for micro businesses across web, mobile and social platforms.40 million websites created since inceptionStarted as a tool for anyone to easily create a website for free and evolved to focus on online solutions for micro businessesRecently acquired and relaunchedPagemodo.com (Freemium Facebook page creation tool)We run an average of 3 different A/B tests at any given time

4. Freemium – A Success Story4As a disruptive business model, freemium has been the driving force behind many successful startups and $billions in value creation

5. Simple Fundamentals5Freemium is a business model that works well in scenarios where the marginal cost of users can be driven towards 0 over time or through economies of scaleARPU$ Per Active UserVariable CostTime (or # users)

6. What Is a Freemium Product/Service?Freemium is NOT…Free trial or sample – user forced to pay after period of timeShareware – user forced to pay to realize actual benefit of product6

7. What Is a Freemium Product/Service?Freemium IS…A free, usable product/serviceThat attracts users through its price tag and free user baseAnd monetizes with optional paidpremium features(typically subscription based)7

8. Freemium is merely one option: Popular web business models:Sell a product to customers (premium)Upsell a product to your users (freemium)Sell other people’s products to your users (ads)Sell your users’ data (ex: addthis)8Thanks!

9. Why we “chose” Freemium 10 yrs agoBroke - No marketing budgetClueless - Uncertainty about subscription/pricing modelFrugal- Cost per user was low9

10. Challenges for Freemium PlayersLow Conversion - People who may have been willing to pay normally do not pay because the free product is sufficientLow ARPU – makes it difficult to accelerate growth via traditional marketing methodsInfrastructure– millions of free users can create non-trivial and costly infrastructure problemsCustomer Support – free users often still demand support10

11. Benefits of the Freemium ModelViral – Rapid adoption, opens new, affordable, channels for getting usersDisruptive – Compete with even the most entrenched premium competitors by giving it away for freeSuperior UI/UX – Offer a better product through rapid testing / customer development11

12. The Freemium Offering on Webs12

13. Leveraging our Free Users13Direct Marketing from Free UsersLinks on bottom of sitesBranding - Logo on bottom of sitesSEO value from websites back to Webs.comURL has webs.com in itIndirect Marketing from Free UsersWord of mouthSize / Credibility

14. How Premium Businesses Can Leverage Freemium14Offer a limited version for free (but be careful NOT to cannibalize)Create new, related free or freemium product, and use as marketing funnel for your paid productPartner with a freemium product, offer their premium product to your audience

15. 15Part II: Testing… Testing… 1 2 3HostingCon 2011

16. The Freemium Conversion Funnel16“What gets measured gets improved”

17. Conversion Timeline17Understanding When Your Users Convert Is KeyOur likelihood of conversion decreases over timeWhy? Users are publishing a new website and want all features at time of launchImplications: Focus on upselling EARLY in customer lifecycleOther models:User more likely to convert as time goes on:Dropbox (space fills up)Mailchimp (email list grows)Implications: Deemphasize upsell up front, and focus on getting as much usage as possible

18. Simple A/B TestingDefine some goal eventCreate test treatmentsTest - randomly split visitors into test & control groups; measure success to goalGood for conversion events with immediacyEx: Homepage, signup page, shopping cart18

19. Simple A/B TestingWhat to test?Calls-to-action (buttons)Copy, Color, Size, Placement, Hover statesForms/FieldsDescriptors, Size, Length/number, etc.TextHeadlines, Value Props, Links, Amount (long/short)ImagerySegment results by source, use landing pages* Bonus Points19

20. AbTests.com20Great site for testing inspiration

21. Case Study21ControlTestShortened Signup FlowTested shorter 1-page flow against 2-page flow with more fieldsShortened version created 10,000 more signups over 3 weeksGoing 100% translates to over 25,000 more signups/month!

22. A/B Test Reporting (GWO)Confidence increases over timeLow variation is goodGoogle Website Optimizer is good because:Provides visitors a consistent experienceMeasures & reports confidence / significanceOffers multivariate (combination) testing22

23. Interpreting ResultsWhen A/B Testing is Not EnoughFreemium Problem: Signup != $$$A/B signup test only told us which produced more USERS, but does NOT tell us which produced more $$Example: Asking more questions in signup will reduce signups but may increase user success rate and make more $$23

24. Database Driven Cohort AnalysisSolution:A system that permanently fixes a user to a test group, allowing one to run results analysis on any variable that is also attached to that user (i.e. $ spent)Offers the ability to compare users who received different treatments through your entire funnel over time24

25. Cohort Funnel Analysis ExampleA/B testing would have shown Test2 as the winnerTest 1 makes more money. Tradeoff?25* Dummy data

26. How to Set Up Cohort TestingThanks Eric Ries!DisplayTreatmentYesIn Cohort?User EntersAssign to CohortCoin TossNoQuick & Dirty Shortcut: Mod on userid(ex: show even users one treatment and odd users another)26

27. Visualizing Results27Day to day reporting on metrics like Premium $ can often be lumpy. Looking at a cumulative view helps visualize more smoothlyDiverging lines means one is superior Parallel lines apart means one got lucky;)

28. Segmentation & Cohort AnalysisAdvantagesIntegrated front- and back-end testingComplete ownership of user dataAbility to track impact to user long after user-facing experience has gone awayProvide users with consistent test experience that integrates your back-end - Great for pricing & promotional testsAbility to “slice” data by internal user variables28

29. Advanced Segmentation29Total/Aggregate results can hide the real storyNot all users are the sameSegmenting data and “slicing” can produce surprising gainsTreatments can be applied to different segments accordingly

30. Visualizing Advanced Segmentation30Cumulative ChartsDifferent reports for each segmentCan use end results to find interesting variable combinations, then check for significance

31. Why A/B Testing is Crucial31Lots of Leverage – since your free customers are often your largest referring source, improving your funnel can have compounding effects on both virality and conversion.Users constantly surprise – its impossible to guess what users will do. That’s why all great companies use testing.Validate ideas and time/energy spent on product effortsBetter Serve Your Users – build a better product and improve your intuition about your customers

32. Summary32The freemium model is here to stayWhen implemented properly, freemium is a disruptive business model allowing rapid growth in new or existing marketsImproving conversion is crucial to success, and requires rapid iteration and A/B testingRapid iteration and testing cannot be done without large influx of free users, favoring those who have achieved scaleSuccessful freemium products are always-improving, and tend to have the superior metrics & UX/UI (out of sheer necessity!)Premium players who do not want to disrupt legacy business can can partner with freemium players to leverage their assets

33. 33Q&AHaroon Mokhtarzada, CEO & Co-Founder of Webs@haroon, haroon@webs.com