Your SlideShare is downloading. ×
The Freemium Model and A/B Testing
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

The Freemium Model and A/B Testing


Published on

Learn about the Freemium Model and A/B Testing to drive revenue. Presentation from HostingCon 2011 given by Haroon Mokhtarzada, CEO & Co-founder to

Learn about the Freemium Model and A/B Testing to drive revenue. Presentation from HostingCon 2011 given by Haroon Mokhtarzada, CEO & Co-founder to

Published in: Business, Technology, Design
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide
  • Template change hopefully makes this more attractive
  • Template change hopefully makes this more attractive
  • Transcript

    • 1. The Freemium Model & A/B Testing
      HostingCon 2011
      By Haroon Mokhtarzada
      CEO & Co-Founder,
    • 2. Introduction
      Haroon Mokhtarzada
      Co-Founder and CEO
      Entrepreneur from the age of 10. Co-founded Webs in 2001 with brothers Zeki and Idris while studying Economics at the University of Maryland. Obtained J.D. from Harvard Law School in 2005 and thereafter joined Webs full-time.
      Founding Vision
      “Build a free website creation tool so easy that Mom can use it”
      Raised $12.1 million from Novak Biddle, Columbia Capital and prominent angels in August 2006.
    • 3. About Webs
      Webs is a leading provider of online “front door” solutions for micro businesses across web, mobile and social platforms.
      40 million websites created since inception
      Started as a tool for anyone to easily create a website for free and evolved to focus on online solutions for micro businesses
      Recently acquired and (Freemium Facebook page creation tool)
      We run an average of 3 different A/B tests at any given time
    • 4. Freemium – A Success Story
      As a disruptive business model, freemium has been the driving force behind many successful startups and $billions in value creation
    • 5. Simple Fundamentals
      Freemium is a business model that works well in scenarios where the marginal cost of users can be driven towards 0 over time or through economies of scale
      $ Per Active User
      Variable Cost
      Time (or # users)
    • 6. What Is a Freemium Product/Service?
      Freemium is NOT…
      Free trial or sample – user forced to pay after period of time
      Shareware – user forced to pay to realize actual benefit of product
    • 7. What Is a Freemium Product/Service?
      Freemium IS…
      A free, usable product/service
      That attracts users through its price tag and free user base
      And monetizes with optional paidpremium features(typically subscription based)
    • 8. Freemium is merely one option:
      Popular web business models:
      Sell a product to customers (premium)
      Upsell a product to your users (freemium)
      Sell other people’s products to your users (ads)
      Sell your users’ data (ex: addthis)
    • 9. Why we “chose” Freemium 10 yrs ago
      Broke - No marketing budget
      Clueless - Uncertainty about subscription/pricing model
      Frugal- Cost per user was low
    • 10. Challenges for Freemium Players
      Low Conversion - People who may have been willing to pay normally do not pay because the free product is sufficient
      Low ARPU – makes it difficult to accelerate growth via traditional marketing methods
      Infrastructure– millions of free users can create non-trivial and costly infrastructure problems
      Customer Support – free users often still demand support
    • 11. Benefits of the Freemium Model
      Viral – Rapid adoption, opens new, affordable, channels for getting users
      Disruptive – Compete with even the most entrenched premium competitors by giving it away for free
      Superior UI/UX – Offer a better product through rapid testing / customer development
    • 12. The Freemium Offering on Webs
    • 13. Leveraging our Free Users
      Direct Marketing from Free Users
      Links on bottom of sites
      Branding - Logo on bottom of sites
      SEO value from websites back to
      URL has in it
      Indirect Marketing from Free Users
      Word of mouth
      Size / Credibility
    • 14. How Premium Businesses Can Leverage Freemium
      Offer a limited version for free (but be careful NOT to cannibalize)
      Create new, related free or freemium product, and use as marketing funnel for your paid product
      Partner with a freemium product, offer their premium product to your audience
    • 15. 15
      Part II: Testing… Testing… 1 2 3
      HostingCon 2011
    • 16. The Freemium Conversion Funnel
      “What gets measured gets improved”
    • 17. Conversion Timeline
      Understanding When Your Users Convert Is Key
      Our likelihood of conversion decreases over time
      Why? Users are publishing a new website and want all features at time of launch
      Implications: Focus on upselling EARLY in customer lifecycle
      Other models:
      User more likely to convert as time goes on:
      Dropbox (space fills up)
      Mailchimp (email list grows)
      Implications: Deemphasize upsell up front, and focus on getting as much usage as possible
    • 18. Simple A/B Testing
      Define some goal event
      Create test treatments
      Test - randomly split visitors into test & control groups; measure success to goal
      Good for conversion events with immediacy
      Ex: Homepage, signup page, shopping cart
    • 19. Simple A/B Testing
      What to test?
      Calls-to-action (buttons)
      Copy, Color, Size, Placement, Hover states
      Descriptors, Size, Length/number, etc.
      Headlines, Value Props, Links, Amount (long/short)
      Segment results by source, use landing pages
      * Bonus Points
    • 20.
      Great site for testing inspiration
    • 21. Case Study
      Shortened Signup Flow
      Tested shorter 1-page flow against 2-page flow with more fields
      Shortened version created 10,000 more signups over 3 weeks
      Going 100% translates to over 25,000 more signups/month!
    • 22. A/B Test Reporting (GWO)
      Confidence increases over time
      Low variation is good
      Google Website Optimizer is good because:
      Provides visitors a consistent experience
      Measures & reports confidence / significance
      Offers multivariate (combination) testing
    • 23. Interpreting Results
      When A/B Testing is Not Enough
      Freemium Problem: Signup != $$$
      A/B signup test only told us which produced more USERS, but does NOT tell us which produced more $$
      Example: Asking more questions in signup will reduce signups but may increase user success rate and make more $$
    • 24. Database Driven Cohort Analysis
      A system that permanently fixes a user to a test group, allowing one to run results analysis on any variable that is also attached to that user (i.e. $ spent)
      Offers the ability to compare users who received different treatments through your entire funnel over time
    • 25. Cohort Funnel Analysis Example
      A/B testing would have shown Test2 as the winner
      Test 1 makes more money. Tradeoff?
      * Dummy data
    • 26. How to Set Up Cohort Testing
      Thanks Eric Ries!
      In Cohort?
      User Enters
      Assign to Cohort
      Coin Toss
      Quick & Dirty Shortcut: Mod on userid(ex: show even users one treatment and odd users another)
    • 27. Visualizing Results
      Day to day reporting on metrics like Premium $ can often be lumpy.
      Looking at a cumulative view helps visualize more smoothly
      Diverging lines means one is superior
      Parallel lines apart means one got lucky;)
    • 28. Segmentation & Cohort Analysis
      Integrated front- and back-end testing
      Complete ownership of user data
      Ability to track impact to user long after user-facing experience has gone away
      Provide users with consistent test experience that integrates your back-end - Great for pricing & promotional tests
      Ability to “slice” data by internal user variables
    • 29. Advanced Segmentation
      Total/Aggregate results can hide the real story
      Not all users are the same
      Segmenting data and “slicing” can produce surprising gains
      Treatments can be applied to different segments accordingly
    • 30. Visualizing Advanced Segmentation
      Cumulative Charts
      Different reports for each segment
      Can use end results to find interesting variable combinations, then check for significance
    • 31. Why A/B Testing is Crucial
      Lots of Leverage – since your free customers are often your largest referring source, improving your funnel can have compounding effects on both virality and conversion.
      Users constantly surprise – its impossible to guess what users will do. That’s why all great companies use testing.
      Validate ideas and time/energy spent on product efforts
      Better Serve Your Users – build a better product and improve your intuition about your customers
    • 32. Summary
      The freemium model is here to stay
      When implemented properly, freemium is a disruptive business model allowing rapid growth in new or existing markets
      Improving conversion is crucial to success, and requires rapid iteration and A/B testing
      Rapid iteration and testing cannot be done without large influx of free users, favoring those who have achieved scale
      Successful freemium products are always-improving, and tend to have the superior metrics & UX/UI (out of sheer necessity!)
      Premium players who do not want to disrupt legacy business can can partner with freemium players to leverage their assets
    • 33. 33
      Haroon Mokhtarzada, CEO & Co-Founder of Webs