• Like
The Freemium Model and A/B Testing
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

The Freemium Model and A/B Testing


Learn about the Freemium Model and A/B Testing to drive revenue. Presentation from HostingCon 2011 given by Haroon Mokhtarzada, CEO & Co-founder to Webs.com

Learn about the Freemium Model and A/B Testing to drive revenue. Presentation from HostingCon 2011 given by Haroon Mokhtarzada, CEO & Co-founder to Webs.com

Published in Business , Technology , Design
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads


Total Views
On SlideShare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide
  • Template change hopefully makes this more attractive
  • Template change hopefully makes this more attractive


  • 1. The Freemium Model & A/B Testing
    HostingCon 2011
    By Haroon Mokhtarzada
    CEO & Co-Founder, Webs.com
  • 2. Introduction
    Haroon Mokhtarzada
    Co-Founder and CEO
    Entrepreneur from the age of 10. Co-founded Webs in 2001 with brothers Zeki and Idris while studying Economics at the University of Maryland. Obtained J.D. from Harvard Law School in 2005 and thereafter joined Webs full-time.
    Founding Vision
    “Build a free website creation tool so easy that Mom can use it”
    Raised $12.1 million from Novak Biddle, Columbia Capital and prominent angels in August 2006.
  • 3. About Webs
    Webs is a leading provider of online “front door” solutions for micro businesses across web, mobile and social platforms.
    40 million websites created since inception
    Started as a tool for anyone to easily create a website for free and evolved to focus on online solutions for micro businesses
    Recently acquired and relaunchedPagemodo.com (Freemium Facebook page creation tool)
    We run an average of 3 different A/B tests at any given time
  • 4. Freemium – A Success Story
    As a disruptive business model, freemium has been the driving force behind many successful startups and $billions in value creation
  • 5. Simple Fundamentals
    Freemium is a business model that works well in scenarios where the marginal cost of users can be driven towards 0 over time or through economies of scale
    $ Per Active User
    Variable Cost
    Time (or # users)
  • 6. What Is a Freemium Product/Service?
    Freemium is NOT…
    Free trial or sample – user forced to pay after period of time
    Shareware – user forced to pay to realize actual benefit of product
  • 7. What Is a Freemium Product/Service?
    Freemium IS…
    A free, usable product/service
    That attracts users through its price tag and free user base
    And monetizes with optional paidpremium features(typically subscription based)
  • 8. Freemium is merely one option:
    Popular web business models:
    Sell a product to customers (premium)
    Upsell a product to your users (freemium)
    Sell other people’s products to your users (ads)
    Sell your users’ data (ex: addthis)
  • 9. Why we “chose” Freemium 10 yrs ago
    Broke - No marketing budget
    Clueless - Uncertainty about subscription/pricing model
    Frugal- Cost per user was low
  • 10. Challenges for Freemium Players
    Low Conversion - People who may have been willing to pay normally do not pay because the free product is sufficient
    Low ARPU – makes it difficult to accelerate growth via traditional marketing methods
    Infrastructure– millions of free users can create non-trivial and costly infrastructure problems
    Customer Support – free users often still demand support
  • 11. Benefits of the Freemium Model
    Viral – Rapid adoption, opens new, affordable, channels for getting users
    Disruptive – Compete with even the most entrenched premium competitors by giving it away for free
    Superior UI/UX – Offer a better product through rapid testing / customer development
  • 12. The Freemium Offering on Webs
  • 13. Leveraging our Free Users
    Direct Marketing from Free Users
    Links on bottom of sites
    Branding - Logo on bottom of sites
    SEO value from websites back to Webs.com
    URL has webs.com in it
    Indirect Marketing from Free Users
    Word of mouth
    Size / Credibility
  • 14. How Premium Businesses Can Leverage Freemium
    Offer a limited version for free (but be careful NOT to cannibalize)
    Create new, related free or freemium product, and use as marketing funnel for your paid product
    Partner with a freemium product, offer their premium product to your audience
  • 15. 15
    Part II: Testing… Testing… 1 2 3
    HostingCon 2011
  • 16. The Freemium Conversion Funnel
    “What gets measured gets improved”
  • 17. Conversion Timeline
    Understanding When Your Users Convert Is Key
    Our likelihood of conversion decreases over time
    Why? Users are publishing a new website and want all features at time of launch
    Implications: Focus on upselling EARLY in customer lifecycle
    Other models:
    User more likely to convert as time goes on:
    Dropbox (space fills up)
    Mailchimp (email list grows)
    Implications: Deemphasize upsell up front, and focus on getting as much usage as possible
  • 18. Simple A/B Testing
    Define some goal event
    Create test treatments
    Test - randomly split visitors into test & control groups; measure success to goal
    Good for conversion events with immediacy
    Ex: Homepage, signup page, shopping cart
  • 19. Simple A/B Testing
    What to test?
    Calls-to-action (buttons)
    Copy, Color, Size, Placement, Hover states
    Descriptors, Size, Length/number, etc.
    Headlines, Value Props, Links, Amount (long/short)
    Segment results by source, use landing pages
    * Bonus Points
  • 20. AbTests.com
    Great site for testing inspiration
  • 21. Case Study
    Shortened Signup Flow
    Tested shorter 1-page flow against 2-page flow with more fields
    Shortened version created 10,000 more signups over 3 weeks
    Going 100% translates to over 25,000 more signups/month!
  • 22. A/B Test Reporting (GWO)
    Confidence increases over time
    Low variation is good
    Google Website Optimizer is good because:
    Provides visitors a consistent experience
    Measures & reports confidence / significance
    Offers multivariate (combination) testing
  • 23. Interpreting Results
    When A/B Testing is Not Enough
    Freemium Problem: Signup != $$$
    A/B signup test only told us which produced more USERS, but does NOT tell us which produced more $$
    Example: Asking more questions in signup will reduce signups but may increase user success rate and make more $$
  • 24. Database Driven Cohort Analysis
    A system that permanently fixes a user to a test group, allowing one to run results analysis on any variable that is also attached to that user (i.e. $ spent)
    Offers the ability to compare users who received different treatments through your entire funnel over time
  • 25. Cohort Funnel Analysis Example
    A/B testing would have shown Test2 as the winner
    Test 1 makes more money. Tradeoff?
    * Dummy data
  • 26. How to Set Up Cohort Testing
    Thanks Eric Ries!
    In Cohort?
    User Enters
    Assign to Cohort
    Coin Toss
    Quick & Dirty Shortcut: Mod on userid(ex: show even users one treatment and odd users another)
  • 27. Visualizing Results
    Day to day reporting on metrics like Premium $ can often be lumpy.
    Looking at a cumulative view helps visualize more smoothly
    Diverging lines means one is superior
    Parallel lines apart means one got lucky;)
  • 28. Segmentation & Cohort Analysis
    Integrated front- and back-end testing
    Complete ownership of user data
    Ability to track impact to user long after user-facing experience has gone away
    Provide users with consistent test experience that integrates your back-end - Great for pricing & promotional tests
    Ability to “slice” data by internal user variables
  • 29. Advanced Segmentation
    Total/Aggregate results can hide the real story
    Not all users are the same
    Segmenting data and “slicing” can produce surprising gains
    Treatments can be applied to different segments accordingly
  • 30. Visualizing Advanced Segmentation
    Cumulative Charts
    Different reports for each segment
    Can use end results to find interesting variable combinations, then check for significance
  • 31. Why A/B Testing is Crucial
    Lots of Leverage – since your free customers are often your largest referring source, improving your funnel can have compounding effects on both virality and conversion.
    Users constantly surprise – its impossible to guess what users will do. That’s why all great companies use testing.
    Validate ideas and time/energy spent on product efforts
    Better Serve Your Users – build a better product and improve your intuition about your customers
  • 32. Summary
    The freemium model is here to stay
    When implemented properly, freemium is a disruptive business model allowing rapid growth in new or existing markets
    Improving conversion is crucial to success, and requires rapid iteration and A/B testing
    Rapid iteration and testing cannot be done without large influx of free users, favoring those who have achieved scale
    Successful freemium products are always-improving, and tend to have the superior metrics & UX/UI (out of sheer necessity!)
    Premium players who do not want to disrupt legacy business can can partner with freemium players to leverage their assets
  • 33. 33
    Haroon Mokhtarzada, CEO & Co-Founder of Webs
    @haroon, haroon@webs.com