The Freemium Model and A/B Testing
Upcoming SlideShare
Loading in...5

The Freemium Model and A/B Testing



Learn about the Freemium Model and A/B Testing to drive revenue. Presentation from HostingCon 2011 given by Haroon Mokhtarzada, CEO & Co-founder to

Learn about the Freemium Model and A/B Testing to drive revenue. Presentation from HostingCon 2011 given by Haroon Mokhtarzada, CEO & Co-founder to



Total Views
Views on SlideShare
Embed Views



1 Embed 2 2


Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment
  • Template change hopefully makes this more attractive
  • Template change hopefully makes this more attractive

The Freemium Model and A/B Testing The Freemium Model and A/B Testing Presentation Transcript

  • The Freemium Model & A/B Testing
    HostingCon 2011
    By Haroon Mokhtarzada
    CEO & Co-Founder,
  • Introduction
    Haroon Mokhtarzada
    Co-Founder and CEO
    Entrepreneur from the age of 10. Co-founded Webs in 2001 with brothers Zeki and Idris while studying Economics at the University of Maryland. Obtained J.D. from Harvard Law School in 2005 and thereafter joined Webs full-time.
    Founding Vision
    “Build a free website creation tool so easy that Mom can use it”
    Raised $12.1 million from Novak Biddle, Columbia Capital and prominent angels in August 2006.
  • About Webs
    Webs is a leading provider of online “front door” solutions for micro businesses across web, mobile and social platforms.
    40 million websites created since inception
    Started as a tool for anyone to easily create a website for free and evolved to focus on online solutions for micro businesses
    Recently acquired and (Freemium Facebook page creation tool)
    We run an average of 3 different A/B tests at any given time
  • Freemium – A Success Story
    As a disruptive business model, freemium has been the driving force behind many successful startups and $billions in value creation
  • Simple Fundamentals
    Freemium is a business model that works well in scenarios where the marginal cost of users can be driven towards 0 over time or through economies of scale
    $ Per Active User
    Variable Cost
    Time (or # users)
  • What Is a Freemium Product/Service?
    Freemium is NOT…
    Free trial or sample – user forced to pay after period of time
    Shareware – user forced to pay to realize actual benefit of product
  • What Is a Freemium Product/Service?
    Freemium IS…
    A free, usable product/service
    That attracts users through its price tag and free user base
    And monetizes with optional paidpremium features(typically subscription based)
  • Freemium is merely one option:
    Popular web business models:
    Sell a product to customers (premium)
    Upsell a product to your users (freemium)
    Sell other people’s products to your users (ads)
    Sell your users’ data (ex: addthis)
  • Why we “chose” Freemium 10 yrs ago
    Broke - No marketing budget
    Clueless - Uncertainty about subscription/pricing model
    Frugal- Cost per user was low
  • Challenges for Freemium Players
    Low Conversion - People who may have been willing to pay normally do not pay because the free product is sufficient
    Low ARPU – makes it difficult to accelerate growth via traditional marketing methods
    Infrastructure– millions of free users can create non-trivial and costly infrastructure problems
    Customer Support – free users often still demand support
  • Benefits of the Freemium Model
    Viral – Rapid adoption, opens new, affordable, channels for getting users
    Disruptive – Compete with even the most entrenched premium competitors by giving it away for free
    Superior UI/UX – Offer a better product through rapid testing / customer development
  • The Freemium Offering on Webs
  • Leveraging our Free Users
    Direct Marketing from Free Users
    Links on bottom of sites
    Branding - Logo on bottom of sites
    SEO value from websites back to
    URL has in it
    Indirect Marketing from Free Users
    Word of mouth
    Size / Credibility
  • How Premium Businesses Can Leverage Freemium
    Offer a limited version for free (but be careful NOT to cannibalize)
    Create new, related free or freemium product, and use as marketing funnel for your paid product
    Partner with a freemium product, offer their premium product to your audience
  • 15
    Part II: Testing… Testing… 1 2 3
    HostingCon 2011
  • The Freemium Conversion Funnel
    “What gets measured gets improved”
  • Conversion Timeline
    Understanding When Your Users Convert Is Key
    Our likelihood of conversion decreases over time
    Why? Users are publishing a new website and want all features at time of launch
    Implications: Focus on upselling EARLY in customer lifecycle
    Other models:
    User more likely to convert as time goes on:
    Dropbox (space fills up)
    Mailchimp (email list grows)
    Implications: Deemphasize upsell up front, and focus on getting as much usage as possible
  • Simple A/B Testing
    Define some goal event
    Create test treatments
    Test - randomly split visitors into test & control groups; measure success to goal
    Good for conversion events with immediacy
    Ex: Homepage, signup page, shopping cart
  • Simple A/B Testing
    What to test?
    Calls-to-action (buttons)
    Copy, Color, Size, Placement, Hover states
    Descriptors, Size, Length/number, etc.
    Headlines, Value Props, Links, Amount (long/short)
    Segment results by source, use landing pages
    * Bonus Points
    Great site for testing inspiration
  • Case Study
    Shortened Signup Flow
    Tested shorter 1-page flow against 2-page flow with more fields
    Shortened version created 10,000 more signups over 3 weeks
    Going 100% translates to over 25,000 more signups/month!
  • A/B Test Reporting (GWO)
    Confidence increases over time
    Low variation is good
    Google Website Optimizer is good because:
    Provides visitors a consistent experience
    Measures & reports confidence / significance
    Offers multivariate (combination) testing
  • Interpreting Results
    When A/B Testing is Not Enough
    Freemium Problem: Signup != $$$
    A/B signup test only told us which produced more USERS, but does NOT tell us which produced more $$
    Example: Asking more questions in signup will reduce signups but may increase user success rate and make more $$
  • Database Driven Cohort Analysis
    A system that permanently fixes a user to a test group, allowing one to run results analysis on any variable that is also attached to that user (i.e. $ spent)
    Offers the ability to compare users who received different treatments through your entire funnel over time
  • Cohort Funnel Analysis Example
    A/B testing would have shown Test2 as the winner
    Test 1 makes more money. Tradeoff?
    * Dummy data
  • How to Set Up Cohort Testing
    Thanks Eric Ries!
    In Cohort?
    User Enters
    Assign to Cohort
    Coin Toss
    Quick & Dirty Shortcut: Mod on userid(ex: show even users one treatment and odd users another)
  • Visualizing Results
    Day to day reporting on metrics like Premium $ can often be lumpy.
    Looking at a cumulative view helps visualize more smoothly
    Diverging lines means one is superior
    Parallel lines apart means one got lucky;)
  • Segmentation & Cohort Analysis
    Integrated front- and back-end testing
    Complete ownership of user data
    Ability to track impact to user long after user-facing experience has gone away
    Provide users with consistent test experience that integrates your back-end - Great for pricing & promotional tests
    Ability to “slice” data by internal user variables
  • Advanced Segmentation
    Total/Aggregate results can hide the real story
    Not all users are the same
    Segmenting data and “slicing” can produce surprising gains
    Treatments can be applied to different segments accordingly
  • Visualizing Advanced Segmentation
    Cumulative Charts
    Different reports for each segment
    Can use end results to find interesting variable combinations, then check for significance
  • Why A/B Testing is Crucial
    Lots of Leverage – since your free customers are often your largest referring source, improving your funnel can have compounding effects on both virality and conversion.
    Users constantly surprise – its impossible to guess what users will do. That’s why all great companies use testing.
    Validate ideas and time/energy spent on product efforts
    Better Serve Your Users – build a better product and improve your intuition about your customers
  • Summary
    The freemium model is here to stay
    When implemented properly, freemium is a disruptive business model allowing rapid growth in new or existing markets
    Improving conversion is crucial to success, and requires rapid iteration and A/B testing
    Rapid iteration and testing cannot be done without large influx of free users, favoring those who have achieved scale
    Successful freemium products are always-improving, and tend to have the superior metrics & UX/UI (out of sheer necessity!)
    Premium players who do not want to disrupt legacy business can can partner with freemium players to leverage their assets
  • 33
    Haroon Mokhtarzada, CEO & Co-Founder of Webs