• Like
The Freemium Model and A/B Testing
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

The Freemium Model and A/B Testing

  • 1,919 views
Published

Learn about the Freemium Model and A/B Testing to drive revenue. Presentation from HostingCon 2011 given by Haroon Mokhtarzada, CEO & Co-founder to Webs.com

Learn about the Freemium Model and A/B Testing to drive revenue. Presentation from HostingCon 2011 given by Haroon Mokhtarzada, CEO & Co-founder to Webs.com

Published in Business , Technology , Design
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,919
On SlideShare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
67
Comments
0
Likes
6

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • Template change hopefully makes this more attractive
  • Template change hopefully makes this more attractive

Transcript

  • 1. The Freemium Model & A/B Testing
    HostingCon 2011
    By Haroon Mokhtarzada
    CEO & Co-Founder, Webs.com
  • 2. Introduction
    2
    Haroon Mokhtarzada
    Co-Founder and CEO
    Entrepreneur from the age of 10. Co-founded Webs in 2001 with brothers Zeki and Idris while studying Economics at the University of Maryland. Obtained J.D. from Harvard Law School in 2005 and thereafter joined Webs full-time.
    Founding Vision
    “Build a free website creation tool so easy that Mom can use it”
    Funding
    Raised $12.1 million from Novak Biddle, Columbia Capital and prominent angels in August 2006.
  • 3. About Webs
    3
    Webs is a leading provider of online “front door” solutions for micro businesses across web, mobile and social platforms.
    40 million websites created since inception
    Started as a tool for anyone to easily create a website for free and evolved to focus on online solutions for micro businesses
    Recently acquired and relaunchedPagemodo.com (Freemium Facebook page creation tool)
    We run an average of 3 different A/B tests at any given time
  • 4. Freemium – A Success Story
    4
    As a disruptive business model, freemium has been the driving force behind many successful startups and $billions in value creation
  • 5. Simple Fundamentals
    5
    Freemium is a business model that works well in scenarios where the marginal cost of users can be driven towards 0 over time or through economies of scale
    ARPU
    $ Per Active User
    Variable Cost
    Time (or # users)
  • 6. What Is a Freemium Product/Service?
    Freemium is NOT…
    Free trial or sample – user forced to pay after period of time
    Shareware – user forced to pay to realize actual benefit of product
    6
  • 7. What Is a Freemium Product/Service?
    Freemium IS…
    A free, usable product/service
    That attracts users through its price tag and free user base
    And monetizes with optional paidpremium features(typically subscription based)
    7
  • 8. Freemium is merely one option:
    Popular web business models:
    Sell a product to customers (premium)
    Upsell a product to your users (freemium)
    Sell other people’s products to your users (ads)
    Sell your users’ data (ex: addthis)
    8
    Thanks!
  • 9. Why we “chose” Freemium 10 yrs ago
    Broke - No marketing budget
    Clueless - Uncertainty about subscription/pricing model
    Frugal- Cost per user was low
    9
  • 10. Challenges for Freemium Players
    Low Conversion - People who may have been willing to pay normally do not pay because the free product is sufficient
    Low ARPU – makes it difficult to accelerate growth via traditional marketing methods
    Infrastructure– millions of free users can create non-trivial and costly infrastructure problems
    Customer Support – free users often still demand support
    10
  • 11. Benefits of the Freemium Model
    Viral – Rapid adoption, opens new, affordable, channels for getting users
    Disruptive – Compete with even the most entrenched premium competitors by giving it away for free
    Superior UI/UX – Offer a better product through rapid testing / customer development
    11
  • 12. The Freemium Offering on Webs
    12
  • 13. Leveraging our Free Users
    13
    Direct Marketing from Free Users
    Links on bottom of sites
    Branding - Logo on bottom of sites
    SEO value from websites back to Webs.com
    URL has webs.com in it
    Indirect Marketing from Free Users
    Word of mouth
    Size / Credibility
  • 14. How Premium Businesses Can Leverage Freemium
    14
    Offer a limited version for free (but be careful NOT to cannibalize)
    Create new, related free or freemium product, and use as marketing funnel for your paid product
    Partner with a freemium product, offer their premium product to your audience
  • 15. 15
    Part II: Testing… Testing… 1 2 3
    HostingCon 2011
  • 16. The Freemium Conversion Funnel
    16
    “What gets measured gets improved”
  • 17. Conversion Timeline
    17
    Understanding When Your Users Convert Is Key
    Our likelihood of conversion decreases over time
    Why? Users are publishing a new website and want all features at time of launch
    Implications: Focus on upselling EARLY in customer lifecycle
    Other models:
    User more likely to convert as time goes on:
    Dropbox (space fills up)
    Mailchimp (email list grows)
    Implications: Deemphasize upsell up front, and focus on getting as much usage as possible
  • 18. Simple A/B Testing
    Define some goal event
    Create test treatments
    Test - randomly split visitors into test & control groups; measure success to goal
    Good for conversion events with immediacy
    Ex: Homepage, signup page, shopping cart
    18
  • 19. Simple A/B Testing
    What to test?
    Calls-to-action (buttons)
    Copy, Color, Size, Placement, Hover states
    Forms/Fields
    Descriptors, Size, Length/number, etc.
    Text
    Headlines, Value Props, Links, Amount (long/short)
    Imagery
    Segment results by source, use landing pages
    * Bonus Points
    19
  • 20. AbTests.com
    20
    Great site for testing inspiration
  • 21. Case Study
    21
    Control
    Test
    Shortened Signup Flow
    Tested shorter 1-page flow against 2-page flow with more fields
    Shortened version created 10,000 more signups over 3 weeks
    Going 100% translates to over 25,000 more signups/month!
  • 22. A/B Test Reporting (GWO)
    Confidence increases over time
    Low variation is good
    Google Website Optimizer is good because:
    Provides visitors a consistent experience
    Measures & reports confidence / significance
    Offers multivariate (combination) testing
    22
  • 23. Interpreting Results
    When A/B Testing is Not Enough
    Freemium Problem: Signup != $$$
    A/B signup test only told us which produced more USERS, but does NOT tell us which produced more $$
    Example: Asking more questions in signup will reduce signups but may increase user success rate and make more $$
    23
  • 24. Database Driven Cohort Analysis
    Solution:
    A system that permanently fixes a user to a test group, allowing one to run results analysis on any variable that is also attached to that user (i.e. $ spent)
    Offers the ability to compare users who received different treatments through your entire funnel over time
    24
  • 25. Cohort Funnel Analysis Example
    A/B testing would have shown Test2 as the winner
    Test 1 makes more money. Tradeoff?
    25
    * Dummy data
  • 26. How to Set Up Cohort Testing
    Thanks Eric Ries!
    Display
    Treatment
    Yes
    In Cohort?
    User Enters
    Assign to Cohort
    Coin Toss
    No
    Quick & Dirty Shortcut: Mod on userid(ex: show even users one treatment and odd users another)
    26
  • 27. Visualizing Results
    27
    Day to day reporting on metrics like Premium $ can often be lumpy.
    Looking at a cumulative view helps visualize more smoothly
    Diverging lines means one is superior
    Parallel lines apart means one got lucky;)
  • 28. Segmentation & Cohort Analysis
    Advantages
    Integrated front- and back-end testing
    Complete ownership of user data
    Ability to track impact to user long after user-facing experience has gone away
    Provide users with consistent test experience that integrates your back-end - Great for pricing & promotional tests
    Ability to “slice” data by internal user variables
    28
  • 29. Advanced Segmentation
    29
    Total/Aggregate results can hide the real story
    Not all users are the same
    Segmenting data and “slicing” can produce surprising gains
    Treatments can be applied to different segments accordingly
  • 30. Visualizing Advanced Segmentation
    30
    Cumulative Charts
    Different reports for each segment
    Can use end results to find interesting variable combinations, then check for significance
  • 31. Why A/B Testing is Crucial
    31
    Lots of Leverage – since your free customers are often your largest referring source, improving your funnel can have compounding effects on both virality and conversion.
    Users constantly surprise – its impossible to guess what users will do. That’s why all great companies use testing.
    Validate ideas and time/energy spent on product efforts
    Better Serve Your Users – build a better product and improve your intuition about your customers
  • 32. Summary
    32
    The freemium model is here to stay
    When implemented properly, freemium is a disruptive business model allowing rapid growth in new or existing markets
    Improving conversion is crucial to success, and requires rapid iteration and A/B testing
    Rapid iteration and testing cannot be done without large influx of free users, favoring those who have achieved scale
    Successful freemium products are always-improving, and tend to have the superior metrics & UX/UI (out of sheer necessity!)
    Premium players who do not want to disrupt legacy business can can partner with freemium players to leverage their assets
  • 33. 33
    Q&A
    Haroon Mokhtarzada, CEO & Co-Founder of Webs
    @haroon, haroon@webs.com