Your SlideShare is downloading. ×
The Freemium Model and A/B Testing
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

The Freemium Model and A/B Testing


Published on

Learn about the Freemium Model and A/B Testing to drive revenue. Presentation from HostingCon 2011 given by Haroon Mokhtarzada, CEO & Co-founder to

Learn about the Freemium Model and A/B Testing to drive revenue. Presentation from HostingCon 2011 given by Haroon Mokhtarzada, CEO & Co-founder to

Published in: Business, Technology, Design

  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide
  • Template change hopefully makes this more attractive
  • Template change hopefully makes this more attractive
  • Transcript

    • 1. The Freemium Model & A/B Testing
      HostingCon 2011
      By Haroon Mokhtarzada
      CEO & Co-Founder,
    • 2. Introduction
      Haroon Mokhtarzada
      Co-Founder and CEO
      Entrepreneur from the age of 10. Co-founded Webs in 2001 with brothers Zeki and Idris while studying Economics at the University of Maryland. Obtained J.D. from Harvard Law School in 2005 and thereafter joined Webs full-time.
      Founding Vision
      “Build a free website creation tool so easy that Mom can use it”
      Raised $12.1 million from Novak Biddle, Columbia Capital and prominent angels in August 2006.
    • 3. About Webs
      Webs is a leading provider of online “front door” solutions for micro businesses across web, mobile and social platforms.
      40 million websites created since inception
      Started as a tool for anyone to easily create a website for free and evolved to focus on online solutions for micro businesses
      Recently acquired and (Freemium Facebook page creation tool)
      We run an average of 3 different A/B tests at any given time
    • 4. Freemium – A Success Story
      As a disruptive business model, freemium has been the driving force behind many successful startups and $billions in value creation
    • 5. Simple Fundamentals
      Freemium is a business model that works well in scenarios where the marginal cost of users can be driven towards 0 over time or through economies of scale
      $ Per Active User
      Variable Cost
      Time (or # users)
    • 6. What Is a Freemium Product/Service?
      Freemium is NOT…
      Free trial or sample – user forced to pay after period of time
      Shareware – user forced to pay to realize actual benefit of product
    • 7. What Is a Freemium Product/Service?
      Freemium IS…
      A free, usable product/service
      That attracts users through its price tag and free user base
      And monetizes with optional paidpremium features(typically subscription based)
    • 8. Freemium is merely one option:
      Popular web business models:
      Sell a product to customers (premium)
      Upsell a product to your users (freemium)
      Sell other people’s products to your users (ads)
      Sell your users’ data (ex: addthis)
    • 9. Why we “chose” Freemium 10 yrs ago
      Broke - No marketing budget
      Clueless - Uncertainty about subscription/pricing model
      Frugal- Cost per user was low
    • 10. Challenges for Freemium Players
      Low Conversion - People who may have been willing to pay normally do not pay because the free product is sufficient
      Low ARPU – makes it difficult to accelerate growth via traditional marketing methods
      Infrastructure– millions of free users can create non-trivial and costly infrastructure problems
      Customer Support – free users often still demand support
    • 11. Benefits of the Freemium Model
      Viral – Rapid adoption, opens new, affordable, channels for getting users
      Disruptive – Compete with even the most entrenched premium competitors by giving it away for free
      Superior UI/UX – Offer a better product through rapid testing / customer development
    • 12. The Freemium Offering on Webs
    • 13. Leveraging our Free Users
      Direct Marketing from Free Users
      Links on bottom of sites
      Branding - Logo on bottom of sites
      SEO value from websites back to
      URL has in it
      Indirect Marketing from Free Users
      Word of mouth
      Size / Credibility
    • 14. How Premium Businesses Can Leverage Freemium
      Offer a limited version for free (but be careful NOT to cannibalize)
      Create new, related free or freemium product, and use as marketing funnel for your paid product
      Partner with a freemium product, offer their premium product to your audience
    • 15. 15
      Part II: Testing… Testing… 1 2 3
      HostingCon 2011
    • 16. The Freemium Conversion Funnel
      “What gets measured gets improved”
    • 17. Conversion Timeline
      Understanding When Your Users Convert Is Key
      Our likelihood of conversion decreases over time
      Why? Users are publishing a new website and want all features at time of launch
      Implications: Focus on upselling EARLY in customer lifecycle
      Other models:
      User more likely to convert as time goes on:
      Dropbox (space fills up)
      Mailchimp (email list grows)
      Implications: Deemphasize upsell up front, and focus on getting as much usage as possible
    • 18. Simple A/B Testing
      Define some goal event
      Create test treatments
      Test - randomly split visitors into test & control groups; measure success to goal
      Good for conversion events with immediacy
      Ex: Homepage, signup page, shopping cart
    • 19. Simple A/B Testing
      What to test?
      Calls-to-action (buttons)
      Copy, Color, Size, Placement, Hover states
      Descriptors, Size, Length/number, etc.
      Headlines, Value Props, Links, Amount (long/short)
      Segment results by source, use landing pages
      * Bonus Points
    • 20.
      Great site for testing inspiration
    • 21. Case Study
      Shortened Signup Flow
      Tested shorter 1-page flow against 2-page flow with more fields
      Shortened version created 10,000 more signups over 3 weeks
      Going 100% translates to over 25,000 more signups/month!
    • 22. A/B Test Reporting (GWO)
      Confidence increases over time
      Low variation is good
      Google Website Optimizer is good because:
      Provides visitors a consistent experience
      Measures & reports confidence / significance
      Offers multivariate (combination) testing
    • 23. Interpreting Results
      When A/B Testing is Not Enough
      Freemium Problem: Signup != $$$
      A/B signup test only told us which produced more USERS, but does NOT tell us which produced more $$
      Example: Asking more questions in signup will reduce signups but may increase user success rate and make more $$
    • 24. Database Driven Cohort Analysis
      A system that permanently fixes a user to a test group, allowing one to run results analysis on any variable that is also attached to that user (i.e. $ spent)
      Offers the ability to compare users who received different treatments through your entire funnel over time
    • 25. Cohort Funnel Analysis Example
      A/B testing would have shown Test2 as the winner
      Test 1 makes more money. Tradeoff?
      * Dummy data
    • 26. How to Set Up Cohort Testing
      Thanks Eric Ries!
      In Cohort?
      User Enters
      Assign to Cohort
      Coin Toss
      Quick & Dirty Shortcut: Mod on userid(ex: show even users one treatment and odd users another)
    • 27. Visualizing Results
      Day to day reporting on metrics like Premium $ can often be lumpy.
      Looking at a cumulative view helps visualize more smoothly
      Diverging lines means one is superior
      Parallel lines apart means one got lucky;)
    • 28. Segmentation & Cohort Analysis
      Integrated front- and back-end testing
      Complete ownership of user data
      Ability to track impact to user long after user-facing experience has gone away
      Provide users with consistent test experience that integrates your back-end - Great for pricing & promotional tests
      Ability to “slice” data by internal user variables
    • 29. Advanced Segmentation
      Total/Aggregate results can hide the real story
      Not all users are the same
      Segmenting data and “slicing” can produce surprising gains
      Treatments can be applied to different segments accordingly
    • 30. Visualizing Advanced Segmentation
      Cumulative Charts
      Different reports for each segment
      Can use end results to find interesting variable combinations, then check for significance
    • 31. Why A/B Testing is Crucial
      Lots of Leverage – since your free customers are often your largest referring source, improving your funnel can have compounding effects on both virality and conversion.
      Users constantly surprise – its impossible to guess what users will do. That’s why all great companies use testing.
      Validate ideas and time/energy spent on product efforts
      Better Serve Your Users – build a better product and improve your intuition about your customers
    • 32. Summary
      The freemium model is here to stay
      When implemented properly, freemium is a disruptive business model allowing rapid growth in new or existing markets
      Improving conversion is crucial to success, and requires rapid iteration and A/B testing
      Rapid iteration and testing cannot be done without large influx of free users, favoring those who have achieved scale
      Successful freemium products are always-improving, and tend to have the superior metrics & UX/UI (out of sheer necessity!)
      Premium players who do not want to disrupt legacy business can can partner with freemium players to leverage their assets
    • 33. 33
      Haroon Mokhtarzada, CEO & Co-Founder of Webs