16/05/2014 by RusseWeb
Learn A/B testing
Definition of A/B testing and Case Studies by Optimizely
01
Conversion Rate
✤ Conversion Rate - the
percentage of visitors who take a
desired action.!
✤ The average conversion rat...
What is A/B testing ?
✤ A/B testing is the simple idea of showing several
different versions of a web page to live trafic.!...
The Main Goal is to Improve your Conversion Rate
What to test first ?
Optimization guide in 5 steps
The hardest part of A/B testing is
determine what to test first
✤ Step One : Define success or the reasons of your site’s
e...
Define success of your site
✤ What is your website for ? !
✤ Identify what constitutes - and does not constitues - a
« con...
Identify Bottlenecks
✤ Bottlenecks are the places where your users are
dropping off
Construct a Hypothesis
✤ After you found the bottlenecks of your site, try to
understand why your potential costumers leav...
Prioritize
✤ You should keep a sense of your testing priorities in
mind!
✤ Your projected ROI (Return On Investment) from ...
Test !
✤ Randomly selected users!
✤ Different site versions!
✤ Before finish the test check if it has reached its
statistic...
Local and Global Maximum
Definitions
✤ The Local Maximum - the nearest target or your
optimization!
✤ The Global Maximum - the main goal of your
op...
Break from the Status Quo :ABC Family
✤ Experiment of Disney on their ABC Family homepage
using Optimizely platform :
The main page displayed a large promotion for a television show you might be interested in. After some
researches Disney t...
By being open to this big, fundamental change
Disney was able to effect an engagement increase of
more than 600% !
✤ Use the insights from small tests to guide and inform
your thinking about bigger changes.!
✤ Consider entirely new alter...
Case Study
5 Quick MultipageTestsTo Maximize
Landing Page Conversions
Multipage strategy
✤ Site that uses templates to design pages instead of
designing each page. !
✤ Testing one element acro...
Travel company VacationRoost has several travel-
lodging sites that propose searching for rental
properties, destinations,...
01
VacationRoost and Results
after A/B testing
✤ Security seals and logos boost user engagement!
✤ Large and orange call-to-a...
Testing methods ofVacationRoost
✤ Test and iterate: first large-scale design changes then
more granular, subtle revisions l...
Optimizely
A/B testing you’ll actually use
01
About
Optimizely:
✤ Optimizely is a world-class
website optimization platform
that enables users with and
without techn...
Optimizely By the Numbers
(As of Oct. 2013)
2.2 Billion visitors tested across all customer websites
300,000+ experiments ...
References
✤ http://www.marketingterms.com/dictionary/
conversion_rate/!
✤ http://blog.optimizely.com/!
✤ https://s3.amazo...
RusseWeb
Blog sur l’internet russe et l’actualité du web russe. Les particularités du référencement sur le moteur de
reche...
Upcoming SlideShare
Loading in …5
×

Definition of A/B testing and Case Studies by Optimizely

1,401 views

Published on

A/B testing is the simple idea of showing several different versions of a web page to live trafic.

Published in: Marketing, Technology, Design
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,401
On SlideShare
0
From Embeds
0
Number of Embeds
458
Actions
Shares
0
Downloads
13
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Definition of A/B testing and Case Studies by Optimizely

  1. 1. 16/05/2014 by RusseWeb Learn A/B testing Definition of A/B testing and Case Studies by Optimizely
  2. 2. 01 Conversion Rate ✤ Conversion Rate - the percentage of visitors who take a desired action.! ✤ The average conversion rate on the web - the rate at which visitors convert into customers - is at only 2%. ! ✤ That means that the 98% of potential customers that reste never will convert on YOUR web site.
  3. 3. What is A/B testing ? ✤ A/B testing is the simple idea of showing several different versions of a web page to live trafic.! ✤ You can then mesure the effect the each version has on your customers! ✤ By doing so you can drastically improve the effectiveness of your marketing and user experience
  4. 4. The Main Goal is to Improve your Conversion Rate
  5. 5. What to test first ? Optimization guide in 5 steps
  6. 6. The hardest part of A/B testing is determine what to test first ✤ Step One : Define success or the reasons of your site’s existence! ✤ Step Two : Identify bottlenecks or where do you lose your customers ?! ✤ Step Three : Construct a hypothesis! ✤ Step Four : Prioritize! ✤ Step Five : Test !
  7. 7. Define success of your site ✤ What is your website for ? ! ✤ Identify what constitutes - and does not constitues - a « conversion » for your web site. ! ✤ Depending on your business model defining a success conversion metrics can be a tricky task.
  8. 8. Identify Bottlenecks ✤ Bottlenecks are the places where your users are dropping off
  9. 9. Construct a Hypothesis ✤ After you found the bottlenecks of your site, try to understand why your potential costumers leave at this step ? ! ✤ Make a hypothesis, try to think as your clients! ✤ To understand your customers vision you can use interviews, feedback forms, groupes.! ✤ If you are running tests without a hypothesis beforehand, you might gather information that’s helpful anecdotally while missing the deeper lesson.
  10. 10. Prioritize ✤ You should keep a sense of your testing priorities in mind! ✤ Your projected ROI (Return On Investment) from each test will itself be derived from a combination of your core success metrics that you determine in Step One, the Bottlenecks (Step two) and your hypothesis about user’s behavior (Step Three).
  11. 11. Test ! ✤ Randomly selected users! ✤ Different site versions! ✤ Before finish the test check if it has reached its statistical significance! ✤ Remember : often a completed test brings not only answers but more questions.
  12. 12. Local and Global Maximum
  13. 13. Definitions ✤ The Local Maximum - the nearest target or your optimization! ✤ The Global Maximum - the main goal of your optimization campaign! Think BIG !
  14. 14. Break from the Status Quo :ABC Family ✤ Experiment of Disney on their ABC Family homepage using Optimizely platform :
  15. 15. The main page displayed a large promotion for a television show you might be interested in. After some researches Disney team discovered that a lot of people were searching for the exact title of shows and specific episodes. ! The team decided to reevaluate their entire approach : alternative view, less visual and more hierarchical. Users can search from top to bottom for a specific show.!
  16. 16. By being open to this big, fundamental change Disney was able to effect an engagement increase of more than 600% !
  17. 17. ✤ Use the insights from small tests to guide and inform your thinking about bigger changes.! ✤ Consider entirely new alternative approach to your principal business goals. ! ✤ if you’re working on a major site redesign or overhaul, don’t wait until the new design is live to A/B test it. A/B test the design itself. Test each feature step by step.
  18. 18. Case Study
  19. 19. 5 Quick MultipageTestsTo Maximize Landing Page Conversions
  20. 20. Multipage strategy ✤ Site that uses templates to design pages instead of designing each page. ! ✤ Testing one element across all pages of one type : product pages, content pages, funnel pages.! ✤ Build a change one time and you can use it for several pages
  21. 21. Travel company VacationRoost has several travel- lodging sites that propose searching for rental properties, destinations, or deals. ✤ Visitors enter on sites via targeted landing page based on researches they made earlier! ✤ Tested changes on the site : Security seals and logos, call-to-action buttons and copy, content length and format, number of calls-to-action displayed, search filter options! ✤ Two primary goals : CTR to search results page and bounce rate
  22. 22. 01
  23. 23. VacationRoost and Results after A/B testing ✤ Security seals and logos boost user engagement! ✤ Large and orange call-to-action button provides the biggest conversion! ✤ Longer pages have bigger conversion (more content and personalization)! ✤ More Calls-to-action increase the number of users who entered the funnel! ✤ Adding mini-filters or search options increase engagement
  24. 24. Testing methods ofVacationRoost ✤ Test and iterate: first large-scale design changes then more granular, subtle revisions like CTA copy or security seal placement. ! ✤ For the fast results, test to highly trafficked pages.! ✤ Test across multiple pages at once.
  25. 25. Optimizely A/B testing you’ll actually use
  26. 26. 01 About Optimizely: ✤ Optimizely is a world-class website optimization platform that enables users with and without technical expertise to make dynamic changes to their websites, test the variations to live traffic, gather immediate results and start achieving goals. ! ✤ Optimizely helps more than 6,000 clients drive more leads, sales and conversions through simple website testing and optimization
  27. 27. Optimizely By the Numbers (As of Oct. 2013) 2.2 Billion visitors tested across all customer websites 300,000+ experiments run since launch 6,000+ customers
  28. 28. References ✤ http://www.marketingterms.com/dictionary/ conversion_rate/! ✤ http://blog.optimizely.com/! ✤ https://s3.amazonaws.com/optimizely-marketing/ customer-stories/vacationroost-casestudy.pdf! ✤ Book « A/B testing. The most powerful way to turn clicks into customers » by Dan Siroker and Pete Koomen
  29. 29. RusseWeb Blog sur l’internet russe et l’actualité du web russe. Les particularités du référencement sur le moteur de recherche russe Yandex http://russeweb.fr

×