Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
A/B Testing
what’s that and how it works

Konrad Synoradzki Allegro Group 2013
What is A/B testing?
A/B testing is comparing two versions of a web
page to see which one performs better.

Konrad Synorad...
web-page visitors

Konrad Synoradzki Allegro Group 2013
slice visitors

Konrad Synoradzki Allegro Group 2013
Then compare two web pages by showing the two
variants (let's call them A and B) to similar visitors at
the same time.
Kon...
example

Konrad Synoradzki Allegro Group 2013
example

Konrad Synoradzki Allegro Group 2013
example

Konrad Synoradzki Allegro Group 2013
What can you test?
Headlines
Sub headlines
Paragraph Text
Testimonials
Call to Action text
Call to Action Button
Links
Ima...
Types of tests

Konrad Synoradzki Allegro Group 2013
A/B testing
Best used to measure the
impact of a two to four
variables on interactions with
the page.

Examples:
• new lay...
Multivariate Testing
• compares a higher
number of variables,
• reveals more information
about how these
variables interac...
Multi-Page Testing
• A/B test on multi-page
Example:
• Colors on primary
action in process
• Badges on promo items

Konrad...
Getting Started
What is and how to use optimizely?

Konrad Synoradzki Allegro Group 2013
Getting Started
Step 0: Implement the snippet

Konrad Synoradzki Allegro Group 2013
Getting Started
Step 1: Enter your URL on Optimizely.com

Konrad Synoradzki Allegro Group 2013
Getting Started
Step 2: Add page variations

Konrad Synoradzki Allegro Group 2013
Getting Started
Step 3: Set up goals for tracking

Konrad Synoradzki Allegro Group 2013
Getting Started
Step 4: Target and allocate

Konrad Synoradzki Allegro Group 2013
Report page

Konrad Synoradzki Allegro Group 2013
Short movie
https://fast.wistia.net/embed/iframe/gxttknv8aa?auto
Play=true&controlsVisibleOnLoad=true&version=v1&vo
lumeCo...
Technical details
but how??

Konrad Synoradzki Allegro Group 2013
Organizing Experiments

Konrad Synoradzki Allegro Group 2013
User Roles

Konrad Synoradzki Allegro Group 2013
Each Project Has Its Own
Project Code (Snippet)

or asynchronous version

Konrad Synoradzki Allegro Group 2013
Optimizely Snippet Contents
•
•
•
•
•

Active Experiments
Draft Experiments
Project Goals
Optimizely execution and trackin...
Two modes

Konrad Synoradzki Allegro Group 2013
Optimizely translates every change made in the visual editor
into a line of jQuery code, which we call Variation Code:

Ko...
Optimizely Testing Process

Konrad Synoradzki Allegro Group 2013
Order of Execution

Konrad Synoradzki Allegro Group 2013
Snippet Timing Logic

Konrad Synoradzki Allegro Group 2013
Optimizely Architecture Overview

Konrad Synoradzki Allegro Group 2013
Optimizely Architecture Overview
• Google App Engine is used to store and manage your account and
experiment configuration...
Collecting data
The visitor’s browser makes an asynchronous request to
https://log.optimizely.com, which logs the followin...
that’s all

Konrad Synoradzki Allegro Group 2013
question?

Konrad Synoradzki Allegro Group 2013
Konrad Synoradzki Allegro Group 2013
Upcoming SlideShare
Loading in …5
×

A/B optimizely technical details

1,611 views

Published on

Technical overview about how optimizely works.

Published in: Technology, Education
  • Login to see the comments

A/B optimizely technical details

  1. 1. A/B Testing what’s that and how it works Konrad Synoradzki Allegro Group 2013
  2. 2. What is A/B testing? A/B testing is comparing two versions of a web page to see which one performs better. Konrad Synoradzki Allegro Group 2013
  3. 3. web-page visitors Konrad Synoradzki Allegro Group 2013
  4. 4. slice visitors Konrad Synoradzki Allegro Group 2013
  5. 5. Then compare two web pages by showing the two variants (let's call them A and B) to similar visitors at the same time. Konrad Synoradzki Allegro Group 2013
  6. 6. example Konrad Synoradzki Allegro Group 2013
  7. 7. example Konrad Synoradzki Allegro Group 2013
  8. 8. example Konrad Synoradzki Allegro Group 2013
  9. 9. What can you test? Headlines Sub headlines Paragraph Text Testimonials Call to Action text Call to Action Button Links Images Content above or below the fold • Social proof • • • • • • • • • Konrad Synoradzki Allegro Group 2013 • • • • • • Media mentions Awards and badges Banners efficiency Rotation speed New version of page Removing ads
  10. 10. Types of tests Konrad Synoradzki Allegro Group 2013
  11. 11. A/B testing Best used to measure the impact of a two to four variables on interactions with the page. Examples: • new layout • Labels on button Konrad Synoradzki Allegro Group 2013
  12. 12. Multivariate Testing • compares a higher number of variables, • reveals more information about how these variables interact with one another. • traffic to a page is split between different versions of the design. Examples • Test headline, button color, button text and image on landing page Konrad Synoradzki Allegro Group 2013
  13. 13. Multi-Page Testing • A/B test on multi-page Example: • Colors on primary action in process • Badges on promo items Konrad Synoradzki Allegro Group 2013
  14. 14. Getting Started What is and how to use optimizely? Konrad Synoradzki Allegro Group 2013
  15. 15. Getting Started Step 0: Implement the snippet Konrad Synoradzki Allegro Group 2013
  16. 16. Getting Started Step 1: Enter your URL on Optimizely.com Konrad Synoradzki Allegro Group 2013
  17. 17. Getting Started Step 2: Add page variations Konrad Synoradzki Allegro Group 2013
  18. 18. Getting Started Step 3: Set up goals for tracking Konrad Synoradzki Allegro Group 2013
  19. 19. Getting Started Step 4: Target and allocate Konrad Synoradzki Allegro Group 2013
  20. 20. Report page Konrad Synoradzki Allegro Group 2013
  21. 21. Short movie https://fast.wistia.net/embed/iframe/gxttknv8aa?auto Play=true&controlsVisibleOnLoad=true&version=v1&vo lumeControl=true Konrad Synoradzki Allegro Group 2013
  22. 22. Technical details but how?? Konrad Synoradzki Allegro Group 2013
  23. 23. Organizing Experiments Konrad Synoradzki Allegro Group 2013
  24. 24. User Roles Konrad Synoradzki Allegro Group 2013
  25. 25. Each Project Has Its Own Project Code (Snippet) or asynchronous version Konrad Synoradzki Allegro Group 2013
  26. 26. Optimizely Snippet Contents • • • • • Active Experiments Draft Experiments Project Goals Optimizely execution and tracking logic Optimizely jQuery Settings (optional – version 1.6.4 included by default) Konrad Synoradzki Allegro Group 2013
  27. 27. Two modes Konrad Synoradzki Allegro Group 2013
  28. 28. Optimizely translates every change made in the visual editor into a line of jQuery code, which we call Variation Code: Konrad Synoradzki Allegro Group 2013
  29. 29. Optimizely Testing Process Konrad Synoradzki Allegro Group 2013
  30. 30. Order of Execution Konrad Synoradzki Allegro Group 2013
  31. 31. Snippet Timing Logic Konrad Synoradzki Allegro Group 2013
  32. 32. Optimizely Architecture Overview Konrad Synoradzki Allegro Group 2013
  33. 33. Optimizely Architecture Overview • Google App Engine is used to store and manage your account and experiment configuration settings, including your account login and password information. Passwords are hashed and stored securely using 160 bit cryptographic hashes. • Amazon Elastic Compute Cloud and Simple Storage Service is used to store your conversion/traffic logging data, which Optimizely uses to compute experiment results. • Akamai is used to store and serve your Optimizely snippet JavaScript file(s). This file is publicly accessible by necessity. Konrad Synoradzki Allegro Group 2013
  34. 34. Collecting data The visitor’s browser makes an asynchronous request to https://log.optimizely.com, which logs the following information: • • • • • • • The Optimizely­­assigned unique user ID IDs of the experiments and variations this visitor has seen Your Optimizely account number Visitor’s browser/OS version Visitor’s IP address, with last octet removed A string identifying this conversion goal, such as the page URL Date and time Konrad Synoradzki Allegro Group 2013
  35. 35. that’s all Konrad Synoradzki Allegro Group 2013
  36. 36. question? Konrad Synoradzki Allegro Group 2013
  37. 37. Konrad Synoradzki Allegro Group 2013

×