This document discusses A/B testing and introduces Google Website Optimizer as a tool for performing A/B tests. It explains that A/B testing involves testing two variations of a page, such as original and modified versions, to determine which performs better. An example is provided of an A/B test conducted on a product page to determine the best button color and text for increasing conversions. Steps for setting up an A/B test using Google Website Optimizer and Miva Merchant are outlined, including creating page variations, tracking codes, and validating/starting the test.

1. A/B Testing with Website Optimizer
Brennan Heyde
Technical Director – Professional Services
3/10/2012

2. Today’s Overview
• What is A/B Testing?
• Why You Should be A/B Testing
• Real World Examples
• Introduction to Google Website Optimizer
• Walk though A Real Test Using Miva
Merchant
• Other Tools / Resources / Q.A.

3. What is A/B Testing?
• Also called split testing
• Complicated: Testing a baseline control sample
with a variety of single-variable tests to improve
conversion rates.
• Easy: Testing two different versions of a page to
see which one converts better.

4. What is A/B Testing?
• 3 Parts to Any A/B test
• 1. Original Page (A)
• 2. Variation Page (B)
• 3. Conversion Page

5. What is A/B Testing
• Other Types of Testing:
• Multivariate Testing – Testing
multiple changes on the SAME page
Examples:
• Button Colors
• Headlines
• Placement of Calls to Action

6. Why you should be A/B testing
• Data Driven Decisions
• Allows you to take personal preference
(yours, your developers, your designers) out of
the equation and help you make a data driven
decision.
• It will Increase your conversion rate!

7. Examples – WhichTestWon.com
A

8. Examples
B

9. A

10. B

11. A

12. B

13. What Should You Test?
• Different Product Page Layouts
• Checkout Variations
• Alternate Payment Methods
• New Features
• Promotions
Create a plan before you begin.

14. Scottevest.com • Inside Zoom Vs Outside Zoom

15. Scottevest.com • Inside Zoom Vs Outside Zoom

16. Scottevest.com • Category Quick View

17. Scottevest.com • Runway Videos – 32% Improvement

18. Scottevest.com • Paypal vs No Paypal

19. What is Google Website Optimizer?
• Free Tool from Google
• Allows you to easily
perform A/B tests on
your website.
• Track Results and make
data driven decisions
• Increase your conversion
rate
• Use your same Google
Account Login

20. Our Test Experiment
• Question: What add to
basket button color and
text converts best for my
customers?
Original: Grey – Add to Basket
Test Cases
1. Green – Add to Cart
2. Red – Add to Cart
3. Greem – Add to Basket
4. Green – Add to Basket

22. Website Optimizer Walk-Thru

23. Website Optimizer Walk-Thru

24. A/B Checklist

25. How to Create Page Variations in Miva Merchant?
• Trick: Any parameters passed in the URL
automatically become global variables in Miva.
• Ex: www.mystore.com/mm5/merchant.mvc?button=1
• I can now use &mvt:global:button; OR
<mvt:if expr=“g.button EQ 1”>
<img src=“/images/button1.jpg” />
<mvt:elseif expr=“g.button EQ 2">
<img src=“/images/button2.jpg” />
</mvt:if>

26. How to create page variations in Miva?
• This allows us to use these different URLs in Website Optimizer
Orginal:
http://www.mystore.com/product/product_code.html
Variations:
http://www.mystore.com/product/product_code.html?button=1
http://www.mystore.com/product/product_code.html?button=2
http://www.mystore.com/product/product_code.html?button=3
http://www.mystore.com/product/product_code.html?button=4

27. Edit PROD
Then Product Display Layout
Add the conditional statement
where your add to cart button
appears

28. Conversion URL: http://store20.coolcommerce.net/store/merchant.mvc?Screen=INVC

29. Adding the JavaScript Code

30. Tracking Code – Step 1
Original Page – Control Script

32. Head Tag Conditional Statement
<mvt:if expr="g.Screen EQ 'PROD' AND g.Product_Code
EQ '3583' AND ISNULL(g.button)">
// control script goes here
<mvt:elseif expr="NOT ISNULL(g.button)">
// Variation tracking code goes here
<mvt:elseif expr="g.Screen EQ 'INVC'">
// Conversion tracking code goes here
</mvt:if>

33. Validate Your Pages, Start The Test!

34. View Results

35. Additional Notes
• You can run multiple tests at the same time
• GWO will give you improvement on conversion
rate. However, you can use Google Analytics
custom variables to also track:
• Average Order Value
• Number of Transactions
• Per Visit Revenue

36. Additional Resources / Questions
• http://blog.kissmetrics.com/guide-google-
website-optimizer/
• http://www.google.com/intl/en/websiteoptimizer/
tutorials.html
• http://www.google.com/intl/en/websiteoptimizer/
articles.html