Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
A/B Testing with Website OptimizerBrennan HeydeTechnical Director – Professional Services3/10/2012
Today’s Overview  • What is A/B Testing?  • Why You Should be A/B Testing  • Real World Examples  • Introduction to Google...
What is A/B Testing?   • Also called split testing   • Complicated: Testing a baseline control sample     with a variety o...
What is A/B Testing?   • 3 Parts to Any A/B test      • 1. Original Page (A)      • 2. Variation Page (B)      • 3. Conver...
What is A/B Testing   • Other Types of Testing:     • Multivariate Testing – Testing       multiple changes on the SAME pa...
Why you should be A/B testing  • Data Driven Decisions  • Allows you to take personal preference    (yours, your developer...
Examples – WhichTestWon.comA
ExamplesB
A
B
A
B
What Should You Test?  •   Different Product Page Layouts  •   Checkout Variations  •   Alternate Payment Methods  •   New...
Scottevest.com • Inside Zoom Vs Outside Zoom
Scottevest.com • Inside Zoom Vs Outside Zoom
Scottevest.com • Category Quick View
Scottevest.com   • Runway Videos – 32% Improvement
Scottevest.com • Paypal vs No Paypal
What is Google Website Optimizer?• Free Tool from Google• Allows you to easily  perform A/B tests on  your website.• Track...
Our Test Experiment  •    Question: What add to       basket button color and       text converts best for my       custom...
Website Optimizer Walk-Thru
Website Optimizer Walk-Thru
A/B Checklist
How to Create Page Variations in Miva Merchant?   • Trick: Any parameters passed in the URL     automatically become globa...
How to create page variations in Miva?   • This allows us to use these different URLs in Website Optimizer   Orginal:   ht...
Edit PRODThen Product Display LayoutAdd the conditional statementwhere your add to cart buttonappears
Conversion URL: http://store20.coolcommerce.net/store/merchant.mvc?Screen=INVC
Adding the JavaScript Code
Tracking Code – Step 1Original Page – Control Script
Head Tag Conditional Statement  <mvt:if expr="g.Screen EQ PROD AND g.Product_Code    EQ 3583 AND ISNULL(g.button)">    // ...
Validate Your Pages, Start The Test!
View Results
Additional Notes   • You can run multiple tests at the same time   • GWO will give you improvement on conversion     rate....
Additional Resources / Questions  • http://blog.kissmetrics.com/guide-google-    website-optimizer/  • http://www.google.c...
A/B Testing with Website Optimizer
A/B Testing with Website Optimizer
Upcoming SlideShare
Loading in …5
×

A/B Testing with Website Optimizer

1,667 views

Published on

Miva Merchant Conference 2012 Breakout Session by Brennan Heyde.

Published in: Technology, Design
  • Hi! On slides 14 and 15 you have a zoom experiment. Could you please let me know which version won and why. Or point me to a source about that experiment. Thanks!
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

A/B Testing with Website Optimizer

  1. 1. A/B Testing with Website OptimizerBrennan HeydeTechnical Director – Professional Services3/10/2012
  2. 2. Today’s Overview • What is A/B Testing? • Why You Should be A/B Testing • Real World Examples • Introduction to Google Website Optimizer • Walk though A Real Test Using Miva Merchant • Other Tools / Resources / Q.A.
  3. 3. What is A/B Testing? • Also called split testing • Complicated: Testing a baseline control sample with a variety of single-variable tests to improve conversion rates. • Easy: Testing two different versions of a page to see which one converts better.
  4. 4. What is A/B Testing? • 3 Parts to Any A/B test • 1. Original Page (A) • 2. Variation Page (B) • 3. Conversion Page
  5. 5. What is A/B Testing • Other Types of Testing: • Multivariate Testing – Testing multiple changes on the SAME page Examples: • Button Colors • Headlines • Placement of Calls to Action
  6. 6. Why you should be A/B testing • Data Driven Decisions • Allows you to take personal preference (yours, your developers, your designers) out of the equation and help you make a data driven decision. • It will Increase your conversion rate!
  7. 7. Examples – WhichTestWon.comA
  8. 8. ExamplesB
  9. 9. A
  10. 10. B
  11. 11. A
  12. 12. B
  13. 13. What Should You Test? • Different Product Page Layouts • Checkout Variations • Alternate Payment Methods • New Features • Promotions Create a plan before you begin.
  14. 14. Scottevest.com • Inside Zoom Vs Outside Zoom
  15. 15. Scottevest.com • Inside Zoom Vs Outside Zoom
  16. 16. Scottevest.com • Category Quick View
  17. 17. Scottevest.com • Runway Videos – 32% Improvement
  18. 18. Scottevest.com • Paypal vs No Paypal
  19. 19. What is Google Website Optimizer?• Free Tool from Google• Allows you to easily perform A/B tests on your website.• Track Results and make data driven decisions• Increase your conversion rate• Use your same Google Account Login
  20. 20. Our Test Experiment • Question: What add to basket button color and text converts best for my customers? Original: Grey – Add to Basket Test Cases 1. Green – Add to Cart 2. Red – Add to Cart 3. Greem – Add to Basket 4. Green – Add to Basket
  21. 21. Website Optimizer Walk-Thru
  22. 22. Website Optimizer Walk-Thru
  23. 23. A/B Checklist
  24. 24. How to Create Page Variations in Miva Merchant? • Trick: Any parameters passed in the URL automatically become global variables in Miva. • Ex: www.mystore.com/mm5/merchant.mvc?button=1 • I can now use &mvt:global:button; OR <mvt:if expr=“g.button EQ 1”> <img src=“/images/button1.jpg” /> <mvt:elseif expr=“g.button EQ 2"> <img src=“/images/button2.jpg” /> </mvt:if>
  25. 25. How to create page variations in Miva? • This allows us to use these different URLs in Website Optimizer Orginal: http://www.mystore.com/product/product_code.html Variations: http://www.mystore.com/product/product_code.html?button=1 http://www.mystore.com/product/product_code.html?button=2 http://www.mystore.com/product/product_code.html?button=3 http://www.mystore.com/product/product_code.html?button=4
  26. 26. Edit PRODThen Product Display LayoutAdd the conditional statementwhere your add to cart buttonappears
  27. 27. Conversion URL: http://store20.coolcommerce.net/store/merchant.mvc?Screen=INVC
  28. 28. Adding the JavaScript Code
  29. 29. Tracking Code – Step 1Original Page – Control Script
  30. 30. Head Tag Conditional Statement <mvt:if expr="g.Screen EQ PROD AND g.Product_Code EQ 3583 AND ISNULL(g.button)"> // control script goes here <mvt:elseif expr="NOT ISNULL(g.button)"> // Variation tracking code goes here <mvt:elseif expr="g.Screen EQ INVC"> // Conversion tracking code goes here </mvt:if>
  31. 31. Validate Your Pages, Start The Test!
  32. 32. View Results
  33. 33. Additional Notes • You can run multiple tests at the same time • GWO will give you improvement on conversion rate. However, you can use Google Analytics custom variables to also track: • Average Order Value • Number of Transactions • Per Visit Revenue
  34. 34. Additional Resources / Questions • http://blog.kissmetrics.com/guide-google- website-optimizer/ • http://www.google.com/intl/en/websiteoptimizer/ tutorials.html • http://www.google.com/intl/en/websiteoptimizer/ articles.html

×