A/B Testing with Website Optimizer

  • 770 views
Uploaded on

Miva Merchant Conference 2012 Breakout Session by Brennan Heyde.

Miva Merchant Conference 2012 Breakout Session by Brennan Heyde.

More in: Technology , Design
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
  • Hi! On slides 14 and 15 you have a zoom experiment. Could you please let me know which version won and why. Or point me to a source about that experiment. Thanks!
    Are you sure you want to
    Your message goes here
    Be the first to like this
No Downloads

Views

Total Views
770
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
10
Comments
1
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. A/B Testing with Website OptimizerBrennan HeydeTechnical Director – Professional Services3/10/2012
  • 2. Today’s Overview • What is A/B Testing? • Why You Should be A/B Testing • Real World Examples • Introduction to Google Website Optimizer • Walk though A Real Test Using Miva Merchant • Other Tools / Resources / Q.A.
  • 3. What is A/B Testing? • Also called split testing • Complicated: Testing a baseline control sample with a variety of single-variable tests to improve conversion rates. • Easy: Testing two different versions of a page to see which one converts better.
  • 4. What is A/B Testing? • 3 Parts to Any A/B test • 1. Original Page (A) • 2. Variation Page (B) • 3. Conversion Page
  • 5. What is A/B Testing • Other Types of Testing: • Multivariate Testing – Testing multiple changes on the SAME page Examples: • Button Colors • Headlines • Placement of Calls to Action
  • 6. Why you should be A/B testing • Data Driven Decisions • Allows you to take personal preference (yours, your developers, your designers) out of the equation and help you make a data driven decision. • It will Increase your conversion rate!
  • 7. Examples – WhichTestWon.comA
  • 8. ExamplesB
  • 9. A
  • 10. B
  • 11. A
  • 12. B
  • 13. What Should You Test? • Different Product Page Layouts • Checkout Variations • Alternate Payment Methods • New Features • Promotions Create a plan before you begin.
  • 14. Scottevest.com • Inside Zoom Vs Outside Zoom
  • 15. Scottevest.com • Inside Zoom Vs Outside Zoom
  • 16. Scottevest.com • Category Quick View
  • 17. Scottevest.com • Runway Videos – 32% Improvement
  • 18. Scottevest.com • Paypal vs No Paypal
  • 19. What is Google Website Optimizer?• Free Tool from Google• Allows you to easily perform A/B tests on your website.• Track Results and make data driven decisions• Increase your conversion rate• Use your same Google Account Login
  • 20. Our Test Experiment • Question: What add to basket button color and text converts best for my customers? Original: Grey – Add to Basket Test Cases 1. Green – Add to Cart 2. Red – Add to Cart 3. Greem – Add to Basket 4. Green – Add to Basket
  • 21. Website Optimizer Walk-Thru
  • 22. Website Optimizer Walk-Thru
  • 23. A/B Checklist
  • 24. How to Create Page Variations in Miva Merchant? • Trick: Any parameters passed in the URL automatically become global variables in Miva. • Ex: www.mystore.com/mm5/merchant.mvc?button=1 • I can now use &mvt:global:button; OR <mvt:if expr=“g.button EQ 1”> <img src=“/images/button1.jpg” /> <mvt:elseif expr=“g.button EQ 2"> <img src=“/images/button2.jpg” /> </mvt:if>
  • 25. How to create page variations in Miva? • This allows us to use these different URLs in Website Optimizer Orginal: http://www.mystore.com/product/product_code.html Variations: http://www.mystore.com/product/product_code.html?button=1 http://www.mystore.com/product/product_code.html?button=2 http://www.mystore.com/product/product_code.html?button=3 http://www.mystore.com/product/product_code.html?button=4
  • 26. Edit PRODThen Product Display LayoutAdd the conditional statementwhere your add to cart buttonappears
  • 27. Conversion URL: http://store20.coolcommerce.net/store/merchant.mvc?Screen=INVC
  • 28. Adding the JavaScript Code
  • 29. Tracking Code – Step 1Original Page – Control Script
  • 30. Head Tag Conditional Statement <mvt:if expr="g.Screen EQ PROD AND g.Product_Code EQ 3583 AND ISNULL(g.button)"> // control script goes here <mvt:elseif expr="NOT ISNULL(g.button)"> // Variation tracking code goes here <mvt:elseif expr="g.Screen EQ INVC"> // Conversion tracking code goes here </mvt:if>
  • 31. Validate Your Pages, Start The Test!
  • 32. View Results
  • 33. Additional Notes • You can run multiple tests at the same time • GWO will give you improvement on conversion rate. However, you can use Google Analytics custom variables to also track: • Average Order Value • Number of Transactions • Per Visit Revenue
  • 34. Additional Resources / Questions • http://blog.kissmetrics.com/guide-google- website-optimizer/ • http://www.google.com/intl/en/websiteoptimizer/ tutorials.html • http://www.google.com/intl/en/websiteoptimizer/ articles.html