A/B Testing with Website OptimizerBrennan HeydeTechnical Director – Professional Services3/10/2012
Today’s Overview • What is A/B Testing? • Why You Should be A/B Testing • Real World Examples • Introduction to Google Website Optimizer • Walk though A Real Test Using Miva Merchant • Other Tools / Resources / Q.A.
What is A/B Testing? • Also called split testing • Complicated: Testing a baseline control sample with a variety of single-variable tests to improve conversion rates. • Easy: Testing two different versions of a page to see which one converts better.
What is A/B Testing? • 3 Parts to Any A/B test • 1. Original Page (A) • 2. Variation Page (B) • 3. Conversion Page
What is A/B Testing • Other Types of Testing: • Multivariate Testing – Testing multiple changes on the SAME page Examples: • Button Colors • Headlines • Placement of Calls to Action
Why you should be A/B testing • Data Driven Decisions • Allows you to take personal preference (yours, your developers, your designers) out of the equation and help you make a data driven decision. • It will Increase your conversion rate!
What is Google Website Optimizer?• Free Tool from Google• Allows you to easily perform A/B tests on your website.• Track Results and make data driven decisions• Increase your conversion rate• Use your same Google Account Login
Our Test Experiment • Question: What add to basket button color and text converts best for my customers? Original: Grey – Add to Basket Test Cases 1. Green – Add to Cart 2. Red – Add to Cart 3. Greem – Add to Basket 4. Green – Add to Basket
How to Create Page Variations in Miva Merchant? • Trick: Any parameters passed in the URL automatically become global variables in Miva. • Ex: www.mystore.com/mm5/merchant.mvc?button=1 • I can now use &mvt:global:button; OR <mvt:if expr=“g.button EQ 1”> <img src=“/images/button1.jpg” /> <mvt:elseif expr=“g.button EQ 2"> <img src=“/images/button2.jpg” /> </mvt:if>
How to create page variations in Miva? • This allows us to use these different URLs in Website Optimizer Orginal: http://www.mystore.com/product/product_code.html Variations: http://www.mystore.com/product/product_code.html?button=1 http://www.mystore.com/product/product_code.html?button=2 http://www.mystore.com/product/product_code.html?button=3 http://www.mystore.com/product/product_code.html?button=4
Edit PRODThen Product Display LayoutAdd the conditional statementwhere your add to cart buttonappears
Tracking Code – Step 1Original Page – Control Script
Head Tag Conditional Statement <mvt:if expr="g.Screen EQ PROD AND g.Product_Code EQ 3583 AND ISNULL(g.button)"> // control script goes here <mvt:elseif expr="NOT ISNULL(g.button)"> // Variation tracking code goes here <mvt:elseif expr="g.Screen EQ INVC"> // Conversion tracking code goes here </mvt:if>
Additional Notes • You can run multiple tests at the same time • GWO will give you improvement on conversion rate. However, you can use Google Analytics custom variables to also track: • Average Order Value • Number of Transactions • Per Visit Revenue