TERMINALFOUR t44u 2010 - WEM Variation impact analysis


Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • 06/12/11
  • 06/12/11
  • 06/12/11
  • 06/12/11
  • 06/12/11
  • 06/12/11
  • 06/12/11
  • 06/12/11
  • 06/12/11
  • 06/12/11
  • TERMINALFOUR t44u 2010 - WEM Variation impact analysis

    1. 1. Raphaele Beyssac Variation impact analysis - A/B Testing Testing your content t44u User & Partner Conference 2010
    2. 2. What Is A/B Testing? <ul><li>A/B testing allows you to measure the most successful variation of a page. </li></ul><ul><li>This variation can be a subtle difference in text, colour, design or could be a very different structure. </li></ul><ul><li>Using Site Manager you can easily: </li></ul><ul><ul><li>create the variations, </li></ul></ul><ul><ul><li>add the code to measure the impact, </li></ul></ul><ul><ul><li>implement the most successful variant after you have completed your testing. </li></ul></ul><ul><li>Today we will cover a simple method that can be used to conduct A/B testing. </li></ul>t44u User & Partner Conference 2010
    3. 3. Method <ul><li>In our example, we want to find out which is the best label for a pdf download link. </li></ul><ul><li>This method uses code on a landing page to randomly show users version A or version B of a page. </li></ul><ul><li>Each of these versions show a slightly different label for the pdf download link. </li></ul><ul><li>Results can be seen in Google Analytics interface. </li></ul>t44u User & Partner Conference 2010
    4. 4. Step 1 – The Structure <ul><li>A “Marketing Campaign” section (our landing page). </li></ul><ul><li>Two sub-sections with slightly different content (our 2 variations). </li></ul><ul><li>View Site Manager Site Structure >> </li></ul>t44u User & Partner Conference 2010 Landing Page Option A: URI = variation-a Option B: URI = variation-b
    5. 5. Step 2 – The Landing Page <ul><li>We added a piece of code to the “Marketing Campaign” section to randomly display content of “Whitepaper 1” or “Whitepaper 2”. </li></ul><ul><li>It is also possible to have 10% of users see one variation and 90% another </li></ul><ul><li>What ever the version of content, the page always show the same URL, i.e.: http://www.my-site.com/news/marketingcampaign/ </li></ul><ul><li>We use Server Side (PHP/JSP/ASP.NET) A/B testing code added as template to the Landing page section </li></ul>t44u User & Partner Conference 2010
    6. 6. Step 3 – The Variations <ul><li>We created content for option A in the “Whitepaper 1” section. </li></ul><ul><li>Then it can be duplicated to the “Whitepaper 2” section and slightly amended to create the option B. </li></ul><ul><li>See the landing page >> </li></ul>t44u User & Partner Conference 2010 Option A Option B
    7. 7. Step 3 – The Variations Pages <ul><li>Our Goal: find out which link label works better </li></ul><ul><ul><li>Which file is downloaded the most: pdf in Option A or Option B? </li></ul></ul><ul><ul><li>Google Analytics: Events Tracking/Goal Tracking can help us find out. </li></ul></ul><ul><li>For our test we did: </li></ul><ul><ul><li>Set up a Google Analytics account and added the relevant JavaScript Analytics code to our style header. </li></ul></ul><ul><ul><li>Update the media formatter so that the media can have an onclick attribute. </li></ul></ul><ul><ul><li>Add onclick attribute to the pdf links: </li></ul></ul><ul><ul><li>Option A: onClick=&quot;_gaq.push(['_trackEvent', 'Downloads', 'PDF', 'W hite Paper A ']);&quot; </li></ul></ul><ul><ul><li>Option B: onClick=&quot;_gaq.push(['_trackEvent', 'Downloads', 'PDF', 'W hite Paper B ']);&quot; </li></ul></ul>t44u User & Partner Conference 2010
    8. 8. Step 3 – The Variations Pages (media formatter code) <ul><li>Updated media formatter (so it can have an onclick attribute): </li></ul><ul><ul><li><a href=&quot;<t4 type=&quot;content&quot; output=&quot;file&quot; modifiers=&quot;nav_sections&quot; name=&quot;Media&quot; />&quot; title=&quot;<t4 type=&quot;content&quot; output=&quot;normal&quot; modifiers=&quot;nav_sections&quot; name=&quot;Description&quot; />&quot; <t4 type=&quot;media&quot; attribute=&quot;onClick&quot; format=&quot;onClick=&quot;$value&quot;&quot; /> target=&quot;_blank&quot;><t4 type=&quot;content&quot; name=&quot;Name&quot; output=&quot;normal&quot; modifiers=&quot;&quot; /></a> </li></ul></ul>t44u User & Partner Conference 2010
    9. 9. Step 4 - Measuring <ul><li>In the Google Analytics interface, results can be seen under Content > Event Tracking: </li></ul>t44u User & Partner Conference 2010
    10. 10. Conclusion <ul><li>A/B testing can also be used to test variations in colour, copy or any element of the page layout. </li></ul><ul><li>A/B testing is more efficient if you test single-variable variations rather than variations of multiple variables. </li></ul><ul><li>Note that what we have demonstrated is just one of many possible methods that can be used. Just Google “A/B testing tools”  </li></ul>t44u User & Partner Conference 2010