Content Testing & Optimization (CTO)<br />April 29, 2010<br /> One to One Interactive, 2010. All rights reserved.<br />
Agenda<br />
What Is Content Testing & Optimization?<br />%?<br />Sale!<br />(But how many made it through?)<br />
The Main Question<br />“Can we rapidly test new landing page elements -- or entirely new pages -- to continue to lift our ...
Introduction<br />Who should consider Content Testing & Optimization?<br />Marketers looking to increase online conversion...
	How A/B/N and multivariate test strategies are designed
	How CTO can lead to significant lifts in conversion, revenue and ROI
	How optimization learnings can improve ongoing marketing strategy</li></li></ul><li>One To One’s Approach To Content Test...
Approach To Content Testing and Optimization<br />Objectives<br />Create ROI positive sustainable improvements in content ...
A/B/N Redesign Test<br /><ul><li>The “A/B/…N” approach provides the simplest approach to Content Testing and Optimization<...
A/B/N Redesign Test<br /><ul><li>Present highly varied, competing page concepts designed to demonstrate as widely differen...
A/B/N Redesign Test<br /><ul><li>The greater potential to affect overall results temporarily supersedes your ability measu...
A/B/N Redesign Test<br /><ul><li>Don’t get trapped trying to keep any elements “consistent”. Your visitors will only ever ...
Most popular complaint to A/B/N testing<br />“But I don’t know why that page won!”<br />Solution?<br />Eye Tracking<br />
Using Eye Tracking To Identify Areas of Interest<br />B<br />A<br />B<br />A<br />B<br /><ul><li>Understanding which eleme...
Multivariate Testing (MVT) Completion<br /><ul><li>Once an A/B/N test’s winning design is determined, the losing designs a...
Any findings are determined via methods such as eye tracking or link analysis, though the specific “what’s most important ...
At this point “multivariate” or “MVT” testing of specific page elements on the new Champion is recommended</li></li></ul><...
Design style
Text surrounding, “prompting” CTAs
Color
Location
Size
Prominence
Highlighted feature content
Size
Theme
Location
Quantity
Prominence
Style – lifestyle, product
Headline within imagery
Clickability of image
Style – story v. straight
Size
Location
Color
Length
Case sensitivity
Upcoming SlideShare
Loading in...5
×

Content Testing & Optimization (CTO) Webinar, April 29 2010

908

Published on

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
908
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
25
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Content Testing & Optimization (CTO) Webinar, April 29 2010

  1. 1. Content Testing & Optimization (CTO)<br />April 29, 2010<br /> One to One Interactive, 2010. All rights reserved.<br />
  2. 2. Agenda<br />
  3. 3. What Is Content Testing & Optimization?<br />%?<br />Sale!<br />(But how many made it through?)<br />
  4. 4. The Main Question<br />“Can we rapidly test new landing page elements -- or entirely new pages -- to continue to lift our site’s conversion rate?”<br />
  5. 5. Introduction<br />Who should consider Content Testing & Optimization?<br />Marketers looking to increase online conversion, revenue and ROI by leveraging the unique opportunities offered by the digital space to improve visitor response<br />What will we cover today?<br />The philosophy, planning and implementation that drives a Content Testing and Optimization (CTO) strategy and share a few of One To One’s successes<br />What will be my takeaways?<br /><ul><li>The digital marketing strategy behind Content Testing and Optimization
  6. 6. How A/B/N and multivariate test strategies are designed
  7. 7. How CTO can lead to significant lifts in conversion, revenue and ROI
  8. 8. How optimization learnings can improve ongoing marketing strategy</li></li></ul><li>One To One’s Approach To Content Testing and Optimization(CTO)<br />
  9. 9. Approach To Content Testing and Optimization<br />Objectives<br />Create ROI positive sustainable improvements in content performance against desired actions<br />Build ongoing optimization engine<br />Approach<br />Conduct test in waves against specific objectives <br />Design tests to always achieve statistical significance with rapid turnaround times (2-6 weeks)<br />Test Formats<br />Content development (A/B/N testing)<br />Content elements (multivariate optimization testing)<br />
  10. 10. A/B/N Redesign Test<br /><ul><li>The “A/B/…N” approach provides the simplest approach to Content Testing and Optimization</li></ul>Redesign B<br />Champion Page<br />Redesign A<br />
  11. 11. A/B/N Redesign Test<br /><ul><li>Present highly varied, competing page concepts designed to demonstrate as widely different designs as possible</li></ul>Redesign B<br />Champion<br />Redesign A<br />
  12. 12. A/B/N Redesign Test<br /><ul><li>The greater potential to affect overall results temporarily supersedes your ability measure the effect of specific elements on the page</li></ul>Redesign B<br />Champion<br />Redesign A<br />
  13. 13. A/B/N Redesign Test<br /><ul><li>Don’t get trapped trying to keep any elements “consistent”. Your visitors will only ever see one version!</li></ul>Redesign B<br />Champion<br />Redesign A<br />
  14. 14. Most popular complaint to A/B/N testing<br />“But I don’t know why that page won!”<br />Solution?<br />Eye Tracking<br />
  15. 15. Using Eye Tracking To Identify Areas of Interest<br />B<br />A<br />B<br />A<br />B<br /><ul><li>Understanding which elements on your winning page generated the most fixation and longest gaze duration can help provide the “why” of A/B/N testing</li></ul>A<br />
  16. 16. Multivariate Testing (MVT) Completion<br /><ul><li>Once an A/B/N test’s winning design is determined, the losing designs are turned off and the winner is declared the new Champion
  17. 17. Any findings are determined via methods such as eye tracking or link analysis, though the specific “what’s most important on this page?” answer may not be available… yet
  18. 18. At this point “multivariate” or “MVT” testing of specific page elements on the new Champion is recommended</li></li></ul><li>Multivariate Testing (MVT)<br /><ul><li>Elements within a layout such as imagery, copy, calls to action are varied and tested against each other in differing combinations, or “recipes”</li></li></ul><li>Multivariate Testing (MVT)<br /><ul><li>The best performing recipe (for example, Headline “1” + Image “3” + Call To Action “2,”) becomes the winner and is rolled out into the next ongoing campaign</li></li></ul><li>Multivariate Test Dimensions<br />Sample elements for test consideration<br />Calls To Action<br />Imagery<br />Headlines<br />Content/Message<br /><ul><li>Quantity
  19. 19. Design style
  20. 20. Text surrounding, “prompting” CTAs
  21. 21. Color
  22. 22. Location
  23. 23. Size
  24. 24. Prominence
  25. 25. Highlighted feature content
  26. 26. Size
  27. 27. Theme
  28. 28. Location
  29. 29. Quantity
  30. 30. Prominence
  31. 31. Style – lifestyle, product
  32. 32. Headline within imagery
  33. 33. Clickability of image
  34. 34. Style – story v. straight
  35. 35. Size
  36. 36. Location
  37. 37. Color
  38. 38. Length
  39. 39. Case sensitivity
  40. 40. Style – story v. straight
  41. 41. Search keyword matching
  42. 42. Banner ad image matching
  43. 43. Search / banner message or offer matching
  44. 44. Trust message
  45. 45. Testimonials/Statistics
  46. 46. Geo/demo message
  47. 47. Feature information in pop-ups
  48. 48. Offers</li></li></ul><li>Test Wave Approach<br /><ul><li>Try to project out ongoing lift prior to launching any test, then benchmark against your projections to ensure a continued positive return on investment</li></ul>Impacting Success Metrics<br />Action rate after Wave 6: 2.60%<br />MVT Optimizations<br />Redesign Tests<br />Lift: 63%<br />Base action rate before testing: 1.60%<br />
  49. 49. Accounting For Interaction Effect<br /><ul><li>Maybe surprisingly, the sum of the winning MVT elements is often NOT the best possible page</li></ul>Test 1<br />Test 3<br />Test 2<br />Test 4<br />Test 5<br />Theoretical Best Recipe<br />#2 Headline<br /><ul><li>Image 1
  50. 50. Headline 3
  51. 51. Message 4
  52. 52. CTA 4
  53. 53. Secondary CTA 2</li></li></ul><li>Accounting For Interaction Effect<br /><ul><li>Tests can be completed more rapidly using the popular “one element at a time” approach… but at the risk of missing better pages</li></ul>Find Out If Service Is Available In Your Area!<br /><ul><li>Lorem ipsum dolor sit amet, consectetur
  54. 54. Adipisicingelit, sed do eiusmodtempor
  55. 55. Incididuntutlabore et dolore magna
  56. 56. Aliqua. Utenim ad minim veniam</li></ul>Sign Up To Hear Our Latest Offers!<br /><ul><li>Lorem ipsum dolor sit amet, consectetur
  57. 57. Adipisicingelit, sed do eiusmodtempor
  58. 58. Incididuntutlabore et dolore magna
  59. 59. Aliqua. Utenim ad minim veniam</li></ul>Recipe A<br />All Winning Elements Options From Wave 1<br />Conversion Rate: 16.7%<br />Recipe D<br />All Winning Element Options exceptHeadline #2<br />Conversion Rate: 23.5%<br />
  60. 60. OTOi’s Content Testing & Optimization Philosophy<br /><ul><li>Determine test objectives, e.g. Clickthrough From Page vs. Action Rate vs. Conversion Rate
  61. 61. Start with the ideal optimization process</li></ul>Optimization Wave: Ideal<br />Redesign<br />Optimize<br />Optimize<br />Redesign<br />Optimize<br />Optimize<br /><ul><li>The greatest growth in conversion is usually seen with alternating waves of redesign and optimization
  62. 62. Estimate actionable volume based on projected traffic
  63. 63. Extend and/or remove tests due to insufficient volume or time</li></ul>Optimization Wave: Lower-Volume Traffic Source<br />Redesign<br />Removed<br />Optimize<br />Removed<br />
  64. 64. Recommended Success Metrics<br />Typical testing success metrics include:<br /><ul><li>Conversion rate (share of visitors completing a sale)
  65. 65. Revenue per visitor
  66. 66. Average shopping cart value</li></ul>Secondary success metrics include: <br /><ul><li>Bounce Rate
  67. 67. Action Rate (percentage of visitors completing a valued non-sale action)
  68. 68. Funnel Depth (average number of steps through the purchase funnel taken)</li></li></ul><li>Beware Incomplete Fallout<br /><ul><li>Some tests may be rightfully set up to only measure clicks off the test page
  69. 69. Be careful that success isn’t determined by a point too early in the funnel</li></ul>Reach Step<br />40%<br />Reach Step<br />34%<br />Reach Step<br />28%<br />Convert<br />18%<br />Landing Page A<br />40% Clickthrough<br />Step 1<br />Step 2<br />Step 3<br />Conversion<br />Reach Step<br />30%<br />Reach Step<br />27%<br />Reach Step<br />25%<br />Convert<br />23%<br />Landing Page B<br />30% Clickthrough<br />Step 1<br />Step 2<br />Step 3<br />Conversion<br />
  70. 70. Forecasting Incremental Revenue <br />Conversion Rate<br />Projected Conversions<br />1-Mo Visits<br />Projected Annual Visits<br />=<br />2.00%<br />Winning Design<br />24,000<br />x 12 =<br />x<br />100,000<br />1,200,000<br />=<br />1.80%<br />Champion Design<br />21,600<br />Key Questions<br />How many visits does your tested page receive per month?<br />How long will you continue to award credit to the new, winning page? 12 months?<br />How many conversions or other desired outcomes is your winning page expected to provide over your Champion?<br />What’s the value of each desired outcome -- in terms of average lifetime revenue if available?<br />Projected Additional Annual Conversions<br />+2,400<br />x<br />Sample Return Estimate<br />Estimated LifetimeValue Per Conversion<br />$600<br />Projected Total Incremental Revenue<br />$1,440,000<br /> Revenue - Cost<br />= ROI<br />Cost<br />
  71. 71. Case Studies<br />
  72. 72. Case Studies: Comcast 2009 Results<br />Test Champions<br />Test Winners<br />Conversion Rate Metric<br />11% Lift In Conversion Rate<br />Wave 1<br />24% Lift In Conversion Rate<br />Wave 2<br />66% Lift In Conversion Rate<br />Wave 3<br />36% Lift In Action Rate<br />(over lowest performer)<br />Wave 4<br />
  73. 73. Case Study: 2009 Wave 3 Multivariate Test<br /><ul><li>A 2009 Comcast multivariate study focused on testing:</li></ul>Image<br />Headline / <br />Call To Action<br /><ul><li> The test audience was all visitors who had clicked on a Comcast Paid Search ad
  74. 74. All versions of the new page’s design were tested against the prior Champion page</li></li></ul><li>Case Study: Comcast 2009 Wave 3 Results<br />Image Results<br />% Lift In Conversion Over Champion<br />“Married Man”<br />“Orange Woman”<br />Headline/Copy Results<br />% Lift In Conversion Over Champion<br /><ul><li>The redesign’s template seemed to significantly beat the Champion no matter which combination of Image and Headline/Copy were used
  75. 75. Among the winning Images, those with the people facing toward the headline and next step button seemed to perform the best (see above)</li></li></ul><li>Case Study: Comcast Business Class<br />“Champion” Landing Page<br />Test Hypothesis:<br />“Versioning a banner ad’s landing page to match the design of the banners will increase clicks to the next page”<br />Methodology:<br />Version “Champion” elements against variations designed to match the banner ad creative more closely<br />Elements Matched:<br /><ul><li>Primary Headline Viewed
  76. 76. Primary Image
  77. 77. Call To Action Colors</li></ul>1<br />2<br />3<br />
  78. 78. Case Study:Variable Elements<br />“Champion” Elements<br />“Small vs. Big” Elements<br />“Dream Big” Elements<br />“Compare” Elements<br />Headline<br />Primary Image<br />Call To Action Colors<br />
  79. 79. Case Study: Winning Pages<br />winner<br />winner<br />Question: “Does matching landing page content to the campaign creative shown make a difference in clickthrough?”<br />Answer: For all creatives tested: yes<br />Mixed Landing Pages<br />Creative Clicked On<br />Fully Matching<br />Champion<br />winner<br />winner<br />
  80. 80. Building Your Library<br />
  81. 81. Building Your Library<br /><ul><li>Firms should view Content Testing & Optimization as an ongoing process which yields an ongoing lift in conversion
  82. 82. Never forget that you’re looking to build out a library of best practices to inform future designs</li></li></ul><li>Building Your Library<br />Past studies have produce insights which have radically changed the prior approach to content, as ongoing testing has driven substantial lift in conversion via such findings such as:<br /><ul><li>Images perform far more successfully on the right side of the page, potentially due to the natural “inverse C” path of eye tracking</li></li></ul><li>Building Your Library<br />Past studies have produce insights which have radically changed the prior approach to content, as ongoing testing has driven substantial lift in conversion via such findings such as:<br /><ul><li>For consumer products, images of people are far more successful in driving conversions than images of products – especially of faces</li></li></ul><li>Building Your Library<br />Past studies have produce insights which have radically changed the prior approach to content, as ongoing testing has driven substantial lift in conversion via such findings such as:<br /><ul><li>The newer an offering, the less successful hard-sell messaging such as “Sign Up Now!” is to drive page interaction</li></ul>Click To Sign Up Now!<br />Check Out Products And Features!<br />
  83. 83. Building Your Library<br />Past studies have produce insights which have radically changed the prior approach to content, as ongoing testing has driven substantial lift in conversion via such findings such as:<br /><ul><li>Images of happy consumers looking in the direction of Calls To Action rather than out of the screen generally generate higher conversion</li></ul>Offer!<br />Offer!<br />
  84. 84. Building Your Library<br />Past studies have produce insights which have radically changed the prior approach to content, as ongoing testing has driven substantial lift in conversion via such findings such as:<br /><ul><li>Matching landing page messaging to the copy in a paid search advertisement clicked is far more important when the copy includes an offer</li></li></ul><li>In Conclusion<br /><ul><li>A Content Testing & Optimization strategy effectively leverages the unique opportunities offered by the digital marketplace and provides a foundation of best practices for ongoing content development
  85. 85. By engaging in the ongoing process of content testing, firms can rapidly implement new content to generate increased response, resulting in steady lifts in conversion, revenue and return on investment (ROI)
  86. 86. GIVE IT A TRY!</li>
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×