Content Testing & Optimization (CTO) Webinar, April 29 2010
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Content Testing & Optimization (CTO) Webinar, April 29 2010

on

  • 1,193 views

 

Statistics

Views

Total Views
1,193
Views on SlideShare
1,190
Embed Views
3

Actions

Likes
1
Downloads
24
Comments
0

2 Embeds 3

http://www.lmodules.com 2
http://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Content Testing & Optimization (CTO) Webinar, April 29 2010 Presentation Transcript

  • 1. Content Testing & Optimization (CTO)
    April 29, 2010
     One to One Interactive, 2010. All rights reserved.
  • 2. Agenda
  • 3. What Is Content Testing & Optimization?
    %?
    Sale!
    (But how many made it through?)
  • 4. The Main Question
    “Can we rapidly test new landing page elements -- or entirely new pages -- to continue to lift our site’s conversion rate?”
  • 5. Introduction
    Who should consider Content Testing & Optimization?
    Marketers looking to increase online conversion, revenue and ROI by leveraging the unique opportunities offered by the digital space to improve visitor response
    What will we cover today?
    The philosophy, planning and implementation that drives a Content Testing and Optimization (CTO) strategy and share a few of One To One’s successes
    What will be my takeaways?
    • The digital marketing strategy behind Content Testing and Optimization
    • 6. How A/B/N and multivariate test strategies are designed
    • 7. How CTO can lead to significant lifts in conversion, revenue and ROI
    • 8. How optimization learnings can improve ongoing marketing strategy
  • One To One’s Approach To Content Testing and Optimization(CTO)
  • 9. Approach To Content Testing and Optimization
    Objectives
    Create ROI positive sustainable improvements in content performance against desired actions
    Build ongoing optimization engine
    Approach
    Conduct test in waves against specific objectives
    Design tests to always achieve statistical significance with rapid turnaround times (2-6 weeks)
    Test Formats
    Content development (A/B/N testing)
    Content elements (multivariate optimization testing)
  • 10. A/B/N Redesign Test
    • The “A/B/…N” approach provides the simplest approach to Content Testing and Optimization
    Redesign B
    Champion Page
    Redesign A
  • 11. A/B/N Redesign Test
    • Present highly varied, competing page concepts designed to demonstrate as widely different designs as possible
    Redesign B
    Champion
    Redesign A
  • 12. A/B/N Redesign Test
    • The greater potential to affect overall results temporarily supersedes your ability measure the effect of specific elements on the page
    Redesign B
    Champion
    Redesign A
  • 13. A/B/N Redesign Test
    • Don’t get trapped trying to keep any elements “consistent”. Your visitors will only ever see one version!
    Redesign B
    Champion
    Redesign A
  • 14. Most popular complaint to A/B/N testing
    “But I don’t know why that page won!”
    Solution?
    Eye Tracking
  • 15. Using Eye Tracking To Identify Areas of Interest
    B
    A
    B
    A
    B
    • Understanding which elements on your winning page generated the most fixation and longest gaze duration can help provide the “why” of A/B/N testing
    A
  • 16. Multivariate Testing (MVT) Completion
    • Once an A/B/N test’s winning design is determined, the losing designs are turned off and the winner is declared the new Champion
    • 17. Any findings are determined via methods such as eye tracking or link analysis, though the specific “what’s most important on this page?” answer may not be available… yet
    • 18. At this point “multivariate” or “MVT” testing of specific page elements on the new Champion is recommended
  • Multivariate Testing (MVT)
    • Elements within a layout such as imagery, copy, calls to action are varied and tested against each other in differing combinations, or “recipes”
  • Multivariate Testing (MVT)
    • The best performing recipe (for example, Headline “1” + Image “3” + Call To Action “2,”) becomes the winner and is rolled out into the next ongoing campaign
  • Multivariate Test Dimensions
    Sample elements for test consideration
    Calls To Action
    Imagery
    Headlines
    Content/Message
    • Quantity
    • 19. Design style
    • 20. Text surrounding, “prompting” CTAs
    • 21. Color
    • 22. Location
    • 23. Size
    • 24. Prominence
    • 25. Highlighted feature content
    • 26. Size
    • 27. Theme
    • 28. Location
    • 29. Quantity
    • 30. Prominence
    • 31. Style – lifestyle, product
    • 32. Headline within imagery
    • 33. Clickability of image
    • 34. Style – story v. straight
    • 35. Size
    • 36. Location
    • 37. Color
    • 38. Length
    • 39. Case sensitivity
    • 40. Style – story v. straight
    • 41. Search keyword matching
    • 42. Banner ad image matching
    • 43. Search / banner message or offer matching
    • 44. Trust message
    • 45. Testimonials/Statistics
    • 46. Geo/demo message
    • 47. Feature information in pop-ups
    • 48. Offers
  • Test Wave Approach
    • Try to project out ongoing lift prior to launching any test, then benchmark against your projections to ensure a continued positive return on investment
    Impacting Success Metrics
    Action rate after Wave 6: 2.60%
    MVT Optimizations
    Redesign Tests
    Lift: 63%
    Base action rate before testing: 1.60%
  • 49. Accounting For Interaction Effect
    • Maybe surprisingly, the sum of the winning MVT elements is often NOT the best possible page
    Test 1
    Test 3
    Test 2
    Test 4
    Test 5
    Theoretical Best Recipe
    #2 Headline
    • Image 1
    • 50. Headline 3
    • 51. Message 4
    • 52. CTA 4
    • 53. Secondary CTA 2
  • Accounting For Interaction Effect
    • Tests can be completed more rapidly using the popular “one element at a time” approach… but at the risk of missing better pages
    Find Out If Service Is Available In Your Area!
    • Lorem ipsum dolor sit amet, consectetur
    • 54. Adipisicingelit, sed do eiusmodtempor
    • 55. Incididuntutlabore et dolore magna
    • 56. Aliqua. Utenim ad minim veniam
    Sign Up To Hear Our Latest Offers!
    • Lorem ipsum dolor sit amet, consectetur
    • 57. Adipisicingelit, sed do eiusmodtempor
    • 58. Incididuntutlabore et dolore magna
    • 59. Aliqua. Utenim ad minim veniam
    Recipe A
    All Winning Elements Options From Wave 1
    Conversion Rate: 16.7%
    Recipe D
    All Winning Element Options exceptHeadline #2
    Conversion Rate: 23.5%
  • 60. OTOi’s Content Testing & Optimization Philosophy
    • Determine test objectives, e.g. Clickthrough From Page vs. Action Rate vs. Conversion Rate
    • 61. Start with the ideal optimization process
    Optimization Wave: Ideal
    Redesign
    Optimize
    Optimize
    Redesign
    Optimize
    Optimize
    • The greatest growth in conversion is usually seen with alternating waves of redesign and optimization
    • 62. Estimate actionable volume based on projected traffic
    • 63. Extend and/or remove tests due to insufficient volume or time
    Optimization Wave: Lower-Volume Traffic Source
    Redesign
    Removed
    Optimize
    Removed
  • 64. Recommended Success Metrics
    Typical testing success metrics include:
    • Conversion rate (share of visitors completing a sale)
    • 65. Revenue per visitor
    • 66. Average shopping cart value
    Secondary success metrics include:
    • Bounce Rate
    • 67. Action Rate (percentage of visitors completing a valued non-sale action)
    • 68. Funnel Depth (average number of steps through the purchase funnel taken)
  • Beware Incomplete Fallout
    • Some tests may be rightfully set up to only measure clicks off the test page
    • 69. Be careful that success isn’t determined by a point too early in the funnel
    Reach Step
    40%
    Reach Step
    34%
    Reach Step
    28%
    Convert
    18%
    Landing Page A
    40% Clickthrough
    Step 1
    Step 2
    Step 3
    Conversion
    Reach Step
    30%
    Reach Step
    27%
    Reach Step
    25%
    Convert
    23%
    Landing Page B
    30% Clickthrough
    Step 1
    Step 2
    Step 3
    Conversion
  • 70. Forecasting Incremental Revenue
    Conversion Rate
    Projected Conversions
    1-Mo Visits
    Projected Annual Visits
    =
    2.00%
    Winning Design
    24,000
    x 12 =
    x
    100,000
    1,200,000
    =
    1.80%
    Champion Design
    21,600
    Key Questions
    How many visits does your tested page receive per month?
    How long will you continue to award credit to the new, winning page? 12 months?
    How many conversions or other desired outcomes is your winning page expected to provide over your Champion?
    What’s the value of each desired outcome -- in terms of average lifetime revenue if available?
    Projected Additional Annual Conversions
    +2,400
    x
    Sample Return Estimate
    Estimated LifetimeValue Per Conversion
    $600
    Projected Total Incremental Revenue
    $1,440,000
    Revenue - Cost
    = ROI
    Cost
  • 71. Case Studies
  • 72. Case Studies: Comcast 2009 Results
    Test Champions
    Test Winners
    Conversion Rate Metric
    11% Lift In Conversion Rate
    Wave 1
    24% Lift In Conversion Rate
    Wave 2
    66% Lift In Conversion Rate
    Wave 3
    36% Lift In Action Rate
    (over lowest performer)
    Wave 4
  • 73. Case Study: 2009 Wave 3 Multivariate Test
    • A 2009 Comcast multivariate study focused on testing:
    Image
    Headline /
    Call To Action
    • The test audience was all visitors who had clicked on a Comcast Paid Search ad
    • 74. All versions of the new page’s design were tested against the prior Champion page
  • Case Study: Comcast 2009 Wave 3 Results
    Image Results
    % Lift In Conversion Over Champion
    “Married Man”
    “Orange Woman”
    Headline/Copy Results
    % Lift In Conversion Over Champion
    • The redesign’s template seemed to significantly beat the Champion no matter which combination of Image and Headline/Copy were used
    • 75. Among the winning Images, those with the people facing toward the headline and next step button seemed to perform the best (see above)
  • Case Study: Comcast Business Class
    “Champion” Landing Page
    Test Hypothesis:
    “Versioning a banner ad’s landing page to match the design of the banners will increase clicks to the next page”
    Methodology:
    Version “Champion” elements against variations designed to match the banner ad creative more closely
    Elements Matched:
    • Primary Headline Viewed
    • 76. Primary Image
    • 77. Call To Action Colors
    1
    2
    3
  • 78. Case Study:Variable Elements
    “Champion” Elements
    “Small vs. Big” Elements
    “Dream Big” Elements
    “Compare” Elements
    Headline
    Primary Image
    Call To Action Colors
  • 79. Case Study: Winning Pages
    winner
    winner
    Question: “Does matching landing page content to the campaign creative shown make a difference in clickthrough?”
    Answer: For all creatives tested: yes
    Mixed Landing Pages
    Creative Clicked On
    Fully Matching
    Champion
    winner
    winner
  • 80. Building Your Library
  • 81. Building Your Library
    • Firms should view Content Testing & Optimization as an ongoing process which yields an ongoing lift in conversion
    • 82. Never forget that you’re looking to build out a library of best practices to inform future designs
  • Building Your Library
    Past studies have produce insights which have radically changed the prior approach to content, as ongoing testing has driven substantial lift in conversion via such findings such as:
    • Images perform far more successfully on the right side of the page, potentially due to the natural “inverse C” path of eye tracking
  • Building Your Library
    Past studies have produce insights which have radically changed the prior approach to content, as ongoing testing has driven substantial lift in conversion via such findings such as:
    • For consumer products, images of people are far more successful in driving conversions than images of products – especially of faces
  • Building Your Library
    Past studies have produce insights which have radically changed the prior approach to content, as ongoing testing has driven substantial lift in conversion via such findings such as:
    • The newer an offering, the less successful hard-sell messaging such as “Sign Up Now!” is to drive page interaction
    Click To Sign Up Now!
    Check Out Products And Features!
  • 83. Building Your Library
    Past studies have produce insights which have radically changed the prior approach to content, as ongoing testing has driven substantial lift in conversion via such findings such as:
    • Images of happy consumers looking in the direction of Calls To Action rather than out of the screen generally generate higher conversion
    Offer!
    Offer!
  • 84. Building Your Library
    Past studies have produce insights which have radically changed the prior approach to content, as ongoing testing has driven substantial lift in conversion via such findings such as:
    • Matching landing page messaging to the copy in a paid search advertisement clicked is far more important when the copy includes an offer
  • In Conclusion
    • A Content Testing & Optimization strategy effectively leverages the unique opportunities offered by the digital marketplace and provides a foundation of best practices for ongoing content development
    • 85. By engaging in the ongoing process of content testing, firms can rapidly implement new content to generate increased response, resulting in steady lifts in conversion, revenue and return on investment (ROI)
    • 86. GIVE IT A TRY!