Content Optimisation<br />Which Test Won<br />
HubSpot Tests Homepage Copy: Which Got More Free Trial Downloads – Goal-Focused vs. Role-Focused?<br />
Version B convinced 49.1% more first-time homepage visitors to download a free trial.<br />Version A tailored copy to two ...
Nonprofit A/B Test: Which Page Got A 10.2% Lift In Average Donation Amounts for Haiti?<br />
Version A got a 10.2% lift in average donation amounts per page view, contributing a total of $1.02 million<br />Traffic t...
Wal-Mart’s UK Chain Tests Radical Homepage Redesign: Which Layout (not Content) Reduced Bounce Rates by Double Digits?<br />
The Version A layout, featuring non-traditional navigation elements, reduced homepage abandons by an average 19%.<br />Sho...
Ecommerce A/B Test: Which Shoe Page Version Got 39.3% More Add-to-Cart Clicks?<br />
39.3% more European 20- to 30-somethings interested in trendy street wear preferred Version A over Version B. <br />The ve...
Which Product Shots Got More Purchases: Just the Product vs. a Model Wearing the Product (in this case, shirts)? <br />
Version B’s product shots with human models not only boosted paid purchases 44%, but also gained a 67% rise in revenue per...
Avis UK Tests its PPC Landing Page: Which Got More Car Rental Bookings? <br />
Version B got a 74% lift in completed car rental bookings (sales)<br />Good example of when a stripped-down, highly-focuse...
Which Page Got More Lead Gen Form Submits?<br />
Version A convinced 66% more visitors to fill out and submit the lead gen form. <br />For the two versions we showed you, ...
PPC Landing Page Test: Which Version Boosted Free Software Sign-Ups By 59%?<br />
Version A boosted free software sign-ups by 59%… even though visitors had to click through to an additional page to sign u...
A/B Copy Test: Which PPC Landing Page Got 93% More ‘Add-to-Cart’ Clicks? <br />
Which LP got more “Add to Cart” clicks<br />The more urgent, ‘take control’ tone of Version B’s copy lifted add-to-cart bu...
Homepage Image A/B Test: “Sad” Young Woman vs. “Haggard” House Wife – <br />Which Got More Clicks on the Big Yellow Button...
Clear Debt – Varied Image – Click Test<br />Version B got 33% more visitors to click the ‘Take the Debt Analyser’ button<b...
Alpha Ecological Tests Homepage Copy, Images, and Layout: Which Got More Email Leads? <br />
Version B got a huge 114% lift in email leads from the homepage. <br />Although the winner is obvious, this isn’t an easy ...
Email A/B Test: Which Customer Survey Invite Got 51% More Clickthroughs?<br />
Which Customer Survey Got More ClickThrus<br />Version B got a whopping 51% more clicks than Version A.<br />Version A rea...
Fragrance Direct Tests the Layout of its Checkout Page: Which Got More Purchases?<br />
Version B, with images and info about the shoppers’ cart contents placed at the top of this third page in the checkout pro...
Does Size Really Matter When It Comes to Email Opt-In <br />Form Overlays? Please Vote For Which Got More Opt-Ins.<br />
FaveCrafts.com – Email Test<br />Version A increased email subscriptions by 8.8%<br />WhicTestWon professionals voted <br ...
Which Navigation Bar Copy Boosted Webinar Attendees – “Webinars” vs. “Online Training”?<br />
Version B’s “Online Training” wording increased clickthroughs from the support page to the webinar page 10.4%.<br />Webina...
Which Banner Ad Landing Page Got More Form Submissions … on Tuesday Afternoons?<br />
National University – Form Submits<br />Version B won on Tuesday, Wed, Thurs PMs by converting 17% More Form Submits<br />...
DHL Express Tests Male vs. Female Image: Which Generated 108% More Norwegian Leads on this PPC Landing Page?<br />
DHL Test<br />
Version B, with the female model in the header, convinced 108% more visitors in Norway to fill out and submit the lead gen...
All PPC traffic was sent to one of these two landing pages. The test results reached a 95% confidence level<br />The only ...
Westwood College Tests Adding a Privacy Policy Statement to its Landing Pages: Which Got More Leads?<br />
Version B, with the privacy policy statement in fine print under the button, convinced 19.1% more visitors to fill out and...
The Official 2010 Vancouver Olympic Store Tests One-Page Checkout vs. Four-Page Checkout: Which Got More Purchases? <br />
Version A, the one-page checkout, gained a 21.8% lift in actual purchases.<br />Should the checkout process be one page or...
Highlight Step One or Step Two? Which Placement of a Blue Box Increased Google Gadget Installations 50.5%?<br />
Version B increased gadget installations by 50.5%.<br />Version A featured the blue box around step one, the address box. ...
Which Landing Page Got More Form Completes? (Both Forms Ask the Same Questions)<br />
Which LP got more form completes?<br />Version B racked up 16% more form completions, making it the solid winner<br />Vers...
Product Tour Page A/B Test: Which Version Boosted Free Trial Software Downloads by 85%?<br />
YNAB – Product Tour Page<br />Version A – increased downloads by 87% (97% confidence rate)<br />Authoritative Quote at the...
Which Product Page Got More Purchases?<br />
Version B got an 11% lift in paid purchases. <br />Good example of why ecommerce marketers should not assume “standard” pr...
“Free” Offer vs. Statement – Which Call to Action Button Copy Got a 171% Lift in Clicks?<br />
Contrary to the notion that the word “Free” always wins, Version B convinced 171% more visitors to click the call to actio...
Subscription Site Homepage A/B Test: Which Design Got More Sign-Up Button Clicks?<br />
Version B got 21.3% more clicks on the sign up button than Version A<br />Everything on the pages tested was identical exc...
Which Email Newsletter Opt-In Form Boosted Sign-Ups by 52.8%?<br />
NYPL Email Subscription Form<br />Version A boosted email newsletter sign-ups 52.8%<br />The winning form’s headline said ...
PPC Landing Page Test: Which Got 73.15% More <br />Visitors to Download the Free Toolbar?<br />
Which Landing Page Performed Best<br />Version B was the winner with 73.15% more toolbar downloads<br />However, Version A...
Shopping Cart Page One A/B Test: Which Convinced <br />More Shoppers to Click to the Next Checkout Step?<br />
Which Shopping Cart Performed Best<br />Version A got 40.7% more page viewers to clickthrough to the next page in the chec...
Upcoming SlideShare
Loading in …5
×

IBAT-ContentOptimisation-AbTest

544 views

Published on

IBAT-Content Optimisation & Ab Testing

Published in: Business, Technology, Design
1 Comment
1 Like
Statistics
Notes
  • Pretty much long slideshare, but the point is: A/B testing is neccessary. A tool I suggest: http://www.coreminer.com/calculators/ab-test
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
544
On SlideShare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
11
Comments
1
Likes
1
Embeds 0
No embeds

No notes for slide

IBAT-ContentOptimisation-AbTest

  1. 1. Content Optimisation<br />Which Test Won<br />
  2. 2. HubSpot Tests Homepage Copy: Which Got More Free Trial Downloads – Goal-Focused vs. Role-Focused?<br />
  3. 3.
  4. 4. Version B convinced 49.1% more first-time homepage visitors to download a free trial.<br />Version A tailored copy to two target demographics (business owners and marketers) based on their goals.<br />Version B addressed the demographics based on their identities or roles.<br />If your homepage gets two or more types of visitors, you absolutely should copy this test!<br />
  5. 5. Nonprofit A/B Test: Which Page Got A 10.2% Lift In Average Donation Amounts for Haiti?<br />
  6. 6.
  7. 7. Version A got a 10.2% lift in average donation amounts per page view, contributing a total of $1.02 million<br />Traffic to the page came from a huge (one of the biggest we’ve seen) “Donate” button on the homepage<br />Version A had a two-column layout with a large image vs. Version B’s single-column, no image layout.<br />Neither version displayed the button above the fold, Version A had two less form fields and used “Support Haiti” for its button copy vs. “Submit” on Version B’s button.<br />
  8. 8. Wal-Mart’s UK Chain Tests Radical Homepage Redesign: Which Layout (not Content) Reduced Bounce Rates by Double Digits?<br />
  9. 9.
  10. 10. The Version A layout, featuring non-traditional navigation elements, reduced homepage abandons by an average 19%.<br />Shows that the tabs-across-the-top homepage may not be the best for all ecommerce sites.<br />Sometimes a list of text-links in the center of the page work better. <br />In fact, perhaps other types of sites (even content and B2B sites) should also test alternate navigational elements on their homepages <br />
  11. 11. Ecommerce A/B Test: Which Shoe Page Version Got 39.3% More Add-to-Cart Clicks?<br />
  12. 12.
  13. 13. 39.3% more European 20- to 30-somethings interested in trendy street wear preferred Version A over Version B. <br />The versions were identical except for the background color (black vs. white), copy (“Outstanding.” vs. “Out of the Box.”), and shoe color (rich vs. dull). <br />Possibly, the black background made the product and the add-to-cart button stand out on the page.<br />This is a much bigger product image than most American ecommerce sites use. <br />Image size, alone, is worth testing<br />
  14. 14. Which Product Shots Got More Purchases: Just the Product vs. a Model Wearing the Product (in this case, shirts)? <br />
  15. 15.
  16. 16. Version B’s product shots with human models not only boosted paid purchases 44%, but also gained a 67% rise in revenue per purchase. )<br />Every time we’ve seen ecommerce sites test variations on their product shots — size, background color, and in this case the use of human models — the results have been noteworthy.<br />This test is a fabulous example of why ecommerce sites can’t rely on “standard” images without leaving money on the table. You have to test product images!<br />
  17. 17. Avis UK Tests its PPC Landing Page: Which Got More Car Rental Bookings? <br />
  18. 18.
  19. 19. Version B got a 74% lift in completed car rental bookings (sales)<br />Good example of when a stripped-down, highly-focused landing page can sell a lot more than a standard home page<br />
  20. 20. Which Page Got More Lead Gen Form Submits?<br />
  21. 21.
  22. 22. Version A convinced 66% more visitors to fill out and submit the lead gen form. <br />For the two versions we showed you, the pages used an identical template, aside from the content in the center.<br />Version A’s headline read “Your translation requirements:” whereas Version B had no headline at all.<br />Version A separated the form into required and optional fields, and featured instructional copy in the right column instead of an oddly-colored woman’s face.<br />Lastly, Version A did not have a “back” button (and neither should any form on the Internet we can think of.<br />
  23. 23. PPC Landing Page Test: Which Version Boosted Free Software Sign-Ups By 59%?<br />
  24. 24.
  25. 25. Version A boosted free software sign-ups by 59%… even though visitors had to click through to an additional page to sign up<br />Version B’s very prominent left-side form may have distracted visitors from reading copy that explained the offer<br />Version A emphasized copy so that prospects understood what they were getting before they had to make a final sign-up decision. <br />
  26. 26. A/B Copy Test: Which PPC Landing Page Got 93% More ‘Add-to-Cart’ Clicks? <br />
  27. 27.
  28. 28. Which LP got more “Add to Cart” clicks<br />The more urgent, ‘take control’ tone of Version B’s copy lifted add-to-cart button clicks a whopping 93% over Version A’s comfort-focused tone.<br />Shorter more focused bullet points<br />All traffic came from the same keyword searches, but PPC ad serving was split to test which PPC ad copy and its matching landing page would do best.<br />The 93% lift in data represents just the improvement on ‘add to cart’ button clicks on the landing page, irrespective of PPC ad clickthrough rates.<br />
  29. 29. Homepage Image A/B Test: “Sad” Young Woman vs. “Haggard” House Wife – <br />Which Got More Clicks on the Big Yellow Button?<br />
  30. 30.
  31. 31. Clear Debt – Varied Image – Click Test<br />Version B got 33% more visitors to click the ‘Take the Debt Analyser’ button<br />Most of traffic came from natural search and the results reached a 98.2% confidence level.<br />Version B showed a “haggard house wife” looking towards the button.<br />Version A showed a “sad” young woman looking away from the button<br />Also age and context could have been a factor – users feel empathy with “Mother and Kids” as opposed to a young girl <br />Setting of image – older woman shopping Vs youg woman sitting in “cinema”<br />Interesting to know about “Target Audience” and source – ppc, email etc.<br />
  32. 32. Alpha Ecological Tests Homepage Copy, Images, and Layout: Which Got More Email Leads? <br />
  33. 33.
  34. 34. Version B got a huge 114% lift in email leads from the homepage. <br />Although the winner is obvious, this isn’t an easy test to sell to a HiPPO (highly paid person).<br />The winning page had a lead capture form front and center, images of the actual pest control team, and more benefits copy.<br />It also had images of the pests customers could be dealing with above the fold<br />
  35. 35. Email A/B Test: Which Customer Survey Invite Got 51% More Clickthroughs?<br />
  36. 36.
  37. 37. Which Customer Survey Got More ClickThrus<br />Version B got a whopping 51% more clicks than Version A.<br />Version A read, “Give us your best Campaign Monitor ideas!” Version B read, “Tell us what we can do better.” The more direct wording won.<br />Active language trumps passive language<br />Easier to “Tell” opinionsrather than “Give” ideas away<br />
  38. 38. Fragrance Direct Tests the Layout of its Checkout Page: Which Got More Purchases?<br />
  39. 39.
  40. 40. Version B, with images and info about the shoppers’ cart contents placed at the top of this third page in the checkout process, won a 22.5% boost in purchases<br />Do customers deep into the checkout process still need to see prominent images and info on what they’re buying on each page?<br />Turns out maintaining “scent” or buyer excitement by showing the product as prominently as possible won in this case<br />
  41. 41. Does Size Really Matter When It Comes to Email Opt-In <br />Form Overlays? Please Vote For Which Got More Opt-Ins.<br />
  42. 42.
  43. 43. FaveCrafts.com – Email Test<br />Version A increased email subscriptions by 8.8%<br />WhicTestWon professionals voted <br />34% - Version A<br />66% - Version B<br />Part of A/B/N Test – versions A & B were identical – and out performed the control <br />Form Headline was consistent with keyword search - <br />Smaller More Focused Box possibly led the eye path and controlled user actions<br />Test Carried Out by: WiderFunnel using GWO<br />Site – Favecrafts.com ->> http://www.favecrafts.com/<br />
  44. 44. Which Navigation Bar Copy Boosted Webinar Attendees – “Webinars” vs. “Online Training”?<br />
  45. 45.
  46. 46. Version B’s “Online Training” wording increased clickthroughs from the support page to the webinar page 10.4%.<br />Webinar attendees doubled the first week the winning navigation link went live<br />The team also ran A/B tests replacing “Webinars” with “Live Training” and “MailChimp Training.” In both cases, the word “Webinars” lost!<br />Certain words resonated better with the audience<br />
  47. 47. Which Banner Ad Landing Page Got More Form Submissions … on Tuesday Afternoons?<br />
  48. 48.
  49. 49. National University – Form Submits<br />Version B won on Tuesday, Wed, Thurs PMs by converting 17% More Form Submits<br />Version A won on all Weekday and Weekend AMs by 22%<br />Shows the importance of dayparting<br />Version B – had a Clear header – “Get on the Fast Track to your Degree” (possibly more relevant to a specific cohort – need to check)<br />Version A – heading “Your Career..on your terms” – more relevant to work crowd = morning<br />Colours – Typically Morning = Bright :: Afternoon = Soft <br />
  50. 50. DHL Express Tests Male vs. Female Image: Which Generated 108% More Norwegian Leads on this PPC Landing Page?<br />
  51. 51. DHL Test<br />
  52. 52. Version B, with the female model in the header, convinced 108% more visitors in Norway to fill out and submit the lead gen form.<br />
  53. 53. All PPC traffic was sent to one of these two landing pages. The test results reached a 95% confidence level<br />The only difference between Version A and Version B was the photograph of the model in the page header<br />Version A had a Caucasian male who looked like one of Norway’s local couriers. Version B had an Asian female.<br />Results show the impact of testing people images in your online marketing.<br />Eyetracking studies find that people love to look at human faces — so DHL was smart to highlight the 30% savings below the model’s faces. <br />
  54. 54. Westwood College Tests Adding a Privacy Policy Statement to its Landing Pages: Which Got More Leads?<br />
  55. 55.
  56. 56. Version B, with the privacy policy statement in fine print under the button, convinced 19.1% more visitors to fill out and submit the lead gen form. <br />Privacy reassurances can help raise response rates (but you have to test, because the wording and visibility can help or hurt you.)<br />Gen Y kids really do care about privacy!<br />Don’t assume that because you’re marketing to any particular demographic that privacy is not a concern<br />
  57. 57. The Official 2010 Vancouver Olympic Store Tests One-Page Checkout vs. Four-Page Checkout: Which Got More Purchases? <br />
  58. 58.
  59. 59.
  60. 60. Version A, the one-page checkout, gained a 21.8% lift in actual purchases.<br />Should the checkout process be one page or multiple pages?<br />Results of similar tests are significant but not predictable – so always test<br />
  61. 61. Highlight Step One or Step Two? Which Placement of a Blue Box Increased Google Gadget Installations 50.5%?<br />
  62. 62.
  63. 63. Version B increased gadget installations by 50.5%.<br />Version A featured the blue box around step one, the address box. Version B featured the blue box around step two, the call to action button.<br />Likely that this test reiterates the importance of drawing attention to call to action buttons.<br />Test copy, color, size, or any other way to make it stand out<br />Version A’s blue box seemed to pull attention away from the CTA button, and toward the large example images below. Version B’s blue box balanced page. <br />
  64. 64. Which Landing Page Got More Form Completes? (Both Forms Ask the Same Questions)<br />
  65. 65.
  66. 66. Which LP got more form completes?<br />Version B racked up 16% more form completions, making it the solid winner<br />Version B form is tightly grouped, green background with white field is visually appealing, Orange Button pops off bground.<br />Version A – Button is below the fold. Version B above the fold<br />Version B used a multi-column approach to form (makes seem shorter) than the single column Version A<br />Possibly poor use of Left hand side with clip art image – readers read from LTR – make use of that space<br />Good money image connected directly with big button <br />Need to do MVT test to see which elements affected most<br />
  67. 67. Product Tour Page A/B Test: Which Version Boosted Free Trial Software Downloads by 85%?<br />
  68. 68.
  69. 69. YNAB – Product Tour Page<br />Version A – increased downloads by 87% (97% confidence rate)<br />Authoritative Quote at the Top of Page – “If you need financial discipline – consider YNAB”<br />Large Thumbnails – handwritten notes Vs Small Thumbs requiring click throughs<br />Even though “Download Now” button way down below fold <br />Shows users will scroll through a clear “story” <br />Always consider the way a user “uses’ your page – as opposed to purely aesthetic rationale<br />
  70. 70. Which Product Page Got More Purchases?<br />
  71. 71.
  72. 72. Version B got an 11% lift in paid purchases. <br />Good example of why ecommerce marketers should not assume “standard” product page layout is going to give them the most possible sales<br />Version B feels “calmer” in general won. <br />Price is over button, button at bottom of page, smaller gold review stars, and no cross-sales.<br />
  73. 73. “Free” Offer vs. Statement – Which Call to Action Button Copy Got a 171% Lift in Clicks?<br />
  74. 74.
  75. 75. Contrary to the notion that the word “Free” always wins, Version B convinced 171% more visitors to click the call to action button, which led visitors to a ‘contact us’ form.<br />The test reached a 98.6% confidence level. Traffic to the page was mostly from organic search<br />Everything about the homepage was identical except for the language on the call to action button.<br />Version A’s button copy said “Free Consultation.” Version B’s said “Work With Us.”<br />
  76. 76. Subscription Site Homepage A/B Test: Which Design Got More Sign-Up Button Clicks?<br />
  77. 77.
  78. 78. Version B got 21.3% more clicks on the sign up button than Version A<br />Everything on the pages tested was identical except for removing the top horizontal navigation bar and decluttering the bottom by stripping out content and links<br />The main lesson here is that simplicity often gets more responses, and fewer links can win<br />For landing pages, try getting rid of your nav bar and anything distracting in your footer or extraneous columns.<br />
  79. 79. Which Email Newsletter Opt-In Form Boosted Sign-Ups by 52.8%?<br />
  80. 80.
  81. 81. NYPL Email Subscription Form<br />Version A boosted email newsletter sign-ups 52.8%<br />The winning form’s headline said “Stay up-to-date” vs. “Subscribe” – more specific / better reason to submit<br />Fewer form fields and a shorter privacy policy<br />Featured a thumbnail graphic for the newsletter, a link to preview, and benefits copy<br />Could do a MVT Test to figure out which specific elements caused the greatest uplift – another test…<br />
  82. 82. PPC Landing Page Test: Which Got 73.15% More <br />Visitors to Download the Free Toolbar?<br />
  83. 83.
  84. 84. Which Landing Page Performed Best<br />Version B was the winner with 73.15% more toolbar downloads<br />However, Version A won on an initial A/B test to determine the best page layoutresulting in a 21.13% lift in toolbar downloads<br />Next, a MVT test was run with 16 different test elements – Version B won that with 73.15% increase in downloads<br />
  85. 85. Shopping Cart Page One A/B Test: Which Convinced <br />More Shoppers to Click to the Next Checkout Step?<br />
  86. 86.
  87. 87. Which Shopping Cart Performed Best<br />Version A got 40.7% more page viewers to clickthrough to the next page in the checkout process<br />About 18% of shopping cart views convert to purchase, a 40.7% lift contributed greatly to the bottom line<br />Moving the 100% satisfaction guarantee icon to a more prominent location<br />Adding site security copy and links to the guarantee and testimonials<br />Making a big free shipping offer visible above the product box<br />Changing the main call to action buttons from blend-in to highly contrasting colors<br />

×