Ad Testing Research & Findings
What are the Keys to Successful Testing?

•   Design your test around a single question
•   Have a large differentiation b...
Test 1: Is Running Ads on Brand Queries a
Waste of Money?
Example SERP w/o Ad   Example SERP w/ Ad




                   ...
Test Design

•   1 week ads on 1 week off
•   No issues of seasonality or promotion
•   30 Branded Keywords
•   1 version ...
Did the Ads Drive Traffic?




                     NO
               • Google -1.3%
               • Yahoo -8.7%
        ...
Did the Ads Increase Conversion Rate?




                  YES
             • Google +17.9%
             • Yahoo +32.1%
 ...
Did the Ads Increase RPV?




                  YES
             • Google +15.4%
             • Yahoo +47.0%
             ...
New Questions

• Right Time? Marketing to Intent?
• Power of Persuasion?
• Power of Brand in Search?




                 ...
Test 2: What is the Best Ad to Increase
Conversion?
       Ad Testing                          Element 1
       Multivaria...
Test Design

• 9 Ads Created and Tagged w/ URL Parameter
• Landing and Thank You Page Tagged
• Ad Serving Optimization Tur...
What does an L9 Array Really Look Like?
  Recipe (9)    Title (3)             D1 (3)                       D2 (3)         ...
What Were the Results?
 Recipe        Visitors    Conversion   Lift
 A - Control   11.80%      5.88%        ---
 B        ...
What is the breakdown?

• Other Ads the Used Winning URL
   – Control
   – +4.72%
• Other Ads the Used Winning D1
   – +3....
What Ad Elements Influenced Conversion?
  Element   Winning Alternative            Influence on
                          ...
What’s Better?

  Winning Recipe                  Best Predicted

{keyword} catalog            {keyword} suppliers
Find pa...
What’s Better?

  Winning Recipe                  Best Predicted

{keyword} catalog            {keyword} suppliers
Find pa...
Test 3: What are the effects of Quality Score?




                      16
Who & What Determines Relevance?
                Recipe A-C
                Title: “{keyword}”

     Recipe     Impression...
Who & What Determines Relevance?
                Recipe D-F
                Title: “ Find {keyword}”

     Recipe      Imp...
Who & What Determines Relevance?
                Recipe G-I
                Title: “The Wait is Over”

     Recipe       I...
Who Decides Relevance?

  Group G-I had
• 17% of the Impressions of A-C
• 32% of the Conversions of A-C
• 82% higher Conve...
Jonathan Mendez, Chief Strategy Officer

                   Thanks!
      • optimizeandprophesize.com - My Blog
      • ot...
Jonathan Mendez, Chief Strategy Officer

                   Thanks!
      • optimizeandprophesize.com - My Blog
      • ot...
Upcoming SlideShare
Loading in...5
×

Ad Testing Research and Findings - SES NYC 2007

2,086

Published on

Published in: Economy & Finance, Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,086
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Ad Testing Research and Findings - SES NYC 2007

  1. 1. Ad Testing Research & Findings
  2. 2. What are the Keys to Successful Testing? • Design your test around a single question • Have a large differentiation b/w tested elements • Account for changes in temporal behavior • Collect enough data to reduce margin of error • Segment results to provide added insight • Act on the results! 2
  3. 3. Test 1: Is Running Ads on Brand Queries a Waste of Money? Example SERP w/o Ad Example SERP w/ Ad 3
  4. 4. Test Design • 1 week ads on 1 week off • No issues of seasonality or promotion • 30 Branded Keywords • 1 version of ad 4
  5. 5. Did the Ads Drive Traffic? NO • Google -1.3% • Yahoo -8.7% • Live -7.3% 5
  6. 6. Did the Ads Increase Conversion Rate? YES • Google +17.9% • Yahoo +32.1% • Live +11.7% 6
  7. 7. Did the Ads Increase RPV? YES • Google +15.4% • Yahoo +47.0% • Live +27.5% 7
  8. 8. New Questions • Right Time? Marketing to Intent? • Power of Persuasion? • Power of Brand in Search? 8
  9. 9. Test 2: What is the Best Ad to Increase Conversion? Ad Testing Element 1 Multivariate Testing Works Element 2 Increase Conversion Rates Element 3 ottodigital.com/ads Element 4 4x3 L9 Orthogonal Array 4 Elements 3 Variations 9
  10. 10. Test Design • 9 Ads Created and Tagged w/ URL Parameter • Landing and Thank You Page Tagged • Ad Serving Optimization Turned OFF 10
  11. 11. What does an L9 Array Really Look Like? Recipe (9) Title (3) D1 (3) D2 (3) URL (3) A - Control {keyword} Search {AdGroup} Compare Products www.brand.com suppliers Today B {keyword} Find {keyword} suppliers Search Thousands of www.brand.com/ Products {keyword} C {keyword} Find Parts fast at (brand) {keyword} catalog www.brand.com/ search catalogs D {keyword} suppliers Search {AdGroup} Compare Products www.brand.com/ suppliers Today catalogs E {keyword} suppliers Find {keyword} suppliers Search Thousands of www.brand.com Products F {keyword} suppliers Find Parts fast at (brand) {keyword} catalog www.brand.com/ search {keyword} G {keyword} catalog Search {AdGroup} Compare Products www.brand.com/ suppliers Today {keyword} H {keyword} catalog Find {keyword} suppliers Search Thousands of www.brand.com/ Products catalogs I {keyword} catalog Find Parts fast at (brand) {keyword} catalog www.brand.com search 11
  12. 12. What Were the Results? Recipe Visitors Conversion Lift A - Control 11.80% 5.88% --- B 14.30% 4.29% -27.05% C 14.04% 6.08% 3.42% D 9.34% 2.57% -56.29% E 13.00% 6.16% 4.72% F 9.42% 5.38% -8.50% G 10.06% 5.04% -14.32% H 10.41% 5.13% -12.82% I 7.63% 8.04% 36.71% 12
  13. 13. What is the breakdown? • Other Ads the Used Winning URL – Control – +4.72% • Other Ads the Used Winning D1 – +3.42% – -8.5% • Other Ads the Used Winning Title – -14% – -12% 13
  14. 14. What Ad Elements Influenced Conversion? Element Winning Alternative Influence on Conversion Title {keyword} suppliers 13.54% D1 Find {keyword} suppliers 53.53% D2 Search Thousands of Products 23.56% URL 9.37% www.brand.com/{keyword} 14
  15. 15. What’s Better? Winning Recipe Best Predicted {keyword} catalog {keyword} suppliers Find parts fast at (brand) Find {keyword} suppliers {keyword} catalog search Search Thousands of Products www.brand.com www.brand.com/{keyword} 15
  16. 16. What’s Better? Winning Recipe Best Predicted {keyword} catalog {keyword} suppliers Find parts fast at (brand) Find {keyword} suppliers {keyword} catalog search Search Thousands of Products www.brand.com www.brand.com/{keyword} Best Predicted +52% Lift in Conversion Rate 15
  17. 17. Test 3: What are the effects of Quality Score? 16
  18. 18. Who & What Determines Relevance? Recipe A-C Title: “{keyword}” Recipe Impressions Conversion Rate Recipe A 265,909 5.72% Recipe B 265,045 5.09% Recipe C 264,784 5.49% 17
  19. 19. Who & What Determines Relevance? Recipe D-F Title: “ Find {keyword}” Recipe Impressions Conversion Rate Recipe D 47,093 8.59% Recipe E 47,289 5.24% Recipe F 47,066 7.76% 18
  20. 20. Who & What Determines Relevance? Recipe G-I Title: “The Wait is Over” Recipe Impressions Conversion Rate Recipe G 46,518 9.84% Recipe H 46,795 8.11% Recipe I 46,579 11.76% 19
  21. 21. Who Decides Relevance? Group G-I had • 17% of the Impressions of A-C • 32% of the Conversions of A-C • 82% higher Conversion Rate than A-C 20
  22. 22. Jonathan Mendez, Chief Strategy Officer Thanks! • optimizeandprophesize.com - My Blog • ottodigital.com/blog - OTTO Blog • offermatica.com - Technology & Parent 21
  23. 23. Jonathan Mendez, Chief Strategy Officer Thanks! • optimizeandprophesize.com - My Blog • ottodigital.com/blog - OTTO Blog • offermatica.com - Technology & Parent 21

×