This document outlines Brad Geddes' framework for testing Google ads. It recommends beginning with multi-ad group testing to answer broad questions across ad segments. Key insights from this testing should inform single ad group A/B testing of top-performing ad groups. Mobile testing and image ad testing are also discussed. The framework stresses an ongoing cycle of big question identification, broad testing, analysis, and repeated focused testing.

1. Brad Geddes
Founder, Certified Knowledge
Founder, AdAlysis
Author, Advanced Google
AdWords
@bgTheory
Creating an Ad Testing
Framework
MnSearch Summit
@bgTheoryCOPYRIGHT CERTIFIED KNOWLEDGE 2014

2. Creating an Ad Testing Framework
Single vs Multi-Ad Group Testing
@bgTheoryCopyright AdAlysis

3. @bgTheoryCOPYRIGHT ADALYSIS 3
A/B Testing – Known as Single Ad Group
Testing

4. @bgTheoryCOPYRIGHT ADALYSIS 4
Multi-Ad Group Testing
Idea 1
Idea 2
Idea 3
Idea 4
Idea 5

5. @bgTheoryCOPYRIGHT ADALYSIS 5
Use Cases
Single Ad Group Testing
- Insights within an ad
group
- Best for high traffic ad
groups
- Best for Brand ad groups
- Good for other ad
groups after you’ve done
some multi-ad group
testing for global
insights
Multi-Ad Group Testing
- Insights across a
segment
- Best for template
created ads
- Best for small data
accounts
- Best for market research

6. Creating an Ad Testing Framework
The Brainstorm
@bgTheoryCOPYRIGHT CERTIFIED KNOWLEDGE 2014

7. @bgTheoryCOPYRIGHT CERTIFIED KNOWLEDGE 2014 7
Research Your Initial Ad Ideas
Ad Ideas Inspirations
• Social mentions
• Customer support inquiries
• Your forums
• Company reviews
• Competitive research
• Brainstorming
• Previous Marketing Efforts
• Customer onboarding interview
• Search query data
• Etc…

8. • Are consumers price sensitive? (discounts,
prices in ads)
• Does Location matter? (use geographies in
ads)
• Does shipping matter? (Free or Rush shipping)
• Is selection important? (large, exclusive,
variety, etc)
• How much does quality matter? (cheap, luxury)
• What’s your best CTA? (Call, Shop, Find,
Discover, etc)
• Should your display URL be products, geo,
CTAs, or something else?
• Do we need to pre-qualify clicks? (b2b vs b2c)
@bgTheoryCopyright Certified Knowledge 2014 8
Ask The Big Questions First

9. @bgTheoryCOPYRIGHT CERTIFIED KNOWLEDGE 2014 9
Optional: Spreadsheets, Keyword Density,
Tag Clouds

10. @bgTheoryCopyright AdAlysis 10
Determine How You Want to Segment
Ad Tests
Basic Segments:
• Your brand
• 3rd party brands
• Products
• Informational
• Long tail
• By buying cycle entrance
• Etc…
Ad Type:
• Text ads
• DSAs
• Image ads
• Call only ads
• HSAs
• etc…
Devices:
• Mobile
• Desktops
• All

11. Creating an Ad Testing Framework
Start With Multi-Ad Group
Testing
@bgTheoryCOPYRIGHT CERTIFIED KNOWLEDGE 2014

12. • Examples:
GEO vs Non-GEO headlines
Call us vs Learn More CTAs
Brand vs Non-Brand Headlines
Rush vs Free Shipping
@bgTheoryCOPYRIGHT CERTIFIED KNOWLEDGE 2014
List Out Your Ad Tests
12

13. @bgTheoryCOPYRIGHT CERTIFIED KNOWLEDGE 2014
Multi-Ad Group Testing Setup
Ad Group D2
1 Rush Shipping!
1 Free Shipping!
1 Price $
2 Rush Shipping!
2 Free Shipping!
2 Price $
3 Rush Shipping!
3 Free Shipping!
3 Price $
Etc… Etc…
13

14. @bgTheoryCOPYRIGHT CERTIFIED KNOWLEDGE 2014 14
(Optional) Label Ads for Easy Filtering

15. @bgTheoryCOPYRIGHT CERTIFIED KNOWLEDGE 2014 15
Use Pivot Tables or Software to Analyze
the Data
Idea 1
Idea 2
Idea 3
Idea 4
Idea 5

16. •Don’t use brand names in non-branded ad
groups
•GEOs increase CPI; break out campaigns by
GEO
•Call Us works best on a mobile device
•Learn more works best on a desktop device
•Etc…
@bgTheoryCopyright Certified Knowledge 2014 16
You Now Have Answers to Big
Questions

17. @bgTheoryCopyright Certified Knowledge 2015 17
If a “Losing Line” is an Outlier for
CTR/CPA:
Test it as an ad extension

18. •If you have more big questions to answer; do
more rounds of multi-ad group testing
•If not; then move to A/B testing
Easiest to start with your top ad groups (by
impressions)
@bgTheoryCOPYRIGHT CERTIFIED KNOWLEDGE 2014 18
Next Steps

19. Creating an Ad Testing Framework
Quick Case Study
@bgTheoryCOPYRIGHT CERTIFIED KNOWLEDGE 201419

20. @bgTheoryCOPYRIGHT CERTIFIED KNOWLEDGE 2014 20
Using Your Customer’s Words

21. @bgTheoryCOPYRIGHT CERTIFIED KNOWLEDGE 2014 21
Rush Shipping Vs Fast Shipping
Ad CTR Conv
Rate
CPI Avg
Order
Fast &
Free
Shipping
4.96% 4.22% 0.209% $71.32
Rush
Shipping
5.13% 4.32% 0.225% $357.9
1

22. @bgTheoryCOPYRIGHT CERTIFIED KNOWLEDGE 2014 22
Segmenting by Consumer Type: B2B
vs B2C
Type Ad CTR Conv Rate RPI
B2B Fast & Free
Shipping
2.31% 3.46% 0.286328
B2B Rush Shipping 5.47% 5.30% 1.037939
B2C Fast & Free
Shipping
4.02% 4.73% 0.137408
B2C Rush Shipping 1.96% 3.05% 0.043392

23. •Segment by B2B vs B2C
For B2B – use ‘rush shipping’
For B2C – use ‘free shipping’
Next Steps:
•Examine the branded terms, RLSAs and
remarketing lists and see if there’s a good way
to match the shipping type to the ads.
•Can we change the landing page to match the
shipping?
Test it to see if it makes a difference
@bgTheoryCopyright Certified Knowledge 2014 23
You Now Have Answers to Big
Questions

24. @bgTheoryCOPYRIGHT CERTIFIED KNOWLEDGE 2014 24
Outlier Data is Good for Extension
Testing

25. @bgTheoryCOPYRIGHT ADALYSIS 25
Campaign Changes w/ New
Extensions
Pre-Extension Changes Post-Extension Changes

26. Creating an Ad Testing Framework
Mobile Ad Testing
@bgTheoryCOPYRIGHT CERTIFIED KNOWLEDGE 2014

27. @bgTheoryCOPYRIGHT ADALYSIS 27
You Can’t Ignore Mobile

28. @bgTheoryCOPYRIGHT ADALYSIS 28
Google Often Changes How Ads Are
Displayed on Mobile

29. @bgTheoryCOPYRIGHT ADALYSIS 29
Mobile Preferred Ads

30. @bgTheoryCOPYRIGHT ADALYSIS 30
Easy Effective Device Analysis
Count of
Ad
Row Labels Mobile All
Ad Group
1 2
Ad Group
2
Ad Group
3 1 2
Ad Group
4 3 1
Ad Group
5 10
Ad Group
6 2 6
Ad Group

31. @bgTheoryCOPYRIGHT CERTIFIED KNOWLEDGE 2014 31
If on Mobile: Segment Ideas by Device
Mobile CTR Desktop CTR
Call Us 11.2% (includes
calls)
5.2%
Contact Us 4.1% 6.9%
Example.com/Things 4.2% 5.3%
M.example.com/things
(or
example.com/mobile)
8.2% N/A
Find the Nearest Store 7.6% 4.5%
Shop Now 2.1% 5.8%

32. •Simple option:
Just flip d1 & d2 and test as usual
•Start with multi-ad group testing for CTA and
other minor changes
•For single ad group testing, start with your top
mobile ad groups
Segment by device and start with most impressions
on mobile devices
•If you’re ignoring mobile, remove -100% bid
modifiers on brand terms and start with your
brand keywords
@bgTheoryCOPYRIGHT ADALYSIS 32
Getting Started With Mobile Testing

33. Creating an Ad Testing Framework
Image Ad Testing
@bgTheoryCOPYRIGHT CERTIFIED KNOWLEDGE 2014

34. @bgTheoryCOPYRIGHT CERTIFIED KNOWLEDGE 2014 34
What’s Your Theme?
• List Your Themes
• Create a label per theme
• Examine how each theme performs
• Pick your winning theme and continue

35. @bgTheoryCOPYRIGHT CERTIFIED KNOWLEDGE 2014 35
Test Your Main Themes in Contrasting
Colors

36. @bgTheoryCOPYRIGHT ADALYSIS 36
Learn the Pivot Table Analysis
Note: Old AdWords UI – Analysis is still the same
https://youtu.be/0IBbNO0v-8c

37. •Start with a big brainstorm / research session
to gather ad ideas
•Determine your ad testing segments
•Ask the ‘big questions’ first through multi-ad
group testing
•Use that data to create rules around ad creation
•Move to A/B testing on your top ad groups
Test mobile as appropriate
•Repeat forever
@bgTheoryCOPYRIGHT ADALYSIS 37
Recap

38. Founder, Certified Knowledge
Co-Founder, AdAlysis
Author, Advanced Google
AdWords
Brad Geddes
Thank You! MnSearch Summit
@bgTheoryCOPYRIGHT CERTIFIED KNOWLEDGE 201538
Let’s Connect
@bgTheory
in/ewhisper Powerful Ad Testing Made Si
Ad Testing
Software
Learn PPC
PPC Training & Tools