A WordStream & KISSmetrics Webinar: A/B Testing; You’re Most Likely Doing It Wrong

FROM THE CLICK AND…
BEY ND
A/B Testing; You’re Most Likely Doing It Wrong
Jess Armstrong & Tomasz Borys
06.11.2015
Copyright 2015 WordStream, Inc. & KISSmetrics All rights reserved.

• Customer Onboarding Specialist
• UMass Amherst- Economics
• Foodie- Traveler- Gymnast
@Jess_Armstrong_
About
Jess

What Makes a Worthwhile Test?
Creating a Fail-Safe A/B Test
3 A/B Test to Consider
A/B Testing in PPC

Why PPC?
• No Cost Set Up
• The Data You need is already there!
• High Commercial Intent
LIVE WEBINAR#BeyondTheClick

Ignore the Trivial Things!
Tiny Optimizations are a waste of time!

Ignore the Trivial Things!
Gains from small changes tend to
flatten out over time.

Fix Obvious Problems!

The Mountain Analogy
Goal to get here
You are
here

Aim for results
that are 2-3x
Higher than your
baseline, not 2-3%
Team
Effort!

Define Success
There are so many
metrics in AdWords, You
need to prioritize ahead
of time!

Which One Wins?
0
5
10
15
20
25
30
35
40
45
50
Week 1 Week 2 Week 3 Week 4 Week 5 Week 6
Total conversions by Week
Control Test
0
10
20
30
40
50
60
Week 1 Week 2 Week 3 Week 4 Week 5 Week 6
Average CPA by Week
Control Test
Conflicting Winners!?

Quantity vs. Quality
Example

Save Yourself the Frustration

Set a Timeline and
Stick to It

Set Up a Safety Net
• Remarketing Campaigns
• Catch lost conversions missed
during your test
• Users are 2x more likely to
convert after seeing your ad 7+
times

1. Change Your Call to Action
• This is a Business Decision
• Ads and Landing Pages
• What makes them click and
convert?
• Collaborate with other teams

2. Test Mobile Strategies
Some Business see huge increases in
conversions on Mobile!

….Others don’t.

Things to consider:
• Mobile Bid Modifier
• Mobile Text-Ads
• Call Extensions
• Mobile Friendly Landing Pages
• Set up an experiment in AdWords
• 50/50 impression share

3. Change Your Flow
• Think about where on the funnel people are
falling off and change it

Key Take-Aways
 Think Big! Aim for 2-3x conversion rate
 Set Clear Goals and Priorities in advance
 Capture lost conversions with remarketing
 Tests to try:
o Change Call-to-Action
o Test Mobile Strategies
o Change the Conversion Flow

Tomasz Borys
Director of Marketing
KISSmetrics
@tbcali

How We A/B Test at KISSmetrics
• Set a Goal
• Baby Steps
• Aim for Statistical Significance
• Never Lose Sight
• Blind Eye for 1 Week

Baby Steps
People want to move the
needle and see results fast, by
implementing multiple changes
at once.
But how can you be sure what
element had an impact?

Aim for Statistical Significance
SET A GOAL
Aim
for Statistical
Significance
getdatadriven.com

LIVE WEBINAR
Blind Eye
for 1 Week
Data can be very erratic the
first several days, so it’s
easy to hit the panic button
#BeyondTheClick

Never Lose
Sight
We can caught up on
what we think is best
to optimize
conversions or
traffic…but data
doesn’t lie

A/B Testing Beyond Click Conversions

Original

Variant

Signups

Signup Install JS
Custom
Data
Opps

Signups
Installed JS

Signups
Install JS

Custom Data

Custom Data
Opportunities Created

Social Ad
Funnel

Keeping a Pulse on Your Funnel

Signup Install JS
Custom
Data
Opps
LIVE WEBINAR

Ways of Improving Funnel
• Find the Bottleneck
• Learn why there is a Bottleneck

Learn Why There is a Bottleneck
• A/B Testing
• Surveys and Reach
• Usability Testing

@tbcali
@ Jess_Armstrong_
Questions?

A WordStream & KISSmetrics Webinar: A/B Testing; You’re Most Likely Doing It Wrong

More Related Content

What's hot

Similar to A WordStream & KISSmetrics Webinar: A/B Testing; You’re Most Likely Doing It Wrong

More from Internet Marketing Software - WordStream

Recently uploaded

A WordStream & KISSmetrics Webinar: A/B Testing; You’re Most Likely Doing It Wrong