1555 track1 goward

Profitable ‘A-ha!’ moments everyday.™

© 2007-2017 WiderFunnel Marketing Inc.
Tweet this: @chrisgoward #Awesome
How to A/B Test Like a Pro
Drive Business-Impacting Insights
from Your Growth Marketing Program

Tweet to: @chrisgoward #Awesome1981

Tweet to: @chrisgoward #Awesome

Var A Var B Var C Var D

Goal: sign up for the
monthly subscription

Hypothesis: Adding a
social inclusion button will
lift orders.
Hypothesis: Adding a sense
of urgency button will lift
orders.
ê 6.5% Signups
ê 0.5% Signups
Var A: Social Signal
Var B: Urgency
Control
Goal: sign up for the
monthly subscription
?

You should be skeptical of so-called
“best-practices”

(Want to know which button worked better?)

23.4% more orders for IBM Softlayer servers

115% more qualified leads for Magento Enterprise

$1 million per month additional revenue for BuildDirect.com

What we hear over and over again...
•  How can we get consistent optimization results?
•  Will we hit an optimization ceiling?
•  How should we respond to losing tests?
•  Will our test results fade over time?
•  How do we build the best growth process?

Analyst

•  Use landing page templates
•  Follow conversion “best practices”
•  Choose tools before strategy
•  No experiments (!)
•  No personalization strategy
•  Misunderstand statistical signiﬁcance
•  Pollute test results with improper
technical setup
•  Little cognitive bias training
•  Unaware of Design Of Experiments
(DOE)

§  New brand & website launch, big marketing spend
§  Result: declining sales!
§  Conversion rates were poor on good quality traﬃc
§  Tie spend to direct sales to know part of the return
§  Focus on Return on Ad Spend (ROAS)
§  Outsource Conversion Optimization for expertise

9,000 pixels
or 9’8” tall

Landing page redesign?
Personalization?
Price anchoring?
Eyeflow?
Button colour?
Headlines?
Testimonials?
Segmentation?
Reduce form fields?
Reduce form fields?
Reduce form steps? Add form momentum?
Long copy landing page?
Short copy landing page?
Create Scarcity?
Use Reciprocity?
Mention Social Proof?
Appeal to Authority?
Font selection?
Urgency wording?
Put your CTA above
the fold?
Include your inbound
keywords in your
headline?
Make your headline action-oriented?
Include testimonials, reviews, awards?
Use specific custom landing pages?
Reduce your outbound links on your pages?
Increase the size of your product images?
Include a photo of a smiling person?
Reduce your product options to max five?
Use high contrast text on white background pages?
Reduce the length of your privacy policy?
Move your SSL secure icons below the fold?
Embed lead gen forms on your landing pages?
Emphasize features over benefits?
Add ‘action captions’ below images?
Include filter and sorting on category pages?
Reduce your page columns to max two?
Use video product overviews?
Use a grid view on category page templates?
Always use orange CTA buttons?
Always use the word ‘Free’ somewhere?Always use the word ‘Free’ somewhere?
Put your CTA above the fold?
ys use the word ‘Free’ somewhere?
ed lead gen forms on your landing pages?
uce your outbound links on your pages?
Always use the word ‘Free’ somewhere?
Use video product overviews
Reduce your outbound links on your page
Make your headline action-oriented
Reduce your outbound links on your pages
lways use the word ‘Free’ somewhere?
Use video product overview
Embed lead gen forms on your landing pages
Reduce your outbound links on you
Embed lead gen forms on your landing p
Reduce your page columns to max tw
Make your headline action-orien
the word ‘Free’ somewhere?
ke your headline action-oriented?Put your CTA above the fold?
use the word ‘Free’ somewhere?
o product overviews?
Embed lead gen forms on your landing page
r outbound links on your pages?
your CTA above the fold?
Reduce your outbound links on your pages
lways use the word ‘Free’ somewhere?
Embed lead gen forms on your landing pages
Reduce your outbound links on you
ke your headline action-oriented?Put your CTA above the fold?
r outbound links on your pages?
word ‘Free’ somewhere?
ce your page columns to max two?
your headline action-oriented?
above the fold?
oduct overviews?
forms on your landing pages?
utbound links on your pages?
ur CTA above the fold?
our CTA above the fold?
se the word ‘Free’ somewhere?
somewhere?
se video product overviews?
n forms on your landing pages?
r page columns to max two?
ne action-oriented?Put your CTA above the fold?
iews?
s on your pages?
ke your headline action-oriented?
e the fold?
Use video prod
Embed lead gen forms on yo
Reduce your outboun
Reduce your outbound links on your pag
Make your head
Always use the word ‘Free’ somewhe
Use vid
Embed lead gen forms o
Reduce your o
Reduce your page colum
Make your headline acti
Always use the word ‘Free’ somew
Embed lead gen for
Reduce your outbound links on y
Reduce your pag
Make your
Put your CTA above the
Embed lead gen forms o
Reduce your page columns to m
Reduce your outbound links on your page
Make your headline action-oriented
Reduce your outbound links on yo
Embed lead gen forms on your landing
Make your headline action-orie
ake your headline action-oriented?Put your CTA above the fold?
eo product overviews?
ur outbound links on your pages?
t your CTA above the fold?
e word ‘Free’ somewhere?
uce your page columns to max two?
your headline action-oriented?
A above the fold?
product overviews?
n forms on your landing pages?
our CTA above the fold?
e the word ‘Free’ somewhere?
eo product overviews?
’ somewhere?
en forms on your landing pages?
ur page columns to max two?
ne action-oriented?Put your CTA above the fold?
views?
ks on your pages?
ake your headline action-oriented?
ve the fold?
Use video pro
Embed lead gen forms on y
Reduce your outboun
Make your headline action-orie
Reduce your outbound links on your pa
Make your head
Always use the word ‘Free’ somewh
Use vi
Embed lead gen forms
Reduce your
Reduce your page colu
Make your headline ac
Always use the word ‘Free’ somew
Embed lead gen fo
Reduce your outbound links on
Reduce your pa
Make your
Put your CTA above the
Embed lead gen forms on your landing
Embed lead gen forms
Reduce your page columns to
Tips & tricks are dead ends.

Common mistake:
Implementing “gut feeling” ideas
without framework thinking
or experiment validation.

They need a total website redesign, right?

Happiness with latest website redesign
Extremely Not so much
24%
76%

They needed:
A growth marketing program that produces
ongoing and proven sales lift.

Expert
optimization
team
Proven
optimization
process
Meaningful
customer
insights
The three success factors

What makes a great
optimization process?

Proven
Quantitative
Logical
Solid
Intuitive
Qualitative
Inspired
FuzzyYang
Yin

The best optimization process
delivers inspiration and proof.

Infinity Optimization Process

Explore

Explore
The LIFT Model helps
you look at marketing
touchpoints through
customers’ perspective.

TM
LIFT Model

CLARITY: Based on scroll and
heuristic analysis, the ﬁrst
block of content created a
false bottom.
CLARITY: White/black text on
dark purple hard to quickly
read.
VALUE PROPOSITION: Crisp,
clear value proposition
messaging.
DISTRACTION: Overwhelming
content creates cognitive load.

“Cognitive strain
makes people wary
of their actions.”
– Daniel Kahneman

source: NNGroup
Decreased conﬁdence
in one’s actions/choices
Decreased trust
in the site/application
Decreased likelihood
of taking desirable actions

Variation A Variation BControl
?

Clear segmentation
module.
Redesign Variation A
Best-selling products move
higher on the page.

Redesign Variation B
Tabs provide more info
and data on engagement.
Segmenting options to
diﬀerentiate home and
business shoppers.

Variation A Variation BControl
é15.8% Sales é41.4% Sales
?

Common mistake:
Assuming A/B testing only works for
“tweaking & tuning”

Conversion optimization can create
dramatic and fast design improvements

Video VariationNew Control
é153.6% Video views
ê10.6% Sales

Get a free chapter at www.YouShouldTestThat.com

Validate
Analyzing results often
leads to surprising
insights for future
experiments.

Validate
Hypothesis:
Reducing prominence
of videos will increase
sales.

ê 29.4% Video views
Control Variation A
é 33% Sales

Common mistake:
Missing powerful insights from
the losing experiments

The goals of experimentation are:
Growth + Insights

Common mistake:
Wasting resources on insignificant tests.

Prioritizing your experiments is your
first critical decision.

Validate

The PIE™ Prioritization Framework
1.  Potential
2.  Importance
3.  Ease

Zone Potential Importance Ease
PIE
Score
Homepage 10 10 8 9.3
Checkout 8 10 9 9.0
Web analysis
Heuristic analysis
Voice of customer
Technical
“Political”
Cost
Traﬃc volume
Return on investment

Zone Potential Importance Ease
PIE
Score
Homepage 10 10 8 9.3
Checkout 8 10 9 9.0
Product page 10 9 7 8.7

How’s WeBoost doing now?

Before After

104% conversion rate lift (e.g. doubled)
215% revenue lift across entire website
71% increase in site traffic
400%+ increase in ad impressions
Business Results:

Common mistake:
Viewing experimentation in isolation.

It’s a way to make the entire
digital advertising platform for a brand
more effective and profitable.
Conversion Optimization
is not just about website sales

The Situation When We Started
•  High Volume website with a one-person optimization team
•  Mish-mash of ad server technology, Optimizely, and Adobe Analytics for
A/B testing
•  “Hunt and peck” optimization planning
•  4 - 8 tests per month
•  Little conﬁdence in test results
•  Unknown optimization opportunity

Clarity: Repeats previous step headline
Clarity: Low button prominence
Distraction: Too many colors
Distraction: Gradient background
Distraction:
Three column
layout adds
complexity
Distraction: Too
much content for
a simple action
Anxiety: Unable
to select multiple
providers at once

Control Variation A
Urgency subhead
Variation B
Choice Isolation (vs A)
Variation C
Lovability Isolation (vs B)
?

Control Variation A
Urgency subhead
Variation B
Choice Isolation (vs A)
Variation C
Lovability Isolation (vs B)
- 9.7%
+ 19.2% + 11.8%

Profitable ‘A-ha!’ Moment

Where do these ideas come from?
The process generates an unending stream of them.

Explore
We have nearly 200
cognitive biases (so far)
to test in varying
situations.

The “choice is yours” effect
A Meta-Analysis of the Eﬀectiveness of
the “But You Are Free” Compliance-
Gaining Technique, 2012
– Christopher J Carpenter
Analysis of 42 studies involving 22,000
participants.
Insight: Telling the person they have the
freedom to choose nearly doubled sales
success rates.

TM
LIFT Model
(Note: For more info, google WiderFunnel Lift )

Common mistake:
Viewing your business from
your own perspective.

Gain insights from the
Voice of your Customer

Feedback Polls

Post-purchase surveys

é 23% Sales é 46% Sales

+7.34% Revenue Lift

The LIFT Model™
(Note: For more info, google WiderFunnel Lift )
Anxiety
Distraction
Relevance
Clarity
Urgency

Micro-survey
Qualify Social Proof focus
Control Variation A
Variation B Variation C
?

Micro-survey
Qualify Social Proof focus
Control Variation A
Variation B Variation C
+ 14.0%
No Change + 40.39%

Social Proof (Bandwagon Effect)
We tend to put our trust in
something because we
believe many other people
have put their trust in it
ﬁrst.
There is safety in numbers.

The Bandwagon Effect works
For this audience and context!

Bandwagon Effect

You can’t use all the tactics all the time.

0.0% ConversionRate

Self-consistency Bias

Self-consistency Bias
We like to keep consistent with
what we think, say and do, and will
change to ensure this is so.
- Cialdini, Inﬂuence.
(2006)

Cognitive Dissonance
Cognitive Dissonance is mental stress
experienced when a person holds
contradictory beliefs, or acts in
contradiction to existing beliefs.
- Leon Festinger
(1957)

Conversion Rate: +1.97%
Average donation value: +3.07%
Revenue per visitor: +5.10%

14% of visitors
68% of donors
86% of visitors
32% of donors
Selected
“Regular Donor”
Selected
“Occasional Donor”
Selected
“First Time Donor”
Non-Responders

Now, we’re building on these insights to deliver
Personalized data-informed experiences.

0%
18.8% 0% 11.8% 0% 0%
1 2 3 4 5 6
Six months of conversion optimization
32.8%

0%
18.8% 0% 11.8% 0% 0%
1 2 3 4 5 6
Six months of conversion optimization
32.8%36.8%

0%
18.8% 0%
11.8% 0% 0%
11.4% 0% 0% 0%
19.7%
13.4%
1 2 3 4 5 6 7 8 9 10 11 12
12 months of conversion optimization
Ouch!
100.8%

0%
18.8% 0%
11.8% 0% 0%
11.4% 0% 0% 0%
19.7%
13.4%
1 2 3 4 5 6 7 8 9 10 11 12
Ouch!
100.8%123.7%

18.8%
11.8%
11.4%
19.7%
13.4%
7.3%
15.9%
12.8%
27.7%
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
259.8%

18.8%
11.8%
11.4%
19.7%
13.4%
7.3%
15.9%
12.8%
27.7%
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
259.8%282.2%

Here’s where DMV.org is now
•  The right tech – with Adobe, Optimizely, Liftmap integrated
•  The right process – using Inﬁnity Optimization Process
•  The right team – integrated DMV.org and WiderFunnel teams
•  The right pace – now running 500+ tests per year
•  The right results – revenue doubled four years in a row

Become an optimization champion
Get the new 32-page
“Optimization Champion’s Handbook”
Send an email to: iwant@WiderFunnel.com
Subject: Champion Handbook Please!

1555 track1 goward

More Related Content

What's hot

Similar to 1555 track1 goward

More from Rising Media, Inc.

Recently uploaded

1555 track1 goward