Conversion World 2015 Keynote by Ton Wesseling

C o n v e r s i o n W o r l d K e y n o t e , A p r i l 2 0 t h 2 0 1 5
To n W e s s e l i n g – C E O – Te s t i n g . A g e n c y
# C R O w o r l d - @ To n W
Growth by Online Experiments

Email: ton@t esting.agency!
Linkedin: t on wesseling!
Tw it ter: @tonw!
!
Ton
Wesseling!

Ton Wesseling | @ TonW
Growth by Online Experiments #CROworld
Online Optimizers

Online Dialogue

Testing.agency

Conversion Conferences worldwide

ConversionHotel.com

The 1st Conversion World

Conversion World speakers

More experiments
“If you double the number of
experiments you do per year
you’re going to double your
inventiveness”

Jeﬀ Bezos, CEO Amazon

Like Amazon, Like Zalando
GermanEcommercemarket,thankyouAndréMorys

Experiments per year
1000+

Why not?
Compare to the past?

Why can this lead to failures?

A big weekly change!

A big weekly change!
26 out of 44 weeks: over 5% change!

Methods used to improve conversion

Methods companies plan to use

It’s just
HOT

Lack of resources

Test more

Importance of a structured approach

A structured approach

FA C T & A C T
&
Tell
Conclude
Analyze
Test
Create
Analyze
Find

The negative effect trend
Time span!

Time span!
Moneyspendon!
Optimizationworldwide!

Time span!
Averageeffectofoptimzation!
atcompaniesapplyingoptimization!
Moneyspendon!
Optimizationworldwide!

It’s not a solution
It’s a method

A/B-testing is a method

It’s a way of working
It’s DNA

So it’s a method:
When
How
What

Growth curve – how it was teached

Optimization phases - ROAR
Time span!
Conversionspermonth!

Time span!
Risk!

Time span!
Risk! + Optimization!

Time span!
Risk! + Optimization! + Automation!

Time span!
Risk! + Optimization! + Automation! Re-think!

Time span!
1.000 conversions!
per month!

Time span!
10.000 conversions
per month!
1.000 conversions!
per month!

Phase 1: bridge the gap – Risk

Fake adwords testing
Do they
click
on your idea?

Fake facebook testing
Do they
like
your idea?

Fake landing page testing
Do they
subscribe
to your idea?

Get out of the ofﬁce

Risk

ROAR - Optimization
Time span!
1.000 conversions!
per month!
10.000 conversions
per month!

Low number testing?
Why can it hurt you?

Warning: low signiﬁcance levels
ü Big chance the winner is not a real winner

Warning: low power levels
ü Big chance a real winner is not recognized

Example test set-up

Example test set-up
ü 90% significance level

Example test set-up
ü 100 conversions per variation

Example test set-up
ü Average conversion rate: 2%

Example test set-up
ü 20 out of 100 ideas is a winner…

Example test set-up
ü 20 out of 100 ideas is a winner…
ü …with an average uplift of +- 10%

Average: 75% signiﬁcance & 40% power

So:

So:
ü 100 tests will give 10 false positives (90% significance)

So:
ü 40% of the 20 winners are recognized(40% power)

So:
ü 8 real winners & 10 false positives

So:
ü 8 real winners & 10 false positives
ü NOT 18 improvements of 10%+

You need big numbers per variation
Time span!
1.000 conversions!
per month!
10.000 conversions
per month!

With 1000 conversions + a month
20+ tests a year

Tests with impact, on average
1 out of 3
Simply truth: you’re just not always able to create a winner with enough impact!

So in this case, you’ve impact
Every 6 weeks

Slower? Energy will run out!

What to test? Business cases!
Delivery –test with
in stock. 4 weeks
or 2 days?!

What to test? Business cases!
Service charge, do
it or not? Include
in price or not?!

What to test? Big design changes

ROAR – Optimization – moving up
Time span!
1.000 conversions!
per month!
10.000 conversions
per month!

Finally you can test details…
Is Multi Variatie Testing (which is also A/B-testing but with more varations) a good idea? Almost never!!

You should grow to
Customer Buying Reasons

Reasons behind the customer journey

Why is your user here?

How can you

help your user’s brain

to fullﬁl it’s needs?

Data & Psychology
Digital Data
Persuasion Psychology
Analyze
&
Experiment

FA C T & A C T – i t e n d s w i t h Te l l
&
Tell
Conclude
Analyze
Test
Create
Analyze
Find

Remember your lack of resources?

Testing takes time
Use it to LEARN

Create a test schedule

A n d t h e n s t a r t : FA C T & A C T
&
Tell
Conclude
Analyze
Test
Create
Analyze
Find
Find
Your optimization team needs a good analyst / researcher!

Test which dialogue has impact

A/B-testing is the ultimate resource

But ﬁrst: map your user journeys

Based on data

Based on: asking

Based on: listening

Many sources out there to Find

FA C T & A C T
&
Tell
Conclude
Analyze
Test
Create
Analyze
Find
Analyze
Really a good analyst / researcher!!

Use your analytics data

On page behaviour

More details needed?

Use event ﬂow reporting

Create heatmaps

Create screen recordings

To ﬁnd out: The critical locations

FA C T & A C T
&
Tell
Conclude
Analyze
Test
Create
Analyze
Find
Create
And yes, your team needs an economic / consumer Psychologyst and a UX designer!

Source: Psychology

Use that persuasion knowledge
Wheel-of-Persuasion.com
Resource & courses

Bart Schutz: April 21 – 16:00 CEST

Design your new dialogue!

FA C T & A C T
&
Tell
Conclude
Analyze
Test
Create
Analyze
Find
Test
And your team needs a good front-end developer!

But my A/B-test software has drag & drop?

No, you need Front end development

Development resources

We measure in Analytics

They do have analytics – but not enough

Event tracking variation behavior

Calculate results: ABTestGuide.com/calc

So you can A/B-test with Tag Managers

FA C T & A C T
&
Tell
Conclude
Analyze
Test
Create
Analyze
Find
Analyze
Your analyst will analyze the results!

Not signiﬁcant
ü It’s not a winner, stop

ü But, it’s does not mean it’s a losing one!

ü Average: 33% of your tests will have winners
If you are optimizing dialogues!

Choose what to analyze up front
ü You just go and dig unitil you find something
(you will always find something that seems true BUT is not!)

Don’t stop the test once it’s signiﬁcant!

You just let it run until it’s signiﬁcant

You just let it run until it’s signiﬁcant
Your test has a fixed lenght!
It’s even a ﬁxed amount of test population, but we love to test business cycles and/or weeks!

ABTestGuide.com/calc

Your winners are spare?
Less than 25%?
Please higher your signiﬁcance level to 95% or more if it drops below this 25 number!!

Tip: Your test takes too long
ü Cookie deletion
(you will loose 10% in two weeks)
They will re-enter the the and pollute your samples!

Tip: Don’t confuse cross device users
ü  Only test 1 device at a time!

Tip: Maybe you’re testing broken stuff
ü  Always check browser compatibility!
ü  Dont’ break dynamic stuff!

Tip: You forgot to leave out s o m e visitors
ü Only test on those who can be changed
(if you test a sales promotion – leave out your current clients who just logon)

Tip: You’re not slowing the control down
ü  Add the same code to the control

Tip: You’re not testing on fresh people
ü Start with control for a full purchase cycle
(and then start sending traffic to your variations – setting the control to 0%)

Tip: You forgot to give them time to buy
ü Stop getting fresh visitors after your test time
(send new traffic to the control, but give you tested people time to finish converting)

FA C T & A C T
&
Tell
Combine
Analyze
Test
Create
Analyze
Find
Tell
Conclude
Your psychologyst should conclude and your conversion lead should tell!

Conclude

Conclude
ü Do nothing?

Conclude
ü Do nothing?
or
ü Re-test?

Conclude
ü Do nothing?
or
ü Re-test?
or
ü Implement?

Conclude
ü Do nothing?
or
ü Re-test?
or
ü Implement?
or
ü Scale?

Conclude
ü Do nothing?
or
ü Re-test?
or
ü Implement?
or
ü Scale?
and
ü Continue or change test roadmap

Optimize and redesign

Knowledge library: ABtestguide.com

Everyone should use your knowledge!
Data Insights Action

Now you’re testing
On your way to
4 tests a week
Max capacity with 1 teammember on each discipline: analyst, ux, developer, psychologyst!

ROAR – Automation
Time span!
1.000 conversions!
per month!
10.000 conversions
per month!

You’re already thinking of more teams
And still have test capacity
*test capacity – the number of tests you can start per week based on your audience & conversions!

Algorithms

You tested which locations are critical

You know which dialogues are critical

Automation

Now you’ve moved to
Algorithm testing
Which internal and external inﬂuencers do have an impact on which segment of users?!

+ Optimization! + Automation!
ROAR – Automation
Time span!
Risk! Re-think!
1.000 conversions!
per month!
10.000 conversions
per month!

ROAR – You don’t stop optimizing
Time span!
1.000 conversions!
per month!
10.000 conversions
per month!

ROAR – Maybe you’ll never automate
Time span!
1.000 conversions!
per month!
10.000 conversions
per month!

ROAR – Re-think
Time span!
1.000 conversions!
per month!
10.000 conversions
per month!

Optimization is a way of working

CRO brings so much user knowledge

You will make big impact on Discover

Customer knowledge center
Making money from conversion improvements

Creating more and more customer insights
Which can be monetized
on even a larger scale

Now, that’s
HOT

So you keep on groing
GermanEcommercemarket,thankyouAndréMorys

Done testing? No!

Done testing? No!
TESTING NEVER STOPS
Okay, end of life, cash cow, no need for any further growth accelaration or user knowledge!

I f y o u u s e a m e t h o d o l o g y
&
Tell
Conclude
Analyze
Test
Create
Analyze
Find
Otherwise the company will not grow enough on your efforts!

ROAR – when to start with what
Time span!
1.000 conversions!
per month!
10.000 conversions
per month!

Conversion World 2015 Keynote by Ton Wesseling

More Related Content

Similar to Conversion World 2015 Keynote by Ton Wesseling

More from Online Dialogue

Recently uploaded

Conversion World 2015 Keynote by Ton Wesseling