Conversionboost - Bart Schutz

Bart.Schutz@onlinedialogue.com
#
Marshmallows, "
& the Psychology"
behind Evidence-Based Growth
#

#
The Marshmallow Test

#
The Marshmallow Test
Self-control:
health, wealth & even happiness

#
Homo Sapiens often is better off when
excerting self-control

#
Part 1) "
Psychology

#
We are a Dual-Processing Brain
Fast & automated
Intuitive & associative
Effortless
Large capacity
Reasons & rules
Requires attention
Effortful
Small capacity
Subconscious,
emotional
processes
Conscious,  
rational
processes
(always on)
(lazy controller)
System 1 

System 2

#
Feel your own 2 systems

#
A user = "
2 processing systems

#
The inconvenient truth:
The systems create ‘illusions of control’

#
Do we have Conscious Control?
A couple has 2 kids. "
One of them is a boy.

How big is the chance that
the other kid is a boy too?
33%
?

#
Do we have Conscious Control?
A couple has 2 kids. "
One of them is a boy.

How big is the chance that
the other kid is a boy too?
33%
?
‘Conscious control is often an illusion‘

#
Do we have Reasons?

#
Do we have Reasons?
‘Reasons are often post-decision alibi’s‘

#
Do we have Free Will?
I Know! Calculations are hard work.
But it’s nearly over.

Come on, one more…
123 + 5
QUICK!
THINK ABOUT A
COLOR AND A
TOOL!
At the end of this test,
I ask a question.
Don’t stop and think about it
Answer is immediately!
How much is:
15 + 6
3 + 56
89 + 2
12 + 53
75 + 26
25 + 52
63 + 32

#

#
‘Free-will is often an illusion‘

#
Do we have a reliable Memory?

#
Do we have a reliable Memory?
‘Experience & memory are quite distinct’

#
You and your users usually "
CANNOT help you with optimizing!

#
Usability Test insights?
B
A
Winner

#
Usability Test insights?
B
A
Winner
User Testing is often
Useless Testing

#
What if users had designed Booking.com?

#
CRO = User-centric
Psychology

#
Part 2) "
Data & Statistics

#
Behavior has become data

#
Behavioral data analysis

#
Behavioral data analysis
The convenient truth:
1st party data!

#
Behavior has become testable

#
Behavior has become testable
The convenient truth:
!!! 1st party evidence !!!

#
Evidence has become FREE
Free A/B-testing: GTMtesting.com

#
Tools easy: lie!
Evidence has become Easy

#
You must have the basics of "
Statistics & Data Quality right!

#
CRO = ‘The illusion of Growth‘

#
POWERRRR!!!! (over signiﬁcance)
Sample Size & Uplift calculator: abtestguide.com/abtestsize/

#
Lack of power:
Big issue in CRO!

#
Calculate
Minimum Required Uplift

#
Frequentistic (t-test): abtestguide.com/calc/
Which statistics for analyzing?
Bayesian: abtestguide.com/bayesian

#
Frequentistic (t-test): abtestguide.com/calc/
Which statistics for analyzing?
Bayesian: abtestguide.com/bayesian
Use Bayesian to grow
Use Frequentist to learn

#
But: Do you meet the assumptions of the statistics?
effect

#
effect
Track users 100%, OR "
incorporate the pollution

#
‘Ceteris Paribus’

#
Check for
Skewed Samples
‘Ceteris Paribus’

#
OMG,  
All my A/B-tests  
were useless 
!?

#
Part 3) "
Evidence-Based Growth

#
&
Evidence Based Growth
√ Data√ Psychology

#
CRO => "
Evidence-Based Growth (EBG)

#
3 levels of EBG!

#
EARN
adding direct value
LEARN
user behavior
TURN
Innovate
3 levels of CRO EBG

#
Level 1: Transactional"

#
Level 1: Transactional"
Earn – Earn - Earn

#
1st Bandwidth: How many experiments can you run?

#
Type 3: Loss
preven.on
Type 2: Earn Type 1: Learn
Prob of upli+ > 0% = 75% Prob of upli+ > 3% = 75%
(CO1/2/3 > 2%)
Frequen>st
(1-sided, 95%)
Bandwidth 20 per week 8 per week 5 per week
1st Bandwidth: How many experiments can you run?

#
“Done, 
absolutely everything
Is tested
REALLY
Booking.com
Evidence Everything

#
Evidence Everything

#
“Our success at Amazon
is a function of
how many experiments
we do per year, per month, per
week, per day…”

Jeﬀ Bezos, CEO Amazon
Focus on Regret & Velocity

#
“Move fast and break things.
Unless you are breaking stuﬀ, you
are not moving fast enough.
…”

Mark Zuckerberg, FaceBook
Done is better than Perfect

#
Earn = Transactional

#
Quantity over Quality

#
Quantity over Quality
The BIG ﬁsh will not eat the SMALL, "
The FAST ﬁsh will eat the SLOW

#
Data Driven

Transactional
C!
CREATE
A!
ANALYZE
T!
TEST

#
EARN => CTA–CTA –CTA!

#
Level 2: Intellectual"

#
Level 2: Intellectual"
Learn – Learn – Learn

#
Evidence Everything

Customer Intelligence
A!
ANALYZE
C!
CONCLUDE
C!
CREATE
A!
ANALYZE
T!
TEST

#
Heatmaps
Recordings
Web Analytics
Contextual Data
Previous tests
Algorithm outcomes
Mission
Vision
Strategy
Positioning
Goals
Scientific research
Competitors
View
(1st party)
Value
Customer Service
Surveys
Feedback tools
User & Market research
Social Media
Behavior & Experience Determinant Study
VoiceVerified
(1st party)
Verified

#
get more insights from science:

#

#
Behavior & Experience Insights Grid

#
Case: Hostelworld

#
A
B
A
B
Winner
Loser
Different brain systems per device

#
System 2
||
/
controling  
System 1
System 1
/
||
Little control of  
System 2
Learn = Customer Intelligence

#
1st order learning: Local maximum

#
LEARN => FACT – FACT – FACT!

#
Level 3: Transformational"
Innovate & Disrupt

#
F!
FIND
T!
TELL
A!
ANALYZE
C!
COMBINE
C!
CREATE
A!
ANALYZE
T!
TEST
F!
FIND
T!
TRANSFORM
A!
ANALYZE
C!
CONCLUDE
C!
CREATE
A!
ANALYZE
T!
TEST
Evidence Everything

Transformational

#
B
A
Winner
Implicit safety

#
Case: Hostelworld On–The–Go?

#
The future of CRO = EBG"
A transformational understanding "
of what drives homo sapiens

#
Let’s rewind

#
EARN => ACT – ACT – ACT!

#
You got to have your
Data & Statistics right

#
Internet = The
Biggest Psychological experiment ever

#
Internet = Data & Experiments
>100 billion ‘people’ are
in a test every month

#
An abundance of data is tracked

#
Knowledge
Knowledge
a/b-test a/b-test a/b-test a/b-test
I am in control
No, you’re not, "
I know you better than
you know yourself

#
Knowledge
Knowledge

I know how to inject my
goal into your mind,
separating you from your
Money & Time & Free-will

#
With great intelligence and inﬂuence,
comes great responsibility

•  Bart@OnlineDialogue.com 

•  nl.linkedin.com/in/bartschutz/ 

•  @BartS
STATISTICS

Use =>
Abtestguide.com/calc
CONSUMER PSYCHOLOGY

Learn =>
wheelofpersuasion.com/course
Download these slides => http://ondi.me/cboostdk
EVIDENCE BASED GROWTH

Incorporate =>
info@OnlineDialogue.com
Co-founder

Advisory Board Dutch  
Institute for Psychologists

EU Federation of Psychology Associations 
Chair Task Force ‘Internet & Psychology’
Tak!
Bart Schutz

Conversionboost - Bart Schutz

Recommended

Recommended

More Related Content

What's hot

What's hot (7)

Viewers also liked

Viewers also liked (10)

Similar to Conversionboost - Bart Schutz

Similar to Conversionboost - Bart Schutz (20)

More from Online Dialogue

More from Online Dialogue (20)

Recently uploaded

Recently uploaded (20)

Conversionboost - Bart Schutz