Ton Wesseling, profile picture
Uploaded byTon Wesseling
PDF, PPTX783 views

Run more experiments - Ilive 2015 Latvia, Riga

The document discusses optimization methodologies in testing, emphasizing the importance of A/B testing and continuous optimization for improving conversion rates in businesses. It introduces the ROAR model for structuring testing and optimization phases, highlighting the significance of data analysis and psychological insights in understanding customer behavior. Additionally, it outlines the resources and team structure needed to facilitate effective testing and decision-making.

Data & Analytics◦
Related topics:
A/B Testing•

Embed presentation

Download as PDF, PPTX
1 / 149
2 / 149
3 / 149
4 / 149
5 / 149
6 / 149
7 / 149
8 / 149
9 / 149
10 / 149
11 / 149
12 / 149
13 / 149
14 / 149
15 / 149
16 / 149
17 / 149
18 / 149
19 / 149
20 / 149
21 / 149
22 / 149
23 / 149
24 / 149
25 / 149
26 / 149
27 / 149
28 / 149
29 / 149
30 / 149
31 / 149
32 / 149
33 / 149
34 / 149
35 / 149
36 / 149
37 / 149
38 / 149
39 / 149
40 / 149
41 / 149
42 / 149
43 / 149
44 / 149
45 / 149
46 / 149
47 / 149
48 / 149
49 / 149
50 / 149
51 / 149
52 / 149
53 / 149
54 / 149
55 / 149
56 / 149
57 / 149
58 / 149
59 / 149
60 / 149
61 / 149
62 / 149
63 / 149
64 / 149
65 / 149
66 / 149
67 / 149
68 / 149
69 / 149
70 / 149
71 / 149
72 / 149
73 / 149
74 / 149
75 / 149
76 / 149
77 / 149
78 / 149
79 / 149
80 / 149
81 / 149
82 / 149
83 / 149
84 / 149
85 / 149
86 / 149
87 / 149
88 / 149
89 / 149
90 / 149
91 / 149
92 / 149
93 / 149
94 / 149
95 / 149
96 / 149
97 / 149
98 / 149
99 / 149
100 / 149
101 / 149
102 / 149
103 / 149
104 / 149
105 / 149
106 / 149
107 / 149
108 / 149
109 / 149
110 / 149
111 / 149
112 / 149
113 / 149
114 / 149
115 / 149
116 / 149
117 / 149
118 / 149
119 / 149
120 / 149
121 / 149
122 / 149
123 / 149
124 / 149
125 / 149
126 / 149
127 / 149
128 / 149
129 / 149
130 / 149
131 / 149
132 / 149
133 / 149
134 / 149
135 / 149
136 / 149
137 / 149
138 / 149
139 / 149
140 / 149
141 / 149
142 / 149
143 / 149
144 / 149
145 / 149
146 / 149
147 / 149
148 / 149
149 / 149
i L i v e L a t v i a , R i g a , N o v e m b e r 1 2 t h 2 0 1 5 To n W e s s e l i n g – C E O – Te s t i n g . A g e n c y # i L i v e 2 0 1 5 - @ To n W Run more experiments
Ton Wesseling | @ TonW Run more experiments #ilive2015 Thank you for having me
Ton Wesseling | @ TonW Run more experiments #ilive2015 On this stage
Ton Wesseling | @ TonW Run more experiments #ilive2015 Visiting Riga
Ton Wesseling | @ TonW Run more experiments #ilive2015 On november 11th
Ton Wesseling | @ TonW Run more experiments #ilive2015 http://testing.agency
Ton Wesseling | @ TonW Run more experiments #ilive2015 Meeting you all
Ton Wesseling | @ TonW Run more experiments #ilive2015 Conversion people
Ton Wesseling | @ TonW Run more experiments #ilive2015 Always fun
Ton Wesseling | @ TonW Run more experiments #ilive2015 When we start cooking
Ton Wesseling | @ TonW Run more experiments #ilive2015 High energy & fun
Ton Wesseling | @ TonW Run more experiments #ilive2015 We are Evidence Based
Ton Wesseling | @ TonW Run more experiments #ilive2015 And LEAN
Ton Wesseling | @ TonW Run more experiments #ilive2015 It has been a long day though
Ton Wesseling | @ TonW Run more experiments #ilive2015 What about your energy?
Ton Wesseling | @ TonW Run more experiments #ilive2015 Value for your time and money
Ton Wesseling | @ TonW Run more experiments #ilive2015 Reboost your energy
Ton Wesseling | @ TonW Run more experiments #ilive2015 Please stand up
Ton Wesseling | @ TonW Run more experiments #ilive2015 Raise your hands
Ton Wesseling | @ TonW Run more experiments #ilive2015 Raise your hands
Ton Wesseling | @ TonW Run more experiments #ilive2015 High energy conversion fun
Ton Wesseling | @ TonW Run more experiments #ilive2015 Calm down and focus
Ton Wesseling | @ TonW Run more experiments #ilive2015 Why this song? ROAR
Ton Wesseling | @ TonW Run more experiments #ilive2015 Because of Katy Perry?
Ton Wesseling | @ TonW Run more experiments #ilive2015 No, it’s the name of the song!
Ton Wesseling | @ TonW Run more experiments #ilive2015 Today the ROAR model
Ton Wesseling | @ TonW Run more experiments #ilive2015 20 years ago
Ton Wesseling | @ TonW Run more experiments #ilive2015 I learned about him
Ton Wesseling | @ TonW Run more experiments #ilive2015 Quality circle – the start of LEAN
Ton Wesseling | @ TonW Run more experiments #ilive2015 LEAN
Ton Wesseling | @ TonW Run more experiments #ilive2015 So I looked at data
Ton Wesseling | @ TonW Run more experiments #ilive2015 Made some changes
Ton Wesseling | @ TonW Run more experiments #ilive2015 And measured if it was successful
Ton Wesseling | @ TonW Run more experiments #ilive2015 Why can this lead to failures?
Ton Wesseling | @ TonW Run more experiments #ilive2015 Average conversion rate looks stable
Ton Wesseling | @ TonW Run more experiments #ilive2015 But in fact
Ton Wesseling | @ TonW Run more experiments #ilive2015 There is a big weekly change!
Ton Wesseling | @ TonW Run more experiments #ilive2015 I only did small improvements
Ton Wesseling | @ TonW Run more experiments #ilive2015 So I made many wrong decisions 26 out of 44 weeks: over 5% change!
Ton Wesseling | @ TonW Run more experiments #ilive2015 Then I learned about this story…
Ton Wesseling | @ TonW Run more experiments #ilive2015 Bill Murray in Groundhog Day
Ton Wesseling | @ TonW Run more experiments #ilive2015 Trying to get the girl
Ton Wesseling | @ TonW Run more experiments #ilive2015 He has a stable testing environment A/B-Testing
Ton Wesseling | @ TonW Run more experiments #ilive2015
Ton Wesseling | @ TonW Run more experiments #ilive2015 Methods used to improve conversion
Ton Wesseling | @ TonW Run more experiments #ilive2015 Methods companies plan to use
Ton Wesseling | @ TonW Run more experiments #ilive2015 A/B-testing is HOT
Ton Wesseling | @ TonW Run more experiments #ilive2015 Everyone is pushing to test more
Ton Wesseling | @ TonW Run more experiments #ilive2015 Like Amazon, Like Zalando GermanEcommercemarket,thankyouAndréMorys
Ton Wesseling | @ TonW Run more experiments #ilive2015 More experiments “If you double the number of experiments you do per year you’re going to double your inventiveness” Jeff Bezos, CEO Amazon
Ton Wesseling | @ TonW Run more experiments #ilive2015 Never satisfied with conversion rates
Ton Wesseling | @ TonW Run more experiments #ilive2015 Start testing or test more
Ton Wesseling | @ TonW Run more experiments #ilive2015 NO Growth problems will disappear…
Ton Wesseling | @ TonW Run more experiments #ilive2015 A/B-testing is not a solution It’s a methodology
Ton Wesseling | @ TonW Run more experiments #ilive2015 It’s part of a: Continuous Optimization Program
Ton Wesseling | @ TonW Run more experiments #ilive2015 It’s a way of working It’s company DNA
Ton Wesseling | @ TonW Run more experiments #ilive2015 You need to know: When How What
Ton Wesseling | @ TonW Run more experiments #ilive2015 Growth curve – how it was teached
Ton Wesseling | @ TonW Run more experiments #ilive2015 the ROAR model
Ton Wesseling | @ TonW Run more experiments #ilive2015 Optimization phases - ROAR Time span! Conversionspermonth!
Ton Wesseling | @ TonW Run more experiments #ilive2015 Optimization phases - ROAR Time span! Conversionspermonth!
Ton Wesseling | @ TonW Run more experiments #ilive2015 Optimization phases - ROAR Time span! Conversionspermonth!
Ton Wesseling | @ TonW Run more experiments #ilive2015 Optimization phases - ROAR Time span! Conversionspermonth! Risk!
Ton Wesseling | @ TonW Run more experiments #ilive2015 Optimization phases - ROAR Time span! Conversionspermonth! Risk! + Optimization!
Ton Wesseling | @ TonW Run more experiments #ilive2015 Optimization phases - ROAR Time span! Conversionspermonth! Risk! + Optimization! + Automation!
Ton Wesseling | @ TonW Run more experiments #ilive2015 Optimization phases - ROAR Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think!
Ton Wesseling | @ TonW Run more experiments #ilive2015 Optimization phases - ROAR Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 1.000 conversions! per month!
Ton Wesseling | @ TonW Run more experiments #ilive2015 Optimization phases - ROAR Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 10.000 conversions per month! 1.000 conversions! per month!
Ton Wesseling | @ TonW Run more experiments #ilive2015 How much impact do you need? http://ondi.me/size!
Ton Wesseling | @ TonW Run more experiments #ilive2015 Optimization phases - ROAR Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 10.000 conversions per month! 1.000 conversions! per month! 15% impact needed! 5% impact! needed! Impact needed: all conversions with an average conversion rate of 2%, a test length of 3 weeks and a power of 80%!
Ton Wesseling | @ TonW Run more experiments #ilive2015 Optimization phases - ROAR Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 1.000 conversions! per month!
Ton Wesseling | @ TonW Run more experiments #ilive2015 Phase 1: bridge the gap – Risk
Ton Wesseling | @ TonW Run more experiments #ilive2015 Phase 1: bridge the gap – Risk
Ton Wesseling | @ TonW Run more experiments #ilive2015 Get out of the office
Ton Wesseling | @ TonW Run more experiments #ilive2015 Risk
Ton Wesseling | @ TonW Run more experiments #ilive2015 Yes, I’ve got those 1000+ conversions Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 1.000 conversions! per month! 10.000 conversions per month!
Ton Wesseling | @ TonW Run more experiments #ilive2015 With 1000+ conversions a month Your test capacity is max. 20+ tests a year with a statistical power of 80% on predicted uplifts of 15%
Ton Wesseling | @ TonW Run more experiments #ilive2015 Tests with measurable impact On average 1 out of 3 Simple truth: you’re just not always able to create a winner with enough impact
Ton Wesseling | @ TonW Run more experiments #ilive2015 So in this case, you’ve got impact Every 7 or 8 weeks
Ton Wesseling | @ TonW Run more experiments #ilive2015 Slower? Energy will run out! And your continous optimization program will die
Ton Wesseling | @ TonW Run more experiments #ilive2015 What to test? Business cases! Delivery –test with in stock. 4 weeks or 2 days?!
Ton Wesseling | @ TonW Run more experiments #ilive2015 What to test? Business cases! Service charge, do it or not? Include in price or not?!
Ton Wesseling | @ TonW Run more experiments #ilive2015 What to test? Big design changes
Ton Wesseling | @ TonW Run more experiments #ilive2015 ROAR – Optimization – moving up Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 1.000 conversions! per month! 10.000 conversions per month!
Ton Wesseling | @ TonW Run more experiments #ilive2015 Finally you can test details… Is Multi Variate Testing (which is also A/B-testing but with more varations) a good idea? Almost never!!
Ton Wesseling | @ TonW Run more experiments #ilive2015 You should grow to Customer Buying Reasons
Ton Wesseling | @ TonW Run more experiments #ilive2015 Reasons behind the customer journey
Ton Wesseling | @ TonW Run more experiments #ilive2015 A continuous optimization team Analyst Psychologyst Designer Developer Team Lead
Ton Wesseling | @ TonW Run more experiments #ilive2015 They keep on asking: Why are these users here?
Ton Wesseling | @ TonW Run more experiments #ilive2015 And they want to know How they can help their users’ brains to fullfil their needs?
Ton Wesseling | @ TonW Run more experiments #ilive2015 Data & Psychology Digital Data Persuasion Psychology Analyze & Experiment
Ton Wesseling | @ TonW Run more experiments #ilive2015 A lack of resources
Ton Wesseling | @ TonW Run more experiments #ilive2015 Testing takes resources Use it to LEARN
Ton Wesseling | @ TonW Run more experiments #ilive2015 FA C T & A C T & Tell Conclude Analyze Test Create Analyze Find
Ton Wesseling | @ TonW Run more experiments #ilive2015 A n d t h e n s t a r t : FA C T & A C T & Tell Conclude Analyze Test Create Analyze Find Find Your optimization team needs a good analyst / researcher!
Ton Wesseling | @ TonW Run more experiments #ilive2015 Many sources out there to Find
Ton Wesseling | @ TonW Run more experiments #ilive2015 FA C T & A C T & Tell Conclude Analyze Test Create Analyze Find Analyze Really a good analyst / researcher!!
Ton Wesseling | @ TonW Run more experiments #ilive2015 Use your analytics data
Ton Wesseling | @ TonW Run more experiments #ilive2015 Create heatmaps
Ton Wesseling | @ TonW Run more experiments #ilive2015 Create screen recordings
Ton Wesseling | @ TonW Run more experiments #ilive2015 To find out: The critical spots
Ton Wesseling | @ TonW Run more experiments #ilive2015 FA C T & A C T & Tell Conclude Analyze Test Create Analyze Find Create And yes, your team needs an economic / consumer Psychologyst and a UX designer!
Ton Wesseling | @ TonW Run more experiments #ilive2015 Source: Psychology
Ton Wesseling | @ TonW Run more experiments #ilive2015 Two systems
Ton Wesseling | @ TonW Run more experiments #ilive2015 Two systems that intervene
Ton Wesseling | @ TonW Run more experiments #ilive2015 Use that persuasion knowledge Wheel-of-Persuasion.com Resource & courses
Ton Wesseling | @ TonW Run more experiments #ilive2015 Tests that can be done
Ton Wesseling | @ TonW Run more experiments #ilive2015 Tests that can be done
Ton Wesseling | @ TonW Run more experiments #ilive2015 Tests that can be done
Ton Wesseling | @ TonW Run more experiments #ilive2015 Tests that can be done
Ton Wesseling | @ TonW Run more experiments #ilive2015 Tests that can be done
Ton Wesseling | @ TonW Run more experiments #ilive2015 Tests that can be done
Ton Wesseling | @ TonW Run more experiments #ilive2015 FA C T & A C T & Tell Conclude Analyze Test Create Analyze Find Test And your team needs a good front-end developer!
Ton Wesseling | @ TonW Run more experiments #ilive2015 Get your A/B-solution in place
Ton Wesseling | @ TonW Run more experiments #ilive2015 But you need Front-end resources!
Ton Wesseling | @ TonW Run more experiments #ilive2015 FA C T & A C T & Tell Conclude Analyze Test Create Analyze Find Analyze Your analyst will analyze the results!
Ton Wesseling | @ TonW Run more experiments #ilive2015 Look at details
Ton Wesseling | @ TonW Run more experiments #ilive2015 FA C T & A C T & Tell Combine Analyze Test Create Analyze Find Tell Conclude Your psychologyst should conclude and your conversion lead should tell!
Ton Wesseling | @ TonW Run more experiments #ilive2015 Store your learnings
Ton Wesseling | @ TonW Run more experiments #ilive2015 Push your behavioral insights
Ton Wesseling | @ TonW Run more experiments #ilive2015 Everyone should use your knowledge! Data Insights Action
Ton Wesseling | @ TonW Run more experiments #ilive2015 + Optimization! + Automation! Back to the ROAR model Time span! Conversionspermonth! Risk! Re-think! 1.000 conversions! per month! 10.000 conversions per month!
Ton Wesseling | @ TonW Run more experiments #ilive2015 ROAR – Optimization – moving up Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 1.000 conversions! per month! 10.000 conversions per month!
Ton Wesseling | @ TonW Run more experiments #ilive2015 Now you’re testing On your way to 4 tests a week Max capacity with 1 teammember on each discipline: analyst, ux, developer, psychologyst!
Ton Wesseling | @ TonW Run more experiments #ilive2015 Test capacity maximum number of experiments per year that your site or app can handle
Ton Wesseling | @ TonW Run more experiments #ilive2015 Regret amount of money you are losing per year by not using your full test capacity
Ton Wesseling | @ TonW Run more experiments #ilive2015 ROAR – Automation Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 1.000 conversions! per month! 10.000 conversions per month!
Ton Wesseling | @ TonW Run more experiments #ilive2015 Automation
Ton Wesseling | @ TonW Run more experiments #ilive2015 You tested which locations are critical
Ton Wesseling | @ TonW Run more experiments #ilive2015 You know which dialogues are critical
Ton Wesseling | @ TonW Run more experiments #ilive2015 Automation
Ton Wesseling | @ TonW Run more experiments #ilive2015 Now you’ve moved to Algorithm testing Which internal and external influencers do have an impact on which segment of users?!
Ton Wesseling | @ TonW Run more experiments #ilive2015 + Optimization! + Automation! ROAR – Automation Time span! Conversionspermonth! Risk! Re-think! 1.000 conversions! per month! 10.000 conversions per month!
Ton Wesseling | @ TonW Run more experiments #ilive2015 ROAR – You don’t stop optimizing Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 1.000 conversions! per month! 10.000 conversions per month!
Ton Wesseling | @ TonW Run more experiments #ilive2015 ROAR – Re-think Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 1.000 conversions! per month! 10.000 conversions per month!
Ton Wesseling | @ TonW Run more experiments #ilive2015 A/B-testing is the ultimate resource
Ton Wesseling | @ TonW Run more experiments #ilive2015 More experiments “If you double the number of experiments you do per year you’re going to double your inventiveness” Jeff Bezos, CEO Amazon
Ton Wesseling | @ TonW Run more experiments #ilive2015 CRO brings user knowledge
Ton Wesseling | @ TonW Run more experiments #ilive2015 Store your learnings
Ton Wesseling | @ TonW Run more experiments #ilive2015 You will make big impact on Discover
Ton Wesseling | @ TonW Run more experiments #ilive2015 Now, that’s HOT
Ton Wesseling | @ TonW Run more experiments #ilive2015 Done testing? No! TESTING NEVER STOPS Okay, end of life, cash cow, no need for any further growth accelaration or user knowledge!
Ton Wesseling | @ TonW Run more experiments #ilive2015 A/B-testing Minimize regret
Ton Wesseling | @ TonW Run more experiments #ilive2015 Test as much as you can Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 10.000 conversions per month! 1.000 conversions! per month! 15% impact needed! 5% impact! needed! Impact needed: all conversions with an average conversion rate of 2%, a test length of 3 weeks and a power of 80%!
Ton Wesseling | @ TonW Run more experiments #ilive2015 Like Amazon, Like Zalando GermanEcommercemarket,thankyouAndréMorys
Ton Wesseling | @ TonW Run more experiments #ilive2015 Enjoy the high energy fun!
Ton Wesseling | @ TonW Run more experiments #ilive2015 Experimenting is a way of working It’s company DNA
Ton Wesseling | @ TonW Run more experiments #ilive2015 Run more experiments
i L i v e L a t v i a , R i g a , N o v e m b e r 1 2 t h 2 0 1 5 To n W e s s e l i n g – C E O – Te s t i n g . A g e n c y # i L i v e 2 0 1 5 - @ To n W Run more experiments

More Related Content

PDF
Keynote Digital Elite Camp Estonia 2015 - ROAR - when to run more experiments
byTon Wesseling
 
PDF
Bang! 15 cases of real CRO impact by applying data & psychology
byTon Wesseling
 
PPT
Kennissessie Sea Is Geen Advertising!
byValtech
 
PPTX
Famous Person
byPaul Ariza
 
DOCX
La importancia del internet en las tecnologías de la educación es muy importa...
byannolia12
 
PPTX
Online marketing plan Fermo
bymarjolein90
 
PPTX
De Waterkant, Rumst
byCAAAP
 
PPT
Social media for dummies
bySelina Roskam
 
Keynote Digital Elite Camp Estonia 2015 - ROAR - when to run more experiments
byTon Wesseling
 
Bang! 15 cases of real CRO impact by applying data & psychology
byTon Wesseling
 
Kennissessie Sea Is Geen Advertising!
byValtech
 
Famous Person
byPaul Ariza
 
La importancia del internet en las tecnologías de la educación es muy importa...
byannolia12
 
Online marketing plan Fermo
bymarjolein90
 
De Waterkant, Rumst
byCAAAP
 
Social media for dummies
bySelina Roskam
 

Similar to Run more experiments - Ilive 2015 Latvia, Riga

PPTX
When to run more experiments?
byiLive Conference
 
PPTX
Myths, Lies and Illusions of AB and Split Testing
byCraig Sullivan
 
PPTX
Ecommerce Conversion World, London March 23 2017 - Ton Wesseling keynote
byOnline Dialogue
 
PDF
Conversion World 2015 Keynote by Ton Wesseling
byOnline Dialogue
 
PDF
[CXL Live 16] The Grand Unified Theory of Conversion Optimization by John Ekman
byCXL
 
PPTX
Brighton CRO Meetup #1 - Oh Boy These AB tests Sure Look Like Bullshit to Me
byCraig Sullivan
 
PDF
Craig Sullivan - Oh Boy! These A/B tests look like total bullshit! MKTFEST 2014
byMarketing Festival
 
PDF
A/B Mythbusters: Common Optimization Objections Debunked
byOptimizely
 
PDF
Build a Winning Conversion Optimization Strategy
bySavage Marketing
 
PDF
Getting Started with Experimentation
byOptimizely
 
PPTX
#Measurefest : 20 Simple Ways to Fuck Up your AB tests
byCraig Sullivan
 
PDF
The Optimisation Grand Unified Theory @ ConversionXL Live
byConversionista
 
PDF
The Optimizely Experience Keynote by Matt Althauser - Optimizely Experience L...
byOptimizely
 
PPTX
Digital Impact 2014 - Oh Boy These AB tests sure look like Bullshit to me
byCraig Sullivan
 
PPTX
John lewis lego craig sullivan 1335
byCharlie Lines
 
PDF
When to run how many A/B-tests (based on data) - MeasureCamp Amsterdam - Ton ...
byOnline Dialogue
 
PDF
Understand A/B Testing in 9 use cases & 7 mistakes
byTheFamily
 
PDF
Webinar: Common Mistakes in A/B Testing
byOptimizely
 
PPTX
#Measurecamp : 18 Simple Ways to F*** up Your AB Testing
byCraig Sullivan
 
PDF
Intuit - How to Scale Your Experimentation Program
byOptimizely
 
When to run more experiments?
byiLive Conference
 
Myths, Lies and Illusions of AB and Split Testing
byCraig Sullivan
 
Ecommerce Conversion World, London March 23 2017 - Ton Wesseling keynote
byOnline Dialogue
 
Conversion World 2015 Keynote by Ton Wesseling
byOnline Dialogue
 
[CXL Live 16] The Grand Unified Theory of Conversion Optimization by John Ekman
byCXL
 
Brighton CRO Meetup #1 - Oh Boy These AB tests Sure Look Like Bullshit to Me
byCraig Sullivan
 
Craig Sullivan - Oh Boy! These A/B tests look like total bullshit! MKTFEST 2014
byMarketing Festival
 
A/B Mythbusters: Common Optimization Objections Debunked
byOptimizely
 
Build a Winning Conversion Optimization Strategy
bySavage Marketing
 
Getting Started with Experimentation
byOptimizely
 
#Measurefest : 20 Simple Ways to Fuck Up your AB tests
byCraig Sullivan
 
The Optimisation Grand Unified Theory @ ConversionXL Live
byConversionista
 
The Optimizely Experience Keynote by Matt Althauser - Optimizely Experience L...
byOptimizely
 
Digital Impact 2014 - Oh Boy These AB tests sure look like Bullshit to me
byCraig Sullivan
 
John lewis lego craig sullivan 1335
byCharlie Lines
 
When to run how many A/B-tests (based on data) - MeasureCamp Amsterdam - Ton ...
byOnline Dialogue
 
Understand A/B Testing in 9 use cases & 7 mistakes
byTheFamily
 
Webinar: Common Mistakes in A/B Testing
byOptimizely
 
#Measurecamp : 18 Simple Ways to F*** up Your AB Testing
byCraig Sullivan
 
Intuit - How to Scale Your Experimentation Program
byOptimizely
 

More from Ton Wesseling

PDF
Landing page competition on CROday 2016
byTon Wesseling
 
PDF
Keynote Ton Wesseling at Superweek 2020: How an analyst can add value!
byTon Wesseling
 
PDF
Online Dialogue - Content is your conversation - Measure & Improve
byTon Wesseling
 
PDF
Conversion strategy & tips: Ton Wesseling A4Uexpo 2009 Europe, Amsterdam
byTon Wesseling
 
PPT
Opening presentatie CSNconference208.com 19 juni 2008
byTon Wesseling
 
PPT
Presentatie Livegang webanalisten.nl 29 mei 2008 op het webanalyticscongres.nl
byTon Wesseling
 
PPT
Internet Macht Aan De Consument
byTon Wesseling
 
PPT
Banking Future : Web & Mobile
byTon Wesseling
 
PPT
How To Create Online Sales presentation at rimc 2008
byTon Wesseling
 
Landing page competition on CROday 2016
byTon Wesseling
 
Keynote Ton Wesseling at Superweek 2020: How an analyst can add value!
byTon Wesseling
 
Online Dialogue - Content is your conversation - Measure & Improve
byTon Wesseling
 
Conversion strategy & tips: Ton Wesseling A4Uexpo 2009 Europe, Amsterdam
byTon Wesseling
 
Opening presentatie CSNconference208.com 19 juni 2008
byTon Wesseling
 
Presentatie Livegang webanalisten.nl 29 mei 2008 op het webanalyticscongres.nl
byTon Wesseling
 
Internet Macht Aan De Consument
byTon Wesseling
 
Banking Future : Web & Mobile
byTon Wesseling
 
How To Create Online Sales presentation at rimc 2008
byTon Wesseling
 

Recently uploaded

PDF
McCrindle 2026 Trends of the Year
byMark McCrindle
 
PPTX
CMMI High Maturity (L4/L5) Consulting.pptx
bySunil Yadav
 
PPTX
[DSC Europe 25] Miodrag Pesovic & Vladislav Radonjic - Federated Data Archite...
byDataScienceConferenc1
 
PDF
[DSC Europe 25] Francisco Prado Moreno - Model Validation in the Age of AI: T...
byDataScienceConferenc1
 
PDF
Terminals into Throughways: Amtrak Through Service to Long Island
bykrv2009
 
PPTX
Data analysis 2 presentation at ipmc.pptx
byabdualimohammed960
 
PDF
DiaryofaWebSDKImplemenation-LaunchDebugging.pdf
bygmuir1066
 
PDF
[DSC Europe 25] Uros Pesic - The Reality of AI in Marketing.pdf
byDataScienceConferenc1
 
PPTX
[DSC Europe 25] Nikolay Burlutskiy - Best Practices for Building Enterprise M...
byDataScienceConferenc1
 
DOCX
A SURVEY ABOUT USING CRYPTOGRAPHIC ALGORITHM
byjaskarans90411
 
PPTX
[DSC Europe 25] Djordje Hirs - Revolutionizing Telco Customer Experience with...
byDataScienceConferenc1
 
PDF
Oracle Data Breach_ Causes, Impact, and Security Lessons.pdf
bydexpose Lastdexpose
 
PPTX
[DSC Europe 25] Jakub Stech - AI for Public Good: How Data and AI Can Transfo...
byDataScienceConferenc1
 
PPT
Fundamentals OF TUBERCULOSIS AND SIGNS.ppt
bymedsjpalomo
 
PPTX
[DSC Europe 25] Branko Urosevic -Rethinking Financial Talent: Integrating Cod...
byDataScienceConferenc1
 
PPTX
[DSC Europe 25] Dobrica Cosic - From Electrons to Innovation: How Granular Da...
byDataScienceConferenc1
 
PPTX
Integrated matching and rebalancing for ride-hailing services.pptx
byWenjiaShen1
 
PPTX
[DSC Europe 25] Tatevik Maytesyan - How to actually use AI in marketing: gett...
byDataScienceConferenc1
 
PPTX
[DSC Europe 25] Ivan Petrovic - Is it really that expensive to build an AI sy...
byDataScienceConferenc1
 
PDF
[DSC Europe 25] Danica Soc - The Science Behind Marketing: Experimentation me...
byDataScienceConferenc1
 
McCrindle 2026 Trends of the Year
byMark McCrindle
 
CMMI High Maturity (L4/L5) Consulting.pptx
bySunil Yadav
 
[DSC Europe 25] Miodrag Pesovic & Vladislav Radonjic - Federated Data Archite...
byDataScienceConferenc1
 
[DSC Europe 25] Francisco Prado Moreno - Model Validation in the Age of AI: T...
byDataScienceConferenc1
 
Terminals into Throughways: Amtrak Through Service to Long Island
bykrv2009
 
Data analysis 2 presentation at ipmc.pptx
byabdualimohammed960
 
DiaryofaWebSDKImplemenation-LaunchDebugging.pdf
bygmuir1066
 
[DSC Europe 25] Uros Pesic - The Reality of AI in Marketing.pdf
byDataScienceConferenc1
 
[DSC Europe 25] Nikolay Burlutskiy - Best Practices for Building Enterprise M...
byDataScienceConferenc1
 
A SURVEY ABOUT USING CRYPTOGRAPHIC ALGORITHM
byjaskarans90411
 
[DSC Europe 25] Djordje Hirs - Revolutionizing Telco Customer Experience with...
byDataScienceConferenc1
 
Oracle Data Breach_ Causes, Impact, and Security Lessons.pdf
bydexpose Lastdexpose
 
[DSC Europe 25] Jakub Stech - AI for Public Good: How Data and AI Can Transfo...
byDataScienceConferenc1
 
Fundamentals OF TUBERCULOSIS AND SIGNS.ppt
bymedsjpalomo
 
[DSC Europe 25] Branko Urosevic -Rethinking Financial Talent: Integrating Cod...
byDataScienceConferenc1
 
[DSC Europe 25] Dobrica Cosic - From Electrons to Innovation: How Granular Da...
byDataScienceConferenc1
 
Integrated matching and rebalancing for ride-hailing services.pptx
byWenjiaShen1
 
[DSC Europe 25] Tatevik Maytesyan - How to actually use AI in marketing: gett...
byDataScienceConferenc1
 
[DSC Europe 25] Ivan Petrovic - Is it really that expensive to build an AI sy...
byDataScienceConferenc1
 
[DSC Europe 25] Danica Soc - The Science Behind Marketing: Experimentation me...
byDataScienceConferenc1
 

Run more experiments - Ilive 2015 Latvia, Riga

  • 1.
    i L iv e L a t v i a , R i g a , N o v e m b e r 1 2 t h 2 0 1 5 To n W e s s e l i n g – C E O – Te s t i n g . A g e n c y # i L i v e 2 0 1 5 - @ To n W Run more experiments
  • 2.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Thank you for having me
  • 3.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 On this stage
  • 4.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Visiting Riga
  • 5.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 On november 11th
  • 6.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 http://testing.agency
  • 7.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Meeting you all
  • 8.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Conversion people
  • 9.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Always fun
  • 10.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 When we start cooking
  • 11.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 High energy & fun
  • 12.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 We are Evidence Based
  • 13.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 And LEAN
  • 14.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 It has been a long day though
  • 15.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 What about your energy?
  • 16.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Value for your time and money
  • 17.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Reboost your energy
  • 18.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Please stand up
  • 19.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Raise your hands
  • 20.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Raise your hands
  • 21.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 High energy conversion fun
  • 22.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Calm down and focus
  • 23.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Why this song? ROAR
  • 24.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Because of Katy Perry?
  • 25.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 No, it’s the name of the song!
  • 26.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Today the ROAR model
  • 27.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 20 years ago
  • 28.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 I learned about him
  • 29.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Quality circle – the start of LEAN
  • 30.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 LEAN
  • 31.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 So I looked at data
  • 32.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Made some changes
  • 33.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 And measured if it was successful
  • 34.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Why can this lead to failures?
  • 35.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Average conversion rate looks stable
  • 36.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 But in fact
  • 37.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 There is a big weekly change!
  • 38.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 I only did small improvements
  • 39.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 So I made many wrong decisions 26 out of 44 weeks: over 5% change!
  • 40.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Then I learned about this story…
  • 41.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Bill Murray in Groundhog Day
  • 42.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Trying to get the girl
  • 43.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 He has a stable testing environment A/B-Testing
  • 44.
    Ton Wesseling |@ TonW Run more experiments #ilive2015
  • 45.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Methods used to improve conversion
  • 46.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Methods companies plan to use
  • 47.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 A/B-testing is HOT
  • 48.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Everyone is pushing to test more
  • 49.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Like Amazon, Like Zalando GermanEcommercemarket,thankyouAndréMorys
  • 50.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 More experiments “If you double the number of experiments you do per year you’re going to double your inventiveness” Jeff Bezos, CEO Amazon
  • 51.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Never satisfied with conversion rates
  • 52.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Start testing or test more
  • 53.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 NO Growth problems will disappear…
  • 54.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 A/B-testing is not a solution It’s a methodology
  • 55.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 It’s part of a: Continuous Optimization Program
  • 56.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 It’s a way of working It’s company DNA
  • 57.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 You need to know: When How What
  • 58.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Growth curve – how it was teached
  • 59.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 the ROAR model
  • 60.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Optimization phases - ROAR Time span! Conversionspermonth!
  • 61.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Optimization phases - ROAR Time span! Conversionspermonth!
  • 62.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Optimization phases - ROAR Time span! Conversionspermonth!
  • 63.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Optimization phases - ROAR Time span! Conversionspermonth! Risk!
  • 64.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Optimization phases - ROAR Time span! Conversionspermonth! Risk! + Optimization!
  • 65.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Optimization phases - ROAR Time span! Conversionspermonth! Risk! + Optimization! + Automation!
  • 66.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Optimization phases - ROAR Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think!
  • 67.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Optimization phases - ROAR Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 1.000 conversions! per month!
  • 68.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Optimization phases - ROAR Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 10.000 conversions per month! 1.000 conversions! per month!
  • 69.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 How much impact do you need? http://ondi.me/size!
  • 70.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Optimization phases - ROAR Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 10.000 conversions per month! 1.000 conversions! per month! 15% impact needed! 5% impact! needed! Impact needed: all conversions with an average conversion rate of 2%, a test length of 3 weeks and a power of 80%!
  • 71.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Optimization phases - ROAR Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 1.000 conversions! per month!
  • 72.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Phase 1: bridge the gap – Risk
  • 73.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Phase 1: bridge the gap – Risk
  • 74.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Get out of the office
  • 75.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Risk
  • 76.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Yes, I’ve got those 1000+ conversions Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 1.000 conversions! per month! 10.000 conversions per month!
  • 77.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 With 1000+ conversions a month Your test capacity is max. 20+ tests a year with a statistical power of 80% on predicted uplifts of 15%
  • 78.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Tests with measurable impact On average 1 out of 3 Simple truth: you’re just not always able to create a winner with enough impact
  • 79.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 So in this case, you’ve got impact Every 7 or 8 weeks
  • 80.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Slower? Energy will run out! And your continous optimization program will die
  • 81.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 What to test? Business cases! Delivery –test with in stock. 4 weeks or 2 days?!
  • 82.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 What to test? Business cases! Service charge, do it or not? Include in price or not?!
  • 83.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 What to test? Big design changes
  • 84.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 ROAR – Optimization – moving up Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 1.000 conversions! per month! 10.000 conversions per month!
  • 85.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Finally you can test details… Is Multi Variate Testing (which is also A/B-testing but with more varations) a good idea? Almost never!!
  • 86.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 You should grow to Customer Buying Reasons
  • 87.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Reasons behind the customer journey
  • 88.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 A continuous optimization team Analyst Psychologyst Designer Developer Team Lead
  • 89.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 They keep on asking: Why are these users here?
  • 90.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 And they want to know How they can help their users’ brains to fullfil their needs?
  • 91.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Data & Psychology Digital Data Persuasion Psychology Analyze & Experiment
  • 92.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 A lack of resources
  • 93.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Testing takes resources Use it to LEARN
  • 94.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 FA C T & A C T & Tell Conclude Analyze Test Create Analyze Find
  • 95.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 A n d t h e n s t a r t : FA C T & A C T & Tell Conclude Analyze Test Create Analyze Find Find Your optimization team needs a good analyst / researcher!
  • 96.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Many sources out there to Find
  • 97.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 FA C T & A C T & Tell Conclude Analyze Test Create Analyze Find Analyze Really a good analyst / researcher!!
  • 98.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Use your analytics data
  • 99.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Create heatmaps
  • 100.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Create screen recordings
  • 101.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 To find out: The critical spots
  • 102.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 FA C T & A C T & Tell Conclude Analyze Test Create Analyze Find Create And yes, your team needs an economic / consumer Psychologyst and a UX designer!
  • 103.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Source: Psychology
  • 104.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Two systems
  • 105.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Two systems that intervene
  • 106.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Use that persuasion knowledge Wheel-of-Persuasion.com Resource & courses
  • 107.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Tests that can be done
  • 108.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Tests that can be done
  • 109.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Tests that can be done
  • 110.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Tests that can be done
  • 111.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Tests that can be done
  • 112.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Tests that can be done
  • 113.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 FA C T & A C T & Tell Conclude Analyze Test Create Analyze Find Test And your team needs a good front-end developer!
  • 114.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Get your A/B-solution in place
  • 115.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 But you need Front-end resources!
  • 116.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 FA C T & A C T & Tell Conclude Analyze Test Create Analyze Find Analyze Your analyst will analyze the results!
  • 117.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Look at details
  • 118.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 FA C T & A C T & Tell Combine Analyze Test Create Analyze Find Tell Conclude Your psychologyst should conclude and your conversion lead should tell!
  • 119.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Store your learnings
  • 120.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Push your behavioral insights
  • 121.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Everyone should use your knowledge! Data Insights Action
  • 122.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 + Optimization! + Automation! Back to the ROAR model Time span! Conversionspermonth! Risk! Re-think! 1.000 conversions! per month! 10.000 conversions per month!
  • 123.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 ROAR – Optimization – moving up Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 1.000 conversions! per month! 10.000 conversions per month!
  • 124.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Now you’re testing On your way to 4 tests a week Max capacity with 1 teammember on each discipline: analyst, ux, developer, psychologyst!
  • 125.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Test capacity maximum number of experiments per year that your site or app can handle
  • 126.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Regret amount of money you are losing per year by not using your full test capacity
  • 127.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 ROAR – Automation Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 1.000 conversions! per month! 10.000 conversions per month!
  • 128.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Automation
  • 129.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 You tested which locations are critical
  • 130.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 You know which dialogues are critical
  • 131.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Automation
  • 132.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Now you’ve moved to Algorithm testing Which internal and external influencers do have an impact on which segment of users?!
  • 133.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 + Optimization! + Automation! ROAR – Automation Time span! Conversionspermonth! Risk! Re-think! 1.000 conversions! per month! 10.000 conversions per month!
  • 134.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 ROAR – You don’t stop optimizing Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 1.000 conversions! per month! 10.000 conversions per month!
  • 135.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 ROAR – Re-think Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 1.000 conversions! per month! 10.000 conversions per month!
  • 136.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 A/B-testing is the ultimate resource
  • 137.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 More experiments “If you double the number of experiments you do per year you’re going to double your inventiveness” Jeff Bezos, CEO Amazon
  • 138.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 CRO brings user knowledge
  • 139.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Store your learnings
  • 140.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 You will make big impact on Discover
  • 141.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Now, that’s HOT
  • 142.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Done testing? No! TESTING NEVER STOPS Okay, end of life, cash cow, no need for any further growth accelaration or user knowledge!
  • 143.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 A/B-testing Minimize regret
  • 144.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Test as much as you can Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 10.000 conversions per month! 1.000 conversions! per month! 15% impact needed! 5% impact! needed! Impact needed: all conversions with an average conversion rate of 2%, a test length of 3 weeks and a power of 80%!
  • 145.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Like Amazon, Like Zalando GermanEcommercemarket,thankyouAndréMorys
  • 146.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Enjoy the high energy fun!
  • 147.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Experimenting is a way of working It’s company DNA
  • 148.
    Ton Wesseling |@ TonW Run more experiments #ilive2015 Run more experiments
  • 149.
    i L iv e L a t v i a , R i g a , N o v e m b e r 1 2 t h 2 0 1 5 To n W e s s e l i n g – C E O – Te s t i n g . A g e n c y # i L i v e 2 0 1 5 - @ To n W Run more experiments