SlideShare a Scribd company logo
1 of 36
Download to read offline
Matt Althauser
GM, Optimizely EMEA 
@malthauser
7,000



500,000



7,000,000,000
Join the
Community:



www.optiverse.com

February 27th, 2000
Holy A/B Testing!
…?
One to Many
 One to Few
 One to One
Evolution of Optimization
What s the best experience for
the average user?	

One to Many
2/27/2000-Present 	

=	

Simple Tools
Original	

Variation	

+606% 
Engagement
Video	

Images
+0.2 % 
Conversions
Video	

Images
2006-Present 	

=	

One to Few
What s the best experience for
segments of users?	

Fire!
Fire!
Flow Analysis…
+15% 
Task Completion
2010-Beyond 	

=	

Wheel!	

One to One
What s the best
experience for each user
on each device?
Donation Button Experiment
0.0%	

 0.0%	

 0.0%	

2.3%	

 27.8%	

 16.3%	

-27.8	

 N/A	

 N/A	

15.2%	

 -24.6%	

 11.9%	

8.5%	

 2.9%	

 18.4%	

Donation Button Experiment
Never
Signed-Up
0.0%	

 0.0%	

 0.0%	

2.3%	

 27.8%	

 16.3%	

-27.8	

 N/A	

 N/A	

15.2%	

 -24.6%	

 11.9%	

8.5%	

 2.9%	

 18.4%	

Donation Button Experiment
Never
Signed-Up
Signed-Up
No Donate
0.0%	

 0.0%	

 0.0%	

2.3%	

 27.8%	

 16.3%	

-27.8	

 N/A	

 N/A	

15.2%	

 -24.6%	

 11.9%	

8.5%	

 2.9%	

 18.4%	

Donation Button Experiment
Never
Signed-Up
Signed-Up
No Donate
Previously
Donated
2%
Return Visitors
10%
Mobile Visitors
1%
Amsterdam Visitors
0.2%
New Visitors
2%
+28% 
Revenue/Visitor
App Store Blues
mobile.optimizely.com



@malthauser

More Related Content

Similar to Digital Data Tips Tuesday #2 - A/B-testing: Matt Althauser - Optimizely

360Bind Automate Business Objects regression testing
360Bind Automate Business Objects regression testing360Bind Automate Business Objects regression testing
360Bind Automate Business Objects regression testing
Sebastien Goiffon
 
The pushing of programs and operating systems
The pushing of programs and operating systemsThe pushing of programs and operating systems
The pushing of programs and operating systems
Joseph Jones
 
Jevgeni Kabanov in GeekOut: Redefining redeploys
Jevgeni Kabanov in GeekOut: Redefining redeploysJevgeni Kabanov in GeekOut: Redefining redeploys
Jevgeni Kabanov in GeekOut: Redefining redeploys
ZeroTurnaround
 

Similar to Digital Data Tips Tuesday #2 - A/B-testing: Matt Althauser - Optimizely (20)

UserVoice Summit 2013 Fall Keynote
UserVoice Summit 2013 Fall KeynoteUserVoice Summit 2013 Fall Keynote
UserVoice Summit 2013 Fall Keynote
 
How Atlassian Broke Down the Walls Between Support & Development
How Atlassian Broke Down the Walls Between Support & DevelopmentHow Atlassian Broke Down the Walls Between Support & Development
How Atlassian Broke Down the Walls Between Support & Development
 
#PDR15 - Data Analytics and Pebble
#PDR15 - Data Analytics and Pebble#PDR15 - Data Analytics and Pebble
#PDR15 - Data Analytics and Pebble
 
New Product Development Financial Analysis PowerPoint Presentation Slides
New Product Development Financial Analysis PowerPoint Presentation SlidesNew Product Development Financial Analysis PowerPoint Presentation Slides
New Product Development Financial Analysis PowerPoint Presentation Slides
 
User experience(UX)
User experience(UX)User experience(UX)
User experience(UX)
 
360Bind Automate Business Objects regression testing
360Bind Automate Business Objects regression testing360Bind Automate Business Objects regression testing
360Bind Automate Business Objects regression testing
 
[Elite Camp 2016] Phil Nottingham - CRO with Video: Tips, Tricks and Tactics
[Elite Camp 2016] Phil Nottingham - CRO with Video: Tips, Tricks and Tactics[Elite Camp 2016] Phil Nottingham - CRO with Video: Tips, Tricks and Tactics
[Elite Camp 2016] Phil Nottingham - CRO with Video: Tips, Tricks and Tactics
 
MeasureWorks - Windesheim Almere - Why Performance matters?
MeasureWorks  - Windesheim Almere - Why Performance matters?MeasureWorks  - Windesheim Almere - Why Performance matters?
MeasureWorks - Windesheim Almere - Why Performance matters?
 
MIE: Impactvolle Data Analist
MIE: Impactvolle Data AnalistMIE: Impactvolle Data Analist
MIE: Impactvolle Data Analist
 
A/B testing at Spotify
A/B testing at SpotifyA/B testing at Spotify
A/B testing at Spotify
 
Google Analytics Alternatives
Google Analytics AlternativesGoogle Analytics Alternatives
Google Analytics Alternatives
 
MVT Presentation
MVT PresentationMVT Presentation
MVT Presentation
 
Olcan Sercinoglu
Olcan SercinogluOlcan Sercinoglu
Olcan Sercinoglu
 
The pushing of programs and operating systems
The pushing of programs and operating systemsThe pushing of programs and operating systems
The pushing of programs and operating systems
 
How We Make Apps And Services
How We Make Apps And ServicesHow We Make Apps And Services
How We Make Apps And Services
 
Jevgeni Kabanov in GeekOut: Redefining redeploys
Jevgeni Kabanov in GeekOut: Redefining redeploysJevgeni Kabanov in GeekOut: Redefining redeploys
Jevgeni Kabanov in GeekOut: Redefining redeploys
 
Taming the Wild West
Taming the Wild West Taming the Wild West
Taming the Wild West
 
Preparing for AI - Measurefest
Preparing for AI - MeasurefestPreparing for AI - Measurefest
Preparing for AI - Measurefest
 
[Webinar] Optimise your website with Analytics & A/B Testing
[Webinar] Optimise your website with Analytics & A/B Testing[Webinar] Optimise your website with Analytics & A/B Testing
[Webinar] Optimise your website with Analytics & A/B Testing
 
The Super User in SAP
The Super User in SAPThe Super User in SAP
The Super User in SAP
 

More from Webanalisten .nl

CH2019 keynote: Guy Yalif - AI and personalization demystified
CH2019 keynote: Guy Yalif - AI and personalization demystifiedCH2019 keynote: Guy Yalif - AI and personalization demystified
CH2019 keynote: Guy Yalif - AI and personalization demystified
Webanalisten .nl
 
CH2019 keynote: Lucia van den Brink - Keynote pitch winner
CH2019 keynote: Lucia van den Brink - Keynote pitch winnerCH2019 keynote: Lucia van den Brink - Keynote pitch winner
CH2019 keynote: Lucia van den Brink - Keynote pitch winner
Webanalisten .nl
 
CH2019 keynote: Elke Geraerts - Authenthic intelligence
CH2019 keynote: Elke Geraerts - Authenthic intelligenceCH2019 keynote: Elke Geraerts - Authenthic intelligence
CH2019 keynote: Elke Geraerts - Authenthic intelligence
Webanalisten .nl
 

More from Webanalisten .nl (20)

DDTT11: Hans Willemse - 21-01-20
DDTT11: Hans Willemse - 21-01-20DDTT11: Hans Willemse - 21-01-20
DDTT11: Hans Willemse - 21-01-20
 
DDTT11: Ton Wesseling - 21-01-20
DDTT11: Ton Wesseling - 21-01-20DDTT11: Ton Wesseling - 21-01-20
DDTT11: Ton Wesseling - 21-01-20
 
CH2019 keynote: Guy Yalif - AI and personalization demystified
CH2019 keynote: Guy Yalif - AI and personalization demystifiedCH2019 keynote: Guy Yalif - AI and personalization demystified
CH2019 keynote: Guy Yalif - AI and personalization demystified
 
CH2019 keynote: Aurélie Pols - Data governance
CH2019 keynote: Aurélie Pols - Data governanceCH2019 keynote: Aurélie Pols - Data governance
CH2019 keynote: Aurélie Pols - Data governance
 
CH2019 keynote: Lucia van den Brink - Keynote pitch winner
CH2019 keynote: Lucia van den Brink - Keynote pitch winnerCH2019 keynote: Lucia van den Brink - Keynote pitch winner
CH2019 keynote: Lucia van den Brink - Keynote pitch winner
 
CH2019 keynote: Elke Geraerts - Authenthic intelligence
CH2019 keynote: Elke Geraerts - Authenthic intelligenceCH2019 keynote: Elke Geraerts - Authenthic intelligence
CH2019 keynote: Elke Geraerts - Authenthic intelligence
 
CH2019 keynote: Lukas Vermeer - One neat trick to run better experiments
CH2019 keynote: Lukas Vermeer - One neat trick to run better experimentsCH2019 keynote: Lukas Vermeer - One neat trick to run better experiments
CH2019 keynote: Lukas Vermeer - One neat trick to run better experiments
 
CH2019 keynote: Roger Dooley - Friction
CH2019 keynote: Roger Dooley - FrictionCH2019 keynote: Roger Dooley - Friction
CH2019 keynote: Roger Dooley - Friction
 
Conversion Hotel 2018 Keynote: Ton Wesseling
Conversion Hotel 2018 Keynote: Ton WesselingConversion Hotel 2018 Keynote: Ton Wesseling
Conversion Hotel 2018 Keynote: Ton Wesseling
 
Conversion Hotel 2018 Keynote: Nir Eyal
Conversion Hotel 2018 Keynote: Nir EyalConversion Hotel 2018 Keynote: Nir Eyal
Conversion Hotel 2018 Keynote: Nir Eyal
 
Conversion Hotel 2018 Keynote: Erin Weigel
Conversion Hotel 2018 Keynote: Erin WeigelConversion Hotel 2018 Keynote: Erin Weigel
Conversion Hotel 2018 Keynote: Erin Weigel
 
Conversion Hotel 2018 Keynote: Lea Pica
Conversion Hotel 2018 Keynote: Lea PicaConversion Hotel 2018 Keynote: Lea Pica
Conversion Hotel 2018 Keynote: Lea Pica
 
Conversion Hotel 2018 Keynote: Greg Shapiro
Conversion Hotel 2018 Keynote: Greg ShapiroConversion Hotel 2018 Keynote: Greg Shapiro
Conversion Hotel 2018 Keynote: Greg Shapiro
 
Conversion Hotel 2018 Keynote: TU Delft Solar Team
Conversion Hotel 2018 Keynote: TU Delft Solar TeamConversion Hotel 2018 Keynote: TU Delft Solar Team
Conversion Hotel 2018 Keynote: TU Delft Solar Team
 
Conversion Hotel 2018 Keynote: Karl Gilis
Conversion Hotel 2018 Keynote: Karl GilisConversion Hotel 2018 Keynote: Karl Gilis
Conversion Hotel 2018 Keynote: Karl Gilis
 
Conversion Hotel 2018 Keynote: Lisa Pierson
Conversion Hotel 2018 Keynote: Lisa PiersonConversion Hotel 2018 Keynote: Lisa Pierson
Conversion Hotel 2018 Keynote: Lisa Pierson
 
Conversion Hotel 2018 Keynote: Chad Sanderson
Conversion Hotel 2018 Keynote: Chad SandersonConversion Hotel 2018 Keynote: Chad Sanderson
Conversion Hotel 2018 Keynote: Chad Sanderson
 
Conversion Hotel 2018 Keynote: Aleksander Fabijan
Conversion Hotel 2018 Keynote: Aleksander FabijanConversion Hotel 2018 Keynote: Aleksander Fabijan
Conversion Hotel 2018 Keynote: Aleksander Fabijan
 
Conversion Hotel 2018 Keynote: Lizzie Eardley
Conversion Hotel 2018 Keynote: Lizzie EardleyConversion Hotel 2018 Keynote: Lizzie Eardley
Conversion Hotel 2018 Keynote: Lizzie Eardley
 
Conversion Hotel 2018 Keynote: Michael Aagaard
Conversion Hotel 2018 Keynote: Michael AagaardConversion Hotel 2018 Keynote: Michael Aagaard
Conversion Hotel 2018 Keynote: Michael Aagaard
 

Recently uploaded

在线制作约克大学毕业证(yu毕业证)在读证明认证可查
在线制作约克大学毕业证(yu毕业证)在读证明认证可查在线制作约克大学毕业证(yu毕业证)在读证明认证可查
在线制作约克大学毕业证(yu毕业证)在读证明认证可查
ydyuyu
 
VIP Call Girls Himatnagar 7001035870 Whatsapp Number, 24/07 Booking
VIP Call Girls Himatnagar 7001035870 Whatsapp Number, 24/07 BookingVIP Call Girls Himatnagar 7001035870 Whatsapp Number, 24/07 Booking
VIP Call Girls Himatnagar 7001035870 Whatsapp Number, 24/07 Booking
dharasingh5698
 
6.High Profile Call Girls In Punjab +919053900678 Punjab Call GirlHigh Profil...
6.High Profile Call Girls In Punjab +919053900678 Punjab Call GirlHigh Profil...6.High Profile Call Girls In Punjab +919053900678 Punjab Call GirlHigh Profil...
6.High Profile Call Girls In Punjab +919053900678 Punjab Call GirlHigh Profil...
@Chandigarh #call #Girls 9053900678 @Call #Girls in @Punjab 9053900678
 
💚😋 Bilaspur Escort Service Call Girls, 9352852248 ₹5000 To 25K With AC💚😋
💚😋 Bilaspur Escort Service Call Girls, 9352852248 ₹5000 To 25K With AC💚😋💚😋 Bilaspur Escort Service Call Girls, 9352852248 ₹5000 To 25K With AC💚😋
💚😋 Bilaspur Escort Service Call Girls, 9352852248 ₹5000 To 25K With AC💚😋
nirzagarg
 

Recently uploaded (20)

VVIP Pune Call Girls Sinhagad WhatSapp Number 8005736733 With Elite Staff And...
VVIP Pune Call Girls Sinhagad WhatSapp Number 8005736733 With Elite Staff And...VVIP Pune Call Girls Sinhagad WhatSapp Number 8005736733 With Elite Staff And...
VVIP Pune Call Girls Sinhagad WhatSapp Number 8005736733 With Elite Staff And...
 
Sarola * Female Escorts Service in Pune | 8005736733 Independent Escorts & Da...
Sarola * Female Escorts Service in Pune | 8005736733 Independent Escorts & Da...Sarola * Female Escorts Service in Pune | 8005736733 Independent Escorts & Da...
Sarola * Female Escorts Service in Pune | 8005736733 Independent Escorts & Da...
 
在线制作约克大学毕业证(yu毕业证)在读证明认证可查
在线制作约克大学毕业证(yu毕业证)在读证明认证可查在线制作约克大学毕业证(yu毕业证)在读证明认证可查
在线制作约克大学毕业证(yu毕业证)在读证明认证可查
 
Katraj ( Call Girls ) Pune 6297143586 Hot Model With Sexy Bhabi Ready For S...
Katraj ( Call Girls ) Pune  6297143586  Hot Model With Sexy Bhabi Ready For S...Katraj ( Call Girls ) Pune  6297143586  Hot Model With Sexy Bhabi Ready For S...
Katraj ( Call Girls ) Pune 6297143586 Hot Model With Sexy Bhabi Ready For S...
 
Hire↠Young Call Girls in Tilak nagar (Delhi) ☎️ 9205541914 ☎️ Independent Esc...
Hire↠Young Call Girls in Tilak nagar (Delhi) ☎️ 9205541914 ☎️ Independent Esc...Hire↠Young Call Girls in Tilak nagar (Delhi) ☎️ 9205541914 ☎️ Independent Esc...
Hire↠Young Call Girls in Tilak nagar (Delhi) ☎️ 9205541914 ☎️ Independent Esc...
 
VIP Call Girls Himatnagar 7001035870 Whatsapp Number, 24/07 Booking
VIP Call Girls Himatnagar 7001035870 Whatsapp Number, 24/07 BookingVIP Call Girls Himatnagar 7001035870 Whatsapp Number, 24/07 Booking
VIP Call Girls Himatnagar 7001035870 Whatsapp Number, 24/07 Booking
 
WhatsApp 📞 8448380779 ✅Call Girls In Mamura Sector 66 ( Noida)
WhatsApp 📞 8448380779 ✅Call Girls In Mamura Sector 66 ( Noida)WhatsApp 📞 8448380779 ✅Call Girls In Mamura Sector 66 ( Noida)
WhatsApp 📞 8448380779 ✅Call Girls In Mamura Sector 66 ( Noida)
 
"Boost Your Digital Presence: Partner with a Leading SEO Agency"
"Boost Your Digital Presence: Partner with a Leading SEO Agency""Boost Your Digital Presence: Partner with a Leading SEO Agency"
"Boost Your Digital Presence: Partner with a Leading SEO Agency"
 
2nd Solid Symposium: Solid Pods vs Personal Knowledge Graphs
2nd Solid Symposium: Solid Pods vs Personal Knowledge Graphs2nd Solid Symposium: Solid Pods vs Personal Knowledge Graphs
2nd Solid Symposium: Solid Pods vs Personal Knowledge Graphs
 
6.High Profile Call Girls In Punjab +919053900678 Punjab Call GirlHigh Profil...
6.High Profile Call Girls In Punjab +919053900678 Punjab Call GirlHigh Profil...6.High Profile Call Girls In Punjab +919053900678 Punjab Call GirlHigh Profil...
6.High Profile Call Girls In Punjab +919053900678 Punjab Call GirlHigh Profil...
 
VIP Model Call Girls Hadapsar ( Pune ) Call ON 9905417584 Starting High Prof...
VIP Model Call Girls Hadapsar ( Pune ) Call ON 9905417584 Starting  High Prof...VIP Model Call Girls Hadapsar ( Pune ) Call ON 9905417584 Starting  High Prof...
VIP Model Call Girls Hadapsar ( Pune ) Call ON 9905417584 Starting High Prof...
 
Russian Call Girls in %(+971524965298 )# Call Girls in Dubai
Russian Call Girls in %(+971524965298  )#  Call Girls in DubaiRussian Call Girls in %(+971524965298  )#  Call Girls in Dubai
Russian Call Girls in %(+971524965298 )# Call Girls in Dubai
 
Call Girls Ludhiana Just Call 98765-12871 Top Class Call Girl Service Available
Call Girls Ludhiana Just Call 98765-12871 Top Class Call Girl Service AvailableCall Girls Ludhiana Just Call 98765-12871 Top Class Call Girl Service Available
Call Girls Ludhiana Just Call 98765-12871 Top Class Call Girl Service Available
 
Shikrapur - Call Girls in Pune Neha 8005736733 | 100% Gennuine High Class Ind...
Shikrapur - Call Girls in Pune Neha 8005736733 | 100% Gennuine High Class Ind...Shikrapur - Call Girls in Pune Neha 8005736733 | 100% Gennuine High Class Ind...
Shikrapur - Call Girls in Pune Neha 8005736733 | 100% Gennuine High Class Ind...
 
Call Now ☎ 8264348440 !! Call Girls in Green Park Escort Service Delhi N.C.R.
Call Now ☎ 8264348440 !! Call Girls in Green Park Escort Service Delhi N.C.R.Call Now ☎ 8264348440 !! Call Girls in Green Park Escort Service Delhi N.C.R.
Call Now ☎ 8264348440 !! Call Girls in Green Park Escort Service Delhi N.C.R.
 
APNIC Updates presented by Paul Wilson at ARIN 53
APNIC Updates presented by Paul Wilson at ARIN 53APNIC Updates presented by Paul Wilson at ARIN 53
APNIC Updates presented by Paul Wilson at ARIN 53
 
Trump Diapers Over Dems t shirts Sweatshirt
Trump Diapers Over Dems t shirts SweatshirtTrump Diapers Over Dems t shirts Sweatshirt
Trump Diapers Over Dems t shirts Sweatshirt
 
VIP Model Call Girls NIBM ( Pune ) Call ON 8005736733 Starting From 5K to 25K...
VIP Model Call Girls NIBM ( Pune ) Call ON 8005736733 Starting From 5K to 25K...VIP Model Call Girls NIBM ( Pune ) Call ON 8005736733 Starting From 5K to 25K...
VIP Model Call Girls NIBM ( Pune ) Call ON 8005736733 Starting From 5K to 25K...
 
Microsoft Azure Arc Customer Deck Microsoft
Microsoft Azure Arc Customer Deck MicrosoftMicrosoft Azure Arc Customer Deck Microsoft
Microsoft Azure Arc Customer Deck Microsoft
 
💚😋 Bilaspur Escort Service Call Girls, 9352852248 ₹5000 To 25K With AC💚😋
💚😋 Bilaspur Escort Service Call Girls, 9352852248 ₹5000 To 25K With AC💚😋💚😋 Bilaspur Escort Service Call Girls, 9352852248 ₹5000 To 25K With AC💚😋
💚😋 Bilaspur Escort Service Call Girls, 9352852248 ₹5000 To 25K With AC💚😋
 

Digital Data Tips Tuesday #2 - A/B-testing: Matt Althauser - Optimizely