Your SlideShare is downloading. ×
0
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Web Analytics Congress 2013 a/b-test workshop
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Web Analytics Congress 2013 a/b-test workshop

1,328

Published on

Presentation by Ton Wesseling (90 minute workshop) at #wac2013.

Presentation by Ton Wesseling (90 minute workshop) at #wac2013.

Published in: Business
0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,328
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
27
Comments
0
Likes
5
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. ONLINE DIALOGUEGuiding you to digital maturity A/B-testing Workshop Web Analytics Congress 2013/03/13, Utrecht, the Netherlands Slides: http://ondi.me/wac2013
  • 2. A/B-testing 1.  M e t h o d o l o g y 2.  To o l s 3.  Te s t i n g t i p s To n W e s s e l i n g Chief Conversion Officer Online DialogueSlides: http://ondi.me/wac2013
  • 3. 1995@tonwesseling@onlinedialogue The first graphical web browsers
  • 4. #measure@tonwesseling@onlinedialogue Logfile analyses
  • 5. Why?@tonwesseling@onlinedialogue To o p t i m i z e
  • 6. To improve@tonwesseling@onlinedialogue To g r o w
  • 7. We as ked   Where can we improve?   We took small steps   So we could act now!@tonwesseling@onlinedialogue …and it was fun to do!
  • 8. @tonwesseling @onlinedialogue They did something else…
  • 9. Knowlegde   People   @tonwesseling @onlinedialogue Amazing
  • 10. @tonwesseling @onlinedialogue Out of reach
  • 11. 1999@tonwesseling@onlinedialogue Overstock.com design change
  • 12. 2002@tonwesseling@onlinedialogue Overstock.com design change
  • 13. 2005@tonwesseling@onlinedialogue Overstock.com design change
  • 14. 2010@tonwesseling@onlinedialogue Overstock.com design change
  • 15. 2013@tonwesseling@onlinedialogue Overstock.com design change
  • 16. What happ end to:   Where can we improve?   We took small steps   So we could act now!@tonwesseling@onlinedialogue …and it was fun to do!
  • 17. W hat’s the d ifference? @tonwesseling @onlinedialogue With the owner of the local bakery store?
  • 18. Company  goals   €€€                     Customer  goals   Company   Prospect  /  client   Time  &  Money   Knowledge   Behavior   Behavior   Knowledge   ACtude   Mul<   Mul<   Mul<-­‐   Personality   Tools   purpose     phase   channel   Internal     PlaHorm   Profile   influencers   Internal   Internal   Mo<va<on   influencers   Context   influencers   External  influencers   Compe<tors   Media   Social  environment   @tonwesseling @onlinedialogue Dialogue model – improve dialogue insights
  • 19. H ow to communicate @tonwesseling @onlinedialogue With the brains of your visitors
  • 20. Be like B ill @tonwesseling @onlinedialogue In Groundhog day
  • 21. Be like B ill @tonwesseling @onlinedialogue And convince the brain
  • 22. Do you know this man? @tonwesseling @onlinedialogue W. E d w a r d s D e m i n g
  • 23. Deming Cycl e @tonwesseling @onlinedialogue Quality management
  • 24. Focus on q ua lity results of work efforts quality = ___________________ total cost @tonwesseling @onlinedialogue When it is defined by this ratio it leads to results
  • 25. Focus on q ua lity “When you focus just on costs they will rise”@tonwesseling@onlinedialogue W. E d w a r d s D e m i n g
  • 26. Obser vation  +O PDCA   Like the Toyota production system @tonwesseling @onlinedialogue Japan started to learn
  • 27. The Toyota way   Challenge  (We form a long-term vision, meeting challenges with courage and creativity to realize our dreams.)   Kaizen  (We improve our business operations continuously, always driving for innovation and evolution.)   Genchi Genbutsu  (Go to the source to find the facts to make correct decisions.)@tonwesseling@onlinedialogue Continuous improvement
  • 28. Kaizen 改善@tonwesseling@onlinedialogue continuous improvement process
  • 29. “ Change for the better ”@tonwesseling@onlinedialogue Kaizen – Good change
  • 30. How to perform experiments using the scientific method@tonwesseling@onlinedialogue Kaizen – Good change
  • 31. Hypothes es Hypot hesis" Alternative" @tonwesseling @onlinedialogue Statistical hypothesis testing
  • 32. A/B-testing Default" Variation A" Variation B" Winner!" @tonwesseling @onlinedialogue Small steps to learn how to improve the dialogue
  • 33. A /B-te sting maturity Fase 5" Fase 4" Gesegmenteerde dialogen" Fase 3" Gespreksontwerp toepassen" Fase 2" Welke informatie zoekt de gebruiker?" Hoe halen we Niet dezelfde Fase 1" drempels weg?" De stap van dialoog voor Yeah, we kunnen pagina elke bezoeker" testen!" Willen begrijpen optimalisatie hoe je de naar user flow bezoeker beter optimalisatie" Waar kunnen we nu direct het kunt servicen" Maak niet uit Personaliseren   wat we testen verschil maken?" als we maar Mo<veren   testen!" Leren   Direct  resultaat   Laaghangend  fruit   @tonwesseling @onlinedialogue Yo u s h o u l d s t a r t a t l e v e l 1 b u t m o v e f a s t t o l e v e l 3
  • 34. SAME N mod el @tonwesseling @onlinedialogue L e t ’s d o i t t o g e t h e r
  • 35. SAME N mod el S" A" M" E" N" Start reason" Analyze" Make" Evaluate" Notify" @tonwesseling @onlinedialogue Te l l t h e s t o r y “ n a v e r t e l l e n ”
  • 36. “Without an analyzed reason to start, it does not make any sense to start at all.”! @tonwesseling @onlinedialogue Start reason
  • 37. “Do youunderstand whatis happening?”! @tonwesseling @onlinedialogue Analyze
  • 38. “You don’t make perfect designs in 1 time.! ! Step by step we’ll grow to the final result.”! ! Hypotheses based design!! @tonwesseling @onlinedialogue Make
  • 39. “Not only the conversion, look for insights!”! @tonwesseling @onlinedialogue Evaluate
  • 40. “If someone can not reproduce your learnings… ! Then, there were no learnings...”! @tonwesseling @onlinedialogue Notify (Navertellen)
  • 41. Tools! @tonwesseling @onlinedialogue Everything is already here as SaaS
  • 42. Tools! @tonwesseling @onlinedialogue The tools we use these days
  • 43. Start reasonGebruikers Onderzoek (SurveyMonkey), Gebruikers wensen (Uservoice), Continue gebruikers feedback (GetSatisfaction, Kampyle), Analytics trendbreuk notificaties (analytics pakket), Benchmark vergelijken (Trendspot), Markpositie veranderd (Mozenda / Yahoo Pipes), Social (Finchline, Momentum), Intern (Basecamp / Yammer). "@tonwesseling@onlinedialogue To o l s
  • 44. Start reason@tonwesseling@onlinedialogue Surveymonkey
  • 45. Start reason@tonwesseling@onlinedialogue Getsatisfaction.com
  • 46. Start reason@tonwesseling@onlinedialogue Kampyle.com
  • 47. Start reason@tonwesseling@onlinedialogue Tr e s h o l d s ! ( G o o g l e A n a l y t i c s )
  • 48. Start reason@tonwesseling@onlinedialogue Tr e n d s p o t ( B e n c h m a r k i n g )
  • 49. Start reason@tonwesseling@onlinedialogue Mozenda.com: competitve information
  • 50. Start reason@tonwesseling@onlinedialogue Momentum (social insights)
  • 51. Start reason@tonwesseling@onlinedialogue I n t e r n a l i n s i g h t s ( Ya m m e r. c o m )
  • 52. AnalyzeAnalytics pakket, Mousemaps en Screen recordings (Clicktale), Clickmaps(Crazyegg, Clicktale), Survey (SurveyMonkey), tijdelijke gebruikersfeedback(Qualaroo, 4Q en eigen tool (met Google Analytics)), Usability feedback(Feedbackarmy.com, Usertesting.com, Conceptfeedback.com), Internefeedback (notableapp.com)." @tonwesseling @onlinedialogue To o l s
  • 53. Analyze@tonwesseling@onlinedialogue Get your tag management right! (Qubit open tag)
  • 54. Analyze@tonwesseling@onlinedialogue Clicktale.com
  • 55. Analyze@tonwesseling@onlinedialogue Qualaroo.com
  • 56. Analyze@tonwesseling@onlinedialogue Usertesting.com
  • 57. Analyze@tonwesseling@onlinedialogue Notableapp.com
  • 58. MakeWire-Frames (Mockflow.com), Concept testing (Verifyapp.com), Clickabledemo’s / usability (Soldifyapp.com), Usability feedback (FeedbackarmyUsertesting, Conceptfeedback, usability test), Interne feedback(Notableapp.com), Test bouwen / uitvoeren (via CMS & script of VWO ofOptimizely), Targeting (Personyze, Conductrics)." @tonwesseling @onlinedialogue To o l s
  • 59. Make@tonwesseling@onlinedialogue M o c k fl o w. c o m
  • 60. Make@tonwesseling@onlinedialogue Ve r i f y a p p . c o m
  • 61. Make@tonwesseling@onlinedialogue Soldifyapp.com
  • 62. Make@tonwesseling@onlinedialogue V i s u a l w e b s i t e o p t i m i z e r. c o m ( t e s t i n g )
  • 63. Make@tonwesseling@onlinedialogue Personyze.com (targeting)
  • 64. Make@tonwesseling@onlinedialogue Conductrics.com (RTD)
  • 65. E valuateAnalytics pakket (segmenten), Mousemaps en Screen recordings (Clicktale),Clickmaps (Crazyegg, Clicktale), Survey voor NPS / CES (SurveyMonkey)." @tonwesseling @onlinedialogue To o l s
  • 66. E valuate@tonwesseling@onlinedialogue Dive in that data!
  • 67. E valuate@tonwesseling@onlinedialogue Don’t forget the why!
  • 68. E valuate@tonwesseling@onlinedialogue Measure lifetime value indicators
  • 69. NotifyLearnings bord, Learnings boekje, Powerpoint single slide learning (voormail, maar ook voor op TV’s), Gedrags persona’s / Huisstijl update, Test /learnings archief (spreadsheet), intern (intranet, yammer, basecamp) enexterne publicaties." @tonwesseling @onlinedialogue To o l s
  • 70. Notify Publish & Promote your insights!@tonwesseling@onlinedialogue Spread knowledge
  • 71. Test tips “ Not every test will improve conversion ”@tonwesseling@onlinedialogue But you will learn from every test!
  • 72. Test tips “ You will learn you should target ”@tonwesseling@onlinedialogue Move on and you’ll see
  • 73. Psychology @tonwesseling @onlinedialogue Wheel of persuasion
  • 74. Test tips@tonwesseling@onlinedialogue Abnamro.nl: Landing pages
  • 75. Test tips@tonwesseling@onlinedialogue Californiaclosets.com: Landing pages
  • 76. Test tips@tonwesseling@onlinedialogue Perrysport.nl – landing page messaging
  • 77. Test tips@tonwesseling@onlinedialogue Alphaecological.com: Form location
  • 78. Test tips@tonwesseling@onlinedialogue Jules.fr: Shop photos
  • 79. Test tips@tonwesseling@onlinedialogue Kras.nl: Footer
  • 80. Test tips@tonwesseling@onlinedialogue Wehkamp.nl: Princing
  • 81. Test tips@tonwesseling@onlinedialogue Kickz.com: price focus?
  • 82. Test tips@tonwesseling@onlinedialogue D e s k a w a y. c o m – “ u g l y b r o t h e r ” p r o d u c t
  • 83. Test tips@tonwesseling@onlinedialogue CDFnetworks.com – Form lay-out
  • 84. Test tips@tonwesseling@onlinedialogue G s m s t u n t e r. n l – i n t e r n a l s e a r c h
  • 85. Test tips@tonwesseling@onlinedialogue Bijenkorf.nl: point-of-actions
  • 86. Test tips@tonwesseling@onlinedialogue A c t i e v a n d e d a g . n l : C TA m o b i l e p r o o f
  • 87. Test tips@tonwesseling@onlinedialogue Basecamphq.com – next action
  • 88. Test tips@tonwesseling@onlinedialogue Kras.nl - checkmarks
  • 89. Test tips@tonwesseling@onlinedialogue VGZ.nl - checkmarks
  • 90. Test tips@tonwesseling@onlinedialogue Hotelschiphol.nl - checkmarks
  • 91. Test tips@tonwesseling@onlinedialogue T h e c o r k s c r e w. i e - d i s c o u n t
  • 92. Test tips@tonwesseling@onlinedialogue Bol.com – like positioning
  • 93. Test tips@tonwesseling@onlinedialogue Inshared.nl – Sticky content
  • 94. To d o:   Optimize your dialogue   Focus on quality   Spread your knowledge@tonwesseling@onlinedialogue Thank you! Slides: http://ondi.me/wac2013

×