A/B-testing #leanconf Manchester 2014: WHEN - HOW - EXAMPLES - PITFALLS
•
1 like•1,255 views
Keynote of Ton Wesseling, CEO Testing.Agency on A/B-testing. November 17th 2014, Lean Conference Manchester. Is this keynote Ton shares why A/B-testing is user knowledge and how you use testing to outgrow the competition. In the slides are some tool and article tips, some A/B-test examples and some analytics pitfalls - which will be pittfalls with an extra t, if you don't know them :-)
Recommended
Recommended
More Related Content
Similar to A/B-testing #leanconf Manchester 2014: WHEN - HOW - EXAMPLES - PITFALLS
Similar to A/B-testing #leanconf Manchester 2014: WHEN - HOW - EXAMPLES - PITFALLS (20)
More from Online Dialogue
More from Online Dialogue (20)
Recently uploaded
Recently uploaded (20)
A/B-testing #leanconf Manchester 2014: WHEN - HOW - EXAMPLES - PITFALLS
- 1. A/B-testing HOW, W H E N , E X A M P L E S & P I T T FA L L S L e a n C o n f e r e n c e – N o v e m b e r 1 7 t h - M a n c h e s t e r To n Wes s e l i n g – C E O – Te s t i n g . A g e n c y
- 2. Manchester Ton Wesseling | @ TonW #leanconf Manchester 2014
- 3. Should I go for City? Ton Wesseling | @ TonW #leanconf Manchester 2014
- 4. Should I go for United? Ton Wesseling | @ TonW #leanconf Manchester 2014
- 5. Manchester Ton Wesseling | @ TonW #leanconf Manchester 2014
- 6. A/B-testing A/B-testing != CRO Ton Wesseling | @ TonW #leanconf Manchester 2014
- 7. A/B-testing CRO != user knowledge Ton Wesseling | @ TonW #leanconf Manchester 2014
- 8. A/B-testing A/B-testing == user knowledge Ton Wesseling | @ TonW #leanconf Manchester 2014
- 9. A/B-testing HOW & WHEN Ton Wesseling | @ TonW #leanconf Manchester 2014
- 10. A/B-testing EXAMPLES Ton Wesseling | @ TonW #leanconf Manchester 2014
- 11. A/B-testing PITTFALLS Ton Wesseling | @ TonW #leanconf Manchester 2014
- 12. Ton Wesseling | @ TonW Ton Wesseling | @ TonW #leanconf Manchester 2014 ü More testing ü Quality Assurance ü Lower pricing
- 13. Online Experiments – A/B-testing Default" Variation B" Variation Winner!" A" Ton Wesseling | @ TonW #leanconf Manchester 2014
- 14. Hyperlink junky Ton Wesseling | @ TonW #leanconf Manchester 2014
- 15. Digging through log files Ton Wesseling | @ TonW #leanconf Manchester 2014
- 16. Trying to improve Ton Wesseling | @ TonW #leanconf Manchester 2014
- 17. Over and over again Ton Wesseling | @ TonW #leanconf Manchester 2014
- 18. What most start-ups do Ton Wesseling | @ TonW #leanconf Manchester 2014
- 19. That is: 0 / 1 Testing Ton Wesseling | @ TonW #leanconf Manchester 2014
- 20. Why can this lead to failures? Ton Wesseling | @ TonW #leanconf Manchester 2014
- 21. 0/1 Testing: weekly change? 26 out of 44 weeks: over 5% change! Ton Wesseling | @ TonW #leanconf Manchester 2014
- 22. Testing should be like: Ton Wesseling | @ TonW #leanconf Manchester 2014
- 23. Scientific approach A / B Testing Live Online Research Ton Wesseling | @ TonW #leanconf Manchester 2014
- 24. Experiments “If you double the number of experiments you do per year you’re going to double your Ton Wesseling | @ TonW #leanconf Manchester 2014 inventiveness” Jeff Bezos, CEO Amazon – 2004!
- 25. Growth like Amazon Ton Wesseling | @ TonW #leanconf Manchester 2014 German Ecommerce market, thank you André Morys
- 26. Experiments per year 1000+ (but it did not start like that) Ton Wesseling | @ TonW #leanconf Manchester 2014
- 27. Growth curve Ton Wesseling | @ TonW #leanconf Manchester 2014
- 28. Phase 1: test solutions Ton Wesseling | @ TonW #leanconf Manchester 2014
- 29. Fake adwords testing Do they click on your idea? Ton Wesseling | @ TonW #leanconf Manchester 2014
- 30. Fake facebook testing Ton Wesseling | @ TonW #leanconf Manchester 2014 Do they like your idea?
- 31. Fake landing page testing Do they subscribe to your idea? Ton Wesseling | @ TonW #leanconf Manchester 2014
- 32. Make sure it’s not 0 / 1 Testing Ton Wesseling | @ TonW #leanconf Manchester 2014
- 33. Once you bridged the gap Ton Wesseling | @ TonW #leanconf Manchester 2014
- 34. The growth phase! Ton Wesseling | @ TonW #leanconf Manchester 2014
- 35. Duplicate? Ton Wesseling | @ TonW #leanconf Manchester 2014
- 36. It’s the web, it’s scalable! Ton Wesseling | @ TonW #leanconf Manchester 2014
- 37. You need to optimize! Ton Wesseling | @ TonW #leanconf Manchester 2014
- 38. Over and over again Ton Wesseling | @ TonW #leanconf Manchester 2014
- 39. It takes many experiments to win Ton Wesseling | @ TonW #leanconf Manchester 2014
- 40. Maximize your test capacity Time Ton Wesseling | @ TonW #leanconf Manchester 2014 Conversion%
- 41. Speed up – more people Ton Wesseling | @ TonW #leanconf Manchester 2014
- 42. Growth like Amazon Ton Wesseling | @ TonW #leanconf Manchester 2014 German Ecommerce market, thank you André Morys
- 43. Real growth Ton Wesseling | @ TonW #leanconf Manchester 2014
- 44. It takes heavy experimenting 250+ in a year Ton Wesseling | @ TonW #leanconf Manchester 2014
- 45. Not every experiment is successful Ton Wesseling | @ TonW #leanconf Manchester 2014
- 46. It takes a full stadium a week for 250+ Ton Wesseling | @ TonW #leanconf Manchester 2014
- 47. Or roughly 1000+ conversion points Ton Wesseling | @ TonW #leanconf Manchester 2014
- 48. Too slow? Energy will run out! Ton Wesseling | @ TonW #leanconf Manchester 2014
- 49. And you will never get this Ton Wesseling | @ TonW #leanconf Manchester 2014
- 50. You need the real fast paced culture Ton Wesseling | @ TonW #leanconf Manchester 2014
- 51. How does Bill win? Ton Wesseling | @ TonW #leanconf Manchester 2014
- 52. Emotion vs Ratio Ton Wesseling | @ TonW #leanconf Manchester 2014
- 53. Ton Wesseling | @ TonW #leanconf Manchester 2014
- 54. Why is your user here? Ton Wesseling | @ TonW #leanconf Manchester 2014
- 55. How can you help your user’s brain to fullfil it’s needs? Ton Wesseling | @ TonW #leanconf Manchester 2014
- 56. It’s not about A/B-testing Growth is about learning how to serve brains! Ton Wesseling | @ TonW #leanconf Manchester 2014
- 57. Testing costs time Use it to LEARN Ton Wesseling | @ TonW #leanconf Manchester 2014
- 58. FACT & ACT – 7 s t e p s t o l e a r n Find Analyze Create Test & Analyze Conclude Tell Ton Wesseling | @ TonW #leanconf Manchester 2014
- 59. Now: f o c u s o n C TA Create Test Analyze Ton Wesseling | @ TonW #leanconf Manchester 2014
- 60. Create = Data & Psychology Analyze & Experiment Digital Data Persuasion Psychology Ton Wesseling | @ TonW #leanconf Manchester 2014
- 61. Wheel-of-Persuasion.com Ton Wesseling | @ TonW #leanconf Manchester 2014
- 62. Persuasion on forms +25% sales Ton Wesseling | @ TonW #leanconf Manchester 2014
- 63. Same technique Significant more submits, but.. Less registrations Ton Wesseling | @ TonW #leanconf Manchester 2014
- 64. A small sentence +18% sales Ton Wesseling | @ TonW #leanconf Manchester 2014
- 65. But also +10% applications Ton Wesseling | @ TonW #leanconf Manchester 2014
- 66. Motivation Winner (sales) Ton Wesseling | @ TonW #leanconf Manchester 2014
- 67. Win your own form A/B-test http://testing.agency/manchester Ton Wesseling | @ TonW #leanconf Manchester 2014
- 68. And social proof? Ton Wesseling | @ TonW #leanconf Manchester 2014 Many went before you In november 1.675 people applied for a loan at MoneYou. Winner (sales)
- 69. Chunk your USP’s +14% sales Ton Wesseling | @ TonW #leanconf Manchester 2014
- 70. With learning: Phase 3 becomes easier Ton Wesseling | @ TonW #leanconf Manchester 2014
- 71. Testing becomes user knowledge Conversion)Op,miza,on) Market'Research' Ton Wesseling | @ TonW #leanconf Manchester 2014
- 72. Pittfalls How NOT to test Ton Wesseling | @ TonW #leanconf Manchester 2014
- 73. A/B-testing can be Ton Wesseling | @ TonW #leanconf Manchester 2014
- 74. Don’t trust your A/B-test analytics Ton Wesseling | @ TonW #leanconf Manchester 2014
- 75. It keeps you away from this failure Ton Wesseling | @ TonW #leanconf Manchester 2014
- 76. Enough traffic & conversions? Ton Wesseling | @ TonW #leanconf Manchester 2014
- 77. Low significance levels ü Big chance the winner is not a real winner Ton Wesseling | @ TonW #leanconf Manchester 2014
- 78. Low power levels ü Big chance a real winner is not recognized Ton Wesseling | @ TonW #leanconf Manchester 2014
- 79. Tested percentage is a spread! Ton Wesseling | @ TonW #leanconf Manchester 2014
- 80. Example test-setup Ton Wesseling | @ TonW #leanconf Manchester 2014
- 81. Average power: 40% Ton Wesseling | @ TonW #leanconf Manchester 2014
- 82. So: Ton Wesseling | @ TonW #leanconf Manchester 2014
- 83. Another error warning: ü You just go and dig unitil you find something (you will always find something that seems true BUT is not!) Ton Wesseling | @ TonW #leanconf Manchester 2014
- 84. OK, results are in Ton Wesseling | @ TonW #leanconf Manchester 2014
- 85. Not significant ü It’s not a winner, stop ü But, it’s does not mean it’s a loser! ü Average: 30% of your tests will have winners Ton Wesseling | @ TonW #leanconf Manchester 2014
- 86. More? http://ondi.me/contentverve http://ondi.me/conversionxl Ton Wesseling | @ TonW #leanconf Manchester 2014
- 87. Conclude ü Do nothing? or ü Re-test? or ü Implement? or ü Scale? and ü Continue or change test roadmap Ton Wesseling | @ TonW #leanconf Manchester 2014
- 88. Knowledge library: ABtestguide.com Ton Wesseling | @ TonW #leanconf Manchester 2014
- 89. It’s not about A/B-testing Growth is about learning how to serve brains! Ton Wesseling | @ TonW #leanconf Manchester 2014
- 90. So you will get this curve Ton Wesseling | @ TonW #leanconf Manchester 2014
- 91. Growth like Amazon Ton Wesseling | @ TonW #leanconf Manchester 2014 German Ecommerce market, thank you André Morys
- 92. A/B-testing HOW & WHEN Ton Wesseling | @ TonW #leanconf Manchester 2014
- 93. A/B-testing EXAMPLES Ton Wesseling | @ TonW #leanconf Manchester 2014
- 94. A/B-testing PITTFALLS Ton Wesseling | @ TonW #leanconf Manchester 2014
- 95. Slides & more http://testing.agency/manchester Ton Wesseling | @ TonW #leanconf Manchester 2014
- 96. Be like Bill, and win! Ton Wesseling | @ TonW #leanconf Manchester 2014
- 97. http://testing.agency/manchester HOW, W H E N , E X A M P L E S & P I T T FA L L S L e a n C o n f e r e n c e – N o v e m b e r 1 7 t h - M a n c h e s t e r To n Wes s e l i n g – C E O – Te s t i n g . A g e n c y