#Conversion2015 Amsterdam keynote Ton Wesseling

1. ONLINE DIALOGU E G u i d i n g y o u t o d i g i t a l m a t u r i t y D e m y t h e n , d e l e u g e n s e n d e i l l u s i e s # C o n v e r s i o n 2 0 1 5 – A m s t e r d a m – A p r i l 1 4 t h 2 0 1 5 To n W e s s e l i n g A /B-testen

2. Email: ton@wesseling. net! Linkedin: t on wesseling! Tw it ter: @tonw! ! Ton Wesseling!

3. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Webanalisten.nl

4. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Online Dialogue

5. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Testing.agency

6. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Emerce magazine: conversion column

7. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Conversion Conferences worldwide

8. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Conversion speakers

9. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Where is Craig Sullivan?

10. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Replacing Craig…

11. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 At least he quotes me

12. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 So..

13. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 More experiments “If you double the number of experiments you do per year you’re going to double your inventiveness” Jeﬀ Bezos, CEO Amazon – 2004!

14. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Like Amazon, Like Zalando GermanEcommercemarket,thankyouAndréMorys

15. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Experiments per year 1000+

16. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015

17. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Why not? Compare to the past?

18. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Why can this lead to failures?

19. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 A big weekly change! 26 out of 44 weeks: over 5% change!

20. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Methods used to improve conversion

21. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Methods companies plan to use

22. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 It’s just HOT

23. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Lack of resources

24. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Test more

25. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Importance of a structured approach

26. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 More experiments

27. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 The negative effect trend Time span! Averageeffectofoptimzation! atcompaniesapplyingoptimization! Moneyspendon! Optimizationworldwide!

28. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 It’s not a solution It’s a method

29. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 It’s a way of working It’s DNA

30. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 So it’s a method: When How What

31. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Growth curve – how it was teached

32. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Optimization phases - ROAR Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 1.000 conversions! per month! 10.000 conversions per month!

33. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Phase 1: bridge the gap – Risk

34. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Fake adwords testing Do they click on your idea?

35. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Fake facebook testing Do they like your idea?

36. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Fake landing page testing Do they subscribe to your idea?

37. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Phase 1: bridge the gap – Risk

38. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Get out of the ofﬁce

39. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Risk

40. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 ROAR - Optimization Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 1.000 conversions! per month! 10.000 conversions per month!

41. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Warning: low signiﬁcance levels ü Big chance the winner is not a real winner

42. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Warning: low power levels ü Big chance a real winner is not recognized

43. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Example test set-up ü 90% significance level ü 100 conversions per variation ü Average conversion rate: 2% ü 20 out of 100 ideas is a winner… ü …with an average uplift of 10%

44. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Average: 75% signiﬁcance & 40% power

45. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 So: ü 100 tests will give 10 false positives (90% significance) ü 40% of the 20 winners are recognized(40% power) ü 8 real winners & 10 false positives ü NOT 18 improvements of 10%+

46. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 You need big numbers per variation Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 1.000 conversions! per month! 10.000 conversions per month!

47. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 With 1000 conversions + a month 20+ tests a year

48. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Tests with impact, on average 1 out of 3 Simply truth: you’re just not always able to create a winner with enough impact!

49. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 So in this case, you’ve impact Every 6 weeks

50. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Slower? Energy will run out!

51. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 What to test? Business cases! Delivery –test with in stock. 4 weeks or 2 days?!

52. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 What to test? Business cases! Service charge, do it or not? Include in price or not?!

53. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 What to test? Big design changes

54. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Warning: choose what to analyze up front ü You just go and dig unitil you find something (you will always find something that seems true BUT is not!)

55. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Don’t stop the test once it’s signiﬁcant!

56. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Don’t stop the test once it’s signiﬁcant!

57. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 You just let it run until it’s signiﬁcant

58. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 You just let it run until it’s signiﬁcant Your test has a fixed lenght! It’s even a ﬁxed amount of test population, but we love to test business cycles and/or weeks!

59. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 ABTestGuide.com/calc

60. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 ROAR – Optimization – moving up Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 1.000 conversions! per month! 10.000 conversions per month!

61. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Finally you can test details… Is Multi Variatie Testing (which is also A/B-testing but with more varations) a good idea? Almost never!!

62. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 You should grow to Customer Buying Reasons

63. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Reasons behind the customer journey

64. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Why is your user here?

65. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 How can you help your user’s brain to fullﬁl it’s needs?

66. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Data & Psychology Digital Data Persuasion Psychology Analyze & Experiment

67. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Testing takes time Use it to LEARN

68. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Lack of resources

69. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Create a test schedule

70. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 FA C T & A C T – 7 s t e p s t o l e a r n & Tell Conclude Analyze Test Create Analyze Find

71. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Source: Psychology

72. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Source: Ask

73. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Source: Listen

74. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Many sources out there to ﬁnd & analyze

75. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Test which dialogue has impact

76. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Analytics

77. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Heatmaps

78. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Which locations are critical

79. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 FA C T & A C T – 7 s t e p s t o l e a r n & Tell Conclude Analyze Test Create Analyze Find

80. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Now you’re testing On your way to 4 tests a week Max capacity with 1 teammember on each discipline: analyst, ux, developer, psychologyst!

81. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Your winners are still spare? Less than 25%? Please higher your signiﬁcance level to 95% or more if it drops below this 25 number!!

82. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Tip: Your test takes too long ü Cookie deletion (you will loose 10% in two weeks) They will re-enter the the and pollute your samples!

83. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Tip: Don’t confuse cross device users ü  Only test 1 device at a time!

84. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Tip: Maybe you’re testing broken stuff ü  Always check browser compatibility! ü  Dont’ break dynamic stuff!

85. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Tip: You forgot to leave out some visitors ü Only test on those who can be changed (if you test a sales promotion – leave out your current clients who just logon)

86. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Tip: You’re not slowing the control down ü  Add the same code to the control

87. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Tip: You’re not testing on fresh people ü Start with control for a full purchase cycle (and then start sending traffic to your variations – setting the control to 0%)

88. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Tip: You forgot to give them time to buy ü Stop getting fresh visitors after your test time (send new traffic to the control, but give you tested people time to finish converting)

89. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 ROAR – Automation Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 1.000 conversions! per month! 10.000 conversions per month!

90. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 You’re already thinking of more teams And still have test capacity *test capacity – the number of tests you can start per week based on your audience & conversions!

91. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Algorithms

92. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 You tested which locations are critical

93. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Automation

94. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Now you’ve moved to Algorithm testing Which internal and external inﬂuencers do have an impact on which segment of users?!

95. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 ROAR – Automation Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 1.000 conversions! per month! 10.000 conversions per month!

96. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 CRO becomes user knowledge

97. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 More impact on discover

98. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 A way of life

99. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Customer knowledge center Making money from conversion improvements Creating more and more customer insights Which can be monetized on even a larger scale

100. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Now, that’s HOT

101. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 So you keep on groing GermanEcommercemarket,thankyouAndréMorys

102. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Done testing? No! TESTING NEVER STOPS Okay, end of life, cash cow, no need for any further growth accelaration or user knowledge!

103. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 ROAR – when to start with what Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 1.000 conversions! per month! 10.000 conversions per month!

104. ONLINE DIALOGU E G u i d i n g y o u t o d i g i t a l m a t u r i t y D e m y t h e n , d e l e u g e n s e n d e i l l u s i e s # C o n v e r s i o n 2 0 1 5 – A m s t e r d a m – A p r i l 1 4 t h 2 0 1 5 To n W e s s e l i n g A /B-testen

105. Email: ton@wesseling. net! Linkedin: t on wesseling! Tw it ter: @tonw! ! Ton Wesseling!

#Conversion2015 Amsterdam keynote Ton Wesseling

You are reading a preview.

Check these out next

#Conversion2015 Amsterdam keynote Ton Wesseling

More Related Content

Slideshows for you (16)

Viewers also liked (9)

Similar to #Conversion2015 Amsterdam keynote Ton Wesseling (20)

More from Online Dialogue (20)

Recently uploaded (20)