Keynote on Online Experiments by Ton Wesseling - Online Dialogue at the Web Analytics Wednesday Amsterdam September 6th 2017

1. SUBTITLE BELOW! Online Experiments: ! Transactions! Information! Transformation!

2. Ton.Wesseling@onlinedialogue.com Key takeaways:! ! 1. Know your testing bandwidth!! 2. Hypothesize ﬁrst,! then apply to journeys/pages to test on ! 3. Prioritize, win and scale up! to learn and transform

3. Ton.Wesseling@onlinedialogue.com Since 1999! Ton Wesseling! @TonW!

4. Ton.Wesseling@onlinedialogue.com Challenge!

5. Ton.Wesseling@onlinedialogue.com Optimization!

6. Ton.Wesseling@onlinedialogue.com Innovation! ! Optimization!

8. Ton.Wesseling@onlinedialogue.com We believe!

9. Ton.Wesseling@onlinedialogue.com

10. Ton.Wesseling@onlinedialogue.com We believe in!

11. Ton.Wesseling@onlinedialogue.com First Party Data!

14. Ton.Wesseling@onlinedialogue.com Analyzing! First Party Data!

17. Ton.Wesseling@onlinedialogue.com Running! Online Experiments! ! on First Party Data!

18. Ton.Wesseling@onlinedialogue.com Online Experiments!

19. Ton.Wesseling@onlinedialogue.com Digital growth optimization! Analytics Psychology

20. Ton.Wesseling@onlinedialogue.com Business cases of our clients  Investment Revenue after implementation in 6 months! Company X! €82.000 €1.200.000! Company X! €89.000 €1.800.000 ! Company X! €150.000 €5.500.000! Company X! €166.000 €5.800.000! Company X! €230.000 €16.500.000!

21. Ton.Wesseling@onlinedialogue.com We believe!

22. Ton.Wesseling@onlinedialogue.com Innovation! ! Optimization! Online Experiments!

23. Ton.Wesseling@onlinedialogue.com Innovation!Optimization! Online Experiments!

24. Ton.Wesseling@onlinedialogue.com Innovation!Optimization! Online Experiments!

25. Ton.Wesseling@onlinedialogue.com Growth!

26. Ton.Wesseling@onlinedialogue.com “If you double the number of experiments you do per year you’re going to double your inventiveness” Jeﬀ Bezos, CEO Amazon

29. Ton.Wesseling@onlinedialogue.com How many! online experiments! are you running! per year?!

30. Ton.Wesseling@onlinedialogue.com “Our success at Amazon is a function of how many experiments we do per year, per month, per week, per day…” Jeﬀ Bezos, CEO Amazon

33. Ton.Wesseling@onlinedialogue.com William Edwards Deming

34. Ton.Wesseling@onlinedialogue.com Quality circle

36. Ton.Wesseling@onlinedialogue.com Methods used to improve conversion

37. Ton.Wesseling@onlinedialogue.com Methods companies plan to use

38. Ton.Wesseling@onlinedialogue.com A/B-testing is HOT

39. Ton.Wesseling@onlinedialogue.com More tests, more growth?

40. Ton.Wesseling@onlinedialogue.com Never satisﬁed with conversion rates

41. Ton.Wesseling@onlinedialogue.com Start testing or test more

42. Ton.Wesseling@onlinedialogue.com NO Growth problems will disappear…

43. Ton.Wesseling@onlinedialogue.com A/B-testing is not a solution It’s a methodology

44. Ton.Wesseling@onlinedialogue.com It’s part of a: Continuous Optimization Program

45. Ton.Wesseling@onlinedialogue.com Quality circle

46. Ton.Wesseling@onlinedialogue.com This is how Deming used it

48. Ton.Wesseling@onlinedialogue.com A 2009

49. Ton.Wesseling@onlinedialogue.com B 2009

50. Ton.Wesseling@onlinedialogue.com B A Winner

51. Ton.Wesseling@onlinedialogue.com Page tests Hypothesis

52. Ton.Wesseling@onlinedialogue.com A 2012

53. Ton.Wesseling@onlinedialogue.com B 2012

54. Ton.Wesseling@onlinedialogue.com Winner B A

55. Ton.Wesseling@onlinedialogue.com Page tests Customer Journey Hypothesis

57. Ton.Wesseling@onlinedialogue.com Did you already decide which hotel you want to stay in? Are you currently comparing the price of the rooms in this hotel at other websites?

59. Ton.Wesseling@onlinedialogue.com Book with your Valk account and get a €5 bonus Easily control your booking with your Valk account

60. Ton.Wesseling@onlinedialogue.com Take the next step

64. Ton.Wesseling@onlinedialogue.com the  Methodology  of Online Experiments

65. Ton.Wesseling@onlinedialogue.com Deming model

66. Ton.Wesseling@onlinedialogue.com Scientiﬁc method

67. Ton.Wesseling@onlinedialogue.com F! FIND T! TELL A! ANALYZE C! COMBINE C! CREATE A! ANALYZE T! TEST FACT & ACT

68. Ton.Wesseling@onlinedialogue.com F! FIND T! TELL A! ANALYZE C! COMBINE C! CREATE A! ANALYZE T! TEST F! FIND T! TELL A! ANALYZE C! COMBINE C! CREATE A! ANALYZE T! TEST

69. Ton.Wesseling@onlinedialogue.com ü User segments and customer journeys Science Competitors Veriﬁed" (2nd party) Webanalytics Heatmaps Surveys Feedback tools Group discussions View Voice Previous tests Veriﬁed" (1st party) Strategy KPI’s Value F! FIND 1) What is going on?

70. Ton.Wesseling@onlinedialogue.com get more insights from science

72. Ton.Wesseling@onlinedialogue.com ü Determinant study F! FIND 1) What will be the starting strategy?

73. Ton.Wesseling@onlinedialogue.com A! ANALYZE 2) Hypothesize I APPLY THISIf , then THIS BEHAVORIAL CHANGE will happen, ( among THIS GROUP ), THIS REASONbecause of . PSY DATA DATA PSY

74. Ton.Wesseling@onlinedialogue.com ü By challenging your hypothesis! 5V A! ANALYZE 2) Prioritize your hypothesis

75. Ton.Wesseling@onlinedialogue.com A! ANALYZE 2) Calculate your testing bandwidth

76. Ton.Wesseling@onlinedialogue.com 2) What to test where: Behavioral Intelligence Grid A! ANALYZE

77. Ton.Wesseling@onlinedialogue.com A! ANALYZE 2) Prioritize your hypothesis

78. Ton.Wesseling@onlinedialogue.com A! ANALYZE 2) Plot your hypothesis

79. Ton.Wesseling@onlinedialogue.com A! ANALYZE 2) Prioritize on potential impact

80. Ton.Wesseling@onlinedialogue.com ü Based on potential impact and power A! ANALYZE 2) Experiments roadmap

81. Ton.Wesseling@onlinedialogue.com Utilize your experiment capacity!

82. Ton.Wesseling@onlinedialogue.com A/B-test development resources

83. Ton.Wesseling@onlinedialogue.com A! ANALYZE C! COMBINE C! CREATE A! ANALYZE T! TEST T! TELL A! ANALYZE C! COMBINE C! CREATE A! ANALYZE T! TEST F! FIND

84. Ton.Wesseling@onlinedialogue.com C! COMBINE 6) Track experiments & store your insights

85. Ton.Wesseling@onlinedialogue.com ü With your new ﬁrst party data knowledge! 5V 6) Re-prioritize your hypothesis C! COMBINE

86. Ton.Wesseling@onlinedialogue.com Experiments with signiﬁcant insights months

87. Ton.Wesseling@onlinedialogue.com Growth!

88. Ton.Wesseling@onlinedialogue.com the  Methodology  of Online Experiments

90. Ton.Wesseling@onlinedialogue.com T! TELL 7) Transform by telling stories

92. Ton.Wesseling@onlinedialogue.com First Party Data!

93. Ton.Wesseling@onlinedialogue.com Running! Online Experiments! ! on First Party Data!

95. Ton.Wesseling@onlinedialogue.com Based on research

103. Ton.Wesseling@onlinedialogue.com This is how Continuous Optimization Efforts! are ACCELERATING the Growth of a Company!

104. Ton.Wesseling@onlinedialogue.com How many! online experiments! are you running! per year?!

105. Ton.Wesseling@onlinedialogue.com Do you know how many! online experiments! you could run ! per year?!

106. Ton.Wesseling@onlinedialogue.com Key takeaways:! ! 1. Know your testing bandwidth!! 2. Hypothesize ﬁrst,! then apply to journeys/pages to test on ! 3. Prioritize, win and scale up! to learn and transform

107. SUBTITLE BELOW! Online Experiments: ! Transactions! Information! Transformation!

108. Ton.Wesseling@onlinedialogue.com Ton Wesseling! @TonW!

Keynote on Online Experiments by Ton Wesseling - Online Dialogue at the Web Analytics Wednesday Amsterdam September 6th 2017

You are reading a preview.

Check these out next

Keynote on Online Experiments by Ton Wesseling - Online Dialogue at the Web Analytics Wednesday Amsterdam September 6th 2017

More Related Content

Slideshows for you (20)

Similar to Keynote on Online Experiments by Ton Wesseling - Online Dialogue at the Web Analytics Wednesday Amsterdam September 6th 2017 (20)

More from Online Dialogue (20)

Recently uploaded (20)