Nhtv gastcollege - Methoden van onderzoek
•
0 likes•116 views
Gastcollega aan de NHTV over moderne technieken van onderzoek voor online bedrijven. Over o.a. A/Btesting, mousetracking, online surveys.
Recommended
Recommended
More Related Content
Similar to Nhtv gastcollege - Methoden van onderzoek
Similar to Nhtv gastcollege - Methoden van onderzoek (7)
More from Annemarie Klaassen
More from Annemarie Klaassen (10)
Nhtv gastcollege - Methoden van onderzoek
- 1. SUBTITLE BELOW Methoden van onderzoek @OnlineDialogue
- 3. Onze klanten
- 6. Wat is Conversie Optimalisatie? Een gestructureerd en systematisch proces Gebaseerd op inzichten uit kwantitatief en kwalitatief onderzoek Om meer bezoekers op de website een conversie te laten maken
- 9. Wat is een A/B-test? Een A/B-test is een experiment, waarbij 2 of meer variaties gelijktijdig worden getest en random verdeeld over bezoekers om te zien welke variant het beste presteert/converteert
- 11. A B
- 15. Waar kunnen we testen?
- 16. Tijd Conversiespermaand Risico Optimalisatie Automatisering Re-think 10.000 conversies per maand 1.000 conversies per maand
- 17. Waar kunnen we testen? Benoem alle verschillende pagina(type)s en bepaal: - # unieke bezoekers op die pagina - # unieke bezoekers met een conversie via de pagina Het je genoeg bezoekers en conversies (= test power) om een significant effect te herkennen?
- 18. Wat gaan we testen?
- 20. Opinions are like **sholes everyone has one
- 21. Without an analyzed reason to start, it doesn’t make any sense to start at all
- 23. Conversie studie Wat doen bezoekers nu op de website? Wat zouden ze moeten doen? Waarom doen bezoekers niet wat ze zouden moeten doen? Combineer kwantitatieve en kwalitatieve data
- 24. Verzamel klant inzichten Interview je klanten Bekijk feedback op Twitter, Facebook etc Luister mee/naar de klantenservice Vraag om feedback online
- 25. (Korte) online vragenlijsten Verzamel antwoorden in je analytics tool Koppel antwoorden aan meer gegevens
- 27. Sessie playbacks
- 28. Heat & scroll maps
- 32. Wetenschappelijk onderzoek A. Bogdanovych, H. Berger, S. Simoff, and C. Sierra, “Travel agents vs. online booking: Tackling the shortcomings of nowadays online tourism portals,” Information and Communication Technologies in Tourism 2006, pp. 418-428, 2006. V.P. Magnini, and K. Karande, “Understanding consumer services buyers based upon their purchase channel,” Journal of Business Research, vol. 64, no.6SI, pp. 543-550, 2011. W.G. Kim, X. Ma and D.J. Kim, “Determinants of Chinese hotel customers' e-satisfaction and purchase intentions,” Tourism Management, vol. 27, no. 5, pp. 890-900, 2006. A. Steinbauer, and H. Werthner, “Consumer Behaviour in e-Tourism, ” Information and Communication Technologies in Tourism 2007, no. 2, pp. 65- 76, 2007. H. Kim, T.T. Kim and S.W. Shin, “Modeling roles of subjective norms and eTrust in customers' acceptance of airline B2C eCommerce websites,” Tourism Management, vol. 30, no. 2, pp. 266-277, 2009. K. Matzler, and M. Waiguny, “Consequences of customer confusion in online hotel booking,” Information and Communication Technologies in Tourism 2005, pp. 306-317, 2005. C.C. Chen, and Z. Schwartz, “Room rate patterns and customers' propensity to book a hotel room,” Journal of Hospitality & Tourism Research, vol. 32, no. 2, pp. 287-306, 2008. C.C. Chen, Z. Schwartz and P. Vargas, “The search for the best deal: How hotel cancellation policies affect the search and booking decisions of deal- seeking customers,” International Journal of Hospitality Management, vol. 30, no. 1, pp. 129-135, 2011. R. Law, and R. Wong, “Analysing room rates and terms and conditions for the online booking of hotel rooms,” Asia Pacific Journal of Tourism Research, vol. 15, no. 1, pp. 43-56, 2010. C.O. Seneler, N. Basoglu and T. Daim, “Interface feature prioritization for web services: Case of online flight reservations,” Computers in Human Behavior, vol. 25, no. 4, pp. 862-877, 2009. S. Beldona, A.M. Morrison and J. O'Leary, “Online shopping motivations and pleasure travel products: a correspondence analysis,” Tourism Management, vol. 26, no. 4, pp. 561-570, 2005. H.Y. Lee, H. Qu and Y.S. Kim, “A study of the impact of personal innovativeness on online travel shopping behavior--A case study of Korean travelers,” Tourism Management, vol. 28, no. 2, pp. 886-897, 2007. H.S. Martin, and A. Herrero, “Influence of the user's psychological factors on the online purchase intention in rural tourism: Integrating innovativeness to the UTAUT framework,” Tourism Management, vol. 33, no. 2, pp. 341-350, 2012. C.C. Chen, and Z. Schwartz, “Timing matters: Travelers' advanced-booking expectations and decisions,” Journal of Travel Research, 47, no. 1, pp. 35- 42, 2008. R. Law, “Disintermediation of hotel reservations: The perception of different groups of online buyers in Hong Kong,” International Journal of Hospitality Management, vol. 21, no. 6-7, pp. 766-772, 2009. Y.S. Wang, and Y.W. Liao, “Understanding individual adoption of mobile booking service: An empirical investigation,” CyberPsychology & Behavior, vol. 11, no. 5, pp. 603-605, 2008.
- 35. Stel hypotheses op "If ___[I do this] ___, then ___[this behavioral change]___ will happen, because of ___[this reason]___ ”
- 36. Stel hypotheses op "If ___[I do this] ___, then ___[this behavioral change]___ will happen, because of ___[this reason]___ ”
- 37. Psychologie
- 40. Systeem 1 de baas Weinig vertrouwen in eigen kunnen (informatie overload, geen kennis, afleiding) Lage motivatie (beslissing is niet heel belangrijk / geen consequenties) Altijd aan en is emotioneel and impulsief
- 43. Systeem 2 de baas Veel vertrouwen in eigen kunnen (duidelijke informatie, veel kennis) Hoge motivatie (beslissing is erg belangrijk / heeft consequenties) Vaak afwezig, beperkte capaciteit en kost veel moeite
- 44. Voorbeeld Systeem 2 17 x 24=
- 45. GEEL BLAUW ORANJE ZWART ROOD GROEN PAARS GEEL ROOD ORANJE GROEN ZWART BLAUW ROOD PAARS Voorbeeld Systeem 2
- 46. Systeem 1 Systeem 2
- 48. Systeem 2 Je moet de aandacht trekken en behouden van bezoekers Focus op de belangrijkste taak
- 52. The Law of Distraction “Our rational thinking can’t deal with distraction”
- 53. 46% 5% 14% 4% 0,5% 20% Exit The Law of Distraction Bezoekers scrollen (vrijwel) niet 46% gebruikt de zoekbox 70% is prijzen aan het vergelijken
- 54. Winnaar A B The Law of Distraction
- 55. The Law of Distraction A Doelgericht: al gezocht op bestemming en specifieke datum Main banner kan een afleiding zijn Zoekresultaten staan onder de vouw
- 56. Verliezer A B Winnaar The Law of Distraction
- 57. # Dagen geboekt voor aankomst Systeem 1 Systeem 2
- 58. Focusing Effect “We can only consciously pay attention to one thing”
- 59. A Bezoekers hebben nog geen datum gekozen 50% klikt meteen op een hostel, slechts 18% vult datum in Datum selectie zit verstopt achter een link Focusing Effect
- 61. Paradox of Choice “If we are offered too many choices we tend not to make a choice at all”
- 62. Paradox of Choice Erg veel keuzes op 1 pagina: startabonnement (2 opties) / hoeveel internet? (6 opties / hoeveel belminuten? (3 options) / extra bundels? (6 opties) Erg klein percentage gaat door naar bestellen A
- 64. Systeem 1 Je moet de intrinsieke motivatie verhogen van de bezoekers Voeg affectieve/emotionele elementen toe
- 65. Autonomy “We prefer situations that we have control over”
- 66. Autonomy >35% gaat niet verder met bestellen Vrijwel alle velden zijn verplicht om in te vullen
- 68. Need for certainty “People want to be sure that a decision is the right decision”
- 69. A Need for certainty 45% gaat niet door naar de volgende stap Veel bezoekers wijzigen of verwijderen de net gekozen pas
- 70. A B Need for certainty Winnaar
- 71. Affect heuristic “We decide differently depending on our emotional state”
- 72. Affect heuristic A Bestelproces voor nieuwe klanten >50% gaat niet verder met bestellen
- 74. Mental accounting “We assign money to ‘mental categories’, and we spend money according to these categories.”
- 75. Mental accounting A Dure producten worden minder verkocht online Boxsprings hebben een veel lagere conversie ratio dan bijvoorbeeld beddengoed
- 82. Blije klanten!
Editor's Notes
- AN We do this for a bunch of clients in the Netherlands and also for some pretty cool international clients. For most we do high velocity testing. Which means , we run multiple tests per week for them.
- We combine data insights with psychological insights for evidence based growth.
- In stead of throwing spaghetti on the wall and hope something sticks
- In stead of throwing spaghetti on the wall and hope something sticks
- With every client we use this framework for conversion optimization. First we look at the data and determine the pages with the highest test power: which pages have enough visitors and conversions to be able to test on? Then we look at the paths visitors take on the website to make a booking or place an order. So, what are the main online customer journeys? And where are the biggest leaks in this process? These data findings are then send to the psychologist. He or she combines this data with scientific research to come up with hypothesis to test These hypothesis are then briefed to the designer who will come up with test variations. These variations are then tested in several A/B-tests (since you cannot prove an hypothesis based on one experiment) The learnings of these A/Btests are then combined in overall learnings which can then be shared with the rest of the organization.
- AN As you might expect we run lots and lots of experiment.
- The purpose of A/B-testing is of course to add direct value, but we still want to learn about user behavior. If you really want to learn from user behavior then you need to test very strict (say with >95%). Otherwise you only have a hunch, but you don’t have proof. A/B testing is straightforward: based on facts not opinions You can make a solid business case: by changing this, you will get x% more profit You can run A/B tests before making expensive, and difficult-to-change, technical investments
- AN
- First of all, do you know which pages will have the biggest impact on revenu. You want to test on those pages, because these tests will really make a difference to the business Secondly, if someone approaches you to do a test on page x can you tell him or her if that’s possible? And how much uplift you need from a test to declare it a winner? In other words, is this page a feasable page to test on?
- The first thing you do is map out all the different page types you have on your website, then look at the weekly unique visitors you have on that page type and the conversions through that page as well. Then you determine whether the pages have enough test power – based on these numbers.
- So if you where you can test, you need to figure out what you should test.
- If you see this website, you would probably want to change everything. It’s like a circus. But guess what:this is UK’s most favourite website to lease a car! They lease over 80 million dollars in cars every year.
- This is the Analyze phase. We want to know what the customer journey is. Where people have difficulty on the site etc.
- In stead of throwing spaghetti on the wall and hope something sticks
- AN
- JORRIN
- AN
- AN
- AN Vanuit de psychologie is bekend dat we beslissingen nemen op basis van 2 systemen: ofwel vanuit system 1 of system 2. Systeem 1 is de onderbewuste en emotionele route Systeem 2 is de bewuste en meer rationele route Voordat we starten met A/B-testen proberen we uit de data te achterhalen welk type systeem de overhand heeft in de customer journey. Afhankelijk namelijk van welk systeem de overhand heeft worden er andere psychologische technieken ingezet in het testen.
- Strooptaak in English!
- You could answer 2+2 in an instant, but this one is much harder. You really have to think and use your system 2 (but it’s slow and takes a lot of energy). The answer is 408
- Lees eerst de tekst: gaat heel snel en makkelijk (is system 1: zijn we gewend – als we tekst zien gaat we lezen) Nu niet de tekst lezen, maar de kleur van de tekst benoemen zijn we niet gewend / moeten we bij nadenken. Systeem 1 wil de overhand nemen.
- AN The most important system is thus system 1: this system is always on and needs to be overruled by system 2 if needed. But normally system 2 stays in the background: it’s lazy.
- Marshmallow test: good example of system 2: if you wait you can get two marshmallows, but system 1 is on and wants to eat the first one. They have an immense struggle to not eat the first marshmallow.
- We zien namelijk niet alles wat er is
- En we zien dingen anders dan ze daadwerkelijk zijn
- We hebben dus de volle aandacht nodig van de bezoeker
- “Our rational thinking can’t deal with distraction”
- This was actually a loser on desktop, but a clear winner on mobile.
- Make sure your most important feature gets all the attention
- AN OD05: Searchbox open Hypothesis Misschien zien bezoekers de mogelijkheid nu niet om de datum in te vullen. Het meer prominent maken van de datum selectie bespaart mentale energie, waardoor boekingen toenemen. Results Veel meer bezoekers vullen een datum in: +50% op mobile en +39% op tablet Conversies op tablet waren 7% lager, maar op mobile 4% hoger! Insights Voor doelgerichte bezoekers helpt het om de datum z.s.m in te vullen.
- Make sure your most important feature gets all the attention
- Ook getest voor telefoons, maar daar deed het niks. Is meer emotioneel product.
- Make sure your most important feature gets all the attention
- Make sure your most important feature gets all the attention
- When you feel sad you will make a more conservative choice, if you are happy you take more risks.
- JORRIN Results The chance that the variation will lead to more visitors on the enter details page is 99,8% In the test an uplift in conversion rate is measured of +2,61% (11,74% versus 11,44%). The probability that variation B leads to more bookers is 99,5% Insights Attention is an important factor on different hostelworld pages – adding a micro interaction had a positive effect on conversion
- JORRIN
- Zoektocht naar data driven inzichten op basis van veel vooronderzoek en verschillende testen.