The document discusses the importance of market research for understanding customers, driving business growth, and preparing for the future. It recommends several methods for conducting market research including desk research, user empathy exercises, monitoring website usage, A/B testing, becoming a customer ("dogfooding"), and open interviews. Empathy exercises involve observing how real customers interact with a product or service without intervening. Dogfooding means using a company's own product as a customer to experience pains and gains. Remote monitoring and user testing capture unbiased, actual user sessions. A/B testing compares two variants to see which performs better. The goal is to better understand customers and homeowners to improve products, expand audiences, and address needs.

1. Increase Bookings, Guest Satisfaction,
and Owner Happiness through
Actionable (and fun) Market Research
Thibault Masson

2. These are tech employees in charge of developing our newest website features at our
Amsterdam headquarters.
Here, they were sent out to wait for Booking.com guests at the Amsterdam airport.
-> Why would we waste a day of these busy and expensive people to meet and greet guests who
had booked a holiday rental on our website?

3. Booking.com Homes & Apartments
Booking.com Homes & Apartments
● 795,332 holiday rental properties
● 3,000,000+ holiday rental units
● All whole places, all instant bookable
● Not including B&Bs
Thibault Masson
Commercial Owner,
Booking.com Homes & Apartments

4. Agenda
1. Why is market research important?
2. Internet resources you can use now
3. Empathy
4. Dogfooding (guests and homeowners)
5. Website observation
6. On Steroids: A/B testing

5. Why care about Market Research?
This is not just about buying expensive market reports.

6. When was the last time that you did market research?

7. And how?

9. Why is market research important?
● Understand your consumers
● Expand to new audiences
● Drive business growth
● Drive business development
● Get ready for the future

11. Common types of market research that we do
Desk Research
Reading up and staying informed
about our industry and trends.
User Empathy
Actually joining guests or partners
on their journey.
Monitoring / Recording
Look how actual users engage
with our platform.
A/B tests
Split an audience in two and
present them a different variant
each.
Dogfooding
Become a guest and experience
Booking.com from their
perspective.
Open Labs
Interviews with actual guests or
partners.

12. Common types of market research that we do
Desk Research
Reading up and staying informed
about our industry and trends.
User Empathy
Actually joining guests or partners
on their journey.
Monitoring / Recording
Look how actual users engage
with our platform.
A/B tests
Split an audience in two and
present them a different variant
each.
Dogfooding
Become a guest and experience
Booking.com from their
perspective.
Open Labs
Interviews with actual guests or
partners.

13. Need renowned research institutions, local industry stats, influencers, how-to tutorials to make
your business grow? All of this is available within an arm’s reach today.
Desk research

14. The digital age at your disposal...
● Research institutions
● Local travel Statistics
● Blogs, forums etc.
● Social Media
● Google

15. Common types of market research that we do
Desk Research
Reading up and staying informed
about our industry and trends.
User Empathy
Actually joining guests or partners
on their journey.
Monitoring / Recording
Look how actual users engage
with our platform.
A/B tests
Split an audience in two and
present them a different variant
each.
Dogfooding
Become a guest and experience
Booking.com from their
perspective.
Open Labs
Interviews with actual guests or
partners.

16. Observe users in their real environment
How are vacation rental guests and owners using your website? How do they plan a stay? How
do they search on your site? What is on their mind? What are the consequences of your website
features on an actual stay?

17. Be a silent observer
Watch and listen, do not
react, do not help.
Involve everyone
Help developers or web
agency develop empathy for
your users
Present findings
Gain buy-in by presenting
findings face to face to
stakeholders
Follow up
Involving key people from the
start makes easier to get the
issues in their agenda.

18. Example: Empathy exercise at Booking.com
Developers, marketers, designers sent out to welcome guests at the Amsterdam airport and
Central Station to silently observe how they find a property and go through checking in.

19. Waiting for guests

20. Diverse profiles of bookers handpicked.
Leigh & friends
7 friends from Ireland,
students, 1 couldn’t
make it in time
Philip & Denise
2 retired couples
from UK (4 people)
Coralie & Brian
Couple from AU,
*daughter joining
later
Jamie & Gavin
2 friends from the UK,
joining Andrew’s family.
*Andrew booked

21. Following them.

22. Figuring out the route.

23. Check-in experience: Why are guests waiting outside the property?
Figuring out where the door is? Are they early or is the host late?

24. When to use it?
Tools
How do travelers search for a
property on your site?
Checking-in at one of your properties
Benchmarking your competitors
What does it take to list a property on
your site in real-life?

25. Common types of market research that we do
Desk Research
Reading up and staying informed
about our industry and trends.
User Empathy
Actually joining guests or partners
on their journey.
Monitoring / Recording
Look how actual users engage
with our platform.
A/B tests
Split an audience in two and
present them a different variant
each.
Dogfooding
Become a guest and experience
Booking.com from their
perspective.
Open Labs
Interviews with actual guests or
partners.

26. Dogfooding
Dogfooding is a slang term used to reference a scenario in which a company uses its own product to validate the quality
and capabilities of the product.

27. First-hand experience of user pains
Stepping into the shoes of your guests or of your homeowners can help notice user pains that
can be hard to catch through data.

28. Photo of Japan Photo of Japan
2
Photo of Jeremie’s wedding
3
Tip: Dogfooding works if you have skin in the game
Dogfooding works even better if your team does this in a “real context”:
● Booking a a short-term rental for actual business trip in Japan,
● Booking a place for your friends and family for your engagement weekend.

29. Getting to the property

30. Getting to the property
was easier than we
thought!
Using host-provided instructions
like video and guides

31. 4 babies
with us.
Can we cook for
them?
Need to buy
food
Where are supermarkets
/ bakery?
Easy check-
in
No check-in time
Just a lock box
Need to store
food
Is there a large freezer?

32. Document everything
From searching a property to
checking out. Yet have only 1
question in mind.
Involve everyone
Have your developers or web
agency be part of a
dogfooding trip.
Present findings
Gain buy-in by presenting
findings face to face to
stakeholders
Follow up
Involving key people from the
start makes easier to get the
issues in their agenda.

33. Dogfooding as homeowners
Your employees who have listed their place on your website are an amazing source of feedback.
They can probably also compare your service with that of some of your competitors ;-)

34. Online forum for Booking.com employees who are also partners

35. When to use it?
Tools
Booking on your website
Listing a property on your website
Checking-in at one of your properties
Benchmarking your competitors

36. Common types of market research that we do
Desk Research
Reading up and staying informed
about our industry and trends.
User Empathy
Actually joining guests or partners
on their journey.
Monitoring / Recording
Look how actual users engage
with our platform.
A/B tests
Split an audience in two and
present them a different variant
each.
Dogfooding
Become a guest and experience
Booking.com from their
perspective.
Open Labs
Interviews with actual guests or
partners.

37. Remotely get feedback
When a user has a task to do your website, like updating their calendar, how do they go about
it? Record their online sessions or give them a task to do.

38. Record online sessions (anonymous data)
-> Get your developer to install code

39. Pay users to test your site
-> No code required, just give a task to do

40. Video sessions with your users (Open Lab)
-> Even 6 user video interviews can bring you a lot

41. Tools
● Tools like UserTesting.com,
UserTest.io and TestmyUi. : paid
user are asked to do a task (i.e.
book a trip for a family of 3)
● Tools like Hotjar, Fullstory:
unbiased, actual sessions
● Open Labs: present a concept or
look at how they use current
features -> Skype with recording
option
Tools Observe
Record Act

42. Common types of market research that we do
Desk Research
Reading up and staying informed
about our industry and trends.
User Empathy
Actually joining guests or partners
on their journey.
Monitoring / Recording
Look how actual users engage
with our platform.
A/B tests
Split an audience in two and
present them a different variant
each.
Dogfooding
Become a guest and experience
Booking.com from their
perspective.
Open Labs
Interviews with actual guests or
partners.

44. All experiments
OFF
All experiments ON

45. How hard is it to predict
customer behaviour?

46. A: No tabs opened from search results
B: New tab opened for each hotel clicked
C: No difference
Test 1

47. A: No suggestions
B: Suggestions for alternative weeks
C: No difference
Test 2

48. A: Shadow
B: No shadow
C: No difference
Test 3

49. USER-CENTERED DESIGN.
“We tested that, and it failed.”
● Making large, small and/or unnecessary changes not inherently linked to the raw
concept, which may have unintended consequences
● Noisy tracking
● Who’s the audience?
● Low traffic
● An idea ahead of its time?
http://blog.booking.com/concept-dne-execution.html

50. A/B testing
1. Failure is inevitable, so be transparent about it
2. Iteration is inherent in innovation
3. Ideas can come from anybody
4. Don’t be afraid to retest and mix things up
5. Testing isn’t just about technology development
http://www.cmo.com.au/article

51. Conclusion
Show that you care about guests and homeowners by better understanding their
needs and pains.
Empathy and dogfooding are easy, yet record everything and act on it.
Use available technology and services (e.g. uster testing websites).
If advanced and high traffic, A/B testing gets you live research results.

52. Thank you!
All references to “Booking.com", including any mention of “us”, “we” and “our” refer to Booking.com BV, the company behind Booking.com™