Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Where to Go on Your Next Trip? Optimizing Travel Destinations Based on User Preferences

1,616 views

Published on

Recommendation based on user preferences is a common
task for e-commerce websites. New recommendation algorithms
are often evaluated by offline comparison to baseline
algorithms such as recommending random or the most
popular items. Here, we investigate how these algorithms
themselves perform and compare to the operational production
system in large scale online experiments in a real-world
application. Specifically, we focus on recommending travel
destinations at Booking.com, a major online travel site, to
users searching for their preferred vacation activities. To
build ranking models we use multi-criteria rating data provided
by previous users after their stay at a destination. We
implement three methods and compare them to the current
baseline in Booking.com: random, most popular, and Naive
Bayes. Our general conclusion is that, in an online A/B test
with live users, our Naive-Bayes based ranker increased user
engagement significantly over the current online system

Published in: Internet
  • Be the first to comment

Where to Go on Your Next Trip? Optimizing Travel Destinations Based on User Preferences

  1. 1. Where to Go on Your Next Trip? Optimizing Travel Destination Based on User Preferences Julia Kiseleva, Melanie J.I. Mueller, Lucas Bernardi, Chad Davis, Ivan Kovacek, Mats Stafseng Einarsen, Jaap Kamps, Alexander Tuzhilin, Djoerd Hiemstra SIGIR 2015, Chile
  2. 2. Booking.com
  3. 3. Booking.com • World largest online travel agent • > 220 countries • > 81.000 destinations • > 710.000 bookable hotels worldwide • > 30.000.000 unique users • >> 100.000 unique visitors per day
  4. 4. Destination Finder
  5. 5. Destination Finder
  6. 6. Destination Finder
  7. 7. How Do We Get Endorsements? • Only users who stayed at a hotel in a destination can endorse it • Free text endorsements since 2013 • Since 2014 free text endorsements standardized to 256 canonical tags • Used NLP techniques to extract the canonical base More Numbers: • > 13.000.000 total unique endorsements • > 60.000 destinations
  8. 8. Endorsement Histogram
  9. 9. Endorsement Histogram
  10. 10. Endorsement-Destination Histogram
  11. 11. Endorsement-Destination Histogram
  12. 12. Endorsements for Santiago
  13. 13. Destination Finder Problem: How to Optimize ranking of recommended destinations?
  14. 14. Problem Setup • Challenge: o Recommender or Information Retrieval System? • Information Retrieval: o Users have information need which is expressed as a query o System has to satisfy this information needs • Recommender: o System predicts what users might be interested We have both: 1) Users search for activities 2) Users don’t know where they want to go
  15. 15. Why is It Hard? Problem Characteristics: • S - Sparsity: new or rare users/destinations • V - Volatility: users’ interests/endorsements of destinations change over time • I - Identity: a failure to match data from the same users • P - Personas: users have different interests at different, possibly closely points in time Continuous Cold Start Problem!
  16. 16. Ranking Destinations for ‘Beach’ • Keep it simple!!! We care about performance! • Naïve Bayes: P(Miami, Beach) = P(Miami) * P(Miami | Beach) P(Miami | Beach) = # ‘Beach’ endorsements for Miami # ‘Beach’ endorsements
  17. 17. What and How to Compare? • Booking.com Baseline • Random • Most Popular Destination • Naïve Bayes Objection: Increase User Engagement (Clickers per SERP)
  18. 18. A/B Testing Setup • 50/50 traffic split • Experiments run for N full weeks according to desired power and significance levels • Hypothesis tests are performed according to targeted metrics (G-test in our case)
  19. 19. A/B Testing Version A Version B Version C
  20. 20. A/B Testing Ranker # Users Engagement Baseline 9928 Random 10079 Popularity 9838 Naïve Bayes 9895 Amount of clickers
  21. 21. A/B Testing Ranker # Users Engagement Baseline 9928 25.61% +/- 0.72% Random 10079 24.46% +/- 0.71% Popularity 9838 25.50% +/- 0.73% Naïve Bayes 9895 26.73% +/- 0.73%
  22. 22. A/B Testing Ranker # Users Engagement Baseline 9928 25.61% +/- 0.72% Random 10079 24.46% +/- 0.71% Popularity 9838 25.50% +/- 0.73% Naïve Bayes 9895 26.73% +/- 0.73%
  23. 23. Conclusion and Future Work • Interesting application • Surprise 1: Keep simple baseline in production system • Surprise 2: Random performed not bad => effect serendipity For the Future: • Improve the ranking by taking contexts into account
  24. 24. Conclusion and Future Work • Interesting application • Surprise 1: Keep simple baseline in production system • Surprise 2: Random performed not bad => effect serendipity For the Future: • Improve the ranking by taking contexts into account

×