At Expedia Group, providing a customized experience for travellers is key to unlocking the best possibilities for each individual traveller and each type of trip.Contextual multi-armed bandits provide a natural approach to develop personalization of user experience and improve content relevancy. In this talk,we present the end-to-end scalable system developed to democratize the use of contextual bandits at EG.The architecture comprises of an online inference component as well as a continuous feedback loop that tracks the users’ affinity towards certain content or page layouts. Kafka is the backbone of our system, powering high-performance streaming jobs that provide bandits with real-time feedback signals to learn from over time. We describe our experience using Kafka for the user interactions events and bandit feedback messages at scale. Lastly,we look at how we plan to expand our use of Kafka to build an off policy evaluation framework to evaluate the effectiveness of new algorithms.

1. Building Adaptive User
Experiences Using
Contextual Multi-Armed
Bandits with Apache Kafka
Kafka Summit Americas 2021
© 2021 Expedia Group. All rights reserved.
Anca Sarb, Expedia Group
Gyula Magyar, EPAM Systems / Expedia Group

2. Let's start with a use case!
possible candidates
2
Expedia Group Proprietary and Confidential

3. 3
Multi-armed Bandits in a Nutshell
Expedia Group Proprietary and Confidential

4. 4
Core Use Cases for Multi-armed Bandits
Template Selection Module Ordering Hero Image Selection
Carousel Personalization Chatbot Intent Disambiguation Email/Push Targeting
0.901
0.907
0.899
Expedia Group Proprietary and Confidential

5. 5
Building the Bandit Learning Loop using Apache Kafka
client
application
Inference
Engine
Training
Service
Feedback
Gatherer
Streaming
Engine
user features, context
action, recommendation
user
interaction
events
bandit
feedback, rewards
bandit
feedback,
rewards
bandit
visualization
samples using
most up-to-date bandit
model parameters
updated
bandit model
parameters
relevant
raw bandit
events
bandit
model
parameters
raw
bandit
events
offline policy
evaluation
bandit
model
datastore
bandit
proposed
action
event
Expedia Group Proprietary and Confidential

6. 6
Benefits of using Apache Kafka
Expedia Group Proprietary and Confidential
§ scalability, can handle a large volume of data
§ replaying or re-processing older messages
§ schema management
§ schema registry
§ Avro for Kafka records
§ rich ecosystem
§ broad connector support
§ variety of producers and consumers written in a number of languages

7. You
Thank

8. © 2021 Expedia Group. All rights reserved.