SlideShare a Scribd company logo
Recent Trends in
Personalization:
A Netflix Perspective
Justin Basilico
ICML 2019 Adaptive & Multi-Task Learning Workshop
2019-06-15
@JustinBasilico
Why do we personalize?
Help members find content
to watch and enjoy to maximize
member satisfaction and retention
Spark joy
What do we personalize?
Ordering of videos is personalized
From what we recommend
Ranking
Selection and placement of rows is personalized
... to how we construct a pageRows
Personalized images
... to what images to select
... to reaching out to our members
Everything is a recommendation!
Over 80% of what
people watch
comes from our
recommendations
Overview in [Gomez-Uribe & Hunt, 2016]
Isn’t this solved yet?
○ Every person is unique with a variety of interests
○ Help people find what they want when they’re not sure what they want
○ Large datasets but small data per user
… and potentially biased by the output of your system
○ Cold-start problems on all sides
○ Non-stationary, context-dependent, mood-dependent
○ More than just accuracy: Diversity, novelty, freshness, fairness, ...
○ ...
No, personalization is hard!
Some recent trends in approaching these challenges:
1. Deep Learning
2. Causality
3. Bandits & Reinforcement Learning
4. Fairness
5. Experience Personalization
Trending Now
Trend 1: Deep Learning in
Recommendations
What~2012 ~2017
Deep Learning becomes popular in
Machine Learning
Deep Learning becomes popular in
Recommender Systems
What took so long?
Traditional Recommendations
Collaborative Filtering:
Recommend items that
similar users have chosen
0 1 0 1 0
0 0 1 1 0
1 0 0 1 1
0 1 0 0 0
0 0 0 0 1
Users
Items
U≈R
V
A Matrix Factorization view
2
U
A Feed-Forward Network view
V
2
U
A (deeper) feed-forward view
V
Mean
squared loss?
… isn’t always the best
U
V
Mean squared
loss
?
V
… but opens up many possibilities
Softmax
Avg / Stack/
Sequence
DNN / RNN / CNN
Input
interactions
(X)
(X)
p(Y)
2018-12-2319:32:10
2018-12-2412:05:53
2019-01-0215:40:22
Sequence prediction
● Treat recommendations as a
sequence classification problem
○ Input: sequence of user actions
○ Output: next action
● E.g. Gru4Rec [Hidasi et. al., 2016]
○ Input: sequence of items in a sessions
○ Output: next item in the session
● Also co-evolution: [Wu et al.,
2017], [Dai et al., 2017]
Leveraging other data
● Example: YouTube Recommender
[Covington et. al., 2016]
● Two stage ranker: candidate
generation (shrinking set of items
to rank) and ranking (classifying
actual impressions)
● Two feed-forward, fully
connected, networks with
hundreds of features
Contextual sequence data
2017-12-10 15:40:22
2017-12-23 19:32:10
2017-12-24 12:05:53
2017-12-27 22:40:22
2017-12-29 19:39:36
2017-12-30 20:42:13
Context ItemSequence
per user
?
Time
Time-sensitive sequence prediction
● Proper modeling of time and system dynamics is critical
○ Recommendations are actions at a moment in time
● Experiment on a Netflix internal dataset
○ Input: Sequence of past plays and time context
■ Discrete time: Day-of-week (Mon, Tue, …) & Hour-of-day
■ Continuous time (aka timestamp)
○ Label: Predict next play (temporal split data)
Results
Trend 2: Causality
From Correlation to Causation
● Most recommendation
algorithms are correlational
○ Some early recommendation
algorithms literally computed
correlations between users
and items
● Did you watch a movie
because you liked it? Or
because we showed it to
you? Or both? p(Y|X) → p(Y|X, do(R))
(from http://www.tylervigen.com/spurious-correlations)
Feedback loops
Impression bias
inflates plays
Leads to inflated
item popularity
More plays
More
impressions
Oscillations in
distribution of genre
recommendations
Feedback loops can cause biases to be
reinforced by the recommendation system!
[Chaney et al., 2018]: simulations showing that this can reduce the
usefulness of the system
Lots of feedback loops...
Closed Loop
Training
Data
Watches Model
Recs
Closed Loop
Training
Data
Watches Model
Recs
Danger Zone
Closed Loop
Training
Data
Watches Model
Recs
Danger Zone
Search
Training
Data
Watches Model
Recs
Open Loop
Closed Loop
Training
Data
Watches Model
Recs
Danger Zone
Search
Training
Data
Watches Model
Recs
Open Loop
Debiasing Recommendations
● IPS Estimator for MF [Schnabel et al., 2016]
○ Train a debiasing model and reweight the data
● Causal Embeddings [Bonner & Vasile, 2018]
○ Jointly learn debiasing model and task model
○ Regularize the two towards each other
● Doubly-Robust MF [Wang et al., 2019]
Trend 3: Bandits &
Reinforcement Learning in
Recommendations
● Uncertainty around user interests and new items
● Sparse and indirect feedback
● Changing trends
● Break feedback loops
● Want to explore to learn
Why contextual bandits for recommendations?
▶Early news example: [Li et al., 2010]
Bart [McInerney et al., 2018]
● Bandit selecting both items and explanations for
Spotify homepage
● Factorization Machine with epsilon-greedy explore
over personalized candidate set
● Counterfactual risk minimization to train the bandit
Which artwork to show?
Artwork Personalization as
Contextual Bandit
● Environment: Netflix homepage
● Context: Member, device, page, etc.
● Learner: Artwork selector for a show
● Action: Display specific image for show
● Reward: Member has positive engagement
Artwork Selector
▶
Offline Replay Results
● Bandit finds good images
● Personalization is better
● Artwork variety matters
● Personalization wiggles
around best images
Lift in Replay in the various algorithms as
compared to the Random baseline
More info in our blog post
Going Long-Term
● Want to maximize long-term user satisfaction and retention
● Involves many user visits, recommendation actions and delayed reward
● … sounds like Reinforcement Learning
● High-dimensional action space: Recommending a single item is O(|C|);
typically want to do ranking or page construction, which is combinatorial
● High-dimensional state space: Users are represented in the state, along
with the relevant history
● Off-policy training: Need to learn from existing system actions
● Concurrency: Don’t observe full trajectories, need to learn simultaneously
from many interactions
● Changing action space: New actions (items) become available and need to
be cold-started.
● No good simulator: Requires knowing feedback for user on recommended
items
Challenges of Reinforcement Learning for
Recommendations
List-wise [Zhao et al., 2017] or Page-wise recommendation [Zhao et al. 2018]
based on [Dulac-Arnold et al., 2016]
Embeddings for actions
● Generator to choose user action from recommendation
● Reward trained like a discriminator
● LSTM or Position-Weight architecture
● Learning over sets via cascading Deep Q Networks
○ Different Q function per position
GAN-inspired as a user simulator
[Chen et al., 2019]
● Train candidate generator using
REINFORCE
● Exploration done using softmax with
temperature
● Off-policy correction with adaptation for
top-k recommendations
● Trust region policy optimization to keep
close to logging policy
Policy Gradient for YouTube
Recommendations [Chen et al., 2019]
Trend 4: Fairness
Personalization has a big impact in people’s lives
How do we make sure that it is fair?
Calibrated Recommendations [Steck, 2018]
● Fairness as matching distribution of user interests
● Accuracy as an objective can lead to unbalanced predictions
● Simple example:
● Many recommendation algorithms exhibit this behavior of exaggerating the
dominant interests and crowd out less frequent ones
30 action70 romance
30% action70% romance
User:
Expectation:
100% romanceReality: Maximizes accuracy
- Genre-distribution of each item is given:
- Genre-distribution of user’s play history:
… add prior for other genres:
- Genre-distribution of recommended list:
(for diversity)
(or other categorization)
Calibration Metric
Calibration Results (MovieLens 20M)
Baseline model (wMF):
Many users receive
uncalibrated rec’s
After reranking:
Rec’s are much more
calibrated (smaller )
Userdensity
More calibrated (KL divergence)
Submodular
Reranker:
Fairness through Pairwise Comparisons
[Beutel et al., 2019]
● Recommendations are fair if likelihood of clicked item being ranked above
an unclicked item is the same across two groups
○ Intra-group pairwise accuracy - Restrict to pairs within group
○ Inter-group pairwise accuracy - Restrict to pairs between groups
● Training: Add pairwise regularizer based on randomized data to collect
fairness feedback
Trend 5:
Experience Personalization
Personalizing how we recommend
(not just what we recommend…)
● Algorithm level: Ideal balance of diversity, popularity,
novelty, freshness, etc. may depend on the person
● Display level: How you present items or explain
recommendations can also be personalized
● Interaction level: Balancing the needs of lean-back
users and power users
Page/Slate Optimization
● Select multiple actions that go together and
receive feedback on group
● Personalizing based on within-session
browsing behavior [Wu et al., 2015]
● Off-policy evaluation for slates
[Swaminathan, et al., 2016]
● Slate optimization as VAE [Jiang et al., 2019]
● Marginal posterior sampling for slate bandits
[Dimakopoulou et al., 2019]
More dimensions to personalize
Rows
Trailer
Evidence
Synopsis
Image
Row Title
Metadata
Ranking
More Adaptive UI
Rating Ranking Pages
4.7
Experience
Evolution of our Personalization Approach
Potential Connections with
Multi-Task / Meta Learning?
Applications as tasks
● Many related personalization tasks in a
recommender system
● Examples:
○ [Zhao et al., 2015] - Outputs for different tasks
○ [Bansal et al., 2016] - Jointly learn to recommend
and predict metadata for items
○ [Ma et al., 2018] - Jointly learn watch and enjoy
○ [Lu et al., 2018] - Jointly learn for rating prediction
and explanation
○ [Hadash et al., 2018] - Jointly learn ranking and
rating prediction
User
history
Ranking
Page
Rating
Explanation
Search
Image
Context ...
Other views
● Users-as-tasks: Treat each user as a task and learn from others users
○ Example: [Ning & Karapis, 2010] finds similar users and does
support vector regression
● Items-as-tasks: Treat each item as a separate model to learn
● Contexts-as-tasks: Treat different contexts (time, device, region, …)
as separate tasks
● Domains-as-tasks: Leverage representations of users in one domain
to help in another (e.g. different kinds of items, different genres)
○ Example: [Li et al., 2009] on movies <-> books
Conclusion
1. Deep Learning
2. Causality
3. Bandits & Reinforcement Learning
4. Fairness
5. Experience Personalization
6. Multi-task & Meta Learning?
Lots of opportunity for Machine Learning
in Personalization
Thank you
Questions?
@JustinBasilico Yes, we’re hiring...
Justin Basilico

More Related Content

What's hot

Tutorial on Deep Learning in Recommender System, Lars summer school 2019
Tutorial on Deep Learning in Recommender System, Lars summer school 2019Tutorial on Deep Learning in Recommender System, Lars summer school 2019
Tutorial on Deep Learning in Recommender System, Lars summer school 2019
Anoop Deoras
 
Deeper Things: How Netflix Leverages Deep Learning in Recommendations and Se...
 Deeper Things: How Netflix Leverages Deep Learning in Recommendations and Se... Deeper Things: How Netflix Leverages Deep Learning in Recommendations and Se...
Deeper Things: How Netflix Leverages Deep Learning in Recommendations and Se...
Sudeep Das, Ph.D.
 
Contextualization at Netflix
Contextualization at NetflixContextualization at Netflix
Contextualization at Netflix
Linas Baltrunas
 
Making Netflix Machine Learning Algorithms Reliable
Making Netflix Machine Learning Algorithms ReliableMaking Netflix Machine Learning Algorithms Reliable
Making Netflix Machine Learning Algorithms Reliable
Justin Basilico
 
Time, Context and Causality in Recommender Systems
Time, Context and Causality in Recommender SystemsTime, Context and Causality in Recommender Systems
Time, Context and Causality in Recommender Systems
Yves Raimond
 
Missing values in recommender models
Missing values in recommender modelsMissing values in recommender models
Missing values in recommender models
Parmeshwar Khurd
 
Learning a Personalized Homepage
Learning a Personalized HomepageLearning a Personalized Homepage
Learning a Personalized Homepage
Justin Basilico
 
A Multi-Armed Bandit Framework For Recommendations at Netflix
A Multi-Armed Bandit Framework For Recommendations at NetflixA Multi-Armed Bandit Framework For Recommendations at Netflix
A Multi-Armed Bandit Framework For Recommendations at Netflix
Jaya Kawale
 
Déjà Vu: The Importance of Time and Causality in Recommender Systems
Déjà Vu: The Importance of Time and Causality in Recommender SystemsDéjà Vu: The Importance of Time and Causality in Recommender Systems
Déjà Vu: The Importance of Time and Causality in Recommender Systems
Justin Basilico
 
Homepage Personalization at Spotify
Homepage Personalization at SpotifyHomepage Personalization at Spotify
Homepage Personalization at Spotify
Oguz Semerci
 
Recommending for the World
Recommending for the WorldRecommending for the World
Recommending for the World
Yves Raimond
 
Personalized Page Generation for Browsing Recommendations
Personalized Page Generation for Browsing RecommendationsPersonalized Page Generation for Browsing Recommendations
Personalized Page Generation for Browsing Recommendations
Justin Basilico
 
Artwork Personalization at Netflix Fernando Amat RecSys2018
Artwork Personalization at Netflix Fernando Amat RecSys2018 Artwork Personalization at Netflix Fernando Amat RecSys2018
Artwork Personalization at Netflix Fernando Amat RecSys2018
Fernando Amat
 
Calibrated Recommendations
Calibrated RecommendationsCalibrated Recommendations
Calibrated Recommendations
Harald Steck
 
RecSys 2020 A Human Perspective on Algorithmic Similarity Schendel 9-2020
RecSys 2020 A Human Perspective on Algorithmic Similarity Schendel 9-2020RecSys 2020 A Human Perspective on Algorithmic Similarity Schendel 9-2020
RecSys 2020 A Human Perspective on Algorithmic Similarity Schendel 9-2020
Zachary Schendel
 
Recent Trends in Personalization at Netflix
Recent Trends in Personalization at NetflixRecent Trends in Personalization at Netflix
Recent Trends in Personalization at Netflix
Förderverein Technische Fakultät
 
Data council SF 2020 Building a Personalized Messaging System at Netflix
Data council SF 2020 Building a Personalized Messaging System at NetflixData council SF 2020 Building a Personalized Messaging System at Netflix
Data council SF 2020 Building a Personalized Messaging System at Netflix
Grace T. Huang
 
Recommendation at Netflix Scale
Recommendation at Netflix ScaleRecommendation at Netflix Scale
Recommendation at Netflix Scale
Justin Basilico
 
Personalizing the listening experience
Personalizing the listening experiencePersonalizing the listening experience
Personalizing the listening experience
Mounia Lalmas-Roelleke
 
Interactive Recommender Systems with Netflix and Spotify
Interactive Recommender Systems with Netflix and SpotifyInteractive Recommender Systems with Netflix and Spotify
Interactive Recommender Systems with Netflix and Spotify
Chris Johnson
 

What's hot (20)

Tutorial on Deep Learning in Recommender System, Lars summer school 2019
Tutorial on Deep Learning in Recommender System, Lars summer school 2019Tutorial on Deep Learning in Recommender System, Lars summer school 2019
Tutorial on Deep Learning in Recommender System, Lars summer school 2019
 
Deeper Things: How Netflix Leverages Deep Learning in Recommendations and Se...
 Deeper Things: How Netflix Leverages Deep Learning in Recommendations and Se... Deeper Things: How Netflix Leverages Deep Learning in Recommendations and Se...
Deeper Things: How Netflix Leverages Deep Learning in Recommendations and Se...
 
Contextualization at Netflix
Contextualization at NetflixContextualization at Netflix
Contextualization at Netflix
 
Making Netflix Machine Learning Algorithms Reliable
Making Netflix Machine Learning Algorithms ReliableMaking Netflix Machine Learning Algorithms Reliable
Making Netflix Machine Learning Algorithms Reliable
 
Time, Context and Causality in Recommender Systems
Time, Context and Causality in Recommender SystemsTime, Context and Causality in Recommender Systems
Time, Context and Causality in Recommender Systems
 
Missing values in recommender models
Missing values in recommender modelsMissing values in recommender models
Missing values in recommender models
 
Learning a Personalized Homepage
Learning a Personalized HomepageLearning a Personalized Homepage
Learning a Personalized Homepage
 
A Multi-Armed Bandit Framework For Recommendations at Netflix
A Multi-Armed Bandit Framework For Recommendations at NetflixA Multi-Armed Bandit Framework For Recommendations at Netflix
A Multi-Armed Bandit Framework For Recommendations at Netflix
 
Déjà Vu: The Importance of Time and Causality in Recommender Systems
Déjà Vu: The Importance of Time and Causality in Recommender SystemsDéjà Vu: The Importance of Time and Causality in Recommender Systems
Déjà Vu: The Importance of Time and Causality in Recommender Systems
 
Homepage Personalization at Spotify
Homepage Personalization at SpotifyHomepage Personalization at Spotify
Homepage Personalization at Spotify
 
Recommending for the World
Recommending for the WorldRecommending for the World
Recommending for the World
 
Personalized Page Generation for Browsing Recommendations
Personalized Page Generation for Browsing RecommendationsPersonalized Page Generation for Browsing Recommendations
Personalized Page Generation for Browsing Recommendations
 
Artwork Personalization at Netflix Fernando Amat RecSys2018
Artwork Personalization at Netflix Fernando Amat RecSys2018 Artwork Personalization at Netflix Fernando Amat RecSys2018
Artwork Personalization at Netflix Fernando Amat RecSys2018
 
Calibrated Recommendations
Calibrated RecommendationsCalibrated Recommendations
Calibrated Recommendations
 
RecSys 2020 A Human Perspective on Algorithmic Similarity Schendel 9-2020
RecSys 2020 A Human Perspective on Algorithmic Similarity Schendel 9-2020RecSys 2020 A Human Perspective on Algorithmic Similarity Schendel 9-2020
RecSys 2020 A Human Perspective on Algorithmic Similarity Schendel 9-2020
 
Recent Trends in Personalization at Netflix
Recent Trends in Personalization at NetflixRecent Trends in Personalization at Netflix
Recent Trends in Personalization at Netflix
 
Data council SF 2020 Building a Personalized Messaging System at Netflix
Data council SF 2020 Building a Personalized Messaging System at NetflixData council SF 2020 Building a Personalized Messaging System at Netflix
Data council SF 2020 Building a Personalized Messaging System at Netflix
 
Recommendation at Netflix Scale
Recommendation at Netflix ScaleRecommendation at Netflix Scale
Recommendation at Netflix Scale
 
Personalizing the listening experience
Personalizing the listening experiencePersonalizing the listening experience
Personalizing the listening experience
 
Interactive Recommender Systems with Netflix and Spotify
Interactive Recommender Systems with Netflix and SpotifyInteractive Recommender Systems with Netflix and Spotify
Interactive Recommender Systems with Netflix and Spotify
 

Similar to Recent Trends in Personalization: A Netflix Perspective

Big & Personal: the data and the models behind Netflix recommendations by Xa...
 Big & Personal: the data and the models behind Netflix recommendations by Xa... Big & Personal: the data and the models behind Netflix recommendations by Xa...
Big & Personal: the data and the models behind Netflix recommendations by Xa...
BigMine
 
Qualitative Research vs Quantitative Research - a QuestionPro Academic Webinar
Qualitative Research vs Quantitative Research - a QuestionPro Academic WebinarQualitative Research vs Quantitative Research - a QuestionPro Academic Webinar
Qualitative Research vs Quantitative Research - a QuestionPro Academic Webinar
QuestionPro
 
Recommender Systems In Industry
Recommender Systems In IndustryRecommender Systems In Industry
Recommender Systems In Industry
Xavier Amatriain
 
Pivotal Tracker - Research Findings
Pivotal Tracker - Research FindingsPivotal Tracker - Research Findings
Pivotal Tracker - Research Findings
Paulina Galindo
 
Recommandation systems -
Recommandation systems - Recommandation systems -
Recommandation systems -
Yousef Fadila
 
Introduction to Recommendation Systems
Introduction to Recommendation SystemsIntroduction to Recommendation Systems
Introduction to Recommendation Systems
Zia Babar
 
Research portfolio
Research portfolioResearch portfolio
Research portfolio
ElkaMao
 
Past, present, and future of Recommender Systems: an industry perspective
Past, present, and future of Recommender Systems: an industry perspectivePast, present, and future of Recommender Systems: an industry perspective
Past, present, and future of Recommender Systems: an industry perspective
Xavier Amatriain
 
Applied Data Science for E-Commerce
Applied Data Science for E-CommerceApplied Data Science for E-Commerce
Applied Data Science for E-Commerce
Arul Bharathi
 
Recommendation engines
Recommendation enginesRecommendation engines
Recommendation engines
Georgian Micsa
 
Recommender systems
Recommender systems Recommender systems
Recommender systems
Mahmoud Khaled
 
Recommender.system.presentation.pjug.01.21.2014
Recommender.system.presentation.pjug.01.21.2014Recommender.system.presentation.pjug.01.21.2014
Recommender.system.presentation.pjug.01.21.2014
rpbrehm
 
Aiinpractice2017deepaklongversion
Aiinpractice2017deepaklongversionAiinpractice2017deepaklongversion
Aiinpractice2017deepaklongversion
Deepak Agarwal
 
Testing, Testing, 1-2-3: Gathering and Leveraging Audience Usability Data in ...
Testing, Testing, 1-2-3: Gathering and Leveraging Audience Usability Data in ...Testing, Testing, 1-2-3: Gathering and Leveraging Audience Usability Data in ...
Testing, Testing, 1-2-3: Gathering and Leveraging Audience Usability Data in ...
czavisca
 
Social Media Fellowship
Social Media Fellowship Social Media Fellowship
Social Media Fellowship
Lauren Brodeur
 
Machine Learning - Startup weekend UCSB 2018
Machine Learning - Startup weekend UCSB 2018Machine Learning - Startup weekend UCSB 2018
Machine Learning - Startup weekend UCSB 2018
Raul Eulogio
 
Designed by Committee: An Analytics and User-Focused Approach to the Overhaul...
Designed by Committee: An Analytics and User-Focused Approach to the Overhaul...Designed by Committee: An Analytics and User-Focused Approach to the Overhaul...
Designed by Committee: An Analytics and User-Focused Approach to the Overhaul...
nealw6971
 
From “Selena Gomez” to “Marlon Brando”: Understanding Explorative Entity Search
From “Selena Gomez” to “Marlon Brando”: Understanding Explorative Entity SearchFrom “Selena Gomez” to “Marlon Brando”: Understanding Explorative Entity Search
From “Selena Gomez” to “Marlon Brando”: Understanding Explorative Entity Search
Mounia Lalmas-Roelleke
 
Qcon SF 2013 - Machine Learning & Recommender Systems @ Netflix Scale
Qcon SF 2013 - Machine Learning & Recommender Systems @ Netflix ScaleQcon SF 2013 - Machine Learning & Recommender Systems @ Netflix Scale
Qcon SF 2013 - Machine Learning & Recommender Systems @ Netflix Scale
Xavier Amatriain
 
Udacity webinar on Recommendation Systems
Udacity webinar on Recommendation SystemsUdacity webinar on Recommendation Systems
Udacity webinar on Recommendation Systems
Axel de Romblay
 

Similar to Recent Trends in Personalization: A Netflix Perspective (20)

Big & Personal: the data and the models behind Netflix recommendations by Xa...
 Big & Personal: the data and the models behind Netflix recommendations by Xa... Big & Personal: the data and the models behind Netflix recommendations by Xa...
Big & Personal: the data and the models behind Netflix recommendations by Xa...
 
Qualitative Research vs Quantitative Research - a QuestionPro Academic Webinar
Qualitative Research vs Quantitative Research - a QuestionPro Academic WebinarQualitative Research vs Quantitative Research - a QuestionPro Academic Webinar
Qualitative Research vs Quantitative Research - a QuestionPro Academic Webinar
 
Recommender Systems In Industry
Recommender Systems In IndustryRecommender Systems In Industry
Recommender Systems In Industry
 
Pivotal Tracker - Research Findings
Pivotal Tracker - Research FindingsPivotal Tracker - Research Findings
Pivotal Tracker - Research Findings
 
Recommandation systems -
Recommandation systems - Recommandation systems -
Recommandation systems -
 
Introduction to Recommendation Systems
Introduction to Recommendation SystemsIntroduction to Recommendation Systems
Introduction to Recommendation Systems
 
Research portfolio
Research portfolioResearch portfolio
Research portfolio
 
Past, present, and future of Recommender Systems: an industry perspective
Past, present, and future of Recommender Systems: an industry perspectivePast, present, and future of Recommender Systems: an industry perspective
Past, present, and future of Recommender Systems: an industry perspective
 
Applied Data Science for E-Commerce
Applied Data Science for E-CommerceApplied Data Science for E-Commerce
Applied Data Science for E-Commerce
 
Recommendation engines
Recommendation enginesRecommendation engines
Recommendation engines
 
Recommender systems
Recommender systems Recommender systems
Recommender systems
 
Recommender.system.presentation.pjug.01.21.2014
Recommender.system.presentation.pjug.01.21.2014Recommender.system.presentation.pjug.01.21.2014
Recommender.system.presentation.pjug.01.21.2014
 
Aiinpractice2017deepaklongversion
Aiinpractice2017deepaklongversionAiinpractice2017deepaklongversion
Aiinpractice2017deepaklongversion
 
Testing, Testing, 1-2-3: Gathering and Leveraging Audience Usability Data in ...
Testing, Testing, 1-2-3: Gathering and Leveraging Audience Usability Data in ...Testing, Testing, 1-2-3: Gathering and Leveraging Audience Usability Data in ...
Testing, Testing, 1-2-3: Gathering and Leveraging Audience Usability Data in ...
 
Social Media Fellowship
Social Media Fellowship Social Media Fellowship
Social Media Fellowship
 
Machine Learning - Startup weekend UCSB 2018
Machine Learning - Startup weekend UCSB 2018Machine Learning - Startup weekend UCSB 2018
Machine Learning - Startup weekend UCSB 2018
 
Designed by Committee: An Analytics and User-Focused Approach to the Overhaul...
Designed by Committee: An Analytics and User-Focused Approach to the Overhaul...Designed by Committee: An Analytics and User-Focused Approach to the Overhaul...
Designed by Committee: An Analytics and User-Focused Approach to the Overhaul...
 
From “Selena Gomez” to “Marlon Brando”: Understanding Explorative Entity Search
From “Selena Gomez” to “Marlon Brando”: Understanding Explorative Entity SearchFrom “Selena Gomez” to “Marlon Brando”: Understanding Explorative Entity Search
From “Selena Gomez” to “Marlon Brando”: Understanding Explorative Entity Search
 
Qcon SF 2013 - Machine Learning & Recommender Systems @ Netflix Scale
Qcon SF 2013 - Machine Learning & Recommender Systems @ Netflix ScaleQcon SF 2013 - Machine Learning & Recommender Systems @ Netflix Scale
Qcon SF 2013 - Machine Learning & Recommender Systems @ Netflix Scale
 
Udacity webinar on Recommendation Systems
Udacity webinar on Recommendation SystemsUdacity webinar on Recommendation Systems
Udacity webinar on Recommendation Systems
 

More from Justin Basilico

Recap: Designing a more Efficient Estimator for Off-policy Evaluation in Band...
Recap: Designing a more Efficient Estimator for Off-policy Evaluation in Band...Recap: Designing a more Efficient Estimator for Off-policy Evaluation in Band...
Recap: Designing a more Efficient Estimator for Off-policy Evaluation in Band...
Justin Basilico
 
Is that a Time Machine? Some Design Patterns for Real World Machine Learning ...
Is that a Time Machine? Some Design Patterns for Real World Machine Learning ...Is that a Time Machine? Some Design Patterns for Real World Machine Learning ...
Is that a Time Machine? Some Design Patterns for Real World Machine Learning ...
Justin Basilico
 
Recommendations for Building Machine Learning Software
Recommendations for Building Machine Learning SoftwareRecommendations for Building Machine Learning Software
Recommendations for Building Machine Learning Software
Justin Basilico
 
Recommendations for Building Machine Learning Software
Recommendations for Building Machine Learning SoftwareRecommendations for Building Machine Learning Software
Recommendations for Building Machine Learning Software
Justin Basilico
 
Lessons Learned from Building Machine Learning Software at Netflix
Lessons Learned from Building Machine Learning Software at NetflixLessons Learned from Building Machine Learning Software at Netflix
Lessons Learned from Building Machine Learning Software at Netflix
Justin Basilico
 
Learning to Personalize
Learning to PersonalizeLearning to Personalize
Learning to Personalize
Justin Basilico
 

More from Justin Basilico (6)

Recap: Designing a more Efficient Estimator for Off-policy Evaluation in Band...
Recap: Designing a more Efficient Estimator for Off-policy Evaluation in Band...Recap: Designing a more Efficient Estimator for Off-policy Evaluation in Band...
Recap: Designing a more Efficient Estimator for Off-policy Evaluation in Band...
 
Is that a Time Machine? Some Design Patterns for Real World Machine Learning ...
Is that a Time Machine? Some Design Patterns for Real World Machine Learning ...Is that a Time Machine? Some Design Patterns for Real World Machine Learning ...
Is that a Time Machine? Some Design Patterns for Real World Machine Learning ...
 
Recommendations for Building Machine Learning Software
Recommendations for Building Machine Learning SoftwareRecommendations for Building Machine Learning Software
Recommendations for Building Machine Learning Software
 
Recommendations for Building Machine Learning Software
Recommendations for Building Machine Learning SoftwareRecommendations for Building Machine Learning Software
Recommendations for Building Machine Learning Software
 
Lessons Learned from Building Machine Learning Software at Netflix
Lessons Learned from Building Machine Learning Software at NetflixLessons Learned from Building Machine Learning Software at Netflix
Lessons Learned from Building Machine Learning Software at Netflix
 
Learning to Personalize
Learning to PersonalizeLearning to Personalize
Learning to Personalize
 

Recently uploaded

The History of Embeddings & Multimodal Embeddings
The History of Embeddings & Multimodal EmbeddingsThe History of Embeddings & Multimodal Embeddings
The History of Embeddings & Multimodal Embeddings
Zilliz
 
Opencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of MünsterOpencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of Münster
Matthias Neugebauer
 
Integrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecaseIntegrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecase
shyamraj55
 
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision MakingConnector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
DianaGray10
 
UX Webinar Series: Aligning Authentication Experiences with Business Goals
UX Webinar Series: Aligning Authentication Experiences with Business GoalsUX Webinar Series: Aligning Authentication Experiences with Business Goals
UX Webinar Series: Aligning Authentication Experiences with Business Goals
FIDO Alliance
 
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
sunilverma7884
 
Acumatica vs. Sage Intacct _Construction_July (1).pptx
Acumatica vs. Sage Intacct _Construction_July (1).pptxAcumatica vs. Sage Intacct _Construction_July (1).pptx
Acumatica vs. Sage Intacct _Construction_July (1).pptx
BrainSell Technologies
 
Types of Weaving loom machine & it's technology
Types of Weaving loom machine & it's technologyTypes of Weaving loom machine & it's technology
Types of Weaving loom machine & it's technology
ldtexsolbl
 
UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...
UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...
UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...
FIDO Alliance
 
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
shanihomely
 
Communications Mining Series - Zero to Hero - Session 3
Communications Mining Series - Zero to Hero - Session 3Communications Mining Series - Zero to Hero - Session 3
Communications Mining Series - Zero to Hero - Session 3
DianaGray10
 
Tailored CRM Software Development for Enhanced Customer Insights
Tailored CRM Software Development for Enhanced Customer InsightsTailored CRM Software Development for Enhanced Customer Insights
Tailored CRM Software Development for Enhanced Customer Insights
SynapseIndia
 
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
alexjohnson7307
 
Redefining Cybersecurity with AI Capabilities
Redefining Cybersecurity with AI CapabilitiesRedefining Cybersecurity with AI Capabilities
Redefining Cybersecurity with AI Capabilities
Priyanka Aash
 
How UiPath Discovery Suite supports identification of Agentic Process Automat...
How UiPath Discovery Suite supports identification of Agentic Process Automat...How UiPath Discovery Suite supports identification of Agentic Process Automat...
How UiPath Discovery Suite supports identification of Agentic Process Automat...
DianaGray10
 
Intel Unveils Core Ultra 200V Lunar chip .pdf
Intel Unveils Core Ultra 200V Lunar chip .pdfIntel Unveils Core Ultra 200V Lunar chip .pdf
Intel Unveils Core Ultra 200V Lunar chip .pdf
Tech Guru
 
Vulnerability Management: A Comprehensive Overview
Vulnerability Management: A Comprehensive OverviewVulnerability Management: A Comprehensive Overview
Vulnerability Management: A Comprehensive Overview
Steven Carlson
 
Sonkoloniya documentation - ONEprojukti.pdf
Sonkoloniya documentation - ONEprojukti.pdfSonkoloniya documentation - ONEprojukti.pdf
Sonkoloniya documentation - ONEprojukti.pdf
SubhamMandal40
 
Discovery Series - Zero to Hero - Task Mining Session 1
Discovery Series - Zero to Hero - Task Mining Session 1Discovery Series - Zero to Hero - Task Mining Session 1
Discovery Series - Zero to Hero - Task Mining Session 1
DianaGray10
 
COVID-19 and the Level of Cloud Computing Adoption: A Study of Sri Lankan Inf...
COVID-19 and the Level of Cloud Computing Adoption: A Study of Sri Lankan Inf...COVID-19 and the Level of Cloud Computing Adoption: A Study of Sri Lankan Inf...
COVID-19 and the Level of Cloud Computing Adoption: A Study of Sri Lankan Inf...
AimanAthambawa1
 

Recently uploaded (20)

The History of Embeddings & Multimodal Embeddings
The History of Embeddings & Multimodal EmbeddingsThe History of Embeddings & Multimodal Embeddings
The History of Embeddings & Multimodal Embeddings
 
Opencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of MünsterOpencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of Münster
 
Integrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecaseIntegrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecase
 
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision MakingConnector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
 
UX Webinar Series: Aligning Authentication Experiences with Business Goals
UX Webinar Series: Aligning Authentication Experiences with Business GoalsUX Webinar Series: Aligning Authentication Experiences with Business Goals
UX Webinar Series: Aligning Authentication Experiences with Business Goals
 
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
 
Acumatica vs. Sage Intacct _Construction_July (1).pptx
Acumatica vs. Sage Intacct _Construction_July (1).pptxAcumatica vs. Sage Intacct _Construction_July (1).pptx
Acumatica vs. Sage Intacct _Construction_July (1).pptx
 
Types of Weaving loom machine & it's technology
Types of Weaving loom machine & it's technologyTypes of Weaving loom machine & it's technology
Types of Weaving loom machine & it's technology
 
UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...
UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...
UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...
 
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
 
Communications Mining Series - Zero to Hero - Session 3
Communications Mining Series - Zero to Hero - Session 3Communications Mining Series - Zero to Hero - Session 3
Communications Mining Series - Zero to Hero - Session 3
 
Tailored CRM Software Development for Enhanced Customer Insights
Tailored CRM Software Development for Enhanced Customer InsightsTailored CRM Software Development for Enhanced Customer Insights
Tailored CRM Software Development for Enhanced Customer Insights
 
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
 
Redefining Cybersecurity with AI Capabilities
Redefining Cybersecurity with AI CapabilitiesRedefining Cybersecurity with AI Capabilities
Redefining Cybersecurity with AI Capabilities
 
How UiPath Discovery Suite supports identification of Agentic Process Automat...
How UiPath Discovery Suite supports identification of Agentic Process Automat...How UiPath Discovery Suite supports identification of Agentic Process Automat...
How UiPath Discovery Suite supports identification of Agentic Process Automat...
 
Intel Unveils Core Ultra 200V Lunar chip .pdf
Intel Unveils Core Ultra 200V Lunar chip .pdfIntel Unveils Core Ultra 200V Lunar chip .pdf
Intel Unveils Core Ultra 200V Lunar chip .pdf
 
Vulnerability Management: A Comprehensive Overview
Vulnerability Management: A Comprehensive OverviewVulnerability Management: A Comprehensive Overview
Vulnerability Management: A Comprehensive Overview
 
Sonkoloniya documentation - ONEprojukti.pdf
Sonkoloniya documentation - ONEprojukti.pdfSonkoloniya documentation - ONEprojukti.pdf
Sonkoloniya documentation - ONEprojukti.pdf
 
Discovery Series - Zero to Hero - Task Mining Session 1
Discovery Series - Zero to Hero - Task Mining Session 1Discovery Series - Zero to Hero - Task Mining Session 1
Discovery Series - Zero to Hero - Task Mining Session 1
 
COVID-19 and the Level of Cloud Computing Adoption: A Study of Sri Lankan Inf...
COVID-19 and the Level of Cloud Computing Adoption: A Study of Sri Lankan Inf...COVID-19 and the Level of Cloud Computing Adoption: A Study of Sri Lankan Inf...
COVID-19 and the Level of Cloud Computing Adoption: A Study of Sri Lankan Inf...
 

Recent Trends in Personalization: A Netflix Perspective

  • 1. Recent Trends in Personalization: A Netflix Perspective Justin Basilico ICML 2019 Adaptive & Multi-Task Learning Workshop 2019-06-15 @JustinBasilico
  • 2. Why do we personalize?
  • 3. Help members find content to watch and enjoy to maximize member satisfaction and retention
  • 5. What do we personalize?
  • 6. Ordering of videos is personalized From what we recommend Ranking
  • 7. Selection and placement of rows is personalized ... to how we construct a pageRows
  • 8. Personalized images ... to what images to select
  • 9. ... to reaching out to our members
  • 10. Everything is a recommendation! Over 80% of what people watch comes from our recommendations Overview in [Gomez-Uribe & Hunt, 2016]
  • 12. ○ Every person is unique with a variety of interests ○ Help people find what they want when they’re not sure what they want ○ Large datasets but small data per user … and potentially biased by the output of your system ○ Cold-start problems on all sides ○ Non-stationary, context-dependent, mood-dependent ○ More than just accuracy: Diversity, novelty, freshness, fairness, ... ○ ... No, personalization is hard!
  • 13. Some recent trends in approaching these challenges: 1. Deep Learning 2. Causality 3. Bandits & Reinforcement Learning 4. Fairness 5. Experience Personalization Trending Now
  • 14. Trend 1: Deep Learning in Recommendations
  • 15. What~2012 ~2017 Deep Learning becomes popular in Machine Learning Deep Learning becomes popular in Recommender Systems What took so long?
  • 16. Traditional Recommendations Collaborative Filtering: Recommend items that similar users have chosen 0 1 0 1 0 0 0 1 1 0 1 0 0 1 1 0 1 0 0 0 0 0 0 0 1 Users Items
  • 19. U A (deeper) feed-forward view V Mean squared loss?
  • 20. … isn’t always the best U V Mean squared loss ?
  • 21. V … but opens up many possibilities Softmax Avg / Stack/ Sequence DNN / RNN / CNN Input interactions (X) (X) p(Y) 2018-12-2319:32:10 2018-12-2412:05:53 2019-01-0215:40:22
  • 22. Sequence prediction ● Treat recommendations as a sequence classification problem ○ Input: sequence of user actions ○ Output: next action ● E.g. Gru4Rec [Hidasi et. al., 2016] ○ Input: sequence of items in a sessions ○ Output: next item in the session ● Also co-evolution: [Wu et al., 2017], [Dai et al., 2017]
  • 23. Leveraging other data ● Example: YouTube Recommender [Covington et. al., 2016] ● Two stage ranker: candidate generation (shrinking set of items to rank) and ranking (classifying actual impressions) ● Two feed-forward, fully connected, networks with hundreds of features
  • 24. Contextual sequence data 2017-12-10 15:40:22 2017-12-23 19:32:10 2017-12-24 12:05:53 2017-12-27 22:40:22 2017-12-29 19:39:36 2017-12-30 20:42:13 Context ItemSequence per user ? Time
  • 25. Time-sensitive sequence prediction ● Proper modeling of time and system dynamics is critical ○ Recommendations are actions at a moment in time ● Experiment on a Netflix internal dataset ○ Input: Sequence of past plays and time context ■ Discrete time: Day-of-week (Mon, Tue, …) & Hour-of-day ■ Continuous time (aka timestamp) ○ Label: Predict next play (temporal split data)
  • 28. From Correlation to Causation ● Most recommendation algorithms are correlational ○ Some early recommendation algorithms literally computed correlations between users and items ● Did you watch a movie because you liked it? Or because we showed it to you? Or both? p(Y|X) → p(Y|X, do(R)) (from http://www.tylervigen.com/spurious-correlations)
  • 29. Feedback loops Impression bias inflates plays Leads to inflated item popularity More plays More impressions Oscillations in distribution of genre recommendations Feedback loops can cause biases to be reinforced by the recommendation system! [Chaney et al., 2018]: simulations showing that this can reduce the usefulness of the system
  • 30. Lots of feedback loops...
  • 33. Closed Loop Training Data Watches Model Recs Danger Zone Search Training Data Watches Model Recs Open Loop
  • 34. Closed Loop Training Data Watches Model Recs Danger Zone Search Training Data Watches Model Recs Open Loop
  • 35. Debiasing Recommendations ● IPS Estimator for MF [Schnabel et al., 2016] ○ Train a debiasing model and reweight the data ● Causal Embeddings [Bonner & Vasile, 2018] ○ Jointly learn debiasing model and task model ○ Regularize the two towards each other ● Doubly-Robust MF [Wang et al., 2019]
  • 36. Trend 3: Bandits & Reinforcement Learning in Recommendations
  • 37. ● Uncertainty around user interests and new items ● Sparse and indirect feedback ● Changing trends ● Break feedback loops ● Want to explore to learn Why contextual bandits for recommendations? ▶Early news example: [Li et al., 2010]
  • 38. Bart [McInerney et al., 2018] ● Bandit selecting both items and explanations for Spotify homepage ● Factorization Machine with epsilon-greedy explore over personalized candidate set ● Counterfactual risk minimization to train the bandit
  • 40. Artwork Personalization as Contextual Bandit ● Environment: Netflix homepage ● Context: Member, device, page, etc. ● Learner: Artwork selector for a show ● Action: Display specific image for show ● Reward: Member has positive engagement Artwork Selector ▶
  • 41. Offline Replay Results ● Bandit finds good images ● Personalization is better ● Artwork variety matters ● Personalization wiggles around best images Lift in Replay in the various algorithms as compared to the Random baseline More info in our blog post
  • 42. Going Long-Term ● Want to maximize long-term user satisfaction and retention ● Involves many user visits, recommendation actions and delayed reward ● … sounds like Reinforcement Learning
  • 43. ● High-dimensional action space: Recommending a single item is O(|C|); typically want to do ranking or page construction, which is combinatorial ● High-dimensional state space: Users are represented in the state, along with the relevant history ● Off-policy training: Need to learn from existing system actions ● Concurrency: Don’t observe full trajectories, need to learn simultaneously from many interactions ● Changing action space: New actions (items) become available and need to be cold-started. ● No good simulator: Requires knowing feedback for user on recommended items Challenges of Reinforcement Learning for Recommendations
  • 44. List-wise [Zhao et al., 2017] or Page-wise recommendation [Zhao et al. 2018] based on [Dulac-Arnold et al., 2016] Embeddings for actions
  • 45. ● Generator to choose user action from recommendation ● Reward trained like a discriminator ● LSTM or Position-Weight architecture ● Learning over sets via cascading Deep Q Networks ○ Different Q function per position GAN-inspired as a user simulator [Chen et al., 2019]
  • 46. ● Train candidate generator using REINFORCE ● Exploration done using softmax with temperature ● Off-policy correction with adaptation for top-k recommendations ● Trust region policy optimization to keep close to logging policy Policy Gradient for YouTube Recommendations [Chen et al., 2019]
  • 48. Personalization has a big impact in people’s lives How do we make sure that it is fair?
  • 49. Calibrated Recommendations [Steck, 2018] ● Fairness as matching distribution of user interests ● Accuracy as an objective can lead to unbalanced predictions ● Simple example: ● Many recommendation algorithms exhibit this behavior of exaggerating the dominant interests and crowd out less frequent ones 30 action70 romance 30% action70% romance User: Expectation: 100% romanceReality: Maximizes accuracy
  • 50. - Genre-distribution of each item is given: - Genre-distribution of user’s play history: … add prior for other genres: - Genre-distribution of recommended list: (for diversity) (or other categorization) Calibration Metric
  • 51. Calibration Results (MovieLens 20M) Baseline model (wMF): Many users receive uncalibrated rec’s After reranking: Rec’s are much more calibrated (smaller ) Userdensity More calibrated (KL divergence) Submodular Reranker:
  • 52. Fairness through Pairwise Comparisons [Beutel et al., 2019] ● Recommendations are fair if likelihood of clicked item being ranked above an unclicked item is the same across two groups ○ Intra-group pairwise accuracy - Restrict to pairs within group ○ Inter-group pairwise accuracy - Restrict to pairs between groups ● Training: Add pairwise regularizer based on randomized data to collect fairness feedback
  • 54. Personalizing how we recommend (not just what we recommend…) ● Algorithm level: Ideal balance of diversity, popularity, novelty, freshness, etc. may depend on the person ● Display level: How you present items or explain recommendations can also be personalized ● Interaction level: Balancing the needs of lean-back users and power users
  • 55. Page/Slate Optimization ● Select multiple actions that go together and receive feedback on group ● Personalizing based on within-session browsing behavior [Wu et al., 2015] ● Off-policy evaluation for slates [Swaminathan, et al., 2016] ● Slate optimization as VAE [Jiang et al., 2019] ● Marginal posterior sampling for slate bandits [Dimakopoulou et al., 2019]
  • 56. More dimensions to personalize Rows Trailer Evidence Synopsis Image Row Title Metadata Ranking
  • 58. Rating Ranking Pages 4.7 Experience Evolution of our Personalization Approach
  • 60. Applications as tasks ● Many related personalization tasks in a recommender system ● Examples: ○ [Zhao et al., 2015] - Outputs for different tasks ○ [Bansal et al., 2016] - Jointly learn to recommend and predict metadata for items ○ [Ma et al., 2018] - Jointly learn watch and enjoy ○ [Lu et al., 2018] - Jointly learn for rating prediction and explanation ○ [Hadash et al., 2018] - Jointly learn ranking and rating prediction User history Ranking Page Rating Explanation Search Image Context ...
  • 61. Other views ● Users-as-tasks: Treat each user as a task and learn from others users ○ Example: [Ning & Karapis, 2010] finds similar users and does support vector regression ● Items-as-tasks: Treat each item as a separate model to learn ● Contexts-as-tasks: Treat different contexts (time, device, region, …) as separate tasks ● Domains-as-tasks: Leverage representations of users in one domain to help in another (e.g. different kinds of items, different genres) ○ Example: [Li et al., 2009] on movies <-> books
  • 63. 1. Deep Learning 2. Causality 3. Bandits & Reinforcement Learning 4. Fairness 5. Experience Personalization 6. Multi-task & Meta Learning? Lots of opportunity for Machine Learning in Personalization
  • 64. Thank you Questions? @JustinBasilico Yes, we’re hiring... Justin Basilico