Déjà Vu: The Importance of Time and Causality in Recommender Systems

Justin Basilico
Justin BasilicoResearch/Engineering Director at Netflix - Machine Learning and Recommendation Systems
Déjà Vu
The Importance of Time and
Causality in Recommender
Systems
Justin Basilico & Yves Raimond
August 29, 2017
@JustinBasilico @moustaki
But first…
Goodbye
& Cinematch
But first…
Goodbye
& Cinematch
% Match
Hello
&
But first…
Goodbye
Why?
+200% ratings volume
Clear link to personalization
& Cinematch
% Match
Hello
&
Image from Domiriel (cc by-nc)
● This moment can be controlled
by the user
○ Visit time
○ Session length
● … or influenced by the system
○ Notifications, emails
● And must choose an action
○ … that has consequences
Recommendations are actions
at a moment in time
● Time and causality are critical
aspects in any recommender
system
○ Data collection
○ Experiment design (offline & online)
○ Algorithm & objective design
○ System design
Time-aware data
collection
Data collection
Observed
labels
Training
time
Serving
time
Serving
input data
collected
Violation of the space-time
continuum!
Observed
labels
Training
input data
collected
Training
time
Serving
time
Serving
input data
collected
Data collection
Observed
labels
Training
input data
collected
Training
time
Serving
time
Serving
input data
collected
Time machines
Observed
labels
Training
input data
collected
Training
time
Serving
time
Serving
input data
collected
Distributed Time Travel for Feature Generation
● Be careful when splitting dataset
○ Don’t overfit the past
○ Predict the future
● Rule of thumb: Split across what you need to generalize
○ Time!
○ Users or Items?
● May need to train/test at multiple distinct time points to see
generalization across time (e.g. [Lathia et. al., 2009])
● Simulate system behaviors (e.g. training and publishing
delays) in evaluation pipeline
○ Helps capture trade-off between accuracy and responsiveness
Experiment design
Train
Time
Test
Time-aware
recommendation
algorithms
R ≈ UM
?
?
Users changing over time
Nonstationarity
Items changing over time
popularity
time
Learned item
bias
Actual item popularity
Item launch
Item becomes
available
● Aggregation
○ Decay functions (e.g. [Ding, Li, 2005])
○ Buckets (e.g. [Zimdars, Chickering, Meek, 2001])
● Extrapolation (e.g. [Koren, 2009])
● Sequences
○ Markov (e.g. [Rendle, et al., 2010])
○ Last N (e.g. [Shani, Heckerman, Brafman, 2005])
○ RNNs (e.g. [Hidasi et al., 2015])
● Features
○ Discretized (e.g. [Baltrunas, Amatriain, 2009])
○ Continuous (e.g. example age in [Covington et. al., 2016])
Some modeling approaches time
● Generalizing to future behaviors through
temporal extrapolation
● Time exhibits many periodicities
○ Daily
○ Weekly
○ Seasonally
○ … and even longer: Olympics, elections, etc.
● Additional periodic time context features
can be added or extracted
Time as context Experiment on a Netflix
internal dataset
● Recommendation systems are a means to an end
○ Reward = enjoyment - interaction cost
○ Enjoyment integrated over time (e.g. goodness * length of view)
○ Interaction cost integrated over time
○ Don’t waste your users time
○ Magnitudes of enjoyment and cost may be user-specific
● Maximize enjoyment of the selected item while minimizing time it
takes to find the item
Minimizing interaction time
Hangul alphabet, 3 syllables but
requires 7 (2 + 3 + 2) interactionsClick
With a model optimized to minimize
interaction time: one interaction
Click
Time-aware
recommender systems
Déjà Vu: The Importance of Time and Causality in Recommender Systems
Algorithms changing
Idea
Offline
experimentation
Online
experimentation
(A/B)
Rollout
Algorithm C
Algorithm B
Algorithm A
Algorithms changing
Idea
Offline
experimentation
Online
experimentation
(A/B)
Rollout
Algorithm C
Algorithm B
Algorithm A
Algorithms changing
Idea
Offline
experimentation
Online
experimentation
(A/B)
Rollout
Assumes stationarity! A change in other parts of the system
might invalidate previous (offline or online) results.
Holdback A/B tests as part of rollout can help.
UX changing over time
% Match&
Feedback loops
Impression bias
inflates plays
Leads to inflated
item popularity
More plays
More
impressions
Oscillations in
distribution of genre
recommendations
Feedback loops can cause biases to be
reinforced by the recommendation system!
Déjà Vu: The Importance of Time and Causality in Recommender Systems
Closed Loop
Training
Data
Watches Model
Recs
Closed Loop
Training
Data
Watches Model
Recs
Danger Zone
Closed Loop
Training
Data
Watches Model
Recs
Danger Zone
Search
Training
Data
Watches Model
Recs
Open Loop
Closed Loop
Training
Data
Watches Model
Recs
Danger Zone
Search
Training
Data
Watches Model
Recs
Open Loop
Open vs. Closed Loops
[Based on Steck, 2013 with system as selector]
Watch when
rec
Probability
of rec
Watch when
not rec
Probability
of not rec
Open vs. Closed Loops
[Based on Steck, 2013 with system as selector]
Watch when
rec
Probability
of rec
Watch when
not rec
Probability
of not rec
Closed loop: 0
Open loop: > 0
Open vs. Closed Loops
[Based on Steck, 2013 with system as selector]
Watch when
rec
Probability
of rec
Watch when
not rec
Probability
of not rec
Closed loop: 0
Open loop: > 0
We have control
over this
● Maintain some controlled exploration to break
feedback loop and handle non-stationarities
● Explore with -greedy, Thompson Sampling, etc.
● Control to avoid significantly degrading user
experience
● Log as much as possible
○ Include counterfactuals: What maximal action
system wanted to do (e.g. [Bottou et al., 2013])
Controlled stochasticity
Explore
Explore
Replay Metrics
Observed
reward
Existing
recommendation
algorithm (with
stochasticity)
Observed
reward
New recommendation
algorithm
[Li et al., 2011; Dudik, Langford, Li, 2014]
Simulate online metrics, offline!
● Stochasticity opens the door to using causal inference
● Inverse Propensity Weighting
○ Reduce production bias by reweighting train and test data
○ Know probability of user receiving an impression
○ Doesn’t handle simultaneity and other endogeneity
● Covariate shift
○ Use explore data to estimate bias in other data
○ Use all data to train
● Instrumental variables for more general settings
Causality
[Schnabel et al., 2016; Liang, Charlin, Blei, 2016; Smola, 2011, Sugiyama, Kawanabe, 2012]
● Most recommendations (and ML) models are correlational
○ These items are correlated with these types of users
● But we seek causal actions
○ Showing this item is rewarding for this user
● Our recommendation action should have an incremental
effect in reward: E[r(a)] - E[r(∅)]
○ Application-dependent choice of ∅
○ Sometimes it may be better not to provide a recommendation that
simply maximizes p(vi
|u)
○ May provide less obvious recommendations
Incrementality
p(vi
|∅) p(vi
|a)
Incremental
effect
● Gold standard of causality
○ Random assignment
○ Measured across time
○ Incremental benefit of treatment
● Causality safety net?
○ Hard to test with full feedback loop effects
○ An algorithm may behave differently when
training off its own data
○ Holdback tests
A/B Testing
Time
A
(Control)
B
(Treatment)
Significant?
Metrics
Conclusions.
● After users and items, time is usually the next most important
factor in recommendation systems
○ Model it as such
○ Evaluate it as such
○ Make it central to your system and infrastructure
● Recommender systems act in a causal loop
○ Influenced by themselves and others
○ Be thoughtful about feedback effects
Takeaways
Thank you.
@JustinBasilico @moustaki
Justin Basilico & Yves Raimond
Yes, we’re hiring...
Déjà Vu
The Importance of Time and
Causality in Recommender
Systems
Justin Basilico & Yves Raimond
August 29, 2017
@JustinBasilico @moustaki
1 of 45

Recommended

Sequential Decision Making in Recommendations by
Sequential Decision Making in RecommendationsSequential Decision Making in Recommendations
Sequential Decision Making in RecommendationsJaya Kawale
2.1K views32 slides
Calibrated Recommendations by
Calibrated RecommendationsCalibrated Recommendations
Calibrated RecommendationsHarald Steck
4.2K views24 slides
Personalizing "The Netflix Experience" with Deep Learning by
Personalizing "The Netflix Experience" with Deep LearningPersonalizing "The Netflix Experience" with Deep Learning
Personalizing "The Netflix Experience" with Deep LearningAnoop Deoras
1.1K views41 slides
Time, Context and Causality in Recommender Systems by
Time, Context and Causality in Recommender SystemsTime, Context and Causality in Recommender Systems
Time, Context and Causality in Recommender SystemsYves Raimond
5.9K views35 slides
Data council SF 2020 Building a Personalized Messaging System at Netflix by
Data council SF 2020 Building a Personalized Messaging System at NetflixData council SF 2020 Building a Personalized Messaging System at Netflix
Data council SF 2020 Building a Personalized Messaging System at NetflixGrace T. Huang
447 views31 slides
Missing values in recommender models by
Missing values in recommender modelsMissing values in recommender models
Missing values in recommender modelsParmeshwar Khurd
118 views29 slides

More Related Content

What's hot

Contextualization at Netflix by
Contextualization at NetflixContextualization at Netflix
Contextualization at NetflixLinas Baltrunas
7.6K views31 slides
Deep Learning for Recommender Systems by
Deep Learning for Recommender SystemsDeep Learning for Recommender Systems
Deep Learning for Recommender SystemsYves Raimond
15.4K views33 slides
Past, present, and future of Recommender Systems: an industry perspective by
Past, present, and future of Recommender Systems: an industry perspectivePast, present, and future of Recommender Systems: an industry perspective
Past, present, and future of Recommender Systems: an industry perspectiveXavier Amatriain
11K views71 slides
Recent Trends in Personalization at Netflix by
Recent Trends in Personalization at NetflixRecent Trends in Personalization at Netflix
Recent Trends in Personalization at NetflixJustin Basilico
24.2K views57 slides
Recent Trends in Personalization: A Netflix Perspective by
Recent Trends in Personalization: A Netflix PerspectiveRecent Trends in Personalization: A Netflix Perspective
Recent Trends in Personalization: A Netflix PerspectiveJustin Basilico
30.3K views64 slides
Artwork Personalization at Netflix Fernando Amat RecSys2018 by
Artwork Personalization at Netflix Fernando Amat RecSys2018 Artwork Personalization at Netflix Fernando Amat RecSys2018
Artwork Personalization at Netflix Fernando Amat RecSys2018 Fernando Amat
3.5K views24 slides

What's hot(20)

Deep Learning for Recommender Systems by Yves Raimond
Deep Learning for Recommender SystemsDeep Learning for Recommender Systems
Deep Learning for Recommender Systems
Yves Raimond15.4K views
Past, present, and future of Recommender Systems: an industry perspective by Xavier Amatriain
Past, present, and future of Recommender Systems: an industry perspectivePast, present, and future of Recommender Systems: an industry perspective
Past, present, and future of Recommender Systems: an industry perspective
Xavier Amatriain11K views
Recent Trends in Personalization at Netflix by Justin Basilico
Recent Trends in Personalization at NetflixRecent Trends in Personalization at Netflix
Recent Trends in Personalization at Netflix
Justin Basilico24.2K views
Recent Trends in Personalization: A Netflix Perspective by Justin Basilico
Recent Trends in Personalization: A Netflix PerspectiveRecent Trends in Personalization: A Netflix Perspective
Recent Trends in Personalization: A Netflix Perspective
Justin Basilico30.3K views
Artwork Personalization at Netflix Fernando Amat RecSys2018 by Fernando Amat
Artwork Personalization at Netflix Fernando Amat RecSys2018 Artwork Personalization at Netflix Fernando Amat RecSys2018
Artwork Personalization at Netflix Fernando Amat RecSys2018
Fernando Amat3.5K views
Past, Present & Future of Recommender Systems: An Industry Perspective by Justin Basilico
Past, Present & Future of Recommender Systems: An Industry PerspectivePast, Present & Future of Recommender Systems: An Industry Perspective
Past, Present & Future of Recommender Systems: An Industry Perspective
Justin Basilico65K views
A Multi-Armed Bandit Framework For Recommendations at Netflix by Jaya Kawale
A Multi-Armed Bandit Framework For Recommendations at NetflixA Multi-Armed Bandit Framework For Recommendations at Netflix
A Multi-Armed Bandit Framework For Recommendations at Netflix
Jaya Kawale11.1K views
Deep Learning for Recommender Systems by Justin Basilico
Deep Learning for Recommender SystemsDeep Learning for Recommender Systems
Deep Learning for Recommender Systems
Justin Basilico21.1K views
Artwork Personalization at Netflix by Justin Basilico
Artwork Personalization at NetflixArtwork Personalization at Netflix
Artwork Personalization at Netflix
Justin Basilico28.1K views
Lessons Learned from Building Machine Learning Software at Netflix by Justin Basilico
Lessons Learned from Building Machine Learning Software at NetflixLessons Learned from Building Machine Learning Software at Netflix
Lessons Learned from Building Machine Learning Software at Netflix
Justin Basilico14.5K views
Crafting Recommenders: the Shallow and the Deep of it! by Sudeep Das, Ph.D.
Crafting Recommenders: the Shallow and the Deep of it! Crafting Recommenders: the Shallow and the Deep of it!
Crafting Recommenders: the Shallow and the Deep of it!
Sudeep Das, Ph.D.1.8K views
Netflix Recommendations - Beyond the 5 Stars by Xavier Amatriain
Netflix Recommendations - Beyond the 5 StarsNetflix Recommendations - Beyond the 5 Stars
Netflix Recommendations - Beyond the 5 Stars
Xavier Amatriain21.1K views
Context Aware Recommendations at Netflix by Linas Baltrunas
Context Aware Recommendations at NetflixContext Aware Recommendations at Netflix
Context Aware Recommendations at Netflix
Linas Baltrunas5.7K views
Making Netflix Machine Learning Algorithms Reliable by Justin Basilico
Making Netflix Machine Learning Algorithms ReliableMaking Netflix Machine Learning Algorithms Reliable
Making Netflix Machine Learning Algorithms Reliable
Justin Basilico11.8K views
Learning a Personalized Homepage by Justin Basilico
Learning a Personalized HomepageLearning a Personalized Homepage
Learning a Personalized Homepage
Justin Basilico6.5K views
Personalized Page Generation for Browsing Recommendations by Justin Basilico
Personalized Page Generation for Browsing RecommendationsPersonalized Page Generation for Browsing Recommendations
Personalized Page Generation for Browsing Recommendations
Justin Basilico5.3K views
Netflix talk at ML Platform meetup Sep 2019 by Faisal Siddiqi
Netflix talk at ML Platform meetup Sep 2019Netflix talk at ML Platform meetup Sep 2019
Netflix talk at ML Platform meetup Sep 2019
Faisal Siddiqi4.4K views
Deep Learning for Personalized Search and Recommender Systems by Benjamin Le
Deep Learning for Personalized Search and Recommender SystemsDeep Learning for Personalized Search and Recommender Systems
Deep Learning for Personalized Search and Recommender Systems
Benjamin Le37.4K views

Viewers also liked

Why [Mobile] [In-app] Programmatic? A Marketer's Guide by
Why [Mobile] [In-app] Programmatic? A Marketer's GuideWhy [Mobile] [In-app] Programmatic? A Marketer's Guide
Why [Mobile] [In-app] Programmatic? A Marketer's GuideMoPub
17.5K views10 slides
Dressipi - Personalised recommendation engine for fashion consumers by
Dressipi - Personalised recommendation engine for fashion consumersDressipi - Personalised recommendation engine for fashion consumers
Dressipi - Personalised recommendation engine for fashion consumersProject Juno
1.7K views21 slides
Machine Intelligence Showcase Feb 2017 by
Machine Intelligence Showcase Feb 2017Machine Intelligence Showcase Feb 2017
Machine Intelligence Showcase Feb 2017Project Juno
814 views7 slides
BenevolentTech - Harnessing the power of AI to accelerate global scientific d... by
BenevolentTech - Harnessing the power of AI to accelerate global scientific d...BenevolentTech - Harnessing the power of AI to accelerate global scientific d...
BenevolentTech - Harnessing the power of AI to accelerate global scientific d...Project Juno
1.6K views10 slides
Personalized Job Recommendation System at LinkedIn: Practical Challenges and ... by
Personalized Job Recommendation System at LinkedIn: Practical Challenges and ...Personalized Job Recommendation System at LinkedIn: Practical Challenges and ...
Personalized Job Recommendation System at LinkedIn: Practical Challenges and ...Benjamin Le
4.5K views34 slides
Bootstrapping a Destination Recommendation Engine by
Bootstrapping a Destination Recommendation EngineBootstrapping a Destination Recommendation Engine
Bootstrapping a Destination Recommendation EngineNeal Lathia
1.5K views22 slides

Viewers also liked(12)

Why [Mobile] [In-app] Programmatic? A Marketer's Guide by MoPub
Why [Mobile] [In-app] Programmatic? A Marketer's GuideWhy [Mobile] [In-app] Programmatic? A Marketer's Guide
Why [Mobile] [In-app] Programmatic? A Marketer's Guide
MoPub17.5K views
Dressipi - Personalised recommendation engine for fashion consumers by Project Juno
Dressipi - Personalised recommendation engine for fashion consumersDressipi - Personalised recommendation engine for fashion consumers
Dressipi - Personalised recommendation engine for fashion consumers
Project Juno1.7K views
Machine Intelligence Showcase Feb 2017 by Project Juno
Machine Intelligence Showcase Feb 2017Machine Intelligence Showcase Feb 2017
Machine Intelligence Showcase Feb 2017
Project Juno814 views
BenevolentTech - Harnessing the power of AI to accelerate global scientific d... by Project Juno
BenevolentTech - Harnessing the power of AI to accelerate global scientific d...BenevolentTech - Harnessing the power of AI to accelerate global scientific d...
BenevolentTech - Harnessing the power of AI to accelerate global scientific d...
Project Juno1.6K views
Personalized Job Recommendation System at LinkedIn: Practical Challenges and ... by Benjamin Le
Personalized Job Recommendation System at LinkedIn: Practical Challenges and ...Personalized Job Recommendation System at LinkedIn: Practical Challenges and ...
Personalized Job Recommendation System at LinkedIn: Practical Challenges and ...
Benjamin Le4.5K views
Bootstrapping a Destination Recommendation Engine by Neal Lathia
Bootstrapping a Destination Recommendation EngineBootstrapping a Destination Recommendation Engine
Bootstrapping a Destination Recommendation Engine
Neal Lathia1.5K views
Recommendations for Building Machine Learning Software by Justin Basilico
Recommendations for Building Machine Learning SoftwareRecommendations for Building Machine Learning Software
Recommendations for Building Machine Learning Software
Justin Basilico6.2K views
Personalization Challenges in E-Learning by Roberto Turrin
Personalization Challenges in E-LearningPersonalization Challenges in E-Learning
Personalization Challenges in E-Learning
Roberto Turrin1.1K views
Building Recommender Systems for Fashion by Nick Landia
Building Recommender Systems for FashionBuilding Recommender Systems for Fashion
Building Recommender Systems for Fashion
Nick Landia3.3K views
Is that a Time Machine? Some Design Patterns for Real World Machine Learning ... by Justin Basilico
Is that a Time Machine? Some Design Patterns for Real World Machine Learning ...Is that a Time Machine? Some Design Patterns for Real World Machine Learning ...
Is that a Time Machine? Some Design Patterns for Real World Machine Learning ...
Justin Basilico8.8K views
Applying deep learning to medical data by Hyun-seok Min
Applying deep learning to medical dataApplying deep learning to medical data
Applying deep learning to medical data
Hyun-seok Min6.7K views
AI and Machine Learning Demystified by Carol Smith at Midwest UX 2017 by Carol Smith
AI and Machine Learning Demystified by Carol Smith at Midwest UX 2017AI and Machine Learning Demystified by Carol Smith at Midwest UX 2017
AI and Machine Learning Demystified by Carol Smith at Midwest UX 2017
Carol Smith4.9M views

Similar to Déjà Vu: The Importance of Time and Causality in Recommender Systems

HT2014 Tutorial: Evaluating Recommender Systems - Ensuring Replicability of E... by
HT2014 Tutorial: Evaluating Recommender Systems - Ensuring Replicability of E...HT2014 Tutorial: Evaluating Recommender Systems - Ensuring Replicability of E...
HT2014 Tutorial: Evaluating Recommender Systems - Ensuring Replicability of E...Alejandro Bellogin
3.1K views67 slides
Strata 2016 - Lessons Learned from building real-life Machine Learning Systems by
Strata 2016 -  Lessons Learned from building real-life Machine Learning SystemsStrata 2016 -  Lessons Learned from building real-life Machine Learning Systems
Strata 2016 - Lessons Learned from building real-life Machine Learning SystemsXavier Amatriain
5.9K views51 slides
Recsys 2016 tutorial: Lessons learned from building real-life recommender sys... by
Recsys 2016 tutorial: Lessons learned from building real-life recommender sys...Recsys 2016 tutorial: Lessons learned from building real-life recommender sys...
Recsys 2016 tutorial: Lessons learned from building real-life recommender sys...Xavier Amatriain
16.5K views46 slides
BIG2016- Lessons Learned from building real-life user-focused Big Data systems by
BIG2016- Lessons Learned from building real-life user-focused Big Data systemsBIG2016- Lessons Learned from building real-life user-focused Big Data systems
BIG2016- Lessons Learned from building real-life user-focused Big Data systemsXavier Amatriain
3.4K views48 slides
Replicable Evaluation of Recommender Systems by
Replicable Evaluation of Recommender SystemsReplicable Evaluation of Recommender Systems
Replicable Evaluation of Recommender SystemsAlejandro Bellogin
5.9K views74 slides
Recommender Systems by
Recommender SystemsRecommender Systems
Recommender SystemsFrancesco Casalegno
2.3K views35 slides

Similar to Déjà Vu: The Importance of Time and Causality in Recommender Systems(20)

HT2014 Tutorial: Evaluating Recommender Systems - Ensuring Replicability of E... by Alejandro Bellogin
HT2014 Tutorial: Evaluating Recommender Systems - Ensuring Replicability of E...HT2014 Tutorial: Evaluating Recommender Systems - Ensuring Replicability of E...
HT2014 Tutorial: Evaluating Recommender Systems - Ensuring Replicability of E...
Alejandro Bellogin3.1K views
Strata 2016 - Lessons Learned from building real-life Machine Learning Systems by Xavier Amatriain
Strata 2016 -  Lessons Learned from building real-life Machine Learning SystemsStrata 2016 -  Lessons Learned from building real-life Machine Learning Systems
Strata 2016 - Lessons Learned from building real-life Machine Learning Systems
Xavier Amatriain5.9K views
Recsys 2016 tutorial: Lessons learned from building real-life recommender sys... by Xavier Amatriain
Recsys 2016 tutorial: Lessons learned from building real-life recommender sys...Recsys 2016 tutorial: Lessons learned from building real-life recommender sys...
Recsys 2016 tutorial: Lessons learned from building real-life recommender sys...
Xavier Amatriain16.5K views
BIG2016- Lessons Learned from building real-life user-focused Big Data systems by Xavier Amatriain
BIG2016- Lessons Learned from building real-life user-focused Big Data systemsBIG2016- Lessons Learned from building real-life user-focused Big Data systems
BIG2016- Lessons Learned from building real-life user-focused Big Data systems
Xavier Amatriain3.4K views
Replicable Evaluation of Recommender Systems by Alejandro Bellogin
Replicable Evaluation of Recommender SystemsReplicable Evaluation of Recommender Systems
Replicable Evaluation of Recommender Systems
Alejandro Bellogin5.9K views
Strata NYC: Building turn-key recommendations for 5% of internet video by Kamil Sindi
Strata NYC: Building turn-key recommendations for 5% of internet videoStrata NYC: Building turn-key recommendations for 5% of internet video
Strata NYC: Building turn-key recommendations for 5% of internet video
Kamil Sindi130 views
Recent and Robust Query Auto-Completion - WWW 2014 Conference Presentation by stewhir
Recent and Robust Query Auto-Completion - WWW 2014 Conference PresentationRecent and Robust Query Auto-Completion - WWW 2014 Conference Presentation
Recent and Robust Query Auto-Completion - WWW 2014 Conference Presentation
stewhir857 views
[UPDATE] Udacity webinar on Recommendation Systems by Axel de Romblay
[UPDATE] Udacity webinar on Recommendation Systems[UPDATE] Udacity webinar on Recommendation Systems
[UPDATE] Udacity webinar on Recommendation Systems
Axel de Romblay195 views
Udacity webinar on Recommendation Systems by Axel de Romblay
Udacity webinar on Recommendation SystemsUdacity webinar on Recommendation Systems
Udacity webinar on Recommendation Systems
Axel de Romblay742 views
10 Lessons Learned from Building Machine Learning Systems by Xavier Amatriain
10 Lessons Learned from Building Machine Learning Systems10 Lessons Learned from Building Machine Learning Systems
10 Lessons Learned from Building Machine Learning Systems
Xavier Amatriain378.2K views
Video Recommendation Engines as a Service by Kamil Sindi
Video Recommendation Engines as a ServiceVideo Recommendation Engines as a Service
Video Recommendation Engines as a Service
Kamil Sindi362 views
Recommending Sequences RecTour 2017 by Gunjan Kumar
Recommending Sequences RecTour 2017Recommending Sequences RecTour 2017
Recommending Sequences RecTour 2017
Gunjan Kumar150 views
Mini datathon by Kunal Jain
Mini datathonMini datathon
Mini datathon
Kunal Jain639 views
Scott Clark, Software Engineer, Yelp at MLconf SF by MLconf
Scott Clark, Software Engineer, Yelp at MLconf SFScott Clark, Software Engineer, Yelp at MLconf SF
Scott Clark, Software Engineer, Yelp at MLconf SF
MLconf4.3K views
A Plan for Sustainable MIR Evaluation by Julián Urbano
A Plan for Sustainable MIR EvaluationA Plan for Sustainable MIR Evaluation
A Plan for Sustainable MIR Evaluation
Julián Urbano557 views
Dowhy: An end-to-end library for causal inference by Amit Sharma
Dowhy: An end-to-end library for causal inferenceDowhy: An end-to-end library for causal inference
Dowhy: An end-to-end library for causal inference
Amit Sharma5.9K views

Recently uploaded

Future of AR - Facebook Presentation by
Future of AR - Facebook PresentationFuture of AR - Facebook Presentation
Future of AR - Facebook Presentationssuserb54b561
22 views27 slides
TrustArc Webinar - Managing Online Tracking Technology Vendors_ A Checklist f... by
TrustArc Webinar - Managing Online Tracking Technology Vendors_ A Checklist f...TrustArc Webinar - Managing Online Tracking Technology Vendors_ A Checklist f...
TrustArc Webinar - Managing Online Tracking Technology Vendors_ A Checklist f...TrustArc
72 views29 slides
Info Session November 2023.pdf by
Info Session November 2023.pdfInfo Session November 2023.pdf
Info Session November 2023.pdfAleksandraKoprivica4
15 views15 slides
Zero to Automated in Under a Year by
Zero to Automated in Under a YearZero to Automated in Under a Year
Zero to Automated in Under a YearNetwork Automation Forum
22 views23 slides
ESPC 2023 - Protect and Govern your Sensitive Data with Microsoft Purview in ... by
ESPC 2023 - Protect and Govern your Sensitive Data with Microsoft Purview in ...ESPC 2023 - Protect and Govern your Sensitive Data with Microsoft Purview in ...
ESPC 2023 - Protect and Govern your Sensitive Data with Microsoft Purview in ...Jasper Oosterveld
27 views49 slides
Ransomware is Knocking your Door_Final.pdf by
Ransomware is Knocking your Door_Final.pdfRansomware is Knocking your Door_Final.pdf
Ransomware is Knocking your Door_Final.pdfSecurity Bootcamp
66 views46 slides

Recently uploaded(20)

Future of AR - Facebook Presentation by ssuserb54b561
Future of AR - Facebook PresentationFuture of AR - Facebook Presentation
Future of AR - Facebook Presentation
ssuserb54b56122 views
TrustArc Webinar - Managing Online Tracking Technology Vendors_ A Checklist f... by TrustArc
TrustArc Webinar - Managing Online Tracking Technology Vendors_ A Checklist f...TrustArc Webinar - Managing Online Tracking Technology Vendors_ A Checklist f...
TrustArc Webinar - Managing Online Tracking Technology Vendors_ A Checklist f...
TrustArc72 views
ESPC 2023 - Protect and Govern your Sensitive Data with Microsoft Purview in ... by Jasper Oosterveld
ESPC 2023 - Protect and Govern your Sensitive Data with Microsoft Purview in ...ESPC 2023 - Protect and Govern your Sensitive Data with Microsoft Purview in ...
ESPC 2023 - Protect and Govern your Sensitive Data with Microsoft Purview in ...
"Running students' code in isolation. The hard way", Yurii Holiuk by Fwdays
"Running students' code in isolation. The hard way", Yurii Holiuk "Running students' code in isolation. The hard way", Yurii Holiuk
"Running students' code in isolation. The hard way", Yurii Holiuk
Fwdays24 views
Piloting & Scaling Successfully With Microsoft Viva by Richard Harbridge
Piloting & Scaling Successfully With Microsoft VivaPiloting & Scaling Successfully With Microsoft Viva
Piloting & Scaling Successfully With Microsoft Viva
HTTP headers that make your website go faster - devs.gent November 2023 by Thijs Feryn
HTTP headers that make your website go faster - devs.gent November 2023HTTP headers that make your website go faster - devs.gent November 2023
HTTP headers that make your website go faster - devs.gent November 2023
Thijs Feryn26 views
Webinar : Desperately Seeking Transformation - Part 2: Insights from leading... by The Digital Insurer
Webinar : Desperately Seeking Transformation - Part 2:  Insights from leading...Webinar : Desperately Seeking Transformation - Part 2:  Insights from leading...
Webinar : Desperately Seeking Transformation - Part 2: Insights from leading...
Special_edition_innovator_2023.pdf by WillDavies22
Special_edition_innovator_2023.pdfSpecial_edition_innovator_2023.pdf
Special_edition_innovator_2023.pdf
WillDavies2218 views
"Node.js Development in 2024: trends and tools", Nikita Galkin by Fwdays
"Node.js Development in 2024: trends and tools", Nikita Galkin "Node.js Development in 2024: trends and tools", Nikita Galkin
"Node.js Development in 2024: trends and tools", Nikita Galkin
Fwdays17 views
STKI Israeli Market Study 2023 corrected forecast 2023_24 v3.pdf by Dr. Jimmy Schwarzkopf
STKI Israeli Market Study 2023   corrected forecast 2023_24 v3.pdfSTKI Israeli Market Study 2023   corrected forecast 2023_24 v3.pdf
STKI Israeli Market Study 2023 corrected forecast 2023_24 v3.pdf
Business Analyst Series 2023 - Week 3 Session 5 by DianaGray10
Business Analyst Series 2023 -  Week 3 Session 5Business Analyst Series 2023 -  Week 3 Session 5
Business Analyst Series 2023 - Week 3 Session 5
DianaGray10345 views
The Forbidden VPN Secrets.pdf by Mariam Shaba
The Forbidden VPN Secrets.pdfThe Forbidden VPN Secrets.pdf
The Forbidden VPN Secrets.pdf
Mariam Shaba20 views

Déjà Vu: The Importance of Time and Causality in Recommender Systems

  • 1. Déjà Vu The Importance of Time and Causality in Recommender Systems Justin Basilico & Yves Raimond August 29, 2017 @JustinBasilico @moustaki
  • 4. But first… Goodbye Why? +200% ratings volume Clear link to personalization & Cinematch % Match Hello &
  • 5. Image from Domiriel (cc by-nc)
  • 6. ● This moment can be controlled by the user ○ Visit time ○ Session length ● … or influenced by the system ○ Notifications, emails ● And must choose an action ○ … that has consequences Recommendations are actions at a moment in time ● Time and causality are critical aspects in any recommender system ○ Data collection ○ Experiment design (offline & online) ○ Algorithm & objective design ○ System design
  • 9. Violation of the space-time continuum! Observed labels Training input data collected Training time Serving time Serving input data collected
  • 12. ● Be careful when splitting dataset ○ Don’t overfit the past ○ Predict the future ● Rule of thumb: Split across what you need to generalize ○ Time! ○ Users or Items? ● May need to train/test at multiple distinct time points to see generalization across time (e.g. [Lathia et. al., 2009]) ● Simulate system behaviors (e.g. training and publishing delays) in evaluation pipeline ○ Helps capture trade-off between accuracy and responsiveness Experiment design Train Time Test
  • 15. ? ? Users changing over time Nonstationarity
  • 16. Items changing over time popularity time Learned item bias Actual item popularity Item launch Item becomes available
  • 17. ● Aggregation ○ Decay functions (e.g. [Ding, Li, 2005]) ○ Buckets (e.g. [Zimdars, Chickering, Meek, 2001]) ● Extrapolation (e.g. [Koren, 2009]) ● Sequences ○ Markov (e.g. [Rendle, et al., 2010]) ○ Last N (e.g. [Shani, Heckerman, Brafman, 2005]) ○ RNNs (e.g. [Hidasi et al., 2015]) ● Features ○ Discretized (e.g. [Baltrunas, Amatriain, 2009]) ○ Continuous (e.g. example age in [Covington et. al., 2016]) Some modeling approaches time
  • 18. ● Generalizing to future behaviors through temporal extrapolation ● Time exhibits many periodicities ○ Daily ○ Weekly ○ Seasonally ○ … and even longer: Olympics, elections, etc. ● Additional periodic time context features can be added or extracted Time as context Experiment on a Netflix internal dataset
  • 19. ● Recommendation systems are a means to an end ○ Reward = enjoyment - interaction cost ○ Enjoyment integrated over time (e.g. goodness * length of view) ○ Interaction cost integrated over time ○ Don’t waste your users time ○ Magnitudes of enjoyment and cost may be user-specific ● Maximize enjoyment of the selected item while minimizing time it takes to find the item Minimizing interaction time
  • 20. Hangul alphabet, 3 syllables but requires 7 (2 + 3 + 2) interactionsClick
  • 21. With a model optimized to minimize interaction time: one interaction Click
  • 25. Algorithm C Algorithm B Algorithm A Algorithms changing Idea Offline experimentation Online experimentation (A/B) Rollout
  • 26. Algorithm C Algorithm B Algorithm A Algorithms changing Idea Offline experimentation Online experimentation (A/B) Rollout Assumes stationarity! A change in other parts of the system might invalidate previous (offline or online) results. Holdback A/B tests as part of rollout can help.
  • 27. UX changing over time % Match&
  • 28. Feedback loops Impression bias inflates plays Leads to inflated item popularity More plays More impressions Oscillations in distribution of genre recommendations Feedback loops can cause biases to be reinforced by the recommendation system!
  • 32. Closed Loop Training Data Watches Model Recs Danger Zone Search Training Data Watches Model Recs Open Loop
  • 33. Closed Loop Training Data Watches Model Recs Danger Zone Search Training Data Watches Model Recs Open Loop
  • 34. Open vs. Closed Loops [Based on Steck, 2013 with system as selector] Watch when rec Probability of rec Watch when not rec Probability of not rec
  • 35. Open vs. Closed Loops [Based on Steck, 2013 with system as selector] Watch when rec Probability of rec Watch when not rec Probability of not rec Closed loop: 0 Open loop: > 0
  • 36. Open vs. Closed Loops [Based on Steck, 2013 with system as selector] Watch when rec Probability of rec Watch when not rec Probability of not rec Closed loop: 0 Open loop: > 0 We have control over this
  • 37. ● Maintain some controlled exploration to break feedback loop and handle non-stationarities ● Explore with -greedy, Thompson Sampling, etc. ● Control to avoid significantly degrading user experience ● Log as much as possible ○ Include counterfactuals: What maximal action system wanted to do (e.g. [Bottou et al., 2013]) Controlled stochasticity Explore Explore
  • 38. Replay Metrics Observed reward Existing recommendation algorithm (with stochasticity) Observed reward New recommendation algorithm [Li et al., 2011; Dudik, Langford, Li, 2014] Simulate online metrics, offline!
  • 39. ● Stochasticity opens the door to using causal inference ● Inverse Propensity Weighting ○ Reduce production bias by reweighting train and test data ○ Know probability of user receiving an impression ○ Doesn’t handle simultaneity and other endogeneity ● Covariate shift ○ Use explore data to estimate bias in other data ○ Use all data to train ● Instrumental variables for more general settings Causality [Schnabel et al., 2016; Liang, Charlin, Blei, 2016; Smola, 2011, Sugiyama, Kawanabe, 2012]
  • 40. ● Most recommendations (and ML) models are correlational ○ These items are correlated with these types of users ● But we seek causal actions ○ Showing this item is rewarding for this user ● Our recommendation action should have an incremental effect in reward: E[r(a)] - E[r(∅)] ○ Application-dependent choice of ∅ ○ Sometimes it may be better not to provide a recommendation that simply maximizes p(vi |u) ○ May provide less obvious recommendations Incrementality p(vi |∅) p(vi |a) Incremental effect
  • 41. ● Gold standard of causality ○ Random assignment ○ Measured across time ○ Incremental benefit of treatment ● Causality safety net? ○ Hard to test with full feedback loop effects ○ An algorithm may behave differently when training off its own data ○ Holdback tests A/B Testing Time A (Control) B (Treatment) Significant? Metrics
  • 43. ● After users and items, time is usually the next most important factor in recommendation systems ○ Model it as such ○ Evaluate it as such ○ Make it central to your system and infrastructure ● Recommender systems act in a causal loop ○ Influenced by themselves and others ○ Be thoughtful about feedback effects Takeaways
  • 44. Thank you. @JustinBasilico @moustaki Justin Basilico & Yves Raimond Yes, we’re hiring...
  • 45. Déjà Vu The Importance of Time and Causality in Recommender Systems Justin Basilico & Yves Raimond August 29, 2017 @JustinBasilico @moustaki