Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Recent Trends in Personalization: A Netflix Perspective

10,190 views

Published on

Invited talk at the Adaptive and Multi-Task Learning (AMTL) workshop at ICML 2019 on 2019-06-15 in Long Beach, CA.

Published in: Technology
  • DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT (2019 Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://tinyurl.com/sjhjjwg } ......................................................................................................................... Download Full EPUB Ebook here { https://tinyurl.com/sjhjjwg } ......................................................................................................................... Download Full doc Ebook here { https://tinyurl.com/sjhjjwg } ......................................................................................................................... Download PDF EBOOK here { https://tinyurl.com/sjhjjwg } ......................................................................................................................... Download EPUB Ebook here { https://tinyurl.com/sjhjjwg } ......................................................................................................................... Download doc Ebook here { https://tinyurl.com/sjhjjwg } ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • @paolostu If you go full screen and click on the links it works then!
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Do you have the video of this presentation ?
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • It would be good to have access to the various links in the presentation, currently it is not possible
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD THAT BOOKS INTO AVAILABLE FORMAT (2019 Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download Full EPUB Ebook here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download Full doc Ebook here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download PDF EBOOK here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download EPUB Ebook here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download doc Ebook here { http://bit.ly/2m6jJ5M } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book that can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer that is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBooks .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story That Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money That the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths that Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Recent Trends in Personalization: A Netflix Perspective

  1. 1. Recent Trends in Personalization: A Netflix Perspective Justin Basilico ICML 2019 Adaptive & Multi-Task Learning Workshop 2019-06-15 @JustinBasilico
  2. 2. Why do we personalize?
  3. 3. Help members find content to watch and enjoy to maximize member satisfaction and retention
  4. 4. Spark joy
  5. 5. What do we personalize?
  6. 6. Ordering of videos is personalized From what we recommend Ranking
  7. 7. Selection and placement of rows is personalized ... to how we construct a pageRows
  8. 8. Personalized images ... to what images to select
  9. 9. ... to reaching out to our members
  10. 10. Everything is a recommendation! Over 80% of what people watch comes from our recommendations Overview in [Gomez-Uribe & Hunt, 2016]
  11. 11. Isn’t this solved yet?
  12. 12. ○ Every person is unique with a variety of interests ○ Help people find what they want when they’re not sure what they want ○ Large datasets but small data per user … and potentially biased by the output of your system ○ Cold-start problems on all sides ○ Non-stationary, context-dependent, mood-dependent ○ More than just accuracy: Diversity, novelty, freshness, fairness, ... ○ ... No, personalization is hard!
  13. 13. Some recent trends in approaching these challenges: 1. Deep Learning 2. Causality 3. Bandits & Reinforcement Learning 4. Fairness 5. Experience Personalization Trending Now
  14. 14. Trend 1: Deep Learning in Recommendations
  15. 15. What~2012 ~2017 Deep Learning becomes popular in Machine Learning Deep Learning becomes popular in Recommender Systems What took so long?
  16. 16. Traditional Recommendations Collaborative Filtering: Recommend items that similar users have chosen 0 1 0 1 0 0 0 1 1 0 1 0 0 1 1 0 1 0 0 0 0 0 0 0 1 Users Items
  17. 17. U≈R V A Matrix Factorization view 2
  18. 18. U A Feed-Forward Network view V 2
  19. 19. U A (deeper) feed-forward view V Mean squared loss?
  20. 20. … isn’t always the best U V Mean squared loss ?
  21. 21. V … but opens up many possibilities Softmax Avg / Stack/ Sequence DNN / RNN / CNN Input interactions (X) (X) p(Y) 2018-12-2319:32:10 2018-12-2412:05:53 2019-01-0215:40:22
  22. 22. Sequence prediction ● Treat recommendations as a sequence classification problem ○ Input: sequence of user actions ○ Output: next action ● E.g. Gru4Rec [Hidasi et. al., 2016] ○ Input: sequence of items in a sessions ○ Output: next item in the session ● Also co-evolution: [Wu et al., 2017], [Dai et al., 2017]
  23. 23. Leveraging other data ● Example: YouTube Recommender [Covington et. al., 2016] ● Two stage ranker: candidate generation (shrinking set of items to rank) and ranking (classifying actual impressions) ● Two feed-forward, fully connected, networks with hundreds of features
  24. 24. Contextual sequence data 2017-12-10 15:40:22 2017-12-23 19:32:10 2017-12-24 12:05:53 2017-12-27 22:40:22 2017-12-29 19:39:36 2017-12-30 20:42:13 Context ItemSequence per user ? Time
  25. 25. Time-sensitive sequence prediction ● Proper modeling of time and system dynamics is critical ○ Recommendations are actions at a moment in time ● Experiment on a Netflix internal dataset ○ Input: Sequence of past plays and time context ■ Discrete time: Day-of-week (Mon, Tue, …) & Hour-of-day ■ Continuous time (aka timestamp) ○ Label: Predict next play (temporal split data)
  26. 26. Results
  27. 27. Trend 2: Causality
  28. 28. From Correlation to Causation ● Most recommendation algorithms are correlational ○ Some early recommendation algorithms literally computed correlations between users and items ● Did you watch a movie because you liked it? Or because we showed it to you? Or both? p(Y|X) → p(Y|X, do(R)) (from http://www.tylervigen.com/spurious-correlations)
  29. 29. Feedback loops Impression bias inflates plays Leads to inflated item popularity More plays More impressions Oscillations in distribution of genre recommendations Feedback loops can cause biases to be reinforced by the recommendation system! [Chaney et al., 2018]: simulations showing that this can reduce the usefulness of the system
  30. 30. Lots of feedback loops...
  31. 31. Closed Loop Training Data Watches Model Recs
  32. 32. Closed Loop Training Data Watches Model Recs Danger Zone
  33. 33. Closed Loop Training Data Watches Model Recs Danger Zone Search Training Data Watches Model Recs Open Loop
  34. 34. Closed Loop Training Data Watches Model Recs Danger Zone Search Training Data Watches Model Recs Open Loop
  35. 35. Debiasing Recommendations ● IPS Estimator for MF [Schnabel et al., 2016] ○ Train a debiasing model and reweight the data ● Causal Embeddings [Bonner & Vasile, 2018] ○ Jointly learn debiasing model and task model ○ Regularize the two towards each other ● Doubly-Robust MF [Wang et al., 2019]
  36. 36. Trend 3: Bandits & Reinforcement Learning in Recommendations
  37. 37. ● Uncertainty around user interests and new items ● Sparse and indirect feedback ● Changing trends ● Break feedback loops ● Want to explore to learn Why contextual bandits for recommendations? ▶Early news example: [Li et al., 2010]
  38. 38. Bart [McInerney et al., 2018] ● Bandit selecting both items and explanations for Spotify homepage ● Factorization Machine with epsilon-greedy explore over personalized candidate set ● Counterfactual risk minimization to train the bandit
  39. 39. Which artwork to show?
  40. 40. Artwork Personalization as Contextual Bandit ● Environment: Netflix homepage ● Context: Member, device, page, etc. ● Learner: Artwork selector for a show ● Action: Display specific image for show ● Reward: Member has positive engagement Artwork Selector ▶
  41. 41. Offline Replay Results ● Bandit finds good images ● Personalization is better ● Artwork variety matters ● Personalization wiggles around best images Lift in Replay in the various algorithms as compared to the Random baseline More info in our blog post
  42. 42. Going Long-Term ● Want to maximize long-term user satisfaction and retention ● Involves many user visits, recommendation actions and delayed reward ● … sounds like Reinforcement Learning
  43. 43. ● High-dimensional action space: Recommending a single item is O(|C|); typically want to do ranking or page construction, which is combinatorial ● High-dimensional state space: Users are represented in the state, along with the relevant history ● Off-policy training: Need to learn from existing system actions ● Concurrency: Don’t observe full trajectories, need to learn simultaneously from many interactions ● Changing action space: New actions (items) become available and need to be cold-started. ● No good simulator: Requires knowing feedback for user on recommended items Challenges of Reinforcement Learning for Recommendations
  44. 44. List-wise [Zhao et al., 2017] or Page-wise recommendation [Zhao et al. 2018] based on [Dulac-Arnold et al., 2016] Embeddings for actions
  45. 45. ● Generator to choose user action from recommendation ● Reward trained like a discriminator ● LSTM or Position-Weight architecture ● Learning over sets via cascading Deep Q Networks ○ Different Q function per position GAN-inspired as a user simulator [Chen et al., 2019]
  46. 46. ● Train candidate generator using REINFORCE ● Exploration done using softmax with temperature ● Off-policy correction with adaptation for top-k recommendations ● Trust region policy optimization to keep close to logging policy Policy Gradient for YouTube Recommendations [Chen et al., 2019]
  47. 47. Trend 4: Fairness
  48. 48. Personalization has a big impact in people’s lives How do we make sure that it is fair?
  49. 49. Calibrated Recommendations [Steck, 2018] ● Fairness as matching distribution of user interests ● Accuracy as an objective can lead to unbalanced predictions ● Simple example: ● Many recommendation algorithms exhibit this behavior of exaggerating the dominant interests and crowd out less frequent ones 30 action70 romance 30% action70% romance User: Expectation: 100% romanceReality: Maximizes accuracy
  50. 50. - Genre-distribution of each item is given: - Genre-distribution of user’s play history: … add prior for other genres: - Genre-distribution of recommended list: (for diversity) (or other categorization) Calibration Metric
  51. 51. Calibration Results (MovieLens 20M) Baseline model (wMF): Many users receive uncalibrated rec’s After reranking: Rec’s are much more calibrated (smaller ) Userdensity More calibrated (KL divergence) Submodular Reranker:
  52. 52. Fairness through Pairwise Comparisons [Beutel et al., 2019] ● Recommendations are fair if likelihood of clicked item being ranked above an unclicked item is the same across two groups ○ Intra-group pairwise accuracy - Restrict to pairs within group ○ Inter-group pairwise accuracy - Restrict to pairs between groups ● Training: Add pairwise regularizer based on randomized data to collect fairness feedback
  53. 53. Trend 5: Experience Personalization
  54. 54. Personalizing how we recommend (not just what we recommend…) ● Algorithm level: Ideal balance of diversity, popularity, novelty, freshness, etc. may depend on the person ● Display level: How you present items or explain recommendations can also be personalized ● Interaction level: Balancing the needs of lean-back users and power users
  55. 55. Page/Slate Optimization ● Select multiple actions that go together and receive feedback on group ● Personalizing based on within-session browsing behavior [Wu et al., 2015] ● Off-policy evaluation for slates [Swaminathan, et al., 2016] ● Slate optimization as VAE [Jiang et al., 2019] ● Marginal posterior sampling for slate bandits [Dimakopoulou et al., 2019]
  56. 56. More dimensions to personalize Rows Trailer Evidence Synopsis Image Row Title Metadata Ranking
  57. 57. More Adaptive UI
  58. 58. Rating Ranking Pages 4.7 Experience Evolution of our Personalization Approach
  59. 59. Potential Connections with Multi-Task / Meta Learning?
  60. 60. Applications as tasks ● Many related personalization tasks in a recommender system ● Examples: ○ [Zhao et al., 2015] - Outputs for different tasks ○ [Bansal et al., 2016] - Jointly learn to recommend and predict metadata for items ○ [Ma et al., 2018] - Jointly learn watch and enjoy ○ [Lu et al., 2018] - Jointly learn for rating prediction and explanation ○ [Hadash et al., 2018] - Jointly learn ranking and rating prediction User history Ranking Page Rating Explanation Search Image Context ...
  61. 61. Other views ● Users-as-tasks: Treat each user as a task and learn from others users ○ Example: [Ning & Karapis, 2010] finds similar users and does support vector regression ● Items-as-tasks: Treat each item as a separate model to learn ● Contexts-as-tasks: Treat different contexts (time, device, region, …) as separate tasks ● Domains-as-tasks: Leverage representations of users in one domain to help in another (e.g. different kinds of items, different genres) ○ Example: [Li et al., 2009] on movies <-> books
  62. 62. Conclusion
  63. 63. 1. Deep Learning 2. Causality 3. Bandits & Reinforcement Learning 4. Fairness 5. Experience Personalization 6. Multi-task & Meta Learning? Lots of opportunity for Machine Learning in Personalization
  64. 64. Thank you Questions? @JustinBasilico Yes, we’re hiring... Justin Basilico

×