Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Netflix talk at ML Platform meetup Sep 2019

997 views

Published on

In this talk at the Netflix Machine Learning Platform Meetup on 12 Sep 2019, Fernando Amat and Elliot Chow from Netflix talk about the Bandit infrastructure for Personalized Recommendations

Published in: Technology
  • Be the first to comment

Netflix talk at ML Platform meetup Sep 2019

  1. 1. Bandit Infrastructure for Personalized Recommendations Elliot Chow & Fernando Amat Gil ML Platform Meetup September 12th 2019
  2. 2. Recommendations at Netflix User is going to abandon session in...
  3. 3. Recommendations at Netflix ● Personalized Homepage for each member ● Goal: quickly help members find content they love ● Challenge: ○ 150M+ members in 190 countries ○ New content added daily ● Recommendations Valued at: $1B* *Carlos A. Gomez-Uribe, Neil Hunt: The Netflix Recommender System: Algorithms, Business Value, and Innovation. ACM Trans. Management Inf. Syst. 6(4): 13:1-13:19 (2016)
  4. 4. Meaning ● What is this title? ● Why should I watch it?
  5. 5. Evidence to convey meaning
  6. 6. Evidence example I
  7. 7. Evidence example I: web homepage
  8. 8. Evidence example II Billboard iOS
  9. 9. Evidence personalization ● Multiple choices for each (show, evidence) pair ● For each title, hard to predict what resonates before launch ● Taste can change over time
  10. 10. Evidence personalization (cont’d) ● Asset choreography (slate): image, video, text, cast, etc Bandit
  11. 11. Bandit setup ● Generic framework to obtain unbiased training data ● Very vast literature (beyond scope) ● Every session is a mini A/B test Pred 1-Pred reward?
  12. 12. Bandit setup for evidence ● Goal: for each mini A/B test collect a sample with ○ Context x ○ Selected action (or treatment) a ○ Propensity p ○ Set of possible actions s ○ Reward r ● Once you have this, very similar to other ML approaches ● Obtaining this data is the hard part infrastructure-wise (Elliot’s section)
  13. 13. System requirements ● Scalable: at Netflix’s global size ● Generic: common policy (or model) and offline metrics API ● Flexible: ○ Each recommendation problems has different attribution, reward and context definition ○ Easy to add new canvases to the bandit system (image, video, text, cards, etc)
  14. 14. Policy API ● Class Slate (List[Items] items) ○ Vector of ids to recreate a composition (“row_10”, “img_124”, “img_1037”,..., “synopsis_32”, “card_101”) 1 2 3 4 5 7 6 8 9
  15. 15. Policy API ● Slate select(List[List[Item]] items) ○ Given a list of possible slates, return one of them (explore-exploit trade-off) ● Map[Slate, Double] propensity(List[List[Item] items) ○ Return propensity of each possible slate to debias data
  16. 16. Offline policy evaluation ● Unbiased offline evaluation from logged samples ● Inverse Propensity Scoring (IPS), Doubly Robust, etc E[Reward] = 2 / 3 User 1 User 2 User 3 User 4 User 5 User 6 Logged stochastic treatment Play? (binary reward) New policy assignment
  17. 17. Offline policy evaluation system ● Input: ○ Logged sample (x, a, s, r, p) ○ Trained policy ● Output: ○ Metric (IPS, SNIPS, DM, DR, RECAP, etc) ● Observations: ○ Trivially parallelizable (decomposes per sample) ○ Each sample needs to consider all possible candidates
  18. 18. Data
  19. 19. Actions taken by a Netflix member - Log-in - Search for “stra” - Play Stranger Things ... Data: Member Activity
  20. 20. Describes Netflix’s desire to take a specific action - Recommend Stranger Things - Display Image ... Data: Intent-to-Treat (ITT)
  21. 21. The actual experience as seen by the Netflix member - Showed Stranger Things on Home Page - Displayed Image ... Data: Treatment
  22. 22. Log In Play Stranger Things Post-PlayIntent-to-treatHome Page 13 Reasons Why on Popular on Netflix Stranger Things on My List Treatment Member Actions
  23. 23. “Closing the Loop”
  24. 24. “Closing the Loop”: Join Intent-to- Treat with Treatment - Did the intent-to-treat take effect as desired? - Which policy was used? - What were the propensities? - What features were used?
  25. 25. “Closing the Loop”: Attribution / Reward Computation - Define reward based on short-/long-term member activity - Compute in “near” real-time
  26. 26. “Closing the Loop”: Update and Publish Policy - Incorporate incoming observations - Evaluate offline replay metrics - Push updated policy
  27. 27. Infrastructure Components
  28. 28. Standardized Logging Standardize logging to simplify on- boarding - Common logging format - Centralized landing topics/tables
  29. 29. Real-time Processing with Flink Process member activity, treatment, and intent-to-treat events in real-time - Join intent-to-treat and treatment events - Prepare data in format amenable to reward computation - Kafka and Hive outputs
  30. 30. Spark Client For Attribution and Reward Computation Provide flexible processing of the joined member activity, treatment, and intent-to-treat data - Events sorted by time - Unbounded windowing support - Optimizations for common access patterns
  31. 31. Configurable, Common Attribution Templatize common attribution methods for simpler access to data - Select treatment T events - Attribute member actions A to treatments - Within time window W - Match treatment and member action based on key K - Materialize as Hive tables
  32. 32. Challenges
  33. 33. Microservice Architecture Edge Home Page Service Image Selection Service ... ... ... ... ...
  34. 34. Microservice Architecture Edge Home Page Service ... Image Selection Service ... ... ... ID1=ITT1: Image-123, Features... ID1 ID1 ...
  35. 35. Microservice Architecture Edge Home Page Service ... Image Selection Service ... ... ... ID1, Image-123 ID1→ITT1: Image-123, Features... ID1 ID1 ... ID1, Image- 123 ID1, Image- 123 ID1, Image- 123
  36. 36. Microservice Architecture Edge Home Page Service ... Image Selection Service ... ... ... ID1, Image-123 ID1→ITT1: Image-123, Features... ID1 ID1 ... ID1, Image- 123 ID1, Image- 123 ID1, Image- 123
  37. 37. Microservice Architecture Edge Home Page Service ... Image Selection Service ... ... ... Cache ITT1: Image-123, Features.. ...
  38. 38. Microservice Architecture Edge Home Page Service ... Image Selection Service ... ... ... Cache ITT1: Image-123, Features.. ID2 ... ID2, Image-123 ID2, Image- 123 ID2, Image- 123
  39. 39. Microservice Architecture Edge Home Page Service ... Image Selection Service ... ... ... Cache ITT1: Image-123, Features... ID2, Image-123 ID2 ID2 → ITT1 ... ID2, Image- 123 ID2, Image- 123
  40. 40. Stream Processing - Out-of-order data - Maintaining state for joins - Widely-varying traffic patterns
  41. 41. Scale - 150M+ members - 190 countries - 450B+ unique events per day - Many microservices - Many personalization algorithms
  42. 42. Used in Production as well as multiple A/B tests
  43. 43. Thank you

×