Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Contextualization at Netflix

994 views

Published on

At Netflix we take context of the member seriously.

In this keynote talk we will see how modeling contextual factors such as time or device can help members to find the right content at the right moment

At the end, the goal is to maximize member satisfaction and retention

These slides will go through which contextual factors matters for the video service and why we choose to use them or not.



Published in: Technology
  • DOWNLOAD THAT BOOKS INTO AVAILABLE FORMAT (2019 Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://urlzs.com/UABbn } ......................................................................................................................... Download Full EPUB Ebook here { https://urlzs.com/UABbn } ......................................................................................................................... Download Full doc Ebook here { https://urlzs.com/UABbn } ......................................................................................................................... Download PDF EBOOK here { https://urlzs.com/UABbn } ......................................................................................................................... Download EPUB Ebook here { https://urlzs.com/UABbn } ......................................................................................................................... Download doc Ebook here { https://urlzs.com/UABbn } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book that can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer that is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBooks .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story That Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money That the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths that Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Contextualization at Netflix

  1. 1. Contextualization @ Netflix CARS@RecSys19, Sep20 Contextualization at Netflix Linas Baltrunas @LinasTw
  2. 2. Goal: to maximize member satisfaction and retention by helping members discover content they enjoy Netflix is Entertainment.
  3. 3. 1 Impact of Context
  4. 4. Technical Definition of Context Features that describe user’s experience and can rapidly change their states
  5. 5. Size of Opportunity of Contextual Factors are Determined by How do we Test? ○ Fraction of sessions with specific contextual condition ○ Magnitude of change in the taste patterns ○ Correlation with other features ○ Systems complexity, cost 1. Formulating hypothesis 2. Looking for a signal in important taste dimensions 3. Focus groups 4. AB test - moment of truth
  6. 6. Hour of the Day — Kids titles played wrt hour of the day
  7. 7. Day of the week — Fraction of movies (vs tv shows) discovered wrt day of the week
  8. 8. — Netflix runs a lot of user studies ○ Test new bold ideas ○ Understand members beyond observable metrics — Our Consumer Insights Team ran a study with 20K members ○ Goal: understand device usage patterns Device
  9. 9. TV ○ Median member characteristics ◆ Older male, full time employed, higher income ○ US, Mexico ○ Longer sessions ○ More discovery sessions ○ High on Kids & Family Mobile ● Median member characteristics ○ Younger, female~male ● India, South Korea ● Short sessions ● More continuation sessions ● Lower on Kids and Family
  10. 10. Country — Recommendations for the same member in JP & MX JAPAN MEXICO
  11. 11. Location — Regions have different viewing patterns — Las Chicas del Cable
  12. 12. Context Relevance to Netflix Members ○ Country ○ Device ○ Time ○ Cultural/Religious/National festivities *only some of these variables are used in production ● Time available ● Attention (background vs focus) ● Companion ● Outside events (sports, elections) ● Weather ● Seasonality ● Location (within country) ● User daily patterns (commute, etc)
  13. 13. 2 Models: CARS via Classification
  14. 14. Context Free, MF Context Aware, TF
  15. 15. Feature Based Context-Aware Model ● Continue Watching row ○ Time ○ Device ● Title ranking ● Row ranking
  16. 16. Title Ranking Model ● P(titleX=continue_watch | current_time, current_device, some_play_happens) Time, tPast Today t1,iOS t2,web t3,web t4,iOS ? ?
  17. 17. ● P(titleX=continue_watch | current_time, current_device, some_play_happens) ● Construction of the data set and feature extraction is the key ● Model matters, but it is a secondary concern Title Ranking Model Time, tPast Today t1,iOS t2,web t3,web t4,iOS
  18. 18. Data Set Construction t3,web,user1,item1 t3,web,user1,item2 t4,iOS,user1,item1 t4,iOS,user1,item2
  19. 19. Performance
  20. 20. 3 Models: Contextual RNN
  21. 21. ● Representation (Deep) Learning promises to do feature engineering for you ● Time is a complex contextual dimension that needs special attention ● Time exhibits many periodicities ○ Daily ○ Weekly ○ Seasonally ○ … and even longer: Olympics, elections, etc. ● Generalizing to future behaviors through temporal extrapolation Representation Learning
  22. 22. Sequence prediction ● Treat recommendations as a sequence classification problem ● Gru4Rec: ○ Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, Domonkos Tikk: Session-based Recommendations with Recurrent Neural Networks. ○ Input: sequence of items in a user history ○ Output: next item in the history
  23. 23. Contextual Sequence Data 2017-12-10 15:40:22 2017-12-23 19:32:10 2017-12-24 12:05:53 2017-12-27 22:40:22 2017-12-29 19:39:36 2017-12-30 20:42:13 Context ActionSequence per user ? Time
  24. 24. Training Contextual RNN 2017-12-23 19:32:10 2017-12-24 12:05:53 2017-12-27 22:40:22 2017-12-29 19:39:36 2017-12-30 20:42:13 Context ActionSequence per user ? Time 2017-12-10 15:40:22
  25. 25. Extrapolating Time 9AM in UK: :11PM in UK
  26. 26. Time-sensitive sequence prediction ● Experiment on a Netflix internal dataset ○ Context: ■ Discrete time ● Day-of-week: Sunday, Monday, … ● Hour-of-day ■ Continuous time (Timestamp) ■ Device ■ Country ○ Predict next play (temporal split data)
  27. 27. Results
  28. 28. Takeaways from Deep Learning ● Think beyond solving existing problems with new tools and instead think what new problems the new tools can solve ● Deep Learning works well for Recommendations... ○ When you go beyond the classic problem definition ○ Use more complex data such as contextual factors ● Lots of open areas to improve recommendations using deep learning ● Scales well
  29. 29. ● Contextual signals can be as strong as personal preferences ○ Model them as such ○ Evaluate them as such ○ Make them central to your system and infrastructure Final Note
  30. 30. Thank you Linas Baltrunas @LinasTw
  31. 31. Learn More ● Relevant presentations ● Blog post on continue watching ● Netflix research page ● Consumer Insights Team ● Localization ● Jobs ● @LinasTw

×