SlideShare a Scribd company logo
11
Past, Present & Future of Recommender Systems:
An Industry Perspective
Xavier Amatriain (Quora)
Justin Basilico (Netflix) RecSys 2016
@xamat @JustinBasilico
DeLorean image by JMortonPhoto.com & OtoGodfrey.com
2
1. Past
3
Netflix Prize
2006
4
For more information ...
5
2. Present
6
Recommender Systems in Industry
Recommender Systems are used pervasively across application
domains
7
Recommender Systems in Industry
click
upvote
downvote
expand
share
8
Beyond explicit feedback
▪ Applications typically oriented around an action: click, buy,
read, listen, watch, …
▪ Implicit Feedback
▪ More data: Implicit feedback comes as part of normal use
▪ Better data: Matches with actions we want to predict
▪ Augment with contextual information
▪ Content for cold-start
▪ Hybrid: Combine together when you can
9
Ranking
▪ Ranking items is central
to recommending
▪ News feeds
▪ Items in catalogs
▪ …
▪ Most recsys can be
assimilated to:
▪ A learning-to-rank approach
▪ A feature engineering
problem
10
Everything is a RecommendationRows
Ranking
11
3. Future
12
Many interesting future directions
1. Indirect feedback
2. Value-awareness
3. Full-page optimization
4. Personalizing the how
▪ Others
▪ Intent/session awareness
▪ Interactive recommendations
▪ Context awareness
▪ Deep learning for
recommendations
▪ Conversational interfaces/bots
for recommendations
▪ …
13
Indirect Feedback
Challenges
▪ User can only click on what you show
▪ But, what you show is the result of what your model
predicted is good
▪ No counterfactuals
▪ Implicit data has no real “negatives”
Potential solutions
▪ Attention models
▪ Context is also indirect/implicit feedback
▪ Explore/exploit approaches and learning across
time
▪ ...
click
upvote
downvote
expand
share
14
Value-aware recommendations
▪ Recsys optimize for probability of action
▪ Not all clicks/actions have the same “reward”
▪ Different margin in ecommerce
▪ Different “quality” of content
▪ Long-term retention vs. short-term clicks (clickbait)
▪ …
▪ In Quora, the value of showing a story to a user is
approximated by weighted sum of actions:
v = ∑a
va
1{ya
= 1}
▪ Extreme application of value-aware recommendations:
suggest items to create that have the highest value
▪ Netflix: Which shows to produce or license
▪ Quora: Answers and questions that are not in the service
15
Full page optimization
▪ Recommendations are rarely displayed in
isolation
▪ Rankings are combined with many other elements
to make a page
▪ Want to optimize the whole page
▪ Means jointly solving for set of items and
their placement
▪ While incorporating
▪ Diversity, freshness, exploration
▪ Depth and coverage of the item set
▪ Non-recommendation elements (navigation,
editorial, etc.)
▪ Needs work hand-in-hand with the UX
16
Personalizing How We Recommend (… not just what we recommend)
▪ Algorithm level: Ideal balance of diversity, novelty,
popularity, freshness, etc. may depend on the person
▪ Display level: How you present items or explain
recommendations can also be personalized
▪ Select the best information and presentation for a user to quickly
decide whether or not they want an item
▪ Interaction level: Balancing the needs of lean-back users and
power users
17
Rows
Example: Rows & Beyond
Hero
Image
Predicted
rating
Evidence
Synopsis
Horizontal
Image
Row Title
Metadata
Ranking
18
4. Conclusions
19
Conclusions
▪ Approaches have evolved a lot in the past 10 years
▪ Looking forward to the next 10
▪ Industry and academia working together has advanced the
field since the beginning, we should make sure that
continues
20
Thank You
Justin Basilico
jbasilico@netflix.com
@JustinBasilico
Xavier Amatriain
xavier@quora.com
@xamat

More Related Content

What's hot

Lessons Learned from Building Machine Learning Software at Netflix
Lessons Learned from Building Machine Learning Software at NetflixLessons Learned from Building Machine Learning Software at Netflix
Lessons Learned from Building Machine Learning Software at Netflix
Justin Basilico
 
Recommendation system
Recommendation system Recommendation system
Recommendation system
Vikrant Arya
 
Recommender system introduction
Recommender system   introductionRecommender system   introduction
Recommender system introduction
Liang Xiang
 
Shallow and Deep Latent Models for Recommender System
Shallow and Deep Latent Models for Recommender SystemShallow and Deep Latent Models for Recommender System
Shallow and Deep Latent Models for Recommender System
Anoop Deoras
 
Recommender systems: Content-based and collaborative filtering
Recommender systems: Content-based and collaborative filteringRecommender systems: Content-based and collaborative filtering
Recommender systems: Content-based and collaborative filtering
Viet-Trung TRAN
 
Deep Learning for Recommender Systems
Deep Learning for Recommender SystemsDeep Learning for Recommender Systems
Deep Learning for Recommender Systems
Justin Basilico
 
Recommender Systems
Recommender SystemsRecommender Systems
Recommender Systems
Girish Khanzode
 
Sequential Decision Making in Recommendations
Sequential Decision Making in RecommendationsSequential Decision Making in Recommendations
Sequential Decision Making in Recommendations
Jaya Kawale
 
Overview of recommender system
Overview of recommender systemOverview of recommender system
Overview of recommender system
Stanley Wang
 
Learning to Rank for Recommender Systems - ACM RecSys 2013 tutorial
Learning to Rank for Recommender Systems -  ACM RecSys 2013 tutorialLearning to Rank for Recommender Systems -  ACM RecSys 2013 tutorial
Learning to Rank for Recommender Systems - ACM RecSys 2013 tutorial
Alexandros Karatzoglou
 
Recommender Systems (Machine Learning Summer School 2014 @ CMU)
Recommender Systems (Machine Learning Summer School 2014 @ CMU)Recommender Systems (Machine Learning Summer School 2014 @ CMU)
Recommender Systems (Machine Learning Summer School 2014 @ CMU)
Xavier Amatriain
 
Learning to Rank - From pairwise approach to listwise
Learning to Rank - From pairwise approach to listwiseLearning to Rank - From pairwise approach to listwise
Learning to Rank - From pairwise approach to listwise
Hasan H Topcu
 
Recommender Systems In Industry
Recommender Systems In IndustryRecommender Systems In Industry
Recommender Systems In Industry
Xavier Amatriain
 
Past, present, and future of Recommender Systems: an industry perspective
Past, present, and future of Recommender Systems: an industry perspectivePast, present, and future of Recommender Systems: an industry perspective
Past, present, and future of Recommender Systems: an industry perspective
Xavier Amatriain
 
Tutorial: Context In Recommender Systems
Tutorial: Context In Recommender SystemsTutorial: Context In Recommender Systems
Tutorial: Context In Recommender Systems
YONG ZHENG
 
Introduction to Recommendation Systems
Introduction to Recommendation SystemsIntroduction to Recommendation Systems
Introduction to Recommendation Systems
Trieu Nguyen
 
Recommendation system
Recommendation systemRecommendation system
Recommendation system
Akshat Thakar
 
Past, Present & Future of Recommender Systems: An Industry Perspective
Past, Present & Future of Recommender Systems: An Industry PerspectivePast, Present & Future of Recommender Systems: An Industry Perspective
Past, Present & Future of Recommender Systems: An Industry Perspective
Justin Basilico
 
Deep Learning for Recommender Systems
Deep Learning for Recommender SystemsDeep Learning for Recommender Systems
Deep Learning for Recommender Systems
Yves Raimond
 
Recommender Systems
Recommender SystemsRecommender Systems
Recommender Systems
T212
 

What's hot (20)

Lessons Learned from Building Machine Learning Software at Netflix
Lessons Learned from Building Machine Learning Software at NetflixLessons Learned from Building Machine Learning Software at Netflix
Lessons Learned from Building Machine Learning Software at Netflix
 
Recommendation system
Recommendation system Recommendation system
Recommendation system
 
Recommender system introduction
Recommender system   introductionRecommender system   introduction
Recommender system introduction
 
Shallow and Deep Latent Models for Recommender System
Shallow and Deep Latent Models for Recommender SystemShallow and Deep Latent Models for Recommender System
Shallow and Deep Latent Models for Recommender System
 
Recommender systems: Content-based and collaborative filtering
Recommender systems: Content-based and collaborative filteringRecommender systems: Content-based and collaborative filtering
Recommender systems: Content-based and collaborative filtering
 
Deep Learning for Recommender Systems
Deep Learning for Recommender SystemsDeep Learning for Recommender Systems
Deep Learning for Recommender Systems
 
Recommender Systems
Recommender SystemsRecommender Systems
Recommender Systems
 
Sequential Decision Making in Recommendations
Sequential Decision Making in RecommendationsSequential Decision Making in Recommendations
Sequential Decision Making in Recommendations
 
Overview of recommender system
Overview of recommender systemOverview of recommender system
Overview of recommender system
 
Learning to Rank for Recommender Systems - ACM RecSys 2013 tutorial
Learning to Rank for Recommender Systems -  ACM RecSys 2013 tutorialLearning to Rank for Recommender Systems -  ACM RecSys 2013 tutorial
Learning to Rank for Recommender Systems - ACM RecSys 2013 tutorial
 
Recommender Systems (Machine Learning Summer School 2014 @ CMU)
Recommender Systems (Machine Learning Summer School 2014 @ CMU)Recommender Systems (Machine Learning Summer School 2014 @ CMU)
Recommender Systems (Machine Learning Summer School 2014 @ CMU)
 
Learning to Rank - From pairwise approach to listwise
Learning to Rank - From pairwise approach to listwiseLearning to Rank - From pairwise approach to listwise
Learning to Rank - From pairwise approach to listwise
 
Recommender Systems In Industry
Recommender Systems In IndustryRecommender Systems In Industry
Recommender Systems In Industry
 
Past, present, and future of Recommender Systems: an industry perspective
Past, present, and future of Recommender Systems: an industry perspectivePast, present, and future of Recommender Systems: an industry perspective
Past, present, and future of Recommender Systems: an industry perspective
 
Tutorial: Context In Recommender Systems
Tutorial: Context In Recommender SystemsTutorial: Context In Recommender Systems
Tutorial: Context In Recommender Systems
 
Introduction to Recommendation Systems
Introduction to Recommendation SystemsIntroduction to Recommendation Systems
Introduction to Recommendation Systems
 
Recommendation system
Recommendation systemRecommendation system
Recommendation system
 
Past, Present & Future of Recommender Systems: An Industry Perspective
Past, Present & Future of Recommender Systems: An Industry PerspectivePast, Present & Future of Recommender Systems: An Industry Perspective
Past, Present & Future of Recommender Systems: An Industry Perspective
 
Deep Learning for Recommender Systems
Deep Learning for Recommender SystemsDeep Learning for Recommender Systems
Deep Learning for Recommender Systems
 
Recommender Systems
Recommender SystemsRecommender Systems
Recommender Systems
 

Similar to Past present and future of Recommender Systems: an Industry Perspective

Make Customers Fall in Love with Your Salesforce Self-service Community
Make Customers Fall in Love with Your Salesforce Self-service CommunityMake Customers Fall in Love with Your Salesforce Self-service Community
Make Customers Fall in Love with Your Salesforce Self-service Community
Perficient, Inc.
 
Mini-training: Personalization & Recommendation Demystified
Mini-training: Personalization & Recommendation DemystifiedMini-training: Personalization & Recommendation Demystified
Mini-training: Personalization & Recommendation Demystified
Betclic Everest Group Tech Team
 
Gaps May 2011 Presentation1
Gaps May 2011 Presentation1Gaps May 2011 Presentation1
Gaps May 2011 Presentation1
Steve Rath
 
Warming Up to Analytics
Warming Up to AnalyticsWarming Up to Analytics
Warming Up to Analytics
Lewandog, Inc,
 
Recommendation at Netflix Scale
Recommendation at Netflix ScaleRecommendation at Netflix Scale
Recommendation at Netflix Scale
Justin Basilico
 
SharePoint 2013 Preview
SharePoint 2013 PreviewSharePoint 2013 Preview
SharePoint 2013 Preview
Regroove
 
Higher Ed Site Redesign Done Right
Higher Ed Site Redesign Done RightHigher Ed Site Redesign Done Right
Higher Ed Site Redesign Done Right
mStoner, Inc.
 
Modern Perspectives on Recommender Systems and their Applications in Mendeley
Modern Perspectives on Recommender Systems and their Applications in MendeleyModern Perspectives on Recommender Systems and their Applications in Mendeley
Modern Perspectives on Recommender Systems and their Applications in Mendeley
Maya Hristakeva
 
Design Thinking User Group Feedback
Design Thinking User Group FeedbackDesign Thinking User Group Feedback
Design Thinking User Group Feedback
Thomas Morrell
 
Configuring share point 2010 just do it
Configuring share point 2010   just do itConfiguring share point 2010   just do it
Configuring share point 2010 just do it
Marianne Sweeny
 
Ola ei nov. 22 2103
Ola ei nov. 22 2103Ola ei nov. 22 2103
Ola ei nov. 22 2103
Stephen Abram
 
Forget the Fairy Dust - How to Create Content That (Actually) Works
Forget the Fairy Dust - How to Create Content That (Actually) WorksForget the Fairy Dust - How to Create Content That (Actually) Works
Forget the Fairy Dust - How to Create Content That (Actually) Works
Joel Klettke
 
tagSurf
tagSurftagSurf
tagSurf
Paul Lacey
 
10 mistakes when you move to topic-based authoring
10 mistakes when you move to topic-based authoring10 mistakes when you move to topic-based authoring
10 mistakes when you move to topic-based authoring
Sharon Burton
 
UT_Analytics_Meeting
UT_Analytics_MeetingUT_Analytics_Meeting
UT_Analytics_Meeting
Nate Valentin
 
Content Strategy: Do It For Your Users
Content Strategy: Do It For Your UsersContent Strategy: Do It For Your Users
Content Strategy: Do It For Your Users
Andrea Sarther
 
6 Keys To Effective Content That Ranks High On SERPs Right Now
6 Keys To Effective Content That Ranks High On SERPs Right Now6 Keys To Effective Content That Ranks High On SERPs Right Now
6 Keys To Effective Content That Ranks High On SERPs Right Now
Search Engine Journal
 
qw3ries flf earlybird
qw3ries flf earlybirdqw3ries flf earlybird
qw3ries flf earlybird
qw3ries
 
Modern Perspectives on Recommender Systems and their Applications in Mendeley
Modern Perspectives on Recommender Systems and their Applications in MendeleyModern Perspectives on Recommender Systems and their Applications in Mendeley
Modern Perspectives on Recommender Systems and their Applications in Mendeley
Kris Jack
 
Search and Filter Interface Round Up - Userability Marathon 2009 - Amy Cueva
Search and Filter Interface Round Up - Userability Marathon 2009 - Amy CuevaSearch and Filter Interface Round Up - Userability Marathon 2009 - Amy Cueva
Search and Filter Interface Round Up - Userability Marathon 2009 - Amy Cueva
Mad*Pow
 

Similar to Past present and future of Recommender Systems: an Industry Perspective (20)

Make Customers Fall in Love with Your Salesforce Self-service Community
Make Customers Fall in Love with Your Salesforce Self-service CommunityMake Customers Fall in Love with Your Salesforce Self-service Community
Make Customers Fall in Love with Your Salesforce Self-service Community
 
Mini-training: Personalization & Recommendation Demystified
Mini-training: Personalization & Recommendation DemystifiedMini-training: Personalization & Recommendation Demystified
Mini-training: Personalization & Recommendation Demystified
 
Gaps May 2011 Presentation1
Gaps May 2011 Presentation1Gaps May 2011 Presentation1
Gaps May 2011 Presentation1
 
Warming Up to Analytics
Warming Up to AnalyticsWarming Up to Analytics
Warming Up to Analytics
 
Recommendation at Netflix Scale
Recommendation at Netflix ScaleRecommendation at Netflix Scale
Recommendation at Netflix Scale
 
SharePoint 2013 Preview
SharePoint 2013 PreviewSharePoint 2013 Preview
SharePoint 2013 Preview
 
Higher Ed Site Redesign Done Right
Higher Ed Site Redesign Done RightHigher Ed Site Redesign Done Right
Higher Ed Site Redesign Done Right
 
Modern Perspectives on Recommender Systems and their Applications in Mendeley
Modern Perspectives on Recommender Systems and their Applications in MendeleyModern Perspectives on Recommender Systems and their Applications in Mendeley
Modern Perspectives on Recommender Systems and their Applications in Mendeley
 
Design Thinking User Group Feedback
Design Thinking User Group FeedbackDesign Thinking User Group Feedback
Design Thinking User Group Feedback
 
Configuring share point 2010 just do it
Configuring share point 2010   just do itConfiguring share point 2010   just do it
Configuring share point 2010 just do it
 
Ola ei nov. 22 2103
Ola ei nov. 22 2103Ola ei nov. 22 2103
Ola ei nov. 22 2103
 
Forget the Fairy Dust - How to Create Content That (Actually) Works
Forget the Fairy Dust - How to Create Content That (Actually) WorksForget the Fairy Dust - How to Create Content That (Actually) Works
Forget the Fairy Dust - How to Create Content That (Actually) Works
 
tagSurf
tagSurftagSurf
tagSurf
 
10 mistakes when you move to topic-based authoring
10 mistakes when you move to topic-based authoring10 mistakes when you move to topic-based authoring
10 mistakes when you move to topic-based authoring
 
UT_Analytics_Meeting
UT_Analytics_MeetingUT_Analytics_Meeting
UT_Analytics_Meeting
 
Content Strategy: Do It For Your Users
Content Strategy: Do It For Your UsersContent Strategy: Do It For Your Users
Content Strategy: Do It For Your Users
 
6 Keys To Effective Content That Ranks High On SERPs Right Now
6 Keys To Effective Content That Ranks High On SERPs Right Now6 Keys To Effective Content That Ranks High On SERPs Right Now
6 Keys To Effective Content That Ranks High On SERPs Right Now
 
qw3ries flf earlybird
qw3ries flf earlybirdqw3ries flf earlybird
qw3ries flf earlybird
 
Modern Perspectives on Recommender Systems and their Applications in Mendeley
Modern Perspectives on Recommender Systems and their Applications in MendeleyModern Perspectives on Recommender Systems and their Applications in Mendeley
Modern Perspectives on Recommender Systems and their Applications in Mendeley
 
Search and Filter Interface Round Up - Userability Marathon 2009 - Amy Cueva
Search and Filter Interface Round Up - Userability Marathon 2009 - Amy CuevaSearch and Filter Interface Round Up - Userability Marathon 2009 - Amy Cueva
Search and Filter Interface Round Up - Userability Marathon 2009 - Amy Cueva
 

More from Xavier Amatriain

Data/AI driven product development: from video streaming to telehealth
Data/AI driven product development: from video streaming to telehealthData/AI driven product development: from video streaming to telehealth
Data/AI driven product development: from video streaming to telehealth
Xavier Amatriain
 
AI-driven product innovation: from Recommender Systems to COVID-19
AI-driven product innovation: from Recommender Systems to COVID-19AI-driven product innovation: from Recommender Systems to COVID-19
AI-driven product innovation: from Recommender Systems to COVID-19
Xavier Amatriain
 
AI for COVID-19 - Q42020 update
AI for COVID-19 - Q42020 updateAI for COVID-19 - Q42020 update
AI for COVID-19 - Q42020 update
Xavier Amatriain
 
AI for COVID-19: An online virtual care approach
AI for COVID-19: An online virtual care approachAI for COVID-19: An online virtual care approach
AI for COVID-19: An online virtual care approach
Xavier Amatriain
 
Lessons learned from building practical deep learning systems
Lessons learned from building practical deep learning systemsLessons learned from building practical deep learning systems
Lessons learned from building practical deep learning systems
Xavier Amatriain
 
AI for healthcare: Scaling Access and Quality of Care for Everyone
AI for healthcare: Scaling Access and Quality of Care for EveryoneAI for healthcare: Scaling Access and Quality of Care for Everyone
AI for healthcare: Scaling Access and Quality of Care for Everyone
Xavier Amatriain
 
Towards online universal quality healthcare through AI
Towards online universal quality healthcare through AITowards online universal quality healthcare through AI
Towards online universal quality healthcare through AI
Xavier Amatriain
 
From one to zero: Going smaller as a growth strategy
From one to zero: Going smaller as a growth strategyFrom one to zero: Going smaller as a growth strategy
From one to zero: Going smaller as a growth strategy
Xavier Amatriain
 
Learning to speak medicine
Learning to speak medicineLearning to speak medicine
Learning to speak medicine
Xavier Amatriain
 
ML to cure the world
ML to cure the worldML to cure the world
ML to cure the world
Xavier Amatriain
 
Medical advice as a Recommender System
Medical advice as a Recommender SystemMedical advice as a Recommender System
Medical advice as a Recommender System
Xavier Amatriain
 
Recsys 2016 tutorial: Lessons learned from building real-life recommender sys...
Recsys 2016 tutorial: Lessons learned from building real-life recommender sys...Recsys 2016 tutorial: Lessons learned from building real-life recommender sys...
Recsys 2016 tutorial: Lessons learned from building real-life recommender sys...
Xavier Amatriain
 
Staying Shallow & Lean in a Deep Learning World
Staying Shallow & Lean in a Deep Learning WorldStaying Shallow & Lean in a Deep Learning World
Staying Shallow & Lean in a Deep Learning World
Xavier Amatriain
 
Machine Learning for Q&A Sites: The Quora Example
Machine Learning for Q&A Sites: The Quora ExampleMachine Learning for Q&A Sites: The Quora Example
Machine Learning for Q&A Sites: The Quora Example
Xavier Amatriain
 
BIG2016- Lessons Learned from building real-life user-focused Big Data systems
BIG2016- Lessons Learned from building real-life user-focused Big Data systemsBIG2016- Lessons Learned from building real-life user-focused Big Data systems
BIG2016- Lessons Learned from building real-life user-focused Big Data systems
Xavier Amatriain
 
Strata 2016 - Lessons Learned from building real-life Machine Learning Systems
Strata 2016 -  Lessons Learned from building real-life Machine Learning SystemsStrata 2016 -  Lessons Learned from building real-life Machine Learning Systems
Strata 2016 - Lessons Learned from building real-life Machine Learning Systems
Xavier Amatriain
 
Barcelona ML Meetup - Lessons Learned
Barcelona ML Meetup - Lessons LearnedBarcelona ML Meetup - Lessons Learned
Barcelona ML Meetup - Lessons Learned
Xavier Amatriain
 
10 more lessons learned from building Machine Learning systems - MLConf
10 more lessons learned from building Machine Learning systems - MLConf10 more lessons learned from building Machine Learning systems - MLConf
10 more lessons learned from building Machine Learning systems - MLConf
Xavier Amatriain
 
10 more lessons learned from building Machine Learning systems
10 more lessons learned from building Machine Learning systems10 more lessons learned from building Machine Learning systems
10 more lessons learned from building Machine Learning systems
Xavier Amatriain
 
Machine Learning to Grow the World's Knowledge
Machine Learning to Grow  the World's KnowledgeMachine Learning to Grow  the World's Knowledge
Machine Learning to Grow the World's Knowledge
Xavier Amatriain
 

More from Xavier Amatriain (20)

Data/AI driven product development: from video streaming to telehealth
Data/AI driven product development: from video streaming to telehealthData/AI driven product development: from video streaming to telehealth
Data/AI driven product development: from video streaming to telehealth
 
AI-driven product innovation: from Recommender Systems to COVID-19
AI-driven product innovation: from Recommender Systems to COVID-19AI-driven product innovation: from Recommender Systems to COVID-19
AI-driven product innovation: from Recommender Systems to COVID-19
 
AI for COVID-19 - Q42020 update
AI for COVID-19 - Q42020 updateAI for COVID-19 - Q42020 update
AI for COVID-19 - Q42020 update
 
AI for COVID-19: An online virtual care approach
AI for COVID-19: An online virtual care approachAI for COVID-19: An online virtual care approach
AI for COVID-19: An online virtual care approach
 
Lessons learned from building practical deep learning systems
Lessons learned from building practical deep learning systemsLessons learned from building practical deep learning systems
Lessons learned from building practical deep learning systems
 
AI for healthcare: Scaling Access and Quality of Care for Everyone
AI for healthcare: Scaling Access and Quality of Care for EveryoneAI for healthcare: Scaling Access and Quality of Care for Everyone
AI for healthcare: Scaling Access and Quality of Care for Everyone
 
Towards online universal quality healthcare through AI
Towards online universal quality healthcare through AITowards online universal quality healthcare through AI
Towards online universal quality healthcare through AI
 
From one to zero: Going smaller as a growth strategy
From one to zero: Going smaller as a growth strategyFrom one to zero: Going smaller as a growth strategy
From one to zero: Going smaller as a growth strategy
 
Learning to speak medicine
Learning to speak medicineLearning to speak medicine
Learning to speak medicine
 
ML to cure the world
ML to cure the worldML to cure the world
ML to cure the world
 
Medical advice as a Recommender System
Medical advice as a Recommender SystemMedical advice as a Recommender System
Medical advice as a Recommender System
 
Recsys 2016 tutorial: Lessons learned from building real-life recommender sys...
Recsys 2016 tutorial: Lessons learned from building real-life recommender sys...Recsys 2016 tutorial: Lessons learned from building real-life recommender sys...
Recsys 2016 tutorial: Lessons learned from building real-life recommender sys...
 
Staying Shallow & Lean in a Deep Learning World
Staying Shallow & Lean in a Deep Learning WorldStaying Shallow & Lean in a Deep Learning World
Staying Shallow & Lean in a Deep Learning World
 
Machine Learning for Q&A Sites: The Quora Example
Machine Learning for Q&A Sites: The Quora ExampleMachine Learning for Q&A Sites: The Quora Example
Machine Learning for Q&A Sites: The Quora Example
 
BIG2016- Lessons Learned from building real-life user-focused Big Data systems
BIG2016- Lessons Learned from building real-life user-focused Big Data systemsBIG2016- Lessons Learned from building real-life user-focused Big Data systems
BIG2016- Lessons Learned from building real-life user-focused Big Data systems
 
Strata 2016 - Lessons Learned from building real-life Machine Learning Systems
Strata 2016 -  Lessons Learned from building real-life Machine Learning SystemsStrata 2016 -  Lessons Learned from building real-life Machine Learning Systems
Strata 2016 - Lessons Learned from building real-life Machine Learning Systems
 
Barcelona ML Meetup - Lessons Learned
Barcelona ML Meetup - Lessons LearnedBarcelona ML Meetup - Lessons Learned
Barcelona ML Meetup - Lessons Learned
 
10 more lessons learned from building Machine Learning systems - MLConf
10 more lessons learned from building Machine Learning systems - MLConf10 more lessons learned from building Machine Learning systems - MLConf
10 more lessons learned from building Machine Learning systems - MLConf
 
10 more lessons learned from building Machine Learning systems
10 more lessons learned from building Machine Learning systems10 more lessons learned from building Machine Learning systems
10 more lessons learned from building Machine Learning systems
 
Machine Learning to Grow the World's Knowledge
Machine Learning to Grow  the World's KnowledgeMachine Learning to Grow  the World's Knowledge
Machine Learning to Grow the World's Knowledge
 

Recently uploaded

Google I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged SlidesGoogle I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged Slides
Google Developer Group - Harare
 
Computer HARDWARE presenattion by CWD students class 10
Computer HARDWARE presenattion by CWD students class 10Computer HARDWARE presenattion by CWD students class 10
Computer HARDWARE presenattion by CWD students class 10
ankush9927
 
The Impact of the Internet of Things (IoT) on Smart Homes and Cities
The Impact of the Internet of Things (IoT) on Smart Homes and CitiesThe Impact of the Internet of Things (IoT) on Smart Homes and Cities
The Impact of the Internet of Things (IoT) on Smart Homes and Cities
Arpan Buwa
 
Using LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and MilvusUsing LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and Milvus
Zilliz
 
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
Priyanka Aash
 
Patch Tuesday de julio
Patch Tuesday de julioPatch Tuesday de julio
Patch Tuesday de julio
Ivanti
 
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
alexjohnson7307
 
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
bhumivarma35300
 
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptxUse Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
SynapseIndia
 
Tailored CRM Software Development for Enhanced Customer Insights
Tailored CRM Software Development for Enhanced Customer InsightsTailored CRM Software Development for Enhanced Customer Insights
Tailored CRM Software Development for Enhanced Customer Insights
SynapseIndia
 
(CISOPlatform Summit & SACON 2024) Cyber Insurance & Risk Quantification.pdf
(CISOPlatform Summit & SACON 2024) Cyber Insurance & Risk Quantification.pdf(CISOPlatform Summit & SACON 2024) Cyber Insurance & Risk Quantification.pdf
(CISOPlatform Summit & SACON 2024) Cyber Insurance & Risk Quantification.pdf
Priyanka Aash
 
Uncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in LibrariesUncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in Libraries
Brian Pichman
 
Mule Experience Hub and Release Channel with Java 17
Mule Experience Hub and Release Channel with Java 17Mule Experience Hub and Release Channel with Java 17
Mule Experience Hub and Release Channel with Java 17
Bhajan Mehta
 
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Nicolás Lopéz
 
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision MakingConnector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
DianaGray10
 
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
alexjohnson7307
 
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
sunilverma7884
 
Acumatica vs. Sage Intacct _Construction_July (1).pptx
Acumatica vs. Sage Intacct _Construction_July (1).pptxAcumatica vs. Sage Intacct _Construction_July (1).pptx
Acumatica vs. Sage Intacct _Construction_July (1).pptx
BrainSell Technologies
 
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptxMAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
janagijoythi
 
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
BrainSell Technologies
 

Recently uploaded (20)

Google I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged SlidesGoogle I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged Slides
 
Computer HARDWARE presenattion by CWD students class 10
Computer HARDWARE presenattion by CWD students class 10Computer HARDWARE presenattion by CWD students class 10
Computer HARDWARE presenattion by CWD students class 10
 
The Impact of the Internet of Things (IoT) on Smart Homes and Cities
The Impact of the Internet of Things (IoT) on Smart Homes and CitiesThe Impact of the Internet of Things (IoT) on Smart Homes and Cities
The Impact of the Internet of Things (IoT) on Smart Homes and Cities
 
Using LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and MilvusUsing LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and Milvus
 
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
 
Patch Tuesday de julio
Patch Tuesday de julioPatch Tuesday de julio
Patch Tuesday de julio
 
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
 
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
 
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptxUse Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
 
Tailored CRM Software Development for Enhanced Customer Insights
Tailored CRM Software Development for Enhanced Customer InsightsTailored CRM Software Development for Enhanced Customer Insights
Tailored CRM Software Development for Enhanced Customer Insights
 
(CISOPlatform Summit & SACON 2024) Cyber Insurance & Risk Quantification.pdf
(CISOPlatform Summit & SACON 2024) Cyber Insurance & Risk Quantification.pdf(CISOPlatform Summit & SACON 2024) Cyber Insurance & Risk Quantification.pdf
(CISOPlatform Summit & SACON 2024) Cyber Insurance & Risk Quantification.pdf
 
Uncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in LibrariesUncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in Libraries
 
Mule Experience Hub and Release Channel with Java 17
Mule Experience Hub and Release Channel with Java 17Mule Experience Hub and Release Channel with Java 17
Mule Experience Hub and Release Channel with Java 17
 
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024
 
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision MakingConnector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
 
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
 
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
 
Acumatica vs. Sage Intacct _Construction_July (1).pptx
Acumatica vs. Sage Intacct _Construction_July (1).pptxAcumatica vs. Sage Intacct _Construction_July (1).pptx
Acumatica vs. Sage Intacct _Construction_July (1).pptx
 
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptxMAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
 
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
 

Past present and future of Recommender Systems: an Industry Perspective

  • 1. 11 Past, Present & Future of Recommender Systems: An Industry Perspective Xavier Amatriain (Quora) Justin Basilico (Netflix) RecSys 2016 @xamat @JustinBasilico DeLorean image by JMortonPhoto.com & OtoGodfrey.com
  • 6. 6 Recommender Systems in Industry Recommender Systems are used pervasively across application domains
  • 7. 7 Recommender Systems in Industry click upvote downvote expand share
  • 8. 8 Beyond explicit feedback ▪ Applications typically oriented around an action: click, buy, read, listen, watch, … ▪ Implicit Feedback ▪ More data: Implicit feedback comes as part of normal use ▪ Better data: Matches with actions we want to predict ▪ Augment with contextual information ▪ Content for cold-start ▪ Hybrid: Combine together when you can
  • 9. 9 Ranking ▪ Ranking items is central to recommending ▪ News feeds ▪ Items in catalogs ▪ … ▪ Most recsys can be assimilated to: ▪ A learning-to-rank approach ▪ A feature engineering problem
  • 10. 10 Everything is a RecommendationRows Ranking
  • 12. 12 Many interesting future directions 1. Indirect feedback 2. Value-awareness 3. Full-page optimization 4. Personalizing the how ▪ Others ▪ Intent/session awareness ▪ Interactive recommendations ▪ Context awareness ▪ Deep learning for recommendations ▪ Conversational interfaces/bots for recommendations ▪ …
  • 13. 13 Indirect Feedback Challenges ▪ User can only click on what you show ▪ But, what you show is the result of what your model predicted is good ▪ No counterfactuals ▪ Implicit data has no real “negatives” Potential solutions ▪ Attention models ▪ Context is also indirect/implicit feedback ▪ Explore/exploit approaches and learning across time ▪ ... click upvote downvote expand share
  • 14. 14 Value-aware recommendations ▪ Recsys optimize for probability of action ▪ Not all clicks/actions have the same “reward” ▪ Different margin in ecommerce ▪ Different “quality” of content ▪ Long-term retention vs. short-term clicks (clickbait) ▪ … ▪ In Quora, the value of showing a story to a user is approximated by weighted sum of actions: v = ∑a va 1{ya = 1} ▪ Extreme application of value-aware recommendations: suggest items to create that have the highest value ▪ Netflix: Which shows to produce or license ▪ Quora: Answers and questions that are not in the service
  • 15. 15 Full page optimization ▪ Recommendations are rarely displayed in isolation ▪ Rankings are combined with many other elements to make a page ▪ Want to optimize the whole page ▪ Means jointly solving for set of items and their placement ▪ While incorporating ▪ Diversity, freshness, exploration ▪ Depth and coverage of the item set ▪ Non-recommendation elements (navigation, editorial, etc.) ▪ Needs work hand-in-hand with the UX
  • 16. 16 Personalizing How We Recommend (… not just what we recommend) ▪ Algorithm level: Ideal balance of diversity, novelty, popularity, freshness, etc. may depend on the person ▪ Display level: How you present items or explain recommendations can also be personalized ▪ Select the best information and presentation for a user to quickly decide whether or not they want an item ▪ Interaction level: Balancing the needs of lean-back users and power users
  • 17. 17 Rows Example: Rows & Beyond Hero Image Predicted rating Evidence Synopsis Horizontal Image Row Title Metadata Ranking
  • 19. 19 Conclusions ▪ Approaches have evolved a lot in the past 10 years ▪ Looking forward to the next 10 ▪ Industry and academia working together has advanced the field since the beginning, we should make sure that continues