SlideShare a Scribd company logo
1 of 32
Download to read offline
Browsemap: 
Collaborative Filtering At LinkedIn 
Lili Wu, Sam Shah, Sean Choi, Mitul Tiwari, Christian Posse 
RSWeb 2014 
with RecSys 
Recruiting Solutions 1
2 
Agenda 
§ Motivation 
§ Architecture 
§ Applications 
§ Lessons Learned
Collaborative filtering for member profile 
Profile Browsemap: 
People who viewed 
this profile 
also viewed… 
3 
Count co-views
4 
Collaborative filtering for job page 
Job Browsemap: 
People who viewed 
this job 
also viewed… 
Count co-views
5 
… many CF based recommenders 
group company portfolio
• Many different entities 
• Similar problems with different requirement 
• Fast product development cycle 
• Hybrid recommender systems 
• Handle LinkedIn data volume and traffic 
6 
Challenges
• Many different entities 
• Similar problems with different requirement 
• Fast product development cycle 
• Hybrid recommender systems 
• Handle LinkedIn data volume and traffic 
7 
Challenges 
è Horizontal Platform
8 
Browsemap 
Collaborative Filtering Platform 
at LinkedIn
9 
Browsemap Platform 
• Scalability 
Ø Online/offline architecture 
Ø Hundreds of millions of entities, billions of 
monthly page views 
• Browsemap Domain Specific Language (DSL) 
Ø Code reuse through modular components 
Ø Flexible computation workflow construction 
• Data are used by hybrid recommenders
10 
Browsemap Architecture 
Frontend 
Services 
User 
Activity 
Data 
HDFS 
Queries 
Results 
Hadoop 
Browsemap 
DSL 
Browsemap 
Engine 
Online 
Query 
API 
Key-value 
storage 
Voldemort
11 
Browsemap Architecture 
Frontend 
Services 
HDFS 
Queries 
Results 
Hadoop 
Browsemap 
DSL 
Browsemap 
Engine 
Online 
Query 
API 
Key-value 
storage 
Voldemort 
User 
Activity 
High Data 
Throughput
12 
Browsemap Architecture 
Frontend 
Services 
HDFS 
Queries 
Results 
Hadoop 
Browsemap 
DSL 
Browsemap 
Engine 
Online 
Query 
API 
Key-value 
storage 
Voldemort 
User 
Activity 
Data 
Low 
Latency
13 
Browsemap Domain Specific Language 
(DSL) 
Module 
Collection 
Expired Job 
Filtering 
Spam User 
Filtering 
Co-view 
counting 
 
Job  
Expired Job 
Filtering 
Cold-start 
techniques 
… 
Co-view 
counting 
… 
Cold-start 
techniques 
Job browsemap 
Company 
… 
Spam User 
Filtering 
Spam User 
Filtering 
… 
Co-view 
counting 
… 
Cold-start 
techniques 
Company 
browsemap
• Support all entity types 
• Adjust to each product requirement 
• Scale 
14 
Recap 
Voldemort
15 
Agenda 
ü Motivation 
ü Architecture 
§ Applications 
§ Lessons Learned
16 
Applications – CF based recommenders 
Profile Browsemap 
Job Browsemap Group Browsemap 
Portfolio Browsemap 
Hiring Browsemap 
Company Browsemap 
Influencer Browsemap
17 
Applications – Hybrid Recommender Systems 
Suggested 
Profile 
Update 
Swee Lim
18 
Applications – Hybrid Recommender Systems 
Suggested 
Profile 
Update 
Goal: for each member, 
find companies he may want to follow
19 
Applications – Hybrid Recommender Systems 
Member info: 
• Content-based features 
title, industry, location, … 
• Collaborative filtering feature 
Google Cisco 
Member 
followed 
companies 
Linkedin, 
Facebook 
Juniper, 
Arista Companies user may 
be interested in 
… 
… 
Co-follow Browsemaps: 
People who follow 
this company also 
follow these companies
20 
Applications – Hybrid Recommender Systems 
Question: 
For a company C, will member M like it? 
Approach: 
Logistic regression 
Features: 
member location company location 
1 if yes, 0 if no 
company is in the list of the co-follow browsemaps ? 
1 if yes, 0 if no 
…
21 
Applications – Hybrid Recommender Systems 
Collaborative Filtering is important: 
• Surface implicit connection between companies 
• Based on Member’s preference
22 
Agenda 
ü Motivation 
ü Architecture 
ü Applications 
§ Lessons Learned
Lesson 1: Tall oaks grow from little acorns 
23
Lesson 1: Tall oaks grow from little acorns 
24
Lesson 1: Tall oaks grow from little acorns 
25
Lesson 1: Tall oaks grow from little acorns 
26 
A generic horizontal platform is essential
Lesson 2: One hand washes the other 
27 
Job Browsemap 
Similar Jobs 
Collaborative filtering: 
“Follower audience” 
Content based: 
“Leader audience”
Lesson 3: You can’t get blood out of a stone 
28 
Need to handle cold start problem 
Job 1 Job 2 Job 3 
(new) 
(view 
time) 
merge 
Leverage Browsing History Personalized Backfill
Lesson 4: A chain is only as strong as its weakest link 
29 
CF: Relies solely on user activities 
Good data is crucial 
§ Mistakes can be hard to detect / debug 
§ Simple mistakes can have big impact 
e.g. “jobid” à “id” 
§ Need prevention mechanism 
Ø Improve tracking 
Ø Unit test 
Ø Browsemap platform data-check : 
input volume, coverage/metrics analysis
Lesson 5: User experience matters 
50% 
CTR 
500% 
more 
applications 
30 
ª Put recommendations in user’s flow
31 
Conclusion 
§ Collaborative filtering is important for 
LinkedIn 
§ Browsemap is in production for 3+ years 
§ Horizontal platform is crucial
32 
Thank you ! 
§ Questions?

More Related Content

What's hot

Recommendation system by_arpit_sharma
Recommendation system by_arpit_sharmaRecommendation system by_arpit_sharma
Recommendation system by_arpit_sharmaEr. Arpit Sharma
 
Recommender Systems (Machine Learning Summer School 2014 @ CMU)
Recommender Systems (Machine Learning Summer School 2014 @ CMU)Recommender Systems (Machine Learning Summer School 2014 @ CMU)
Recommender Systems (Machine Learning Summer School 2014 @ CMU)Xavier Amatriain
 
Overview of recommender system
Overview of recommender systemOverview of recommender system
Overview of recommender systemStanley Wang
 
Recommandation systems -
Recommandation systems - Recommandation systems -
Recommandation systems - Yousef Fadila
 
Past, present, and future of Recommender Systems: an industry perspective
Past, present, and future of Recommender Systems: an industry perspectivePast, present, and future of Recommender Systems: an industry perspective
Past, present, and future of Recommender Systems: an industry perspectiveXavier Amatriain
 
Social Recommender Systems
Social Recommender SystemsSocial Recommender Systems
Social Recommender Systemsguest77b0cd12
 
Qcon SF 2013 - Machine Learning & Recommender Systems @ Netflix Scale
Qcon SF 2013 - Machine Learning & Recommender Systems @ Netflix ScaleQcon SF 2013 - Machine Learning & Recommender Systems @ Netflix Scale
Qcon SF 2013 - Machine Learning & Recommender Systems @ Netflix ScaleXavier Amatriain
 
Recommendation System Explained
Recommendation System ExplainedRecommendation System Explained
Recommendation System ExplainedCrossing Minds
 
Information Retrieval Models for Recommender Systems - PhD slides
Information Retrieval Models for Recommender Systems - PhD slidesInformation Retrieval Models for Recommender Systems - PhD slides
Information Retrieval Models for Recommender Systems - PhD slidesDaniel Valcarce
 
Recent advances in deep recommender systems
Recent advances in deep recommender systemsRecent advances in deep recommender systems
Recent advances in deep recommender systemsNAVER Engineering
 
Recommenders Systems
Recommenders SystemsRecommenders Systems
Recommenders SystemsTariq Hassan
 
Scalable advertising recommender systems
Scalable advertising recommender systemsScalable advertising recommender systems
Scalable advertising recommender systemsJoaquin Delgado PhD.
 
Introduction to Recommendation System
Introduction to Recommendation SystemIntroduction to Recommendation System
Introduction to Recommendation SystemMinha Hwang
 
Movie recommender system using the user's psychological profile
Movie recommender system using the user's psychological profileMovie recommender system using the user's psychological profile
Movie recommender system using the user's psychological profileUniversity Politehnica Bucharest
 
Movie lens recommender systems
Movie lens recommender systemsMovie lens recommender systems
Movie lens recommender systemsKapil Garg
 
DIY ERM (Do-It-Yourself Electronic Resources Management) for the Small Library
DIY ERM (Do-It-Yourself Electronic Resources Management) for the Small LibraryDIY ERM (Do-It-Yourself Electronic Resources Management) for the Small Library
DIY ERM (Do-It-Yourself Electronic Resources Management) for the Small LibraryNASIG
 

What's hot (19)

Recommendation system by_arpit_sharma
Recommendation system by_arpit_sharmaRecommendation system by_arpit_sharma
Recommendation system by_arpit_sharma
 
Recommender Systems (Machine Learning Summer School 2014 @ CMU)
Recommender Systems (Machine Learning Summer School 2014 @ CMU)Recommender Systems (Machine Learning Summer School 2014 @ CMU)
Recommender Systems (Machine Learning Summer School 2014 @ CMU)
 
Recommender Systems
Recommender SystemsRecommender Systems
Recommender Systems
 
Overview of recommender system
Overview of recommender systemOverview of recommender system
Overview of recommender system
 
Recsys 2016
Recsys 2016Recsys 2016
Recsys 2016
 
Recommandation systems -
Recommandation systems - Recommandation systems -
Recommandation systems -
 
Past, present, and future of Recommender Systems: an industry perspective
Past, present, and future of Recommender Systems: an industry perspectivePast, present, and future of Recommender Systems: an industry perspective
Past, present, and future of Recommender Systems: an industry perspective
 
Social Recommender Systems
Social Recommender SystemsSocial Recommender Systems
Social Recommender Systems
 
Qcon SF 2013 - Machine Learning & Recommender Systems @ Netflix Scale
Qcon SF 2013 - Machine Learning & Recommender Systems @ Netflix ScaleQcon SF 2013 - Machine Learning & Recommender Systems @ Netflix Scale
Qcon SF 2013 - Machine Learning & Recommender Systems @ Netflix Scale
 
Recommender Systems
Recommender SystemsRecommender Systems
Recommender Systems
 
Recommendation System Explained
Recommendation System ExplainedRecommendation System Explained
Recommendation System Explained
 
Information Retrieval Models for Recommender Systems - PhD slides
Information Retrieval Models for Recommender Systems - PhD slidesInformation Retrieval Models for Recommender Systems - PhD slides
Information Retrieval Models for Recommender Systems - PhD slides
 
Recent advances in deep recommender systems
Recent advances in deep recommender systemsRecent advances in deep recommender systems
Recent advances in deep recommender systems
 
Recommenders Systems
Recommenders SystemsRecommenders Systems
Recommenders Systems
 
Scalable advertising recommender systems
Scalable advertising recommender systemsScalable advertising recommender systems
Scalable advertising recommender systems
 
Introduction to Recommendation System
Introduction to Recommendation SystemIntroduction to Recommendation System
Introduction to Recommendation System
 
Movie recommender system using the user's psychological profile
Movie recommender system using the user's psychological profileMovie recommender system using the user's psychological profile
Movie recommender system using the user's psychological profile
 
Movie lens recommender systems
Movie lens recommender systemsMovie lens recommender systems
Movie lens recommender systems
 
DIY ERM (Do-It-Yourself Electronic Resources Management) for the Small Library
DIY ERM (Do-It-Yourself Electronic Resources Management) for the Small LibraryDIY ERM (Do-It-Yourself Electronic Resources Management) for the Small Library
DIY ERM (Do-It-Yourself Electronic Resources Management) for the Small Library
 

Viewers also liked

Hypothesis-Based Collaborative Filtering
Hypothesis-Based Collaborative FilteringHypothesis-Based Collaborative Filtering
Hypothesis-Based Collaborative FilteringAmancio Bouza
 
Collaborative Filtering
Collaborative FilteringCollaborative Filtering
Collaborative FilteringTayfun Sen
 
Collaborative Filtering in Map/Reduce
Collaborative Filtering in Map/ReduceCollaborative Filtering in Map/Reduce
Collaborative Filtering in Map/ReduceOle-Martin Mørk
 
My talk about recommendation and search to the Hive
My talk about recommendation and search to the HiveMy talk about recommendation and search to the Hive
My talk about recommendation and search to the HiveTed Dunning
 
Collaborative filtering at scale
Collaborative filtering at scaleCollaborative filtering at scale
Collaborative filtering at scalehuguk
 
Tutorial 14 (collaborative filtering)
Tutorial 14 (collaborative filtering)Tutorial 14 (collaborative filtering)
Tutorial 14 (collaborative filtering)Kira
 
Machine Learning to Grow the World's Knowledge
Machine Learning to Grow  the World's KnowledgeMachine Learning to Grow  the World's Knowledge
Machine Learning to Grow the World's KnowledgeXavier Amatriain
 
China construction coating industry market demand forecast and investment str...
China construction coating industry market demand forecast and investment str...China construction coating industry market demand forecast and investment str...
China construction coating industry market demand forecast and investment str...Qianzhan Intelligence
 
International OnQ PM 200 - Groups Level Two
International OnQ PM 200 - Groups Level TwoInternational OnQ PM 200 - Groups Level Two
International OnQ PM 200 - Groups Level Twomhtar
 
MORE Vision 8: Crisis & Retail Consumption
MORE Vision 8: Crisis & Retail ConsumptionMORE Vision 8: Crisis & Retail Consumption
MORE Vision 8: Crisis & Retail ConsumptionMIPIMWorld
 
อุปกรณ์เครือข่ายคอมพิวเตอร์
อุปกรณ์เครือข่ายคอมพิวเตอร์อุปกรณ์เครือข่ายคอมพิวเตอร์
อุปกรณ์เครือข่ายคอมพิวเตอร์I'Tay Tanawin
 
Marketo At Enterprise Scale: How to Tame The Chaos and Maximize System Perfor...
Marketo At Enterprise Scale: How to Tame The Chaos and Maximize System Perfor...Marketo At Enterprise Scale: How to Tame The Chaos and Maximize System Perfor...
Marketo At Enterprise Scale: How to Tame The Chaos and Maximize System Perfor...DemandGen
 
Підручник
ПідручникПідручник
ПідручникKPG_KPG
 
China construction quality testing industry market forecast and competition s...
China construction quality testing industry market forecast and competition s...China construction quality testing industry market forecast and competition s...
China construction quality testing industry market forecast and competition s...Qianzhan Intelligence
 
Presentation23 (2)
Presentation23 (2)Presentation23 (2)
Presentation23 (2)MATC
 
อุปกรณ์พื้นฐานคอมพิวเตอร์
อุปกรณ์พื้นฐานคอมพิวเตอร์อุปกรณ์พื้นฐานคอมพิวเตอร์
อุปกรณ์พื้นฐานคอมพิวเตอร์I'Tay Tanawin
 
glue.things – a Mashup Platform for wiring the Internet of Things with the In...
glue.things – a Mashup Platform for wiring the Internet of Things with the In...glue.things – a Mashup Platform for wiring the Internet of Things with the In...
glue.things – a Mashup Platform for wiring the Internet of Things with the In...Robert Kleinfeld
 
China dredging engineering industry development prospect and investment strat...
China dredging engineering industry development prospect and investment strat...China dredging engineering industry development prospect and investment strat...
China dredging engineering industry development prospect and investment strat...Qianzhan Intelligence
 
China electric power construction industry market forecast and investment str...
China electric power construction industry market forecast and investment str...China electric power construction industry market forecast and investment str...
China electric power construction industry market forecast and investment str...Qianzhan Intelligence
 

Viewers also liked (19)

Hypothesis-Based Collaborative Filtering
Hypothesis-Based Collaborative FilteringHypothesis-Based Collaborative Filtering
Hypothesis-Based Collaborative Filtering
 
Collaborative Filtering
Collaborative FilteringCollaborative Filtering
Collaborative Filtering
 
Collaborative Filtering in Map/Reduce
Collaborative Filtering in Map/ReduceCollaborative Filtering in Map/Reduce
Collaborative Filtering in Map/Reduce
 
My talk about recommendation and search to the Hive
My talk about recommendation and search to the HiveMy talk about recommendation and search to the Hive
My talk about recommendation and search to the Hive
 
Collaborative filtering at scale
Collaborative filtering at scaleCollaborative filtering at scale
Collaborative filtering at scale
 
Tutorial 14 (collaborative filtering)
Tutorial 14 (collaborative filtering)Tutorial 14 (collaborative filtering)
Tutorial 14 (collaborative filtering)
 
Machine Learning to Grow the World's Knowledge
Machine Learning to Grow  the World's KnowledgeMachine Learning to Grow  the World's Knowledge
Machine Learning to Grow the World's Knowledge
 
China construction coating industry market demand forecast and investment str...
China construction coating industry market demand forecast and investment str...China construction coating industry market demand forecast and investment str...
China construction coating industry market demand forecast and investment str...
 
International OnQ PM 200 - Groups Level Two
International OnQ PM 200 - Groups Level TwoInternational OnQ PM 200 - Groups Level Two
International OnQ PM 200 - Groups Level Two
 
MORE Vision 8: Crisis & Retail Consumption
MORE Vision 8: Crisis & Retail ConsumptionMORE Vision 8: Crisis & Retail Consumption
MORE Vision 8: Crisis & Retail Consumption
 
อุปกรณ์เครือข่ายคอมพิวเตอร์
อุปกรณ์เครือข่ายคอมพิวเตอร์อุปกรณ์เครือข่ายคอมพิวเตอร์
อุปกรณ์เครือข่ายคอมพิวเตอร์
 
Marketo At Enterprise Scale: How to Tame The Chaos and Maximize System Perfor...
Marketo At Enterprise Scale: How to Tame The Chaos and Maximize System Perfor...Marketo At Enterprise Scale: How to Tame The Chaos and Maximize System Perfor...
Marketo At Enterprise Scale: How to Tame The Chaos and Maximize System Perfor...
 
Підручник
ПідручникПідручник
Підручник
 
China construction quality testing industry market forecast and competition s...
China construction quality testing industry market forecast and competition s...China construction quality testing industry market forecast and competition s...
China construction quality testing industry market forecast and competition s...
 
Presentation23 (2)
Presentation23 (2)Presentation23 (2)
Presentation23 (2)
 
อุปกรณ์พื้นฐานคอมพิวเตอร์
อุปกรณ์พื้นฐานคอมพิวเตอร์อุปกรณ์พื้นฐานคอมพิวเตอร์
อุปกรณ์พื้นฐานคอมพิวเตอร์
 
glue.things – a Mashup Platform for wiring the Internet of Things with the In...
glue.things – a Mashup Platform for wiring the Internet of Things with the In...glue.things – a Mashup Platform for wiring the Internet of Things with the In...
glue.things – a Mashup Platform for wiring the Internet of Things with the In...
 
China dredging engineering industry development prospect and investment strat...
China dredging engineering industry development prospect and investment strat...China dredging engineering industry development prospect and investment strat...
China dredging engineering industry development prospect and investment strat...
 
China electric power construction industry market forecast and investment str...
China electric power construction industry market forecast and investment str...China electric power construction industry market forecast and investment str...
China electric power construction industry market forecast and investment str...
 

Similar to Browsemap: Collaborative Filtering at LinkedIn

Disrupting Data Discovery
Disrupting Data DiscoveryDisrupting Data Discovery
Disrupting Data Discoverymarkgrover
 
Related searches at LinkedIn
Related searches at LinkedInRelated searches at LinkedIn
Related searches at LinkedInMitul Tiwari
 
Data council sf amundsen presentation
Data council sf    amundsen presentationData council sf    amundsen presentation
Data council sf amundsen presentationTao Feng
 
Large scale social recommender systems and their evaluation
Large scale social recommender systems and their evaluationLarge scale social recommender systems and their evaluation
Large scale social recommender systems and their evaluationMitul Tiwari
 
Strata sf - Amundsen presentation
Strata sf - Amundsen presentationStrata sf - Amundsen presentation
Strata sf - Amundsen presentationTao Feng
 
CROSSMINER Project at OW2con'19
CROSSMINER Project at OW2con'19CROSSMINER Project at OW2con'19
CROSSMINER Project at OW2con'19OW2
 
TLC2018 Thomas Haver: Transform with Enterprise Automation
TLC2018 Thomas Haver: Transform with Enterprise AutomationTLC2018 Thomas Haver: Transform with Enterprise Automation
TLC2018 Thomas Haver: Transform with Enterprise AutomationAnna Royzman
 
Relevancy and Search Quality Analysis - Search Technologies
Relevancy and Search Quality Analysis - Search TechnologiesRelevancy and Search Quality Analysis - Search Technologies
Relevancy and Search Quality Analysis - Search Technologiesenterprisesearchmeetup
 
Personalized Search-Building a prototype to infer the user's interest
Personalized Search-Building a prototype to infer the user's interestPersonalized Search-Building a prototype to infer the user's interest
Personalized Search-Building a prototype to infer the user's interestTom Burgmans
 
Employees, Business Partners and Bad Guys: What Web Data Reveals About Person...
Employees, Business Partners and Bad Guys: What Web Data Reveals About Person...Employees, Business Partners and Bad Guys: What Web Data Reveals About Person...
Employees, Business Partners and Bad Guys: What Web Data Reveals About Person...Connotate
 
VIVO Team Builder - VIVO conference 2014
VIVO Team Builder - VIVO conference 2014VIVO Team Builder - VIVO conference 2014
VIVO Team Builder - VIVO conference 2014Anup Sawant
 
How Lyft Drives Data Discovery
How Lyft Drives Data DiscoveryHow Lyft Drives Data Discovery
How Lyft Drives Data DiscoveryNeo4j
 
Identity Management: Tools, processes & services
Identity Management: Tools, processes & servicesIdentity Management: Tools, processes & services
Identity Management: Tools, processes & servicesJISC Netskills
 
Requirement Elicitation
Requirement ElicitationRequirement Elicitation
Requirement ElicitationRavikanth-BA
 
How Lyft Drives Data Discovery
How Lyft Drives Data DiscoveryHow Lyft Drives Data Discovery
How Lyft Drives Data DiscoveryNeo4j
 
Organizational Overlap on Social Networks and its Applications
Organizational Overlap on Social Networks and its ApplicationsOrganizational Overlap on Social Networks and its Applications
Organizational Overlap on Social Networks and its ApplicationsMitul Tiwari
 
Neo4j GraphTour Santa Monica 2019 - Amundsen Presentation
Neo4j GraphTour Santa Monica 2019 - Amundsen PresentationNeo4j GraphTour Santa Monica 2019 - Amundsen Presentation
Neo4j GraphTour Santa Monica 2019 - Amundsen PresentationTamikaTannis
 
KSU IT4983 Capstone Projects Report 2017 Update
KSU IT4983 Capstone Projects Report 2017 UpdateKSU IT4983 Capstone Projects Report 2017 Update
KSU IT4983 Capstone Projects Report 2017 UpdateJack Zheng
 
Fast & Cheap UX Research
Fast & Cheap UX ResearchFast & Cheap UX Research
Fast & Cheap UX ResearchJodi Bollaert
 

Similar to Browsemap: Collaborative Filtering at LinkedIn (20)

Disrupting Data Discovery
Disrupting Data DiscoveryDisrupting Data Discovery
Disrupting Data Discovery
 
Related searches at LinkedIn
Related searches at LinkedInRelated searches at LinkedIn
Related searches at LinkedIn
 
Data council sf amundsen presentation
Data council sf    amundsen presentationData council sf    amundsen presentation
Data council sf amundsen presentation
 
Large scale social recommender systems and their evaluation
Large scale social recommender systems and their evaluationLarge scale social recommender systems and their evaluation
Large scale social recommender systems and their evaluation
 
Strata sf - Amundsen presentation
Strata sf - Amundsen presentationStrata sf - Amundsen presentation
Strata sf - Amundsen presentation
 
CROSSMINER Project at OW2con'19
CROSSMINER Project at OW2con'19CROSSMINER Project at OW2con'19
CROSSMINER Project at OW2con'19
 
TLC2018 Thomas Haver: Transform with Enterprise Automation
TLC2018 Thomas Haver: Transform with Enterprise AutomationTLC2018 Thomas Haver: Transform with Enterprise Automation
TLC2018 Thomas Haver: Transform with Enterprise Automation
 
Meetup SF - Amundsen
Meetup SF  -  AmundsenMeetup SF  -  Amundsen
Meetup SF - Amundsen
 
Relevancy and Search Quality Analysis - Search Technologies
Relevancy and Search Quality Analysis - Search TechnologiesRelevancy and Search Quality Analysis - Search Technologies
Relevancy and Search Quality Analysis - Search Technologies
 
Personalized Search-Building a prototype to infer the user's interest
Personalized Search-Building a prototype to infer the user's interestPersonalized Search-Building a prototype to infer the user's interest
Personalized Search-Building a prototype to infer the user's interest
 
Employees, Business Partners and Bad Guys: What Web Data Reveals About Person...
Employees, Business Partners and Bad Guys: What Web Data Reveals About Person...Employees, Business Partners and Bad Guys: What Web Data Reveals About Person...
Employees, Business Partners and Bad Guys: What Web Data Reveals About Person...
 
VIVO Team Builder - VIVO conference 2014
VIVO Team Builder - VIVO conference 2014VIVO Team Builder - VIVO conference 2014
VIVO Team Builder - VIVO conference 2014
 
How Lyft Drives Data Discovery
How Lyft Drives Data DiscoveryHow Lyft Drives Data Discovery
How Lyft Drives Data Discovery
 
Identity Management: Tools, processes & services
Identity Management: Tools, processes & servicesIdentity Management: Tools, processes & services
Identity Management: Tools, processes & services
 
Requirement Elicitation
Requirement ElicitationRequirement Elicitation
Requirement Elicitation
 
How Lyft Drives Data Discovery
How Lyft Drives Data DiscoveryHow Lyft Drives Data Discovery
How Lyft Drives Data Discovery
 
Organizational Overlap on Social Networks and its Applications
Organizational Overlap on Social Networks and its ApplicationsOrganizational Overlap on Social Networks and its Applications
Organizational Overlap on Social Networks and its Applications
 
Neo4j GraphTour Santa Monica 2019 - Amundsen Presentation
Neo4j GraphTour Santa Monica 2019 - Amundsen PresentationNeo4j GraphTour Santa Monica 2019 - Amundsen Presentation
Neo4j GraphTour Santa Monica 2019 - Amundsen Presentation
 
KSU IT4983 Capstone Projects Report 2017 Update
KSU IT4983 Capstone Projects Report 2017 UpdateKSU IT4983 Capstone Projects Report 2017 Update
KSU IT4983 Capstone Projects Report 2017 Update
 
Fast & Cheap UX Research
Fast & Cheap UX ResearchFast & Cheap UX Research
Fast & Cheap UX Research
 

Recently uploaded

GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationMichael W. Hawkins
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Drew Madelung
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking MenDelhi Call girls
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfEnterprise Knowledge
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024Results
 
SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024Scott Keck-Warren
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
 
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 3652toLead Limited
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Miguel Araújo
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024The Digital Insurer
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘RTylerCroy
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...shyamraj55
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Allon Mureinik
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slidespraypatel2
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Igalia
 
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024BookNet Canada
 
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...gurkirankumar98700
 

Recently uploaded (20)

GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024
 
SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slides
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
 
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
 

Browsemap: Collaborative Filtering at LinkedIn

  • 1. Browsemap: Collaborative Filtering At LinkedIn Lili Wu, Sam Shah, Sean Choi, Mitul Tiwari, Christian Posse RSWeb 2014 with RecSys Recruiting Solutions 1
  • 2. 2 Agenda § Motivation § Architecture § Applications § Lessons Learned
  • 3. Collaborative filtering for member profile Profile Browsemap: People who viewed this profile also viewed… 3 Count co-views
  • 4. 4 Collaborative filtering for job page Job Browsemap: People who viewed this job also viewed… Count co-views
  • 5. 5 … many CF based recommenders group company portfolio
  • 6. • Many different entities • Similar problems with different requirement • Fast product development cycle • Hybrid recommender systems • Handle LinkedIn data volume and traffic 6 Challenges
  • 7. • Many different entities • Similar problems with different requirement • Fast product development cycle • Hybrid recommender systems • Handle LinkedIn data volume and traffic 7 Challenges è Horizontal Platform
  • 8. 8 Browsemap Collaborative Filtering Platform at LinkedIn
  • 9. 9 Browsemap Platform • Scalability Ø Online/offline architecture Ø Hundreds of millions of entities, billions of monthly page views • Browsemap Domain Specific Language (DSL) Ø Code reuse through modular components Ø Flexible computation workflow construction • Data are used by hybrid recommenders
  • 10. 10 Browsemap Architecture Frontend Services User Activity Data HDFS Queries Results Hadoop Browsemap DSL Browsemap Engine Online Query API Key-value storage Voldemort
  • 11. 11 Browsemap Architecture Frontend Services HDFS Queries Results Hadoop Browsemap DSL Browsemap Engine Online Query API Key-value storage Voldemort User Activity High Data Throughput
  • 12. 12 Browsemap Architecture Frontend Services HDFS Queries Results Hadoop Browsemap DSL Browsemap Engine Online Query API Key-value storage Voldemort User Activity Data Low Latency
  • 13. 13 Browsemap Domain Specific Language (DSL) Module Collection Expired Job Filtering Spam User Filtering Co-view counting Job Expired Job Filtering Cold-start techniques … Co-view counting … Cold-start techniques Job browsemap Company … Spam User Filtering Spam User Filtering … Co-view counting … Cold-start techniques Company browsemap
  • 14. • Support all entity types • Adjust to each product requirement • Scale 14 Recap Voldemort
  • 15. 15 Agenda ü Motivation ü Architecture § Applications § Lessons Learned
  • 16. 16 Applications – CF based recommenders Profile Browsemap Job Browsemap Group Browsemap Portfolio Browsemap Hiring Browsemap Company Browsemap Influencer Browsemap
  • 17. 17 Applications – Hybrid Recommender Systems Suggested Profile Update Swee Lim
  • 18. 18 Applications – Hybrid Recommender Systems Suggested Profile Update Goal: for each member, find companies he may want to follow
  • 19. 19 Applications – Hybrid Recommender Systems Member info: • Content-based features title, industry, location, … • Collaborative filtering feature Google Cisco Member followed companies Linkedin, Facebook Juniper, Arista Companies user may be interested in … … Co-follow Browsemaps: People who follow this company also follow these companies
  • 20. 20 Applications – Hybrid Recommender Systems Question: For a company C, will member M like it? Approach: Logistic regression Features: member location company location 1 if yes, 0 if no company is in the list of the co-follow browsemaps ? 1 if yes, 0 if no …
  • 21. 21 Applications – Hybrid Recommender Systems Collaborative Filtering is important: • Surface implicit connection between companies • Based on Member’s preference
  • 22. 22 Agenda ü Motivation ü Architecture ü Applications § Lessons Learned
  • 23. Lesson 1: Tall oaks grow from little acorns 23
  • 24. Lesson 1: Tall oaks grow from little acorns 24
  • 25. Lesson 1: Tall oaks grow from little acorns 25
  • 26. Lesson 1: Tall oaks grow from little acorns 26 A generic horizontal platform is essential
  • 27. Lesson 2: One hand washes the other 27 Job Browsemap Similar Jobs Collaborative filtering: “Follower audience” Content based: “Leader audience”
  • 28. Lesson 3: You can’t get blood out of a stone 28 Need to handle cold start problem Job 1 Job 2 Job 3 (new) (view time) merge Leverage Browsing History Personalized Backfill
  • 29. Lesson 4: A chain is only as strong as its weakest link 29 CF: Relies solely on user activities Good data is crucial § Mistakes can be hard to detect / debug § Simple mistakes can have big impact e.g. “jobid” à “id” § Need prevention mechanism Ø Improve tracking Ø Unit test Ø Browsemap platform data-check : input volume, coverage/metrics analysis
  • 30. Lesson 5: User experience matters 50% CTR 500% more applications 30 ª Put recommendations in user’s flow
  • 31. 31 Conclusion § Collaborative filtering is important for LinkedIn § Browsemap is in production for 3+ years § Horizontal platform is crucial
  • 32. 32 Thank you ! § Questions?