Personalizing Search at LinkedIn
•Download as PPTX, PDF•
2 likes•1,646 views
LinkedIn's vision is to create economic opportunity for the global workforce by connecting members to other members, knowledge, and opportunities through their economic graph. Their search functionality powers searching across people, jobs, companies, and schools, and aims to understand user intent to provide personalized and relevant results. They use a learning to rank approach trained on clickstream data to rank results based on inferred searcher interests and other features. Federated page construction combines results from different verticals and ranks them based on predicted click probabilities learned from past user behavior and intents.
Recommended
More Related Content
Viewers also liked
Similar to Personalizing Search at LinkedIn
Similar to Personalizing Search at LinkedIn (20)
Recently uploaded
Recently uploaded (14)
Personalizing Search at LinkedIn
- 1. Recruiting SolutionsRecruiting SolutionsRecruiting Solutions Ganesh Venkataraman Viet Ha-Thuc Personalizing Search @ LinkedIn
- 2. Bigger Picture ▪ LinkedIn’s vision – Create economic opportunity for every member of the global workforce ▪ Connect members to other members, knowledge and opportunity and help them be great at what they do
- 3. Economic Graph ▪ Organize people, companies, jobs, knowledge and map out the economic graph 3
- 4. Role of Search ▪ At the heart of the economic graph, search makes the economic graph accessible, useful and actionable ▪ Powers searching people, jobs, companies, schools etc. ▪ On linkedin.com consumer, recruiter, sales solutions 4
- 6. Basic Nomenclature 6 TypeAhead/TYAH Full Search/SERP
- 8. 8
- 9. Search is about understanding the user intent 9
- 10. LinkedIn Search - An Overview 10 Query Processing Retrieval Ranking Federated Page Construction Search Assist ● Instant Results ● Guided suggestions ● Autocomplete suggestions Entity View/Action
- 11. Let’s talk intent - Navigational ▪ Navigational - exactly one result in mind 11
- 12. Two types of Intent - Exploratory ▪ Exploratory - Typically more than one entity in mind 12
- 13. How to handle navigational queries? Be Fast Type Less Be Lenient 13
- 14. Handling Navigational Queries ▪ Type Less – Index prefixes (‘ga’, ‘gan’, ‘gane’ => ‘ganesh’) ▪ Be Fast – Do not retrieve all documents – Order documents in posting list by static rank – Modify query for targeted retrieval ▪ Be Lenient – Smart spell correction 14
- 15. Exploratory Queries ▪ If possible guide users to more structured queries ▪ Above query could go into different verticals if these are selected ▪ User intent becomes much clearer 15
- 17. Unclear intent - Federating TYAH results 17
- 18. LinkedIn Search - Bird’s eye view 18 Query Processing Retrieval Ranking Federated Page Construction Search Assist ● Instant Results ● Guided suggestions ● Autocomplete suggestions Entity View/Action
- 19. Query Processing - things not strings 1919 TITLE CO GEO TITLE-237 software engineer software developer programmer … CO-1441 Google Inc. Industry: Internet GEO-7583 Country: US Lat: 42.3482 N Long: 75.1890 W (RECOGNIZED TAGS: NAME, TITLE, COMPANY, SCHOOL, GEO, SKILL )
- 20. Retrieval ▪ Custom search engine to handle 100’s of millions of documents (Galene) ▪ Key Features: – Offline indexing pipeline – Supports live updates with fine granularity – Static Ranking ▪ Posting list organized by static rank for each document ▪ Enables early termination 20
- 21. LinkedIn Search - Bird’s eye view 21 Query Processing Retrieval Ranking Federated Page Construction Search Assist ● Instant Results ● Guided suggestions ● Autocomplete suggestions Entity View/Action
- 22. Ranking ▪ Manually tuning vs. Learning to Rank (LTR) ▪ Why Learning to Rank? – Hard to manually tune with very large number of features – Challenging to personalize – LTR allows leveraging large volume of click data in an automated way 22
- 23. Training Data: Human Label
- 24. What if the searcher is a job seeker? Or a recruiter? Training Data: Human Label
- 25. ▪ Relevance depends on who’s searching ▪ Difficult to scale Training Data: Human Label
- 26. Training Data: Click Stream Approach: Clicked = Relevant, Skipped = Not Relevant User eye scan direction Unfair penalized
- 27. Training Data: Click Stream Approach: Graded relevance Uncertain (middle level) Non-relevant Relevant
- 28. Feature Overview ▪ Textual features ▪ Social features ▪ Homophily features – Geo – Industry ▪ Inferred Searcher Interests ▪ etc.
- 29. Inferred Searcher Interests Interests * Locations * Industry ...
- 30. Learning Algorithm ▪ Coordinate Ascent Algorithm – Listwise approach ▪ Objective function: Normalized Discounted Cumulative Gain (NDCG) – Defined on graded relevance – Intuition: more useful to show more-relevant documents at higher positions
- 31. LinkedIn Search - Bird’s eye view 31 Query Processing Retrieval Ranking Federated Page Construction Search Assist ● Instant Results ● Guided suggestions ● Autocomplete suggestions Entity View/Action
- 33. ▪ Why do we need this? – Not to overwhelm the user with too much information –Make results personally relevant 33 Motivation
- 34. ▪ Challenges –Query can be ambiguous –Incomparability across vertical objects ▪Compare objects of different nature: individual job vs. people cluster ▪Objects associate with different signals 34 Motivation
- 36. Learning Federation Model ▪ Predicts: p(click| individual result OR vertical cluster, query, searcher) ▪ Training data: click logs ▪ Features –Relevance scores from base rankers –Searcher intent –Query intent –etc.
- 37. Features ▪ Searcher Intents – Mine searcher profiles and past behavior to infer intent ▪ Title recruiter -> recruiting intent ▪ Search for jobs -> job seeking intent – Machine-learned models predict member intents: ▪Job seeking ▪Recruiting ▪Content consuming 37
- 38. Features ▪ Query Intents: e.g. p(job vertical| “software engineer”) –Mine from historical searches and actions 38
- 39. Features ▪ Query Intents: e.g. p(job vertical| “software engineer”) –Mine from historical searches and actions ▪ Personalized Query Intents –p(job vertical| “software engineer”, searcher) 39
- 40. Features ▪ Query Intents: e.g. p(job vertical| “software engineer”) –Mine from historical searches and actions ▪ Personalized Query Intents –p(job vertical| “software engineer”, searcher) –Individual searcher → searcher group ▪p(job vertical| “software engineer”, job seeking searcher) 40
- 41. Calibrate Signals across Verticals ▪ Relevance scores from vertical rankers are incomparable 41
- 42. Calibrate Signals across Verticals ▪ Relevance scores from vertical rankers are incomparable ▪ Construct composite features People relevance score of searcher if result is People f 1= ⎨0, otherwise 42
- 43. Calibrate Signals across Verticals ▪ Verticals associate with different signals 43 People Result Job Result Group Result Recruiting Intent Job Seeking Intent Content Consuming Intent
- 44. Calibrate Signals across Verticals ▪ Verticals associate with different signals 44 People Result Job Result Group Result Recruiting Intent Job Seeking Intent Content Consuming Intent
- 45. Calibrate Signals across Verticals ▪ Verticals associate with different signals 45 People Result Job Result Group Result Recruiting Intent Job Seeking Intent Content Consuming Intent
- 46. Conclusions ▪ Search personalization is at the core of our economic graph vision –Connect talent with opportunity at massive scale ▪ Click data is useful sources for personalized training data –Need to correct position bias ▪ Personalized features are keys ▪ Create composite features to calibrate across verticals