Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Personalizing Search at LinkedIn

1,070 views

Published on

Bay Area search meetup (June 2015)

Published in: Social Media
  • Be the first to comment

Personalizing Search at LinkedIn

  1. 1. Recruiting SolutionsRecruiting SolutionsRecruiting Solutions Ganesh Venkataraman Viet Ha-Thuc Personalizing Search @ LinkedIn
  2. 2. Bigger Picture ▪ LinkedIn’s vision – Create economic opportunity for every member of the global workforce ▪ Connect members to other members, knowledge and opportunity and help them be great at what they do
  3. 3. Economic Graph ▪ Organize people, companies, jobs, knowledge and map out the economic graph 3
  4. 4. Role of Search ▪ At the heart of the economic graph, search makes the economic graph accessible, useful and actionable ▪ Powers searching people, jobs, companies, schools etc. ▪ On linkedin.com consumer, recruiter, sales solutions 4
  5. 5. Powered by Search 5
  6. 6. Basic Nomenclature 6 TypeAhead/TYAH Full Search/SERP
  7. 7. Search is ... 7
  8. 8. 8
  9. 9. Search is about understanding the user intent 9
  10. 10. LinkedIn Search - An Overview 10 Query Processing Retrieval Ranking Federated Page Construction Search Assist ● Instant Results ● Guided suggestions ● Autocomplete suggestions Entity View/Action
  11. 11. Let’s talk intent - Navigational ▪ Navigational - exactly one result in mind 11
  12. 12. Two types of Intent - Exploratory ▪ Exploratory - Typically more than one entity in mind 12
  13. 13. How to handle navigational queries? Be Fast Type Less Be Lenient 13
  14. 14. Handling Navigational Queries ▪ Type Less – Index prefixes (‘ga’, ‘gan’, ‘gane’ => ‘ganesh’) ▪ Be Fast – Do not retrieve all documents – Order documents in posting list by static rank – Modify query for targeted retrieval ▪ Be Lenient – Smart spell correction 14
  15. 15. Exploratory Queries ▪ If possible guide users to more structured queries ▪ Above query could go into different verticals if these are selected ▪ User intent becomes much clearer 15
  16. 16. Exploratory Queries 16
  17. 17. Unclear intent - Federating TYAH results 17
  18. 18. LinkedIn Search - Bird’s eye view 18 Query Processing Retrieval Ranking Federated Page Construction Search Assist ● Instant Results ● Guided suggestions ● Autocomplete suggestions Entity View/Action
  19. 19. Query Processing - things not strings 1919 TITLE CO GEO TITLE-237 software engineer software developer programmer … CO-1441 Google Inc. Industry: Internet GEO-7583 Country: US Lat: 42.3482 N Long: 75.1890 W (RECOGNIZED TAGS: NAME, TITLE, COMPANY, SCHOOL, GEO, SKILL )
  20. 20. Retrieval ▪ Custom search engine to handle 100’s of millions of documents (Galene) ▪ Key Features: – Offline indexing pipeline – Supports live updates with fine granularity – Static Ranking ▪ Posting list organized by static rank for each document ▪ Enables early termination 20
  21. 21. LinkedIn Search - Bird’s eye view 21 Query Processing Retrieval Ranking Federated Page Construction Search Assist ● Instant Results ● Guided suggestions ● Autocomplete suggestions Entity View/Action
  22. 22. Ranking ▪ Manually tuning vs. Learning to Rank (LTR) ▪ Why Learning to Rank? – Hard to manually tune with very large number of features – Challenging to personalize – LTR allows leveraging large volume of click data in an automated way 22
  23. 23. Training Data: Human Label
  24. 24. What if the searcher is a job seeker? Or a recruiter? Training Data: Human Label
  25. 25. ▪ Relevance depends on who’s searching ▪ Difficult to scale Training Data: Human Label
  26. 26. Training Data: Click Stream Approach: Clicked = Relevant, Skipped = Not Relevant User eye scan direction Unfair penalized
  27. 27. Training Data: Click Stream Approach: Graded relevance Uncertain (middle level) Non-relevant Relevant
  28. 28. Feature Overview ▪ Textual features ▪ Social features ▪ Homophily features – Geo – Industry ▪ Inferred Searcher Interests ▪ etc.
  29. 29. Inferred Searcher Interests Interests * Locations * Industry ...
  30. 30. Learning Algorithm ▪ Coordinate Ascent Algorithm – Listwise approach ▪ Objective function: Normalized Discounted Cumulative Gain (NDCG) – Defined on graded relevance – Intuition: more useful to show more-relevant documents at higher positions
  31. 31. LinkedIn Search - Bird’s eye view 31 Query Processing Retrieval Ranking Federated Page Construction Search Assist ● Instant Results ● Guided suggestions ● Autocomplete suggestions Entity View/Action
  32. 32. 32 Federated Search Page
  33. 33. ▪ Why do we need this? – Not to overwhelm the user with too much information –Make results personally relevant 33 Motivation
  34. 34. ▪ Challenges –Query can be ambiguous –Incomparability across vertical objects ▪Compare objects of different nature: individual job vs. people cluster ▪Objects associate with different signals 34 Motivation
  35. 35. 35 Overall Approach
  36. 36. Learning Federation Model ▪ Predicts: p(click| individual result OR vertical cluster, query, searcher) ▪ Training data: click logs ▪ Features –Relevance scores from base rankers –Searcher intent –Query intent –etc.
  37. 37. Features ▪ Searcher Intents – Mine searcher profiles and past behavior to infer intent ▪ Title recruiter -> recruiting intent ▪ Search for jobs -> job seeking intent – Machine-learned models predict member intents: ▪Job seeking ▪Recruiting ▪Content consuming 37
  38. 38. Features ▪ Query Intents: e.g. p(job vertical| “software engineer”) –Mine from historical searches and actions 38
  39. 39. Features ▪ Query Intents: e.g. p(job vertical| “software engineer”) –Mine from historical searches and actions ▪ Personalized Query Intents –p(job vertical| “software engineer”, searcher) 39
  40. 40. Features ▪ Query Intents: e.g. p(job vertical| “software engineer”) –Mine from historical searches and actions ▪ Personalized Query Intents –p(job vertical| “software engineer”, searcher) –Individual searcher → searcher group ▪p(job vertical| “software engineer”, job seeking searcher) 40
  41. 41. Calibrate Signals across Verticals ▪ Relevance scores from vertical rankers are incomparable 41
  42. 42. Calibrate Signals across Verticals ▪ Relevance scores from vertical rankers are incomparable ▪ Construct composite features People relevance score of searcher if result is People f 1= ⎨0, otherwise 42
  43. 43. Calibrate Signals across Verticals ▪ Verticals associate with different signals 43 People Result Job Result Group Result Recruiting Intent Job Seeking Intent Content Consuming Intent
  44. 44. Calibrate Signals across Verticals ▪ Verticals associate with different signals 44 People Result Job Result Group Result Recruiting Intent Job Seeking Intent Content Consuming Intent
  45. 45. Calibrate Signals across Verticals ▪ Verticals associate with different signals 45 People Result Job Result Group Result Recruiting Intent Job Seeking Intent Content Consuming Intent
  46. 46. Conclusions ▪ Search personalization is at the core of our economic graph vision –Connect talent with opportunity at massive scale ▪ Click data is useful sources for personalized training data –Need to correct position bias ▪ Personalized features are keys ▪ Create composite features to calibrate across verticals
  47. 47. 47 We are hiring!

×