Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Recruiting SolutionsRecruiting SolutionsRecruiting Solutions
Recommending Jobs You May Be Interested In
Anuj Goyal
Recomme...
§  About LinkedIn
§  Search vs. Recommendations
§  Recommendation Opportunities
§  Evaluating Job Recommendations
§  ...
270+ M
Company Pages
>3M
*
Professional searches in 2012
~5.7B
90%Fortune 100 Companies
use LinkedIn to hire
*
*as of Marc...
4
Recommendation Versus Search
Explicit
Implicit
The Recommendations Opportunity
5
6
50%7
Value of Recommendations
Job Recommendations
8
9
Jobs You May be Interested In
Evaluation
§  Upside metrics
–  Are users getting relevant jobs?
§  Downside metrics
–  Are users getting offending jobs...
Evaluation - Upside
§  Total Job Views (Clicks)
§  Total Applications
§  Total Viewers
§  Total Applicants
11
Evaluation - Downside
§  Applications per Click
§  Clicks per Impression
§  Applications per Impression
§  Expert Judg...
User Features & Recommendation Algorithm
14
Positions
Education
Summary
Experience
Skills
User Features
Corpus StatsCandidate Jobs
User Base
title
geo
company
industry
description
functional area
…
Candidate
General
expertise
...
Challenges & Feature Engineering
Challenge: Entity Resolution
17
‘IBM’ has 13000+ variations
-  ibm – ireland
-  ibm research
-  T J Watson Labs
-  International Bus. Machines
Are All Com...
-  Software Engineer
-  Technical Yahoo
-  Member Technical Staff
-  Software Development Engineer
-  SDE
Are All Titles T...
Same company with
different name
Same name but
different companies
Name Variations for IBM?
“Orion” refers to 20 diff. com...
§  Binary classifier (LR), not
ranker
§  P({position, company
entity} is a match)
§  Features
§  Content
§  Social
§...
Challenges – Geo Location
22
§  Zip code mapped to Regions
§  How sticky are those locations?
Feature Engineering – Sticky locations
23
§  Open to relocation ?
§  Region similarity based on profiles or network
§  Region transition probability
§  Predict ...
Feature Engineering – The Network effect
25
Hybrid Recommendation
Title : Research Engineer
Company : Yahoo!
Location : CA,USA
Skills : Stats, ML, Java
Title : Data S...
Information Gain
Pick Top K overrepresented features from the
applicants distribution
A representative projection of the j...
§  Why Jobs Recommendations are Different
§  Recommendation Algorithm
§  Challenges
–  Entity Resolution
–  Location Re...
Questions?
Contact:
agoyal@linkedin.com
We’re Hiring!
http://data.linkedin.com/
Thank You!
29
Upcoming SlideShare
Loading in …5
×

Big data innovation_summit_2014

The talk I gave at the Big Data Innovation Summit.

  • Be the first to comment

Big data innovation_summit_2014

  1. 1. Recruiting SolutionsRecruiting SolutionsRecruiting Solutions Recommending Jobs You May Be Interested In Anuj Goyal Recommendations at LinkedIn Anuj
  2. 2. §  About LinkedIn §  Search vs. Recommendations §  Recommendation Opportunities §  Evaluating Job Recommendations §  Job Recommendation Algorithm §  Challenges §  Summary 2 Overview
  3. 3. 270+ M Company Pages >3M * Professional searches in 2012 ~5.7B 90%Fortune 100 Companies use LinkedIn to hire * *as of March 31, 2014 New Members joining ~2/sec 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 3 World’s largest professional network Over 65% of members are now international
  4. 4. 4 Recommendation Versus Search Explicit Implicit
  5. 5. The Recommendations Opportunity 5
  6. 6. 6
  7. 7. 50%7 Value of Recommendations
  8. 8. Job Recommendations 8
  9. 9. 9 Jobs You May be Interested In
  10. 10. Evaluation §  Upside metrics –  Are users getting relevant jobs? §  Downside metrics –  Are users getting offending jobs? 10
  11. 11. Evaluation - Upside §  Total Job Views (Clicks) §  Total Applications §  Total Viewers §  Total Applicants 11
  12. 12. Evaluation - Downside §  Applications per Click §  Clicks per Impression §  Applications per Impression §  Expert Judgments 12
  13. 13. User Features & Recommendation Algorithm
  14. 14. 14 Positions Education Summary Experience Skills User Features
  15. 15. Corpus StatsCandidate Jobs User Base title geo company industry description functional area … Candidate General expertise specialties education headline geo experience Current Position title summary tenure length industry functional area … Similarity (candidate expertise, job description) 0.56 Similarity (candidate specialties, job description) 0.2 Transition probability (candidate industry, job industry) 0.43 Title Similarity 0.8 Similarity (headline, title) 0.7 . . . derived Matching Binary Exact matches: geo, industry, … Soft transition probabilities, similarity, … Text Recommendation Algorithm Transition probabilities Connectivity yrs of experience to reach title education needed for this title … 15 Job Collection
  16. 16. Challenges & Feature Engineering
  17. 17. Challenge: Entity Resolution 17
  18. 18. ‘IBM’ has 13000+ variations -  ibm – ireland -  ibm research -  T J Watson Labs -  International Bus. Machines Are All Companies The Same? 18
  19. 19. -  Software Engineer -  Technical Yahoo -  Member Technical Staff -  Software Development Engineer -  SDE Are All Titles The Same? 19
  20. 20. Same company with different name Same name but different companies Name Variations for IBM? “Orion” refers to 20 diff. companies large scale: 100M+ members, 2M+ company entities IBM: Intl Brotherhood of Magicians ~ 13000 Challenges – Entity Resolution 20
  21. 21. §  Binary classifier (LR), not ranker §  P({position, company entity} is a match) §  Features §  Content §  Social §  Behavior §  Company candidate set leveraged from Social graph and cosine similarity 97% Precision at 50% Coverage Asonam’11, KDD’11 Challenges – Entity Resolution 21 Precision Coverage
  22. 22. Challenges – Geo Location 22
  23. 23. §  Zip code mapped to Regions §  How sticky are those locations? Feature Engineering – Sticky locations 23
  24. 24. §  Open to relocation ? §  Region similarity based on profiles or network §  Region transition probability §  Predict individuals propensity to migrate and most likely migration target Feature Engineering – Sticky locations 24
  25. 25. Feature Engineering – The Network effect 25
  26. 26. Hybrid Recommendation Title : Research Engineer Company : Yahoo! Location : CA,USA Skills : Stats, ML, Java Title : Data Scientist Company : Samsung Location : PA,USA Skills : Stats, R Title : Analyst Company : Microsoft Location : CA, USA Skills : R, ML Title : Research Engineer <1>, Data Scientist <1>, Analyst <1> Company : Yahoo<1>, Samsung<1>, Microsoft<1> Location : CA,USA <2>, PA,USA<1> Skills : Stats<2>, ML<2>, R<2>, Java<1> Applicant Features Distribution Data Scientist / Senior Data Scientist San Jose 26
  27. 27. Information Gain Pick Top K overrepresented features from the applicants distribution A representative projection of the job in the member feature space 27 Hybrid Recommendation
  28. 28. §  Why Jobs Recommendations are Different §  Recommendation Algorithm §  Challenges –  Entity Resolution –  Location Resolution 28 Summary
  29. 29. Questions? Contact: agoyal@linkedin.com We’re Hiring! http://data.linkedin.com/ Thank You! 29

×