Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn


Published on

Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Daniel Tunkelang (LinkedIn)

LinkedIn operates the world's largest professional network on the Internet with more than 100 million members in over 200 countries. In order to connect its users to the people, opportunities, and content
that best advance their careers, LinkedIn has developed a variety of algorithms that surface relevant content, offer personalized recommendations, and establish topic-sensitive reputation -- all at a
massive scale. In this talk, I will discuss some of the most challenging technical problems we face at LinkedIn, and the approaches we are taking to address them.

Note: This talk was presented at the Carnegie Mellon University School of Computer Science Intelligence Seminar on September 20, 2011. As of May 2013, LinkedIn has over 225 million members.

Published in: Technology, Business
1 Comment
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • LinkedIn is all about connecting professionals with opportunity. We are taking the tools that made people successful in the 20th century and bringing them into the 21st century.
  • Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn

    1. Daniel<br />Keeping It Professional:Relevance, Recommendations, and Reputation.<br />Daniel Tunkelang<br />Principal Data Scientist at LinkedIn <br />1<br />
    2. Overview<br /><ul><li>What is LinkedIn?
    3. Hard problems we’re tackling in:
    4. Relevance
    5. Recommendations
    6. Reputation
    7. Open problems</li></ul>2<br />
    8. Identity<br />Connect, find and be found<br />Rolodex, Resume, Business Card<br />LinkedIn Profile, Address Book, Search<br />Insights<br />Be great at what you do<br />Newspapers, <br />Trade Magazines, Events<br />Homepage, LinkedIn Today, Groups<br />Everywhere<br />Desktop<br />Work wherever our members work<br />Mobile, APIs, Plug-Ins<br />What is LinkedIn?<br />3<br />
    9. Identity: Profile of Record<br />4<br />
    10. Identity: Connect with Others<br />5<br />
    11. Identity: Join the Conversation<br />6<br />
    12. Insights: Power of Aggregation<br />Before<br />employees worked at<br />Yahoo! (169)<br /> Google (96)<br /> Oracle (78)<br /> Microsoft (72)<br /> IBM (43)<br />Before<br />employees worked at<br />Google(475)<br /> Microsoft (448)<br /> LinkedIn (169)<br /> Apple, Inc. (154)<br />ebay(133)<br />7<br />
    13. Insights: Market Research<br />8<br />
    14. Insights: Data Stories<br />9<br />
    15. Everywhere<br />10<br />
    16. Hard Problems: Examples<br /><ul><li>Relevance
    17. People Search
    18. Recommendations
    19. Job Matching
    20. Reputation
    21. Skills</li></ul>11<br />
    22. People Search<br />12<br />
    23. People Search: Scale<br /><ul><li>120M+ members
    24. 2B searches in 2010
    25. Based on (cf.</li></ul>13<br />
    26. People Search: Faceted Search<br />14<br />
    27. People Search: Network Facet<br />15<br />
    28. People Search: Type-Ahead<br />16<br />
    29. People Search: Relevance<br /><ul><li>Query-Independent Signals
    30. Network Rank, Profile Quality
    31. Query-Dependent Signals
    32. Field-Based Relevance
    33. Personalized Signals
    34. Network Distance</li></ul>17<br />
    35. People Search: Query-Independent Signals<br />18<br />
    36. People Search: Network Rank<br />19<br />
    37. People Search: Profile Quality<br />20<br />
    38. People Search: SEO<br />21<br />
    39. People Search: Query-Dependent Signals<br />22<br />
    40. People Search: Inferring Structure<br />23<br />
    41. People Search: Ambiguity<br />vs.<br />vs.<br />24<br />
    42. People Search: HMM + Segmentation<br />for i in [1..n]<br /> s  w1 w2 … wi<br /> if Pc(s) > 0<br /> a  new Segment()<br />a.segs {s}<br />a.prob Pc(s)<br /> B[i]  {a}<br /> for j in [1..i-1]<br /> for b in B[j]<br /> s wjwj+1 … wi<br /> if Pc(s) > 0<br /> a  new Segment()<br />a.segsb.segs U {s}<br />a.probb.prob * Pc(s)<br /> B[i]  B[i] U {a}<br /> sort B[i] by prob<br /> truncate B[i] to size k<br />25<br />
    43. People Search: Personalized Signals<br />26<br />
    44. People Search: Further Reading<br /><ul><li>QCon 2010 presentation by John Wang on “LinkedIn Search: Searching the Social Graph in Real Time”
    45. SIGIR 2011 Workshop on Entity-Oriented Search
    46. HCIR 2011 paper by Jonathan Koren on “Faceted Search Query Log Analysis” (forthcoming)</li></ul>27<br />
    47. Job Matching<br />28<br />
    48. Job Matching: Overview<br /><ul><li>Job Features
    49. Job Description, Location, Similar Jobs,
    50. Candidate Features
    51. Profile Data, Network, Activity, …
    52. Standardization
    53. Companies, Job Titles, Education, …</li></ul>29<br />
    54. …<br />industry<br />description<br />functional area<br />Current Position<br />title<br />summary<br />tenure length<br />industry<br />functional area<br />…<br />Job Matching: Algorithm<br />Corpus Stats<br />Job<br />Matching<br />Transition probabilities<br />Connectivity<br />yrs of experience to reach title <br />education needed for this title<br />…<br />Binary<br /> Exact matches:<br /> geo, industry,<br /> …<br />Soft<br /> transition<br /> probabilities,<br /> similarity,<br /> … <br />Text<br />title<br />geo<br />company<br />User Base<br />Similarity <br />(candidate expertise, job description)<br />0.56<br />Filtered<br />Similarity <br />(candidate specialties, job description)<br />0.2<br />Candidate<br />Transition probability<br />(candidate industry, job industry)<br />0.43<br />General<br />expertise<br />specialties<br />education<br />headline<br />geo<br />experience<br />Title Similarity<br />0.8<br />Similarity (headline, title)<br />0.7<br />derived<br />.<br />.<br />.<br />30<br />
    55. Job Matching: Challenges<br /><ul><li>Most people aren't looking for jobs.
    56. Complicates evaluation, training.
    57. Important not to offend users.
    58. e.g., by offering Peter Norvig a postdoc.
    59. You can’t always get what you want
    60. Every employer wants the hottest candidates.</li></ul>31<br />
    61. Job Matching: Further Reading<br /><ul><li>KDD 2011 paper by Bekkerman & Gavish on “High-Precision Phrase-based Document Classification”
    62. SIGIR 2011 paper by Cetintas et al. on “Identifying Similar People in Professional Social Networks”
    63. Blog post on LinkedIn’s recommendation engine</li></ul>32<br />
    64. Skills<br />33<br />
    65. Skills: What are Skills?<br />34<br />
    66. Skills: Identifying Skills<br />35<br />
    67. Skills: Cluster and Disambiguate<br />angel<br />36<br />
    68. Skills: Assigning Skills to People<br />37<br />
    69. Skills: Who are the Experts?<br />38<br />
    70. Summary: The 3 Rs<br /><ul><li>Relevance
    71. Combine query-independent, query-dependent, and personalized features.
    72. Recommendations
    73. Match people to jobs, groups, news, …
    74. Reputation
    75. Expertise relative to professional skills.</li></ul>39<br />
    76. ¿<br />Open Problems<br />?<br />40<br />
    77. Exploratory Search<br />Exploratory Search<br />Investigate<br />Lookup<br />Learn<br />Accretion<br />Analysis<br />Exclusion/Negation<br />Synthesis<br />Evaluation<br />Discovery<br />Planning/Forecasting<br />Transformation<br />Fact retrieval<br />Known item search<br />Navigation<br />Transition<br />Verification<br />Question answering<br />Knowledge acquisition<br />Comprehension/Interpretation<br />Comparison<br />Aggregation/Integration<br />Socialize<br />41<br />
    78. Explore / Exploit<br />42<br />
    79. Incentives for Online Reputation<br />43<br />
    80. Thank You!<br />Questions?<br />Contact:<br /><br />We’re Hiring!<br /><br />44<br />