Your SlideShare is downloading. ×
0
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn

8,455

Published on

Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn …

Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Daniel Tunkelang (LinkedIn)

LinkedIn operates the world's largest professional network on the Internet with more than 100 million members in over 200 countries. In order to connect its users to the people, opportunities, and content
that best advance their careers, LinkedIn has developed a variety of algorithms that surface relevant content, offer personalized recommendations, and establish topic-sensitive reputation -- all at a
massive scale. In this talk, I will discuss some of the most challenging technical problems we face at LinkedIn, and the approaches we are taking to address them.

Note: This talk was presented at the Carnegie Mellon University School of Computer Science Intelligence Seminar on September 20, 2011. As of May 2013, LinkedIn has over 225 million members.

Published in: Technology, Business
1 Comment
16 Likes
Statistics
Notes
No Downloads
Views
Total Views
8,455
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
100
Comments
1
Likes
16
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • LinkedIn is all about connecting professionals with opportunity. We are taking the tools that made people successful in the 20th century and bringing them into the 21st century.
  • Transcript

    • 1. Daniel<br />Keeping It Professional:Relevance, Recommendations, and Reputation.<br />Daniel Tunkelang<br />Principal Data Scientist at LinkedIn <br />1<br />
    • 2. Overview<br /><ul><li>What is LinkedIn?
    • 3. Hard problems we’re tackling in:
    • 4. Relevance
    • 5. Recommendations
    • 6. Reputation
    • 7. Open problems</li></ul>2<br />
    • 8. Identity<br />Connect, find and be found<br />Rolodex, Resume, Business Card<br />LinkedIn Profile, Address Book, Search<br />Insights<br />Be great at what you do<br />Newspapers, <br />Trade Magazines, Events<br />Homepage, LinkedIn Today, Groups<br />Everywhere<br />Desktop<br />Work wherever our members work<br />Mobile, APIs, Plug-Ins<br />What is LinkedIn?<br />3<br />
    • 9. Identity: Profile of Record<br />4<br />
    • 10. Identity: Connect with Others<br />5<br />
    • 11. Identity: Join the Conversation<br />6<br />
    • 12. Insights: Power of Aggregation<br />Before<br />employees worked at<br />Yahoo! (169)<br /> Google (96)<br /> Oracle (78)<br /> Microsoft (72)<br /> IBM (43)<br />Before<br />employees worked at<br />Google(475)<br /> Microsoft (448)<br /> LinkedIn (169)<br /> Apple, Inc. (154)<br />ebay(133)<br />7<br />
    • 13. Insights: Market Research<br />8<br />
    • 14. Insights: Data Stories<br />9<br />
    • 15. Everywhere<br />10<br />
    • 16. Hard Problems: Examples<br /><ul><li>Relevance
    • 17. People Search
    • 18. Recommendations
    • 19. Job Matching
    • 20. Reputation
    • 21. Skills</li></ul>11<br />
    • 22. People Search<br />12<br />
    • 23. People Search: Scale<br /><ul><li>120M+ members
    • 24. 2B searches in 2010
    • 25. Based on (cf. http://sna-projects.com/)</li></ul>13<br />
    • 26. People Search: Faceted Search<br />14<br />
    • 27. People Search: Network Facet<br />15<br />
    • 28. People Search: Type-Ahead<br />16<br />
    • 29. People Search: Relevance<br /><ul><li>Query-Independent Signals
    • 30. Network Rank, Profile Quality
    • 31. Query-Dependent Signals
    • 32. Field-Based Relevance
    • 33. Personalized Signals
    • 34. Network Distance</li></ul>17<br />
    • 35. People Search: Query-Independent Signals<br />18<br />
    • 36. People Search: Network Rank<br />19<br />
    • 37. People Search: Profile Quality<br />20<br />
    • 38. People Search: SEO<br />21<br />
    • 39. People Search: Query-Dependent Signals<br />22<br />
    • 40. People Search: Inferring Structure<br />23<br />
    • 41. People Search: Ambiguity<br />vs.<br />vs.<br />24<br />
    • 42. People Search: HMM + Segmentation<br />for i in [1..n]<br /> s  w1 w2 … wi<br /> if Pc(s) > 0<br /> a  new Segment()<br />a.segs {s}<br />a.prob Pc(s)<br /> B[i]  {a}<br /> for j in [1..i-1]<br /> for b in B[j]<br /> s wjwj+1 … wi<br /> if Pc(s) > 0<br /> a  new Segment()<br />a.segsb.segs U {s}<br />a.probb.prob * Pc(s)<br /> B[i]  B[i] U {a}<br /> sort B[i] by prob<br /> truncate B[i] to size k<br />25<br />
    • 43. People Search: Personalized Signals<br />26<br />
    • 44. People Search: Further Reading<br /><ul><li>QCon 2010 presentation by John Wang on “LinkedIn Search: Searching the Social Graph in Real Time”http://www.infoq.com/presentations/LinkedIn-Search
    • 45. SIGIR 2011 Workshop on Entity-Oriented Searchhttp://research.microsoft.com/en-us/um/beijing/events/eos2011/
    • 46. HCIR 2011 paper by Jonathan Koren on “Faceted Search Query Log Analysis” (forthcoming)http://hcir.info/hcir-2011/</li></ul>27<br />
    • 47. Job Matching<br />28<br />
    • 48. Job Matching: Overview<br /><ul><li>Job Features
    • 49. Job Description, Location, Similar Jobs, …
    • 50. Candidate Features
    • 51. Profile Data, Network, Activity, …
    • 52. Standardization
    • 53. Companies, Job Titles, Education, …</li></ul>29<br />
    • 54. …<br />industry<br />description<br />functional area<br />Current Position<br />title<br />summary<br />tenure length<br />industry<br />functional area<br />…<br />Job Matching: Algorithm<br />Corpus Stats<br />Job<br />Matching<br />Transition probabilities<br />Connectivity<br />yrs of experience to reach title <br />education needed for this title<br />…<br />Binary<br /> Exact matches:<br /> geo, industry,<br /> …<br />Soft<br /> transition<br /> probabilities,<br /> similarity,<br /> … <br />Text<br />title<br />geo<br />company<br />User Base<br />Similarity <br />(candidate expertise, job description)<br />0.56<br />Filtered<br />Similarity <br />(candidate specialties, job description)<br />0.2<br />Candidate<br />Transition probability<br />(candidate industry, job industry)<br />0.43<br />General<br />expertise<br />specialties<br />education<br />headline<br />geo<br />experience<br />Title Similarity<br />0.8<br />Similarity (headline, title)<br />0.7<br />derived<br />.<br />.<br />.<br />30<br />
    • 55. Job Matching: Challenges<br /><ul><li>Most people aren't looking for jobs.
    • 56. Complicates evaluation, training.
    • 57. Important not to offend users.
    • 58. e.g., by offering Peter Norvig a postdoc.
    • 59. You can’t always get what you want
    • 60. Every employer wants the hottest candidates.</li></ul>31<br />
    • 61. Job Matching: Further Reading<br /><ul><li>KDD 2011 paper by Bekkerman & Gavish on “High-Precision Phrase-based Document Classification”http://www.stanford.edu/~gavish/documents/phrase_based.pdf
    • 62. SIGIR 2011 paper by Cetintas et al. on “Identifying Similar People in Professional Social Networks”http://dl.acm.org/citation.cfm?id=2010123
    • 63. Blog post on LinkedIn’s recommendation enginehttp://blog.linkedin.com/2011/03/02/linkedin-products-you-may-like/</li></ul>32<br />
    • 64. Skills<br />33<br />
    • 65. Skills: What are Skills?<br />34<br />
    • 66. Skills: Identifying Skills<br />35<br />
    • 67. Skills: Cluster and Disambiguate<br />angel<br />36<br />
    • 68. Skills: Assigning Skills to People<br />37<br />
    • 69. Skills: Who are the Experts?<br />38<br />
    • 70. Summary: The 3 Rs<br /><ul><li>Relevance
    • 71. Combine query-independent, query-dependent, and personalized features.
    • 72. Recommendations
    • 73. Match people to jobs, groups, news, …
    • 74. Reputation
    • 75. Expertise relative to professional skills.</li></ul>39<br />
    • 76. ¿<br />Open Problems<br />?<br />40<br />
    • 77. Exploratory Search<br />Exploratory Search<br />Investigate<br />Lookup<br />Learn<br />Accretion<br />Analysis<br />Exclusion/Negation<br />Synthesis<br />Evaluation<br />Discovery<br />Planning/Forecasting<br />Transformation<br />Fact retrieval<br />Known item search<br />Navigation<br />Transition<br />Verification<br />Question answering<br />Knowledge acquisition<br />Comprehension/Interpretation<br />Comparison<br />Aggregation/Integration<br />Socialize<br />41<br />
    • 78. Explore / Exploit<br />42<br />
    • 79. Incentives for Online Reputation<br />43<br />
    • 80. Thank You!<br />Questions?<br />Contact:<br />dtunkelang@linkedin.com<br />We’re Hiring!<br />http://engineering.linkedin.com/<br />44<br />

    ×