Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn

  • 8,266 views
Uploaded on

Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn …

Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn
Daniel Tunkelang (LinkedIn)

LinkedIn operates the world's largest professional network on the Internet with more than 100 million members in over 200 countries. In order to connect its users to the people, opportunities, and content
that best advance their careers, LinkedIn has developed a variety of algorithms that surface relevant content, offer personalized recommendations, and establish topic-sensitive reputation -- all at a
massive scale. In this talk, I will discuss some of the most challenging technical problems we face at LinkedIn, and the approaches we are taking to address them.

Note: This talk was presented at the Carnegie Mellon University School of Computer Science Intelligence Seminar on September 20, 2011. As of May 2013, LinkedIn has over 225 million members.

More in: Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
8,266
On Slideshare
0
From Embeds
0
Number of Embeds
3

Actions

Shares
Downloads
96
Comments
1
Likes
16

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • LinkedIn is all about connecting professionals with opportunity. We are taking the tools that made people successful in the 20th century and bringing them into the 21st century.

Transcript

  • 1. Daniel
    Keeping It Professional:Relevance, Recommendations, and Reputation.
    Daniel Tunkelang
    Principal Data Scientist at LinkedIn
    1
  • 2. Overview
    • What is LinkedIn?
    • 3. Hard problems we’re tackling in:
    • 4. Relevance
    • 5. Recommendations
    • 6. Reputation
    • 7. Open problems
    2
  • 8. Identity
    Connect, find and be found
    Rolodex, Resume, Business Card
    LinkedIn Profile, Address Book, Search
    Insights
    Be great at what you do
    Newspapers,
    Trade Magazines, Events
    Homepage, LinkedIn Today, Groups
    Everywhere
    Desktop
    Work wherever our members work
    Mobile, APIs, Plug-Ins
    What is LinkedIn?
    3
  • 9. Identity: Profile of Record
    4
  • 10. Identity: Connect with Others
    5
  • 11. Identity: Join the Conversation
    6
  • 12. Insights: Power of Aggregation
    Before
    employees worked at
    Yahoo! (169)
    Google (96)
    Oracle (78)
    Microsoft (72)
    IBM (43)
    Before
    employees worked at
    Google(475)
    Microsoft (448)
    LinkedIn (169)
    Apple, Inc. (154)
    ebay(133)
    7
  • 13. Insights: Market Research
    8
  • 14. Insights: Data Stories
    9
  • 15. Everywhere
    10
  • 16. Hard Problems: Examples
    11
  • 22. People Search
    12
  • 23. People Search: Scale
    • 120M+ members
    • 24. 2B searches in 2010
    • 25. Based on (cf. http://sna-projects.com/)
    13
  • 26. People Search: Faceted Search
    14
  • 27. People Search: Network Facet
    15
  • 28. People Search: Type-Ahead
    16
  • 29. People Search: Relevance
    • Query-Independent Signals
    • 30. Network Rank, Profile Quality
    • 31. Query-Dependent Signals
    • 32. Field-Based Relevance
    • 33. Personalized Signals
    • 34. Network Distance
    17
  • 35. People Search: Query-Independent Signals
    18
  • 36. People Search: Network Rank
    19
  • 37. People Search: Profile Quality
    20
  • 38. People Search: SEO
    21
  • 39. People Search: Query-Dependent Signals
    22
  • 40. People Search: Inferring Structure
    23
  • 41. People Search: Ambiguity
    vs.
    vs.
    24
  • 42. People Search: HMM + Segmentation
    for i in [1..n]
    s  w1 w2 … wi
    if Pc(s) > 0
    a  new Segment()
    a.segs {s}
    a.prob Pc(s)
    B[i]  {a}
    for j in [1..i-1]
    for b in B[j]
    s wjwj+1 … wi
    if Pc(s) > 0
    a  new Segment()
    a.segsb.segs U {s}
    a.probb.prob * Pc(s)
    B[i]  B[i] U {a}
    sort B[i] by prob
    truncate B[i] to size k
    25
  • 43. People Search: Personalized Signals
    26
  • 44. People Search: Further Reading
    • QCon 2010 presentation by John Wang on “LinkedIn Search: Searching the Social Graph in Real Time”http://www.infoq.com/presentations/LinkedIn-Search
    • 45. SIGIR 2011 Workshop on Entity-Oriented Searchhttp://research.microsoft.com/en-us/um/beijing/events/eos2011/
    • 46. HCIR 2011 paper by Jonathan Koren on “Faceted Search Query Log Analysis” (forthcoming)http://hcir.info/hcir-2011/
    27
  • 47. Job Matching
    28
  • 48. Job Matching: Overview
    • Job Features
    • 49. Job Description, Location, Similar Jobs, …
    • 50. Candidate Features
    • 51. Profile Data, Network, Activity, …
    • 52. Standardization
    • 53. Companies, Job Titles, Education, …
    29
  • 54.
    industry
    description
    functional area
    Current Position
    title
    summary
    tenure length
    industry
    functional area

    Job Matching: Algorithm
    Corpus Stats
    Job
    Matching
    Transition probabilities
    Connectivity
    yrs of experience to reach title
    education needed for this title

    Binary
    Exact matches:
    geo, industry,

    Soft
    transition
    probabilities,
    similarity,

    Text
    title
    geo
    company
    User Base
    Similarity
    (candidate expertise, job description)
    0.56
    Filtered
    Similarity
    (candidate specialties, job description)
    0.2
    Candidate
    Transition probability
    (candidate industry, job industry)
    0.43
    General
    expertise
    specialties
    education
    headline
    geo
    experience
    Title Similarity
    0.8
    Similarity (headline, title)
    0.7
    derived
    .
    .
    .
    30
  • 55. Job Matching: Challenges
    • Most people aren't looking for jobs.
    • 56. Complicates evaluation, training.
    • 57. Important not to offend users.
    • 58. e.g., by offering Peter Norvig a postdoc.
    • 59. You can’t always get what you want
    • 60. Every employer wants the hottest candidates.
    31
  • 61. Job Matching: Further Reading
    • KDD 2011 paper by Bekkerman & Gavish on “High-Precision Phrase-based Document Classification”http://www.stanford.edu/~gavish/documents/phrase_based.pdf
    • 62. SIGIR 2011 paper by Cetintas et al. on “Identifying Similar People in Professional Social Networks”http://dl.acm.org/citation.cfm?id=2010123
    • 63. Blog post on LinkedIn’s recommendation enginehttp://blog.linkedin.com/2011/03/02/linkedin-products-you-may-like/
    32
  • 64. Skills
    33
  • 65. Skills: What are Skills?
    34
  • 66. Skills: Identifying Skills
    35
  • 67. Skills: Cluster and Disambiguate
    angel
    36
  • 68. Skills: Assigning Skills to People
    37
  • 69. Skills: Who are the Experts?
    38
  • 70. Summary: The 3 Rs
    • Relevance
    • 71. Combine query-independent, query-dependent, and personalized features.
    • 72. Recommendations
    • 73. Match people to jobs, groups, news, …
    • 74. Reputation
    • 75. Expertise relative to professional skills.
    39
  • 76. ¿
    Open Problems
    ?
    40
  • 77. Exploratory Search
    Exploratory Search
    Investigate
    Lookup
    Learn
    Accretion
    Analysis
    Exclusion/Negation
    Synthesis
    Evaluation
    Discovery
    Planning/Forecasting
    Transformation
    Fact retrieval
    Known item search
    Navigation
    Transition
    Verification
    Question answering
    Knowledge acquisition
    Comprehension/Interpretation
    Comparison
    Aggregation/Integration
    Socialize
    41
  • 78. Explore / Exploit
    42
  • 79. Incentives for Online Reputation
    43
  • 80. Thank You!
    Questions?
    Contact:
    dtunkelang@linkedin.com
    We’re Hiring!
    http://engineering.linkedin.com/
    44