Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn

Daniel
Keeping It Professional:Relevance, Recommendations, and Reputation.
Daniel Tunkelang
Principal Data Scientist at LinkedIn
1

Overview
What is LinkedIn?
Hard problems we’re tackling in:
Relevance
Recommendations
Reputation
Open problems
2

Identity
Connect, find and be found
Rolodex, Resume, Business Card
LinkedIn Profile, Address Book, Search
Insights
Be great at what you do
Newspapers,
Trade Magazines, Events
Homepage, LinkedIn Today, Groups
Everywhere
Desktop
Work wherever our members work
Mobile, APIs, Plug-Ins
What is LinkedIn?
3

Identity: Profile of Record
4

Identity: Connect with Others
5

Identity: Join the Conversation
6

Insights: Power of Aggregation
Before
employees worked at
Yahoo! (169)
Google (96)
Oracle (78)
Microsoft (72)
IBM (43)
Before
employees worked at
Google(475)
Microsoft (448)
LinkedIn (169)
Apple, Inc. (154)
ebay(133)
7

Insights: Market Research
8

Insights: Data Stories
9

Everywhere
10

Hard Problems: Examples
Relevance
People Search
Recommendations
Job Matching
Reputation
Skills
11

People Search
12

People Search: Scale
120M+ members
2B searches in 2010
Based on (cf. http://sna-projects.com/)
13

People Search: Faceted Search
14

People Search: Network Facet
15

People Search: Type-Ahead
16

People Search: Relevance
Query-Independent Signals
Network Rank, Profile Quality
Query-Dependent Signals
Field-Based Relevance
Personalized Signals
Network Distance
17

People Search: Query-Independent Signals
18

People Search: Network Rank
19

People Search: Profile Quality
20

People Search: SEO
21

People Search: Query-Dependent Signals
22

People Search: Inferring Structure
23

People Search: Ambiguity
vs.
vs.
24

People Search: HMM + Segmentation
for i in [1..n]
s  w1 w2 … wi
if Pc(s) > 0
a  new Segment()
a.segs {s}
a.prob Pc(s)
B[i]  {a}
for j in [1..i-1]
for b in B[j]
s wjwj+1 … wi
if Pc(s) > 0
a  new Segment()
a.segsb.segs U {s}
a.probb.prob * Pc(s)
B[i]  B[i] U {a}
sort B[i] by prob
truncate B[i] to size k
25

People Search: Personalized Signals
26

People Search: Further Reading
QCon 2010 presentation by John Wang on “LinkedIn Search: Searching the Social Graph in Real Time”http://www.infoq.com/presentations/LinkedIn-Search
SIGIR 2011 Workshop on Entity-Oriented Searchhttp://research.microsoft.com/en-us/um/beijing/events/eos2011/
HCIR 2011 paper by Jonathan Koren on “Faceted Search Query Log Analysis” (forthcoming)http://hcir.info/hcir-2011/
27

Job Matching
28

Job Matching: Overview
Job Features
Job Description, Location, Similar Jobs, …
Candidate Features
Profile Data, Network, Activity, …
Standardization
Companies, Job Titles, Education, …
29

…
industry
description
functional area
Current Position
title
summary
tenure length
industry
functional area
…
Job Matching: Algorithm
Corpus Stats
Job
Matching
Transition probabilities
Connectivity
yrs of experience to reach title
education needed for this title
…
Binary
Exact matches:
geo, industry,
…
Soft
transition
probabilities,
similarity,
…
Text
title
geo
company
User Base
Similarity
(candidate expertise, job description)
0.56
Filtered
Similarity
(candidate specialties, job description)
0.2
Candidate
Transition probability
(candidate industry, job industry)
0.43
General
expertise
specialties
education
headline
geo
experience
Title Similarity
0.8
Similarity (headline, title)
0.7
derived
.
.
.
30

Job Matching: Challenges
Most people aren't looking for jobs.
Complicates evaluation, training.
Important not to offend users.
e.g., by offering Peter Norvig a postdoc.
You can’t always get what you want
Every employer wants the hottest candidates.
31

Job Matching: Further Reading
KDD 2011 paper by Bekkerman & Gavish on “High-Precision Phrase-based Document Classification”http://www.stanford.edu/~gavish/documents/phrase_based.pdf
SIGIR 2011 paper by Cetintas et al. on “Identifying Similar People in Professional Social Networks”http://dl.acm.org/citation.cfm?id=2010123
Blog post on LinkedIn’s recommendation enginehttp://blog.linkedin.com/2011/03/02/linkedin-products-you-may-like/
32

Skills
33

Skills: What are Skills?
34

Skills: Identifying Skills
35

Skills: Cluster and Disambiguate
angel
36

Skills: Assigning Skills to People
37

Skills: Who are the Experts?
38

Summary: The 3 Rs
Relevance
Combine query-independent, query-dependent, and personalized features.
Recommendations
Match people to jobs, groups, news, …
Reputation
Expertise relative to professional skills.
39

¿
Open Problems
?
40

Exploratory Search
Exploratory Search
Investigate
Lookup
Learn
Accretion
Analysis
Exclusion/Negation
Synthesis
Evaluation
Discovery
Planning/Forecasting
Transformation
Fact retrieval
Known item search
Navigation
Transition
Verification
Question answering
Knowledge acquisition
Comprehension/Interpretation
Comparison
Aggregation/Integration
Socialize
41

Explore / Exploit
42

Incentives for Online Reputation
43

Thank You!
Questions?
Contact:
dtunkelang@linkedin.com
We’re Hiring!
http://engineering.linkedin.com/
44

http://thenoisychannel.com	2402
http://us-w1.rockmelt.com	31
http://paper.li	16
http://www.linkedin.com	11
http://www.onlydoo.com	10
http://recsys.acm.org	10
http://tweetedtimes.com	7
http://a0.twimg.com	7
http://contentmanagementconnection.com	7
http://internetbusiness.collected.info	6
http://www.borjasantaolalla.com	5
http://translate.googleusercontent.com	4
http://blog.contentmanagementconnection.com	4
http://thenoisychannel.com.	3
http://cmc.l50sw.com	2
http://recsys.hosting.acm.org	2
https://twitter.com	2
http://thenoisychannel.com.sixxs.org	2
http://www.slideshare.net	2
http://feeds2.feedburner.com	1
http://twitter.com	1
http://twitter.com	1
http://webcache.googleusercontent.com	1
http://www.google.com	1
http://www.nonrelevant.net	1
http://www.tumblr.com	1
http://www.zoominfo.com	1

Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn

by Daniel Tunkelang

Accessibility

Categories

Upload Details

Usage Rights

27 Embeds 2,541

Statistics

Keeping It Professional: Relevance, Recommendations, and Reputation at LinkedIn Presentation Transcript