Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Large-Scale Social Recommender
Systems at LinkedIn
Mitul Tiwari
Search, Network, and Analytics (SNA)
LinkedIn
Who am I
2
Outline
• About LinkedIn
• Social Recommender Systems at LinkedIn
• Social Graph Analysis
• Virality in Social Recommender...
LinkedIn by the numbers
4
259M members 2 new members/sec
Broad Range of Products
5
Member Profile
6
Contacts
7
Talent Solutions
8
Job Search
9
Company Pages
10
Outline
• About LinkedIn
• Social Recommender Systems at LinkedIn
• Social Graph Analysis
• Virality in Social Recommender...
LinkedIn Homepage
• Powered by
recommendations
12
Recommender Ecosystem
13
Similar Profiles
Connections
News
Skill Endorsements
Outline
• Social Recommender Systems at LinkedIn
• LinkedIn Today: Recommend News
• People You May Know and Social Graph A...
LinkedIn Today: News Recommendation
• Objective: serve valuable professional news, leading to
higher engagement as measure...
News Recommendation: Explore/Exploit
16Agarwal et. al 2012
News Recommendations: Challenges
• Drop in CTR wrt Time
17
News Recommendation: Challenges
• Same item shown to the same users: drop in CTR
18
News Recommendations: Revised Algorithm
• Explore/Exploit scheme
• Explore: choose an item at random with a small probabil...
Outline
• Social Recommender Systems at LinkedIn
• LinkedIn Today: Recommend News
• People You May Know and Social Graph A...
PYMK: Network is Important
21
PYMK: Link Prediction over social Graph
22
People You May Know
• > 50% of total connections and invitations
• Challenges
• Feature Engineering
• Machine Learning
• S...
People You May Know: Feature Engineering
Alice
Bob Carol
24
How do people
know each other?
People You May Know: Feature Engineering
Alice
Bob Carol
25
How do people
know each other?
People You May Know: Feature Engineering
Alice
Bob Carol
Triangle closing
26
How do people
know each other?
People You May Know: Feature Engineering
Alice
Bob Carol
Triangle closing
Prob(Bob knows Carol) ~ the # of common
connecti...
Triangle Closing in Pig
-- connections in (source_id, dest_id) format in both directions
connections = LOAD `connections` ...
People You May Know: Feature Engineering
• Member profile contains various types of organizations
• Company, Schools, Grou...
Organizational Overlap: Feature Engineering
• Insight 1: Connection density increases with organizational
time overlap
30
...
Organizational Overlap: Feature Engineering
• Insight 2: Connection density decreases with the size of
the organizational
...
Organizational Overlap Model
• Empirical connection
density fits our model
32
How does PYMK work?
• Combine features using a Machine Learning model
33
How does diversity affects Conversion in PYMK
• Graph Structural Diversity Study
• Measure the effects of Structural Diver...
How does diversity affects Conversion in PYMK
• Members in recommendation set mapped to a graph G
• Vertices represent mem...
Structural Diversity in PYMK
• A connected component
• any pair of vertices are connected by a path or an isolated vertex
...
Structural Diversity in PYMK
37
• Invitation rate increases as the number of components
decreases
Structural Diversity in PYMK
• Lower structural diversity among recommendation set results
in a higher invitation rate
• D...
Outline
• Social Recommender Systems at LinkedIn
• LinkedIn Today: Recommend News
• People You May Know and Social Graph A...
Related Searches Recommendation
• Millions of Searches everyday
• Help users to explore and refine their queries
40
Reda e...
Related Searches Recommendation
41
Related Searches Recommendations
• Signals
• Collaborative Filtering
• Query-Result Click graph
• Overlapping terms
• Leng...
Outline
• Social Recommender Systems at LinkedIn
• LinkedIn Today: Recommend News
• People You May Know and Social Graph A...
Suggested Skill Endorsement
44
Skills Endorsements
45
Viral Growth: 3B Skills Endorsements
• One of the fastest growing product in LinkedIn’s history
46
Skill Tagging
• Tagging: extract potential skills from
profile using skills taxonomy
• Standardize skill phrase variants
P...
Skill Recommendation
• Predict a skill even if not
present in the profile
• Based on likelihood of
member having a skill
•...
Suggested Skill Endorsements
• Binary Classification
• Features
• Company overlap, School overlap, Industrial
and function...
Social Recommendation and tagging
Skill Tagging
Skill Recommendation
Suggested Skill Endorsements
50
Find influencers in Venture Capital?
51
Skills Important for Data Scientists?
52
Outline
• Social Recommender Systems at LinkedIn
• LinkedIn Today: Recommend News
• People You May Know and Social Graph A...
Scaling Challenges: Related Searches
Example
• Kafka: publish-subscribe messaging system
• Hadoop: MapReduce data processi...
A Production Azkaban Hadoop Workflow
55
Voldemort Read-Only Store
56
Summary
• Social Recommender Systems at LinkedIn
• LinkedIn Today: Recommend News
• People You May Know and Social Graph A...
References
58
Acknowledgement
• Thanks to Data Team at LinkedIn: http://data.linkedin.com
• We are hiring!
• Contact: mtiwari[at]linkedi...
Questions?
60
Upcoming SlideShare
Loading in …5
×

of

Large scale social recommender systems at LinkedIn Slide 1 Large scale social recommender systems at LinkedIn Slide 2 Large scale social recommender systems at LinkedIn Slide 3 Large scale social recommender systems at LinkedIn Slide 4 Large scale social recommender systems at LinkedIn Slide 5 Large scale social recommender systems at LinkedIn Slide 6 Large scale social recommender systems at LinkedIn Slide 7 Large scale social recommender systems at LinkedIn Slide 8 Large scale social recommender systems at LinkedIn Slide 9 Large scale social recommender systems at LinkedIn Slide 10 Large scale social recommender systems at LinkedIn Slide 11 Large scale social recommender systems at LinkedIn Slide 12 Large scale social recommender systems at LinkedIn Slide 13 Large scale social recommender systems at LinkedIn Slide 14 Large scale social recommender systems at LinkedIn Slide 15 Large scale social recommender systems at LinkedIn Slide 16 Large scale social recommender systems at LinkedIn Slide 17 Large scale social recommender systems at LinkedIn Slide 18 Large scale social recommender systems at LinkedIn Slide 19 Large scale social recommender systems at LinkedIn Slide 20 Large scale social recommender systems at LinkedIn Slide 21 Large scale social recommender systems at LinkedIn Slide 22 Large scale social recommender systems at LinkedIn Slide 23 Large scale social recommender systems at LinkedIn Slide 24 Large scale social recommender systems at LinkedIn Slide 25 Large scale social recommender systems at LinkedIn Slide 26 Large scale social recommender systems at LinkedIn Slide 27 Large scale social recommender systems at LinkedIn Slide 28 Large scale social recommender systems at LinkedIn Slide 29 Large scale social recommender systems at LinkedIn Slide 30 Large scale social recommender systems at LinkedIn Slide 31 Large scale social recommender systems at LinkedIn Slide 32 Large scale social recommender systems at LinkedIn Slide 33 Large scale social recommender systems at LinkedIn Slide 34 Large scale social recommender systems at LinkedIn Slide 35 Large scale social recommender systems at LinkedIn Slide 36 Large scale social recommender systems at LinkedIn Slide 37 Large scale social recommender systems at LinkedIn Slide 38 Large scale social recommender systems at LinkedIn Slide 39 Large scale social recommender systems at LinkedIn Slide 40 Large scale social recommender systems at LinkedIn Slide 41 Large scale social recommender systems at LinkedIn Slide 42 Large scale social recommender systems at LinkedIn Slide 43 Large scale social recommender systems at LinkedIn Slide 44 Large scale social recommender systems at LinkedIn Slide 45 Large scale social recommender systems at LinkedIn Slide 46 Large scale social recommender systems at LinkedIn Slide 47 Large scale social recommender systems at LinkedIn Slide 48 Large scale social recommender systems at LinkedIn Slide 49 Large scale social recommender systems at LinkedIn Slide 50 Large scale social recommender systems at LinkedIn Slide 51 Large scale social recommender systems at LinkedIn Slide 52 Large scale social recommender systems at LinkedIn Slide 53 Large scale social recommender systems at LinkedIn Slide 54 Large scale social recommender systems at LinkedIn Slide 55 Large scale social recommender systems at LinkedIn Slide 56 Large scale social recommender systems at LinkedIn Slide 57 Large scale social recommender systems at LinkedIn Slide 58 Large scale social recommender systems at LinkedIn Slide 59 Large scale social recommender systems at LinkedIn Slide 60
Upcoming SlideShare
Large scale social recommender systems and their evaluation
Next

5 Likes

Share

Large scale social recommender systems at LinkedIn

Keynote talk at QCON SF 2014 on large-scale social recommender systems at LinkedIn

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Large scale social recommender systems at LinkedIn

  1. 1. Large-Scale Social Recommender Systems at LinkedIn Mitul Tiwari Search, Network, and Analytics (SNA) LinkedIn
  2. 2. Who am I 2
  3. 3. Outline • About LinkedIn • Social Recommender Systems at LinkedIn • Social Graph Analysis • Virality in Social Recommender Systems • Scaling Challenges 3
  4. 4. LinkedIn by the numbers 4 259M members 2 new members/sec
  5. 5. Broad Range of Products 5
  6. 6. Member Profile 6
  7. 7. Contacts 7
  8. 8. Talent Solutions 8
  9. 9. Job Search 9
  10. 10. Company Pages 10
  11. 11. Outline • About LinkedIn • Social Recommender Systems at LinkedIn • Social Graph Analysis • Virality in Social Recommender Systems • Scaling Challenges 11
  12. 12. LinkedIn Homepage • Powered by recommendations 12
  13. 13. Recommender Ecosystem 13 Similar Profiles Connections News Skill Endorsements
  14. 14. Outline • Social Recommender Systems at LinkedIn • LinkedIn Today: Recommend News • People You May Know and Social Graph Analysis • Related Searches Recommendation • Virality in Social Recommender Systems • Skills Endorsements Suggestions and Social Virality • Scaling Challenges 14
  15. 15. LinkedIn Today: News Recommendation • Objective: serve valuable professional news, leading to higher engagement as measured by metrics such as CTR 15
  16. 16. News Recommendation: Explore/Exploit 16Agarwal et. al 2012
  17. 17. News Recommendations: Challenges • Drop in CTR wrt Time 17
  18. 18. News Recommendation: Challenges • Same item shown to the same users: drop in CTR 18
  19. 19. News Recommendations: Revised Algorithm • Explore/Exploit scheme • Explore: choose an item at random with a small probability (e.g., 5%) • Exploit: choose highest scoring CTR item (e.g., 95%) • Temporal smoothing: more weight to recent data • Impression discounting: discount items with repeat views • Segmented model: segment users in CTR estimation 19
  20. 20. Outline • Social Recommender Systems at LinkedIn • LinkedIn Today: Recommend News • People You May Know and Social Graph Analysis • Related Searches Recommendation • Virality in Social Recommender Systems • Skills Endorsements Suggestions and Social Virality • Scaling Challenges 20
  21. 21. PYMK: Network is Important 21
  22. 22. PYMK: Link Prediction over social Graph 22
  23. 23. People You May Know • > 50% of total connections and invitations • Challenges • Feature Engineering • Machine Learning • Scaling 23
  24. 24. People You May Know: Feature Engineering Alice Bob Carol 24 How do people know each other?
  25. 25. People You May Know: Feature Engineering Alice Bob Carol 25 How do people know each other?
  26. 26. People You May Know: Feature Engineering Alice Bob Carol Triangle closing 26 How do people know each other?
  27. 27. People You May Know: Feature Engineering Alice Bob Carol Triangle closing Prob(Bob knows Carol) ~ the # of common connections 27 How do people know each other?
  28. 28. Triangle Closing in Pig -- connections in (source_id, dest_id) format in both directions connections = LOAD `connections` USING PigStorage(); group_conn = GROUP connections BY source_id; pairs = FOREACH group_conn GENERATE generatePair(connections.dest_id) as (id1, id2); common_conn = GROUP pairs BY (id1, id2); common_conn = FOREACH common_conn GENERATE flatten(group) as (source_id, dest_id), COUNT(pairs) as common_connections; STORE common_conn INTO `common_conn` USING PigStorage(); 28
  29. 29. People You May Know: Feature Engineering • Member profile contains various types of organizations • Company, Schools, Groups, ... • Can we compute edge affinity based on these organization information? • Useful for many applications: • Recommending members to connect (link prediction) • Recommending other entities from the same community (community detection) 29
  30. 30. Organizational Overlap: Feature Engineering • Insight 1: Connection density increases with organizational time overlap 30 Hsieh et. al, WWW’13
  31. 31. Organizational Overlap: Feature Engineering • Insight 2: Connection density decreases with the size of the organizational 31
  32. 32. Organizational Overlap Model • Empirical connection density fits our model 32
  33. 33. How does PYMK work? • Combine features using a Machine Learning model 33
  34. 34. How does diversity affects Conversion in PYMK • Graph Structural Diversity Study • Measure the effects of Structural Diversity in PYMK recommendation • Conversion: a connection invitation is sent to one of PYMK recommendation 34 Huang et. al, RecSys RSWeb’13
  35. 35. How does diversity affects Conversion in PYMK • Members in recommendation set mapped to a graph G • Vertices represent members in the recommendation set • Edges are the connections between those members on LinkedIn social graph • 3 measures of structural diversity • Number of connected components • Number of triangles • Average local node degree 35 Huang et. al, RecSys RSWeb’13
  36. 36. Structural Diversity in PYMK • A connected component • any pair of vertices are connected by a path or an isolated vertex • Number of connected components • a measure of structural diversity [Ugander et al. 2012] • Smaller number of components => less structural diversity • Effect on Invitation rate or conversion rate • ratio of the number of invitations sent and size of recommended set 36
  37. 37. Structural Diversity in PYMK 37 • Invitation rate increases as the number of components decreases
  38. 38. Structural Diversity in PYMK • Lower structural diversity among recommendation set results in a higher invitation rate • Different form Facebook data study [Ugander et al. 2012] • Use case is slightly different • Effect of structural diversity on recommender system highly depends on the use • Don’t generalize structural diversity effects on one recommender system to all 38
  39. 39. Outline • Social Recommender Systems at LinkedIn • LinkedIn Today: Recommend News • People You May Know and Social Graph Analysis • Related Searches Recommendation • Virality in Social Recommender Systems • Skills Endorsements Suggestions and Social Virality • Scaling Challenges 39
  40. 40. Related Searches Recommendation • Millions of Searches everyday • Help users to explore and refine their queries 40 Reda et. al, CIKM’12
  41. 41. Related Searches Recommendation 41
  42. 42. Related Searches Recommendations • Signals • Collaborative Filtering • Query-Result Click graph • Overlapping terms • Length-bias • Ensemble approach for unified recommendation • Practical considerations 42
  43. 43. Outline • Social Recommender Systems at LinkedIn • LinkedIn Today: Recommend News • People You May Know and Social Graph Analysis • Related Searches Recommendation • Virality in Social Recommender Systems • Skills Endorsements Suggestions and Social Virality • Scaling Challenges 43
  44. 44. Suggested Skill Endorsement 44
  45. 45. Skills Endorsements 45
  46. 46. Viral Growth: 3B Skills Endorsements • One of the fastest growing product in LinkedIn’s history 46
  47. 47. Skill Tagging • Tagging: extract potential skills from profile using skills taxonomy • Standardize skill phrase variants Profile Tokenize Skills Tagger Phrases Skills 47
  48. 48. Skill Recommendation • Predict a skill even if not present in the profile • Based on likelihood of member having a skill • Features: company, industry, skills, ... 48 Profile Tokenize Skills Tagger Phrases Skills Skills Classifier Profile features Recommended Skills
  49. 49. Suggested Skill Endorsements • Binary Classification • Features • Company overlap, School overlap, Industrial and functional area similarity, Title similarity, Site interactions, Co-interactions, ... Candidate generation Classifier Features - Company - Title - Industry ... Suggested Endorsement s 49
  50. 50. Social Recommendation and tagging Skill Tagging Skill Recommendation Suggested Skill Endorsements 50
  51. 51. Find influencers in Venture Capital? 51
  52. 52. Skills Important for Data Scientists? 52
  53. 53. Outline • Social Recommender Systems at LinkedIn • LinkedIn Today: Recommend News • People You May Know and Social Graph Analysis • Related Searches Recommendation • Virality in Social Recommender Systems • Skills Endorsements Suggestions and Social Virality • Scaling Challenges 53
  54. 54. Scaling Challenges: Related Searches Example • Kafka: publish-subscribe messaging system • Hadoop: MapReduce data processing system • Azkaban: Hadoop workflow management tool • Voldemort: Key-value store 54
  55. 55. A Production Azkaban Hadoop Workflow 55
  56. 56. Voldemort Read-Only Store 56
  57. 57. Summary • Social Recommender Systems at LinkedIn • LinkedIn Today: Recommend News • People You May Know and Social Graph Analysis • Related Searches Recommendation • Virality in Social Recommender Systems • Skills Endorsements Suggestions and Social Virality • Scaling Challenges 57
  58. 58. References 58
  59. 59. Acknowledgement • Thanks to Data Team at LinkedIn: http://data.linkedin.com • We are hiring! • Contact: mtiwari[at]linkedin.com • Follow: @mitultiwari on Twitter 59
  60. 60. Questions? 60
  • olamideogundeji5

    Mar. 17, 2021
  • kennethowino9

    Jul. 15, 2018
  • holysoros

    Dec. 15, 2017
  • zhutounong

    Sep. 23, 2017
  • seraph115

    Jan. 8, 2016

Keynote talk at QCON SF 2014 on large-scale social recommender systems at LinkedIn

Views

Total views

910

On Slideshare

0

From embeds

0

Number of embeds

9

Actions

Downloads

0

Shares

0

Comments

0

Likes

5

×