Your SlideShare is downloading. ×
0
Skills, Reputation, and SearchPete SkomorochPrincipal Data Scientist, LinkedIn
Vision: Create Economic Opportunity for Every Professional2Location
LinkedIn: The Professional Profile of Record©2012 LinkedIn Corporation. All Rights Reserved. 3200+MMembers 200M MemberProf...
LinkedIn Search: Connecting Talent with Opportunity4
Skills Correlated with the Job Title “Data Scientist”5
Skills Related to “Big Data”6
Information Retrieval7
Soul Retrieval8
9
Lucene on LinkedIn10
Lucene Endorsement Graph11
Solr on LinkedIn12
Solr Endorsement Graph13
Reputation: Building the Endorsement Graph14
15Viral Growth: 1 Billion Endorsements in 5 Months
How Did We Gather this Data?161. Desire + Social Proof2. Viral Loops + Network Effects3. Data Foundation + Recommendation ...
171) Desire & Social Proof
AendorsesBBnotifiedB “accepts”endorsementBendorsesCBendorsesDEndorsementrecommendationsEmail NotificationNews Feed2) Viral...
3) Data Foundation: Skills & Suggested Skills19
Data Foundation: LinkedIn Skills20
Social Tagging Accelerates AdoptionSuggestedendorsementsSkill recommendationsSkill marketing©2012 LinkedIn Cororation. All...
Outline22Skill discoverySkill taggingSkill recommendationsSuggested endorsements
Skill Discovery: Unsupervised Topics from Profiles23Extract
Topic Clustering & Phrase Sense Disambiguation24
Deduplication Signals from Mechanical Turk25
Sample Task for Mechanical Turk Workers26
Skill Phrase Deduplication27
Outline28Skill discoverySkill taggingSkill recommendationsSuggested endorsements
Lead designer and engineer for the implementation of a user-centric, fully-configurable UI for data aggregation and report...
Outline30Skill discoverySkill taggingSkill recommendationsSuggested endorsements
Skill Inference How suggested/inferred skills work:– The skill likelihood is a conditional model– Probabilities are combi...
Skill Recommendations for Your LinkedIn Profile3749% Conversion4% Conversion
Outline38Skill discoverySkill taggingSkill recommendationsSuggested endorsements
Social Tagging via Skill Endorsements39
Social Tagging Accelerates AdoptionSkill endorsementsSkill recommendationsSkill marketing©2012 LinkedIn Cororation. All Ri...
Data Amplifies Desire411. Desire + Social Proof2. Viral Loops + Network Effects3. Data Catalyst + Recommendation Algorithms
Over 58 Million Profiles are now Tagged with Skills42
All This Data Flows Back Into Our Lucene Index43
Helping us Connect Talent & Opportunity44Location
Questions?We’re hiring: data.linkedin.com@peteskomoroch©2012 LinkedIn Corporation. All Rights Reserved. 45
CONTACTPete Skomoroch@peteskomorochhttp://data.linkedin.com
Skills, Reputation, and Search
Skills, Reputation, and Search
Skills, Reputation, and Search
Skills, Reputation, and Search
Skills, Reputation, and Search
Upcoming SlideShare
Loading in...5
×

Skills, Reputation, and Search

30,752

Published on

This keynote presentation describes the critical role that search and Lucene has in building next generation products that understand reputation and relevance. We also describe how data science and machine learning have been applied at LinkedIn to collect, interpret, and index data around topical reputation.

Lucene Revolution is the biggest open source conference dedicated to Apache Lucene/Solr.

Published in: Technology, Education
0 Comments
20 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
30,752
On Slideshare
0
From Embeds
0
Number of Embeds
37
Actions
Shares
0
Downloads
33
Comments
0
Likes
20
Embeds 0
No embeds

No notes for slide

Transcript of "Skills, Reputation, and Search"

  1. 1. Skills, Reputation, and SearchPete SkomorochPrincipal Data Scientist, LinkedIn
  2. 2. Vision: Create Economic Opportunity for Every Professional2Location
  3. 3. LinkedIn: The Professional Profile of Record©2012 LinkedIn Corporation. All Rights Reserved. 3200+MMembers 200M MemberProfiles
  4. 4. LinkedIn Search: Connecting Talent with Opportunity4
  5. 5. Skills Correlated with the Job Title “Data Scientist”5
  6. 6. Skills Related to “Big Data”6
  7. 7. Information Retrieval7
  8. 8. Soul Retrieval8
  9. 9. 9
  10. 10. Lucene on LinkedIn10
  11. 11. Lucene Endorsement Graph11
  12. 12. Solr on LinkedIn12
  13. 13. Solr Endorsement Graph13
  14. 14. Reputation: Building the Endorsement Graph14
  15. 15. 15Viral Growth: 1 Billion Endorsements in 5 Months
  16. 16. How Did We Gather this Data?161. Desire + Social Proof2. Viral Loops + Network Effects3. Data Foundation + Recommendation Algorithms
  17. 17. 171) Desire & Social Proof
  18. 18. AendorsesBBnotifiedB “accepts”endorsementBendorsesCBendorsesDEndorsementrecommendationsEmail NotificationNews Feed2) Viral Loops & Network Effects
  19. 19. 3) Data Foundation: Skills & Suggested Skills19
  20. 20. Data Foundation: LinkedIn Skills20
  21. 21. Social Tagging Accelerates AdoptionSuggestedendorsementsSkill recommendationsSkill marketing©2012 LinkedIn Cororation. All Rights Reserved.Virality only
  22. 22. Outline22Skill discoverySkill taggingSkill recommendationsSuggested endorsements
  23. 23. Skill Discovery: Unsupervised Topics from Profiles23Extract
  24. 24. Topic Clustering & Phrase Sense Disambiguation24
  25. 25. Deduplication Signals from Mechanical Turk25
  26. 26. Sample Task for Mechanical Turk Workers26
  27. 27. Skill Phrase Deduplication27
  28. 28. Outline28Skill discoverySkill taggingSkill recommendationsSuggested endorsements
  29. 29. Lead designer and engineer for the implementation of a user-centric, fully-configurable UI for data aggregation and reporting.Developed over 20 SaaS custom applications using Python,Javascript and RoR.Tagging Skill Phrases Tagging: Extract potential skill phrases from text Standardize unambiguous phrase variants29JavaScript RoR SaaS Pythonrorrubyonrailsruby on rails developmentruby railsruby on railRuby on RailsDocument(ex: Profile)TokenizationSkills TaggerPhrases(up to 6 words)Skills ClassifierSkills(unordered)Skills(ranked by relevance)
  30. 30. Outline30Skill discoverySkill taggingSkill recommendationsSuggested endorsements
  31. 31. Skill Inference How suggested/inferred skills work:– The skill likelihood is a conditional model– Probabilities are combined using a Naïve BayesClassifier If you are an engineer at Apple, you probably knowabout iPhone Development.31ProfileExtractattributes- Company ID- Title ID- Groups ID- Industry ID- …Skills ClassifierSkills(ranked by likelihood)FeatureVectors
  32. 32. Skill Recommendations for Your LinkedIn Profile3749% Conversion4% Conversion
  33. 33. Outline38Skill discoverySkill taggingSkill recommendationsSuggested endorsements
  34. 34. Social Tagging via Skill Endorsements39
  35. 35. Social Tagging Accelerates AdoptionSkill endorsementsSkill recommendationsSkill marketing©2012 LinkedIn Cororation. All Rights Reserved.
  36. 36. Data Amplifies Desire411. Desire + Social Proof2. Viral Loops + Network Effects3. Data Catalyst + Recommendation Algorithms
  37. 37. Over 58 Million Profiles are now Tagged with Skills42
  38. 38. All This Data Flows Back Into Our Lucene Index43
  39. 39. Helping us Connect Talent & Opportunity44Location
  40. 40. Questions?We’re hiring: data.linkedin.com@peteskomoroch©2012 LinkedIn Corporation. All Rights Reserved. 45
  41. 41. CONTACTPete Skomoroch@peteskomorochhttp://data.linkedin.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×