Downloaded 286 times
Uploaded bySam Shah
Strata 2013 - LinkedIn Endorsements: Reputation, Virality, and Social Tagging
This document summarizes LinkedIn's new skill endorsement feature, which allows members to endorse each other's skills. It discusses how skill endorsements have grown virally with over 800 million endorsements in 4 months. It also describes how LinkedIn uses data from profiles and relationships to suggest relevant skill endorsements to members.
More Related Content
Similar to Strata 2013 - LinkedIn Endorsements: Reputation, Virality, and Social Tagging
![Getting Started with Apache Spark: Big Data Made Simple [Free Meetup]](https://cdn.slidesharecdn.com/ss_thumbnails/apachesparkgettingstarted-260203175547-8361bcc3-thumbnail.jpg?width=640&height=640&fit=bounds)
Strata 2013 - LinkedIn Endorsements: Reputation, Virality, and Social Tagging
- 1. LinkedIn Endorsements: Reputation,Virality, and Social Tagging O‟Reilly Strata - February 28, 2013 Sam Shah @sam_shah Pete Skomoroch @peteskomoroch ©2012 LinkedIn Corporation. All Rights Reserved.
- 2. Sam Shah Principal Engineer and Engineering Manager @sam_shah www.linkedin.com/in/shahsam Peter Skomoroch Principal Data Scientist @peteskomoroch www.linkedin.com/in/peterskomoroch ©2012 LinkedIn Corporation. All Rights Reserved.
- 3. LinkedIn: The ProfessionalProfile of Record 200+M Members 200M Member Profiles ©2012 LinkedIn Corporation. All Rights Reserved. 3
- 4. LinkedIn‟s Latest DataProduct: Skill Endorsements 4
- 5. Viral Growth: 800MEndorsements in 4 Months 5
- 6. Data Amplifies Desire 1.Desire + Social Proof 2. Viral Loops + Network Effects 3. Data Foundation + Recommendation Algorithms 6
- 7. 1) Desire &Social Proof 7
- 8. Email News Feed Notification 2) Viral Loops & Network Effects A B B “accepts” endorses notified endorsement B Endorsement recommendations B B endorses endorses C D
- 9. 3) Data Foundation:Skills & Suggested Skills 9
- 10. Data Foundation: LinkedInSkills 10
- 11. Social Tagging AcceleratesAdoption Skill marketing Skill recommendations Virality only Suggested endorsements ©2012 LinkedIn Cororation. All Rights Reserved.
- 12. Outline Skill discovery Skill tagging Skill recommendations Suggested endorsements 12
- 13. Unsupervised Topic Discoveryfrom Profiles Extract 13
- 14. Profile Building the SkillsDictionary (specialties) What is the skills dictionary? – A growing taxonomy of skills Tokenization Clustering – Generated by mining profiles and maintained by the Skills team at LinkedIn Crowdsourcing – Created using clustering and crowdsourcing. – Multiple phrases, acronyms, and misspellings map to a single standardized skill. 250+ different phrases map to “Microsoft Office” Taxonomy 14
- 15. Topic Clustering &Phrase Sense Disambiguation 15
- 16. Skills Dictionary: MicrosoftOffice – ms office – ms office suite – computer skills including ms office – office 97 – microsoft office user Microsoft Office – mac office – microsoft office 2003 & 2007 (Skill ID = 366) – microsoft office suits – microsoft ofice – microsoft ofiice – ms office certified – office 98 – … 16
- 17. Deduplication Signals fromMechanical Turk 17
- 18. Sample Task forMechanical Turk Workers 18
- 19.
- 20. Outline Skill discovery Skill tagging Skill recommendations Suggested endorsements 20
- 21. Skills Classification Use skill dictionary metadata to tag, standardize and infer skills Run classifiers for each skill on member profiles Public Speaking Ruby on Rails Entrepreneurship Microsoft Office AP Style 21
- 22. Document Tagging SkillPhrases (ex: Profile) Tagging: Extract potential skill phrases from text Lead designer and engineer for the implementation of a user- centric, fully-configurable UI for data aggregation and reporting. Developed over 20 SaaS custom applications using Python, Javascript and RoR. Tokenization Phrases JavaScript RoR SaaS Python (up to 6 words) Standardize unambiguous phrase variants Skills Tagger ror rubyonrails Skills ruby on rails development Ruby on Rails (unordered) ruby rails ruby on rail Skills Classifier Skills (ranked by relevance) 22
- 23. Outline Skill discovery Skill tagging Skill recommendations Suggested endorsements 23
- 24. Skills Classification onMember Profiles The skills classifier computes the likelihood of a member to have a skill based on the member’s profile, other profiles which share common attributes and their connections. Tagging Standardization Inference Profile Tokenize free Transform tags Rank skills by text text into phrase tags into potential skills likelihood Profile attributes & network signals 24
- 25. Profile Skill Inference How suggested/inferred skills work: Extract – Profiles with skills help build a massive dataset of attributes (attribute: skills). Feature - Company ID Example with a title: Vectors - Title ID - Groups ID Software Engineer Java 100 000 - Industry ID Software Engineer C++ 88 000 -… … Skills Classifier Title Skill Occurrences Skills (ranked by likelihood) 25
- 26. Profile Skill Inference How suggested/inferred skills work: Extract – The skill likelihood is a conditional model attributes Feature - Company ID – Probabilities are combined using a Naïve Bayes Vectors - Title ID Classifier - Groups ID - Industry ID -… Skills Classifier If you are an engineer at Apple, you probably know about iPhone Development. Skills (ranked by likelihood) 26
- 29. Skill Suggestions forYour LinkedIn Profile 4% Conversion 49% Conversion 29
- 30. Outline Skill discovery Skill tagging Skill recommendations Suggested endorsements 30
- 31. Social Tagging viaSkill Endorsements 31
- 32. Suggesting Endorsements Candidate People-skill combinations in a member‟s network generation Binary classification Feature - Company Features Vectors - Title – Skill inference score - Groups – Company overlap - Industry – School overlap -… – Group overlap – Industry and functional area similarity Classifier – Title similarity – Site interactions – Co-interactions Suggested Endorsements (ranked by likelihood) 32
- 33. Social Tagging AcceleratesAdoption Skill marketing Skill recommendations Skill endorsements ©2012 LinkedIn Cororation. All Rights Reserved.
- 34. Can We FindInfluencers In Venture Capital? 34
- 35. Which Skills AreImportant for a Data Scientist? 35
- 36. What Technologies areProfessionals Adopting? 36
- 37. Data Amplifies Desire 1.Desire + Social Proof 2. Viral Loops + Network Effects 3. Data Catalyst + Recommendation Algorithms 37
- 38. Infrastructure • Apache Hadoop: Parallel processing architecture • Apache Kafka: Ingress pipes • Azkaban: Hadoop scheduler • Voldemort: Egress database • Apache Pig: High-level MR language • DataFu: Convenience routines http://data.linkedin.com R. Sumbaly, J. Kreps, and S. Shah. “The „Big Data‟ ecosystem at LinkedIn”. In SIGMOD 2013 (to appear). ©2012 LinkedIn Corporation. All Rights Reserved. 38
- 39.