Connecting Talent to Opportunity.. at scale @ LinkedIn

7,447 views

Published on

Text Analytics Summit 2012, San Francisco.

Published in: Technology
0 Comments
9 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
7,447
On SlideShare
0
From Embeds
0
Number of Embeds
247
Actions
Shares
0
Downloads
30
Comments
0
Likes
9
Embeds 0
No embeds

No notes for slide
  • Speak to the growth.
  • Mission: For us, fundamentally changing the way the world works begins with our mission statement: To connect the world’s professionals to make them more productive and successful. This means not only helping people to find their dream jobs, but also enabling them to be great at the jobs they’re already in. Vision: But, we’re just getting started. By our measure,there are more than 640 million professionals in the world. And roughly 3.3 billion people in the global workforce. Ultimately, our vision is to create economic opportunity for every professional, which we believe is an especially crucial objective in light of current macroeconomic trends.Our most important core value is that members come first.
  • Speak to the growth.
  • Speak to the growth.
  • Talent Match: job posting flow: When recruiters post jobs we in real time suggest top candidates fit for the job
  • Similar Profiles: For recruiters, useful in sourcingIf a recruiter finds a profile that he is interested in, this helps him in finding other similar profiles that may be of interest
  • As part of Talent pipeline, this new features helps recruiters in importing resumesWhen recruiters import resume we suggest in real-time LinkedIn profile that might correspond to those resumes so they can link the prospect with a LinkedIn account.
  • As part of Talent pipeline, this new features helps recruiters in importing resumesWhen recruiters import resume we suggest in real-time LinkedIn profile that might correspond to those resumes so they can link the prospect with a LinkedIn account.
  • As part of Talent pipeline, this new features helps recruiters in importing resumesWhen recruiters import resume we suggest in real-time LinkedIn profile that might correspond to those resumes so they can link the prospect with a LinkedIn account.
  • Job recommendations are a feedback on how well they write their profile on Li. Main problem definition because there are secondary problem definitions.
  • As part of Talent pipeline, this new features helps recruiters in importing resumesWhen recruiters import resume we suggest in real-time LinkedIn profile that might correspond to those resumes so they can link the prospect with a LinkedIn account.
  • 8000 name variants of IBMWe use the definition of entity resolution terminology k−ambiguous and k−variant from [10]. Same company name can denote multiple company entities but each occurrence of a company name references a single entity only. A name referring to k different entities is called k − ambigous. Additionally, An entity which can be referred to by k different names is called k − variant.Ranker approach does not work. A given name may not be resolvable in the sense that the company entity has not being created yet…Classification problemGiven a pair of (member position, company entity), a binary classifier would determine whether there is enough evidence to resolve the member position to the company entity. This would address the problem of the ranking approach in that an unresolvable member position would most likely remain unresolved because the classifier has insufficient evidence for any company entity. It is certainly possible that there could be multiple company entities with sufficient evidence for a member position.
  • Unreasonable effectiveness of Big Data.. This chart shows the probability of holding a title across all titles, plotted vs number of months after graduation. Notice the spikes.. They are ~12 month almost perfectly aligned.. Remember the itch that you had when you finished 2 years at your company 
  • Huge impact on the business: Target more aggressively active JS while not spamming the others. Personalize the set of recommendations- Ads
  • Exposing a % of users to a new treatmentMeasuring the effect on metrics of interestRunning statistical tests to determine whether the differences are statistically significant, thus establishing causality
  • Connecting Talent to Opportunity.. at scale @ LinkedIn

    1. 1. Connecting Talent  Opportunity.. at scaleAnmol Bhasin Text Analytics Summit San FranciscoDirector of Engineering November 14, 2012Recommendations & PersonalizationLinkedIn Confidential ©2013 All Rights Reserved
    2. 2. World’s Largest Professional Network180 M+ 2 new 100M+ 2M+ …..Members Worldwide Members Per Second Monthly Unique Visitors Company Pages LinkedIn Confidential ©2013 All Rights Reserved 3
    3. 3. Our Mission Connect the world’s professionals to make them more productive and successful.Our VisionCreate economic opportunity for every professional in the world.Members First!
    4. 4. 85% *175M+ 2Use Linkedin to Hire.. ….. new Members Per Second Monthly Unique VisitorsMembers Worldwide Company Pages LinkedIn Confidential ©2013 All Rights Reserved 6
    5. 5. Real Time Talent Match 10
    6. 6. 11
    7. 7. Automated Resume to Profile Link12
    8. 8. 14
    9. 9. Problem Definition Address job seeker* need: find dream job – Huge cost of consumption Lag between view and application is in hours/days – Extremely high level of expectation – No forgiveness for less than perfect recommendations  Accuracy is key! (*) 20% active, 60% receptive -- 10/12 Job Seeker Survey, 20K in 7 countrie
    10. 10. How LinkedIn matches people to jobs Job Corpus Stats Matching Transition probabilities Connectivity Binary yrs of experience to reach titletitle industry … Exact matches: education needed for this titlegeo description …company functional area geo, industry, … User Base Soft Similarity (candidate expertise, job description) transition Filtered 0.56 probabilities, Similarity Candidate similarity, (candidate specialties, job description) … 0.2 Transition probability Text (candidate industry, job industry) General Current Position 0.43 expertise title specialties summary Title Similarity education tenure length 0.8 headline industry Similarity (headline, title) geo functional area experience … 0.7 Ensemble . . Scorings . ~250B Member Job Pairs a day! 17
    11. 11. Feature Engineering – Entity Resolution Companies ‘IBM’ has 8000+ variations - ibm – ireland - ibm research - T J Watson Labs - International Bus. Machines K-Ambiguous - Deep Blue Huge impact on the business and UE  Ad targeting  TalentMatch  Referrals Asonam’11, KDD’11 19
    12. 12. Feature Engineering – Would you move Open to relocation ?  Region similarity based on profiles or network  Region transition probability  predict individuals propensity to migrate and most likely migration target Impact on job recommendations  20% lift in views/viewers/applications/applicants
    13. 13. What should you transition to .. and when ? Probability of switch Months since graduation 21
    14. 14. Where are you likely to stay ? 23
    15. 15. Power of aggregation.. Before employees worked at Yahoo! (247) Google (139) Microsoft (105) Oracle (93) IBM (68) Before employees worked at Microsoft (1379) IBM (939) Yahoo! (608) Oracle (558) 24
    16. 16. Segmented Models Demographic Segmentation  Students (or recent grads)  US vs International members  Industry Specific models  e.g. Finance vs Technology Behavioral Segmentation  Job Seekers (Active)  Daily Users vs Monthly Users
    17. 17. Job Seeking  Types  Active  Passive receptive  Not a job seeker  Modeling  Ordered logistic reg.  Impact  ~10x application rate between Active and Passive receptive27
    18. 18. Job Seeking Socially Contagious? [Zhang, 2012] 28
    19. 19. Capturing User Interests Social GraphsContent http://inmaps.linkedinlabs.com/ Behavior PVs Queries Actions (clicks) … 29
    20. 20. A/B TestingIs A better than B.. Let’s test
    21. 21. A/B Testing Is A better than B.. Let’s test Beware of - novelty effect - cannibalization - potential biases (time, targeted population) - random sampling destroying the network effectEnjoy testing furiously! Don’t forget to A/A test first job views per 5% bucket range - 6/5/11 job views 6/19/11 9,000 7,000 8,000 7,000 6,000 6,000 5,000 5,000 4,000 job views per 5% bucket range - 4,000 6/5/11 3,000 job views 6/19/11 3,000 2,000 2,000 1,000 1,000 0 0 0 5 10 15 20 25 0 5 10 15 20 25 (“Seven Pitfalls to Avoid when Running Controlled Experiments on the Web”, KDD’09 “Framework and Algorithms for Network Bucket Testing” WWW’12 submission) 32 32
    22. 22. Some final remarks Most people arent actively looking for jobs. – Many people are but most aren’t – Complicates evaluation and training Important not to offend – JYMBII: I am more senior than that! – What is the price of a bad recommendation? (PYMK vs. JYMBII) You can’t always get what you want – Every employer wants the hottest candidate. – The perfect candidate already works for you. 33
    23. 23. CreditsEngineering : Abhishek Gupta, Adam Smyczek, AdilAijaz, Alan Li, Baoshi Yan, Bee-Chung Chen, DeepakAgarwal, Ethan Zhang, Haishan Liu, Igor Perisic, JonathanTraupman, Liang Zhang, Lokesh Bajaj, MarioRodriguez, Mitul Tiwari, Mohammad Amin, MonicaRogati, Parul Jain, Paul Ogilvie, Sam Shah, SanjayDubey, Tarun Kumar, Trevor Walker, Utku IrmakProduct : Andrew Hill, Christian posse, GyandaSachdeva, Mike Grishaver, Parker Barrile, Sachit Kamat Alphabetically sorted 
    24. 24. Contact: abhasin@linkedin.comhttp://data.linkedin.com/

    ×