Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Tutorial on People Recommendations in Social Networks - ACM RecSys 2013,Hong Kong

15,392 views

Published on

Tutorials at ACM RecSys 2013

Social Networks
Learning to Rank
Beyond Friendship
Pref. Handling

Beyond Friendship:­ The Art, Science and Applications of Recommending People to People in Social Networks

by Luiz Augusto Pizzato (University of Sydney, Australia)
& Anmol Bhasin (LinkedIn, USA)

While Recommender Systems are powerful drivers of engagement and transactional utility in social networks, People recommenders are a fairly involved and diverse subdomain. Consider that movies are recommended to be watched, news is recommended to be read, people however, are recommended for a plethora of reasons – such as recommendation of people to befriend, follow, partner, targets for an advertisement or service, recruiting, partnering romantically and to join thematic interest groups.

This tutorial aims to first describe the problem domain, touch upon classical approaches like link analysis and collaborative filtering and then take a rapid deep dive into the unique aspects of this problem space like Reciprocity, Intent understanding of recommender and the recomendee, Contextual people recommendations in communication flows and Social Referrals – a paradigm for delivery of recommendations using the Social Graph. These aspects will be discussed in the context of published original work developed by the authors and their collaborators and in many cases deployed in massive-scale real world applications on professional networks such as LinkedIn.

Introduction
The basics of Social Recommenders
People recommender systems
Special Topics in People Recommenders
Why reciprocal (people) recommenders are different to traditional (product) recommendations
Multi-Objective Optimization
Intent Understanding
Feature Engineering
Social Referral
Pathfinding
Concluding remarks

The pre-requisite for this tutorial is some familiarity with foundational Recommender Systems, Data Mining, Machine Learning and Social Network Analysis literature.
Date

Oct 13, 2013 (08:30 – 10:15)

  • Be the first to comment

Tutorial on People Recommendations in Social Networks - ACM RecSys 2013,Hong Kong

  1. Beyond Friendship The science, applications & quirks of People Recommenders in Social Networks ACM RecSys - 2013 Hong Kong LinkedIn Confidential ©2013 All Rights Reserved Anmol Bhasin Director of Engineering Recommendations, Personalization & A/B Jointly presented with Luiz Augusto Pizza from University of Sydney
  2. Disclaimer  Material presented references publicly shared research work done at LinkedIn  Opinions expressed however are mine and do not represent the official position of LinkedIn
  3. Outline  Introduction  The basics of Social Recommenders  People recommender systems  Reciprocity & its quirks o Cornerstones o Special Topics in People Recommenders o Motivating Examples o Intent Understanding o Reciprocity & Multi-Objective Optimization o Evaluation o Some Novel approaches & Applications o Social Lens & Referrals o Virtual Profiles o Endorsements o Conclusions 3
  4. Cornerstones o Accuracy & Precision are key  Revenue at stake o Reciprocity throws a wrench o Three actors at play  Multiple (possibly competing) objectives to optimize recommendee recommendation recommender system o Evaluations are delayed  Conversion of a lead takes days/weeks/months o Not deployed in Isolation  Usually co-located work with other recommenders, search and Social Streams 4
  5. Outline  Introduction  The basics of Social Recommenders  People recommender systems  Reciprocity & its quirks  Cornerstones  Motivating Examples o Special Topics in People Recommenders o Intent Understanding o Multi-Objective Optimization o Evaluation Quirks o Some Novel approaches & Applications o Social Lens & Referrals o Virtual Profiles o Endorsements o Conclusions 5
  6. Motivating Examples o People You May Know o Utility To Recommendee, Recommender System, Reciprocity 6
  7. Motivating Examples o People You May Want to Follow  Utility To Recommendee, Recommender System 7
  8. Motivating Examples o InMail Suggest  Reciprocity, MOO, Utility to Recommendee, Recommended 8
  9. Motivating Examples o Talent Match o Reciprocity, MOO, Utility to all parties 9
  10. Motivating Examples o Endorsements o Reciprocity ?, Utility to Recommender System 10
  11. Motivating Examples Email News Feed Notification o Endorsements A o endorses B No Reciprocity, Utility B Recommender System to notified B “accepts” endorsement Endorsement recommendations B endorses C B endorses D
  12. Outline  Introduction  The basics of Social Recommenders  People recommender systems  Reciprocity & its quirks  Cornerstones  Motivating Examples o Special Topics in People Recommenders o Intent Understanding o Multi-Objective Optimization o Some Novel approaches & Applications o Social Lens & Referrals o Virtual Profiles o Endorsements o Conclusions 12
  13. Intent Understanding Image credit: http://www.acquisio.com/
  14. Recruiting Intent
  15. Recruiting Intent o Look-alike Models  Well Researched technique in Computational Advertising  Finding/Ranking behavioral look-alikes Performance at a certain reach Reference : http://www.theguardian.com/media-network/media-networkblog/2013/sep/06/lookalike-modelling-advertising-demystified
  16. Recruiting Intent o Target Definition is crucial  How do we define targets/labels to predict? It is a waste of time to develop features and learning algorithms without carefully defining the right target T: Profile Based Recruiters U: Non-Recruiters VT: Recruiters not showing Recruiting activity CT: Recruiters showing activity CU: Showing Recruiting activity VU: Not showing Recruiting activity : positives : negatives
  17. Recruiting Intent o As always, magic is in the features  Who are you? - industry, title, seniority, function, skill, groups …  What are you doing ? - page views, searches, invitations, news reads, group memberships  Temporal Behavioral featues.. target window t0 tn tn+1 target time feature window 17
  18. Recruiting Intent o L2 Regularized Logistic Regression o Model derives a response score for each user from his static profile and past online activities o Score indicates the likelihood that this user will respond to the ad campaign (clicks or conversion) 18
  19. Job Seeking Intent 2008.02 2010.05 Given 1) the member started job a at time ta 2) the member hasn’t change from job a till now 3) various information (x) we have about the member Predict the probability of the user changing to job b at time y 2013.09
  20. Job Seeking - Survival Analysis Review of Survival Analysis  is the time of death/event/purchase  is the survival time  Probability density distribution of event  Survival function  Hazards function
  21. Job Seeking – Survival Analysis Cox Proportional Hazards Model for Survival Analysis  How to incorporate covariates/additional information? – Covariates are multiplicatively relative to hazards (Cox Proportional Hazards) – Another way to do this is to have covariates multiplicatively related to Survival (Accelerated Failure time)  What can be included in x? – Time independent variables  Titles of Jobs, Companies at play, long term user preferences – Time dependent interval variables  Mean time to switch between jobs in an area, industry – Time dependent external variables  Seasonal softness – Time independent external variables  Economic conditions
  22. Job Seeking Intent Weibull Distribution Basic Weibull distribution Proportional hazards model with Weibull distribution Scale of the curve Reference : http://data.princeton.edu/pop509/ParametricSurvival.pdf
  23. Probability of switch Job Seeking – Feature Engineering Months since graduation What should you transition to .. and when ? 23
  24. Job Seeking – Feature Engineering  Open to relocation ?  Region similarity based on profiles or network  Region transition probability  Model individuals propensity to migrate and most likely migration target
  25. Job Seeking Bayesian Proportional Hazards Model  A hazards model for each transition pair m: {ja -> jb}  Hierarchical Bayesian models: handle transitions without much training data data Transition Reference : Jian Wang, Yi Zhang, Christian Posse, A Bhasin. Is it time for a career switch? Proceedings of the 22nd World Wide Web conference, 2013
  26. Job Seeking Intent What Can the Model Tell Us?  Tenure-based Decision Probability – The probability that user  make a job transition from to  at time between and to (in the near future)  given that the user doesn’t change job from till now
  27. Job Seeking Intent H-one • Single set of parameters H-Source • Multiple sets of parameters for transitions H-SourceDest • Multiple sets of parameters for transitions H-SourceDestCov • Further incorporates covariates
  28. Multi-Objective Optimization
  29. Multi-Objective Optimization Recommender EDITORIAL AD SERVER content Clicks on FP links influence downstream supply distribution PREMIUM DISPLAY (GUARANTEED) NETWORK PLUS (Non-Guaranteed) Downstream engagement (Time spent)
  30. Multi-Objective Optimization Serving Content on Y! Front Page : Click Shaping  What do we want to optimize?  Maximize clicks (maximize downstream supply from FP)  But consider the following  Article 1: CTR=5%, utility per click = 5  Article 2: CTR=4.9%, utility per click=10  By promoting 2, we lose 1 click/100 visits, gain 5 utils  If we do this for a large number of visits --- lose some clicks but obtain significant gains in utility?  E.g. lose 5% relative CTR, gain 40% in utility (revenue, engagement, etc)
  31. Multi-Objective Optimization other Why call it Click Shaping? other video videogames tv buzz autos finance gmy.news health autos travel hotjobs travel buzz video videogames tv hotjobs tech movies movies tech finance gmy.news health AFTER new.music new.music BEFORE sports sports shopping shopping news shine shine rivals omg realestate realestate omg 10.00% 8.00% 6.00% 4.00% 2.00% -8.00% -10.00% es othe r gam tv vide o vide o -6.00% om g rea le s tat e rival s -4.00% buzz finan ce gmy .ne w s heal th hotjo bs mov ie s new .mus ic new s 0.00% -2.00% aut o s Supply distribution Changes shin e shop ping spor ts te ch tra ve l rivals news SHAPING can happen with respect to any downstream metrics (like engagement)
  32. Multi-Objective Optimization n articles K properties m user segments A1 S1 A2 S2 news finance … … … omg An Sm CTR of user segment i on article j: pij Time duration of i on j: dij 32
  33. Multi-Objective Optimization  Scalarization Goal Programming Simplex constraints on xiJ is always applied Constraints are linear Every 10 mins, solve x Use this x as the serving scheme in the next 10 mins Reference : Deepak Agarwal, Bee-Chung Chen, Pradheep Elango, Xuanhui Wang. Click shaping to optimize multiple objectives. Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining (KDD’11)
  34. Multi-Objective Optimization Pareto-optimal solution 34
  35. Real Time Talent Match
  36. Multi-Objective Optimization Increase TalentMatch Utility fn(booking rate, email rate, reply rate) Job-Seeking Intent: actives & passives 16x reply rate on career-related mail Reply Rate Reference : Mario Rodriguez, Christian Posse, Ethan Zhang. Multiple Objective Optimization in Recommender Systems. Proceedings of the Sixth ACM conference on Recommender systems (RecSys '12)
  37. Multi-Objective Optimization Match Score Distributions Talent Match ranking Match Score 1, Item X, 0.98, Non-Seeker 2, Item Y, 0.91, Non-Seeker --------------------------------------3, Item Z, 0.89, Active Divergence score Divergence Function Δ() Perturbation Function f() Perturbed ranking Match Score, Perturbed Score 1, Item X, 0.98, 0.98, Non-Seeker 2, Item Z, 0.89, 0.93, Active -----------------------------------------------3, Item Y, 0.91, 0.91, Non-Seeker Objective Function g() How: Controlled Perturbation Objective score
  38. Multi-Objective Optimization  Perturbation Function  Divergence Function  Objective Function
  39. Multi-Objective Optimization  Loss Function  Objective and divergence depend on a sort/rank, so gradient-based optimization not directly applicable
  40. Connecting Talent to Opportunity MOO Pareto Optimization
  41. Connecting Talent to Opportunity MOO 0 54 27 100 Match Score Histogram Divergence
  42. Multi-Objective Optimization Experiments  A/B Test – Treatment 1: 1.15 boost – Treatment 2: 1.07 boost – Control: 1.0 boost  Expectations – 50% increase in reply rate for 1.07 boost – 100% increase in reply rate for 1.15 boost – Expected booking rate and email rate to remain unchanged or minimally affected
  43. Multi-Objective Optimization Booking rate α = β = 1.07 0% α = β = 1.15 -0.4% Email rate α = β = 1.07 31% α = β = 1.15 25% Reply rate α = β = 1.07 22% α = β = 1.15 42%
  44. Evaluation quirks  Days to act on Recommendation  Weeks to reciprocate  Does not work in isolation - Success only if - Reciprocation comes from first impression from recommender - First impression : Did not see that result on any channel “K” days before seeing it on the Recommender
  45. Outline  Introduction  The basics of Social Recommenders  People recommender systems  Reciprocity & its quirks  Cornerstones  Motivating Examples  Special Topics in People Recommenders  Intent Understanding  Multi-Objective Optimization  Evaluation quirks o Some Novel approaches & Applications o Social Lens & Referrals o Virtual Profiles o Endorsements o Conclusions 45
  46. Social Referral
  47. Social Referral Formulation When user ui interacts with Group g j Define C = f , the candidate neighbor set Foreach uk Î neighbor(ui ) Guk = {g0 , g1,...gk } - Generate - If g j Î Gu then k the top-k group recommendations C Å (uk , g j ) Rank order C using  Connection strength between ui &uk  Probability ofuk joiningg j  Combined score using the above two factors
  48. Social Referral Linkedin Group: Text Analytics From: Deepak Agarwal – Engineering Director, LinkedIn I found this group interesting, and I think you will too Deepak Linkedin Group: Text Analytics 2X > 2X conversion Conversion Reference : Mohammad Amin, Baoshi Yan, Sripad Sriram, Anmol Bhasin, Christian Posse. Social Referral : Using network connections to deliver recommendations. Proceedings of the Sixth ACM conference on Recommender systems (RecSys '12)
  49. Social Referral Quirks and Cautionary points  Controlled number of referral nudges to the source user - If nudged too many times, it may degrade the experience  Controlled number of referrals to the target user - Presumably degrades the experience of the target user as well  Only useful to use social referrals to individuals not engaged with the product - If the target already interacts with many items, the referral has marginal utility  Referred items of high quality - If the item referred is of poor quality, the entire exercise is futile
  50. Virtual Profiles
  51. Virtual Profiles Title : Eng Dir Company : LinkedIn Location : CA,USA Skills : ML, RecSys Title : Sr. Manager Company : Netflix Location : CA, USA Skills : Machine Learning, Data Mining Title : Eng Mgr Company : Linkedin Location : PA, USA Skills : Machine Learning, Statistics, Data Mining Title : Sr. Mgr<1>, Eng Dir<1>, Eng Mgr <1> Company : LinkedIn<1>, Netflix<1> Google<1>, Location : CA,USA <2>, PA, USA<1> Skills : ML<2>, RecSys<1>, Stats<1>, DM<1>
  52. Virtual Profiles Point-wise Mutual Information  Pick Top K overrepresented features (f) from the Group Join distribution vs the overall userpopulation feature distribution A representative projection of the item (Group) in the user feature space
  53. Virtual Profiles – Group join propensity Ranker MEMBER FEATURES Group virtual profile Group Features Pjoin Social Information  Match feature pair includes  Group Virtual Profile features, Group popularity features  Member Profile features  Contextual features (device, location)  Interaction featues  L2 regularized Logistic Regression (Liblinear, VW, Mahout, ADMM) for Ranking Reference : Haishan Liu, Mohammad Amin, Baoshi Yan, Anmol Bhasin. Generating Supplemental Content Information using Virtual Profiles.To appear at ACM RecSys’13
  54. Endorsements 54
  55. Endorsements  Rank Ordered Candidates with LR with L2 penalty  Features – – – – – – – Company overlap School overlap Group overlap Industry and functional area similarity Title similarity Site interactions Co-interactions  Open Questions – Do they share the same skill ? – Validity of the endorsement ? Candidate generation Feature Vectors - Company - Title - Groups - Industry -… Classifier Suggested Endorsements (ranked by likelihood) 55
  56. Endorsements Skill marketing Skill recommendations Skill endorsements ©2012 LinkedIn Cororation. All Rights Reserved.
  57. Endorsements Can We Find Influencers In Venture Capital? 57
  58. Big Challenge (Shameless plug)  Detect when we don’t have “ANY” good items to show to a particular user Top K WTF  Personalized Thresholds for users – Cost of Consumption  Marginal utility of showing a particular item to a particular user is –ve  How to use crowdsourcing to rate WTF for a particular user When not to show.. 58
  59. Outline  Introduction  The basics of Social Recommenders  People recommender systems  Reciprocity & its quirks  Cornerstones  Motivating Examples  Special Topics in People Recommenders  Intent Understanding  Multi-Objective Optimization  Evaluation quirks  Some Novel approaches & Applications  Social Lens & Referrals  Virtual Profiles  Endorsements o Conclusions 59
  60. Conclusions o Accuracy & Precision are key  Revenue at stake o Reciprocity throws a wrench o Three actors at play  Multiple (possibly competing) objectives to optimize o Evaluations are delayed recommendee recommendation recommender system  Conversion of a lead takes days/weeks/months o Not deployed in Isolation  Usually co-located work with other recommenders, search and Social Streams 60
  61. It takes a village! LinkedIn Engineering : Abhishek Gupta, Adam Smyczek, Adil Aijaz, Alan Li, Baoshi Yan, Bee-Chung Chen, Deepak Agarwal, Ethan Zhang, Haishan Liu, Igor Perisic, Jonathan Traupman, Liang Zhang, Lokesh Bajaj, Mario Rodriguez, Mitul Tiwari, Mohammad Amin, Parul Jain, Paul Ogilvie, Sam Shah, Sanjay Dubey, Tarun Kumar, Trevor Walker, Utku Irmak LinkedIn Product : Andrew Hill, Christian posse, Gyanda Sachdeva, Parker Barrile, Sachit Kamat External Partners : Christian Posse, Mike Grishaver, Monica Rogati, Luiz Augusto Pizzato, Yi Zhang Alphabetically sorted 
  62. Contact: abhasin@linkedin.com http://data.linkedin.com/ http://engineering.linkedin.com/

×