Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Beyond ratings and followers (RecSys 2012)

10,296 views

Published on

Industry session talk on LinkedIn Recommender System @ RecSys 2012

Published in: Technology, Business
  • Be the first to comment

Beyond ratings and followers (RecSys 2012)

  1. 1. Beyond Ratings & Followers Anmol Bhasin Sr. ManagerAnalytics Engineering www.linkedin.com
  2. 2. The answer is 4
  3. 3. The Recommender Ecosystem Similar Profiles Connections Network updates Events You May Be Interested In News 11
  4. 4. LinkedIn Recommendation Engine Jobs Groups PeopleRecommendation … Ads Entities Companies Searches be interested in Similar Groups Jobs You May Jobs Browse Browse Map Similar Jobs News Browse Map TalentMatch Groups Events GYML Referral Profiles People Similar Center Map … and more Products A/B APIRecommen-dation Types Behavior Collaborative Popularity User Feedback Analysis Filtering Shared, Dynamic, Unified (R-T) Feature Extraction, Entity (R-T) matching computations Core Resolution & Enrichment Offline data munging (hadoop) Service
  5. 5. Cloning
  6. 6. Possible Approaches Naïve K Nearest Neighbor solution  Complexity is O(n 2 ) Clustering  Latent Factor Models like PLSI or LDA  Hierarchical Agglomerative clustering Self Organizing Maps Item based Collaborative Filtering  Find pairs of Users viewed in the same session
  7. 7. Challenges Scale  175+ M profiles Dimensionality  ~2M companies  ~200K schools  ~147 industries  ~200 countries  ~25K titles  ~40K Skills  ~200 Job Functions Similar means different things to different people  Similar Behavior doesn’t mean you can replace me at my job  Accuracy vs Relevance (me & my boss.. ) Realtime.. It’s a problem of accuracy.. Not recall..
  8. 8. Approach  Focus attention only on pairs likely to be similar  Filter out the possibly dis-similar pairs  Run Similarity Functions on filtered in pairs FILTERCluster Rank
  9. 9. Locality Sensitive Hashing LSH function family for Cosine Distance
  10. 10. Approach  Focus attention only on pairs likely to be similar  Filter out the possibly dis-similar pairs  Run Similarity Functions on filtered in pairs FILTERCluster Rank
  11. 11. Similarity Functions Different bands of attributes  Boolean, Jaccard or Cosine Similarities across attribute pairs.• Logisitic Regression with Elastic Penalty  Learn model params on a set of hand labeled data points  Predicted value interpreted as score
  12. 12. Ad Ranking Given U j ,{(c0, b0 ), (c1, b1 ), (c2, b2 ), (c3, b3 )..(cn, bn )}, H Objective argmax(pCTR i *bidi ) iÎC Goal:  Increase revenue  Respect daily budgets of Advertisers  Good user experience
  13. 13. Campaign creation
  14. 14. Virtual Profiling Title : Eng Mgr Company : LinkedIn Location : CA,USA Skills : ML, RecSys Title : Vice President Company : Twitter Location : CA,USA Skills : DM, ML, RecSys ……………….
  15. 15. Virtual Profiling Title :Title : Eng Mgr Sr. SE<1>, Eng Mgr<1>,Company : LinkedIn Eng Dir<1>Location : CA,USASkills : ML, RecSys Company :Title : Sr. SE LinkedIn<2>, Google<1>,Company : GoogleLocation : PA, USA Location :Skills : ML, DM CA,USA <2>, PA, USA<1> Skills :Title : Eng Dir ML<2>, RecSys<1>,Company : Linkedin Stats<1>, DM<1>Location : PA, USASkills : ML, Stats, DM
  16. 16. Virtual ProfilingInformation Gain Pick Top K overrepresented features from the clicker distribution vs the target segment A representative projection of the item in the member feature space
  17. 17. CTR Prediction – CF Similarity Ranker MEMBER FEATURESAD CREATIVE VIRTUAL PROFILE Creative Score to features pCTR pCTRi correction L2 regularized Logistic Regression (Liblinear, VW, Mahout, ADMM) For new ad creatives back-off to the advertiser / ad category nodes till they reach critical impression/click volume (explore/exploit)
  18. 18. Feature Engineering – Entity Resolution Companies ‘IBM’ has 8000+ variations - ibm – ireland - ibm research - T J Watson Labs - International Bus. Machines K-Ambiguous - Deep Blue Huge impact on the business and UE  Ad targeting  TalentMatch  Referrals Asonam’11, KDD’11 30
  19. 19. Feature Engineering – Sticky Locations Open to relocation ?  Region similarity based on profiles or network  Region transition probability  predict individuals propensity to migrate and most likely migration target Impact on job recommendations  20% lift in views/viewers/applications/applicants
  20. 20. What should you transition to .. and when ? Probability of switch Months since graduation 32
  21. 21. Social Referral
  22. 22. Social ReferralLinkedin Group: Text Analytics From: Deepak Agarwal – Engineering Director, LinkedInI found this group interesting, and I think you will tooDeepak 2X conversionLinkedin Group: Text Analytics > 2X Conversion Mohammad Amin, Baoshi Yan, Sripad Sriram, Anmol Bhasin, Christian Posse. Social Referral : Using network connections to deliver recommendations. To appear in Proceedings of the Sixth ACM conference on Recommender systems (RecSys 12)
  23. 23. Orthogonality in A/B
  24. 24. Beware of some A/B testing pitfalls1. Novelty effect  E.g., new job recommendation algorithms have week-long novelty effect that shows lifts twice the stationary (real) one job views per 5% bucket range - 6/5/11 job views 6/19/11 9,000 7,000 8,000 7,000 6,000 6,000 5,000 5,000 4,000 job views per 5% bucket range - 4,000 6/5/11 3,000 job views 6/19/11 3,000 2,000 2,000 1,000 1 week lifts 1,000 2weeks lifts 0 0 0 5 10 15 20 25 0 5 10 15 20 251. Cannibalization  Zero-sum game or real lift?2. Random sampling destroys network effect 38 38
  25. 25. Open Source Technologies BoboZoie Voldemort Kafka http://data.linkedin.com 40
  26. 26. CreditsEngineering : Abhishek Gupta, Adam Smyczek, Adil Aijaz,Alan Li, Baoshi Yan, Bee-Chung Chen, Deepak Agarwal,Ethan Zhang, Haishan Liu, Igor Perisic, JonathanTraupman, Liang Zhang, Lokesh Bajaj, Mario Rodriguez,Mitul Tiwari, Mohammad Amin, Monica Rogati, ParulJain, Paul Ogilvie, Sam Shah, Sanjay Dubey, Tarun Kumar,Trevor Walker, Utku IrmakProduct : Andrew Hill, Christian posse, Gyanda Sachdeva,Mike Grishaver, Parker Barrile, Sachit Kamat Alphabetically sorted 
  27. 27. A Recommendation for you.. Picture yourself with this New Job: You Applied Researcher / Research Engineer
  28. 28. Contact: abhasin@linkedin.comhttp://data.linkedin.com/

×