Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

of

Related searches at LinkedIn Slide 1 Related searches at LinkedIn Slide 2 Related searches at LinkedIn Slide 3 Related searches at LinkedIn Slide 4 Related searches at LinkedIn Slide 5 Related searches at LinkedIn Slide 6 Related searches at LinkedIn Slide 7 Related searches at LinkedIn Slide 8 Related searches at LinkedIn Slide 9 Related searches at LinkedIn Slide 10 Related searches at LinkedIn Slide 11 Related searches at LinkedIn Slide 12 Related searches at LinkedIn Slide 13 Related searches at LinkedIn Slide 14 Related searches at LinkedIn Slide 15 Related searches at LinkedIn Slide 16 Related searches at LinkedIn Slide 17 Related searches at LinkedIn Slide 18 Related searches at LinkedIn Slide 19 Related searches at LinkedIn Slide 20 Related searches at LinkedIn Slide 21 Related searches at LinkedIn Slide 22 Related searches at LinkedIn Slide 23 Related searches at LinkedIn Slide 24 Related searches at LinkedIn Slide 25 Related searches at LinkedIn Slide 26 Related searches at LinkedIn Slide 27 Related searches at LinkedIn Slide 28 Related searches at LinkedIn Slide 29 Related searches at LinkedIn Slide 30
Upcoming SlideShare
Catálogo de Debbie Reynolds
Next

0 Likes

Share

Related searches at LinkedIn

Search plays an important role in online social networks as it provides an essential mechanism for discovering members and content on the network. Related search recommendation is one of several mechanisms used for improving members’ search experience in finding relevant results to their queries. This paper describes the design, implementation, and deployment of Metaphor, the related search recommendation system on LinkedIn, a professional social networking site with over 175 million members worldwide. Metaphor builds on a number of signals and filters that capture several dimensions of relatedness across member search activity.
The system, which has been in live operation for over a year, has gone through multiple iterations and evaluation cycles. This paper makes three contributions. First, we provide a discussion of a large-scale related search recommendation system. Second, we describe a mechanism for effectively combining several signals in building a unified dataset for related search recommendations. Third, we introduce a query length model for capturing bias in recommendation
click behavior. We also discuss some of the practical concerns in deploying related search recommendations.

  • Be the first to like this

Related searches at LinkedIn

  1. 1. 1 Related Searches at LinkedIn Mitul Tiwari Joint work with Azarias Reda, Yubin Park, Christian Posse, and Sam Shah LinkedIn SIGIR Industrial Track 2013
  2. 2. 2 Who am I
  3. 3. 3 Outline • About LinkedIn • Related Searches ‣ Design ‣ Implementation ‣ Evaluation
  4. 4. 4 LinkedIn by the numbers • 175M+ members • 2+ new user registrations per second • 4.2 Billion people searches in 2011 • 9.3 Billion page views in Q2 2012 • 100+ million monthly active users in Q2 2012
  5. 5. Broad Range of Products Profile Search Hiring Solutions People You May Know Skills News
  6. 6. 6 Related Searches at LinkedIn • Millions of searches everyday • Goal: Build related searches system at LinkedIn • To help users to explore and refine their queries
  7. 7. Related Searches at LinkedIn
  8. 8. 8 Related Searches • Design • Implementation • Evaluation
  9. 9. 9 Related Searches • Design • Implementation • Evaluation
  10. 10. 10 Design • Signals ‣ Collaborative Filtering ‣ Query-Result Click graph ‣ Overlapping terms • Length-bias • Ensemble approach for unified recommendation • Practical considerations
  11. 11. 11 Design: Collaborative Filtering • Searches correlated by time ‣ Searches done in the same session by the same user ‣ Collaborative filtering: implicit feedback ‣ TFIDF scoring to take care of popular queries (e.g. `Obama’) Q1 Q2 Q3 Q4 Time
  12. 12. 12 Design: Query-Result Clicks • Searches correlated by result clicks Q1 Qn R1 Rm
  13. 13. 14 Design: Overlapping Terms • Searches with overlapping terms ‣ TFIDF scoring to give importance to terms Software Developer Software Engineer Q1 Q2
  14. 14. 15 Design: Length Bias • Insight: clicks on suggestions one term longer
  15. 15. Design: Length Bias • Insight: clicks on suggestions one term longer • Corresponds to refining the initial query • Statistical biasing model to score a longer query higher
  16. 16. 18 Design: Ensemble Approach • Need to generate unified recommendation dataset • Analysis to figure out engagement of each signal • Attempted ML approach ‣ Minimal overlap across different signals
  17. 17. 19 Design: Ensemble Approach • Step-wise unionization • Importance based on individual signal performance ‣ First, collaborative filter ‣ Second, queries correlated by query-result clicks ‣ Third, queries overlapping terms
  18. 18. 20 Design: Practical Considerations • System designed for public consumption ‣ Strong profanity filters ‣ Need to deal with misspellings ‣ Languages ‣ Remove spammy search queries
  19. 19. 21 Related Searches • Design • Implementation • Evaluation
  20. 20. 22 Implementation Challenge • Scale ‣ 175M+ members ‣ Billions of searches ‣ Terabytes of data to process
  21. 21. Implementation • Kafka: publish-subscribe messaging system • Hadoop: MapReduce data processing system • Azkaban: Hadoop workflow management tool • Voldemort: Key-value store
  22. 22. Implementation: Workflow
  23. 23. 29 Related Searches • Design • Implementation • Evaluation
  24. 24. 30 Evaluation • Performance of each signal and combination • How does the system scale?
  25. 25. 31 Evaluation Cont’d • Offline evaluation ‣ Precision-Recall • Online evaluation ‣ A/B testing to measure engagement ‣ Performance evaluation
  26. 26. 32 Offline Evaluation • Correct set: set of searches performed by a user in the following K minutes, here K=10
  27. 27. 33 Online Evaluation • Used A/B testing • Metrics ‣ Coverage: queries with recommendations ‣ Impressions: # of recommendations shown ‣ Clicks: Clicks on recommendations ‣ Click-through rate (CTR): Clicks per impression
  28. 28. Online Evaluation
  29. 29. 35 Evaluation: System Runtime
  30. 30. 36 Details • Metaphor: a System for Related Search Recommendations, Azarias Reda, Yubin Park, Mitul Tiwari, Christian Posse, and Sam Shah. In Proceedings of the CIKM, 2012.

Search plays an important role in online social networks as it provides an essential mechanism for discovering members and content on the network. Related search recommendation is one of several mechanisms used for improving members’ search experience in finding relevant results to their queries. This paper describes the design, implementation, and deployment of Metaphor, the related search recommendation system on LinkedIn, a professional social networking site with over 175 million members worldwide. Metaphor builds on a number of signals and filters that capture several dimensions of relatedness across member search activity. The system, which has been in live operation for over a year, has gone through multiple iterations and evaluation cycles. This paper makes three contributions. First, we provide a discussion of a large-scale related search recommendation system. Second, we describe a mechanism for effectively combining several signals in building a unified dataset for related search recommendations. Third, we introduce a query length model for capturing bias in recommendation click behavior. We also discuss some of the practical concerns in deploying related search recommendations.

Views

Total views

500

On Slideshare

0

From embeds

0

Number of embeds

8

Actions

Downloads

0

Shares

0

Comments

0

Likes

0

×