Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.

Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.

Like this presentation? Why not share!

955 views

849 views

849 views

Published on

RecSys 2012 Dublin Conference Slides - Multiple Objective Optimization in Recommender Systems

Published in:
Data & Analytics

No Downloads

Total views

955

On SlideShare

0

From Embeds

0

Number of Embeds

79

Shares

0

Downloads

0

Comments

0

Likes

4

No embeds

No notes for slide

- 1. Mario Rodriguez Christian Posse Ethan Zhang MARIO Multiple Objective Optimization in Recommender Systems
- 2. Motivation 1. Value of a recommender system given by its multi-faceted utility function: utility = fn(relevance, engagement, …) 2. We want to efficiently improve the utility of the system by focusing on the most promising facet(s)
- 3. Outline • TalentMatch case study oOverview oUtility function – Multiple Objectives! oApproach details • Problem formulation & Optimization oA/B test results
- 4. Talent Match TalentMatch Job Posting Member Profiles Ranked Talent
- 5. Job Posting
- 6. Member Profile
- 7. Job Posting title geo company industry description functional area … Candidate General expertise specialties education headline geo experience Current Position title summary tenure length industry functional area … Matching Transition probability Cosine similarity … TalentMatch Model
- 8. TalentMatch Teaser Snippet
- 9. TalentMatch Utility = fn(booking rate, email rate, reply rate) Booking Rate
- 10. TalentMatch Utility = fn(booking rate, email rate, reply rate) Booking Rate Email Rate
- 11. TalentMatch Utility = fn(booking rate, email rate, reply rate) Booking Rate Email Rate Reply Rate
- 12. TalentMatch Utility = fn(booking rate, email rate, reply rate) Booking Rate Email Rate Reply Rate Problem!
- 13. TalentMatch Utility = fn(booking rate, email rate, reply rate) Booking Rate Email Rate Reply Rate Problem! Job seeker?
- 14. Flightmeter: Job Seeker Intent Model • Propensity Score o p(switch jobs in next month) • Model o Survival Analysis of Positions o Accelerated failure time (AFT) model log Ti = Σkβkxik+σεi ACTIVE PASSIVE NON-JOB- SEEKER
- 15. Flightmeter Feature Example: Industry Attrition Probability Time
- 16. Flightmeter: actives & passives 16x reply rate on career-related mail Reply Rate What: Increase TalentMatch Utility fn(booking rate, email rate, reply rate)
- 17. Job Posting title geo company industry description functional area … Candidate General expertise specialties education headline geo experience Current Position title summary tenure length industry functional area … Matching Transition probability Cosine similarity Flightmeter Flightmeter as Another Feature?
- 18. Job Posting title geo company industry description functional area … Candidate General expertise specialties education headline geo experience Current Position title summary tenure length industry functional area … Matching Transition probability Cosine similarity Flightmeter Flightmeter as Another Feature?
- 19. Match Score Histogram of 12th Rank t
- 20. Talent Match ranking Match Score 1, Item X, 0.98, Non-Seeker 2, Item Y, 0.91, Non-Seeker --------------------------------------- 3, Item Z, 0.89, Active Perturbed ranking Match Score, Perturbed Score 1, Item X, 0.98, 0.98, Non-Seeker 2, Item Z, 0.89, 0.93, Active ------------------------------------------------ 3, Item Y, 0.91, 0.91, Non-Seeker Perturbation Function f() Divergence Function Δ() Divergence score Objective Function g() Objective score How: Controlled Ranking Perturbation Match Score Distributions
- 21. Talent Match ranking Match Score 1, Item X, 0.98, Non-Seeker 2, Item Y, 0.91, Non-Seeker --------------------------------------- 3, Item Z, 0.89, Active Perturbed ranking Match Score, Perturbed Score 1, Item X, 0.98, 0.98, Non-Seeker 2, Item Z, 0.89, 0.93, Active ------------------------------------------------ 3, Item Y, 0.91, 0.91, Non-Seeker Perturbation Function f() Divergence Function Δ() Divergence score Objective Function g() Objective score How: Controlled Ranking Perturbation Match Score Distributions
- 22. Talent Match ranking Match Score 1, Item X, 0.98, Non-Seeker 2, Item Y, 0.91, Non-Seeker --------------------------------------- 3, Item Z, 0.89, Active Perturbed ranking Match Score, Perturbed Score 1, Item X, 0.98, 0.98, Non-Seeker 2, Item Z, 0.89, 0.93, Active ------------------------------------------------ 3, Item Y, 0.91, 0.91, Non-Seeker Perturbation Function f() Divergence Function Δ() Divergence score Objective Function g() Objective score How: Controlled Ranking Perturbation Match Score Distributions
- 23. Problem Formulation • Perturbation Function • Divergence Function • Objective Function
- 24. Problem Formulation • Perturbation Function • Divergence Function • Objective Function TalentMatch Score
- 25. Finding a Good Perturbation Function • Loss Function • Objective and divergence depend on a sort/rank, so gradient-based optimization not directly applicable • Lambda value?
- 26. Pareto Optimization
- 27. Pareto Optimization
- 28. Match Score Histogram Divergence 0 27 54 100
- 29. Computational Approaches • Grid Search • Gradient-based techniques
- 30. Gradient-based Techniques λ = 0.076 …0.076 > λ > 0
- 31. Gradient-based Techniques, cont. • Smooth approximations to popular ranking metrics amenable to gradient-descent o Normalized Discounted Cumulative Gain (NDCG) o Average Precision (AP) • Re-frame the Multi-Objective Optimization problem using those approximations, and apply SmoothRank
- 32. Experiments • A/B Test o Treatment 1: 1.15 boost (8/12) o Treatment 2: 1.07 boost (6/12) o Control: 1.0 boost (4/12) • Expectations o 50% increase in reply rate for 1.07 boost o 100% increase in reply rate for 1.15 boost o Expected booking rate and email rate to remain unchanged or minimally affected
- 33. A/B Test Results (% increase over control) Booking rate α = β = 1.07 0% α = β = 1.15 -0.4% Email rate α = β = 1.07 31% α = β = 1.15 25% Reply rate α = β = 1.07 22% α = β = 1.15 42%
- 34. Conclusion • Consider the multiple facets of your system’s utility function to improve utility efficiently o Handle competing objectives carefully • Know your tradeoff(s)! o A/B test furiously
- 35. 2 4 8 17 32 55 90 2004 2005 2006 2007 2008 2009 2010 2011 LinkedIn Members (Millions) 175M+ 25thMost visit website worldwide (Comscore 6-12) Company pages >2M 62% non U.S. 2/sec 85% Fortune 500 Companies use LinkedIn to hire Thank You! We’re Hiring! mrodriguez@linkedin.com

No public clipboards found for this slide

×
### Save the most important slides with Clipping

Clipping is a handy way to collect and organize the most important slides from a presentation. You can keep your great finds in clipboards organized around topics.

Be the first to comment