Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Multiple Objectives in Collaborative Filtering
Tamas Jambor and Jun Wang
University College London
Structure of the talk
• Motivation
• Multiple objectives
• User perspective
– Promoting less popular items
• System perspe...
Motivation
• In the RecSys community, many research efforts
are focused on recommendation accuracy
• And yet accuracy is n...
Improved Accuracy != Improved User experience
Algorithm
Additional
factors
Available resources
Cost of delivery
User inter...
Improved user experience
Available resources
Cost of delivery
User interface
Diverse choices
Profitability per item
Advert...
Handling Multiple objectives
• Accuracy is the main objective
– Defined in the baseline algorithm
• User perspective
– Def...
Where to optimize?
• In the objective function or as a post-filter?
• Post-filters have the advantage to
– Add to any base...
The proposed optimization framework
(for each user)
• Add additional constraints of w
0
11:tosubject
ˆmax
w
w
rw
T
T
w

Properties of the framework
• Linear optimization problem
• Recommendation as a ranking problem
• Constraints provide the ...
User case – Promoting the Long Tail
Current systems are biased towards popular items
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4...
Promoting the Long Tail
• Does that reflect real user needs?
• Popular items might not be interesting for the user
• Disco...
Promoting the Long Tail
• Extending the optimization framework
0
11
:tosubject
ˆmax
w
w
mwm
rw
T
u
T
T
w

 
Promoting the Long Tail and Diversification
• Diversifying the results
0
11
:tosubject
ˆmax
w
w
mwm
wwrw
T
u
T
TT
w


...
Diversification
• Increase the covariance between recommended
items
– Reduce the risk of expanding the system
– Provide a ...
Experimental setup
• MovieLens 1m dataset
• 3900 movies, 6040 users
• Five-fold cross validation
Evaluation metrics
• Recommendation as a ranking problem
• IR measures
– Normalized discounted cumulative gain (NDCG)
– Pr...
Results: Promoting the Long Tail
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
1 2 3 4 5 6 7 8 9 10
Probabilityofanyof100...
Results: Promoting the Long Tail
Baseline (SVD) LTC LTC and Div (λ=6)
NDCG@10 0.8808 0.8780 (-0.3%) 0.8715 (-1.0%)
P@10 0....
System case – Resource Constraint
• Introducing external factors to the system
• Stock availability of recommended items
•...
Simulation
• Online DVD-Rental company
– Operates a warehouse
– Only a limited number of items are available
• Recommend i...
Simulation
• User choice is based purely on recommendation
• Simulating the stock level for 50 days
– Present a list of it...
Cut-off point
• Threshold c controls the cut-off point from which
the system starts re-ranking items


 


cp
cps
s ...
Resource Constraint
• Extending the optimization framework
0
11
:tosubject
ˆmax


w
w
sws
rw
T
u
T
w


Evaluation: Monitoring the waiting list size
• Waiting list
– If item is not in stock, user puts it on their waiting list
...
Results: Resource Constraint
0
20
40
60
80
100
120
140
160
180
0 5 10 15 20
Numberofitemsonthewaitinglist
Time (days)
base...
Results: Resource Constraint
• Trade-off between low waiting list size and good
performance
0.75
0.77
0.79
0.81
0.83
0.85
...
Results: Resource Constraint
c=0 c=0.4 c=0.8 c=1.2 c=1.6
NDCG@3(mean) -12.3% -4.32% -1.03% -0.43% -0.13%
NDCG@3(max) -14.7...
Conclusion
• Recommender systems have multiple objectives
• Multiple optimization framework
– Expand the system with minor...
Future plan
• Personalized digital content delivery
– Reduce delivery cost
• Diversification and the long tail
– Does reco...
Thank you.
References
• Deshpande, M., Karypis, G.: Item-based top-N recommendation algorithms.
ACM Trans. Inf. Syst. 22(1) (2004)
• ...
Upcoming SlideShare
Loading in …5
×

Multiple objectives in Collaborative Filtering (RecSys 2010)

1,190 views

Published on

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

Multiple objectives in Collaborative Filtering (RecSys 2010)

  1. 1. Multiple Objectives in Collaborative Filtering Tamas Jambor and Jun Wang University College London
  2. 2. Structure of the talk • Motivation • Multiple objectives • User perspective – Promoting less popular items • System perspective – Stock management
  3. 3. Motivation • In the RecSys community, many research efforts are focused on recommendation accuracy • And yet accuracy is not a only concern • Practical recommender systems might have multiple goals
  4. 4. Improved Accuracy != Improved User experience Algorithm Additional factors Available resources Cost of delivery User interface Diverse choices Profitability per item Advertisement
  5. 5. Improved user experience Available resources Cost of delivery User interface Diverse choices Profitability per item Advertisement Additional factors Accuracy Improved user experience
  6. 6. Handling Multiple objectives • Accuracy is the main objective – Defined in the baseline algorithm • User perspective – Define and consider user satisfaction as priority • System perspective – Consider additional system related objectives • Objectives of the system might contradict
  7. 7. Where to optimize? • In the objective function or as a post-filter? • Post-filters have the advantage to – Add to any baseline algorithm – Extend easily – Add multiple goals
  8. 8. The proposed optimization framework (for each user) • Add additional constraints of w 0 11:tosubject ˆmax w w rw T T w 
  9. 9. Properties of the framework • Linear optimization problem • Recommendation as a ranking problem • Constraints provide the means of biasing the ranking
  10. 10. User case – Promoting the Long Tail Current systems are biased towards popular items 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 1 2 3 4 5 6 7 8 9 10 Probabilityofanyof100mostpopularitembeingat rankingposition Ranking Position SVD User-based Item-based Random Sample
  11. 11. Promoting the Long Tail • Does that reflect real user needs? • Popular items might not be interesting for the user • Discovering unknown item could be more valuable • The aim is to reduce recommending popular items – if the user is likely to be an interested in alternative choices – keep recommending popular items otherwise
  12. 12. Promoting the Long Tail • Extending the optimization framework 0 11 :tosubject ˆmax w w mwm rw T u T T w   
  13. 13. Promoting the Long Tail and Diversification • Diversifying the results 0 11 :tosubject ˆmax w w mwm wwrw T u T TT w     
  14. 14. Diversification • Increase the covariance between recommended items – Reduce the risk of expanding the system – Provide a wider range of choice
  15. 15. Experimental setup • MovieLens 1m dataset • 3900 movies, 6040 users • Five-fold cross validation
  16. 16. Evaluation metrics • Recommendation as a ranking problem • IR measures – Normalized discounted cumulative gain (NDCG) – Precision – Mean reciprocal rank (MRR) • Constraint specific measures
  17. 17. Results: Promoting the Long Tail 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 1 2 3 4 5 6 7 8 9 10 Probabilityofanyof100mostpopularitembeingat rankingposition Ranking Position Baseline (SVD) Long Tail Constraint Long Tail Constraint and Diversification (λ=6) Random Sample
  18. 18. Results: Promoting the Long Tail Baseline (SVD) LTC LTC and Div (λ=6) NDCG@10 0.8808 0.8780 (-0.3%) 0.8715 (-1.0%) P@10 0.8204 0.8207 (+0.2%) 0.8177 (-0.3%) MRR 0.9518 0.9453 (-0.6%) 0.9349 (-1.7%)
  19. 19. System case – Resource Constraint • Introducing external factors to the system • Stock availability of recommended items • The aim is to rank items lower, if less of them are available • Minimizing performance loss
  20. 20. Simulation • Online DVD-Rental company – Operates a warehouse – Only a limited number of items are available • Recommend items that are in stock higher in the ranking list
  21. 21. Simulation • User choice is based purely on recommendation • Simulating the stock level for 50 days – Present a list of items to a random number of users – The probability that the item is taken depends on the rank – Cumulative probability depends on how many times the item was shown and at which rank position
  22. 22. Cut-off point • Threshold c controls the cut-off point from which the system starts re-ranking items       cp cps s ti titi ti , ,, , if0 if
  23. 23. Resource Constraint • Extending the optimization framework 0 11 :tosubject ˆmax   w w sws rw T u T w  
  24. 24. Evaluation: Monitoring the waiting list size • Waiting list – If item is not in stock, user puts it on their waiting list – When item returns, it goes out to the next user • Waiting list size represents how long a user has to wait to get their favourite items
  25. 25. Results: Resource Constraint 0 20 40 60 80 100 120 140 160 180 0 5 10 15 20 Numberofitemsonthewaitinglist Time (days) baseline c=1.6 c=1.2 c=0.0
  26. 26. Results: Resource Constraint • Trade-off between low waiting list size and good performance 0.75 0.77 0.79 0.81 0.83 0.85 0.87 0.89 0.91 0 5 10 15 20 NDCG@3 Time (days) baseline c=1.6 c=1.2 c=0.0
  27. 27. Results: Resource Constraint c=0 c=0.4 c=0.8 c=1.2 c=1.6 NDCG@3(mean) -12.3% -4.32% -1.03% -0.43% -0.13% NDCG@3(max) -14.7% -5.12% -1.34% -0.56% -0.50% P@10(mean) -6.42% -3.37% -0.86% -0.06% -0.03% P@10(max) -8.42% -3.91% -1.11% -0.24% -0.18% Performance loss over 50 days
  28. 28. Conclusion • Recommender systems have multiple objectives • Multiple optimization framework – Expand the system with minor performance loss – It is designed to add objectives flexibly – It can be added to any recommender system • Two scenarios that offer practical solutions – Long-tail items – Stock simulation
  29. 29. Future plan • Personalized digital content delivery – Reduce delivery cost • Diversification and the long tail – Does recommendation kill diversity? • Evaluate improved user experience – User studies
  30. 30. Thank you.
  31. 31. References • Deshpande, M., Karypis, G.: Item-based top-N recommendation algorithms. ACM Trans. Inf. Syst. 22(1) (2004) • Herlocker, J.L., Konstan, J.A., Borchers, A., Riedl, J.: An algorithmic framework for performing collaborative filtering. In: SIGIR '99. (1999) • Koren, Y., Bell, R., Volinsky, C.: Matrix factorization techniques for recommender systems. Computer 42(8) (2009) • Wang, J., de Vries, A.P., Reinders, M.J.T.: Unifying user-based and item- based collaborative filtering approaches by similarity fusion. In: SIGIR '06: Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval, New York, NY, ACM Press

×