CLiMF: Learning to Maximize                  Reciprocal Rank with                Collaborative Less-is-More               ...
Top-k Recommendations                RecSys 2012, Dublin, Ireland, September 13, 2012                                     ...
Implicit feedback                    RecSys 2012, Dublin, Ireland, September 13, 2012                                     ...
Models for Implicit feedback•  Classification or Learning to Rank•  Binary pairwise ranking loss function (Hinge, AUC loss...
Learning to Rank in CF•  The Point-wise Approach      f (user, item) → R   •  Reduce Ranking to Regression, Classification...
Ranking metricsList-wise ranking measures for implicit feedback:                                         1Reciprocal Rank:...
Ranking metrics                      RR = 1                       AP = 1                  RecSys 2012, Dublin, Ireland, Se...
Ranking metrics                      RR = 1                   AP = 0.66                  RecSys 2012, Dublin, Ireland, Sep...
Less is more•  Focus at the very top of the list•  Try to get at least one interesting item at the top of the list•  MRR p...
Ingredients•  What kind of model should we use?    •  Factor model•  Which Ranking measure do we need to optimize to have ...
Model              +                      +         +           +               +             +           +         +     ...
The Non-smoothness of Reciprocal    Rank•  Reciprocal Rank (RR) of a ranked list of items for a given user       N       Y...
Non-smoothness     0.81   2     0.75   1     0.64   3Fi   0.61   4                         RR = 0.5     0.58   5     0.55 ...
Non-smoothness     0.84   2     0.82   1     0.56   3Fi   0.50   4                        RR = 0.5     0.45   5     0.40  ...
Non-smoothness     0.85   1     0.84   2     0.56   3Fi   0.50   4                           RR = 1     0.45   5     0.40 ...
RecSys 2012, Dublin, Ireland, September 13, 2012                                                   16
How can we get a smooth-MRR?•  Borrow techniques from learning-to-rank:      I(Rik  Rij ) ≈ g(fik − fij )       1         ...
MRR Loss function        N                           N                                                                    ...
MRR loss function II  •  Use concavity and monotonicity of log function,               N                                  ...
Optimization                     M N                                         E(U, V ) =              Yij ln g(UiT Vj )    ...
What’s different ?•  CLiMF reciprocal rank loss essentially pushes relevant items   apart•  In the process at least one it...
Conventional loss                    RecSys 2012, Dublin, Ireland, September 13, 2012                                     ...
CLiMF MRR-loss                 RecSys 2012, Dublin, Ireland, September 13, 2012                                           ...
Experimental Evaluation         Data sets•  Epinions :    •  346K observations of trust relationship    •  1767 users; 492...
Experimental Evaluation         Data sets•  Tuenti :    •  798K observations of trust relationship    •  11392 users; 5000...
Experimental Evaluation           Experimental Protocol       Given 5       Friends/                                      ...
Experimental Evaluation           Experimental Protocol           Given 10       Friends/                                 ...
Experimental Evaluation           Experimental Protocol                    Given 15       Friends/                        ...
Experimental Evaluation           Experimental Protocol                     Given 20       Friends/                       ...
Experimental Evaluation   Evaluation Metrics            |S|        1   1M RR =       |S| i=1 rankiP @5                  ra...
Experimental Evaluation    Scalability                  RecSys 2012, Dublin, Ireland, September 13, 2012                  ...
Experimental Evaluation      Competition •  Pop: Naive, recommend based on popularity of each item •  iMF (Hu and Koren, I...
Epinions MRR      0.4                Pop        iMF             BPR−MF           CLiMF      0.3MRR      0.2      0.1      ...
Epinions P@5      0.30                 Pop        iMF            BPR−MF           CLiMF      0.25      0.20P@5      0.15  ...
Epinions 1−call@5           0.8                     Pop         iMF             BPR−MF           CLiMF           0.61−call...
Tuenti MRR      0.20                 Pop           iMF             BPR−MF           CLiMF      0.15MRR      0.10      0.05...
Tuenti P@5      0.10                 Pop           iMF              BPR−MF           CLiMF      0.08      0.06P@5      0.0...
Tuenti 1−call@5           0.30                      Pop        iMF             BPR−MF           CLiMF           0.25      ...
Conclusions and Future Work•  Contribution   •  Novel method for implicit data with some nice properties (Top-k, speed)•  ...
Thank you !Telefonica Research is looking for interns!        Contact: alexk@tid.es or linas@tid.es                       ...
Upcoming SlideShare
Loading in …5
×

CLiMF: Collaborative Less-is-More Filtering

2,689 views

Published on

RecSys presentation slides of CLiMF,
a Collaborative Filtering algorithm based on a novel ranking algorithm

Published in: Technology, Travel, Business
0 Comments
9 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,689
On SlideShare
0
From Embeds
0
Number of Embeds
31
Actions
Shares
0
Downloads
135
Comments
0
Likes
9
Embeds 0
No embeds

No notes for slide

CLiMF: Collaborative Less-is-More Filtering

  1. 1. CLiMF: Learning to Maximize Reciprocal Rank with Collaborative Less-is-More FilteringYue Shia, Alexandros Karatzogloub (Presenter), Linas Baltrunasb, MarthaLarsona, Alan Hanjalica, Nuria OliverbaDelft University of Technology, NetherlandsbTelefonica Research, Spain RecSys 2012, Dublin, Ireland, September 13, 2012 1
  2. 2. Top-k Recommendations RecSys 2012, Dublin, Ireland, September 13, 2012 2
  3. 3. Implicit feedback RecSys 2012, Dublin, Ireland, September 13, 2012 3
  4. 4. Models for Implicit feedback•  Classification or Learning to Rank•  Binary pairwise ranking loss function (Hinge, AUC loss)•  Sample from non-observed/irrelevant entries Friend Not a Friend RecSys 2012, Dublin, Ireland, September 13, 2012 4
  5. 5. Learning to Rank in CF•  The Point-wise Approach f (user, item) → R •  Reduce Ranking to Regression, Classification, or Ordinal Regression problem, [OrdRec@Recys 2011]•  The Pairwise Approach f (user, item1 , item2 ) → R •  Reduce Ranking to pair-wise classification [BPR@UAI 2010]•  List-wise Approach f(user, item1 , . . . , itemn ) → R •  Direct optimization of IR measures, List-wise loss minimization [CoFiRank@NIPS 2008] RecSys 2012, Dublin, Ireland, September 13, 2012 5
  6. 6. Ranking metricsList-wise ranking measures for implicit feedback: 1Reciprocal Rank: RR = ranki |S| Average precision: P (k) k=1 [TFMAP@SIGIR2012] AP = |S| RecSys 2012, Dublin, Ireland, September 13, 2012 6
  7. 7. Ranking metrics RR = 1 AP = 1 RecSys 2012, Dublin, Ireland, September 13, 2012 7
  8. 8. Ranking metrics RR = 1 AP = 0.66 RecSys 2012, Dublin, Ireland, September 13, 2012 8
  9. 9. Less is more•  Focus at the very top of the list•  Try to get at least one interesting item at the top of the list•  MRR particularly important measure in domains that usually provide users with only few recommendations, i.e. Top-3 or Top-5 RecSys 2012, Dublin, Ireland, September 13, 2012 9
  10. 10. Ingredients•  What kind of model should we use? •  Factor model•  Which Ranking measure do we need to optimize to have a good Top-k recommender? •  MRR captures the quality of Top-k recommendations•  But MRR is not smooth so what can we do? •  We can perhaps find a smooth version of MRR•  How to ensure the proposed solution scalable? •  A fast learning algorithm (SGD), smoothness - gradients RecSys 2012, Dublin, Ireland, September 13, 2012 10
  11. 11. Model + + + + + + + + + fij = Ui , Vj RecSys 2012, Dublin, Ireland, September 13, 2012 11
  12. 12. The Non-smoothness of Reciprocal Rank•  Reciprocal Rank (RR) of a ranked list of items for a given user N Yij NRRi = (1 − Yik I(Rik Rij )) j=1 Rij k=1 RecSys 2012, Dublin, Ireland, September 13, 2012 12
  13. 13. Non-smoothness 0.81 2 0.75 1 0.64 3Fi 0.61 4 RR = 0.5 0.58 5 0.55 6 0.49 7 0.43 8 RecSys 2012, Dublin, Ireland, September 13, 2012 13
  14. 14. Non-smoothness 0.84 2 0.82 1 0.56 3Fi 0.50 4 RR = 0.5 0.45 5 0.40 6 0.32 7 0.31 8 RecSys 2012, Dublin, Ireland, September 13, 2012 14
  15. 15. Non-smoothness 0.85 1 0.84 2 0.56 3Fi 0.50 4 RR = 1 0.45 5 0.40 6 0.32 7 0.31 8 RecSys 2012, Dublin, Ireland, September 13, 2012 15
  16. 16. RecSys 2012, Dublin, Ireland, September 13, 2012 16
  17. 17. How can we get a smooth-MRR?•  Borrow techniques from learning-to-rank: I(Rik Rij ) ≈ g(fik − fij ) 1 ≈ g(fij ) Rij g(x) = 1/(1 + e−x ) RecSys 2012, Dublin, Ireland, September 13, 2012 17
  18. 18. MRR Loss function N N RRi ≈ Yij g(fij ) 1 − Yik g(fik − fij ) j=1 k=1fij = Ui , Vj Ui , V = arg max{RRi } Ui ,V 2 O(N ) RecSys 2012, Dublin, Ireland, September 13, 2012 18
  19. 19. MRR loss function II •  Use concavity and monotonicity of log function, N N L(Ui , V ) = Yij ln g(fij ) + ln 1 − Yik g(fik − fij ) j=1 k=1 +2 O(n ) RecSys 2012, Dublin, Ireland, September 13, 2012 19
  20. 20. Optimization M N E(U, V ) = Yij ln g(UiT Vj ) i=1 j=1 N + ln 1 − Yik g(UiT Vk − UiT Vj ) k=1 λ − (U 2 + V 2 ) 2 ∂E ∂E•  Objective is smooth we can compute: ∂Ui ∂Vj•  We use Stochastic Gradient Descent•  Overall scalability linear to # of relevant items O(dS) RecSys 2012, Dublin, Ireland, September 13, 2012 20
  21. 21. What’s different ?•  CLiMF reciprocal rank loss essentially pushes relevant items apart•  In the process at least one items ends up high-up in the list RecSys 2012, Dublin, Ireland, September 13, 2012 21
  22. 22. Conventional loss RecSys 2012, Dublin, Ireland, September 13, 2012 22
  23. 23. CLiMF MRR-loss RecSys 2012, Dublin, Ireland, September 13, 2012 23
  24. 24. Experimental Evaluation Data sets•  Epinions : •  346K observations of trust relationship •  1767 users; 49288 trustees •  99.85% Sparseness •  Avg. friends/trustees per user 73.34 RecSys 2012, Dublin, Ireland, September 13, 2012 24
  25. 25. Experimental Evaluation Data sets•  Tuenti : •  798K observations of trust relationship •  11392 users; 50000 friends •  99.86% Sparseness •  Avg. friends/trustees per user 70.06 RecSys 2012, Dublin, Ireland, September 13, 2012 25
  26. 26. Experimental Evaluation Experimental Protocol Given 5 Friends/ Friends/ Trustees Trusteesdata Test data (Items holdout) Training data RecSys 2012, Dublin, Ireland, September 13, 2012 26
  27. 27. Experimental Evaluation Experimental Protocol Given 10 Friends/ Friends/ Trustees Trusteesdata Test data (Items holdout) Training data RecSys 2012, Dublin, Ireland, September 13, 2012 27
  28. 28. Experimental Evaluation Experimental Protocol Given 15 Friends/ Friends/ Trustees Trusteesdata Test data (Items holdout) Training data RecSys 2012, Dublin, Ireland, September 13, 2012 28
  29. 29. Experimental Evaluation Experimental Protocol Given 20 Friends/ Friends/ Trustees Trusteesdata Test data (Items holdout) Training data RecSys 2012, Dublin, Ireland, September 13, 2012 29
  30. 30. Experimental Evaluation Evaluation Metrics |S| 1 1M RR = |S| i=1 rankiP @5 ratio of test users who have at1 − call@5 least one relevant item in their Top-5 RecSys 2012, Dublin, Ireland, September 13, 2012 30
  31. 31. Experimental Evaluation Scalability RecSys 2012, Dublin, Ireland, September 13, 2012 31
  32. 32. Experimental Evaluation Competition •  Pop: Naive, recommend based on popularity of each item •  iMF (Hu and Koren, ICDM 08): Optimizes Squared error loss •  BPR-MF (Rendle et al., UAI 09): Optimizes AUC RecSys 2012, Dublin, Ireland, September 13, 2012 32
  33. 33. Epinions MRR 0.4 Pop iMF BPR−MF CLiMF 0.3MRR 0.2 0.1 0.0 5 10 15 20 RecSys 2012, Dublin, Ireland, September 13, 2012 33
  34. 34. Epinions P@5 0.30 Pop iMF BPR−MF CLiMF 0.25 0.20P@5 0.15 0.10 0.05 0.00 5 10 15 20 RecSys 2012, Dublin, Ireland, September 13, 2012 34
  35. 35. Epinions 1−call@5 0.8 Pop iMF BPR−MF CLiMF 0.61−call@5 0.4 0.2 0.0 5 10 15 20 RecSys 2012, Dublin, Ireland, September 13, 2012 35
  36. 36. Tuenti MRR 0.20 Pop iMF BPR−MF CLiMF 0.15MRR 0.10 0.05 0.00 5 10 15 20 RecSys 2012, Dublin, Ireland, September 13, 2012 36
  37. 37. Tuenti P@5 0.10 Pop iMF BPR−MF CLiMF 0.08 0.06P@5 0.04 0.02 0.00 5 10 15 20 RecSys 2012, Dublin, Ireland, September 13, 2012 37
  38. 38. Tuenti 1−call@5 0.30 Pop iMF BPR−MF CLiMF 0.25 0.201−call@5 0.15 0.10 0.05 0.00 5 10 15 20 RecSys 2012, Dublin, Ireland, September 13, 2012 38
  39. 39. Conclusions and Future Work•  Contribution •  Novel method for implicit data with some nice properties (Top-k, speed)•  Future work •  Use CLiMF to avoid duplicate or very similar recommendations in the top-k part of the list •  To optimize other evaluation metrics for top-k recommendation •  To take the social network of users into account RecSys 2012, Dublin, Ireland, September 13, 2012 39
  40. 40. Thank you !Telefonica Research is looking for interns! Contact: alexk@tid.es or linas@tid.es RecSys 2012, Dublin, Ireland, September 13, 2012 40

×