View stunning SlideShares in full-screen with the new iOS app!Introducing SlideShare for AndroidExplore all your favorite topics in the SlideShare appGet the SlideShare app to Save for Later — even offline
View stunning SlideShares in full-screen with the new Android app!View stunning SlideShares in full-screen with the new iOS app!
Goal-driven Collaborative FilteringA Directional Error Based Approach Tamas Jambor and Jun Wang University College London
Structure of the talk• Background/Problem description• Goal-driven design• Experimental results• Conclusions
Collaborative filtering• Predicting user preference towards unknown items• Based on previously expressed preferences Love Pulp Crazy White Up in A Single Actually Fiction Heart Ribbon the Air Man Sophie ? ? ? Peter ? ? ? Jaden ? ? ?
Evaluation metrics ˆ• Root Mean Squared Error E (( r r ) 2 )• Netflix recommendation competition adopted this metric• The objective function for some of the SVD implementations is equivalent to the performance measure [Koren et al 2009]• Criticism – Error criterion is uniform across rating scales – Is it consistent with users’ satisfactions?
Goal-driven design• We argue that – Measure does not always reflect user needs – Different user needs require different performance measures• The algorithm should be defined based on user needs – Start from the user point of view, define measure and algorithm accordingly
The two dimensional weighting function r = 1,2 r=3 r = 4,5 p <= 2.5 w1 w2 w3 2.5<p<=3.5 w4 w5 w6 P > 3.5 w7 w8 w9
Two-stage Optimization (in General) Learning the Directional Errors Feedback/IR Learning the Metrics Recom. Model Testing
Two-stage Optimization (An example) Genetic algorithm NDCG as fitness function Plug in the learned Weights in SVD Training T 2 2 2 argmin w(rui q pu ) i ( qi pu ) q, p ui
Genetic algorithms• Search algorithms that work via the process of natural selection• Start with a sample set of potential solutions (a set of weights)• Evolve towards a set of more optimal solutions• Poor solutions tend to die out (smaller NDCG)• Better solutions remain in the population (higher NDCG)
Experiments• MovieLens 100k dataset• 1862 movies, 943 users• Only using ratings• Five-fold cross validation
Evaluation metrics• Recommendation as a ranking problem• IR measures – Normalized discounted cumulative gain (NDCG) – Mean average precision (MAP) – Mean reciprocal rank (MRR)
Results – Experiment I Baseline SVD r = 1,2 r=3 r = 4,5 p <= 2.5 0.0517 0.0193 0.0106 2.5<p<=3.5 0.0904 0.1461 0.1391 p > 3.5 0.0299 0.1012 0.4115 SVD with weights where w7>w8>w4 r = 1,2 r=3 r = 4,5 p <= 2.5 0.0759 0.0407 0.0264 2.5<p<=3.5 0.0837 0.1676 0.2381 p > 3.5 0.0125 0.0583 0.2966
Results – Experiment II r = 1,2 r=3 r = 4,5 p <= 2.5 w1 w2 w3 2.5<p<=3.5 w4 w5 w6 P > 3.5 w7 w8 w9
Probability of correct prediction within sectorsProbability of predicting non-relevant items relevant
Improved user experience• More likely to receive relevant items on their recommendation list• Less likely that lower rated items receive higher predictions• But it is more likely that higher rated items receive lower predictions
Conclusion• Optimize algorithm from the user point of view• Identify directional errors• Assign risk to each direction• Approach can be changed depending on how items are presented
Future work• Taste boundaries might be user dependent• Directional error across items or users• Different recommender goals
References• Deshpande, M., Karypis, G.: Item-based top-N recommendation algorithms. ACM Trans. Inf. Syst. 22(1) (2004)• Herlocker, J.L., Konstan, J.A., Borchers, A., Riedl, J.: An algorithmic framework for performing collaborative filtering. In: SIGIR 99. (1999)• Koren, Y., Bell, R., Volinsky, C.: Matrix factorization techniques for recommender systems. Computer 42(8) (2009)• Wang, J., de Vries, A.P., Reinders, M.J.T.: Unifying user-based and item- based collaborative filtering approaches by similarity fusion. In: SIGIR 06: Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval, New York, NY, ACM Press