Telefonica Lunch Seminar

Slide 1: Exploring Recommender Systems: trust, graphs, and experts neal lathia telefonica i+d lunch seminar july 8, 2008

Slide 2: neal lathia (2nd yr phd): stephen hailes & licia capra department of computer science, university college london intern @ telefonica i+d: xavier amatriain

Slide 3: warning: what follows is a slide mash-up from multiple sources

Slide 4: utiforo project “pervasive support for market trading” what should I buy?

Slide 5: recommender systems source: O. Celma & P. Lamere “Music Recommendation Tutorial” ISMIR 2007

Slide 6: design methods collaborative filtering research  to recommend stuff

Slide 7: …a method to classify content correctly predicted intelligent  ratings data  process our focus: k-nearest neighbours (kNN)

Slide 8: kNN collaborative filtering ? 1. find neighbours 2. make predictions 3. recommend source: N. Lathia “Computing Recommendations with Collaborative Filtering” To appear, Collaborative and Social Information Retrieval and Access: Techniques for Improved User Modeling

Slide 9: who does the research? computer supported collaborative work machine learning human computer interaction trust e-commerce management agents artificial reputation systems communities mobile systems m-commerce distributed systems social networks

Slide 10: example? computer supported collaborative work machine learning human computer interaction trust e-commerce management agents artificial reputation systems communities mobile systems m-commerce distributed systems social networks

Slide 11: distributed content dissemination via trustworthy peers filter out the spammers source: D. Quercia, S. Hailes, L. Capra “Lightweight Distributed Trust Propagation” ICDM 2007

Slide 12: problem: who do you trust? spammer? uncooperative? thief?

Slide 13: bootstrap cooperation with privacy method: estimate transitive similarity evaluation: netflix dataset & simulated profiles source: N. Lathia, S. Hailes, L. Capra “Private Distributed Collaborative Filtering Using Estimated Concordance Measures” RecSys 2007, Minn, USA

Slide 14: effect on simulated profiles High Estimation Error high correlation estimation error, but prediction accuracy remains?

Slide 15: [2] [4] [3] [1]  my profile [1] [3] your profile  [5] [2] [3] [3] similarity values depend on the method used: there is no agreement between measures bad pearson -0.50 near zero weighted- pearson -0.05 good cosine angle 0.76 very good co-rated proportion 1.00 near zero concordance -0.06

Slide 16: Proportion (-1 .0 , 0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 (-0 -0.9 .9 5 ) , (-0 -0.8 .8 5 ) ,- (-0 0.7 .7 5 ) , (-0 -0.6 .6 5 ) , (-0 -0.5 .5 5 ) ,- (-0 0.4 .4 5 ) , (-0 -0.3 .3 5 ) , (-0 -0.2 .2 5 ) ,- (-0 0.1 .1 5 ) ,-0 ( 0 .0 5 ) .0 ,0 .0 (0 process .1 5) ,0 intelligent Range ( 0 .1 5 ) .2 Pearson Distribution ,0 ( 0 .2 5 ) .3 ,0 .3 (0 .4 5) ,0 ( 0 .4 5 ) .5 ,0 ( 0 .5 5 ) .6 ,0 .6 (0 …the pearson distribution .7 5) ,0  ( 0 .7 5 ) .8 ,0 .8 (0 .9 5) ,0 .9 5)

Slide 17: Proportion (-1 .0 , 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 (-0 -0.9 .9 5 ) , (-0 -0.8 .8 5 ) , (-0 -0.7 .7 5 ) ,- (-0 0.6 .6 5 ) , (-0 -0.5 .5 5 ) , (-0 -0.4 .4 5 ) ,- (-0 0.3 .3 5 ) , (-0 -0.2 .2 5 ) , (-0 -0.1 .1 5 ) ,-0 weighted-PCC, constrained-PCC ( 0 .0 5 ) .0 ,0 Weighted-PCC .0 (0 .1 5) ,0 Range ( 0 .1 5 ) .2 ,0 ( 0 .2 5 ) .3 ,0 Modified Pearson Distributions .3 (0 .4 5) ,0 ( 0 .4 5 Constrained-PCC ) .5 ,0 .5 (0 .6 5) ,0 ( 0 .6 5 ) .7 ,0 .7 (0 .8 5) ,0 ( 0 .8 5 ) .9 ,0 .9 …the modified pearson distributions 5)

Slide 18: Proportion (-1 .0 , (-0 -0.9 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 .9 5 ) , (-0 -0.8 .8 5 ) , (-0 -0.7 .7 5 ) , (-0 -0.6 .6 5 ) , (-0 -0.5 .5 5 ) , (-0 -0.4 .4 5 ) , (-0 -0.3 .3 5 ) , (-0 -0.2 .2 5 ) , (-0 -0.1 .1 5 ) 5 (-0 ,-0. somers’ d, co-rated, cosine angle .0 1 ) 5 Co-Rated (0 ,0.0 .0 process 5, ) ( 0 0 .1 intelligent ) Range .1 Other Distributions 5, 0. (0 Somers .2 2) 5, ( 0 0 .3 ) .3 VSS 5, 0. (0 .4 4) 5, ( 0 0 .5 ) .5 …and other measures 5, 0. (0 .6 6) 5, ( 0 0 .7 ) .7  5, 0. (0 .8 8) 5, ( 0 0 .9 ) .9 5, 1. 0)

Slide 19: …what if..? java.util.Random r = new java.util.Random() for all neighbours i { similarity(i) = (r.nextDouble()*2.0)-1.0); } source: N. Lathia, S. Hailes, L. Capra “The Effect of Correlation Coefficients on Communities of Recommenders” ACM SAC TRECK 2008, Fortaleza, Brazil.

Slide 20: r  p a ,i accuracy  MAE  a ,i N Neighborh ood Co Rated Somers’ d PCC wPCC R(0.5, 1.0) R(-1.0, 1.0) Constant(1.0) 0.9449 0.9492 1.1150 0.9596 1.0665 1.0406 1.0341 1 0.9595 0.9689 10 0.8498 0.8355 1.0455 0.8277 0.9495 0.8903 0.8848 30 0.7979 0.7931 0.9464 0.7847 0.9108 0.8584 0.8498 50 0.7852 0.7817 0.9007 0.7733 0.8922 0.8222 0.8153 100 0.7759 0.7728 0.8136 0.7647 0.8511 0.8053 0.8024 153 0.7726 0.7727 0.7817 0.7638 0.8243 0.7919 0.8058 229 0.7717 0.7771 0.7716 0.7679 0.7992 459 0.7718 0.7992 0.8073 0.8025 0.7773 0.7769 0.7811

Slide 21: a) our error measures are not good enough?

Slide 22: a) our error measures are not good enough? b) is there something wrong with the dataset?

Slide 23: a) our error measures are not good enough? ] T S b) is there something wrong with the dataset? U R T O c) current user-similarity is not strong enough to [N capture the best recommender relationships?

Slide 24: trust enables interaction in environments that are characterised by uncertainty read more: N. Lathia “Learning to Trust on the Move” International Workshop on Trust in Mobile Environments (TIME). IFIPTM, June 2008, Trondheim, Norway.

Slide 25: weight users based on how like-minded you have been to them in the past (similarity) to: weight users based on the quality of the opinions you receive from them (trust) filtering is a trust problem

Slide 26: 1. find neighbours 2. make predictions trust risk engine engine 4. get feedback 3. recommend source: N. Lathia, S. Hailes, L. Capra “Trust-Based Collaborative Filtering” Joint iTrust and PST Conference (IFIPTM), June 2008, Trondheim, Norway.

Slide 27: (trust-based) select valuable neighbours 1 value(a, b, i )  ra ,i  rb ,i  1 5

Slide 28: (trust-based) select valuable neighbours 1 1 0.9 0.8 value(a, b, i )  ra ,i  rb ,i  1 0.7 5 0.6 Value 0.5 0.4 0.3 0.2 0.1 0 0 1 2 3 4 5 Difference

Slide 29: (trust-based) select valuable neighbours  value(a, b, i) a-symmetric trust (a, b)  i value-added N reward information

Slide 30: you 1* 2* 3* 4* 5* 1* 1 2 5 3 1 2* 0 2 11 2 2 me 3* 0 1 13 26 5 4* 1 1 10 30 19 5* 0 0 4 18 26 (movielens movie-rating data example)

Slide 31: you 1* 2* 3* 4* 5* 1* 1 2 5 3 1 2* 0 2 11 2 2 me 3* 0 1 13 26 5 4* 1 1 10 30 19 5* 0 0 4 18 26 (movielens movie-rating data example)

Slide 32: you 1* 2* 3* 4* 5* 1* 1 2 5 3 1 2* 0 2 11 2 2 me 3* 0 1 13 26 5 4* 1 1 10 30 19 5* 0 0 4 18 26 can we learn to interpret neighbour recommendations?

Slide 33: (rating transpose) 5 4 you 3 2 1 1 2 3 4 5 me “do what I mean, not what I say” semantic distance

Slide 34: extensions? no profile? trust bootstrapping [see: Massa et al]    value(a, b, i ) trust (a, b)  i N 1 uncertain? award potential recommenders 1 value(a, b, i )  ra ,i  1...5 , ra ,i  rb ,i  1 : rb ,i  1...5 0.1 5 Note (a): a [computed] subset of the community of users seems to be good recommenders for everybody. analysis?

Slide 35: a user kNN algorithm generates a graph source: N. Lathia, S. Hailes, L. Capra “kNN CF: A Temporal Social Network” To appear, ACM RecSys ’08. October 2008, Lausanne, Switzerland

Slide 36: what properties does the graph have? • growth • preferential attachment • converging neighbourhoods • very short diameter • low reciprocity • fixed out-degree • power-law in-degree • similarity-dependent evolution

Slide 37: similarity evolution

Slide 38: similarity evolution

Slide 39: what properties does the graph have? • growth • preferential attachment • converging neighbourhoods • very short diameter • low reciprocity • fixed out-degree • power-law in-degree • similarity-dependent evolution

Slide 40: Note (b): a [kNN structure] subset of the community of users seems to be good recommenders for everybody.

Slide 41: Exploring Recommender Systems: the wisdom of the few

Slide 42: situation: sparse users ratings items

Slide 43: complication: privacy uncertainty coverage accuracy noise sparse users ratings items

Slide 44: one solution: more information (difficult) sparse users ratings items

Slide 45: our solution: more information sparse users ratings items

Slide 46: our solution: experts: dense ratings rotten tomatoes flixster items users sparse ratings items

Slide 47: dense ratings source set test set training set sparse ratings items

Slide 48: different characteristics (data)…

Slide 49: different characteristics (users)…

Slide 50: changing collaborative filtering before: find best method data ? users

Slide 51: changing collaborative filtering now: find best data source Goal: adaptive collaborative filtering ? user-dependent problem can provably generate an RMSE < 0.856!

Slide 52: Exploring Recommender Systems: trust, graphs, and experts neal lathia n.lathia@ cs.ucl.ac.uk http:/ / www.cs.ucl.ac.uk/ staff/ n.lathia