Evaluation of Collaborative Filtering Algorithms for Recommending Articles on CiteULike June 29th, 2009 HT 2009, Workshop ...
Outline <ul><li>Motivation </li></ul><ul><li>Methods </li></ul><ul><ul><li>CCF </li></ul></ul><ul><ul><li>NwCF </li></ul><...
Motivation <ul><li>Based on information available on CiteULike : </li></ul><ul><li>Develop user-centered recommendations o...
CiteULike
Methods: CCF (1 / 2) <ul><li>Classic Collaborative Filtering  (CCF): user-based recommendations, using Pearson Correlation...
Methods: CCF (2 / 2) 3 4 1 4 4 1 1 3 3 2 5 3 4 2 1 3 2 2 5 3 3 2
Methods: NwCF (1 / 2) <ul><li>Neighbor weighted Collaborative Filtering  (NwCF): Similar to CCF, yet incorporates  the “am...
Methods: NwCF (2 / 2) 3 4 1 4 4 1 1 3 3 2 5 3 4 2 1 3 2 2 5 3 3 2
Methods: BM25 (1 / 2) <ul><li>BM25 :  We obtain the similarity between users (neighbors) using their set of tags as “docum...
Methods: BM25 (2 / 2) Query terms Doc_1 Doc_2 Doc_3
The Study <ul><li>7 subjects </li></ul><ul><li>To each subject, four lists of 10 recommendations (each list) were created ...
Description of the Data <ul><li>Crawl CUL for 20 “center users” (only 7 were used for the study) </li></ul><ul><li>Annotat...
Results (a) nDCG  (b) Average Novelty  (c) Precision_2@5 (d) Precision_2@10  (e) Precision_2_1@5  (f) Precision_2_1@10
Conclusions <ul><li>The rating scale must be considered carefully in a CF approach. </li></ul><ul><li>NwCF, which incorpor...
Questions?
Bibliography <ul><li>[1] Schafer, J., Frankowski, D., Herlocker, J. and Sen, S. 2007 Collaborative Filtering Recommender S...
Upcoming SlideShare
Loading in...5
×

Evaluation of Collaborative Filtering Algorithms for Recommending Articles on CiteULike

2,486

Published on

Presentation given at the Workshop "Web 3.0: Merging Semantic Web and Social Web" in the conference Hypertext 2009, Torino, Italy.

The workshop online proceedings are here:http://ftp1.de.freebsd.org/Publications/CEUR-WS/Vol-467/

Published in: Education, Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,486
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
89
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Evaluation of Collaborative Filtering Algorithms for Recommending Articles on CiteULike

  1. 1. Evaluation of Collaborative Filtering Algorithms for Recommending Articles on CiteULike June 29th, 2009 HT 2009, Workshop “Web 3.0: Merging Semantic Web and Social Web” Dr. Peter Brusilovsky, Associate Professor Denis Parra, PhD Student School of Information Sciences University of Pittsburgh
  2. 2. Outline <ul><li>Motivation </li></ul><ul><li>Methods </li></ul><ul><ul><li>CCF </li></ul></ul><ul><ul><li>NwCF </li></ul></ul><ul><ul><li>BM25 </li></ul></ul><ul><li>The Study </li></ul><ul><li>Description of the Data </li></ul><ul><li>Results </li></ul><ul><li>Conclusions </li></ul>
  3. 3. Motivation <ul><li>Based on information available on CiteULike : </li></ul><ul><li>Develop user-centered recommendations of scientific articles. </li></ul><ul><li>Investigate the potential of users’ tags in collaborative tagging systems to provide recommendations . </li></ul><ul><li>Compare the accuracy of user-based collaborative filtering methods . </li></ul><ul><li>Why CiteULike? </li></ul><ul><li>Popular collaborative tagging system more topic-oriented than delicious: article references. </li></ul><ul><li>Familiarity with the system. </li></ul>
  4. 4. CiteULike
  5. 5. Methods: CCF (1 / 2) <ul><li>Classic Collaborative Filtering (CCF): user-based recommendations, using Pearson Correlation (users’ similarity) and adjusted ratings to rank items to recommend [1] </li></ul>
  6. 6. Methods: CCF (2 / 2) 3 4 1 4 4 1 1 3 3 2 5 3 4 2 1 3 2 2 5 3 3 2
  7. 7. Methods: NwCF (1 / 2) <ul><li>Neighbor weighted Collaborative Filtering (NwCF): Similar to CCF, yet incorporates the “amount of neighbors rating an item” in the ranking formula of recommended items </li></ul>
  8. 8. Methods: NwCF (2 / 2) 3 4 1 4 4 1 1 3 3 2 5 3 4 2 1 3 2 2 5 3 3 2
  9. 9. Methods: BM25 (1 / 2) <ul><li>BM25 : We obtain the similarity between users (neighbors) using their set of tags as “documents” and performing an Okapi BM25 (probabilistic IR model) Retrieval Status Value [2] calculation. </li></ul>
  10. 10. Methods: BM25 (2 / 2) Query terms Doc_1 Doc_2 Doc_3
  11. 11. The Study <ul><li>7 subjects </li></ul><ul><li>To each subject, four lists of 10 recommendations (each list) were created (CCF, NwCF, BM25_10, BM25_20) </li></ul><ul><li>The four lists were combined and sorted randomly (due to overlapping of recommendations, less than 40 items) </li></ul><ul><li>Subjects were asked to evaluate relevance (relevant/somewhat relevant/not relevant) and novelty (novel/ somewhat novel/ not novel) </li></ul>
  12. 12. Description of the Data <ul><li>Crawl CUL for 20 “center users” (only 7 were used for the study) </li></ul><ul><li>Annotation: tuple {user, article, tag} </li></ul>Item # of unique instances users 358 articles 186,122 tags 51,903 annotations 902,711
  13. 13. Results (a) nDCG (b) Average Novelty (c) Precision_2@5 (d) Precision_2@10 (e) Precision_2_1@5 (f) Precision_2_1@10
  14. 14. Conclusions <ul><li>The rating scale must be considered carefully in a CF approach. </li></ul><ul><li>NwCF, which incorporates the number of raters, decreases the uncertainty produced by items with too few ratings. </li></ul><ul><li>The tag-based user similarity approach shows interesting results, which can lead us to consider it a valid approach to Pearson-correlation when using CF algorithms. </li></ul><ul><li>We will incorporate more users in our future studies to make the results more conclusive. </li></ul>
  15. 15. Questions?
  16. 16. Bibliography <ul><li>[1] Schafer, J., Frankowski, D., Herlocker, J. and Sen, S. 2007 Collaborative Filtering Recommender Systems. The Adaptive Web. (May 2007), 291-324. </li></ul><ul><li>[2] Manning, C., Raghavan, P. and Schutze, H. 2008 Introduction to Information Retrieval. Cambridge University Press. </li></ul>
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×