Your SlideShare is downloading. ×
0
Social Book Search: A Combination of Personalized Recommendations and Retrieval
Social Book Search: A Combination of Personalized Recommendations and Retrieval
Social Book Search: A Combination of Personalized Recommendations and Retrieval
Social Book Search: A Combination of Personalized Recommendations and Retrieval
Social Book Search: A Combination of Personalized Recommendations and Retrieval
Social Book Search: A Combination of Personalized Recommendations and Retrieval
Social Book Search: A Combination of Personalized Recommendations and Retrieval
Social Book Search: A Combination of Personalized Recommendations and Retrieval
Social Book Search: A Combination of Personalized Recommendations and Retrieval
Social Book Search: A Combination of Personalized Recommendations and Retrieval
Social Book Search: A Combination of Personalized Recommendations and Retrieval
Social Book Search: A Combination of Personalized Recommendations and Retrieval
Social Book Search: A Combination of Personalized Recommendations and Retrieval
Social Book Search: A Combination of Personalized Recommendations and Retrieval
Social Book Search: A Combination of Personalized Recommendations and Retrieval
Social Book Search: A Combination of Personalized Recommendations and Retrieval
Social Book Search: A Combination of Personalized Recommendations and Retrieval
Social Book Search: A Combination of Personalized Recommendations and Retrieval
Social Book Search: A Combination of Personalized Recommendations and Retrieval
Social Book Search: A Combination of Personalized Recommendations and Retrieval
Social Book Search: A Combination of Personalized Recommendations and Retrieval
Social Book Search: A Combination of Personalized Recommendations and Retrieval
Social Book Search: A Combination of Personalized Recommendations and Retrieval
Social Book Search: A Combination of Personalized Recommendations and Retrieval
Social Book Search: A Combination of Personalized Recommendations and Retrieval
Social Book Search: A Combination of Personalized Recommendations and Retrieval
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Social Book Search: A Combination of Personalized Recommendations and Retrieval

287

Published on

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
287
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Social Book Search: A Combination of Personalized Recommendations and RetrievalAuthor:Justin van WeesSupervisor: Master ThesisMarijn Koolen Information ScienceSecond assessor: Human Centered MultimediaFrank Nack August 23, 2012
  • 2. Outline1. Background2. Research questions3. Data collection4. Experiments and results5. Conclusions6. Discussion and future work7. Questions
  • 3. Current situation• Traditional information retrieval (IR) models: • developed for use on small collections • contain only officially published documents, annotated by professionals• Many modern web (2.0) applications still use traditional models for search• Millions of documents• Combination of user-generated content (UDG) and professional metadata
  • 4. Current situation• User uses IR system to find those documents that are topically relevant to her information need• Queries can lead to thousands of relevant documents• Evaluating large number of results expensive for user• Other notions of relevance, i.e. how well-written, popular, recent, fun is the document• Combination of professional and user-generated metadata
  • 5. Social Book Search Track• Evaluate relative value of controlled book metadata versus social metadata (Koolen et al., 2012)• Amazon.com and LibraryThing (LT) corpus• ~2.8 million book records, both social and professional metadata• book search requests from LT discussion forums as topics, suggestions by other users as relevance judgements
  • 6. Recommender Systems• Recommender Systems (RSs) suggest items of interest to individuals or groups of users (Resnick and Varian, 1997)• Assumes that individual’s taste or interest in a particular item can be explained by features recorded by the RS (demographics, previous interactions, etcetera)• Different strategies: collaborative filtering (CF), content-, community-, knowledge-based, hybrid (Burke, 2007)• Differs from traditional retrieval in terms of query formulation, source of relevance feedback and personalization (Furner, 2002)
  • 7. Research QuestionsDoes a combination of techniques from the field of IR withthose from RSs improve retrieval performance when searchingfor works in a large scale on-line collaborative mediacatalogue?• What data are we able to collect?• Can we automatically make accurate predictions of a user’s preference for an unknown book?• How do we combine results from IR system with RSs?• Social Book Search scenario and data
  • 8. Crawling LibraryThing • Perform four different crawls of user profiles and personal catalogues • For each crawl, also crawl links to other profiles • Compare crawls to determine representativeness for entire LT user-base • All crawl combined approximately 6% of LT userbaseCrawl seed list profiles unique works profile overlapForum users 1,104 60,131 4,354,387Random – 211 works 1,306 8,040 2,537,065 7,048Random – 1,000 works 5,577 18,381 3,580,296 14,262Random – 10,000 works 35,671 64,379 5,122,848 37,300Total - 89,693 5,299,399 -
  • 9. Crawling LibraryThing Crawl min. max. median mean std. dev. Forum users Friends 0 172 3.0 8.47 16.31 Groups 0 10 9.0 6.79 3.74 Interesting Libraries 0 510 2.0 11.19 26.46 Random – 211 works Friends 0 79 0.0 2.61 7.46 Groups 0 10 0.0 1.70 3.05 Interesting Libraries 0 394 0.0 3.30 17.80 Random – 1,000 works Friends 0 84 0.0 2.18 6.07 Groups 0 10 0.0 1.64 3.02 Interesting Libraries 0 574 0.0 2.74 14.41 Random – 10,000 works Friends 0 2,858 0.0 1.73 17.49 Groups 0 10 0.0 1.24 2.61 Interesting Libraries 0 855 0.0 1.69 10,40 Total Friends 0 2,858 1.0 2.14 12.77 Groups 0 10 0.0 1.18 2.44 Interesting Libraries 0 855 0.0 1.27 8.00
  • 10. Crawling LibraryThing Crawl min. max. median mean std. dev. sum Forum users Unrated 0 28,402 84.0 397.22 929.70 23,885,23 Rated 0 12,190 3.0 78.80 238.53 4,738,018 Total 0 28,402 148.00 476.02 980.88 28,623,249 Random – 211 works Unrated 0 28,402 458.00 1,112.81 1,835.81 8,946,997 Rated 0 12,190 10.00 182.77 472.08 1,469,531 Total 0 28,402 657.00 1,295.58 1,908.65 10,416,528 Random – 1,000 works Unrated 0 28,402 331.00 864.32 1,480.98 15,887,025 Rated 0 12,190 3.00 130.20 369.06 2,393,233 Total 0 28,402 475.00 994.52 1539.15 18,280,258 Random – 10,000 works Unrated 0 28,402 163.00 486.63 955.86 31,328,971 Rated 0 12,190 1.00 74.04 237.01 4,766,750 Total 0 28,402 201.00 560.68 1,000.50 36,095,721 Total Unrated 0 28,402 102.00 378.18 834.94 33,920,353 Rated 0 12,190 1.00 62.85 206.40 5,637,097 Total 0 28,402 156.00 441.03 876.76 39,557,450
  • 11. Generating Recommendations• Collaborative filtering approach• Unary and rated transactions• Memory- and model-based recommenders• Randomly split transactions (80% train/20% test) for performance evaluation
  • 12. Generating Recommendations• Neighbourhood (Desrosiers and Karypis, 2011): • Directly use user-item ratings to predict ratings for ‘unseen’ items • Find n most similar neighbours (Pearson correlation) • Use the weighted average rating given by the user’s neighbours • Let neighbours ‘vote’ on unary transactions
  • 13. Generating Recommendations• Singular Value Decomposition (SVD) (Schafer et al., 2007): • Reduce domain complexity by mapping item space to k dimensions • Remaining dimensions represent the latent topics: preferences classes of users, categorical classes of items • Currently considered ‘state of the art’
  • 14. Recommender PerformanceMethod MAE RMSE P@5 P@10 P@50Neighbourhood (N=25) 0.7813 1.0286 0.0712 0.0661 0.0614Neighbourhood (N=50) 0.7721 1.0105 0.0376 0.0371 0.0339Neighbourhood (N=100) 0.7633 0.9927 0.0246 0.0239 0.0232SVD (K=50) 0.6210 0.8139 0.0021 0.0019 0.0026SVD (K=100) 0.6203 0.8131 0.0025 0.0022 0.0028SVD (K=150) 0.6192 0.8122 0.0281 0.0107 0.0030 Method Accuracy P@5 P@10 P@50 Neighbourhood (N=25) 0.2430 0.3711 0.2425 0.1829 Neighbourhood (N=50) 0.3014 0.3824 0.2561 0.1861 Neighbourhood (N=100) 0.3621 0.3640 0.2422 0.1812 SVD (K=50) 0.2240 0.0214 0.0198 0.0216 SVD (K=100) 0.2601 0.0219 0.0203 0.0229 SVD (K=150) 0.2676 0.0424 0.0212 0.0234
  • 15. Retrieving Works• Setup used for INEX 2012; top performing run• Index consists of user-generated content• Removed stopwords• Stemming with Krovetz• Topic titles as queries• Language model• Pseudo relevance feedback, 50 terms of top 10 results
  • 16. Combining IR and RS• Retrieval system: ranked list, probability score between 0 and 1 per work• Recommendations: estimated preference of user for work between 0.5 and 5.0 or 0 or 1 (unary)• Normalise ratings• ‘Boost’ works with estimated preference, CombSUM (Fox and Shaw, 19994)• Use average rating when no prediction can be made• Introduce weight (λ) between systems
  • 17. ResultsMethod nDCG@10 P@10 R@10Baseline - 0.1437 0.1219 0.1494Neighbourhood Rated (n=25) 0.0001700 0.1709 (18.93%) 0.1490 (22.23%) 0.1899 (27.11%) Rated (n=50) 0.0001855 0.1778 (23.73%) 0.1500 (23.05%) 0.1913 (28.05%) Rated (n=100) 0.0001800 0.1669 (16.14%) 0.1490 (22.23%) 0.1878 (25.70%) Unary (n=25) 0.0001500 0.1446 (0.63%) 0.1229 (0.82%) 0.1520 (1.74%) Unary (n=50) 0.0001500 0.1441 (0.28%) 0.1229 (0.82%) 0.152 (1.74%) Unary (n=100) 0.0001500 0.1441 (0.28%) 0.1229 (0.82%) 0.152 (1.74%)SVD Rated (K=50) 0.0001800 0.1718 (19.55%) 0.149 (22.23%) 0.1866 (24.9%) Rated (K=100) 0.0001850 0.1721 (19.76%) 0.149 (22.23%) 0.1866 (24.9%) Rated (K=150) 0.0001850 0.172 (19.69%) 0.149 (22.23%) 0.1866 (24.90%) Unary (K=50) 0.0001500 0.1449 (0.84%) 0.124 (1.72%) 0.1541 (3.15%) Unary (K=100) 0.0001550 0.1441 (0.28%) 0.1229 (0.82%) 0.1520 (1.74%) Unary (K=150) 0.0001550 0.1424 (-0.9%) 0.1250 (2.54%) 0.1561 (4.48%)
  • 18. Conclusions• Collected representative sample of user profiles• Collaborative filtering obvious choice• SVD best at estimating rated preference• Poor performance on unary transactions• Successfully combined retrieval with personalized recommendations• Rated transactions most useful• Personal preference is relevance evidence that can highly improve retrieval performance in SBS
  • 19. Discussion and Future Work• Popularity as relevance evidence• Value of λ depending on IR score distribution• Other (mixtures of) RS setups• Scaling, cold-start problems• Trust and transparency of the system
  • 20. Questions?
  • 21. References• R. Burke. Hybrid web recommender systems. In The adaptive web, pages 377–408. Springer-Verlag, 2007.• C. Desrosiers and G. Karypis. A comprehensive survey of neighborhood-based recommendation methods. Recommender Systems Handbook, pages 107–144, 2011.• E. Fox and J. Shaw. Combination of multiple searches. NIST SPECIAL PUBLICATION SP, pages 243–243, 1994.• J. Furner. On recommending. Journal of the American Society for Information Science and Technology, 53(9):747–763, 2002.• M. Koolen, G. Kazai, J. Kamps, A. Doucet, and M. Landoni. Overview of the INEX 2011 books and social search track. In S. Geva, J. Kamps, and R. Schenkel, editors, Focused Retrieval of Content and Structure: 10th International Workshop of the Initiative for the Evaluation of XML Retrieval (INEX 2011), volume 7424 of LNCS. Springer, 2012.• P. Resnick and H.Varian. Recommender systems. Communi- cations of the ACM, 40(3):56–58, 1997.• J. B. Schafer, D. Frankowski, J. Herlocker, and S. Sen. Collaborative Filtering Recommender Systems. Inter- national Journal of Electronic Business, 2(1):77, 2007. ISSN 14706067. doi: 10.1504/IJEB.2004.004560. URL http://www.springerlink.com/index/ t87386742n752843.pdf.
  • 22. Number of Books in Catalogue (a) Unrated works (b) Rated works
  • 23. Document scoring S(d) = (1 )PRet (d|q) + PCF (d)• PRet (d|q): work’s score obtained through IR system• PCF : estimated rating of current user for work obtained through RS• : weight between systems
  • 24. Estimating preference (rated) P wuv rvi v2Ni (u) rui = ˆ P |wuv | v2Ni (u)• rui : estimated preference of user u for item i ˆ• wuv : preference similarity between users v and u• Ni (u): k-NN of u that rated item i Desrosiers and Karypis, 2011
  • 25. Estimating preference (unary) X vir = (rvi = r)Wuv v2Ni (u) Desrosiers and Karypis, 2011

×