Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Keynote: Capturing User Interests for Content-Based Recommendations

1,937 views

Published on

Keynote presentation at the 2nd Workshop on New Trends in Content-Based Recommender Systems (CBRecSys'15, http://recsys.acm.org/recsys15/cbrecsys/), held in conjunction with ACM RecSys'15 in Vienna, Austria.

Abstract:
Given the information overload that we are facing nowadays, tools and systems are required that help us to filter through large information and product spaces. Recommender systems approach this task by predicting the preference or rating that a user would give to an item. Two methods (or combinations thereof) to provide these predictions dominate the field - collaborative filtering and content-based recommendation. Collaborative filtering methods exploit information on users’ behaviours or preferences to identify their interests and predict items that similar users showed interest in. On the other hand, content-based recommender systems aim to identify users’ interests based on analysing the actual content of items that they interacted with.
This talk first introduces the conceptual idea behind content-based recommendation. Representative systems and studies are presented that illustrate the advantage of semantic metadata, as well as the challenges that come with an automated analysis of content, especially in the multimedia domain.
Following this overview, two methods to capture users’ interests in items are introduced, namely explicit and implicit user profiling. Explicit user profiles are created by asking users to rate items in a collection. Implicit user profiles are created by gathering user interest based on implicit relevance feedback such as viewing or clicking behaviour online. Examples and case studies are presented that illustrate the advantages and limitations of both techniques.
The talk ends with an overview of CLEF NewsREEL, an evaluation campaign that allows researchers to benchmark news article recommendation algorithms in an offline and an online setting. Given the content rich nature of news articles, as well as the large numbers of users within NewsREEL who access news online, the lab can serve as a training ground to improve both content-based and collaborative filtering techniques.

Published in: Data & Analytics

Keynote: Capturing User Interests for Content-Based Recommendations

  1. 1. Capturing User Interests for Content- Based Recommendations Frank Hopfgartner @OkapiBM25
  2. 2. Recommendation algorithm Item profile User profile Recommendations Content-Based Recommendation Recommend most relevant items based on content features Item profile is a collection of content features describing an item User profile is an aggregation of all item profiles of a user
  3. 3. Recommendation algorithm Item profile User profile Recommendations Outline
  4. 4. Item profiles Collection of content features to describe an item Item profile
  5. 5. Creating Item profiles Transformation from text to Term-Vector Model and TF-IDF Title Genre Director Actor Storyplot ... Collateral Damage Drama Action Davis Schwarzenegger Neri ... Futuristic action about a man who meets a clone of himself... Drama Action Sci-Fi Davis Schwarzenegger Neri Hamilton Term 1 1 0 1 1 1 0 Vector 0.8 1.2 0 0.7 1.2 0.8 0 TF-IDF
  6. 6. Creating Item profiles Transformation from text to Term-Vector Model and TF-IDF Title Genre Director Actor Storyplot ... The Terminator Sci-Fi Action Cameron Schwarzenegger Hamilton ... A human-looking indestructible cyborg is sent from 2029 to 1984... Drama Action Sci-Fi Davis Schwarzenegger Neri Hamilton Term 0 1 1 0 1 0 1 Vector 0 1.2 2.4 0 1.2 0 0.7 TF-IDF
  7. 7. • Semantic Recommender System that supports users in finding movies • Recommendations are based on various semantic data sources • Available sources in semantic format: • Internet Movie Database • Freebase • Rotten Tomatoes • Dbpedia Example: Semantic Movie Recommender A. Lommatzsch, B. Kille, S. Albayrak. A framework for learning and analyzing hybrid recommenders based on heterogeneous semantic data. In Proc. OAIR’13, Lisbon, Portugal, pp. 137-140, 2013.
  8. 8. Semantic sources
  9. 9. Graphical User Interface
  10. 10. Multimedia metadata - Challenge 1101100101010100 1101110001011010 1000111001010111 0010111010011101 0010110100111010 1010101 Semantic Gap
  11. 11. Creating Multimedia Metadata The Semantic Gap Human Annotation Automated Content Analysis
  12. 12. Human annotation http://www.soundcloud.com/ Challenge: Use timed comments on the SoundCloud platform. Let’s activate the crowd to assess the annotation for us
  13. 13. Annotation Interface This is tedious work. Let’s add some gamification to make it more fun! Graphic: Courtesy of M. Meder
  14. 14. Gamification “Gamification refers to the use of design elements characteristic for games in a non-game context.” Deterding et al., 2011
  15. 15. Gamified annotation assessment F. Bütow. Akquirierung von Metadaten zu Soundcloud-Timed-Comments: Entwicklung einer Crowdsourcing-Plattform zur Optimierung von textbasiertem Audio-Retrieval. Bachelor Thesis, TU Berlin, 2015.
  16. 16. Content analysis • Using low-level features based on automatic extraction. • Understand the Meaning… –Semantic Concepts – the objects –Inferred abstract concepts – the meaning Signal Processing Computer Vision AI Recommender System Low level features Semantic Concepts Abstract Attributes User
  17. 17. Example: Content-Based Image Recommendation • Extract low-level features to represent images • Create item profiles based on these features • Recommend similar items while considering – Diversity – Feedback time }Low-Level Features Colour Texture Shape T. Leelanupab, F. Hopfgartner, J. Jose. User Centred Evaluation of a Recommendation Based Image Browsing System. In Proc. Indian International Conference on Artificial Intelligence, Tumkur, India, 2009.
  18. 18. Example: Content-Based Image Recommendation T. Leelanupab, F. Hopfgartner, J. Jose. User Centred Evaluation of a Recommendation Based Image Browsing System. In Proc. Indian International Conference on Artificial Intelligence, Tumkur, India, 2009.
  19. 19. Recommendation algorithm Item profile User profile Recommendations Outline
  20. 20. User profiling techniques Two types of user profiles are used to capture user interest: • Ask users to explicitly rate items in a collection (Explicit user profiling) • Gather user interest based on implicit relevance feedback such as viewing or clicking behaviour. (Implicit user profiling) User profile
  21. 21. Example: Explicit User Profiling
  22. 22. User profile of “Mr. Blue“ Mr. Blue The Terminator Drama Action Sci-Fi Davis Schwarzenegger Neri Hamilton 0 0 2.4 0 1.2 0 0.7 x 4 0 0 0 6 0 3.5 Collateral Damage Drama Action Sci-Fi Davis Schwarzenegger Neri Hamilton 0.8 1.2 0 0.7 1.2 0.8 0 x 5 4 6 0 3.5 6 4 0 Profile Drama Action Sci-Fi Davis Schwarzenegger Neri Hamilton 2 3 1.2 1.75 6 2 1.75Æ Collateral Damage The Terminator
  23. 23. Recommendation task • Question: – How would Mr. Blue rate the movie “Maggie“? • Idea: – Predict items that match user preferences • Approach: – Content: Information about items – User profile: Preferences of the user – Learn user preferences – Match content with user profile and recommend movie
  24. 24. Item profile of “Maggie“ Transformation from text to Term-Vector Model and TF-IDF Title Genre Director Actor Storyplot ... Maggie Drama Horror Hobson Schwarzenegger Breslin ... A teenage girl becomes infected by an outbreak of a disease... Drama Action Sci-Fi Davis Schwarzenegger Neri Hamilton Term 1 0 0 0 1 0 0 Vector 0.8 0 0 0 1.2 0 0 TF-IDF
  25. 25. Determine similarity of user profile and item profile Profile Drama Action Sci-Fi Davis Schwarzenegger Neri Hamilton 2 3 1.2 1.75 6 2 1.75 Maggie Drama Action Sci-Fi Davis Schwarzenegger Neri Hamilton 0.8 0 0 0 1.2 0 0 Mr. Blue Maggie
  26. 26. User profiling techniques Two types of user profiles are used to capture user interest: • Ask users to explicitly rate items in a collection (Explicit user profiling) • Gather user interest based on implicit relevance feedback such as viewing or clicking behaviour. (Implicit user profiling) User profile
  27. 27. Challenges and approaches Relevance feedback (RF) is crucial for understanding users’ interests. But users tend not to provide explicit RF. Usually, users are interested in more than one topic or item, e.g., sports and politics or football and tennis. Users’ interest can evolve over time, i.e., interest fades over time. Identify users’ interest by treating their interactions with the GUI as implicit indication of relevance. Represent interest in diverse topics in dynamic user profile by categorising items based on content. Develop time aware (context aware) user profiling approach.
  28. 28. Example: Implicit User Profiling F. Hopfgartner and J. M. Jose. Semantic User Profiling Techniques for personalised multimedia recommendation. Multimedia Systems 14(4-5):255-274, 2010.
  29. 29. Implicit feedback elements Qi Ri Nodes Query - Query terms Document - Document ID Actions General metadata: - Timestamp - User id q p b n … Query Action Navigation Action Browsing Action Play Action - Time of play c Click Action t Tooltip Action
  30. 30. Q1 D1 D2bq c D3 c Q2q c D4b Q3q c c Viewing and clicking behaviour
  31. 31. • Creation of individual test collections • Mimic user interactions to create simulated user profiles User Simulation • Users included the system in their daily information gathering routine • Questionnaires, logfile analysis Longitudinal Study Evaluation • Television news (BBC One O‘Clock news and iTV News) • 2*30 min broadcasts • Recorded every day during the experiment • Segmented into news stories, represented by keyframes • Index updated automatically once video processing is finished. DAILYNEWSBROADCASTS F. Hopfgartner and J. M. Jose. Semantic User Profiling Techniques for personalised multimedia recommendation. Multimedia Systems 14(4-5):255-274, 2010. F. Hopfgartner and J. M. Jose. An experimental evaluation of ontology-based user profiles. Multimedia Tools and Applications 73(2):1029-1051, 2014.
  32. 32. User Simulation pnb Q D1 D2 DN … R1 RNr … Qr c User session Interaction results Nr=5
  33. 33. Results
  34. 34. User Study • Deployed system on university campus network. • Asked users to include system into their daily news gathering routine and follow simulated search task situation for two weeks • Interactions logged • Had to answer online questionnaire every other day of the experiment
  35. 35. Results Questionnaires Logfile analysis
  36. 36. Case Study: The Role of Feedback Time • Analysis of one year log files of Campus wide IPTV provider • Live and VoD content • 16 genres • 33 channels • Over 7000 different programme names • Over 500 unique users J. Yuan, F. Sikrivaya, F. Hopfgartner, A. Lommatzsch, M. Mu. Context-Aware LDA: Balancing Relevance and Diversity in TV Content Recommenders. In Proc. RecSysTV workshop, Vienna, Austria, 2015.
  37. 37. 1 2 3 4 5 6 7 0246810121416182022 ARTS CHILDRENS COMEDY DRAMA ENTERTAINMENT FACTUAL FILM LEARNING LIFESTYLE MUSIC NEWS NULL RELIGIONANDETHICS SPORT SPORTS WEATHER day of w eek Category Distribution time of day categories categories chosen count 20 40 60 80 100 120 140 News, Entertainment, Drama und Comedy are the most popular genres in VoD services Case Study: The Role of Feedback Time
  38. 38. Case Study: The Role of Feedback Time J. Yuan, F. Sikrivaya, S. Marx, F. Hopfgartner. When to recommend what? A study on the role of contextual factors in IP-based TV services. In Proc. of the Mind The Gap Workshop, Berlin, Germany, pp. 12-18, 2014.
  39. 39. When to recommend? 40 9:00~12:00 18:00~21:00 The Big Bang Theory The Big Bang Theory BBC News Coronation Street This Morning The Simpsons The Jeremy Kyle Show Hollyoaks Frasier Emmerdale How I Met Your Mother How I Met Your Mother Rules of Engagement Come Dine with Me Top Gear EastEnders The Simpsons BBC News Winter Olympics ITV News & Weather usreId:1105 path_lambda:[0.00025 0.9995 0.00025] Top 10 recommended items for anonymous user #1105, when using context-aware LDA, at different “time of day” J. Yuan, F. Sikrivaya, F. Hopfgartner, A. Lommatzsch, M. Mu. Context-Aware LDA: Balancing Relevance and Diversity in TV Content Recommenders. In Proc. RecSysTV workshop, Vienna, Austria, 2015.
  40. 40. Recommendation algorithm Item profile User profile Recommendations Outline
  41. 41. News Article Recommendation Source(Image):T.Brodtofplista.com F. Hopfgartner, B. Kille, A. Lommatzsch, T. Plumbaum, T. Brodt, T. Heintz. Benchmarking News Recommendations in a Living Lab. In Proc. CLEF’14, Sheffield, UK, 250-267, 2014.
  42. 42. Recommendation scenario Publisher A Publisher n … plista
  43. 43. CLEF NewsREEL In CLEF NewsREEL, participants can develop stream- based news recommendation algorithms and have them benchmarked (a) online by millions of users over the period of a few months, and (b) offline by simulating a live stream.
  44. 44. Conferences and Labs of the Evaluation Forum (CLEF) N. Ferro “CLEF 15th Birthday: Past, Present, and Future,” in SIGIR Forum, 48(2): 31-55, 2014.
  45. 45. Who are the users?
  46. 46. CLEF NewsREEL Tasks • Benchmark News Recommendation in a Simulated Environment Offline Evaluation • Benchmark News Recommendations in a Live System Online Evaluation TASK1TASK2
  47. 47. Task 1: Offline Evaluation • Traffic and content updates of nine German-language news content provider websites • Traffic: Reading article, clicking on recommendations • Updates: adding and updating news articles • Simulation of data stream using Idomaar framework • Participants have to predict interactions with data stream • Quality measured by the ratio of successful predictions by the total number of predictions
  48. 48. Example results (Task 1) B. Kille, A. Lommatzsch, R. Turrin, A. Sereny, M. Larson, T. Brodt, J. Seiler, F. Hopfgartner “Overview of CLEF NewsREEL 2015: News Recommendation Evaluation Lab,” in Working Notes of CLEF 2015, Toulouse, France, 2015.
  49. 49. Task 2: Online Evaluation • Provide recommendations for visitors of the news portals of plista’s customers • Ten portals (local news, sports, business, technology) • Communication via Open Recommendation Platform (ORP) • Benchmark own performance with other participants and baseline algorithm • We determine best performing algorithms during three pre-defined evaluation periods • Standard evaluation metrics B. Kille, A. Lommatzsch, R. Turrin, A. Sereny, M. Larson, T. Brodt, J. Seiler, F. Hopfgartner “Overview of CLEF NewsREEL 2015: News Recommendation Evaluation Lab,” in Working Notes of CLEF 2015, Toulouse, France, 2015.
  50. 50. Example results (Task 2) B. Kille, A. Lommatzsch, R. Turrin, A. Sereny, M. Larson, T. Brodt, J. Seiler, F. Hopfgartner “Overview of CLEF NewsREEL 2015: News Recommendation Evaluation Lab,” in Working Notes of CLEF 2015, Toulouse, France, 2015.
  51. 51. Registration opens on November 2nd “NewsREEL provided an excellent opportunity for my students to gain experience in using web services, processing streams, and developing information access algorithms.” (Senior Academic) http://clef-newsreel.org
  52. 52. Thank you Frank Hopfgartner, PhD www.hopfgartner.co.uk Frank.Hopfgartner@glasgow.ac.uk @OkapiBM25

×