Successfully reported this slideshow.

Narrative-Driven Recommendation for Casual Leisure Needs

1

Share

1 of 32
1 of 32

Narrative-Driven Recommendation for Casual Leisure Needs

1

Share

Download to read offline

Recommender systems typically generate recommendations for a user based on their profile, or for an item given its user interactions, but there are many scenarios especially in leisure domains such as books, movies, games and music, where users have specific recommendation needs, where they want to steer the recommendation process towards certain aspects they find relevant. Currently, there are few recommender or search systems that can deal with the complexity of such directed needs, nor do we know well which data types (metadata, user ratings and reviews, item content) are useful to match against different aspects of recommendation needs. There are many discussion forums where users describe their needs and their frustration with current search and recommender systems. In this talk I will summarize our work on analyzing relevance aspects for these needs and describe experiments on dealing with these.

Recommender systems typically generate recommendations for a user based on their profile, or for an item given its user interactions, but there are many scenarios especially in leisure domains such as books, movies, games and music, where users have specific recommendation needs, where they want to steer the recommendation process towards certain aspects they find relevant. Currently, there are few recommender or search systems that can deal with the complexity of such directed needs, nor do we know well which data types (metadata, user ratings and reviews, item content) are useful to match against different aspects of recommendation needs. There are many discussion forums where users describe their needs and their frustration with current search and recommender systems. In this talk I will summarize our work on analyzing relevance aspects for these needs and describe experiments on dealing with these.

More Related Content

Related Audiobooks

Free with a 14 day trial from Scribd

See all

Narrative-Driven Recommendation for Casual Leisure Needs

  1. 1. Narrative-driven Recommendation for Casual- Leisure Needs Marijn Koolen Royal Netherlands Academy of Arts and Sciences Humanities Cluster RecSysNL Meetup, Amsterdam, 2019-02-19
  2. 2. Overview 1.Scenario: Narrative-Driven Recommendation 2.Analyzing Forum Discussions for Casual Leisure Domains 3.Narrative- and Example-Driven Algorithms 4.Lessons & Outlook
  3. 3. 1. Scenario: Narrative- Driven Recommendation
  4. 4. History of NDR • Evolved from Social Book Search evaluation campaign (2011-2016) • Evolved into analysing forum requests as complex information needs • Books: Koolen et al. (CIKM 2012, ECIR 2014, ECIR 2015) • Movies and Books: Bogers et al. (iConference 2018) • Games: Bogers et al. (iConference 2019) • Books, Movies, Games, Music: Bogers et al. (in preparation) • Narrative-Driven Recommendation • Bogers, Koolen (ACM RecSys 2017, KaRS 2018).
  5. 5. Scenario • Narrative-Driven Recommendation (NDR, Bogers & Koolen 2017, 2018) is a complex scenario: • narrative description of desired aspects of items • user preference info (user profile or example items) • Related to Conversational Recommendation • But different: e.g. human-directed, complex and often vaguely expressed (latent) needs • Interactions with conversational agents tend to be much simpler • E.g. more concrete aspects (genre, creator, title, year, …)
  6. 6. 2. Analyzing Forum Discussions
  7. 7. ANALYSIS – WHAT DO NDR BOOKS REQUESTS LOOK LIKE? ▸ Annotated 1,457 book NDR request narratives with seven relevance aspects of Reuters (JASIS 2007) – Accessibility Accessibility in terms of the language, length, or level of difficulty. – Content Aspects such as topic, plot, genre, style, or comprehensiveness. – Engagement Books that fit a particular mood or interest or provide a particular reading experience. – Familiarity Books that are similar to known books or related to a previous experience. – Metadata Books with a certain title or by a certain author, editor, illustrator, publisher, in a particular format, or written or published in certain year or period. – Novelty Books with content that is novel to the reader, unusual or quirky. – Socio-Cultural Books related to the user’s socio-cultural background or values, books that are popular or obscure, or books that have had a particular impact. 8
  8. 8. ANALYSIS – WHAT DO THEY LOOK LIKE? ▸ Distribution of the seven narrative aspects 9 A C E F M N S Accessibility 137 96 41 48 28 8 27 Content 598 157 267 176 26 98 Engagement 196 88 40 11 24 Familiarity 326 74 17 45 Metadata 179 11 25 Novelty 34 10 Socio-cultural 133
  9. 9. Search-Recommendation Continuum • Some requests are pure search • “sci-fi books about space traders” • Some requests are almost pure recommendation • “Something as good as David Copperfield“ • Many requests mix search and recommendation • “historical fiction set in 17th c. England that I’ll like based on my profile”
  10. 10. Continuum and Latent Interests • Latent factors are related to amount of reading experience • Novice readers ask for recommendations based on examples (interests are latent) • Experienced readers describe detailed content aspects (interests are known) • How does this work in other domains?
  11. 11. Setting the scene
  12. 12. Comparing Domains • We developed a relevance aspect model for leisure needs • Five annotators together analysed forum posts in four domains • Data from a range of discussion forums • Books: 503 requests (LibraryThing) • Movies: 538 requests (IMDB forums) • Games: 521 requests (reddit) • Music: 589 requests (reddit)
  13. 13. Domain Differences • Noticeable differences • NDR Movie requests often mention plot and characters, book and game requests mention experience • Book and game information needs are more complex, NDR requests more prevalent on forums than movie and music needs • Songs are short, low cost to access and consumption • Books and games are long, high cost to access and consumption • Movies fall somewhere in between
  14. 14. 3. Narrative- and Example-Driven Algorithms
  15. 15. Operationalizing Forum Request Context I’m looking for manly books about manly issues, that aren’t too gritty, but make you think as much as you laugh. So far these examples I have on my bookshelf: ‘About a Boy’ and ‘High Fidelity’ by Nick Hornby, ‘Train Man’ by Hitori Nakano. Have you any other manly books for manly men such as I? Requester Post 1 ( = original request) Thread … Well, of course, many of the early works of Ernest Hemingway are an intense meditation on manliness. Esp. The Sun Also Rises, the Nick Adams stories, the story "Fifty Grand," but many of his other short stories too.User X Post 2 (= 1st reply) User preferences Books Metadata User-generated content Curated metadata Narrative Mini-profile Book(s) contains Author(s) Author(s) Book(s) User suggestions represented by for has has Context: requests in LibraryThing (LT) discussion forums
  16. 16. Methodology & Setup • Requests • 974 NDR forum requests (Bogers & Koolen, RecSys 2017) • 331 requests have 1) examples, 2) suggestions by forum members and 3) user profile of requester • Datasets • 2.8M Amazon/LT book records (metadata + user reviews/tags) • 66K LibraryThing user profiles, 29M trans., 4.4M ratings • Relevance judgements • Recommendations posted by forum members • Graded: higher relevance value if requester catalogues recommendation
  17. 17. (1) Book(s) Mini-profile Requester (g) (a) (b) (e) (d) (c) (f) Similar books Authored book(s) (2) Requester Content-based filtering 1. 2. 3. Narrative Metadata User-generated content Curated metadata (a) (b) (e) (d) (c) (f) EDR-1 Authored book(s) Author(s)Book(s) Mini-profile Requester (g) (a) (b) (e) (d) (c) (f) Similar authors Similar books EDR-2 1. 2. 3. 1. 2. 3. (1) Authored book(s) Author(s)Book(s) Mini-profile Requester (a) (b) (e) (d) (c) (f) Similar authors Similar books (2) 1. 2. 3. 1. 2. 3. ed s) (s) r rs 3) Narrative Driven Recommendation NDR: narrative request Example Driven Recommendation EDR-1: example books EDR-2: example authors
  18. 18. Baselines • Collaborative Filtering • LightFM toolkit (Kula 2015) • Loss function: WARP (Weston et al. 2011), 300 factors • Too few ratings, so transaction as implicit feedback • 50/50 train/test (due to small no. of requests) • Content Based Filtering • Indri 5.4, LM with JM smoothing, Krovetz stemming, stopword removal • optimized based on earlier experiments (Bogers & Petras 2015, 2017) • Narrative Driven: Narrative request as text query • Example Driven: book metadata of examples as query
  19. 19. NDR Baselines • CF less effective than CBF (also on standard recsys scenario) so domain is atypical • CBF effective with tags and esp. reviews (uses similar language as requests?)
  20. 20. Example-Driven Recommenders • EDR less effective than NDR • CBF: esp. tags are effective • CF: using examples much more effective than using full profile, less than CBF • Example books more effective than example authors (drift?) • requests with only example authors more difficult
  21. 21. Hybrid EDR + NDR • EDR books + authors better than EDR books and EDR authors • EDR + NDR more effective than NDR alone: • Examples complement narrative (latent factors), narrative more important
  22. 22. 4. Lessons & Outlook
  23. 23. Popularity • Forum recommendations that users follow up on are lower range popularity • People don’t need a recommender to find popular stuff • Collab. Filtering picks mid-high range popularity items • can use personalized popularity priors to rerank • but still not directed towards current need • Content-based picks more low range popularity items • and can deal with directed needs
  24. 24. The Power of Reviews • User reviews are highly effective • not only for recommendation but many search tasks (Koolen ECIR 2014) • written in the language of the user (same as request) • discuss broad range of aspects… • … including reading/watching/listening/playing experience!
  25. 25. Models for Capturing Needs - Conversational Recommenders • Complex narrative is difficult to interpret algorithmically • Possible interaction: conversational models for iterative structuring • Kang et al., (RecSys 2017) look at queries in conversational movie recommendation • Objective: genre (“superhero movies”), deep features (“movies with open endings or plot twists”) • subjective: emotion (“sad movie”), quality (“interesting characters, clever plot”), familiarity (“what would you recommend to a fan of Big Lebowski?”)
  26. 26. Explanations From Reviews • Lu et al. (RecSys 2018) used adversarial s2s learning to generate explanations from user reviews • generate review for new item based on own reviews of consumed items • Possible interaction: • select reviews that discuss aspects and experiences you’re interested in • let system generate recommendations + explanations
  27. 27. Experience, Appeal and Impact • Many requests indicate that users are looking for a certain reading experience or impact • Saricks (2005) identified appeal elements based on style, characterization, plot, pace • We’re currently developing an impact model • Identify and extract expressions of impact from user reviews (with colleague Peter Boot) • Impact type: emotional (not just binary sentiment) and cerebral (changing your views, motivating you, bringing up memories) • Impact cause: e.g. style, narrative
  28. 28. Conclusions • Narrative-Driven Recommendation is a challenging task. • Requires combination of data sources and algorithms to solve. • User-generated content essential for good performance • Little explored scenario • Use well-understood baselines to understand nature of scenario, requests and data • Next steps: other domains, deeper analysis, interaction models
  29. 29. References (1/2) • Bogers, Koolen (RecSys 2017). Defining and Supporting Narrative-driven Recommendation • Bogers, Koolen (KaRS 2018). “I’m looking for something like …”: Combining Narratives and Example Items for Narrative-driven Book Recommendation • Bogers et al. (iConf 2018). "What was this Movie About this Chick?" - A Comparative Study of Relevance Aspects in Book and Movie Discovery • Bogers et al. (iConf 2019). "Looking for an amazing game I can relax and sink hours into..." - A Study of Relevance Aspects in Video Game Discovery • Bogers, Petras (iConf 2015). Tagging vs. Controlled Vocabulary: Which is More Helpful for Book Search? • Bogers, Petras (iConf 2017). An in-depth analysis of tags and controlled metadata for book search • Kang et al. (RecSys 2017). Understanding How People Use Natural Language to Ask for Recommendations
  30. 30. References (2/2) • Koolen et al. (CIKM 2012). Social book search: comparing topical relevance judgements and book suggestions for evaluation • Koolen, M. (ECIR 2014). “User reviews in the search index? That’ll never work!” • Koolen et al. (ECIR 2015). Looking for Books in Social Media: An Analysis of Complex Search Requests • Kula (CBRecSys 2015). Metadata Embeddings for User and Item Cold-start Recommendations • Lu et al. (RecSys 2018). Why I like it: Multi-task Learning for Recommendation and Explanation • Reuter (JASIS 2007). Assessing aesthetic relevance: Children's book selection in a digital library • Saricks (2005). Readers' advisory service in the public library • Weston et al. (IJCAI 2011). WSABIE: Scaling Up To Large Vocabulary Image Annotation
  31. 31. Thank You! • Acknowledgements: • collaborative work with Toine Bogers, Peter Boot, Maria Gäde, Jaap Kamps, Vivien Petras, Mette Skov • Questions?

×