SlideShare a Scribd company logo
Narrative-Driven Recommendation for
Complex Leisure Needs
Marijn Koolen
Royal Netherlands Academy of Arts and Sciences
Humanities Cluster
Google CSR Workshop, London, 2019-08-28
Narrative-Driven Recommendation for Casual Leisure Needs
Narrative-Driven Recommendation for Casual Leisure Needs
Overview
1. Scenario: Narrative-Driven Recommendation
2. Analyzing Forum Discussions for 4 Leisure Domains
3. Lessons for System Design
1. Narrative-Driven Recommendation
History of NDR
● Evolved from Social Book Search evaluation campaign (INEX/CLEF 2011-2016)
● Evolved into analysing forum requests as complex information needs
○ Books: Koolen et al. (CIKM 2012, ECIR 2014, ECIR 2015)
○ Movies and Books: Bogers et al. (iConference 2018)
○ Games: Bogers et al. (iConference 2019)
○ Books, Movies, Games, Music: Bogers et al. (in preparation)
● Scenario: Narrative-Driven Recommendation
○ Bogers, Koolen (ACM RecSys 2017, KaRS 2018).
Scenario
● NDR (Bogers & Koolen 2017, 2018) is a complex scenario:
○ Narrative description of desired aspects of items
○ User preference info (user profile or example items)
● Related to Conversational Search & Recommendation
○ But human-directed, complex and often vaguely expressed (latent) needs
○ Interactions with conversational agents tend to be simpler (Kang et al. 2017)
■ E.g. more concrete aspects (genre, creator, title, year, …)
Search-Recommendation Continuum
● Some requests are pure search
○ “sci-fi books about space traders”
● Some requests are almost pure recommendation
○ “Something as good as David Copperfield“
● Majority of requests mix search and recommendation
○ “historical fiction set in 17th c. England that I’ll like based on my profile”
● In the book domain latent factors are related to amount of reading experience
(Koolen et al. 2015)
○ Novice readers ask for recommendations based on example books (interests are latent)
○ Experienced readers describe detailed content aspects (interests are known)
○ In developing conversation on forums: start from latent factors and discuss examples to tease
out more concrete aspects of interest.
● How does this work in other leisure domains?
● And what relevance aspects do discussion forum users mention?
Continuum and Latent Interests
2. Analyzing Forum Discussions
● We developed a relevance aspect model for leisure needs
○ Grounded in actual forum requests
● Data from a range of discussion forums
○ Books: 503 requests (LibraryThing forums)
○ Movies: 538 requests (IMDB forums)
○ Games: 521 requests (reddit)
○ Music: 589 requests (reddit)
Comparing Domains
Narrative-Driven Recommendation for Casual Leisure Needs
Narrative-Driven Recommendation for Casual Leisure Needs
Narrative-Driven Recommendation for Casual Leisure Needs
Narrative-Driven Recommendation for Casual Leisure Needs
Narrative-Driven Recommendation for Casual Leisure Needs
Narrative-Driven Recommendation for Casual Leisure Needs
Narrative-Driven Recommendation for Casual Leisure Needs
3. Lessons for System Design
Capturing Needs - Conversational Recommenders
● Complex narrative is difficult to interpret algorithmically
○ Possible interaction: conversational models for iterative structuring
● Kang et al., (RecSys 2017) look at queries in conversational movie
recommendation
○ Many follow-up queries for refinement (clarify, constrain), reformulation
○ Objective: genre (“superhero movies”), deep features (“movies with open endings or plot
twists”)
○ Subjective: emotion (“sad movie”), quality (“interesting characters, clever plot”), familiarity
(“what would you recommend to a fan of Big Lebowski?”)
● User reviews are highly effective
○ for Narrative-Driven Recommendation (Bogers & Koolen 2018) ...
○ ... and also many search tasks (Koolen et al. 2012, Koolen 2014, Koolen et al. 2015)
● Why?
○ written in the language of the user (same as request)
○ discuss broad range of aspects…
○ … including reading/watching/listening/playing experience!
● Potential for conversational mode:
○ Background model for how people talk about books/films/games/music
○ “This review makes me want to buy this film.”
i. “What aspect of this review triggered your interest?”
The Effectiveness of User Reviews
Explanations From Reviews
● Reviews as a source of explanations
○ “Several reviewers say this book/film changed their views.”
● Lu et al. (RecSys 2018) used adversarial s2s learning to generate
explanations from user reviews
○ generate review for new item based on own reviews of consumed items
● Possible interaction:
○ User mentions examples, systems generates recommendations + explanations
○ User points out aspects of explanations they’re interested in, system refines recommendation
and explanations
Experience, Appeal and Impact
● How to extract information about experience or impact
○ Saricks (2005) identified appeal elements based on style, characterization, plot, pace
● We’re currently developing an impact model (for reviews in Dutch)
○ Identify and extract expressions of impact from user reviews (with Peter Boot, in preparation)
i. “I couldn’t put the book down and finished it in one go.”
ii. “I really started to understand the main character.”
○ Impact type: emotional (not just binary sentiment) and cerebral (changing your views,
motivating you, bringing up memories)
○ Impact cause: e.g. style, narrative, reflection
○ On 400K reviews, inverse relationship between narrative absorption and stylistic/reflective
impact
Conclusions
● Narrative-Driven Recommendation is a challenging task.
○ Complex but common recommendation/search need
○ Requires combination of data sources and algorithms to solve.
○ Conversational mode useful for identifying and ranking relevance aspects
○ User-generated content essential for good performance
● Next steps: other domains, more advanced NLU analysis, interaction models
● Bogers, Koolen (RecSys 2017). Defining and Supporting Narrative-driven Recommendation
● Bogers, Koolen (KaRS 2018). “I’m looking for something like …”: Combining Narratives and Example Items for Narrative-driven
Book Recommendation
● Bogers et al. (iConf 2018). "What was this Movie About this Chick?" - A Comparative Study of Relevance Aspects in Book and
Movie Discovery
● Bogers et al. (iConf 2019). "Looking for an amazing game I can relax and sink hours into..." - A Study of Relevance Aspects in
Video Game Discovery
● Bogers, Petras (iConf 2015). Tagging vs. Controlled Vocabulary: Which is More Helpful for Book Search?
● Bogers, Petras (iConf 2017). An in-depth analysis of tags and controlled metadata for book search
● Kang et al. (RecSys 2017). Understanding How People Use Natural Language to Ask for Recommendations
● Koolen et al. (CIKM 2012). Social book search: comparing topical relevance judgements and book suggestions for evaluation
● Koolen, M. (ECIR 2014). “User reviews in the search index? That’ll never work!”
● Koolen et al. (ECIR 2015). Looking for Books in Social Media: An Analysis of Complex Search Requests
● Kula (CBRecSys 2015). Metadata Embeddings for User and Item Cold-start Recommendations
● Lu et al. (RecSys 2018). Why I like it: Multi-task Learning for Recommendation and Explanation
● Reuter (JASIS 2007). Assessing aesthetic relevance: Children's book selection in a digital library
● Saricks (2005). Readers' advisory service in the public library
● Weston et al. (IJCAI 2011). WSABIE: Scaling Up To Large Vocabulary Image Annotation
References
Thank You!
● Acknowledgements:
○ collaborative work with Toine Bogers, Peter Boot, Maria Gäde, Jaap Kamps, Vivien Petras,
Mette Skov
● Questions?

More Related Content

Similar to Narrative-Driven Recommendation for Casual Leisure Needs

Design case methods AECT 2013
Design case methods AECT 2013Design case methods AECT 2013
Design case methods AECT 2013
Lisa Yamagata-Lynch
 
Patron-Driven Programming: Creating a Culture of Participatory Learning in Li...
Patron-Driven Programming: Creating a Culture of Participatory Learning in Li...Patron-Driven Programming: Creating a Culture of Participatory Learning in Li...
Patron-Driven Programming: Creating a Culture of Participatory Learning in Li...
ALAeLearningSolutions
 
Ontology based opinion mining for book reviews
Ontology based opinion mining for book reviewsOntology based opinion mining for book reviews
Ontology based opinion mining for book reviews
firzhan naqash
 
What's Appropriate - Research in HCI
What's Appropriate - Research in HCIWhat's Appropriate - Research in HCI
What's Appropriate - Research in HCI
Xin(Jason) Wang
 
1:1 Community Interview Examples & Tips for Libraries
1:1 Community Interview Examples & Tips for Libraries1:1 Community Interview Examples & Tips for Libraries
1:1 Community Interview Examples & Tips for Libraries
WiLS
 
Starting from Scratch: Meaningful Integration of Information Literacy through...
Starting from Scratch: Meaningful Integration of Information Literacy through...Starting from Scratch: Meaningful Integration of Information Literacy through...
Starting from Scratch: Meaningful Integration of Information Literacy through...
Chris Sweet
 
Data Scopes - Towards transparent data research in digital humanities (Digita...
Data Scopes - Towards transparent data research in digital humanities (Digita...Data Scopes - Towards transparent data research in digital humanities (Digita...
Data Scopes - Towards transparent data research in digital humanities (Digita...
Marijn Koolen
 
Multimedia Academic Literacy
Multimedia Academic LiteracyMultimedia Academic Literacy
Multimedia Academic Literacy
Spelman College
 
Estratégia e Empreendedorismo Assessment 01.pptx
Estratégia e Empreendedorismo Assessment 01.pptxEstratégia e Empreendedorismo Assessment 01.pptx
Estratégia e Empreendedorismo Assessment 01.pptx
Sthefan Berwanger
 
Ethnography Research Paper Upcoming Events 1. Philippine Inde.docx
Ethnography Research Paper Upcoming Events 1. Philippine Inde.docxEthnography Research Paper Upcoming Events 1. Philippine Inde.docx
Ethnography Research Paper Upcoming Events 1. Philippine Inde.docx
SANSKAR20
 
Writing good case studies
Writing good case studiesWriting good case studies
Writing good case studies
Miia Kosonen
 
MCN2017 | From Research to Action: Translating User Feedback into Digital Pro...
MCN2017 | From Research to Action: Translating User Feedback into Digital Pro...MCN2017 | From Research to Action: Translating User Feedback into Digital Pro...
MCN2017 | From Research to Action: Translating User Feedback into Digital Pro...
Susan Wigodner
 
MCN2017 | From Research to Action: Translating User Feedback into Digital Pro...
MCN2017 | From Research to Action: Translating User Feedback into Digital Pro...MCN2017 | From Research to Action: Translating User Feedback into Digital Pro...
MCN2017 | From Research to Action: Translating User Feedback into Digital Pro...
Susan Wigodner
 
Google is NOT a Verb
Google is NOT a VerbGoogle is NOT a Verb
Google is NOT a Verb
The International Academy - Amman
 
presentationlitreview-2ND COT PR2.pdf
presentationlitreview-2ND COT PR2.pdfpresentationlitreview-2ND COT PR2.pdf
presentationlitreview-2ND COT PR2.pdf
AnalieCabanlit1
 
Literature Review (Review of Related Literature - Research Methodology)
Literature Review (Review of Related Literature - Research Methodology)Literature Review (Review of Related Literature - Research Methodology)
Literature Review (Review of Related Literature - Research Methodology)
Dilip Barad
 
Review of Related Literature- Step by Step Guide.pptx
Review of Related Literature- Step by Step Guide.pptxReview of Related Literature- Step by Step Guide.pptx
Review of Related Literature- Step by Step Guide.pptx
Hermenio Jr. Cabusog
 
Search in Research, Let's Make it More Complex!
Search in Research, Let's Make it More Complex!Search in Research, Let's Make it More Complex!
Search in Research, Let's Make it More Complex!
Marijn Koolen
 
Kansei cards: A visual tool supporting the investigation, discussion, and rep...
Kansei cards: A visual tool supporting the investigation, discussion, and rep...Kansei cards: A visual tool supporting the investigation, discussion, and rep...
Kansei cards: A visual tool supporting the investigation, discussion, and rep...
Alexandre Gentner
 
IxD Works Miniworkshop: Introduction
IxD Works Miniworkshop: IntroductionIxD Works Miniworkshop: Introduction
IxD Works Miniworkshop: Introduction
InteractionDesign
 

Similar to Narrative-Driven Recommendation for Casual Leisure Needs (20)

Design case methods AECT 2013
Design case methods AECT 2013Design case methods AECT 2013
Design case methods AECT 2013
 
Patron-Driven Programming: Creating a Culture of Participatory Learning in Li...
Patron-Driven Programming: Creating a Culture of Participatory Learning in Li...Patron-Driven Programming: Creating a Culture of Participatory Learning in Li...
Patron-Driven Programming: Creating a Culture of Participatory Learning in Li...
 
Ontology based opinion mining for book reviews
Ontology based opinion mining for book reviewsOntology based opinion mining for book reviews
Ontology based opinion mining for book reviews
 
What's Appropriate - Research in HCI
What's Appropriate - Research in HCIWhat's Appropriate - Research in HCI
What's Appropriate - Research in HCI
 
1:1 Community Interview Examples & Tips for Libraries
1:1 Community Interview Examples & Tips for Libraries1:1 Community Interview Examples & Tips for Libraries
1:1 Community Interview Examples & Tips for Libraries
 
Starting from Scratch: Meaningful Integration of Information Literacy through...
Starting from Scratch: Meaningful Integration of Information Literacy through...Starting from Scratch: Meaningful Integration of Information Literacy through...
Starting from Scratch: Meaningful Integration of Information Literacy through...
 
Data Scopes - Towards transparent data research in digital humanities (Digita...
Data Scopes - Towards transparent data research in digital humanities (Digita...Data Scopes - Towards transparent data research in digital humanities (Digita...
Data Scopes - Towards transparent data research in digital humanities (Digita...
 
Multimedia Academic Literacy
Multimedia Academic LiteracyMultimedia Academic Literacy
Multimedia Academic Literacy
 
Estratégia e Empreendedorismo Assessment 01.pptx
Estratégia e Empreendedorismo Assessment 01.pptxEstratégia e Empreendedorismo Assessment 01.pptx
Estratégia e Empreendedorismo Assessment 01.pptx
 
Ethnography Research Paper Upcoming Events 1. Philippine Inde.docx
Ethnography Research Paper Upcoming Events 1. Philippine Inde.docxEthnography Research Paper Upcoming Events 1. Philippine Inde.docx
Ethnography Research Paper Upcoming Events 1. Philippine Inde.docx
 
Writing good case studies
Writing good case studiesWriting good case studies
Writing good case studies
 
MCN2017 | From Research to Action: Translating User Feedback into Digital Pro...
MCN2017 | From Research to Action: Translating User Feedback into Digital Pro...MCN2017 | From Research to Action: Translating User Feedback into Digital Pro...
MCN2017 | From Research to Action: Translating User Feedback into Digital Pro...
 
MCN2017 | From Research to Action: Translating User Feedback into Digital Pro...
MCN2017 | From Research to Action: Translating User Feedback into Digital Pro...MCN2017 | From Research to Action: Translating User Feedback into Digital Pro...
MCN2017 | From Research to Action: Translating User Feedback into Digital Pro...
 
Google is NOT a Verb
Google is NOT a VerbGoogle is NOT a Verb
Google is NOT a Verb
 
presentationlitreview-2ND COT PR2.pdf
presentationlitreview-2ND COT PR2.pdfpresentationlitreview-2ND COT PR2.pdf
presentationlitreview-2ND COT PR2.pdf
 
Literature Review (Review of Related Literature - Research Methodology)
Literature Review (Review of Related Literature - Research Methodology)Literature Review (Review of Related Literature - Research Methodology)
Literature Review (Review of Related Literature - Research Methodology)
 
Review of Related Literature- Step by Step Guide.pptx
Review of Related Literature- Step by Step Guide.pptxReview of Related Literature- Step by Step Guide.pptx
Review of Related Literature- Step by Step Guide.pptx
 
Search in Research, Let's Make it More Complex!
Search in Research, Let's Make it More Complex!Search in Research, Let's Make it More Complex!
Search in Research, Let's Make it More Complex!
 
Kansei cards: A visual tool supporting the investigation, discussion, and rep...
Kansei cards: A visual tool supporting the investigation, discussion, and rep...Kansei cards: A visual tool supporting the investigation, discussion, and rep...
Kansei cards: A visual tool supporting the investigation, discussion, and rep...
 
IxD Works Miniworkshop: Introduction
IxD Works Miniworkshop: IntroductionIxD Works Miniworkshop: Introduction
IxD Works Miniworkshop: Introduction
 

More from Marijn Koolen

Recommender Systems NL Meetup
Recommender Systems NL MeetupRecommender Systems NL Meetup
Recommender Systems NL Meetup
Marijn Koolen
 
Digital History - Maritieme Carrieres bij de VOC
Digital History - Maritieme Carrieres bij de VOCDigital History - Maritieme Carrieres bij de VOC
Digital History - Maritieme Carrieres bij de VOC
Marijn Koolen
 
Facilitating reusable third-party annotations in the digital edition
Facilitating reusable third-party annotations in the digital editionFacilitating reusable third-party annotations in the digital edition
Facilitating reusable third-party annotations in the digital edition
Marijn Koolen
 
Scholary Web Annotation - HuC Live 2018
Scholary Web Annotation - HuC Live 2018Scholary Web Annotation - HuC Live 2018
Scholary Web Annotation - HuC Live 2018
Marijn Koolen
 
A hands-on approach to digital tool criticism: Tools for (self-)reflection
A hands-on approach to digital tool criticism: Tools for (self-)reflectionA hands-on approach to digital tool criticism: Tools for (self-)reflection
A hands-on approach to digital tool criticism: Tools for (self-)reflection
Marijn Koolen
 
Lessons Learned from a Digital Tool Criticism Workshop
Lessons Learned from a Digital Tool Criticism WorkshopLessons Learned from a Digital Tool Criticism Workshop
Lessons Learned from a Digital Tool Criticism Workshop
Marijn Koolen
 

More from Marijn Koolen (6)

Recommender Systems NL Meetup
Recommender Systems NL MeetupRecommender Systems NL Meetup
Recommender Systems NL Meetup
 
Digital History - Maritieme Carrieres bij de VOC
Digital History - Maritieme Carrieres bij de VOCDigital History - Maritieme Carrieres bij de VOC
Digital History - Maritieme Carrieres bij de VOC
 
Facilitating reusable third-party annotations in the digital edition
Facilitating reusable third-party annotations in the digital editionFacilitating reusable third-party annotations in the digital edition
Facilitating reusable third-party annotations in the digital edition
 
Scholary Web Annotation - HuC Live 2018
Scholary Web Annotation - HuC Live 2018Scholary Web Annotation - HuC Live 2018
Scholary Web Annotation - HuC Live 2018
 
A hands-on approach to digital tool criticism: Tools for (self-)reflection
A hands-on approach to digital tool criticism: Tools for (self-)reflectionA hands-on approach to digital tool criticism: Tools for (self-)reflection
A hands-on approach to digital tool criticism: Tools for (self-)reflection
 
Lessons Learned from a Digital Tool Criticism Workshop
Lessons Learned from a Digital Tool Criticism WorkshopLessons Learned from a Digital Tool Criticism Workshop
Lessons Learned from a Digital Tool Criticism Workshop
 

Recently uploaded

20240710 ACMJ Diagrams Set 3.docx . Apache, Csharp, Mysql, Javascript stack a...
20240710 ACMJ Diagrams Set 3.docx . Apache, Csharp, Mysql, Javascript stack a...20240710 ACMJ Diagrams Set 3.docx . Apache, Csharp, Mysql, Javascript stack a...
20240710 ACMJ Diagrams Set 3.docx . Apache, Csharp, Mysql, Javascript stack a...
Sharon Liu
 
The Dynamical Origins of the Dark Comets and a Proposed Evolutionary Track
The Dynamical Origins of the Dark Comets and a Proposed Evolutionary TrackThe Dynamical Origins of the Dark Comets and a Proposed Evolutionary Track
The Dynamical Origins of the Dark Comets and a Proposed Evolutionary Track
Sérgio Sacani
 
smallintestinedisorders-causessymptoms-240626042532-363e8392.pptx
smallintestinedisorders-causessymptoms-240626042532-363e8392.pptxsmallintestinedisorders-causessymptoms-240626042532-363e8392.pptx
smallintestinedisorders-causessymptoms-240626042532-363e8392.pptx
muralinath2
 
Review Article:- A REVIEW ON RADIOISOTOPES IN CANCER THERAPY
Review Article:- A REVIEW ON RADIOISOTOPES IN CANCER THERAPYReview Article:- A REVIEW ON RADIOISOTOPES IN CANCER THERAPY
Review Article:- A REVIEW ON RADIOISOTOPES IN CANCER THERAPY
niranjangiri009
 
Anatomy, and reproduction of Gnetum.pptx
Anatomy, and reproduction of Gnetum.pptxAnatomy, and reproduction of Gnetum.pptx
Anatomy, and reproduction of Gnetum.pptx
karthiksaran8
 
antenna-fundamentals an introductions to basics
antenna-fundamentals an introductions to basicsantenna-fundamentals an introductions to basics
antenna-fundamentals an introductions to basics
drphrao1
 
[1] Data Mining - Concepts and Techniques (3rd Ed).pdf
[1] Data Mining - Concepts and Techniques (3rd Ed).pdf[1] Data Mining - Concepts and Techniques (3rd Ed).pdf
[1] Data Mining - Concepts and Techniques (3rd Ed).pdf
PANDURANGLAWATE1
 
ANTIGENS_.pptx ( Ranjitha SL) PRESENTATION SLIDE
ANTIGENS_.pptx ( Ranjitha SL) PRESENTATION SLIDEANTIGENS_.pptx ( Ranjitha SL) PRESENTATION SLIDE
ANTIGENS_.pptx ( Ranjitha SL) PRESENTATION SLIDE
RanjithaSL
 
seed drying lecture, different types of dryers
seed drying lecture, different types of dryersseed drying lecture, different types of dryers
seed drying lecture, different types of dryers
Rammehargahlot1
 
Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, ...
Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, ...Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, ...
Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, ...
Hossein Fani
 
BIOPHYSICS Interactions of molecules in 3-D space-determining binding and.pptx
BIOPHYSICS Interactions of molecules in 3-D space-determining binding and.pptxBIOPHYSICS Interactions of molecules in 3-D space-determining binding and.pptx
BIOPHYSICS Interactions of molecules in 3-D space-determining binding and.pptx
alishyt102010
 
Phytoremediation: Harnessing Nature's Power with Phytoremediation
Phytoremediation: Harnessing Nature's Power with PhytoremediationPhytoremediation: Harnessing Nature's Power with Phytoremediation
Phytoremediation: Harnessing Nature's Power with Phytoremediation
Gurjant Singh
 
largeintestinepathologiesconditions-240627071428-3c936a47 (2).pptx
largeintestinepathologiesconditions-240627071428-3c936a47 (2).pptxlargeintestinepathologiesconditions-240627071428-3c936a47 (2).pptx
largeintestinepathologiesconditions-240627071428-3c936a47 (2).pptx
muralinath2
 
A mature quasar at cosmic dawn revealed by JWST rest-frame infrared spectroscopy
A mature quasar at cosmic dawn revealed by JWST rest-frame infrared spectroscopyA mature quasar at cosmic dawn revealed by JWST rest-frame infrared spectroscopy
A mature quasar at cosmic dawn revealed by JWST rest-frame infrared spectroscopy
Sérgio Sacani
 
Modelling, Simulation, and Computer-aided Design in Computational, Evolutiona...
Modelling, Simulation, and Computer-aided Design in Computational, Evolutiona...Modelling, Simulation, and Computer-aided Design in Computational, Evolutiona...
Modelling, Simulation, and Computer-aided Design in Computational, Evolutiona...
University of Maribor
 
THE ESSENCE OF CHANGE CHAPTER ,energy,conversion,life is easy,laws of physics
THE ESSENCE OF CHANGE CHAPTER ,energy,conversion,life is easy,laws of physicsTHE ESSENCE OF CHANGE CHAPTER ,energy,conversion,life is easy,laws of physics
THE ESSENCE OF CHANGE CHAPTER ,energy,conversion,life is easy,laws of physics
Dr. sreeremya S
 
Possible Anthropogenic Contributions to the LAMP-observed Surficial Icy Regol...
Possible Anthropogenic Contributions to the LAMP-observed Surficial Icy Regol...Possible Anthropogenic Contributions to the LAMP-observed Surficial Icy Regol...
Possible Anthropogenic Contributions to the LAMP-observed Surficial Icy Regol...
Sérgio Sacani
 
Introduction to Artificial Intelligence.pdf
Introduction to Artificial Intelligence.pdfIntroduction to Artificial Intelligence.pdf
Introduction to Artificial Intelligence.pdf
kaavyashreegoskula
 
SCIENCEgfvhvhvkjkbbjjbbjvhvhvhvjkvjvjvjj.pptx
SCIENCEgfvhvhvkjkbbjjbbjvhvhvhvjkvjvjvjj.pptxSCIENCEgfvhvhvkjkbbjjbbjvhvhvhvjkvjvjvjj.pptx
SCIENCEgfvhvhvkjkbbjjbbjvhvhvhvjkvjvjvjj.pptx
WALTONMARBRUCAL
 
Probing the northern Kaapvaal craton root with mantle-derived xenocrysts from...
Probing the northern Kaapvaal craton root with mantle-derived xenocrysts from...Probing the northern Kaapvaal craton root with mantle-derived xenocrysts from...
Probing the northern Kaapvaal craton root with mantle-derived xenocrysts from...
James AH Campbell
 

Recently uploaded (20)

20240710 ACMJ Diagrams Set 3.docx . Apache, Csharp, Mysql, Javascript stack a...
20240710 ACMJ Diagrams Set 3.docx . Apache, Csharp, Mysql, Javascript stack a...20240710 ACMJ Diagrams Set 3.docx . Apache, Csharp, Mysql, Javascript stack a...
20240710 ACMJ Diagrams Set 3.docx . Apache, Csharp, Mysql, Javascript stack a...
 
The Dynamical Origins of the Dark Comets and a Proposed Evolutionary Track
The Dynamical Origins of the Dark Comets and a Proposed Evolutionary TrackThe Dynamical Origins of the Dark Comets and a Proposed Evolutionary Track
The Dynamical Origins of the Dark Comets and a Proposed Evolutionary Track
 
smallintestinedisorders-causessymptoms-240626042532-363e8392.pptx
smallintestinedisorders-causessymptoms-240626042532-363e8392.pptxsmallintestinedisorders-causessymptoms-240626042532-363e8392.pptx
smallintestinedisorders-causessymptoms-240626042532-363e8392.pptx
 
Review Article:- A REVIEW ON RADIOISOTOPES IN CANCER THERAPY
Review Article:- A REVIEW ON RADIOISOTOPES IN CANCER THERAPYReview Article:- A REVIEW ON RADIOISOTOPES IN CANCER THERAPY
Review Article:- A REVIEW ON RADIOISOTOPES IN CANCER THERAPY
 
Anatomy, and reproduction of Gnetum.pptx
Anatomy, and reproduction of Gnetum.pptxAnatomy, and reproduction of Gnetum.pptx
Anatomy, and reproduction of Gnetum.pptx
 
antenna-fundamentals an introductions to basics
antenna-fundamentals an introductions to basicsantenna-fundamentals an introductions to basics
antenna-fundamentals an introductions to basics
 
[1] Data Mining - Concepts and Techniques (3rd Ed).pdf
[1] Data Mining - Concepts and Techniques (3rd Ed).pdf[1] Data Mining - Concepts and Techniques (3rd Ed).pdf
[1] Data Mining - Concepts and Techniques (3rd Ed).pdf
 
ANTIGENS_.pptx ( Ranjitha SL) PRESENTATION SLIDE
ANTIGENS_.pptx ( Ranjitha SL) PRESENTATION SLIDEANTIGENS_.pptx ( Ranjitha SL) PRESENTATION SLIDE
ANTIGENS_.pptx ( Ranjitha SL) PRESENTATION SLIDE
 
seed drying lecture, different types of dryers
seed drying lecture, different types of dryersseed drying lecture, different types of dryers
seed drying lecture, different types of dryers
 
Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, ...
Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, ...Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, ...
Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, ...
 
BIOPHYSICS Interactions of molecules in 3-D space-determining binding and.pptx
BIOPHYSICS Interactions of molecules in 3-D space-determining binding and.pptxBIOPHYSICS Interactions of molecules in 3-D space-determining binding and.pptx
BIOPHYSICS Interactions of molecules in 3-D space-determining binding and.pptx
 
Phytoremediation: Harnessing Nature's Power with Phytoremediation
Phytoremediation: Harnessing Nature's Power with PhytoremediationPhytoremediation: Harnessing Nature's Power with Phytoremediation
Phytoremediation: Harnessing Nature's Power with Phytoremediation
 
largeintestinepathologiesconditions-240627071428-3c936a47 (2).pptx
largeintestinepathologiesconditions-240627071428-3c936a47 (2).pptxlargeintestinepathologiesconditions-240627071428-3c936a47 (2).pptx
largeintestinepathologiesconditions-240627071428-3c936a47 (2).pptx
 
A mature quasar at cosmic dawn revealed by JWST rest-frame infrared spectroscopy
A mature quasar at cosmic dawn revealed by JWST rest-frame infrared spectroscopyA mature quasar at cosmic dawn revealed by JWST rest-frame infrared spectroscopy
A mature quasar at cosmic dawn revealed by JWST rest-frame infrared spectroscopy
 
Modelling, Simulation, and Computer-aided Design in Computational, Evolutiona...
Modelling, Simulation, and Computer-aided Design in Computational, Evolutiona...Modelling, Simulation, and Computer-aided Design in Computational, Evolutiona...
Modelling, Simulation, and Computer-aided Design in Computational, Evolutiona...
 
THE ESSENCE OF CHANGE CHAPTER ,energy,conversion,life is easy,laws of physics
THE ESSENCE OF CHANGE CHAPTER ,energy,conversion,life is easy,laws of physicsTHE ESSENCE OF CHANGE CHAPTER ,energy,conversion,life is easy,laws of physics
THE ESSENCE OF CHANGE CHAPTER ,energy,conversion,life is easy,laws of physics
 
Possible Anthropogenic Contributions to the LAMP-observed Surficial Icy Regol...
Possible Anthropogenic Contributions to the LAMP-observed Surficial Icy Regol...Possible Anthropogenic Contributions to the LAMP-observed Surficial Icy Regol...
Possible Anthropogenic Contributions to the LAMP-observed Surficial Icy Regol...
 
Introduction to Artificial Intelligence.pdf
Introduction to Artificial Intelligence.pdfIntroduction to Artificial Intelligence.pdf
Introduction to Artificial Intelligence.pdf
 
SCIENCEgfvhvhvkjkbbjjbbjvhvhvhvjkvjvjvjj.pptx
SCIENCEgfvhvhvkjkbbjjbbjvhvhvhvjkvjvjvjj.pptxSCIENCEgfvhvhvkjkbbjjbbjvhvhvhvjkvjvjvjj.pptx
SCIENCEgfvhvhvkjkbbjjbbjvhvhvhvjkvjvjvjj.pptx
 
Probing the northern Kaapvaal craton root with mantle-derived xenocrysts from...
Probing the northern Kaapvaal craton root with mantle-derived xenocrysts from...Probing the northern Kaapvaal craton root with mantle-derived xenocrysts from...
Probing the northern Kaapvaal craton root with mantle-derived xenocrysts from...
 

Narrative-Driven Recommendation for Casual Leisure Needs

  • 1. Narrative-Driven Recommendation for Complex Leisure Needs Marijn Koolen Royal Netherlands Academy of Arts and Sciences Humanities Cluster Google CSR Workshop, London, 2019-08-28
  • 4. Overview 1. Scenario: Narrative-Driven Recommendation 2. Analyzing Forum Discussions for 4 Leisure Domains 3. Lessons for System Design
  • 6. History of NDR ● Evolved from Social Book Search evaluation campaign (INEX/CLEF 2011-2016) ● Evolved into analysing forum requests as complex information needs ○ Books: Koolen et al. (CIKM 2012, ECIR 2014, ECIR 2015) ○ Movies and Books: Bogers et al. (iConference 2018) ○ Games: Bogers et al. (iConference 2019) ○ Books, Movies, Games, Music: Bogers et al. (in preparation) ● Scenario: Narrative-Driven Recommendation ○ Bogers, Koolen (ACM RecSys 2017, KaRS 2018).
  • 7. Scenario ● NDR (Bogers & Koolen 2017, 2018) is a complex scenario: ○ Narrative description of desired aspects of items ○ User preference info (user profile or example items) ● Related to Conversational Search & Recommendation ○ But human-directed, complex and often vaguely expressed (latent) needs ○ Interactions with conversational agents tend to be simpler (Kang et al. 2017) ■ E.g. more concrete aspects (genre, creator, title, year, …)
  • 8. Search-Recommendation Continuum ● Some requests are pure search ○ “sci-fi books about space traders” ● Some requests are almost pure recommendation ○ “Something as good as David Copperfield“ ● Majority of requests mix search and recommendation ○ “historical fiction set in 17th c. England that I’ll like based on my profile”
  • 9. ● In the book domain latent factors are related to amount of reading experience (Koolen et al. 2015) ○ Novice readers ask for recommendations based on example books (interests are latent) ○ Experienced readers describe detailed content aspects (interests are known) ○ In developing conversation on forums: start from latent factors and discuss examples to tease out more concrete aspects of interest. ● How does this work in other leisure domains? ● And what relevance aspects do discussion forum users mention? Continuum and Latent Interests
  • 10. 2. Analyzing Forum Discussions
  • 11. ● We developed a relevance aspect model for leisure needs ○ Grounded in actual forum requests ● Data from a range of discussion forums ○ Books: 503 requests (LibraryThing forums) ○ Movies: 538 requests (IMDB forums) ○ Games: 521 requests (reddit) ○ Music: 589 requests (reddit) Comparing Domains
  • 19. 3. Lessons for System Design
  • 20. Capturing Needs - Conversational Recommenders ● Complex narrative is difficult to interpret algorithmically ○ Possible interaction: conversational models for iterative structuring ● Kang et al., (RecSys 2017) look at queries in conversational movie recommendation ○ Many follow-up queries for refinement (clarify, constrain), reformulation ○ Objective: genre (“superhero movies”), deep features (“movies with open endings or plot twists”) ○ Subjective: emotion (“sad movie”), quality (“interesting characters, clever plot”), familiarity (“what would you recommend to a fan of Big Lebowski?”)
  • 21. ● User reviews are highly effective ○ for Narrative-Driven Recommendation (Bogers & Koolen 2018) ... ○ ... and also many search tasks (Koolen et al. 2012, Koolen 2014, Koolen et al. 2015) ● Why? ○ written in the language of the user (same as request) ○ discuss broad range of aspects… ○ … including reading/watching/listening/playing experience! ● Potential for conversational mode: ○ Background model for how people talk about books/films/games/music ○ “This review makes me want to buy this film.” i. “What aspect of this review triggered your interest?” The Effectiveness of User Reviews
  • 22. Explanations From Reviews ● Reviews as a source of explanations ○ “Several reviewers say this book/film changed their views.” ● Lu et al. (RecSys 2018) used adversarial s2s learning to generate explanations from user reviews ○ generate review for new item based on own reviews of consumed items ● Possible interaction: ○ User mentions examples, systems generates recommendations + explanations ○ User points out aspects of explanations they’re interested in, system refines recommendation and explanations
  • 23. Experience, Appeal and Impact ● How to extract information about experience or impact ○ Saricks (2005) identified appeal elements based on style, characterization, plot, pace ● We’re currently developing an impact model (for reviews in Dutch) ○ Identify and extract expressions of impact from user reviews (with Peter Boot, in preparation) i. “I couldn’t put the book down and finished it in one go.” ii. “I really started to understand the main character.” ○ Impact type: emotional (not just binary sentiment) and cerebral (changing your views, motivating you, bringing up memories) ○ Impact cause: e.g. style, narrative, reflection ○ On 400K reviews, inverse relationship between narrative absorption and stylistic/reflective impact
  • 24. Conclusions ● Narrative-Driven Recommendation is a challenging task. ○ Complex but common recommendation/search need ○ Requires combination of data sources and algorithms to solve. ○ Conversational mode useful for identifying and ranking relevance aspects ○ User-generated content essential for good performance ● Next steps: other domains, more advanced NLU analysis, interaction models
  • 25. ● Bogers, Koolen (RecSys 2017). Defining and Supporting Narrative-driven Recommendation ● Bogers, Koolen (KaRS 2018). “I’m looking for something like …”: Combining Narratives and Example Items for Narrative-driven Book Recommendation ● Bogers et al. (iConf 2018). "What was this Movie About this Chick?" - A Comparative Study of Relevance Aspects in Book and Movie Discovery ● Bogers et al. (iConf 2019). "Looking for an amazing game I can relax and sink hours into..." - A Study of Relevance Aspects in Video Game Discovery ● Bogers, Petras (iConf 2015). Tagging vs. Controlled Vocabulary: Which is More Helpful for Book Search? ● Bogers, Petras (iConf 2017). An in-depth analysis of tags and controlled metadata for book search ● Kang et al. (RecSys 2017). Understanding How People Use Natural Language to Ask for Recommendations ● Koolen et al. (CIKM 2012). Social book search: comparing topical relevance judgements and book suggestions for evaluation ● Koolen, M. (ECIR 2014). “User reviews in the search index? That’ll never work!” ● Koolen et al. (ECIR 2015). Looking for Books in Social Media: An Analysis of Complex Search Requests ● Kula (CBRecSys 2015). Metadata Embeddings for User and Item Cold-start Recommendations ● Lu et al. (RecSys 2018). Why I like it: Multi-task Learning for Recommendation and Explanation ● Reuter (JASIS 2007). Assessing aesthetic relevance: Children's book selection in a digital library ● Saricks (2005). Readers' advisory service in the public library ● Weston et al. (IJCAI 2011). WSABIE: Scaling Up To Large Vocabulary Image Annotation References
  • 26. Thank You! ● Acknowledgements: ○ collaborative work with Toine Bogers, Peter Boot, Maria Gäde, Jaap Kamps, Vivien Petras, Mette Skov ● Questions?