Recommended
More Related Content
Similar to A domain-independent Framework for building Conversational Recommender Systems
Similar to A domain-independent Framework for building Conversational Recommender Systems (20)
Recently uploaded
Recently uploaded (20)
A domain-independent Framework for building Conversational Recommender Systems
- 1. A domain-independent Framework for building Conversational Recommender Systems Fedelucio Narducci, Pierpaolo Basile, Andrea Iovine, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro Oct 7, 2018 - VANCOUVER
- 2. BACKGROUND ➤ Conversational Recommender Systems (CoRSs) belong to the class of dialog agents and interact with the user during the recommendation process ➤ CoRSs guide the users through an interactive dialog ➤ The preference acquisition is an incremental process that does not have to be necessarily finalized in one step A domain-independent Framework for building Conversational Recommender Systems
- 3. PROBLEMS ➤ Developing dialog agents is becoming more and more popular for customer care, health care, medical diagnoses ➤ The implementation is a complex task since it requires knowledge about NLP, HCI, machine learning A domain-independent Framework for building Conversational Recommender Systems
- 4. A POSSIBLE SOLUTION ➤ a domain- independent framework for building conversational recommender systems A domain-independent Framework for building Conversational Recommender Systems
- 5. CAPABILITIES ➤ acquire preferences ➤ explore user profile ➤ provide feedback ➤explain recommendations ➤interaction modes ➤ natural language ➤ buttons ➤ hybrid A domain-independent Framework for building Conversational Recommender Systems
- 6. ARCHITECTURE A domain-independent Framework for building Conversational Recommender Systems Intent Recognizer Dialog Manager Entity Recognizer Sentiment Analyzer Recommendation Services
- 7. ARCHITECTURE: DIALOG MANAGER A domain-independent Framework for building Conversational Recommender Systems Intent Recognizer Dialog Manager Entity Recognizer Sentiment Analyzer Recommendation Services ➤core component of the framework ➤supervises the whole recommendation process ➤ keeps track of the dialog state ➤receives/sends messages from/to the user ➤ is completely independent from the client (JSON message)
- 8. ARCHITECTURE: INTENT RECOGNIZER A domain-independent Framework for building Conversational Recommender Systems Intent Recognizer Dialog Manager Entity Recognizer Sentiment Analyzer Recommendation Services ➤ defines the intent of the user expressed by a natural language sentence ➤ is based on DialogFlow ➤ four user intents ➤ preference ➤ recommendation ➤ show profile ➤ help
- 9. ARCHITECTURE: INTENT RECOGNIZER A domain-independent Framework for building Conversational Recommender Systems Intent Recognizer Dialog Manager Entity Recognizer Sentiment Analyzer Recommendation Services ➤ each intent can be composed of set of sub-intents ➤ show profile ➤ delete preference ➤ update preference ➤ reset profile
- 10. ARCHITECTURE: SENTIMENT ANALYZER A domain-independent Framework for building Conversational Recommender Systems Intent Recognizer Dialog Manager Entity Recognizer Sentiment Analyzer Recommendation Services ➤ Sentiment Tagger of Stanford CoreNLP ➤ returns the sentiment tags identified in a sentence ➤ links the sentiment tag to an entity in the sentence ➤ I like The Matrix, but I hate Keanu Reeves
- 11. ARCHITECTURE: SENTIMENT ANALYZER A domain-independent Framework for building Conversational Recommender Systems Intent Recognizer Dialog Manager Entity Recognizer Sentiment Analyzer Recommendation Services ➤ Sentiment Tagger of Stanford CoreNLP ➤ returns the sentiment tags identified in a sentence ➤ links the sentiment tag to an entity in the sentence ➤ I like The Matrix, but I hate Keanu Reeves
- 12. ARCHITECTURE: ENTITY RECOGNIZER A domain-independent Framework for building Conversational Recommender Systems Intent Recognizer Dialog Manager Entity Recognizer Sentiment Analyzer Recommendation Services ➤finds entities in the user sentence ➤links them to the Knowledge Base ➤Wikidata ➤annotated data not required for training ➤recognizes alias Steven Allan Spielberg, Spielberg, Steven Spielberg
- 13. ARCHITECTURE: ENTITY RECOGNIZER - PROBLEMS A domain-independent Framework for building Conversational Recommender Systems Intent Recognizer Dialog Manager Entity Recognizer Sentiment Analyzer Recommendation Services ➤ different surfaces forms can refer to the same entity ➤ Steven Spielberg, Spielberg -> Steven_Spielberg:director ➤ the same surface form can refer to more than one entities ➤ Spielberg -> Steven_Spielberg:director Sasha_Spielberg:actor
- 14. ARCHITECTURE: ENTITY RECOGNIZER@WORK A domain-independent Framework for building Conversational Recommender Systems ➤Step 1 ➤ I like Spielberg and Jurassic Park ~ I like Jurassic Park and its director Spielberg Steven_Spielberg:director Sasha_Spielberg:actor
- 15. ARCHITECTURE: ENTITY RECOGNIZER@WORK A domain-independent Framework for building Conversational Recommender Systems ➤Step 1 ➤ I like Spielberg and Jurassic Park ~ I like Jurassic Park and its director Spielberg Steven_Spielberg:director Sasha_Spielberg:actor
- 16. ARCHITECTURE: ENTITY RECOGNIZER@WORK A domain-independent Framework for building Conversational Recommender Systems ➤Step 2 ➤ I like Spielberg and Jurassic Park ➤ Surface form: Spielberg ➤ Candidate entities: Steven_Spielberg:director, Sasha_Spielberg:actor ➤ Context: Jurassic Park sim1 (Jurassic Park,Steven_Spielberg:director) = 0.90 sim (Jurassic Park, Sasha_Spielberg:actor)= 0.15 Spielberg = Steven_Spielberg:director 1Maximilian Nickel, Lorenzo Rosasco, Tomaso A Poggio, and others. 2016. Holographic Embeddings of Knowledge Graphs. In The Thirtieth AAAI Conference on Artificial Intelligence (AAAI-16). 1955–1961.
- 17. ARCHITECTURE: RECOMMENDATION SERVICES A domain-independent Framework for building Conversational Recommender Systems Intent Recognizer Dialog Manager Entity Recognizer Sentiment Analyzer Recommendation Services ➤ recommendation algorithm is PageRank with Priors ➤nodes are entities from DBpedia (eg, actors, movies, directors) ➤ Explanation ➤ exploits links between liked items and recommendations in the DBpedia graph
- 18. ARCHITECTURE: RECOMMENDATION SERVICES A domain-independent Framework for building Conversational Recommender Systems Intent Recognizer Dialog Manager Entity Recognizer Sentiment Analyzer Recommendation Services ➤critiquing ➤ the user can provides complex feedback on the recommended items ➤ I like the movie Titanic, but I don’t like the actor Bill Paxton
- 19. EXPLANATION@WORK American Epic Films Tom Hanks Dystopian Films The Wachowskis I recommend you Cloud Atlas because you often like films with Tom Hanks as Saving Private Ryan and Da Vinci Code. In addition, you sometimes like films directed by The Wachowskis as The Matrix. dbpedia-owl:starring dbpedia-owl:starring dcterms:subject! dcterms:subject! dbpedia-owl:director dbpedia-owl:director dcterm s:subject! dbpedia-owl:starring dcterms:subject! A domain-independent Framework for building Conversational Recommender Systems
- 20. THE FRAMEWORK@WORK A domain-independent Framework for building Conversational Recommender Systems ➤ three instances on Telegram (movie, music, and book) ➤ @movierecsys2_bot ➤ @musicrecsys_bot ➤ @bookrecsys_bot
- 21. EXPERIMENTAL EVALUATION Goal to assess the accuracy of each component of our framework and its influence on the recommendation process A domain-independent Framework for building Conversational Recommender Systems
- 22. EXPERIMENTAL EVALUATION: GOAL Test separately ➤ Intent Recognizer ➤ Entity Recognizer ➤ Sentiment Recognizer A domain-independent Framework for building Conversational Recommender Systems
- 23. EXPERIMENTAL EVALUATION: DATASET bAbI by Facebook Research collects utterances like Beauty and the Beast, Aladdin, Schindler’s List, and The Silence of the Lambs are movies I loved. Would you recommend something I might like? A domain-independent Framework for building Conversational Recommender Systems
- 24. EXPERIMENTAL EVALUATION Intent Recognizer test Beauty and the Beast, Aladdin, Schindler’s List, and The Silence of the Lambs are movies I loved. (Preference) Would you recommend something I might like? (Recommendation request) Entities and Sentiments are set programmatically A domain-independent Framework for building Conversational Recommender Systems
- 25. EXPERIMENTAL EVALUATION Entity Recognizer test Beauty and the Beast, Aladdin, Schindler’s List, and The Silence of the Lambs are movies I loved. Would you recommend something I might like? Intents and Sentiments are set programmatically A domain-independent Framework for building Conversational Recommender Systems
- 26. EXPERIMENTAL EVALUATION Sentiment Recognizer test Beauty and the Beast, Aladdin, Schindler’s List, and The Silence of the Lambs are movies I loved. Would you recommend something I might like? Intents and Entities are set programmatically A domain-independent Framework for building Conversational Recommender Systems
- 27. EXPERIMENTAL EVALUATION ➤ Intent Recognizer Test ➤ Entity Recognizer Test ➤ Sentiment Recognizer Test compared to ➤ Upper bound recommendations generated by setting intents, entities, and sentiments by code A domain-independent Framework for building Conversational Recommender Systems
- 28. EXPERIMENTAL EVALUATION: METRIC AND RESULTS HitRate@n: hits/#recommended items n= 5,10,20 HR@5 HR@10 HR@20 Upper Bound 0.75 1.21 1.93 Loss@5 Loss@10 Loss@20 Intent Recognizer -34.00% -30.86% -24.03% Entity Recognizer -46.00% -35.80% -27.13% Sentiment Recognizer -20.00% -16.05% -14.73% ER~ 85% accuracy IR ~ 77% accuracy SR ~ 83% accuracy A domain-independent Framework for building Conversational Recommender Systems
- 29. EXPERIMENTAL EVALUATION: SECOND EXPERIMENT ➤ Dataset released by Grouplens1 ➤ collects recommendation requests of real users to a conversational recommender system ➤ 694 sentences ➤ Results ➤ 7.4% intents (very difficult task, requests like “action movies”,”exploitations films”, ”film with sharks”, ”i’m looking for a hard sci-fi movie”) ➤ 64.39% entities ➤ 1Kang, J., Condiff, K., Chang, S., Konstan, J. A., Terveen, L., & Harper, F. M. (2017, August). Understanding how people use natural langua ask for recommendations. In Proceedings of the Eleventh ACM Conference on Recommender Systems (pp. 229-237). ACM. A domain-independent Framework for building Conversational Recommender Systems
- 30. CONCLUSION AND FUTURE WORK ➤ The component with the highest impact in a conversational recommender is the entity recognizer ➤ Intents are expressed in very variable forms ➤ carry out experiments with real users in different domains ➤ release the framework as open source software ➤ investigate the different capabilities of the framework (critiquing, explanation) A domain-independent Framework for building Conversational Recommender Systems
- 31. QUESTIONS? A domain-independent Framework for building Conversational Recommender Systems fedelucio.narducci@uniba.it