1. A CASE BASED REASONING
MODEL FOR MULTILINGUAL
LANGUAGE GENERATION IN
DIALOGUES
Víctor López Salazar, Eduardo M. Eisman Cabeza, Juan Luis Castro
Peña, Jose Manuel Zurita López, 2012
Zelia Blaga
2. Definitions
• Conversational Agent = computer system meant to have a
dialogue with the user
• Dialogue = text, speech, graphics, haptics, gestures
• Natural Language Generation = translator that converts data to a
natural language representation
• Natural language = any language that has evolved naturally in
humans
• Case Based Reasoning = solving new problems based on solutions
of past similar problems
2
Introduction
3. Examples
• Ada and Grace: Responsive
Virtual Human Museum Guides,
Boston, 2009 [1]
• Make a visit to a museum more
interesting by answering visitor
questions, suggesting exhibitions
and explaining the technology
behind the products
3
Introduction
4. Examples
• Freudbot [2]
• Constructed to determine whether a famous person application
of chatbot technology could improve student-content interaction
in distance education
4
Introduction
5. Examples
• Cassandra [3]
• Coordinates all aspects of the conversation: speech recognition,
speech synthesis and avatar display
• Has a persistent memory of previous conversational states and
data, inheritance of base behaviors and response generation
variation.
5
Introduction
6. Examples
• Siri [4], Alexa [5], Cortana [6]
• Goal-oriented
• Help people to solve everyday
problems using natural language
• Based on deep neural networks
6
Introduction
7. Examples
• MicrosoftTay bot [7]
• General conversation
• Attempt to talk with people on a wide range of topics
• Based on deep neural networks
7
Introduction
8. Motivation
• ConversationalAgents - display intelligent behavior by assisting
users to use the interface, asking remedial questions, and giving
relevant answers
• Shape the users’ behavior by using alternative phrasing of
utterances, engaging the user through display of embodiment and
by responding to user affect through text, voice and gesture
• ConversationalAgents are not necessarily web-based; also present
on various platforms, such as mobile phones, personal digital
assistants and other mobile devices
8
Motivation
9. Background
• The term ConversationalAgent was introduced in 1994, in a
paper ”AnimatedConversation: Rule-BasedGeneration of Facial
Expression, Gesture and Spoken Intonation for Multiple
Conversational Agents”, written by Justine Cassell, Catherine
Pelachaud and their colleagues from Center for Human Modeling
and Simulation at the University of Pennsylvania [8]
9
Motivation
10. Usage
• Embodied Conversational Agents (ECA)- interfaces that allow
users to interact with their devices in a natural way
• Educational field:
Tutoring systems
Conversational games testing emotional abilities
Embodied agents to simulate different roles in a professional
environment
10
Related work
11. Autotutor
• Graesser, Chipman, Haynes, &Olney, 2005 [9]
• Intelligent tutoring system that holds conversations with the
human in natural language
• Multiple domains (e.g., computer literacy, physics, critical thinking)
11
Related work
13. Tasks
• Natural Language Understanding
• Dialogue Management
• Natural LanguageGeneration (NLG)
13
Related work
14. Approaches for NLG
• There is no standard technique, depends a lot on problem domain
Canned text
Templates
Symbolic approaches employing knowledge representations
at different linguistic levels and rules
14
Related work
15. Canned text
• Advantage: simple approach; only needs the final text to be
generated; copy & paste
• Disadvantage: not reusable
• Example: horoscope machines or generators of personalized
business letters
15
Related work
16. Templates
• Advantage: mix fixed text with variable length text; general – does
not need to pre-generate all the systems answers
• Disadvantage: not reusable for other situations than what it was
designed for initially
• Example: ELIZA
16
Related work
17. Symbolic representations
• Advantage: linguistic knowledge as grammars or rhetorical
operators to describe the part of the language used by the system;
more generic
• Disadvantage: raised complexity
• Example: evidence, justification, antithesis, concession,
circumstance, elaboration, background, purpose, condition,
contrast etc.
17
Related work
18. AIML
• Artificial Intelligence Markup Language [10]
• Well-known language to develop conversational agents
• Answer generation based on stimulus-response scheme
• Uses pattern matching for recognizing user input
• Tags: “topic” and “that”
• Disadvantage: insufficient to establish the contents of the answer
18
Related work
19. AIML
• The most important units of AIML are:
<aiml>: the tag that begins and ends anAIML document
<category>: the tag that marks a "unit of knowledge" in an
Alicebot's knowledge base
<pattern>: used to contain a simple pattern that matches what a
user may say or type to an Alicebot
<template>: contains the response to a user input
19
Related work
20. AIML
<category>
<pattern>WHAT ARE YOU</pattern>
<template>
<think><set name="topic">Me</set></think>
I am the latest result in artificial intelligence, which can
reproduce the capabilities of the human brain with greater speed
and accuracy.
</template>
</category>
20
Related work
21. AIML extension
• Kimura and Kitamura, 2006 [11]
• Incorporates SPARQL queries to extract sentences from web pages
annotated with RDF
• The agent is more dynamic
21
Related work
22. Genetic programming algorithm
• Lim and Cho, 2005 [12]
• The answers are more varied using Sentence PlanTrees (STP)
• STP = binary trees containing templates in their leaves and joint
operators joining these sentences in their parent nodes
• Crossing and mutating operators to create new sentences
22
Related work
23. ProtoPropp
• Gervas, Diaz-Agudo, Peinado and Hervas, 2005 [13]
• Story plot generation
• CBR technique to build a story from an initial description of the plot
• Allows following the history until a certain point (e.g. until an
objective has been reached)
• NLG module – selects the content to be included, structures the
discourse, presentation of facts based on priorities
23
Related work
24. PERSONAGE
• Mairesse andWalker, 2010 [14]
• Psychologically motivated, parameterizable NLG able to produce
texts with different styles associated with different aspects of
personality
• Components: content planner, sentence planner and realization
module
24
Related work
26. Literature review
• [15]
• Symbolic conversational systems must have a semantic level and a
lexical level
• Hard to adapt from one domain to another
• Requires theoretic knowledge about linguistics
• Must adapt the rules for each language
26
Related work
27. Objectives
• Develop a NLG system used for dialogues which does not require a
deep linguistic knowledge and which could be reused in many
domains without changing the system’s functionality
• Context: virtual simulated patient that maintains dialogues in
several languages
27
Solution
28. Tasks
• Macro-planning = content determination + discourse planning
• Micro-planning – how to describe an entity; reference,
aggregation, lexicalization
• Surface Realization – impose grammatical constructions
• Task is to maintain a conversation, keeping track of the intentions
and beliefs of the hearer and our own ones => micro-planning tasks
are primary, and the produced text is shorter
28
Solution
29. Terminology
• Communication Objective = intention of the sentence; has priority
indicating the order between objectives; e.g. “Greeting” or
“Answer”
• Communicative Action = speech acts; has priority indicating the
order between actions; e.g. “Informing” or “Complaining”
• Phrase Restriction = concrete natural language realization of an
entity; e.g. Human -> person, Human -> “homo sapiens”;
constraints indicating the situation
• Phrase = template representing a natural language sentence;
domain entities between ‘#’;
• Case = complex template; communicative objective + set of
ordered phrases references in a text template
29
Solution
34. 1. Case indexation
• Cases indexed by:
1. Communication objective
2. Number of phrases in the case
34
Stages
35. 1. Case indexation
• Steps:
1. Group cases by objective
2. Regroup cases by number of phrases
3. Regroup phrases that share the same actions
• Examples
35
Stages
37. 1. Case indexation
37
Stages
• “I’ve got a strong headache on the left side”
• Template: “I’ve got a #Intensity# #Location##Symptom# on the
#Location#”
• The words “strong”, “head”, “ache”, “left side” are replaced with
references to phrase restrictions linked to the phrase
• Restrictions are associated to the domain entities: Intensity,
Location and Symptom
• These have constraints
• For Location = head we have jointloc = true
• For Location = left side we have dir = left
• Reference to communicative action “Answer”
39. 2. Case retrieving
• Input: semantic answer = communication objective + actions
• Actions have domain entities specifying the content of the action
• The process retrieves the set of cases with the same
communication objective as the semantic answer
• Then it retrieves the subset with the same number of phrases as
the objective’s actions
• Match the action and the entities associated to each phrase
restriction in the cases’ phrase with the action and their entities in
the semantic answer
39
Stages
43. 3. Case adaptation
• After the cases have been retrieved, generate a concrete phrase
• “fill the gaps” of each template of the phrases with concrete phrase
restrictions that meet the action’s entities of the semantic answer
• Final phrase = fill in the template of the phrase in the selected case
with the restrictions
43
Stages
44. 3. Case adaptation
• E.g.
• Take first case from the list,CaseMotive1
• Take templates:
• “I’ve got a #Intensity##Location##Symptom# on the #Location#”
• “#Frequency# my #Location# and #Location# become
#Symptom#’’
44
Stages
45. 3. Case adaptation
• E.g. cont’d
• Get phrase restrictions
• “I’ve got a strong head-ache on the left side”
• “sometimes my hands and tongue become numb”
• Repeat for the other cases
• CaseDisease
• Final answer of the agent: “I’ve got a strong headache on the left
side and sometimes my hands and tongue become numb. I think I
have a cold”
45
Stages
46. Testing
• MultilingualVirtual Simulated Patient (MVSP)
• Simulating a patient in the context of doctor-patient appointment
• The agent takes some sentences in natural language input by
the user and builds a semantic answer that contains references
to entities in a domain of diseases and human body locations
• Languages: Spanish, English,German, Italian, Hungarian, Bulgarian
and Portuguese
46
Results
47. Testing
• Partners did not have specific education in linguistics or
computational grammars
• Task: create an appropriate and natural dialogue
• Cases: introduction, information, error and farewell
47
Results
49. Interface
• Pose question in a semantic way: select from the list-boxes a symptom, a
location (if the symptom has any) and a property (e.g intensity, frequency,
duration)
49
Results
50. Project approach
1. Training – how to use the tool (how to input data, and how to
use the patient as a training tool)
2. Development – ontology generation (‘teaching’ the patient how
to speak, by inputting vocabulary, phrase combinations and
other required information so as to allow the patient to
understand and answer questions)
3. Piloting – usage of the virtual patients in real-life settings
through pilots held in medical classrooms around Europe
• [16]
50
Results
51. Evaluation
• 10 min clinical interview between students (first or second year
college students in medicine) and virtual patient
• Put questions to set the diagnostic (Horton’s migraine)
51
Results
53. Present work
• The results were “acceptable to good”
• Works for multiple languages
• Does not need deep understanding of grammar or linguistic theory
• More practical for students
• Actual functionality: search in a database for a query
• Disadvantage: introduction of new cases is time consuming
• Similar:TheraSim, vSim
53
Conclusions
55. Future work
• Samantha, intelligent computer operating system from the movie
Her (2013), dir. Spike Jonze
• understood how to engage in real dialog, ask questions, follow up
on statements, and even appreciate and offer humor
55
Conclusions
56. Future work
• “A user-perception based approach to create smiling embodied
conversational agents”, MagalieOchs,Catherine Pelachaud and
Gary McKeown (2017) - to improve the social capabilities of
embodied conversational agents [17]
56
Conclusions
59. Bibliography
1. http://ict.usc.edu/prototypes/museum-guides/
2. http://www.chatbots.org/conversational_agent/freudbot/
3. https://www.chatbots.org/chatterbot/cassandra/
4. https://www.apple.com/ios/siri/
5. https://www.alexa.com/
6. https://support.microsoft.com/en-us/help/17214/windows-10-what-is
7. https://en.wikipedia.org/wiki/Tay_(bot)
8. https://www.chatbots.org/conversational_agent/
9. Graesser, A. C., Chipman, P., Haynes, B. C., & Olney, A. (2005). Autotutor: An intelligent tutoring system with mixed-initiative
dialogue. IEEE Transactions on Education, 48, 612–618
10. Wallace, R. (2000). http://www.pandorabots.com/pandora/pics/wallaceaiml tutorial.html
11. Kimura, M., & Kitamura, Y. (2006). Embodied conversational agent based on semantic web. In Z. Z. Shi & R. Sadananda (Eds.),
Agent Computing and Multi-Agent Systems. Lecture Notes in Artificial Intelligence (vol. 4088, pp. 734–741). Berlin: Springer-
Verlag Berlin
12. Lim, S., & Cho, S.B. (2005). Language generation for conversational agent by evolution of plan trees with genetic programming.
In Torra, V., Narukawa, Y., & Miyamoto S., (Eds.), Modeling decisions for artificial intelligence proceedings, Lecture Notes in
Artificial Intelligence (Vol. 3558, pp. 305–315).
13. Gervas, P., Diaz-Agudo, B., Peinado, F., & Hervas, R. (2005). Story plot generation based on cbr. Knowledge-Based Systems, 18,
235–242.
14. Mairesse, F., & Walker, M. (2010). Towards personality-based user adaptation: psychologically informed stylistic language
generation. User Modeling and User-Adapted Interaction, 20, 227–278
15. Víctor López Salazar, Eduardo M. Eisman Cabeza, Juan Luis Castro Peña, and Jose Manuel Zurita López. 2012. A case based
reasoning model for multilingual language generation in dialogues. Expert Syst. Appl. 39, 8 (June 2012), 7330-7337.
16. Multilingual Virtual Simulated Patient Project, Progress Report
17. Magalie Ochs, Catherine Pelachaud, and Gary Mckeown. 2017. A User Perception--Based Approach to Create Smiling Embodied
Conversational Agents. ACM Trans. Interact. Intell. Syst. 7, 1, Article 4 (January 2017), 33 pages
59
