The document discusses creating empathetic conversational experiences through affective intelligence and mental state inference. It describes how sentiment analysis, emotion detection, and mental state tracking can be used to develop empathic dialog policies that improve the user's affective state. The system utilizes commonsense knowledge bases and reasoning to predict how events affect a user's emotions and motivations. The goal is to create more engaging experiences for users by understanding their perspective and responding supportively and understandingly.

1. Beyond Sentiment Analysis: Creating Engaging
Conversational Experiences through Empathy
Will Kearns
Co-founder & CTO, COCO

2. A telenursing and virtual therapy platform
for family caregivers:
● On-demand caregiving support
● Interactive health and wellness counseling

3. Overview
● Affective Intelligence
● Empathetic Dialog
● Integrating with Rasa

4. Terminology
● Natural Language Understanding (NLU)
● Intents
● Slots
● State Tracker
● Dialog Policies
● Natural Language Generation (NLG)

5. Affective Empathy 💞😱
The ability to experience how
someone else may feel in response
to an event.
Cognitive Empathy 🧠🧐
The ability to reason about how an
event may likely affect the mental
state of someone else.

6. Sentiment Analysis
Custom
Action
Oh hi - I’m working late
and haven't been able to
get the paperwork to my
child's doctor.
Sentiment
Analysis
Text
Response
-0.75
-0.75 < -0.50
I’m sorry to hear
that.

7. Button Selection
Dialog
Policy
Tired
Stretch Exercise
Got it! Would you like to take a
moment to do a stretching
exercise to reboot your
energy?
Button
Response

8. Emotion Detection
Dialog
Policy
Tired
It sounds like you may be
feeling tired. Would you like to
take a moment to do a
stretching exercise to reboot
your energy?
Emotion
Detection
Text
Response
Oh hi - I’m working late
and haven't been able to
get the paperwork to my
child's doctor.
Stretch Exercise

9. Beyond Intents

10. Empathy Driven Dialog
xReact: tired, overwhelmed
xAttr: busy, dedicated
xWant: to go home, finish work
reschedule
It sounds like you're busy and want
to finish work and go home. Would
you like me to check in again after
dinner around 8pm with a
meditation exercise to help ease
your stress?
Mental State
Inference
Empathic
Policy
Text
Response
Oh hi - I’m working late
and haven't been able to
get the paperwork to my
child's doctor.

11. Mental State Inference
Predict the effect of events on a
user’s emotions, motivations,
and perceptions.
Context: What solutions have you tried?
Utterance: I’m trying everything for Connor.
Inference: The caregiver feels overwhelmed.
(caregiver, feels, overwhelmed)

12. Language models (LMs) predict
the next word in a sentence
based on their distribution in a
training dataset which may
lack implicit knowledge
Problem with LMs

13. A commonsense knowledge
base of if-then relationships
between actors and agents
ATOMIC

14. A commonsense reasoning from
transformer based language
models to infer knowledge
beyond ATOMIC
COMET

15. We customize our NLU pipeline with
a COMET Featurizer that extends
the LanguageModelFeaturizer as
well as another custom component
for Mental State Inference.
Mental State Tracking

16. Given a dialog context the
empathic policy determines the
best way to improve the user’s
affective state
Empathic Policy
Inference: The caregiver feels overwhelmed.
(caregiver, feels, overwhelmed)
Action: Normalize
Style: Supportive, Understanding

17. The act of communicating that
users are not alone in their
experience and that their
reactions to situations are
shared by others.
Normalizing

18. The act of putting words to our
emotions can shift activity from
the amygdala to the left
prefrontal cortex, reducing
anxiety and anger.
Affect Labeling

19. Ensure a smooth transition for
the customer through
contextual knowledge transfer
from the bot to the operator
Human Handoff

20. Summary
● Empathy can drastically change a
user’s experience
● Empathy can be enhanced with AI
both in human agents and chatbots

21. Weichao Yuwen, PhD, RN
Clinical Co-Founder & CEO
Nursing Informatics
Sunny Cheng, PhD, RN
Myra Divina, MS-CIPCT
Meet Our Team
Will Kearns, PhD Candidate
Technical Co-Founder & CTO
Human Centered Design
Honson Ling
Lukas Sexton
Rosanna Liu
Kelly Hou
Software Development
Nora Wang
Stanley Wang
Advisors/Mentors
Teresa Ward
Trevor Cohen
Sponsors
UW Comotion
UW Population Health Initiative

22. Email: will@coco.health
Linkedin: kearnsw
Clubhouse: kearnsw
Website: https://coco.health
Contact info