Towards Queryable User Profiles: Introducing Conversational Agents in a Platform for Holistic User Modeling

@cataldomusto
cataldo.musto@uniba.it
Towards Queryable User Profiles:
Introducing Conversational Agents in
a Platform for Holistic User Modeling
CATALDO MUSTO*, FEDELUCIO NARDUCCI^, MARCO POLIGNANO*, PASQUALE LOPS*, MARCO DE GEMMIS*, GIOVANNI SEMERARO*
*UNIVERSITÀ DEGLI STUDI DI BARI ‘ALDO MORO’ – ITALY
^POLYTECHNIC UNIVERSITY OF BARI - ITALY
ExUM 2020 – International
Workshop on Explainable User
Models and Personalised Systems
jointly held with UMAP 2020
Online from Genoa (Italy) – July 18, 2020

2
User Modeling (in the ‘egosystem’ era)
Cataldo Musto, Fedelucio Narducci, Marco Polignano, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Towards Queryable User Profiles:
Introducing Conversational Agents in a Platform for Holistic User Modeling. ExUM Workshop@UMAP 2020 – Online – July 18, 2020

Holistic User Model*
Affects
Demographics
Interests
Behaviors
Social
Relations
Knowledge
and Skills
Physical
States
Psychological
Aspects
6
(*) Musto, C., Polignano, M.,
Semeraro, G. et al. MYRROR: a
platform for holistic user
modeling. User Modeling and
User-Adapted
Interaction (2020).

7
Myrror: a platform for holistic user modeling*
(*) Musto, C., Polignano, M., Semeraro, G. et al. MYRROR: a platform for holistic
user modeling. User Modeling and User-Adapted Interaction (2020).

8

10

11

13

14

15

• Current Issues
1. Overall transparency may be improved
◦ Users should be aware of the information encoded in their
profiles
◦Interaction with a web-based dashboard can be difficult,
especially for beginners
◦ «How can I check my current interests?»
2. It is difficult to get personal «insights» from the current
dashboard
◦ «Did I sleep enough?»
16

Introducing Conversational Agents in a Platform for Holistic User Modeling. ExUM Workshop@UMAP 2020 – Online – July 18, 2020 17
Goals
To improve the way people access to the
information encoded in their profiles

18
Intuition: Conversational Agents
What is a conversational
agent?
• A tool that understands
natural-language
requests and provides
natural-language answers
to general-purposes
services
• Popularized by Alexa, Siri,
Google Now, etc.
Alexa, play Africa
by Toto.
Here’s Africa by Toto
from Amazon Music.

19
Intuition: Conversational Agents
Intuition
• To exploit Conversational
Agents to allow the user
to query her holistic user
model in natural
language
• Easier to access to the
information encoded
in the profile
• Easier to get insights
Alexa, play Africa
by Toto.
from Amazon Music.
Did I sleep enough?
No, your sleep is not
enough. Currently, you
sleep 6h per day.
A first step towards Queryable User Profiles

21
Contribution: MyrrorBot
MyrrorBot is a conversational
agent built on top of Myrror
Based on HUMs

Based on HUMs
Answers to natural-language
requests concerning the
information encoded in the
HUM
Provides personalized services
Alexa-like (Music, News,
Recipes, etc.)
22

23
Based on HUMs
HUM
Recipes, etc.)

24
MyrrorBot: Intent Recognizer
Based on HUMs
HUM
Recipes, etc.)

25
Our natural language
understanding module is based
on Google DialogFlow.
Input:
Natural Language
Request
Output:
Intent (i.e., a
representation of the
informative need)
Alexa, play
Africa by Toto.
from Amazon Music.
Intent=music

26
Our natural language
understanding module is based
on Google DialogFlow.
Input:
Natural Language
Request
Output:
Intent (i.e., a
representation of the
informative need)
What kind of
questions can
we handle?

29
Intents in MyrrorBot
Demographics
Interests
Affects
Physical States
Social Relations
Behaviors
Knowledge and Skills
Psychological Traits
Intents are directly mapped to the facets of HUMs

30
Intents in MyrrorBot
Demographics
Interests
Affects
Physical States
Social Relations
Behaviors
Knowledge and Skills
Psychological Traits
Each intent is further split into two sub-intents
Informative
(«How much did I
sleep today?»)
Insights
(«Did I sleep
enough today?»)

35
Training the Intent Recognizer
Intent Recognizer is
based on Machine
Learning
For each intent, a set of
training sentences
covering all the
possible utterances
should be provided
We feed the algorithm
with 20-30 sentences
for each intent
Behaviors - Insights Behaviors - Insights

36
Intent Recognizer @Work
Affects - Informative
Psychological Traits - Informative
Behaviors - Insights
Physical States - Insights
Demographics - Informative
What is my mood right now, according
to your data?
Tell me about my personality traits
Did I burn enough calories?
Did I sleep enough today?
Where do I live?
Intent
Recognizer
(NLU)

38
MyrrorBot: Workflow
Based on HUMs
HUM
Recipes, etc.)

40
MyrrorBot: Generator
The Generator module produces a natural
language answer that fulfill user’s
information need.
In case of Informative intent, no computation
is carried out. The profile is queried and a
template is filled in with the values
returned by the HUM.

41
MyrrorBot: Generator
The Generator module
produces a natural language
answer that fulfill user’s
information need.
Differently, intents that aim to
get some insights require some
computation (i.e., compare
current values with some
threshold value) before the
answer is generated.

MyrrorBot: Recap
Acquires a HUM
Recognizes users’ requests
Generate the answer

Experimental Evaluation
Research Question 1
What is the opinion of the users, in terms of usability, ease of use and learning curve?
Research Question 2
Does a conversational interface reduces the time needed to access to the personal information
encoded in a user profile?
Experimental Design
User Study with a Web Application
76 subjects (previous experience with chatbots= 35.5%)
Metrics RQ1: Questions based on SUS (System Usability Survey) Questionnaire
5-points Likert scale
Metrics RQ2: Seconds required to complete a task (web-based vs. conversational interface)
Randomized Order of interaction, Wilcoxon Test

Results – Research Question 1
Metric Question %Compl. Agree %Agree %Others
Learning Curve «I became familiar with
the system very quickly»
51.3% 38.2% 10.5%
Interaction «It was easy to formulate
my questions»
40.8% 48.7% 11.8%
Accuracy «The system provided me
with the right answers»
55.3% 35.5% 9.2%
Daily Use «I would use the system
for my daily tasks»
28.9% 40.8% 29.2%
Switching «I would prefer the system
to a web platform»
40.8% 40.8% 18.4%

Results – Research Question 2
Facet Task MyrrorBot
(sec.)
Myrror
(sec.)
Gap%
Demographics Check your birthday 7.72 10.31 -25.12%
(p<0.001)
Interests Check your interests 9.62 22.11 -56.49%
(p<0.001)
Affects Check your latest mood 10.34 20.02 -48.35%
(p<0.001)
Psychological
Aspects
Check your personality
traits
9.98 12.33 -19.10%
(p<0.05)
45
Significant reduction of the task completion time

Results – Research Question 2 (cont.)
Facet Task MyrrorBot
(sec.)
Myrror
(sec.)
Gap%
Behaviors
Check your daily activities 13.25 24.99 -47.20%
(p<0.001)
Check if you burnt
enough calories
14.18 23.17 -19.05%
(p<0.05)
Physical States
Check if you slept enough 13.44 19.20 -30.47%
(p<0.001)
Check your heart rate 10.20 16.49 -38.14%
(p<0.001)
Significant reduction of the task completion time
46

Research Question 2 – Recap
Avg. Reduction40.3%
11,09
18,58
0,00 2,00 4,00 6,00 8,00 10,00 12,00 14,00 16,00 18,00 20,00
MyrrorBot
Myrror
Average Task Completion Time (sec.)
47

Recap
A first step towards Queryable User Profiles
Starting Point: Myrror
◦ Holistic User Modeling platform
◦ Collects data from several heterogeneous sources
◦ Processes the data and aggregates them to eight pre-defined facets
◦ A Conversational agent built on top of MyrrorBot
◦ Acquires a HUM and recognizes users’ requests (based on an intent recognizer)
◦ Generate natural language answers fulfilling the requests
◦ Informative and Insight-based intents
◦ Good impact on the users. Significant reduction of task completion time.
48

Demo Available!
@cataldomusto
Want to try MyrrorBot?
Contact us!
49

Future Work
Evaluation of personalized services
Evaluation of new interaction
methods
◦ Integration with social robots (e.g.,
Pepper)
◦ Integration with voice-based interfaces
(e.g., Amazon Alexa-like)
Training of more (and more
sophisticated) intents
◦ Goal: to get more useful insights from the
platform (e.g., is my lifestyle correct?)
50

Thank you!
@cataldomusto
Contacts
51

Towards Queryable User Profiles: Introducing Conversational Agents in a Platform for Holistic User Modeling

More Related Content

More from Cataldo Musto

Recently uploaded

Towards Queryable User Profiles: Introducing Conversational Agents in a Platform for Holistic User Modeling