11. question:
Connection between the arguments proposed by the
participants of a debate and their emotional status?
• correlation of polarity of arguments and
polarity of detected emotions?
• relation between kinds and amount of arguments,
and the engagement of participants?
13. argumentation
support decision-making and persuasion
e-democracy and online debates
abstract bipolar argumentation
[Dung, 1995; Cayrol and Lagasquie-Schiex, 2013]
support and attack relations
14. e.g.
A. Elena: This information is important, we must publish it
B. Serena: It is a private information about a person who does
not want to publish it
C. Elena: This person is the Prime Minister so the
information is not private
D. Fabien: Yes, being a governmental officer makes the
information about him public
attack
attack
support
22. Seempad Joint Research Lab
Emotions play an important role in decision making
[Quartz, 2009]
Assess connection between argumentation and emotions
Final goal = Detect on the Web…
• a debate turning into a flame war,
• a content reaching an agreement,
• a good or bad emotion spreading in a community…
+
23. 1st Experiment & Public dataset
focus on associating participants’
arguments and the relations among them
mental engagement detected by EEG
facial emotions detected via Face Emotion Recognition
24. protocol of the experiment
topics from popular discussions in iDebate & DebateGraph
12 debates - 4 participants and 2 moderators each
participants equipped with emotions detection tools
messages in plain English through IRC
participants are anonymous
debate for 20 minutes
debrief questionnaire
25. participants
6 sessions of 4 participants (-1 session)
20 participants (7 women, 13 men)
age range from 22 to 35 years
not all of them were native English speakers
students in a North American university
signed an ethical agreement
good computer skills
26. data collection during the experiment
minimum, average and maximum engagement of every
participant in a debate
most dominant emotion (having maximum value)
pleased/unpleased valence
active/inactive arousal
27. data annotation after the experiment
synchronize arguments, relations and emotional indexes
bipolar argumentation labelled with : sources, arguments,
emotional states
two independent annotators with agreement of 91%
Cohen’s kappa 0.82 >> 0.6
=
Pr 𝑎 − Pr(𝑒)
1 − Pr(e)
29. dataset content
xml structure of debate flow
598 arguments in 12 different debates
263 argument pairs
127 supports
136 attacks
gender, age and personality type
dominant emotion, valence and arousal
mental engagement levels
30. dataset extract (flow)
<argument id="2" debate_id="4" participant="4" time-
from="20:30" time-to="20:30">The religion is an
independent factor, it should not be a dissociative
factor separating people. </argument>
<argument id="3" debate_id="4" participant="1" time-
from="20:32" time-to="20:32">The religion gives to his
followers hope and help them to overcome some problem
of the life so it's not all bad. </argument>
<argument id="4" debate_id="4" participant="4" time-
from="20:32" time-to="20:32">Here in Canada it is
appreciable to find the liberty of religion a practice
in a peaceful way. </argument>
31. dataset extract (relations)
<debate id="4" title="Religion" task="relation">
<pair id="1" relation="support">
<argument id="2" debate_id="4" participant="4" time-
from="20:30" time-to="20:30">The religion is an independent factor,
it should not be a dissociative factor separating people.
</argument>
<argument id="3" debate_id="4" participant="1" time-
from="20:32" time-to="20:32">The religion gives to his followers
hope and help them to overcome some problems of the life so it's
not all bad. </argument>
</pair>
<pair id="2" relation="attack">
<argument id="3" debate_id="4" participant="1" time-
from="20:32" time-to="20:32">The religion gives to his followers
hope and help them to overcome some problems of the life so it's
not all bad. </argument>
<argument id="5" debate_id="4" participant="3" time-
from="20:32" time-to="20:32">During all the existence of human
being, religion makes a lot of issue. It make more hurts than curs.
</argument>
</pair>
32. dataset extract (emotions)
<argument id="30" debate_id="4" participant="4" time-from="20:43"
time-to="20:43" emotion_p1="neutral" emotion_p2="neutral"
emotion_p3="neutral" emotion_p4="neutral">
Indeed but there exist some advocates of the devil like Bernard
Levi who is decomposing arabic countries. </argument>
<argument id="31" debate_id="4" participant="1" time-from="20:43"
time-to="20:43" emotion_p1="angry" emotion_p2="neutral"
emotion_p3="angry" emotion_p4="disgusted">
I don’t totally agree with you Participant2: science and religion
don’t explain each other, they tend to explain the world but in two
different ways.
</argument>
<argument id="32" debate_id="4" participant="3" time-from="20:44"
time-to="20:44" emotion_p1="angry" emotion_p2="happy"
emotion_p3="surprised" emotion_p4="angry">
Participant4: for recent wars ok but what about wars happened 3 or
4 centuries ago? </argument>
33. initial analysis
H1 : some emotional and behavioral trends can be
extracted from a set of debates.
H2 : the number and the strength of arguments, attacks
and supports exchanged between the debaters are
correlated with particular emotions.
H3 : the number of expressed arguments is connected to
the degree of mental engagement and social interactions.
34. initial analysis
mean percentage of appearance of each basic emotion
(with 95% confidence interval)
that the most frequent : anger 8:15% to 15:6%
second most frequent : disgust 7:52% to 14:8%
negativity effect [Rozin and Royzman, 2001]
high level engagement (70:2% to 87:7% of time)
correlated with anger (r=0.306)
36. correlation table for Session 3
“Distribute condoms at schools”
“Encourage fewer people to go to the university”
number of attacks increased linearly with disgust (H2)
37. aggregated correlation table
supports increase linearly with engagement (r=0.31)
more pronounced when no conflict (r=0.80)
high engagement for most participative participants (H3)
38. to wrap-up
connection between the arguments proposed by the
participants of a debate and their emotional status?
H1 : emotional trends can be extracted from debates
H2 : arguments, attacks, supports correlated with emotions
H3 : the number of arguments is connected to the engagement
39. current work
granularity: from sessions to debates
existence or not of a priori positive/negative opinion
actual changes of opinion
impact of personality (big 5 test)
dynamics of the debate and emotion changes
40. perspectives
deeper analysis, different time scales and granularity
new experiment with improved protocol
NLP analysis and correlations
http://fabien.infohttp://project.inria.fr/seempad/ 40/40/…40 :-)
Editor's Notes
Good morning
My name is Fabien Gandon, I am a research director at Inria Sophia Antipolis France and presenting here on behalf of Serena and Elena who recently became mothers.
The general context of this work is one of the many evolutions of the Web namely the fact it is becoming a world-wide agora
Initially the Web was perceived as a documentary space for human consumption
With the advent of Wikis in 94 we reopen the Web in writing mode
This also starts the social Web evolution with forums, blogs, social networks and so on
Among the contributions made we now have a number of techniques to detect and analyze an argument made in a contribution
And link these arguments to reconstruct debate graphs
We also have more and more means to analyze the emotions of the users from direct sensors to indirect sentiment analysis
In parallel the semantic Web allows us to formally represent knowledge and data and also annotate all the resources identified on the Web.
And we can leverage these formalisms to represent and merge these debate traces with other linked data on the Web, to exchange, query, reason on the debate traces.
So the research question we are considering in this paper is : is there a connection between the arguments proposed by the participants of a debate and their emotional status?
And in particular is there correlation of polarity of arguments and polarity of detected emotions?
And is there a relation between the kinds and amount of arguments, and the engagement of participants?
For this we first leverage results from Argumentation theory
Argumentation theory is used for instance to support decision-making and persuasion for example in scenarios of e-democracy and online debates
We rely on abstract bipolar argumentation to capture support and attack relations
For instance here is a small debate …
Behind this dialog is an argumentation graph with support and attack relations that we can capture and represent with KR formalism
Then it can be analyzed to extract for instance the claims that still stand in a debate and give an overview of the current state of the debate.
The second domain we rely on is emotion analysis.
Here the idea is to go beyond purely rational behavior by detecting emotional states of users to adapt the reactions of the system
This is used for instance in designing and adapting games
In this work we rely on two detection means:
facial expressions analysis with FACEREADER
and physiological sensors EEG
physiological sensors EEG are used here to compute real–time engagement using Pope’s metric based on theta, alpha and Beta EEG frequency bands
This gives us a clue of the engagement of a user in a task.
The real-time facial analysis classifies 500 key points in facial muscles and uses a neural network trained on 10 000 examples to indicate if the user is
happy, sad, angry, surprised, scared, disgusted.
The valence and arousal of his emotion
And a probability of neutral emotion
So in the context of a joint lab between Inria and University Montreal called Seempad we combine Argumentation theory and Emotion detections
We believe emotions play an important role in decision making and we want to assess connection between argumentation and emotions
Our final goal would be to be able to detect on the Web…
a debate turning into a flame war,
a content reaching an agreement,
a good or bad emotion spreading in a community
The article presents the first result of this project which is an experiment to produce a publicly available dataset of recorded debates associating participants’
arguments and the relations among them
mental engagement detected by EEG
facial emotions detected via Face Emotion Recognition
Topics obtained from
during data collection we recorded
After the experiment we synchronize arguments, relations and emotional indexes
The bipolar argumentation is labelled with sources, arguments, emotional states by two independent annotators with a verified good level of agreement
Cohen's kappa coefficient is a statistic which measures inter-rater agreement for qualitative (categorical) items. It is generally thought to be a more robust measure than simple percent agreement calculation, since κ takes into account the agreement occurring by chance
Pr(a) is the relative observed agreement among raters, and Pr(e) is the hypothetical probability
Here is the table of content of the publicly available dataset with topics and stats.
In a nutshell this public dataset contains:
An XML structure of debate flow
598 arguments in 12 different debates
263 argument pairs
127 supports
136 attacks
gender, age and personality type
dominant emotion, valence and arousal
mental engagement levels
Here is what it looks like if you open this dataset.
You have here the markup for the argument flow
the markup for the relations between argument (support/attack)
the markup for the synchronized emotions detected
This first dataset allowed us perform so preliminary analysis
The general hypothesis was some emotional and behavioral trends can be extracted from a set of debates.
With two sub hypothesis:
the number and the strength of arguments, attacks and supports exchanged between the debaters are correlated with particular emotions.
the number of expressed arguments is connected to the degree of mental engagement and social interactions
The details are in the paper and I will only give some extracts here
Looking at the mean percentage of emotions, most frequent ones are anger and disgusts and you can verify the negativity effect namely that negative emotions last longer.
There is also a correlation between anger and engagement
Here is a extract of the emotion evolution for surprise and disgust for participant 1 in debate 1.
And what you see here is what happen when an argument is rejected.
Here is the correlation table for session 3 with two debates on
“Distributing condoms at schools”
and
“Encouraging fewer people to go to the university”
An example of interesting things happening here is to see that the number of attacks increases linearly with disgust
Now here is the aggregated correlation table for all sessions
And you can see for instance that supports increase linearly with engagement and this is even more pronounced when there is no conflict
And there is high engagement for most participative participants
To wrap-up our question was is there connection between the arguments proposed by the participants of a debate and their emotional status?
and we have evidences for three initial hypothesis that are
We are now performing deeper analysis
First of all at smaller granularity at the level of debates
We are considering the existence or not of a priori positive/negative opinion
We are checking the actual changes of opinion
We are considering the impact of the personality using the big 5 test
And looking at the dynamics of the debate and emotion changes
On a longer term we target deeper analysis, different time scales and granularity
We are preparing a new experiment with an improved protocol
We intend to perform NLP analysis on the debate corpus and consider the results in the correlations