Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Emotions in Argumentation
an Empirical Evaluation
Sahbi Benlamine, Maher Chaouachi, Claude Frasson
Serena Villata, Elena C...
Web Agora
web
read-write web
social web
argument web
debate web
emotion web
semantic web
debate traces
question:
Connection between the arguments proposed by the
participants of a debate and their emotional status?
• correlat...
Web Agora
Argumentation ( )
argumentation
 support decision-making and persuasion
 e-democracy and online debates
 abstract bipolar argumentation
[...
e.g.
A. Elena: This information is important, we must publish it
B. Serena: It is a private information about a person who...
argument network analysis
attack
attack
support
A.
B.
C.
D.
Web Agora
Argumentation
Emotions ( )
emotion computing
 beyond purely rational behavior
 detect emotional state to adapt reactions
 e.g.(serious) games
emotion detection
 webcams for facial expressions analysis
[FACEREADER 6.0]
 physiological sensors
(EEG) for cognitive
s...
real–time engagement
 engagement index [Pope et al.,1995]
𝑒𝑛𝑔 =


 EEG frequency bands
 (4-8Hz)
α (8-13Hz)
β (13-22...
real-time facial analysis
 classifying 500 key points in facial muscles
 neural network
 trained on 10 000 examples
 h...
Web Agora
Argumentation
Emotions
Seempad project
Seempad Joint Research Lab
 Emotions play an important role in decision making
[Quartz, 2009]
 Assess connection between...
1st Experiment & Public dataset
focus on associating participants’
 arguments and the relations among them
 mental engag...
protocol of the experiment
 topics from popular discussions in iDebate & DebateGraph
 12 debates - 4 participants and 2 ...
participants
 6 sessions of 4 participants (-1 session)
 20 participants (7 women, 13 men)
 age range from 22 to 35 yea...
data collection during the experiment
 minimum, average and maximum engagement of every
participant in a debate
 most do...
data annotation after the experiment
 synchronize arguments, relations and emotional indexes
 bipolar argumentation labe...
publicly available dataset
dataset content
 xml structure of debate flow
 598 arguments in 12 different debates
 263 argument pairs
 127 supports...
dataset extract (flow)
<argument id="2" debate_id="4" participant="4" time-
from="20:30" time-to="20:30">The religion is a...
dataset extract (relations)
<debate id="4" title="Religion" task="relation">
<pair id="1" relation="support">
<argument id...
dataset extract (emotions)
<argument id="30" debate_id="4" participant="4" time-from="20:43"
time-to="20:43" emotion_p1="n...
initial analysis
 H1 : some emotional and behavioral trends can be
extracted from a set of debates.
 H2 : the number and...
initial analysis
 mean percentage of appearance of each basic emotion
(with 95% confidence interval)
 that the most freq...
emotional evolution participant 1 debate 1
before and after argument rejection
(hypothesis 2)
correlation table for Session 3
“Distribute condoms at schools”
“Encourage fewer people to go to the university”
number of...
aggregated correlation table
 supports increase linearly with engagement (r=0.31)
more pronounced when no conflict (r=0.8...
to wrap-up
connection between the arguments proposed by the
participants of a debate and their emotional status?
 H1 : em...
current work
 granularity: from sessions to debates
 existence or not of a priori positive/negative opinion
 actual cha...
perspectives
 deeper analysis, different time scales and granularity
 new experiment with improved protocol
 NLP analys...
Upcoming SlideShare
Loading in …5
×

Emotions in Argumentation: an Empirical Evaluation @ IJCAI 2015

1,596 views

Published on

IJCAI 2015 of the first result of the SEEMPAD project on emotions in argumetation.

Published in: Science
  • Be the first to comment

Emotions in Argumentation: an Empirical Evaluation @ IJCAI 2015

  1. 1. Emotions in Argumentation an Empirical Evaluation Sahbi Benlamine, Maher Chaouachi, Claude Frasson Serena Villata, Elena Cabrio, Fabien Gandon
  2. 2. Web Agora
  3. 3. web
  4. 4. read-write web
  5. 5. social web
  6. 6. argument web
  7. 7. debate web
  8. 8. emotion web
  9. 9. semantic web
  10. 10. debate traces
  11. 11. question: Connection between the arguments proposed by the participants of a debate and their emotional status? • correlation of polarity of arguments and polarity of detected emotions? • relation between kinds and amount of arguments, and the engagement of participants?
  12. 12. Web Agora Argumentation ( )
  13. 13. argumentation  support decision-making and persuasion  e-democracy and online debates  abstract bipolar argumentation [Dung, 1995; Cayrol and Lagasquie-Schiex, 2013]  support and attack relations
  14. 14. e.g. A. Elena: This information is important, we must publish it B. Serena: It is a private information about a person who does not want to publish it C. Elena: This person is the Prime Minister so the information is not private D. Fabien: Yes, being a governmental officer makes the information about him public attack attack support
  15. 15. argument network analysis attack attack support A. B. C. D.
  16. 16. Web Agora Argumentation Emotions ( )
  17. 17. emotion computing  beyond purely rational behavior  detect emotional state to adapt reactions  e.g.(serious) games
  18. 18. emotion detection  webcams for facial expressions analysis [FACEREADER 6.0]  physiological sensors (EEG) for cognitive states [Chaouachi et al., 2010]
  19. 19. real–time engagement  engagement index [Pope et al.,1995] 𝑒𝑛𝑔 =    EEG frequency bands  (4-8Hz) α (8-13Hz) β (13-22Hz)
  20. 20. real-time facial analysis  classifying 500 key points in facial muscles  neural network  trained on 10 000 examples  happy, sad, angry, surprised, scared, disgusted.  valence, arousal  neutral probability.
  21. 21. Web Agora Argumentation Emotions Seempad project
  22. 22. Seempad Joint Research Lab  Emotions play an important role in decision making [Quartz, 2009]  Assess connection between argumentation and emotions  Final goal = Detect on the Web… • a debate turning into a flame war, • a content reaching an agreement, • a good or bad emotion spreading in a community… +
  23. 23. 1st Experiment & Public dataset focus on associating participants’  arguments and the relations among them  mental engagement detected by EEG  facial emotions detected via Face Emotion Recognition
  24. 24. protocol of the experiment  topics from popular discussions in iDebate & DebateGraph  12 debates - 4 participants and 2 moderators each  participants equipped with emotions detection tools  messages in plain English through IRC  participants are anonymous  debate for 20 minutes  debrief questionnaire
  25. 25. participants  6 sessions of 4 participants (-1 session)  20 participants (7 women, 13 men)  age range from 22 to 35 years  not all of them were native English speakers  students in a North American university  signed an ethical agreement  good computer skills
  26. 26. data collection during the experiment  minimum, average and maximum engagement of every participant in a debate  most dominant emotion (having maximum value)  pleased/unpleased valence  active/inactive arousal
  27. 27. data annotation after the experiment  synchronize arguments, relations and emotional indexes  bipolar argumentation labelled with : sources, arguments, emotional states  two independent annotators with agreement of 91% Cohen’s kappa 0.82 >> 0.6  = Pr 𝑎 − Pr(𝑒) 1 − Pr(e)
  28. 28. publicly available dataset
  29. 29. dataset content  xml structure of debate flow  598 arguments in 12 different debates  263 argument pairs  127 supports  136 attacks  gender, age and personality type  dominant emotion, valence and arousal  mental engagement levels
  30. 30. dataset extract (flow) <argument id="2" debate_id="4" participant="4" time- from="20:30" time-to="20:30">The religion is an independent factor, it should not be a dissociative factor separating people. </argument> <argument id="3" debate_id="4" participant="1" time- from="20:32" time-to="20:32">The religion gives to his followers hope and help them to overcome some problem of the life so it's not all bad. </argument> <argument id="4" debate_id="4" participant="4" time- from="20:32" time-to="20:32">Here in Canada it is appreciable to find the liberty of religion a practice in a peaceful way. </argument>
  31. 31. dataset extract (relations) <debate id="4" title="Religion" task="relation"> <pair id="1" relation="support"> <argument id="2" debate_id="4" participant="4" time- from="20:30" time-to="20:30">The religion is an independent factor, it should not be a dissociative factor separating people. </argument> <argument id="3" debate_id="4" participant="1" time- from="20:32" time-to="20:32">The religion gives to his followers hope and help them to overcome some problems of the life so it's not all bad. </argument> </pair> <pair id="2" relation="attack"> <argument id="3" debate_id="4" participant="1" time- from="20:32" time-to="20:32">The religion gives to his followers hope and help them to overcome some problems of the life so it's not all bad. </argument> <argument id="5" debate_id="4" participant="3" time- from="20:32" time-to="20:32">During all the existence of human being, religion makes a lot of issue. It make more hurts than curs. </argument> </pair>
  32. 32. dataset extract (emotions) <argument id="30" debate_id="4" participant="4" time-from="20:43" time-to="20:43" emotion_p1="neutral" emotion_p2="neutral" emotion_p3="neutral" emotion_p4="neutral"> Indeed but there exist some advocates of the devil like Bernard Levi who is decomposing arabic countries. </argument> <argument id="31" debate_id="4" participant="1" time-from="20:43" time-to="20:43" emotion_p1="angry" emotion_p2="neutral" emotion_p3="angry" emotion_p4="disgusted"> I don’t totally agree with you Participant2: science and religion don’t explain each other, they tend to explain the world but in two different ways. </argument> <argument id="32" debate_id="4" participant="3" time-from="20:44" time-to="20:44" emotion_p1="angry" emotion_p2="happy" emotion_p3="surprised" emotion_p4="angry"> Participant4: for recent wars ok but what about wars happened 3 or 4 centuries ago? </argument>
  33. 33. initial analysis  H1 : some emotional and behavioral trends can be extracted from a set of debates.  H2 : the number and the strength of arguments, attacks and supports exchanged between the debaters are correlated with particular emotions.  H3 : the number of expressed arguments is connected to the degree of mental engagement and social interactions.
  34. 34. initial analysis  mean percentage of appearance of each basic emotion (with 95% confidence interval)  that the most frequent : anger 8:15% to 15:6%  second most frequent : disgust 7:52% to 14:8%  negativity effect [Rozin and Royzman, 2001]  high level engagement (70:2% to 87:7% of time) correlated with anger (r=0.306)
  35. 35. emotional evolution participant 1 debate 1 before and after argument rejection (hypothesis 2)
  36. 36. correlation table for Session 3 “Distribute condoms at schools” “Encourage fewer people to go to the university” number of attacks increased linearly with disgust (H2)
  37. 37. aggregated correlation table  supports increase linearly with engagement (r=0.31) more pronounced when no conflict (r=0.80)  high engagement for most participative participants (H3)
  38. 38. to wrap-up connection between the arguments proposed by the participants of a debate and their emotional status?  H1 : emotional trends can be extracted from debates  H2 : arguments, attacks, supports correlated with emotions  H3 : the number of arguments is connected to the engagement
  39. 39. current work  granularity: from sessions to debates  existence or not of a priori positive/negative opinion  actual changes of opinion  impact of personality (big 5 test)  dynamics of the debate and emotion changes
  40. 40. perspectives  deeper analysis, different time scales and granularity  new experiment with improved protocol  NLP analysis and correlations http://fabien.infohttp://project.inria.fr/seempad/ 40/40/…40 :-)

×