By: Muhammad Abdul-Mageed, Hassan Alhuzali, Dua'a Abu-Elhij'a and Mona Diab Abstract Although there has been a surge of research on sentiment analysis, less work has been done on the related task of emotion detection. Especially for the Arabic language, there is no literature that we know of for the computational treatment of emotion. This situation is due partially to lack of labelled data, a bottleneck that we seek to ease. In this work, we report efforts to acquire and annotate a multi-dialect dataset for Arabic emotion analysis.

1. DINA: A Multi-Dialect Dataset for
Arabic Emotion Analysis
Muhammad Abdul-Mageed1,2, Hassan AlHuzliy1,
Duaa’ Abu Elhija1, Mona Diab2
Indiana University1, The George Washington
University2

2. Emotions
• Categories of emotion:
– Ekman (e.g., 1992) proposes there are 6 basic
emotions: anger, disgust, fear, happiness, sadness,
and surprise
– Plutchik (1980, 1985, 1994) adds trust and
anticipation
• Emotion on 3 dimensions:
– e.g., Francisco and Gervas (2006) mark the attributes
of pleasantness, activation, and dominance in the
genre of fairy tales.
– DINA is focused on the Ekman emotions.
2

3. Motivations
• Opinion Mining:
– Provides an enriching component beyond the mere binary
valence (i.e. positive and negative) of most sentiment
analysis systems.
• Health & Wellness
– Early detection of certain emotional disorders such as depression.
– Improving the well-being of people by exposing them to desired
emotions (since emotion is contagious [Kramer et al., 2014]).
• Education:
– Integrating emotionally-aware agents in intelligent
computer-assisted language learning, for example, should
prove useful and enhance the naturalness of the
pedagogical experience. 3

4. Motivations Cont.
• Marketing:
– e.g., emotion-sensitive language generation can help with
marketing (Heath et al., 2001; Tan et al., 2014), political
campaigning, etc.
• Security:
– Deflect potential hazards and anticipate dangerous
behaviors
• Author Profiling:
– Useful for predicting age and gender (Meina et al., 2013;
Flekova and Gurevych, 2013; Farias et al., 2013; Bamman
et al., 2014; Forner et al., 2013) and personality
(Mohammad and Kiritchenko, 2013)
4

5. Related Work
• SemEval-2007 Affective Text task (Strapparava and
Mihalcea, 2007) [SEM07]:
– Collection and classification of emotion and
valence in news headlines
• Aman and Szpakowicz (2007):
– Annotation and detection of emotions from blogs
• Qadir and Riloff (2014), Mohammad (2012), Wang
et al. (2012):
– use hashtags as an approximation of emotion
categories to collect emotion data
5

6. Arabic: Motivations
• Morphologically Rich Language
– Highly inflected: person, number, gender, case,
mood, aspect, voice
• Strategic Language:
– One of the 6 languages of UN, with ~ 300M
speakers worldwide
• Exponential Web growth:
– More than 2000% growth rate on the Web in 2010
onwards (www.internetworldstats.com).
6

7. Arabic Dialects
7

8. Data Collection
8
• Crawled Twitter data using a seed set of size < 10
phrases for each of the six Ekman emotion types.
• Each phrase is composed of an emotion word (e.g.,
“happy”) and the first personal pronoun “I”.
• We collect only tweets where a seed phrase occurs
in the tweet body text.
• This approach does not depend on hashtags.
• We collect 500 tweets from each of the 6 emotion
types. Total = 3,000.
• Seeds capture various Arabic dialects.

9. Seeds
9
Table 1. Example seeds

10. Annotation
10
• To verify the utility of this seeds approach, two
college-educated native speakers of Arabic
labeled the data.
• For labeling, we use one of four tags from the set
{“no-emotion/zero”, “weak-emotion”,
“moderate/fair-emotion”, “strong-emotion”}.
• We measure inter-annotator agreement as to
these intensity labels in Cohen’s Kappa.
• We also calculate the % of emotion-carrying
tweets per category (those that did not end up
assigned the label “no-emotion/zero”).

11. DINA: Agreement & % Emotion
11
Table 3. Agreement in fine-grained annotation and
average percentage of emotion

12. Gold Labels from Happiness Class
12
Table 2. Agreement in happiness annotation

13. Examples: Anger
13

14. Examples: Disgust
14

15. Examples: Fear
15

16. Examples: Happiness
16

17. Examples: Sadness
17

18. Examples: Surprise
18

19. Context of No- and Mixed Emotions
19
• Even with a list of well-crafted seeds, both
annotators assign “no-emotion” for 7.5% of the
data.
• This is a function of emotion being a
pragmatics-level phenomenon.
• Contexts for “no-emotion” include:
– Reported speech
– Sarcasm

20. Reported Speech
20

21. Sarcasm
21

22. Conclusion
22
• Emotion is like other pragmatic-level phenomena;
hence a seed-collection approach is useful, but not
perfect.
• Phenomena like reported speech and sarcasm interact
with our method for emotion data collection.
• DINA is multidialectal, but we do not have exact
dialect labels on the tweets.
• DINA is at 3,000 tweets, and we plan to grow the size.
• Full evaluation of DINA is only possible when we build
models exploiting these data, which we plan to do.