Recently, major social media companies have utilized multiple resources in effort to censor offensive language, yet it seems very challenging to successfully handle this issue. In this talk, I introduce the field of abusive language detection, as well as my research on comparing different machine learning models in classifying abusiveness of social media texts.

1. ABUSIVE LANGUAGE
DETECTION—
COMPARATIVE STUDY AND
ITS APPLICATIONS
YOUNGHUN LEE

2. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
BEFORE WE BEGIN..
▸ B.S. in Electrical and Computer Engineering
at Seoul National University
▸ M.S. in Information at the University of
Michigan
▸ (Current) Research Intern at Machine
Intelligence Lab @ Seoul National University
▸ (Current) Ph.D. applicant in Machine
Learning and Natural Language Processing
2

3. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
WHAT THIS PRESENTATION IS COVERING
I. Introduction to Abusive Language Detection
▸ What is it? Why is it important to study?
▸ Recent studies
II. About the paper, “Comparative Studies of Detecting Abusive Language on Twitter”
▸ Research idea and its novelty
▸ Empirical results and findings
III. Future relevance with Naver
▸ Research possibilities and their challenges
3

4. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
ACKNOWLEDGEMENTS
▸ Zeerak Waseem
▸ Ph.D. Candidate at the University of Sheffield
▸ Organizer of the Abusive Language Online Workshop
4

5. I. INTRODUCTION TO
ABUSIVE LANGUAGE DETECTION

6. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
I. INTRODUCTION TO ABUSIVE LANGUAGE DETECTION
▸ Varying definition and terminology
6
EARLY DEFINITION
Messages that most users consider to
be annoying or upsetting1

7. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
▸ Varying definition and terminology
7
RECENT
Messages that under-rate a person or a
group on the basis of characteristics
(race, ethnicity, gender, religion, etc.)2
EARLY DEFINITION
Messages that most users consider to
be annoying or upsetting1
I. INTRODUCTION TO ABUSIVE LANGUAGE DETECTION

8. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
▸ Varying definition and terminology
8
EXAMPLE
1) Go fucking kill yourself useless
scumbag
2) Hell yeah! Go bitches!
3) Jews are lower class pigs
EARLY DEFINITION
Messages that most users consider to
be annoying or upsetting1
RECENT
Messages that under-rate a person or a
group on the basis of characteristics
(race, ethnicity, gender, religion, etc.)2
I. INTRODUCTION TO ABUSIVE LANGUAGE DETECTION

9. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
▸ Varying definition and terminology
9
EXAMPLE
1) Go fucking kill yourself useless
scumbag
2) Hell yeah! Go bitches!
3) Jews are lower class pigs
EARLY DEFINITION
Messages that most users consider to
be annoying or upsetting1
RECENT
Messages that under-rate a person or a
group on the basis of characteristics
(race, ethnicity, gender, religion, etc.)2
I. INTRODUCTION TO ABUSIVE LANGUAGE DETECTION

10. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
▸ Varying definition and terminology
10
EXAMPLE
1) Go fucking kill yourself useless
scumbag
2) Hell yeah! Go bitches!
3) Jews are lower class pigs
EARLY DEFINITION
Messages that most users consider to
be annoying or upsetting1
RECENT
Messages that under-rate a person or a
group on the basis of characteristics
(race, ethnicity, gender, religion, etc.)2
I. INTRODUCTION TO ABUSIVE LANGUAGE DETECTION

11. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
▸ Varying definition and terminology
11
TERMINOLOGY
abusive language
hate speech
offensive language
cyberbullying
hostile flames
vulgar language
insults, profanity
…
EARLY DEFINITION
Messages that most users consider to
be annoying or upsetting1
RECENT
Messages that under-rate a person or a
group on the basis of characteristics
(race, ethnicity, gender, religion, etc.)2
I. INTRODUCTION TO ABUSIVE LANGUAGE DETECTION

12. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
▸ Varying definition and terminology
12
TERMINOLOGY
abusive language
hate speech
offensive language
cyberbullying
hostile flames
vulgar language
insults, profanity
…
EARLY DEFINITION
Messages that most users consider to
be annoying or upsetting1
RECENT
Messages that under-rate a person or a
group on the basis of characteristics
(race, ethnicity, gender, religion, etc.)2
I. INTRODUCTION TO ABUSIVE LANGUAGE DETECTION
PRO)
Enables considering
diverse situations
CON)
Confusing definition
-> hard to annotate

13. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
▸ Why is it important these days?
13
1) Increasing usage of social media3
I. INTRODUCTION TO ABUSIVE LANGUAGE DETECTION

14. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
▸ Why is it important these days?
14
I. INTRODUCTION TO ABUSIVE LANGUAGE DETECTION
1) Increasing usage of social media
2) Social media significantly affects
current day society

15. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
▸ Why is it important these days?
15
4
I. INTRODUCTION TO ABUSIVE LANGUAGE DETECTION

16. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
▸ Why is it important these days?
16
5
I. INTRODUCTION TO ABUSIVE LANGUAGE DETECTION
1) Increasing usage of social media
2) Social media significantly affects
current day society
3) More and more users have been
experiencing online harassment

17. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
▸ Why is it important these days?
17
1) Increasing usage of social media
2) Social media significantly affects
current day society
3) More and more users have been
experiencing online harassment
4) Yet, major social media companies
fail to successfully resolve the issueArtist stencils hate speech tweets
outside Twitter HQ to highlight failure
to deal with offensive messages
I. INTRODUCTION TO ABUSIVE LANGUAGE DETECTION

18. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
▸ Characteristics of Abusive Language Online
▸ Abusive messages might be influenced by…6
18
- its discourse context
- its co-occurring media (images, videos)
- world events
- identity of the author and target
I. INTRODUCTION TO ABUSIVE LANGUAGE DETECTION

19. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
▸ Characteristics of Abusive Language Online
▸ Abusive messages might be influenced by…6
▸ What is it different from other advanced text classification tasks such as
sarcasm detection and fake news detection?
19
- its discourse context
- its co-occurring media (images, videos)
- world events
- identity of the author and target
I. INTRODUCTION TO ABUSIVE LANGUAGE DETECTION
- CONTEXT is its core (e.g. ‘nigger’, ‘bitch’)

20. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
I. INTRODUCTION TO ABUSIVE LANGUAGE DETECTION
▸ Recent studies
▸ Data crawling and annotation
▸ Feature-engineering models for accurate classification
▸ Using meta-information such as age, gender, location as additional features
▸ Applying deep models
▸ Concerning fairness issues
▸ Resolving unintentional bias issue (Woman, Jew being abusive?)
20

21. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
I. INTRODUCTION TO ABUSIVE LANGUAGE DETECTION
▸ Future focus
▸ Creating a gold-standard test set
▸ More reliable annotations
▸ Getting more context information
▸ Multi-modal abusive detection
21

22. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
I. INTRODUCTION TO ABUSIVE LANGUAGE DETECTION
▸ Future focus
▸ Creating a gold-standard test set
▸ More reliable annotations
▸ Getting more context information
▸ Multi-modal abusive detection
22

23. II. ABOUT THE PAPER,
“COMPARATIVE STUDIES OF DETECTING
ABUSIVE LANGUAGE ON TWITTER”

24. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
II. COMPARATIVE STUDIES OF DETECTING ABUSIVE LANGUAGE ON TWITTER
▸ Brief overview of the paper
Comparing the accuracy (F1 scores) of different machine learning models and
different features in detecting abusive language on a recently released Twitter
dataset
24

25. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
II. COMPARATIVE STUDIES OF DETECTING ABUSIVE LANGUAGE ON TWITTER
▸ About the dataset, “Hate and Abusive Speech on Twitter” 7
▸ Approximately 100K labeled tweets (other datasets: 10K to 35K)
▸ Reduced overlapping labels by calculating correlation coefficients
▸ E.g. ‘abusive’, ‘offensive’, ‘aggressive’, ‘cyberbullying’ ‘abusive’
▸ Multi-label dataset (4 labels: abusive, hateful, spam, none)
25
Label distribution of crawled tweets

26. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
II. COMPARATIVE STUDIES OF DETECTING ABUSIVE LANGUAGE ON TWITTER
▸ Implemented models and features
26
• Naïve Bayes
• Logistic Regression
• Support Vector Machine
• Random Forests
• Gradient Boosted Trees
• Word-level / Char-level
FEATURE-ENGINEERING ML NEURAL NETWORKS
• Conv Neural Network
• Recurrent Neural Network
• Word-level / Char-level
• Hybrid CNN
• Self-matching attention RNN
• Latent Topic Clustering
• Context Tweets

27. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
II. COMPARATIVE STUDIES OF DETECTING ABUSIVE LANGUAGE ON TWITTER
▸ Implemented models and features
27
NEURAL NETWORKS
• Conv Neural Network
• Recurrent Neural Network
• Word-level / Char-level
• Hybrid CNN
• Self-matching attention RNN
• Latent Topic Clustering
• Context Tweets
Architecture of Hybrid CNN8

28. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
II. COMPARATIVE STUDIES OF DETECTING ABUSIVE LANGUAGE ON TWITTER
▸ Implemented models and features
28
NEURAL NETWORKS
• Conv Neural Network
• Recurrent Neural Network
• Word-level / Char-level
• Hybrid CNN
• Self-matching attention RNN
• Latent Topic Clustering
• Context Tweets
Gated self-matching attention networks (Wang et al.,)9

29. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
II. COMPARATIVE STUDIES OF DETECTING ABUSIVE LANGUAGE ON TWITTER
▸ Implemented models and features
29
NEURAL NETWORKS
• Conv Neural Network
• Recurrent Neural Network
• Word-level / Char-level
• Hybrid CNN
• Self-matching attention RNN
• Latent Topic Clustering
• Context Tweets
Hierarchical Recurrent Dual Encoder with
Latent Topic Clustering module10

30. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
II. COMPARATIVE STUDIES OF DETECTING ABUSIVE LANGUAGE ON TWITTER
▸ Implemented models and features
30
NEURAL NETWORKS
• Conv Neural Network
• Recurrent Neural Network
• Word-level / Char-level
• Hybrid CNN
• Self-matching attention RNN
• Latent Topic Clustering
• Context Tweets
WHAT ARE CONTEXT TWEETS?
‣ Looking at the tweet one has
replied to or has quoted provides
significant contextual information
‣ Assumption: ML models benefit
from taking context tweets into
account in detecting abusive
language.

31. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
II. COMPARATIVE STUDIES OF DETECTING ABUSIVE LANGUAGE ON TWITTER
▸ Implemented models and features
31
NEURAL NETWORKS
• Conv Neural Network
• Recurrent Neural Network
• Word-level / Char-level
• Hybrid CNN
• Self-matching attention RNN
• Latent Topic Clustering
• Context Tweets
SAMPLE CONTEXT TWEET
Who the HELL is “LIKE” ING this post?
Sick people….
@user_B LABELED TWEET

32. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
II. COMPARATIVE STUDIES OF DETECTING ABUSIVE LANGUAGE ON TWITTER
▸ Implemented models and features
32
NEURAL NETWORKS
• Conv Neural Network
• Recurrent Neural Network
• Word-level / Char-level
• Hybrid CNN
• Self-matching attention RNN
• Latent Topic Clustering
• Context Tweets
SAMPLE CONTEXT TWEET
Survivors of #Syria Gas Attack Recount
‘a Cruel Scene’.
@user_A
Who the HELL is “LIKE” ING this post?
Sick people….
@user_B
CONTEXT TWEET
LABELED TWEET

33. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
II. COMPARATIVE STUDIES OF DETECTING ABUSIVE LANGUAGE ON TWITTER
▸ Implemented models and features
33
NEURAL NETWORKS
• Conv Neural Network
• Recurrent Neural Network
• Word-level / Char-level
• Hybrid CNN
• Self-matching attention RNN
• Latent Topic Clustering
• Context Tweets
HOW TO INTEGRATE CONTEXT TWEETS
LABELED
TWEET
CONTEXT
TWEET
RNN
CNN
max_pooled
max_pooled
last_hidden
last_hidden
σ
σ

34. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
II. COMPARATIVE STUDIES OF DETECTING ABUSIVE LANGUAGE ON TWITTER
34
Experimental results of learning models and their variants, followed by the context tweet models.
The top 2 scores are marked as bold for each metric.
‣ Neural network
models with word-
level features are
accurate in general
compared to feature-
engineering models

35. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
II. COMPARATIVE STUDIES OF DETECTING ABUSIVE LANGUAGE ON TWITTER
35
Experimental results of learning models and their variants, followed by the context tweet models.
The top 2 scores are marked as bold for each metric.
‣ Neural network
models with word-
level features are
accurate in general
compared to feature-
engineering models
‣ Char-level features
benefit feature-
engineering ML
models while cripple
neural models

36. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
II. COMPARATIVE STUDIES OF DETECTING ABUSIVE LANGUAGE ON TWITTER
36
Experimental results of learning models and their variants, followed by the context tweet models.
The top 2 scores are marked as bold for each metric.
‣ Neural network
models with word-
level features are
accurate in general
compared to feature-
engineering models
‣ Char-level features
benefit feature-
engineering ML
models while cripple
neural models
‣ Context data doesn’t
improve the overall
accuracy, however, it
is more effective in
classifying lower-
distributed labels

37. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
II. COMPARATIVE STUDIES OF DETECTING ABUSIVE LANGUAGE ON TWITTER
37
▸ Discussion
BASELINE CONTEXT
p_normal
.1614
p_abusive
.5958
p_normal
.3709
p_abusive
.2582
LABEL: abusive
8 dudes added me in a group chat
telling me I should kill myself,
lmfaooo I think I reached a whole
new level of twitter.
@user_A
They hatin cause you a damn
queen who doesn’t answer to
anybody. Let em be mad. Keep
rolling mamas.
@user_B
‣ Only baseline model
was correct, but is
‘abusive’ label for
this tweet accurate?

38. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
II. COMPARATIVE STUDIES OF DETECTING ABUSIVE LANGUAGE ON TWITTER
38
▸ Discussion
BASELINE CONTEXT
LABEL: normal
Oklahoma home invasion shooting:
No charges against man who killed
3 intruders.
@user_C
You have every right to protect your
family and home from worthless
thieves.
@user_D
‣ Context tweets gave
more information to
accurately classify
the tweet
p_normal
.3287
p_abusive
.3718
p_normal
.8104
p_abusive
.0962

39. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
II. COMPARATIVE STUDIES OF DETECTING ABUSIVE LANGUAGE ON TWITTER
39
▸ Discussion
▸ How to efficiently train the classifier for imbalanced datasets?
▸ How can we better incorporate context tweets into the original data? How to
handle ‘no_context’?
▸ Can we annotate the dataset regarding context data? Specifically, can we
make annotators consider the content of its context tweet when deciding
abusiveness of the tweet?
▸ What are other applications that could benefit from using context data?

40. III. FUTURE RELEVANCE
WITH NAVER

41. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
III. FUTURE RELEVANCE WITH NAVER
▸ Hate Speech in Korea
▸ 남성혐오 / 여성혐오 (워마드, 일베저장소)
▸ 극단적 진보 / 보수 정치 성향
▸ 사회적 문제로 확장
41

42. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
III. FUTURE RELEVANCE WITH NAVER
▸ Possible Contribution
▸ DATASET!!!
42

43. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
III. FUTURE RELEVANCE WITH NAVER
43

44. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
III. FUTURE RELEVANCE WITH NAVER
44

45. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
III. FUTURE RELEVANCE WITH NAVER
45
RICH TEXTUAL & CONTEXTUAL
INFORMATION FROM ONE NEWS ARTICLE
1) Article, title, and its topic
2) Comments and comments of comments
2) Emotions on the article
3) Number of comments, demographic
distribution of commenters
4) Upvotes and downvotes of comments
5) User meta-information

46. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
III. FUTURE RELEVANCE WITH NAVER
▸ Possible applications
▸ Detecting abusive language in news article comments
▸ Enabling personalized settings for not encountering toxic messages
▸ CLOVA: read comments of news articles that are not abusive
▸ Challenges
▸ Korean text: not extensively studied compared to English
▸ Lack of user meta-information
46

47. / 48
ABUSIVE LANGUAGE DETECTION—COMPARATIVE STUDY AND ITS APPLICATIONS
REFERENCE
1. Ellen Spertus. 1997. Smokey: Automatic recognition of hostile messages. In AAAI/IAAI, pages 1058– 1065.
2. John T. Nockleby. 2000. Hate Speech. In Leonard W. Levy, Kenneth L. Karst, and Dennis J. Mahoney, editors, Encyclopedia of
the American Constitution, pages 1277–1279. Macmillan, 2nd edition.
3. Aaron Smith and Monica Anderson. 2018. Social Media Use in 2018. Pew Research Center; accessed 8-December-2018.
4. Monica Anderson and Skye Toor. How social media users have discussed sexual harassment since #MeToo went viral. Pew
Research Center; accessed 8-December-2018.
5. Maeve Duggan. 2017. Online harassment 2017. Pew Research Center; accessed 8-December-2018.
6. Anna Schmidt and Michael Wiegand. 2017. A survey on hate speech detection using natural language pro- cessing. In
Proceedings of the Fifth International Workshop on Natural Language Processing for So- cial Media, pages 1–10.
7. Antigoni Founta, Constantinos Djouvas, Despoina Chatzakou, Ilias Leontiadis, Jeremy Blackburn, Gianluca Stringhini, Athena
Vakali, Michael Sirivianos, and Nicolas Kourtellis. 2018. Large scale crowdsourcing and characterization of twitter abusive
behavior. In Proceedings of the International AAAI Conference on Web and Social Media.
8. Ji Ho Park and Pascale Fung. 2017. One-step and two- step classification for abusive language detection on twitter. In
Proceedings of the First Workshop on Abusive Language Online, pages 41–45.
9. Wenhui Wang, Nan Yang, Furu Wei, Baobao Chang, and Ming Zhou. 2017. Gated self-matching networks for reading
comprehension and question answering. In Proceedings of the 55th Annual Meeting of the Association for Computational
Linguistics, volume 1, pages 189–198.
10.Seunghyun Yoon, Joongbo Shin, and Kyomin Jung. 2018. Learning to rank question-answer pairs using hierarchical recurrent
encoder with latent topic clustering. In Proceedings of the 2018 Conference of the North American Chapter of the Association
for Computational Linguistics: Human Language Technologies, volume 1, pages 1575–1584.
47

48. THANK YOU!!