Presentation of paper at IIMA 2022 conference. Abstract of the paper: Purpose: This short paper describes the dashboard design process for online hate speech monitoring for multiple languages and platforms. Methodology/approach: A case study approach was adopted in which the authors followed a research & development project for a multilingual and multiplatform online dashboard monitoring online hate speech. The case under study is the project for the European Observatory of Online Hate (EOOH). Results: We outline the process taken for design and prototype development for which a design thinking approach was followed, including multiple potential user groups of the dashboard. The paper presents this process's outcome and the dashboard's initial use. The identified issues, such as obfuscation of the context or identity of user accounts of social media posts limiting the dashboard's usability while providing an important trade-off in privacy protection, may contribute to the discourse on privacy and data protection in (big data) social media analysis for practitioners. Research limitations/implications: The results are from a single case study from the dashboard development's first one and half years. Still, they may be relevant for other social listening or online hate speech detection and monitoring projects involving big data analysis and human annotation. Practical implications: The study emphasises the need to involve diverse user groups and a multidisciplinary team in developing a dashboard for online hate speech. The context in which potential online hate is disseminated and the network of accounts distributing or interacting with that hate speech seems relevant for analysis by a part of the user groups of the dashboard. Keywords: online hate speech, social media analysis, big data, anonymisation, social listening.

1. EOOH: the
Development of a
Multiplatform and
Multilingual Online
Hate Speech
Monitoring Dashboard
IIMA 2022 October 25 – Session 9, paper 1888
Anand Sheombar*, Pascal Ravesteijn & Gijs van Beek
* Presenter’s correspondence email: anand.sheombar@hu.nl
HU University of Applied Sciences Utrecht, The Netherlands
IIMA 2022 Conference iima.org
The State of Digital
Transformation
October 23 – 26, 2022 Seattle
Pacific University

2. Paper abstract
Purpose: This short paper describes the dashboard design process for online hate speech monitoring for multiple languages and platforms.
Methodology/approach: A case study approach was adopted in which the authors followed a research & development project for a
multilingual and multiplatform online dashboard monitoring online hate speech. The case under study is the project for the European
Observatory of Online Hate (EOOH).
Results: We outline the process taken for design and prototype development for which a design thinking approach was followed, including
multiple potential user groups of the dashboard. The paper presents this process's outcome and the dashboard's initial use. The identified
issues, such as obfuscation of the context or identity of user accounts of social media posts limiting the dashboard's usability while
providing an important trade-off in privacy protection, may contribute to the discourse on privacy and data protection in (big data) social
media analysis for practitioners.
Research limitations/implications: The results are from a single case study from the dashboard development's first one and half years.
Still, they may be relevant for other social listening or online hate speech detection and monitoring projects involving big data analysis and
human annotation.
Practical implications: The study emphasises the need to involve diverse user groups and a multidisciplinary team in developing a
dashboard for online hate speech. The context in which potential online hate is disseminated and the network of accounts distributing or
interacting with that hate speech seems relevant for analysis by a part of the user groups of the dashboard.
Keywords: online hate speech, social media analysis, big data, anonymisation, social listening.
2
EOOH: the Development of a
Multiplatform and Multilingual Online
Hate Speech Monitoring Dashboard
Session 9, paper 1888

3. Online Hate
3
IIMA2022, session 9, paper 1888
• Increased use of mobile devices and social media (Digital 2022 report)
• More people encounter online insults or threats.
• To counter online hate speech: challenging due to ever-shifting
dynamics of our online environment.
• The problem of doing this in multiple languages ánd on multiple social
media platforms…
• Big data research & (semi-) automated detection & monitoring
NEEDED (Laaksonen et al., 2020).

4. Hate speech
• Concept of (online) hate speech has arbitrary definitions .
• Various launched to automate detection of online hate speech with
mixed results (Laaksonen et al., 2020, Paschalides et al., 2020,
Salminen et al., 2020).
• European Union's definition (EC, 2022): "hate speech is defined in
EU law as the public incitement to violence or hatred on the basis of
certain characteristics, including race, colour, religion, descent and
national or ethnic origin." - adopted in EOOH project (the case of
study for this paper).
4
IIMA2022, session 9, paper 1888

5. Dashboard & Social Listening
• Dashboard: "a visual display of the most important
information needed to achieve one or more
objectives; consolidated and arranged on a single
screen so the information can be monitored at a
glance." (Few, 2006).
• Social Listening / SMAT (Social Media Analysis
Tools): "tools that enable users to make sense of
the interactions on social media, as well as to
listen, monitor and analyse the information that is
generated via social media applications."
(Anson et al., 2017)
5
IIMA2022, session 9, paper 1888

6. Research gap on…
• Development of multilingual and multiplatform online hate
speech dashboard &
• also touches upon the research gap on cross-platform evaluation
of online hate classifiers (Salminen et al., 2020)
6
IIMA2022, session 9, paper 1888

7. Research Approach
• Research question: What issues, pitfalls and
insights are encountered when developing a
multiplatform and multilingual dashboard for
online hate speech?
• Exploratory nature of the study: qualitative
research
• Case study – analytical units: social listening,
big data analysis, and the development of digital
services (dashboard).
• Case selection: European Observatory of
Online Hate (EOOH), a two-year EU-funded
research project. Data from 1st year.
• Main data sources: interviews, participatory
research/observations, supported by various
types of secondary data from the project. 7
IIMA2022, session 9, paper 1888

8. Findings 1: Initial User Interface Development
• Five user groups identified for the dashboard:
1. Policy makers, 2. Scientists, 3. Law enforcement, 4. Social
workers/NGOs, and 5. Campaigners/media organisations.
8
IIMA2022, session 9, paper 1888

9. Findings 2A: Technical Social Media Data Collection
and Dashboard Development
• Written in Python, own software, no parts used of
proprietary software vendors.
• Scrapes multiple social media platforms in real-time and
predicts toxicity in EU's 24 official languages.
• GDPR compliant storage system for data, hosted in
Germany ISO 27001 data centre.
• AI tools for social media data anonymisation..
• Visualising insights and generating written summaries
9
IIMA2022, session 9, paper 1888

10. Findings 2-B: Technical Social Media Data Collection & Dashboard Development
• Visualisation of online hate speech collection and processing.
10
IIMA2022, session 9, paper 1888
Project website EOOH.eu

11. SCORE 0 = no problem, 4 = very problematic THREATS
RACISM
universal translation
thousands of known expressions
Findings 2C: xAI: Annotation Process - ongoing - categorising toxic language
IIMA2022, session 9, paper 1888 11

12. Findings 2D: Continuously learns new unknown expressions using ethical xAI
12

13. Findings 3A: Pilot Use By Expert Groups
• Impression of start page of the EOOH dashboard .
13
IIMA2022, session 9, paper 1888
118 experts involved
727 channels (532
active, 195 archived)
53 million keyword
related messages
detected
4 Baseline channels
(‘neutral’ channels):
•Ukraine baseline
•Refugees baseline
•Islam baseline
•Judaism baseline

14. Findings 3B: Example of Dashboard Use by Experts
14
IIMA2022, session 9, paper 1888

15. Findings 3C: EOOH expert community channels
15

16. Discussion
• EOOH dashboard provides and improves online hate speech
monitoring functionalities, such as multilingual &
multiplatform hate speech detection (De Smedt et al., 2020)
• Ethical principle by design, built-in upfront data
anonymisation, similar to suggestions by Kiritchenko and
Nejadgholi (2020).
• Further research is needed on overcoming the tension
between GDPR compliance and fulfilling the needs of
dashboard users
16
IIMA2022, session 9, paper 1888

17. Q&A
17
Anand Sheombar*, Pascal Ravesteijn & Gijs van Beek
* Presenter’s correspondence email: anand.sheombar@hu.nl
IIMA2022, session 9, paper 1888

18. Online Gender-Based Violence
Source: Hinson et al. (2018)
18
Technology-Facilitated or Online Gender-Based
Violence “is action by one or more people that
harms others based on their sexual or gender
identity or by enforcing harmful gender norms.
This action is carried out using the internet and/or
mobile technology and includes stalking, bullying,
sexual harassment, defamation, hate speech and
exploitation.” (Hinson et al., 2018)
Misogynoir: anti-Black and misogynistic
representation of Black women, also in media &
digital space. (Dr. Moya Bailey)
IIMA2022, session 9, paper 1888

19. Online violence against women during COVID19
Infographic source: UN Women. (2021).
19
Increased online
violence facilitated by
ICT during COVID-19
may impact:
• Women’s access to
online services
• Education and
employment
opportunities
• Women’s
participation as
active digital
citizens
Source: UN Women (2021)
IIMA2022, session 9, paper 1888

20. Keyword(s) used in EOOH Channel
Data collection only with Dutch keywords
- For this presentation an approximate
English translation is shown
Platform(s)
Racist, aap, slaaf, slavin, neger, zwarte piet,
kotsmisselijk, racismekaart, kutwijf, zeikwijf,
knettergek
Twitter
Racist, ape, slave, negro, blackface, sick to my
stomach, racism card, cunt, bitch, lunatic / crazy
woman
IIMA2022, session 9, paper 1888

21. Some Initial Dashboard Findings
Finding(s) Your Insight(s)
Many toxic content
combined.
Qualitative analysis
suggests focal points
lie in intimidation,
threats and hate
speech
Events on which Black
Politician reacted were
used to spread hate
towards her.
For example the war in
Ukraine where Black
people and POC were
mistreated.
Hate speech shows
sexism, racism and
dehumanisation
IIMA2022, session 9, paper 1888

22. Refugees baseline
channel
Islam(ophobia) baseline
channel
Intersectional trend: Antisemitism prevalent across toxic topics

23. AI and Data Collection
• When developing system that collects social media data for analysis
using artificial intelligence, ethical considerations need to be taken
into account (Kiritchenko and Nejadgholi, 2020)
• Mitigation of unintended biases
• What constitutes hatespeech
• Sampling/topic bias
• Annotator bias
• Transparency and explainability
23
IIMA2022, session 9, paper 1888