Slide deck used during a presentation at STI Forum side event on "Innovating for Peace", hosted by the UN missions of Turkey and Qatar, as well as UN DPPA. More about the STI Forum at https://www.un.org/ecosoc/en/events/2021/multi-stakeholder-forum-science-technology-and-innovation-sustainable-development-goals. The presentation features work by QCRI scientists, including Muhammad Imran and Preslav Nakov, and many others. See the last slide for references.

1. Data on Polarization,
Peace, and Propaganda
STI Forum Side Event – Innovating for Peace
May 4, 2021
Ingmar Weber
@ingmarweber

2. QATAR COMPUTING RESEARCH INSTITUTE
Example Studies
Using Twitter to Study the Antecedents of ISIS Support
Secular vs. Islamist Polarization in Egypt on Twitter
Mapping Education Insecurity Using Twitter
Monitoring the Venezuelan Exodus with Facebook Advertising Data
Computational Propaganda Detection

3. QATAR COMPUTING RESEARCH INSTITUTE
Using Twitter to Study the Antecedents of
ISIS Support
01

4. User’s first reference to ISIS
#IslamicState something else #IslamicState again
Down with the tyrants!
Pre-ISIS Period Post-ISIS Period
What were ISIS supporters tweeting in the pre-ISIS period?
How does it differ from what ISIS opponents were tweeting?
Twitter as a Time Machine to Look Into the Past

5. ‫داعش‬# ... #ISIS ... ‫الدولة‬
‫اإلسالمية‬ # … #IslamicState
In 93% of tweets, using the long form
(“Islamic State”) indicates support for ISIS.
In 77% of tweets, using the short form
(“ISIS”) indicates opposition to ISIS.
How to Tell Current ISIS Support from Opposition?

6. QATAR COMPUTING RESEARCH INSTITUTE
Data Collection
Final data set
57K active Twitter accounts with 10+ ISIS-related tweets
(123 million tweets in total)
• 46K opposing ISIS (#ISIS)
• 11K supporting ISIS (#IslamicState)
Build classifier to tell the two groups apart using only
the pre-ISIS tweets
Predicts (in retrospect) #ISIS vs. #IslamicState with ~90%
accuracy

7. QATAR COMPUTING RESEARCH INSTITUTE
Close Look at “Predictive” (in Retrospect) Hashtags
Pro-ISIS Anti-ISIS
Before #
‫انتخبوا‬
_
‫العرص‬
(
# electThePimp)
# ‫اعتصام‬
_
‫بريده‬
(
# BuraidaProtest)
#libya (#Libya)
# ‫مرسي‬
(
# Morsy)
# ‫ال‬
_
‫سعود‬
(
# FamilyOfSaud)
# ‫الربيع‬
_
‫العراقي‬
(
# IraqiSpring)
#feb17 (#Feb17 -- launch of Libyan revolution against
Ghaddafi)
# ‫الشعب‬
_
‫يقول‬
_
‫كلمته‬
(
# thePeopleSayTheirWord)
# ‫جبهه‬
_
‫النصره‬
(
# AnnusraFront)
# ‫حسم‬
(
# Hasm -- an Egyptian anti-government armed group)
# ‫مسيره‬
_
‫كرامه‬
_
‫وطن‬
(
# nationalDignityMarch)
#kwu89 (#Kuwait)
# ‫رابعه‬
_
‫العدويه‬
(
# Rabae -- site of anti-coup protest)
→ Most top discriminating hashtags refers to revolutions and
opposition to dictatorial regimes in different Arabic countries
#
‫معرض‬
_
‫ابوظبي‬
_
‫للصيد‬
_
‫والفروسيه‬
(
# AbuDhabiHuntingAndEquestrianExpo)
# ‫تحيا‬
_
‫مصر‬
(
# longLiveEgypt -- slogan of pro-coup
camp)
# ‫الحفاظ‬
_
‫علي‬
_
‫مواردنا‬
_
‫سلوك‬
_
‫وطني‬
(
# preservingOurResourcesIsAPatrioticBehavior)
# ‫غزه‬
(
# Gaza)
# ‫غزه‬
_
‫تحت‬
_
‫القصف‬
(
# GazaUnderShelling)
→ More general, with some density from UAE,
supporters of military coup in Egypt, and general
support for Gaza

8. QATAR COMPUTING RESEARCH INSTITUTE
Secular vs. Islamist Polarization in Egypt on
Twitter
02

9. QATAR COMPUTING RESEARCH INSTITUTE
How Polarized is a Hashtag (in Egypt in 2012/2013)?
#music
#Morsi_is_a_big_liar ( )
<- used by all types of users
<- used only by Secularists
New question: how to identify Secularists and Islamists?

10. QATAR COMPUTING RESEARCH INSTITUTE
Retweeting = Endorsement?
Asked two judges to label 100 users
Judges labeled 38% as “unknown” – not easy!
For the non-“unknown” labels …
77% agreement of inferred label with judges
80% inter-judge agreement
Some noise at individual level
Strong signal at aggregate level
In the end, 5,215 Secularists and 1,719 Islamists

11. QATAR COMPUTING RESEARCH INSTITUTE
Examples of Polarized Hashtags

12. QATAR COMPUTING RESEARCH INSTITUTE
A Hashtag Barometer?
Hashtag polarization score = distance from uniform usage
For a given week, average the polarization score across all hashtags
a - Assailants with rocks and firebombs gather outside Ministry of Defence to call for an end to military rule.
b - Demonstrations and clashes break out after President Morsi grants himself increased power to protect the nation.
c,d - Continuing protests after the November 22nd declaration.
e - Demonstrations in Tahrir square, Port Said and all across the country.
f,g - Demonstrations at Tahrir square.
more
polarized
less
polarized

13. QATAR COMPUTING RESEARCH INSTITUTE
Mapping Education Insecurity Using Twitter
03

14. QATAR COMPUTING RESEARCH INSTITUTE
Mapping Education Insecurity
Many incidents not covered in media are potentially reported on social media
Education insecurity reports:
• Killing, murder, assassination
• Injured, wounded
• Abduction, kidnapping, detention,
imprisoned
• Rape, sexual violence or harassment
• Threats, extorted, abused
• Forced labor
• School/college/university damage

15. QATAR COMPUTING RESEARCH INSTITUTE
Real-time processing of Twitter
Multilingual classifiers:
Arabic, French, English
https://data.humdata.org/visualization/mapping-education-insecurity/

16. QATAR COMPUTING RESEARCH INSTITUTE
Monitoring the Venezuelan Exodus with Facebook
Advertising Data
04

17. QATAR COMPUTING RESEARCH INSTITUTE
Facebook Advertising Platform

18. Correcting for non-Facebook users assuming same
Facebook penetration as in Colombia in general
Different UN estimates for Latin
America
Number of monthly active Facebook users who “lived
in Venezuela” and are living in Colombia at the time
Real-Time Estimates of Venezuelan Migrants and Refugees
Correcting for Facebook penetration, Facebook-derived estimates are larger than UN estimates.

19. QATAR COMPUTING RESEARCH INSTITUTE
Validation w/ (Few) Available Data
Registro de Administrativo de
Migrantes Venezolanos (RAMV)
- Jun, 2018
Facebook - Jun, 2018
Kendall's τ = .71 (n=31)

20. QATAR COMPUTING RESEARCH INSTITUTE
Previously Unavailable Estimates
Brazil - Facebook. Feb 2019 Peru - Facebook. Feb 2019 Ecuador - Facebook. Feb 2019

21. QATAR COMPUTING RESEARCH INSTITUTE
Predicted Income Based on OS
3% 54%

22. QATAR COMPUTING RESEARCH INSTITUTE
Computational Propaganda Detection
05

23. QATAR COMPUTING RESEARCH INSTITUTE
What is Propaganda?
“Communications that deliberately misrepresent symbols, appealing to emotions
and prejudices and bypassing rational thought, to influence its audience
towards a specific goal”*
• Not all influence has ill intent
• Tries to avoid detection
• Pursue an agenda
*definition re-elaborated from Institute for Propaganda Analysis (Ed.). (1938). How to Detect Propaganda. In
Propaganda Analysis. Volume I of the Publications of the Institute for Propaganda Analysis (pp. 210–218).

24. QATAR COMPUTING RESEARCH INSTITUTE
Different Propaganda Techniques
Name Calling Slogan

25. QATAR COMPUTING RESEARCH INSTITUTE
Many Different Propaganda Techniques Exist

26. QATAR COMPUTING RESEARCH INSTITUTE
Fine-Grained Propaganda Detection

27. QATAR COMPUTING RESEARCH INSTITUTE
PRTA and Tanbih Demos
https://www.tanbih.org/prta and https://www.tanbih.org/

28. QATAR COMPUTING RESEARCH INSTITUTE
High Level Reflections
06

29. QATAR COMPUTING RESEARCH INSTITUTE
(Selected) Opportunities and Challenges
Opportunities
Make citizen-government interaction more scalable
- Through social listening and/or ways to actively solicit feedback
Have real-time insights into conflicts and their impact
- Don’t have to wait until census
Alternative channels for data collection
- In-person surveys difficult due to covid-19
Challenges
Power structures
- Control of the data and analysis probably far from those it is supposed to serve
Bias in whose voice is amplified
- Not everybody has a digital footprint
Streetlamp bias
- Just because it’s convenient to collect, doesn’t make X the best data for the job
Author copies and preprints of the articles are openly accessible on the authors’ homepages or at https://www.researchgate.net/.

30. QATAR COMPUTING RESEARCH INSTITUTE
References
07

31. QATAR COMPUTING RESEARCH INSTITUTE
References
Using Twitter to Study the Antecedents of ISIS Support
First Monday 2016: #FailedRevolutions: Using Twitter to study the antecedents of ISIS support. Walid Magdy, Kareem Darwish, Ingmar Weber
HyperText 2015: “I like ISIS, but I want to watch Chris Nolan’s new movie” – Exploring ISIS Supporters on Twitter. Walid Magdy, Kareem Darwish, Ingmar Weber
Secular vs. Islamist Polarization in Egypt on Twitter
ASONAM 2013: Secular vs. Islamist Polarization in Egypt on Twitter. Ingmar Weber, Venkata Rama Kiran Garimella, Alaa Batayneh
ASONAM 2013: #Egypt: Visualizing Islamist vs. Secular tension on Twitter. Ingmar Weber, Venkata Rama Kiran Garimella
CSCW 2015: Content and Network Dynamics Behind Egyptian Political Polarization on Twitter. Javier Borge-Holthoefer, Walid Magdy, Kareem Darwish, Ingmar Weber
Mapping Education Insecurity Using Twitter
WWW 2014: AIDR: Artificial Intelligence for Disaster Response. Muhammad Imran, Carlos Castillo, Ji Lucas, Patrick Meier, Sarah Vieweg
CACM 2021: Non-traditional data sources: providing insights into sustainable development. Ingmar Weber, Muhammad Imran, Ferda Ofli, Fouad Mrad, Jennifer Colville, Mehdi Fathallah, Alissar Chaker, Wigdan Seed Ahmed
Monitoring the Venezuelan Exodus with Facebook Advertising Data
PLOS ONE 2018: Monitoring of the Venezuelan Exodus through Facebook’s Advertising Platform. Joao Palotti, Natalia Adler, Alfredo Morales-Guzman, Jeffrey Villaveces, Vedran Sekara, Manuel Garcia Herranz, Musa Al-Asad, Ingmar Weber
PDR 2017: Leveraging Facebooks Advertising Platform to Monitor Stocks of Migrants. Emilio Zagheni, Ingmar Weber, Krishna Gummadi
Computational Propaganda Detection
IJCAI 2020: A Survey on Computational Propaganda Detection. Giovanni Da San Martino, Stefano Cresci, Alberto Barrón-Cedeño, Seunghak Yu, Roberto Di Pietro, Preslav Nakov
EMNLP 2019: Fine-Grained Analysis of Propaganda in News Articles. Giovanni Da San Martino, Seunghak Yu, Alberto Barrón-Cedeño, Rostislav Petrov, Preslav Nakov
ACL 2020: Prta: A System to Support the Analysis of Propaganda Techniques in the News. Giovanni Da San Martino, Shaden Shaar, Yifan Zhang, Seunghak Yu, Alberto Barrón-Cedeño, Preslav Nakov
Author copies and preprints of the articles are openly accessible on the authors’ homepages or at https://www.researchgate.net/.

32. QATAR COMPUTING RESEARCH INSTITUTE
Thanks!
iweber@hbku.edu.qa