Understanding Online Misinformation: Major Challenges Ahead. By Kalina Bontcheva. Berlin, January 2020. Keynote at Qurator Conference.

1. Understanding Online Misinformation:
Major Challenges Ahead
Kalina Bontcheva
K.Bontcheva@sheffield.ac.uk
Keynote talk @ QURATOR Conference, Berlin, 21 January 2020
1WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297

2. Is Online Misinformation a Big Problem for Citizens?
2WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
Source: Eurobarometer
2018

3. Do citizens know how to spot it?
3WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
Source: Eurobarometer
2018

4. 4WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
Information disorder theoretical framework
(Wardle, 2017; Wardle & Derakshan, 2017)
Very difficult to distinguish
🡪 mainly between MIS-
and DISinformation

5. Misinformation lifecycle?
5WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
Source: STOA report 2019

6. The 6 Questions of Misinformation Analysis
What is being spread?
Where it spreads?
How it spreads?
Who is spreading it?
Why it spreads?
When it spreads?
6WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297

7. The 6 Questions of Misinformation Analysis
What is being spread?
Where it spreads?
How it spreads?
Who is spreading it?
Why it spreads?
When it spreads?
7WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297

8. Examples
8WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
https://www.buzzfeednews.com/article/ishmaeldaro/roundup-of-misinformation-on-youtube-shooting

9. Examples
9WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297

10. Examples
10WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
British news parody show The Day Today - BBC2 in
1994
Heathrow airport website
https://www.heathrow.com/noise/what-you-can- do/ice-falls-from-aircraft
Despite popular beliefs, modern aircraft do not have the facility to
eject toilet waste whilst they are airborne. Waste collection
happens when the aircraft lands at an airport and is disposed of
responsibly.

11. The 6 Questions of Misinformation Analysis
What is being spread?
Where it spreads?
How it spreads?
Who is spreading it?
Why it spreads?
When it spreads?
11WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297

12. 12
WeVerify project has received funding from the European Union's Horizon 2020 research and innovation
programme under grant agreement No 825297
print
& debunk
Source: https://medium.com/1st-draft/5-lessons-for-reporting-in-an-age-of-disinformation-9d98f0441722
WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
The Trumpet of Amplification

13. Low Credibility Website Networks
13WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
A network of seemingly UK-based
far-right news sites (operated from
Eastern Europe)
Shared and amplified through
thirteen related Facebook pages
2.4 million likes (more than any
UK Facebook political page)
Source: (Reynolds, 2018)

14. Misinformation in search results
14WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
Source: (Albright, 2018)

15. Closed Networks
15WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
Harmless
Harmful

16. The 6 Questions of Misinformation Analysis
What is being spread?
Where it spreads?
How it spreads?
Who is spreading it?
Why it spreads?
When it spreads?
16WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297

17. Fake Profile, Advertising and Clickbait
Fake Profiles
• IRA operated cyborg accounts spread misinformation during the 2016 US
elections
• Jenna Abrams - widely considered a real American
• Fake accounts try to gain credibility by following genuine accounts
• As many as 60% of Donald Trump’s followers being suspected fake
accounts (Campoy, 2018)
Advertising and Clickbait
• Online advertising used extensively to make money from junk news sites
• A clickbait post is designed to provoke emotional response in its readers
17WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297

18. Micro-targetting or dark ads
Online adverts that are visible only to the
users that are being targeted
• e.g. voters in a marginal UK
constituency (Cadwalladr, 2017a)
Do not appear on the advertiser’s timeline
or in the feeds of the advertiser’s followers
Micro-targeting - fine-grained ad targeting,
based on job titles or demographic data
18WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
https://www.parliament.uk/documents/commons-committees/culture-media-and-sport/Fake_news_evidence/Ads-supplied-by-Fac
ebook-to-the-DCMS-Committee.pdf

19. Misinformation in Political Advertising Online
Russia Today & related acc promoted just under 2,000 election-related tweets
• generated around 53.5 million impression on U.S. based users (Edgett,2017)
Issues around digital campaigning by political organisations during the 2016 UK EU
membership referendum (DCMS report, 2018)
Questions also around the Trump presidential campaign use of over 5.9 million
Facebook adverts (Frier, 2018)
£350m false claim - 10.2 times more tweets than the 3,200 tweets by the
Russia-linked accounts suspended by Twitter
• More than 1,500 tweets from different voter
19WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297

20. Mainstream Media
20WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
https://www.ipso.co.uk/news-press-releases/press-releases/ipso-upholds-complaint-against-the-sun-s-queen-backs-brexit-headline/
https://revistacrescer.globo.com/Criancas/Seguranca/noticia/2019
/03/momo-aparece-em-videos-de-slime-do-youtube-kids-e-ensina-
criancas-se-suicidarem.html

21. False Amplifiers: Bots
Bots that spread malware and unsolicited content
disseminated anti-vaccine messages
Russian trolls promoted discord
Accounts masquerading as legitimate users create false
equivalency, eroding public consensus on vaccination
21WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297

22. False Amplifiers: Fake Groups
Astroturfing 🡪 creating artificial appearance of “grass-roots” support
Initially seeded with fake accounts, before drawing in genuine users
Other fake groups are created to “spread sensationalistic or heavily biased news
or headlines, often distorting facts to fit a narrative” (Weedon, Nulan & Stamos,
2017)
22WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297

23. Genuine Amplifiers
The main amplifiers behind viral misinformation and propaganda are genuine human
users (Vosoughi et al, 2018)
Confirmation bias 🡪 reading news that conforms to the individual’s political views
Homophily
• Individual’s information sharing and commenting behaviour is influenced by the
behaviour of their online social connections
Confirmation bias and homophily lead to the creation of online echo chambers
(Quattrociocchi et al. 2016)
Polarisation
• “social networks and search engines are associated with an increase in the mean
ideological distance between individuals” (Flaxman et al, 2018)
23WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297

24. The 6 Questions of Misinformation Analysis
What is being spread?
Where it spreads?
How it spreads?
Who is spreading it?
Why it spreads?
When it spreads?
24WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297

25. Online Rumours Analysis
25WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297

26. Online Rumours Analysis
26WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297

27. PHEME Project
Rumour is “a circulating story of questionable veracity, which is
apparently credible but hard to verify, and produces sufficient
skepticism and/or anxiety”
Examples:
True rumour: "10 people dead in Charlie Hebdo according to
witnesses”
False rumour: "GERMAN NEWS REPORT: Co-Pilot of
Germanwings Airbus Was MUSLIM CONVERT …’Hero of Islamic
State’?”
Unverified rumour: "Police in Ferguson claimed that Mike Brown
had been involved in a robbery”
27WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297

28. Rumour detection and verification
28WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297

29. Detecting a rumour
First step in rumour analysis
• Can be omitted if the rumour is already known!
• Binary classification
Can be identified through reactions from “skeptical users”
• If a post has a considerable amount of enquiring tweets 🡪 information is rumourous
Datasets from PHEME project
• Includes a collection of 1,972 rumours and 3,830 non-rumours
• Associated with five breaking news stories
29WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297

30. Tracking a rumour
Second step in rumour analysis
• Previous work uses already identified rumours 🡪 no studies for emerging rumours
• Can involve network monitoring
Widely used dataset for rumour tracking (Qazvinian et al., 2011)
• Includes over 10,000 tweets, associated with 5 different rumours
• Each tweet annotated for relevance towards the rumour as related or unrelated
30WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297

31. Stance classification
Third step in rumour analysis
• Determining the type of orientation that each individual post expresses towards the
disputed veracity of a rumour
• Relevant in the context of social media where unverified reports are continually being
posted and discussed
• Can guide veracity classification
31WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297

32. Stance classification
32WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297

33. Stance classification
PHEME dataset
33WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297

34. Stance classification
RumourEval 2019 shared task
• Task A: stance classification (Comment/Support/Deny/Query)
34WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
Task B
Macro F
System A 0.6187
System B 0.6067
System X 0.5776
https://competitions.codalab.org/competitions/19938

35. Veracity classification
Final step in rumour analysis
• Determine the veracity of an entire rumour
• Binary or three-class problem (true/false/unverified)
• Potentially less useful for professionals
RumourEval 2019 shared task
• Three-class: true/false/unverified
35WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
Task A
Macro F RMSE
System X 0.5765 0.6078
System Y 0.2856 0.7642
System Z 0.2620 0.8012

36. WeVerify project
36WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297

37. 37
Credit: Denis Teyssou (AFP)
Facebook vs Twitter misinformation

38. 38
Multilingual Search
Credit: Denis Teyssou (AFP)

39. 39
Google getting confused
Credit: Denis Teyssou (AFP)

40. 40
Google getting confused
Credit: Denis Teyssou (AFP)
Database of
Known Fakes can
help!

41. WeVerify
Cross-modal and cross-platform approach
• Evidence from text, images, and videos; deep fakes
Overcome effort fragmentation
• A blockchain-based, authoritative database of already debunked fake items
Verification made simple
• A digital companion for citizens and an advanced version for journalists
Look beyond the content
• Analysis of disinformation source, spread strategies, and community networks
Build a community, not just technology
• Pilot with large stakeholder communities
41WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297

42. Challenges on rumour detection and verification
Scarcity of data
Several levels of veracity (a claim can be “half true”, “inaccurate”)
Ethical approval 🡪 social data is a kind of personal data
Requirements for data storage and usage
Difficulty to reach closed channels (e.g. WhatsApp)
42WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297

43. Other Challenges
Deep fakes
• Synthetic videos and images “look and sound like a real person saying
something that that person has never said.” (Lucas, 2018)
Preserve important social media content for future studies
Establish policies for ethical, privacy-preserving research and data
analytics
Measure the effectiveness of technological solutions implemented by
social media platforms
43WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297

44. Thank you!
Kalina Bontcheva
K.Bontcheva@Sheffield.ac.uk
QURATOR Conference, Berlin, 21 Jan 2020
44WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297