Understanding Online Misinformation: Major Challenges Ahead, Rome,
1. Understanding Online Misinformation:
Major Challenges Ahead
Carolina Scarton
c.scarton@Sheffield.ac.uk
Keynote @ ROME Workshop, Paris, 25/07/2019
1WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
2. Is Online Misinformation a Big Problem for Citizens?
2WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
Source: Eurobarometer
2018
3. Do citizens know how to spot it?
3WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
Source: Eurobarometer
2018
4. 4WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
Information disorder theoretical framework
(Wardle, 2017; Wardle & Derakshan, 2017)
Very difficult to distinguish
🡪 mainly between MIS-
and DISinformation
5. Misinformation lifecycle?
5WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
Source: STOA report 2019
6. The 6 Questions of Misinformation Analysis
What is being spread?
Where it spreads?
How it spreads?
Who is spreading it?
Why it spreads?
When it spreads?
6WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
7. The 6 Questions of Misinformation Analysis
What is being spread?
Where it spreads?
How it spreads?
Who is spreading it?
Why it spreads?
When it spreads?
7WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
8. Examples
8WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
https://www.buzzfeednews.com/article/ishmaeldaro/roundup-of-misinformation-on-youtube-shooting
9. Examples
9WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
10. Examples
10WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
British news parody show The Day Today - BBC2 in
1994
Heathrow airport website
https://www.heathrow.com/noise/what-you-can- do/ice-falls-from-aircraft
Despite popular beliefs, modern aircraft do not have the facility to
eject toilet waste whilst they are airborne. Waste collection
happens when the aircraft lands at an airport and is disposed of
responsibly.
11. The 6 Questions of Misinformation Analysis
What is being spread?
Where it spreads?
How it spreads?
Who is spreading it?
Why it spreads?
When it spreads?
11WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
12. 12
WeVerify project has received funding from the European Union's Horizon 2020 research and innovation
programme under grant agreement No 825297
print
& debunk
Source: https://medium.com/1st-draft/5-lessons-for-reporting-in-an-age-of-disinformation-9d98f0441722
WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
The Trumpet of Amplification
13. Low Credibility Website Networks
13WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
A network of seemingly UK-based
far-right news sites (operated from
Eastern Europe)
Shared and amplified through
thirteen related Facebook pages
2.4 million likes (more than any
UK Facebook political page)
Source: (Reynolds, 2018)
14. Misinformation in search results
14WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
Source: (Albright, 2018)
15. Closed Networks
15WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
Harmless
Harmful
16. The 6 Questions of Misinformation Analysis
What is being spread?
Where it spreads?
How it spreads?
Who is spreading it?
Why it spreads?
When it spreads?
16WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
17. Fake Profile, Advertising and Clickbait
Fake Profiles
• IRA operated cyborg accounts spread misinformation during the 2016 US
elections
• Jenna Abrams - widely considered a real American
• Fake accounts try to gain credibility by following genuine accounts
• As many as 60% of Donald Trump’s followers being suspected fake
accounts (Campoy, 2018)
Advertising and Clickbait
• Online advertising used extensively to make money from junk news sites
• A clickbait post is designed to provoke emotional response in its readers
17WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
18. Micro-targetting or dark ads
Online adverts that are visible only to the
users that are being targeted
• e.g. voters in a marginal UK
constituency (Cadwalladr, 2017a)
Do not appear on the advertiser’s timeline
or in the feeds of the advertiser’s followers
Micro-targeting - fine-grained ad targeting,
based on job titles or demographic data
18WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
https://www.parliament.uk/documents/commons-committees/culture-media-and-sport/Fake_news_evidence/Ads-supplied-by-Fac
ebook-to-the-DCMS-Committee.pdf
19. Misinformation in Political Advertising Online
Russia Today & related acc promoted just under 2,000 election-related tweets
• generated around 53.5 million impression on U.S. based users (Edgett,2017)
Issues around digital campaigning by political organisations during the 2016 UK EU
membership referendum (DCMS report, 2018)
Questions also around the Trump presidential campaign use of over 5.9 million
Facebook adverts (Frier, 2018)
£350m false claim - 10.2 times more tweets than the 3,200 tweets by the
Russia-linked accounts suspended by Twitter
• More than 1,500 tweets from different voter
19WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
20. Mainstream Media
20WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
https://www.ipso.co.uk/news-press-releases/press-releases/ipso-upholds-complaint-against-the-sun-s-queen-backs-brexit-headline/
https://revistacrescer.globo.com/Criancas/Seguranca/noticia/2019
/03/momo-aparece-em-videos-de-slime-do-youtube-kids-e-ensina-
criancas-se-suicidarem.html
21. False Amplifiers: Bots
Bots that spread malware and unsolicited content
disseminated anti-vaccine messages
Russian trolls promoted discord
Accounts masquerading as legitimate users create false
equivalency, eroding public consensus on vaccination
21WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
22. False Amplifiers: Fake Groups
Astroturfing 🡪 creating artificial appearance of “grass-roots” support
Initially seeded with fake accounts, before drawing in genuine users
Other fake groups are created to “spread sensationalistic or heavily biased news
or headlines, often distorting facts to fit a narrative” (Weedon, Nulan & Stamos,
2017)
22WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
23. Genuine Amplifiers
The main amplifiers behind viral misinformation and propaganda are genuine human
users (Vosoughi et al, 2018)
Confirmation bias 🡪 reading news that conforms to the individual’s political views
Homophily
• Individual’s information sharing and commenting behaviour is influenced by the
behaviour of their online social connections
Confirmation bias and homophily lead to the creation of online echo chambers
(Quattrociocchi et al. 2016)
Polarisation
• “social networks and search engines are associated with an increase in the mean
ideological distance between individuals” (Flaxman et al, 2018)
23WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
24. The 6 Questions of Misinformation Analysis
What is being spread?
Where it spreads?
How it spreads?
Who is spreading it?
Why it spreads?
When it spreads?
24WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
25. Online Rumours Analysis
25WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
26. Online Rumours Analysis
26WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
27. PHEME Project
Rumour is “a circulating story of questionable veracity, which is
apparently credible but hard to verify, and produces sufficient
skepticism and/or anxiety”
Examples:
True rumour: "10 people dead in Charlie Hebdo according to
witnesses”
False rumour: "GERMAN NEWS REPORT: Co-Pilot of
Germanwings Airbus Was MUSLIM CONVERT …’Hero of Islamic
State’?”
Unverified rumour: "Police in Ferguson claimed that Mike Brown
had been involved in a robbery”
27WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
28. Rumour detection and verification
28WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
29. Detecting a rumour
First step in rumour analysis
• Can be omitted if the rumour is already known!
• Binary classification
Can be identified through reactions from “skeptical users”
• If a post has a considerable amount of enquiring tweets 🡪 information is rumourous
Datasets from PHEME project
• Includes a collection of 1,972 rumours and 3,830 non-rumours
• Associated with five breaking news stories
29WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
30. Tracking a rumour
Second step in rumour analysis
• Previous work uses already identified rumours 🡪 no studies for emerging rumours
• Can involve network monitoring
Widely used dataset for rumour tracking (Qazvinian et al., 2011)
• Includes over 10,000 tweets, associated with 5 different rumours
• Each tweet annotated for relevance towards the rumour as related or unrelated
30WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
31. Stance classification
Third step in rumour analysis
• Determining the type of orientation that each individual post expresses towards the
disputed veracity of a rumour
• Relevant in the context of social media where unverified reports are continually being
posted and discussed
• Can guide veracity classification
31WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
32. Stance classification
32WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
34. Stance classification
RumourEval 2019 shared task
• Task A: stance classification (Comment/Support/Deny/Query)
34WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
Task B
Macro F
System A 0.6187
System B 0.6067
System X 0.5776
https://competitions.codalab.org/competitions/19938
35. Veracity classification
Final step in rumour analysis
• Determine the veracity of an entire rumour
• Binary or three-class problem (true/false/unverified)
• Potentially less useful for professionals
RumourEval 2019 shared task
• Three-class: true/false/unverified
35WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
Task A
Macro F RMSE
System X 0.5765 0.6078
System Y 0.2856 0.7642
System Z 0.2620 0.8012
36. WeVerify project
36WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
41. WeVerify
Cross-modal and cross-platform approach
• Evidence from text, images, and videos; deep fakes
Overcome effort fragmentation
• A blockchain-based, authoritative database of already debunked fake items
Verification made simple
• A digital companion for citizens and an advanced version for journalists
Look beyond the content
• Analysis of disinformation source, spread strategies, and community networks
Build a community, not just technology
• Pilot with large stakeholder communities
41WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
42. Challenges on rumour detection and verification
Scarcity of data
Several levels of veracity (a claim can be “half true”, “inaccurate”)
Ethical approval 🡪 social data is a kind of personal data
Requirements for data storage and usage
Difficulty to reach closed channels (e.g. WhatsApp)
42WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297
43. Other Challenges
Deep fakes
• Synthetic videos and images “look and sound like a real person saying
something that that person has never said.” (Lucas, 2018)
Preserve important social media content for future studies
Establish policies for ethical, privacy-preserving research and data
analytics
Measure the effectiveness of technological solutions implemented by
social media platforms
43WeVerify project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825297