Presentation "Automatic detection of online misinformation: Are we there yet?" By Kalina Bontcheva An internal workshop at the Alan Turing Institute, London, 19 January 2019

1. Automatic detection of
online misinformation:
Are we there yet?
Kalina Bontcheva
The University of Sheffield
An internal workshop at the Alan Turing Institute, London
19 January 2019
© The University of Sheffield
This work is licensed under the
Creative Commons Attribution-NonCommercial-ShareAlike Licence

2. PHEME: Analysing Online Rumours
@PhemeEU
● Memes are thematic motifs that spread through
social media in ways analogous to genetic traits
● We coined the term phemes to add truthfulness
and deception to the mix
● Named after ancient Greek Pheme, “embodiment
of fame and notoriety, her favour being notability,
her wrath being scandalous rumours"

3. PHEME: Analysing Online Rumours
@PhemeEU

4. PHEME: Rumour Sources, Veracity,...
@PhemeEU

5. PHEME: ...Veracity, and Spread
@PhemeEU

6. PHEME: Analysing Rumours Automatically
@PhemeEU

7. PHEME: Analysing Rumours Automatically
● Rumour Stance Classification
○ Four way classification on unseen events -
78.4% accuracy with deep learning
7

8. Rumour Veracity Classification
● Built on PHEME results
● Rumour Veracity Classification
○ 3-way veracity classification of newly
emerging rumours
■ 60.7% accuracy and 61.6% F1 measure on
unseen rumours and no stance
■ 75% accuracy and 74.3% F1 measure with a
few examples from the target rumour
8
@comradesproject

9. Rumour Analysis: Are We There Yet?
● 75% accuracy = being wrong ¼ of the time!
● We need:
○ More annotated data to train better models
○ UIs to assist, not replace the professionals
9

10. InVID: Verifying A Deep Fake Video
Source:
Denis
Teyssou,
AFP
@dteyssou

11. InVID: Verifying A Deep Fake Video (2)

12. Analysing Disinformation Campaigns
https://twitter.com/dteyssou/status/1052075694694559745
Target:
Belgian MEP
Guy
Verhofstadt
Agent:
Hungarian
government

13. Analysing Disinformation Campaigns
● Impact of Inaccurate Claims by Politicians
○ £350m false claim - 10.2 times more tweets
than the 3,200 tweets by the Russia-linked
accounts suspended by Twitter
○ More than 1,500 tweets from different voters
■ I am with @Vote_leave because we should stop
sending £350 million per week to Brussels, and spend
our money on our NHS instead.
■ I just voted to leave the EU by postal vote! Stop
sending our tax money to Europe, spend it on the NHS
instead! #VoteLeave #EUreferendum
○ Ipsos Mori (22/06/2016) - for 9% the NHS was the most
important issue in the campaign

14. What’s Next?
14

15. WeVerify: Next Generation Verification Tools
@WeVerify

16. Better and Simpler Verification Tools

17. Better and Simpler Verification Tools

18. Other Challenges Ahead
● Preserving important social media content for
future studies
● Establish policies for ethical, privacy-preserving
research and data analytics
● More funding for inter-disciplinary research
● Measure the effectiveness of technological
solutions implemented by social media platforms
● Strengthening media and improving journalism and
political campaigning standards

19. Thank you!
Questions?
Details in this STOA report:
Alaphilippe, A., Bontcheva, K., Gizikis, A. Automated
tackling of disinformation: Major challenges ahead.
http://www.europarl.europa.eu/thinktank/en/docum
ent.html?reference=EPRS_STU(2019)624278
email: K.Bontcheva@sheffield.ac.uk
twitter: @kbontcheva

20. Acknowledgements
Work partially support by funding from the European Union’s Horizon 2020
research and innovation programme WeVerify(825297), SoBigData(654024),
COMRADES (687847).
WeVerify: https://weverify.eu/
COMRADES: https://www.comrades-project.eu/
SoBigData: http://sobigdata.eu/index
20
This document does not represent the opinion of the European Community, and the European Community is not responsible for any use
that might be made of its content