A discussion of some approaches taken to detecting and tracking misinformation at scale in the Internet age

1. 20 February 2020
Detecting and tracking
misinformation in the
Internet age

2. FACT CHECKING APPROACH:
UNCOVERING THE TRUTH
Detecting and tracking fake news and misinformation at scale

3. • An excellent approach and something to be deployed with vigour in any
situation where it can usefully be applied
but ...
• Problem #1 It rarely happens and when it does, it’s often an accident
• Problem #2 It’s takes a lot of effort for humans to do it
• Problem #3 It’s impossible (more or less) for computers
Detecting and tracking fake news and misinformation at scale

4. Solution #1 Do it directly with machines anyway!

5. Solution #2 Try to spot it indirectly with correlating proxies

6. Solution #2 Other correlating proxies are available ... ‘canary accounts’

7. • Solution #3 Use machines to increase human productivity
Detecting and tracking fake news and misinformation at scale

8. Case Study: Fact checking your article

9. Case Study: Fact checking your article
Read
article
Identify
claims
Collect
evidence
Rank
evidence
Output

10. Detecting and tracking fake news and misinformation at scale
• Problem #4 It’s always something new
• Problem #5 True believers don’t care
• Problem #6 It’s just not good politics

11. CONFLICT APPROACH:
UNCOVERING THE CONTEST
Detecting and tracking fake news and misinformation at scale

12. • Characterise the conflict
• Identify the activities
Detecting and tracking fake news and misinformation at scale

13. Case Study: Disrupting Daesh – Golden Age
• 2014-2015 Golden Age on Twitter for Islamic State
• Thriving online community (50,000 – 70,000) active accounts
• Very easy access to contact and content
• Obvious markers of support (avatars, screen names, hashtags)
• Strong and supportive ideological community and sub-
communities (e.g. Chechens, ‘Sisters’)

14. Case Study: Disrupting Daesh – late 2015 disruption begins
• From mid 2015 - community disruption begins
o Account suspensions and takedowns
o Disruption of hashtags
• Reactions:
o Flight to Telegram
o May have strengthened community cohesion
• Late 2016: what was left?
o Impact on online Twitter community?
o Activities on Twitter?

15. Method52: A platform for agile modular investigation

16. Method52 allows user to 'fail fast' and iterate to find patterns of use
• Grounded theory (Glaser et al., 1968)
• "Unbiased examination of the available data"
• Iterative exploration of what fits
Scheme 1 Scheme 2 Scheme 3

17. Case Study: Disrupting Daesh. Build bespoke pipelines that are
adapted to the specific scenario
Data
Store
social
media
data
Construction,
maintenance &
analysis tools
Disruption
Monitoring
System
Visualisation
& Evaluation
Daesh
propaganda
analysis system
Visualisation
Engine
Pipeline
Construction
Engine
1
2 3
4
5
seed
accounts
A
B

18. Case Study: Disrupting Daesh
Data Store
-Account details
-Tweet details
-Link details
tweets
Score & analyse
confirmed accounts
Assess
relevancy
(i)
seed
accounts
seed
search
terms
Analyse links in
flagged tweets
Identify new
terms
(ii)
(iii)
(iv)
(v)
(vi)

19. CANDIDATE ACCOUNTS
Strategies for identifying accounts
• Content of tweets
• Generic words (qa'idin, bay'ah, nifaq, mushrik)
• Current topics (tabqa, Suwaydiya, Abu Ali al-Turki)
• Presence of generic coms links (Telegram, YouTube etc.)
• Specific known links (images, YouTube, other videos)
• Specific known hashtags (#tabqa)
• Mentions of specific 'canary' accounts (@39_nas)
• Network analysis
• Build out and understand network. Possible typology: 'source',
'canary', 'news gathering', 'signpost' and 'protected chat'
accounts
• Followers of known 'source' accounts (p_vanostaeyen)
• Followers of known 'canary' accounts (whoamidude)
• Followed by or followers of network members (protected chat
network, 'news gathering' accounts, 'signpost' accounts)
Case Study: Disrupting Daesh - strategies for identifying accounts

20. Tweets Followers Friends
IS 51 14 33
Other Jihadi 320 189 122
Case Study: Disrupting Daesh - account suspension rate

21. Case Study: Disrupting Daesh – seeding propaganda
11 12 13 14 15 16 17 18 19 20 21 22 23 00 01 02 03 04 05 06 07
DocPakistan
justpaste.it
KronaThe
omar_367
KronaThe
KronaThe
DocPakistan
justpaste.it
lhfg08
nycijm
sendvid.com
693mstafa
el_cvk
AllyOfTruth
justpaste.it
DocPakistan
justpaste.it
JulUllil
el_uhj
el_bhv x2
el_gyf
y8m...
qfg...
7wv...
wmy...
onk...
y8m...
skdj...
njnj...
0 3 3 4 31 41 61 81 121 137 525 632 697 766 776 777 790 794 798 835 842
archive.org vid
justpaste.it x2
sendvid.com
youtube.com
youtu.be
justpaste.it
Google photos
Googledrive
yadi.sk
youtu.be
archive (justpaste.it)
Language Key
English
Somali
Arabic

22. Case Study: Disrupting Daesh - account suspension rate

23. * Excludes 7 Mar which had 240 URLs (Rumiyah release)
0%
20%
40%
60%
80%
100%
Feb Mar Apr
URLs per day (mean)
Others (26 domains)
vimple.co, store6.up, pc.cd,
4shared.com
the vid.net
Google Drive
YouTube
sendvid.com
archive.org
IS’s own server
justpaste.it
4 Feb – 8 Feb 4 Mar – 8 Mar* 4 Apr – 8 Apr
cloud.mail.ru, addpost.it, vid.me
Case Study: Disrupting Daesh – URLs used as destinations

24. Note: All accounts
tracked were created
before 0600Z on Tuesday
4 April. Data set created
at 0600Z
Case Study: Disrupting Daesh – intercepting the propaganda

25. *Print media, websites, forums, social media
Inbound
Data*
Assess
relevancy
Sites and
accounts
Analyse
message
Search
terms
Identify
accounts
Identify
narrative
Cluster
narratives
Identify
attributes
Identify
networks
Emerging general methodology: the first iteration

26. CONFLICT APPROACH:
UNCOVERING THE CONTEST
Detecting and tracking fake news and misinformation at scale

27. Characterising conflict: The concept of ’Information Operations’
• Information operations are vast in scale and numerous in strategies and tactics
• A focus on ‘fake news’ or ‘misinformation’ is myopic
• Most information is not ‘fake’, but the selective amplification of reputable stories
• Information operations are characterised by erratic bursts of activity
• Information operations exploit cultural and social division
• Although information operations are coordinated, they are inconsistent, presenting a
challenge to third-party identification of inauthentic accounts.

28. Case Study: Internet Research Agency operations in the UK
Phase 1: Spam and the process of building credible accounts
I'm ready to eat healthy and workout.
@xhibellamy @William_Stokes @guru_paul
@ThomasAmor1 @jennyc08318 @richtweten
http://t.co/TAZ9Co1QF9
.@pedrareyes148 pedra @Chloe0354
ASDFGchloeHJKLL? @pulmonxry Yeezus
@Nick281051 Nick @puffylore163 lore
http://t.co/ZLpIlrsV33

29. Case Study: Internet Research Agency operations in the UK
Phase 2: Brexit Vote
Those who are still EU members can enjoy
their political correctness and tolerance
#BrexitVote https://t.co/VeMW7bagDQ
This is the simplest explanation. Just like UK we
too want to stop globalist liberals from ruining
us! #BrexitVote https://t.co/XkNFpNof1c

30. Case Study: Internet Research Agency operations in the UK
Phase 3: London Terror Attacks
Welcome To The New Europe! Muslim
migrants shouting in London “This is our
country now, GET OUT!” #Rapefugees
https://t.co/GCiFT96h76
Sharia NO-GO areas in BRITAIN. Citizens
blocked from their own suburbs. Only #Trump
can stop this here! https://t.co/IuQDe8rvPA

31. Case Study: Internet Research Agency operations in Europe

32. DISINFO: GEARING UP FOR THE
US PRESIDENTIAL ELECTIONS
Detecting and tracking fake news and misinformation at scale

33. Colleagues who participated in the work and/or developed Method52

34. www.taglaboratory.org
Jeremy Reffin