Slides from Disinformation talk at Disclosure conference 2020

1. Distributed Defense Against Disinformation
Lessons Learned in The CTI League’s Disinformation Team
SJ Terp & Pablo Breuer, CogSecCollab, Disclosure 2020
1

2. 2020: NOT YOUR
GRANDMA’S
DISINFORMATION
ECOSYSTEM
2
Find them, Fix them,
F<del><del>do something
about them

3. DISINFORMATION. DIS. NOT MIS.
“deliberate promotion… of false,
misleading or mis-attributed information
focus on online creation, propagation,
consumption of disinformation
We are especially interested in
disinformation designed to change
beliefs or emotions in a large number of
people”
3

4. MISINFORMATION: COVID19 NARRATIVES
• Origin myths
• Escaped bioweapon
• Country x created Covid
• US soldiers took Covid to China
• Covid isn’t serious
• Covid doesn’t exist
• Individual medical targets
• Medical scams
• MMS prevents
• Alcohol prevents
• etc
• Resolution myths
• Country x has a Covid cure
● Crossover with conspiracies
○ Covid and 5G
○ Covid and antivax / anti-Gates
○ Helicopters spraying for covid
○ Depopulation conspiracy
● Crossover with ‘freedom rights’
○ Anti- stayathome
○ 2nd amendment
○ anti-immigration
○ Usual far-rightwing groups
○ Antivax+2a+covid+
● Geopolitics
○ China, Iran: covert + overt
○ “Blue check” disinfo
4

5. MORE FRAMING: INTENT
• Geopolitical gain
• Countries
• Power gain
• Follow the divisions
• Financial gain
• Follow the money
• Attention seeking
• Sharks, satire and other LOLs
• Online discussion
• opinion
• conspiracies
• protest
• nazis
• all the above
Blahblahbla
hblah
5

6. MOTIVATIONS: MONEY
Covid19 hucksters are selling:
• T-shirts
• “Cures”
• Blood
• Views and clicks
• And a bunch of other stuff
Follow the money!
6

7. MONEY NEEDS URLS
7

8. EVOLUTION: #WeWontStayHome
8

9. EVOLUTION: Face Mask Exemption Cards
9

10. EVOLUTION: Face Mask Countermeasures
10

11. Evolution: Disinformation and Business
11

12. HONE YOUR UNDERSTANDING
Blue team: respond to disinformation
• response plan
• playbook
• tracking: campaigns, incidents, narratives, artifacts
Red team: learn from examples
• run a "disinformation as a service"/alternative marketing company,
• run a hostile social media platform,
• mix disinformation with coordinated sensor spoofs
• extend an existing narrative (e.g. anti-medical safety)

13. 2020: COGNITIVE
SECURITY IS REAL NOW
13
Join us

14. THOUSAND-POINT SOLUTION TO A THOUSAND-POINT PROBLEM
Collaborative, Heterogeneous, Connected

15. AXIS: DISINFORMATION LAYERS
1
5
Campaigns
Incidents
Narratives
Artifacts

16. AXIS: THREAT INTELLIGENCE TASKS
Find recent, current and future threats
• Intelligence analysis:
• Situation: what’s going on, who’s involved, best guesses about now and future
• Intent: why
• Attribution: who. Attribution is hard, and rarely necessary to act
• OSINT:
• Using public data, what can we tell responders, in time for them to act?
• (most disinformation is public and lends itself well to OSINT)
• Data science:
• find patterns and information in data
• 3V (volume, velocity, variety): too big, fast, different for humans to respond in time

17. AXIS: TIME

18. AXIS: RESPONSE TIMESCALES
• Strategic - weeks/months/years.
• Issue focussed. e.g. long-form journalism. [Stanford Internet
Observatory, UWashington, Shorenstein Center, Bellingcat, DFRlab,
Grafika, platforms].
• Operational - days/weeks.
• Project focussed. Usually embedded in dev team. Support behaviour/
hypothesis-driven development, lean enterprise (pruning value trees
etc). [AI/ML-based disinformation data and tool companies].
• Tactical - hours/days.
• Incident focussed. Includes many data journalists. [NY Times, CTI
League team, some of MLSEC, some of the crisis-mappers].

19. DISINFORMATION
INCIDENT
RESPONSE
Threat Intelligence with brains and feels
19

20. COMMUNITY: COGSECCOLLAB
10 years ago: Disaster Data Today: Disinformation
• Covid19Activation
• Covid19Disinformation
• CTI League Disinformation
• CogSecCollab
• Threet

21. PRACTICE: CTI LEAGUE’S DISINFORMATION TEAM
First Global Volunteer emergency response Community, defending
and neutralizing cybersecurity threats and vulnerabilities to the life-
saving sectors related to the current COVID-19 pandemic.
• Active Cyber-attack Neutralization
• Cyber-attack Prevention
• Cyber-supporting the Life-saving and Health-related Sectors
• Monitoring Cyberspace for Potential Danger to Public Health and
Safety.

22. DISINFORMATION SERVICES
• Neutralise: Disinformation incident response: triage, takedown, escalation.
• Clearinghouse: Collate and share incident data, including with organizations
focusing on response and countercampaigns.
• Prevent: Collate disinformation indicators of compromise (IoCs) and
vulnerabilities; supply to organisations.
• Support: Assess the possibility of direct attack, and ways to be ready.
22

23. INCIDENT RESPONSE REQUIREMENTS
• People
• Enough people to make a difference, in time
• Enough connections / levers to make a difference
• Culture
• Safety processes: mental health and opsec
• Process
• Understand disinformation, understand threat response
• Fast, lightweight processes
• Technology
• Speed - supporting analysis, storage etc
• Sharing - get data to responders in ways they understand (whatever works)

24. Response Support: Process, Tools, Training
Training: e.g. Data Science for Disinformation Response
• Introduction to the disinformation team
• What are we chasing? Digital harm training
• Getting set up for disinformation data science
• Data sources (process, sources, stores)
• Disinfo Data Science examples
• Social text analysis
• Image data analysis
• Relationships as data
• Extending your analysis with machine learning skillz
• Communicating results (style, narratives, visualisations)
24
Tools:
• HIVE/D3PO - case tracking
• Data tools - analysis support (scrapers etc)
• MISP - intelligence sharing (AMITT TTPs etc)
Process:

25. GOING BACK TO BASICS

26. OBSERVE:
2020’S DECADE OF
TRACKING COVID19
DISINFORMATION
trust us, it feels that way
26

27. TRIAGE: deciding whether to start an incident
• Is this potentially doing harm?
• What effects might this have?
• Is it large? Coordinated? Targetted (to demographics etc)?
• Is this disinformation
• Is the content false (e.g. misinformation?)
• Is it e.g. phishing rather than disinformation?
• Does it include fake groups, fake profiles, fake amplification etc
• Is our team the best one to respond to this?
• Is this in our area (e.g. CTI = currently working on Covid19 / medically-related?)
• Is someone else already tracking and responding to this?
• Do we have the resources to respond?
27

28. (Some) Tactical Tasks
• Activity analysis
• Track artefacts (messages, images, urls, accounts, groups etc), e.g.
• find artefact origins,
• track how artefact moves across channels/groups etc
• find related artefacts
• Detect AMITT Techniques, e.g.
• detect computational amplification
• detect, track and analyse narratives
• Network detection
• Find inauthentic website networks (pinkslime)
• Find inauthentic account and group networks (including botnets)
• Credibility/ Verification
• Fact-checking: verify article, image, video etc doesn’t contain disinformation.
• Source-checking: verify source (publisher, domain etc) doesn’t distribute disinformation.

29. Machine Learning / AI
• Graph analysis
• Find super-spreaders
• Find rumor origins
• Uncover new artefacts
• Track movement over time
• Text Analysis
• Find themes
• Classify to narratives
• Cluster text to narratives
• Search for similar text/narratives
• Image, video, audio analysis
• Cluster images
• Search for similar images
• Detect shallowfakes

30. ORIENT: GETTING
THE SITUATION
PICTURE
Sensemaking
30

31. Shared Awareness: MISP Threat Intelligence Platform
• Threat sharing standard with large
community
• EU funded (ENISA, CIRCL)
• ISAC, ISAO, CERTs, CSIRTs
• NATO, Military, Intelligence
• Fortune 500’s
• Open data standards
• MISP Core, STIX
• Connections
• API push/pull
• Email
• Anomali ThreatStream, ThreatConnect,
OSQuery
31

32. Data Standards: STIX Extension
Disinformation STIX Description Level Infosec STIX
Report communication to other responders Communication Report
Campaign Longer attacks (Russia’s interference in the 2016 US elections is
a “campaign”)
Strategy Campaign
Incident Shorter-duration attacks, often part of a campaign Strategy Intrusion Set
Course of Action Response Strategy Course of Action
Identity Actor (individual, group, organisation etc): creator, responder,
target, useful idiot etc.
Strategy Identity
Threat actor Incident creator Strategy Threat Actor
Attack pattern Technique used in incident (see framework for examples) TTP Attack pattern
Narrative Malicious narrative (story, meme) TTP Malware
Tool bot software, APIs, marketing tools TTP Tool
Observed Data artefacts like messages, user accounts, etc Artefact Observed Data
Indicator posting rates, follow rates etc Artefact Indicator
Vulnerability Cognitive biases, community structural weakness etc Vulnerability Vulnerability
32https://github.com/cogsec-collaborative/amitt_cti

33. Disinformation TTPs: AMITT Framework
33(TTP Framework adopted by MITRE, upstream development by CogSecCollab)

34. Other MISP Disinformation Objects
• taxonomies
● DFRLab Dichotomies of Disinformation
● NATO Disinformation Taxonomy (WIP)
• objects
• forged-document, publication, etc.
• twitter-post, facebook-post, reddit-post, etc.
• twitter-group, facebook-group, reddit-
subreddit, etc
• twitter-account, facebook-account, reddit-
account, etc.
• relationships
• authored-by
• relates-to, etc.
34

35. DECIDE: OPTIONS ON
COUNTERMEASURES,
MITIGATIONS,
RESILIENCE
Not just admiring the problem any
more
35

36. Response: Actors
• Platform
• Law Enforcement
• Government
• Elves
• Public
• Influencer
• Media
• Nonprofit
• Educator
• Corporation
36

37. Response: Mitigations and Countermeasures
37
● Detect: find them
● Deny: stop them getting in
● Disrupt: interrupt them
● Degrade: slow them down
● Deceive: divert them
● Destroy: damage them
● Deter: discourage them
https://github.com/cogsec-collaborative/amitt_counters

38. ACT: GETTING
COLLABORATIVE,
GETTING
INTERACTIVE
And the waves to come
38

39. Back To Multiplayer Games

40. THANK YOU
@bodaceacat
@ngree_h0bit
https://cogsec-collab.org/
40