Paper presented at ECREA 2022, Aarhus, 21 Oct. 2022.

1. CRICOS No.00213J
‘Fake News’ on Facebook: A Large-Scale,
Longitudinal Study of Problematic
Information Dissemination between
2016 and 2021
Axel Bruns1, Daniel Angus1, Xue Ying (Jane) Tan1, Edward Hurcombe1,
Nadia Jude1, Phoebe Matich1, Stephen Harrington1,
Jennifer Stromer-Galley2, Karin Wahl-Jorgensen3, Scott Wright4
1Queensland University of Technology, Australia
2Syracuse University, USA
3Cardiff University, Wales
4Bournemouth University, England
@snurb_dot_info @antmandan @qutdmrc

2. CRICOS No.00213J

3. CRICOS No.00213J
Evaluating the Challenge of ‘Fake News’ and Other Malinformation
Prof Axel Bruns, Prof Daniel Angus, A/Prof Stephen Harrington, Dr Edward Hurcombe, Ms Jane Tan,
Prof Scott Wright (Bournemouth), Prof Jennifer Stromer-Galley (Syracuse), Prof Karin Wahl-
Jorgensen (Cardiff)
ARC Discovery Project (2020 - 2022)
This project conducts a systematic, large-scale, mixed-methods analysis of empirical evidence on
the dissemination of, engagement with, and impact of ‘fake news’ and other malinformation in
public debate, in Australia and beyond.

4. CRICOS No.00213J
Objectives
• to identify and thematically categorise, for Facebook, the public pages, groups, and verified
profiles that are most active in linking to identified sources of problematic information;
• to identify and rank the influence of the sources of problematic information shared by these
public spaces on Facebook, using Facebook’s own engagement metrics;
• to examine the themes and topics addressed, and the sources linked to, by the most active such
public pages and groups, in their day-to-day activities beyond the sharing of problematic news
content; and
• to examine and analyse the patterns of such activity over a five-year timeframe, and identify the
impact of major political and other events during that time on posting and sharing activity.

5. CRICOS No.00213J
Finding relevant data (FakeNIX)
• Iteratively updated masterlist of domains listed in existing studies of ‘fake news’
• 2,314 domains to date (from Shao et al., 2016; Starbird et al., 2017; Allcott et al., 2018; Grinberg et al., 2019;
Guess et al., 2018; 2019; etc.)
• Data from CrowdTangle: any Facebook posts from public pages / groups / verified profiles that contained
links to any of these domains
• 1 Jan. 2016 to 31 Mar. 2021: 42.6million posts from 918,760 pages/groups
Limitations:
• ‘Fake news’ domain lists largely US- / Anglocentric
• Crowdtangle’s coverage is not complete

6. CRICOS No.00213J
US progressives
US conservatives
France /
Germany
Italy
Brazil
India
alternative
health
conspiracies
UK
alternative
finance
Nodes: public pages, groups, verified profiles / domains in posts
Size: weighted in-degree
Colour: weighted in-degree
FakeNIX domain posts, 1 Jan. 2016 to 31 Mar. 2021
Angus, D., Bruns, A., Hurcombe, E., & Harrington, S. (2021). ‘Fake news’ on Facebook: a
large-scale longitudinal study of problematic link-sharing practices from 2016 to 2020.
In Selected Papers in Internet Research 2021: Research from the Annual Conference of the
Association of Internet Researchers AoIR - Association of Internet
Researchers. https://doi.org/10.5210/spir.v2021i0.12089

7. CRICOS No.00213J
Thematic mapping of pages/groups
• Too many pages/groups in our collection to capture all content (918,760 in total)
…instead…
• What characterises the top 500 pages, and 500 groups in our collection?
• Dimensions of interest:
• the subscriber count;
• count of distinct FakeNix domains it has shared at least one link to;
• number of links to any FakeNix domain; and
• total engagement from users towards this content.

8. CRICOS No.00213J
Rank product
• Technique from genetics (Breitling et. al., 2004), useful in combining multiple quantities into a
single rank.
• Uses a simple geometric mean of the ranks of each quantity.
• Rank product = (subscriber rank * domains rank * links rank * engagement rank) ^ 1/4
Breitling, R., Armengaud, P., Amtmann, A., & Herzyk, P. (2004). Rank products: a simple, yet powerful, new method to detect differentially
regulated genes in replicated microarray experiments. FEBS letters, 573(1-3), 83–92. https://doi.org/10.1016/j.febslet.2004.07.055

9. CRICOS No.00213J
Analysing the ‘prominent 1000’ … actually 954
• Collect all posts from these pages/groups over a six year period: 2016 – 2021*
• Text from posts (not in images) combined into a single text field
• Text collated into yearly quarters per page/group: 2016Q1_1, 2016Q2_1, … , 2021Q4_954
• Latent Dirichlet Allocation (n = 40) to generate topic model for this entire corpus
• Keyword analysis (top terms, bigrams, trigrams)
• Qualitative exploration of emergent themes
• Thematic prominence over time
• Dynamic Networks of thematic similarity between pages/groups
*481 pages, 473 groups = 954 total, ~70million posts

10. CRICOS No.00213J
Earth, wellness, and conspiratuality with
strong African and Indigenous themes
Italian right-wing populism Vietnamese politics
Natural food and recipes Romanian politics Arabic politics
Issues from a Catholic and Evangelical
perspective
Brazilian politics (leans pro-
Bolsonaro)
Arabic news
Entertainment, media and sport, with a focus
on Black and LGBTQI+ perspectives
Filipino politics (leans pro-Duterte)
General US politics
and news
Bernie Sanders
Turkish, Norwegian, and Swedish
politics (leans pro-Russia)
Hindi news
MAGA German politics Nigerian politics
Israel and Palestine Russian politics Italian news

11. CRICOS No.00213J

12. CRICOS No.00213J
Topic Graph
• Each page/group characterised by it’s
association with each topic category
• Semantic network map is constructed where
pages/groups are linked according to
strength of conceptual similarity (Angus &
Wiles, 2018)
Angus, D. & Wiles, J. (2018). Social semantic networks: Measuring topic
management in discourse using a pyramid of conceptual recurrence metrics,
Chaos, 28, https://doi.org/10.1063/1.5024809

13. CRICOS No.00213J

14. CRICOS No.00213J
Conclusions and Future Work
• Spectacle of significant world events reflected in upticks in online interest
• Collapse of boundaries between conspiratorial communities and political communities
• Ongoing and deeper qualitative analysis into specific pages/groups and clusters
• Linking practice analysis is ongoing

15. CRICOS No.00213J
https://research.qut.edu.au/dmrc/
@qutdmrc