Co-Spread of Misinformation
and Fact-CheckingContent
during the Covid-19 Pandemic
Grégoire Burel, Tracie Farrell, Martino Mensio,
Prashant Khare, and Harith Alani
SocIinfo 2020, 6-9 October 2020
• During the Covid-19 pandemic the amount of misinformation shared on
social media has risen significantly with some tragic results.
• Fact-checking organisations have been correcting misinforming content.
However, the effectiveness of corrective information remains largely
unknown.
Misinformationand Fact-Checkingspread duringCovid-19
Amount of new fact-checks per day. (Source: Poynter)
Peak: 143 per day.
“Rumours about
coronavirus – that it
can be prevented by
drinking alcohol; that it
is killed by cold or
heat – pose serious
risk.”
The United Nations
17 Apr 2020
Did fact-checking during the Covid-19 pandemic have a positive impact in reducing misinformation spread?
2
Co-Spread of Misinformationand Fact-
Checking
• Research Questions:
1. Are misinformation and fact-checking
information spread similarly?
2. How do misinformation and fact-checking
spread patterns vary in relation to each other,
and to the pandemic level?
3. Does fact-checking spread affect the diffusion
of misinformation about Covid-19?
Misinformation is false or inaccurate
information. Examples of misinformation
include false rumours or more deliberate
disinformation such as malicious content
such as hoaxes and computational
propaganda.
Fact-checking is the process of verifying
information in non-fictional text in order to
determine its veracity and correctness.
3
Data collection
• We collect tweets mentioning misinforming and fact-
checking URLs.
• Fact-checks and misinforming URLs are obtained from the
misinfo.me tool that collect websites belonging to the
International Fact-Checking Network (IFCN).
• ClaimReview is a tagging system that fact-checkers use
to identify their articles (i.e., claims, ratings fact-checked
content and fact-checking URL).
twitter
misinfo.me
Stacked cumulative spread of misinforming and corrective information.
2,830 Misinforming URLs
734 Fact-checking URLs
21,394 Tweets
Data collected until 4th May 2020
4
Methodology
1. Misinformation and fact-checks spread analysis over
time:
• Relative and pandemic-level analysis.
• Non-parametric MANOVA/ANOVA (Analysis of Variance)
→ Identifies if there is a significant variance in means
between the misinformation and fact-check spreads.
• Identifies if there are significant differences in how
misinformation and fact-checking information
spreads in different time periods.
2. Relational analysis between misinformation and fact-
checking spread:
• Relative analysis.
• Causation analysis (Granger causality) model) for
estimating if the spread of a given information type can
be used to predict the spread of another type of
information.
• Impulse response analysis and Forecast Error
Variance Decomposition (FEVD) to evaluate the
spread response of a given information type
depending on change in misinformation or fact-check
spread.
Relative analysis: Data is aligned based on
their initial sharing date and then divided in
initial, early and late periods (using linear
regressions and inflection points).
Pandemic-level analysis: Divide pandemic
in the same three different time periods
using worldwide cases.
AnalysisLevels
0 – 2 days 2 – 14 days 14+ days
initial early late
< 14th March 14/3 – 2/4 > 2nd April
initial early late
5
• Results:
• Pandemic-level analysis:
• Misinformation and Fact-checks spread
differently globally (MANOVA, p = 0.01)
• At individual periods, significant
differences are observed for the initial
and late periods.
• Misinformation is shared much more than
fact-checks.
Stacked cumulative spread of misinforming and corrective information.
Multivariate Analysis
Initial onset period
until 14th March .
• Relative analysis:
• There are significant differences in how
misinforming URLs and fact-checking
URLs spread globally (MANOVA, P <
0.001).
• Significant differences are only
observed in the early and late periods.
• The highest difference in terms of
mean and standard deviation between
the different URL types appears to be
mostly during the initial phase.
Late period from
2nd April
Ramp-up period from
14th March until 2nd
April .
6
Fact-checking
delayed spread
response
Downward
misinformation spread
trend
Self initial response
(spread drop soon
after initial increase)
• Results:
• Misinformation spread can be predicted from
fact-checking spread (Granger causality, p = 0.02
vs. p = 0.93).
• Fact-checking impulse generates a slow
downward trend in misinformation spread.
• Misinformation impulse generates a delayed
fact-checking spread response.
• FEVD analysis confirms that misinformation
spread is influenced by fact-checking.
Causalityand Impulse Analysis
Misinformation
spread is affected by
Fact-checks.
Fact-checks spread
marginally affected
by misinformation.. 7
Conclusions and FutureWork
8 8
• Limitations and future work:
• Limited but accurate data sample (restricted by
Fact-checking data) → Automatic identification of fact-
checked claims.
• Granular features necessary for better
understanding of spread variance (e.g.,
demographics and topics) → Topic analysis and
demographics extraction.
• Conclusions:
• Fact-checking spread has a positive impact in
reducing misinformation.
• However, the impact of fact-checking is seriously
impeded by: 1) the amount of shared
misinformation, and; 2) the short period of time in
which fact-checks are likely to spread.
• Creating fact-checking content that is more
spreadable may be the key to reduce misinformation
spread.
- Improving Covid-19 fact-
check database (7,100+
fact-checks) and data
collection (226,000+
tweets).
- Topic and demographics
analysis.
- Automatic tracking and
reporting (fc-observatory).
FC-Observatory
prototype.
CurrentWork
Thank you.
Grégoire Burel.
@evhart
github.com/evhart
g.burel@open.ac.uk
Co-Spread of Misinformation and Fact-Checking Content during the Covid-19 Pandemic
Background illustrations: Myth busters created by Redgirl Lee for United Nations Global Call Out To Creatives.

Co-Spread of Misinformation and Fact-Checking Content during the Covid-19 Pandemic

  • 1.
    Co-Spread of Misinformation andFact-CheckingContent during the Covid-19 Pandemic Grégoire Burel, Tracie Farrell, Martino Mensio, Prashant Khare, and Harith Alani SocIinfo 2020, 6-9 October 2020
  • 2.
    • During theCovid-19 pandemic the amount of misinformation shared on social media has risen significantly with some tragic results. • Fact-checking organisations have been correcting misinforming content. However, the effectiveness of corrective information remains largely unknown. Misinformationand Fact-Checkingspread duringCovid-19 Amount of new fact-checks per day. (Source: Poynter) Peak: 143 per day. “Rumours about coronavirus – that it can be prevented by drinking alcohol; that it is killed by cold or heat – pose serious risk.” The United Nations 17 Apr 2020 Did fact-checking during the Covid-19 pandemic have a positive impact in reducing misinformation spread? 2
  • 3.
    Co-Spread of MisinformationandFact- Checking • Research Questions: 1. Are misinformation and fact-checking information spread similarly? 2. How do misinformation and fact-checking spread patterns vary in relation to each other, and to the pandemic level? 3. Does fact-checking spread affect the diffusion of misinformation about Covid-19? Misinformation is false or inaccurate information. Examples of misinformation include false rumours or more deliberate disinformation such as malicious content such as hoaxes and computational propaganda. Fact-checking is the process of verifying information in non-fictional text in order to determine its veracity and correctness. 3
  • 4.
    Data collection • Wecollect tweets mentioning misinforming and fact- checking URLs. • Fact-checks and misinforming URLs are obtained from the misinfo.me tool that collect websites belonging to the International Fact-Checking Network (IFCN). • ClaimReview is a tagging system that fact-checkers use to identify their articles (i.e., claims, ratings fact-checked content and fact-checking URL). twitter misinfo.me Stacked cumulative spread of misinforming and corrective information. 2,830 Misinforming URLs 734 Fact-checking URLs 21,394 Tweets Data collected until 4th May 2020 4
  • 5.
    Methodology 1. Misinformation andfact-checks spread analysis over time: • Relative and pandemic-level analysis. • Non-parametric MANOVA/ANOVA (Analysis of Variance) → Identifies if there is a significant variance in means between the misinformation and fact-check spreads. • Identifies if there are significant differences in how misinformation and fact-checking information spreads in different time periods. 2. Relational analysis between misinformation and fact- checking spread: • Relative analysis. • Causation analysis (Granger causality) model) for estimating if the spread of a given information type can be used to predict the spread of another type of information. • Impulse response analysis and Forecast Error Variance Decomposition (FEVD) to evaluate the spread response of a given information type depending on change in misinformation or fact-check spread. Relative analysis: Data is aligned based on their initial sharing date and then divided in initial, early and late periods (using linear regressions and inflection points). Pandemic-level analysis: Divide pandemic in the same three different time periods using worldwide cases. AnalysisLevels 0 – 2 days 2 – 14 days 14+ days initial early late < 14th March 14/3 – 2/4 > 2nd April initial early late 5
  • 6.
    • Results: • Pandemic-levelanalysis: • Misinformation and Fact-checks spread differently globally (MANOVA, p = 0.01) • At individual periods, significant differences are observed for the initial and late periods. • Misinformation is shared much more than fact-checks. Stacked cumulative spread of misinforming and corrective information. Multivariate Analysis Initial onset period until 14th March . • Relative analysis: • There are significant differences in how misinforming URLs and fact-checking URLs spread globally (MANOVA, P < 0.001). • Significant differences are only observed in the early and late periods. • The highest difference in terms of mean and standard deviation between the different URL types appears to be mostly during the initial phase. Late period from 2nd April Ramp-up period from 14th March until 2nd April . 6
  • 7.
    Fact-checking delayed spread response Downward misinformation spread trend Selfinitial response (spread drop soon after initial increase) • Results: • Misinformation spread can be predicted from fact-checking spread (Granger causality, p = 0.02 vs. p = 0.93). • Fact-checking impulse generates a slow downward trend in misinformation spread. • Misinformation impulse generates a delayed fact-checking spread response. • FEVD analysis confirms that misinformation spread is influenced by fact-checking. Causalityand Impulse Analysis Misinformation spread is affected by Fact-checks. Fact-checks spread marginally affected by misinformation.. 7
  • 8.
    Conclusions and FutureWork 88 • Limitations and future work: • Limited but accurate data sample (restricted by Fact-checking data) → Automatic identification of fact- checked claims. • Granular features necessary for better understanding of spread variance (e.g., demographics and topics) → Topic analysis and demographics extraction. • Conclusions: • Fact-checking spread has a positive impact in reducing misinformation. • However, the impact of fact-checking is seriously impeded by: 1) the amount of shared misinformation, and; 2) the short period of time in which fact-checks are likely to spread. • Creating fact-checking content that is more spreadable may be the key to reduce misinformation spread. - Improving Covid-19 fact- check database (7,100+ fact-checks) and data collection (226,000+ tweets). - Topic and demographics analysis. - Automatic tracking and reporting (fc-observatory). FC-Observatory prototype. CurrentWork
  • 9.
    Thank you. Grégoire Burel. @evhart github.com/evhart g.burel@open.ac.uk Co-Spreadof Misinformation and Fact-Checking Content during the Covid-19 Pandemic Background illustrations: Myth busters created by Redgirl Lee for United Nations Global Call Out To Creatives.