Slides from our presentation at the International Conference on Computational Social Science http://www.iccss2015.eu/

1. Political Fact Checking on
Twitter: When Do Corrections
Have an Effect?
Aniko Hannak, Drew Margolin, Ingmar Weber
6/12/2015 IC2S2

2. Proposition
• People are more likely to pay attention to facts when
group members tell them to than when non-group
members tell them to
• Factual challenges on Twitter:
• One Tweeter “snopes” another
• Do they follow each other?
Operationalization
6/12/2015 IC2S2

3. Fact-checking websites
• Snopes.com
• Politifact.com
• Factcheck.org
6/12/2015 IC2S2

4. Fact-checking websites
Advantages
• Clear true /
false judgment
• Same ultimate
sources for all
subjects
6/12/2015 IC2S2

5. Method
• Look for “snoping” tweets
1. Begin with @mention (reply)
2. Contain Fact-check url
Presumption: “Snoper” is replying to “Snopee” to trying to correct
them
6/12/2015 IC2S2

6. The Real Snopes of Social Media
😄
👮
6/12/2015 IC2S2
@Snopee
@Snopee
@Snoper

7. Method – Dataset #1
• 1,369 snoping events
• from “garden hose” sample of Twitter (April 2012-March 2013)1
• where both snoper and snopee history available1
• 303 where snope url is explicitly political (mentions Romney or Obama or is
categorized as such)
1See further details in ICWSM 2014 paper:
http://www.brianckeegan.com/papers/ICWSM_2014.pdf
6/12/2015 IC2S2

8. What are Snopers doing?
• Crowdflower coders code as “correction”
6/12/2015 IC2S2

9. Corrections by Snope Topic
6/12/2015 IC2S2
Not-Political
(sample)
Political
(all)
Non-Correction 83 (31%) 64 (21%)
Correction 182 (69%) 236 (78%)
Not codable 0 3 (1%)
Total 265 303
Chi-Sq. = 7.29, p < .01
Political snopes are more likely to be “corrections”

10. Independent Variables
6/12/2015 IC2S2

11. Incidence – Who Snopes Whom on Politics?
Relationship Political
Cases
% of Cases Correcting % Correcting
Stranger 152 50% 117 78%
Followee 14 5% 11 85%
Follower 49 16% 41 84%
Friend (mutual following) 88 29% 67 76%
Total 303 100% 236
Chi-Sq. = 1.38, p = .71
6/12/2015 IC2S2
Friends/strangers are equivalent in propensity to use snopes to correct

12. Are Political Corrections Acknowledged?
“Snopee”
Ignores
“Snopee”
Responds
Snoped by
Stranger
Snoped by
Friend
N = 97
N = 57
95% 5%
86% 14%
p =.05
When snope is a
correction about politics,
people are 2.8 times more
likely to respond when it
comes from a friend
6/12/2015 IC2S2

13. Replies to Political vs. Non-Political Corrections
• When snope is a correction not about politics
• People are 3-5 times more likely to respond
• But difference between stranger vs. friend is less
• only 1.5 times more likely to respond to snope from a friend vs. a stranger
• No difference in response rates for non-correcting snopes between
friends and strangers (political or non-political)
6/12/2015 IC2S2

14. Method – Dataset #2
• 4,733 snoping events
• Explicitly “political” snopes according to the categorization of the rumor
• from “fire hose”
• 2,037 where both snoper and snopee history and friend / follower
information available
6/12/2015 IC2S2

15. Consistent w/prior results
“Snopee”
Ignores
“Snopee”
Responds
Snoped by
Stranger
Snoped by
Friend
N = 1532
N = 505
Tweeters are significantly more likely to reply (at any time) to snoping tweet
when it comes from a friend (someone they follow)
78% 22%
55% 45%
p < .0016/12/2015
IC2S2

16. What Type of Replies?
Author (Margolin) and research assistant
hand coded 445 snopes where the snopee
replied
• 351 (79%) were “correcting snopes”, of
these…
• 173 (49%) the snopee “accepted
the snope fact as true”
6/12/2015
IC2S2
👮
😄
😄
@Snopee
@Snopee
@Snopee
@Snoper
@Snoper

17. Among the replies?
“Snopee” Does
Not Accept
“Snopee”
Accepts
Snoped by
Stranger
Snoped by
Friend
N = 219
N = 132
Friends more likely to “accept” snope fact than strangers
(remains significant when controlling for friends/followers in common)
58% 42%
39% 61%
p < .0016/12/2015
IC2S2

18. Other kinds of reactions?
Coded for “defends original position”
• 94 (27%) of “correcting snopes”
Early results suggest snopees are more
likely to defend in this way when:
1. Snoper is stranger (p < .05)
2. Snoper is unpopular (has fewer
followers) (p < .05)[“double-down
against weaker opponents”?]
Number of snopee followers did not
matter
6/12/2015
IC2S2
👮
😄
😄
@Snopee
@Snopee
@Snoper
@Snopee
@Snoper

19. Other kinds of reactions?
Coded for “defends original position”
• 94 (27%) of “correcting snopes”
Coded for “defends original position with
new basis”
• 67 (19%) of “correcting snopes”
6/12/2015
IC2S2
👮
😄
😄
Acknowledgement of
snope fact truth
Asserts new basis
@Snopee
@Snopee
@Snoper
@Snopee
@Snoper

20. Other kinds of reactions?
Coded for “defends original position”
• 94 (27%) of “correcting snopes”
Coded for “defends original position with
new basis”
• 67 (19%) of “correcting snopes”
Early results suggest snopees are more
likely to defend in this way when:
1. Snoper is popular (more followers) (p
< .01) [“shift against authority”?]
Friend/following between snoper-snopee
did not matter
6/12/2015
IC2S2
👮
😄
😄
@Snopee
@Snopee
@Snoper
@Snopee
@Snoper

21. Ongoing …
• Political leaning
• What happens before
• Change of relationship due to snoping
• Closeness of relationship
6/12/2015 IC2S2