Dr Scott A Hale presented these slides at Truth and Trust Online in October 2019. https://truthandtrustonline.com/

1. No master algorithm
Human-machine intelligence and the real-world
needs of fact-checking and content moderation
Dr Scott A. Hale, @computermacgyve

2. Toward Automated Fact-Checking:
...Detecting Check-worthy Factual Claims…
...Developing an Annotation Schema...for Claim Detection

3. Academic vs. practitioner goals
Academic:
Build a system to identify claims needing to be fact-checked / moderated
Build a system to do fact-checking / moderation

4. Academic vs. practitioner goals
Academic:
Build a system to identify claims needing to be fact-checked / moderated
Build a system to do fact-checking / moderation
Practitioner:
Wants to select claims to fact-check based on organisational context, own
knowledge/expertise, etc.
Needs assistance prioritising claims (based on reach, impact, domain, type)
Needs assistance managing pool of experts, workload, etc.

5. We need to work together
Problem motivation
Data collection, analysis
Method development
Extracting and clustering claims
Matching similar images

6. Check Message

7. Building a data trust to ensure data is open for research
Partnerships
Harvard CyberLaw Clinic -Kendra Albert
Wolfe and Hwang -Tim Hwang
Digital Public -Sean McDonald
Conduct in-depth research focused on
usability, scalability, and security
Establish best practices for a new type of
data governance in the era of closed
networks.
Image source: Open Data Institute

8. Claim extraction and clustering
Konstantinovskiy et al. (2018) arXiv:1809.08193

9. Image matching
PDQ
Perceptual hashing
Neural network
representations
pHash distance 6

10. pHash distance 4

11. pHash distance 4

12. Dialogue
Credibility Coalition (CredCo): https://credibilitycoalition.org/
Working groups on UX, InfoSec, Images & Memes, Health, Literacy,....
Content Moderation Group (Como)

13. Content Moderation Database

16. Original research and research partnerships
Image and claim matching
UX
Health fact-checking
Importance of local knowledge in content moderation

17. Thank You
Dr Scott Hale
@computermacgyve