The document discusses the challenges of determining what is true in an age where most of our knowledge comes from sources other than direct experience. It notes that political campaigns have long been dishonest, and examines how we evaluate the reliability of testimony from sources like journalists, experts, and social media. It emphasizes the importance of considering the independence of sources, evaluating each link in the "testimonial chain", and getting as close as possible to the original event or documentation when assessing the veracity of claims.

4. “the … perception has emerged that ours is a particularly dark time for
political ‘truth.’It isn’t. Throughout history, presidential campaigns have
been consistently dishonest.” Michael Moynihan, The Daily Beast

5. “The Zionist regime's establishment was
based on numerous deceptions and lies and
one of the biggest lies was the Holocaust.”
- Mahmoud Ahmadinejad

6. The vast majority of
facts we know about
the world, we know
through what other
people tell us.

7. For example:
• Abraham Lincoln was President
• Admedinejad actually said that
• Admedinejad is President of Iran
• The Nazis killed six million Jews

8. How do we
know whether
“testimony” is
true?

10. References
Journalists
Experts
Authoritative Sources

11. Event
Source
You
Opaque
testimonial
chain
Trust

12. “Several articles I wrote
or co-wrote were based
on this faulty
intelligence, and in May
2004, The Times
concluded in an editors'
note that its coverage
should have reflected
greater editorial and
reportorial skepticism.”
“It’s now common
knowledge that
[Snopes is] owned by
an ultra-liberal San
Francisco Bay area
couple, but until now
only a handful of
people knew they
were bankrolled by
George Soros, one of
Obama’s primary
financial supporters.”
Judith Miller
“‘ News
reporting’ from
well-established
news outlets is
generally
considered to be
reliable for
statements of
fact.”
‘test’

14. “Networks of knowledge on the Net have
no shape because the Net has no outer
edge. Besides, it doesn’t stay still long
enough.”
- David Weinberger, Too Big to Know

15. I disagree.
Knowledge has a
shape, and it is
extremely rich and
interesting.

16. Event Evidence
Witness
Witness
*
Analyst
Evidence
Memory
Documentation
Testimony
Perception

17. Attention
Distortion
Failure Modes of Perception
Interpretation

18. Forgetting
Failure Modes of Memory
Confusion
Confabulation
Encoding

19. Objectivity
Competence
Sincerity
Failure Modes of Testimony

20. Event
Witness
Witness
*
AnalystReporter
98%
90%
80% 95%
After analysis of the failure
modes, we could assign
confidence levels to each step.
All of the steps need to
transmit the truth, so we
multiply to get the overall
confidence in the chain.
.98 x .90 x .80 x .95 = .67

21. Event AnalystReporter
If we have multiple independent confirming
sources, we increase our confidence.
(1-.8) x (1-.2) x (1-.95) = .008 [99.2% confidence]
Reporter
Reporter
95%
20%
80%

22. Collaborative
Synoptic
Correlative
Failure Modes of Independence

24. Google Search
Showed coverage of this in:
Forbes
Esquire
Huffington Post
International Business Times
RawStory
MSN Now
and dozens of others

25. So it’s probably true:
Independent, trusted
sources. Right?
But there was one
problem.

26. All of these articles
were based on a single
blog post on MSNBC

27. And all it said about
credit cards is:
“Aides taking cabs home late that night got
rude awakenings when they found the
credit cards linked to the campaign no
longer worked. ‘Fiscally conservative,’
sighed one aide the next day.”

28. Event
Analyst
Reporter
*
Reporter
Reporter
Witness
Reporter
Reporter
Reporter
Synoptic Failure

29. Event
Analyst
Reporter
*
Reporter
Reporter
Witness
Reporter
Reporter
Reporter
Synoptic Failure

30. • Garrett Haake, reporter for MSNBC
• Embedded with Romney campaign
• Nominated for four Emmys
• Makes one biased comment in article
• MSNBC body of work: generally biased
• No named source attributed
• Suggests more than justified by known facts
• And I tweeted him for good measure
Evaluating the Witness / Reporter

31. Quality
Context
Genuineness
Failure Modes of Documentation

32. It has structure.
It mostly exists
online.
Could it be partly
automated?

34. “Take no one’s word for it.”

35. “Take no one’s word for it.”

36. Credulous Trusting Skeptical Paranoid
How much it matters

37. When evaluating claims:
• Find multiple, independent testimony chains
– Evaluate the independence
– Use Google, not just links
• Evaluate the quality of each chain, at each step
• Get as close as possible to the original event,
including looking for original documentation
What we should insist on:
• Writers state and link to all their sources!
• Digitize and post original documentation!
Join the Truth Brigade!

38. If you repeat
what you hear
from liars, fools,
and clowns,
what does that
make you?

39. • Seth Herd (CU Boulder)
• Blur: How to Know What’s True in the Age of
Information Overload (Kovach, Rosenstiel)
• Too Big To Know (Weinberger)
• Testimony and Epistemic Authority (Fricker)
• Pathologies of Testimony (Coady)
Acknowledgements and References