Talk given August 29, 2018 at the 1st Biannual Conference on Design of Experimental Search & Information Retrieval Systems (DESIRES 2018). Paper: https://www.ischool.utexas.edu/~ml/papers/lease-desires18.pdf
“Truthiness” is an Old Problem
“Truthiness is tearing apart our country... It used to be,
everyone was entitled to their own opinion, but not
their own facts. But that’s not the case anymore.”
– Stephen Colbert (Jan. 25, 2006)
“You furnish the pictures and I’ll furnish the war.”
– William Randolph Hearst (Jan. 25, 1898)
National Information Literacy Awareness Month,
US Presidential Proclamation, October 1, 2009.
“Though we may know how to find the information
we need, we must also know how to evaluate it.
Over the past decade, we have seen a crisis of
authenticity emerge. We now live in a world where
anyone can publish an opinion or perspective, whether
true or not, and have that opinion amplified…”
A Few Questions (1 of 2)
• What can IR bring to fact checking that NLP and ML do not,
and what new questions does fact checking raise for IR?
• Which results should we return, how should we present
them, what modes of interaction should we provide, and
how should we evaluate success?
• How do ranking & filtering decisions potentially impacting
user trust of the system and fears of being manipulated?
• How to diversify political (or other forms of) bias to provide
diverse perspectives, esp. on controversial topics? How
should we rank & evaluate results based on such diversity?
A Few Questions (2 of 2)
• How do we balance giving users search results
which match their existing beliefs vs. challenging
users with alternative perspectives?
• Just as people choose news outlets in part due to
political leanings, will new vertical search engines
rank and filter results to match audience views?
• How do we frame, measure, and address potential
harm of search results (e.g. “alternative” facts), be
they errors or intended diversity? How do we
evaluate information gain alongside potential
varying severity of harm to some number of users?