Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Designing Human-AI Partnerships to Combat Misinfomation

61 views

Published on

Talk presented September 6, 2019 at the University of Texas at Austin, USA

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Designing Human-AI Partnerships to Combat Misinfomation

  1. 1. Designing Human-AI Partnerships to Combat Misinfomation Matt Lease School of Information University of Texas at Austin ml@utexas.edu @mattlease Slides: slideshare.net/mattlease
  2. 2. • Background: UT iSchool and Good Systems • Prototype: A Human + AI Partnership for Misinformation • User Bias: Detecting & Mitigating User Bias • Wrap-up Discussion 2 Roadmap Matt Lease (University of Texas at Austin)
  3. 3. “The place where people & technology meet” ~ Wobbrock et al., 2009 “iSchools” now exist at over 100 universities around the world What’s an Information School? 3
  4. 4. Matt Lease (University of Texas at Austin) 4 Good Systems: 8-Year UT “Moonshot” Project Goal: design a future of Artificial Intelligence (AI) technologies to meet society’s needs and values. http://goodsystems.utexas.edu
  5. 5. Misinformation: a long history “Truthiness is tearing apart our country... It used to be, everyone was entitled to their own opinion, but not their own facts. But that’s not the case anymore.” – Stephen Colbert (Jan. 25, 2006) “You furnish the pictures and I’ll furnish the war.” – William Randolph Hearst (Jan. 25, 1898) 5
  6. 6. Matt Lease (University of Texas at Austin) Fact Checking by Journalists 6
  7. 7. Automatic Fact Checking (by AI) 7Matt Lease (University of Texas at Austin)
  8. 8. Matt Lease (University of Texas at Austin) Design Challenges for AI Fact Checking Fair, Accountable, & Transparent (“FAT”) AI 8
  9. 9. Matt Lease (University of Texas at Austin) Design Challenges for AI Fact Checking Fair, Accountable, & Transparent (“FAT”) AI • Is “fact” vs. “fake” all people want to know? 9
  10. 10. Matt Lease (University of Texas at Austin) Design Challenges for AI Fact Checking Fair, Accountable, & Transparent (“FAT”) AI • Is “fact” vs. “fake” all people want to know? • Why trust a “black box” classifier? 10
  11. 11. Matt Lease (University of Texas at Austin) Design Challenges for AI Fact Checking Fair, Accountable, & Transparent (“FAT”) AI • Is “fact” vs. “fake” all people want to know? • Why trust a “black box” classifier? • How to detect & mitigate bias (user or AI)? 11
  12. 12. Matt Lease (University of Texas at Austin) Design Challenges for AI Fact Checking Fair, Accountable, & Transparent (“FAT”) AI • Is “fact” vs. “fake” all people want to know? • Why trust a “black box” classifier? • How to detect & mitigate bias (user or AI)? • How can we combine human knowledge & experience with AI speed & scalability? – Goals: human insights, reduced AI error, & personalization 12
  13. 13. Matt Lease (University of Texas at Austin) Design Challenge: Human Interaction with AI? 13
  14. 14. Believe it or not: Designing a Human-AI Partnership for Mixed-Initiative Fact-Checking Joint work with An Thanh Nguyen (UT), Byron Wallace (Northeastern), & more!
  15. 15. Matt Lease (UT Austin) • Believe it or not: Designing a Human-AI Partnership for Mixed-Initiative Fact-Checking 15 Demo! Nguyen et al., UIST’18
  16. 16. Matt Lease (University of Texas at Austin) Empowering Users: Vision 16
  17. 17. Matt Lease (University of Texas at Austin) Empowering Users: Challenge 17
  18. 18. Matt Lease (University of Texas at Austin) Empowering Users: Challenge 18
  19. 19. Matt Lease (University of Texas at Austin) A new form of echo chamber? User control promotes transparency & trust, but what about bias? 19
  20. 20. 20 Anubrata Das, Kunjan Mehta and Matthew Lease SIGIR 2019 Workshop on Fair, Accountable, Confidential, Transparent, and Safe Information Retrieval (FACTS-IR). July 25, 2019 CobWeb: A Research Prototype for Exploring User Bias in Political Fact-Checking
  21. 21. Matt Lease (University of Texas at Austin) 21 We introduce “political leaning” (bias) as a function of the adjusted reputation of the news sources
  22. 22. 22 We introduce “political leaning” (bias) as a function of the adjusted reputation of the news sources CobWeb; Anubrata Das, Kunjan Mehta and Matthew Lease Stance Sources
  23. 23. 23 We introduce “political leaning” (bias) as a function of the adjusted reputation of the news sources CobWeb; Anubrata Das, Kunjan Mehta and Matthew Lease Imaginary Source Bias Stance Sources
  24. 24. 24 We introduce “political leaning” (bias) as a function of the adjusted reputation of the news sources CobWeb; Anubrata Das, Kunjan Mehta and Matthew Lease A user can alter the source reputation Imaginary Source Bias Stance Sources
  25. 25. 25 We introduce “political leaning” (bias) as a function of the adjusted reputation of the news sources CobWeb; Anubrata Das, Kunjan Mehta and Matthew Lease A user can alter the source reputation Changing reputation scores changes the predicted correctness Imaginary Source Bias Stance Sources
  26. 26. 26 We introduce “political leaning” (bias) as a function of the adjusted reputation of the news sources CobWeb; Anubrata Das, Kunjan Mehta and Matthew Lease A user can alter the source reputation Changing reputation scores changes the predicted correctness Changing reputation scores changes the overall political leaning Imaginary Source Bias Stance Sources
  27. 27. 27 We introduce “political leaning” (bias) as a function of the adjusted reputation of the news sources CobWeb; Anubrata Das, Kunjan Mehta and Matthew Lease Imaginary Source Bias Stance Sources
  28. 28. 28 We introduce “political leaning” (bias) as a function of the adjusted reputation of the news sources CobWeb; Anubrata Das, Kunjan Mehta and Matthew Lease A user can alter the overall political leaning Imaginary Source Bias Stance Sources
  29. 29. 29 We introduce “political leaning” (bias) as a function of the adjusted reputation of the news sources CobWeb; Anubrata Das, Kunjan Mehta and Matthew Lease A user can alter the overall political leaning Imaginary Source Bias Stance Sources Changing the overall political leaning changes the source reputations
  30. 30. 30 We introduce “political leaning” (bias) as a function of the adjusted reputation of the news sources CobWeb; Anubrata Das, Kunjan Mehta and Matthew Lease Changing the overall political leaning changes the predicted correctness A user can alter the overall political leaning Imaginary Source Bias Stance Sources Changing the overall political leaning changes the source reputations
  31. 31. Matt Lease (University of Texas at Austin) Wrap-up Discussion • Black-box AI is not enough! Want interaction, exploration, & trust. – Useful problem to ground work on Fair, Accountable, & Transparent (FAT) AI • Design challenge: human + AI partnership • Search engines need more than fact-checking – How to diversify search results for controversial topics? – How to assess potential harm? (eg, vaccination & autism) • AI: new risks of harm as well as potential for good – Potential added confusion, data / algorithmic bias – Potential for personal “echo chamber” – Adversarial applications 31
  32. 32. Matt Lease (University of Texas at Austin) – ml@utexas.edu Thank You! 32 Lab: ir.ischool.utexas.edu @mattlease Slides: slideshare.net/mattlease

×