Class Presentation

1. How Does a “Personal
Informatics Tool” Impacts
Reader Responses to Online
News?
Prerana Khatiwada, Haritha Varkala,
Dileep Reddy Nimma, Yongho Cho
As a requirement for a class project
Human-centered Computing (HCC)
April 15, 2023

2. Problem Statement

3. Problem
Statement
False information spreads quickly
through online platforms and can
mislead people, causing serious
consequences for individuals and
society.
Online news consumption is
increasingly becoming a
personalized and individualized
experience, with users having
access to a wide variety of sources
and platforms.

4. Introduction

5. Introduction
• In the digital age, fake news is a widespread problem.
• News consumption is becoming increasingly personalized and fragmented due to the
abundance of sources and platforms available.
• The rise of fake news and misinformation has led to a growing need for effective
interventions to help users identify and avoid false information.
• Technological interventions may be effective in helping individuals distinguish between real and fake
news, but their impact on readers' perception and understanding of news content remains unclear.
• The study aims to investigate the impact of a personal informatics tool (PI) on reader responses to online
news by examining its effect on people's reading and attention when reading news articles on a
desktop.

6. Research Questions

7. Research
Questions
•Does reading habits have an impact
on the consumption of misinformation?
RQ1
•What is the impact of personalized
visualizations and informatics
dashboards on readers' perception
and understanding of news content?
RQ2

8. Hypothesis
"Participants who use the personal
informatics tool while reading news
articles will show a higher level of
engagement with the articles, a
greater ability to identify false and
misleading information, and more
positive attitudes and perceptions
toward the articles than those who
read news articles without the tool."

9. Purpose

10. Purpose
• Gain insights into how technological
interventions can help individuals make more
informed decisions about the news they
consume.
• Understand how the tool affects readers'
attention and engagement with news articles
and whether it can lead to more critical
thinking and better understanding of news
content.

11. Method

12. Methodology
In Lab study
Online survey
Analysis of metrics

13. User Study

14. User Study
• Participants will be asked to complete a digital consent form and pre-study survey.
• Assigned a set of articles to read, followed by a short quiz and post-study survey on
the high-level content of the article.
• The study will take place in person in a controlled laboratory setting with the
assistance of trained research staff.
• The entire study is expected to take approximately 60 minutes, including time for
instructions and feedback.

15. Study
Approach and
Protocol
Fill out All participants must fill out a digital consent form and
complete the pre-survey.
Read The control group will read news articles followed by a quiz
without the PI tool.
Read The intervention group will read articles with the PI tool to
enhance their reading experience.
Assess In-situ answers will be compared to assess the impact of the
tool on participants' overall opinions.
Installat
ion
The PI tool will be installed on the lab computers instead of
having participants install it on their own.
Allow This change will allow us to ensure consistency in the
installation and usage of the tool.

16. Methodology | Post Study Survey
A post-study survey will be
conducted to assess
participants' attitudes and
perceptions toward the
articles they read.
Questions will be added to
evaluate the effectiveness of
specific features of the tool.
Survey questions will ask
participants how likely they
are to recommend the tool
to a friend and rate their
experience reading the news
article with the intervention
tool.

17. Sample Survey
Recruitment form
Survey for the study

18. Compensation
• If Participants complete all study components (surveys and reading tasks), they
will be compensated $10.
• They will be paid via a digital Amazon Gift Card sent to the email address at the
end of the testing session unless they are an employee of the University of
Delaware.
• If employed by the University of Delaware, you will receive direct payment via the
payroll system.

19. Prototyping

20. Images for
PI Tool

21. Images for PI Tool
SOURCE : Shows its source based on web domain
BIAS : Political bias of news articles will be determined by whether the publisher
exists in the AllSides Dataset (https://www.allsides.com/media-bias).

22. Images for PI Tool
Reading Articles : Total number of articles you have read
Leaning : Shows you how much you lean left or right based on the bias statistics
of the articles you read
Chart : Real-time statistics on how many articles were read categorized by bias
Red : Left bias
Orange : Left-center bias
Blue : Right bias
Cyan : Right-center bias

23. Images for PI Tool
Avg Reading Time(s) : Displays the average amount of time it took you to read
each article.

24. PI Tool development currently
in progress
Scroll Time
Click Through Rate
Pop-up alert on fake news
detection

25. Insight of Intervention
User Impact
The change in users' perception can impact their judgment of whether an
article is accurate or not and its overall reliability.
Can the system's intervention by providing real-time relevant information
change the user's perception of the article?

26. Alternative Approaches

27. Data Analysis
1
Compute mean scores
for each survey
question for both
groups and compare to
identify significant
differences.
2
Conduct t-tests to
compare mean scores
between control and
intervention groups for
each survey question
and determine
statistical significance of
differences.
3
ANOVA: Conduct an
ANOVA (analysis of
variance) to compare
the mean scores of
multiple groups (e.g.,
different types of PI
tools) and identify any
significant differences.
4
Regression analysis to
assess the impact of
independent variables,
such as personalized
visualizations and bias
charts, on dependent
variables, such as
comprehension and
engagement.
5
Evaluate participants'
sessions by comparing
elements such as time
taken to read each
article, number of
articles read, click-
through rates, scrolling
speed, and accuracy
of responses to assess
the effectiveness of the
dashboard.

28. Expected
Outcomes

29. Expected Outcomes
• The average time spent reading news articles in the control group was XYZ minutes, while
it was ABC minutes in the intervention group.
• Participants in the intervention group reported feeling less/more overwhelmed by the
amount of news they read (ABC%), whereas the control group expressed
frustration/satisfaction with the amount of news they had to read.
• The percentage of participants in the intervention group who reported feeling more
confident in their ability to identify biased or false news articles was XYZ%, compared to
ABC% in the control group.
• The percentage of participants who reported feeling more informed about the news
after using the PI tool was ABC% in the intervention group, while it was XYZ% in the control
group.
• The intervention group reported that the PI tool helped them/did not help them identify
and avoid biased or false news articles, while the control group found it challenging/easy
to identify biased or false news articles.

30. Timeline/Gantt Chart

31. Blockers and Challenges
• Recruitment of Participants: It can be challenging to recruit participants who meet the
study's specific inclusion and exclusion criteria, especially if the sample size is small.
• Technology Limitations: The PI tool was integrated with a Chrome-based plugin, which may
limit its accessibility to users not using the Chrome browser and mobile devices.
• Time and Resources: The study requires significant time and resources, including participant
recruitment, data collection, and analysis.

32. Future
Work
Expand sample size to increase
generalizability
Conduct follow-up studies like
semi structured and need finding
interviews to assess the
effectiveness of interventions
Examine how reading habits and
media consumption differ among
different demographic groups

33. References
Axelsson, C.A.W.; Guath, M.; Nygren, T. Learning How to Separate Fake From Real News: Scalable
Digital Tutorials Promoting Students’ Civic Online Reasoning.
Nygren, T., M. Guath, C.-A. W. Axelsson, and D. Frau-Meigs. 2021. “Combatting Visual
Fake News with a Professional Fact-Checking Tool in Education in France, Romania,
Spain and Sweden.” Information 12 (5): 201–225.
Ku, K.Y.; Kong, Q.; Song, Y.; Deng, L.; Kang, Y.; Hu, A. What predicts adolescents’
critical thinking about real-life news? The roles of social media news consumption and
news media literacy. Think. Ski. Creat. 2019, 33, 100570
Jason VanTol, Maxim Nikitin, Noah Helzerman, Mark Allison, and Matthew Spradling.
2022. OpenLabel: An Open-Source Media Labeling Web Browser Extension

34. Thank you!
QA