4. Agenda Setting Theory (McCombs & Shaw, 1972)
• Media affects audiences by having an influence on
• What to think about
• How to think about it
• Examples of traditional media studies
• Media affects the outcome of presidential elections (Perloff and Krauss,
1985)
• Media coverage influences the control of infectious diseases (Cui et al.,
2008)
• Tone of news articles affects the number of visitors to museums
(Zyglidopoulos et al., 2012)
5. 1.Use of traditional off-line newspapers and TV as target media
• Analysis is limited to a small volume over a short duration
• Issues are arbitrarily chosen
2.Use of off-line MIP (Most Important Problems) surveys
• Self-reports are not reliable
• Only a small subset of the population can be surveyed
3.Use of manual coding for content analysis
• You need experts
• It is difficult to replicate and generalize to other domains
Limitation of Traditional Media Studies
6. Computational Analysis of Agenda Setting Theory
1.Use of traditional off-line newspapers and TV as target media
• Crawl online news to get several years’ data
• Use machine learning to automatically discover the important issues
2.Use of off-line MIP (Most Important Problems) surveys
• Look at counts of social media shares
• Look at counts of user comments
3.Use of manual coding for content analysis
• Use unsupervised machine learning to analyze content for tone (polarity) of
articles and comments
• Try it for different issues to see whether ML approach can generalize over
many domains
9. 8
Section #Articles #Comments #Commenters #Shares
Politics 1,863 174,680 14,106 2,080,889
Business 2,043 130,921 17,791 3,657,544
Opinion 4,820 149,618 30,556 6,620,489
Sports 814 17,282 5,484 712,507
Technology 456 13,571 4,993 570,732
Science 945 50,113 11,114 4,709,041
World 3,673 134,572 14,882 3,534,637
Health 3,060 92,964 18,185 6,001,082
Total 17,674 763,721 117,111 27,886,921
From http://www.npr.org/
2011.01 – 2013.04
DATA STATISTICS
10. 9
Section Issue (Labeled by using Mturk) #Articles
Politics presidential election
infringement of human rights
race for Washington
government economics
presidential campaigns and money
candidate-marriage & immigration
political viewpoints
575
195
167
274
163
261
157
Business economic decline under Obama
employment and paid slavery
agriculture
banks and loan
stock market and business
housing market
tax and business
energy and finance
new business and running
514
218
131
198
166
170
180
222
138
Health health care reform laws
vaccination
HIV and treatment
medication
healthcare and costs
food and obesity
sleep study and children
food and safety
health tech and new treatment
mental health in families
349
189
496
197
224
245
210
223
125
117
Issue Detection using HDP
Detected Issue list and the number of articles of each issue for three sections out of eight
sections.
11. Correlation between the volume of articles (per week) and audience’s interests (following comments). Correlation value for (a) is 0.786
and it shows strong agenda setting effects., also correlation value for (b) is 0.418 and it shows weak agenda setting effects.
10
▶ Effects from media exposure
CORRELATION IN ISSUE
Jan 01 2011
12. 11
Section Keywords Issue (Labeled by using Mturk) commentcomment shareshareSection Keywords Issue (Labeled by using Mturk)
corr. effect size corr. effect size
Politics romney gingrich republican santorum
president obama house people political
state republican election party walker
president obama tax house congress
romney campaign obama money million
court law state ice supreme marriage
romney obama president republican voters
presidential election
infringement of human rights
race for Washington
government economics
presidential campaigns and money
candidate-marriage & immigration
political viewpoints
0.873****
0.845****
0.855****
0.903****
0.836****
0.895****
0.878****
Large
Large
Large
Large
Large
Large
Large
0.161
0.562****
0.511****
0.347***
0.367**
0.417**
0.372*
none
Large
Large
Medium
Medium
Medium
Medium
Business percent economy year jobs debt rate tax
people job can work time jobs year years
food farmers year beer corn prices new
bank banks financial money new company
news new company people can stock now
people city new can home like now housing
tax can people state new like year get
new oil gas company american car industry
like can people new company get year
economic decline under Obama
employment and paid slavery
agriculture
banks and loan
stock market and business
housing market
tax and business
energy and finance
new business and running
0.870****
0.732****
0.634****
0.786****
0.736****
0.670****
0.767****
0.702****
0.750****
Large
Large
Large
Large
Large
Large
Large
Large
Large
0.304***
0.346***
0.230*
0.268**
0.441**
0.360*
0.278**
0.423**
0.278**
Medium
Medium
Small
Small
Medium
Medium
Small
Medium
Small
Health health law care insurance people federal
health people vaccine virus new flu
cancer women people percent risk hiv
drug drugs fda people can new patients
care health patients hospital hospitals
food people can health new like weight
can people study sleep kids children
food can people health like may new get
can patients people cancer brain new
people can like life health get know
health care reform laws
vaccination
HIV and treatment
medication
healthcare and costs
food and obesity
sleep study and children
food and safety
health tech and new treatment
mental health in families
0.564****
0.640****
0.399*
0.447**
0.706****
0.702****
0.541****
0.428**
0.544****
0.418**
Large
Large
Medium
Medium
Large
Large
Large
Medium
Large
Mediu,m
0.241**
0.341***
0.279**
0.149
0.615****
0.162*
0.456**
0.330***
0.172
0.360*
Small
Medium
Small
none
Large
Small
Medium
Medium
none
Medium
CORRELATION & EFFECT SIZE
13. Q: Why do we see larger agenda setting effects for some issues?
H: Previous studies argue for a relationship between agenda setting and
relevance and/or uncertainty.
Previous research in media studies argue that larger agenda setting affects can
be seen for issues with
• Low relevance and High uncertainty
(Schonbach & Weaver, 1985)
• High relevance and High uncertainty (McCombs, 2004)
12
RELEVANCE & UNCERTAINTY
Relevance:The relevance of an issue to the audience
Uncertainty:The degree of uncertainty by the audience about the issue
14. GOAL
Measure the average relevance and uncertainty of each issue to analyze the
correlation to the level of agenda setting effect
EXPERIMENT SETTINGS
2 sets of randomly distributed issues
(each set contains 27 issues)
PARTICIPANTS
- 26 American MTurkers, 13 for each set
-Various ages from 20s to 50s
13
Measuring Relevance & Uncertainty with MTurk
19. 17
Results of MTurk
1
2
3
4
5
1 2 3 4 5
Uncertainty
Relevance
20s
Internet and privacy 1
2
3
4
5
1 2 3 4 5
30s-50s
Issue
world and football
Internet and privacy
1
2
3
4
5
1 2 3 4 5
60s
world and football
world and football
Internet and privacy
20. 18
Correlations with Agenda Setting Effect
share_corr = 0.480* Relevance
1
2
3
4
5
1 2 3 4 5
Uncertainty
Relevance
Set A
Issue
22. INFLUENTIAL FACTOR
Tone (Polarity) of article
GOAL
Identify the effects of article tone, positive and negative, on the commenting
and sharing behaviors of the audience
20
Content Polarity & Audience Behavior
23. 21
Positive and Negative Articles
Proportion of sharing behavior to commenting behavior. Audience tends to leave more comments on negative article set, on the
other hand, audience shares more articles in positive article set.
24. 22
DETECTED POS./NEG.WORDS
The sets of positive and negative words obtained from model analysis for news articles. Words
depending on sections differentiate positive and negative traits of each section.
BUSINESS HEALTH OPINION POLITICS
Positive
joined
viral
smoothly
better
balance
respect
forward
empower
fair
moderate
Negative
cutthroat
axed
lawsuit
beating
lose
opposite
battle
unjust
fuming
sequester
Positive
care
respect
admit
clarify
essential
healthy
repair
benign
hope
repaired
Negative
tough
severe
emergency
affected
risk
dying
war
spitting
tricks
abnormal
Positive
spectacular
useful
created
prize
confirm
love
sublime
win
confident
mellow
Negative
weird
fog
distressing
slam
doubted
fail
wrong
fears
slippery
peril
Positive
expert
forward
proud
consent
carol
rights
great
worth
integrity
truth
Negative
ironic
heinous
arguing
dick
undo
grinding
outlaw
meaningless
theft
lost
SCIENCE SPORTS TECHNOLOGY WORLD
Positive
fortunate
cleanup
essential
credit
safety
comforting
milestone
learn
gang
dim
Negative
spill
crude
busted
upset
concern
problems
dark
smash
prize
creating
Positive
victory
won
grace
fun
champion
passion
ace
belief
luck
balance
Negative
chase
shock
busted
beating
defeat
thwart
lost
alleged
assault
cockeyed
Positive
best
fancy
easy
help
intelligence
strong
improve
fit
trust
fame
Negative
blocks
shabby
shy
wicked
rash
shaky
mortal
grave
pity
unfinished
Positive
free
respected
support
moderate
consistent
prompt
afford
gratitude
joined
affluent
Negative
tension
protest
heavy
raging
slam
war
crime
oppress
poverty
poor
25. • Presented preliminary research in using computational methods for media
studies
• Crawled a corpus of articles including user comments and social sharing
counts from NPR over a period of three years
• Showed that sharing patterns and commenting patterns are quite different
• Showed the effects of agenda setting for 57 issues over 8 sections of NPR
• Looked at relevance and uncertainty as two dimensions to explain the various
degree of agenda setting for different issues
• Looked at the tone of article (pos, neg) to see whether people react differently
• Identified lots of loose ends
• Please contact me if you are interested in collaborating
Contributions & Future Work
26. Set ASet A Set BSet B
#Person Kappa #Person Kappa
Entire set 13 0.0436** 13 0.0192
20s 2 -0.225. 1 One person
30s 3 0.0228 6 0.00883
40s 2 0.0189 2 0.074
50s 4 -0.0382 3 -0.0588
60s 2 0.211. 1 One person
30s-50s 9 0.0448* 11 0.0162
Male 4 0.0158 7 0.0239
Female 9 0.0616** 6 -0.0048
24
DATA:AGREEMENT & SIGNIFICANCE
Calculate Fleiss’ Kappa value for each data set. .p<0.1, *p<0.05,**p<0.01, ***p<0.001,****p<0.0001
