he Agenda Setting process has been studied extensively and meta-analyses have consistently found relationships between the media, public and political agendas. However, before studying the interactions of different agendas, they have to be thoroughly operationalized and described. For media and political agendas, studies have relied mostly on manual content analyses, yet these approaches are very labor extensive, thereby only capturing only parts of party or media communication. The goal of this paper is to offer a new method of measuring agendas by using a broader set of public communication with a replicable and scalable computational approach. Specifically, this study contributes to this methodological challenge by proposing the use of neural networks to measure political parties’ agendas.
Several recent studies have demonstrated the useful applicability of computational approaches to study agenda-setting processes, yet the importance of careful application and validation has also been emphasized. A political, symbolic agenda can be defined as “the list of issues to which political actors pay attention” and measured by looking at issues a party talks about. Using neural networks, we propose to employ unsupervised machine learning for calculating vector representations of words or documents and locating them in a semantic space to describe parties' political agendas. Parties have many possibilities to reach different audiences, the sum of which will serve as a proxy of a party’s public agenda. Thus, we purposely take a cross-domain approach and include multiple text kinds. Publicly available press releases, parliamentary speeches, and Facebook and Twitter posts of Austrian parties from the last ten years will be used to study how the parties’ public agendas differ regarding a) the topics present and b) the words most closely connected to them.
This study goes beyond studying limited timeframes, types of communication, or predefined topics, and consequently will lead to a more encompassing and accurate description of agendas.
1. Measuring party agendas: A neural topic modeling approach
Jana Bernhard, Hajo Boomgaarden
ECREA 2022, 19 – 22 Oktober 2022
2. Disclaimer: Please be aware that this study is a work in progress.
To further this research, we would be most interested in feedback
on, and the discussion of our methodological approach.
3. What is a party agenda?
“the list of issues to which political actors pay attention”
(Walgrave et al., 2008, p. 815)
which issues are on political agendas?
(Soroka, 2002)
• Symbolic agendas
• Substantive political agendas
• Prominent issues
• Sensational issues
• Governmntal issues
21.10.2022 Modeling party agendas: A neural network approach Page 3
4. What is a party agenda?
“the list of issues to which political actors pay attention”
(Walgrave et al., 2008, p. 815)
which issues are on political agendas?
(Soroka, 2002)
• Symbolic agendas
• Substantive political agendas
• Prominent issues
• Sensational issues
• Governmntal issues
21.10.2022 Modeling party agendas: A neural network approach Page 4
5. Why use an automated approach to measure party agendas?
• Manual Content analysis is time and labour intensive
• „Past work concentrated on a limited number of political agendas“
(Walgraave et al. 2008, p. 815)
1. (often) limited to one channel
2. (often) limited to few parties
3. (often) limited to short timeframe
21.10.2022 Modeling party agendas: A neural network approach Page 5
6. Automated text analysis is scalable and repetition is cheap.
21.10.2022 Modeling party agendas: A neural network approach Page 6
• Austrian Parties
(SPOE, OEVP, GRUE, FPOE, NEOS)
• National Level, County Level
and Chairpersons
• 2012 – 2021
• Multiple Channels
◦ Tweets
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
SPOE GRUE NEOS FPOE OEVP
7. Automated text analysis is scalable and repetition is cheap.
21.10.2022 Modeling party agendas: A neural network approach Page 7
• Austrian Parties
(SPOE, OEVP, GRUE, FPOE, NEOS)
• National Level, County Level
and Chairpersons
• 2012 – 2021
• Multiple Channels
◦ Tweets
◦ Facebook Posts
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
SPOE GRUE NEOS FPOE OEVP
8. Automated text analysis is scalable and repetition is cheap.
21.10.2022 Modeling party agendas: A neural network approach Page 8
• Austrian Parties
(SPOE, OEVP, GRUE, FPOE, NEOS)
• National Level, County Level
and Chairpersons
• 2012 – 2021
• Multiple Channels
◦ Tweets
◦ Facebook Posts
◦ Parliamentary Speeches
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
SPOE GRUE NEOS FPOE OEVP
9. Automated text analysis is scalable and repetition is cheap.
21.10.2022 Modeling party agendas: A neural network approach Page 9
• Austrian Parties
(SPOE, OEVP, GRUE, FPOE, NEOS)
• National Level, County Level
and Chairpersons
• 2012 – 2021
• Multiple Channels
◦ Tweets
◦ Facebook Posts
◦ Parliamentary Speeches
◦ Press Releases
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
SPOE GRUE NEOS FPOE OEVP
10. Automated text analysis is scalable and repetition is cheap.
21.10.2022 Modeling party agendas: A neural network approach Page 10
• Austrian Parties
(SPOE, OEVP, GRUE, FPOE, NEOS)
• National Level, County Level
and Chairpersons
• 2012 – 2021
• Multiple Channels
◦ Tweets
◦ Facebook Posts
◦ Parliamentary Speeches
◦ Press Releases
0
20000
40000
60000
80000
100000
120000
140000
SPOE GRUE NEOS FPOE OEVP
12. Neural Topic Modeling for Issue Detection?
• Topic Modeling detects Topics
(Angelov, 2022)
◦ A topic is the theme, matter or subject of a text
◦ topics are continuous
◦ each document has its own topic with a value in that continuum
• Topics in Parties Communication can be interpreted as the issues they publicly pay
attention to.
21.10.2022 Modeling party agendas: A neural network approach Page 12
13. 1. We look at one parties communication in different channels.
21.10.2022 Modeling party agendas: A neural network approach Page 13
14. 2. they are ordered by the semantic similarity of their words
(based on pre-trained / self-trained word embeddings).
21.10.2022 Modeling party agendas: A neural network approach Page 14
15. 3. Dense clusters in this semantic space are found and denoted
topics.
21.10.2022 Modeling party agendas: A neural network approach Page 15
T1
T2
T3
T4
T5
T6
T7
16. 4. We interpret these topics by looking at the words that are
closest to them.
21.10.2022 Modeling party agendas: A neural network approach Page 16
T1
T2
T3
T4
T5
T6
T7
17. How can we use these word
and document lists to
measure party agendas?
18. 1. Explore the found issues (excerpt of 5 largest topics)
21.10.2022 Modeling party agendas: A neural network approach Page 20
Green Party Social Democrats Peoples Party Freedom Party Neoliberal Party
Climate Change
Taxes &
Corruption
Taxes & Budgets
Taxes &
Corruption
Reforms
Climate Crisis EU Policy EU Policy EU Budgets EU Policy
EU Policy Youth Policy
Crisis
Communication
Scandals
Environmental
Policy
Education Policy
Environmental
Policy
Education Policy
Attacks on
Government
Climate Crisis
Human Rights Social Work
Internal Party
Politics
Christianity Corruption
19. 2. See how issues develop over time: Climate Change
21.10.2022 Modeling party agendas: A neural network approach Page 21
0
500
1000
1500
2000
2500
2012 2013 2014 2015 2016 2017 2018 2019 2020 2021
Nationalrat Press Releases Facebook Tweets
20. 3. See how channels differ over time
21.10.2022 Modeling party agendas: A neural network approach Page 22
0
500
1000
1500
2000
2500
2012 2013 2014 2015 2016 2017 2018 2019 2020 2021
Nationalrat Press Releases Facebook Tweets
21. 3. See how channels differ over time
21.10.2022 Modeling party agendas: A neural network approach Page 23
0
500
1000
1500
2000
2500
2012 2013 2014 2015 2016 2017 2018 2019 2020 2021
Nationalrat Press Releases Facebook Tweets
22. 3. See how channels differ over time
21.10.2022 Modeling party agendas: A neural network approach Page 24
0
500
1000
1500
2000
2500
2012 2013 2014 2015 2016 2017 2018 2019 2020 2021
Nationalrat Press Releases Facebook Tweets
23. 3. See how channels differ over time
21.10.2022 Modeling party agendas: A neural network approach Page 25
0
500
1000
1500
2000
2500
2012 2013 2014 2015 2016 2017 2018 2019 2020 2021
Nationalrat Press Releases Facebook Tweets
26. 4. Compare how issues are talked about.
• Which words are the closes to a specific issue
◦ Mathematical similarity measures (cosine similarity)
◦ Qualitative interpretation
• In a next step:
◦ how words differ between channels
◦ How words differ over time
21.10.2022 Modeling party agendas: A neural network approach Page 28
28. Can we use neural topic modeling to measure party agendas?
1. Exploratory issue detection
2. Novel issue definition (continuous)
3. Methodological:
a. No Pre-Processing
b. Full set / no sampling
c. Easy replication and data addition
4. Issues over time
5. Comparison on Word Level
6. (word changes over time)
21.10.2022 Modeling party agendas: A neural network approach Page 30
YES!
29. Can we use neural topic modeling to measure party agendas?
1. Validate, validate, validate
2. Easier approaches also possible
a. Manual Analysis
b. Dictionary Approaches
c. Clustering Approaches
3. Application of computational models should be
considered in light of its usefulness towards the
research question.
21.10.2022 Modeling party agendas: A neural network approach Page 31
BUT …
30. Neural Topic Models can be a
fruitful approach to
measuring party agendas,
when validated carefully and
useful to the research
question.
