Tim Draws, Jody Liu, and Nava Tintarev explore using joint topic models to help users discover perspectives in opinionated text corpora related to debates. They conducted an experiment comparing regular topic models and joint topic models on a dataset of 600 documents annotated with stances and perspectives on abortion. Their Transformed Aspect Model (TAM) performed the best at identifying the correct perspectives, extracting more keywords that explicitly expressed perspectives. However, they found no evidence that users were more likely to identify perspectives that aligned with their own stance over opposing stances. The study provides implications for applications involving journalism, policy-making, and generating explanations.

1. 1
WIS
Web
Information
Systems
Helping users discover
perspectives
Enhancing opinion mining with joint topic models
Tim Draws, Jody Liu, Nava Tintarev
TU Delft, The Netherlands
t.a.draws@tudelft.nl
https://timdraws.net

2. 2
WIS
Web
Information
Systems
Discovering perspectives

3. 3
WIS
Web
Information
Systems
Discovering perspectives
Unstructured set of
textual opinions
?
Perspective 1
Supporting
Perspective 2
Perspective 3
Perspective 4
Perspective 5
Perspective 6
Perspective 7
Perspective 8
Perspective 9
Perspective 10
Opposing
Structured set of
perspectives

4. 4
WIS
Web
Information
Systems
Topic models
• Topic model = unsupervised model to discover
hidden structures (i.e., topics) in corpora of text
– Example: Latent Dirichlet Allocation (LDA) [1]
– Topics are probability distributions over words
– If applied to a corpus of documents related to a debate,
topics could be interpreted as perspectives
• Joint topic model = adding additional components
(e.g., sentiment analysis) to a classical topic
model (e.g., LDA)

5. 5
WIS
Web
Information
Systems
Our paper
RQ1. Can joint topic models support users in discovering
perspectives in a corpus of opinionated documents?
RQ2. Do users interpret the output of joint topic models
in line with their personal pre-existing stance?
Contributions:
1. Perspective-annotated data set
2. User study

6. 6
WIS
Web
Information
Systems
Data
Document Stance Perspective
You cannot be a
Christian and support
abortion…
Against Abortion is the killing of a human
being, which defies the word of
God.
No one in the world has
any right to judge over
what someone else
does with their body, …
For Reproductive choice empowers
women by giving them control over
their own bodies.
Why put a child through
the pain of an unloving
mother…
For A baby should not come into the
world unwanted.
… … …
Final data set: 600 documents; 6 perspectives

7. 7
WIS
Web
Information
Systems
Experimental setup
1
2
3
4
5
• Ran each model on the final
data set (i.e., for 6 topics)
• Between-subjects study: each
participant sees output of one
of the models
• Participants need to identify
the correct 6 perspectives from
the model output

8. 8
WIS
Web
Information
Systems
Procedure
Step 1 Step 2 Step 3
Participants state:
• Age
• Gender
• Personal stance
towards abortion
• Familiarity with the
abortion debate
Participants state:
• Perceived usefulness
• Perceived awareness
increase
• Confidence in task
performance

9. 9
WIS
Web
Information
Systems
Results: descriptive
• 158 participants (recruited from Prolific)
– After excluding 12 participants due to failing both honeypot topics
– 150 required according to power analysis
• 50.6% female, 49.4% male
• 33.3 years old on average (range 18 to 64)
• Most (57.8%) at least somewhat familiar with the topic
• Sample skewed towards the supporting viewpoint

10. 10
WIS
Web
Information
Systems
Results: hypothesis tests
H1: Users find more correct perspectives when being exposed
to the output of a joint topic model compared to the output of a
regular topic model or baseline.
– We find a difference between models (p < 0.001, η2 = 0.126)
– TAM is the only one that performs significantly better than the baseline
3
4
5
TF−IDF LDA JST VODUM TAM LAM
Model
MeannCor

11. 11
WIS
Web
Information
Systems
Results: hypothesis tests
H2: Users are more likely to identify sets of keywords as
perspectives that are in line with their personal stance compared
to perspectives that they do not agree with.
– No evidence for for such a relationship (ρ = 0.122, p = 0.163)

12. 13
WIS
Web
Information
Systems
Discussion and future work
• Why did TAM perform better?
– It extracted more keywords that appeared explicitly in the perspective expression
Abortion is the killing of a human
being, which defies the word of God.
Reproductive choice empowers
women by giving them control over
their own bodies.
A baby should not come into the
world unwanted.
• Future work: different domains, novel topic models

13. 14
WIS
Web
Information
Systems
Take home
• Joint topic models such as TAM can perform
perspective discovery
• No evidence for tendency of users to interpret
output in line with their personal stance
• Implications for several areas: journalism,
policy-making, generating explanations
(All supplementary materials are openly available at
https://osf.io/uns63/.)

14. 15
WIS
Web
Information
Systems
References
[1] D. Blei, A. Ng, and M. Jordan, “Latent dirichlet allocation,” Journal of Machine Learning Research, vol. 3, pp. 993–
1022, 05 2003.
[2] M. Paul and R. Girju, “A two-dimensional topic-aspect model for discovering multi-faceted topics.” in AAAI, vol. 1, 01
2010. [Online]. Available: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.
226.3550&rep=rep1&type=pdf
[3] C. Lin and Y. He, “Joint sentiment/topic model for sentiment analysis,” in Proceedings of the 18th ACM Conference on
Information and Knowledge Management, ser. CIKM ’09. New York, NY, USA: Association for Computing
Machinery, 2009, p. 375–384. [Online]. Available: https://doi.org/10.1145/1645953.1646003
[4] T. Thonet, G. Cabanac, M. Boughanem, and K. Pinel-Sauvagnat, “Vodum: A topic model unifying viewpoint, topic and
opinion discovery,” in ECIR, vol. 9626. Toulouse, France: Springer, 03 2016, pp. 533– 545.
[5] D. Vilares and Y. He, “Detecting perspectives in political debates,” in EMNLP. Association for Computational
Linguistics, 01 2017, pp. 1573–1582.