1
WIS
Web
Information
Systems
Helping users discover
perspectives
Enhancing opinion mining with joint topic models
Tim Draws, Jody Liu, Nava Tintarev
TU Delft, The Netherlands
t.a.draws@tudelft.nl
https://timdraws.net

2
WIS
Web
Information
Systems
Discovering perspectives

3
WIS
Web
Information
Systems
Discovering perspectives
Unstructured set of
textual opinions
?
Perspective 1
Supporting
Perspective 2
Perspective 3
Perspective 4
Perspective 5
Perspective 6
Perspective 7
Perspective 8
Perspective 9
Perspective 10
Opposing
Structured set of
perspectives

4
WIS
Web
Information
Systems
Topic models
• Topic model = unsupervised model to discover
hidden structures (i.e., topics) in corpora of text
– Example: Latent Dirichlet Allocation (LDA) [1]
– Topics are probability distributions over words
– If applied to a corpus of documents related to a debate,
topics could be interpreted as perspectives
• Joint topic model = adding additional components
(e.g., sentiment analysis) to a classical topic
model (e.g., LDA)

5
WIS
Web
Information
Systems
Our paper
RQ1. Can joint topic models support users in discovering
perspectives in a corpus of opinionated documents?
RQ2. Do users interpret the output of joint topic models
in line with their personal pre-existing stance?
Contributions:
1. Perspective-annotated data set
2. User study

6
WIS
Web
Information
Systems
Data
Document Stance Perspective
You cannot be a
Christian and support
abortion…
Against Abortion is the killing of a human
being, which defies the word of
God.
No one in the world has
any right to judge over
what someone else
does with their body, …
For Reproductive choice empowers
women by giving them control over
their own bodies.
Why put a child through
the pain of an unloving
mother…
For A baby should not come into the
world unwanted.
… … …
Final data set: 600 documents; 6 perspectives

7
WIS
Web
Information
Systems
Experimental setup
1
2
3
4
5
• Ran each model on the final
data set (i.e., for 6 topics)
• Between-subjects study: each
participant sees output of one
of the models
• Participants need to identify
the correct 6 perspectives from
the model output

8
WIS
Web
Information
Systems
Procedure
Step 1 Step 2 Step 3
Participants state:
• Age
• Gender
• Personal stance
towards abortion
• Familiarity with the
abortion debate
Participants state:
• Perceived usefulness
• Perceived awareness
increase
• Confidence in task
performance

9
WIS
Web
Information
Systems
Results: descriptive
• 158 participants (recruited from Prolific)
– After excluding 12 participants due to failing both honeypot topics
– 150 required according to power analysis
• 50.6% female, 49.4% male
• 33.3 years old on average (range 18 to 64)
• Most (57.8%) at least somewhat familiar with the topic
• Sample skewed towards the supporting viewpoint

10
WIS
Web
Information
Systems
Results: hypothesis tests
H1: Users find more correct perspectives when being exposed
to the output of a joint topic model compared to the output of a
regular topic model or baseline.
– We find a difference between models (p < 0.001, η2 = 0.126)
– TAM is the only one that performs significantly better than the baseline
3
4
5
TF−IDF LDA JST VODUM TAM LAM
Model
MeannCor

11
WIS
Web
Information
Systems
Results: hypothesis tests
H2: Users are more likely to identify sets of keywords as
perspectives that are in line with their personal stance compared
to perspectives that they do not agree with.
– No evidence for for such a relationship (ρ = 0.122, p = 0.163)

13
WIS
Web
Information
Systems
Discussion and future work
• Why did TAM perform better?
– It extracted more keywords that appeared explicitly in the perspective expression
Abortion is the killing of a human
being, which defies the word of God.
Reproductive choice empowers
women by giving them control over
their own bodies.
A baby should not come into the
world unwanted.
• Future work: different domains, novel topic models

14
WIS
Web
Information
Systems
Take home
• Joint topic models such as TAM can perform
perspective discovery
• No evidence for tendency of users to interpret
output in line with their personal stance
• Implications for several areas: journalism,
policy-making, generating explanations
(All supplementary materials are openly available at
https://osf.io/uns63/.)

15
WIS
Web
Information
Systems
References
[1] D. Blei, A. Ng, and M. Jordan, “Latent dirichlet allocation,” Journal of Machine Learning Research, vol. 3, pp. 993–
1022, 05 2003.
[2] M. Paul and R. Girju, “A two-dimensional topic-aspect model for discovering multi-faceted topics.” in AAAI, vol. 1, 01
2010. [Online]. Available: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.
226.3550&rep=rep1&type=pdf
[3] C. Lin and Y. He, “Joint sentiment/topic model for sentiment analysis,” in Proceedings of the 18th ACM Conference on
Information and Knowledge Management, ser. CIKM ’09. New York, NY, USA: Association for Computing
Machinery, 2009, p. 375–384. [Online]. Available: https://doi.org/10.1145/1645953.1646003
[4] T. Thonet, G. Cabanac, M. Boughanem, and K. Pinel-Sauvagnat, “Vodum: A topic model unifying viewpoint, topic and
opinion discovery,” in ECIR, vol. 9626. Toulouse, France: Springer, 03 2016, pp. 533– 545.
[5] D. Vilares and Y. He, “Detecting perspectives in political debates,” in EMNLP. Association for Computational
Linguistics, 01 2017, pp. 1573–1582.