SIGIR '18 Short paper poster

1. Hmm.. Why is the new
tax bill controversial?
1. Motivation
• Online controversies often emerge and evolve quickly due to the nature of
social media.
• Navigating social media platforms to learn about a new controversy is an
overwhelming task
• Users have to linearly scan postings to understand two conflicting stances
while fighting against:
• Filter bubble phenomenon
• A lot of noisy postings
2. Problem Definition
3. What makes a tweet a good summary?
4. Probabilistic Ranking Model 5. Experimental Setup
6. Evaluation and conclusion
Given a controversial topic, find a summary of k tweets that
best explains why the topic is controversial by ranking them
We assume that there are always two conflicting stances,
namely stance A and stance B, for a controversial topic, and
aim to generate a summary around those stances.
Application Scenario
• We know it’s not okay that for 40 yrs politicians have denied a woman
coverage of abortion just because she’s poor #BoldTheVote #BeBoldEndHyde
Topic
tweet2Vec summary
# tweets # users # tweets RT ratio
Election 10.8M 4.3M 10,000 70.9%
#TakeAKnee 565K 692K 44,167 71.1%
Abortion 692K 539K 3,477 57.6%
Feminism 1.7M 1.7M 50,323 41.3%
Climate Change 546K 360K 10,234 60.1%
0.0
0.2
0.4
0.6
0.8
1.0
Abortion
2016/11/4
Election
2016/10/14
Feminism
2016/3/8
Climate Change
2016/11/30
US Anthem
Protest
2017/09/30
Fractionoftimespreferred
Random mostRT Sumbasic-C
SumSAT SumSAT-C HastagSumSAT
Dataset
Methods
Evaluation
Summary by
method A
Summary by
method Bvs
Asked 10 people on AMT which
summary better explains the
controversy in a blind fashion.
• Random
• MostRT
• SumBasic (Nenkova and Vanderwende, 2005)
• SumSAT (-C)
• HashtagSumSAT (-C)
• Don’t support #RapeCulture by calling it #LockerroomTalk
• If you’re so pro life then go and f****ing get one?
Good summary tweets are characterized by a clear stance on the
controversial aspect of the topic described by assertive, non-vulgar
language and supported by ”stance hashtags” while being relevant.
We propose that a tweet is likely to be part of the good summary if:
• Stance-indicative
• Articulation
• Topic Relevance
a probability model
a regression model
a language model
score
4.1 Estimating Stance Indication
• Retweet communities are disconnected for
controversial topics (Garimella et al, 2016)
• Define stance hashtags as the hashtags that
are exclusively used in the retweet communities
Extract Stance
Hashtags
# #
#
#
#
Stance A
Stance B
Probability that a tweet
has hashtags of stance A
Probability that a tweet has
hashtags of stance B
#
Estimating latent Hashtags Probability
• Hashtags are incomplete user-annotated labels.
• An absence of a hashtag doesn’t necessarily mean that the hashtag isn’t the
right label.
• We trained tweet2vec (Dhingra et al, 2016) to obtain the embedding of the
tweets and hashtags to estimate the probability of P(h|t) for all hashtags.
• SUMSAT ranks the tweets by the score of the ranking function and take the top
k/2 tweets for two conflicting stances for the summary.
• HashtagSumSAT takes the top k/2 stance hashtags and find the top summary
tweet among those that contain the given stance hashtag.
4.2 Estimating Articulation
4.3 Summary Generation
• Trained a logistic regression model to predict how well-written tweets are
• Labeled 300 tweets with a binary label (articulate vs non-articulate)
• Features of the ratio of tweet POS Tags, OOV words, offensive words, POS Tags
N-grams, stop words as well as tweet length, avg. word length are used.
Myungha Jang and James Allan
Center for Intelligent Information Retrieval, University of Massachusetts Amherst
Explaining Controversy on Social Media via Stance Summarization
• Hashtag-based summarization seems to be more effective for event-
based controversies.
0.42 0.42
0.26
0.08
0.54
0.68
0.61 0.59
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Fractionoftimespreferred
random mostRT Sumbasic-C Sumbasic
SumSAT-C SumSAT HastagSumSAT-C HastagSumSAT
• SumSAT generates the summaries that were preferred the most
followed by HashtagSumSAT-C.
• A normal summarization technique fails: controversy summarization
is a new task.
• Social features are more useful than content features.
Original figure source: Garimella et al. (2015), Quantifying Controversy on Social Media
Sample tweets on ”Abortion” on Oct 21, 2017