This document summarizes several approaches for sentiment analysis of tweets. It discusses basic machine learning approaches using features like n-grams, part-of-speech tags, and relationships between tweets. Advanced approaches exploit social and topical contexts, learn sentiment-specific word embeddings, and use recursive neural networks and convolutional neural networks. Deep learning methods like recursive neural tensor networks and convolutional neural networks achieved state-of-the-art performance. Open challenges remain in handling sarcasm, ambiguity and incorporating contextual information.

1. Образец заголовка
Survey on Approaches of
Sentiment Analysis of Tweets
by Swapna Lekkala
Prepared as an assignment for CS410: Text Information Systems in Spring 2016
netid: lekkala2@illinois.edu

2. Образец заголовкаSentiments
• Huge explosion of sentiments on social media like
Facebook, Twitter, blogs, message boards and
user forums
500 million tweets per day currently
95 million yelp reviews on Dec 2015
510 comments per minute, 293 K status
updates per minute and 136 K photo
upload per minute on Facebook

3. Образец заголовкаApplications of Sentiments
• Political elections to gauge the sentiment of
people on issues and campaign speeches of a
candidate
• Financial Markets to predict the stock price
movements based on sentiment analysis of tweets
• Product improvement by monitoring reviews
of users in real time
• Predictive Analysis provides actionable items
for government and businesses to dispel rumors
and negative sentiments
3

4. Образец заголовка
What is Sentiment
Analysis(SA)?
SA is the task of mining opinions of authors on
specific entities
4
Hottest research area in Computer Science
Over 7000 research articles on SA by 2013
SA techniques
Track
sentiments/opinions
on social media
Feedback for Products
and actions

5. Образец заголовкаTwitter sentiment analysis
Tweets contain
• emoticons, abbreviations, Colloquial
expressions
• opinions of multiple entities
• Procuring labeled data of tweets is
also laborious
5
Is Sentiment Analysis used for large opinionated corpora
applicable to tweets?

6. Образец заголовка
Basic Approaches of SA of
tweets
1.Machine learning based approaches
• using unigram or bigrams features for term level
analysis
• Part-of-Speech tags features are used for sentence
or phrase level analysis
• supervised learning based approaches using
classifiers such as Naives Bayes, Maximum Entropy,
Support Vector Machines, KNN or Logistic Regression
• relationships between tweets, target dependent
features, social network relations are exploited for
improvements
6

7. Образец заголовкаBasic Approaches Contd.
2. Unsupervised learning approaches
• Lexicon based approaches: Sentiment
or polarity of opinion is expressed as a
function of opinion words in the document.
• Statistical approaches: calculate word
co-occurrences to infer semantic
orientation of words or word frequency in a
syntactic context
7
Semantic Orientation = Difference of PMI scores of
phrase with two sentiment words(“poor” or “excellent”)

8. Образец заголовка
Drawbacks of basic
approaches
• Lexicon based approaches have low recall but
good precision
• Conventional methods for documents do not work
well for tweets
• Lack of large labelled training data sets for
learning techniques
• Simple bag of word features using classifiers like
SVM, Naives Bayes etc are good for document
topic classification but do not work well for
predictive sentiment analysis.
• Larger phrases as features are not handled
8

9. Образец заголовкаSentiment analysis for different
types needs different granularity
Document - Level Sentiment Analysis: Opinion of one object
Sentence- Level Sentiment Analysis: Multiple opinions of
entities
Aspect - Based Sentiment Analysis: Aspects of same entity
Comparative Sentiment Analysis: Comparative opinion
Sentiment Lexicon Acquisition:
9
Different types of SA need different techniques

10. Образец заголовкаAdvanced Approaches
Document level SA and Sentence level SA:
Unsupervised and Supervised machine learning
methods work
Aspect based SA: Noun phrases in a document are
extracted and PMI is calculated with phrases
related to product category
Comparative SA: comparative/superlative
adjectives/adverbs like “outperform”, “prefer” are
extracted
Sentiment Lexicon as a Resource: corpus based
algorithms to capture domain specificity
10

11. Образец заголовкаEntity level SA for tweets
1) Tweets are tokenized and POS tagged
2) Aggregated Opinion for entities is obtained by
the score function score(e).
wi is word in sentence s,L is opinion lexicon,
dis(wi,e) is distance between entity and wi
3) Additional Opinionated tweets are extracted
using Pearson Chi square test and
4) SVM classifier is learned
11
in the
{what,
uld be-
o, did,
in the
entity)
though
well for
d have
tweet,
!”. We
fers to
closest
al. 2008)). The basic idea is as follows. Given a sen-
tence s containing the user-given entity, opinion words
in the sentence are first identified by matching with
the words in the opinion lexicon. We then compute
an orientation score for the entity e. A positive word
is assigned the semantic orientation score of +1, and
a negative word is assigned the semantic orientation
score of 1. All the scores are then summed up using
the following score function:
score(e) = ⌃wi:wi2Lwi2s
wi · so
dis(wi, e)
(1)
where wiis an opinion word, L is the opinion lexicon
and s is the sentence that contains the entity e, and
dis(wi, e) is the distance between entity e and opinion
word wi in the sentence s. wi · so is the semantic
orientation score of the word wi. The multiplicative
inverse in the formula is used to give low weights to

12. Образец заголовка
Collective Sentiment
dynamics of Tweets
Dynamics of sentiments in social media is given
by r
Prediction model parameter: history window,
prediction bandwidth, response time
Various Machine learning
methods are applied for
different values of these
parameters
12
The goal of this work is to predict the sentiment change
over time rather than the absolute sentiment values (e.g.,
the change of number of positive tweets at a certain time).
We quantify the dynamics of the sentiment in social media
through measuring the ratio r between positive tweets and
tweets with either positive or negative polarity for a partic-
ular time interval. r is defined as:
r =
#tweets+
#tweets+ + #tweets
(1)
r ranges between 0 and 1. It is 0 when there are no positive
tweets and it is 1 when there are no negative tweets at a
certain time slice. Positive and negative tweets are classi-
fied by the sentiment analysis described in more details in
Section 4. r is a ratio and it does not depend on the absolute
number of tweets. This is important as we are comparing
the sentiment changes over multiple topics (namely, iPhone,
Android and Blackberry) and di↵erent topics have di↵erent
number of tweets (Figure 3), trying to create a model pre-
dicting absolute values for iPhone might for example not
work for Android or for other domains such as politics, etc.
Figure 1 shows the r ratio of iPhone over 7 days (168 hours).
Rati
Dyn
Table
time
The
namic
is con
wind
time
featur
the se
“respo
describe the statistical model to predict the change based on
features extracted from the time-series social media dynam-
ics.
3.1 Sentiment Change
The goal of this work is to predict the sentiment change
over time rather than the absolute sentiment values (e.g.,
the change of number of positive tweets at a certain time).
We quantify the dynamics of the sentiment in social media
through measuring the ratio r between positive tweets and
tweets with either positive or negative polarity for a partic-
ular time interval. r is defined as:
r =
#tweets+
#tweets+ + #tweets
(1)
r ranges between 0 and 1. It is 0 when there are no positive
tweets and it is 1 when there are no negative tweets at a
certain time slice. Positive and negative tweets are classi-
fied by the sentiment analysis described in more details in
Section 4. r is a ratio and it does not depend on the absolute
number of tweets. This is important as we are comparing
the sentiment changes over multiple topics (namely, iPhone,
Android and Blackberry) and di↵erent topics have di↵erent
number of tweets (Figure 3), trying to create a model pre-
dicting absolute values for iPhone might for example not
work for Android or for other domains such as politics, etc.
Figure 1 shows the r ratio of iPhone over 7 days (168 hours).
Figure 1: Ratio between positive tweets and tweets
User Number of followers
Number of friends
Number of posted statuses
Number of lists a user belongs to
Sentiment #positive : #negative tweets
Ratio #positive : #(positive+negative) tweets
#negative : #(positive+negative) tweets
#neutral : #(positive+negative) tweets
#(positive+negative) : #all tweets
#neutral : #all tweets
Dynamics First and second order
derivatives of all above features
Table 1: Features extracted from the social media
time series to model the dynamics of sentiment.
The goal of this research is to predict the sentiment dy-
namics in social media in the future. The prediction process
is conditioned on three random variables, namely, history
window size ↵, prediction bandwidth and response
time (as shown in Figure 2). Our prediction model uses
features extracted from the history window ↵ and predict
the sentiment changes in a future window which is after
“response time” from now.
Figure 2: Parameters of the prediction: history win-
dow size ↵, prediction bandwidth and response
time . Prediction model extracts features from his-
tory window and predict the sentiment change of the
social media in a future window of size which is
hours after the current time t.

13. Образец заголовкаFindings
1) Long history suppressed important immediate
occurrences before prediction time
2) Machine learning models performed well for
Sentiments between 12 and 24 hours
3) SVM, logistic regression outperformed decision
trees in F1 scores.
4) 85 % accuracy was observed in prediction of
directional sentiment ratio.
13

14. Образец заголовка
Predicting user-topic opinion in
twitter using social & topical context
• Collaborative filtering task
• The formal problem definition is
Given user topic matrix OL , Graph GS with adjacency
matrix S of social relations among user and Graph GT
with adjacency matrix T of topic relations, predict the
user topic opinions OU
• Matrix factorization framework incorporates social
and topical context as regularization constraints to
minimize the objective function.
• Adding social and topical context improved user-topic
opinion prediction compared to other methods.
14

15. Образец заголовка
Recursive Deep models for
sentiment compositonality
• Stanford Sentiment treebank with fully labeled
parse trees of syntactically plausible phrases
• Recursive Neural Tensor Network(RNTN) take
input word vectors and uses same tensor based
compositionally function to calculate higher nodes
in the tree.
• RNTN outperforms other machine learning and
recursive models with simple bag of words as
features
• RNTN achieves 85% accuracy and outperform
baseline binary classifiers at 80 %
15

16. Образец заголовкаRNTN is the state of the art
RNTN also achieves 80.7 % accuracy in
• fine grained sentiment prediction across all
phrases
• captures negation of different sentiments and
scope more accurately compared to other
methods
16
cursive Deep Models for Semantic Compositionality
Over a Sentiment Treebank
Richard Socher, Alex Perelygin, Jean Y. Wu, Jason Chuang,
hristopher D. Manning, Andrew Y. Ng and Christopher Potts
Stanford University, Stanford, CA 94305, USA
ard@socher.org,{aperelyg,jcchuang,ang}@cs.stanford.edu
{jeaneis,manning,cgpotts}@stanford.edu
Abstract
spaces have been very use-
xpress the meaning of longer
ncipled way. Further progress
tanding compositionality in
sentiment detection requires
d training and evaluation re-
re powerful models of com-
emedy this, we introduce a
ank. It includes fine grained
for 215,154 phrases in the
1,855 sentences and presents
–
0
0
This
0
film
–
–
–
0
does
0
n’t
0
+
care
+
0
about
+
+
+
+
+
cleverness
0
,
0
wit
0
or
+
0
0
any
0
0
other
+
kind
+
0
of
+
+
intelligent
+ +
humor
0
.
Figure 1: Example of the Recursive Neural Tensor Net-

17. Образец заголовка
Semi-Supervised Recursive
Autoencoders(RAE) for Predicting
Sentiment Distributions
• RAE learns semantic vector representations of
phrases
• Hierarchical structure and compositional
semantics are leveraged unlike bag of words
representation
• RAE does not need Sentiment Lexica. Can be
trained on unlabeled data 17
Indices
Words
Semantic
Representations
Recursive Autoencoder
i walked into a parked car
Sorry, Hugs You Rock Teehee I Understand Wow, Just Wow
Predicted
Sentiment
Distribution

18. Образец заголовкаPerformance of RAE
RAE on tasks involving complex broad range
human sentiment outperforms
- approached based on sentiment lexical that lack
in coverage
- bag of word representations not robust enough
RAE outperforms state of the art dependency tree
based classification method
18

19. Образец заголовка
Exploiting Social Relations for
Sentiment Analysis in
• Sociological approach to handle noisy and short
text data(SANT) for sentiment classification
• The problem is formally defined as:
given microblogging message corpus T with
content X and corresponding sentiment labels Y,
and sociological information in the form of user
message relation U and user-user following
relation, a classifier W is learned to assign
sentiment labels for unseen messages
19

20. Образец заголовкаPerformance of SANT
• SANT outperforms text- based
methods like Least squares(LS),
Lasso, Mincuts and LexRatio
• SANT is independent of training data
set sizes and incorporates sociological
information in a most efficient way.
20

21. Образец заголовка
Learning SentimentSpecific Word Embedding for
Twitter Sentiment Classification
• Sentiment information is incorporated into neural
networks to learn sentiment specific word
embedding (SSWE) of distant supervised tweets
• SSWE outperforms all baseline methods in
identification of positive and negative tweets
• SSWE also outperforms other word embedding
like C&W,Word2Vec, ReEmb and WVSA as these
do not exploit sentiment information in tweets.
21

22. Образец заголовка
Deep Convolutional Neural Networks
for Sentiment Analysis of Short Texts
• Character to Sentence Convolution Neural
Network (CharSCNN) is proposed
Convolution layers are used to extract relevant
features from character level to sentence level
• Does not need hand crafted inputs
• CharSCNN outperformed RNTN, RNN, SVM and
Naives Bayes for fine-grained and binary
classification of Stanford Twitter Sentiment
Treebank
• Character-level information is found to have
more impact for short texts like tweets
22

23. Образец заголовка
Open Research Challenges and
Future Work
1) Word ambiguity Improving accuracy in identifying relevant
text
2) Classification methods to identify sarcasm
3) Need Algorithms that use context to attach sentiment scores
to objective statements.
4)comprehensive knowledge base is required for concept
based approaches and this can place bounds on inferences of
semantic
5) Spatial-temporal patterns in tweets to determine overall
sentiment of people in different regions.
6) Study how sentiments diffuse on online networks as
compared to real world
7) Fully explored multimodal analysis of audio, video and
linguistic information can serve the areas in which textual
transcripts are unavailable for mining opinions
23

24. Образец заголовкаReferences
1) Techniques and Applications for Sentiment Analysis Author: Ronen Feldman,
Communications of the Acm, 2013
2) Combining Lexiconbased and Learningbased Methods for Twitter Sentiment
Analysis, Lei Zhang, Riddhiman Ghosh, Mohamed Dekhil, Meichun Hsu, Bing
Liu, 2011, HP Laboratories HPL201189
3) Predicting collective sentiment dynamics from times eries social media, Le T.
Nguyen, Pang Wu, William Chan, Wei Peng and Ying Zhang, Proceedings of the
first international workshop on issues of sentiment discovery and opinion mining
ACM, 2012
4) Predicting UserTopic Opinions in Twitter with Social and Topical Context
Author: Fuji Ren, Senior Member, IEEE, and Ye Wu, 2013, IEEE
TRANSACTIONS ON AFFECTIVE COMPUTING
5) Recursive Deep Models for Semantic Compositionality Over a Sentiment
Treebank, Richard Socher, Alex Perelygin, Jean Y. Wu, Jason Chuang,
Christopher D. Manning, Andrew Y. Ng and Christopher Potts, 2013, Proceedings
of the conference on empirical methods in natural language processing (EMNLP)
24

25. Образец заголовкаReferences contd.
6) SemiSupervised Recursive Autoencoders for Predicting Sentiment Distributions
Authors: Richard Socher, Jeffrey Pennington, Eric H. Huang, Andrew Y. Ng,
Christopher D. Manning, Conference: P roceedings of the 2011 Conference on
Empirical Methods in Natural Language Processing
Year: 2011
7) New Avenues in Opinion Mining and Sentiment Analysis Author: Erik Cambria,
Bjorn Schuller, Yunqing Xia and Catherine Havasi Year: 2013 Conference : IEEE
Computer Society
8) Exploiting Social Relations for Sentiment Analysis in Microblogging, Xia Hu, Lei
Tang, Jiliang Tang, Huan Liu, Proceedings of the sixth ACM international conference
on Web search and data mining, 2013
9) Learning SentimentSpecific Word Embedding for Twitter Sentiment
Classification,Duyu Tang, Furu Wei, Nan Yang, Ming Zhou, Ting Liu, Bing Qin,
Proceedings of the 52nd Annual Meeting of the Association for Computational
Linguistics, 2014
10)Deep Convolutional Neural Networks for Sentiment Analysis of Short Texts,
Cicero Nogueira dos Santos and Maira Gatti, Proceedings of COLING 2014, the
25th International Conference on Computational Linguistics 25