I was graciously invited by Data Society to talk a bit about data and specifically sentiment analysis. These are my slides.

1. DATA SOCIETY © 2015
TM
“If you can’t explain it simply, you don’t understand it well enough.”
- Albert Einstein

2. DATA SOCIETY © 2015
Contrarian Question
con·trar·i·an
kəәnˈtre(əә)rēəәn,kän-/
noun
noun: contrarian; plural noun: contrarians
1.
a person who opposes or rejects popular opinion, especially in stock exchange dealing.
adjective
adjective: contrarian
1.
opposing or rejecting popular opinion; going against current practice.
”the comment came more from a contrarian disposition than moral conviction"

3. DATA SOCIETY © 2015
Contrarian Question
Data has NO intrinsic value!!

4. DATA SOCIETY © 2015
Facts vs. Opinion(s)
Two main types of textual information.
• Facts and Opinions
Search engines are optimized for Facts;
Sentiment Analysis is a growing attempt (not completely
solved) to optimize the discovery of opinions.
Opinions Mining or Sentiment Analysis is an attempt to
recognize the opinion or sentiment that a person holds
toward an object.

5. DATA SOCIETY © 2015
Let's have a look at this*:
• 91 percent of people report having gone into a store because of an
online experience.
• 89 percent of consumers conduct research using search engines.
• 78 percent of consumers say that posts made by companies on
social media influence their purchases.
• 72 percent of consumers trust online reviews as much as personal
recommendations.
• 62 percent of consumers end up making a purchase in a store after
researching it online.
*Blogs, Comments (i.e. YouTube, Facebook), Reviews, Forums, Microblogging (i.e. Twitter) http://www.investopedia.com/terms/a/asymmetricinformation.asp
Information Asymmetry

6. DATA SOCIETY © 2015
Information Asymmetry
seller
buyer
information
information
information $
$
$
$
$
$
$
$

7. DATA SOCIETY © 2015
Where do we find sentiment?
• Movie / Books: Are the reviews on this movie/book positive/negative?
• Product Sales: What is thought of the new iPhone?
• Public Sentiment: How do consumers feel about the economy? How is
consumer sentiment effecting sales by sector?
• Politics: How are voters polarized, if at all around a candidate or policy?
• Prediction: Stock Prices, Election Outcomes, Market Trends, Product Sales

8. DATA SOCIETY © 2015
Scherer’s Typology of Emotions:
Scherer's typology of emotions is briefly explained as follows:
• Emotion: This is a brief, organically synchronized evaluation of a major event, for
example, being angry, sad, joyful, ashamed, proud, and elated i.e. fired or promoted at
work.
• Mood: This is a diffused, non-caused, low-intensity, long-duration change in subjective
feeling, for example, being cheerful, gloomy, irritable, listless, depressed, and buoyant
• Interpersonal stance: This is an affective stance towards another person in a specific
interaction, for example, being friendly, flirtatious, distant, cold, warm, supportive, and
contemptuous
• Attitudes: This is enduring, affectively colored beliefs or dispositions towards objects or
persons, for example, being liking, loving, hating, valuing, and desiring
• Personality traits: These are stable personality dispositions and typical behavior
tendencies, for example, being nervous, anxious, reckless, morose, hostile, and jealous

9. DATA SOCIETY © 2015
Goal: all measurement is to arrange items on a continuum (observed or unobserved).
Measurement:

10. DATA SOCIETY © 2015
1. Dictionary Based Sentiment Analysis
• i.e. Is an attitude toward an object positive or negative?
• e.g. Jeffrey Breen’s method, qdap
2. Supervised Learning for Sentiment Analysis.
• i.e. Given data we have seen in the past, can we predict class
assignment for our polarity measure (positive/neutral/negative)
• e.g. Naive Bayes, MaxEnt, SVM
3. Unsupervised Sentiment Analysis
• i.e. No dictionaries. No labeled data. No training algorithms. And,
scale words (often bi-grams) and users on a single dimension.
• e.g. latent variable models - IRT
Sentiment Analysis: Ordered Sophistication Lexicon Based
(Supervised)

11. DATA SOCIETY © 2015
• The Beige Book (http://www.federalreserve.gov/monetarypolicy/beigebook), more formally
called the Summary of Commentary on Current Economic Conditions, is a report published
by the United States Federal Research Board (FRB) eight times a year.
• The Beige Book has been in publication since 1985 and is now published online.
• The report is published by each (n=12) of the Federal Reserve Bank districts (e.g. Beige
Book (October 2013) is below)
• The content is rather anecdotal. The report interviews key business contacts, economists,
market experts, and others to get their opinion about the economy.
• The data used in this book can be found on GitHub (https://github.com/
SocialMediaMininginR/beigebook), as well as the Python code for all the scraping and
parsing.
Beige Book

12. DATA SOCIETY © 2015
“Consumer spending grew modestly in most Districts. Auto sales
continued to be strong, particularly in the New York District where they
were said to be increasingly robust. In contrast, Chicago, Kansas City,
and Dallas indicated slower growth in auto sales in September.”
Beige Book

13. DATA SOCIETY © 2015
qdap
Determine polarity (a few ways to do this… none of which are perfect). qdap uses word clusters.
> pol.bb<- polarity(bb$text, grouping.var = bb$location, polarity.frame = POLKEY,
constrain = TRUE, negators = qdapDictionaries::negation.words,
amplifiers = qdapDictionaries::amplification.words,
deamplifiers = qdapDictionaries::deamplification.words, question.weight = 0,
amplifier.weight = .3, n.before = 4, n.after = 2, rm.incomplete = FALSE, digits = 3)
xi = word of polarity
xit = neutral (xi
0
); negator (xi
N
); amplifier(xi
a
); de-amplifier (xi
d
)
Each polarized word (xi) is then weighted w based on the weights from polarity frame
xi
T
xixi-1xi-2xi-3xi-4 xi+1 xi+2

14. DATA SOCIETY © 2015
Summary
Beige Book (1996 - 2013) with recession bars (pink)

15. DATA SOCIETY © 2015
thank you
Blog: http://socialmediaminingr.com
Twitter: @rheimann
http://datatactics.blogspot.com/2015/02/modern-approaches-to-sentiment-analysis.html