1. Deep Dive:
Consumer Sentiment
Rating & Analysis
white paper
Discover the philosophy behind DataRank’s propriety sentiment
model, and our methodology for applying, and testing,
automated sentiment analysis.

2. Consumer Sentiment
Rating & Analysis
> introduction
Businesses collect online conversation about their brand and products to get insights from
consumers who are freely sharing their opinions on social media, and review sites.
Quickly learning how a person feels about a product can boost businesses toward improved
customer service, refined marketing campaigns, and product innovation. It can also provide
insights for competitor benchmarking, and industry trends.
Most social media monitoring tools are equipped with the technology to perform sentiment
rating. The quality of machine sentiment rated comments, however, varies greatly between each
monitoring tool.
Industry leading social media monitoring companies don’t settle for average results. They are
constantly redrawing the boundaries of automated sentiment analysis, and providing their
customers with the highest possible sentiment rating accuracy.
In this guide you will learn how DataRank performs automated, and manual sentiment rating and
analyses on the comments that we collect, and how our method is producing industry leading
results.
Sentiment describes the feeling that comes from within a comment or review. Did a customer
have a bad experience with a product? Did a brand meet their expectations? Did a customer prefer
a competitor’s product?
While sentiment is usually described as having a binary opposition it is often more complex.
There are comments and reviews that offer neither a good or bad opinion, and are described as
having neutral sentiment.
Sentiment Analysis aims to determine the attitude of the author of a specific piece of content with
respect to the topic of interest. Comments and content can be referred to as Positive, Negative,
Neutral or having no sentiment at all.
> what is sentiment analysis?
©DataRank>DeepDive:ConsumerSentimentRating&Analysis
1

3. > our sentiment model
2
> original sentiment
The DataRank Sentiment Model has 3 core components:
• Original Sentiment
• Machine/Automated Sentiment
• Human/Manual Sentiment
Most social media monitoring companies have produced their own system for conducting
sentiment analysis.
At DataRank we use a combination of original sentiment, machine learning based sentiment
analysis and manual, human-rated sentiment.
This allows us to rate large data sets of thousands of comments, while also controlling the quality
of the sentiment analysis process.
Original sentiment occurs when a comment already has some kind of sentiment rating applied
to it from the original source. For example, when you collect reviews from a site like Walmart the
customer gives a star rating to the product. Rating methods vary by site. (Walmart uses a 5 star
system, whereas Amazon uses a 10 star system).
To manage these varying original sentiment ratings we’ve made rules that automatically classifies
a 1 star review into negative sentiment, a 3 star as neutral and 5 star as positive. The mapping of
sentiment changes depending on the number of overall stars a product can be given.
Regardless of how ratings are represented on the original datasource, they are mapped to a value
between one and five for valid sentiments. Invalid sentiments are mapped to zero.
If original sentiment is provided with a comment it is the most preferred sentiment and takes
precedence over all other sentiment ratings.
©DataRank>DeepDive:ConsumerSentimentRating&Analysis

4. Machine sentiment or automated sentiment analysis is the process of applying a heuristic model
to automatically measure the sentiment of a comment.
Our model currently implements a Bayesian classifier largely due to it’s simplicity and relative
accuracy. Our current model reports an accuracy of approximately 80% for both positive
classification and negative classification. This number can fluctuate depending on the comment,
and its datasource.
The goal of automated sentiment is to be 100% accurate but we know that is likely not going to
occur for some time yet. To reach 100% accuracy, the machine will have to become better than
a human at determining sentiment. As you will go on to read later, human sentiment rating
accuracy is largely dependant on the context surrounding the comment rating, and the emotions
that the human perceives the comment to exhibit.
Between 80-85% is currently the highest industry average for automated sentiment rating.
Sentiment analysis accuracy also depends largely on the datasource. Tweets for example are
full of single-sentence, slang-filled, misspellings, hashtags, mentions, urls and images. Whereas
reviews published on Amazon, Walmart and Goodreads are often lengthy paragraphs employing
grammatical structure, and proper vocabulary.
We collect a large sample size of comments of human rated eCommerce reviews, comments,
Tweets, emails, and various other human rated data that our global Bayesian Sentiment classifier is
trained on.
Problems occur in comments that suggest both positive and negative sentiment, which can usually
only be resolved by taking one of two stances:
• Human rate the comment
• Machine rating takes the overall language and judges whether the comment is overly positive
or negative.
Take this comment for example:
©DataRank>DeepDive:ConsumerSentimentRating&Analysis
3
> machine sentiment /
automated sentiment analysis
“Was so excited to pick up the new Dr. Pepper after work. Got home to
discover I’d grabbed Coca-Cola by mistake. I hate Coke!!”

5. Choosing a sentiment rating for a comment that exhibits both positive and negative sentiment can
be complicated. Generally we will take the sentiment of the author. But when we create topics for
a customer the sentiment is determined by the analyst. The Dr Pepper analyst, for example, would
classify the above tweet as positive. On a topic run by a Coca-Cola analyst, the comment would be
negative.
DataRank’s engineering team is hard at work on future innovations for improved automated
sentiment analysis. We take pride in providing our customers with tools backed by the best
technology in the industry.
There are two ways that DataRank allows for manual human sentiment rating.
The first is to do a batch export and submit it to MTurk (Amazon Mechanical Turks) where
we can quickly obtain sentiment for thousands of comments from paid sentiment raters who
manually determine if a comment is positive, negative, or neutral.
The second is to manually rate comments one at a time via HitFarm. Our DataRank team have
become adept at understanding the sentiment behind a review or comment and time is dedicated
every day for comment rating.
Sentiment rating is an integral part of determining the success of a product or brand. It is,
however, only one part of a number of tools that marketers and analysts are using to uncover
valuable insights about their business from online conversation.
Comment sentiment used in conjunction with demographics data, for example, can help
marketers uncover how different genders and age groups respond to their marketing campaigns.
And geotagged tweets with sentiment rating can reveal key locations for brand growth and
product innovation.
If you have any questions regarding our sentiment rating and analysis process or any of our other
DataRank features, please contact your account manager or email us at contact@datarank.com
©DataRank>DeepDive:ConsumerSentimentRating&Analysis
4
> human sentiment rating
> conclusion

6. thank you
To see how DataRank can help give your business a competitive edge
visit our website and book a live demo with our team.
www.datarank.com/demo
about datarank
DataRank is a leading social listening platform bringing the world’s
online conversation to brands both big and small. Marketers and
researchers are provided with industry leading analytical tools
designed to uncover profitable business insights.
contact
To find out more, please contact:
Candice Evans Gray
candice@datarank.com
Ryan Frazier
ryan@datarank.com
Josephine Hardy
josie@datarank.com
Copyright © 2015 DataRank All rights reserved.
white paper