Methodological study of opinion mining and sentiment analysis techniques
Analyzing Samsung Galaxy Mega 5.8 Reviews
1. 1
TEXT ANALYSIS TECHNIQUES TO
ANALYZE REVIEWS OF SAMSUNG
GALAXY MEGA 5.8 I9152
SUBMITTED BY
KOUSHIK RAKSHIT
ROLL NO:-A14034
2. 2
CONTENTS
1. Introduction------------------------------------------------------------------------ 3
2. Problem Statement---------------------------------------------------------------- 3
3. Key features------------------------------------------------------------------------ 3
4. Research Design------------------------------------------------------------------- 4
5. Research Methodology---------------------------------------------------------- 4
A. Insights from Web Crawling & Word Cloud--------------------------5
B. Latent Semantic Analysis (LSA) and Cluster Analysis--------------7
C. Reviews Ratings Analysis-----------------------------------------------14
D. Classification using Support Vector Machine(SVM)---------------14
E. Reviews Sentiment Analysis-------------------------------------------16
6. Business Perspective------------------------------------------------------------18
7. Appendix-------------------------------------------------------------------------19
3. 3
1. INTRODUCTION:- TEXT ANALYTICS
Text mining,referred to as text data mining, roughly equivalent to text analytics,
refers to the process of deriving high-quality information from text. High-quality
information is typically derived through the devising of patterns and trends through
means such as statistical pattern learning.
2. PROBLEM STATEMENT
ANALYZING REVIEWS FOR SAMSUNG GALAXY MEGA 5.8 I9152 (BLACK, WITH
BLACK)
From flipkart.com reviews for Samsung Galaxy Mega 5.8 I9152 (at least
100 reviews) were downloaded and a thorough analysis using text
analysis techniques was carried out.
3. KEY FEATURES OF SAMSUNG GALAXY MEGA 5.8 I9152
Wi-Fi Enabled
Expandable Storage Capacity of 64 GB
5.8-inch TFT Capacitive Touchscreen
Android v4.2.2 (Jelly Bean) OS
8 MP Primary Camera
1.9 MP Secondary Camera
1.4 GHz Dual Core Processor
Full HD Recording
4. 4
4. RESEARCH DESIGN
To analyze the user’s responses we had to collect primary and secondary
information from user’s mobile reviews from the website http://www.flipkart.com.
To analyze the user’s perception the about the phone we took 100 reviews from
the review section from flipkart.
5. RESEARCH METHODOLOGY
To analyze the user reviews following research analysis procedures where undertaken:
A. Web Crawling & Word Cloud
B. Latent Semantic Analysis (LSA) and Clustering Analysis
C. Rating Analysis
D. Classification Analysis using Support Vector Machine(SVM)
E. Reviews Sentiment Analysis
A. INSIGHT FROM WEB CRAWLING & WORDCLOUD
A tag cloud (word cloud, or weighted list in visual design) is a visual representation
for text data, typically used to depict keyword metadata (tags) on websites, or to
visualize free form text. Tags are usually single words, and the importance of each tag
is shown with font size or color. This format is useful for quickly perceiving the most
prominent terms and for locating a term alphabetically to determine its relative
prominence. When used as website navigation aids, the terms are hyperlinked to items
associated with the tag.
R packages used for Word cloud:-RCurl , XML , rvest , word cloud , tm
5. 5
1. Fetching reviews from FLIPKART.COM
FLIPKART<-"http://www.flipkart.com/samsung-galaxy-mega-5-8-i9152/product-
reviews/ITMEYFRTWAXZXTUT?pid=MOBDZSDJAPQXGAWN&type=all"
2.Word Cloud Creation
wordcloud(d$words,d$freq,max.words=300,colors=brewer.pal(8,"Dark2"),scale=c(3,0.5),
random.order=F)
INFERENCE DRAWN:- The word that took prominence in this Word Cloud gave a
clear idea that the mobile at the point of discussion is good & may be known for
screen size , display, camera ,battery . But it does not give proper idea if the product is
worth buying or users of the said mobile are satisfied or not. So to gain more insight
into our data we had to analyze their ratings (based out of 5).
B.LATENT SEMANTIC ANALYSIS AND CLUSTERANALYSIS:
For Latent Semantic Analysis, in which we break the term document matrix into 3 matrices:
6. 6
Word-Dimension Matrix
Documents dimension Matrix
Diagonal Matrix(Identity)
Word-Dimension Matrix:-
PLOTTING X1 vs X2
Inferences:
When we break the term document matrix into dimension-word vector space chart it is
clearly visible that the positive words like good, features like screen, battery etc, are
occurring mostly at dimension-1.
Grand, Mega , phone occurring mostly in dimension-2
Display, price, money, quality of phone are more or less occurring in both the
dimensions.
7. 7
PLOTTING X1 vs X3
Inferences:
Grand, Mega , phone occurring mostly in dimension-1
quality related words are more or less occurring in both the dimensions.
PLOTTING X2 vs X3
8. 8
Inferences:
Grand, Mega occurring mostly in dimension-1
Camera, samsung are more or less occurring in both the dimensions.
PLOTTING FOR DOCUMENT MATRIX
Inference:-
Document no. 71, 67 49 are close to dimension-1.
Document no.68 is close to dimension-2.
HIERARCHIAL CLUSTERING TO DETERMINE OPTIMUM NO. OF
CLUSTERS
In data mining, hierarchical clustering (also called hierarchical cluster analysis or
HCA) is a method of cluster analysis which seeks to build a hierarchy of clusters.
Strategies for hierarchical clustering generally fall into two types:
Agglomerative: This is a "bottom up" approach: each observation starts in its
own cluster, and pairs of clusters are merged as one moves up the hierarchy.
Divisive: This is a "top down" approach: all observations start in one cluster,
and splits are performed recursively as one moves down the hierarchy.
In general, the merges and splits are determined in a greedy manner. The
results of hierarchical clustering are usually presented in a dendrogram.
9. 9
As per the plot, optimum values could be either 3,4 or 5.
CLUSTER ANALYSIS:
From the above LSA analysis we got 5 optimum numbers of clusters,
which shows that there are 5 categories of reviews out of total 100
reviews.
So as if now we will concentrate on 5 review-clusters which are helpful to
club different types of reviews from the users.
CLUSTER-1
10. 10
There are total of 57 observations in this cluster.
This cluster consists of words related to price and the look of the phone
CLUSTER-2
There are 32 observations in this cluster
This cluster consists of words from reviews that are from customers who have good
experience with this mobile.
WORD CLOUD FOR CLUSTER-1
11. 11
CLUSTER-3
There are 38 observations in this cluster
This cluster consists of words from reviews that are from customers who have good faith in
the company.
WORD CLOUD FOR CLUSTER-2
12. 12
CLUSTER-4
There are only 2 observations in this cluster
This cluster doesnot throw any light on the nature of the cluster.
WORD CLOUD FOR CLUSTER-4
WORD CLOUD FOR CLUSTER-3
13. 13
CLUSTER-5
The above cluster has 1449 number of observations.
This cluster consists of words related to product features & quality.
WORD CLOUD FOR CLUSTER-5
14. 14
INFERENCE DRAWN FROM CLUSTERING
Apart from Cluster 1 , all other cluster does not give sufficient information about the
customer base/type. Moreover Cluster 2,3,4,5 is substantially smaller than Cluster 1
and no constructive storyline can be curved out of them.
Whereas Cluster 1 reflects almost anything & everything about the various features of
the phone that the customers might have liked.
C.REVIEWS RATINGS ANALYSIS
Total Reviews=100
Satisfied Reviews: 73
Dissatisfied Reviews: 27
Checking The Ratings Gives A Better Idea That Most Users Are Satisfied With
This Mobile.
D. CLASSIFICATION USING SUPPORT VECTOR MACHINE(SVM)
In machine learning, support vector machines (SVMs, also support vector
networks) are supervised learning models with associated learning algorithms
that analyze data and recognize patterns, used for classification and regression
analysis. Given a set of training examples, each marked as belonging to one of
two categories, an SVM training algorithm builds a model that assigns new
examples into one category or the other, making it a non-probabilistic binary
linear classifier. An SVM model is a representation of the examples as points in
space, mapped so that the examples of the separate categories are divided by a
clear gap that is as wide as possible. New examples are then mapped into that
same space and predicted to belong to a category based on which side of the
gap they fall on.
The generalization properties of an SVM do not depend on the dimensionality of
the space. You can bound the generalization error by a term depending on the
quotient of radius of a ball which contains all the data and the margin realized on
that data, but not on the dimensionality of the space. Many extensions exist, but
the answer is essentially the same: The generalization does not depend on the
15. 15
dimensionality.
An extended explanation is that you can generalize well even in high-dimensional
spaces because the data occupies only a low-dimensional subspace of the feature
space, and regularization results in the learner dealing only with that subspace.
You can see this for yourself if you look at the eigenvalues of the kernel matrix
which typically decay quickly, meaning that you can project your data to a low-
dimensional subspace with negligible error.
So even if you have, for example, a Gaussian kernel, where the feature space is
infinite-dimensional, you are actually dealing with an essentially finite dimensional
kernel feature space where you are learning a linear decision function, which is
statistically tractable. Note that you need to regularize, though.
From the above fig. we can infer that out of 100 data points , 95 data points contribute
to the formation of marginal plane .
6 words displayed at the
head has negative coefficient
6 words displayed at the tail
has positive coefficient
16. 16
Using SVM, we have classified reviews into two categories .
Since “dissatisfied” is the first level, the words with negative coefficient have positive
impact & vice versa
Snapshot of Data Frame containing list of words & their
frequency count
SENTIMENT ANALYSIS
Sentiment essentially relates to feelings; attitudes, emotions and opinions.
Sentiment Analysis refers to the practice of applying Natural Language
Processing and Text Analysis techniques to identify and extract subjective
information from a piece of text. A person‟s opinion or feelings are for the
most part subjective and not facts. Which means to accurately analyze an
individual‟s opinion or mood from a piece of text can be extremely difficult.
With Sentiment Analysis from a text analytics point of view, we are essentially
looking to get an understanding of the attitude of a writer with respect to a
17. 17
topic in a piece of text and its polarity; whether it‟s positive, negative or
neutral.
In recent years there has been a steady increase in interest from brands,
companies and researchers in Sentiment Analysis and its application to
business analytics.
The business world today, as is the case in many data analytics streams, is
looking for “business insight.”
● Installing „qdap‟ package
● We decided the threshold value for polarity to classify between satisfied
& dissatisfied on the basis of the plot in the next page.
● This tree plot was done using library “party”.
● Output of the popularity check or sentiment analysis gives the clear
message that 65% of the buyers are satisfied with the purchase of the
mobile
18. 18
CONCLUSION & BUSINESS PERSPECTIVE
• Output of our text analytics techniques brings out the fact that
Samsung Galaxy Mega 5.8 I9152 is a mobile worth buying.
• Most of the customers who bought it are extremely satisfied with the
various features it offers.
• Customer segmentation is possible but the very clear classification is
not possible as there are lot many features that are equally liked by all
across clusters.
• But of course buyers can be classified in terms of their satisfaction
level.
