• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Semi Automatic Sentiment Analysis
 

Semi Automatic Sentiment Analysis

on

  • 1,652 views

Web 2.0 applications for social networking provide data about users’ mood and opinions in almost real time. Many applications are taking advantage of these data to derive business intelligence. ...

Web 2.0 applications for social networking provide data about users’ mood and opinions in almost real time. Many applications are taking advantage of these data to derive business intelligence. However, the volume of data makes it hard and error-prone to classify sentiments and opinions manually. The combination of data mining techniques and a pipeline to process data from Web 2.0 applications, such as Twitter, Facebook, and Wordpress, makes it possible to apply natural language processing and machine learning techniques to automate partially this task. Therefore, the amount of manual classification is reduced, as the incoming data has already a classification tag that can be easily changed, feeding back the classifier. There is room for improvements and a Brazilian Portuguese Corpus was created to do the initial training of the classifier. The code used for this testing was based on open source libraries and is available as a test bed for different corpora and new algorithms.

Statistics

Views

Total Views
1,652
Views on SlideShare
1,453
Embed Views
199

Actions

Likes
2
Downloads
14
Comments
0

2 Embeds 199

http://www.scoop.it 198
http://webcache.googleusercontent.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Semi Automatic Sentiment Analysis Semi Automatic Sentiment Analysis Presentation Transcript

    • Semi Automatic Sentiment AnalysisResults from a case study in Brazilian Portuguese web 2.0 sites Gleicon Moraes, Marco Aurélio Gerosa gleicon@gmail.com, gerosa@ime.usp.br
    • Introduction•  Popular Web 2.0 applications are based on social networking: Facebook, Twitter, Orkut, Flickr, LinkedIn•  Status messages, user information, wall posts, like/unline votes, scraps, recommendations are created and exchanged between users.•  Symetric and Assymetric relationships broadcast these messages between friends (direct connections) and friends of friends.•  Sentiment and opinions might be objective (up/down votes, recommendations) or subjective (free text) Gleicon Moraes, Marco Aurélio Gerosa 2/20
    • Sentiment Classification•  Find out what users in a social network think about product, tendency or brand.•  Compute or help to compute the Return of Investment of a marketing campaign•  Create or compose product and services recommendations to other users•  To measure user satisfaction and experience about a service. Gleicon Moraes, Marco Aurélio Gerosa 3/20
    • Goals•  Opinion mining / subjectivity and sentiment analysis review [1]•  Automate opinion classification (tweet, scrap, message, wall post) using Machine Learning and Information Retrieval techniques.•  To apply a Bayesian filter (and also try a SVM classifier) to identify Positive and Negative sentiment on brazilian Portuguese texts.•  To build a corpus to train and test the classifiers•  To find out how to measure the filter efficiency.[1] Pang e Lee - Opinion Mining and Sentiment Analysis Gleicon Moraes, Marco Aurélio Gerosa 4/20
    • Related work•  Thumbs Up? Sentiment Classification using Machine Learning Techniques: Bayesian filter, Maximum Entropy filter and SVM filter. Training was made with Movielens dataset, splitting between 70% corpus to training and 30% to test. This corpus is already marked as positive and negative. Conclusion was that sarcasm on opinions made it difficult to classify the sentiments. There was no smaller text classification (e.g. A tweet/140 chars) and feedback with outside text to the classifiers. [1][1] Pang, B. Lee L., Cornell University, Vaithyanathan S, IBM: Thumbs Up? Sentiment Classification using Machine LearningTechniques Gleicon Moraes, Marco Aurélio Gerosa 5/20
    • Related work•  Content-based book Recommendation Using Learning for Text Categorization and information extracted from the internet to train a classifier, with a database per user. The combination between collaborative filtering and content filtering complete each other and help improve the results. [1][2][1] Mooney R. J., Roy L., “Content-Based Book Recommendation Using Learning for Text Categorization” (Proceedings ofACM Conference on Digital Libraries, 2000)[2] Dˇzeroski S., Zenko B. “Is Combining Classifiers Better than Selecting the Best One?” Gleicon Moraes, Marco Aurélio Gerosa 6/20
    • Semi-Automatic Sentiment Classification•  Trained Bayesian Filter on two categories: “positive” and “negative”•  Feedback feature so false positives and false negatives could be trained back to improve the filter•  Problem: There is not a brazilian portuguese data matching text to sentiment to do the initial classificator training.•  Problem: Text composition varies between social networks and groups within these networks. Feeding back data to keep the classificator database updated is fundamental Gleicon Moraes, Marco Aurélio Gerosa 7/20
    • Semi-Automatic Sentiment Classification•  English language training corpus uses movie reviews in most papers, associated with ratings to tell what that text block express [1]•  An initial training corpus was made using consumer review data from Brazilian websites like iVox, ReclameAqui, opiniões do MercadoLivre•  After scrapping each opinion and its rating (stars, rating, or positive/negative indication), stored it on folders ranging from 0.0 to 5.0, each opinion a file inside the proper folder[1] MovieLens dataset: http://www.grouplens.org/node/73 Gleicon Moraes, Marco Aurélio Gerosa 8/20
    • Training composition findings•  Number of words in negative opinions is bigger than on positive opinions: 67.575 words in 712 positive opinions versus 81.747 words in 507 negative opinions.•  Distribution of reviews between minimum and maximum ratings: more opinions on the extremes (0.0 to 0.5 and 4.5 to 5.0). Gleicon Moraes, Marco Aurélio Gerosa 9/20
    • Composição da base de treinamento - iVox Gleicon Moraes, Marco Aurélio Gerosa 10/22
    • Domain•  Language domain varies between communities/sites Gleicon Moraes, Marco Aurélio Gerosa 11/20
    • Opinion Sample (Mercadolivre)positivo (rating 5):"Este alto-falante faz o baile tremer... comprei para montar uma mini-saveiro”negativo (rating 1):"Apesar de custar muito barato recomendo economizar e comprarfalantes de marcas conhecidas. Bravox, Selenium.O produto parece recondicionado, e não tem 90Wats nem na china,meu triaxial Pionner de 60Wats aquenta muito mais grave que esseUnlike.Não faça besteira economize mais R$60,00 e compre um Kit 2 viasSelenium ou até Sony ou Bomber que custa quase o mesmo aqui noMercado Livre" Gleicon Moraes, Marco Aurélio Gerosa 12/20
    • Opinion Sample (iVox)positivo (rating 5.0):"Economica não tem Adquiri uma web.evo Sundown,à moto é bonita,gostei tantoda Sundown que adquiri mais uma moto Sundow a hunter 90cc. estou com 2motos e estou muito satisfeito. Quanto ao pessoal da grappa, todos sem exceçãosempre bem atenciósos comigo; só tenho a agradecer. "negativo (rating 1):"Contra Todas Não sei o motivo de sua defesa a esta empresa, pois fuienganado a pouco tempo e o engraçado é que liguei para reclamar,bem na hora que o vendedor estava enganando outra pessoa, por umdeslize do mesmo o cliente verificou o numero e me ligou dizendo quetambém havia sido enganado. Entramos com denúncia conjunta naDECON do DF. Razoável Muito Ruim Razoável Muito Ruim" Gleicon Moraes, Marco Aurélio Gerosa 13/20
    • Opinion Sample (Reclame Aqui)positivo:"Olá, estou passando apenas para parabenizar ao ótimo e sério trabalho daequipe do site reclameaqui.com.br, pois já fui atendido em duas ocasiõesreclamadas no site e foi algo bem melhor e mais rápido do que partir para outrasatitudes. Parabéns e que cada vez mais possamos ter meios iguais parapodermos agilizar o processo de negociação.Obrigado,"negativo:"Fiz 2 reclamações contra a MOTOROLA DO BRASIL por propagandaenganosa em seu site www.motorola.com.br sobre o aparelho V3m queno site diz ACOPMPANHA cartão enquanto no meu aparelho nao veioNADA !!! Eles me ligaram e tiram o deles da reta dizendo que a culpa éda VIVO ! MAis perai quem faz o aparelho nao é eles ??? A VIVO sorevende !!!! Ah MOTOROLA POR FAVOR NE !!!!! QUERO MEUCARTAO !!!" Gleicon Moraes, Marco Aurélio Gerosa 14/20
    • Domain•  Language Domain [1]: "go read a book” has different meaning related to each social network. In a book related network might be a positive meaning. In others might mean a negative sentiment.•  Feeding back data also helps to keep the database updated with new slangs and combinations that also might cover sarcasm expressions.•  Events like world cup and television shows might introduce new words and expressions.[1] Pang e Lee - Opinion Mining and Sentiment Analysis Gleicon Moraes, Marco Aurélio Gerosa 15/20
    • Training•  Split the database between negative (rating: 0.0) and positive (rating 5.0). Later steps added ratings 4.5, 4.0 to positive while negative rating kept the same.•  Training/Classifying applied on raw data and on data processed a pipeline of taking out stop words and extracting the stem of remaining words•  Raw data biased towards negative sentiment, processed data biased towards positive sentiments. Gleicon Moraes, Marco Aurélio Gerosa 16/20
    • Results – raw data iVox ReclameAqui False results Ratings Negative/Positive Negative Positive Negative Positive No training No messages 1635 268 0 0 0.0 e 5.0 506/720 1634 6 262 1 0.0 e 4.5 + 5.0 506/873 1587 99 169 48 0.0 e 4.0 + 4.5 + 5.0 506/973 1365 165 105 270 Gleicon Moraes, Marco Aurélio Gerosa 17/20
    • Results – filtered data iVox ReclameAqui False results Ratings Negative/Positive Negative Positive Negative Positive No training No messages 1635 268 0 0 0.0 and 5.0 506/720 1635 0 268 0 0.0 and 4.5 + 5.0 506/873 0 261 0 1627 0.0 and 4.0 + 4.5 + 5.0 506/973 0 268 0 1635 Gleicon Moraes, Marco Aurélio Gerosa 18/20
    • Measuring efficiency•  Metrics: Accuracy, Precision Recall•  Token extraction: words (bag of words) and bigrams.•  Test between languages and domain: trained and tested the same classifiers and extractors with the Movielens dataset [1][1] The MovieLens dataset: http://www.grouplens.org/node/73 Gleicon Moraes, Marco Aurélio Gerosa 19/20
    • EfficiencyMovie Review (en) Feature Extractor Accuracy Positive Negative Positive Negative Precision Precision Recall Recall Bag of Words 0.7280 0.6516 0.9597 0.9800 0.4760 Bigrams 0.8240 0.7613 0.9263 0.9440 0.7040Consumer Opinion (pt_br) Feature Extractor Accuracy Positive Negative Positive Negative Precision Precision Recall Recall Bag of Words 0.5984 1.0000 0.5100 0.3099 1.000 Bigrams 0.7049 1.0000 0.5862 0.4930 1.000 Gleicon Moraes, Marco Aurélio Gerosa 20/20
    • Conclusion•  Consumer review database helped on initial training.•  O keep the messages as is helps makes the database richer with different forms of the same expression•  Token extraction influences the end result•  Feeding back helps to keep the database up to date•  To combine classifiers helps the end results and the precision•  Contribution: Brazilian portuguese database and scripts used to extract data and to reproduce the experiment at: https://github.com/gleicon/ sentiment_analysis Gleicon Moraes, Marco Aurélio Gerosa 21/20