Download as PDF, PPTX
Uploaded byWei-Ting Kuo
Sentiment analysis-by-nltk
This document discusses sentiment analysis using NLTK (Natural Language Toolkit) in Python. It begins with an overview of sentiment analysis and examples of determining sentiment from texts. Then it demonstrates the basics of using a sentiment dictionary to analyze sentences. It discusses challenges with real texts, like handling punctuation and splitting into sentences. NLTK tools for tokenization, sentence splitting, and counting positive and negative words are presented. Finally, it briefly introduces machine learning approaches to sentiment analysis using training data to build a model that can predict sentiment for new texts.
Sentiment analysis-by-nltk
- 1. Sentiment Analysis by NLTK Wei-TingKuo PyconApac2015
- 2.
- 3. Sentiment Analysis? Aim todetermine the attitude of a speaker/writer with respect to some text.
- 4.
- 5.
- 6. Positive or Negative? Thisis a good book! This is a good book! I like it! Positive The first chapter is good, but the rest is terrible Negative (more) Positive This is a bad book! Negative
- 7.
- 8. This is agood book! From scouter: This is a postive review
- 9. Why is sentiment analysisuseful?
- 10. • This isa terrible book. Because it’s important, so I mention three times, terrible, terrible, terrible!!!
- 11. Let’s begin withthe easiest way!
- 12. Sentiment Dictionary like 1 good2 bad -2 terrible -3
- 13. This is agood book! Dictionary like 1 good 2 bad -2 terrible -3 2 This is a good book! I like it! Positive 3 The first chapter is good, but the rest is terrible -1 Negative (more) Positive This is a bad book! -2 Negative
- 14. AFINN-111 • http://www2.imm.dtu.dk/pubdb/ views/publication_details.php? id=6010 • Alist of words rated between -5 (neg) to 5 (pos)
- 15. Let’s build thedictionary in Python
- 16. Let’s split thesentence first
- 17. And compute thescore •
- 18.
- 19. What if thetext is long? And have many punctuation? • Nice book! Though it is lack of advanced topics. It’s still good for beginners.
- 20.
- 22. NLTK to therescue • Natural Language ToolKit • Works with Python3!
- 23. Tokenization the process ofbreaking a stream of text up into words, phrases, symbols, or other meaningful elements called tokens
- 24.
- 25. It seems welose some information • Nice book! Though it is lack of advanced topics. It’s still good for beginners. Positive: 3
- 26. Separate to multiple sentencesfirst • Nice book! • Though it is lack of advanced topics. • It’s still good for beginners. Positive Negative Positive
- 27. How to splitthe text to sentences?
- 28.
- 29. Compute score for eachsentence
- 30. But we stillmiss some information in another case It’s a bad idea to buy this book. Negative Negative The first chapter is good, but the rest is terrible and confusing At least the customer mentioned something good, but it’s not recorded
- 31. Let’s count Pos& Neg separately The first chapter is good, but the rest is terrible and confusing It’s a bad idea to buy this book. Neg: -5 Neg: -3 Pos:3
- 32.
- 33. how about newwords? how about domain specific term?
- 34.
- 35. Traing Data (with Labels) Model TrainedModelReal Data Prediction
- 36. Training Data • Thisis a good book! Postive • This is a awesome book! Postive • This is a bad book! Negative • This is a terrible book! Negative
- 37. • This isa good book! Postive • This is a awesome book! Postive • This is a bad book! Negative • This is a terrible book Negative Model This is a good article This is a bad article Trained Model Postive Negative
- 38.
- 39.
- 40. Build the model,and train it!
- 41. Real Case • MovieReview Data http://www.cs.cornell.edu/people/pabo/movie-review-data/ • 5331 positive reviews & 5331 negative reviews labelled by human.
- 42.
- 43.
- 44.
- 45. Separate the data TrainingData Testing Data Training data to train the model Testing data to compute the accuracy
- 46.
- 47.
- 48. Compute the accuracy accuracy= number of correct / total
- 49. How to enhance? •Use the most frequent 1000 words only • Use different model, maybe SVC • Read more paper about the latest research
- 50.