1. Sentiment Analysis
by NLTK
Wei-Ting Kuo
PyconApac2015

2. http://goo.gl/wJelD4

3. Sentiment Analysis?
Aim to determine the attitude of a
speaker/writer with respect to some text.

4. Amazon’s Customer Review

5. Twitter Follower’s replies

6. Positive or Negative?
This is a good book!
This is a good book!
I like it!
Positive
The first chapter is good,
but the rest is terrible
Negative
(more)
Positive
This is a bad book! Negative

7. How to compute it?

8. This is a good book!
From scouter: This is a postive review

9. Why is sentiment
analysis useful?

10. • This is a terrible book. Because it’s important, so I
mention three times, terrible, terrible, terrible!!!

11. Let’s begin with the
easiest way!

12. Sentiment Dictionary
like 1
good 2
bad -2
terrible -3

13. This is a good book!
Dictionary
like 1
good 2
bad -2
terrible -3
2
This is a good book!
I like it!
Positive
3
The first chapter is good,
but the rest is terrible
-1 Negative
(more)
Positive
This is a bad book! -2 Negative

14. AFINN-111
• http://www2.imm.dtu.dk/pubdb/
views/publication_details.php?
id=6010
• A list of words rated between
-5 (neg) to 5 (pos)

15. Let’s build the dictionary
in Python

16. Let’s split the sentence first

17. And compute the score
•

18. Recap

19. What if the text is long?
And have many punctuation?
• Nice book! Though it is lack of advanced topics.
It’s still good for beginners.

20. Doesn’t work!

22. NLTK to the rescue
• Natural Language ToolKit
• Works with Python3!

23. Tokenization
the process of breaking a stream of text up into
words, phrases, symbols, or other meaningful
elements called tokens

24. Word tokenize

25. It seems we lose some
information
• Nice book! Though it is lack of advanced
topics. It’s still good for beginners.
Positive: 3

26. Separate to multiple
sentences first
• Nice book!
• Though it is lack of advanced topics.
• It’s still good for beginners.
Positive
Negative
Positive

27. How to split the text to
sentences?

28. Sentence tokenize

29. Compute score for
each sentence

30. But we still miss some
information in another case
It’s a bad idea to buy this book.
Negative
Negative
The first chapter is good,
but the rest is terrible and confusing
At least the customer mentioned something good,
but it’s not recorded

31. Let’s count Pos & Neg
separately
The first chapter is good,
but the rest is terrible and confusing
It’s a bad idea to buy this book.
Neg: -5
Neg: -3
Pos:3

32. In Python

33. how about new words?
how about domain specific term?

34. Machine Learning!!!

35. Traing Data
(with Labels)
Model
Trained ModelReal Data Prediction

36. Training Data
• This is a good book! Postive
• This is a awesome book! Postive
• This is a bad book! Negative
• This is a terrible book! Negative

37. • This is a good book! Postive
• This is a awesome book! Postive
• This is a bad book! Negative
• This is a terrible book Negative
Model
This is a good article
This is a bad article
Trained Model
Postive
Negative

38. The format NLTK use

39. Prepare the Training Set

40. Build the model, and train it!

41. Real Case
• Movie Review Data
http://www.cs.cornell.edu/people/pabo/movie-review-data/
• 5331 positive reviews & 5331 negative reviews
labelled by human.

42. Positive Reviews

43. Negative Reviews

44. Read our data

45. Separate the data
Training Data Testing Data
Training data to train the model
Testing data to compute the accuracy

46. Separate our data

47. Train the data

48. Compute the accuracy
accuracy = number of correct / total

49. How to enhance?
• Use the most frequent 1000 words only
• Use different model, maybe SVC
• Read more paper about the latest research

50. Q & A