Machine Learning Prague 2016 www.mlprague.com

1. Machine Learning Approaches
to
Sentiment Analysis
Kateřina Veselovská
Ataccama/Matfyz
@kveselovska

3. IDC: “Companies that can figure out a way to properly collect,
synthetize and use unstructured data can improve their bottom
line, reduce costs and help organizations respond more quickly
to changing market and customer sentiment.”

4. Good quality!
Nice price.
Holy crap!!!
It’s @#*$%!

5. Black ivory coffee is made from
elephant dung.
Black ivory coffee is the best.

6. Black ivory coffee is the best.
Black ivory coffee is a crap.

7. [Black ivory coffee is the best.] +
[Black ivory coffee is a crap.] -

9. I find the famous poop coffee disgusting.

10. • unsupervised learning
• Turney’s three steps algorithm
• LDA
• supervised learning
• classifier (non-polar x polar, pos/neg)
• regression

12. Kappa = 0.66

13. “Every time I fire a linguist, the
performance of our system goes up.”

14. • preprocessing:
lemmatization, desambiguation
• sentence-level classifiers
• bag-of-words model
• word-based features
• various filters

15. • frequency, statistical significance
• selected parts of speech only
• negation (reverse polarity if verb is negative)
• for specified POS
• for whole segment or only after verb
• can plug in an external lexicon

16. Classifier 1
• features: lemma presence in a segment
Clasisifier 2
• features: lemma presence in a segment
+ rules

17. • both classifiers estimate the predictive
power of individual lemmas w.r.t.
polarity class
• training ~ building a lexicon of all present
lemmas and their predictive power

18. Model Acc Recall Precision F-score
Baseline 63.0 37.0 23.3 28.6
NB + rules 88.9 88.9 89.4 89.0
NB 82.7 74.5 84.7 78.1

19. I love black ivory coffee, but civet coffee is horrible.
?

20. Data:
•marked on sentiment targets
Pipeline:
• preprocessing
• marking known-aspects
• morpho-tagging + parsing
• syntactic rules

21. • linear-chain conditional random fields
• features:
a) surface – surface form of the word +
2 preceding and 2 following
b) morpho-syntactic – uni-bi-trigram around +
lemma, morpho tag,
analytical function
c) subjectivity lexicon
d) rule features

22. Feature set Short segments
P R F
Long segments
P R F
surface 85.2 36.9 51.5 47.2 8.0 13.8
+ morpho-syntactic 75.9 54.2 63.2 40.2 23.1 29.3
+ sublex 78.2 55.2 64.6 58.7 18.1 28.7
+ rules 76.5 57.7 65.8 51.7 21.4 30.3

23. • language-independent approaches: neural
networks
• tried on seven languages
• recurrent neural networks with long-short term
memory cells
• pre-trained embeddings, Wikipedia dump
• comparison with logistic regression model

24. Domain Restaurants Hotels Laptops
Language English Spanish French Dutch Russian Turkish Arabic English
Network 59.3 58.8 49.9 53.9 64.8 61.0 47.3 27.0
Baseline 58.0 62.2 54.8 54.7 60.8 34.4 49.4 35.0

25. system errors
x
human annotator errors

26. Nothing. Price. I don’t know.
Quality.
Long washing programs.

27. I cannot review this, I sent the goods back since
it was damaged.
I don’t know how I could have lived without the
dryer. Only those who don’t have it defame it, or
the turned down wives whose husbands don’t
want to buy it for them. It saves time, some
things still need to be ironed, but very little. I
dried the bed linen during Saturday, including
the mattress and bed linen cases (pillows,
blankets) and I still had plenty of time.

28. It is not good.

29. It is not good, but I still like it.

30. She is pretty annoying.

31. He is one sandwich short of a picnic.

33. *GREEEEEAT*!!!:-DD

34. OMG, she is such a @#%$!!!

35. Go read the book!

36. USA: Awesome!
SWE: Inget speciellt

39. Special thanks:
Aleš Tamchyna
Jan Hajič, jr.