https://github.com/morris821028/NLP-SentimentClassification

1. Sentiment Classification
N-grams as Linguistic Features

2. Why high order n-gram ?
• Negative views
• “highly recommend staying away …“
• Positive views
• “recommend …“
• “highly recommend …“
• Fuzzy n-gram
• In addition, while it is difficult to model positive and
negative expressions by lexico-syntactic patterns due to
extreme variety.

3. Make helpful
high order n-gram
• n-gram violates the independence ?
• Composite model
combining unigrams and bigrams gives much higher
performance than using only bigrams.
• Classifier Definition
The classifiers we employ do not require independent
features.

4. Make helpful
high order n-gram
• Reduce n-gram data
• reduce computational complexity
• offline operation

5. Method – pick features (1)
• term t
• class c
• A be the number of times t and c co-occur.
• B be the number of times t occurs without c.
• C be the number of times c occurs without t.
• D be the number of times neither t nor c occurs,
• N be the number of documents.

6. Method – pick features (2)
Features Meaning
A + C ↑ ↓ Class c is large, dilute features
Ｂ +Ｄ ↑ ↓ Without c is large, class c may not important
Ａ +Ｂ ↑ ↓ Item t in more class
C + D ↑ ↓ Item t not in more class
AD ↑ ↑ More features in class c (frequently appear)
CB ↑ ↑ More features in class c (rare)

7. Method – pick features (3)
• Take top M ranked n-grams as features in the
classification experiments.
• Example. (in positive comments)
• Score 0.517334 (of the best)
• Score 0.325458 (as well as)
• Score 0.200934 (lot of fun)
• Score 0.197970 (nice to see)
• … ignore
• w(0, …, 0) = w(`of the best`, `as well as`, …)
w(1, …, 0) mean which comment appears `of the best`
• erase n-gram record which not in top M ranked n-grams in
Language Model.

8. Experiment – Simple
• Up to Down
• LM
• PA
• Winnow
•Training 400
•Testing 400

9. Experiment – Observe
• Class `Negative`
• LM performance depend strongly `training data` P = 10%~90%
Table `Language Model Class Negative`
|TruthClassifier| Classifier no| Classifier yes|
| ------| ------| ------|
| Truth no| 400| 0|
| Truth yes| 360| 40|
Table `Passive-Aggressive Class Negative`
|TruthClassifier| Classifier no| Classifier yes|
| ------| ------| ------|
| Truth no| 289| 111|
| Truth yes| 54| 346|

10. Experiment – Three-Ways
• 三個不同的模型進行投票
• 根據其信任度
PA ≒ LM > Winnow
• 三者持相同意見
• PA 持不同意見於 LM 和 Winnow
LM: POS Winnow: POS
PA: POS
LM: NEG Winnow: NEG
PA: NEG
LM: NEG Winnow: NEG
PA: POS
Occur 55 % ↑ , Precision 92% ↑
If LM-option = Winnow-option
final-option = LM-option
Else
final-option = PA-option
Precision 72% ↑

11. Experiment – mix n-grams
• Three-Ways Online
• Precision 71% - 83%
• Training 400 items
• Testing 400 items
• Distinct n-grams = n × 100K
• Top M = 10000
• features ratio < 10%
• If performance PA ≒ LM, increasing precision 2% ↑.

12. Experiment – LM filter
• When Language Model testing
• Remove objective sentence by Language Model predict
function.
• “it's a comedy , and teenagers have little clout , but for
my money, …”
• If Predict(sentence) < threshold, then remove it.
• Not helpful, Precision ↓

13. Experiment – Weight Vector
• When using Passive-Aggressive and Winnow Algorithm
• AFINN-111.txt
• Score(n-grams) = sum weight(w_{i})
• Robustness ↑
AFINN-111.txt
abhors -3
abilities 2
ability 2
aboard 1
absentee -1
absentees -1
absolve 2
absolved 2
…