Recognizing Strong and Weak
Opinion Clauses
Lucas Rizoli
2007-11-22
CPSC 503

Wilson, Wiebe, & Hwa, 2006
Recognizing Strong and Weak Opinion Clauses

Motivation
Opinion is more than binary
Intensity of expression
How negative or positive?
“I'm fond of you” vs. “I love you”
Many applications
Surveillance, marketing, retrieval...

Subjective expressions
Surface-level phrases
Express an internal state
Opinion, emotion, sentiment, belief, etc.
Can describe state directly
“I love the taste of Zima”

Sometimes indirect
“Rahul is so full of crap”
Called an expressive subjective element
Physical manifestation
“Mayukh applauded the stripper”
Called a private state action
Vary in intensity
neutral; low, medium, high, extreme

Annotation
3 annotators; ~10 000 sentences
Who said it or expressed it
“Sudeep said Sheelagh is a laugh”
Objective and subjective
“Sudeep said Sheelagh is a laugh”

Direct subjective element
Source, span, intensity, expression intensity,
implicit
Expressive Subjective element
Source, span, intensity
Objective speech event element
Source, span, implicit

Direct subjective element
Source, span, intensity, expression intensity,
implicit
Expressive Subjective element
Source, span, intensity
Objective speech event element
Source, span, implicit

Annotation notes
Differing spans
“Edwin is a silly canker-sore of a man.”
“Edwin is a silly canker-sore of a man.”
Considered any overlap as agreement
Agreement on sub/objective elements
82%, 72% (Wiebe et al., 2005)

Agreement on intensity
Direct & objective frames
Intensity 0.79 (75%)
Expression intensity 0.75 (62%)
Subjective frames
Intensity 0.46 (53%)
Used Krippendorff's α

Subjective clues
Prev lexicon and rules
Verb classes, FrameNet, adjective sets, etc.
Culled from literature & experience
Syntax rules
From dependency representation
Part-of-speech in/sensitive

Clue groups
Type grouping
29 Prev groups, by source
15 Syntax groups, by class and reliability
Intensity grouping
Grouped by P(intensity)
Groups as classifier features

Classification
Mean squared error, not accuracy
Allows partially correct labeling
Many classifiers
Boosting, rules, SVM
Clause-level intensity labels
Highest intensity in a clause

Medium
Medium
Neutral

Boosting
Bag-of-words + Intensity
StdError 0.99–1.211
Rules
Intensity
StdError 0.99–1.21
SVM
Bag-of-words + Type or Intensity
StdError 0.75–1.07

Easier to classify high-level clauses
More information, training examples
Boosting & Rules
Off by 1 degree of intensity
SVM
Ordinal classifier
Better by StdError, not Accuracy

Results
SVM performs best
Intensity and Type groups work best
Syntax groups detrimental to all classifiers