What Business Innovators Need to Know about Sentiment Analysis, Claire Cardie
Upcoming SlideShare
Loading in...5
×
 

What Business Innovators Need to Know about Sentiment Analysis, Claire Cardie

on

  • 2,142 views

What Business Innovators Need to Know about Sentiment Analysis, presented by Prof. Claire Cardie of Cornell University.

What Business Innovators Need to Know about Sentiment Analysis, presented by Prof. Claire Cardie of Cornell University.

Statistics

Views

Total Views
2,142
Views on SlideShare
2,137
Embed Views
5

Actions

Likes
2
Downloads
92
Comments
0

1 Embed 5

http://www.slideshare.net 5

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

What Business Innovators Need to Know about Sentiment Analysis, Claire Cardie What Business Innovators Need to Know about Sentiment Analysis, Claire Cardie Presentation Transcript

  • What Business Innovators Need to Know about Sentiment Analysis Claire Cardie Department of Computer Science Chair, Information Science Department Cornell University Co-founder Chief Scientist
  • Plan for the Talk Subjectivity and sentiment in language Continuum of capabilities – Surface-level in-depth understanding – Document-level phrase-level Next steps…
  • Subjective Language Subjective text expresses speculations, beliefs, emotions, evaluations, goals, opinions, judgments, … • Jill said, "I hate Bill." • John thought about whom to vote for. • Seth knew his symposium would go well.
  • Subjectivity vs. Sentiment Sentiment-bearing text expresses positive and negative speculations, beliefs, emotions, evaluations, goals, opinions, judgments,… • Jill said, "I hate Bill." - • John thought about whom to vote for. ~ • Seth knew his symposium would go well. + sentiment analysis tome [Pang & Lee, 2008]
  • A Word on Polarity (tone, valence) Positive “I love NY.” Negative “I hate NY.” Neither positive nor negative – Objective? “I thought about NY.” – Neutral? “I’m ambivalent about NY.” – Mixed polarity? “Sometimes I love NY; other times I hate it.”
  • And What About Intensity? Strength/intensity “I love NY.” “I absolutely adore NY!” – Low, medium, high, very high, extreme – ratings – rotten tomatoes
  • Plan for the Talk Subjectivity and sentiment in language Continuum of capabilities – Surface-level in-depth understanding – Document-level phrase-level Next steps…
  • Document-level Sentiment Analysis Is the overall sentiment in the Document document positive? negative? neutral?
  • Identifying Tone of a Collection Sentiment (w.r.t. a topic) – Example: Tone on “economic stimulus”
  • Detecting “chatter” or “buzz” Chatter (w.r.t. a topic) – Example: Buzz on “economic stimulus”
  • Keyword-based Approaches Search the text for the presence of specific terms from a manually created “sentiment lexicon” – +: “great”, “praise”, “peace”, “superb”, … – -: “war”, “dull”, “messy”, “criticize”, … Sentiment is based on the counts – E.g., If more positive terms than negative terms, then return +, else return –
  • Keyword-based Approaches Complications – Inherent ambiguities of language… – This laptop is a great deal. – A great deal of media attention surrounded the release of the new laptop model. – If you think this laptop is a great deal, I’ve got a nice bridge for you to buy. [Examples from Lillian Lee] [Pang & Lee, 2008]
  • Machine-learning Approaches Learn from training data Are better able to take advantage of context to disambiguate terms examples ML Algorithm statistical model (novel) examples class (program)
  • Measuring Performance Precision: #correct / #attempted Recall: #correct / #possible F-measure: harmonic mean of P and R 1. _______ P = 3 / 4 = .75 2. _______ P = 3 / 3 = 1.00 3. _______ R = 3 / 4 = .75 4. _______ accuracy
  • Measuring Performance How well do document-level sentiment analysis systems work? It depends… – Product reviews easier than Movie reviews, easier than News/editorials – Shorter documents harder than longer ones – Messy documents harder than clean ones ~75 F - ~85 F
  • This is actually quite good… Comparison is not vs. 100% P/R…but vs. human sentiment analysis accuracy – Cohen’s kappa Machine-learning methods for sentiment analysis approach human agreement levels – ~85 F: for positive/negative – ~75 F: when neutrals are included
  • Sentiment Analysis at Passage Level Passage tone The suggestion that the White House never took seriously an – Optionally w.r.t. a issue that infuriated millions of topic Americans was supported by – E.g., AIG or Geithner Senator Robert Menendez, a New Jersey Democrat who claimed that several weeks earlier he warned Timothy Geithner, the Treasury secretary, that AIG was planning to use taxpayer funds to pay out $165m in bonuses… speculation that Obama will have to replace him, despite the president’s insistence to Leno that Geithner is doing "an outstanding job“.
  • Sentiment Analysis at Phrase Level Fine-grained opinion analysis Identify who is saying what about what
  • Fine-Grained Sentiment Extraction The suggestion that the White House never took seriously an issue that infuriated millions of Americans was supported by Senator Robert Menendez, a New Jersey Democrat who claimed that several weeks earlier he warned Timothy Geithner, the Treasury secretary, that AIG was planning to use taxpayer funds to pay out $165m in bonuses… speculation that Obama will have to replace him, despite the president’s insistence to Leno that Geithner is doing "an outstanding job".
  • Fine-Grained Sentiment Extraction …the president insisted to Leno that Geithner is doing "an outstanding job". – Opinion trigger – Polarity Opinion Frame – Intensity Polarity: positive – Opinion holder Intensity: high Opinion Holder: “the president” – Target (topic) Target: “Geithner”
  • Example – fine-grained opinions opinion frame opinion frame opinion frame opinion frame opinion frame opinion frame The suggestion that the White House never took seriously an issue that infuriated millions of Americans was supported by Senator Robert Menendez, a New Jersey Democrat who claimed that several weeks earlier he warned Timothy Geithner, the Treasury secretary, that AIG was planning to use taxpayer funds to pay out $165m in bonuses…the president insisted to Leno that Geithner is doing "an outstanding job". opinion frame
  • Example – Opinion Summary AIG Obama Geithner Americans Menendez
  • Example – Opinion Summary Summarize thoughts and views across documents – Critical addition: opinion holder AIG
  • What makes this hard? Same issues of ambiguity as before plus… Need to associate opinion with topic and with opinion holder Requires different machine learning methods Requires many language-processing modules
  • Noun Phrase Coreference Resolution The suggestion that the White House never took seriously an issue that infuriated millions of Americans was supported by Senator Robert Menendez, a New Jersey Democrat who claimed that several weeks earlier he warned Timothy Geithner, the Treasury secretary, that AIG was planning to use taxpayer funds to pay out $165m in bonuses…speculation that Obama will have to replace Geithner, despite the president’s insistence to Leno that he is doing "an outstanding job". Ng & Cardie [2002, 2003]; Stoyanov & Cardie [2006, 2008]
  • Performance 82F opinion OH 79F extraction extractor 69F link classifier –<opinion holder> expresses <opinion> Choi, Breck & Cardie [2006, 2007]
  • Plan for the Talk Subjectivity and sentiment in language Continuum of capabilities – Surface-level in-depth understanding – Document-level phrase-level Next steps…
  • Next Steps… Predicting business outcomes from opinions – Doable in some settings Determining the key influencers
  • Thank you! Questions?