Your SlideShare is downloading. ×
0
What Business Innovators Need to
 Know about Sentiment Analysis
                Claire Cardie

        Department of Compu...
Plan for the Talk
Subjectivity and sentiment in language
Continuum of capabilities
– Surface-level  in-depth understanding...
Subjective Language
Subjective text expresses speculations,
beliefs, emotions, evaluations, goals,
opinions, judgments, …
...
Subjectivity vs. Sentiment
Sentiment-bearing text expresses
positive and negative speculations, beliefs,
emotions, evaluat...
A Word on Polarity (tone, valence)
Positive “I love NY.”
Negative “I hate NY.”

Neither positive nor negative
– Objective?...
And What About Intensity?
Strength/intensity

        “I love NY.”

        “I absolutely adore NY!”

– Low, medium, high,...
Plan for the Talk
Subjectivity and sentiment in language
Continuum of capabilities
– Surface-level  in-depth understanding...
Document-level Sentiment Analysis


            Is the overall
            sentiment in the
Document    document
         ...
Identifying Tone of a Collection

Sentiment (w.r.t. a topic)
– Example: Tone on “economic stimulus”
Detecting “chatter” or “buzz”

Chatter (w.r.t. a topic)
– Example: Buzz on “economic stimulus”
Keyword-based Approaches
Search the text for the presence of
specific terms from a manually created
“sentiment lexicon”
– ...
Keyword-based Approaches
Complications
– Inherent ambiguities of language…

– This laptop is a great deal.
– A great deal ...
Machine-learning Approaches
  Learn from training data
  Are better able to take advantage of
  context to disambiguate te...
Measuring Performance
Precision: #correct / #attempted
Recall:    #correct / #possible
F-measure: harmonic mean of P and R...
Measuring Performance
 How well do document-level sentiment
 analysis systems work?

It depends…
  – Product reviews easie...
This is actually quite good…
Comparison is not vs. 100% P/R…but vs.
human sentiment analysis accuracy
– Cohen’s kappa
Mach...
Sentiment Analysis at Passage Level

Passage tone              The suggestion that the White
                          Hou...
Sentiment Analysis at Phrase Level
Fine-grained opinion analysis
Identify who is saying what about what
Fine-Grained Sentiment Extraction



The suggestion that the White House never took
seriously an issue that infuriated mil...
Fine-Grained Sentiment Extraction
…the president insisted to Leno that Geithner is doing "an
outstanding job".


–   Opini...
Example – fine-grained opinions
opinion frame
                                         opinion frame
                 opin...
Example – Opinion Summary


                         AIG

Obama

             Geithner




             Americans
Menendez
Example – Opinion Summary
Summarize thoughts and views across
documents
– Critical addition: opinion holder




          ...
What makes this hard?
Same issues of ambiguity as before plus…
Need to associate opinion with topic and
with opinion holde...
Noun Phrase Coreference Resolution


The suggestion that the White House never took
seriously an issue that infuriated mil...
Performance


82F    opinion                     OH           79F
      extraction                extractor




         6...
Plan for the Talk
Subjectivity and sentiment in language
Continuum of capabilities
– Surface-level  in-depth understanding...
Next Steps…
Predicting business outcomes from
opinions
– Doable in some settings
Determining the key influencers
Thank you!


Questions?
What Business Innovators Need to Know about Sentiment Analysis, Claire Cardie
Upcoming SlideShare
Loading in...5
×

What Business Innovators Need to Know about Sentiment Analysis, Claire Cardie

1,686

Published on

What Business Innovators Need to Know about Sentiment Analysis, presented by Prof. Claire Cardie of Cornell University.

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,686
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
93
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Transcript of "What Business Innovators Need to Know about Sentiment Analysis, Claire Cardie"

  1. 1. What Business Innovators Need to Know about Sentiment Analysis Claire Cardie Department of Computer Science Chair, Information Science Department Cornell University Co-founder Chief Scientist
  2. 2. Plan for the Talk Subjectivity and sentiment in language Continuum of capabilities – Surface-level in-depth understanding – Document-level phrase-level Next steps…
  3. 3. Subjective Language Subjective text expresses speculations, beliefs, emotions, evaluations, goals, opinions, judgments, … • Jill said, "I hate Bill." • John thought about whom to vote for. • Seth knew his symposium would go well.
  4. 4. Subjectivity vs. Sentiment Sentiment-bearing text expresses positive and negative speculations, beliefs, emotions, evaluations, goals, opinions, judgments,… • Jill said, "I hate Bill." - • John thought about whom to vote for. ~ • Seth knew his symposium would go well. + sentiment analysis tome [Pang & Lee, 2008]
  5. 5. A Word on Polarity (tone, valence) Positive “I love NY.” Negative “I hate NY.” Neither positive nor negative – Objective? “I thought about NY.” – Neutral? “I’m ambivalent about NY.” – Mixed polarity? “Sometimes I love NY; other times I hate it.”
  6. 6. And What About Intensity? Strength/intensity “I love NY.” “I absolutely adore NY!” – Low, medium, high, very high, extreme – ratings – rotten tomatoes
  7. 7. Plan for the Talk Subjectivity and sentiment in language Continuum of capabilities – Surface-level in-depth understanding – Document-level phrase-level Next steps…
  8. 8. Document-level Sentiment Analysis Is the overall sentiment in the Document document positive? negative? neutral?
  9. 9. Identifying Tone of a Collection Sentiment (w.r.t. a topic) – Example: Tone on “economic stimulus”
  10. 10. Detecting “chatter” or “buzz” Chatter (w.r.t. a topic) – Example: Buzz on “economic stimulus”
  11. 11. Keyword-based Approaches Search the text for the presence of specific terms from a manually created “sentiment lexicon” – +: “great”, “praise”, “peace”, “superb”, … – -: “war”, “dull”, “messy”, “criticize”, … Sentiment is based on the counts – E.g., If more positive terms than negative terms, then return +, else return –
  12. 12. Keyword-based Approaches Complications – Inherent ambiguities of language… – This laptop is a great deal. – A great deal of media attention surrounded the release of the new laptop model. – If you think this laptop is a great deal, I’ve got a nice bridge for you to buy. [Examples from Lillian Lee] [Pang & Lee, 2008]
  13. 13. Machine-learning Approaches Learn from training data Are better able to take advantage of context to disambiguate terms examples ML Algorithm statistical model (novel) examples class (program)
  14. 14. Measuring Performance Precision: #correct / #attempted Recall: #correct / #possible F-measure: harmonic mean of P and R 1. _______ P = 3 / 4 = .75 2. _______ P = 3 / 3 = 1.00 3. _______ R = 3 / 4 = .75 4. _______ accuracy
  15. 15. Measuring Performance How well do document-level sentiment analysis systems work? It depends… – Product reviews easier than Movie reviews, easier than News/editorials – Shorter documents harder than longer ones – Messy documents harder than clean ones ~75 F - ~85 F
  16. 16. This is actually quite good… Comparison is not vs. 100% P/R…but vs. human sentiment analysis accuracy – Cohen’s kappa Machine-learning methods for sentiment analysis approach human agreement levels – ~85 F: for positive/negative – ~75 F: when neutrals are included
  17. 17. Sentiment Analysis at Passage Level Passage tone The suggestion that the White House never took seriously an – Optionally w.r.t. a issue that infuriated millions of topic Americans was supported by – E.g., AIG or Geithner Senator Robert Menendez, a New Jersey Democrat who claimed that several weeks earlier he warned Timothy Geithner, the Treasury secretary, that AIG was planning to use taxpayer funds to pay out $165m in bonuses… speculation that Obama will have to replace him, despite the president’s insistence to Leno that Geithner is doing "an outstanding job“.
  18. 18. Sentiment Analysis at Phrase Level Fine-grained opinion analysis Identify who is saying what about what
  19. 19. Fine-Grained Sentiment Extraction The suggestion that the White House never took seriously an issue that infuriated millions of Americans was supported by Senator Robert Menendez, a New Jersey Democrat who claimed that several weeks earlier he warned Timothy Geithner, the Treasury secretary, that AIG was planning to use taxpayer funds to pay out $165m in bonuses… speculation that Obama will have to replace him, despite the president’s insistence to Leno that Geithner is doing "an outstanding job".
  20. 20. Fine-Grained Sentiment Extraction …the president insisted to Leno that Geithner is doing "an outstanding job". – Opinion trigger – Polarity Opinion Frame – Intensity Polarity: positive – Opinion holder Intensity: high Opinion Holder: “the president” – Target (topic) Target: “Geithner”
  21. 21. Example – fine-grained opinions opinion frame opinion frame opinion frame opinion frame opinion frame opinion frame The suggestion that the White House never took seriously an issue that infuriated millions of Americans was supported by Senator Robert Menendez, a New Jersey Democrat who claimed that several weeks earlier he warned Timothy Geithner, the Treasury secretary, that AIG was planning to use taxpayer funds to pay out $165m in bonuses…the president insisted to Leno that Geithner is doing "an outstanding job". opinion frame
  22. 22. Example – Opinion Summary AIG Obama Geithner Americans Menendez
  23. 23. Example – Opinion Summary Summarize thoughts and views across documents – Critical addition: opinion holder AIG
  24. 24. What makes this hard? Same issues of ambiguity as before plus… Need to associate opinion with topic and with opinion holder Requires different machine learning methods Requires many language-processing modules
  25. 25. Noun Phrase Coreference Resolution The suggestion that the White House never took seriously an issue that infuriated millions of Americans was supported by Senator Robert Menendez, a New Jersey Democrat who claimed that several weeks earlier he warned Timothy Geithner, the Treasury secretary, that AIG was planning to use taxpayer funds to pay out $165m in bonuses…speculation that Obama will have to replace Geithner, despite the president’s insistence to Leno that he is doing "an outstanding job". Ng & Cardie [2002, 2003]; Stoyanov & Cardie [2006, 2008]
  26. 26. Performance 82F opinion OH 79F extraction extractor 69F link classifier –<opinion holder> expresses <opinion> Choi, Breck & Cardie [2006, 2007]
  27. 27. Plan for the Talk Subjectivity and sentiment in language Continuum of capabilities – Surface-level in-depth understanding – Document-level phrase-level Next steps…
  28. 28. Next Steps… Predicting business outcomes from opinions – Doable in some settings Determining the key influencers
  29. 29. Thank you! Questions?
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×