Your SlideShare is downloading. ×
0
Polling the Blogosphere: a Rule-Based Approach to Belief Classification, By Jason Kessler
Polling the Blogosphere: a Rule-Based Approach to Belief Classification, By Jason Kessler
Polling the Blogosphere: a Rule-Based Approach to Belief Classification, By Jason Kessler
Polling the Blogosphere: a Rule-Based Approach to Belief Classification, By Jason Kessler
Polling the Blogosphere: a Rule-Based Approach to Belief Classification, By Jason Kessler
Polling the Blogosphere: a Rule-Based Approach to Belief Classification, By Jason Kessler
Polling the Blogosphere: a Rule-Based Approach to Belief Classification, By Jason Kessler
Polling the Blogosphere: a Rule-Based Approach to Belief Classification, By Jason Kessler
Polling the Blogosphere: a Rule-Based Approach to Belief Classification, By Jason Kessler
Polling the Blogosphere: a Rule-Based Approach to Belief Classification, By Jason Kessler
Polling the Blogosphere: a Rule-Based Approach to Belief Classification, By Jason Kessler
Polling the Blogosphere: a Rule-Based Approach to Belief Classification, By Jason Kessler
Polling the Blogosphere: a Rule-Based Approach to Belief Classification, By Jason Kessler
Polling the Blogosphere: a Rule-Based Approach to Belief Classification, By Jason Kessler
Polling the Blogosphere: a Rule-Based Approach to Belief Classification, By Jason Kessler
Polling the Blogosphere: a Rule-Based Approach to Belief Classification, By Jason Kessler
Polling the Blogosphere: a Rule-Based Approach to Belief Classification, By Jason Kessler
Polling the Blogosphere: a Rule-Based Approach to Belief Classification, By Jason Kessler
Polling the Blogosphere: a Rule-Based Approach to Belief Classification, By Jason Kessler
Polling the Blogosphere: a Rule-Based Approach to Belief Classification, By Jason Kessler
Polling the Blogosphere: a Rule-Based Approach to Belief Classification, By Jason Kessler
Polling the Blogosphere: a Rule-Based Approach to Belief Classification, By Jason Kessler
Polling the Blogosphere: a Rule-Based Approach to Belief Classification, By Jason Kessler
Polling the Blogosphere: a Rule-Based Approach to Belief Classification, By Jason Kessler
Polling the Blogosphere: a Rule-Based Approach to Belief Classification, By Jason Kessler
Polling the Blogosphere: a Rule-Based Approach to Belief Classification, By Jason Kessler
Polling the Blogosphere: a Rule-Based Approach to Belief Classification, By Jason Kessler
Polling the Blogosphere: a Rule-Based Approach to Belief Classification, By Jason Kessler
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Polling the Blogosphere: a Rule-Based Approach to Belief Classification, By Jason Kessler

950

Published on

ICWSM 2008 Talk

ICWSM 2008 Talk

Published in: Technology, Education
2 Comments
0 Likes
Statistics
Notes
  • Be the first to like this

No Downloads
Views
Total Views
950
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
14
Comments
2
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  1. Polling the Blogosphere: a Rule-Based Approach to Belief Classification Jason Kessler Indiana University, Bloomington
  2. Belief Analysis of Blogs <ul><li>Polling the blogosphere on a controversial proposition </li></ul><ul><li>Literal search on a proposition (e.g., “Obama is electable”) </li></ul><ul><li>Which blog entries contain assert it? Which deny it? </li></ul><ul><li>Aggregate results </li></ul><ul><ul><li>243 bloggers assert it </li></ul></ul><ul><ul><li>616 bloggers deny it </li></ul></ul>
  3. Motivating Example <ul><li>Polling for “the Moon landings were staged” </li></ul><ul><li>“ The theory that the Moon landings were staged is complete nonsense.” </li></ul><ul><li>The writer denies “the Moon landings were staged.” </li></ul>
  4. Motivating Example <ul><li>If Obama is electable , the country is in good shape. </li></ul><ul><li>Writer takes no stance toward “Obama is electable”. </li></ul>
  5. Problem <ul><li>When a writer uses a declarative finite clause, does that writer assert, deny, or take no stance toward its truth value? </li></ul><ul><li>This is the problem of identifying a writer’s stance toward a proposition . </li></ul><ul><li>Veridicity or facticity of a proposition. </li></ul>
  6. Example <ul><li>Everybody is sad that the bar closed. </li></ul><ul><li>The writer asserts “the bar closed.” </li></ul><ul><li>Belief != Sentiment </li></ul><ul><ul><li>Negative sentiment toward “the bar closed” </li></ul></ul><ul><ul><li>Positive stance. </li></ul></ul>
  7. Outline <ul><li>System Description </li></ul><ul><ul><li>Given a proposition, sentence </li></ul></ul><ul><ul><li>Dependency Parse </li></ul></ul><ul><ul><ul><li>Syntactic Representation </li></ul></ul></ul><ul><ul><li>Hand written patterns over semantic classes </li></ul></ul><ul><ul><ul><li>Veridicality Elements </li></ul></ul></ul><ul><ul><ul><li>Veridicality Transformations </li></ul></ul></ul><ul><li>Evaluation </li></ul><ul><ul><li>Proof of concept </li></ul></ul><ul><ul><li>Promising results </li></ul></ul>
  8. Dependency Parse <ul><li>Pipeline Stages: </li></ul><ul><li>Dependency Parse </li></ul><ul><li>Tag Veridicality Elements </li></ul><ul><li>Apply Veridicality Transformations </li></ul>The theory that the Moon landings were staged is complete nonsense.
  9. Veridicality Elements (VEs) <ul><li>Pipeline Stages: </li></ul><ul><li>Dependency Parse </li></ul><ul><li>Tag Veridicality Elements </li></ul><ul><li>Apply Veridicality Transformations </li></ul>The theory that the Moon landings were staged is complete nonsense.
  10. Veridicality Transformations (VTs) <ul><li>Pipeline Stages: </li></ul><ul><li>Dependency Parse </li></ul><ul><li>Tag Veridicality Elements </li></ul><ul><li>Apply Veridicality Transformations </li></ul>The theory that the Moon landings were staged is complete nonsense.
  11. Veridicality Transformations (VTs) <ul><li>Pipeline Stages: </li></ul><ul><li>Dependency Parse </li></ul><ul><li>Tag Veridicality Elements </li></ul><ul><li>Apply Veridicality Transformations </li></ul>The theory that the Moon landings were staged is complete nonsense.
  12. System Structure: Veridicality Elements <ul><li>Find expressions that have the potential of changing the truth-value of a proposition or referring to it </li></ul><ul><li>Different classes affect truth values differently </li></ul><ul><li>Examples: </li></ul><ul><ul><li>Assertion – Positive </li></ul></ul><ul><ul><ul><li>The assertion that the sky is blue </li></ul></ul></ul><ul><ul><li>Nonsense – Negative </li></ul></ul><ul><ul><ul><li>The idea that the sky is orange is nonsense </li></ul></ul></ul><ul><ul><li>If – Neutral </li></ul></ul><ul><ul><li>Pretend – Counter-factive </li></ul></ul>
  13. Finding Veridicality Elements <ul><li>Manually created seed sets </li></ul><ul><li>Search web for patterns likely to contain VEs </li></ul><ul><li>“ I agree with the assertion that” </li></ul><ul><ul><li>“ I * with the assertion that” </li></ul></ul><ul><ul><ul><li>“ I quibble with the assertion that” </li></ul></ul></ul><ul><ul><ul><li>“ I take issue with the assertion that” </li></ul></ul></ul><ul><li>Manually classify matches, form new queries </li></ul><ul><ul><li>“ I take issue with the * that” </li></ul></ul><ul><ul><ul><li>“ I take issue with the argument that” </li></ul></ul></ul><ul><li>Similar to Brin (1998) </li></ul>
  14. System Structure: Veridicality Transformations <ul><li>Relate these expressions to propositions </li></ul><ul><ul><li>Some expressions won’t be related to propositions </li></ul></ul><ul><ul><li>Why bag-of-Veridicality-Elements fails </li></ul></ul><ul><li>Templates over dependency graphs </li></ul><ul><ul><li>Select for a VE class and a proposition </li></ul></ul>
  15. System Structure: Veridicality Transformations <ul><li>Examples </li></ul><ul><ul><li>Expression is a main verb, proposition is its comp. clause </li></ul></ul><ul><ul><ul><li>John pretended the monkey was harmless . </li></ul></ul></ul><ul><ul><li>Cleft construction, expression is an adjective </li></ul></ul><ul><ul><ul><li>It is inconceivable that two plus two equals five . </li></ul></ul></ul>
  16. Another Example <ul><li>Pipeline Stages: </li></ul><ul><li>Dependency Parse </li></ul><ul><li>Tag Veridicality Elements </li></ul><ul><li>Apply Veridicality Transformations </li></ul>If Bob goes to school, he realizes the Earth is round.
  17. Another Example <ul><li>Pipeline Stages: </li></ul><ul><li>Dependency Parse </li></ul><ul><li>Tag Veridicality Elements </li></ul><ul><li>Apply Veridicality Transformations </li></ul>If Bob goes to school, he realizes the Earth is round.
  18. Another Example <ul><li>Pipeline Stages: </li></ul><ul><li>Dependency Parse </li></ul><ul><li>Tag Veridicality Elements </li></ul><ul><li>Apply Veridicality Transformations </li></ul>If Bob goes to school, he realizes the Earth is round.
  19. Evaluation <ul><li>Primitive, proof-of-concept evaluation </li></ul><ul><li>Can we poll the blogosphere ? </li></ul><ul><li>Google blog search for “abortion is murder” </li></ul><ul><ul><li>Unseen data </li></ul></ul><ul><li>Run the system on the first 100 hits. </li></ul><ul><li>See if it does better baseline. </li></ul>
  20. Evaluation <ul><li>Exclude a number of results: </li></ul><ul><ul><li>Spam blogs </li></ul></ul><ul><ul><li>Long, unparsable sentences </li></ul></ul><ul><ul><li>Trivial sentences (no VEs) </li></ul></ul><ul><ul><ul><li>Abortion is murder! </li></ul></ul></ul><ul><ul><li>Questions </li></ul></ul>
  21. Evaluation <ul><li>Corpus Statistics: </li></ul><ul><ul><li>48 Sentences </li></ul></ul><ul><ul><ul><li>27 positive </li></ul></ul></ul><ul><ul><ul><li>3 negative </li></ul></ul></ul><ul><ul><ul><li>18 neutral </li></ul></ul></ul><ul><ul><li>39 classified correctly (81% accuracy) </li></ul></ul><ul><ul><li>Majority class was positive, giving a baseline of 56% accuracy </li></ul></ul>
  22. Related Work <ul><li>Nairn et al. (2006) focused on main verbs </li></ul><ul><ul><li>Complex behavior under negation </li></ul></ul><ul><li>Work on contextual polarity for sentiment analysis. </li></ul><ul><ul><li>Wilson et al. (2005) </li></ul></ul><ul><ul><ul><li>Statistical approach </li></ul></ul></ul><ul><ul><li>Polanyi and Zaenen (2006) </li></ul></ul><ul><ul><ul><li>Theoretical approach </li></ul></ul></ul>
  23. Related Work <ul><li>Somasundaran et al. (2007) </li></ul><ul><ul><li>Statistical techniques used to detect presence of “arguing” in a sentence. </li></ul></ul><ul><ul><li>Arguing = writer takes a non-neutral stance toward some content </li></ul></ul>
  24. Future Work <ul><li>Annotate corpus </li></ul><ul><ul><li>Further testing </li></ul></ul><ul><ul><li>Statistical approaches </li></ul></ul><ul><li>Augment VE/VTs </li></ul><ul><li>Integrate Nairn et al. (2006) </li></ul><ul><li>Take into account questions </li></ul>
  25. Takeaways <ul><li>Belief analysis is a young field </li></ul><ul><li>Bag-of-words is not enough </li></ul><ul><li>Shallow linguistic methods show promise </li></ul>
  26. Questions? <ul><li>Thank you. </li></ul><ul><li>References: </li></ul><ul><li>Brin, S. 1998. Extracting patterns and relations from the world wide web. In WebDB Workshop at 6 th International Conference on Extending Database Technology, EDBT’98. </li></ul><ul><li>Nairn, R.; Condoravdi, C.; and Karttunen, L. 2006. Computing relative polarity for textual inference. In ICoS-5 . </li></ul><ul><li>Polanyi, L.; and Zaenen, A. 2005. Contextual valence shifters. In Shanahan, J. G.; Qu, Y.; and Wiebe J., eds,. Computing Attitude and Affect in Text. </li></ul><ul><li>Somasundaran, S.; Wilson, T.; Wiebe, J.; and Stoyanov, V. 2007. QA with attitude: Exploiting opinion type analysis for improving question answering in on-line discussions and the news. In ICWSM. </li></ul><ul><li>Wilson, T.; Wiebe, J.; and Hoffmann, P. 2005. Recognizing contextual polarity in phrase-level sentiment analysis. In HLT/EMNLP. </li></ul>
  27. Implementation Veridicality Element Classes:
  28. Veridicality Transformations

×