Polling the Blogosphere: a Rule-Based Approach to Belief Classification Jason Kessler Indiana University, Bloomington
Belief Analysis of Blogs <ul><li>Polling the blogosphere on a controversial proposition </li></ul><ul><li>Literal search o...
Motivating Example <ul><li>Polling for “the Moon landings were staged” </li></ul><ul><li>“ The theory that  the Moon landi...
Motivating Example <ul><li>If  Obama is electable , the country is in good shape. </li></ul><ul><li>Writer takes no stance...
Problem <ul><li>When a writer uses a declarative finite clause, does that writer assert, deny, or take no stance toward it...
Example <ul><li>Everybody is sad that the bar closed. </li></ul><ul><li>The writer asserts “the bar closed.” </li></ul><ul...
Outline <ul><li>System Description </li></ul><ul><ul><li>Given a proposition, sentence </li></ul></ul><ul><ul><li>Dependen...
Dependency Parse <ul><li>Pipeline Stages: </li></ul><ul><li>Dependency Parse </li></ul><ul><li>Tag Veridicality Elements <...
Veridicality Elements (VEs) <ul><li>Pipeline Stages: </li></ul><ul><li>Dependency Parse </li></ul><ul><li>Tag Veridicality...
Veridicality Transformations (VTs) <ul><li>Pipeline Stages: </li></ul><ul><li>Dependency Parse </li></ul><ul><li>Tag Verid...
Veridicality Transformations (VTs) <ul><li>Pipeline Stages: </li></ul><ul><li>Dependency Parse </li></ul><ul><li>Tag Verid...
System Structure:  Veridicality Elements <ul><li>Find expressions that have the potential of changing the truth-value of a...
Finding Veridicality Elements <ul><li>Manually created seed sets </li></ul><ul><li>Search web for patterns likely to conta...
System Structure: Veridicality Transformations <ul><li>Relate these expressions to propositions </li></ul><ul><ul><li>Some...
System Structure: Veridicality Transformations <ul><li>Examples </li></ul><ul><ul><li>Expression is a main verb, propositi...
Another Example <ul><li>Pipeline Stages: </li></ul><ul><li>Dependency Parse </li></ul><ul><li>Tag Veridicality Elements </...
Another Example <ul><li>Pipeline Stages: </li></ul><ul><li>Dependency Parse </li></ul><ul><li>Tag Veridicality Elements </...
Another Example <ul><li>Pipeline Stages: </li></ul><ul><li>Dependency Parse </li></ul><ul><li>Tag Veridicality Elements </...
Evaluation <ul><li>Primitive, proof-of-concept evaluation </li></ul><ul><li>Can we  poll the blogosphere ? </li></ul><ul><...
Evaluation <ul><li>Exclude a number of results:  </li></ul><ul><ul><li>Spam blogs </li></ul></ul><ul><ul><li>Long, unparsa...
Evaluation <ul><li>Corpus Statistics: </li></ul><ul><ul><li>48 Sentences </li></ul></ul><ul><ul><ul><li>27 positive </li><...
Related Work <ul><li>Nairn et al. (2006) focused on main verbs </li></ul><ul><ul><li>Complex behavior under negation </li>...
Related Work <ul><li>Somasundaran et al. (2007)  </li></ul><ul><ul><li>Statistical techniques used to detect presence of “...
Future Work <ul><li>Annotate corpus </li></ul><ul><ul><li>Further testing </li></ul></ul><ul><ul><li>Statistical approache...
Takeaways <ul><li>Belief analysis is a young field </li></ul><ul><li>Bag-of-words is not enough </li></ul><ul><li>Shallow ...
Questions? <ul><li>Thank you. </li></ul><ul><li>References: </li></ul><ul><li>Brin, S. 1998.  Extracting patterns and rela...
Implementation Veridicality Element Classes:
Veridicality Transformations
Upcoming SlideShare
Loading in …5
×

Polling the Blogosphere: a Rule-Based Approach to Belief Classification, By Jason Kessler

1,128 views
1,087 views

Published on

ICWSM 2008 Talk

Published in: Technology, Education
2 Comments
0 Likes
Statistics
Notes
  • Be the first to like this

No Downloads
Views
Total views
1,128
On SlideShare
0
From Embeds
0
Number of Embeds
48
Actions
Shares
0
Downloads
16
Comments
2
Likes
0
Embeds 0
No embeds

No notes for slide

Polling the Blogosphere: a Rule-Based Approach to Belief Classification, By Jason Kessler

  1. 1. Polling the Blogosphere: a Rule-Based Approach to Belief Classification Jason Kessler Indiana University, Bloomington
  2. 2. Belief Analysis of Blogs <ul><li>Polling the blogosphere on a controversial proposition </li></ul><ul><li>Literal search on a proposition (e.g., “Obama is electable”) </li></ul><ul><li>Which blog entries contain assert it? Which deny it? </li></ul><ul><li>Aggregate results </li></ul><ul><ul><li>243 bloggers assert it </li></ul></ul><ul><ul><li>616 bloggers deny it </li></ul></ul>
  3. 3. Motivating Example <ul><li>Polling for “the Moon landings were staged” </li></ul><ul><li>“ The theory that the Moon landings were staged is complete nonsense.” </li></ul><ul><li>The writer denies “the Moon landings were staged.” </li></ul>
  4. 4. Motivating Example <ul><li>If Obama is electable , the country is in good shape. </li></ul><ul><li>Writer takes no stance toward “Obama is electable”. </li></ul>
  5. 5. Problem <ul><li>When a writer uses a declarative finite clause, does that writer assert, deny, or take no stance toward its truth value? </li></ul><ul><li>This is the problem of identifying a writer’s stance toward a proposition . </li></ul><ul><li>Veridicity or facticity of a proposition. </li></ul>
  6. 6. Example <ul><li>Everybody is sad that the bar closed. </li></ul><ul><li>The writer asserts “the bar closed.” </li></ul><ul><li>Belief != Sentiment </li></ul><ul><ul><li>Negative sentiment toward “the bar closed” </li></ul></ul><ul><ul><li>Positive stance. </li></ul></ul>
  7. 7. Outline <ul><li>System Description </li></ul><ul><ul><li>Given a proposition, sentence </li></ul></ul><ul><ul><li>Dependency Parse </li></ul></ul><ul><ul><ul><li>Syntactic Representation </li></ul></ul></ul><ul><ul><li>Hand written patterns over semantic classes </li></ul></ul><ul><ul><ul><li>Veridicality Elements </li></ul></ul></ul><ul><ul><ul><li>Veridicality Transformations </li></ul></ul></ul><ul><li>Evaluation </li></ul><ul><ul><li>Proof of concept </li></ul></ul><ul><ul><li>Promising results </li></ul></ul>
  8. 8. Dependency Parse <ul><li>Pipeline Stages: </li></ul><ul><li>Dependency Parse </li></ul><ul><li>Tag Veridicality Elements </li></ul><ul><li>Apply Veridicality Transformations </li></ul>The theory that the Moon landings were staged is complete nonsense.
  9. 9. Veridicality Elements (VEs) <ul><li>Pipeline Stages: </li></ul><ul><li>Dependency Parse </li></ul><ul><li>Tag Veridicality Elements </li></ul><ul><li>Apply Veridicality Transformations </li></ul>The theory that the Moon landings were staged is complete nonsense.
  10. 10. Veridicality Transformations (VTs) <ul><li>Pipeline Stages: </li></ul><ul><li>Dependency Parse </li></ul><ul><li>Tag Veridicality Elements </li></ul><ul><li>Apply Veridicality Transformations </li></ul>The theory that the Moon landings were staged is complete nonsense.
  11. 11. Veridicality Transformations (VTs) <ul><li>Pipeline Stages: </li></ul><ul><li>Dependency Parse </li></ul><ul><li>Tag Veridicality Elements </li></ul><ul><li>Apply Veridicality Transformations </li></ul>The theory that the Moon landings were staged is complete nonsense.
  12. 12. System Structure: Veridicality Elements <ul><li>Find expressions that have the potential of changing the truth-value of a proposition or referring to it </li></ul><ul><li>Different classes affect truth values differently </li></ul><ul><li>Examples: </li></ul><ul><ul><li>Assertion – Positive </li></ul></ul><ul><ul><ul><li>The assertion that the sky is blue </li></ul></ul></ul><ul><ul><li>Nonsense – Negative </li></ul></ul><ul><ul><ul><li>The idea that the sky is orange is nonsense </li></ul></ul></ul><ul><ul><li>If – Neutral </li></ul></ul><ul><ul><li>Pretend – Counter-factive </li></ul></ul>
  13. 13. Finding Veridicality Elements <ul><li>Manually created seed sets </li></ul><ul><li>Search web for patterns likely to contain VEs </li></ul><ul><li>“ I agree with the assertion that” </li></ul><ul><ul><li>“ I * with the assertion that” </li></ul></ul><ul><ul><ul><li>“ I quibble with the assertion that” </li></ul></ul></ul><ul><ul><ul><li>“ I take issue with the assertion that” </li></ul></ul></ul><ul><li>Manually classify matches, form new queries </li></ul><ul><ul><li>“ I take issue with the * that” </li></ul></ul><ul><ul><ul><li>“ I take issue with the argument that” </li></ul></ul></ul><ul><li>Similar to Brin (1998) </li></ul>
  14. 14. System Structure: Veridicality Transformations <ul><li>Relate these expressions to propositions </li></ul><ul><ul><li>Some expressions won’t be related to propositions </li></ul></ul><ul><ul><li>Why bag-of-Veridicality-Elements fails </li></ul></ul><ul><li>Templates over dependency graphs </li></ul><ul><ul><li>Select for a VE class and a proposition </li></ul></ul>
  15. 15. System Structure: Veridicality Transformations <ul><li>Examples </li></ul><ul><ul><li>Expression is a main verb, proposition is its comp. clause </li></ul></ul><ul><ul><ul><li>John pretended the monkey was harmless . </li></ul></ul></ul><ul><ul><li>Cleft construction, expression is an adjective </li></ul></ul><ul><ul><ul><li>It is inconceivable that two plus two equals five . </li></ul></ul></ul>
  16. 16. Another Example <ul><li>Pipeline Stages: </li></ul><ul><li>Dependency Parse </li></ul><ul><li>Tag Veridicality Elements </li></ul><ul><li>Apply Veridicality Transformations </li></ul>If Bob goes to school, he realizes the Earth is round.
  17. 17. Another Example <ul><li>Pipeline Stages: </li></ul><ul><li>Dependency Parse </li></ul><ul><li>Tag Veridicality Elements </li></ul><ul><li>Apply Veridicality Transformations </li></ul>If Bob goes to school, he realizes the Earth is round.
  18. 18. Another Example <ul><li>Pipeline Stages: </li></ul><ul><li>Dependency Parse </li></ul><ul><li>Tag Veridicality Elements </li></ul><ul><li>Apply Veridicality Transformations </li></ul>If Bob goes to school, he realizes the Earth is round.
  19. 19. Evaluation <ul><li>Primitive, proof-of-concept evaluation </li></ul><ul><li>Can we poll the blogosphere ? </li></ul><ul><li>Google blog search for “abortion is murder” </li></ul><ul><ul><li>Unseen data </li></ul></ul><ul><li>Run the system on the first 100 hits. </li></ul><ul><li>See if it does better baseline. </li></ul>
  20. 20. Evaluation <ul><li>Exclude a number of results: </li></ul><ul><ul><li>Spam blogs </li></ul></ul><ul><ul><li>Long, unparsable sentences </li></ul></ul><ul><ul><li>Trivial sentences (no VEs) </li></ul></ul><ul><ul><ul><li>Abortion is murder! </li></ul></ul></ul><ul><ul><li>Questions </li></ul></ul>
  21. 21. Evaluation <ul><li>Corpus Statistics: </li></ul><ul><ul><li>48 Sentences </li></ul></ul><ul><ul><ul><li>27 positive </li></ul></ul></ul><ul><ul><ul><li>3 negative </li></ul></ul></ul><ul><ul><ul><li>18 neutral </li></ul></ul></ul><ul><ul><li>39 classified correctly (81% accuracy) </li></ul></ul><ul><ul><li>Majority class was positive, giving a baseline of 56% accuracy </li></ul></ul>
  22. 22. Related Work <ul><li>Nairn et al. (2006) focused on main verbs </li></ul><ul><ul><li>Complex behavior under negation </li></ul></ul><ul><li>Work on contextual polarity for sentiment analysis. </li></ul><ul><ul><li>Wilson et al. (2005) </li></ul></ul><ul><ul><ul><li>Statistical approach </li></ul></ul></ul><ul><ul><li>Polanyi and Zaenen (2006) </li></ul></ul><ul><ul><ul><li>Theoretical approach </li></ul></ul></ul>
  23. 23. Related Work <ul><li>Somasundaran et al. (2007) </li></ul><ul><ul><li>Statistical techniques used to detect presence of “arguing” in a sentence. </li></ul></ul><ul><ul><li>Arguing = writer takes a non-neutral stance toward some content </li></ul></ul>
  24. 24. Future Work <ul><li>Annotate corpus </li></ul><ul><ul><li>Further testing </li></ul></ul><ul><ul><li>Statistical approaches </li></ul></ul><ul><li>Augment VE/VTs </li></ul><ul><li>Integrate Nairn et al. (2006) </li></ul><ul><li>Take into account questions </li></ul>
  25. 25. Takeaways <ul><li>Belief analysis is a young field </li></ul><ul><li>Bag-of-words is not enough </li></ul><ul><li>Shallow linguistic methods show promise </li></ul>
  26. 26. Questions? <ul><li>Thank you. </li></ul><ul><li>References: </li></ul><ul><li>Brin, S. 1998. Extracting patterns and relations from the world wide web. In WebDB Workshop at 6 th International Conference on Extending Database Technology, EDBT’98. </li></ul><ul><li>Nairn, R.; Condoravdi, C.; and Karttunen, L. 2006. Computing relative polarity for textual inference. In ICoS-5 . </li></ul><ul><li>Polanyi, L.; and Zaenen, A. 2005. Contextual valence shifters. In Shanahan, J. G.; Qu, Y.; and Wiebe J., eds,. Computing Attitude and Affect in Text. </li></ul><ul><li>Somasundaran, S.; Wilson, T.; Wiebe, J.; and Stoyanov, V. 2007. QA with attitude: Exploiting opinion type analysis for improving question answering in on-line discussions and the news. In ICWSM. </li></ul><ul><li>Wilson, T.; Wiebe, J.; and Hoffmann, P. 2005. Recognizing contextual polarity in phrase-level sentiment analysis. In HLT/EMNLP. </li></ul>
  27. 27. Implementation Veridicality Element Classes:
  28. 28. Veridicality Transformations

×