Jodi Schneider
Frontiers and Connections between Argumentation Theory
and Natural Language Processing
BiCi seminar series,...
Topics
• Examples of argumentation support
• Supporting Collaborative Online Arguing
• Structuring scientific argument:
Mi...
Create new spaces for civic debate
http://Consider.It
"Supporting reflective public thought with ConsiderIt." CSCW 2012
Tr...
Create new spaces for civic debate
http://Consider.It
"Supporting reflective public thought with ConsiderIt." CSCW 2012
Tr...
Choose
task-
appropriate
formalisms:
SEAS
"Template-based structured argumentation." In Knowledge Cartography, Springer Lo...
Choose
task-
appropriate
formalisms
discoursedb.
org
Support
incremental
formalization
argunet.org
ClimateCoLab
climatecolab.org
http://www.climatecolab.org/ Circa 2010
Topics
• Examples of argumentation support
• Supporting Collaborative Online Arguing
• Structuring scientific argument:
Mi...
General Approach (from Informatics)
1. Analyze requirements
2. Consider which argumentation models to use
3. Build a proto...
500 Wikipedia debates each week:
Should we delete this article?
Requirements:
Support argumentation tasks
• Convince others of your position, using
community norms
• Determine the overal...
Compare two argumentation theories
• Walton’s Argumentation Schemes
(Walton, Reed, and Macagno 2008)
– Informal argumentat...
Walton’s Argumentation Schemes
Example Argumentation Scheme:
Argument from Rules – “we apply rule X”
Critical Questions
1....
“Rule” Argumentation Scheme
“Arguments about Deletion: How Experience Improves the Acceptability of Arguments in Ad-hoc On...
“Evidence” Argumentation Scheme
“Arguments about Deletion: How Experience Improves the Acceptability of Arguments in Ad-ho...
Evidence + Rule -> Conclusion
“Arguments about Deletion: How Experience Improves the Acceptability of Arguments in Ad-hoc ...
Supporting Tasks with Walton
• Convince others of your position, using
community norms
– To win an argument, use popular s...
Factors/Dimensions Analysis
• Factors (case-based reasoning)
– All or nothing
• Either present ("applicable") or absent
• ...
Example factors analysis
(Aleven 1997)
Aleven 1997
Wikipedia Factors Analysis
Factors determined
by iterative annotation
4 Factors cover
– 91% of comments
– 70% of discussio...
Wikipedia Factors Analysis
Factors determined
by iterative annotation
4 Factors cover
– 91% of comments
– 70% of discussio...
Wikipedia Factors Analysis
Factor Example (used to justify 'keep') Example (used to justify 'delete'
Notability Anyone cov...
Supporting Tasks with Factors
• Convince others of your position, using
community norms
– To win an argument, talk about t...
Factor-based Summarization
Argument Schemes vs. Factors?
• Argument Schemes (kappa=.48)
Details of how to put together an argument
– Could support WR...
Argument prevalence depends on the
corpus
• Wikipedia
– Argument from Evidence to Hypothesis (19%)
– Argument from Rules (...
Topics
• Examples of argumentation support
• Supporting Collaborative Online Arguing
• Micropublications model: Structurin...
Micropublications
Model Data,
Methods,
Materials,
References
Micropublications: a Semantic Model for
Claims, Evidence, Arguments and
Annotat...
Direct Annotation with Domeo
http://swan.mindinformatics.org/ Paolo N Ciccarese
Micropublication: Claim + Support
(e.g. Attribution)
Micropublications: a Semantic Model for Claims, Evidence, Arguments a...
Constructs claim-argument network
across scientific papers
Micropublications: a Semantic Model for Claims, Evidence, Argum...
Argumentation Mining papers
Arguing on Wikipedia
• “Arguments about Deletion: How Experience Improves the Acceptability of...
Argumentation mining today
• No unified vision of the field. Multiple:
– Interrelated problems
– Application domains
– Too...
Example: "Stop at a red light"
1. Does the rule require carrying out this type of action?
Were you driving a vehicle?
2. A...
None of Wikipedia's top-used schemes
are prevalent in Arucaria.
Classifying Arguments by Scheme. Vanessa Wei Feng. Master'...
Goal: large-scale arguing
• Search for issues, claims, and opinion clusters
• Link to evidence when writing your opinion
•...
Online argumentation support
interfaces can:
• Promote "listening" in online conversations
• Support incremental formaliza...
Argumentation mining could be the
basis for support tools
• Help participants write persuasive arguments
– How: provide pe...
Supporting collaborative online arguing -- talk for BICI Frontiers and Connections between Argumentation Theory and Natura...
Upcoming SlideShare
Loading in...5
×

Supporting collaborative online arguing -- talk for BICI Frontiers and Connections between Argumentation Theory and Natural Language Processing 2014-07-24

272

Published on

Seminar talk: Frontiers and Connections between Argumentation Theory and Natural Language Processing
BiCI seminar series, Bertinoro (Forlì-Cesena), Italy, 2014-07-24
http://www-sop.inria.fr/members/Serena.Villata/BiCi2014/program/index.html

Topics:
- Examples of argumentation support
- Supporting Collaborative Online Arguing
- Structuring scientific argument: Micropublications model

My paper covers related (but not identical) ground:
http://jodischneider.com/pubs/frontiersargnlp2014.pdf


Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
272
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • 20 min

    http://www-sop.inria.fr/members/Serena.Villata/BiCi2014/frontiersARG-NLP.html
    BiCi
    Frontiers and Connections between Argumentation Theory and Natural Language Processing
  • ConsiderIt

    https://livingvotersguide.org/wash__1240/results

    Videolyzer
  • ConsiderIt

    https://livingvotersguide.org/wash__1240/results

    Videolyzer
  • ArguNet
    Climate CoLab (formerly Deliberatorium)


  • When the argumentation scheme used in a draft message is not generally accepted, the author could be warned that their message might not be persuasive, and given personalized suggestions

    Listing these questions in concrete and contextualized form (drawing on the premises, inference rules, and conclusions to instantiate and contextualize them) would encourage participants to consider the pos- sible flaws in reasoning and might prompt partici- pants to request answers within the debate.

    Macro- argumentation, such as the factors analysis de- scribed above, would be a natural choice for sum- marization, as it has already proven useful for fil- tering discussions. A more reasoning-intensive approach would be to calculate consistent out- comes (Wyner and van Engers, 2010), if debates can be easily formalized.

  • Transcript of "Supporting collaborative online arguing -- talk for BICI Frontiers and Connections between Argumentation Theory and Natural Language Processing 2014-07-24"

    1. 1. Jodi Schneider Frontiers and Connections between Argumentation Theory and Natural Language Processing BiCi seminar series, Bertinoro (Forlì-Cesena), Italy 24 July 2014
    2. 2. Topics • Examples of argumentation support • Supporting Collaborative Online Arguing • Structuring scientific argument: Micropublications model
    3. 3. Create new spaces for civic debate http://Consider.It "Supporting reflective public thought with ConsiderIt." CSCW 2012 Travis Kriplean, Jonathan Morgan, Deen Freelon, Alan Borning, and Lance Bennett.
    4. 4. Create new spaces for civic debate http://Consider.It "Supporting reflective public thought with ConsiderIt." CSCW 2012 Travis Kriplean, Jonathan Morgan, Deen Freelon, Alan Borning, and Lance Bennett.
    5. 5. Choose task- appropriate formalisms: SEAS "Template-based structured argumentation." In Knowledge Cartography, Springer London, 2008. John Lowrance, Ian Harrison, Andres Rodriguez, Eric Yeh, Tom Boyce, Janet Murdock, Jerome Thomere, and Ken Murray.
    6. 6. Choose task- appropriate formalisms discoursedb. org
    7. 7. Support incremental formalization argunet.org
    8. 8. ClimateCoLab climatecolab.org http://www.climatecolab.org/ Circa 2010
    9. 9. Topics • Examples of argumentation support • Supporting Collaborative Online Arguing • Structuring scientific argument: Micropublications model
    10. 10. General Approach (from Informatics) 1. Analyze requirements 2. Consider which argumentation models to use 3. Build a prototype support tool 4. Evaluate and iterate
    11. 11. 500 Wikipedia debates each week: Should we delete this article?
    12. 12. Requirements: Support argumentation tasks • Convince others of your position, using community norms • Determine the overall consensus decision
    13. 13. Compare two argumentation theories • Walton’s Argumentation Schemes (Walton, Reed, and Macagno 2008) – Informal argumentation (philosophical & computational argumentation) – Identify & prevent errors in reasoning (fallacies) – 60 patterns • Factors/Dimensions Analysis (Ashley 1991; Bench-Capon and Rissland, 2001) – Case-based reasoning – E.g. factors for deciding cases in trade secret law, favoring either party (the plaintiff or the defendant).
    14. 14. Walton’s Argumentation Schemes Example Argumentation Scheme: Argument from Rules – “we apply rule X” Critical Questions 1. Does the rule require carrying out this type of action? 2. Are there other established rules that might conflict with or override this one? 3. Are there extenuating circumstances or an excuse for noncompliance? Walton, Reed, and Macagno 2008
    15. 15. “Rule” Argumentation Scheme “Arguments about Deletion: How Experience Improves the Acceptability of Arguments in Ad-hoc Online Task Groups” CSCW 2013
    16. 16. “Evidence” Argumentation Scheme “Arguments about Deletion: How Experience Improves the Acceptability of Arguments in Ad-hoc Online Task Groups” CSCW 2013
    17. 17. Evidence + Rule -> Conclusion “Arguments about Deletion: How Experience Improves the Acceptability of Arguments in Ad-hoc Online Task Groups” CSCW 2013
    18. 18. Supporting Tasks with Walton • Convince others of your position, using community norms – To win an argument, use popular schemes: • Argument from Evidence to Hypothesis (19%) • Argument from Rules (17%) • Determine the overall consensus decision – Ask critical questions to check others' arguments
    19. 19. Factors/Dimensions Analysis • Factors (case-based reasoning) – All or nothing • Either present ("applicable") or absent • When present, a factor always favors the same side • Dimensions – More complex/subtle • Can be applicable to a varying degree ("sliding scale") • Favor plantiff on one extreme; defendant on the other Ashley 1991; Bench-Capon and Rissland, 2001
    20. 20. Example factors analysis (Aleven 1997) Aleven 1997
    21. 21. Wikipedia Factors Analysis Factors determined by iterative annotation 4 Factors cover – 91% of comments – 70% of discussions “Other” as 5th catchall Factor Example (used to justify `keep') Notability Anyone covered by another encyclopedic reference is considered notable enough for inclusion in Wikipedia. Sources Basic information about this album at a minimum is certainly verifiable, it's a major label release, and a highly notable band. Maintenance …this article is savable but at its current state, needs a lot of improvement. Bias It is by no means spam (it does not promote the products). **Other I'm advocating a blanket "hangon" for all articles on newly-drafted players
    22. 22. Wikipedia Factors Analysis Factors determined by iterative annotation 4 Factors cover – 91% of comments – 70% of discussions “Other” as 5th catchall Factor Example (used to justify `keep') Notability Anyone covered by another encyclopedic reference is considered notable enough for inclusion in Wikipedia. Sources Basic information about this album at a minimum is certainly verifiable, it's a major label release, and a highly notable band. Maintenance …this article is savable but at its current state, needs a lot of improvement. Bias It is by no means spam (it does not promote the products). **Other I'm advocating a blanket "hangon" for all articles on newly-drafted players
    23. 23. Wikipedia Factors Analysis Factor Example (used to justify 'keep') Example (used to justify 'delete' Notability Anyone covered by another encyclopedic reference is considered notable enough for inclusion in Wikipedia. There is simply no coverage in reliable sources to establish notability. Sources Basic information about this album at a minimum is certainly verifiable, it's a major label release, and a highly notable band. There are no independent secondary sources (books, magazine articles, documentaries, etc.) about her. Maintenance …this article is savable but at its current state, needs a lot of improvement. Too soon for a page likely to be littered with rumour and speculation. Bias It is by no means spam (it does not promote the products). The article seems to have been created by her or her agent as a promotional device. **Other I'm advocating a blanket "hangon" for all articles on newly-drafted players it appears to be original research by synthesis Deletion Discussions in Wikipedia: Decision Factors and Outcomes. WikiSym 2012.
    24. 24. Supporting Tasks with Factors • Convince others of your position, using community norms – To win an argument, talk about the right topics • Notability, Sources, Maintenance, Bias • Determine the overall consensus decision – Group messages by factor – Summarize prevalence
    25. 25. Factor-based Summarization
    26. 26. Argument Schemes vs. Factors? • Argument Schemes (kappa=.48) Details of how to put together an argument – Could support WRITING detailed arguments – Critical Questioning • Factors (kappa=.64-.82, based on factor) Topics of discussion – Basic support for writing arguments – Summarization supports decision-making
    27. 27. Argument prevalence depends on the corpus • Wikipedia – Argument from Evidence to Hypothesis (19%) – Argument from Rules (17%) • Arucaria – Argument from example (38%) – Argument from cause to effect (27%) – Practical reasoning (14%) – Argument from consequences (11%) – Argument from verbal classification (10%)
    28. 28. Topics • Examples of argumentation support • Supporting Collaborative Online Arguing • Micropublications model: Structuring scientific argument
    29. 29. Micropublications
    30. 30. Model Data, Methods, Materials, References Micropublications: a Semantic Model for Claims, Evidence, Arguments and Annotations in Biomedical Communications Tim Clark, Paolo N. Ciccarese, Carole A. Goble http://arxiv.org/abs/1305.3506
    31. 31. Direct Annotation with Domeo http://swan.mindinformatics.org/ Paolo N Ciccarese
    32. 32. Micropublication: Claim + Support (e.g. Attribution) Micropublications: a Semantic Model for Claims, Evidence, Arguments and Annotations in Biomedical Communications Tim Clark, Paolo N. Ciccarese, Carole A. Goble http://arxiv.org/abs/1305.3506
    33. 33. Constructs claim-argument network across scientific papers Micropublications: a Semantic Model for Claims, Evidence, Arguments and Annotations in Biomedical Communications Tim Clark, Paolo N. Ciccarese, Carole A. Goble http://arxiv.org/abs/1305.3506
    34. 34. Argumentation Mining papers Arguing on Wikipedia • “Arguments about Deletion: How Experience Improves the Acceptability of Arguments in Ad-hoc Online Task Groups” CSCW 2013. • “Deletion Discussions in Wikipedia: Decision Factors and Outcomes” WikiSym2012. Arguing in Social Media • “Dimensions of Argumentation in Social Media" EKAW 2012 • “Why did they post that argument? Communicative intentions of Web 2.0 arguments.” Arguing on the Web 2.0 at ISSA 2014 Arguing in Reviews • “Identifying Consumers' Arguments in Text” SWAIE 2012 • “Semi-Automated Argumentative Analysis of Online Product Reviews" COMMA 2012 • “Arguing from a Point of View” Agreement Technologies 2012 Structuring Arguments on the Social Semantic Web • “A Review of Argumentation for the Social Semantic Web” Semantic Web – Interoperability, Usability, Applicability, 2013. • “Identifying, Annotating, and Filtering Arguments and Opinions in Open Collaboration Systems" 2013 Thesis: purl.org/jsphd • “Modeling Arguments in Scientific Papers” at ArgDiaP 2014 http://jodischneider.com/jodi.html
    35. 35. Argumentation mining today • No unified vision of the field. Multiple: – Interrelated problems – Application domains – Tools handling one aspect of annotation • Few corpora • Need for – Common definition(s) of argumentation – "Challenge problems" – Shared corpora – Applications
    36. 36. Example: "Stop at a red light" 1. Does the rule require carrying out this type of action? Were you driving a vehicle? 2. Are there other established rules that might conflict with or override this one? Did a police officer direct you to continue without stopping? 3. Are there extenuating circumstances or an excuse for noncompliance? Were you driving an ambulance with its siren on? Critical Questions from Argument from Rules based on Walton, Reed, and Macagno 2008
    37. 37. None of Wikipedia's top-used schemes are prevalent in Arucaria. Classifying Arguments by Scheme. Vanessa Wei Feng. Master's thesis, Toronto, 2010.
    38. 38. Goal: large-scale arguing • Search for issues, claims, and opinion clusters • Link to evidence when writing your opinion • Publish and navigate claims networks
    39. 39. Online argumentation support interfaces can: • Promote "listening" in online conversations • Support incremental formalization • Slice and dice the views • Collect crisp examples • Support distributed sensemaking
    40. 40. Argumentation mining could be the basis for support tools • Help participants write persuasive arguments – How: provide personalized feedback on drafts – Requires: knowing which arguments are accepted; identifying argumentation in a drafts • Find weaknesses in others’ arguments – How: suggest & instantiate relevant critical questions – Requires: identifying argumentation schemes • Summarize the overall conclusions of the debate – How: identify the winning and losing rationales – Requires: identifying rationales and contradictions
    1. Gostou de algum slide específico?

      Recortar slides é uma maneira fácil de colecionar informações para acessar mais tarde.

    ×