Jodi Schneider
SICSA Workshop on Argument Mining:
Perspectives from Information Extraction, Information
Retrieval and Comp...
Argumentation mining today
• No unified vision of the field. Multiple:
– Interrelated problems
– Application domains
– Too...
Argumentation mining today
• No unified vision of the field. Multiple:
– Interrelated problems
– Application domains
– Too...
Evidence Informatics
• How do we support people in making
evidence-based decisions?
• This includes supporting their argum...
Informatics Perspective
1. Analyze requirements
2. Consider which argumentation models to use
3. Build a prototype support...
Two examples of evidence informatics
• Evidence about article quality in Wikipedia
(PhD)
• Evidence about pharmaceutical d...
http://purl.org/jsphd
Which content belongs in Wikipedia?
Tasks using evidence & arguments
• Convince others of your position, using
community norms
• Determine the overall consens...
Compare two argumentation theories
• Walton’s Argumentation Schemes
(Walton, Reed, and Macagno 2008)
– Informal argumentat...
Walton’s Argumentation Schemes
Example Argumentation Scheme:
Argument from Rules – “we apply rule X”
Critical Questions
1....
“Rule” Argumentation Scheme
“Arguments about Deletion: How Experience Improves the Acceptability of Arguments in Ad-hoc On...
“Evidence” Argumentation Scheme
“Arguments about Deletion: How Experience Improves the Acceptability of Arguments in Ad-ho...
Evidence + Rule -> Conclusion
“Arguments about Deletion: How Experience Improves the Acceptability of Arguments in Ad-hoc ...
Supporting Tasks with Walton
• Convince others of your position, using
community norms
– To win an argument, use popular s...
Example factors analysis
(Aleven 1997)
Aleven 1997
Wikipedia Factors Analysis
Factor Example (used to justify 'keep') Example (used to justify 'delete'
Notability Anyone cov...
Supporting Tasks with Factors
• Convince others of your position, using
community norms
– To win an argument, talk about t...
Factor-based Summarization
Argument Schemes vs. Factors?
• Argument Schemes
Details of how to put together an argument
– Could support WRITING detail...
Jodi Schneider, Carol Collins, Lisa E Hines,
John R Horn, and Richard Boyce
12th Argumentation, Dialogue, Persuasion
confe...
Goal: Support evidence-based updates
to drug-interaction reference DBs
• Make sense of the EVIDENCE
– New clinical trials
...
Drug Interaction Knowledge Base
(DIKB) - Boyce 2007-2009
– Hand-constructed knowledge base
– Safety issues when 2 drugs ar...
• Evidence
Micropublications
Direct Annotation with Domeo
http://swan.mindinformatics.org/ Paolo N Ciccarese
Textual quotes
Micropublication: Claim + Support
(e.g. Attribution)
Micropublications: a Semantic Model for Claims, Evidence, Arguments a...
Constructs claim-argument network
across scientific papers
Micropublications: a Semantic Model for Claims, Evidence, Argum...
Model Data,
Methods,
Materials,
References
Micropublications: a Semantic Model for
Claims, Evidence, Arguments and
Annotat...
Micropublications Ontology
Micropublications: a Semantic Model for Claims, Evidence, Arguments and Annotations in Biomedic...
Methods
Methods section of challenge graph
"escitalopram does not
inhibit CYP2D6"
Support graph Challenge graph
From individual documents to a
searchable claim-argument network
• "Pay as you go" annotation of source
documents with Dom...
Argumentation Mining papers
Arguing on Wikipedia
• “Arguments about Deletion: How Experience Improves the Acceptability of...
Example: "Stop at a red light"
1. Does the rule require carrying out this type of action?
Were you driving a vehicle?
2. A...
How to win an argument (Arucaria)?
Classifying Arguments by Scheme. Vanessa Wei Feng. Master's thesis, Toronto, 2010.
Argumentation mining could be the
basis for support tools
• Help participants write persuasive arguments
– How: provide pe...
Experts vs. Novices
• Experts were more likely to use
– Argument from Precedent
• Novices were more likely to use
– Argume...
Unsuccessful arguments from novices
• Emsworth Cricket Club is one of the oldest
cricket clubs in the world, and this real...
Newcomers don't understand
how to counterargue
Wikipedia Factors Analysis
Factors determined
by iterative annotation
4 Factors cover
– 91% of comments
– 70% of discussio...
An informatics perspective on argumentation mining - SICSA 2014-07-09
Upcoming SlideShare
Loading in...5
×

An informatics perspective on argumentation mining - SICSA 2014-07-09

448

Published on

Informal talk for the SICSA argumentation mining workshop: http://www.arg-tech.org/index.php/sicsa-workshop-on-argument-mining-2014/

For more details, see two related papers:
(1) Automated argumentation mining to the rescue? Envisioning argumentation and decision-making support for debates in open online collaboration communities.
ACL First Workshop on Argumentation Mining (summary of my PhD work)
http://jodischneider.com/pubs/aclargmining2014.pdf

(2) Modeling Arguments in Scientific Papers
Jodi Schneider, Carol Collins, Lisa Hines, John R Horn and Richard Boyce
ArgDiaP conference
http://jodischneider.com/pubs/argdiap2014.pdf

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
448
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • http://www.sintelnet.eu/content/arguing-web-20-0

    15:50-16:10: 15 min + 5 min for questions

    Full paper online at:
    http://jodischneider.com/pubs/issaargweb2014.pdf

    Abstract:
    The Web is an open platform where users are free to publish their own opinions, to discuss the latest news, to write reviews about a service or product. Given the growing number of online platforms allowing such interactions, it is becoming more and more important to support users in understanding the meaning of such conversations by identifying the context of the discourse, and the evidence and background knowledge required to evaluate the proposed arguments. In this paper, we suggest recording the rough context with communicative intentions to help classify arguments on the Web. We describe five communicative intentions: recreation, information, instruction, discussion, and recommendation. We suggest that this classification can help identify and analyze messages for the Argument Web. In particular, we discuss the suitable combinations of natural language processing techniques and argumentation-based reasoning to support users in understanding the discussions.
  • Envisioning argumentation and decision-making support for debates in open online collaboration communities
  • Categories (Walton’s argumentation schemes) vs. process (factors analysis)

  • Major Premise: If carrying out types of actions including A is the established rule for x, then (unless the case is an exception), a must carry out A.
    Minor Premise: Carrying out types of actions including A is the established rule for a.
    Conclusion: Therefore, a must carry out A.
  • We would count this as an Argument from Rules
  • Factors provide a good way to organize the debate;

    Filtering discussions based on each factor can show the rationale topic by topic, which supported decision making in a pilot user-based evaluation

    16 of 19 participants (84%) preferred
  • When the argumentation scheme used in a draft message is not generally accepted, the author could be warned that their message might not be persuasive, and given personalized suggestions

    Listing these questions in concrete and contextualized form (drawing on the premises, inference rules, and conclusions to instantiate and contextualize them) would encourage participants to consider the pos- sible flaws in reasoning and might prompt partici- pants to request answers within the debate.

    Macro- argumentation, such as the factors analysis de- scribed above, would be a natural choice for sum- marization, as it has already proven useful for fil- tering discussions. A more reasoning-intensive approach would be to calculate consistent out- comes (Wyner and van Engers, 2010), if debates can be easily formalized.

  • An informatics perspective on argumentation mining - SICSA 2014-07-09

    1. 1. Jodi Schneider SICSA Workshop on Argument Mining: Perspectives from Information Extraction, Information Retrieval and Computational Linguistics Dundee, Scotland 9-10 July 2014.
    2. 2. Argumentation mining today • No unified vision of the field. Multiple: – Interrelated problems – Application domains – Tools handling one aspect of annotation • Few corpora • Need for – Common definition(s) of argumentation – "Challenge problems" – Shared corpora – Applications
    3. 3. Argumentation mining today • No unified vision of the field. Multiple: – Interrelated problems – Application domains – Tools handling one aspect of annotation • Few corpora • Need for – Common definition(s) of argumentation – "Challenge problems" – Shared corpora – Applications
    4. 4. Evidence Informatics • How do we support people in making evidence-based decisions? • This includes supporting their arguments!
    5. 5. Informatics Perspective 1. Analyze requirements 2. Consider which argumentation models to use 3. Build a prototype support tool 4. Evaluate and iterate
    6. 6. Two examples of evidence informatics • Evidence about article quality in Wikipedia (PhD) • Evidence about pharmaceutical drug interactions (ArgDiaP 2014)
    7. 7. http://purl.org/jsphd
    8. 8. Which content belongs in Wikipedia?
    9. 9. Tasks using evidence & arguments • Convince others of your position, using community norms • Determine the overall consensus decision
    10. 10. Compare two argumentation theories • Walton’s Argumentation Schemes (Walton, Reed, and Macagno 2008) – Informal argumentation (philosophical & computational argumentation) – Identify & prevent errors in reasoning (fallacies) – 60 patterns • Factors/Dimensions Analysis (Ashley 1991; Bench-Capon and Rissland, 2001) – Case-based reasoning – E.g. factors for deciding cases in trade secret law, favoring either party (the plaintiff or the defendant).
    11. 11. Walton’s Argumentation Schemes Example Argumentation Scheme: Argument from Rules – “we apply rule X” Critical Questions 1. Does the rule require carrying out this type of action? 2. Are there other established rules that might conflict with or override this one? 3. Are there extenuating circumstances or an excuse for noncompliance? Walton, Reed, and Macagno 2008
    12. 12. “Rule” Argumentation Scheme “Arguments about Deletion: How Experience Improves the Acceptability of Arguments in Ad-hoc Online Task Groups” CSCW 2013
    13. 13. “Evidence” Argumentation Scheme “Arguments about Deletion: How Experience Improves the Acceptability of Arguments in Ad-hoc Online Task Groups” CSCW 2013
    14. 14. Evidence + Rule -> Conclusion “Arguments about Deletion: How Experience Improves the Acceptability of Arguments in Ad-hoc Online Task Groups” CSCW 2013
    15. 15. Supporting Tasks with Walton • Convince others of your position, using community norms – To win an argument, use popular schemes: • Argument from Evidence to Hypothesis (19%) • Argument from Rules (17%) • Determine the overall consensus decision – Ask critical questions to check others' arguments
    16. 16. Example factors analysis (Aleven 1997) Aleven 1997
    17. 17. Wikipedia Factors Analysis Factor Example (used to justify 'keep') Example (used to justify 'delete' Notability Anyone covered by another encyclopedic reference is considered notable enough for inclusion in Wikipedia. There is simply no coverage in reliable sources to establish notability. Sources Basic information about this album at a minimum is certainly verifiable, it's a major label release, and a highly notable band. There are no independent secondary sources (books, magazine articles, documentaries, etc.) about her. Maintenance …this article is savable but at its current state, needs a lot of improvement. Too soon for a page likely to be littered with rumour and speculation. Bias It is by no means spam (it does not promote the products). The article seems to have been created by her or her agent as a promotional device. **Other I'm advocating a blanket "hangon" for all articles on newly-drafted players it appears to be original research by synthesis Deletion Discussions in Wikipedia: Decision Factors and Outcomes. WikiSym 2012.
    18. 18. Supporting Tasks with Factors • Convince others of your position, using community norms – To win an argument, talk about the right topics • Notability, Sources, Maintenance, Bias • Determine the overall consensus decision – Group messages by factor – summarize prevalence
    19. 19. Factor-based Summarization
    20. 20. Argument Schemes vs. Factors? • Argument Schemes Details of how to put together an argument – Could support WRITING detailed arguments – Critical Questioning • Factors Topics of discussion – Basic support for writing arguments – Summarization supports decision-making
    21. 21. Jodi Schneider, Carol Collins, Lisa E Hines, John R Horn, and Richard Boyce 12th Argumentation, Dialogue, Persuasion conference (ArgDiaP 2014) Warsaw, Poland 2014-05-25
    22. 22. Goal: Support evidence-based updates to drug-interaction reference DBs • Make sense of the EVIDENCE – New clinical trials – Adverse drug event reports – Drug product labels – Updates to regulatory information (U.S. FDA,…) – … • Significant discrepancies between different drug-interaction reference DBs http://jama.jamanetwork.com/article.aspx?articleid=183454
    23. 23. Drug Interaction Knowledge Base (DIKB) - Boyce 2007-2009 – Hand-constructed knowledge base – Safety issues when 2 drugs are taken together – Focus is on EVIDENCE
    24. 24. • Evidence
    25. 25. Micropublications
    26. 26. Direct Annotation with Domeo http://swan.mindinformatics.org/ Paolo N Ciccarese
    27. 27. Textual quotes
    28. 28. Micropublication: Claim + Support (e.g. Attribution) Micropublications: a Semantic Model for Claims, Evidence, Arguments and Annotations in Biomedical Communications Tim Clark, Paolo N. Ciccarese, Carole A. Goble http://arxiv.org/abs/1305.3506
    29. 29. Constructs claim-argument network across scientific papers Micropublications: a Semantic Model for Claims, Evidence, Arguments and Annotations in Biomedical Communications Tim Clark, Paolo N. Ciccarese, Carole A. Goble http://arxiv.org/abs/1305.3506
    30. 30. Model Data, Methods, Materials, References Micropublications: a Semantic Model for Claims, Evidence, Arguments and Annotations in Biomedical Communications Tim Clark, Paolo N. Ciccarese, Carole A. Goble http://arxiv.org/abs/1305.3506
    31. 31. Micropublications Ontology Micropublications: a Semantic Model for Claims, Evidence, Arguments and Annotations in Biomedical Communications Tim Clark, Paolo N. Ciccarese, Carole A. Goble http://arxiv.org/abs/1305.3506
    32. 32. Methods Methods section of challenge graph
    33. 33. "escitalopram does not inhibit CYP2D6" Support graph Challenge graph
    34. 34. From individual documents to a searchable claim-argument network • "Pay as you go" annotation of source documents with Domeo & Micropublications • Generates claim-argument network – Supports & challenges – Materials, methods, data – Quotes linked into the graph – … within & across documents • Query support
    35. 35. Argumentation Mining papers Arguing on Wikipedia • “Arguments about Deletion: How Experience Improves the Acceptability of Arguments in Ad-hoc Online Task Groups” CSCW 2013. • “Deletion Discussions in Wikipedia: Decision Factors and Outcomes” WikiSym2012. Arguing in Social Media • “Dimensions of Argumentation in Social Media" EKAW 2012 • “Why did they post that argument? Communicative intentions of Web 2.0 arguments.” Arguing on the Web 2.0 at ISSA 2014 Arguing in Reviews • “Identifying Consumers' Arguments in Text” SWAIE 2012 • “Semi-Automated Argumentative Analysis of Online Product Reviews" COMMA 2012 • “Arguing from a Point of View” Agreement Technologies 2012 Structuring Arguments on the Social Semantic Web • “A Review of Argumentation for the Social Semantic Web” Semantic Web – Interoperability, Usability, Applicability, 2013. • “Identifying, Annotating, and Filtering Arguments and Opinions in Open Collaboration Systems" 2013 Thesis: purl.org/jsphd • “Modeling Arguments in Scientific Papers” at ArgDiaP 2014 http://jodischneider.com/jodi.html
    36. 36. Example: "Stop at a red light" 1. Does the rule require carrying out this type of action? Were you driving a vehicle? 2. Are there other established rules that might conflict with or override this one? Did a police officer direct you to continue without stopping? 3. Are there extenuating circumstances or an excuse for noncompliance? Were you driving an ambulance with its siren on? Critical Questions from Argument from Rules based on Walton, Reed, and Macagno 2008
    37. 37. How to win an argument (Arucaria)? Classifying Arguments by Scheme. Vanessa Wei Feng. Master's thesis, Toronto, 2010.
    38. 38. Argumentation mining could be the basis for support tools • Help participants write persuasive arguments – How: provide personalized feedback on drafts – Requires: knowing which arguments are accepted; identifying argumentation in a drafts • Find weaknesses in others’ arguments – How: suggest & instantiate relevant critical questions – Requires: identifying argumentation schemes • Summarize the overall conclusions of the debate – How: identify the winning and losing rationales – Requires: identifying rationales and contradictions
    39. 39. Experts vs. Novices • Experts were more likely to use – Argument from Precedent • Novices were more likely to use – Argumentation from Values – Argumentation from Cause to Effect – Argument from Analogy
    40. 40. Unsuccessful arguments from novices • Emsworth Cricket Club is one of the oldest cricket clubs in the world, and this really is worth a mention. Especially on a website, where pointless people … gets a mention. • Why just because it is a small team and not major does it not deserve it’s (sic) own page on here?
    41. 41. Newcomers don't understand how to counterargue
    42. 42. Wikipedia Factors Analysis Factors determined by iterative annotation 4 Factors cover – 91% of comments – 70% of discussions “Other” as 5th catchall Factor Example (used to justify `keep') Notability Anyone covered by another encyclopedic reference is considered notable enough for inclusion in Wikipedia. Sources Basic information about this album at a minimum is certainly verifiable, it's a major label release, and a highly notable band. Maintenance …this article is savable but at its current state, needs a lot of improvement. Bias It is by no means spam (it does not promote the products). **Other I'm advocating a blanket "hangon" for all articles on newly-drafted players
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×