Informal talk for the SICSA argumentation mining workshop: http://www.arg-tech.org/index.php/sicsa-workshop-on-argument-mining-2014/ For more details, see two related papers: (1) Automated argumentation mining to the rescue? Envisioning argumentation and decision-making support for debates in open online collaboration communities. ACL First Workshop on Argumentation Mining (summary of my PhD work) http://jodischneider.com/pubs/aclargmining2014.pdf (2) Modeling Arguments in Scientific Papers Jodi Schneider, Carol Collins, Lisa Hines, John R Horn and Richard Boyce ArgDiaP conference http://jodischneider.com/pubs/argdiap2014.pdf

1. Jodi Schneider
SICSA Workshop on Argument Mining:
Perspectives from Information Extraction, Information
Retrieval and Computational Linguistics
Dundee, Scotland
9-10 July 2014.

2. Argumentation mining today
• No unified vision of the field. Multiple:
– Interrelated problems
– Application domains
– Tools handling one aspect of annotation
• Few corpora
• Need for
– Common definition(s) of argumentation
– "Challenge problems"
– Shared corpora
– Applications

3. Argumentation mining today
• No unified vision of the field. Multiple:
– Interrelated problems
– Application domains
– Tools handling one aspect of annotation
• Few corpora
• Need for
– Common definition(s) of argumentation
– "Challenge problems"
– Shared corpora
– Applications

4. Evidence Informatics
• How do we support people in making
evidence-based decisions?
• This includes supporting their arguments!

5. Informatics Perspective
1. Analyze requirements
2. Consider which argumentation models to use
3. Build a prototype support tool
4. Evaluate and iterate

6. Two examples of evidence informatics
• Evidence about article quality in Wikipedia
(PhD)
• Evidence about pharmaceutical drug
interactions (ArgDiaP 2014)

7. http://purl.org/jsphd

8. Which content belongs in Wikipedia?

9. Tasks using evidence & arguments
• Convince others of your position, using
community norms
• Determine the overall consensus decision

10. Compare two argumentation theories
• Walton’s Argumentation Schemes
(Walton, Reed, and Macagno 2008)
– Informal argumentation
(philosophical & computational argumentation)
– Identify & prevent errors in reasoning (fallacies)
– 60 patterns
• Factors/Dimensions Analysis
(Ashley 1991; Bench-Capon and Rissland, 2001)
– Case-based reasoning
– E.g. factors for deciding cases in trade secret law,
favoring either party (the plaintiff or the defendant).

11. Walton’s Argumentation Schemes
Example Argumentation Scheme:
Argument from Rules – “we apply rule X”
Critical Questions
1. Does the rule require carrying out this type of action?
2. Are there other established rules that might conflict
with or override this one?
3. Are there extenuating circumstances or an excuse for
noncompliance?
Walton, Reed, and Macagno 2008

12. “Rule” Argumentation Scheme
“Arguments about Deletion: How Experience Improves the Acceptability of Arguments in Ad-hoc Online Task Groups”
CSCW 2013

13. “Evidence” Argumentation Scheme
“Arguments about Deletion: How Experience Improves the Acceptability of Arguments in Ad-hoc Online Task Groups”
CSCW 2013

14. Evidence + Rule -> Conclusion
“Arguments about Deletion: How Experience Improves the Acceptability of Arguments in Ad-hoc Online Task Groups”
CSCW 2013

15. Supporting Tasks with Walton
• Convince others of your position, using
community norms
– To win an argument, use popular schemes:
• Argument from Evidence to Hypothesis (19%)
• Argument from Rules (17%)
• Determine the overall consensus decision
– Ask critical questions to check others' arguments

16. Example factors analysis
(Aleven 1997)
Aleven 1997

17. Wikipedia Factors Analysis
Factor Example (used to justify 'keep') Example (used to justify 'delete'
Notability Anyone covered by another
encyclopedic reference is considered
notable enough for inclusion in
Wikipedia.
There is simply no coverage in
reliable sources to establish
notability.
Sources Basic information about this album
at a minimum is certainly verifiable,
it's a major label release, and a
highly notable band.
There are no independent
secondary sources (books, magazine
articles, documentaries, etc.) about
her.
Maintenance …this article is savable but at its
current state, needs a lot of
improvement.
Too soon for a page likely to be
littered with rumour and
speculation.
Bias It is by no means spam (it does not
promote the products).
The article seems to have been
created by her or her agent as a
promotional device.
**Other I'm advocating a blanket "hangon"
for all articles on newly-drafted
players
it appears to be original research by
synthesis
Deletion Discussions in Wikipedia: Decision Factors and Outcomes. WikiSym 2012.

18. Supporting Tasks with Factors
• Convince others of your position, using
community norms
– To win an argument, talk about the right topics
• Notability, Sources, Maintenance, Bias
• Determine the overall consensus decision
– Group messages by factor
– summarize prevalence

19. Factor-based Summarization

20. Argument Schemes vs. Factors?
• Argument Schemes
Details of how to put together an argument
– Could support WRITING detailed arguments
– Critical Questioning
• Factors
Topics of discussion
– Basic support for writing arguments
– Summarization supports decision-making

21. Jodi Schneider, Carol Collins, Lisa E Hines,
John R Horn, and Richard Boyce
12th Argumentation, Dialogue, Persuasion
conference (ArgDiaP 2014)
Warsaw, Poland
2014-05-25

22. Goal: Support evidence-based updates
to drug-interaction reference DBs
• Make sense of the EVIDENCE
– New clinical trials
– Adverse drug event reports
– Drug product labels
– Updates to regulatory
information (U.S. FDA,…)
– …
• Significant discrepancies between different
drug-interaction reference DBs
http://jama.jamanetwork.com/article.aspx?articleid=183454

23. Drug Interaction Knowledge Base
(DIKB) - Boyce 2007-2009
– Hand-constructed knowledge base
– Safety issues when 2 drugs are taken together
– Focus is on EVIDENCE

24. • Evidence

25. Micropublications

26. Direct Annotation with Domeo
http://swan.mindinformatics.org/ Paolo N Ciccarese

27. Textual quotes

28. Micropublication: Claim + Support
(e.g. Attribution)
Micropublications: a Semantic Model for Claims, Evidence, Arguments and Annotations in Biomedical Communications
Tim Clark, Paolo N. Ciccarese, Carole A. Goble
http://arxiv.org/abs/1305.3506

29. Constructs claim-argument network
across scientific papers
Micropublications: a Semantic Model for Claims, Evidence, Arguments and Annotations in Biomedical Communications
Tim Clark, Paolo N. Ciccarese, Carole A. Goble
http://arxiv.org/abs/1305.3506

30. Model Data,
Methods,
Materials,
References
Micropublications: a Semantic Model for
Claims, Evidence, Arguments and
Annotations in Biomedical
Communications
Tim Clark, Paolo N. Ciccarese, Carole A.
Goble
http://arxiv.org/abs/1305.3506

31. Micropublications Ontology
Micropublications: a Semantic Model for Claims, Evidence, Arguments and Annotations in Biomedical Communications
Tim Clark, Paolo N. Ciccarese, Carole A. Goble
http://arxiv.org/abs/1305.3506

32. Methods
Methods section of challenge graph

33. "escitalopram does not
inhibit CYP2D6"
Support graph Challenge graph

34. From individual documents to a
searchable claim-argument network
• "Pay as you go" annotation of source
documents with Domeo & Micropublications
• Generates claim-argument network
– Supports & challenges
– Materials, methods, data
– Quotes linked into the graph
– … within & across documents
• Query support

35. Argumentation Mining papers
Arguing on Wikipedia
• “Arguments about Deletion: How Experience Improves the Acceptability of Arguments
in Ad-hoc Online Task Groups” CSCW 2013.
• “Deletion Discussions in Wikipedia: Decision Factors and Outcomes” WikiSym2012.
Arguing in Social Media
• “Dimensions of Argumentation in Social Media" EKAW 2012
• “Why did they post that argument? Communicative intentions of Web 2.0 arguments.”
Arguing on the Web 2.0 at ISSA 2014
Arguing in Reviews
• “Identifying Consumers' Arguments in Text” SWAIE 2012
• “Semi-Automated Argumentative Analysis of Online Product Reviews" COMMA 2012
• “Arguing from a Point of View” Agreement Technologies 2012
Structuring Arguments on the Social Semantic Web
• “A Review of Argumentation for the Social Semantic Web” Semantic Web –
Interoperability, Usability, Applicability, 2013.
• “Identifying, Annotating, and Filtering Arguments and Opinions in Open Collaboration
Systems" 2013 Thesis: purl.org/jsphd
• “Modeling Arguments in Scientific Papers” at ArgDiaP 2014
http://jodischneider.com/jodi.html

37. Example: "Stop at a red light"
1. Does the rule require carrying out this type of action?
Were you driving a vehicle?
2. Are there other established rules that might conflict
with or override this one?
Did a police officer direct you to continue without
stopping?
3. Are there extenuating circumstances or an excuse for
noncompliance?
Were you driving an ambulance with its siren on?
Critical Questions from Argument from Rules based on Walton, Reed, and Macagno 2008

38. How to win an argument (Arucaria)?
Classifying Arguments by Scheme. Vanessa Wei Feng. Master's thesis, Toronto, 2010.

39. Argumentation mining could be the
basis for support tools
• Help participants write persuasive arguments
– How: provide personalized feedback on drafts
– Requires: knowing which arguments are accepted;
identifying argumentation in a drafts
• Find weaknesses in others’ arguments
– How: suggest & instantiate relevant critical questions
– Requires: identifying argumentation schemes
• Summarize the overall conclusions of the debate
– How: identify the winning and losing rationales
– Requires: identifying rationales and contradictions

40. Experts vs. Novices
• Experts were more likely to use
– Argument from Precedent
• Novices were more likely to use
– Argumentation from Values
– Argumentation from Cause to Effect
– Argument from Analogy

41. Unsuccessful arguments from novices
• Emsworth Cricket Club is one of the oldest
cricket clubs in the world, and this really is
worth a mention. Especially on a website,
where pointless people … gets a mention.
• Why just because it is a small team and not
major does it not deserve it’s (sic) own page
on here?

42. Newcomers don't understand
how to counterargue

43. Wikipedia Factors Analysis
Factors determined
by iterative annotation
4 Factors cover
– 91% of comments
– 70% of discussions
“Other” as 5th catchall
Factor Example (used to justify `keep')
Notability Anyone covered by another
encyclopedic reference is
considered notable enough for
inclusion in Wikipedia.
Sources Basic information about this
album at a minimum is certainly
verifiable, it's a major label
release, and a highly notable
band.
Maintenance …this article is savable but at its
current state, needs a lot of
improvement.
Bias It is by no means spam (it does
not promote the products).
**Other I'm advocating a blanket
"hangon" for all articles on
newly-drafted players