Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Arguments about deletion cscw2013 how experience improves the acceptability of arguments in ad hoc online task groups

1,468 views

Published on

CSCW2013 talk about our analysis of Wikipedia article discussions (Articles for Deletion) using Walton's argumentation schemes.

Paper:
http://jodischneider.com/pubs/cscw2013.pdf

Abstract: Increasingly, ad-hoc online task groups must make decisions about jointly created artifacts such as open source software and Wikipedia articles. Time-consuming and laborious attention to textual discussions is needed to make such decisions, for which computer support would be beneficial. Yet there has been little study of the argumentation patterns that distributed ad-hoc online task groups use in evaluation and decision-making. In a corpus of English Wikipedia deletion discussions, we investigate the argumentation schemes used, the role of the arguer’s experience, and which arguments are acceptable to the audience. We report three main results: First, the most prevalent patterns are the Rules and Evidence schemes from Walton’s catalog of argumentation schemes [34], which comprise 36% of arguments. Second, we find that familiarity with community norms correlates with the novices’ ability to craft persuasive arguments. Third, acceptable arguments use community-appropriate rhetoric that demonstrate knowledge of policies and community values while problematic arguments are based on personal preference and inappropriate analogy to other cases.

Citation:
Jodi Schneider, Krystian Samp, Alexandre Passant, Stefan Decker. “Arguments about Deletion: How Experience Improves the Acceptability of Arguments in Ad-hoc Online Task Groups”. In Computer Supported Cooperative Work and Social Computing. San Antonio, TX, February 23-27, 2013.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Arguments about deletion cscw2013 how experience improves the acceptability of arguments in ad hoc online task groups

  1. 1. Digital Enterprise Research Institute www.deri.ie Arguments about Deletion How Experience Improves the Acceptability of Arguments in Ad-hoc Online Task Groups Jodi Schneider, Krystian Samp, Alexandre Passant, and Stefan Decker @jschneider CSCW 2013: Computer Supported Tuesday 26th February 2013 Cooperative Work and Social Computing San Antonio, Texas Copyright 2011 Digital Enterprise Research Institute. All rights reserved. Enabling Networked Knowledge 1
  2. 2. Ad-hoc online task groups• Open source software development• Collaborative writing groups• Standardization bodies
  3. 3. Let’s do something together at the banquet!
  4. 4. How’s the talk length?
  5. 5. Making an argument
  6. 6. Arguments give a POSITION & RATIONALEPosition: Add more question timeRationale: the flow of people between sessions can be distracting
  7. 7. Arguments are used in decision-makingPosition: Add more question time Attack positionRationale: the flow of people between sessions can be distracting Attack rationale Attack inference
  8. 8. John Danaher
  9. 9. Case Study of Argumentation in Wikipedia Deletion Discussions1. What arguments are given in content deletion discussions?2. Differences in novices’ and experts’ arguments?3. Which argumentation schemes are accepted?
  10. 10. Should we delete this Wikipedia article? [Delete the article]...hasnt played since 2008. His 66-73 record is far from stellar and, in my opinion, does not merit an article. >>He pitched last month and plays for the Venezuelan League. This meets our article criteria.
  11. 11. [Delete the article]...hasnt played since 2008. His 66-73 record is far from stellar and, in my opinion, does not merit an article.>>He pitched last month and plays for the Venezuelan League. This meets our article criteria.
  12. 12. “Rule” Argumentation Scheme
  13. 13. “Evidence” Argumentation Scheme
  14. 14. Evidence + Rule -> Conclusion
  15. 15. Corpus Analysis• English Wikipedia deletion discussions• Representative “typical” day: 72 discussions• Iterative Manual Annotation w/ multiple coders
  16. 16. Which arguments are given?Compared to general arguing,Wikipedia arguing involves more: • Rules • Values • Bias • Precedent • Avoiding Waste
  17. 17. Novices often misunderstand policy“if you folks had been around actively working onthe Web in 2000, you would know when the DotCom Bubble burst, many, many companies wentout of business. Servers with information aboutme... were taken off line.”
  18. 18. Problems with novices’ arguments• Confusion about what “reliable source” (RS) means• Confusion about establishing notability with RS’s• Confusion about verification and need for RS’s
  19. 19. Problems with experts’ arguments• Vagueness or lack of justification• Boilerplate messages lacking in detail• Sourcing (verification vs. importance of topic)• Mixing case-debate with policy development
  20. 20. TypicalPotentially problematic Wikipedia Argumentationargument terminology SchemesPersonal preference Values ILIKEITFew search engine hits Ignorance Google testMany search engine hits Cause to Effect Google test Values or Waste orRequesting a favor PLEASEDONT Practical ReasoningAnalogy to other cases Analogy OTHERNo harm in keeping an article Copyright 2011 Digital Enterprise Research Institute. All rights reserved. Values or Waste NOHARMTopic will be notable in the Practical Reasoning Networked Knowledge Enabling CRYSTALBALLfuture
  21. 21. Novices are more likely to use some problematic arguments • Values o“obviously of interest to the public in general” • Analogy o“just as special as an article on a breed of dog or something similar”
  22. 22. Emotion• Emotional involvement vs. action o“I know Wikipedia has a dislike for all things [article topic]” o“I and others have added several third-party reliable sources to the article.”• Understanding the process helps o “I believe that (much as it would break my heart based on the no of hours I have put in over the years working on the article) it is perhaps sensible that the piece is deleted.”
  23. 23. Future Work• Templates for effective arguments• Semi-automatic argument identification• Reusing argument analysis methods• Incentivizing social sensitivity• Classifying emotional needs and triggers• Understanding impact on newcomers & article creators
  24. 24. Future Work• Templates for effective arguments• Semi-automatic argument identification• Reusing argument analysis methods• Incentivizing social sensitivity• Classifying emotional needs and triggers• Understanding impact on newcomers & article creators Thanks! @jschneider jodi.schneider@deri.org
  25. 25. Acknowledgements• Science Foundation Ireland Grant No. SFI/09/CE/I1380 (Líon2)• Annotators: Laura O’Connor and Lyndia Peters• Trevor Bench-Capon, Luigina Ciolfi, Bernie Hogan, David Randall, Mark Snaith, Adam Wyner• Thanks to reviewers and CSCW revisions process!
  26. 26. Research Questions• [RQ1] What arguments are given?• [RQ2] Do people with different levels of experience with Wikipedia editing or the Wikipedia deletion process provide different types of arguments?• [RQ3] Which argumentation schemes are accepted?
  27. 27. Previous ResearchShallow analysis of large datasets• Redacted content • West & Lee, “What Wikipedia deletes” WikiSym 2011• Vote sequencing • Taraborelli & Ciampaglia “Beyond notability” SASOW 2011• Decision quality • Lam, Karim & Riedl “The effects of group composition on decision quality in a social production community”, GROUP 2010• Who participates, what & how much gets deleted • Priedhorsky, Chen, Lam, Panciera, Terveen, & Riedl. “Creating, destroying, and restoring value in Wikipedia”, GROUP 2007 • Geiger & Ford “Participation in Wikipedia’s article deletion processes”, WikiSym 2011 30
  28. 28. Argument from Rules - From Established RuleMajor Premise: If carrying out types of actions includingA is the established rule for x, then (unless the case is anexception), a must carry out A.Minor Premise: Carrying out types of actions including Aisthe established rule for a.Conclusion: Therefore, a must carry out A.
  29. 29. Find counterarguments with “critical questions”1. Does the rule require carrying out this type of action?2. Are there other established rules that might conflict with or override this one?3. Are there extenuating circumstances or an excuse for noncompliance?
  30. 30. InstantiatingIf stopping at a red light is the established rule fordriving a vehicle, then (unless the case is anexception), drivers must stop at a red light.Stopping at a red light is the established rule fordrivers.Therefore, drivers must stop at a red light.
  31. 31. 1. Were you driving a vehicle?2. Did a police officer direct you to continue without stopping?3. Were you driving an ambulance with its siren on?
  32. 32. Differences in Novices vs. Experts• Experts may read all debates
  33. 33. Case Study of Argumentation in Wikipedia Deletion DiscussionsRQ 1: What arguments are given in contentdeletion discussions?RQ 2: Differences in novices’ and experts’arguments?RQ 3: Which argumentation schemes areaccepted?

×