Thesis summary-arguments-about-deleting-wikipedia-content-paris-2013-04-19


Published on

Overview of my thesis in Wikipedia deletion discussions

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Felipe Ortega via editors are leaving faster than they can be replaced1 in 3 editors begin by creating a new article7 times as likely to stay if their article is kept
  • “only 0.6 percent of those whose articles are met with deletion stayed editing, compared to 4.4 percent of the users whose articles remained”, Signpost/2011-04-04/Editor_retention
  • Interviews via various means: (skype, IRC, in person)The story: understand the problem (analysis / survey), solve it (define method / tools + analysed criteria), evaluate it (prototype)
  • By typical we mean average volume: there are consistently ~500 discussions per week about deleting borderline articles, see our WikiSym paper.
  • Mentoring in discussions is effective: Article creators who receive mentoring seem toMake more edits to the articleContinue editingIncrease understanding of policy
  • Experts argue from precedentNovices: values, analogy, cause to effectJodi Schneider, KrystianSamp, Alexandre Passant, and Stefan Decker. Arguments about Deletion: How Experience Improves the Acceptability of Arguments in Ad-hoc Online Task Groups. Computer-Supported Cooperative Work and Social Computing (CSCW 2013).
  • 3 student annotators (besides me)Iterative refinement of annotation manualGood interannotator agreement
  • 20 novice participants used both systems“The ability to navigate the comments made it a bit easier to filter my mind set and to come to a conclusion.”“summarise and, at the same time, evaluate which factor should be considered determinant for the final decision”
  • Thesis summary-arguments-about-deleting-wikipedia-content-paris-2013-04-19

    1. 1. Copyright 2011 Digital Enterprise Research Institute. All rights reserved.Digital Enterprise Research Institute www.deri.ieEnabling Networked KnowledgeArguments about deleting Wikipedia contentJodi Schneiderjschneider@pobox.comVendredi 19th April 20131Télécom ParisTech
    2. 2. Is Wikipedia Sustainable?
    3. 3. Deletion threatens Wikipedia• 1 in 4 new Wikipedia articles is deleted –within minutes or hours• Demotivating!– 1 in 3 newcomers start by writing a new article– 7X less likely to stay if their article is deleted!• Can we support editor retention?
    4. 4. Ph.D. case study: argumentative dialoguesabout deleting Wikipedia articles• Goals:– Understand collaboration & coordination– Identify “pain points” & new IT support opportunities• Approaches:– Net-ethnography• Interviews of community members• Embedded participation• Reading essays, policies, & written dialogues• Analysing article history, user contributions– Content analysis• Departure point: grounded theory or existing categories. With multipleannotators, iteratively refined annotation manual to achieve strong interannotatoragreement.• Decision factors (WikiSym 2012)• Walton’s argumentation schemes (CSCW 2013)– Prototyping & iterative design• Design (WikiSym 2012 demo)• User study (reported in dissertation)
    5. 5. Corpus• Article deletion dialoguesfrom English Wikipediastarted on a typical-volume day• 72 dialogues (94 A4 pages)
    6. 6. Findings: pain points ofarticle deletion• Article creators• Novices visiting or newly joining Wikipedia• No-consensus dialogues
    7. 7. Article creators• Misunderstand policy– “I do understand that articles on wikipedia need to besourced… it is due to have two [sources] once [ourwebsite goes] live”• Express high levels of emotion– “To be honest its been a real turn off adding articlesto WP and I dont think I will add articles again. Sosmile and enjoy.”• Learn from discussions– “much as it would break my heart … it is perhapssensible that the piece is deleted.”Net-ethnographyin 8th International Symposium on Wikis and Open Collaboration(WikiSym 2012)
    8. 8. Novices’ arguments• Structurally different to experts’ arguments• More problematic arguments from novices– Personal preference– Requesting a favor– Analogy to other cases– No harm in keeping an article– Large number of search engine hitsArgumentation schemes content analysisin 16th ACM Conference on Computer-SupportedCooperative Work and Social Computing (CSCW 2013)
    9. 9. No consensus discussions“What works well is simply the communityagreeing on a verdict.”Otherwise:• Time-consuming & difficult to judge a case• Same case may get raised repeatedly• Emotional upset is more likely– “messy”, “full of hate and pain” when overturnedNet-ethnography & interviewsin 8th International Symposium on Wikis andOpen Collaboration (WikiSym 2012)
    10. 10. Articulate criteriaDecision factors content analysisin 8th International Symposium on Wikis and Open Collaboration(WikiSym 2012)4 Factors cover– 91% ofcomments– 70% ofdiscussionsFactor Example (used to justify `keep)Notability Anyone covered by another encyclopedicreference is considered notable enoughfor inclusion in Wikipedia.Sources Basic information about this album at aminimum is certainly verifiable, its amajor label release, and a highly notableband.Maintenance …this article is savable but at its currentstate, needs a lot of improvement.Bias It is by no means spam (it does notpromote the products).Other Im advocating a blanket "hangon" for allarticles on newly- drafted players
    11. 11. Use criteria to augment interfacePrototype design (RDFa; custom ontology based on FOAF, SIOC)in WikiSym 2012 Demos
    12. 12. 84% prefer our system“Information is structured and I can quickly get anoverview of the key arguments.”“The ability to navigate the comments made it a bit easierto filter my mind set and to come to a conclusion.”“It offers the structure needed to consider each factorseparately, thus making the decision easier. Also, thenumber of comments per factor offers a quick indicationof the relevance and the deepness of the decision.”Based on a formative evaluation user study with 20 novice usersin dissertation “Enabling reuse of arguments and opinions from online social disputes”