Felipe Ortega via http://www.businessinsider.com/chart-of-the-day-wikipedia-editors-2009-11Wikipedia editors are leaving faster than they can be replaced1 in 3 editors begin by creating a new article7 times as likely to stay if their article is kept
“only 0.6 percent of those whose articles are met with deletion stayed editing, compared to 4.4 percent of the users whose articles remained”, http://enwp.org/Wikipedia:Wikipedia_ Signpost/2011-04-04/Editor_retention
Interviews via various means: (skype, IRC, in person)The story: understand the problem (analysis / survey), solve it (define method / tools + analysed criteria), evaluate it (prototype)
By typical we mean average volume: there are consistently ~500 discussions per week about deleting borderline articles, see our WikiSym paper.
Mentoring in discussions is effective: Article creators who receive mentoring seem toMake more edits to the articleContinue editingIncrease understanding of policy
Experts argue from precedentNovices: values, analogy, cause to effectJodi Schneider, KrystianSamp, Alexandre Passant, and Stefan Decker. Arguments about Deletion: How Experience Improves the Acceptability of Arguments in Ad-hoc Online Task Groups. Computer-Supported Cooperative Work and Social Computing (CSCW 2013).
20 novice participants used both systems“The ability to navigate the comments made it a bit easier to filter my mind set and to come to a conclusion.”“summarise and, at the same time, evaluate which factor should be considered determinant for the final decision”
Copyright 2011 Digital Enterprise Research Institute. All rights reserved.Digital Enterprise Research Institute www.deri.ieEnabling Networked KnowledgeArguments about deleting Wikipedia contentJodi Schneiderjschneider@pobox.comVendredi 19th April 20131Télécom ParisTech
Deletion threatens Wikipedia• 1 in 4 new Wikipedia articles is deleted –within minutes or hours• Demotivating!– 1 in 3 newcomers start by writing a new article– 7X less likely to stay if their article is deleted!• Can we support editor retention?
Ph.D. case study: argumentative dialoguesabout deleting Wikipedia articles• Goals:– Understand collaboration & coordination– Identify “pain points” & new IT support opportunities• Approaches:– Net-ethnography• Interviews of community members• Embedded participation• Reading essays, policies, & written dialogues• Analysing article history, user contributions– Content analysis• Departure point: grounded theory or existing categories. With multipleannotators, iteratively refined annotation manual to achieve strong interannotatoragreement.• Decision factors (WikiSym 2012)• Walton’s argumentation schemes (CSCW 2013)– Prototyping & iterative design• Design (WikiSym 2012 demo)• User study (reported in dissertation)
Corpus• Article deletion dialoguesfrom English Wikipediastarted on a typical-volume day• 72 dialogues (94 A4 pages)
Article creators• Misunderstand policy– “I do understand that articles on wikipedia need to besourced… it is due to have two [sources] once [ourwebsite goes] live”• Express high levels of emotion– “To be honest its been a real turn off adding articlesto WP and I dont think I will add articles again. Sosmile and enjoy.”• Learn from discussions– “much as it would break my heart … it is perhapssensible that the piece is deleted.”Net-ethnographyin 8th International Symposium on Wikis and Open Collaboration(WikiSym 2012)
Novices’ arguments• Structurally different to experts’ arguments• More problematic arguments from novices– Personal preference– Requesting a favor– Analogy to other cases– No harm in keeping an article– Large number of search engine hitsArgumentation schemes content analysisin 16th ACM Conference on Computer-SupportedCooperative Work and Social Computing (CSCW 2013)
No consensus discussions“What works well is simply the communityagreeing on a verdict.”Otherwise:• Time-consuming & difficult to judge a case• Same case may get raised repeatedly• Emotional upset is more likely– “messy”, “full of hate and pain” when overturnedNet-ethnography & interviewsin 8th International Symposium on Wikis andOpen Collaboration (WikiSym 2012)
Articulate criteriaDecision factors content analysisin 8th International Symposium on Wikis and Open Collaboration(WikiSym 2012)4 Factors cover– 91% ofcomments– 70% ofdiscussionsFactor Example (used to justify `keep)Notability Anyone covered by another encyclopedicreference is considered notable enoughfor inclusion in Wikipedia.Sources Basic information about this album at aminimum is certainly verifiable, its amajor label release, and a highly notableband.Maintenance …this article is savable but at its currentstate, needs a lot of improvement.Bias It is by no means spam (it does notpromote the products).Other Im advocating a blanket "hangon" for allarticles on newly- drafted players
Use criteria to augment interfacePrototype design (RDFa; custom ontology based on FOAF, SIOC)in WikiSym 2012 Demos
84% prefer our system“Information is structured and I can quickly get anoverview of the key arguments.”“The ability to navigate the comments made it a bit easierto filter my mind set and to come to a conclusion.”“It offers the structure needed to consider each factorseparately, thus making the decision easier. Also, thenumber of comments per factor offers a quick indicationof the relevance and the deepness of the decision.”Based on a formative evaluation user study with 20 novice usersin dissertation “Enabling reuse of arguments and opinions from online social disputes”