Thesis summary-arguments-about-deleting-wikipedia-content-paris-2013-04-19
Upcoming SlideShare
Loading in...5

Thesis summary-arguments-about-deleting-wikipedia-content-paris-2013-04-19



Overview of my thesis in Wikipedia deletion discussions

Overview of my thesis in Wikipedia deletion discussions



Total Views
Slideshare-icon Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment
  • Felipe Ortega via editors are leaving faster than they can be replaced1 in 3 editors begin by creating a new article7 times as likely to stay if their article is kept
  • “only 0.6 percent of those whose articles are met with deletion stayed editing, compared to 4.4 percent of the users whose articles remained”, Signpost/2011-04-04/Editor_retention
  • Interviews via various means: (skype, IRC, in person)The story: understand the problem (analysis / survey), solve it (define method / tools + analysed criteria), evaluate it (prototype)
  • By typical we mean average volume: there are consistently ~500 discussions per week about deleting borderline articles, see our WikiSym paper.
  • Mentoring in discussions is effective: Article creators who receive mentoring seem toMake more edits to the articleContinue editingIncrease understanding of policy
  • Experts argue from precedentNovices: values, analogy, cause to effectJodi Schneider, KrystianSamp, Alexandre Passant, and Stefan Decker. Arguments about Deletion: How Experience Improves the Acceptability of Arguments in Ad-hoc Online Task Groups. Computer-Supported Cooperative Work and Social Computing (CSCW 2013).
  • 3 student annotators (besides me)Iterative refinement of annotation manualGood interannotator agreement
  • 20 novice participants used both systems“The ability to navigate the comments made it a bit easier to filter my mind set and to come to a conclusion.”“summarise and, at the same time, evaluate which factor should be considered determinant for the final decision”

Thesis summary-arguments-about-deleting-wikipedia-content-paris-2013-04-19 Thesis summary-arguments-about-deleting-wikipedia-content-paris-2013-04-19 Presentation Transcript

  • Copyright 2011 Digital Enterprise Research Institute. All rights reserved.Digital Enterprise Research Institute www.deri.ieEnabling Networked KnowledgeArguments about deleting Wikipedia contentJodi Schneiderjschneider@pobox.comVendredi 19th April 20131Télécom ParisTech
  • Is Wikipedia Sustainable?
  • Deletion threatens Wikipedia• 1 in 4 new Wikipedia articles is deleted –within minutes or hours• Demotivating!– 1 in 3 newcomers start by writing a new article– 7X less likely to stay if their article is deleted!• Can we support editor retention?
  • Ph.D. case study: argumentative dialoguesabout deleting Wikipedia articles• Goals:– Understand collaboration & coordination– Identify “pain points” & new IT support opportunities• Approaches:– Net-ethnography• Interviews of community members• Embedded participation• Reading essays, policies, & written dialogues• Analysing article history, user contributions– Content analysis• Departure point: grounded theory or existing categories. With multipleannotators, iteratively refined annotation manual to achieve strong interannotatoragreement.• Decision factors (WikiSym 2012)• Walton’s argumentation schemes (CSCW 2013)– Prototyping & iterative design• Design (WikiSym 2012 demo)• User study (reported in dissertation)
  • Corpus• Article deletion dialoguesfrom English Wikipediastarted on a typical-volume day• 72 dialogues (94 A4 pages)
  • Findings: pain points ofarticle deletion• Article creators• Novices visiting or newly joining Wikipedia• No-consensus dialogues
  • Article creators• Misunderstand policy– “I do understand that articles on wikipedia need to besourced… it is due to have two [sources] once [ourwebsite goes] live”• Express high levels of emotion– “To be honest its been a real turn off adding articlesto WP and I dont think I will add articles again. Sosmile and enjoy.”• Learn from discussions– “much as it would break my heart … it is perhapssensible that the piece is deleted.”Net-ethnographyin 8th International Symposium on Wikis and Open Collaboration(WikiSym 2012)
  • Novices’ arguments• Structurally different to experts’ arguments• More problematic arguments from novices– Personal preference– Requesting a favor– Analogy to other cases– No harm in keeping an article– Large number of search engine hitsArgumentation schemes content analysisin 16th ACM Conference on Computer-SupportedCooperative Work and Social Computing (CSCW 2013)
  • No consensus discussions“What works well is simply the communityagreeing on a verdict.”Otherwise:• Time-consuming & difficult to judge a case• Same case may get raised repeatedly• Emotional upset is more likely– “messy”, “full of hate and pain” when overturnedNet-ethnography & interviewsin 8th International Symposium on Wikis andOpen Collaboration (WikiSym 2012)
  • Articulate criteriaDecision factors content analysisin 8th International Symposium on Wikis and Open Collaboration(WikiSym 2012)4 Factors cover– 91% ofcomments– 70% ofdiscussionsFactor Example (used to justify `keep)Notability Anyone covered by another encyclopedicreference is considered notable enoughfor inclusion in Wikipedia.Sources Basic information about this album at aminimum is certainly verifiable, its amajor label release, and a highly notableband.Maintenance …this article is savable but at its currentstate, needs a lot of improvement.Bias It is by no means spam (it does notpromote the products).Other Im advocating a blanket "hangon" for allarticles on newly- drafted players
  • Use criteria to augment interfacePrototype design (RDFa; custom ontology based on FOAF, SIOC)in WikiSym 2012 Demos
  • 84% prefer our system“Information is structured and I can quickly get anoverview of the key arguments.”“The ability to navigate the comments made it a bit easierto filter my mind set and to come to a conclusion.”“It offers the structure needed to consider each factorseparately, thus making the decision easier. Also, thenumber of comments per factor offers a quick indicationof the relevance and the deepness of the decision.”Based on a formative evaluation user study with 20 novice usersin dissertation “Enabling reuse of arguments and opinions from online social disputes”