Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Engineering at the intersection of productive efficiency, ideology, and ethical ai in wikipedia

62 views

Published on

Test

Published in: Technology
  • DOWNLOAD FULL. BOOKS INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Engineering at the intersection of productive efficiency, ideology, and ethical ai in wikipedia

  1. 1. Be Nice, Be Respectful: Protecting Online Spaces with Applied Machine Learning
  2. 2. User:ClueBot NG HotCat Engineering at the Intersection of Productive Efficiency, Ideology, and Ethical AI in Wikipedia Wikimedia Research Aaron Halfaker ahalfaker@wikimedia.org
  3. 3. Planet Earth (6.97 billion) data from March 2012
  4. 4. Planet Earth (6.97 billion) data from March 2012 1:10 Monthly Wikipedia Readers (500 million)
  5. 5. Monthly Wikipedia Editors (113,304) Planet Earth (6.97 billion) data from March 2012 1:10000 1:10 Monthly Wikipedia Readers (500 million)
  6. 6. Monthly Wikipedia Editors (113,304) Planet Earth (6.97 billion) data from March 2012 Monthly Wikipedia Readers (500 million) 1:10 1:10000
  7. 7. Monthly Wikipedia Editors (113,304) Planet Earth (6.97 billion) data from March 2012 Monthly Wikipedia Readers (500 million) 1:10 1:10000
  8. 8. [edit] Monthly Wikipedia Editors (100,000) 1:10,000 Wikimedia Foundation Staff (~225) 1:1000 Monthly Wikipedia Readers (500M)
  9. 9. Wikimedia Research
  10. 10. Story Time!
  11. 11. 1. Wikipedia as a socio-technical system ○ Systems-thinking & Biological metaphors ~
  12. 12. 1. Wikipedia as a socio-technical system ○ Systems-thinking & Biological metaphors 2. Critique of algorithmic quality control ○ Standpoint epistemology ○ Encoding of ideology in technology ~
  13. 13. 1. Wikipedia as a socio-technical system ○ Systems-thinking & Biological metaphors 2. Critique of algorithmic quality control ○ Standpoint epistemology ○ Encoding of ideology in technology 3. Infrastructure for socio-technical change ○ “progress catalyst” ○ Hearing to speech vs. Speaking to be heard ○ The dangers of “subjective algorithms” ~
  14. 14. Part 1. The socio-technical of Wikipedia
  15. 15. What is Wikipedia?
  16. 16. The world’s largest encyclopedia ~5 million articles in English
  17. 17. A wiki ● Anyone can edit ● Shared authorship Flipped publication model ● publish first ● review later (maybe)
  18. 18. An online community ● ~100k active volunteer editors WikiProject Video Games WikiProject Medicine
  19. 19. A system Available Human Attention
  20. 20. A system Available Human Attention Input Output
  21. 21. A system Available Human Attention Input Output Socio-technical
  22. 22. Social Technical https://commons.wikimedia.org/wiki/File:Commodore_P ET_4016,_Google_NY_office_computer_museum.jpg https://commons.wikimedia.org/wiki/File:Kadokawa_office_(2).jpg
  23. 23. Social Technical https://commons.wikimedia.org/wiki/File:Marcus_Bai ns_Line.png https://commons.wikimedia.org/wiki/File:Balsa-gpg.png
  24. 24. Socio-technical
  25. 25. Technologist
  26. 26. Techno-biologist?
  27. 27. Bacterium Paramecium
  28. 28. Bacterium Paramecium DNA ribosomes vacuole
  29. 29. Bacterium Paramecium DNA ribosomes vacuole vacuole contractile vacuole endoplasmic reticulum oral groove macronucleus micronucleus
  30. 30. Bacterium Paramecium
  31. 31. Bacterium Paramecium Dunbar’s number: ~150 Via commons. CC-BY-SA 2.0
  32. 32. Bacterium ParameciumBare-amecium Organelles Robots & tools Policies & Guidelines
  33. 33. socio - technical
  34. 34. socio-technical
  35. 35. ~ System with specialized sub-systems System with specialized sub-systems
  36. 36. Work allocation ● Identify, prioritize and assign tasks
  37. 37. Work allocation ● Largely for free due to Linus’ Law
  38. 38. Linus’s Law?
  39. 39. Linus’s Law? Eric Raymond
  40. 40. Linus’s Law? “given enough eyeballs, all bugs are shallow”
  41. 41. Linus’s Law? “given enough eyeballs, all bugs are shallow” yessss… bugssss….
  42. 42. Linus’s Law? “given enough eyeballs, all bugs are shallow”
  43. 43. Linus’s Law? “given enough eyeballs, all bugs are shallow” !
  44. 44. Linus’s Law? visibility is critical to open collaboration
  45. 45. A corollary for Wikipedia Given that enough people see an incomplete article, all potential contributions to that article will be easy for someone. ! Works in theory -- Stvilia, B., et al. (2008). Information quality work organization in Wikipedia. JASIST, 59(6), 983-1001. Part of becoming a Wikipedian -- Bryant, S. L., Forte, A., & Bruckman, A. (2005, November). Becoming Wikipedian. GROUP (pp. 1-10). ACM. We can support visibility with technology -- Cosley, D., et al. (2007, January). SuggestBot: using intelligent task routing to help people find work in wikipedia. IUI (pp. 32-41). ACM. Bad things happen when we take it away -- Schneider, J., et al. (2014, August). Accept, decline, postpone. OpenSym(p. 26). ACM.
  46. 46. Work allocation ● Identify, prioritize and assign tasks Regulation of behavior ● Norm formation, propagation and enforcement
  47. 47. Prescriptive Idea for a Norm Descriptive Observation of consistent behavior
  48. 48. Prescriptive Idea for a Norm Descriptive Observation of consistent behavior ESSAY
  49. 49. Prescriptive Idea for a Norm Descriptive Observation of consistent behavior Requests for Comments ESSAY rabble rabble rabble
  50. 50. Prescriptive Idea for a Norm Descriptive Observation of consistent behavior Requests for Comments GUIDELINE POLICY ESSAY rabble rabble rabble
  51. 51. GUIDELINE POLICY ESSAY Informal Formalized Morgan, J. T., & Zachry, M. (2010, November). Negotiating with angry mastodons: the wikipedia policy environment as genre ecology. In Proceedings of the 16th ACM international conference on Supporting group work (pp. 165-168). ACM. Halfaker, A., Geiger, R. S., Morgan, J. T., & Riedl, J. (2012). The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline. American Behavioral Scientist, 0002764212469365.
  52. 52. Informal Formalized
  53. 53. Growth of regulations Halfaker, A., Geiger, R. S., Morgan, J. T., & Riedl, J. (2012). The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline. American Behavioral Scientist, 0002764212469365.
  54. 54. Growth of regulations Halfaker, A., Geiger, R. S., Morgan, J. T., & Riedl, J. (2012). The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline. American Behavioral Scientist, 0002764212469365.
  55. 55. Growth of regulations Halfaker, A., Geiger, R. S., Morgan, J. T., & Riedl, J. (2012). The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline. American Behavioral Scientist, 0002764212469365.
  56. 56. Work allocation ● Identify, prioritize and assign tasks Regulation of behavior ● Norm formation, propagation and enforcement Quality control ● Identify and remove damage
  57. 57. Quality control
  58. 58. Quality control !
  59. 59. Quality control ! Fully automated Machine learning ● Fast (~ 5 seconds)[1] ● No human effort ● Only obvious vandalism Semi automated Human computation ● Still pretty fast (~ 30 seconds)[1] ● Minimizes human effort ● Humans catch most vandalism at a glance 1. R. Stuart Geiger & Aaron Halfaker. When the Levee Breaks (2013). WikiSym.
  60. 60. Quality control ! Fully automated Semi automated Banning (Admins)
  61. 61. Quality control ! Fully automated Semi automated Banning (Admins) Innate - Fast - General - Local Adaptive - Slow - Specific - Global Halfaker, A. & Riedl, J. (2012) Bots and Cyborgs, IEEE Computing 45(3) (p. 79-82) Geiger, R. S., & Ribes, D. (2010, February). The work of sustaining order in wikipedia: the banning of a vandal. CSCW (pp. 117-126). ACM.
  62. 62. Work allocation ● Identify, prioritize and assign tasks Regulation of behavior ● Norm formation, propagation and enforcement Quality control ● Identify and remove damage Community management ● Newcomer socialization, dispute mediation & training
  63. 63. https://commons.wikimedia.org/wiki/File:Flickr_-_Official_U.S._Navy_Imagery_-_Sailor%27s_daug hter_operates_a_fire_hose_with_crew_member_assistance..jpg 6,000 newcomers per day
  64. 64. https://commons.wikimedia.org/wiki/File:Flickr_-_Official_U.S._Navy_Imagery_-_Sailor%27s_daug hter_operates_a_fire_hose_with_crew_member_assistance..jpg Host Bot vandals good-faith 6,000 newcomers per day
  65. 65. https://commons.wikimedia.org/wiki/File:Flickr_-_Official_U.S._Navy_Imagery_-_Sailor%27s_daug hter_operates_a_fire_hose_with_crew_member_assistance..jpg Host Bot good-faith vandals 6,000 newcomers per day
  66. 66. Work allocation ● Identify, prioritize and assign tasks Regulation of behavior ● Form, propagate and enforce norms Quality control ● Identify and remove damage Community management ● Socialize & train newcomers; mediate disputes Reflection (Adaptation) ● Where are we going? Where do we want to go? How do we want to get there?
  67. 67. Reflection (Adaptation) Where are we going? Where do we want to go? How do we get there? ?
  68. 68. Reflection Adaptation
  69. 69. https://commons.wikimedia.org/wiki/File:Morphine_to_Apomorphine.png CC-SY-SA 4.0
  70. 70. https://commons.wikimedia.org/wiki/File:Morphine_to_Apomorphine.png CC-SY-SA 4.0 https://commons.wikimedia.org/wiki/File:Gear_reducer.gif CC-BY-SA 3.0 Norms User:ClueBot NG
  71. 71. Available Human Attention

×