Research on Social Dynamics in Wikipedia

  • 1,687 views
Uploaded on

presented at Stanford Open Source lab unconference and Recent Changes Camp 2008

presented at Stanford Open Source lab unconference and Recent Changes Camp 2008

More in: Technology , Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
  • Hello, this is very interesting. Do you have any update on this study ? Thank you. -Hyunjung Bae
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
1,687
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
29
Comments
1
Likes
4

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Augmented Social Cognition: Who Edits Wikipedia? Ed H. Chi Augmented Social Cognition Area Palo Alto Research Center 2007-06-17 Ed H. Chi - Who writes Wikipedia? Image from: http://www.flickr.com/photos/ourcommon/480538715/
  • 2. Wikipedia 2007-06-17 Ed H. Chi - Who writes Wikipedia?
  • 3. High-end of the collaboration spectrum
    • Groups utilize systems to make sense and share complex topics and materials.
    • Wikipedia (social status)
    • Slashdot (karma points)
    • eHow.com
    • Lostpedia.com
    2007-06-17 Ed H. Chi - Who writes Wikipedia?
  • 4. Middle of the spectrum
    • Systems that evolve structures that can be used to organize information.
    • Del.icio.us
    • Flickr
    • YouTube
    • Friendster
    2007-06-17 Ed H. Chi - Who writes Wikipedia?
  • 5. Lightweight social processes
    • Counting votes
      • A way to increase signal-to-noise ratio
      • Information faddishness
    • Examples:
      • Digg.com
      • Most bookmarked items on del.icio.us
      • Estimating the weight of an ox or temperature of a room
      • The true value of a stock
      • PageRank or Hub / Authority algorithms
    2007-06-17 Ed H. Chi - Who writes Wikipedia?
  • 6. Layers of Models Needed 2007-06-17 Ed H. Chi - Who writes Wikipedia? Heavier collaboration Voting systems Digg.com Collaborative Creation Wikipedia Col. Information Structures Slashdot eHow.com Del.icio.us IBM dogear PageRank Flickr
    • Understanding of micro-economics
    • of foraging [PARC]
    • Personal vs. group [Huberman, Adamic]
    • Wisdom of Crowd [Surowieki]
    • Information cascades [Anderson and Holt]
    • Understanding of conflicts and coordination
    • Wikipedia coordination costs [PARC]
    • Invisible Colleges [Sandstrom]
    • Interference effects [Pirolli]
    • Co-laboratories [Olson and Olson]
    • Community networks / Col. Problem solving [Carroll]
    • Understanding of info and social networks
    • Tag network analysis [PARC, Golder, Yahoo]
    • Structural holes (info brokerage) [Burt]
    • Network constraints and structure [various]
    • Semantic of semiotic structures / words [IR, LSA]
  • 7. Research Vision Augmented Social Cognition
    • Cognition : the ability to remember, think, and reason; the faculty of knowing.
    • Social Cognition : the ability of a group to remember, think, and reason; the construction of knowledge structures by a group.
      • (not quite the same as in the branch of psychology that studies the cognitive processes involved in social interaction, though included)
    • Augmented Social Cognition : Supported by systems, the enhancement of the ability of a group to remember, think, and reason; the system-supported construction of knowledge structures by a group.
    2007-06-17 Ed H. Chi - Who writes Wikipedia?
  • 8. The first step in solving any interesting problem is to get some paper and pencil. John Tukey (not a direct quote) 2007-06-17 Ed H. Chi - Who writes Wikipedia?
  • 9. Increasing Coordination Cost in Wikipedia (joint work with Niki Kittur, Bongwon Suh, Bryan Pendleton) Published in CHI2007 conference: Aniket Kittur, Bongwon Suh, Bryan Pendleton, Ed H. ChiHe Says, She Says: Conflict and Coordination in Wikipedia. In Proc. of ACM Conference on Human Factors in Computing Systems (CHI2007), pp. 453--462, April 2007. ACM Press. San Jose, CA 
 2007-06-17 Ed H. Chi - Who writes Wikipedia?
  • 10. What is Wikipedia? 2007-06-17 Ed H. Chi - Who writes Wikipedia? “ Wikipedia is the best thing ever. Anyone in the world can write anything they want about any subject, so you know you’re getting the best possible information.” – Steve Carell, The Office
  • 11. Increasing Coordination Costs in Wikipedia
    • Understanding coordination costs is vital for long-term viability of collaborative information environment
    • Data:
      • Entire dump on July 2, 2006
      • 58 million revisions
      • 4.7 million wiki pages
      • 2.4 million article pages
      • 800 gigabytes
    2007-06-17 Ed H. Chi - Who writes Wikipedia?
  • 12. Less direct work
    • Decrease in proportion of edits to article page
    2007-06-17 Ed H. Chi - Who writes Wikipedia? 70%
  • 13. More indirect work
    • Increase in proportion of edits to user talk
    2007-06-17 Ed H. Chi - Who writes Wikipedia? 8%
  • 14. More indirect work
    • Increase in proportion of edits to user talk
    • Increase in proportion of edits to procedure
    2007-06-17 Ed H. Chi - Who writes Wikipedia? 11%
  • 15. More maintenance work
    • Increase in proportion of edits that are reverts
    2007-06-17 Ed H. Chi - Who writes Wikipedia? 7%
  • 16. More wasted work
    • Increase in proportion of edits that are reverts
    • Increase in proportion of edits reverting vandalism
    2007-06-17 Ed H. Chi - Who writes Wikipedia? 1-2%
  • 17. Global level
    • Conflict and coordination costs are growing
      • Less direct work (articles)
      • More indirect work (article talk, user, procedure)
      • More maintenance work (reverts, vandalism)
    2007-06-17 Ed H. Chi - Who writes Wikipedia?
  • 18. Conflict at the article level
    • Conflict is growing at the global level
    • We have some idea about where it is
    • But what defines conflict at the local level?
    • Build a characterization model of article conflict
      • Identify metrics relevant to conflict
      • Automatically identify high-conflict articles
    2007-06-17 Ed H. Chi - Who writes Wikipedia?
  • 19. Measure of controversy
    • “ Controversial” tag
    • Use # revisions tagged controversial
    2007-06-17 Ed H. Chi - Who writes Wikipedia?
  • 20. Page metrics
    • Possible metrics for identifying conflict in articles
    2007-06-17 Ed H. Chi - Who writes Wikipedia? Metric type Page Type Revisions (#) Article, talk, article/talk Page length Article, talk, article/talk Unique editors Article, talk, article/talk Unique editors / revisions Article, talk Links from other articles Article, talk Links to other articles Article, talk Anonymous edits (#, %) Article, talk Administrator edits (#, %) Article, talk Minor edits (#, %) Article, talk Reverts (#, by unique editors) Article
  • 21. Performance: Cross-validation
    • 5x cross-validation, R 2 = 0.897
    2007-06-17 Ed H. Chi - Who writes Wikipedia?
  • 22. Performance: Cross-validation
    • 5x cross-validation, R 2 = 0.897
    2007-06-17 Ed H. Chi - Who writes Wikipedia?
  • 23. Determinants of conflict
    •   Revisions (talk)
    •   Minor edits (talk)
    •   Unique editors (talk)
    •   Revisions (article)
    •   Unique editors (article)
    •   Anonymous edits (talk)
    •   Anonymous edits (article)
    2007-06-17 Ed H. Chi - Who writes Wikipedia?
    • Highly weighted features of conflict model:
  • 24. Model Generalization and Validation survey
    • Applied model to untagged articles (100+ edits)
    • Sampled range of predicted conflict scores
    • Rated by expert Wikipedians
    • Significantly correlated with predicted scores
      • By rank correlation, p < 0.013 (Spearman’s rho)
    • Validates characterization model
      • Detects conflicts even for articles with no ground truth
    2007-06-17 Ed H. Chi - Who writes Wikipedia?
  • 25. Who edits Wikipedia? % of edits made by administrators 2007-06-17 Ed H. Chi - Who writes Wikipedia?
  • 26. % of edits by 10k+ editors 2007-06-17 Ed H. Chi - Who writes Wikipedia?
  • 27. Word changes made by admins 2007-06-17 Ed H. Chi - Who writes Wikipedia?
  • 28. Shifting user population in Wikipedia (more and more bottom driven!) 2007-06-17 Ed H. Chi - Who writes Wikipedia?
  • 29. Proportion of edits made by top editors in Wikipedia 2007-06-17 Ed H. Chi - Who writes Wikipedia?
  • 30. Long tail of participation in Wikipedia 2007-06-17 Ed H. Chi - Who writes Wikipedia?
  • 31. The participation architecture is a power law 2007-06-17 Ed H. Chi - Who writes Wikipedia?
  • 32. Only 60% of top 1% editors stay around month to month! 2007-06-17 Ed H. Chi - Who writes Wikipedia?
  • 33. Living Laboratory: Prototyping Social Applications on the Internet Create a Living Laboratory as a platform to develop, test, and market our innovations, and as a vehicle for creating collaborations and thought leadership. 2007-06-17 Ed H. Chi - Who writes Wikipedia?
  • 34. WikiDashboard Joint work with Bongwon Suh, Aniket Kittur, Bryan Pendleton 2007-06-17 Ed H. Chi - Who writes Wikipedia?
  • 35. Risks for Using Wikipedia
    • Factual accuracy
    • Motives of editors
    • Uncertain expertise
    • Volatility
    • Spotty coverage
    • Unproven/non-independent source
    2007-06-17 Ed H. Chi - Who writes Wikipedia? [Denning et al. 2005]
  • 36. Social Dashboard
    • Social translucent for effective communication and collaboration
      • Make socially significant information visible and salient
      • Support awareness of the rules and constraints
      • Accountability for actions
    • Wikis can be a prime candidate
      • Every edit is logged and retrievable
      • WikiScanner.com
      • WikiRage.com
      • Intellipedia
    2007-06-17 Ed H. Chi - Who writes Wikipedia? [Erickson and Kellogg 2002]
  • 37. WikiDashboard
    • Surfacing hidden social context to users
    • For readers
      • Any incidents in the past e.g. A sudden burst of edits?
      • Who are the editors?
      • What is their motivation / point of views / expertise / topics of interest
      • Help them judging the quality/trustworthiness/usefulness of an article
    • For writers
      • Measure expertise / contribution / reputation
      • Motivate them to be more active / responsible (?)
    2007-06-17 Ed H. Chi - Who writes Wikipedia?
  • 38. Article Dashboard 2007-06-17 Ed H. Chi - Who writes Wikipedia?
  • 39. User Dashboard 2007-06-17 Ed H. Chi - Who writes Wikipedia?
  • 40. Drilling Down
    • List of every edits that a user made
    • Let readers examine each individual revision for validity, which is hard to accomplish when only provided with aggregate visual summaries.
    2007-06-17 Ed H. Chi - Who writes Wikipedia?
  • 41. Augmented Social Cognition: From Social Foraging to Social Sensemaking 2007-06-17 Ed H. Chi - Who writes Wikipedia? Image from: http://www.flickr.com/photos/ourcommon/480538715/
    • Research Vision: Understand how social computing systems enhance the ability of a group of people to remember, think, and reason.
    • Living Laboratory: Create breakthrough applications that harness collective intelligence to improve knowledge capture, transfer, and discovery.