CHI2007 talk on Conflicts in Wikipedia

He Says, She Says: Conflict and Coordination in Wikipedia Aniket Kittur, Bongwon Suh, Bryan Pendleton, Ed Chi UCLA Augmented Social Cognition Group Palo Alto Research Center

What is Wikipedia? “ Wikipedia is the best thing ever. Anyone in the world can write anything they want about any subject, so you know you’re getting the best possible information.” – Steve Carell, The Office

Spreading conflict

Policy and procedure
“ The degree of success that one meets in dealing with conflicts... often depends on the efficiency with which one can quote policy and precedent.”
- Wikipedia admin (survey data)

Collaborative work beneath the surface
Visitors only look at article pages
But much of Wikipedia comprised of other pages
Conflict resolution, coordination, policies and procedures

Characterizing coordination and conflict

Exponential growth

Costs of growth
Increase in conflict and coordination costs
Software development (Boehm, 1981; Brooks, 1975)
MUDs/MOOs (Curtis, 1992; Dibbell, 1993)
Mailing lists (Sproull & Kiesler, 1991)
How has growth affected Wikipedia?
Millions of new users and articles

Infrastructure
Analyze entire history of Wikipedia
Every edit to every article
Large amount of data
4+ million pages
58+ million revisions
800+ Gb
as of June 2006
Distributed processing
Hadoop distributed filesystem
Map/reduce to process data in parallel

Types of work Direct work Immediately consumable Indirect work Coordination, conflict Maintenance work Reverts, vandalism Article Talk, user, procedure

Less direct work
Decrease in proportion of edits to article page
70%

More indirect work
Increase in proportion of edits to user talk
8%

More indirect work
Increase in proportion of edits to user talk
Increase in proportion of edits to procedure
11%

More maintenance work
Increase in proportion of edits that are reverts
7%

More wasted work
Increase in proportion of edits that are reverts
Increase in proportion of edits reverting vandalism
1-2%

Global level
Conflict and coordination costs are growing
Less direct work (articles)
More indirect work (article talk, user, procedure)
More maintenance work (reverts, vandalism)

Conflict at the article level
What defines conflict in articles?
Build a characterization model of article conflict
Identify page features and metrics associated with conflict
Automatically identify high-conflict articles

Page metrics
Chose metrics for identifying conflict in articles
Easily computable, scalable
Article Reverts (#, by unique editors) Article, talk Minor edits (#, %) Article, talk Administrator edits (#, %) Article, talk Anonymous edits (#, %) Article, talk Links to other articles Article, talk Links from other articles Article, talk Unique editors / revisions Article, talk, article/talk Unique editors Article, talk, article/talk Page length Article, talk, article/talk Revisions (#) Page Type Metric type

Defining conflict
Operational definition for conflict
Revisions tagged controversial
Conflict revision count

Machine learning
Predict conflict from page metrics
Training set of “controversial” pages
Support vector machine regression predicting # controversial revisions (SMOreg; Smola & Scholkopf, 1998)
Not just conflict/no conflict, but how much conflict

Performance: Cross-validation
5x cross-validation, R 2 = 0.897

Determinants of conflict
—  Revisions (talk)
—  Minor edits (talk)
˜  Unique editors (talk)
—  Revisions (article)
˜  Unique editors (article)
—  Anonymous edits (talk)
˜  Anonymous edits (article)
Highly weighted metrics of conflict model:

Identifying untagged articles
Detect conflicts for unlabeled articles
Majority of articles have never been conflict tagged
Testing model generalization
Applied model to untagged articles
Sample rated by expert Wikipedians
Significant positive correlation with predicted scores
By rank correlation, p < 0.013 (Spearman’s rho)

Conflict at the user level
How can we identify conflict between users?
Reverts as a proxy for user conflict
Revert patterns between users
Force directed layout to cluster users
Group similar viewpoints
Find conflicts between groups

Dokdo/Takeshima opinion groups Group A Group B Group C Group D

Terry Schiavo Mediators Sympathetic to parents Sympathetic to husband Anonymous (vandals/spammers)

Summary: Characterizing Wikipedia
Coordination costs and conflict are increasing
Global-level: Trend identification
Decrease in direct article work
Increase in indirect coordination work
Increase in maintenance work
Article-level: Prediction using Machine learning
Identify characteristics of article conflict
Detect conflict-heavy articles needing extra attention
User-level: User Conflict Visualization
Make sense of user conflicts and identify shared viewpoints

Future Work
Applied to many domains
Corporate memory (Socialtext)
Intelligence gathering (Intellipedia)
Scholarly research (Scholarpedia)
Collaborative problem solving (Lostpedia)
Application: Social Dashboard
Identify high conflict articles
Surface editing patterns to readers
Route attention to articles that need it most

Future work

He Says, She Says: Conflict and Coordination in Wikipedia Aniket Kittur, Bongwon Suh, Bryan Pendleton, Ed Chi UCLA Augmented Social Cognition Group Palo Alto Research Center Thank you!

CHI2007 talk on Conflicts in Wikipedia

by Ed Chi on Feb 23, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

Statistics

CHI2007 talk on Conflicts in Wikipedia — Presentation Transcript