He Says, She Says: Conflict and Coordination in Wikipedia Aniket Kittur, Bongwon Suh, Bryan Pendleton, Ed Chi UCLA Augmented Social Cognition Group Palo Alto Research Center

What is Wikipedia? “ Wikipedia is the best thing ever. Anyone in the world can write anything they want about any subject, so you know you’re getting the best possible information.” – Steve Carell, The Office

Spreading conflict

Spreading conflict

Spreading conflict

Spreading conflict

Spreading conflict

Policy and procedure “ The degree of success that one meets in dealing with conflicts... often depends on the efficiency with which one can quote policy and precedent.” - Wikipedia admin (survey data)

“ The degree of success that one meets in dealing with conflicts... often depends on the efficiency with which one can quote policy and precedent.”

Collaborative work beneath the surface Visitors only look at article pages But much of Wikipedia comprised of other pages Conflict resolution, coordination, policies and procedures

Visitors only look at article pages

But much of Wikipedia comprised of other pages

Conflict resolution, coordination, policies and procedures

Characterizing coordination and conflict

Characterizing coordination and conflict

Exponential growth

Costs of growth Increase in conflict and coordination costs Software development (Boehm, 1981; Brooks, 1975) MUDs/MOOs (Curtis, 1992; Dibbell, 1993) Mailing lists (Sproull & Kiesler, 1991) How has growth affected Wikipedia? Millions of new users and articles

Increase in conflict and coordination costs

Software development (Boehm, 1981; Brooks, 1975)

MUDs/MOOs (Curtis, 1992; Dibbell, 1993)

Mailing lists (Sproull & Kiesler, 1991)

How has growth affected Wikipedia?

Millions of new users and articles

Infrastructure Analyze entire history of Wikipedia Every edit to every article Large amount of data 4+ million pages 58+ million revisions 800+ Gb as of June 2006 Distributed processing Hadoop distributed filesystem Map/reduce to process data in parallel

Analyze entire history of Wikipedia

Every edit to every article

Large amount of data

4+ million pages

58+ million revisions

800+ Gb

as of June 2006

Distributed processing

Hadoop distributed filesystem

Map/reduce to process data in parallel

Types of work Direct work Immediately consumable Indirect work Coordination, conflict Maintenance work Reverts, vandalism Article Talk, user, procedure

Less direct work Decrease in proportion of edits to article page 70%

Decrease in proportion of edits to article page

More indirect work Increase in proportion of edits to user talk 8%

Increase in proportion of edits to user talk

More indirect work Increase in proportion of edits to user talk Increase in proportion of edits to procedure 11%

Increase in proportion of edits to user talk

Increase in proportion of edits to procedure

More maintenance work Increase in proportion of edits that are reverts 7%

Increase in proportion of edits that are reverts

More wasted work Increase in proportion of edits that are reverts Increase in proportion of edits reverting vandalism 1-2%

Increase in proportion of edits that are reverts

Increase in proportion of edits reverting vandalism

Global level Conflict and coordination costs are growing Less direct work (articles) More indirect work (article talk, user, procedure) More maintenance work (reverts, vandalism)

Conflict and coordination costs are growing

Less direct work (articles)

More indirect work (article talk, user, procedure)

More maintenance work (reverts, vandalism)

Characterizing coordination and conflict

Conflict at the article level What defines conflict in articles? Build a characterization model of article conflict Identify page features and metrics associated with conflict Automatically identify high-conflict articles

What defines conflict in articles?

Build a characterization model of article conflict

Identify page features and metrics associated with conflict

Automatically identify high-conflict articles

Page metrics Chose metrics for identifying conflict in articles Easily computable, scalable Article Reverts (#, by unique editors) Article, talk Minor edits (#, %) Article, talk Administrator edits (#, %) Article, talk Anonymous edits (#, %) Article, talk Links to other articles Article, talk Links from other articles Article, talk Unique editors / revisions Article, talk, article/talk Unique editors Article, talk, article/talk Page length Article, talk, article/talk Revisions (#) Page Type Metric type

Chose metrics for identifying conflict in articles

Easily computable, scalable

Defining conflict Operational definition for conflict Revisions tagged controversial Conflict revision count

Operational definition for conflict

Revisions tagged controversial

Conflict revision count

Machine learning Predict conflict from page metrics Training set of “controversial” pages Support vector machine regression predicting # controversial revisions (SMOreg; Smola & Scholkopf, 1998) Not just conflict/no conflict, but how much conflict

Predict conflict from page metrics

Training set of “controversial” pages

Support vector machine regression predicting # controversial revisions (SMOreg; Smola & Scholkopf, 1998)

Not just conflict/no conflict, but how much conflict

Performance: Cross-validation 5x cross-validation, R 2 = 0.897

5x cross-validation, R 2 = 0.897

Performance: Cross-validation 5x cross-validation, R 2 = 0.897

5x cross-validation, R 2 = 0.897

Determinants of conflict —  Revisions (talk) —  Minor edits (talk) ˜  Unique editors (talk) —  Revisions (article) ˜  Unique editors (article) —  Anonymous edits (talk) ˜  Anonymous edits (article) Highly weighted metrics of conflict model:

—  Revisions (talk)

—  Minor edits (talk)

˜  Unique editors (talk)

—  Revisions (article)

˜  Unique editors (article)

—  Anonymous edits (talk)

˜  Anonymous edits (article)

Identifying untagged articles Detect conflicts for unlabeled articles Majority of articles have never been conflict tagged Testing model generalization Applied model to untagged articles Sample rated by expert Wikipedians Significant positive correlation with predicted scores By rank correlation, p < 0.013 (Spearman’s rho)

Detect conflicts for unlabeled articles

Majority of articles have never been conflict tagged

Testing model generalization

Applied model to untagged articles

Sample rated by expert Wikipedians

Significant positive correlation with predicted scores

By rank correlation, p < 0.013 (Spearman’s rho)

Characterizing coordination and conflict

Conflict at the user level How can we identify conflict between users? Reverts as a proxy for user conflict Revert patterns between users Force directed layout to cluster users Group similar viewpoints Find conflicts between groups

How can we identify conflict between users?

Reverts as a proxy for user conflict

Revert patterns between users

Force directed layout to cluster users

Group similar viewpoints

Find conflicts between groups

Dokdo/Takeshima opinion groups Group A Group B Group C Group D

Terry Schiavo Mediators Sympathetic to parents Sympathetic to husband Anonymous (vandals/spammers)

Summary: Characterizing Wikipedia Coordination costs and conflict are increasing Global-level: Trend identification Decrease in direct article work Increase in indirect coordination work Increase in maintenance work Article-level: Prediction using Machine learning Identify characteristics of article conflict Detect conflict-heavy articles needing extra attention User-level: User Conflict Visualization Make sense of user conflicts and identify shared viewpoints

Coordination costs and conflict are increasing

Global-level: Trend identification

Decrease in direct article work

Increase in indirect coordination work

Increase in maintenance work

Article-level: Prediction using Machine learning

Identify characteristics of article conflict

Detect conflict-heavy articles needing extra attention

User-level: User Conflict Visualization

Make sense of user conflicts and identify shared viewpoints

Future Work Applied to many domains Corporate memory (Socialtext) Intelligence gathering (Intellipedia) Scholarly research (Scholarpedia) Collaborative problem solving (Lostpedia) Application: Social Dashboard Identify high conflict articles Surface editing patterns to readers Route attention to articles that need it most

Applied to many domains

Corporate memory (Socialtext)

Intelligence gathering (Intellipedia)

Scholarly research (Scholarpedia)

Collaborative problem solving (Lostpedia)

Application: Social Dashboard

Identify high conflict articles

Surface editing patterns to readers

Route attention to articles that need it most

Future work

He Says, She Says: Conflict and Coordination in Wikipedia Aniket Kittur, Bongwon Suh, Bryan Pendleton, Ed Chi UCLA Augmented Social Cognition Group Palo Alto Research Center Thank you!