Your SlideShare is downloading. ×
Emotions and dialogue in a peer-production community: the case of Wikipedia
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Emotions and dialogue in a peer-production community: the case of Wikipedia

1,581
views

Published on

Slides presented at WikiSym 2012. …

Slides presented at WikiSym 2012.

This paper presents a large-scale analysis of emotions in conversations among Wikipedia editors. Our focus is on the emotions expressed by editors in talk pages, measured by using the Affective Norms for English Words (ANEW).

We find evidence that to a large extent women tend to participate in discussions with a more positive tone, and that administrators are more positive than non-administrators. Surprisingly, female non-administrators tend to behave like administrators in many aspects.

We observe that replies are on average more positive than the comments they reply to, preventing many discussions from spiralling down into conflict. We also find evidence of emotional homophily: editors having similar emotional styles are more likely to interact with each other.

Our findings offer novel insights into the emotional dimension of interactions in peer-production communities, and contribute to debates on issues such as the flattening of editor growth and the gender gap.

Published in: Technology

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,581
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
2
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Emotions and dialogue in a peer-production community: the case of Wikipedia David Laniado Carlos Castillo Barcelona Media Foundation Qatar Computing Research Institute Andreas Kaltenbrunner Mayo Fuster Morell Barcelona Media Foundation Berkman Center, Cambridge, MA, USA August 27th , 2012 WikiSym ’12, Linz, Austria university-logoLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 1 / 37
  • 2. Outline1 Introduction2 Framework of analysis Data acquisition and pre-processing User gender labelling Sentiment analysis3 Results Emotions and experience Topics and gender Emotional congruence Emotional homophily Emotions in personal (“User talk”) pages4 Conclusions university-logoLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 2 / 37
  • 3. Outline1 Introduction2 Framework of analysis Data acquisition and pre-processing User gender labelling Sentiment analysis3 Results Emotions and experience Topics and gender Emotional congruence Emotional homophily Emotions in personal (“User talk”) pages4 Conclusions university-logoLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 3 / 37
  • 4. Introduction“Like other aspects of culture, emotions can be seen as an aspect ofall social action and social relations. They accompany rational acts asfully as irrational ones, positive experiences as much as negativeones” (Goodwin, 2001)Goal: Shed light on the emotional dimension in a large peerproduction community focus: English Wikipedia extensive analysis of emotions expressed in talk pages Article talk pages → discussions about how to improve articles User talk pages → a kind of public in-boxes university-logoLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 4 / 37
  • 5. Research questions 1 How are the emotional styles of editors affected by their level of experience? 2 How are the emotional styles of editors affected by their gender and the topics they choose to work on? 3 How are the emotional expressions affected by interacting with others in comment threads (emotional congruence)? 4 How are the emotional styles of editors related to those of the editors they interact more frequently with (emotional homophily)? 5 How are the emotional styles of editors expressed in their personal spaces (user talk pages)? university-logoLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 5 / 37
  • 6. Outline1 Introduction2 Framework of analysis Data acquisition and pre-processing User gender labelling Sentiment analysis3 Results Emotions and experience Topics and gender Emotional congruence Emotional homophily Emotions in personal (“User talk”) pages4 Conclusions university-logoLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 6 / 37
  • 7. Outline1 Introduction2 Framework of analysis Data acquisition and pre-processing User gender labelling Sentiment analysis3 Results Emotions and experience Topics and gender Emotional congruence Emotional homophily Emotions in personal (“User talk”) pages4 Conclusions university-logoLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 7 / 37
  • 8. Dataset: conversationsExtracting conversations among editors from the English Wikipedia Articles 3 210 039 Articles with talk page (ATP) 871 485 (27.1%) Total comments in ATP 11 041 246 Comments in ATP linking to policies 460 644 (4.1%) Users who comment articles 350 958 Users with ≥ 100 comments on ATP 12 231 (3.5%)Table: Data extracted from a complete dump of the English Wikipedia, datedMarch 12th, 2010 university-logoLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 8 / 37
  • 9. Outline1 Introduction2 Framework of analysis Data acquisition and pre-processing User gender labelling Sentiment analysis3 Results Emotions and experience Topics and gender Emotional congruence Emotional homophily Emotions in personal (“User talk”) pages4 Conclusions university-logoLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 9 / 37
  • 10. Identifying user gender ≈ 12 000 users wrote ≥ 100 comments in articles talk pages Gender identified through Wikipedia API for ≈ 2 000 of them Out of the remaining ones, a sample of 1 385 users for manual labelling through crowdsourcing (Crowdflower) university-logoLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 10 / 37
  • 11. Manual user gender labellingGender could be identified only for ≈ 50% of users: real name or username (50% of those identified) implicitly stated gender (27% of females, 20% of males) pronoun (15% of females, 10% of males) other indicators: userboxes, pictures, links to personal blogs... Non-admins Admins Total Males 1 087 1 526 2 613 Females 68 97 165 Unknown 6 850 2 603 9 453 Total 8 005 4 226 12 231 Table: Users with ≥ 100 comments by gender and administrator status. Category “unknown” includes: 8 708 users that were not included in the crowd-sourced task university-logo 745 users whose gender could not be identified by evaluatorsLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 11 / 37
  • 12. Outline1 Introduction2 Framework of analysis Data acquisition and pre-processing User gender labelling Sentiment analysis3 Results Emotions and experience Topics and gender Emotional congruence Emotional homophily Emotions in personal (“User talk”) pages4 Conclusions university-logoLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 12 / 37
  • 13. Measuring the Emotional Content of DiscussionsUse ANEW listRates a list of 1060 frequent words on a 9 point scale in threedimensions: Valence: positive vs negative emotions Arousal: strength of the emotions Dominance: feeling in control vs feeling dominated Bradley and Lang. (1999). Affective norms for English words (ANEW) Technical report C-1. The Center for Research in Psychophysiology, University of Florida, FL. university-logoLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 13 / 37
  • 14. Example messagesTable: Example messages with their corresponding Valence, Arousal, andDominance scores (average over the words contained in each message). V A D Sounds like a good challenge - to be proven or disproven. I’m happy 7.4 5.3 6.2 if it can be shown to go further using closed cubic polynomial solu- tions. The nice thing about these are that they are pretty easy to test numerically . . . –in “Exact trigonometric constants” Seems you have not yet seen female lover after having sex who 5.5 7.0 5.2 do not wish to have sex with the same lover any more :) Once you’ve seen it, you understand very well what war of Venus means compared to war of Mars. –in “House (astrology)” What about the whirlie hazing, the alcohol abuse, the emotional 1.6 5.8 3.5 poverty, the suicide in 1995/6, the biotech plans which were stopped by pitzer protests . . . –in “Harvey Mudd College” university-logoLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 14 / 37
  • 15. Outline1 Introduction2 Framework of analysis Data acquisition and pre-processing User gender labelling Sentiment analysis3 Results Emotions and experience Topics and gender Emotional congruence Emotional homophily Emotions in personal (“User talk”) pages4 Conclusions university-logoLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 15 / 37
  • 16. Outline1 Introduction2 Framework of analysis Data acquisition and pre-processing User gender labelling Sentiment analysis3 Results Emotions and experience Topics and gender Emotional congruence Emotional homophily Emotions in personal (“User talk”) pages4 Conclusions university-logoLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 16 / 37
  • 17. Emotions and edit count corr= 0.12, p−value<10−8 1 Male Female mean dominance normalized 0.5 0 −0.5 100 1000 10000 100000 log. of edit count Positive correlation between edit count and dominance university-logo → more experienced users tend to feel “in control” more than newbiesLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 17 / 37
  • 18. Emotions and roleDistribution of the ANEW-valence scores of admins vs non-admins valence 4.8 5 5.2 5.4 5.6 5.8 6 6.2 6.4 6.6 6.8 7 0.25 1 Normal Admin 0.9 Normal: mean ± SEM 0.2 Admin: mean ± SEM 0.8 P(Admin|mean(valence)=x) P(Admin|mean(valence)=x) 0.7 0.15 0.6 pdf 0.5 0.1 0.4 0.3 0.05 0.2 0.1 0 0 4.8 5 5.2 5.4 5.6 5.8 6 6.2 6.4 6.6 6.8 7 mean valence Distribution of the ANEW-valence scores of admins vs non-admins mean value and the interval of mean ± the standard error of the mean (SEM) dashed lines indicate probability of being an administrator given the mean valence of university-logo a user (assuming equal sample sizes)Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 18 / 37
  • 19. Emotions and roleMean (±SEM) of valence and dominance for regular users and administrators valence dominance 5.45 Admin 6.1 Non−admin 5.44 6.08 5.43 6.06 p-values mean dominance mean valence 6.04 5.42 *** p < 0.001 ** p < 0.01 5.41 6.02 * p < 0.05 6 5.4 5.98 5.39 5.96 5.38 All le le All le le ma Ma ma Ma Fe Fe Comments by administrators have higher valence and dominance university-logo No significant difference between female admins and female non-adminsLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 19 / 37
  • 20. Outline1 Introduction2 Framework of analysis Data acquisition and pre-processing User gender labelling Sentiment analysis3 Results Emotions and experience Topics and gender Emotional congruence Emotional homophily Emotions in personal (“User talk”) pages4 Conclusions university-logoLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 20 / 37
  • 21. Emotions and gender Words more used by females and males Size accounts for difference in frequency university-logoLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 21 / 37
  • 22. Emotions and genderMean (±SEM) of valence and dominance form males and females valence dominance 5.45 Male 6.1 Female 5.44 6.08 5.43 6.06 p-values mean dominance mean valence 6.04 5.42 *** p < 0.001 ** p < 0.01 5.41 6.02 * p < 0.05 6 5.4 5.98 5.39 5.96 5.38 All ins ns All ins ns dm mi m mi a Ad ad Ad n− n− No No Females use words associated to more positive emotions university-logo Difference no more significant when normalizing by articleLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 22 / 37
  • 23. Topics, emotions and gender N≥1 ANEW words; corr=−0.64 (p=0.002) 5.6 Computing Arts 5.5 Philosophy Language 5.4 Health Mathematics 5.3 Belief mean valence Sports 5.2 Agriculture Environment 5.1 Techn. & app. sci. Law 5 Society Business Education 4.9 Culture People 4.8 Science Politics 4.7 Geography and places 0.82 0.84 0.86 0.88 0.9 0.92 0.94 0.96 prop. of male comments History and eventsFigure: Mean valence for discussions of articles in different topic categories, university-logovs the proportion of comments written by male editorsLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 23 / 37
  • 24. Links to Wikipedia policiesTable: Percentage of messages including links to Wikipedia policies fordifferent classes of users Non-admins Admins All users Males 2.5% 4.9% 3.9% Females 6.2% 12.4% 9.8% Differences between genders are statistically significant with p < 0.01 Comments containing links to Wikipedia policies have higher valence, higher arousal and higher dominance when editors invoke community norms, they tend to do it with a remarkably positive and dominant tone, and with stronger emotional load university-logoLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 24 / 37
  • 25. Message length and genderFemales write longer messages 83 vs 71 words on averageDifference is accentuated for female and male administrators 85 vs 68 words on average different leadership style?p-values Differences are statistically significant with p < 0.05 university-logoLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 25 / 37
  • 26. Outline1 Introduction2 Framework of analysis Data acquisition and pre-processing User gender labelling Sentiment analysis3 Results Emotions and experience Topics and gender Emotional congruence Emotional homophily Emotions in personal (“User talk”) pages4 Conclusions university-logoLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 26 / 37
  • 27. Emotional congruenceReplies are more positiveOn average, editors tend to reply with: higher valence: +0.05 higher dominance: +0.04 no statistically significant differences for arousal Users tend to be more positive and dominant when replying, but without recurring to words evoking stronger sentiments Results computed on all comments in article talk pages university-logoLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 27 / 37
  • 28. Outline1 Introduction2 Framework of analysis Data acquisition and pre-processing User gender labelling Sentiment analysis3 Results Emotions and experience Topics and gender Emotional congruence Emotional homophily Emotions in personal (“User talk”) pages4 Conclusions university-logoLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 28 / 37
  • 29. Emotional homophilyMixing patterns: do users interact preferentially with similar users? - By activity: users who write more comments tend to reply preferentially to less active users, end viceversa + By gender: females interact more with other females + By style: users interact more with others expressing similar emotions edges connecting users who have exchanged at least 10 replies red nodes → 15% users expressing higher valence in article discussions black nodes → 15% users expressing lower valence in article discussions size → proportional to the number of connections university-logoLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 29 / 37
  • 30. Outline1 Introduction2 Framework of analysis Data acquisition and pre-processing User gender labelling Sentiment analysis3 Results Emotions and experience Topics and gender Emotional congruence Emotional homophily Emotions in personal (“User talk”) pages4 Conclusions university-logoLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 30 / 37
  • 31. Emotions expressed on personal pagesComments written by users on their own talk page written by females: higher valence, dominance and arousal by male administrators: higher valence and dominance, lower arousal male administrators are positive and dominant, but write less emotional content on the contrary women, including female administrators, express stronger emotions on their personal pagesComments received on personal talk pages more positive emotions than in one’s own comments females receive more positive messages, except for female administrators a supposed gender difference seems to be out-weighed by the type of interactions the administrator tasks bring with them university-logoLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 31 / 37
  • 32. Outline1 Introduction2 Framework of analysis Data acquisition and pre-processing User gender labelling Sentiment analysis3 Results Emotions and experience Topics and gender Emotional congruence Emotional homophily Emotions in personal (“User talk”) pages4 Conclusions university-logoLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 32 / 37
  • 33. Conclusions (1)Wikipedia editors express themselves in general with a positiveemotional tone responses tend to be more positive than the comments to which they reply users receive more positive messages than they write on their personal pagesAdministrators and experienced users play a pivotal role they tend to interact especially with less experienced users they express themselves with a more positive tone university-logoLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 33 / 37
  • 34. Conclusions (2)Female editors are different in their emotional expression andrelationships they work on topics in which the discussions have a more positive tone they receive more positive messages on their personal pages they tend to behave similarly to administrators however, there are some differences in the leadership style females write longer messages, link more often Wikipedia policies, and express stronger emotions in personal spacesWikipedians of a feather fly together editors communicate more with others having a similar style females interact more with other females university-logoLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 34 / 37
  • 35. RecommendationsDesign for positive emotional expressions Besides the neutral point of view, provide spaces for emotional expression, favoring positive exchanges and channeling negative feelings in a non-disctructive manner in particular, build upon the positive tone of user talk pagesEncourage experienced editors to contribute to positive emotions leverage contagiousness of emotional styles to “cool down” conflictive discussions positive tone of messages linking to WP policiesGender-aware recruiting it may be advisable that females are invited by other females consider that certain topics may be more attractive to females university-logo seek participation of females where the gender gap is more acuteLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 35 / 37
  • 36. Future workComparison with other metrics preliminary results on the same data using other lexicons are largely consistentLongitudinal analysis how do emotional styles of editors change over time and with increasing experience? how do emotions in the discussion affect participation?Need for qualitative analysis and human annotation include non-textual emotional aspects such as emoticons, barn stars and virtual gifts cover also less experienced users deal with sarcasm, measure the extent of condescending or paternalistic language in comments addressed at female editors university-logoLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 36 / 37
  • 37. Main references I Bradley and Lang. Affective norms for English words (ANEW) Technical report C-1. The Center for Research in Psychophysiology, University of Florida, FL, 2012. B. Collier and J. Bear. Conflict, criticism, or confidence: an empirical examination of the gender gap in wikipedia contributions. In Proc. of CSCW, 2012. O. Kucuktunc, B. B. Cambazoglu, I. Weber, and H. Ferhatosmanoglu. A large-scale sentiment analysis for Yahoo! answers. In Proc. of WSDM, 2012. S. K. Lam, A. Uduwage, Z. Dong, S. Sen, D. R. Musicant, L. Terveen, and J. Riedl. WP:clubhouse? an exploration of Wikipedia’s gender imbalance. In Proc. of WikiSym, 2011. D. Laniado, R. Tasso, Y. Volkovich, and A. Kaltenbrunner. When the Wikipedians talk: Network and tree structure of Wikipedia discussion pages. In Proc. of ICWSM, 2011. university-logoLaniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 37 / 37