Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Exploring cyberbullying and 

other toxic behavior in 

team competition online games
@haewoon Kwak
Qatar Computing Resear...
Bullying that takes place using electronic technology.
This image is retrieved from http://ucrtoday.ucr.edu/24423
http://www.madesimplegroup.com/blog/tutorial-incorporating-social-media-site/
http://www.madesimplegroup.com/blog/tutorial-incorporating-social-media-site/
http://kirby.wikia.com/wiki/Sword
7
Game industry is huge
Active players
(month) 67M 7M
eSports Prize pool* $2M $11M
*2014 US Open Men’s tennis prize pool i...
8
Gamers invest lots of their time and
energy into games
http://techraptor.net/content/imo-video-game-addiction-complicate...
9
Games are popular in younger generation
http://archive.news.softpedia.com/news/Game-Addiction-Similar-to-Drug-Addiction-...
MonumentValley
Competition in games: Pros
• Competition is a good gaming element for enjoyment
• It naturally causes tensions
• and it leads to an increasing concern about negative
behavior (called toxic behavior)
Com...
• The definition of toxic playing is often unclear due to
differences in expected behavior, customs, rules, and ethics
acr...
noob!
uninstall the game
…
What r u doing?
r u sleeping?
NOOB
OMG
surrender at 20
ks
Quiz: which is toxic and which is not...
A case study: RiotGames
The game studio developing League of
Legends, the most popular online game
The Tribunal (May 2011~2014)
A crowdsourcing platform to fight
against habitual toxic players
http://leagueoflegends.wikia....
Reviewers vote for pardon / punish
User reports (anonymized)
Chat logs (anoymized)
Reported matches
General information
7 predefined categories in the Tribunal
• Offensive language
• Verbal abuse
• Assisting enemy team
• Intentional feeding [...
Decision (Punish or pardon)
Agreement level
Outcome (Win or lose)
User reports
Chat logs
In-game performance
Vote results
6 agreement levels in the Tribunal
Pardon
Majority Majority
Strong
Majority
Strong
Majority
Overwhelming
Majority
Overwhel...
23
Data collection from different servers
NA EUWest KR All
Player 590,311 649,419 220,614 1,460,344
Match 2,107,522 2,841,...
• STFU NOOB! Predicting crowdsourced decisions on
toxic behavior in online games (WWW’14)

http://arxiv.org/abs/1404.5905
...
• We explore toxic behavior to answer the question 

“Why is it hard to deal with toxic behavior?” with
guidance of existi...
• Possible biases in reporting and reviewing
• Bystander effect and low deg. of participation in reporting
• In-group favo...
Bystander effect and vague nature of toxic playing:
How do they affect reporting and judging?
Deindividuation theory
Why do people show extreme behavior when immersed in a group?
http://nypost.com/2015/02/20/soccer-h...
Anonymous online environments of LoL
• No presence of face-to-face communication.
• Reduced self-awareness
• Less constrai...
• Degree of participation in reporting toxic playing
directly connects to
✓Feasibility/efficiency of the overall system to...
31
http://lifehacker.com/5930167/cut-through-the-bystander-effect-by-explicitly-telling-someone-to-call-911
H1.1 If there ...
• Unclear definition of toxic playing make it hard not only to
report but also to judge whether it is toxic.
32
Also, vagu...
Low number of reports per match is found
• Mean: 1.812
• Median: 1
• Less than 2 players per match report toxic behavior
•...
Impact of asking reports
• If allies explicitly ask enemies to report the toxic player
through all chats, such as ‘plz rep...
35
Vagueness of toxic playing affects review
For 1 in 4 spamming cases in KR,
reviewers do not find the player toxic.
In-group Favoritism and Out-group Hostility
Does team-competition setting affect reporting?
Ingroup favoritsm
http://www.agefotostock.com/en/Stock-Images/Royalty-Free/FNC-FAN2034560
Ingroup favoritsm
Outgroup hosti...
What can we see from intergroup conflicts?
Ingroup favoritsm
RQ2: What is the difference between reporting behavior
of the...
Proper category for hypothesis testing
• Assisting enemy team
• Intentional feeding [suicide]
• Offensive language
• Verba...
# of matches that toxic players are reported
by ally-only, enemy-only, or both
Category of toxic playing
# of matches
43
Ingroup favoritism is observed!
+47% More reports come from enemies than allies
H2.1 For toxic behavior that affects bo...
Intra-group Conflicts and Socio-political Factors
What/why do conflicts happen within a team?
Community vs. Association
LoL teams are close to associations
• Task-oriented association (Gesellschaft) rather than social
community (Geminshaft)
•...
47
Possible socio-political factors
affecting intragroup conflicts
RQ3: What is the impact of socio-political factors on
t...
• Individualism vs. collectivism
• focus on ones’ self vs. group goal
48
Testable hypotheses about regional diff.
H3.2 Rep...
49
Korea is the most generous region for
cyberbullying.
(O+V)17.1%
(O+V) 14.3%
(O+V) 9.7%
* Effect of region on pardon is ...
50
Korea is the most generous region for
cyberbullying.
H3.1 Due to a more group-success oriented socio-
political environ...
51
Korea players most actively report
assisting enemy and intentional feeding
Korea NA EUW
Avg. 1.75 1.71 1.48
* Effect of...
52
Korea players most actively report
assisting enemy and intentional feeding
* Effect of region on reports from teammates...
53
KR reviewers are more likely to perceive
AE and IF as severe toxic playing
* Effect of region on the percentage of P-O ...
54
KR reviewers are more likely to perceive
AE and IF as severe toxic playing
H3.3 Reports on behavior that largely affect...
Team-cohesion and Performance
How does toxic playing affect team performance?
56
https://www.youtube.com/watch?v=KatTVGAEu80
• Actively studied in sports and organizational science
because a team is an important unit there.
• Significant positive ...
• A negative player degrades team cohesion (and team
performance) as follows:
1. violates interpersonal norms
2. leads to ...
• A negative player degrades team cohesion (and team
performance) as follows:
1. violates interpersonal norms
2. leads to ...
Alternative explanation looks also reasonable
✓Negative outcome can trigger reporting of toxic behavior
• After losing the...
Winning ratio = a proxy of team performance
• If 100 matches are reported for verbal abuse, and a team
with a toxic player...
(We focus on toxic playing reported by teammates.)
Win. ratio of reported cases ≪ 50%
50%
More reports come from losing teams!
50%
H4.1 More reports come from matches where the
accused was on the losing team. is ...
+ one more interesting finding
50%
the win. ratio with inappropriate name < 50%
67
Winning cases are more pardoned
68
Winning cases are more pardoned
(We focus on cases that teammates report.)
H4.2 There are more cases pardoned when the ...
Bystander effect and vague nature of toxic playing
How do they affect reporting and judging?
In-group Favoritism and Out-g...
• LoL is a virtual laboratory to understand human behavior.
• The Tribunal offers a unique chance to study toxic
behavior ...
71
A full paper with an additional executive
summary is available via
http://arxiv.org/abs/1504.02305
Exploring cyberbullying and 
other toxic behavior in 
team competition online games
Exploring cyberbullying and 
other toxic behavior in 
team competition online games
Exploring cyberbullying and 
other toxic behavior in 
team competition online games
Upcoming SlideShare
Loading in …5
×

Exploring cyberbullying and 
other toxic behavior in 
team competition online games

1,201 views

Published on

We explore toxic behavior to answer the question 
“Why is it hard to deal with toxic behavior?” with guidance of existing theories and hypotheses by using large-scale data collection.. This work will be presented in CHI'15.

Published in: Data & Analytics
  • Be the first to comment

Exploring cyberbullying and 
other toxic behavior in 
team competition online games

  1. 1. Exploring cyberbullying and 
 other toxic behavior in 
 team competition online games @haewoon Kwak Qatar Computing Research Institute Doha, Qatar Jeremy Blackburn Telefonica Research Barcelona, Spain Seungyeop Han University of Washington Seattle, USA
  2. 2. Bullying that takes place using electronic technology.
  3. 3. This image is retrieved from http://ucrtoday.ucr.edu/24423
  4. 4. http://www.madesimplegroup.com/blog/tutorial-incorporating-social-media-site/
  5. 5. http://www.madesimplegroup.com/blog/tutorial-incorporating-social-media-site/ http://kirby.wikia.com/wiki/Sword
  6. 6. 7 Game industry is huge Active players (month) 67M 7M eSports Prize pool* $2M $11M *2014 US Open Men’s tennis prize pool is “only” $3M
  7. 7. 8 Gamers invest lots of their time and energy into games http://techraptor.net/content/imo-video-game-addiction-complicated-issue
  8. 8. 9 Games are popular in younger generation http://archive.news.softpedia.com/news/Game-Addiction-Similar-to-Drug-Addiction-100447.shtml
  9. 9. MonumentValley
  10. 10. Competition in games: Pros • Competition is a good gaming element for enjoyment
  11. 11. • It naturally causes tensions • and it leads to an increasing concern about negative behavior (called toxic behavior) Competition in games: Cons
  12. 12. • The definition of toxic playing is often unclear due to differences in expected behavior, customs, rules, and ethics across games. 14 Major hurdle: Vagueness of toxic behavior http://www.slideshare.net/Vince77580/OBDecision-and-Culture1
  13. 13. noob! uninstall the game … What r u doing? r u sleeping? NOOB OMG surrender at 20 ks Quiz: which is toxic and which is not? Fking noob wtf
  14. 14. A case study: RiotGames
  15. 15. The game studio developing League of Legends, the most popular online game
  16. 16. The Tribunal (May 2011~2014) A crowdsourcing platform to fight against habitual toxic players http://leagueoflegends.wikia.com/wiki/The_Journal_of_Justice:_Volume_1,_Issue_18
  17. 17. Reviewers vote for pardon / punish User reports (anonymized) Chat logs (anoymized) Reported matches General information
  18. 18. 7 predefined categories in the Tribunal • Offensive language • Verbal abuse • Assisting enemy team • Intentional feeding [suicide] • Negative attitude • Inappropriate [handle] name • Spamming • Unskilled player • Refusing to communicate with team • Leaving the game / AFK [away from keyboard] Cyberbullying
  19. 19. Decision (Punish or pardon) Agreement level Outcome (Win or lose) User reports Chat logs In-game performance Vote results
  20. 20. 6 agreement levels in the Tribunal Pardon Majority Majority Strong Majority Strong Majority Overwhelming Majority Overwhelming Majority Punish
  21. 21. 23 Data collection from different servers NA EUWest KR All Player 590,311 649,419 220,614 1,460,344 Match 2,107,522 2,841,906 1,066,618 6,016,046 Report 3,441,557 5,559,968 1,893,433 10,898,958 * KR Tribunal starts from November 2012, while other two Tribunals start from May 2011. To study regional difference!
  22. 22. • STFU NOOB! Predicting crowdsourced decisions on toxic behavior in online games (WWW’14)
 http://arxiv.org/abs/1404.5905 • Linguistic analysis of toxic behavior in an online video game (EGG’14 workshop)
 http://arxiv.org/abs/1410.5185 24 Our previous work on the Tribunal
  23. 23. • We explore toxic behavior to answer the question 
 “Why is it hard to deal with toxic behavior?” with guidance of existing theories and hypotheses by using large-scale data collection. 25 Research goal in this work
  24. 24. • Possible biases in reporting and reviewing • Bystander effect and low deg. of participation in reporting • In-group favoritism and out-group hostility • Intra-group conflicts and socio-political factors • Team-cohesion and performance • Summary 26 Outline of the rest of the talk Research Q / formulated H Supported H Not supported H
  25. 25. Bystander effect and vague nature of toxic playing: How do they affect reporting and judging?
  26. 26. Deindividuation theory Why do people show extreme behavior when immersed in a group? http://nypost.com/2015/02/20/soccer-hooligan-rampage-is-romes-terrorist-fears-realized/
  27. 27. Anonymous online environments of LoL • No presence of face-to-face communication. • Reduced self-awareness • Less constraints/responsibility • Thus, observers respond to toxic behavior as bystanders • social control does not work well RQ1a: How active are players in reporting toxic behavior?
  28. 28. • Degree of participation in reporting toxic playing directly connects to ✓Feasibility/efficiency of the overall system to detect toxic playing by voluntary player reports Why is this question important?
  29. 29. 31 http://lifehacker.com/5930167/cut-through-the-bystander-effect-by-explicitly-telling-someone-to-call-911 H1.1 If there is a request asking to report toxic players, the number of reports is increased.
  30. 30. • Unclear definition of toxic playing make it hard not only to report but also to judge whether it is toxic. 32 Also, vague nature of toxic playing RQ1b: How does the vague nature of toxic playing 
 affect tribunal decisions?
  31. 31. Low number of reports per match is found • Mean: 1.812 • Median: 1 • Less than 2 players per match report toxic behavior • Players have different perception • Players are not actively engaged in reporting (remember 5 vs. 5 setting!)
  32. 32. Impact of asking reports • If allies explicitly ask enemies to report the toxic player through all chats, such as ‘plz report’, they really do! • Chi-square test, p < .0001 • Odds ratio shows that the probability of reports by enemies with allies’ requests is 16.37 times higher than that without such requests H1.1 If there is a request asking to report toxic players, the number of reports is increased. is supported.
  33. 33. 35 Vagueness of toxic playing affects review For 1 in 4 spamming cases in KR, reviewers do not find the player toxic.
  34. 34. In-group Favoritism and Out-group Hostility Does team-competition setting affect reporting?
  35. 35. Ingroup favoritsm http://www.agefotostock.com/en/Stock-Images/Royalty-Free/FNC-FAN2034560 Ingroup favoritsm Outgroup hostility
  36. 36. What can we see from intergroup conflicts? Ingroup favoritsm RQ2: What is the difference between reporting behavior of the toxic player’s teammates and his opponents? H2.1 For toxic behavior that affects both teams equally, in-group members (teammates) are less likely to submit reports when compared to out-group members.
  37. 37. Proper category for hypothesis testing • Assisting enemy team • Intentional feeding [suicide] • Offensive language • Verbal abuse • Negative attitude • Inappropriate [handle] name • Spamming • Unskilled player • Refusing to communicate with team • Leaving the game / AFK [away from keyboard] - visible to all players - same impact to both teams
  38. 38. # of matches that toxic players are reported by ally-only, enemy-only, or both
  39. 39. Category of toxic playing
  40. 40. # of matches
  41. 41. 43 Ingroup favoritism is observed! +47% More reports come from enemies than allies H2.1 For toxic behavior that affects both teams equally, in-group members (teammates) are less likely to submit reports when compared to out-group members. is supported.
  42. 42. Intra-group Conflicts and Socio-political Factors What/why do conflicts happen within a team?
  43. 43. Community vs. Association
  44. 44. LoL teams are close to associations • Task-oriented association (Gesellschaft) rather than social community (Geminshaft) • Randomly matched • Short-term/ephemeral • Goal-oriented • Thus, toxic players feel neither a sense of a team nor qualms about harassing poor players in a team
  45. 45. 47 Possible socio-political factors affecting intragroup conflicts RQ3: What is the impact of socio-political factors on toxic behavior? Korea NA / EUW Collectivism Political belief Individualism Group goal Emphasis Ones’ self “There’s no ‘I’ in TEAM!’ Quote “Well… there ain’t no ‘WE’ either”
  46. 46. • Individualism vs. collectivism • focus on ones’ self vs. group goal 48 Testable hypotheses about regional diff. H3.2 Reports on toxic behavior that largely affects the result of the match are more often submitted in Korea than in other regions. H3.3 Reports on behavior that largely affects the result of the match are more likely to be punished in Korea than in other regions. H3.1 Due to a more group-success oriented socio- political environment, cyberbullying offenses are less likely to be punished in Korea than in other regions.
  47. 47. 49 Korea is the most generous region for cyberbullying. (O+V)17.1% (O+V) 14.3% (O+V) 9.7% * Effect of region on pardon is confirmed by Chi-square test (p < .0001) O V
  48. 48. 50 Korea is the most generous region for cyberbullying. H3.1 Due to a more group-success oriented socio- political environment, cyberbullying offenses are less likely to be punished in Korea than in other regions. is supported.
  49. 49. 51 Korea players most actively report assisting enemy and intentional feeding Korea NA EUW Avg. 1.75 1.71 1.48 * Effect of region on reports from teammates is confirmed by a Kruskal- Wallis test & a post-hoc test using Mann-Whitney tests (p < .0001)
  50. 50. 52 Korea players most actively report assisting enemy and intentional feeding * Effect of region on reports from teammates is confirmed by a Kruskal- Wallis test & a post-hoc test using Mann-Whitney tests (p < .0001) Korea NA EUW Avg. 1.75 1.71 1.48 H3.2 Reports on toxic behavior that largely affects the result of the match are more often submitted in Korea than in other regions. is supported.
  51. 51. 53 KR reviewers are more likely to perceive AE and IF as severe toxic playing * Effect of region on the percentage of P-O is confirmed by a Chi-square test (p < .0001) (A+I) 48% (A+I) 27% (A+I) 24%A I
  52. 52. 54 KR reviewers are more likely to perceive AE and IF as severe toxic playing H3.3 Reports on behavior that largely affects the result of the match are more likely to be punished in Korea than in other regions. is supported.
  53. 53. Team-cohesion and Performance How does toxic playing affect team performance?
  54. 54. 56 https://www.youtube.com/watch?v=KatTVGAEu80
  55. 55. • Actively studied in sports and organizational science because a team is an important unit there. • Significant positive correlation has been confirmed. Cohesion-performance relationship
  56. 56. • A negative player degrades team cohesion (and team performance) as follows: 1. violates interpersonal norms 2. leads to negative emotions 3. reduces trust among teammates 4. triggers defensive reactions and sacrifices overall function of the group 58 How toxic playing degrades team performance
  57. 57. • A negative player degrades team cohesion (and team performance) as follows: 1. violates interpersonal norms 2. leads to negative emotions 3. reduces trust among teammates 4. triggers defensive reactions and sacrifice overall function of the group 59 How toxic playing degrades team performance RQ4: What is the relationship between toxic behavior, player reports, and team performance? H4.1 More reports come from matches where the accused was on the losing team.
  58. 58. Alternative explanation looks also reasonable ✓Negative outcome can trigger reporting of toxic behavior • After losing the match, people can report poor players and gain some emotional satisfaction H4.2 There are more cases pardoned when the accused was on the losing team than on the winning team.
  59. 59. Winning ratio = a proxy of team performance • If 100 matches are reported for verbal abuse, and a team with a toxic player wins 40 matches, team performance when verbal abuse occurs is approximated as 40/100=0.4.
  60. 60. (We focus on toxic playing reported by teammates.)
  61. 61. Win. ratio of reported cases ≪ 50% 50%
  62. 62. More reports come from losing teams! 50% H4.1 More reports come from matches where the accused was on the losing team. is supported.
  63. 63. + one more interesting finding 50% the win. ratio with inappropriate name < 50%
  64. 64. 67 Winning cases are more pardoned
  65. 65. 68 Winning cases are more pardoned (We focus on cases that teammates report.) H4.2 There are more cases pardoned when the accused was on the losing team than on the winning team. is not supported.
  66. 66. Bystander effect and vague nature of toxic playing How do they affect reporting and judging? In-group Favoritism and Out-group Hostility Does team-competition setting affect reporting? Intra-group Conflicts and Socio-political Factors What/why do conflicts happen within a team? Team-cohesion and Performance How does toxic playing affect team performance?
  67. 67. • LoL is a virtual laboratory to understand human behavior. • The Tribunal offers a unique chance to study toxic behavior by crowdsourcing. • It is hard to deal with toxic playing • Toxic playing is vague, and it affects reporting and judging. • Players are not actively engaged in reporting. • Various biases (e.g. teammate, culture, and match result) exist. 70 Conclusion
  68. 68. 71 A full paper with an additional executive summary is available via http://arxiv.org/abs/1504.02305

×