1. Exploring cyberbullying and
other toxic behavior in
team competition online games
@haewoon Kwak
Qatar Computing Research Institute
Doha, Qatar
Jeremy Blackburn
Telefonica Research
Barcelona, Spain
Seungyeop Han
University of Washington
Seattle, USA

2. Bullying that takes place using electronic technology.

3. This image is retrieved from http://ucrtoday.ucr.edu/24423

5. http://www.madesimplegroup.com/blog/tutorial-incorporating-social-media-site/

6. http://www.madesimplegroup.com/blog/tutorial-incorporating-social-media-site/
http://kirby.wikia.com/wiki/Sword

7. 7
Game industry is huge
Active players
(month) 67M 7M
eSports Prize pool* $2M $11M
*2014 US Open Men’s tennis prize pool is “only” $3M

8. 8
Gamers invest lots of their time and
energy into games
http://techraptor.net/content/imo-video-game-addiction-complicated-issue

9. 9
Games are popular in younger generation
http://archive.news.softpedia.com/news/Game-Addiction-Similar-to-Drug-Addiction-100447.shtml

10. MonumentValley

12. Competition in games: Pros
• Competition is a good gaming element for enjoyment

13. • It naturally causes tensions
• and it leads to an increasing concern about negative
behavior (called toxic behavior)
Competition in games: Cons

14. • The definition of toxic playing is often unclear due to
differences in expected behavior, customs, rules, and ethics
across games.
14
Major hurdle: Vagueness of toxic
behavior
http://www.slideshare.net/Vince77580/OBDecision-and-Culture1

15. noob!
uninstall the game
…
What r u doing?
r u sleeping?
NOOB
OMG
surrender at 20
ks
Quiz: which is toxic and which is not?
Fking noob
wtf

16. A case study: RiotGames

17. The game studio developing League of
Legends, the most popular online game

18. The Tribunal (May 2011~2014)
A crowdsourcing platform to fight
against habitual toxic players
http://leagueoflegends.wikia.com/wiki/The_Journal_of_Justice:_Volume_1,_Issue_18

19. Reviewers vote for pardon / punish
User reports (anonymized)
Chat logs (anoymized)
Reported matches
General information

20. 7 predefined categories in the Tribunal
• Offensive language
• Verbal abuse
• Assisting enemy team
• Intentional feeding [suicide]
• Negative attitude
• Inappropriate [handle] name
• Spamming
• Unskilled player
• Refusing to communicate with team
• Leaving the game / AFK [away from keyboard]
Cyberbullying

21. Decision (Punish or pardon)
Agreement level
Outcome (Win or lose)
User reports
Chat logs
In-game performance
Vote results

22. 6 agreement levels in the Tribunal
Pardon
Majority Majority
Strong
Majority
Strong
Majority
Overwhelming
Majority
Overwhelming
Majority
Punish

23. 23
Data collection from different servers
NA EUWest KR All
Player 590,311 649,419 220,614 1,460,344
Match 2,107,522 2,841,906 1,066,618 6,016,046
Report 3,441,557 5,559,968 1,893,433 10,898,958
* KR Tribunal starts from November 2012, while other two Tribunals start from May 2011.
To study
regional difference!

24. • STFU NOOB! Predicting crowdsourced decisions on
toxic behavior in online games (WWW’14)
http://arxiv.org/abs/1404.5905
• Linguistic analysis of toxic behavior in an online video
game (EGG’14 workshop)
http://arxiv.org/abs/1410.5185
24
Our previous work on the Tribunal

25. • We explore toxic behavior to answer the question
“Why is it hard to deal with toxic behavior?” with
guidance of existing theories and hypotheses by using
large-scale data collection.
25
Research goal in this work

26. • Possible biases in reporting and reviewing
• Bystander effect and low deg. of participation in reporting
• In-group favoritism and out-group hostility
• Intra-group conflicts and socio-political factors
• Team-cohesion and performance
• Summary
26
Outline of the rest of the talk
Research Q / formulated H
Supported H
Not supported H

27. Bystander effect and vague nature of toxic playing:
How do they affect reporting and judging?

28. Deindividuation theory
Why do people show extreme behavior when immersed in a group?
http://nypost.com/2015/02/20/soccer-hooligan-rampage-is-romes-terrorist-fears-realized/

29. Anonymous online environments of LoL
• No presence of face-to-face communication.
• Reduced self-awareness
• Less constraints/responsibility
• Thus, observers respond to toxic behavior as bystanders
• social control does not work well
RQ1a: How active are players in reporting toxic behavior?

30. • Degree of participation in reporting toxic playing
directly connects to
✓Feasibility/efficiency of the overall system to detect
toxic playing by voluntary player reports
Why is this question important?

31. 31
http://lifehacker.com/5930167/cut-through-the-bystander-effect-by-explicitly-telling-someone-to-call-911
H1.1 If there is a request asking to report toxic players,
the number of reports is increased.

32. • Unclear definition of toxic playing make it hard not only to
report but also to judge whether it is toxic.
32
Also, vague nature of toxic playing
RQ1b: How does the vague nature of toxic playing
affect tribunal decisions?

33. Low number of reports per match is found
• Mean: 1.812
• Median: 1
• Less than 2 players per match report toxic behavior
• Players have different perception
• Players are not actively engaged in reporting
(remember 5 vs. 5 setting!)

34. Impact of asking reports
• If allies explicitly ask enemies to report the toxic player
through all chats, such as ‘plz report’, they really do!
• Chi-square test, p < .0001
• Odds ratio shows that the probability of reports by
enemies with allies’ requests is 16.37 times higher than
that without such requests
H1.1 If there is a request asking to report toxic players,
the number of reports is increased. is supported.

35. 35
Vagueness of toxic playing affects review
For 1 in 4 spamming cases in KR,
reviewers do not find the player toxic.

36. In-group Favoritism and Out-group Hostility
Does team-competition setting affect reporting?

37. Ingroup favoritsm
http://www.agefotostock.com/en/Stock-Images/Royalty-Free/FNC-FAN2034560
Ingroup favoritsm
Outgroup hostility

38. What can we see from intergroup conflicts?
Ingroup favoritsm
RQ2: What is the difference between reporting behavior
of the toxic player’s teammates and his opponents?
H2.1 For toxic behavior that affects both teams equally,
in-group members (teammates) are less likely to submit
reports when compared to out-group members.

39. Proper category for hypothesis testing
• Assisting enemy team
• Intentional feeding [suicide]
• Offensive language
• Verbal abuse
• Negative attitude
• Inappropriate [handle] name
• Spamming
• Unskilled player
• Refusing to communicate with team
• Leaving the game / AFK [away from keyboard]
- visible to all players
- same impact to both teams

40. # of matches that toxic players are reported
by ally-only, enemy-only, or both

41. Category of toxic playing

42. # of matches

43. 43
Ingroup favoritism is observed!
+47% More reports come from enemies than allies
H2.1 For toxic behavior that affects both teams equally,
in-group members (teammates) are less likely to submit
reports when compared to out-group members.
is supported.

44. Intra-group Conflicts and Socio-political Factors
What/why do conflicts happen within a team?

45. Community vs. Association

46. LoL teams are close to associations
• Task-oriented association (Gesellschaft) rather than social
community (Geminshaft)
• Randomly matched
• Short-term/ephemeral
• Goal-oriented
• Thus, toxic players feel neither a sense of a team nor
qualms about harassing poor players in a team

47. 47
Possible socio-political factors
affecting intragroup conflicts
RQ3: What is the impact of socio-political factors on
toxic behavior?
Korea NA / EUW
Collectivism Political belief Individualism
Group goal Emphasis Ones’ self
“There’s no ‘I’ in
TEAM!’
Quote
“Well… there ain’t
no ‘WE’ either”

48. • Individualism vs. collectivism
• focus on ones’ self vs. group goal
48
Testable hypotheses about regional diff.
H3.2 Reports on toxic behavior that largely affects the
result of the match are more often submitted in Korea
than in other regions.
H3.3 Reports on behavior that largely affects the result
of the match are more likely to be punished in Korea
than in other regions.
H3.1 Due to a more group-success oriented socio-
political environment, cyberbullying offenses are less
likely to be punished in Korea than in other regions.

49. 49
Korea is the most generous region for
cyberbullying.
(O+V)17.1%
(O+V) 14.3%
(O+V) 9.7%
* Effect of region on pardon is confirmed by
Chi-square test (p < .0001)
O
V

50. 50
Korea is the most generous region for
cyberbullying.
H3.1 Due to a more group-success oriented socio-
political environment, cyberbullying offenses are less
likely to be punished in Korea than in other regions.
is supported.

51. 51
Korea players most actively report
assisting enemy and intentional feeding
Korea NA EUW
Avg. 1.75 1.71 1.48
* Effect of region on reports from teammates is confirmed by a Kruskal-
Wallis test & a post-hoc test using Mann-Whitney tests (p < .0001)

52. 52
Korea players most actively report
assisting enemy and intentional feeding
* Effect of region on reports from teammates is confirmed by a Kruskal-
Wallis test & a post-hoc test using Mann-Whitney tests (p < .0001)
Korea NA EUW
Avg. 1.75 1.71 1.48
H3.2 Reports on toxic behavior that largely affects the
result of the match are more often submitted in Korea
than in other regions. is supported.

53. 53
KR reviewers are more likely to perceive
AE and IF as severe toxic playing
* Effect of region on the percentage of P-O is confirmed by a Chi-square test (p < .0001)
(A+I) 48%
(A+I) 27%
(A+I) 24%A
I

54. 54
KR reviewers are more likely to perceive
AE and IF as severe toxic playing
H3.3 Reports on behavior that largely affects the result
of the match are more likely to be punished in Korea
than in other regions. is supported.

55. Team-cohesion and Performance
How does toxic playing affect team performance?

56. 56
https://www.youtube.com/watch?v=KatTVGAEu80

57. • Actively studied in sports and organizational science
because a team is an important unit there.
• Significant positive correlation has been confirmed.
Cohesion-performance relationship

58. • A negative player degrades team cohesion (and team
performance) as follows:
1. violates interpersonal norms
2. leads to negative emotions
3. reduces trust among teammates
4. triggers defensive reactions and sacrifices overall
function of the group
58
How toxic playing degrades team
performance

59. • A negative player degrades team cohesion (and team
performance) as follows:
1. violates interpersonal norms
2. leads to negative emotions
3. reduces trust among teammates
4. triggers defensive reactions and sacrifice overall
function of the group
59
How toxic playing degrades team
performance
RQ4: What is the relationship between toxic behavior,
player reports, and team performance?
H4.1 More reports come from matches where the
accused was on the losing team.

60. Alternative explanation looks also reasonable
✓Negative outcome can trigger reporting of toxic behavior
• After losing the match, people can report poor players and
gain some emotional satisfaction
H4.2 There are more cases pardoned when the accused
was on the losing team than on the winning team.

61. Winning ratio = a proxy of team performance
• If 100 matches are reported for verbal abuse, and a team
with a toxic player wins 40 matches, team performance
when verbal abuse occurs is approximated as 40/100=0.4.

62. (We focus on toxic playing reported by teammates.)

64. Win. ratio of reported cases ≪ 50%
50%

65. More reports come from losing teams!
50%
H4.1 More reports come from matches where the
accused was on the losing team. is supported.

66. + one more interesting finding
50%
the win. ratio with inappropriate name < 50%

67. 67
Winning cases are more pardoned

68. 68
Winning cases are more pardoned
(We focus on cases that teammates report.)
H4.2 There are more cases pardoned when the accused
was on the losing team than on the winning team.
is not supported.

69. Bystander effect and vague nature of toxic playing
How do they affect reporting and judging?
In-group Favoritism and Out-group Hostility
Does team-competition setting affect reporting?
Intra-group Conflicts and Socio-political Factors
What/why do conflicts happen within a team?
Team-cohesion and Performance
How does toxic playing affect team performance?

70. • LoL is a virtual laboratory to understand human behavior.
• The Tribunal offers a unique chance to study toxic
behavior by crowdsourcing.
• It is hard to deal with toxic playing
• Toxic playing is vague, and it affects reporting and
judging.
• Players are not actively engaged in reporting.
• Various biases (e.g. teammate, culture, and match result)
exist.
70
Conclusion

71. 71
A full paper with an additional executive
summary is available via
http://arxiv.org/abs/1504.02305