This document analyzes discussion in the citizen science game EyeWire. It finds that real-time chat appears to have a positive effect on player activity, with 'active' players who used chat commands completing many more games on average. Chat is predominantly used for game commands, especially by 'highly active' players. While chat may facilitate interaction and teamwork, it does not seem to support in-depth scientific discussion. Real-time chat and gamification can benefit engagement if coordination is needed within the game, but additional channels may be required for deeper scientific discourse.

1. R. Tinati, M. Luczak-Rösch, E. Simperl, W. Hall, N. Shadbolt
University of Southampton, SOCIAM, Web & Internet Science
Web Science 2015
‘/Command and Conquer’
Analysing Discussion in a Citizen
Science Game

2. background
related work
data
findings
conclusions

3. background
related work
data
findings
conclusions

4. Citizen Science
It’s not new! In the 16th century, scientific discovery was a collaborative process
The Web/crowdsourcing now helps this happen again
Solving scientific problems using crowdsourcing techniques
Emphasis on communities that are volunteers, unpaid, and potentially
untrained in the domain
It has been applied to various domains: astronomy, medicine, nature, history

5. EyeWire
“A game to map the brain”
A gamified citizen science project
Started: Jan 2012 - present
160,000+ active volunteers today
Players from 145 Countries
Sponsored by National
Telecommunications Organisation
in South Korea

6. EyeWire’s Task
Players are tasked with filling in the 3D structures of Brain neurons
Uses gamification components to turn the task into gaming activities
Gamification features as a way to increase interest in the task
Leader boards, points, badges, roles
The original task is still visible, so not full gamification (e.g. FoldIT)

9. EyeWire’s Talk
Players can talk in real-time with each other
The Real-time chat interface is embedded in the main user
interface
Players can issue commands using specific pre-assigned terms
report or list stats and current leaders, communicate directly with
other players, join teams, ask for help, or silence each other

10. EyeWire’s Competitions and Teams
EyeWire runs gaming competitions to speed up the classification
process
Competitions were a grassroots movement led by the active
players
Originally only during competitions, players can join teams and
compete to gain the most points
Real-time chat facilitates the process of coordination and
progress.

12. We seek to understand real-time chat
what is the relationship between real-time chat and the
gaming process?
what do players use real-time chat for?
does real-time chat facilitate discourse and collaboration
between players?
Past studies have shown that passive discussion plays an
important role in the success of citizen projects
Based on these findings, how can EyeWire better support its community
Study Motivations

13. background
related work
data
findings
conclusions

14. related work
Motivation of citizen scientists (Raddick et al. 2010, 2013; Rotman, 2012)
Discussions related to task completion and success
(Hassman et al. 2013, Mugar et al. 2014)
Cross-project participation bolsters engagement and sustained activity
(Luczak-Rösch et al. 2014)
The use of gamification in citizen science projects (Bowser et al. 2013,2014)

15. background
related work
data
findings
conclusions

16. data
EyeWire player gaming and chat log activity between
January 2012 and August 2014
97,000+ Players
4,000,000+ completed games
835,000+ chat messages

17. Analysis Process
General platform analysis
Player analysis (Churn, Player Types)
Real-time chat analysis
Real-time Chat and Task Completion

18. background
related work
data
findings
conclusions

19. general overview
68.2% of games were completed by 10.9% of EyeWire players
… and just over 1% of EyeWire players were responsible for over 50% of the total
games (2 million)
‘active’ players on average completed a higher number of games in comparison to
gaming only players (255 games compared to 15).
‘active’ players on average took longer to complete games in comparison to gaming
only players (293s compared to 258s).
The overall account length (the total time they were active on EyeWire) of ’active’
players was nearly 4 times longer.

20. general platform characteristics
Rela-timechatmessages
Games Completed

21. Player Churn (method)
Purpose:
Measure if players remain active after a given period of time
We separate a player’s gaming session based on task activity
We also separate player chat activity into the corresponding timeframe
(of their gaming)
Activity is segmented into monthly (m) time slices
We examine if players in m are in m-1. If not, players are identified as
non-participating players.

22. gaming activity churn

23. real-time chat churn

24. Types of players
Purpose:
Distinguish different types of players based on how they
interaction with they EyeWire features
How does these types of players relate to their gaming activity
We looked at features including
use of chat messages
use of chat commands
types of commands used
timing of command use during gaming

25. Types of players
Chat + Gaming Players
[‘active players’]
(10.9%)
Command
Players
(3.2%)
All Players
(100%)
‘Highly Active’
(1.1%)

26. Players using commands
12% of messages were made during gaming sessions
29.5% of ‘active players’ used commands
60.3% of ‘highly active players’ used commands
players that used commands within their chat completed over 6
times as many games on average (642 in comparison to 94)
Command-using player were slower to complete games in
comparison to non-command using players (317s compared to
286s)

27. Players using commands
CommandsUsed(log)
Games Completed (log)

28. Chat and command use during playing
Purpose:
Can we examine how the use of chat at different points in time affects
gaming sessions?
The 5 stages that a chat message occurs during the completion of a
game.

29. Chat and command use during playing
12% of chat messages were made at the same time as chatting
Games took on average 464 seconds in comparison to 364 seconds
Stage Messages Associated
Games
Avg. Message
Len. (Chars)
Avg Class.
Duration (Secs)
Commands
Used
Before (Q0 ) 19,942 11,811 26 142.6 11,271
Start (Q1) 13,154 23,583 25 865.6 17,070
During (Q2-3) 29,783 50,143 30 461.9 45,540
End (Q4) 14,497 21,075 30 344.0 22,660
After (Q5) 18,735 12,236 20 177.51 8,972

30. Chat and command use during playing
Stage Messages Associated
Games
Avg. Message
Len. (Chars)
Avg Class.
Duration (Secs)
Commands
Used
Before (Q0 ) 19,942 11,811 26 142.6 11,271
Start (Q1) 13,154 23,583 25 865.6 17,070
During (Q2-3) 29,783 50,143 30 461.9 45,540
End (Q4) 14,497 21,075 30 344.0 22,660
After (Q5) 18,735 12,236 20 177.51 8,972
Messages before gaming sessions (Q0) and at the start (Q1) are shorter in length
than during (Q2-3) or at the end (Q4)
Messages at the start (Q1) or during (Q2-3) increased game completion time

31. Chat and command use during playing
before start during end after
“Introductions,
Welcome”
“/silence,
/team
“Questions,
Help”
“/team,
Interface
“Farewells,
/gm

32. background
related work
data
findings
conclusions

33. conclusions
what is the relationship between real-time chat and the gaming process?
real-time chat appear to have a positive effect on player activity
what do players use real-time chat for?
game commands are the predominate use of real-time chat, especially for the
‘highly active’ players
conversations are short, without scientific vocab (unlike other CS platform)
does real-time chat facilitate discourse and collaboration between players?
real-time chat may provide a mechanism for players to interact with each other,
and work in teams, but we did not detect user flows in chat sessions

34. Implications for citizen science
Real-time chat and gamified features may be beneficial to engagement and player
success
the use of game commands appears to be related with player activity and length of
account activity.
real-time chat may facilitate co-operative playing between members
if co-ordination within the game/platform is required, then a communication
mechanism such as real-time chat is essential.
real-time chat may not be a sufficient mechanism to facilitate in-depth scientific
discourse
additional communication channels may be required (Such as Zooniverse talk)

35. thank you
r.tinati@soton.ac.uk
sociam.org