Poster presented at the Games User research summit 2012

1. The State of the Research on Games User Research: A Brief Review
Salvador Garcia-Martinez
Introduction Key Results Discussion
Problem Conclusions
• Games User Research is an area that has been widely studied •From the authors’ keywords analysis it is possible to conclude
from both, the industry and the academia viewpoints. that two emerging trends are heuristic evaluation and the use
of metrics. However, it is important to point out the lack of
• There is still a need to organize the most relevant literature consistency in use of terminology.
and to identify the current trends and future research in the
field. •Research in the field was categorized in six groups: game
testing research, player research, gameplay experience, game
design, interface design and others.
•Most used data collection methods are: 1) questionnaires and
Figure 1. Authors’ Keywords Tag Cloud surveys; 2) metrics; 3) observations; and 4) biometrics.
Objectives Interface Design:
•There is a special interest for automated data collection
techniques that involve the use of metrics, biometrics and
•Includes selected topics such as
• Create an overview of the foundations of Games User Research numerical analyses.
Game design: accessibility that are directly related to
in Video Games.
• Design and development of a user-interface design.
•There is also large interest for the design and validation of
game. 2%
• Identify key trends and gaps in the body of literature. heuristics that are used in testing methods such as expert
• Key subcategories: goal design, 5% Others: evaluation.
• Stimulate discussion about current research trends in Games level design guidelines and 7% • Includes theoretical foundations, non-
User Research and emergent topics in the field. usability guidelines. traditional research methodologies such •According to their main goals, articles were classified in
14% as field studies, etc. eight categories: research method development, overview
• Answer the research question: What is the state of the Gameplay Experience: 45% articles, game development, game testing, user experience
research related to Games User Research? •Includes all the topics that are Game Testing Research: and behaviour, research tools, articles focused on user
related to the interaction of the • Methodologies and techniques used to experience and others.
player and the game. 27% test games.
•Key subcategories: affection, fun, • Key subcategories: general topics of •The most frequent goal in the literature was to propose or
gameplay and player perceptions. game testing, methodologies for testing, validate new methodologies or heuristics.
data analysis techniques and data
collection methods. •There is a need to continue developing, testing and
Methodology Player Research:
validating existing methods and guidelines. Empirical
validation could be ideal.
•Study and analysis of the player.
•This review was done in a period of four months. •Key subcategories: differences between types •It is also required to conduct longer studies and consider
of users, user research for specific game genres more games spanning different genres and user types.
Articles were retrieved searching in: and user psychology.
•For numerical analyses, it is necessary to continue exploring
•Four academic databases: Figure 2. Emerging Themes from Code Analysis what are the best variables and outcome measures to be
• INSPEC. Citations with abstracts of literature in considered.
physics, electrical engineering, electronics and
computing.
• Web of Science. Multidisciplinary database that
contains literature of the sciences, with a focus on
Engineering and Computer Science.
Future Work and
• ACM Digital. Articles published in the Association for
Computing Machinery (ACM). Limitations
• Academic Search Premier. Multidisciplinary database
•This review presents just the “global picture” of the status of
that includes literature in fields such as Social
the research in Games User Research. The next step is to
Sciences, Sciences and Engineering
explore the details.
•Journals and Magazines: Game Studies, Eludamos,
•The search parameters used for this review are limited. In
Gamasutra, Game Developer Magazine and Wired Magazine.
future work improvement of the search scope using newly
identified key terms and synonyms is expected.
The keywords used were :
• [Video games, computer games OR digital games] AND Figure 3. Representative data collection methodologies (2006 - 2011) •Because the short amount of space in a poster, it was not
• [[User research or player research] OR
possible to provide all the results and necessary arguments to
• [usability or play testing]].
Research Methods Development back up all the review’s conclusions.
User Experience and Behaviour Others • Propose new methodologies or
Inclusion criteria:
• Explore how the user plays. heuristics.
4% 3% • Validate or update existing
•Articles must be related to the research question. 6%
Most used research methods: methodologies and heuristics.
26%
1. Metrics
•Years: 2005 – 2011
2. Biometrics Most used research methods:
•The full version of the article should be available on-line.
3. Questionnaires and Surveys
19%
1. Questionnaires
2. Metrics
Contact Information
Games Testing 3. Expert Evaluation
•Very similar articles from the same author(s) were excluded. • Test specific games usually at the 10% 19%
development stage. Overview articles.
•Repeated articles were deleted. 13% Salvador Garcia-Martinez
Game Development Technoculture, Art and Games (TAG) Research Centre
•Total: 89 articles
• Propose principles that can be incorporated while developing games. Concordia University
• Identify specific recurrent usability or playability problems.
•In order to identify key trends and gaps in the literature, Montreal, QC
articles were coded according to their goals, research
Most used research methods:
methodologies, types of participants (children, adults,
1. Literature reviews and authors’ expertise. salvador.garcia.martinez@gmail.com
university students or others) and results.
2. Questionnaires
3. Observations www.sal-garcia.com
Figure 4. Distribution of articles according to research goals.