GAME RESEARCH
METHODS
AN OVERVIEW
EDITED BY PETRI LANKOSKI & STAFFAN BJÖRK
What
 Overview of game research methods
book
 Different research design
 Pluralistic perspective on research
approaches
 A bit about philosophy of science &
ethics
Research Practices
 Game research can be seen as its own
research field or a research domain
common to many research fields
 Established disciplines typically have a
core set of settled research methods
 Not so for new disciplines or those that
are interdisciplinary
A Very Short Introduction to
Philosophy of science
 Objectivity
 Critique by Hume, Popper, Devey
 Theory-laden (Kuhn)
 Science the most reliable way to form
knowledge
 Even the knowledge may be fallible and
approximate
Unit of Analysis
 What is studied
 Can be different that the data gathered
 Well-established research fields typically
have identified what is the individual
data collected and analysed
Method?
 A method is a specific systematic
procedure for handling data (of the
chosen Unit of Analysis)
 Data gathering methods (e.g., interview,
observation, questionnaire)
 Data analysis methods (e.g., T-test, discourse
analysis)
 Mixed methods
 Qualitative and quantitative mixed
Method, cont.
 Qualitative method
 Rich descriptions of a
phenomenon
 Quantitative methods
 Relations between measured
variables
 Mixed methods
approaches
 Combines qualitative and
quantitative methods in various
ways
?
Images:pixabay(CC0)
Methods, cont.
 Research question implies possible research
methods
 But, you need research to find your research
question
 Sometimes you start with other things that
research questions
 Research domains
 Research projects
 …
 Research is messy business
Overview of our forthcoming
book
For games
 Qualitative approachers
For play and players
 Quantitative approaches
 Quantitative approaches
 Mixed methods
Game development for research
 e.g., design research
What and Not
 We cover various very typical game
research methods and some more
esoteric
 Opportunistic sampling for chapters
 Does (unfortunately) not include all
feasible methods we are aware of, e.g.:
 platform studies
 historical studies methods
Preface
Frans Mäyrä
 “Finding and following a methods means
finding a way. The original etymology of
methodology conveys the same
message: meta hodos in Ancient Greek
meant following a path, as well as
finding the way or means to achieve
certain goal.”
General
Fundamentals for writing a thesis
Carl Magnus Olsson
Argument chains
General
Formal analysis of gameplay
Petri Lankoski & Staffan Björk
Formal analysis of gameplay
Language for describing various
aspects of the game systems
Primitives: components, actions,
goals
Analysis examples
Plant vs Zombies
Ico, combining formal
Qualitativemethodsforstudying
games
Time in videogames
José Zagal & Michael Mateas
Formal analysis of temporal frames
 Language for describing temporal
aspects of the games
Real-world–gameworld–
coordination–fictive
Interaction between frames
Temporal anomalies
Qualitativemethodsforstudying
games
Studying gameplay as interactions
with information
Olle Sköld, Suellen Adams, J.Tuomas Harviainen & Isto
Huvila
Games as information
 How players search and use
information in games
 Analyzing information, kinds of
information and access to information
Qualitativemethodsforstudying
games
Awkward: The importance of reflexivity in
using ethnographic methods
Ashley Brown
Ethnography for games
Focus for MMORPGs
Studying sex using
ethnomethodology
Emotions
Boundary managagement
Qualitativemethodsforstudying
playandplayers
In-Depth interviews for games research
Amanda Cote & Julia G. Raz
When to use interview
Ethical consideration
Participant selection & recruiment
Conducting interviews
Thematic analysis
Qualitativemethodsforstudying
playandplayers
Studying thoughts: Stimulated recall as a
game research method
Jori Pitkänen
Records for supporting interview
E.g., gameplay video
Possible to look at a specific event
and discuss about that
Planning
Conducting
Analyzing
Ethical issues
Qualitativemethodsforstudying
playandplayers
Focus group interviews as a way to evaluate
and understand gameplay experiences
Lina Eklund
Form of interview where people
are interviewed as a group
When to use
Planning
Conducting
How to moderate a group
Transcribing pratices
Analyzing
Qualitativemethodsforstudying
playandplayers
Quantitative methods and analyses for the
study of players and their behaviour
Richard N. Landers & Kristina N. Bauer
General introduction to
quantitative research
Reliability & validity
Correlational designs
Experiment designs
Quasi-experiment designs
Quantitativeapproaches
Sex , Violence and Learning: Doing effect
assessment for games
Andreas Lieberoth, Kaare Bro Wellnitz & Jesper Aagaard
What counts as evidence?
What can be inferred from the
results?
Effect size
P values & confidence intervals
Quantitativeapproaches
Stimulus games
Simo Järvelä, Inger Ekman, J. Matias Kivikangas & Niklas Ravaja
Using games as a stimulus in
psychophysiological studies
 Studying cognition & emotions
Advantages & challenges
Quantitativeapproaches
Audio visual analysis of player
experience
Raphaël Marczak & Gareth R. Schott
How to collected gameplay data
automatically from a game
Quantitativeapproaches
An Introduction to Gameplay Data
Visualization
Günter Wallner & Simone Kriglstein
Quantitativeapproaches
Structural equation modeling
Mattias Svahn & Richard Wahlund
Modeling technique for studying
causal relation between variables
 Directly
 Indirectly
Evaluating models
Assessing models
Case example: Agents Against
Power Waste
Quantitativeapproaches
Mixed methods in game research playing
on strengths and countering weaknesses
Andreas Lieberoth & Andreas Roepstorff
Introduction to mixed methods
designs
Why to use mixed method designs
Downsides of mixed method designs
Mixedmethodsapproaches
Systematic interviews and analysis:
Using the repertory grid technique
Carl Magnus Olsson
Basis in personal construct theory by
Kelly
Elements – constructs – links
Research designs
 Qualitative, mixed methods
Mixedmethodsapproaches
Grounded Theory
Nathan Hook
Grounded theory process
Variations
 Qualitative
 Mixed methods
Examples
Mixedmethodsapproaches
 How to use modding in experiment research
 Possible games for modding
 Best practices

Gamedevelopmentforresearch
Extensive modding for
experimental game research
M. Rohangis Mohseni, Benny Liebold & Daniel
Pietschmann
Experimental Game Design
Annika Waern &Jon Back
When is game design experimental?
Focus on qualitative design research
Example case Codename Heroes
Gamedevelopmentforresearch
Summary
 The Book
 Overview of different methods
 How to use them
 Something about research in general
 Possible advantages
 Toolbox of methods
 Understanding of different methods, which fit your research,
and how to apply them
 Help reflecting on your research
 What is your research question?
 What is your unit of analysis?
Aims and Perspective
 Confusion reigning in game research
 What should be studied? (games, gamers,
gaming, platforms, texts, etc.)
 What are the legitimate methods?
 What does legitimate mean in this context?
 We don’t believe there is one correct
answer, i.e., we see game research as an
interdisciplinary field
 Method selection should be motivated
 This requires awareness of different approaches
Aims and Perspective, cont.
 To advance game research as a field,
research approaches needs to be more
explicit
 This puts demands on researchers to discipline
themselves about being explicit about what they
are doing and why (not why others are wrong)
 Descriptions of approaches to provide a
common source where the approaches can
be compared
 Research methods is one aspect of this
Petri Lankoski
 petri.lankoski@sh.se
Staffan Björk
 staffan.bjork@ait.gu.se
The book at ETC Press
 http://press.etc.cmu.edu/content/game-research-
methods-overview

Game research methods book introduction

  • 1.
    GAME RESEARCH METHODS AN OVERVIEW EDITEDBY PETRI LANKOSKI & STAFFAN BJÖRK
  • 2.
    What  Overview ofgame research methods book  Different research design  Pluralistic perspective on research approaches  A bit about philosophy of science & ethics
  • 3.
    Research Practices  Gameresearch can be seen as its own research field or a research domain common to many research fields  Established disciplines typically have a core set of settled research methods  Not so for new disciplines or those that are interdisciplinary
  • 4.
    A Very ShortIntroduction to Philosophy of science  Objectivity  Critique by Hume, Popper, Devey  Theory-laden (Kuhn)  Science the most reliable way to form knowledge  Even the knowledge may be fallible and approximate
  • 5.
    Unit of Analysis What is studied  Can be different that the data gathered  Well-established research fields typically have identified what is the individual data collected and analysed
  • 6.
    Method?  A methodis a specific systematic procedure for handling data (of the chosen Unit of Analysis)  Data gathering methods (e.g., interview, observation, questionnaire)  Data analysis methods (e.g., T-test, discourse analysis)  Mixed methods  Qualitative and quantitative mixed
  • 7.
    Method, cont.  Qualitativemethod  Rich descriptions of a phenomenon  Quantitative methods  Relations between measured variables  Mixed methods approaches  Combines qualitative and quantitative methods in various ways ? Images:pixabay(CC0)
  • 8.
    Methods, cont.  Researchquestion implies possible research methods  But, you need research to find your research question  Sometimes you start with other things that research questions  Research domains  Research projects  …  Research is messy business
  • 9.
    Overview of ourforthcoming book For games  Qualitative approachers For play and players  Quantitative approaches  Quantitative approaches  Mixed methods Game development for research  e.g., design research
  • 10.
    What and Not We cover various very typical game research methods and some more esoteric  Opportunistic sampling for chapters  Does (unfortunately) not include all feasible methods we are aware of, e.g.:  platform studies  historical studies methods
  • 11.
    Preface Frans Mäyrä  “Findingand following a methods means finding a way. The original etymology of methodology conveys the same message: meta hodos in Ancient Greek meant following a path, as well as finding the way or means to achieve certain goal.” General
  • 12.
    Fundamentals for writinga thesis Carl Magnus Olsson Argument chains General
  • 13.
    Formal analysis ofgameplay Petri Lankoski & Staffan Björk Formal analysis of gameplay Language for describing various aspects of the game systems Primitives: components, actions, goals Analysis examples Plant vs Zombies Ico, combining formal Qualitativemethodsforstudying games
  • 14.
    Time in videogames JoséZagal & Michael Mateas Formal analysis of temporal frames  Language for describing temporal aspects of the games Real-world–gameworld– coordination–fictive Interaction between frames Temporal anomalies Qualitativemethodsforstudying games
  • 15.
    Studying gameplay asinteractions with information Olle Sköld, Suellen Adams, J.Tuomas Harviainen & Isto Huvila Games as information  How players search and use information in games  Analyzing information, kinds of information and access to information Qualitativemethodsforstudying games
  • 16.
    Awkward: The importanceof reflexivity in using ethnographic methods Ashley Brown Ethnography for games Focus for MMORPGs Studying sex using ethnomethodology Emotions Boundary managagement Qualitativemethodsforstudying playandplayers
  • 17.
    In-Depth interviews forgames research Amanda Cote & Julia G. Raz When to use interview Ethical consideration Participant selection & recruiment Conducting interviews Thematic analysis Qualitativemethodsforstudying playandplayers
  • 18.
    Studying thoughts: Stimulatedrecall as a game research method Jori Pitkänen Records for supporting interview E.g., gameplay video Possible to look at a specific event and discuss about that Planning Conducting Analyzing Ethical issues Qualitativemethodsforstudying playandplayers
  • 19.
    Focus group interviewsas a way to evaluate and understand gameplay experiences Lina Eklund Form of interview where people are interviewed as a group When to use Planning Conducting How to moderate a group Transcribing pratices Analyzing Qualitativemethodsforstudying playandplayers
  • 20.
    Quantitative methods andanalyses for the study of players and their behaviour Richard N. Landers & Kristina N. Bauer General introduction to quantitative research Reliability & validity Correlational designs Experiment designs Quasi-experiment designs Quantitativeapproaches
  • 21.
    Sex , Violenceand Learning: Doing effect assessment for games Andreas Lieberoth, Kaare Bro Wellnitz & Jesper Aagaard What counts as evidence? What can be inferred from the results? Effect size P values & confidence intervals Quantitativeapproaches
  • 22.
    Stimulus games Simo Järvelä,Inger Ekman, J. Matias Kivikangas & Niklas Ravaja Using games as a stimulus in psychophysiological studies  Studying cognition & emotions Advantages & challenges Quantitativeapproaches
  • 23.
    Audio visual analysisof player experience Raphaël Marczak & Gareth R. Schott How to collected gameplay data automatically from a game Quantitativeapproaches
  • 24.
    An Introduction toGameplay Data Visualization Günter Wallner & Simone Kriglstein Quantitativeapproaches
  • 25.
    Structural equation modeling MattiasSvahn & Richard Wahlund Modeling technique for studying causal relation between variables  Directly  Indirectly Evaluating models Assessing models Case example: Agents Against Power Waste Quantitativeapproaches
  • 26.
    Mixed methods ingame research playing on strengths and countering weaknesses Andreas Lieberoth & Andreas Roepstorff Introduction to mixed methods designs Why to use mixed method designs Downsides of mixed method designs Mixedmethodsapproaches
  • 27.
    Systematic interviews andanalysis: Using the repertory grid technique Carl Magnus Olsson Basis in personal construct theory by Kelly Elements – constructs – links Research designs  Qualitative, mixed methods Mixedmethodsapproaches
  • 28.
    Grounded Theory Nathan Hook Groundedtheory process Variations  Qualitative  Mixed methods Examples Mixedmethodsapproaches
  • 29.
     How touse modding in experiment research  Possible games for modding  Best practices  Gamedevelopmentforresearch Extensive modding for experimental game research M. Rohangis Mohseni, Benny Liebold & Daniel Pietschmann
  • 30.
    Experimental Game Design AnnikaWaern &Jon Back When is game design experimental? Focus on qualitative design research Example case Codename Heroes Gamedevelopmentforresearch
  • 31.
    Summary  The Book Overview of different methods  How to use them  Something about research in general  Possible advantages  Toolbox of methods  Understanding of different methods, which fit your research, and how to apply them  Help reflecting on your research  What is your research question?  What is your unit of analysis?
  • 32.
    Aims and Perspective Confusion reigning in game research  What should be studied? (games, gamers, gaming, platforms, texts, etc.)  What are the legitimate methods?  What does legitimate mean in this context?  We don’t believe there is one correct answer, i.e., we see game research as an interdisciplinary field  Method selection should be motivated  This requires awareness of different approaches
  • 33.
    Aims and Perspective,cont.  To advance game research as a field, research approaches needs to be more explicit  This puts demands on researchers to discipline themselves about being explicit about what they are doing and why (not why others are wrong)  Descriptions of approaches to provide a common source where the approaches can be compared  Research methods is one aspect of this
  • 34.
    Petri Lankoski  petri.lankoski@sh.se StaffanBjörk  staffan.bjork@ait.gu.se The book at ETC Press  http://press.etc.cmu.edu/content/game-research- methods-overview

Editor's Notes

  • #6 This graph needs some explaining, e.g. the whole social sciences often have non-tangible UoA rather than humans - design research doesn’t fit in here also and in addition changes things while doing research
  • #8 Everything that looks like nail is not nail If you are used to use hammer, everything looks like nail
  • #14 Methods than have focus on the games itself Games as formal systems Games as information system Not gaming In these, the player is a theoretical concept The method abstracts player e.g., as to player actions
  • #17 Methods has focus on players’ behavior or phenomenology Ethnomethodology Interview methods How to apply the method when studying games
  • #27 Mixed methods approaches combines both qualitative and quantitative methods Introduction to mixed methods research Repertory grid technique
  • #28 Mixed methods approaches combines both qualitative and quantitative methods Introduction to mixed methods research Repertory grid technique
  • #29 Mixed methods approaches combines both qualitative and quantitative methods Introduction to mixed methods research Repertory grid technique
  • #30 Modding for experimental research Developing games for experimental research Qualitative game design research Design research is a special where the created artefact is integral part of research
  • #31 Mixed methods approaches combines both qualitative and quantitative methods Introduction to mixed methods research Repertory grid technique