Adaptability is a metacompetency critically important to the United States Department of Defense and is considered a key component of 21st Century skills by the U.S. Department of Labor and the U.S. Department of Education). Video games are seen as learning environments supporting the acquisition of 21st century skills. Can games, then, be used as components of an effective learning environment that support the development of adaptability? Initially this paper describes the metacompetency of adaptability. Next is how adaptability can be functionally and discretely measured by focusing on its most granular or micromomentary level which we describe as cognitive adaptability. Finally, the authors examine both the nature of cognitive adaptability, interventions that support its development, and how those interventions might be translated into game design features. Toward this end, the paper will also discuss how these features are exhibited in a popular commercially available video game and how it could be employed to test the hypothesis that a play frame of 12 consecutive hours, using a video game meeting the design criteria and example previously discussed, will increase cognitive adaptability in the players

1. Supporting Cognitive
Adaptability Through
Game Design
P. Shane Gallagher, Ph.D
Learning Analysis Reasearch Corp
shane@learninganalysis.org
shane.gallagher.ctr@adlnet.gov
Shenan Prestwich
Advanced Distributed Learning (ADL)
shenan.prestwrich.ctr@adlnet.gov

2. Adaptability?
• Adaptability: identified by U.S.
DoD, DoL, DoEd as important
metacompetency.
• Ability to use existing knowledge
to create innovative problem
solutions
• Repeatedly trying new/different
strategies to solve problems while
reflecting on actions and
incorporating feedback
2

3. How can we frame it?
• Scale of macro to micro
– Macro: adaptive stance,
operational adaptability
– Mid: individual adaptive behavior
– Micro: micromomentary cognitive
processes, i.e., cognitive
adaptability
3

4. Cognitive adaptability?
• Micromomentary adaptive
cognitive processes & functions
• Components
– Cognitive
Openness/Creativity
– Focused Attention
– Cognitive
Flexbility/Metacognition
– Critical Thinking & Problem-
Solving
4

5. Requires
Supports

6. Fostering cognitive adaptability
• Metacompetency?
• Learning environment design?
– Conceptual framework
– Intersection with CA = research void
• Had to draw from other areas of research and
posit how these findings can be translated into
design
– Cognitive remediation therapy (CRT)
– Feature overlap theory
– Game design - MDA 6

7. Intersecting domains
• Cognitive Remediation Therapy (CRT)
– Neurocognitive psychotherapy technique aimed at
improving cognitive flexibility, working memory,
and planning in sub-normal populations
– Emphasizes practicing cognitive microskills
– Wisconsin Card Sorting Test, Stroop Color Word
Test, interventions (verbalizing, scaffolding,
errorless learning)
7

8. Intersecting domains
• Feature Overlap Theory (Halpern, Hansen, &
Riefer, 1990)
– If training is too similar on a surface level to the actual
event, students will reach for superficial connections
– If training teaches a deep, causal understanding of the
material but differs or varies on a surface level,
students exercise ability to make deep connections and
adapt knowledge.
8

9. MDA
“…Hunicke, LeBlanc, and Zubek (2002), where
mechanics are the components of a game at the
level of data representation and algorithms,
dynamics are how the game components interact
with the player and vice-versa, and aesthetics
comprise the emotional response evoked by the
mechanics and dynamics…”

10. FOT MDA
CRT Others…
Conceptual Framework for Developing CA
10

11. A variation on MDA: FFMDA
11

12. FFMDA
12

13. FFMDA
13

14. 5 Features To Increase C A
• Unstated/Implicit Rules
• Unstated/Implicit Shifting of Rules
• Dynamic Shifting Environments
• Open-Ended Gameplay
• Implicit Reinforcement for Individual
Actions/Choices to Achieve Final
Goal
14

15. The Study
Research Questions:
(Q1) Will 12 hours of consecutive gameplay of a video
game with the design features previously described
(Portal 2) cause an increase in cognitive adaptability as Will playing a
measured by cognitive function tests (CANTAB)? Ind var commercial off-the-
=Portal 2 playtime
(Q2) Will those playing Portal 2 be more likely to be shelf game with the
metacognitively aware during play than those not playing five identified features
Portal 2? Ind var Portal 2 play history
(Q3) Will those possessing higher metacognitive will increase cognitive
awareness levels (MAL) also score higher on CANTAB adaptability?
after playing Portal 2 than those who didn’t? Ind var
Portal 2 playtime
(Q4) Are there differences of CA between high MAL
(HMAL) and low MAL (LMAL)? Ind var MAL
(Q5) Is a prior history of playing Portal 2 or like games
positively correlated with higher metacognitive
awareness levels? Ind vars Portal 2 play history and MAL
(Q6) Will MA change over time during game play? Ind
var Portal 2 playtime

16. The Game

18. How we did it…
June 8-17 2012 Used FFMDA
Provided: • Portal 2
• 39 Airmen Measures
• touchscreen • CANTAB
laptops • Metacognitive Awareness
• facilities Inventory
• tech support on Protocol
site • Experimental
• 2 groups

19. Sample
Population
N=39
18 – Control Group (MS Games)
21 – Experimental Group (Portal 2)
Age 17-24, Ave age = 21
Ave years in AF – 1.22
Rank – mostly E1-E3
Education – mostly HS Diploma
Gender – 5.1% Female, 94.9% Male

20. Measures
• Cambridge Cognition CANTAB Eclipse
• http://www.cambridgecognition.com/cantab-tests.asp
• Metacognitve Awareness Inventory
• (Haynie, 2005)
• Game History Questionnaire
• Active MA Survey
• FRAPS recorded game sessions (3)

21. Procedure
June 8
CANTAB baseline
administration
Used for mitigating
practice effect in
repeated measures
Rugged touchscreen
laptops in computer
lab

22. Metacognitive Awareness Inventory
Procedure
June 14
CANTAB pre-test
Metacognitive
Awareness Inventory
AMA

23. Procedure
June 15-16 Airmen played either
Portal II (Intervention)
Has the 5 features that should promote CA
Play for 6 hours over 2 days (12 hours total) with breaks every 1.5
hours
AMA questionnaire each break
N=21
OR
Solitaire, Minesweeper, FreeCell, and Mahjong (Control)
Do not have the 5 features that should promote CA
Commonly found in MS Windows 7
Puzzle/Logic games
Play each for 1.5 hours a day over 2 days (12 hours total)
AMA questionnaire each break
N=18

24. Procedure
June 17 (final day)
• CANTAB posttest
• Game History Questionnaire

25. Results
• Average plays 6-9 hrs/wk
– True for Exp and Control
• Most have never played Portal or Portal 2 before
or have spent little time if they did
– True for Exp and Control
• Majority indicated that time spent playing video
games has decreased in the past year and in the
past 5 years
Metacognitive Awareness Inventory
• Average Score = 78%, Range 65%-95%

26. CANTAB Results
Significant difference (increase) on RVPS between the experimental (Portal) and control
groups (n=39, F=6.126, p=.018).
Part of variance explained by high/low gaming
Tests of Within-Subjects Effects
Measure:RVPA
Type III Sum of
Source Squares df Mean Square F Sig.
Trial * GRP Sphericity Assumed .002 1 .002 6.126 .018
Greenhouse-Geisser .002 1.000 .002 6.126 .018
Huynh-Feldt .002 1.000 .002 6.126 .018
Lower-bound .002 1.000 .002 6.126 .018

27. CANTAB Results
Spatial Span (SSP) -Computerized version of the Corsi Blocks task
• Assesses working memory capacity
• measure of frontal lobe functioning
Previous Portal 2 Play
Significant (p =.012) difference (decrease in errors) both pre and post measure on
SSP between those that have played Portal 2 in the past six months and those that
didn’t.
Tests of Between-Subjects Effects
Measure:SSP
Transformed Variable:Average
Type III Sum of
Source Squares df Mean Square F Sig.
Intercept 4075.283 1 4075.283 1702.932 .000
PRVPORTL2_NUM 16.756 1 16.756 7.002 .012
Error 86.152 36 2.393

28. CANTAB Results
High/Low Gaming Experience
Analysis by high/low gamers shows a large and significant difference
(F=16.778, p=0).
Tests of Between-Subjects Effects
Measure:SSP
Transformed Variable:Average
Type III Sum of
Source Squares df Mean Square F Sig.
Intercept 4565.978 1 4565.978 2341.745 .000
TIME_WEEK_BI 32.714 1 32.714 16.778 .000
Error 70.193 36 1.950

29. CANTAB Results
Analysis of SSP by high and low gamers controlling for
previous Portal play shows significant portion of variance is
most likely due to gaming in general rather than Portal.
Tests of Between-Subjects Effects
Measure:SSP
Transformed Variable:Average
Type III Sum of
Source Squares df Mean Square F Sig.
Intercept 183.460 1 183.460 92.447 .000
TIME_WEEK_BI 16.694 1 16.694 8.412 .006
PRVPORTL2_NUM .736 1 .736 .371 .546
Error 69.457 35 1.984

30. CANTAB Results
One Touch Stockings of Cambridge (OTS)
• Tests executive planning
• Critical component for strategic thinking
Analysis of the number of problems correct on the first try (OTS) by high/low
gamers shows strong significant difference between groups.
High game play experience scored higher significantly on both the pre and post-
tests (F= 10.335, p=.003).
Controlling for previous Portal 2 play the significance was still at the .006 level
suggesting that most of the variance occurred because of prior game play in
general and not due to previous Portalof2 play.
Tests Between-Subjects Effects
Measure:OTS
Transformed Variable:Average
Type III Sum of
Source Squares df Mean Square F Sig.
Intercept 10938.936 1 10938.936 2700.817 .000
TIME_WEEK_BI 41.859 1 41.859 10.335 .003
Error 149.859 37 4.050

31. CANTAB Results
Spatial Working Memory (SWM
• Tests the ability to retain spatial information
• manipulate remembered items in working memory
• It is a self-ordered task suggesting heuristic strategies
Analysis by high and low gamers shows large significant decrease in errors made by
high gamers (F=11,872, p=.001
Tests of Between-Subjects Effects
Measure:SWM
Transformed Variable:Average
Type III Sum of
Source Squares df Mean Square F Sig.
Intercept 228139.508 1 228139.508 109.646 .000
TIME_WEEK_BI 24702.892 1 24702.892 11.872 .001
Error 76985.287 37 2080.683

32. Differences by Areas of
Game Interest
Analysis of CANTAB by high/low gamers and previous Portal 2 play taking into account
areas of game interest shows four measures of interest.
• spatial working memory (SWM)
• latency or response for focused attention (RVML*)
• spatial span (SSP)
• execute planning (OTS)
Interactions
SWM (spatial working memory)- significant interaction effect over both trials between
interest in MMOG type games (F=8.965, p=.006).
RVPML - significant interaction for interest in Card games/Simple puzzles (F=5.622,
p=.026).
Between Subjects
Significant effect on SWM scores between interest in Racing games.
*latency measure on the focused attention task and is one of the most sensitive
measures on the CANTAB. It is a significant factor in the score differences of high/low
gamers and previous Portal 2 play.

33. Implications
Design
• Cognitive focused attention was enhanced in Portal 2
players verses MS Games players
• Potential cause is the open ended game play present in the
design of Portal 2.
Gaming
• High frequency gamers scored higher on cognitive tests than
low frequency gamers.
• Design characteristics presents in MMOGs, card games,
puzzles, racing, strategy, and sports games may lead to
these increases and should be explored.
Playing video games may actually lead to cognitive enhancement.

34. Further Research
• Cognitive Task Analysis of Portal 2
and others - present
• Further data collection on Portal
2/non-Portal 2 players and
CANTAB measures Fall 12/Spr 13
• Testing for metacognitive
monitoring on math for
adaptability performance
evidence and transfer tasks.
Fall12/Spr 13

35. Questions?
For more information contact:
shane.gallagher.ctr@adlnet.gov
703-575-3718