1. Eye Tracking Data
Gazeplot of fixation and gaze duration
Landmark (top) vs. Cardinal Directions (bottom)
Japanese Video
Navigation Task
Map Tracing
Dog Identification Task
English Video
Navigation Task
Dog Map Task
NASA-TLX-Post Task
Spatial Skills
Individual Difference Surveys &
Demographics
Results and Discussion
Preliminary results show that most participants could complete the task. However
39% required at least one hint. Overall the navigation setting (Japan, US) did not
influence navigation success.
There were strong differences on the TLX from time 1 before the game and Time 2
after the game. Participants overall scored higher on all TLX dimensions on Time 2
showing that the game was challenging.
Interestingly, the influence of the environment on the TLX depended on gender. For
example women seemed to feel more rushed, especially in the US environment.
(see Figure 1)
Setting also impacted men and women differently on their perception of learning
and progressing in the game. Men felt more self-efficacy in the US version than the
Japanese. Women felt more self-efficacy in the Japanese version.
The eye tracking data have not been fully analyzed but preliminary analysis shows
that the two strategies are easily recognized (see above).
Participants also frequently mentioned the two navigation strategies in their
surveys.
 “I looked at the signs and people. I also looked at the cars and the direction people
were walking.”(female)
 “Remembering the direction of the turns with numbers assigned to the turns.”(male)
 “Paid attention to surroundings such as trees, names of signs, street vendors.”(male)
The Current Study
Spatial thinking involves the ability to imagine and manipulate shapes or situations in the
mind. A meta-analysis by Voyer, Voyer, & Bryden, (1995) asserted that spatial skills have 3
separable dimensions.
• Spatial perception: the ability to determine spatial relationships while ignoring
distracting information.
• Spatial visualization: the ability to manipulate complex spatial information when
several stages are needed to produce the correct solution.
• Mental rotation: the ability to mentally manipulate 2 or 3-dimensional objects in
order to imagine them in a different perspective or orientation.
Wayfinding is the ability to understand one’s place in the environment and to create
mental maps that allow one to navigate the environment.
• Overall men and women are similar in map reading and wayfinding/navigation, but
research suggests they tend to use a different balance of strategies.
• Males favor navigational strategies that involve distal and directional information,
while women favor land mark oriented navigation (Mueller, Jackson, Skelton, 2008).
• Women sustain landmark gaze over time, while men’s gaze decreases over time
(Anderson, Dahmani, Konishi & Bohbot, 2012).
Research Questions:
• Separability of dimensions: Navigation in a 3-D game environment should not
correlate with mental rotation but may be predicted by sense of direction scales.
• Strategy Differences: Women may look more at landmarks and men may look more at
navigational turns.
• Impact of Stress: When navigating in the Japanese environment participants may have
more stressed and make more mistakes on navigation.
References
Blasko, D.G., Holliday-Darr, K., Mace, D., & Blasko-Drabik, H. (2004). VIZ: The visualization assessment and
training website. Behavior Research Methods Instruments & Computers, 36(2), 256-260.
Blasko-Drabik, H., Blasko, D., Lum, H., Erdem, B., Ohashi, M (2013) Investigating the Impact of Self-Efficacy in
Learning Disaster Strategies in an On-Line Serious Game. Proceedings of the Human Factors and ergonomics
society.
Hart, S. G., & Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): Results of empirical and
theoretical research. Advances in psychology,52, 139-183.
Hegarty, M., Richardson, A.E., Montello, D.R., Lovelace, K., & Subbiah, I. (2002). Development of a self-report
measure of environmental spatial ability. Intelligence, 30, 425-447.
Piaget J., & Inhelder, B. (1956). The child’s conception of space. London: Routledge & Kegan Paul.
Voyer, D., Voyer, S., & Bryden, M. P. (1995). Magnitude of sex differences in spatial abilities: A meta-analysis and
consideration of critical variables. Psychological Bulletin, 117, 250-270
Acknowledgements
Many thanks to our entire VIZ team and to Dr. Robert Light at the Office of
Undergraduate Research at Penn State Behrend for funding our work.
Abstract
Navigational skills are important in many situations throughout life. From navigating your
way through an unknown location to finding your car in a parking lot, navigation can be
useful anywhere. Navigation is just one of the many types of spatial skills that we use
every day, and it may be highly correlated with other types such as mental rotation and
spatial working memory. This study was designed to test the effects of a foreign
environment on participants’ navigation abilities and spatial skills. It also looked at their
level of concentration on the primary task with the implementation of a secondary
attention task. This was assessed by using a video game that allowed participants to
navigate a foreign city while attempting to locate certain objects along the way. The
attentional task and the strategies used during the study were analyzed using an eye
tracker program. Participant’s eye movements were tracked as they navigated the virtual
world; this gives us a sensitive measure of critical elements of on-line navigation. The goal
of this design was to elevate task difficulty and evaluate mental, physical, and temporal
capabilities using the NASA TLX survey. Following the navigation task, the participants
then completed spatial tasks on the VIZ website. The tasks included mental rotation of
blocks, and a water level task.
Modifications to Grand Theft Auto V
To create an interactive high fidelity 3D open world setting, we used the RockStar game
engine to modify GTA5 to create a believable foreign city setting.
High resolution textures and realistic environmental effects allow players to fully explore
and interact with the world. For purposes of the study we added boundaries around a
roughly 4 block area. Boundary items included crowds of people around a musician, fences,
utility trucks or decorative planters. They served as a reference point for researches to warn
the participants they needed to turn back if they continued.
261 props, pedestrians, and vehicles were added to the basic game map in order to create a
more realistic foreign or US city feel.
For the Japanese map, an additional 26 vehicle types were changed so that randomly
spawning vehicles would add to the feeling of immersion.
The game has a very rich auditory environment (including the sounds of traffic, people
walking or talking, birds and other animals). Custom auditory soundtracks were added (4 to
each condition and lasting 22 minutes total). These soundtracks were overlaid on top of the
game SFX sounds. Both sets of soundtracks included extra traffic and voices appropriate to
either the Japanese or US version of the game.
Demographics
• 38 undergraduate students (19 females, 19 males);
• M Age 21.1 years
• 65% white, 9% international
• 13 fluent in at least two languages.
• 12% traveled outside of the US.
• Males had more video game experience than
females.
• 26 participants completed the task without hints
and 17 received hints
NASA Total Load Index (TLX)
NASA TLX(Hart & Staveland, 1988)
Subjective, multidimensional
assessment tool to rate perceived
workload on six subscales:
• Mental demand
• Physical demand
• Temporal demand
• Effort
• Frustration
• Performance
Surveys
Video Game Experience, Past Travel, & Demographic Survey
• Demographics: age, ethnicity, gender, languages spoken
• Game Experience: Type of games played, average time playing games each week, previous Grand Theft Auto
experience
• Travel: Previous countries visited, previous travel to Japan
Santa Barbara Sense of Direction Scale (Hegarty, Richardson, Montello, Lovelace & Subbiah, 2002)
• Self-report measure of environmental spatial ability.
• “I can usually remember a new route after I have traveled it only once.”
• “I very easily get lost in a new city.”
Serious Games Experience Measure (Blasko-Drabik, Blasko, Lum, & Ohashi, 2013)
• 42-item questionnaire: 3 important dimensions of serious games
• Affective-focuses on the emotions, enjoyment, and immersion
• “I became unaware of my surroundings while playing the game.”
• Learning- focuses on the player’s perceived self-efficacy, perceived learning and motivation
• “Playing this game, made me think about other ways I could apply what I learned in it.”
• Game Mechanics- concerns the usability and playability of the game itself
• “The goals of the game were presented clearly.”
Spatial Experience Survey (Short V2)
• “How easily you can you construct a mental map of a city”
Spatial Skills
(Blasko, Holliday-Darr, Mace, & Blasko-Drabik, 2004)
• Mental Rotation:
(Vandenburg & Kuse, 1978)
• Water Level (Spatial Perception):
(Piaget & Inhelder, 1956)
Calibration & NASA-TLX
Baseline
Figure 1. NASA-TLX
0
5
10
15
20
25
30
35
40
Japan USA
DifferenceTime2–Time1
Temporal Dimension
male female
Figure 1 shows the difference between Time 1 and Time 2 on how rushed they felt.
Females showed more of an increase than males and this was greater in the
English environment.
Figure 2.Learning Self-Efficacy
Setting (Japan, United States) X Gender
1
1.5
2
2.5
3
3.5
4
4.5
5
Japan USA
Ratinglearningselfefficacy
Learning Self Efficacy
Male Female
There was a significant interaction between the setting (Japan vs US) and the gender
of the participant, F( 1,48) = 11.33, p < .01. Females had more self-efficacy in the
Japanese environment and less than males in the US environment.
Procedure