My poster presentation at the Old Dominion University Graduate Research Appreciation Day 2014 based on my first graduate research project. Participants were given 2 scenarios about different training scenarios - one about traditional PowerPoint lecture training and the other about playing an interactive video game as part of training. Reactions to these scenarios were measured and analyzed. Attitudes towards video games and experience with video games were measured as moderators of the effect of training design on reactions.

1. Reactions to Gamified Training
Michael B. Armstrong & Richard N. Landers, Ph.D.
Department of Psychology
Abstract
This study examined the impact of gamification on anticipated reactions to
workplace training, providing a preliminary evaluation of gamified training
design. The moderating effects of experience with games and technology
and attitudes towards the use of games and technology in training were
explored as well. By exploring the effects of gamified training design on
anticipated reactions, evidence can be gathered to support future research
regarding the effects of gamified training on workplace learning, behavioral
change, and organizational results through a full-scale gamified training
system.
Literature Review
Gamification is the use of game elements in non-game contexts in order to
motivate users to behave a certain way (Deterding, Khaled, Nacke, & Dixon,
2011). Examples include:
• Gamifying exercise: Award points to people for losing weight and display
points on a scoreboard so people can see where they stand in relation to
others in a friendly competition.
• Gamifying school: Utilize a fantasy narrative to exchange taking tests for
slaying dragons and exchange projects for epic quests.
• Gamifying work: Distribute badges or ribbons for completing different
training modules. Race to collect them all!
This study explored how effective gamification could be in a potential
workplace training scenario.
• Training design effectiveness can be evaluated by measuring reactions to
training, which impacts learning, behavioral change, and organizational
outcomes (Kirkpatrick, 1996).
• More experience with a technology or system (e.g. video game) means
people have to expend fewer cognitive resources using it than someone
new to that system who is distracted by learning the system (Landers &
Callan, 2012; Sweller, 1988).
• Individual differences (e.g. attitudes) among people differentiate how
each person will react (Kraiger, 2008).
Method and Hypotheses
• 150 participants
• Participants received two different vignettes: The control vignette described a
typical training scenario via lecture and Microsoft PowerPoint. The experimental
vignette described the same training content but via an interactive video game
instead of the lecture and presentation.
• Participants completed surveys measuring reactions to the training scenarios, as well
as experience with technology/games and their attitudes towards technology/games
in the workplace.
Hypothesis 1. Gamified training design will affect anticipated reactions to training.
Hypothesis 2. Experience with games and technology will moderate the relationship
between training design and anticipated reactions. That is, the relationship between
training design and training reactions will be stronger when trainees have more
experience with games and technology.
Hypothesis 3. Trainee attitudes toward the value of games and technology used for
training will moderate the relationship between training design and anticipated
reactions. That is, the relationship between training design and training reactions will be
stronger when trainees have more positive attitudes toward games and technology.
Conclusions
Figure 1. The present model.
Figure 3. Hypothesis 3: Condition x Attitude Interaction.
• Hypothesis 1 was not supported. Overall, anticipated gamified training did not
significantly affect reactions compared to anticipated traditional training.
• Hypotheses 2 and 3 were supported. With higher ratings of experience with
games and attitudes towards using games in work, reactions increased from
anticipated traditional training to anticipated gamified training.
• This study is limited in its generalizability to actual work training designs. The
results reached in this study are based on possible training scenarios instead of
actual training sessions. Future research should measure reactions and learning
outcomes based on gamified training.
• This study found that gamification in work training deserves further study. For
trainees high in experience with games and high in attitudes towards the use of
games in work, gamified training can produce higher reaction ratings, and
possibly learning, behavioral, and organizational outcomes.
References
Deterding, S., Khaled, R., Nacke, L. E., & Dixon, D.
(2011, May 7-12, 2011). Gamification: Toward a
definition. Paper presented at the 2011 Annual
Conference on Human Factors in Computing
Systems (CHI '11), Vancouver, BC, Canada.
Kirkpatrick, D. (1996). Great Ideas Revisited.
Techniques for Evaluating Training Programs.
Revisiting Kirkpatrick's Four-Level Model. Training
and Development, 50(1), 54-59.
Kraiger, K. (2008). Transforming our models of
learning and development: Web-based instruction
as enabler of third-generation instruction.
Industrial and Organizational Psychology, 1(4),
454-467. doi: 10.1111/j.1754-9434.2008.00086.x
Landers, R. N., & Callan, R. C. (2012). Training
evaluation in virtual worlds: Development of a
model. Journal of Virtual Worlds Research,
5(3), 1-20.
Sweller, J. (1988). Cognitive load during problem
solving: Effects on learning. Cognitive
Science, 12(2), 257-285.
Analyses and Results
Hypothesis 1. To test this hypothesis, we regressed training reactions onto
a dummy-coded indicator of condition. Training design condition did not
significantly affect reactions (b = .105, SE = .117, p = .370, R2 = .081).
Hypotheses 2 & 3. To test these hypotheses, we conducted two separate
hierarchical multiple regressions. In Step 1, we regressed reactions onto
condition and moderator variable. In Step 2, we added in the interaction of
condition and moderator. Results can be found in Tables 1 and 2 and are
illustrated in Figures 2 & 3.
Table 2. Regression Results of Hypothesis 3.
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Traditional Gamification
ReactionstoTraining
Low Experience
High Experience
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Traditional Gamification
ReactionstoTraining
Low Attitude
High Attitude
Figure 2. Hypothesis 2: Condition x Experience Interaction.
Table 1. Regression Results of Hypothesis 2.