TRACK 3(4) | QUALITY IMPROVEMENT | DAY 1 - 31 OCT 2016 Mary Dankbaar, Program manager e-learning | Erasmus University Medical Center (NL) Games for Health Europe 2016

1. The effects of a simulation game on skills
and motivation of doctors and students:
the expertise-reversal effect in medical practice
Games for Health 2016
Mary Dankbaar PhD,
Jelmer Alsma MD, Jan van Saase PhD, Stephanie Schuit PhD,
Erasmus University Medical Center Rotterdam,
Jeroen van Merrienboer PhD, Maastricht University
The Netherlands.

2. Background: training complex cognitive skills
 Simulation games offer challenging & engaging learning tasks.
 Games are becoming increasingly popular in medical education.
 More insight in the effectiveness of simulation games for skills training and
their critical design features is needed (1,2, 3).
 Erasmus University MC developed a simulation game for residents to train
complex cognitive emergency care skills.
(1)Clark, Mayer E-Learning and the science of instruction, 2008.
(2) Akl , Pretorius, Sackett et al. BEME Guide No 14,.2010.
(3) Garris, Ahlers, Driskell, Games, Motivation and Learning: a research practice model, 2002, Dec, 33 (4).

3. Background (2)
 A previous study with residents showed positive effects of the simulation
game on residents’ cognitive emergency care skills and engagement (4).
 For novice learners, high-fidelity cases may cause ‘cognitive overload’ (5) .
 Highly realistic tasks with feedback are engaging, but expensive.
Research question:
Are cognitive skills and motivation of medical students more
improved by adding a simulation game or text-based cases
to an instructional e-module on emergency care?
(4) Dankbaar et al, in Games for Health 2014, Schouten, Fedtke, Schijven, Vosmeer, Gekker (eds)
(5) Van Merrienboer, Kirschner, 2012

4. Simulation game (high fidelity cases)
- Authentic tasks (6 patients cases)
- High level of interactivity & feedbackabcdeSIM on emergency care

5. Text-based cases (low fidelity)
 Ee-module-m
Using the same 6 cases

6. Motivation questionnaire
Control group, n=16
E-module
Game group, n=25 Cases group, n=20
Questionnaire
Knowledge test
24 items
Open Cases
program
Questionnaire
Questionnaires
Text based cases (low fidelity)
Game-based cases (high fidelity)
Cognitive load e-module Cognitive load e-module Cognitive load e-module
Knowledge test Knowledge test Knowledge test
E-module E-module E- module
Cognitive loadCognitive load
Motivation questionnaireMotivation questionnaire
Skill assessment
(2 cases)
Mannequin-based
assessment
Mannequin-based
assessment
Mannequin-based
assessment
Study-designRandomized study-design
E-module on
emergency
care
Text-based cases (low fidelity)

7. Mannequin-based skills assesment
Clinical competency rating scale (6 items, scale 1-7),
‘uses ABCDE method on treatment’
Communication competency scale (3 items, scale 1-7),
‘communicates with patient effectively’
Global performance scale (single scale 1-10)
‘independent function in caring for acutely ill patient’
The Assessment instrument was validated in a separate study showed good validity and
moderate interrater reliability (1)
(1) Dankbaar et al, Assessing the Assessment in Emergency Care Training. Plos-One, Dec. ‘14.
2 scenario-based assessments with blinded assessors;
on diabetic coma and urosepsis with delirium (15 min.)

8. Questionnaires
Cognitive Load (10 items, scale 0-11)
 Intrinsic load ‘the content of the game/cases was very complex’ (3 items)
 Extraneus load ‘the explanations were very unclear’ (3 items)
 Germane load ‘the game/cases really enhanced my understanding’ (4 items)
Evaluation (9 items, scale 1-5)
 Engagement ‘it was fun to work through the material’ (6 items)
 Feedback ‘during learning, I could tell whether I was doing well’ (3 items)

9. Results (n= 61)
 No differences between groups in age, clinical experience, GPA bachelor
 No differences in knowledge scores between groups (M= 20, SD=3.7)
Self-reported learning time in minutes, mean (SD) Logged
game time
E-module Game or Cases Total
Control group 133 (78) - 133
Game group 146 (98) 95 (53) 241 83 (54)
Cases group 177 (118) 70 (46) 247

10. Performance in Clinical skills
5.4 5.2 5.6
1
2
3
4
5
6
7
game group cases group control group
P = 0.13 N=61

11. Cognitive load for game- and text-based cases
6.0
3.3
8.4
5.0
3.4
7.2
0.0
1.0
2.0
3.0
4.0
5.0
6.0
7.0
8.0
9.0
10.0
Intrinsic Load Extraneous Load Germane Load
Game groep
Cases groep
P=0.03
ES=0.8
P=0.01
ES=1.0
P=0.75
N=54

12. Motivation

4.3
3.4
3.5
3.1
3.6
2.7
1
1.5
2
2.5
3
3.5
4
4.5
5
Engagement Feedback
Game group
Cases group
Control group
P < 0.001*, ES =1.4 P < 0.05**, ES =1.0
* Game vs cases+ control
** Game vs. control
N=59

13. Conclusions
 For initial skills acquisition, an e-module with a video demonstration is a
powerful instructional format.
 Novice students do not profit from working on additional open cases, which
appeared to be too complex, but nonetheless challenged them to study longer.
 High fidelity game-based cases enhance motivation and cognitive effort, but
also increases complexity and may impede learning for novices.
 The same game improved clinical skills among residents, with limited self-
study time.
 ‘Expertise-reversal effect’:
A rich learning environment may benefit experts, but is counter-productive for
novice learners (interaction prior knowledge - complexity of formats)

14. Take-home message
 Serious games often are engaging, but careful alignment of
case complexity and fidelity in games with students’ proficiency
level is essential.
 Complexity in cases may be reduced by offering guidance
(hints, worked cases) and gradual increase in fidelity
 More research is needed on the relation between case-
complexity, case-fidelity, motivation and skills development.

15. Questions?
 Contact information
 m.dankbaar@erasmusmc.nl
 Mary Dankbaar
 Erasmus University Medical Center Rotterdam