Does It Work?..and Why?
Research into the effectiveness of simulation as a
teaching tool
Vicki R LeBlanc, PhD
Wilson Centr...
SIMULATION
The representation or
reproduction of
something real
by imitation
 Physical objects (task trainers)
 Process or skill (suturing)
 People (standardized patients, confederates)
 Environm...
• Struggle - resources, access, time, funding, etc…
• Position papers & advocacy (Ziv, Gaba, Reznick, etc..)
• Climate of ...
• National/International Societies
• Simulation Conferences & Courses
• Accreditation Standards
2011: CLIMATE OF
LEGITIMACY
….Important role in acceptance
RESEARCH
• Robust acceptance by trainees
• Enhanced learning
• Transfer to clinical setting
RESEARCH EVIDENCE
Educators and students exposed to
mannequin-based simulation
85% of students & 85% of educators rated session as
excellen...
• Robust acceptance by trainees
• Enhanced learning
• Transfer to clinical setting
RESEARCH EVIDENCE
D Tabak, CA Moulton, A Birze,
H MacRae, D Nestel, R Kneebone, V LeBlanc
ENHANCED
LEARNING USING
HYBRID
SIMULATIONS
Specific Research Aims
To determine the effectiveness of the hybrid
simulation in teaching communication and
interpersonal...
Study Design
4th
Year
Medical
Students
N = 16
PGY1
Surgery
Residents
N = 16
Wound Closure
Drunk and Angry
Urinary
Catheter...
Outcome Measures
Validated communication scale (2 blinded raters)
Empathy
Response to patient’s feelings and needs
Coheren...
0
10
20
30
40
50
60
70
80
90
100
overall emphathy cohesion verbal non-verbal
Communication Scores: Pre Intervention
No Fee...
0
10
20
30
40
50
60
70
80
90
100
overall empathy cohesion verbal non-verbal
* *
Significant effect of feedback p<0.05
Comm...
Technical Scores: GRS
No Feedback
Feedback
%
Pre Post
0
10
20
30
40
50
60
70
80
90
100
catheter-pre wound-pre skin exc-pos...
0
10
20
30
40
50
60
70
80
90
100
Cath Wound Skin Cast
No Feedback
Feedback
Technical Scores: Checklist
%
Pre Post
IPPI can be used to deliver effective formative feedback
Feedback > Exposure
Improvements not at cost of non-intervention ...
The advantages of
simulation-based
education lead to
enhanced performance
and clinical care
TRANSFER TO CLINICAL
SETTING
ENHANCED LEARNING
20 anesthesiology residents; 2-h training session:
- High-fidelity simulation-based training
- Interacti...
Post Test Retention
Non technical
skills
Clinical
checklist
Non technical
skills
Clinical
checklist
Seminar
11.8 75.4 11.7...
TRANSFER TO CLINICAL
SETTING
Simulation-based learning of central line insertions
- Greater success
- Reduced complication...
 BUT…..Some evidence of lack of improvements:
 Olympio (2003):
 no improvement in management of intubation
 Borges (20...
Not possible to state that simulation, itself, IS or IS NOT effective
for learning
Important to understand what elements o...
It’s no longer a question of
IF
But a question of
HOW
FEEDBACK
IMPORTANT CHARACTERISTICS
Key element to learning and skill acquisition
 without debriefing, students fail to learn
Issenberg et al. (2005) Medical...
Medical students learning suturing and knot tying
skill
Control group: practice with no feedback
Concurrent: feedback as s...
Immediate post-test:
• Control < Concurrent = Summative
1-month retention test:
• Control = Concurrent < Summative
RESULTS
Continuous, intensive feedback inhibits learning:
• Over-guidance: learners overly depend on feedback
and perform poorly i...
PRACTICE
IMPORTANT CHARACTERISTICS
Podcast Study- In Progress
 How to optimize podcasts in health professions
education
• Mental practice:
-cognitive rehear...
Pre-Test (Multiple Choice)
Randomization
Podcast
No Mental Practice
No Modeling
(n=15)
Podcast
With Mental Practice
No Mod...
Preliminary results
 Data collection: mid-point, with 7-8 per group
 MCQ improvements:
Control < Mental Practice = Model...
Summary
Significant scholarship in simulation over 20 years
 - Learner reactions
 - Can lead to improved performance
 -...
“The field of patient simulation has gotten to
where it is now because it makes sense, but
we need more science behind it ...
Research into Simulation
Research into Simulation
Upcoming SlideShare
Loading in …5
×

Research into Simulation

284 views
242 views

Published on

Vicki LeBlanc PhD (University of Toronto) introduces us to her data on feedback and enhanced hybrid simulation learning on communication skills. Her expertise is in Human Factors and Decision-making particularly in stressed and non-ideal situations.

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
284
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Growth
  • Did not appear to be at the expense of technical skills
  • Research into Simulation

    1. 1. Does It Work?..and Why? Research into the effectiveness of simulation as a teaching tool Vicki R LeBlanc, PhD Wilson Centre; University of Toronto
    2. 2. SIMULATION The representation or reproduction of something real by imitation
    3. 3.  Physical objects (task trainers)  Process or skill (suturing)  People (standardized patients, confederates)  Environments (virtual worlds, immersive simulation) SIMULATION
    4. 4. • Struggle - resources, access, time, funding, etc… • Position papers & advocacy (Ziv, Gaba, Reznick, etc..) • Climate of scepticism 1980S-MID 2000S
    5. 5. • National/International Societies • Simulation Conferences & Courses • Accreditation Standards 2011: CLIMATE OF LEGITIMACY
    6. 6. ….Important role in acceptance RESEARCH
    7. 7. • Robust acceptance by trainees • Enhanced learning • Transfer to clinical setting RESEARCH EVIDENCE
    8. 8. Educators and students exposed to mannequin-based simulation 85% of students & 85% of educators rated session as excellent > 80% sim-based training should be required for all medical students Gordon et al., Acad Med 2001 ROBUST ACCEPTANCE
    9. 9. • Robust acceptance by trainees • Enhanced learning • Transfer to clinical setting RESEARCH EVIDENCE
    10. 10. D Tabak, CA Moulton, A Birze, H MacRae, D Nestel, R Kneebone, V LeBlanc ENHANCED LEARNING USING HYBRID SIMULATIONS
    11. 11. Specific Research Aims To determine the effectiveness of the hybrid simulation in teaching communication and interpersonal skills to medical students and surgical residents
    12. 12. Study Design 4th Year Medical Students N = 16 PGY1 Surgery Residents N = 16 Wound Closure Drunk and Angry Urinary Catheterization Uncomfortable Patient Feedback Volar Cast Irritated Skin Lesion Blind and Afraid Pre-Test Post -Test No Feedback
    13. 13. Outcome Measures Validated communication scale (2 blinded raters) Empathy Response to patient’s feelings and needs Coherence Plan and organization of the interview Verbal Expression Command of language Non Verbal Expression Engagement with non verbal expression Hodges B. Med Teach 2003; 25:250-254 Validated technical scales
    14. 14. 0 10 20 30 40 50 60 70 80 90 100 overall emphathy cohesion verbal non-verbal Communication Scores: Pre Intervention No Feedback Feedback % No main effect of feedback p=.752
    15. 15. 0 10 20 30 40 50 60 70 80 90 100 overall empathy cohesion verbal non-verbal * * Significant effect of feedback p<0.05 Communication Scores: Post Intervention * * * No Feedback Feedback %
    16. 16. Technical Scores: GRS No Feedback Feedback % Pre Post 0 10 20 30 40 50 60 70 80 90 100 catheter-pre wound-pre skin exc-post cast-post
    17. 17. 0 10 20 30 40 50 60 70 80 90 100 Cath Wound Skin Cast No Feedback Feedback Technical Scores: Checklist % Pre Post
    18. 18. IPPI can be used to deliver effective formative feedback Feedback > Exposure Improvements not at cost of non-intervention skill Conclusions
    19. 19. The advantages of simulation-based education lead to enhanced performance and clinical care TRANSFER TO CLINICAL SETTING
    20. 20. ENHANCED LEARNING 20 anesthesiology residents; 2-h training session: - High-fidelity simulation-based training - Interactive seminar Each trainee then weaned patients from CPB - within 2 weeks (posttest) - within 5 weeks (retention test) Bruppacher et al., Anesthesiology; 2010
    21. 21. Post Test Retention Non technical skills Clinical checklist Non technical skills Clinical checklist Seminar 11.8 75.4 11.7 77.0 Simulation 14.3 89.9 14.1 93.2
    22. 22. TRANSFER TO CLINICAL SETTING Simulation-based learning of central line insertions - Greater success - Reduced complications - Reduced infection rates Barsuk et al; Arch Intern Med; 2009
    23. 23.  BUT…..Some evidence of lack of improvements:  Olympio (2003):  no improvement in management of intubation  Borges (2010):  no changes in management of “cannot intubate, cannot ventilate” scenarios  Wenk (2009):  simulation not better than PBL for intubation
    24. 24. Not possible to state that simulation, itself, IS or IS NOT effective for learning Important to understand what elements of simulation facilitate learning
    25. 25. It’s no longer a question of IF
    26. 26. But a question of HOW
    27. 27. FEEDBACK IMPORTANT CHARACTERISTICS
    28. 28. Key element to learning and skill acquisition  without debriefing, students fail to learn Issenberg et al. (2005) Medical Teacher; Stefanidis et a.l (2007) Surgery Xeroulis et al. (2007) Surgery; Walsh et al. (2009) Academic Medicine FEEDBACK
    29. 29. Medical students learning suturing and knot tying skill Control group: practice with no feedback Concurrent: feedback as student performs the task Summary: feedback after completion of the task SUMMARY VS CONCURRENT FEEDBACK Xeroulis et al., 2006; Surgery
    30. 30. Immediate post-test: • Control < Concurrent = Summative 1-month retention test: • Control = Concurrent < Summative RESULTS
    31. 31. Continuous, intensive feedback inhibits learning: • Over-guidance: learners overly depend on feedback and perform poorly in it’s absence • Inhibits intrinsic learning strategies and problem solving activities necessary to master skills NOT JUST ANY FEEDBACK
    32. 32. PRACTICE IMPORTANT CHARACTERISTICS
    33. 33. Podcast Study- In Progress  How to optimize podcasts in health professions education • Mental practice: -cognitive rehearsal of the steps in a tasks without overt physical movement • Modeling: -Viewing of a demonstration of what needs to be learned Fahad Alam, Sylvain Boet, Vicki LeBlanc
    34. 34. Pre-Test (Multiple Choice) Randomization Podcast No Mental Practice No Modeling (n=15) Podcast With Mental Practice No Modeling (n=15) Podcast No Mental Practice With Modeling (n=15) Podcast With Mental Practice With Modeling (n=15) Post-Test (Multiple Choice) Basic Airway Management Simulation Scenario
    35. 35. Preliminary results  Data collection: mid-point, with 7-8 per group  MCQ improvements: Control < Mental Practice = Modeling < MP and Modeling
    36. 36. Summary Significant scholarship in simulation over 20 years  - Learner reactions  - Can lead to improved performance  - Learning transfers to clinical setting Important focus of current research… …understanding how to optimize simulation-based learning
    37. 37. “The field of patient simulation has gotten to where it is now because it makes sense, but we need more science behind it to guide its use.” Jim Gordon

    ×