The different uses of simulation:
From education to product testing
Adam Dubrowski, PhD
Learning and Research Institutes
W...
www.wordle.net
Chaos in the Brickyard
Forscher, BK. Chaos in the Brickyard.
Science. 1963 Oct 18; 142(3590): 339.
Applying the Scientific Method and
underpinning research methodology to
generate evidence that will be used to build
simul...
• Models of Medical Education
• Introduction to Simulation
• Working Model
• Components of Simulation
– Simulators
– Educa...
Apprenticeship Model
Trainer
Trainee
PatientCare
Shift in responsibility
Ashley, 2000
Apprenticeship Model
Trainer
Trainee
PatientCare
Gap
Ashley, 2000
Apprenticeship Model
Trainer
Trainee
PatientCare
More people die in a given year as a result of medical
errors ( 44,000) t...
Apprenticeship Model
Trainer
Trainee
PatientCare
Simulation
Simulation Based Education
Learn faster
Remember longer
Be safer
The replication of a task or an
event for the purpose of
education and/or assessment.
Simulation Based Education
Simulation Based Education
Working
model
Dubrowski, A., Brydges, R., Satterthwaite, L., Xeroulis, G., Classe, R.
American Journal of Surgery. in press.
Busy Enviro...
Technical
Performance
Communication
Environment
Clinical
Knowledge
Attentional capacity
threshold
Additional
Learning
Simu...
Simulation Based Education
Technical
Performance
Communication
Environment
Clinical
Knowledge
Additional
Learning
Attentio...
Simulation Based Education
Technical
Performance
Communication
Environment
Clinical
Knowledge
Additional
Learning
Attentio...
Simulation Based Education
Technical
Performance
Communication
Environment
Clinical
Knowledge
Additional
Learning
Attentio...
Simulation Based Education
Technical
Performance
Communication
Environment
Clinical
Knowledge
Additional
Learning
Attentio...
Simulation Based Education
Evidence
Kurahashi et al. Surgery. 2010
Study 1: Does it work?
Can simulation based training help in
pre-training?
Practice: Trained
Skills
Test
Written
Test
1 week rest
Study 1: Does it work?
No practice: Novice
Skills Test: Expert Opinion
0
5
10
15
20
25
Written Test: Score
0
5
10
15
20
25
Trained
Novice
Novice
Trained
Novice
Novic...
simulation performanceimproves
Study 1: Does it work?
learningenables
Study 1: Does it work?
simulation
Simulation Based Education
How
to use it?
Simulation Based Education
Components
Simulators Educators Organization
Simulators
Information Flow Framework
Brydges et al., Am J Surg. 2007, Sidhu et al., J Vasc Surg. 2007
Study 2: Realism
Brydges et al., Am J Surg. 2007, Sidhu et al., J Vasc Surg. 2007
High fidelity
Animal model
Low fidelity
Study 2: Realism
Brydges et al., Am J Surg. 2007, Sidhu et al., J Vasc Surg. 2007
ExpertOpinion
0
5
10
15
20
25
Senior
0
5...
Study 2: Realism
simulation appropriate modelsrequires
Study 3: Educators
Errors in learning
Xeroulis et al., Surgery. 2007
Study 3: Educators
Xeroulis et al., Surgery. 2007
ExpertOpinion
0
5
10
15
20
25
Post-training Retention
1 month
Pre-traini...
Study 3: Educators
simulation educated educatorsrequires
Organization
Moulton et al., Ann Surg. 2007
Optimal schedules
Study 4: Organization
Moulton et al., Ann Surg. 2007
1 2 3 4
1 hour
block
1 month
1 2 3 4
1 hour block
1 week
rest
1 month...
Study 4: Organization
Moulton et al., Ann Surg. 2007
distributed
massed
ExpertOpinion
30
20
10
0
Pre-test
Microsurgical dr...
Study 4: Organization
simulation effective curricularequires
Summary of Results
simulation appropriate modelsrequires
simulation educated educatorsrequires
simulation effective curric...
Implications
Trainer
Trainee
PatientCare
Medical Education = Evidence Based Education
Simulation for Assessment
Framework
Preference
For learning
Of learning
Testing
Framework
Results
Behavior
Learning
Reaction
Kirkpatrick
Simulation for Assessment
Results
Behavior
Learning
Reaction
Kirkpatrick
Simulation for Assessment
Results
Behavior
Learning
Reaction
Kirkpatrick
Does
Shows How
Knows How
Knows
Miller
Simulation for Assessment
Results
Behavior
Learning
Reaction
Kirkpatrick
Does
Shows How
Knows How
Knows
Miller
Moore
Simulation for Assessment
Assessment
of what?
Assessment of Performance
Performance
Assessment of Performance
PGY 1-5 Gen Surgery residents
Knot tying: surface (easy) and depth (difficult)
Assessed with expert-based (GRS) and comput...
T
T
T
T T T
T T T
T
400
200
100
0
5
4
3
2
1
0
T T
T
T T
T T T TT
1000
800
600
400
200
0
T
T
T
T T
Surgical Volume
Expert-b...
simulation valid assessmentsrequires
Brydges et al., J Am Coll Surg. 2008
Assessment of Performance
simulation realistic contextprovides
Brydges et al., J Am Coll Surg. 2008
Assessment of Performance
Porte et al., AJS. 2007
Assessment for Learning
Computer
Practice
Pre
Test
Retention
Test
1 week restComputer + Crit
Expert
Post
Test
Porte et al., AJS. 2007
Assessment f...
T T
T
T
T T
T
T T
Pre Post Retention
ExpertOpinion
20
10
0
Computer
Computer + Crit
Expert
Porte et al., AJS. 2007
Assessm...
Porte et al., AJS. 2007
assessments for learning: choices
Assessment for Learning
Assessment of Learning
Learning
Building Skills, Changing
Practice: Simulator Training
for Hand Hygiene Protocols
Canadian Institutes of Health Research
P...
Purpose
• Current models of
knowledge delivery about
proper hand hygiene rely
on didactic session
• Transfer to practice i...
PurposeAssessment of Learning
Results
Behavior
Learning
Reaction
Does
Shows How
Knows How
Knows
PurposeAssessment of Learning
Results
Behavior
Learning
Reaction
Does
Shows How
Knows How
Knows
PurposeAssessment of Learning
Results
Behavior
Learning
Reaction
Does
Shows How
Knows How
Knows
Recruitment
Clinical monitoring
(audits)
Clinical monitoring
(audits)
Δ in Behavior
Δ in
Performance
Reaction
Education
(S...
• Simulation Design Scale
• Educational Practices Questionnaire
PurposeAssessment of Learning
Frequency
100
50
0
Disagree ...
Pre-test Post-test
Hand Hygiene
Compliance
(average of all
opportunities; all 4
moments)
62%
(n=558
opportunities)
64%
(n=...
98
24
51
9292
44
48
93
0
10
20
30
40
50
60
70
80
90
100
Bef-Pat/Env Bef-Asp Aft-Bfl Aft-Pat/Env
PreTest (n=558)
Post Test ...
• No transfer of
knowledge
• No changes in
performance, no
learning
• Ineffective education
and not transfer to
practice
P...
Assessment for Testing
Product and
Process testing
Patient
handoffs
Assessment for Testing
Patient
handoffs
Checklist
Assessment for Testing
Is it an effective tool?
Does it disturb the flow?
What is the frequency of use?
What is the most optimal format?
Assessme...
Assessment for Testing
Porte et al., AJS. 2007
simulation with process testinghelps
Assessment for Testing
Summary
Summary
Best available
evidence
Ongoing
assessment
Educator’s
experience
• Simulators
• Educators
• Schedule
Summary
Best available
evidence
Ongoing
assessment
Educator’s
experience
• Simulators
• Educators
• Schedule
• Performance...
Summary
Best available
evidence
Ongoing
assessment
Educator’s
experience
• Simulators
• Educators
• Schedule
• Performance...
End
Upcoming SlideShare
Loading in...5
×

Dubrowski draft

520

Published on

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
520
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Being trained as a kinesiologyst and neuroscientist the general aim of my research is to apply the scientific method and the underpinning research methodology to generate evidence which can then be used to build simulation augmented education programs
  • I will do this by first introducing the past med education models that are rooted in the principles of apprenticeshipNext I will introduce SimulationAfter that I will present samples of research that look at the optimization of various components of Simulation. Specifically, I will look at evidence that can help us decide what simulators to use, how we should train the teachers or educators, and how the courses which utilize simulation should be organized.Finally I will talk about a few possible future direction
  • For that I will use anon-going study that looked at the use of simulation for the adherence of hand hygiene protocols
  • What we normally want to know is wehther our programs impact learning of skills, which then lead to change in behavour, which can be measured in the clinical setting. This is what is known as transfer to practice. However,
  • Typically what we assess is whether the learning happened based on what the learners know. When we don’t see transfer to practice, we cannot say whether this failure is due to ineffective education or transfer to practice mechanisms. Using simualtion ….
  • … for assessments ofcompetnacy and proficiency at a skill can help us in the understanding of the low transfer to practice phenomenon.
  • In this study …..
  • Dubrowski draft

    1. 1. The different uses of simulation: From education to product testing Adam Dubrowski, PhD Learning and Research Institutes Wilson Centre Department of Paediatrics, UofT
    2. 2. www.wordle.net
    3. 3. Chaos in the Brickyard Forscher, BK. Chaos in the Brickyard. Science. 1963 Oct 18; 142(3590): 339.
    4. 4. Applying the Scientific Method and underpinning research methodology to generate evidence that will be used to build simulation based education programs. General Aim
    5. 5. • Models of Medical Education • Introduction to Simulation • Working Model • Components of Simulation – Simulators – Educators – Organization • Future Directions Outline
    6. 6. Apprenticeship Model Trainer Trainee PatientCare Shift in responsibility Ashley, 2000
    7. 7. Apprenticeship Model Trainer Trainee PatientCare Gap Ashley, 2000
    8. 8. Apprenticeship Model Trainer Trainee PatientCare More people die in a given year as a result of medical errors ( 44,000) than from motor vehicle accidents (43,458), breast cancer (42,297), or AIDS (16,516). "To Err is Human, Building a Safety Health System" (2000).
    9. 9. Apprenticeship Model Trainer Trainee PatientCare Simulation
    10. 10. Simulation Based Education Learn faster Remember longer Be safer
    11. 11. The replication of a task or an event for the purpose of education and/or assessment. Simulation Based Education
    12. 12. Simulation Based Education Working model
    13. 13. Dubrowski, A., Brydges, R., Satterthwaite, L., Xeroulis, G., Classe, R. American Journal of Surgery. in press. Busy Environment: Multitasking Simulation Based Education
    14. 14. Technical Performance Communication Environment Clinical Knowledge Attentional capacity threshold Additional Learning Simulation Based Education
    15. 15. Simulation Based Education Technical Performance Communication Environment Clinical Knowledge Additional Learning Attentional capacity threshold
    16. 16. Simulation Based Education Technical Performance Communication Environment Clinical Knowledge Additional Learning Attentional capacity threshold
    17. 17. Simulation Based Education Technical Performance Communication Environment Clinical Knowledge Additional Learning Attentional capacity threshold
    18. 18. Simulation Based Education Technical Performance Communication Environment Clinical Knowledge Additional Learning Attentional capacity threshold
    19. 19. Simulation Based Education Evidence
    20. 20. Kurahashi et al. Surgery. 2010 Study 1: Does it work? Can simulation based training help in pre-training?
    21. 21. Practice: Trained Skills Test Written Test 1 week rest Study 1: Does it work? No practice: Novice
    22. 22. Skills Test: Expert Opinion 0 5 10 15 20 25 Written Test: Score 0 5 10 15 20 25 Trained Novice Novice Trained Novice Novice Study 1: Does it work?
    23. 23. simulation performanceimproves Study 1: Does it work?
    24. 24. learningenables Study 1: Does it work? simulation
    25. 25. Simulation Based Education How to use it?
    26. 26. Simulation Based Education Components Simulators Educators Organization
    27. 27. Simulators Information Flow Framework Brydges et al., Am J Surg. 2007, Sidhu et al., J Vasc Surg. 2007
    28. 28. Study 2: Realism Brydges et al., Am J Surg. 2007, Sidhu et al., J Vasc Surg. 2007 High fidelity Animal model Low fidelity
    29. 29. Study 2: Realism Brydges et al., Am J Surg. 2007, Sidhu et al., J Vasc Surg. 2007 ExpertOpinion 0 5 10 15 20 25 Senior 0 5 10 15 20 25 Novice Synthetic Cadaver ExpertOpinion Pre-training baseline Synthetic Cadaver
    30. 30. Study 2: Realism simulation appropriate modelsrequires
    31. 31. Study 3: Educators Errors in learning Xeroulis et al., Surgery. 2007
    32. 32. Study 3: Educators Xeroulis et al., Surgery. 2007 ExpertOpinion 0 5 10 15 20 25 Post-training Retention 1 month Pre-training baseline Errors allowed Errors not allowed Independent learning
    33. 33. Study 3: Educators simulation educated educatorsrequires
    34. 34. Organization Moulton et al., Ann Surg. 2007 Optimal schedules
    35. 35. Study 4: Organization Moulton et al., Ann Surg. 2007 1 2 3 4 1 hour block 1 month 1 2 3 4 1 hour block 1 week rest 1 month distributed practice massed practice 1 2 3 41 2 3 41 2 3 41 2 3 41 2 3 4 T T Transfer
    36. 36. Study 4: Organization Moulton et al., Ann Surg. 2007 distributed massed ExpertOpinion 30 20 10 0 Pre-test Microsurgical drill Post-test Rat Transfer
    37. 37. Study 4: Organization simulation effective curricularequires
    38. 38. Summary of Results simulation appropriate modelsrequires simulation educated educatorsrequires simulation effective curricularequires
    39. 39. Implications Trainer Trainee PatientCare Medical Education = Evidence Based Education
    40. 40. Simulation for Assessment Framework Preference For learning Of learning Testing
    41. 41. Framework Results Behavior Learning Reaction Kirkpatrick
    42. 42. Simulation for Assessment Results Behavior Learning Reaction Kirkpatrick
    43. 43. Simulation for Assessment Results Behavior Learning Reaction Kirkpatrick Does Shows How Knows How Knows Miller
    44. 44. Simulation for Assessment Results Behavior Learning Reaction Kirkpatrick Does Shows How Knows How Knows Miller Moore
    45. 45. Simulation for Assessment Assessment of what?
    46. 46. Assessment of Performance Performance
    47. 47. Assessment of Performance
    48. 48. PGY 1-5 Gen Surgery residents Knot tying: surface (easy) and depth (difficult) Assessed with expert-based (GRS) and computer- based (motion analysis device) assessments Brydges et al., J Am Coll Surg. 2008 Assessment of Performance
    49. 49. T T T T T T T T T T 400 200 100 0 5 4 3 2 1 0 T T T T T T T T TT 1000 800 600 400 200 0 T T T T T Surgical Volume Expert-basedComputer-based 1 2 3 4 5 Depth Surface Brydges et al., J Am Coll Surg. 2008 Assessment of Performance
    50. 50. simulation valid assessmentsrequires Brydges et al., J Am Coll Surg. 2008 Assessment of Performance
    51. 51. simulation realistic contextprovides Brydges et al., J Am Coll Surg. 2008 Assessment of Performance
    52. 52. Porte et al., AJS. 2007 Assessment for Learning
    53. 53. Computer Practice Pre Test Retention Test 1 week restComputer + Crit Expert Post Test Porte et al., AJS. 2007 Assessment for Learning
    54. 54. T T T T T T T T T Pre Post Retention ExpertOpinion 20 10 0 Computer Computer + Crit Expert Porte et al., AJS. 2007 Assessment for Learning
    55. 55. Porte et al., AJS. 2007 assessments for learning: choices Assessment for Learning
    56. 56. Assessment of Learning Learning
    57. 57. Building Skills, Changing Practice: Simulator Training for Hand Hygiene Protocols Canadian Institutes of Health Research Partnerships for Health System Improvement A. McGeer, MA. Beduz, A. Dubrowski Assessment of Learning
    58. 58. Purpose • Current models of knowledge delivery about proper hand hygiene rely on didactic session • Transfer to practice is low Assessment of Learning
    59. 59. PurposeAssessment of Learning Results Behavior Learning Reaction Does Shows How Knows How Knows
    60. 60. PurposeAssessment of Learning Results Behavior Learning Reaction Does Shows How Knows How Knows
    61. 61. PurposeAssessment of Learning Results Behavior Learning Reaction Does Shows How Knows How Knows
    62. 62. Recruitment Clinical monitoring (audits) Clinical monitoring (audits) Δ in Behavior Δ in Performance Reaction Education (Sim/traditional) PurposeAssessment of Learning Sim Sim
    63. 63. • Simulation Design Scale • Educational Practices Questionnaire PurposeAssessment of Learning Frequency 100 50 0 Disagree Strongly Agree 90 66
    64. 64. Pre-test Post-test Hand Hygiene Compliance (average of all opportunities; all 4 moments) 62% (n=558 opportunities) 64% (n=550 opportunities) Average Adequacy (Excellent/ Satisfactory/ Unsatisfactory) Satisfactory (n=252 observations) Satisfactory (n=282 observations) Average Duration 13 seconds (n=254 observations) 13 seconds (n=252 observations) # Extra Washes 140 90 PurposeAssessment of Learning
    65. 65. 98 24 51 9292 44 48 93 0 10 20 30 40 50 60 70 80 90 100 Bef-Pat/Env Bef-Asp Aft-Bfl Aft-Pat/Env PreTest (n=558) Post Test (n=550) Moment 1 Moment 2 Moment 3 Moment 4 PurposeAssessment of Learning *
    66. 66. • No transfer of knowledge • No changes in performance, no learning • Ineffective education and not transfer to practice PurposeAssessment of Learning
    67. 67. Assessment for Testing Product and Process testing
    68. 68. Patient handoffs Assessment for Testing
    69. 69. Patient handoffs Checklist Assessment for Testing
    70. 70. Is it an effective tool? Does it disturb the flow? What is the frequency of use? What is the most optimal format? Assessment for Testing
    71. 71. Assessment for Testing
    72. 72. Porte et al., AJS. 2007 simulation with process testinghelps Assessment for Testing
    73. 73. Summary
    74. 74. Summary Best available evidence Ongoing assessment Educator’s experience • Simulators • Educators • Schedule
    75. 75. Summary Best available evidence Ongoing assessment Educator’s experience • Simulators • Educators • Schedule • Performance • For learning • Of learning • Other uses
    76. 76. Summary Best available evidence Ongoing assessment Educator’s experience • Simulators • Educators • Schedule • Performance • For learning • Of learning • Other uses Simulation may not be the answer
    77. 77. End
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×