-Incomplete knowledge of indications-Complexity of a multi-step process-Lack of confidence, concern for treatment risksKnowledge of indications incomplete or misunderstoodPsychomotor sequence more difficult than anticipatedInadequate preparation for stress-induced hesitancy and discombobulation
Curriculum will be divided into 6 modules over an 18 month period, allowing for entire curriculum to be repeated x 1 during a 3 year training period.Deliberate practice: emphasize that simulation scenarios will be multidisciplinary (CC and GI) and comprehensive including procedures (endoscopy, sclerotherapy, biopsy with pathology interpretation…); including pediatric acute liver failure and acute on chronic liver failure-using a standardized set of questions
Format• Discuss project for 30 minutes – Presenter stays at table – Consultants rotate tables x 3• Transition for 5 minutes – Summarize state of consultation – Presenter highlights key questions
1. Chang- Script Concordance LP2. Barry- BVM training3. Kummett- Neonatal Skills4. Mehta- Health literacy5. Sherzer- Epi pen community6. Levy- PALS tool validation7. Maa- PALS performance tool8. Maa- Hybrid-simulator9. Meyer- Donation after Cardiac Death10.Overly- Structured-patient encounter
Multimedia-based ScriptConcordance Testing as a methodof testing Clinical Decision Making Skills Todd Chang, MD email@example.com
Background• Assessing Clinical Decision Making Skills (CDMS) is difficult but necessary• Script Concordance Testing (SCT) – Has been validated as a method of assessing trainees in many subspecialties – Is currently being studied for infant lumbar punctures (LP) to assess infant LP management
Background• Text-based questions and team-based simulations may not isolate individual’s CDMS – Multimedia questions have lower scores than text- based Holtzman KZ, Swanson DB, Ouyang W, Hussie K, Allbee K. Use of Multimedia on the Step 1 and Step 2 Clinical Knowledge Components of USMLE: A Controlled Trial of the Impact on Item Characteristics. Acad Med 2009; 84(10s): s90-s93
SCTA 1-month-old male has a rectal temperature of 40.3 Celsius. There is mild rhinorrhea.Does the following change your likelihood to perform an LP: He is RSV+ -2 Much less likely -1 Less likely 0 No change +1 More likely +2 Much more likely
PICO Question• Population – In pediatric residents, subspecialty fellows, and attendings• Intervention – Does the use of Multimedia depictions of clinical scenarios• Comparison – Compared with text-based depictions• Outcome – Affect SCT scores negatively?
Approach• Create 2 versions of every SCT question: – text-based case scenario – multimedia-based (screen-based simulation) case scenario Q1. An alert 2-week-old infant is RSV+ and has a respiratory rate of 70 and subcostal retractions Q1. 2-week-old RSV+
Approach• Optimize a 15-question SCT set to test 2 CDMS: – Infant Lumbar Puncture – Infant/Pediatric/Adolescent Intubation• Randomize multimedia vs. text-based & administer questions• Evaluate score differences against training status, (sub)specialty status, and self-reported experience
3 Questions• How do we create the optimal multimedia element – VR vs. true patients?• Should randomization be per question or per student?• Are there more optimal methods of validating the SCT question set? Todd Chang, MD firstname.lastname@example.org
Improving the Effectiveness of Bag and Mask Ventilation Training in an Academic Center NICU Jim Barry Medical Director, University of Colorado Hospital NICU
Background• BMV is a simple skill that is simply done wrong frequently• Simulation and Learning (and not forgetting) Theory – Retrieval based testing/learning improves short and long term memory- THE WAY WE TEST MATTERS – Partial task trainers and deliberate practice can improve skill attainment in trainees (LP)• Knowledge Gaps – Can the combination of deliberate practice and retrieval testing improve skill acquisition and retention for BMV ?
PICO Question• Population – Primary- 2011- 2012 Pediatric Residents from Univ Co – Secondary- other NICU staff: RNs, RTs, NNPs, MDs• Intervention – Formal training: deliberate practice, knowledge (retrieval vs recognition), spacing• Comparison – Subjects in 3 groups varied by testing: Recognition, Retrieval, Retrieval+Practice• Outcome – Changes in BMV Knowledge and Skill at 1 month and 6-12 months later
Approach• Randomized controlled study 1. Randomize teams monthly into 1 of 3 study groups 2. Knowledge evaluation with pre/post-test (Retrieval or Recognition) beginning/end of month and 6-12 months 3. Pretest, questionnaire to determine BMV experience and career choice 4. Evaluate BMV skill and equipment knowledge with video- recorded session using apneic neonatal partial task trainer 5. BMV scoring tool and mastery learning applied 6. Pre/PostData collected: Knowledge, BMV skill
3 Questions• What would be the best format/time to evaluate long(er) term retention? 3,6,9 months• Currently at single center, could this be replicated at other sites?• How could this education/intervention be applied to patient outcomes?
ALERT Presentation:Maintaining Proficiency in BagMask Ventilation, a two month recommendation Gary Kummet, Julie Lindower
Background• 60 second delay of effective ventilation in adults after cardiopulmonary arrest decreased survival by 9%.• Bag mask ventilation training – Improves skills immediately post training – Requires significant exposure to achieve proficiency – Skills decay rapidly with time (6-7 months)• Knowledge gaps – Does early establishment of ventilation improve pediatric outcomes? – What is the training interval to maintain proficiency?
PICO Question• Population – Pediatric and Family Practice residents at a tertiary medical center• Intervention – will receive bag mask ventilation training every two months• Comparison – compared to conventional training (NRP/PALS at orientation)• Outcome – Improved procedural skills competency six months after initial training
Approach• Randomized controlled study 1. Provide initial training to all residents to achieve baseline proficiency 2. Randomize residents into one of two study groups 3. Re-training at two month intervals to study group 4. Respiratory arrest scenario at 6 months (videotaped with objective mannequin feedback) 5. Feedback and debriefing
3 Questions• What are the technical requirements of the simulator for this study? – Taking the test to the tester/simulator portability – Obtaining objective data• What issues do we need to consider when standardizing the research across study sites? – Initial training, testing, re-training, re-testing• What are clinically significant differences in performance?
Appropriate Health Literacy Communication study Renuka Mehta
Background• Poor health literacy (HL) is associated with adverse patient outcomes, poor patient satisfaction and possible litigation.• This may be related, in part, to communication mismatches with providers and the healthcare system.• Thus we seek to improve upon this gap by developing a multi-modal, interprofessional communication training method that can be utilized to enhance providers’ communication skills.
P.I.C.O1. Participants will be health care students (medical, nursing, Pharmacy and allied health care providers).2. Online lecture focusing on HL and communication skills, interaction with the VP software, learning strategies for clear and sensitive ways to communicate with parents, and Interprofessional team training simulation workshop where subjects will communicate with a Standardized Patient (SP) acting as a parent .3. Pre and post intervention comparison of appropriate HL communication between control and intervention groups.4. Improvement in low health Literacy sensitive communication between pre and post intervention using standardized parent.
Approach Development of Multi stakeholder case scenarioInterdisciplin Health literacy ary team communicatio Online lecture training n curriculum Virtual parent software
Questions1. What is the best way to set up a interdisciplinary team training2. What are practical outcome3. What assessment tools will be helpful.
ALERT Presentation:Using an Epipen educational module to improve FoodAnaphylaxis Recognition and Response D.J. Scherzer
Background• Incorrect outpatient epinephrine-injector usage leads to preventable mortality among food allergy children in the U.S. – The devices are prescribed with inadequate teaching and follow-up.• Succinct educational modules improve competence when directed towards specific goals (eg. AED).• Competency Gaps – Incomplete knowledge of indications – Complexity of a multi-step process – Lack of confidence, concern for treatment risks
PICO Question• Population – Parents, patients, school personnel, healthcare staff – Prescribers• Intervention – Succinct educational module comprised of MCQ, practicum and video• Comparison – Historical control – before and after• Outcome – Improve ability to know when and how to use epinephrine injector and to be prepared to actually do it. – Improve ability to teach others to do above.
Approach• Prospective longitudinal study; learner outcomes • Subjects serve as their own controls. • Pre- assessment survey of subjects: • Experience and confidence questions • Knowledge and case management questions • 1:1 practical performance assessment. • Post- assessment: • Review practicum and repeat until success with individuals • Group presentation of edu-video c Q&A. • Immediate resurvey of individuals with confidence and case management questions • Follow-up with confidence question and case management questions in 3-6 months. Mock scenario on site?
3 Questions• Do the case management questions and practicum get at the active ingredients of food anaphylaxis competency?• How can we make the practicum feel more realistic?• Can we roll this out in a way that is easier logistically? More eLearning? Mock scenarios on site rather than 1:1 practicum.
Development and validation of a generalizable tool to assess pediatric resuscitationsYasaman Shayan and Arielle Levy Pediatric Emergency Department Sainte-Justine Hospital, Montreal, Canada
Background• Resuscitation of a pediatric patient presents many challenges• Competency in pediatric resuscitation skills gained by: – Formal training (PALS) Rapid decline in skills – Experience Rarity of pediatric cardiopulmonary arrest• These points highlight the importance of simulation as an essential teaching tool – Essential to have a valid and reliable assessment tool
Objectives• To develop a short, objective, easy to use and generalizable scoring tool to assess trainees during simulated pediatric resuscitations scenarios – To determine its validity – To analyse its inter-rater reliability
Tool development Content Validity IdentificationAHA 2010 of specific Review of objective scoring items PALS elements for by subject-curriculum each domain matter experts (C-A-B)
Tool validation• Convenience sample of residents rotating through ped ED First week During the rotation Last week 5 video-taped 12-16 ER shifts 5 video-taped simulated + simulated resuscitation Simulation-based resuscitation scenarios courses scenarios - Asystole/PEA Scored - Arrhythmias Scored - Status epilepticus/ asthmaticus - Shock
Discussion• How to assure content validity• Type of scoring system – Checklist vs. GRS• Inclusion of crisis resource management skills – Communication, leadership…
ALERT presentation: Code team leader assessment tool andcorrelation of event leader performance with team performanceTensing Maa, Ada Lin, Samantha Gee, Aaron Calhoun
Background• Pediatric code blue events are rare = poor experiential learning for trainees• Code team leader simulation training may be helpful.• Knowledge gap/needs – Generalizable scoring tool to assess code team leader competency – Does code leader performance correlate with team performance and event outcome?
PICO question• Population 1. Pediatric healthcare providers (APNs) or trainees (residents, PICU, NICU, EM and anesthesia fellows) who are expected to act as code team leaders. 2. Pediatric healthcare providers who are potential code team members.• Intervention – Evaluation of event leaders’ and code teams’ performance during standardized simulated pediatric codes using our scoring tools.• Comparison – Performance of event leader with that of the whole team• Outcome – Assess interrater reliability and validity of team leader scoring tool – Correlate event leader performance with team performance and event outcome
ApproachProspective observational pilot study1. Fine tune content and determine initial psychometricbenchmarks of event leader scoring tool2. Perform simulations and collect data on event leadersand team performance – Scenarios will be standardized in terms of outcome to assure reproducibility of results – Separate raters will be used for the teamwork and leadership tools to minimize potential biases – Rating will be done based on live and video taped performance – Score results will be statistically compared with each other as well as with the final outcomes of the session to look for correlations
Questions1. Suggestions on the content (domains or behavioral anchors) of the team leader rating tool?2. Can you accurately separate team leader performance from team performance when you have other “experts” (ex. more experienced fellows) on the team?3. What about bias from inexperience with or anxiety from simulation?
Hybrid-learning: a model for a comprehensive curriculum incorporating online self-directed modules and augmented by high fidelity patient simulations Samantha Gee, Ada Lin, and Tensing Maa Nationwide Children’s Hospital, The Ohio State University Section of Pediatric Critical Care
Background• “Traditional pathway” – Didactic learning + patient exposure• Management of pediatric acute and chronic liver failure, including the liver transplant patient, is complex: – Acutely decompensated – Multiorgan system dysfunction – Life-threatening sequelae• Knowledge gap: – Firsthand experience is limited to liver transplant centers – High acuity level requires accurate decision-making to successfully manage this rare patient population – Competency and comfort level of fellows trained by the traditional pathway may not be optimized
PICO QuestionPopulation: Pediatric ICU and GI fellow traineesIntervention: Comprehensive liver failure andtransplantation hybrid-learning curriculum: ◦ Core reading materials for background preparation ◦ Online self-learning modules in didactic form ◦ High fidelity simulation with immediate debriefing sessions for learning recap and feedbackComparison: Trainees taught by the traditional pathwayOutcome: Improved mastery of learning and ease oftransfer to practice for those who participate in the hybridcurriculum as compared to traditional
ApproachDesign: Prospective pilot study involving ICU and GI fellows Revolving curriculum: 6 modules over 18 months Online self-directed learning, including core readings and quizzes A series of interdisciplinary, comprehensive simulations Deliberate practice: Each scenario involves a patient in an acute situation, providing the fellow an opportunity to: Practice critical-thinking skills Manage the patient accurately in a safe setting Tie-in core concepts attained from online learning Evaluation: Measure knowledge base pre-/post-participation in the hybrid-learning curriculum Systematically scored based on the six core competencies outlined by the ABP
QuestionsHow to detect a true effect? Number of fellows Account for test-taking abilityIs there a more accurate way to assess performance? Knowledge base Clinical acumen Comfort level
ALERT Presentation: Donation after Cardiac Death (DCD): Improvingconsent to donate, compliance with institutional protocols and organ procurement Elaine C. Meyer, Ph.D., R.N. Kristen Nelson, M.D. Elizabeth Hunt, M.D., Ph.D.
Background• Patients awaiting organ transplantation far exceed availability of healthy organs• The Institute of Medicine has advocated for Donation after Cardiac Death (DCD) protocols in hospitals to expand the potential donor pool• Staff knowledge and experience with DCD is limited and may negatively impact the implementation of protocols• A simulation-based educational film is available (focusing on ethics & family-staff conversations)
PICO Question• Population – Healthcare and organ procurement providers• Intervention – DCD educational film• Comparison – Each participating site will be compared to itself pre and post educational intervention; aggregate comparison across educational conditions• Outcome – Knowledge, attitudes, sense of preparation, confidence, communicative ability, compliance with established DCD protocols, use & evaluation of film, frequency of family-OPO staff meetings to initiate and discuss organ donation, frequency of consent to donate, frequency of organ procurement
Approach• Survey to all INSPIRE sites to determine absence/presence of DCD, current educational approach and educational needs• Among those with DCD, pre-post comparison after educational intervention with educational arms (film to hospital educators, film to OPO educators, partnership and film to hospital & OPO)• Among those without DCD, pre-post comparison between film and no film• Follow natural use, acceptability and efficacy of film as per outcome measures
3 Questions• How many INSPIRE sites have DCD protocols? If no, what is the likelihood that they would be willing or able to institute DCD protocols in the near future?• Outcomes such as frequency of family-OPO meetings, consent to donate, and organ procurement require partnership with regional OPOs. How realistic is this?• What aspect(s) of this proposal would be possible as part of INSPIRE simulation centers and is external funding needed?
The Structured Patient Encounter: Improving the pediatricpatient/family experience using a structure approach to clinical interactions Frank Overly, Linda Brown, Adam Rojek, Linda Dykstra, Lynn Sweeney
Background• Excellence in healthcare is no longer defined merely by the quality of clinical care offered, but also by the superiority of service provided to those who seek care.• With increasing competition between health care systems, administrators have increased their focus on service and patient satisfaction as a barometer of how successful the institution is at providing a high quality patient experience.• Low satisfaction scores are significantly associated with malpractice activity (United States).• New 5 Step Structured Patient Encounter (SPE) – Incorporated into a larger communication initiative at Rhode Island Hospital designed to optimize the patient experience• Knowledge Gaps – despite a plethora of communication improvement initiatives, there remains a paucity of published data objectively evaluating these programs/interventions
PICO Question• Population – Residents caring for pediatric patients in the outpatient setting• Intervention – Exposure to, or training in the Structured Patient Encounter (SPE)• Comparison – compare baseline performance with performance after exposure to, or training in the Structured Patient Encounter (SPE)• Outcomes – Families’ perception of the experience with the care provider – Families’ perception of the overall clinical experience
Approach• Randomized controlled study 1. Randomize groups into one of 4 study arms 1. Routine education, no exposure to SPE 2. Routine education and SPE cognitive aid 3. Routine education Sim enhanced education on SPE 4. Routine education Sim education and cognitive aid for SPE 2. Baseline scores for all individuals (parents’/patient’s feedback from clinical encounters, sim observation score) 3. Intervention (cognitive aid, sim, sim + cognitive aid) 4. Follow up scores for all individuals (parents’/patient’s feedback from clinical encounters, sim observation score)
Study Design Standardized Patient Encounter(SPE) Baseline measurement of individuals’ performances 1)Actual family/patient feedback 2)SPE Checklist evaluation in simulated case Group 1 Group 2 Group 3 Group 4 Given sim Given sim Routine Given SPE enhanced SPE enhanced SPEEducation cognitive aid training no training and cognitive aid cognitive aid Follow-up measurement of individuals’ performances 1)Actual family/patient feedback 2)Checklist evaluation in simulated case
3 Questions• Is it necessary to gather information in the simulated setting?• What confounding issues might we encounter? How to overcome them? (ED issues: waiting times, nursing issues, residents will have normal progression of skills, acuity level of patient)• How many encounters would you need to record for each individual?