IPSSW Competency Prediction
•Download as PPTX, PDF•
1 like•377 views
Dr. March Auerbach presents data on the ability of the JIT simulation evaluation and SBME evaluation to predict success.
Recommended
More Related Content
What's hot
What's hot (19)
Similar to IPSSW Competency Prediction
Similar to IPSSW Competency Prediction (20)
More from Todd
More from Todd (9)
Recently uploaded
Recently uploaded (20)
IPSSW Competency Prediction
- 1. Simulation-Based Just-In-Time Competency Assessment Predicts Interns Clinical Infant Lumbar Puncture Performance Marc Auerbach, MD, MSc Assistant Professor of Clinical Pediatrics Associate Director of Pediatric Simulation Yale University School of Medicine, New Haven CT, USA Co-director of the POISE research network Marc Auerbach, Todd Chang, Daniel Fein, James Gerard, Renuka Mehta, Daniel Scherzer, Jennifer Reid, Glenda Rabe, Martin Pusic, David Kessleron behalf of the POISE Investigators
- 3. Conflict of Interest On behalf of myself (and any co-presenters of the papers I am responsible for), I declare to my knowledge, there are no conflicts of interest The POISE Network is funded by a grant by
- 4. Background Competence is a developmental process: for each domain/context there is a spectrum of ability from novice to mastery Gained through deliberate practice and reflection Each individual proceeds at a different rate Simulation training/assessment can facilitate skills development to the level required for safe practice
- 6. There are few objective methods for supervisors to assess trainees procedural skills prior to clinical performance
- 8. Unsupervised Unconscious competence Independence Conscious competence Conscious Incompetence Watching Unconscious Incompetence Dreyfus, Maslow, Erricson
- 11. Construct validation in prior studyPrompt = a verbal interjection to either prevent or correct an error
- 12. Validation 60 subjects 20 beginner < 5 LP (medical students) 20 intermediate 10-20 LP (residents) 20 expert > 50 LP (faculty/fellows) Reliability Overall agreement = ICC = 0.71, 95% CI 0.59 – 0.80(p =0.000) Positive correlation for all paired rater comparisons (0.69 – 0.73, p = 0.000) Discriminant validity GSA tool could reliably discriminate between the 3 groups Experts scored the highest, followed respectively by the intermediate and beginner groups (p < 0.05 for all post hoc comparisons)
- 13. Methods 2009-2010 cohort SBME 2010-2011 cohort SBME Orientation Time = 0
- 14. Methods 2009-2010 cohort SBME 2010-2011 cohort SBME Orientation Time = 0
- 15. Methods 2009-2010 cohort SBME LP Clinical encounter #1 JIT 2010-2011 cohort SBME LP Clinical encounter #1 Orientation Time = 0
- 16. Methods 2009-2010 cohort SBME LP Clinical encounter #1 LP#2,3… JIT JIT 2010-2011 cohort SBME* LP#2,3… LP Clinical encounter #1 Orientation Time = 0 0 to 6 months
- 17. Methods 2009-2010 cohort SBME LP Clinical encounter #1 LP#2,3… JIT JIT 2010-2011 cohort SBME* LP#2,3… LP Clinical encounter #1 Orientation Time = 0 0 to 6 months
- 18. Methods 2009-2010 cohort SBME LP Clinical encounter #1 LP#2,3… JIT JIT 2010-2011 cohort SBME LP#2,3… LP Clinical encounter #1 Orientation Time = 0 0 to 6 months
- 19. Results 2009-2010 (SBME) 51 interns report 102 LPs 104 interns enrolled 2010-2011 (SBME + JIT) 501 interns enrolled 161 interns report 228 LPs
- 20. Results 45% (46/102) 45% (102/228) (55%) (41%)
- 21. Results Odds ratio for success if rated “high” on simulator = 1.74 (95%CI 1.01-3.00), p=0.045 (pearson chi square)
- 23. Assessors not blinded to clinical outcome
Editor's Notes
- Congress center PierreBaudis. 10 minute talk. 5 minute Q&ALook at notes on each slide.
- 35 hospitalsPoitiers??
- Using deliberate practice- define?1990 to 2010
- Endpoint or minimum passing scoreUsing deliberate practice1990 to 2010
- Australian Medical AssociationAssociation for Medical Education in EuropeCanada- CanMEDSUnited States- ACGME outcomes projectUnited Kingdom- Royal Colleges Performance ProceduresNetherlandsScotland- The Scottish Doctor
- Progression of competenceFor each aspect or domain of competence, the spectrum of ability from novice to mastery. The goal of medical education is to facilitate the development of a physician to the level of ability required for optimal practice in each domain. At any given point in time, and in a given context, an individual physician will reflect greater or lesser ability in each domain.Deliberate practice- expertise requires 10,000 hours of practice, 3 hours a day for 10 years. (anderserricson)Skill development depends on quality and quantityDREYFUS MODEL OF SKILLS PERFORMANCENovice- No experience, Passive “hands-off” exposureAdvance Beginner- Some “hands-on” experience, Marginally acceptable performance, barely knows stepsCompetent- Significant hands on experience, perceives procedure as chopped up parts, lacks speed, efficiency and flexibility but achieve masteryProficient- perceives whole procedure instead of thinking out each part, can start to trouble shoot and reflect5. Expert- lots of experience, intuitive performance, “it feels right/easy” does not think about performing, Novice to expert in 10,000 hours or 10 yearsMaslow psychology professor of Brandeis until 1970 death20th century4 Stages of learningFirst the learner does not know what they don’t know. And needs to be instructed in this phase.Once they are aware of their mistakes they pass into conscious incompetence.At this stage it takes practice until they become competent in a skillAnd after much practice they stop thinking about it and can deal with higher level tasks as the cerebellum takes over the skill
- CHANGE TEARDROP(that occur before clinical LP attempts)
- Success= atraumatic CSF <1000 rbcs/hpfINTERNS with almost no prior infant LP procedure experienceAnd low skill levels
- videotaped from two angles performing on a simulatorThree blinded expert ratersSpearman’s rank correlation coefficients werecalculated to assess the correlation between individual pairs of raters. To assess discriminant validity,group scores were compared using the Kruskal-Wallis nonparametric comparison of means. For post-hocanalyses, Bonferroni corrected Mann-Whitney U pair-wise comparisons were made.
- Describe SBME using checklist for mastery (add pic?) instructors trained with web based module.All watched audiovisual presentation
- 501 interns completed initial mastery training228 clinical infant lumbar punctures reportedMention interns suck normally (45%)Odds ratio for clinical procedural success if rated high on simulator = 1.74 (95%CI 1.01-3.00), p=0.045 (pearson chi square)