Assessing Procedural Competencies


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Assessing Procedural Competencies

  1. 1. Procedural Assessment: Where do we stand? March 2013 Walter J. Coyle MD, FACG, FASGE
  2. 2. Objectives The apprentice model – The way we were Competency based education – Where we are now Outcomes based learning – Procedural focus – Milestone development
  3. 3. Apprenticeship Successfulfor thousands of years Key properties: – Good mentor – Motivated student – Adequate exposure AND hands on time Problems: – Consistency – Objective measures of success – Low ceiling for promotion The Mystery of Mastery. Psychology Today 1986;20:32
  4. 4. Apprenticeship Works well for very sub-specialized areas and few centers of excellence Still model for advanced endoscopy – AEF match – Variation in level of exposure and mastery – Who monitors the mentors? – How do the graduates do? Medicine resents outside monitoring – Better if we did it ourselves
  5. 5. Competency based training ACGME initiative from the 1990s – Applied to all aspects of training – Knowledge, professionalism, procedures Ineffective for procedural training – GI procedures still in apprentice model – Little consensus on assessment and outcomes – Little data to define milestones  How should a 2nd year fellow scope? Lurie. Med Educ 2012;46:1365
  6. 6. Competency based training Diverse training methods and assessment techniques Small programs vs large; research fellows Silo mentality: no consistent standard Explains why we have this problem now
  7. 7. Outcomes based learning More process oriented Focus on the process not the problem ACGME wants us to move here Starting point: 1st year fellow Ending point: Staff GI Milestone development: easier for knowledge core vs procedures N Engl J Med 2012;3686:1051-56
  8. 8. Milestone development Final milestone: Colonoscopy – >95% cecal intubation rate – >25% ADR – Low complication – Patient satisfaction Stepwise milestones: None with great data or evidence – 1st year vs 2nd vs 3rd Gastrointest Endosc 2010;71:319-24
  9. 9. Procedural Education: initial focus on process Intense didactic – FYF course, DVDs, local resources Intense hands on training with scope – ? Simulators – Training box/tool – Standardized patient – Example of pilot training?
  10. 10. Procedural Education:subsequent focus on process Ongoing, continuous assessment: – Mentor feedback; patient feedback – Objective outcomes based assessment tool – Universal tool ? Development of outcomes based, data driven milestones that apply throughout fellowship – How???
  11. 11. Procedural Education: A Proposal Universal assessment tool agreed upon Web-based submission of assessments – Collection and development of milestone – Feedback to fellow and program – Fellow compared to peers nationally Progressionthrough milestones will be fellow driven, not fixed year driven
  12. 12. Data on Fellow Sedlack, GIE 2010;72:1125-33
  13. 13. Procedural Education: A Proposal Requirements of system – Ease of use: minutes, APP for phone, link on desktop – Secure – Can provide data back to program and fellow in real time – Dynamic and progressive
  14. 14. Procedural Education: A Proposal Cost:GI programs, GI societies, ACGME Web site location and maintenance – ACGME – CORI database like initiative – GIQuik – Endoscopic report generating systems  Provation initiative with Mayo Clinic Time frame
  15. 15. ASGE Proposed Forms
  16. 16. Integrated Assessment Are we ready and committed?? Resource Commitment Staff Commitment Barriers breaking silos Only definite: change is here
  17. 17. Summary Prior models and procedural mentoring are probably inadequate Classic competency based assessment is flawed for procedures Outcomes and milestones are a next step GI directed development of milestones and tools is critical
  18. 18. SummaryNeed to think of the process NOT the problem