Assessing Procedural       CompetenciesAmy S. Oxentenko, MD, FACP, FACG     Associate Professor of MedicineFellowship Dire...
Outline:   Procedural Competency• The Numbers• The Skills• The Future
Competent or Not?•   3rd year fellow         •   3rd year fellow•   285 colonoscopies       •   427 colonoscopies•   Cecal...
What Needs to be     Measured?        CompetencyMinimum              DefinedProcedure            level of Number          ...
The Numbers
Minimum Cited Numbers•   Flex sigs    25-30•   EGDs         130•   Colons       140    – Based on goal of cecal intubation...
Cecal Intubation Rate     Sedlack RE. Gastrointest Endosc 2011;74:355-66.
Spier BJ, et al. Gastrointest Endosc 2010;71:319-24.
Procedure Logs:   Not Just Numbers Anymore!  • “A skilled preceptor must be available to    teach and supervise the fellow...
The Skills
Procedure Logging• “Assessment of procedural competence should include  a formal evaluation process and NOT be based solel...
Multi-Society Evaluation Form            (MSEF)• AASLD, ACG, AGA, ASGE• Part of the GI Core Curriculum• Third Edition, May...
Any Downfalls of the MSEF?• Lacks anchoring characteristics for all points• Not validated for continuous assessment• Grade...
Mayo Colonoscopy SkillsAssessment Tool (MCSAT) •   13-item survey •   Staff completed on each colon •   Took < 1 minute to...
Sedlack RE. Gastrointest Endosc 2010;72:1125-33.
Overall Skill in Colonoscopy         Sedlack RE. Gastrointest Endosc 2011;74:355-66.
Other DateRetrieved Per   Trainee
Barriers of the MCSAT or      Similar Systems?• Many procedures performed  – Assessment needs to be quick/simple• Differin...
Sunrise: ProVation                October 2012Sunset: MERGEOctober 2012
ProVation Tracking: EGD
ProVation Tracking: Colons
Pros/Cons of ProVation• PROS:• Compliance with completion  – Automatic pop-up on all fellow EGDs and colons  – Staff canno...
The Future
The Game Has Changed in       the Setting of NAS • No longer a numbers game • No longer a competency yes/no game • Now it ...
A Blueprint for Milestones         or Competency?                    Unacceptable Competent   IdealAdenoma                ...
A Blueprint for Milestones         or Competency?                    Unacceptable Competent   IdealAdenoma                ...
A Blueprint for Milestones         or Competency?                    Unacceptable Competent   IdealAdenoma             < 1...
Milestones and Narratives
Summary• Procedure numbers are not enough  – An anchor at which competency    assessment should begin• Procedure details a...
Thank you!oxentenko.amy@mayo.edu
Assessing Procedural Competencies
Upcoming SlideShare
Loading in...5
×

Assessing Procedural Competencies

327

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
327
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Assessing Procedural Competencies

  1. 1. Assessing Procedural CompetenciesAmy S. Oxentenko, MD, FACP, FACG Associate Professor of MedicineFellowship Director, Mayo Clinic-Rochester
  2. 2. Outline: Procedural Competency• The Numbers• The Skills• The Future
  3. 3. Competent or Not?• 3rd year fellow • 3rd year fellow• 285 colonoscopies • 427 colonoscopies• Cecal intubation 95% • Cecal intubation 69%• Adenoma detection 35% • Adenoma detection 12%• Withdrawal 10 min • Withdrawal 6 min• Patient tolerance 97% • Patient tolerance 85% fair to excellent fair to excellent YES NO
  4. 4. What Needs to be Measured? CompetencyMinimum DefinedProcedure level of Number ability
  5. 5. The Numbers
  6. 6. Minimum Cited Numbers• Flex sigs 25-30• EGDs 130• Colons 140 – Based on goal of cecal intubation > 90% – Others found competence at 275 cases1 – Others found competence at 500 cases2 1 Using Sedlack data 2 Using Spier data
  7. 7. Cecal Intubation Rate Sedlack RE. Gastrointest Endosc 2011;74:355-66.
  8. 8. Spier BJ, et al. Gastrointest Endosc 2010;71:319-24.
  9. 9. Procedure Logs: Not Just Numbers Anymore! • “A skilled preceptor must be available to teach and supervise the fellows in the performance and interpretation of procedures, which must be documented in each fellows record, including indications, outcomes, diagnoses, and supervisor(s).”IV.A.6.d).(2) on Page 19, GI Program Requirements, “Tracked Changes” document
  10. 10. The Skills
  11. 11. Procedure Logging• “Assessment of procedural competence should include a formal evaluation process and NOT be based solely on a minimum number of procedures performed.• Each program must define criteria for competence for all required and elective procedures.• The record of evaluation must include the fellow’s logbook or an equivalent method to demonstrate that each fellow has achieved competence in the performance of required procedures.”V.A.1.a).(2) and 1.b).(1).(a) on Page 20-21, GI Requirements, “Tracked Changes” document
  12. 12. Multi-Society Evaluation Form (MSEF)• AASLD, ACG, AGA, ASGE• Part of the GI Core Curriculum• Third Edition, May 2007http://www.asge.org/WorkArea/showcontent.aspx?id=3584
  13. 13. Any Downfalls of the MSEF?• Lacks anchoring characteristics for all points• Not validated for continuous assessment• Grade inflation (our problem, not the form’s)• Compare graduates across programs?• What constitutes competent?
  14. 14. Mayo Colonoscopy SkillsAssessment Tool (MCSAT) • 13-item survey • Staff completed on each colon • Took < 1 minute to complete • Embedded in MERGE database – Allowed for recording of procedure # for fellow, fellow name, etc. Sedlack RE. Gastrointest Endosc 2010;72:1125-33.
  15. 15. Sedlack RE. Gastrointest Endosc 2010;72:1125-33.
  16. 16. Overall Skill in Colonoscopy Sedlack RE. Gastrointest Endosc 2011;74:355-66.
  17. 17. Other DateRetrieved Per Trainee
  18. 18. Barriers of the MCSAT or Similar Systems?• Many procedures performed – Assessment needs to be quick/simple• Differing procedures performed – Similar models needed: EGD, PEGs, capsules, etc• Compliance with completion – Too easy for staff to forget or not take the time• Differing endoscopy database systems – No communication across programs
  19. 19. Sunrise: ProVation October 2012Sunset: MERGEOctober 2012
  20. 20. ProVation Tracking: EGD
  21. 21. ProVation Tracking: Colons
  22. 22. Pros/Cons of ProVation• PROS:• Compliance with completion – Automatic pop-up on all fellow EGDs and colons – Staff cannot sign off until complete• CONS:• Yet another database – How long with it be around? – Not everyone has it – Dependent on others to add features desired• Detail desired – Has to fit into radio buttons, brief, succinct
  23. 23. The Future
  24. 24. The Game Has Changed in the Setting of NAS • No longer a numbers game • No longer a competency yes/no game • Now it is all about meeting milestones on the way to becoming competenthttp://www.acgme-nas.org/assets/pdf/NEJMfinal.pdf
  25. 25. A Blueprint for Milestones or Competency? Unacceptable Competent IdealAdenoma >20%Detection RateColonoscopy > 6 minWithdrawal TimeCecal Intubation > 95%RateComplication Rate < 1/200 bleed <1/1000 perfPolyp Retrieval > 95% > 10 mmRate > 80% < 10 mmPatient Tolerance > 90% fair to excellent
  26. 26. A Blueprint for Milestones or Competency? Unacceptable Competent IdealAdenoma > 20% >20%Detection RateColonoscopy 7-15 min > 6 minWithdrawal TimeCecal Intubation > 90% > 95%RateComplication Rate < 1/200 bleed <1/1000 perfPolyp Retrieval > 95% > 10 mmRate > 80% < 10 mmPatient Tolerance > 90% fair to excellent
  27. 27. A Blueprint for Milestones or Competency? Unacceptable Competent IdealAdenoma < 15% > 20% >20%Detection RateColonoscopy > 20 min 7-15 min > 6 minWithdrawal TimeCecal Intubation < 80% > 90% > 95%RateComplication Rate < 1/200 bleed <1/1000 perfPolyp Retrieval > 95% > 10 mmRate > 80% < 10 mmPatient Tolerance > 90% fair to excellent
  28. 28. Milestones and Narratives
  29. 29. Summary• Procedure numbers are not enough – An anchor at which competency assessment should begin• Procedure details are now needed – Indication, findings, complications• Competency tracking is required – Milestones will pave the way, and they need to be carefully developed
  30. 30. Thank you!oxentenko.amy@mayo.edu
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×