Assessing Procedural Competencies
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Assessing Procedural Competencies

on

  • 490 views

 

Statistics

Views

Total Views
490
Views on SlideShare
490
Embed Views
0

Actions

Likes
0
Downloads
3
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Assessing Procedural Competencies Presentation Transcript

  • 1. Assessing Procedural CompetenciesAmy S. Oxentenko, MD, FACP, FACG Associate Professor of MedicineFellowship Director, Mayo Clinic-Rochester
  • 2. Outline: Procedural Competency• The Numbers• The Skills• The Future
  • 3. Competent or Not?• 3rd year fellow • 3rd year fellow• 285 colonoscopies • 427 colonoscopies• Cecal intubation 95% • Cecal intubation 69%• Adenoma detection 35% • Adenoma detection 12%• Withdrawal 10 min • Withdrawal 6 min• Patient tolerance 97% • Patient tolerance 85% fair to excellent fair to excellent YES NO
  • 4. What Needs to be Measured? CompetencyMinimum DefinedProcedure level of Number ability
  • 5. The Numbers
  • 6. Minimum Cited Numbers• Flex sigs 25-30• EGDs 130• Colons 140 – Based on goal of cecal intubation > 90% – Others found competence at 275 cases1 – Others found competence at 500 cases2 1 Using Sedlack data 2 Using Spier data
  • 7. Cecal Intubation Rate Sedlack RE. Gastrointest Endosc 2011;74:355-66.
  • 8. Spier BJ, et al. Gastrointest Endosc 2010;71:319-24.
  • 9. Procedure Logs: Not Just Numbers Anymore! • “A skilled preceptor must be available to teach and supervise the fellows in the performance and interpretation of procedures, which must be documented in each fellows record, including indications, outcomes, diagnoses, and supervisor(s).”IV.A.6.d).(2) on Page 19, GI Program Requirements, “Tracked Changes” document
  • 10. The Skills
  • 11. Procedure Logging• “Assessment of procedural competence should include a formal evaluation process and NOT be based solely on a minimum number of procedures performed.• Each program must define criteria for competence for all required and elective procedures.• The record of evaluation must include the fellow’s logbook or an equivalent method to demonstrate that each fellow has achieved competence in the performance of required procedures.”V.A.1.a).(2) and 1.b).(1).(a) on Page 20-21, GI Requirements, “Tracked Changes” document
  • 12. Multi-Society Evaluation Form (MSEF)• AASLD, ACG, AGA, ASGE• Part of the GI Core Curriculum• Third Edition, May 2007http://www.asge.org/WorkArea/showcontent.aspx?id=3584
  • 13. Any Downfalls of the MSEF?• Lacks anchoring characteristics for all points• Not validated for continuous assessment• Grade inflation (our problem, not the form’s)• Compare graduates across programs?• What constitutes competent?
  • 14. Mayo Colonoscopy SkillsAssessment Tool (MCSAT) • 13-item survey • Staff completed on each colon • Took < 1 minute to complete • Embedded in MERGE database – Allowed for recording of procedure # for fellow, fellow name, etc. Sedlack RE. Gastrointest Endosc 2010;72:1125-33.
  • 15. Sedlack RE. Gastrointest Endosc 2010;72:1125-33.
  • 16. Overall Skill in Colonoscopy Sedlack RE. Gastrointest Endosc 2011;74:355-66.
  • 17. Other DateRetrieved Per Trainee
  • 18. Barriers of the MCSAT or Similar Systems?• Many procedures performed – Assessment needs to be quick/simple• Differing procedures performed – Similar models needed: EGD, PEGs, capsules, etc• Compliance with completion – Too easy for staff to forget or not take the time• Differing endoscopy database systems – No communication across programs
  • 19. Sunrise: ProVation October 2012Sunset: MERGEOctober 2012
  • 20. ProVation Tracking: EGD
  • 21. ProVation Tracking: Colons
  • 22. Pros/Cons of ProVation• PROS:• Compliance with completion – Automatic pop-up on all fellow EGDs and colons – Staff cannot sign off until complete• CONS:• Yet another database – How long with it be around? – Not everyone has it – Dependent on others to add features desired• Detail desired – Has to fit into radio buttons, brief, succinct
  • 23. The Future
  • 24. The Game Has Changed in the Setting of NAS • No longer a numbers game • No longer a competency yes/no game • Now it is all about meeting milestones on the way to becoming competenthttp://www.acgme-nas.org/assets/pdf/NEJMfinal.pdf
  • 25. A Blueprint for Milestones or Competency? Unacceptable Competent IdealAdenoma >20%Detection RateColonoscopy > 6 minWithdrawal TimeCecal Intubation > 95%RateComplication Rate < 1/200 bleed <1/1000 perfPolyp Retrieval > 95% > 10 mmRate > 80% < 10 mmPatient Tolerance > 90% fair to excellent
  • 26. A Blueprint for Milestones or Competency? Unacceptable Competent IdealAdenoma > 20% >20%Detection RateColonoscopy 7-15 min > 6 minWithdrawal TimeCecal Intubation > 90% > 95%RateComplication Rate < 1/200 bleed <1/1000 perfPolyp Retrieval > 95% > 10 mmRate > 80% < 10 mmPatient Tolerance > 90% fair to excellent
  • 27. A Blueprint for Milestones or Competency? Unacceptable Competent IdealAdenoma < 15% > 20% >20%Detection RateColonoscopy > 20 min 7-15 min > 6 minWithdrawal TimeCecal Intubation < 80% > 90% > 95%RateComplication Rate < 1/200 bleed <1/1000 perfPolyp Retrieval > 95% > 10 mmRate > 80% < 10 mmPatient Tolerance > 90% fair to excellent
  • 28. Milestones and Narratives
  • 29. Summary• Procedure numbers are not enough – An anchor at which competency assessment should begin• Procedure details are now needed – Indication, findings, complications• Competency tracking is required – Milestones will pave the way, and they need to be carefully developed
  • 30. Thank you!oxentenko.amy@mayo.edu