The Value of Competency-based Medical Education Across the Continuum


Published on

"The Value of Competency-based Medical Education Across the Continuum." Workshop presented by Dr. Eric S. Holmboe at Memorial University's Faculty of Medicine.

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • A paradigm shift! This is not tinkering around the edges!
  • In the CBME approach, medical education is organized around “competencies”, and not just scientific medical knowledge as in our traditional approach.
  • In this way of thinking, our definition of competence is changed. It is multidimensional and contextual. This also leads to the relative terms “dyscompetence” and “supracompetence”.
  • I specifically have a box around the 18 month milestone in preparation for the next slide…
  • 1.2       Initiating the interview Pre-Clerkship 1 Pre-Clerkship 2 MD 1 MD 2 Junior Resident 1 Junior Resident 2 Senior residency (entry to practice) 1 Senior residency (entry to practice) 2 Advanced Practice 1 Advanced Practice 2                       a)   Greets the patient, and caregivers, and confirms how s/he/they would like to be addressed 3. Demonstrates fully, but may be rule-based in application. 2. Simulation/Role play/Standardized patient 4. Demonstrates proficiency and efficiency and is able to adjust to the specific patient or context 3. Routine clinical 4. Demonstrates proficiency and efficiency and is able to adjust to the specific patient or context 5. Across the spectrum of professional practice 4. Demonstrates proficiency and efficiency and is able to adjust to the specific patient or context 5. Across the spectrum of professional practice 5. Demonstrates innovation, improvisation and deliberate use of best practice 5. Across the spectrum of professional practice
  • This example uses a narrative stream for patient care and demonstrates how narrative milestones could be used in a FasTrack type system. You will notice the language under the narrative stream asks for a judgment about the trainee’s trajectory. Given that there are 23 narrative streams which equates to about 3 – 4 for each general competency, you could envision a trainee progressing at different rates for those streams. Asking for a summative judgment allows a program to identify that a trainee may be lagging in some areas, but is still felt overall to be developing appropriately.
  • This would address the criticism of being reductionistic
  • Landmarks are a little different than EPA’s because they occur during training … Why use them to assess competence? They are a sampling of events that … Mention how the Reed Williams article supports this concept of focused assessment of a particular skill/set of skills allows for a generalized statement of competency for that skill
  • Here is how this may actually develop for any given learner. This does introduce the concept that competency development is not limited to the small window of formal training. One challenge that CBME will need to address is what will be the minimum needed to successfully complete residency or fellowship training?
  • The Value of Competency-based Medical Education Across the Continuum

    1. 1. The Value of Competency-based Medical EducationAcross the Continuum Eric S. Holmboe
    2. 2. Conflicts of Interest• Employed by the American Board of Internal Medicine• Serve on the boards of the National Board of Medical Examiners and Medbiquitous (both non-profit)• Receive royalties from Mosby-Elsevier for a textbook on assessment 2
    3. 3. Objectives for Today• Discuss the roles of assessment in a competency-based medical program• Develop approaches to help local educators effectively use performance-based assessment methods and tools• Discuss how performance-based assessment can be integrated into an education and assessment system for Memorial University. 3
    4. 4. Nostalgialitis Imperfecta Syndrome characterized by the following signs and symptoms: – “When I was an student…<insert superlative>” – “Medicine was so much better 25 years ago” • Reality: Not really… – “Younger physicians today are less professional, skilled, etc. because of <insert favorite complaint>”
    5. 5. Current Model of Training and Practice Retirement Competent
    6. 6. Change in Performance Over Time Lower Performance All OutcomesChoudhry NK, Ann Intern Med, 2005;142:260-73
    7. 7. With your immediate neighbors, discuss whatcompetency-based education and training means to you?
    8. 8. Competency-Based Medical Education Is an outcomes-based approach to the design, implementation, assessment and evaluation of a medical education program using an organizing framework of competencies the unit of progression is mastery of specific knowledge, skills and attitudesFrank, JR, Snell LS, ten Cate O, et. al. Competency-based medicaleducation: theory to practice. Med Teach. 2010; 32: 638–645
    9. 9. So What are the Outcomes and Who Determines Them? The Profession? The Public? Policy Makers?
    10. 10. Determining Outcomes: Perspectives The Profession? – The “core” of a discipline? – Competence in the Can MEDS roles? Safe, effective, patient-centered care? The Public? – Trust that a doctor can do certain things? Policy Makers? – Meeting the needs of the complex and aging Canadian health population?
    11. 11. Traditional versus Competency-based: Start with System NeedsFrenk J, et al. Health professionals for a new century: transformingeducation to strengthen health systems in an interdependent world.Lancet. 2010 11
    12. 12. U.S. Institute of Medicine Competency Framework IOM, 2003
    13. 13. Implications of CBME Curriculum and assessment follows from the competencies and outcomes, not vice versa Requires: – Definition of milestones of competency • What does competency look like? – Robust assessment methods, tools & systems
    14. 14. Educational ProgramVariable Structure/Process Competency-basedDriving force: Content-knowledge Outcome-knowledge curriculum acquisition applicationDriving force: process Teacher LearnerPath of learning Hierarchical Non-hierarchical (Teacher→student) (Teacher↔student)Responsibility: content Teacher Student and TeacherGoal of educ. encounter Knowledge acquisition Knowledge applicationTypical assessment tool Single subject measure Multiple objective measuresAssessment tool Proxy Authentic (mimics real tasks of profession)Setting for evaluation Removed (gestalt) Direct observationEvaluation Norm-referenced Criterion-referencedTiming of assessment Emphasis on summative Emphasis on formativeProgram completion Fixed time Variable timeCarraccio, et al. 2002.
    15. 15. Definitions and Frameworks: The Need for a Shared Understanding
    16. 16. Competency / Competencies An observable ability of a health professional – Reflects a spectrum – Integrates multiple components such as knowledge, skills, values, and attitudes – Multiple competencies can be combined – Measureable with respect to a defined outcome© 2009 Royal College and The International CBME Collaborators
    17. 17. Competent  Possessing the required abilities at a specified stage of medical education  Is always qualified by a frame of reference  …Dyscompetence  …Supracompetence© 2009 Royal College and The International CBME Collaborators
    18. 18. Frameworks Webster’s New Collegiate Dictionary: – A skeletal or structural frame – A basic structure (as of ideas) – Frame of reference
    19. 19. CanMEDS Framework
    20. 20. KSA FrameworkK = KnowledgeS = Skill Information gathering skills – Interview, physical examination, communication Ability to use knowledge and information – Problem solving, clinical judgment Management skills – Diagnosis, treatment, patient education, counseling, procedural skillsA = Attitudes Professionalism, humanism
    21. 21. RIME Model Developed at USUHS – Lou Pangaro and Gordon Noel in the 1980s for use in third year medical student clerkships in internal medicine “Synthetic” Model – Reporter – Interpreter – Manager – Educator
    22. 22. Dreyfus Developmental Model• Novice – Don’t know what they don’t know• Advanced Beginner – Know what they don’t know• Competent – Able to perform the tasks and roles of the discipline – restricted breath and depth• Proficient – Consistent and efficient in performance of the tasks and roles of the discipline - know what they know and don’t know• Expert – In depth knowledge concerning the discipline – often rule based – know what they know• Master – Expert who relishes the unknown, or the situation that breaks the rules – who the experts go to for help – don’t know what they know 1 as presented by Leach, D., modified by Nasca, T.J. American Board of Internal Medicine Summer Retreat, August, 1999
    23. 23. Competence Learning Performance is exhausting Sense of what’s important may be lacking Volume of various aspects can be overwhelming Competent model of decision making: – “is a detached, deliberative, and sometimes agonizing selection among alternatives” Dreyfus and Dreyfus, 1986
    24. 24. Assessment Across the Continuum:Challenges and Principles
    25. 25. Assessment Strategies in Undergraduate and Postgraduate Training
    26. 26. Milestones Definition A significant point in development Merriam-Webster A scheduled event signifying the completion of a major deliverable or a set of related deliverables.
    27. 27. Milestones and Trajectories Milestones should enable the trainee, program and the regulatory bodies to know an individuals trajectory of competency acquisition. The focus is developmental
    28. 28. Approaches to Developing Milestones 1 Discrete – Defining different behaviors in a domain of competence at each stage of training Continuous – A similar ability modified at each stage of training to reflect increasing complexity or sophistication – Parameters: • Setting, players, complexity, supervision 1 From Jason Frank, RCPSC 28
    29. 29. Approaches to Developing Milestones 1 Narrative – Detailed descriptions of stages of development of competency by domain – Short essays and vignettes that describe a “competence story”  EPAs(entrustable professional activity) 1 From Jason Frank, RCPSC 29
    30. 30. Patient CareACGME Developmental Milestones Approximate AssessmentCompetency Informing Time Frame Methods/Tools ACGME Competencies Trainee to Achieve StageClinical skills Historical Data Gathering Standardized and 1. Acquire accurate and relevant history 6 months patient reasoning from the patient in an efficiently Direct customized, prioritized, and Observation Manages hypothesis driven fashion Simulation patients using 2. Seek and obtain appropriate, verified, 9 months clinical skills of and prioritized data from secondary interviewing and sources (e.g. family, records, physical pharmacy) examination 3. Obtain relevant historical subtleties 18 months that inform and prioritize both differential diagnoses and diagnostic plans, including sensitive, complicated, and detailed information that may not often be volunteered by the patient Sub- competency
    31. 31. Communicator Milestones Project Stage 1 Stage 2 Stage 3 Stage 4Greets the Demonstrates Demonstrates Demonstrates Demonstratespatient in a rule-based proficiency; proficiency; proficiency & way; some adapts to efficiency; Simulation or adaptation; many Across role play routine contexts; spectrum of clinical complex practice clinical An Introduction to CBME – Frank, Snell, Harris, Holmboe 2012 31
    32. 32. Patient CareThe resident is demonstrating satisfactory development of the knowledge, skill and attitudes needed to advance in training.He/she is demonstrating a learning trajectory that anticipates the achievement of competency for unsupervised practice that includes the delivery of safe, timely, equitable patient-centered care. ____ Yes ____ No ____ Marginal 32
    33. 33. Milestones Benefits Provide the learner with a clear path of progression. There are no surprises. Allow for rich formative feedback. Learners know where they are and where they need to go. Define specific behaviors that can focus assessment.
    34. 34. Milestones Challenge Synthesizing milestones into larger global representations of competency that reflect those activities that define the profession. These activities have been described as entrustable professional activities or EPAs.
    35. 35. Entrustable Professional Activities  EPAs represent the routine professional-life activities of physicians based on their specialty and subspecialty  The concept of “entrustable” means: – ‘‘a practitioner has demonstrated the necessary knowledge, skills and attitudes to be trusted to independently perform this activity.’’11 Ten Cate O, Scheele F. Competency-based postgraduatetraining: can we bridge the gap between theory andclinical practice? Acad Med. 2007; 82(6):542–547.
    36. 36. Entrustable Trainee Activities ETAs, or entrustable resident or student activities, can help to define important benchmarks in a trainee’s development ETAs in a training program may mean: – A trainee has demonstrated the necessary knowledge, skills and attitudes to be trusted to perform this activity without constant or direct supervision.
    37. 37. Why “ETAs” to Assess Competence? Sampling of events that: - are critical moments in medical training - inform developmental progression - faculty and leaders already implicitly assess - are manageable for busy training programs – are logical of assessment for stakeholders Supported by generalizability theory – 8-12 focused assessments can potentially allow a generalized statement of competency
    38. 38. Synthesize to Physician trusted to meet Analyze to Educate and the health care needs of Understand Evaluate the population Competency Milestones LANDMARK in EPA in Training Practice Medical MK1 Knowledge MK2 Lead a resident Lead a health care team care team Patient Care PC1 PC2Professionalism Prof1 Prof2 Care for clinic patients with Practice Interpersonal ISC1 distance Skills independently ISC2 supervisionSystems-based SBP1 Practice SBP2 Complete an audit Lead Quality of a panel of ImprovementPractice-based PBLI1 clinic patients initiative learning PBLI2 Shared Mental Models and Frameworks
    39. 39. Entrustment Trajectories
    40. 40. Entrustments in Newfoundland With a neighbor(s), discuss an entrustment you make either with medical students or post-graduate trainees How do you arrive at this entrustment judgment?
    41. 41. Questions?Contact