A paradigm shift! This is not tinkering around the edges!
In the CBME approach, medical education is organized around “competencies”, and not just scientific medical knowledge as in our traditional approach.
In this way of thinking, our definition of competence is changed. It is multidimensional and contextual. This also leads to the relative terms “dyscompetence” and “supracompetence”.
I specifically have a box around the 18 month milestone in preparation for the next slide…
1.2 Initiating the interview Pre-Clerkship 1 Pre-Clerkship 2 MD 1 MD 2 Junior Resident 1 Junior Resident 2 Senior residency (entry to practice) 1 Senior residency (entry to practice) 2 Advanced Practice 1 Advanced Practice 2 a) Greets the patient, and caregivers, and confirms how s/he/they would like to be addressed 3. Demonstrates fully, but may be rule-based in application. 2. Simulation/Role play/Standardized patient 4. Demonstrates proficiency and efficiency and is able to adjust to the specific patient or context 3. Routine clinical 4. Demonstrates proficiency and efficiency and is able to adjust to the specific patient or context 5. Across the spectrum of professional practice 4. Demonstrates proficiency and efficiency and is able to adjust to the specific patient or context 5. Across the spectrum of professional practice 5. Demonstrates innovation, improvisation and deliberate use of best practice 5. Across the spectrum of professional practice
This example uses a narrative stream for patient care and demonstrates how narrative milestones could be used in a FasTrack type system. You will notice the language under the narrative stream asks for a judgment about the trainee’s trajectory. Given that there are 23 narrative streams which equates to about 3 – 4 for each general competency, you could envision a trainee progressing at different rates for those streams. Asking for a summative judgment allows a program to identify that a trainee may be lagging in some areas, but is still felt overall to be developing appropriately.
This would address the criticism of being reductionistic
Landmarks are a little different than EPA’s because they occur during training … Why use them to assess competence? They are a sampling of events that … Mention how the Reed Williams article supports this concept of focused assessment of a particular skill/set of skills allows for a generalized statement of competency for that skill
Here is how this may actually develop for any given learner. This does introduce the concept that competency development is not limited to the small window of formal training. One challenge that CBME will need to address is what will be the minimum needed to successfully complete residency or fellowship training?
The Value of Competency-based Medical Education Across the Continuum
The Value of Competency-based Medical EducationAcross the Continuum Eric S. Holmboe
Conflicts of Interest• Employed by the American Board of Internal Medicine• Serve on the boards of the National Board of Medical Examiners and Medbiquitous (both non-profit)• Receive royalties from Mosby-Elsevier for a textbook on assessment 2
Objectives for Today• Discuss the roles of assessment in a competency-based medical program• Develop approaches to help local educators effectively use performance-based assessment methods and tools• Discuss how performance-based assessment can be integrated into an education and assessment system for Memorial University. 3
Nostalgialitis Imperfecta Syndrome characterized by the following signs and symptoms: – “When I was an student…<insert superlative>” – “Medicine was so much better 25 years ago” • Reality: Not really… – “Younger physicians today are less professional, skilled, etc. because of <insert favorite complaint>”
Current Model of Training and Practice Retirement Competent
Change in Performance Over Time Lower Performance All OutcomesChoudhry NK, Ann Intern Med, 2005;142:260-73
With your immediate neighbors, discuss whatcompetency-based education and training means to you?
Competency-Based Medical Education Is an outcomes-based approach to the design, implementation, assessment and evaluation of a medical education program using an organizing framework of competencies the unit of progression is mastery of specific knowledge, skills and attitudesFrank, JR, Snell LS, ten Cate O, et. al. Competency-based medicaleducation: theory to practice. Med Teach. 2010; 32: 638–645
So What are the Outcomes and Who Determines Them? The Profession? The Public? Policy Makers?
Determining Outcomes: Perspectives The Profession? – The “core” of a discipline? – Competence in the Can MEDS roles? Safe, effective, patient-centered care? The Public? – Trust that a doctor can do certain things? Policy Makers? – Meeting the needs of the complex and aging Canadian health population?
Traditional versus Competency-based: Start with System NeedsFrenk J, et al. Health professionals for a new century: transformingeducation to strengthen health systems in an interdependent world.Lancet. 2010 11
U.S. Institute of Medicine Competency Framework IOM, 2003
Implications of CBME Curriculum and assessment follows from the competencies and outcomes, not vice versa Requires: – Definition of milestones of competency • What does competency look like? – Robust assessment methods, tools & systems
Educational ProgramVariable Structure/Process Competency-basedDriving force: Content-knowledge Outcome-knowledge curriculum acquisition applicationDriving force: process Teacher LearnerPath of learning Hierarchical Non-hierarchical (Teacher→student) (Teacher↔student)Responsibility: content Teacher Student and TeacherGoal of educ. encounter Knowledge acquisition Knowledge applicationTypical assessment tool Single subject measure Multiple objective measuresAssessment tool Proxy Authentic (mimics real tasks of profession)Setting for evaluation Removed (gestalt) Direct observationEvaluation Norm-referenced Criterion-referencedTiming of assessment Emphasis on summative Emphasis on formativeProgram completion Fixed time Variable timeCarraccio, et al. 2002.
Definitions and Frameworks: The Need for a Shared Understanding
KSA FrameworkK = KnowledgeS = Skill Information gathering skills – Interview, physical examination, communication Ability to use knowledge and information – Problem solving, clinical judgment Management skills – Diagnosis, treatment, patient education, counseling, procedural skillsA = Attitudes Professionalism, humanism
RIME Model Developed at USUHS – Lou Pangaro and Gordon Noel in the 1980s for use in third year medical student clerkships in internal medicine “Synthetic” Model – Reporter – Interpreter – Manager – Educator
Dreyfus Developmental Model• Novice – Don’t know what they don’t know• Advanced Beginner – Know what they don’t know• Competent – Able to perform the tasks and roles of the discipline – restricted breath and depth• Proficient – Consistent and efficient in performance of the tasks and roles of the discipline - know what they know and don’t know• Expert – In depth knowledge concerning the discipline – often rule based – know what they know• Master – Expert who relishes the unknown, or the situation that breaks the rules – who the experts go to for help – don’t know what they know 1 as presented by Leach, D., modified by Nasca, T.J. American Board of Internal Medicine Summer Retreat, August, 1999
Competence Learning Performance is exhausting Sense of what’s important may be lacking Volume of various aspects can be overwhelming Competent model of decision making: – “is a detached, deliberative, and sometimes agonizing selection among alternatives” Dreyfus and Dreyfus, 1986
Assessment Across the Continuum:Challenges and Principles
Assessment Strategies in Undergraduate and Postgraduate Training
Milestones Definition A significant point in development Merriam-Webster A scheduled event signifying the completion of a major deliverable or a set of related deliverables. mariosalaexandrou.com
Milestones and Trajectories Milestones should enable the trainee, program and the regulatory bodies to know an individuals trajectory of competency acquisition. The focus is developmental
Approaches to Developing Milestones 1 Discrete – Defining different behaviors in a domain of competence at each stage of training Continuous – A similar ability modified at each stage of training to reflect increasing complexity or sophistication – Parameters: • Setting, players, complexity, supervision 1 From Jason Frank, RCPSC 28
Approaches to Developing Milestones 1 Narrative – Detailed descriptions of stages of development of competency by domain – Short essays and vignettes that describe a “competence story” EPAs(entrustable professional activity) 1 From Jason Frank, RCPSC 29
Patient CareACGME Developmental Milestones Approximate AssessmentCompetency Informing Time Frame Methods/Tools ACGME Competencies Trainee to Achieve StageClinical skills Historical Data Gathering Standardized and 1. Acquire accurate and relevant history 6 months patient reasoning from the patient in an efficiently Direct customized, prioritized, and Observation Manages hypothesis driven fashion Simulation patients using 2. Seek and obtain appropriate, verified, 9 months clinical skills of and prioritized data from secondary interviewing and sources (e.g. family, records, physical pharmacy) examination 3. Obtain relevant historical subtleties 18 months that inform and prioritize both differential diagnoses and diagnostic plans, including sensitive, complicated, and detailed information that may not often be volunteered by the patient Sub- competency
Communicator Milestones Project Stage 1 Stage 2 Stage 3 Stage 4Greets the Demonstrates Demonstrates Demonstrates Demonstratespatient in a rule-based proficiency; proficiency; proficiency & way; some adapts to efficiency; Simulation or adaptation; many Across role play routine contexts; spectrum of clinical complex practice clinical An Introduction to CBME – Frank, Snell, Harris, Holmboe 2012 31
Patient CareThe resident is demonstrating satisfactory development of the knowledge, skill and attitudes needed to advance in training.He/she is demonstrating a learning trajectory that anticipates the achievement of competency for unsupervised practice that includes the delivery of safe, timely, equitable patient-centered care. ____ Yes ____ No ____ Marginal 32
Milestones Benefits Provide the learner with a clear path of progression. There are no surprises. Allow for rich formative feedback. Learners know where they are and where they need to go. Define specific behaviors that can focus assessment.
Milestones Challenge Synthesizing milestones into larger global representations of competency that reflect those activities that define the profession. These activities have been described as entrustable professional activities or EPAs.
Entrustable Professional Activities EPAs represent the routine professional-life activities of physicians based on their specialty and subspecialty The concept of “entrustable” means: – ‘‘a practitioner has demonstrated the necessary knowledge, skills and attitudes to be trusted to independently perform this activity.’’11 Ten Cate O, Scheele F. Competency-based postgraduatetraining: can we bridge the gap between theory andclinical practice? Acad Med. 2007; 82(6):542–547.
Entrustable Trainee Activities ETAs, or entrustable resident or student activities, can help to define important benchmarks in a trainee’s development ETAs in a training program may mean: – A trainee has demonstrated the necessary knowledge, skills and attitudes to be trusted to perform this activity without constant or direct supervision.
Why “ETAs” to Assess Competence? Sampling of events that: - are critical moments in medical training - inform developmental progression - faculty and leaders already implicitly assess - are manageable for busy training programs – are logical of assessment for stakeholders Supported by generalizability theory – 8-12 focused assessments can potentially allow a generalized statement of competency
Synthesize to Physician trusted to meet Analyze to Educate and the health care needs of Understand Evaluate the population Competency Milestones LANDMARK in EPA in Training Practice Medical MK1 Knowledge MK2 Lead a resident Lead a health care team care team Patient Care PC1 PC2Professionalism Prof1 Prof2 Care for clinic patients with Practice Interpersonal ISC1 distance Skills independently ISC2 supervisionSystems-based SBP1 Practice SBP2 Complete an audit Lead Quality of a panel of ImprovementPractice-based PBLI1 clinic patients initiative learning PBLI2 Shared Mental Models and Frameworks