Download as PDF, PPTX
More Related Content
Similar to Assessment instruction module
Assessment instruction module
- 1. Educator Development ProgramSchool of MedicineVANDERBILT Assessment & Instruc-onal Strategies Advanced Clinical Elec-ves (ACE) or Ac-ng Internships (AI) at Vanderbilt Assembled by Kim Lomis, MD & Lillian B. Nanney, Ph.D March 2015
- 2. Educator Development ProgramSchool of MedicineVANDERBILT Learning Objectives for this Module After viewing materials in this presentation, a course director should be able to. . . . . 1. Match learning objectives for an ACE or AI course with appropriate instructional and assessment strategies for learners. 2. Design an ACE or AI course that conforms to best practices that have been established by Vanderbilt’s School of Medicine.
- 3. Educator Development ProgramSchool of MedicineVANDERBILT Why our emphasis on assessment? • Assessment drives learning. • Assessment is a 2-way street. It allows both the learner and the instructor to determine if the learning objectives have been achieved.
- 4. Educator Development ProgramSchool of MedicineVANDERBILT Assessment Serves Dual Func-ons Provide an Assessment OF Learning This is summa4ve = used for grading Provide an Assessment FOR Learning This is forma4ve = guides the learner Vanderbilt’s Course Directors are expected to . . . . . .
- 5. Educator Development ProgramSchool of MedicineVANDERBILT A New Compelling Argument for Providing Medical Students with Rigorous Assessments • Years 1 and 2 in the School of Medicine are now foundational. They are graded as pass/fail. • Assessments are critical for appraising a student’s readiness for residency.
- 6. Educator Development ProgramSchool of MedicineVANDERBILT Responsibili-es of the Assessor • Assessment is one of your professional du4es. • Feedback must be based on OBSERVATIONS of PERFORMANCE
- 7. Educator Development ProgramSchool of MedicineVANDERBILT Best Prac-ces Courtesy of the Standing Assessment CommiKee Category: Communica4on to Learners/Course Faculty: • Learning objec-ves MUST be shared with students and par-cipa-ng faculty, residents, fellows. Distribute your materials on Day 1. • Ensure that each assessment is linked to a learning objec-ves. • Disclose your grading procedures to students and par-cipa-ng assessors in print and/or electronic form. Also a Day 1 ac-vity.
- 8. Educator Development ProgramSchool of MedicineVANDERBILT Category: Assessment Methods • Grading must be based on more than one assessment method. Collect different perspec4ves to ensure accuracy and fairness. • Performance data about an individual student should be solicited from mul4ple team members (faculty, residents, other health professionals). Category: Feedback to learners: • Courses must provide forma4ve feedback, 4med such that there is opportunity for the learner to improve. Best Prac-ces Courtesy of the Standing Assessment CommiKee
- 9. Educator Development ProgramSchool of MedicineVANDERBILT • Learners have a right to expect fairness and consistency from course to course. One purpose of this module is to norm the course directors/assessors. • Vanderbilt has a responsibility to ensure that learners are progressing upward along a developmental learning curve. This determination can only be made if learners are observed and assessed. Our Milestones Module is coming soon . . . . . • Learners have a right to know specific areas where they are not progressing or are deficient. To recap - Assessment/observations must be accurate, well-documented, informative and frequent. Assessment Does MaKer
- 10. Educator Development ProgramSchool of MedicineVANDERBILT Ideally, assessment should inform future learning: • The Assessment should describes current performance “You are here” • The Assessment should ar-culate behaviors necessary to aKain the next level of performance. “Go this way” Milestones = GPS (Global Posi-oning System)
- 11. Educator Development ProgramSchool of MedicineVANDERBILT All 3 aspects of course design must be though[ully aligned Learning Objec-ves Assessment Strategies Instruc-onal Strategies
- 12. Educator Development ProgramSchool of MedicineVANDERBILT Let’s focus on Learning Objec-ves Learning Objec-ves Assessment Strategies Instruc-onal Strategies 1. How will you help students aKain them? This is your instruc-onal strategy. 2. How will you ensure they have aKained them? This is your assessment method.
- 13. Educator Development ProgramSchool of MedicineVANDERBILT A Learning Objec-ves can best be achieved by a specific type of instruc-onal tool. Learning Objec-ves Assessment Strategy Instruc-onal Strategy Each learning objec-ves must be measurable (assessable) and reasonable given the learning -me.
- 14. Educator Development ProgramSchool of MedicineVANDERBILT Examine our example below linking learning objectives with likely educational activities and then how the learner will be assessed. Our sketch would be suitable for an ACE course in Neuroradiology or Neurosurgery. Learning Objec-ve Educa-onal Ac-vity Assessment Create a differen-al diagnosis of brain tumors based on tumor loca-on a. par4cipate in weekly brain tumor boards b. complete a comprehensive pa4ent evalua4on (using a template) for one pa4ent of interest during the first 2 weeks c. view a prepared module on this topic d. seek evidence based literature (epidemiological, guideline ar4cles) on this topic. e. par4cipate in the radiology reading room experience when reports are being prepared. f. present assigned pa4ents to the team during daily reports. 1. Student completes a second write-‐up by the end of course for an unknown tes4ng case. 2. A publicized scoring rubric will be used by Resident/ faculty to score two pa4ent presenta4ons during each week. The first 2 weeks are for feedback purposes. The last 2 weeks will be summa4ve.
- 15. Educator Development ProgramSchool of MedicineVANDERBILT Examine another example linking learning objectives with likely educational activities and then with learner assessment. Our sketch would be suitable for an ACE course in Endocrinology (Diabetes). Learning Objec-ve Educa-onal Ac-vity Assessment Gain proficiency in evalua-ng and trea-ng in pa-ents with diabetes mellitus and in formula-ng or modifying a treatment regimen to control blood glucose level. a. Student will aWend the adult endocrinology fellows’ conference b. Student will aWend rounds with the diabetes consulta4ve service in the hospital c. Student will provide glycemic care in 4 seYngs (periopera4ve, post-‐surgical emergency and cardiac pa4ents). d. Students will prac4ce educa4ng pa4ents regarding how to test blood gluose, dose/administer insulin, count carbohydrate grams, use devices such as insulin pens. 1. Assessment of the quality of pa4ent educa4on will be assessed by direct clinical observa4ons on the glucose management service once a day. 2. Assessed by oral exam from a set of prepared cases and graded by a scoring rubric familiar with to the learner. 3. Quality of glycemic care decisions in daily prac4ce will be scored once a day by residents/fellows on the service.
- 16. Educator Development ProgramSchool of MedicineVANDERBILT Now its your turn for prac-ce. Here is a simple table to get you started. Write 3 plans for your course. This VSTAR module also contains a word doc with a table you can download and use. Learning Objec-ve Educa-onal Ac-vity Assessment Strategy 1. 2. 3.
- 17. Educator Development ProgramSchool of MedicineVANDERBILT Who will be the Assessors? Consider: Other Faculty The Residents The Staff 1) Will they need some training so they can be as standardized as possible? 2) Should you furnish them with a checklist or rubric? 3) Do you need to do some “faculty” development? Do they need to see the linkage between learning objectives, instructional strategies, and assessments so they have clarity about expectations for learners at this level? Answers: Yes, Yes, Yes
- 18. Educator Development ProgramSchool of MedicineVANDERBILT Unconscious Bias Share these reminders with your assessors Halo Error Outstanding performance in one area, particularly early in the rotation, tends to blind observers to poorer performance in other domains. Similarly, it can be hard for a student to recover from a poor start. Wave Effect If everyone else has felt that the learner is superior, it can be difficult to give an unsatisfactory assessment. The student is ‘”waved” through the rotation on reputation and not on performance. Hawk or Dove Effect Take a close look at the new grading scale that Vanderbilt will be using. Are you too hard, too easy? Use the rating tools appropriately to ensure fair grading (Stay turned for the Advanced Assessment Module) .
- 19. Educator Development ProgramSchool of MedicineVANDERBILT Similarity/Dissimilarity Bias Recognize something of yourself in the Learner? - Likely a good assessment will result. Likewise, it can be hard to give a high rating to someone with a very different practice style. Prejudices Accents, beards, gender, lip rings, tattoos? Be aware of your prejudicial tendencies (blindspots) that can interfere with fair assessment. Unconscious Bias (con-nued) Share these reminders with your assessors
- 20. Educator Development ProgramSchool of MedicineVANDERBILT Learning Objec-ves Assessment Strategy Instruc-onal Strategy Learning Objective: Interpret clinical information to formulate a prioritized differential diagnosis that guides the creation of a patient-specific management plan. Assessment Strategy: SNAPPS S - summarize the case N - narrow the differential A - analyze the differential P - probe the preceptor P - plan management S - select an issue for self directed learning
- 21. Educator Development ProgramSchool of MedicineVANDERBILT Consider these Using these Tools • 1 minute preceptor technique hWp://www.prac4caldoc.ca/teaching/prac4cal-‐prof/teaching-‐nuts-‐bolts/one-‐ minute-‐preceptor/ • SNAPPS technique (your departmental Master Clinical Teacher can host a teaching session) hWp://www.prac4caldoc.ca/teaching/prac4cal-‐prof/teaching-‐nuts-‐bolts/snapps/ • Mini-‐Clinical Evalua4on Exercise (Mini-‐CEX) hWp://www.abim.org/program-‐directors-‐administrators/assessment-‐tools/mini-‐cex.aspx • S4mulated chart review hWp://www.prac4caldoc.ca/teaching/prac4cal-‐prof/teaching-‐nuts-‐bolts/chart-‐s4mulated-‐ recall/ • Case Discussions • Conference Presenta4ons
- 22. Educator Development ProgramSchool of MedicineVANDERBILT Use Verbal Feedback Techniques § Continue (You are on track) § Do More (of “this”) § Do Less (of “this”) § Stop (Don’t do “this” again!) Considerations: • Keep portions small and feedback spaced out over time • Students tend to selectively “hear” good bites and miss the critique. • Best prac4ce assumes that feedback is delivered in a rela4vely private seYng.
- 23. Educator Development ProgramSchool of MedicineVANDERBILT Standing Assessment CommiKee (recapping the baseline requirements) 1) Use Mul4-‐Modal Tools -‐ to gain many perspec-ves 2) Distribute the plan to students -‐ remove the mystery around “What am I supposed to learn?” 3) Distribute the plan to all the possible assessors -‐ eliminate all mystery around “What am I supposed to do with this student.” 4) Provide frequent feedback -‐mid and end-‐of-‐course at minimum 4) Final grades due < 6 weeks
- 24. Educator Development ProgramSchool of MedicineVANDERBILT Upcoming Modules • Milestones are coming to Clinical Evalua4on We will share which competencies must be assessed. • Master Adap4ve Workplace Learner Cycle We will be helping students improve their prac4ce-‐based learning skills and develop the habits necessary for lifelong learning. Courses will be asked to include assessment of the student’s progress through the Master Adap4ve Workplace Learner cycle. More informa4on to come on this soon.