More Related Content
What's hot
What's hot (20)
Similar to Guidelines for Evaluating Learners Clinical Performance
Similar to Guidelines for Evaluating Learners Clinical Performance (20)
More from TucsonMedicalCenter
More from TucsonMedicalCenter (20)
Recently uploaded
Recently uploaded (20)
Guidelines for Evaluating Learners Clinical Performance
- 1. Guidelines for Evaluating Learners’ Clinical Performance
- 2. Know Do Learning Objectives • The definition and purpose of evaluation. • The preceptor’s role in evaluation. • Strategies to make evaluation efficient and useful. • Eager to make evaluation more student-focused. • Enthusiastic about encouraging learners to self-assess. • Use strategies that will help improve techniques for evaluating learners’ clinical performance. Feel
- 3. Feedback Evaluation Primary Differences between Feedback & Evaluation Schwiebert, L. P., & Bondurant, W. (2000). Summative Feedback, Evaluation, and Grading Students. In P. M. Paulman, J. L. Susman, & C. A. Abboud (Eds.), Precepting Medical Students in the Office (pp. 81-87) Baltimore: The Johns Hopkins University Press. o Conveys information o Formative o Current performance o Neutral (verbs/nouns) o Fosters learning o Conveys judgment o Summative o Past performance o Normative statement (adjectives/adverbs) o Certification
- 4. Preceptor’s Role in Evaluation Before Rotation - Understand program’s expectations - Review goals and objectives During Rotation - Gather information from multiple sources - Provide feedback - Use systematic method of recording End of Rotation - Prepare for final evaluation - Schedule and conduct summary meeting - Complete and submit final evaluation
- 5. The Evaluation Process ① Collect data ② Review and collate data ③ Apply framework (competencies) ④ Determine level of competency ⑤ Share data with learner During the Clinical Rotation
- 6. Use aVariety of Sources to EvaluateYour Learner
- 7. ① Medical Knowledge ② Patient Care ③ Professionalism ④ Interpersonal Communication ⑤ Practice-based Learning: personal improvement ⑥ Systems-based Practice: system improvement ACGME Core Competencies acgme.org
- 8. Guidelines for Preparing for the Final Evaluation ① Evaluation should be based on a systematic observation recorded over a period of time. ② Evaluation should emphasize both changes in behavior (improvement) and progress toward a goal. Example: Julie’s suturing skills are comparable to other first-year trainees. She has mastered proper wound preparation. Her suture spacing and tension are improving.
- 9. Guidelines, cont. ③ Evaluation should be both verbal and written whenever possible. ④ Evaluation should be conducted in an unhurried atmosphere. ⑤ Student should self-assess. ⑥ Evaluation should fulfill due process procedures.
- 10. SampleWritten Comments John is able to get to the important parts of a history. He appeared kind and understanding. He could quickly size up which individuals were difficult patients. Sheila had a bit of a hard time applying and adapting her textbook knowledge to fit the real life cases that are part of every family practice. While this reality threw her at first, I noticed significant improvement by the end of her time here. UNDERSTANDING (problem solving, synthesis of knowledge, originality, analytical ability)
- 11. SampleWritten Comments SKILL (rapport, histories, physical examination, laboratory organization, adaptability, use of hands) Larry needs to work on taking a brief general history, then concentrate on a more detailed history of the current problem. Bill does a good exam but is occasionally casual in his attitude, writes incomplete notes, and is not as thorough as he could be. Not so great with his hands but makes up for it with strength of personality. He will be popular with his patients.
- 12. SampleWritten Comments KNOWLEDGE (scope and depth of faculty information) I felt that Ann had a fairly narrow field of knowledge regarding many of the cases we encounter in this practice. Additional exposure to textbook physical diagnosis would help her improve in this area. Ed worked hard to improve his differential diagnosis skills. By the clerkship’s end, he was performing at an appropriate level for a third-year student.
- 13. SampleWritten Comments ATTITUDE (intellectual curiosity, respect, integrity, recognizes limitations) Susan was excessively self-confident and needs to better understand her limits. Despite giving her specific feedback on this point several times, she still needs to work on this area. Although intelligent, she often jumped to conclusions without weighing all other possibilities. I really got the feeling that Doug had a “just passing through” attitude while he was in my office. He didn’t seem interested in what was happening or in improving in areas where he was weak (like doctor-patient communication skills), despite specific feedback. He asked few questions and did not respond to my efforts to get him thinking about the consulting-referring physician interaction.
- 14. SampleWritten Comments GENERAL COMMENTS (strengths and weaknesses) Stan has good communication skills and establishes rapport easily with a wide range of patients. He was able to get some information from a complicated and uncommunicative patient that has helped me greatly in that patient’s care. He has a gentle style that I predict will make him a sought-after physician once he is in practice. Brad is quiet and reserved. I know that he cares about people but sometimes his natural reserve can come across as uncaring. He needs to continue to work on comfortable ways to demonstrate warmth and build rapport during one-on-one patient encounters. We discussed specific strategies such as concentrating on eye-contact, using more non-verbal prompts, and having a more relaxed posture during the interview.
- 15. A ‘Word Cloud’ of Written Comments
- 16. Pre-Rotation Planning ① Review course materials and evaluation forms from the program. ② Plan ways to observe and measure trainee behaviors. ③ Check in with trainee: a. Review form and your role in grading. b. Consider learner self-assessment. Summary
- 17. Summary Core Rotation Activities ① Observe and provide regular, ongoing trainee feedback. ② Use a system to record trainee observations. ③ Consider mid-rotation review, especially if deficiencies are identified.
- 18. Summary End of Rotation Activities ① Review evaluation criteria and form. ② Review student performance data. ③ Schedule and conduct final evaluation session with trainee. ④ Complete and return evaluation form promptly.
- 19. Use the strategies presented to evaluate learners’ clinical performance during mid- and final- assessments.
- 20. What will you keep the same? What will you do more of? What will you do less of? What will you stop doing? What will you do differently & how will you do it? What will you add? LEARN – REFLECT -TEACH
Editor's Notes
- Let’s make sure we understand the differences between feedback and evaluation. What do you think are the primary differences between “Feedback” and “Evaluation?” For starters … - “Evaluation is used for grades; feedback is used for improvement.” “Evaluation is summative; feedback is formative.” “Evaluation is written down; feedback is spoken.” Compare / contrast “Feedback” vs. “Evaluation” The images capture the essence of each term. “FEEDBACK is an assessment for learning rather than an assessment of learning.” FORMATIVE EVALUTION (or FEEDBACK) … is evaluation which seeks to shape, grow and develop an individual EVALUATION IS JUDGMENTAL (it addresses how well or poorly a learner performed, often in comparison with peers). Inasmuch as it provides a trainer’s distillation of overall trainee performance, evaluation can also be called ‘SUMMATIVE FEEDBACK. Evaluation can be thought of as a CONTINUOUS process of observing behaviors and comparing those behaviors to a standard, such as course criteria, norm of the class, or defined competencies. The DIFFERENCE between evaluation and feedback is the comparison of performance to a standard. CONSIDER … How does this trainee compare with other PGY learners you have taught? CONSIDER … Is there a difference between WRITTEN vs ORAL with giving Feedback or Evaluation? (SCHWIEBERT & BONDURANT – CH 16 PRECEPTING MEDICAL STUDENTS IN THE OFFICE)
- The Preceptor’s Role in Evaluation Before Clinical Rotation Understand program’s expectations Review course materials (goals/objectives) and evaluation forms from the program. Plan ways to observe and measure learner behaviors. Check in with learner: Review form and your role in grading. Consider learner self-assessment. During Clinical Rotation 1. Gather information from multiple sources. 2. Provide feedback. 3. Use systematic method of recording. Then you will be ready, with a little preparation, for the final evaluation meeting with the student at the end of the clerkship. END of Clinical Rotation Prepare for final evaluation Review evaluation criteria and form – if you have questions, contact the program. Review trainee performance data – data you’ve observed and collected from others. Schedule and conduct final evaluation session with trainee – conduct this evaluation in an unhurried manner in a quiet, private place. 5. Complete and return the evaluation form promptly – do this within 24 hours of meeting with the trainee, while the information is still fresh.
- The Evaluation Process During Clinical Rotation 1. Gather information from multiple sources. 2. Review and collate data Apply framework (competencies) Determine the level of competency Share data with learner a. Schedule and conduct final evaluation session with trainee – conduct this evaluation in an unhurried manner in a quiet, private place. b. Complete and return the evaluation form promptly – do this within 24 hours of meeting with the trainee, while the information is still fresh. TIPS for preceptors: Sharing the evaluation form with the trainee early in the rotation is often helpful as trainees don’t always know which criteria are used to rate their performance. The preceptor may want the trainee to do a self-assessment early in the rotation to help identify areas of concentration for their time together. If, for example, a trainee identifies trouble with physical exam skills, the preceptor can emphasize this aspect of patient care.
- It is necessary to gather information from a variety of sources in order to provide a proper evaluation of trainees’ performance. What sources of information do you use? Collect data from a variety of sources: Student self-assessment Questioning Patient logs Reading of patient write-ups (patient notes / records) Observation of trainee doing history/physical Observation of trainee with patient and/or family members Observation of tutorial performance Observation of trainee doing procedures Observation of trainee’s case presentations Input gained from staff and/or patients about trainee’s performance Direct discussion of patient assessment and/or planning Student projects
- 6 ACGME CORE COMPETENCIES: MEDICAL KNOWLEDGE– demonstration of knowledge of the established and evolving biomedical, clinical, epidemiological and social-behavioral sciences, as well as the application of this knowledge to patient care. INTERPERSONAL & COMMUNICATION SKILLS – demonstrates interpersonal & communication skills that result in the effective exchange of information and collaboration with patients, their families, and health professionals PATIENT CARE – the ability to provide patient care that is compassionate, appropriate, and effective for the treatment of health problems and the promotion of health PROFESSIONALISM – demonstrates a commitment to carrying out professional responsibilities and an adherence to ethical principles PRACTICE-BASED LEARNING & IMPROVEMENT – the ability to investigate and evaluate one’s care of patients, to appraise and assimilate scientific evidence, and to continuously improve patient care based on constant self-evaluation and life-long learning SYSTEMS-BASED PRACTICE – demonstrates awareness of and responsiveness to the larger context and system of health care, as well as the ability to call effectively on other resources in the system to provide optimal health care
- In addition to general guidelines related to feedback, the following refer specifically to the process of evaluation: Evaluation should be based on a systematic observation recorded over a period of time. Evaluation should emphasize both changes in behavior (improvement) and progress toward a goal. Example: Julie’s suturing skills are comparable to other third-year medical students. She has mastered proper wound preparation. Her suture spacing and tension are improving.
- Evaluation should be both verbal and written whenever possible. If only verbal evaluations are given, those being evaluated should be asked to review their understanding of the evaluation. 4. Evaluation should be conducted in an unhurried atmosphere. The evaluator should undertake an evaluation only of what can be adequately covered in the available time. The individual being evaluated should have the opportunity to provide input. Evaluation should fulfill due process procedures.
- Sometimes it is difficult to come up with useful written comments about students’ performance in your practice. Here are some samples that you could refer to when you prepare written comments about the students who work in your practice. We’d like to review these from the perspective of the student— HOW this information helps them learn and change their behavior (if necessary). To assess UNDERSTANDING, we’re looking at problem solving, their ability to synthesize information, originality, and their analytic ability. John’s scenario: short, brief, but contains essential elements that would be helpful for the student to know HOW IS THIS HELPFUL TO THE STUDENT? Sheila: Identifying a difficulty and then observing how the student improved over the duration of the clerkship. HOW IS THIS HELPFUL TO THE STUDENT? While the student’s initial difficulty could be viewed as a negative, the experience captures a successful turn around (which could be seen as an advantage).
- We can evaluate SKILL by reporting on their rapport with people, how they conduct histories and physical exams, their adaptability, and use of hands. Laboratory organization would be assessed in a research context. Both statements indicate areas where improvement is needed: Larry … needs to work on taking a brief general history, then focusing on a more detailed history. HOW IS THIS HELPFUL TO THE STUDENT? Bill is complimented on performing a ‘good’ exam, but his attitude is occasionally ‘casual’ … non-professional, and he takes incomplete notes and his hand dexterity is not very good. HOW IS THIS HELPFUL TO THE STUDENT? The statement “He will be popular with his patients.” … is an assumption….. Is this appropriate? Necessary? Helpful?
- Here we have two comments that describe students’ KNOWLEDGE base (scope and depth of faculty information): Ann …the preceptor noted that she had a fairly narrow field of knowledge on many of the cases she encountered and recommended that additional reading on physical diagnosis in a textbook should be of help. HOW IS THIS HELPFUL TO ANN? Ed … the preceptor observed a change in Ed’s ability to come up with a differential … saying that John worked hard to improve his differential diagnostic skills and that at the end of the clerkship, he was performing at the appropriate 3rd-yr student level. HOW IS THE HELPFUL TO ED?
- In this scenario the preceptor is assessing students’ ATTITUDE— the intellectual curiosity, respect, integrity, and their ability to recognize their limitations. Susan … is a student who is excessively self-confident and is still not showing adequate improvement even after specific feedback has been given. What suggestions do you have for handling this situation? How would you rate her performance: 1 or 2? 1. Not acceptable (significant deficiencies exist or are below standards)? 2. Needs improvement ( compared with peers, is at a marginal level)? Doug … a student who didn’t take the rotation very seriously … “just passing through” attitude; didn’t seem interested in the work and wasn’t interested in improving his doctor-patient communication skills (despite feedback given). How would you handle this situation? How would you rate him?
- Under GENERAL COMMENTS we address specific strengths and weaknesses that students have. Here we compare statements about two students—Stan and Brad. Stan … this preceptor has very good comments regarding his strengths … such as his communication skills and easy rapport with patients. Also, his ability to obtain information from a complicated and uncommunicative patient … which helped the preceptor greatly. He predicts that Stan will be a sought-after physician in practice… is this appropriate? HOW IS THIS INFO HELPFUL TO STAN? Brad …on the other hand is quiet and reserved, which can sometimes come across as uncaring. The preceptor identifies specific strategies to help him demonstrate warmth and build rapport with patients. HOW IS THIS INFO HELPFUL TO BRAD? How would you rate Stan and Brad? Not acceptable (significant deficiencies exist or are below standards) Needs Improvement (compared with peers, is at a marginal level) Meets expectations (competence is appropriate for level of training Exceeds expectations (level achieved by only the top 20% of students Unable to assess (unable to evaluate)
- Research in 2012 on written comments for students in a 7-wk General Surgery & Anesthesiology clerkship included the development of a WORD CLOUD of ALL the comments provided … this technique presents individual words in a ‘cloud’ in which the SIZE of the word is directly related to its FREQUENCY of occurrence in the data. This research revealed that: There was a wide range of student performance was recorded (by mostly physicians, also patients, peers, administrators) (4 themes noted: Student as Physician-in-Training; Student as Learner; Student as Team Member; and Student as Person) Assessors provided comments on different aspects of student’s performance Suggesting that comments provided from a SINGLE viewpoint may potentially under-represent or overlook some areas of student performance
- We want to leave you with some key points regarding pre-rotation planning: be sure to … Review course materials and evaluation forms from the program– share with other professional colleagues who may also work with the student. Plan ways to observe and measure trainee behaviors – discuss with your staff. Check in with trainee: Review form and your role in grading – show form to trainee. Consider student self-assessment – daily, mid-point, and end of rotation.
- As far as the core rotation activities are concerned, here are three points to remember: Observe and provide regular, ongoing trainee feedback – on a daily basis. Use system to record trainee observations – record your observations on paper or computer; copy a sample of trainees’ patient notes. Consider mid-clerkship review, especially if deficiencies are identified – remember to use the “Ask” – “Tell” – “Ask” sandwich model and record your notes. Consider how you remember / keep track of your observations and other information you collect on students. Here are pearls on evaluation from experienced preceptors: - Use trainee self-assessment forms - Dictate trainee progress note (end of day, when procedures occur) Consider periodic recording of trainee case presentations Use computer printout of trainee’s patient profile Consider periodic recording of trainee case presentations (with smart phone)
- As far as the core rotation activities are concerned, here are three points to remember: Observe and provide regular, ongoing trainee feedback – on a daily basis. Use system to record trainee observations – record your observations on paper or computer; copy a sample of trainees’ patient notes. Consider mid-clerkship review, especially if deficiencies are identified – remember to use the “Ask” – “Tell” – “Ask” sandwich model and record your notes. Consider how you remember / keep track of your observations and other information you collect on students. Here are pearls on evaluation from experienced preceptors: - Use trainee self-assessment forms - Dictate trainee progress note (end of day, when procedures occur) Consider periodic recording of trainee case presentations Use computer printout of trainee’s patient profile Consider periodic recording of trainee case presentations (with smart phone)