Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Zabar final report cg


Published on

  • Be the first to comment

  • Be the first to like this

Zabar final report cg

  1. 1. Picker Institute/ACGME Challenge Grants Project Name: Emergency Medicine Resident Training in Inter-professionalism Skills Evaluating a Needs-Based Curriculum FINAL REPORT (February 29, 2007 – April 15, 2008)Date of Report: April 15, 2008Grant Number: 16Grantee Institution: New York University School of MedicinePrincipal Investigator Information: Sondra Zabar, MD Associate Professor of Medicine New York University School of Medicine 550 First Avenue, OBV D401 New York, NY 10016 (212) 263-1138 szabar@breitezabar.comCo-Investigator Information: Linda Regan, MD Assistant Professor of Emergency Medicine New York University School of Medicine
  2. 2. TABLE OF CONTENTSA. EXECUTIVE SUMMARY (ABSTRACT)........................................................................................................2B. INTRODUCTION (BACKGROUND)............................................................................................................3C. METHODS (PROJECT IMPLEMENTATION AND ADMINISTRATION) ...........................................4D. RESULTS............................................................................................................................................................9E. DISCUSSION...................................................................................................................................................14F. DISSEMINATION ..........................................................................................................................................16G. FINANCIAL REPORT ...................................................................................................................................16H. ATTACHMENTS ............................................................................................................................................17 ATTACHMENT – SAMPLE CASE AND CHECKLIST (MEDICAL ERROR).........................................................................18 ATTACHMENT – SAMPLE REPORT CARD ..................................................................................................................28 ATTACHMENT – SESSION OBJECTIVES .....................................................................................................................34 ATTACHMENT – SAMPLE POCKET CARD ..................................................................................................................35 ATTACHMENT – GOLD FOUNDATION ABSTRACT .....................................................................................................36
  3. 3. A. EXECUTIVE SUMMARY (ABSTRACT) Since the 1960’s, Emergency Medicine (EM) researchers’ efforts have worked todemonstrate the importance of patient-centered doctor-patient communication, only acknowledgingdecades later that advancing such patient-centered care will require increased and effective providereducation. Having had experience with the development and implementation of a controlled studyon the impact of comprehensive, integrated clinical communication skills curriculum on studentpatient-centered skills, the Section of Primary Care faculty at New York University School ofMedicine’s were prepared and eager to partner with Emergency Medicine faculty on this veryimportant topic. With the commitment of NYUSOM-Bellevue Emergency Medicine Residencyleadership, we created the Emergency Medicine Professionalism and Communication Training(EMPACT) Project. EMPACT aimed to improve EM resident competency in communication andprofessionalism through the development, implementation, and evaluation of new curriculum andassessment measures. Our objectives were to: 1) design, implement and evaluate patient-centeredhealthcare curriculum for all 60 EM residents; 2) evaluate predictive validity of Objective StructuredClinical Examinations (OSCEs) by assessing correlation of OSCE performance with actual residentperformance in emergent care setting for cohort of PGY2 residents (n=15); and 3) disseminate thisPatient-Centered Care educational program to EM programs nationally. We conducted EMPACT infour phases: Phase I) established baseline competency of EM interns using a 5 station OSCE; PhaseII) integrated an interactive skills-based series of five workshops focusing on interpersonal andprofessionalism skills—into monthly required EM seminar series; Phase III) conducted post-curriculum OSCE to evaluate impact of curriculum; and Phase IV) developed and implemented two“Unannounced” Standardized Patient (USP) cases. In completing all four phases of the EMPACT Project, we learned a lot about our residents,how to improve our OSCEs, and how to implement another USP project in the future. Residentsagreed that the curriculum helped them to improve on the strengths and weaknesses identified bythe OSCE. Our comparison of the residents’ pre- and post-OSCE performances has shownsignificant improvement in overall Communication, Relationship Development, and PatientEducation Skills. Also, through our USP pilot, we learned that we will need a better understandingof the system in which we practice before embarking on such an endeavor and more USP cases tobetter gauge how residents perform in reality. Even having taught communication skills in other disciplines, teaching the same skills in EMprovided rich learning opportunities for us as curriculum innovators, evaluators, and administrators.It is clear that learners need and appreciate curricula that are interactive and role model key patientcentered skills. Performance based assessment, OSCE and Unannounced Patients though timeintensive are meaningful assessment tools for both learners and programs.EM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Final Report) Page 2PI: Sondra Zabar, MDNYU School of Medicine
  4. 4. B. INTRODUCTION (BACKGROUND) Since the 1960’s, Emergency Medicine (EM) researchers’ efforts have worked todemonstrate the importance of patient-centered doctor-patient communication, only acknowledgingdecades later that advancing such patient-centered care will require increased and effective providereducation. 12 Having completed the Macy Initiative in Health Communication, a controlled study ofthe impact of comprehensive, integrated clinical communication skills curriculum on studentpatient-centered skills,3 the Section of Primary Care (PC) faculty at New York University School ofMedicine’s (NYUSOM) were prepared and eager to continue such work with the EM faculty on thisvery important topic. Drs. Linda Regan, Jeffrey Manko, and Eric Legome, directors of theNYUSOM-Bellevue Residency in EM, an integrated four-year residency dedicated to training highlycompetent emergency physicians, shared this enthusiasm and began to plan for such an initiative. Our program, entitled Emergency Medicine Professionalism and Communication Training(EMPACT), expands on previous work by assessing and improving EM resident competency incommunication and professionalism through the development, implementation, and evaluation ofnew curriculum and assessment measures. To ensure clinical competency of EM graduates indelivering patient-centered care, we incorporated both ACGME core competency requirements andseveral of the Picker Institute’s Dimensions of Patient-Centered Care into our program/researchdesign. Our objectives were to: 1. Design, implement and evaluate patient-centered healthcare curriculum for all 60 EM residents; 2. Evaluate predictive validity of Objective Structured Clinical Examinations (OSCEs) by assessing correlation of OSCE performance with actual resident performance in emergent care setting for a cohort of PGY2 residents (n=15); and 3. Disseminate this Patient-Centered Care educational program to EM programs nationally.1 Korsch BM, Negrete VF. Doctor-patient communication. Sci Am. 1972 Aug; 227(2):66-74.2 Rhodes KV, Vieth T, He T, Miller A, Howes DS, Bailey O, Walter J, Frankel R, Levinson W. Resuscitating thephysician-patient relationship: emergency department communication in an academic medical center. Ann EmergMed. 2004 Sep; 44(3):262-7.3 Kalet A, Pugnaire MP, Cole-Kelly K, Janicik R, Ferrara E, Schwartz MD, Lipkin M Jr., Lazare A. Teachingcommunication in clinical clerkships: a model from the Macy Initiative in Health Communications. Acad Med.2004; 79(6):511-20.EM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Final Report) Page 3PI: Sondra Zabar, MDNYU School of Medicine
  5. 5. C. METHODS (PROJECT IMPLEMENTATION AND ADMINISTRATION) To achieve our objectives, we conducted EMPACT in four phases. (See Figure 1. ProjectTimeline) In Phase I, we established a baseline competency of EM interns using a 5-station OSCE.Phase II, we developed an interactive skills-based series of five workshops focusing on interpersonaland professionalism skills and integrated them into required monthly EM seminar series. In PhaseIII, we conducted a post-curriculum OSCE to evaluate impact of curriculum. In Phase IV, wedeveloped and implemented two cases for the “unannounced” standardized patient (USP) project.4Figure 1. Project Timeline 3/2007 4/2007 5/2007 6/2007 7/2007 8/2007 9/2007 10/2007 11/2007 12/2008 1/2008 2/2008 Curriculum Curriculum Development Curriculum Implementation Curriculum Packaging OSCE Development (Case Individual Pre- Post- Evaluation development, SP Data Analysis Report Card Generation Remediation of OSCE OSCE Recruitment & Training) Poor Performers Generation of Program“Unannounced” Program Development (Logistics of Data Case Development “Patient” in Implementation in SP Program Implementation) Analysis computer record ER Mid- Project Production of manuscripts, abstract yearDissemination submissions, final summary reports, etc. Report Phase I - Establish baseline competency of EM interns using a 5-station OSCE In order to determine effectiveness of our curriculum, we chose to evaluate a subset ofresident performance in a pre- and post-OSCE. We wrote five cases and developed checklists thatassessed communication skills in scenarios commonly encountered by EM residents (See Table 1.OSCE Cases). The checklists used to evaluate residents’ performance included items that assessedoverall communication skills (information gathering, relationship development, and patienteducation), case-specific skills, and whether patients would recommend seeing the resident as theirphysician. Table 1. OSCE CasesOSCE Case Picker Dimension Communication SkillsInformed Consent Access; Respect for patient’s values, preferences, and Obtaining Informed Consent;Via an Interpreter expressed needs; Information, communication and Patient Education; Dealing with education Challenging PatientDisclosing a Medical Respect for patient’s values, preferences, and expressed Rapport Building; EmotionError needs; Emotional support and alleviation of fear and Handling anxietyDelivering Emotional support and alleviation of fear and anxiety; Emotion Handling; PatientUnexpected Bad Information, communication and education EducationNewsTransferring Care to Coordination and integration of care; Transition and Interdisciplinary Communication;Another Service continuity Telephone SkillsUsing the Emergency Access; Respect for patient’s values, preferences, and Dealing with Challenging Patient;Room for Primary expressed needs; Emotional support and alleviation of Emotion Handling; PatientCare fear and anxiety; Information, communication and Education education4 Kravitz RL, Epstein RM, Feldman MD, Franz CE, Azari R, Wilkes MS, Hinton L, Franks P. Influence of Patients’Requests for Direct-to-Consumer Advertised Antidepressants: A randomized controlled trial. JAMA 2005;293:1995-2002.EM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Final Report) Page 4PI: Sondra Zabar, MDNYU School of Medicine
  6. 6. The preparation for the pre-OSCE included multiple preparatory steps. We trained fivestandardized patients (SPs) to reliably and repeatedly portray their roles for the OSCE. SP trainingsessions allowed the SPs to ask questions about their character, develop the improvisational rangethat should be portrayed in their role, and practice how to consistently respond to participantreactions. Prior to the pre-OSCE, we piloted the five cases and videotaped them to fine tune thecontent of the cases and the checklists. Five EM chief residents, junior faculty, and medical studentswere assessed as the participants. After reviewing the videos of their performances, examining thedata from checklists completed by the SPs, and hearing feedback from the participants in adebriefing session, we adjusted the OSCE and checklist for clarity, timing, and realism. After makingthe appropriate adjustments to the five cases, we were ready to launch the OSCE. We conducted the pre-OSCE in three sessions. At each session, five residents went throughall five stations. All 15 PGY2 EM residents completed the OSCE. We chose to test the PGY2because we believe, developmentally, the intervention will have the most impact at this stage oflearner. 90% of the OSCEs were audio and videotaped for the purposes of assessing inter-raterreliability afterwards. Colleen Gillespie, PhD, our evaluation researcher, compiled the feedback from facultyobservers and checklist data from SPs and summarized them as both a presentation for EM facultyand report cards for each individual resident (See Attachments – Sample Report Card). The reportcard noted each resident’s performance in five core areas: 1) communication, 2) overallrecommendation, 3) ratings of ability to apply expertise, 4) specific skills across cases, and 5) overallcase-specific skill scores. One case was not reliably scored (Delivering Bad News) and so scoresassociated with that case should be interpreted with caution (details of how these scores werecalculated are included in the sample report card provided in the Attachments). Overall, we noted there was room for improvement for all the residents in their DataGathering, Relationship Building, and Patient Education Skills. Residents performed best at DataGathering, less well at Relationship Building, and worst at Patient Education. As a group they alsoscored low on Emotion Handling. Such information was also included in the report cards, whichdemonstrated how the individual performed in comparison to the rest of the participants. This dataguided us in our focus and approach to key topics covered in the curriculum. Residents told theirprogram director that they found the OSCEs enjoyable and educational. Phase II - Integrate an interactive skills-based series of five workshops —focusing oninterpersonal and professionalism skills—into monthly EM seminar series We developed curricula based on the Macy model and other literature that taught five keypatient-care tasks, including: 1) relationship development and maintenance, 2) patient assessment, 3)education and counseling, 4) negotiation and shared decision making, and 5) organization and timemanagement of EM. Our curriculum was composed of five one-hour interactive sessions thataddressed each of the core skills during the OSCE using different teaching modalities. (See Table 2.EMPACT Course Schedule) We clearly delineated cognitive, skills, and affective objectives for eachsession and highlighted them at the beginning of each session. We also created pocket cards thatincluded take-home points and a bibliography of relevant literature for each session. (SeeAttachment X for the Session Objectives) Approximately 40 residents attended each of the session,with ~10 PGY2 residents at each.EM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Final Report) Page 5PI: Sondra Zabar, MDNYU School of Medicine
  7. 7. Table 2. EMPACT Course ScheduleSession Title Date Picker Dimension Communication Teaching Method Skills1. Making Every Session 08/01/2007 Respect for patient’s Patient Education, VideotapeCount: Effective values, preferences, and Rapport Building Reenactment andCommunication Skills in expressed needs; Debriefing, Minithe Emergency Room Information, Lecture communication and education2. Interdisciplinary 09/12/2007 Coordination and Conflict Negotiation; Audiotape Trigger,Communication and integration of care; Telephone Skills Role PlayRespect Transition and continuity3. Delivering Bad News 10/03/2007 Emotional support and Emotion Handling Videotape Triggerin the Emergency alleviation of fear and from Medical TVDepartment anxiety; Information, Show, Rolling Role communication and Play between education Attending and SP4. Dealing with 11/07/2007 Access; Respect for Effective use of an Rolling Role PlayCulturally Diverse patient’s values, interpreter, Elements of between ResidentsPopulations in the preferences, and expressed informed consent and SP, Mini LectureEmergency Department needs; Information, communication and education5. Discussing Medical 12/05/2007 Respect for patient’s Emotion Handling; Videotape TriggerErrors in the values, preferences, and Patient Education; from Medical TVEmergency Department expressed needs; Dealing with Show, Role Play with Emotional support and Challenging Patient Small Groups alleviation of fear and anxiety The first session, entitled “Making Every Session Count: Effective Communication Skills inthe Emergency Room,” aimed to provide residents with tools to maximize the effectiveness of theircommunication with patients and their families. The session began with a videotaped reenactment ofOSCE case as a trigger for discussion. The session also included a PowerPoint presentation of howresidents performed in the OSCE overall and how they can improve their professionalism skills.Residents’ feedback on this first session was very positive. They noted, “I feel the hurriedatmosphere of the ER causes the communication skills to atrophy. I think this was a usefulreminder of that and an effective tool relevant to ER situations.” Our second session, entitled “Interdisciplinary Communication and Respect,” aimed to teachresidents to effectively work with the professionals around them to optimize patient care. Thissession proceeded with a general discussion of how interdisciplinary communication can be bothpositive and negative. Then, we played a re-enacted audiotape of the “Transferring Care to AnotherService” case they experienced in the OSCE, which we used as the trigger for discussion on howinterdisciplinary communication can be made better. A short lecture outlined the key steps and skillsto successful conflict negotiation and effective phone skills. Residents then participated in a role playto practice these skills. We debriefed the role play as a large group to help residents identify whatpersonal traits or attitudes are barriers for successful interdisciplinary communication. We handedout a pocket card summarizing an approach to conflict negotiation and telephone skills. A numberof residents stated that this was the first time these issues were ever addressed as part of theircurriculum. In particular, they said, “Good suggestions on how to approach multidisciplinarycommunication. Short handout with key points helpful. Tape [was] very pertinent and important.”EM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Final Report) Page 6PI: Sondra Zabar, MDNYU School of Medicine
  8. 8. The third session, entitled “Breaking Bad News in the Emergency Department,” aimed toimprove residents’ effectiveness in their delivery of bad news and provide residents with facts aboutpost-death procedures. The session began with the viewing of a trigger video clip from the Foxtelevision series, “House,” where a patient is abruptly given an AIDS diagnosis by the maverick, Dr.Gregory House. This led to a conversation about what contributes to the sensitivities and difficultiesof delivering bad news, regardless of how the residents may perceive the severity of the news to be(e.g. broken limb, new diagnosis of disease, or death of a loved one). Then, the residents directed arolling role play between an SP and Dr. Regan, who had to break the news of a positive HIVdiagnosis. The roll play was stopped a few times midstream to allow for a discussion of possiblestrategies to better manage the situation. The session concluded with the key take-home points,including protocol on how to follow-up on death notification, which residents took with them onpocket cards. The residents notes that this topic "...can be fairly dry, has been done so much in medschool, BUT this was a very strong revisiting of this hard issue.” In particular, they said the sessionwas “excellent because it was DYNAMIC… well prepared, very interactive. The role play was verywell done." The fourth session, entitled “Dealing with Culturally Diverse Populations in the EmergencyDepartment,” aimed to improve residents interactions with culturally diverse patients andunderstand appropriate use of interpreters in the ED. The session began with a discussion of thechallenges of providing cross-cultural care, including how different health beliefs affect patient andprovider behavior and how language can act as the most apparent barrier. The conversation turnedto the challenge of working with various kinds of interpreters and strategies to overcome commonerrors. During this session, a pair of Bengali-speaking SPs participated in a role play with Dr. Regan,who demonstrated a bad version. Residents were asked to strategize on how to improve theinteraction and asked to come up and interact with the sp in front of the group. We used a RollingRole Play as the educational strategy for this session. We concluded the session with a summary onhow to use interpreters better. Residents again took home pocket cards that reviewed the key skills.They enjoyed the use of small group role play and said it was "a refreshing approach to this topic." The fifth session, entitled “Medical Errors in the Emergency Department,” aimed to improveresident’s effectiveness in their disclosure of medical errors. This session began with a viewing of avideoclip from the NBC television series, “Scrubs,” where a resident debates whether or not toexpose a potential medical error he believes was committed by his friend and colleague. Whilecomical, this clip helped the residents to begin broaching the difficult topic. Then, the sessioncontinued with a discussion of frequent barriers to the disclosure of medical errors in general, as wellas specific to the ED. Residents were then given a checklist of items to follow which representedcommon good practice for this sensitive topic. After explicitly discussing the 5Ws (Who, What,Where Why, and When), the session proceeded with a skills practice. Each group of three to fiveresidents were given a scenario where one resident played the patient and another played theresident who had to deliver the news about one of three medical error scenarios. Each group wasfacilitated by a faculty member. The rest of the group observed and scored the scenario with achecklist, similar to that which the SPs would use during the OSCE. Each small group reportedlarger group the key learning points from their scenario. The session ended with the viewing of afinal clip from “Scrubs,” where everyone is relieved to find out an error did not occur and a re-emphasis on the take-home points for the session. Phase III - Conduct a post-curriculum OSCE to evaluate impact of curriculum. Two months following the final EMPACT session, we held the post-OSCE. For comparisonpurposes, we used the same five cases as the pre-OSCE. Due to the availability of the SPs, however,we needed to train new SPs for four of the five cases. However, we purposefully chose SPs whomEM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Final Report) Page 7PI: Sondra Zabar, MDNYU School of Medicine
  9. 9. we have worked with in the past and found to be reliable raters. Consequently, we believe the overallintegrity of the OSCE remains the same. The post-OSCE was held in three sessions, with approximately five residents attending eachsession. All 15 EM PGY2s participated in the post-OSCE and completed all five stations. Again, forinterrater reliability purposes, each station was videotaped, with the exception of the Transfer Case,which was audio taped. Colleen Gillespie and Tavinder Ark, MSc, our research associate, collected feedback fromfaculty observers, checklist data from SPs, and resident satisfaction data relating to both theEMPACT OSCE and curriculum. They summarized all data into report cards for each individualresident, this time with a comparison of how their performance differed in the two OSCEs. Thereport card reported each resident’s comparative performance in five core areas: 1) communication,2) overall recommendation, 3) ratings of ability to apply expertise, 4) specific skills across cases, and5) overall case-specific skill scores. The comparative data of the pre- and post-OSCE are describedlater in the Results section. Phase IV - Develop and implement two cases for the “unannounced” standardizedpatient (USP) project. The USP portion of EMPACT, was both exciting and educational. To our knowledge, basedon an extensive literature search in PubMed and Medline, the use of USPs in emergency clinicalsettings had not been done prior to our attempt. Despite posing us with many labor-intensivechallenges, with full prior consent of residents, support of department and hospital leadership, andapproval from our IRB, we launched the USP program in December 2007 and assessed 12 residentsthrough 17 successful USP encounters in the ER. For comparison purposes and to protect our SPs, we chose to use the Medical Error andRepeat Visitor cases for the USP visits, as they required non-invasive interventions by the residents. Having obtained verbal confirmation from Medical Records, Registration, EM Nurses, EMAttendings, and the radiologists, we were poised to begin this aspect of the project. As the USPs inboth the cases were supposed to have visited the Bellevue ER before, both cases required the entryof previous medical notes, x-rays, MRIs, and labs in the medical record system. We obtainedspecified Medical Record Numbers for the USPs. However, the challenges of this effort soonbecame apparent. The rate limiting step in setting up the Medical Error case was the time frame allowed byMISYS, the medical records system, to enter prior visits into the record history. Because the USPwas supposed to have visited the ER two days prior to the actual USP visit, we needed a visit to beopened two days prior in real time. The system would not allow us to enter future visits. This meantthat the Bellevue Hospital EM Admitting needed to be ready to open the visit when we asked twodays prior to the actual USP visit. This also meant that the PACS team, the group that handled allradiology related issues, had to be ready to upload the X-ray images and reports onto the systemonce the prior visit was opened. Because this was a voluntary effort on the part of the Admittingand PACS, it took a few tries to come up with an efficient system for getting all the requiredinformation adequately noted in the USPs fictitious medical records prior to the actual USP visit. The main challenge of the Repeat Visitor case was the manipulation of the MRI images.Based on the original version of our case, the USP was supposed to have visited the Bellevue ERtwice in the past and have taken MRI images here. In order to have the MRI images reflect the casedetails of each visit (e.g. dates, patient name, etc.), we needed to edit more than 50 images per visit.We consulted Sectra, the company that services our PACS system, who offered to write us aprogram that would quickly do so for $12,000. Since this was not possible given our financialsituation, we ended up editing the USP case. In the new version, the USP visited another ER in NewEM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Final Report) Page 8PI: Sondra Zabar, MDNYU School of Medicine
  10. 10. York City two times and got an MRI at another location. The USP then brought the MRI report tothe actual USP visit at Bellevue. The third most prominent challenge of the USP project resulted from our need to limit thenumber of informed people in the ER, the unpredictability of the ER, and the assignment of theresidents to the USP case on a given day. We tried to limit the number of people in the ER whoknew that a USP was present to avoid detection. Although we tried our best to have the USP triagedexactly to where the targeted resident was supposed to be working on the given day, our efforts wereoften thwarted by eager medical students, rotating orthopedic residents, or unexpected schedulechanges. During a few of our scheduled visits, the USPs were mistakenly examined by another careprovider while the target resident was called away to see a more acutely ill patient. The attendingmay have known about the USP, but at times was engaged in the care of another patient when non-targeted personnel elected to see the USP. After 29 attempts, we successfully evaluated 17 of the 30 planned visits (five residents werevisited by both types of USPs, which accounted for ten of the visits). We audio taped ~71% of theencounters (12/17), which we will use to establish intra- and inter-rater reliability. Following eachvisit, we videotaped the USPs as they debriefed the entire experience and completed the checklists.As the last USP visit was just completed on April 8, 2008, a comprehensive comparison of the USPand OSCE performances is still pending.D. RESULTS The OSCEs assess residents’ clinical skills in two major areas: 1) Communication Skills and2) Case-Specific Skills. The Communication Skills describe residents’ ability in informationgathering, relationship development and patient education skills. The Case-Specific Skills describethe residents’ ability to perform skills specific to each case. They are divided into five broadcategories: 1) managing a difficult case, 2) accountability, 3) delivering bad news, 4) patient educationand 5) treatment plan and management. For the EMPACT OSCE and USP visits, Communication and Case-Specific Skills questionsare rated by the SP on a 3-point scale of “not done” (resident did not perform the task at all),“partially done” (the resident attempted the task, but did not do it entirely correctly), or “well done”(the resident performed the task and did it correctly). In addition, residents’ were rated by the SPson the degree to which they would recommend this doctor to a friend based on their interpersonalskills and expertise on a 4-point scale (1= Not recommend and 4= Highly Recommend). Residents’Communication and Case-Specific Skills are calculated as the percent of items rated as “well done”across all cases. The overall recommendation rating was based on interpersonal skills and expertisewas calculated across all cases as a mean average on a 4-point scale. These score was calculatedacross all 5 cases. A pre and post comparison was conducted. For the USP visits, this score wascomputed only across the repeat visitor case and broken wrist (medical error) and compared to thepre and post of only these two cases. D1. Resident Experience of EMPACT Data on residents’ exposure to actual clinical situations similar to the OSCE cases highlightthe importance of having an opportunity to practice low frequency clinical situations: only 29%reported encountering a situation involving giving bad news since the pre-curriculum OSCE andslightly less than half (43%) reported exposure to a clinical situation involving a medical mistake.Despite evidence reported below that residents made substantial improvements from pre- to post-curriculum in some core clinical areas, from more than a third to close to half of residents reportedEM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Final Report) Page 9PI: Sondra Zabar, MDNYU School of Medicine
  11. 11. that their performance on the post OSCE was “about the same” as their performance on the preOSCE (depending on the case, % ranged from 36% to 50%). Most agreed that the OSCE helpedthem identify their strengths and weaknesses (60%) and provided a good cross-section of cases(74%). However, some skepticism of the value of OSCEs was also apparent as just over half didnot think that the OSCEs taught them something new (54%) or was a fair evaluation of their skills(60%). When asked in an open-ended manner to describe what was most helpful about EMPACTmost focused on the OSCE (perhaps reinforced by having just completed the post OSCE!), focusingon practice (“repeated exposure to clinical scenarios”) and on being able to assess and reflect onone’s skills (“recognizing my triggers for what is a problem for me;” “self reflection about myweaknesses,” “the situations are a good reflection of what we see in the ED and they highlight someof the weaknesses we have in dealing with difficult situations. I know I tend to make the samemistakes over and over again.”). Several residents simply said that the EMPACT “curriculum” wasthe most helpful aspect of EMPACT overall. D2. Impact of the Curriculum: Pre- vs. Post-Curriculum OSCE Results Comparison of the pre- and post-curriculum OSCEs showed significant improvement inresidents’ overall Communication Skills (pre=53.4% SD 14.9% vs. post=65.5% SD 11.5%;p=0.003). In particular, they improved on overall Relationship Development skills (pre=49.2% SD21.5% vs. post=59.8% SD 17.8%; p=0.025) and especially in their overall Patient Education skills(pre=31.6% SD 15.1% vs. post=57.0% SD 15.2%, p<.001). In terms of residents’ case-specific skills, significant improvement from pre- to post-curriculum was seen in the Repeat Visitor case (pre=38.7% SD 18.1% vs. post=73.3% SD 16.7%,p<.001) and close to significant improvement in the Bad News case (pre=54.0% SD 15.5% SD 22.1%; p=.066). SPs rated residents more highly in terms of the degree to which they would recommendthem (using a 4-point scale) for their interpersonal skills (pre=2.84 SD .58 vs. post=3.09 SD .41;p=.066) and for their medical expertise (pre=2.90 SD .48 vs. post=3.19 SD .29; p=.014).EM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Final Report) Page 10PI: Sondra Zabar, MDNYU School of Medicine
  12. 12. Impact of EMPACT: Pre-Curriculum vs. Post-Curriculum OSCE Communication Scores (n=15) 80% 74% Pre Post 70% 70% p<.01 65% p<.05 60% p<.001 60% 57% 53% 49% 50% % Well Done 40% 32% 30% 20% 10% 0% OVERALL Information Gathering Relationship Patient Education COMMUNICATION DevelopmentEM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Final Report) Page 11PI: Sondra Zabar, MDNYU School of Medicine
  13. 13. Impact of EMPACT: Pre-Curriculum vs. Post-Curriculum OSCE Case Specific Scores (n=15) 80% p<.001 73% Pre Post p<.10 70% 67% 59% 60% 54% 54% 53% 54% 53% 50% % Well Done 44% 40% 39% 30% 20% 10% 0% Bad News Interpreter Broken Wrist Repeat Visitor Transfer (Medical Error)EM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Final Report) Page 12PI: Sondra Zabar, MDNYU School of Medicine
  14. 14. Impact of EMPACT: Pre-Curriculum vs. Post-Curriculum Recommendation Ratings (n=15)Highly 4Recommend Pre Post p<.10 p<.01 3.19 3.09 3 2.90Recommend 2.84Recommendw 2ReservationsNot 1Recommend Recommendation - Interpersonal Skills Recommendation - Applic of Expertise D2. Comparison of OSCE and USP Scores A major goal of this project was to begin to explore how residents’ performance in an OSCE relates to their actual clinical performance, at least as assessed by an USP. Given that the pre-OSCE took place in July, the post in March, and the USP visits anytime between mid-January and early April, scores generated from the USP visits were compared with both pre- and post-curriculum OSCE scores. Although, we expected USP scores to be more highly correlated with post-OSCE scores since they generally occurred closer in time. Twelve residents had at least one USP visit and 5 residents were visited by both USPs (Repeat Visitor and Medical Error). We report correlations for both sets of data in order to maximize our sample size (including all 12 residents by reporting whatever USP data is available for each resident be it one or two visits) and maximize our sample of actual clinical performance (including only those 5 residents from whom we have two samples of performance data, i.e., two USP visits). Correlations between OSCE and USP Scores At least 1 USP Visit (n=12) 2 USP Visits (n=5) USP Scores Pre OSCE Post OSCE Pre OSCE Post OSCE Overall .70 .17 .83 .53 Communication (p=.011) (p=.600) (p=.088) (p=.379) Skills Overall Case .63 .17 .64 .85 Specific Skills (p=.029) (p=.598) (p=.249) (p=.066) EM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Final Report) Page 13 PI: Sondra Zabar, MD NYU School of Medicine
  15. 15. Results suggest that the USP scores are strongly correlated with the pre-OSCE scores forboth residents with one or more visits and for only those residents with an adequate sample ofclinical performance (both Repeat Visitor and Medical Error USP visits). However, it is only amongthose with both USP visits that we see strong correlations with post OSCE scores. It may be thatresidents’ performance on the pre-curriculum OSCE best represents how they are in actual clinicalpractice while their performance on the post-curriculum OSCE was more reflective of how theyperform when being evaluated on the basis of clear criteria (as shared through the 5-sessioncurriculum). These exploratory results also demonstrate the importance of including multiplesamples of performance – one USP visit is probably not sufficient to obtain a true and accuratepicture of physician skills. We assessed two additional dimensions of clinical performance: patient-centeredness (e.g.,fully explored my experience of the problem, took a personal interest in me, earned my trust,acknowledge impact of situation on my life) and the degree to which the resident “activated thepatient” (e.g., helped me to understand the nature and causes of my condition, helped me find outabout the different medical treatment options available, made me feel confident I can figure out newsolutions if my situation changes) (Hibbard ref). There is increasing evidence that these skills, alongwith core communication and case-specific skills, are associated with important patient outcomes.Therefore, we examined correlations between average scores residents received from USPs on theseitems and their OSCE scores and found, as above, that both pre and post OSCE communicationand case-specific skills were strongly (albeit not significantly) and positively correlated with patientcenteredness and patient activation. 2 USP Visits (n=5) Overall Communication Skills Overall Case Specific Skills USP Scores Pre OSCE Post OSCE Pre OSCE Post OSCE Patient .56 .78 .79 .84 Centeredness (p=.326) (p=.120) (p=.112) (p=.078) Patient .68 .60 .85 .84 Activation (p=.202) (p=.282) (p=.070) (p=.078)E. DISCUSSION There are many things we can learn from the development and implementation of a newcurriculum designed to help residents with their communication skills. Even having taughtcommunication skills in other disciplines, teaching the same skills in EM provided rich learningopportunities for us as curriculum innovators, evaluators, and administrators First, residents portray an outward confidence about their communication skills, whichlacked grounding in their assessment levels. Despite their relaxed attitude about the OSCE cases,the data showed that they had difficulty with some of the scenarios. This came as a great surprise tosome, though the majority already knew there was some deficiency when questioned. Residentreported they learned that: 1) without listening to what patients have to say about their condition, itis difficult to hear what the patient is actually trying to convey, without appropriately providingpatient education, quality of care may be compromised, 2) without communicating effectively withother disciplines, it will be difficulty to coordinate care, and 3) without demonstrating empathy,kindness, patient satisfaction is hard to achieve. Having the opportunity to step back from the flurryEM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Final Report) Page 14PI: Sondra Zabar, MDNYU School of Medicine
  16. 16. of activities in the EM, residents were able to acknowledge their respective shortcomings incommunication skills and commit to improving them for their patients. Second, residents received their feedback in a much more affirmative manner than we hadhoped. We are struck by their positive feedback for the “much needed” education on “basic skills”that are essential for success as EM physicians. Their enthusiasm for this education is surprising andgladly received. They have been instructive in helping us to design our curriculum so that they canget the most out of the experience for their practical day-to-day use. Third, as measured by a reliable and valid OSCE, the EMPACT project shows that a focusedcurriculum, with five one-hour group interactive sessions on communications and professionalcurriculum, can significantly improve residents’ rapport building and patient education skills. Theseskills were tested months after the curriculum. Our curriculum is unique, not for its topics, butbecause of the variety of educational methods we incorporated (i.e. role play, modeling withstandardized patients, discussion triggered by “TV medical clip” and reenactments of real residents’performances). This approach is highly acceptable and engaging to residents, as evidenced by theirfeedback. Fourth, through the USP aspect of this project, a novel endeavor, we have shown that thismethodology is feasible and acceptable to residents, program directors, and faculty and hospitaladministrations. As noted by the program director, this project has already brought added value tothe resident learning and patient care. By informing the residents that USPs would be visiting themin the ED, the residents seemed to perform at a higher level, not knowing which patients might beevaluating their performance and what measures were being evaluated. One resident commentedthat when he thought a patient was a USP, he washed his hands more frequently, thinking that handwashing was the metric we were evaluating. A faculty member noted that when one resident thoughthe had identified a USP, he seemed more empathic and professional when discussing the dischargeplan and follow-up care. Clearly, the patients also benefited from the study, as higher professionalstandards, including stricter adherence to Joint Commission Safety Initiatives were being executedby the residents to more patients, not only the USPs. We must further analyze our USP results, debriefing tapes, and audio tapes to understandwhat additional information we can learn about our residents’ skills using this innovativemethodology. The fact that our post-OSCE results did not fully match the residents’ USPencounters further supports the need to perform larger USP studies with multiple cases in order tobetter understand the degree to which OSCEs reflect real world skills. It is our hope that we can inwhat ways OSCEs can predict real life performance in order to enable us as educators to use themas efficient and effective tools to help learners become expert physicians. With the ACGME recently placing greater importance on evaluation of patient outcomesand its linkage to medical education, we believe that our project is representative of a new way toassess real-time resident physician performance. As program evaluators working towardenhancement of curricula that better meet patient needs, this project has contributed much to ourlarger efforts. The data collected from these OSCEs have been incorporated into Database forResearch on Education Academic Medicine (DREAM), an initiative of our Research on MedicalEducation Outcomes Unit (ROMEO), which enables long-term, longitudinal assessments ofparticipant performance both in residency and beyond. Further comparison of OSCE evaluationswith USP encounters will enable educators to determine whether or not these commonly usedevaluation tools actually mimic real practice. The current OSCE data will be assessed in conjunctionwith future evaluations and patient outcomes. We eagerly await results of a larger trial. Lastly, this collaboration between NYUSOM Primary Care and Emergency Medicine hasenabled us to further heighten the overall abilities of NYUSOM faculty to teach and communicatewith each other and to our residents. Additionally, we believe this curriculum also provided an addedEM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Final Report) Page 15PI: Sondra Zabar, MDNYU School of Medicine
  17. 17. value as a faculty development opportunity. Faculty members in the Emergency Department havegained a standardized approach to teaching and assessing communications skills after participatingor playing facilitative roles in the curriculum.F. DISSEMINATION We have already begun to share our methods with other departments and institutions.Owing to the success of the EMPACT OSCE, the Gastroenterology fellowship used our cases fortheir OSCE held on October 6, 2007. Their use of our communication skills checklist will enable usto compare performance across disciplines and levels of training. They are planning a second OSCEfor additional fellows in May 2008. Additionally, current plans are under way within the Departmentof Emergency Medicine at Johns Hopkins to apply for funding to support the use of USPs inevaluation of curriculum focusing on disaster education. In terms of publication, the Arnold P. Gold Foundation, which promotes and affirms morecompassionate medical care and caregivers, accepted our abstract (“A Curriculum in Patient-Centeredness for Surgery and Emergency Medicine Residents: Establishing the Baseline.” M.Hochberg, S. Zabar, L. Regan, R. Laponis, R. Richter, A.L. Kalet), for presentation at the GoldFoundation Symposium, How Are We Teaching Humanism in Medicine and What is Working?,which was held on September 27-29, 2007, Chicago, IL. Future plans include submission toAcademic Emergency Medicine, the journal of the Society of Academic Emergency Medicine as wellas to the national Council of Residency Directors (CORD) meeting for Emergency Medicine whichis held annually.G. FINANCIAL REPORT The Financial Report will be provided by the NYUSOM Sponsored ProgramsAdministration under separate cover.EM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Final Report) Page 16PI: Sondra Zabar, MDNYU School of Medicine
  18. 18. H. ATTACHMENTS a. Sample Case and Checklist b. Sample Report Card c. Session Objectives d. Sample Pocket Card e. Sample Feedback f. Dissemination i. Gold Foundation AbstractEM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Final Report) Page 17PI: Sondra Zabar, MDNYU School of Medicine
  19. 19. Attachment – Sample Case and Checklist (Medical Error) STATION OVERVIEW OBJECTIVES To test the resident’s ability to: 1. Admit an error has been made 2. Be empathic 3. Address patient concerns surrounding an error LOGISTICS Personnel: Standardized patient, male, 32 y.o., dressed in regular clothing, sitting in chair. Station Materials: • Resident instructions • SP Instructions • SP evaluation forms • Faculty evaluation forms Room Arrangement: • Station signs • Chair (2) • Exam tableEM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Progress Report) Page 18PI: Sondra Zabar, MDNYU School of Medicine
  20. 20. RESIDENT INSTRUCTIONS PATIENT Name: John McCoy INFORMATION Age: 32 REASON FOR ENCOUNTER • John McCoy came to the ER 2 days ago complaining of right wrist pain after falling while rollerblading near Washington Square Park. • At that time, his hand x-ray was MISREAD by a resident as normal and he was sent home with an Ace bandage and some ibuprofen. • The Radiology Attending re-read the x-ray and found a non-displaced, non-intra-articular right distal radius fracture. • He presents today to the ER after having been called back. YOUR ROLE ER Resident YOUR TASKS 1) See the patient, explain what has occurred, and develop a plan.EM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Progress Report) Page 19PI: Sondra Zabar, MDNYU School of Medicine
  21. 21. STANDARDIZED PATIENT INSTRUCTIONSTHE SCENARIO Your name is John McCoy and you are 32 years old. 2 days ago you were rollerblading in Washington Square Park prior to when your shift started for work at a restaurant (you work as a waiter at the Union Square Cafe). You fell and hit your outstretched right hand on the pavement. Your right wrist hurt a lot and you were afraid that it might have been broken. This was particularly concerning as you work as a jazz pianist occasionally. You went to the Emergency Room and after waiting for 4 hours, finally saw a doctor. They took some x-rays and told you it was just a sprain. You got some pain drugs (ibuprofen) and a bandage to wrap your wrist. You were told to rest your wrist, use ice, and keep it wrapped and raised as much as possible. Because of the wait at the ER, you had to have someone cover for you at work. Because you don’t get sick pay, you decided to work yesterday even though you were in pain. This morning, you got a call from a nurse instructing you to return to the ER as the doctors had some information about your wrist. You again got someone to cover for you (although you still won’t get paid) in order to go back to the ER today. Today, the pain in your right wrist is about 5/10 (10 being the worst pain in your life) and it only gets worse when you bend it back or press on it. The swelling has gone down from 2 days ago and it seems like it is slowly getting better, despite having used it yesterday at work. CHARACTER Objective: • To understand what has occurred and know when DESCIRPTION you can return to work Obstacles: • You are upset about missing work as you are having a tough time making ends meet. Tactics: You are initially somewhat agitated as you are missing work again When you hear the news of the mistake you become further agitated If the resident is empathic, apologizes, and is helpful, you calm down a little. If, however, the resident is at all defensive, argumentative or unhelpful, then your agitation continues to increase.EM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Progress Report) Page 20PI: Sondra Zabar, MDNYU School of Medicine
  22. 22. SINCE YOU Since you left the ER 2 days ago, you have been trying to do what the LEFT THE ER doctor told you to do: rest it, use ice, compress it with the bandage and keep it elevated. You did, however, go to work yesterday after taking a few ibuprofen (Advil) tablets and a strong gin and tonic in order to minimize the pain. You got thru your shift without too much trouble and were able to compensate using your left hand more often than usual. Today, you still have some pain, but the ibuprofen is helping. PERSONALITY You tend to be a little dramatic. When you are happy, you border on gushy and when you are upset, you can get angry. This is partly due to the fact that your financial situation is slightly unstable and it can put you on edge at times.CURRENT LIFE You live with a roommate in the East Village. SITUATION You have no children. You work as a waiter at the Union Square Cafe and play jazz piano intermittently with various local groups. You are still hoping to make it as a pianist, but it hasn’t worked out that well so far.PAST MEDICAL None. You are otherwise very healthy and active.AND SURGICAL HISTORY FAMILY Your mother and father are both living in Ohio. They are healthy as HISTORY far as you know. You have one brother who is healthy and married living in Ohio as well. SOCIAL You smoke ½ pack a day for the past 10 years. HISTORY You drink alcohol at least 3 times per week, usually having 2-3 drinks each time. You do not use recreational drugs. You are sexually active with a girlfriend you have had for the past 6 months. You use condoms for protection. You are eating and sleeping well and staying active by rollerblading and going to the gym occasionally. MEDICATIONS Ibuprofen (Advil) – 2 tablets every 4 hours for pain ALLERGIES None THE When the Resident knocks and enters the room, you are sitting in a ENCOUNTER chair in the exam room talking with a colleague trying to get someone to cover for you as you are missing work. You are upset interruptingEM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Progress Report) Page 21PI: Sondra Zabar, MDNYU School of Medicine
  23. 23. the person on the other end of the phone line and end the conversation about 20-25 seconds after the resident enters the room. When you hang up, you are still upset having had to miss work for the second time this week. You show this by making eye contact with the resident, occasionally breathing deeply and audibly, and have aggravated tone to your voice. You are testy and confrontational the entire interview and occasionally interrupt the resident to voice your frustration. If asked in an open-ended way why you are here, state: “You guys called me. I was here a couple days ago about my wrist, so I assume it’s about that.” With respect to your wrist- Any pain? – “A little, but the Advil helps.” How bad is the pain? – “About 5 out of 10” Any pain with movement? – “Only when I bend it back” Any swelling? – “It’s gotten a lot better.” Any tingling or loss of sensation? – “No” Any redness? – “No” Any tenderness? – “It hurts a little when I push on it.” In general currently: How have you been? – “Fine, I guess. My wrist hurt a bit during work yesterday, but I got through it. But I’ve missed two days because of this stupid thing.” If/when you are told a mistake was made (i.e. someone read the x-ray of your wrist incorrectly and you actually have a bone fracture) regardless of where it occurs in the interview, take a moment to let it set in and then at first become upset. Raise your voice, but do not shout, look the Resident straight in the eye, and impatiently tap your finger on the desk or table to underline your frustration. State: “So my wrist is broken?” “This is so annoying.” “I mean, what’s going on here? I had to miss two days of work because of this.” If then the Resident acknowledges the mistake, states that he/she is sorry that it happened/empathizes, you still remain angry and state in a slightly aggressive tone: “Oh man. I knew it. I knew it was something bad. ThisEM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Progress Report) Page 22PI: Sondra Zabar, MDNYU School of Medicine
  24. 24. always happens to me. Well, will there be any long-term damage?” When you realize the long term damage will be nil or minimal, you are only a little relieved. State in a somewhat frustrated way: “Why did this happen? What if this was something really serious? I mean, my God, does this happen all the time?” Whatever the resident’s response is state: “Well, don’t you think this is a bad system here?” If the Resident remains apologetic and non-confrontational, you calm down a little and ask: “Well, when can I go back to work?” If the Resident acknowledges that a mistake was made, but then becomes defensive, does not empathize or say he/she is sorry, or makes up a bizarre story -> get more upset: “I mean, me missing work today would have been totally unnecessary right? If you guys actually did your job, I wouldn’t have had to come down here.” “I knew I shouldn’t have come to his ER.” If the resident asks if they can write you a note, state sarcastically: “A note? What I am I going to do with a note?” Whenever the Resident changes course and becomes more apologetic/empathic, react accordingly. Adequately challenge the resident. You are upset for a multitude of reasons: losing work pay, being in pain, losing faith in your health care provider, and not being able to play piano. If you feel the resident is making a genuine effort to address your concerns, is empathic and non-confrontational, become less angry, but maintain a baseline of annoyance and frustration. If the resident ever becomes dismissive/confrontational or you don’t feel supported, become more upset. Towards the end of the interview, regardless of the Resident’s reactions, become calm. Your motivation for doing this is as follows: If the Resident has admitted the mistake and acted appropriately, you are satisfied. If the Resident has done poorly by not admitting the mistake or making fabrications you become withdrawn contemplating a lawsuit: (Please note: Do not mention lawsuit, litigation, suing, or anything relating to malpractice unless the Resident brings it up - this is purely an internal cue for you to help you act out the character). If the latter is the case – partially crossEM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Progress Report) Page 23PI: Sondra Zabar, MDNYU School of Medicine
  25. 25. your arms, rest your head on one hand, and avoid eye contact. Once you have calmed down a little, state: “Well, I came all the way down here. Now what?” CHALLENGES • Admit that an error was made FOR THE • Regain patient trust RESIDENT CUES FOR THE Non-verbal 1 At the beginning of the interview, eye contact RESIDENT with occasional audible breathing. Verbal 2: State: Why exactly was I called back? -> Resident to verbally acknowledge your concern and explain reason Verbal-Non- Express anger (state that you are upset, raise Verbal 3: your voice, look at the Resident in angry and accusatory fashion, underline your verbal comment with tapping your fingers on the table) -> Resident to verbally acknowledge your anger/being upset and label it as understandable Verbal-Non- Calm down in last part of encounter; if Resident Verbal 4: acted appropriately: calm down (e.g., appear more relaxed in your posture and voice); if Resident acted inappropriately: withdraw (e.g., cross arms, speak in short sentences, etc). State: “Well, I’m here. What do we do now?”TIMING Initially: You are already a little upset. Ongoing: If the Resident is empathic/truthful/straightforward, become more and more calm. If the Resident is defensive/evasive/making up bizarre stories, become more and more upset. 2 minute warning: Begin to calm down because the Resident is acting appropriately or withdraw because the Resident is acting inappropriately. State: “What do we do now?”EM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Progress Report) Page 24PI: Sondra Zabar, MDNYU School of Medicine
  26. 26. Evaluator’s ChecklistCOMMUNICATION Not Done Partially Done Well DoneInformation GatheringElicited your responses using appropriate Impeded story by asking Used leading/judgmental Asked questions one at a timequestions: leading/judgmental questions questions OR asked more than without leading patient in their AND more than one question at one question at a time responses No leading questions a time Only one question at a timeClarified information by repeating to Did not clarify (did not repeat Repeated information you Repeated information andmake sure he/she understood you on an back to you the information you provided but did not give you a directly invited you to indicate provided) chance to indicate if accurate whether accurateongoing basis Did not interrupt directly BUT cut Did not interrupt AND allowedAllowed you to talk without interrupting Interrupted responses short by not giving time to express thoughts fully enough timeRelationship DevelopmentCommunicated concern or intention to Did not communicate intention to Words OR actions conveyed Actions AND words conveyed help/concern via words orhelp intention to help/concern intention to help/concern actionsNon-verbal behavior enriched Non-verbal behavior was Non-verbal behavior Non-verbal behavior facilitated negative OR interfered withcommunication (e.g., eye contact, posture) demonstrated attentiveness effective communication communicationAcknowledged emotions/feelings Acknowledged & responded to DID NOT acknowledge Acknowledged emotions/feelings emotions/feelings in ways thatappropriately emotions/feelings made you feel better Made comments and Made judgmental comments OR Did not express judgment but didWas accepting/non-judgmental facial expressions not demonstrate respect expressions that demonstrated respectUsed words you understood and/or Consistently used jargon Sometimes used jargon AND did Explained jargon when used, ORexplained jargon WITHOUT further explanation not explain it avoided jargon completelyEducation and CounselingAsked questions to see what you Asked if patient had any Assessed understanding by Did not check for understanding questions BUT did not check for checking in throughout theunderstood understanding encounter Gave confusing OR no Information was somewhat clear Provided small bits of information explanations which made itProvided clear explanations/information impossible to understand BUT still led to some difficulty in at a time AND summarized to understanding ensure understanding informationCollaborated with you in identifying Told patient options, THEN Told patient next steps THEN Told patient next steps/plan mutually developed a plan ofpossible next steps/plan asked patient’s views action EM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Progress Report) Page 25 PI: Sondra Zabar, MD NYU School of Medicine
  27. 27. ADDRESSING MEDICAL ERRORAccountabilityDisclosed error Did not directly disclose the error• Direct (used the words “error” or Did not directly disclose the error (there was a “problem”) NOR (there was a “problem”) OR Directly disclosed the error upfront “mistake”) directly disclosed late in the was the explanation upfront interview• Prompt disclosurePersonally apologized for the error (“I am Did not apologize for error NOR Apologized for the error OR for Apologized for the error AND for for the inconvenience it causedsorry that this happened) the inconvenience it caused you the inconvenience it caused you youShared the cause of the error (i.e., Acknowledged issue with system Did not acknowledge issues with Acknowledged issue with system BUT was dismissive/Explained issues with system) system AND was genuine in addressing it condescending Took no personal responsibility Took a general responsibility as for your present situation (e.g., Took a personal responsibility forTook responsibility for situation assigns your problem to other part of the department for your your situation (“I will…) present situation person/department) Made general suggestion forIdentified future preventative strategies Did not address how situation improvement (e.g., “We’ll look Offered specific strategies forto prevent situation from happening again would be prevented in future into it,” “I’ll make a note of it to potential improvement of system my Attending”)Managing a Difficult Situation Became defensive/ Became defensive/ Remained calm AND did notAvoided assigning blame argumentative AND assigned argumentative OR assigned mention blame someone else blame to a person/department blame to a person/department Maintained a high level ofMaintained professionalism by Unable to control emotions, Attempted to control emotions professionalism in handling your became dismissive and (e.g. was somewhat dismissivecontrolling emotions specific situation, did not show condescending or condescending) anger or frustration Delivering Bad NewsPrepared you to receive the news: Entered room in a manner Entered room in a manner Entered room in a manner befitting unfitting the news AND • Entered room prepared to deliver news physically situated him/herself unfitting the news OR physically the news AND physically situated situated him/herself far from you him/herself close to you • Ensured sufficient time and privacy far from youAssessed your readiness to receive news: Attempted to deliver warning shot, BUT inappropriately (does Gave you a well-timed warning • Gave warning shot (e.g., “I have No warning shot not pause for your assent OR shot some good and bad news for you…”) warning shot too long)Gave you opportunity to emotionallyrespond: Responded inappropriately to Allowed you to emotionally Allowed you to express your your emotional reaction (no respond (vent) BUT did not feelings, fully giving you the feeling • Remained sensitive to your venting of opportunity to vent, cut you off, address/acknowledge response you were being listened to before shock/anger/disbelief/accusations became defensive) before moving on moving on • Attended to emotions before moving on Acknowledged your feelingsDirectly asked what you are feeling: “What (e.g., “I see that you are Did not ask specifically “What Specifically asked you “What are upset…”) BUT did notare you thinking/feeling?” are you thinking/feeling?” you thinking/feeling?” specifically ask you to name your emotions Offered specific next steps (e.g.Provided appropriate “next steps” Did not offer next steps AND Offered only general next steps Orthopedics is going to fit you for a (e.g., I’ll be calling Ortho) OR• Orthopedics for immediate care evaded response as to what will promised to “ask the attending” cast) AND informed you of long happen long-term term care needs (e.g., unable to• What to expect long-term for next steps use arm for 6 weeks) EM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Progress Report) Page 26 PI: Sondra Zabar, MD NYU School of Medicine
  28. 28. Would you recommend this doctor to a friend for his/her interpersonal skills? Recommend with Not Recommend Recommend Highly Recommend Reservation Would you recommend this doctor to a friend for his/her medical competence? Not Recommend Recommend with Recommend Satisfactory Highly Recommend Non -exemplary Physician: Reservation Unexceptional Physician: Model Physician:superficial, artificial demeanor applied appropriate knowledge base applied sophisticated, wise, thoughtful, applied Physician: knowledge base inadequate to my adequately to my specific situation profound knowledge base specifically to awkward, knowledge base only situation my situation somewhat apparent in application to my situation COMMENTS: EM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Progress Report) Page 27 PI: Sondra Zabar, MD NYU School of Medicine
  29. 29. Attachment – Sample Report Card EMPACT OSCE Report of Results – July 2007 Clinical skills were assessed in 5 cases. Your scores in 5 core areas – communication scores, overall recommendation scores, ratings of ability to apply expertise, specific skills across cases, and overall case- specific skill scores -- are reported in the charts that follow. For case-specific skills and recommendation ratings, results for each case are included as well. One case was not reliably scored (Unexpected Death) and so scores associated with that case should be interpreted with caution. Overall communication score: Calculated across all cases as the % of behaviorally-anchored communication items (8-14 items per case) for which you were rated as having performed well (“done well”). Sub-domains include: Information gathering, relationship development, and patient education. Overall recommendation rating: Calculated across all cases on the basis of rating of degree to which “would recommend physician to a friend based on his/her communication skills” with the following response options: Not Recommend – Recommend with Reservations – Recommend – Highly Recommend. Overall rating of application of expertise: Calculated across all cases on the basis of rating of degree to which applied expertise effectively, using a 4-pt scale: Insufficient Application, Slight Application, Sufficient Application, Exceptional Application of Expertise. Selected skills across cases: Calculated as the % of items rated as well done for specific skills measured across at least several cases including: delivering bad news, managing difficult situations, accountability, handling emotions. Overall case-specific skills: Calculated across all cases as the % of items rated as well done for core knowledge and skill items specific to each case.EM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Progress Report) Page 28PI: Sondra Zabar, MDNYU School of Medicine
  30. 30. Communication Scores for Sample Student 100% Error Bars: +/- 1 Std Dev Your Scores Class Mean 90% 80% 70% 64% 61% 60% 56%% Well Done 51% 50% 52% 50% 40% 33% 30% 27% 20% 10% 0% OVERALL Communication - Communication - Communication - Patient COMMUNICATION SCORE Information Gathering Relationship Development Education EM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Progress Report) Page 29 PI: Sondra Zabar, MD NYU School of Medicine
  31. 31. Overall Recommendation Rating for Sample StudentHighly 4 Error Bars: +/- 1 Std DevRecommend 3.35Recommend 3 Informed Consent 2.75 X-Ray Recall Unexpected Death* Transfer of CareRecommend with 2Reservation Not 1 Repeat VisitRecommend OVERALL Recommendation Ratings RECOMMENDATION for Each Case *Unreliable Case - Interpret w/ Caution Your Scores Class Mean EM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Progress Report) Page 30 PI: Sondra Zabar, MD NYU School of Medicine
  32. 32. Overall Rating of Application of Expertise for Sample StudentExceptional Unexpected Death*Application 4 Error Bars: +/- 1 Std Devof Expertise SufficientApplication 3 2.84 Informed Consent Transfer of Care Informed Consent Transfer of Care X-Ray Recall Repeat Visit X-Ray Recall Repeat Visit 2.00 Slight 2Applicationof ExpertiseInsufficient 1Application OVERALL RATING Ratings APPLICATION OF EXPERTISE for Each Case *Unreliable Case - Interpret w/ Caution Your Scores Class Mean EM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Progress Report) Page 31 PI: Sondra Zabar, MD NYU School of Medicine