Successfully reported this slideshow.
Your SlideShare is downloading. ×

Top Articles in Medical Education 2017

Loading in …3

Check these out next

1 of 154 Ad

More Related Content

Slideshows for you (20)

Similar to Top Articles in Medical Education 2017 (20)


Top Articles in Medical Education 2017

  1. 1. Top Articles in Medical Education 2017: Applying the Current Literature to Educational Practice and Scholarship Donna D’Alessandro, MD Steve Paik, MD, EdM Mike Ryan, MD, MEHP Teri Turner, MD, MPH, MEd H. Barrett Fromme, MD, MHPE Leora Mogilner, MD Alix Darden, PhD, Med 7 May 2018
  2. 2. Disclosures •None to report
  3. 3. Goal •Broadly review the medical education literature of 2017 to select the top articles that could impact practice
  4. 4. Objectives By the end of the session, participants will be able to: • List major thematic areas of investigation and publication in medical education for the year 2017 • Discuss the outcomes of the top articles in medical education in 2017 • Formulate approaches to incorporating medical education innovations into their own practice. • Identify areas of scholarly interest for themselves for further reading, curricular application, and innovative scholarship
  5. 5. Road Map for Session •Who We Are •Methods •Topic Areas •Key Point •Methods •Results •Summary •Question and Answers
  6. 6. The Team • H. Barrett Fromme, MD, MHPE, University of Chicago • Director of Faculty Development in Medical Education, Assoc Residency PD • Donna M D’Alessandro, MD, University of Iowa • Educator of Residents and Students • Alix Darden, PhD, MEd, University of Oklahoma Health Sciences Center • Director of Faculty Development, Dept of Pediatrics • Leora Mogilner, MD, Icahn School of Medicine at Mount Sinai • Director of Advocacy and Community Pediatrics • Steve Paik, MD, EdM, Columbia University School of Medicine • Pediatric Residency Program Director • Michael S Ryan, MD, MEHP, Virginia Commonwealth University • Assistant Dean for Clinical Medical Education • Teri L. Turner, MD, MPH, MEd, Baylor College of Medicine • Vice Chair of Education, Department of Pediatrics
  7. 7. Inclusion Criteria •Relevant to academic pediatricians •Could change practice at individual or programmatic/institutional level
  8. 8. Journals Reviewed • Academic Medicine • Academic Pediatrics • BMC Education • Journal of the American Medical Association • Journal of General Internal Medicine • Journal of Graduate Medical Education • Medical Education • Medical Education Online • Medical Science Educator - IAMSE • Medical Teacher • NEJM • Pediatrics • Teaching & Learning in Medicine
  9. 9. Methods, Stage 1 •Seven medical educators reviewed journals •Two authors read all titles/abstracts in each journal to find articles that met inclusion criteria
  10. 10. Methods, Stage 1 •Two authors for each journal selected abstracts of interest •Each reviewed combined article titles and scored: • 0 = not relevant • 1 = may be relevant • 2 = highly relevant •All four point abstracts automatically advanced •All three point abstracts discussed for inclusion
  11. 11. Methods, Stage 2 •Each abstract assigned to two reviewers to read full article •Each full article was scored: • 0 = do not consider, will not change practice • 1 = consider, could change practice • 2 = definitely include, will change practice •All four points articles automatically included •All three point articles discussed for inclusion
  12. 12. Final Stage •Final articles grouped by topics • Summary and slides created • Peer reviewed by two members of group
  13. 13. Selection Process 1682 abstracts 13 journals 114 abstracts selected for discussion 75 articles selected for review 22 articles for presentation 53 not presented 39 not selected 1568 not discussed
  14. 14. Final Topics
  15. 15. Feedback, Coaching and Observation We’ve come a long way since the feedback sandwich Teri L. Turner, MD MPH, M.Ed.
  16. 16. R2C2 in Action: Testing an Evidence-Based Model to Facilitate Feedback and Coaching in Residency Sargeant J, et al. JGME. 2017; 9(2): 165-170
  17. 17. Key Points •The R2C2 model enabled meaningful, collaborative, reflective, goal-oriented feedback discussions. •A defined coaching phase prompted giver and receiver to think differently and more positively about feedback and framed conversations as opportunities to coach for improvement. R=rapport building R= reactions to feedback C=content of feedback C=coaching
  18. 18. What They Did •Purpose: To determine the R2C2 model’s utility and acceptability for engaging residents in their feedback, and in using it to plan for improvement •A qualitative study of 7 IM and Peds residents and their 5 supervisors •Thematic analysis
  19. 19. What They Found •Supervisors valued having a structure for feedback conversations and appreciated the opportunity to coach. •Residents appreciated working collaboratively with their supervisor. •Supervisors needed time to learn to use the R2C2 model.
  20. 20. Why We Chose It
  21. 21. Why We Chose It
  22. 22. “It’s Just Not the Culture”: A qualitative study exploring residents’ perceptions of he impact of institutional culture on feedback Ramani S, et al. Teaching and Learning in Medicine. 2017; 29(2): 153-161
  23. 23. Key Points •Institutional culture is central to resident’s perceptions of the quality, credibility, and acceptability of feedback. •Training on techniques for delivering feedback alone, is unlikely to enhance its impact on resident performance.
  24. 24. What They Did •Purpose: To examine resident opinions on institutional factors that affect the quality of feedback •Qualitative study using a constructivist grounded theory approach •Thematic analysis
  25. 25. What They Found
  26. 26. What They Found • The cultural norm lacks clear expectations and messages around feedback • The prevailing culture of niceness does not facilitate honest feedback • Bidirectional feedback is not part of the culture • Faculty-resident relationships affect credibility and receptivity to feedback • There is a need to establish a culture of longitudinal professional growth
  27. 27. Why We Chose It •Faculty development is not the only solution to feedback problems •Importance of feedback as a bidirectional exchange within a social culture that encourages reflective practice and ongoing professional development for both parties
  28. 28. Gender Differences in Attending Physicians’ Feedback to Residents: A qualitative analysis Mueller AS, et al. JGME. 2017; 9(5): 577-585
  29. 29. Key Points •There are qualitative differences in the kind of feedback that male and female residents receive. •Raise awareness of gender bias in perceptions of residents’ capabilities.
  30. 30. What They Did •Purpose: To examine the feedback that male and female residents received from attending physicians •Qualitative content analysis using a post-positivist paradigm •Collected several variables for analysis
  31. 31. What They Found
  32. 32. What They Found Discordant feedback to female residents generally focused on masculine traits of: • Autonomy • Assertiveness 0 10 20 30 40 50 60 70 Positive and Constructive Feedback No negative comments related to traits Criticised multiple times for lacking traits Gender Difference in Feedback Male Female
  33. 33. Why We Chose It •Gender differences exist in feedback •Raise awareness of both gender bias in perceptions of residents’ capabilities and gender stereotypes of what traits are valued Author unknown but is attributed by some to Emma Watson
  34. 34. Successful Implementation of a Direct Observation Program in an Ambulatory Block Rotation Smith J, et al. JCME. 2017; 9(1): 113-117
  35. 35. Key Points •Despite initial faculty skepticism, the authors achieved high rates of frequent observation on a breadth of clinical skills for assessing residents in an ambulatory setting.
  36. 36. What They Did •Purpose: To evaluate the effects of a novel direct observation evaluation system on both faculty and trainees •Program evaluation research •Several variables analyzed
  37. 37. What They Found
  38. 38. What They Found
  39. 39. Why We Chose It •Challenges the notion that if we asked faculty to do more observations it will increase perceived faculty burden •Discrepancy between residents and faculty regarding which clinical domain was the most useful for observation
  40. 40. Section Summary • The R2C2 feedback model provides a framework for bi- directional meaningful feedback and coaching • To make an impact on trainee performance and ultimately on patient care, we have to change the institutional culture of feedback • We need to be cognizant and work to be consistent in our feedback messages regardless of the gender of our trainees • Frequent direct observation is feasible in the outpatient setting
  41. 41. How Is My Trainee Progressing? Let Me Count the Ways. Alix Darden, PhD, MEd
  42. 42. Justify Your Answer: The Role of Written Think Aloud in Script Concordance Testing Power, A, Teaching and Learning in Medicine. 2017; 29(1):59-67
  43. 43. Key Point •Quantitative assessments of clinical reasoning may not provide a complete assessment of trainee clinical reasoning skills.
  44. 44. What They Did •Purpose: Compare clinical reasoning as assessed by a quantitative test, script concordance test (SCT) and qualitative test, think aloud for SCT cases. • Script Concordance Test (SCT) of Clinical Reasoning -24 • Written Think aloud - 3 Pediatric Residents N=91 • Script Concordance Test of Clinical Reasoning - 24 • Written Think aloud - 3 Panel of Experts (POE) N=12 Compare Residents Answers 1. SCT & Think Aloud 2. POE Think aloud
  45. 45. What They Found Think Aloud illuminates: 1. Incorrect clinical reasoning despite correct SCT response 2. Sound clinical thinking with a suboptimal SCT response 3. Question Misinterpretation
  46. 46. Why We Choose It •Think aloud as a method to document learner thought processes is under utilized in medical education, yet is a powerful tool for enriching quantitative assessment data.
  47. 47. Promoting Responsible Electronic Documentation: Validity Evidence for a Checklist to Assess Progress Notes in the Electronic Health Record Bierman, JA, Teaching and Learning in Medicine. 2017; 29(4):420-432
  48. 48. Key Points •The Responsible Electronic Document checklist, can be used as a tool to systematically review the quality of trainee inpatient EMR progress notes. •Trainee inpatient progress notes need improvement.
  49. 49. What They Did •Purpose: Create and validate a checklist, 18 closed items, 4 open-items, to assess trainee inpatient progress notes in EMR. Response Process Validation Internal Structure Validation
  50. 50. Responsible Electronic Document (RED) Checklist Northwestern University Feinberg School of Medicine Subjective The note contains: 1. Current patient concerns or symptoms No 0 1 Yes 2 N/A Objective The Physical exam contains: 2. Succinct vitals 3. Examination of all systems relevant to today’s positive symptoms 4. Examination different from previous day’s exam The data portion of the note contains: 5. Labs only if they are new 6. Reports of studies only if it is the first day they are included Mark on scale as defined in key: 0 1 2 N/A 7. A summary or impression of study reports
  51. 51. Assessment and Plan The assessment and plan meets these criteria: 8. A summary statement is included. 9. The summary statement is different from previous day’s statement. 10. Positive symptom(s) from subjective section are included. 11. A problem-based assessment is included. Mark on scale as defined in key: 0 1 2 N/A 12. The status of each problem is described. 13. Lab abnormalities are interpreted. 14. Interpretation of studies is included. 15. Problems are written as diagnoses or accompanied by differentials. 16. Active problems are accompanied by clinical reasoning. 17. Problems are associated with brief, clear plans. 18. Assessment and plan is different from previous day’s assessment and plan. No 0 1 Yes 2 N/A Summary – open ended A good progress note is Truthful, Reasoned, Updated and Succinct. Please comment on characteristics of this note that fulfill or lack these.
  52. 52. What They Found • Trainee notes show substantial room for improvement • Average note score – 66.9% (SD=10.6, range=33.4%- 93.3%) • 7 minutes to complete – rater does not need to know patient or review chart. Compare one daily note to the previous days note • Good Interrater reliability – Cohen’s kappa coefficient = 0.67. C-
  53. 53. Why We Chose It •AAMC and ACGME both emphasized the importance of clinical documentation training. •The Responsible Electronic Document Checklist provides a systematic, easy to use assessment tool that also provides feedback to the trainee.
  54. 54. The development of the PARENTS: a tool for parents to assess residents' non-technical skills in pediatric emergency departments. Moreau KA, BMC Medical Education. 2017; 17: 210 - 220
  55. 55. Key Points With a validated tool, PARENTS, 1. parents of pediatric patients can be involved in the assessment of residents, 2. assessment of residents’ non technical skills in Pediatric Emergency Departments can be improved 3. parents perspectives can be used to improve resident training
  56. 56. What They Did •Purpose: Create and validate a 19 item assessment tool of residents’ ER non-technical skills for parents of patients to use. Response Process Validation Internal Structure Validation
  57. 57. PARENTS 1.Did the resident identify him/herself as a resident 2: Was the resident’s ID badge or nametag visible? 3: Did the resident wash his/her hands? Yes No NA 4: ...enter the room with some basic knowledge of your child’s condition? 5: ...listen to you and allow you to speak without interruption? 6: ...appear to understand what you had to say? 7: ...explain what he/she was doing for your child and why? 8: ...interact with you comfortably? 9: ...interact with your child comfortably? 10: flexible in his/her thinking and approach depending on your needs and those of your child? 11: concern for your feelings and those of your child? 12: attention to you and your child during your interactions with him/her? Very poor Poor Fair Good Very good NA
  58. 58. PARENTS 13: ...explain your child’s treatment or prescribed medication, including possible side effects? 14: ...determine next steps about care or treatment with you, including any follow-up plans? 15: ...discuss what to do if your child has any problems or complications related to his/her condition? 16: ...answer your question? 17: ...explain things in a way that you could understand? Very poor Poor Fair Good Very good NA Open ended questions 18. What can the resident do to improve his/her interactions with caregivers and their children? 19. Please use the space below to provide additional comments on the resident’s skills when interacting with you and your child?
  59. 59. Why We Chose It • Parent assessment of trainee non-technical skills provides an important, authentic assessment opportunity for residents.
  60. 60. Mixed Messages or Miscommunication? Investigating the Relationship Between Assessors’ Workplace- Based Assessment Scores and Written Comments Sebok-Syer, SS, et al. Academic Medicine. 2017; 92(12): 1774-1779
  61. 61. Key Point •Narrative comments are more balanced for trainees who are perceived to have a deficiency. Types of comments are not consistent across trainee levels leading to a “hidden code”. Overall Resident Assessment – Clinical Competency Committee 360 Comment Checklist Residency assessment program
  62. 62. What They Did •Purpose: Examine relationships between checklist, task ratings, global ratings, and narrative comments in resident assessment tools. Task ratings 360o Rating Checklist Comments • McMaster Modular Assessment Program • 23 PGY 1 & 2 - EM • Regression Analysis • Content analysis
  63. 63. What They Found – Quantitative data “Significantly associated with qualitative rating of task strengths and weaknesses Regression variable for assessors’ selection of the checklist option “Done, but needs attention. (n=321) Model Variables Beta SE Wald Probability Intercept 4.95 1.32 3.75 <0.001 Task rating -1.06 0.29 -3.61 <0.001 Global rating 0.026 0.27 0.09 0.92 Quality rating of task strengths -0.41 0.21 -3.47 <0.001 Quality rating of task weaknesses 0.36 0.14 2.42 0.016 Task comment length 0.013 0.031 0.40 0.69
  64. 64. What They Found – Qualitative data •~30% of assessors avoid written comments even when required by system • Comments associated with trainees with perceived deficiency(s) • Addressed both strengths and area(s) of improvement • Focus - criterion referencing •Comments associated with high performing trainees • Addressed strengths in global manner • Focus - norm referencing
  65. 65. Why We Chose It • A novel study identifying associations with different resident assessments (qualitative and quantitative). • Can lead to faculty development for members of competency committees in how to interpret narrative comments and various rating models. Resident Assessment C B A
  66. 66. Section Summary •Development of unique competency-based assessment tools and methodology aid in training future physicians. •Think aloud – getting inside the trainee thought process •EMR progress note assessment •Parent assessment of non-technical skills •Methodology for comparing instruments within an assessment plan
  67. 67. Educator Development Across the Continuum Donna M. D’Alessandro, MD
  68. 68. Medical Students’ Professional Development as Educators Revealed Through Reflection on Their Teaching Following a Students-as-Teachers Course Yoon MH, Blatt BC, and Greenberg LW Teaching and Learning in Medicine 2017; 29(4):411-419.
  69. 69. Key Point •Medical student (MS) self-assessment narratives provide an understanding of their developing identities and emerging professional self-concept as educators
  70. 70. What They Did • Purpose: Explore MS4 reflections in a students-as-teachers course using written self-assessment narratives • 1 year elective at George Washington University in Washington DC • 6 workshops and practicum where MS4 co-teach in standardized patient physical diagnosis encounters with younger MS • 2011-13, MS4 were asked to reflect on their course efforts and assign a grade to themselves with written justification • Inductive content analysis
  71. 71. What They Found Educator Identity Growth Professional Development Personal Growth • Using teaching strategies for adult learning • Preparing to teach physical diagnosis • Incorporating clinical correlations • Giving and receiving feedback • Creating a positive learning climate • Growing as educators • Modeling professionalism • Exceeding course requirements • Peer counseling • Many comments related to professionalism • Gained confidence • Increased comfort with teaching • Developed camaraderie with other educators
  72. 72. Why We Chose It •This explores how students translate their experiences in a students-as-teachers course into their identities as a professional and educator •These could be foundational courses for developing a workforce of skilled medical educators
  73. 73. A Multi-Institutional Longitudinal Faculty Development Program in Humanism Supports the Professional Development of Faculty Teachers Branch WT, et al. Academic Medicine. 2017; 92(12):1680-1686.
  74. 74. Key Point •Participation in a longitudinal, multi- institutional, faculty development program facilitated the professional development in humanism for its participants
  75. 75. What They Did •Purpose: To describe a multi-year, longitudinal, multi- institutional faculty development program in humanism • 2005-2017 • 30 institutions in US and Canada • Local facilitator and 8-12 faculty members • 12-18 month program of bi-monthly sessions • Evaluations – faculty and matched-control • Questionnaires and narratives
  76. 76. What They Found •993 faculty participated + some residents •Participants were highly engaged with little drop out and consistent attendance •Participants scored higher on overall humanism questionnaires than matched controls including trends for individual questionnaire items •Participants progressed toward more advanced levels of humanism self-identity as measured by Robert Kegan’s stages of adult development
  77. 77. Why We Chose It •Describes a successful long-term, multi- institutional, faculty development program supporting the professional self-identity growth of medical educators •Used as a institutional model to strengthen the humanistic side of medical education and improve the learning environment
  78. 78. Creating a Medical Education Enterprise: Leveling the Playing Fields of Medical Education vs. Medical Science Research Within Core Missions Thammasitboon S, Ligon BL, Singhal G, Schultze GE, Turner TL. Medical Education Online. 2017;22:1377038
  79. 79. Key Points •Business organizational themes were used to organize and develop the structure of a medical education enterprise •The themes framed practical strategies for empowering and advancing the scholarly endeavors of clinical-educators within a pediatric department
  80. 80. What They Did •Purpose: Create an educational enterprise within the pediatric department at Baylor College of Medicine, that levels the playing field for clinician-educators relative to clinician-scientists and empowers them •Describes the process of enterprise creation using Bowman and Deal’s business organization model with 4 frames
  81. 81. What They Found Framework Meaning Strategies to Support Clinician-Educators Structural Administrative structure, rules, policies, & the organizational reporting structure • Created a Center specifically for medical education • Decentralized leadership organization • Offered grants • Developed promotion pathways • Defined dissemination success for educational scholarship Human Resource Optimizes the organization’s personnel through recruitment, development, empowerment & support • Used the Center to assist less experienced clinician-educators • Endowed Chair • Offered incentives (travel, etc.) Political Aligns power and authority to manage resources, negotiate conflict and form alliances • Financially incentivized educational scholarship • Expanded venues for scholarly engagement and dissemination • Educational awards Symbolic Objects, people, events, or stories, used to communicate missions and values for supporting cultural identity & shared vision • Provided opportunities to recognize all types of scholarship • Reminded leaders about the educational mission’s importance • Provide consistent messaging about educational mission’s value
  82. 82. What They Found •12/15 college-wide educational scholarship awards •Winners of college-wide faculty educational excellence awards increasing •National meeting presentations dramatically increasing •Strongly supports the clinician-educator
  83. 83. Why We Chose It •Flexible structure that offers pragmatic strategies that institutional can use to support clinician-educators •Strategies discussed could be implemented in isolation or in combination, at many (all?) institutions
  84. 84. Section Summary • Educators start to view themselves as educators early in their professional careers • Increased support for medical educators advances their self- identity and the improves the their outcomes as teachers and scholars over their professional careers • Supporting the development of medical educators across their professional lifespan is makes everyone a winner!
  85. 85. Entrustment and Transitions Leora Mogilner, MD
  86. 86. How Supervisor Experience Influences Trust, Supervision, and Trainee Learning: A Qualitative Study Sheu L, Kogan J, and Hauer K Academic Medicine 2017; 92(9): 1320-1327.
  87. 87. Key Point •Supervisors’ approach to trust and supervision varies with their level of experience and this variation can directly affect trainee learning
  88. 88. What They Did •Purpose: to investigate how supervisor experience influences trust, supervision, and trainee learning • Two phase qualitative study: • Phase 1: Reviewed supervisor interviews from 2 institutions (UCSF and HUP) to develop supervisor “exemplars” (early, developing and experienced) • Phase 2: Trainee focus groups at a single institution to validate accuracy of exemplars developed in Phase 1 and explore impact on learning
  89. 89. What They Found – Phase 1 Early Supervisor Developing Supervisor Experienced Supervisor Data Granular: Emphasis on trainee task completion Individualized: Emphasis on trainee skills and ability Holistic: Emphasis on trainee qualities/behaviors Approach Err towards more supervision Tailored supervision Trend towards greater autonomy Perspective Reflect on recent experiences as trainee Reflect on own early supervisor experience Draw on institutional knowledge/global experience Clinical Personal uncertainty Growing confidence Confidence in assessing trainee skills
  90. 90. What They Found – Phase 2 Themes identified: • Shift in trainee preference and learning needs over time • Desire for flexibility and an individualized approach to supervision to promote learning
  91. 91. Why We Chose It •Demonstrates that supervisor experience has a direct impact on trainees’ learning and satisfaction •Faculty development can help supervisors provide the flexible supervision appropriate for trainees’ needs •Opportunity to help trainees learn how to “manage up” and give constructive feedback to their supervisors to address their learning needs
  92. 92. Entrustment of the On-call Senior Medical Resident Role: Implications for Patient Safety and Collective Care Huda N, Faden L and Goldszmidt M BMC Medical Education 2017; 17 (121)
  93. 93. Key Point •Identifies senior medical resident core on-call supervisory tasks that can be used for training and assessing residents prior to making entrustment decisions
  94. 94. What They Did •Purpose: to understand the clinical activities of the on-call senior medical resident (SMR) and provide a model for entrustment decisions for this role • Four-phase constructivist grounded theory approach conducted at 2 academic medical centers in Ontario: • Phase 1: Case study • Phase 2: Focus groups • Phase 3: Literature search • Phase 4: Two return-of-findings focus groups
  95. 95. What They Found 1-Overseeing Ongoing Patient Care 2-Briefing Ensures that the junior trainee has needed information and approach for patient assessment 3-Case Review Ensures that problems have been identified, plan addresses each problem and junior MD understands the plan 4-Documentation Ensures that admit note, SMR note and patient orders are consistent and complete 5-Preparing for Handover Prepares the junior MD for their case presentation to the attending physician • Five core on-call supervisory practices that support collective care and patient safety were identified:
  96. 96. Why We Chose It • Identifies a set of core on-call supervisory tasks of the senior medical resident and their impact on patient safety and trainee learning • These findings can be used for training and assessing trainees prior to making entrustment decisions
  97. 97. The Educational Climate Inventory: Measuring Students’ Perceptions of the Preclerkship and Clerkship Settings Krupat E, Borges N, Brower R, Haidet P, Schroth S, Fleenor Jr T, and Uijtdehaage S. Academic Medicine 2017; 92 (12): 1757-1764.
  98. 98. Key Point • The Educational Climate Inventory (ECI) allows educators to assess students’ perceptions of the medical school learning environment and differentiates between a learning or mastery- oriented climate vs. performance-oriented climate
  99. 99. What They Did • Purpose: to develop and validate an instrument to assess educational climate and examine the relationship between ECI and other variables • A pool of 50 items was rated by 1st, 2nd and 3rd year students at 6 US medical schools • Exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were performed • They explored the relationship between ECI and other factors (pass-fail vs. traditional grades, satisfaction with medical school experience/choice of medicine as a career, and satisfaction with year in medical school)
  100. 100. What They Found • 1441 students completed the surveys (56% response rate) Factor Analysis: • 3 distinct factors resulting in a 20-item scale were identified: Factors Cronbach alpha 1. Centrality of learning and mutual respect Ex. “In this medical school we focus on a sense of discovery and the excitement of inquiry” .88 2. Competitiveness and stress Ex. “The atmosphere here is highly competitive” .80 3. Passive learning and memorization Ex. “Most of what we do here is focused on the passive transfer of knowledge” .71
  101. 101. What They Found Relationship of ECI scores to other variables: • Students’ ratings of the preclerkship learning climate were more performance-oriented in schools with grades compared to schools with a pass-fail system (P=.04) • Clerkship students rated their learning climate as significantly more performance-oriented than preclerkship students (P<.001) • Students perceiving their environment as more performance-oriented were less satisfied with their own medical school and their decision to pursue medicine as a career (P<.001)
  102. 102. Why We Chose It •Validates the ECI, an instrument that can be used by educators to assess the learning environment with the goal of ensuring that a positive learning environment exists •Provides insight into the impact of educational climate on student satisfaction with their school and choice of career
  103. 103. Transition to Residency: Using Specialty-Specific Clinical Tracks and Advanced Competencies to Prepare Medical Students for Internship Khan M, Splinter A, Kman N, Leung C, Rundell K, Davis J and McCallister J Medical Science Educator 2017; 27: 105-112
  104. 104. Key Point • Description of a novel 4th year medical student competency-based, specialty-specific curriculum that prepares students with the skills they need to be successful interns
  105. 105. What They Did •Purpose: redesign of fourth year curriculum based on ACGME Core Competencies and AAMC Entrustable Professional Activities (EPAs) to ensure students are prepared for internship •Curriculum redesign at one medical school (Ohio State University College of Medicine)
  106. 106. What They Found • Acute care setting (ER and subinternship)—8 weeks • Outpatient ambulatory setting—8 weeks Required • Clinical Tracks •Provide set of guidelines for clinical rotations and competency-based assessments of essential intern skills • Advanced Competencies •Enhanced content that maps to core ACGME competencies; interdisciplinary, generalizable to multiple practice areas Elective Revamped4thYearCurriculum
  107. 107. Why We Chose It •Uses ACGME milestones to evaluate performance, ensuring a common language that can translate to readiness assessment at the start of internship •Has the potential to better prepare students for internship and ease the transition from medical school to residency (no evaluation data published yet)
  108. 108. Section Summary • Supervisor experience directly impacts trainees’ learning and satisfaction • Knowledge of core on-call supervisory tasks can be used for training and assessing trainees prior to making entrustment decisions • The Educational Climate Inventory (ECI) allows schools to assess student perceptions of the educational environment, which directly impacts student learning and satisfaction • Competency-based, specialty-specific curricula may help fourth year medical students develop the skills they need to be successful interns
  109. 109. Digging Deeper: Culture and Climate in Medical Education Steve Paik, MD, EdM
  110. 110. The effect of white coats and gender on medical students’ perception of physicians Ladha, M. et al. BMC Medical Education 2017; 17: 93
  111. 111. Key Points •White coats did not change the perception of physician’s ratings by medical students •However gender and possibly race altered students’ ratings of physicians in the perception of: • Trustworthiness • Physician management • Competence • Professionalism • Perception of medical error
  112. 112. What They Did •Purpose: To determine if components of physical appearance of the physician affected students ratings of competence •Cross-sectional study using self-administered questionnaire with four scenarios portraying potential physician error •Data collected: • 5 point Likert scale rating physician qualities, appropriateness of actions and whether the physician committed the error
  113. 113. What They Found •Primary Outcome – White Coat • No difference in altering perception in any domains •Secondary Outcome – Gender and Race • Gender – Male • Trustworthiness • Appropriateness of action • Competence • Medical Error (less correlated) • Case/Caucasian • Competence • Medical Error (less correlated)
  114. 114. Why We Chose It •Medical Schools put effort into evaluations •Implicit Bias is a factor that goes beyond just patient care •There are a many subtleties we may not be addressing in our evaluation system
  115. 115. Acculturation Needs of Pediatric International Medical Graduates: A Qualitative Study Osta AD, et al. Teaching and Learning in Medicine. 2017; 29: 143-152
  116. 116. Key Points •Residency Training is a time of adjustment for all trainees and there are many acculturation issues for IMGs •Pediatric-specific acculturation issues were elicited from this qualitative study which were not previously elucidated.
  117. 117. What They Did • Purpose: To explore if pediatric IMG residents have specific acculturation needs that have not been previously identified. • 90-minute semi structured focus groups with Pediatric non US-IMG residents at one academic medical center • Main focus group questions • What has been the most challenging part of living and practicing medicine in the United States? • What are some of the communication challenges that you have faced while practicing medicine as a pediatric resident? • How is it different being a pediatrician in the United States as compared to your culture?
  118. 118. What They Found
  119. 119. What They Found •Rich qualitative data to inform their practice locally:
  120. 120. Why We Chose It •Residency Training is an acculturation challenge during a stressful transition •This study elucidated acculturation issues that were specific to IMG trainees training in Pediatrics. •The topics would be helpful for addressing curriculum and faculty development enhance training and transitioning IMG trainees
  121. 121. Long-term benefits by a mind-body medicine skills course on perceived stress and empathy among medical and nursing students van Vliet, M., et al. Medical Teacher. 2017; 39 (7): 710-719
  122. 122. Key Points •Mind Body Medicine course had short term and long term benefits on •Fostering Empathy •Decreasing Perceived stress •Preventing increased Personal Distress
  123. 123. What They Did •Purpose: To explore short term and long term effects of the MBM course among medical students and nursing students •Exploratory Controlled, quasi-experimental study to evaluate the effects of Mind body Medicine course using a validated questionnaire over 4 time points •Analysis • Descriptive statistics for baseline characteristics • Effect analysis using linear mixed models
  124. 124. What They Did: Content
  125. 125. What They Found •MBM Course: •Significant decrease in perceived stress in nursing students who had baseline high stress levels •Prevented higher levels of personal stress in medical students who had lower levels of baseline stress •Improved fantasy and empathetic concern compared to control over 12 months
  126. 126. What They Found •MBM Course: •Significant decrease in perceived stress in nursing students who had baseline high stress levels •Prevented higher levels of personal distress in medical students who had lower levels of baseline stress •Improved fantasy and empathetic concern compared to control over 12 months
  127. 127. Why We Chose It •Validated measures were used to show internal and external validity of the MBM course •Preparation of stress may help in transition from the academic to clinical practice •Stress management training and wellbeing early in the professional development process may improve wellness in the future
  128. 128. Section Summary • White coats did not have an effect on evaluations but gender and race may be a factor in faculty assessment and all assessment that we may need to better evaluate • Acculturation issues specific to pediatric training programs are different and important to incorporate into training and faculty development • Mind Body Medicine started early in medical school may have a lasting effect on learners in transitioning to clinical years
  129. 129. Improving the Medical Student Experience: Adding Value, Refining Evaluations, and Helping Them Pee Happy Michael S. Ryan, MD MEHP
  130. 130. Numerical versus narrative: A comparison between methods to measure medical student performance during clinical clerkships Bartels J, et al. Medical Teacher. 2017; 39: 1154-1158
  131. 131. Key Points •Narrative comments from evaluations were at least as reliable, if not more, than quantitative scores on final evaluations •There was a strong correlation between narrative comments and quantitative scores
  132. 132. What They Did •Purpose: To determine whether narrative data would be equal or more reliable than quantitative data from clerkship evaluations •Retrospective analysis of ITERs (in-training evaluation reports, a.k.a. final evaluations) collected from the M3 Neurology clerkship at the University of Rochester •Data collected: • All ITERs from 50 randomly selected students • Original numeric score (Likert-scale, 1-10) • New narrative score (60-100) generated by 5 grading committee members • Analyzed for correlation and inter-rater reliability
  133. 133. What They Found •Inter-rater reliability • Original numeric scores = ICC .62 (95% CI = .41, .83) • New narrative scores = ICC .88 (95% CI = .83, .92) •Original numeric scores were highly positively correlated (r = .81) with the new narrative scores
  134. 134. Why We Chose It Observations Impression Score Evaluation Score Score How Faculty Observations Translate into Student Evaluations Issues 1. Halo effect 2. Poor depth 3. Inflation Elimination of the “middle man!” Traditional systemNarrative-only system
  135. 135. Developing validity evidence for the written Pediatric history and physical exam evaluation rubric King MA, et al. Acad Pediatr. 2017; 17: 68-73
  136. 136. Key Points •P-HAPEE is 10-item instrument which can be used to assess the quality of medical students’ H&Ps •The instrument has strong content, internal structure (inter-rater reliability), and response process validity
  137. 137. What They Did •Purpose: To develop and gather validity evidence for a Pediatric H&P evaluation rubric • Development of P-HAPEE rubric • Content developed from accreditation bodies, published instruments, expert consensus • > 50 educators critically reviewed rubric •Data collected on 30 H&Ps with variable characteristics. Analyzed for validity evidence: • Content validity, Internal structure, response process
  138. 138. What They Did: Content • Content: Traditional elements (Histories, PE, studies, A/P) • Scale: 5-points (None, Some, All) • “A medical student is expected to perform at a score of 3 or better by the end of his/her third year.” • Narratives and Final Overall Rating 1 2 3 4 5 Absent, unsupported, misses many critical findings, includes excessive irrelevant data, fails to include physical exam/diagnostic study findings, and/or restates findings without synthesis Identifies some defining history AND physical exam/diagnostic study findings while omitting most of the irrelevant data. Uses some medical terms and semantic qualifiers to synthesize an assessment Selects critical defining history AND physical exam/diagnostic study findings. Uses appropriate medical terms and semantic qualifiers to synthesize an accurate and concise summary statement. Notes: Example (Assessment Section)
  139. 139. What They Found Category ICC (95% CI) Information-gathering, history 0.81 (0.74-0.87) Information-gathering, physical examination 0.83 (0.77-0.88) Information synthesis and clinical reasoning 0.89 (0.86-0.92) Total score 0.85 (0.83-0.88) Overall assessment 0.89 (0.81-0.95) • Greater IRR for high and low quality (vs. medium quality) H&Ps • Training time: 30-60 minutes • Response process • Time to score each: 15.1 min/19.3 min (attending/resident, p <.001) • Easiest: HPI, PE, Intro • Most difficult: Assessment, problem ID, plan Internal structure : Inter-rater reliability
  140. 140. Why We Chose It •1 of only 2 published instrument developed to assess the quality of the Pediatric H&P •In comparison to other instrument, P-HAPEE had: •Smaller number of items (10 vs. 38) •Greater IRR •Utility in EMR and non-EMR generated notes •Multi-center involvement
  141. 141. Group observed structured encounter (GOSCE) for third-year medical students improves self-assessment of clinical communication Ludwig AB, et al. Medical Teacher. 2017; 39: 931-935
  142. 142. Key Points •Group OSCEs offer a more cost-conscious alternative to the traditional OSCE and allows for both the development and assessment of group/team-based skills •While valuable for formative feedback, there are barriers to their use for summative assessment of individual learners
  143. 143. What They Did •Purpose: To describe the feasibility and effectiveness of a Group OSCE (GOSCE) • Description of a formative educational innovation • Groups of 4-6 M3 students +1 faculty preceptor on Internal Medicine • 4 station SP-based GOSCE, 1 student leads each • Focused on: behavioral change, difficult encounter, shared decision-making, delivering bad news •Data collected: • Pre/post surveys to assess confidence in communication • Survey rating cases and experience
  144. 144. What They Found I feel confident in my ability to… Pre-test % agree or strongly agree N = 155 Post-test % agree or strongly agree N = 155 Absolute difference p value Give feedback to peers 60.00% 80.70% 20.70% <.0001 Respond to patients when they become angry 45.20% 63.90% 18.70% <.0001 Shared decision making 69.70% 86.50% 16.80% <.0001 Elicit all of my patient’s concerns 78.70% 91.00% 12.30% .0009 Deliver bad news 34.20% 46.50% 12.30% .008 Communication with patients within a group 72.30% 83.20% 10.90% .01 Take a smoking history/motivate to quit 64.5% 74.8% 10.3% .02 Communicate using nonverbal cues 87.70% 94.90% 7.20% .02 Clarify patient’s responses when I interview them 92.90% 98.70% 5.80% .007 3/12 were not significant: communicate in general, show respect, and verbal communication
  145. 145. Why We Chose It •Few reports of GOSCE in the literature •Though limited in terms of application for summative and/or individual evaluations, GOSCEs may offer opportunities for: 1. Training/feedback on group dynamics 2. Peer role modeling 3. Processing feedback encounters 4. Lower cost compared to single-learner OSCE
  146. 146. How can medical students add value? Identifying roles, barriers, and strategies to advance the value of undergraduate medical education to patient care and the health system. Gonzalo JD, et al. Acad Med. 2017; 92: 1294-1301
  147. 147. Key Points •The authors propose 7 specific roles and several strategies which could be used to enhance the value of medical students within the healthcare system
  148. 148. What They Did • Purpose: To describe the activities, tasks, and roles that could be afforded to medical students which would add value for the health system • AMA Accelerating Change in Med Ed (2016) conference • Plenary session (large/small group discussions) on medical student value • Qualitative analysis of: • Field notes from AMA staff • Written form completed by small group
  149. 149. What They Found Proposed “new” roles for Medical Students Patient navigator Safety analyst Care transition facilitator Member of QI Team Population Health Manager Patient care tech Scribe
  150. 150. What They Found: Barriers, Strategies, and Outcomes Student engagement, skills, assessment Service vs. Learning Resources, Logistics, Supervision Productivity and Billing Culture Faculty Factors Reshape focus to value- added roles Focus on continuity of learning Increase student “touch points” in the system Student experiences add value Enhanced education in clinical and health systems science Achieve quadruple aim: 1) Improve patient experience 2) Improve population health 3) Decrease cost 4) Improve life of health care workers Barriers General Strategies Outcomes Short-term Long-term
  151. 151. Why We Chose It •Challenges the peripheral/non-essential role that medical students often have on modern healthcare teams •Provides concrete guidance to increase the value medical students have on health care systems and outcomes
  152. 152. Section Summary • Narrative comments are at least as reliable, if not more, than numerical scores obtained on M3 clerkship evaluations • P-HAPEE is an encounter note assessment instrument with substantial inter-rater reliability which can be used to evaluate the quality of medical student Pediatric H&Ps • Group OSCEs provide an opportunity for learners to develop team-based skills, learn from their peers, and obtain formative feedback on interactions with patients and teams • Medical students should be given new roles and responsibilities to increase their value to the healthcare team