Medical decision making edited


Published on

Outline of the problems in training medical decision makers, and what we have learned about improving performance.

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Medical decision making edited

  1. 1. Medical Decision Making Jeffrey Young, MD, FACS Professor of Surgery Director, UVA Trauma Center Chief Quality Officer University of Virginia Health System
  2. 2. Problems <ul><li>How do we train medical students and residents to make effective medical decisions? </li></ul>
  3. 3. Problems <ul><li>Education in Medicine is poorly structured with respect to what is known about medical expertise </li></ul><ul><li>Reliance on standardized tests as a measure of operational competence is, at the least, not supported by any data </li></ul><ul><li>How do we know when trainees are able to accomplish more complicated (and risky) tasks? (other than that the page of the calendar flipped?) </li></ul>
  4. 4. Learning vs. Teaching <ul><li>We need to learn from other high-risk processes </li></ul><ul><ul><li>Military </li></ul></ul><ul><ul><li>Nuclear power </li></ul></ul><ul><ul><li>Maritime </li></ul></ul><ul><li>Many people have died over the years to benefit training in all these industries </li></ul>
  5. 5. Military Domain <ul><li>Simulation essential to preparation </li></ul><ul><li>Recreating mindset that soldier will face in battle is vital </li></ul><ul><li>Use entire range of simulation </li></ul><ul><ul><li>Tabletop to highly complex high-fidelity simulation </li></ul></ul><ul><li>Also after-action examination of decisions is common with aggressive search for errors </li></ul>
  6. 6. Learning vs. Teaching: Medicine <ul><li>No other high risk domain would have people right out of training show up for work, hand them a list of responsibilities, and say “good luck!” </li></ul><ul><li>All other high risk domains to try assess some level of competence in the domain before turning people loose </li></ul>
  7. 7. Learning vs. Teaching <ul><li>Creating sets of simulations, with minimum performance standards, for the medical domain would not be difficult </li></ul><ul><li>Close supervision where the teacher intimately watches the performance of the trainee, and can correct small errors </li></ul><ul><ul><li>Happens in OR, not outside </li></ul></ul>
  8. 8. Expertise
  9. 9. What is expertise? <ul><li>Nate Fick on Marine training </li></ul><ul><ul><li>Hardness = “The ability to face an overwhelming situation with aplomb, smile calmly at it, and them triumph through sheer professional pride” </li></ul></ul><ul><li>Ability to face new situations and bring experience to bear to make effective decisions </li></ul><ul><li>Ability to manipulate environment to carry out tasks </li></ul><ul><ul><li>Just knowing what to do without the ability to implement action is not effective </li></ul></ul>
  10. 10. Deliberate Practice <ul><li>Key concept in gaining expertise </li></ul><ul><li>Practicing situations where the same mental and physical processes will be used </li></ul><ul><li>Ameliorates anxiety, insecurity </li></ul><ul><li>Allows clearer thinking during live events </li></ul>
  11. 11. Lessons from Other Disciplines <ul><li>Baseball </li></ul><ul><li>Music </li></ul><ul><ul><li>Outliers – The Beatles </li></ul></ul><ul><li>Military </li></ul><ul><li>Fire operations </li></ul><ul><li>Dispatch </li></ul>
  12. 12. 9-11 <ul><li>FDNY fire dispatch </li></ul><ul><ul><li>Early response to event was executed according to pre-set plan </li></ul></ul><ul><ul><li>First Chief Officer on scene – First day as Battalion Chief </li></ul></ul><ul><ul><ul><li>Assessed situation rapidly and effectively </li></ul></ul></ul><ul><ul><ul><li>Organized command post </li></ul></ul></ul><ul><ul><ul><li>Dispatched first units </li></ul></ul></ul><ul><ul><li>Initial response was well-organized due to repeated practice, lessons learned from earlier attack, and simulations </li></ul></ul>
  13. 13. Analysis <ul><li>FDNY had simulated major incidents at the WTC for many years </li></ul><ul><li>The commander, though junior, was stationed near the WTC, and had participated in many of the drills </li></ul><ul><li>Dispatch had rigid procedures for major incidents and automated actions that went into effect for different alarms </li></ul>
  14. 14. Errors <ul><li>Though mistakes were made, they mostly occurred when circumstances went far outside practiced parameters </li></ul><ul><ul><li>These situations had not been simulated </li></ul></ul><ul><ul><li>Time pressure turned out to be extreme, where simulations did not routinely include that variable </li></ul></ul>
  15. 15. Lessons <ul><li>“ Tactical catastrophes are rarely the outcome of a single poor decision. Small compromises incrementally close off options until a commander is forced into actions he would never choose freely” </li></ul><ul><ul><li>Rounds vernacular: “good, OK, whatever” </li></ul></ul><ul><li>Rarely will people be able to improvise effectively, under pressure, with unique unpracticed circumstances </li></ul><ul><li>Mistakes most commonly occur in those settings </li></ul><ul><li>Simulation must be robust and complex to be effective </li></ul>
  16. 16. Our Research Purpose <ul><li>To simulate common clinical situations encountered by physicians and nurses to improve medical decision making under pressure </li></ul><ul><li>To avoid epiphenomenon associated with high fidelity simulators </li></ul><ul><li>To provide deliberate practice </li></ul><ul><ul><li>“ Luck is not method, and neither is hope. Hard work is.” </li></ul></ul><ul><li>To improve decision making and confidence, and decrease cognitive errors </li></ul>
  17. 17. Designing medical simulation to enhance decision making <ul><li>Focus on variables in, and orders out </li></ul><ul><ul><li>Not a technical simulation </li></ul></ul><ul><li>Wide variety of cases </li></ul><ul><li>Pressure and stress should be part of simulation </li></ul><ul><li>Should get subjects outside of comfort zone so they can begin to practice for those situations </li></ul><ul><li>Provide feedback on performance and teach subjects how to improve performance through heuristics and other techniques </li></ul>
  18. 18. Proactive vs. Reactive: The Effect of Experience on Performance in a Critical Care Simulator Jeffrey S Young, MD, Robert L Smith, MD, Jayme B Stokes, MD and Chadrick E Denlinger, MD Presented, Association for Surgical Education, 2005 Published in the American Journal of Surgery, 193(1): 100-104, 2007.
  19. 19. Hypothesis <ul><li>Increased experience, as measured by months of clinical rotations, will decrease errors in critical care simulations </li></ul>
  20. 20. Methods <ul><li>Subjects grouped by ICU experience: </li></ul><ul><ul><li>Novice < 9 wks ICU </li></ul></ul><ul><ul><li>Intermediate (IM) ≥ 9 < 20 wks ICU </li></ul></ul><ul><ul><li>Expert > 20 wks ICU </li></ul></ul><ul><li>All scenarios were digitally recorded for later data analysis. </li></ul><ul><li>Recall, cognitive strategy and errors in management were compared. </li></ul>
  21. 21. Demographics * p<0.05 1Y 4.3±3.9 13.9±2.95* 24±5.7 3 GS, 1 EM 32±4.8 4 Expert (>10 weeks ICU experience) 2Y 5.1±2.0 5.4±3.6* 18.2±8.8 4 GS, 1 EM 28±1.6 5 N-I (0-10 weeks ICU experience) AOA Sleep in past 24 hours ICU Weeks Months of Postgraduate Education Residency Age N Group
  22. 22. Weeks of ICU Experience * p <0.05 # p =0.07 * p <0.05 # p =0.07 21.8±9.9* 3±2.7 3.2±1.3 0.75±0.96* 19.8±2.2# Expert (>10) 4±2.6* 4±2 3.2±1.6 5.6±3.5* 15.2±3.96# N-I (0-10) Proactive Steps Backward Reasoning Forward Reasoning Cognitive Errors Key Clinical Tasks ICU Weeks
  23. 23. Months of Residency All comparisons not significant . 9.25±6.4 4±2.9 3.3±0.95 1.75±2.2 17.25±3.9 >24 14±14.5 3.2±1.8 3.2±1.8 4.8±5 17.2±4.4 0-24 Proactive Steps Backward Reasoning Forward Reasoning Cognitive Errors Key Clinical Tasks Months Residency
  24. 24. Conclusions <ul><li>We uncovered a unique finding that residents with >10 weeks of ICU experience initiated a large number of proactive actions immediately after being presented with the patient information, while those with less than 10 weeks experience in the ICU rarely performed these actions. </li></ul><ul><li>In addition, residents with this degree of experience committed significantly fewer cognitive errors. </li></ul><ul><li>These differences may play a role in efficiency, cost, and overall outcome in the care of ICU patients </li></ul>
  25. 25. War Games
  26. 26. Evolution <ul><li>These were instituted as educational sessions for residents on the Trauma and SICU services in 2002 </li></ul><ul><li>They began as an informal session where I would present cases off the top of my head and take the subject through the case </li></ul>
  28. 29. Hypothesis <ul><li>In simulations of straightforward urgent clinical situations, subjects with increased critical care experience will perform a higher percentage of recommended actions that more inexperienced subjects </li></ul>
  29. 30. Methods <ul><li>Medical students and residents rotating on the trauma and surgical ICU services participated. </li></ul><ul><li>Sessions were conducted by an examiner and included the initial nursing report of an unstable patient. Twelve scenarios were created, focusing on basic floor emergencies. Scores were assigned for clinical actions ordered. These were divided into the initial evaluation (ABC’s), secondary evaluation (labs, studies), diagnosis, follow-up, and total score. </li></ul>
  30. 31. Methods <ul><li>The scenarios were validated by two critical care attendings, and EM attending, and two critical care fellows and these scores were used as the expert group. </li></ul><ul><li>Scores were assigned by two examiners, and the average of the grades in each area was used. </li></ul><ul><li>The scores are a ratio of actual to possible correct responses in each section, and in the entire exercise. </li></ul>
  31. 32. Results <ul><li>Eighty subjects participated in the simulations. </li></ul><ul><li>The group was divided into third-year medical students (MS3), fourth-year students (MS4), first-year residents (PGY-1), and residents beyond their first year (PGY2+). There were 20 subjects in each group. </li></ul><ul><li>Five experts were used as validation </li></ul><ul><ul><li>One Emergency Medicine attending, two critical care attendings and two critical care fellows </li></ul></ul>
  32. 34. Conclusions <ul><li>All groups had significant deficits in cognitive performance compared to experts in the areas of secondary evaluation, follow-up of the presenting problem and total performance in simple clinical scenarios. </li></ul><ul><li>We must design educational systems that rapidly enhance the cognitive performance of students and residents before they are left to independently diagnose and intervene in life-threatening clinical situations. </li></ul>
  34. 36. Hypothesis <ul><li>Subjects who have participated in multiple war game sessions will outperform naïve subjects </li></ul><ul><li>Subjects with greater clinical experience will have augmented gains in performance with multiple sessions </li></ul>
  35. 37. Methods <ul><li>Similar to previous study except that we examined all subgroups, including subjects performed after undergoing more than one session </li></ul><ul><li>Each subject always receives a different case at each session </li></ul>
  36. 38. Results <ul><li>There were 20 subjects in each group and five experts (n=85). </li></ul><ul><li>In total, 227 simulations were performed and graded. </li></ul><ul><li>In naïve subjects, MS3, MS4, PGY-1, and PGY-2+ performed significantly worse than experts in the simulations. </li></ul><ul><li>However, in those subjects who completed 3 “war games” session, all in different scenarios, their performance in every category did not differ significantly from the experts. </li></ul>
  37. 40. Discussion <ul><li>All groups initially had significant deficits in cognitive performance compared to experts in these relatively simple high-risk clinical scenarios. </li></ul><ul><li>We have designed an educational system that rapidly enhances the cognitive performance of students and residents. </li></ul><ul><li>The clinical benefit of this laboratory finding is unknown but it may represent an important tool in assessing and enhancing the competencies of trainees in a safe environment </li></ul>
  38. 41. Critical Incident Interviews
  39. 42. Interviews <ul><li>Based on studies performed by Gary Klein in the late 80’s on NICU nurses, fireground commanders, and marines </li></ul><ul><li>Formed basis of “Naturalistic Decision Making” and “Recognition Primed Decision Making” </li></ul>
  40. 43. RPD <ul><li>RPD reveals a critical difference between experts and novices when presented with recurring situations. </li></ul><ul><li>Experienced people will generally be able to come up with quicker decision because the situation may match a prototypical situation they have encountered before. </li></ul><ul><li>Novices, lacking this experience, must cycle through different possibilities, and tend to use the first course of action that they believe will work. </li></ul><ul><li>The inexperienced also have the tendencies of using trial and error through their imagination. </li></ul>
  41. 44. Recognition Primed DM <ul><li>Experts rarely (if ever) weigh options against each other (differential diagnosis) </li></ul><ul><li>They see a situation, compare it to situations they have experienced previously, bring up how they acted in those situations and war game it in their head </li></ul>
  42. 45. Recognition Primed DM <ul><li>If the mental simulation works out well, they proceed </li></ul><ul><li>If they see something wrong in the mental simulation, they alter the plan and simulate it again (may only take 4-5 seconds) </li></ul><ul><li>If they still are not satisfied, they will go back to traditional comparative analysis of options </li></ul>
  43. 46. Critical Incident Interview <ul><li>Ask novice (year 1-2), “expert” residents (year 4-5) and attendings to describe an incident where they feel they made an exceptional good decision, and then an exceptional bad decision </li></ul><ul><li>Interviewer takes the subject through a detailed process eliciting how they made the decision, and what factors were involved </li></ul>
  44. 47. Critical Incident Interview <ul><li>Hope this will help describe the “naturalistic” decision making processes in medicine, which up to know are poorly understood. </li></ul>
  45. 48. Lessons learned
  46. 49. What we have seen from novices <ul><li>Has learning changed? </li></ul><ul><ul><li>One (or several) students finish case </li></ul></ul><ul><ul><li>Next person begins simulation </li></ul></ul><ul><ul><ul><li>As if nothing had previously occurred </li></ul></ul></ul><ul><ul><ul><li>Same exact mistakes are made </li></ul></ul></ul><ul><ul><li>Requires multiple iterations for changes to take </li></ul></ul>
  47. 50. Final Thoughts <ul><li>Should residents be allowed to start on floor or in unit without some basic competency training and evaluation </li></ul><ul><li>Beginning mandatory set of floor and ICU cases for trauma service, hopefully expand to entire residency </li></ul><ul><li>We must find a way to bring this to the subject since getting people together in a room is becoming impossible </li></ul>
  48. 51. Final Thoughts <ul><li>In new world of duty hours, closer examination of outcomes, and decreasing clinical experience, diffuse learning experience may no longer be optimal </li></ul><ul><li>Focused teaching through deliberate practice in isolated high risk clinical situations should be considered </li></ul>
  49. 52. Thank you