Principal investigator: Richard M. Wardrop III, MD, PhD, FAAP, FACP, WakeMed Faculty Physicians, Internal Medicine and Pediatrics, Assistant Professor at Virginia Tech Cailion School of Medicine and the University of North Carolina School of Medicine The Resident Performance project intended to adapt an existing attendant-based evaluation into a patient-centered prototype tool that is concise, valid and reliable, and that enables patients to accurately assess resident performance on 4/6 ACGME competencies. Performance with regard to ACGME core competencies of residents who receive feedback and coaching using the patient-centered tool was compared to that of those who received attending-only feedback.

1. Patient and Family Centered Care (PFCC): Lessons from Graduate Medical Education Medicine Grand Rounds March 15th, 2011 Richard M. Wardrop III, M.D., Ph.D. WakeMed Faculty Physicians

2. Disclosure I had grant support to perform research from the Picker Institute and ACGME I currently serve as an external reviewer to the Picker Institute and their challenge grant program

3. Full Disclosure - My Research Focus Then Now

4. Objectives Introduction Review what PFCC is Review history of PFCC Give resources for PFCC practice Show examples of successful implementation of PFCC in GME and beyond Research Share some data from my experience at Carilion Mixed methods project Speculate on what we can do here in PFCC and in GME

5. Background Setting The PFCC movement is enormous There is no fixed history Multiple players on multiple levels The work of others is very humbling Just good medical practice? We have to start somewhere….

6. How can I get you to practice PFCC? Patients like it? Patients feel empowered? It saves money? It leads to safer care? It does not cost anything extra? Anyone can do it?

7. We are (I was 2008 – 2010)…..Carilion Clinic 500+ physicians in a multi-specialty group practice and eight not-for-profit hospitals. Specializing in patient-centered care, medical education, and clinical research.

8. What is PFCC? “Patient- and family-centered care is an innovative approach to the planning, delivery, and evaluation of health care that is grounded in mutually beneficial partnerships among health care patients, families, and providers.” “Patient- and family-centered care applies to patients of all ages, and it may be practiced in any health care setting “ www.familycenteredcare.org

9. Core Values of PFCC Dignity and Respect Information Sharing Participation Collaboration

10. Core Values of PFCC – Picker Institute Respect for patient values, preferences, and expressed needs Coordination and integration of care Information, communication, and education Emotional support and alleviation of fear and anxiety Inclusion of family in care Transition and continuity Physical Comfort Access to care www.pickerinstitute.org

11. Core Values of PFCC Belief oriented….Planetree Foundation that we are human beings, caring for other human beings we are all caregivers care giving is best achieved through kindness and compassion safe, accessible, high quality care is fundamental to patient centered care is a holistic approach to meeting people's needs of body, mind and spirit families, friends and loved ones are vital to the healing process

12. Core Values of PFCC Belief oriented….cont access to understandable health information can empower individuals to participate in their health. the opportunity for individuals to make personal choices related to their care is essential physical environments can enhance healing, health and wellbeing illness can be a transformational experience for patients, families and caregivers www.planetree.org

13. Pause Pause…. Are you saying…. “I agree with that – why do we need to hear this?” “I already do all these things…I’m good” Or are you saying “I want to do more but the system won’t let me….” “I’m to busy to worry about this…..” Out of respect for my audience… welcome your thoughts

14. Brief History of PFCC – Divergent Events / Efforts Converging on a point…. Started with the Planetree Foundation in 1978 Had many roots in pediatrics and HIV/AIDS population A system in need of major change – Institute of Medicine Reports and C.E. Coop

15. Brief History of PFCC The Institute for Family Centered Care Picker Institute for Patient Centered Care Center for Patient and Family Centered Care - MCG National Priorities Partnership AHRQ, CDC, CMS, IOM, Joint Commission and 20+ others Defined Priorities in Patient Care – many of which adhere to PFCC principles www.qualityforum.org/about/npp

16. Brief History of PFCC Other Organizations involved with guidelines or statements SCCM ABIM American Hospital Association American Academy of Orthopedic Surgeons Countless medical centers recognized

17. Resources Partnering with Patients and Families to Design a Patient- and Family-Centered Health Care System: Recommendations and Promising Practices – free 178 page document Supported by RWJ Other titles available from evidence based architecture to bibliographies for resources and references http://www.familycenteredcare.org/resources/other/index.html

18. Resources Picker Institute Picker Surveys Challenge grants Profiles of successful Centers Downloadable seminars and workshops Other grants and resources http://www.pickerinstitute.org/Research/pickerchallenge.html

19. Reading and Resources for GME GME Assessment and Tips Books Downloads….from ABIM http://www.abimfoundation.org/

20. Making it real….. Real examples from a large teaching institution……

21. Beacon of PFCC Medical College of Georgia http://cpfcc.org Started movement in 1993 by including patients in design of Children's Hospital In 2002 set behavioral standards for all staff

22. MCG – UME and GME Since 1993 Institutional commitment to change clinical environments to PFCC Developed family faculty for patients and families to serve as expert advisors and teachers Foster resident involvement with family and patient advisors Create opportunities that allow residents to apply this to their practice Developed standards for how teaching rounds will take place

23. MCG – UME and GME Teaching examples at MCG MS Clinic Behavioral Health Cystic Fibrosis Family Medicine Clinic Cancer Center Peri-natal clinic Research ePHR HTN project Involve patients and families in faculty and resident recruitment Integrate patients and families into research efforts

24. Relating ACGME Competencies to PFCC at MCG in GME

25. PFCC Success….in a clinical unit Neurosciences Center of Excellence - at MCG: PFCC experience Patient satisfaction 10%-95th%tile Length of stay on Neurosurgery decreased 50% 62% decrease in medication errors Staff vacancy from ~8%  0% (wait list)

26. MCG Institutional Improvement? Increased / Improved Patient satisfaction Clinical quality Payer mix Market share Profitability Decreased Malpractice claims Show me the Money! Featured in PBS Documentary – Remaking of American Medicine

27. Summary PFCC has a 30 year history at least At its deepest levels, we probably all agree with it Differences are in the execution Research exists Success stories exist Real improvements in a medical center can happen It can not only co-exist with GME but also drive GME

28. Resident performance from the patient’s view: A novel prospective assessment of performance and performance improvement in delivering patient-centered care

29. A common motivating theme is some experience….. Dr. Harvey Picker – Picker Institute Angelica Thieriot – Planetree Me – what got me interested?

30. Grand Parents and Parents

31. The questions I started having as I started at Carilion Should doctors finish training without getting feedback from patients? If we give them feedback will they be better in cultivating good communication and productive relationships with patients and families?

32. Medical Educators – Patient Care, Supervision, Teaching, and Feedback

33. The Problem As a young attending I felt a few pieces of information were missing as I went to fill out the ACGME competency based evaluations….

34. The Art of Feedback Make observations and collect facts Time and data dependant Need a framework to standardize feedback Provide a mirror image

35. The Problem It occurred to me during my 7th or 8th two week block… when a patient fired one of the residents I was supervising and then a nurse complained about a different resident then a patient gave the same resident praise…. I was not really sure what was going on between the residents and patients when I was not around to observe them.

36. What to do? Spying? Using a wire tap? Video taping? Interrogating the patients? Sneaking around?

37. What to do? Patient care is complex and emotional Supervising residents is complex There are only 24 hours in a day But I need to know!

38. What to do? Sentinel events (firing by patients, complaints, or praise) cannot dominate the entire recorded experience all the time…. How to empower and enlists the patients in the feedback process consistently?

39. The Resources What did we have at our disposal to solve this problem...?

40. What we had experimented with…. How useful? Learner Centered? Patient Centered?

41. Evolved into this…. Learner Centered? Patient Centered?

42. Opportunity knocks… We had problem / hypothesis Call for grants from OSP Struck me at the right time I had time We had core team We had resources There was money up for grabs

43. Eureka! Make a new mousetrap… Distribute it to everyone Share the feedback with the residents Ask the patients anonymously about the residents using a survey Make it competency based Make it portable Make it easy

44. Make a new mousetrap… Most of all make it patient centered

45. Hypothesis and Aims Hypothesis - the regular use of the a patient-centered, core competency-based survey tool combined with specific learner centered feedback would improve the performance of residents in delivering patient-centered care when compared to the conventional practice of attending-only assessment and feedback.

46. Hypothesis and Aims AIM 1: To adapt the evaluation tool into a concise, valid and reliable instrument that enables patients assess resident performance on 4/6 ACGME competencies. AIM 2: To compare residents who receive feedback and coaching using the tool developed under AIM 1 to traditional attending-only assessment and feedback.

47. Picker Challenge Grant Program Started in 2005 for projects in 2006 4 per year at $25,000 Requires matching funds (in kind) 2008 cycle had 119 applications Grants focused on research in PFCC in GME Picker Principles $50,000 total / project Transitioning Adolescent Patients (TAP) from Pediatric to Adult CarePrincipal investigator: Emily von Scheven, MD, MAS, Pediatric Rheumatology, University of California San Francisco Improving Patient Rounds (IPR)Principal investigator: Walter J. Moore, MD, Center for Patient- and Family-Centered Care, Medical College of Georgia Patient-Centered Training of Residents on a Medical Ward Principal investigator: Robert C. Smith, MD, MS, Internal Medicine,EW Sparrow Hospital/Michigan State University College of Human Medicine Resident Performance from the Patient’s ViewPrincipal investigator: R.M. Wardrop, MD, PhD, FAAP, Director of Resident Research, Internal Medicine, Carillion Clinic, Roanoke, Va.

48. Our team upon receiving award

49. The tool

50. The tool – used by patients, residents and attending

51. The Picker Principles - Assessed Respect for patient values, preferences, and expressed needs Coordination and integration of care Information, communication, and education Emotional support and alleviation of fear and anxiety Transition and continuity

52. Hypothesis and Aims AIM 1: To adapt the evaluation tool into a concise, valid and reliable instrument that enables patients assess resident performance on 4/6 ACGME competencies. AIM 2: To compare residents who receive feedback and coaching using the tool developed under AIM 1 to traditional attending-only assessment and feedback.

53. End points Quantitative Reliable? Valid and on how many scales? Measure performance Effect on performance when combined with feedback? Qualitative Patients appreciate? Residents appreciate? Comments have any meaningful content?

54. Other measures Residents evaluate themselves in these patient centered domains Attending physicians evaluate residents in patient centered domains Conduct patient-centered interviews regarding the process

55. Study Design

56. Study Design Study Attending Intervention – provide usual coaching + coaching derived form PERPS) Attending Intervention – provide usual coaching July 2008-June 2009 Month 1 Month 2 Month 3 Month 4 Feedback 1-1 AERPS, RERPS Feedback 1-2 PERPS, AERPS, RERPS Group 1 (Usual evaluation) End evaluation, correlations Month 1 Month 2 Month 3 Month 4 Group 2 (Patient Centered evaluation + usual evaluation) Feedback 2-1 PERPS, AERPS, RERPS Feedback 2-2 PERPS, AERPS, RERPS Study Attending Intervention – provide usual coaching + coaching derived form PERPS) Study Attending Intervention – provide usual coaching + coaching derived form PERPS)

57. Patient Questionnaire Distribution

58. Patient Questionnaire Collection

59. We did the unthinkable… Selected patient interviews by study faculty To focus survey Have patients tell us what the questions mean to them in their own words.

60. Patient Interview Data

61. Patient Interview Data

62. Patient Interview Data

63. Patient Interview Data

64. Patient Interview Data

65. Scale Validity and Reliability Reliability (consistency) Consistency in the ratings on any measure is important Differences in rating not due to chance Measure Chronbach’s Alpha across all raters and ACGME domains All were >0.75 indicating the scale and the raters were using it consistently Validity (accuracy) Assessing validity determines whether or not the survey is measuring what it was intended to measure The instrument had built-in content validity Structural validity testing only found 1 valid scale with no subscales (patients and attending rated with little variability between domains (good in one good in all)

66. Sample Comments from Patients (over 300 written) The good “I know this looks like we just circled all 5's but Dr B. truly met all of them. Our family was REALLY impressed with Dr B. of all the doctors we have seen. He stands at the top.” +/PC/ICS/P “Dr T shows great leadership, he is a listener and very helpful to me about getting to the problem and ruling out disease and ordering tests and explaining special tests. He LISTENS. That smile also will take him a long way. I'm glad he was on my team of recovery. My heart goes (out to him).” +/P/PC/ICS “Very compassionate, caring and professional. Takes time to listen to pts. (the resident) explained everything to myself and family in layman's terms to understand. Excellent Dr. Wish him success in the world. Thanks for excellent care.” +/PC/P/ICS Other “He (the resident) seems caring my only problem is its hard to understand his when he's talking. Otheriwise he seems OK. I've only seen him once since my husband's been here. Everyone else has been very good to him.” +/-P/ICS “No contact. I have been in the hosp for over 35 hrs in this 3 day stay and never spoke with this Dr. (written by wife)” - P/ICS

67. Content of Comments from Patients (n=100)

68. Next Phase – Learner Feedback Standardized Scripted Based on scores from patients, self, and attending Open ended and closed ended questions

69. Resident Reponses to Feedback

70. Resident Reponses to Feedback

71. Sample Resident Comments Name two distinct things you learned from this process about yourself or about how patients view you. “I thought the patients would be more concerned with my medical decision making” “glad to know I helped my patients so much – it makes it easier to work so hard knowing this” “Patients value giving input into care” “Patient’s know what is going on with their care” “It was more positive than I expected” “the patients view me more favorably than I thought” “I am my own worst critic” “Patients think I am a great doctor” “I need to interrupt patients less” What one thing will you commit to change because of this process? “more discussion of the plan with patients or family” “I will stop interrupting patients so much” “Maybe going back to round on my patients in the afternoon (to update them) “trying to better incorporate patient preferences into management” Comments and suggestions? “Get as many back from patients as possible to assure accuracy” “Worried attending physicians may not be best to hand out the surveys may skew the results towards good evals to not get anyone in trouble” “Pleasant patients may get more attention” “Patients with multiple co-morbidities may not get a form as frequently and this could create bias”

72. Resident Performance – prior to feedback No significant differences between groups prior to feedback

73. Resident Performance – prior to feedback No significant differences between groups prior to feedback

74. Resident Performance improved when associated with having received feedback – communication, patient care, systems based practice * * * * *

75. Challenges for prime time use Curriculum Making sense of the data Validity and reliability testing in your populations Survey collection and distribution Finding meaning in the non-numerical data

76. Conclusions Patients provided regular feedback using the instrument and scales Patient appreciated providing feedback The survey was reliable and valid. Scale validity for one scale Providing feedback during the year improved performance of residents in the patients’ and the attending physicians’ “eyes” in several areas

77. Conclusions Patients regularly provided qualitatively rich and competency-based feedback Residents appreciate the feedback from the patients They feel it helps direct them for self- improvement in these areas

78. Implications for WakeMed Use these resources / ideas at your own level Hospitalist face-card Do what you do only potentially change focus PFCC rounds on Pediatrics Stroke Rounds with MD Know who else feels this is important Ideas?

79. Many thanks to…. The Picker Institute ACGME leadership and staff Carilion Clinic Our team Our faculty Our residents Our patients

80. Core Study Team Richard M. Wardrop III, MD, PHD, FAAP, FACP Chad J. DeMott, MD, FACP Jon M. Sweet , MD, FACP – Program Director David Baker, PhD Robert Herbertson, MS SowjanyaKolluri, MD RoshanBowansingh, MD Study Coordinators Dawn Bowles, RN Jacqueline Baker, RN Grant Management: W. Eryn Perry

81. Thanks and Questions?

82. Selected Bibliography Putting Patients First: Best Practices in Patient Centered Care, 2nd Ed. Susan Frampton and Patrick Charmel eds. Josey-Bass Publishers. San Francisco, 2008. Partnering with Patients and Families to Design a Patient- and Family-Centered Health Care System: Recommendations and Promising Practices Institute for Family-Centered Care – 2008. Available online at www.familycenteredcare.org National Partnership Priorities Executive Report. National Priorities Partnership – at the National Quality Forum. November 2008. www.qualityforum.org Synthesis of Definitions of Patient-, Family-, and Relationship-Centered Care. Amy Cunningham. ABIM Foundation. www.abimfoundation.org Patient- and Family-Centered Care and Graduate Medical Education: A Primer. Beverely H. Johnson. Presented at 2009 ACGME Educational Conference, Grapevine, TX. www.acgme.org Patient- and Family-Centered Care and Resident Learning. Patricia Sodomka. Presented at 2009 ACGME Educational Conference, Grapevine, TX. www.acgme.org

83. Selected WebBibliography Extensive references for specific specialties in PFCC available at the Institute for Family Centered Care www.familycenteredcare.org Planetree Foundation. All in one site with complete model and programs. www.planetree.org Additional resources results, links, survey tools, description of challenge grant awards and results available at www.pickerinstitute.org Center for Patient and Family Centered Care at UCG. Links, research in PFCC and GME research www.cpfcc.org