VA Centers of Excellence in Primary Care Education: Evaluation Design and Findings
•
1 like•916 views
Presentation by Annette L. Gardner, PhD, MPH Nancy Harada, PhD, Philip R. Lee Institute for Health Policy Studies, UCSF
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to VA Centers of Excellence in Primary Care Education: Evaluation Design and Findings
Similar to VA Centers of Excellence in Primary Care Education: Evaluation Design and Findings (20)
More from Philip R. Lee Institute for Health Policy Studies
More from Philip R. Lee Institute for Health Policy Studies (20)
Recently uploaded
Recently uploaded (20)
VA Centers of Excellence in Primary Care Education: Evaluation Design and Findings
- 3. VETERANS HEALTH ADMINISTRATION Veterans Health Administra7on (VHA) Statutory Missions 1. Health Care 2. Health Professions Training* 3. Research related to health condi.ons of Veterans 4. Emergency Backup to DoD & part of Nat’l Disaster Medical System (HHS, DoD, VA, FEMA) • VA is the largest single provider of health professions educa.on in the US • 124 of 152 VA medical centers and 3 of 6 independent outpa.ent clinics have affilia.ons for graduate medical educa.on (GME) and almost all VA facili.es have some health professions trainees • VHA’s educa.onal mission is conducted in collabora.on with: o 130 of 141 allopathic & 22 of 29 osteopathic medical schools. o More than 40 other health professions – represented by over 1,800 colleges and universi.es
- 4. VETERANS HEALTH ADMINISTRATION Scope of Academic Affilia7ons Office of Academic Affiliations * 2013 RCS 10-0161 Report 118,799 Total Trainees* 40,420 Medical Residents 22,541 Medical Students 23,808 Nursing Students 31,380 Associated Health Trainees 1,397 Dental Students & Residents 253 Advanced Fellows
- 5. VETERANS HEALTH ADMINISTRATION Centers of Excellence in Primary Care Educa7on (CoEPCE) Purpose • Foster transforma.on of clinical educa.on by preparing graduates of health professional programs to work in and lead pa.ent-centered interprofessional teams; • Study the impact of new interprofessional educa.onal approaches and models on health professions educa.on and workforce outcomes within and beyond VA. • Improve primary care instruc.onal strategies with emphasis on clinical, workplace learning Goal is culture change and increased understanding of other professions culture, values, and educa5on 4
- 7. VETERANS HEALTH ADMINISTRATION • Each site has a locally-developed curriculum that combines 3 instruc.onal strategies: o Workplace Learning (e.g., clinical, point of care) o Didac.c/Formal Instruc.on (e.g., seminars) o Reflec.ve Prac.ce • All sites provide instruc.on in four core domains: o Interprofessional Collabora.on o Shared Decision Making o Sustained Rela.onships o Performance Improvement • Trainees learn and work together per PACT model • Trainee engagement at least 30%: o Physician residents trainees o Nurse Prac..oner trainees o Post-Doctorate Pharmacy residents o Post-Doctorate Psychology fellows 6
- 9. VETERANS HEALTH ADMINISTRATION 8 1. Transforma7ve center Interprofessional Educa7on and Collabora7ve Prac7ce 2. Expanded par7cipant knowledge, aTtudes, prac7ces, understanding and competencies 3. Increased par7cipant interprofession al collabora7on 4. Improved par7cipant clinical performance 5. Improved center pa7ent outcomes: 6. Improved center par7cipant sa7sfac7on 7. Improved center primary care services 8. Improved center func7on 9. Improved VA primary care delivery system Short-term Outcomes (2011-2012) Intermediate Outcomes (2012-2014) Long-term Outcomes (2015+) CoEPCE Logic Model + Contextual Analysis: health care system (structure, financing, accredita7on, licensure) + Four to six indicators per outcome
- 10. VETERANS HEALTH ADMINISTRATION Data Collec7on Ac7vi7es ü Semi-structured interviews with trainees (N = 35, 16%), faculty (N = 8), CoE leadership (N = 10) to describe par7cipant experience, facilita7ng factors and challenges, and trainee sa7sfac7on (2014) ü Par7cipant Survey administered to Past trainees (N=88, 42%) and Current trainees (N = 130, 82%) to assess sa7sfac7on and impacts (2015) ü Semi-Annual Site Reports and CoE documenta7on of program development, implementa7on and sustainment (2012 – 2015) ü Nurse Prac77oner Resident Competency Tool (2012 – 2015) ü Five Case studies to document successful site interprofessional educa7on and/or clinical ini7a7ves (2015) ü Curriculum Analysis to document CoE models and their evolu7on (2012 – 2015) ü Feasibility Analysis of VA Clinical Data Sets (2015) ü Financial Assessment of CoE costs and resources secured (2015) 9
- 11. VETERANS HEALTH ADMINISTRATION Program Outcome 1: TransformaSve center interprofessional educaSon and collaboraSve (IPEC) pracSce Evidence of Achievement: • Launch of interprofessional, pa.ent-centered medical home model of training and providing new educa.on training opportuni.es for trainees • Launch of workplace, clinical learning opportuni.es, e.g., huddles, models of longitudinal engagement, NP Residency programs • Successful trainee recruitment and increase in number and type of professions Facilita7ng Factors: • CoEPCE resources, e.g., NP s.pends • High level of par.cipa.on by faculty and trainees Challenges: • Aspects of the VA facility—space, culture, protected .me for faculty • Trainee diversity, including schedules, clinical exper.se
- 12. VETERANS HEALTH ADMINISTRATION Program Outcome 1: TransformaSve center interprofessional educaSon and collaboraSve pracSce Profession AY 11-12 AY 12-13 AY 13-14 NP Residents 3 8 12 Nursing Students 20 26 30 NP Psych-Mental Health 1 1 2 Medicine 79 104 121 Pharmacy Residents 14 17 25 Psychology Trainees 7 8 16 Totals 124 164 206 11 * Rugen, K.W., et al., “Veteran Affairs Centers of Excellence in Primary Care Educa.on: Transforming nurse prac..oner educa.on., Nursing Outlook. 62 (2014) 78-88
- 14. VETERANS HEALTH ADMINISTRATION Program Outcomes 2, 3, 4 – ParScipant Knowledge and Performance Over 80% of Past and/or Current trainees learned and prac7ced most of the competencies in the four core domains—sample findings 13 96% 95% 87% 91% 79% 95% 97% 96% 95% 79% Learned Prac.ce % of Past and Current trainees that said they ‘learned’ (Yes) and ‘prac.ced’ a skill (Agree/Strongly Agree) Providing clinical care as a member of PACT (Current trainees) Valuing the unique roles and exper.se of other health professionals Communica.ng my roles and responsibili.es clearly to other health professionals (Current trainees) Developing trus.ng rela.onships with pa.ents and families/caregivers Using performance improvement strategies to increase the efficacy of team-based care
- 22. VETERANS HEALTH ADMINISTRATION Summary of Findings • CoEPCE is succeeding in: – Expanding Center adop.on of interprofessional curriculum and clinic models – Expanding trainee knowledge and skills in four core domains albeit to varying degrees – Increasing trainee interprofessional collabora.on and a team-based approach to pa.ent care – Improving trainee clinical performance and integra.on with PACT team – Improving trainee sa.sfac.on with CoEPCE and many individual components – Improving Center primary care and func.on – Improving VA primary care delivery system • Areas for Improvement: – Trainee knowledge and use of Performance Improvement skills – Trainee mentoring and assessment of progress – Site-level monitoring of specific curriculum components, e.g., classroom didac.c sessions 21
Editor's Notes
- Good morning – I appreciate the opportunity to present the design and findings from an exciting, 5-year Interprofessional Education initiative at the VA
- I won’t talk through all the information on these next few slides describing the VA and its role in health care professional education. Suffice it to say that the VA is a very important component of the U.S. health care delivery system, providing many times exemplary care to over 21 million veterans.
- It also takes very seriously its role in training future health care professionals, ranking it second in its 4-part mission. It is the largest provider of health professions education in the U.S.
- Here is the break-out by profession
- In 2010, it launched a new educational initiative to prepare health care professionals to work in and lead patient-centered, interprofessional teams. It was an opportunity to develop and study interprofessional education strategies, particularly in the primary care setting Really – it is about supporting culture change and cross-profession knowledge and ability to work as teams – teams that share the care. This comes at a time with the VA is implementing its version of the patient-centered medical home or PACT – Patient Aligned Care Teams.
- Here are the five sites. Two new ones were brought on board in 2015, with the program extending another 4 years. All of them had some capacity in curriculum development and commitment to team based care.
- Each sites had significant discretion in developing its curriculum thought there were some common instructional strategies Four core domains drive the curriculum with some differences among sites Moreover, trainees work together as teams to provide care Particularly important is the dose effect or committing enough time to working in the clinic as a team.
- In 2013 we rebooted the enterprise evaluation, paying close attention to the top layer of our pyramid.
- 8
- Here are our eight data collection activities through FY 2015. They are a combination of qualitative and quantitative methods that assess multiple aspects of the program and provide evidence achievement of all 9 outcomes. We relied heavily on triangulation to increase the validity, such as pairing the interviews and the participant survey instruments. Also, since this is a demonstration project with differences across the sites, we looked to case studies.
- We completed the data collection and analysis in fall 2015 and reported the findings by the logic model outcomes: Outcome 1 is really about increased center capacity to do successful interprofessional education. We’ve documented this outcome using a variety of data collection activities, starting early in the program.
- Here is the data on successful trainee recruitment, with Centers reporting more applicants than they can accommodate. We see expansions in: Annual totals Associated Health trainees The launch of a new NP Residency program
- The participant survey was particularly useful for assessing Center capacity. The news was positive and the clinical and team activities were rated highly while the VA facility components were rated less highly in effectiveness.
- Here are sample findings on Outcome 2, 3, 4 and trainee self-report on knowledge and performance, with performance improvement rating lower. Note: the intention is to corroborate this data with administration of the survey to faculty under stage 2. Also, we have another instrument to assess NP trainee knowledge and performance.
- I won’t read through all the findings – we just wanted to illustrate how deeply we drilled and what can be informative to sites.
- The findings on Outcome 6 or trainee satisfaction were positive as well as informative on what could be strengthened.
- The trainee interview data has been very illuminative – not just on what works and doesn’t work but how these trainees are being personally and professionally effected by the program.
- Perhaps the one ‘surprise’ finding was how in an enriched program, trainees still think sites are coming up short in providing targeted assessment and career development – a Center function outcome
- We’re getting toward the end of the logic model and longer-term outcomes or the impact of the program on VA primary care delivery. Modest gains here, with limited differences by trainee cohort, which may be due to other factors than their experience in the program.
- The bigger change was trainee interest in working at the VA. Trainees like working with complex patients
- What are these findings due to?
- Stepping back, what did we learn: Accomplishment of most of the program outcomes, except being able to document improved patient clinical outcomes Findings that can be used to improve the program at the national level and the site level
- What did we learn that can inform evaluation practice more broadly? Don’t underestimate the role of the context, particularly resources and an uncertain funding environment, uneven leadership commitment (to PACT, to IPE), and local VA facility support---even with a VA-wide mandate to implement PACT BUT we were able to parlay our design and instruments and take some steps in strengthening VA evaluation culture.
- Last, this was a huge enterprise and a lot of people deserve a lot of credit for developing this 15-minute presentation—another contextual element.