1. Principal Investigator
Ianita Zlateva | Evaluation Specialist &
Researcher
Research Team
Co-Investigator
Amanda Schiessl | Project Director/Co-PI,
NTTAP, Clinical Workforce Development
Amanda@chc1.com
Co-Investigator
Nashwa Khalid | Project Analyst, NTTAP
Senior Investigator
Margaret Flinter | SVP, Clinical Director &
Co-PI, NTTAP

2. www.nachc.org | 2
Community Health Center Inc.
Locations and Service Sites in Connecticut
CHCI Profile:
 204 sites service delivery sites
 1300 staff
 Patients/year: 105,000
 Visits/year: 600,000
 Specialty access by eConsult to all specialties
Weitzman Institute:
 NTTAP Clinical Workforce Development
 Formal Research
 Leads Quality Improvement /Practice Redesign
 Educational arm for both current and future
workforce nationally
 Consults to other FQHCs and health systems
through WILA
Health Professions Student Education and Training:
Fall 2019: 252 students/20 university/school
relationships across all disciplines and levels,
certificate through doctoral
 Accredited CHC postgraduate training programs
 Postgraduate NP residency (est. 2007):
(family, adult, pedi, psychiatry)
 Postdoctoral clinical psychology (est.
2010)

3. HRSAs Goal
“Develop a Readiness to Train Assessment Tool, which will help
health centers assess and improve their own organization’s
readiness to engage in health professional training programs.
Additionally, a report on the “readiness to train” status of health
centers will also be an outcome of this strategy, for the purpose of
directing HRSA workforce investments to address the priority
needs of health centers in the future”.

4. Study Objectives
Develop and validate a survey instrument that:
i) can measure and assess health center readiness to engage with
and implement health professions training (HPT) programs.
ii) is based on organizational readiness theory and experts’
judgement of the most important factors influencing successful
HPT program implementation.

5. The 3 Pillars of a Well-Designed Measure
Solid theoretical/conceptual
framework
Sound development and
validation methodology
High quality data
If the 3 conditions are met
the measure can be used
for decision-making

6. Assumptions Based on Previous Research
• The extent to which engagement with HPT programs will be
successful will differ between health centers
• Many factors may pose significant challenges to health centers
launching HPT programs
• Evaluation of organizational readiness would allow for early
identification and mitigation of barriers to implementation

7. Study Design
Phase 1
• Extensive literature review by the research team and development of
conceptual framework.
Phase 2
• Conceptual framework validated by 20 experts in two focus groups.
• Conceptual framework used to create the initial survey item pool.
Phase 3
• Survey items selected according to consensus opinions of thirteen
experts in three Delphi rounds.
Phase 4
• Pilot test of survey and statistical validation.
Readiness to Train Assessment Tool™ (RTAT™)
A validated, reliable survey tool

8. Pilot Test Respondents
• Health center employees responding to the survey:
– 212 completed the entire survey
– 76.5% over the age of 40
– 71% female
– White/Non-Hispanic 61.5%, Black/African-American 13.5%
– 36.9% with more than 10 years of HPT experience.
• Represented health centers:
– from 41 US states and US Territories
– designated as urban - 52.1%
– serving under 25,000 patients - 46.5%

9. Theory of Organizational Readiness for Change
“Organizational readiness for change is a multi-level, multi-faceted
construct. As an organization-level construct, readiness for change
refers to organizational members' shared resolve to implement a
change (change commitment) and shared belief in their collective
capability to do so (change efficacy).”
Bryan Weiner, PhD
Author: Theory of Organizational Readiness for Change (2009)

10. ‘Organizational readiness for change’ is the degree to which
health centers are motivated and capable to engage with and
implement health professions training programs.
Construct to Measure

11. • Any formal organized education or training undertaken for the
purposes of gaining knowledge and skills necessary to practice a
specific health profession or role in a healthcare setting.
• Types of HPT programs (e.g., shadowing, rotations, affiliation
agreements, accredited or accreditation-eligible programs)
• At any educational level (certificate, undergraduate, graduate,
professional and/or postgraduate)
• In any clinical discipline
Health Professions Training

12. Conceptual Framework*
Networking,
External
Policies &
Incentives
Available Resources,
Leadership
Engagement,
Culture,
Implementation
Climate
Change
Efficacy,
Knowledge
& Beliefs
about HPT
Planning,
Engaging,
Executing,
Reflecting &
Evaluating
Relative
Advantage,
Complexity
External Context
ProcessCharacteristics
of Individuals
Organizational
Characteristics
Characteristics of
the HPT program
* Based on the Consolidated
Framework for Implementation
Research (Damschroder et al.,
2009)

13. • Followed recommendations for selecting the best and most robust
survey item combination
306 items = initial survey item pool
• Rewording and mapping the items to a relevant domain/subdomain in
the conceptual framework
182 items = for assessment in the Delphi process
Survey Item Pool

14. Delphi Process for Item Selection
• A modified, electronic Delphi technique
• Assessment of the survey items by 13 experts
– Appropriateness and ability to measure the relevant
domain in the conceptual framework (rounds 1 and 2)
– Importance for organizational readiness to engage with HPT
programs (round 3)
– suggest changes, reword, propose new survey items
• Consensus reached for 65 survey items

15. Survey Items
Please respond on a scale of 1-5:
1=strongly disagree; 2=disagree; 3=neutral; 4=agree; 5=strongly agree
Our health center has enough evidence to support that:
The health professions training program is flexible enough to be redesigned to meet the needs of our
health center.
If the organization engages with this health professions training program, it will be beneficial for our
health center workforce.
The program will help trainees better adapt to relevant best practices at health centers.
The health professions training program has clear structure and goals.

16. Rigorous Statistical Testing
• Completed extensive array of statistical tests using high quality data from
the pilot test survey respondents
• Assessed the psychometric properties of the newly developed and
validated survey instrument
• Developed and validated state-of-art survey instrument for
organizational readiness grounded in rigorous analytic methodology

17. Readiness to Train Assessment Tool
• The resultant 41-item, 7-subscale structure of the survey was derived through
exploratory factor analysis. Cronbach’s alphas (.79 -.97) indicated good to
excellent reliability.
• The instrument covers dimensions of health center readiness for engaging with
HPT programs that were deemed critical to evaluate by the project’s subject
matter experts.
• The advantage of the RTAT™ is that it covers organizational readiness dimensions
that are relevant to all kinds of health professions training programs and types of
health centers.

18. Subscales of the RTAT
The seven sub-scales that emerged from the data analysis represent seven
areas of readiness:
– Readiness to engage (8 items),
– Evidence strength and quality of the HPT program (4 items),
– Relative advantage of the HPT program (4 items),
– Financial resources (3 items),
– Additional resources (3 items),
– Implementation team (4 items), and
– Implementation plan (15 items).

19. Overall Readiness Scale and 7 Subscales
Sub-scale Readiness to Engage
Evidence Strength &
Quality of the HPT
Program
Relative Advantage of
the HPT program
Financial Resources Additional Resources Implementation Team Implementation Plan
Brief Description
Indicators of the health
center’s overall
readiness and
commitment
to engage with health
professions training.
Stakeholders’
perceptions of the
quality and validity of
evidence supporting
the belief that the HPT
program will have
desired outcomes at
their health center.
Stakeholders’
perceptions of the
advantage of engaging
with/implementing the
HPT program versus an
alternative solution.
The level of financial
resources dedicated
for implementation
and ongoing
operations.
The level of additional
resources dedicated
for implementation
and on-going
operations, including
appropriate staff and
assistance for staff
(e.g. evaluation
resources, tools,
training, and
coaching).
This subscale is about
the individuals
involved with the HPT
implementation
process who can
formally or informally
influence this process
through their
knowledge, attitudes,
and behaviors. They
are effective in
overcoming
indifference or
resistance that the
implementation of an
HPT program may
provoke in the health
center.
This subscale is
associated with the
implementation
process. Successful
engagement usually
requires an active
change process aimed
to achieve effective
implementation of the
HPT program(s). The
subscale measures the
degree to which a
scheme or method of
behavior and tasks for
implementing an HPT
program are developed
in advance, and the
quality of those
schemes or methods.
Number of Survey
Items
8 4 4 3 3 4 15