The document outlines the development and validation of the Readiness to Train Assessment Tool (RTAT), designed to assess health centers' readiness to implement health professions training programs. It details the study's objectives, methodology, and results from a pilot test, emphasizing the tool's relevance and structured design across various health professions training types. The initiative aims to enhance workforce capabilities by facilitating training partnerships and addressing barriers to program implementation.

1. Readiness to Train Assessment Tool™
The Development and Validation of a Tool to Assess
Health Center Readiness to Train Health Professionals
September 23rd, 2020
3:00 pm EST

2. Principal Investigator
Ianita Zlateva | Evaluation Specialist &
Researcher
Research Team
Co-Investigator
Amanda Schiessl | Project Director/Co-PI,
NTTAP, Clinical Workforce Development
Amanda@chc1.com
Co-Investigator
Nashwa Khalid | Project Analyst, NTTAP
Senior Investigator
Margaret Flinter | SVP, Clinical Director &
Co-PI, NTTAP

3. | 3
Community Health Center Inc. CHCI Profile:
 204 sites service delivery sites
 1300 staff
 Patients/year: 105,000
 Visits/year: 600,000
 Specialty access by eConsult to all specialties
Weitzman Institute:
 NTTAP Clinical Workforce Development
 Formal Research
 Leads Quality Improvement /Practice Redesign
 Educational arm for both current and future
workforce nationally
 Consults to other FQHCs and health systems
through WILA
Health Professions Student Education and Training:
Fall 2020: 168 students/20 university/school
relationships across all disciplines and levels,
certificate through doctoral
 Accredited CHC postgraduate training programs
 Postgraduate NP residency (est. 2007):
(family, adult, pedi, psychiatry)
 Postdoctoral clinical psychology (est.
2010)
Locations and Service Sites in Connecticut

4. HRSA Health Professions Education
and Training Initiative (HP-ET)
HRSA HP-ET Initiative will enhance health centers capabilities to
recruit, develop, and retain their workforce by exposing health
and allied health professions students, trainees, and residents to
education and training programs conducted at health centers.
Today with Macrae: Health Equity Panel Discussion, HRSA Workforce Training Initiative and More! HRSA PHC Digest,
https://content.govdelivery.com/accounts/USHHSHRSA/bulletins/2a0aa86 September 15th, 2020

5. Goal of the HP-ET Initiative
The HP-ET Initiative will use the Readiness to Train Assessment Tool (RTAT)
developed by Community Health Center (CHC), Inc., a HRSA-funded
National Training and Technical Assistance Partner (NTTAP), to help health
centers assess and improve their readiness to engage in health professions
training programs. The tool covers dimensions of health center readiness for
developing and engaging with health professions training programs.
Today with Macrae: Health Equity Panel Discussion, HRSA Workforce Training Initiative and More! HRSA PHC Digest,
https://content.govdelivery.com/accounts/USHHSHRSA/bulletins/2a0aa86 September 15th, 2020

6. Background: Developing RTAT
“Develop a Readiness to Train Assessment Tool, which will help
health centers assess and improve their own organization’s
readiness to engage in health professional training programs.
Additionally, a report on the “readiness to train” status of health
centers will also be an outcome of this strategy, for the purpose of
directing HRSA workforce investments to address the priority
needs of health centers in the future”.

8. Study Objectives
Develop and validate a survey instrument that:
i) can measure and assess health center readiness to engage with
and implement health professions training (HPT) programs.
ii) is based on organizational readiness theory and experts’
judgement of the most important factors influencing successful
HPT program implementation.

9. The 3 Pillars of a Well-Designed Measure
Solid theoretical/conceptual
framework
Sound development and
validation methodology
High quality data
If the 3 conditions are met
the measure can be used
for decision-making

10. Assumptions Based on Previous Research
• The extent to which engagement with HPT programs will
be successful will differ between health centers
• Many factors may pose significant challenges to health
centers launching HPT programs
• Evaluation of organizational readiness would allow for
early identification and mitigation of barriers to
implementation

11. Study Design
Phase 1
• Extensive literature review by the research team and development of
conceptual framework.
Phase 2
• Conceptual framework validated by 20 experts in two focus groups.
• Conceptual framework used to create the initial survey item pool.
Phase 3
• Survey items selected according to consensus opinions of thirteen
experts in three Delphi rounds.
Phase 4
• Pilot test of survey and statistical validation.
Readiness to Train Assessment Tool™ (RTAT™)
A validated, reliable survey tool

12. Pilot Test Respondents
• Health center employees responding to the survey:
– 212 completed the entire survey
– 76.5% over the age of 40
– 71% female
– White/Non-Hispanic 61.5%, Black/African-American 13.5%
– 36.9% with more than 10 years of HPT experience.
• Represented health centers:
– from 41 US states and US Territories
– designated as urban - 52.1%
– serving under 25,000 patients - 46.5%

13. Theory of Organizational Readiness for Change
“Organizational readiness for change is a multi-level, multi-faceted
construct. As an organization-level construct, readiness for change
refers to organizational members' shared resolve to implement a
change (change commitment) and shared belief in their collective
capability to do so (change efficacy).”
Bryan Weiner, PhD
Author: Theory of Organizational Readiness for Change (2009)

14. Construct to Measure
‘Organizational readiness for change’ is the degree
to which health centers are motivated and capable
to engage with and implement health professions
training programs.

15. Health Professions Training
• Any formal organized education or training undertaken for the
purposes of gaining knowledge and skills necessary to practice a
specific health profession or role in a healthcare setting.
• Types of HPT programs (e.g., affiliation agreements, accredited or
accreditation-eligible programs)
• At any educational level (certificate, undergraduate, graduate,
professional and/or postgraduate)
• In any clinical discipline

16. Conceptual Framework*
Networking,
External
Policies &
Incentives
Available Resources,
Leadership
Engagement,
Culture,
Implementation
Climate
Change
Efficacy,
Knowledge
& Beliefs
about HPT
Planning,
Engaging,
Executing,
Reflecting &
Evaluating
Relative
Advantage,
Complexity
External Context
ProcessCharacteristics
of Individuals
Organizational
Characteristics
Characteristics of
the HPT program
* Based on the Consolidated
Framework for Implementation
Research (Damschroder et al.,
2009)

17. Survey Item Pool
• Followed recommendations for selecting the best and most robust
survey item combination
306 items = initial survey item pool
• Rewording and mapping the items to a relevant domain/subdomain in
the conceptual framework
182 items = for assessment in the Delphi process

18. Delphi Process for Item Selection
• A modified, electronic Delphi technique
• Assessment of the survey items by 13 experts
– Appropriateness and ability to measure the relevant
domain in the conceptual framework (rounds 1 and 2)
– Importance for organizational readiness to engage with
HPT programs (round 3)
– suggest changes, reword, propose new survey items
• Consensus reached for 65 survey items

19. Survey Items
Please respond on a scale of 1-5:
1=strongly disagree; 2=disagree; 3=neutral; 4=agree; 5=strongly agree
Our health center has enough evidence to support that:
The health professions training program is flexible enough to be redesigned to meet the needs of our
health center.
If the organization engages with this health professions training program, it will be beneficial for our
health center workforce.
The program will help trainees better adapt to relevant best practices at health centers.
The health professions training program has clear structure and goals.

20. Rigorous Statistical Testing
• Completed extensive array of statistical tests using high quality data from
the pilot test survey respondents
• Assessed the psychometric properties of the newly developed and
validated survey instrument
• Developed and validated state-of-art survey instrument for
organizational readiness grounded in rigorous analytic methodology

21. Readiness to Train Assessment Tool
• The resultant 41-item, 7-subscale structure of the survey was derived through
exploratory factor analysis. Cronbach’s alphas (.79 -.97) indicated good to
excellent reliability.
• The instrument covers dimensions of health center readiness for engaging with
HPT programs that were deemed critical to evaluate by the project’s subject
matter experts.
• The advantage of the RTAT™ is that it covers organizational readiness dimensions
that are relevant to all kinds of health professions training programs and types of
health centers.

22. Subscales of the RTAT
The seven sub-scales that emerged from the data analysis represent seven
areas of readiness:
– Readiness to engage (8 items),
– Evidence strength and quality of the HPT program (4 items),
– Relative advantage of the HPT program (4 items),
– Financial resources (3 items),
– Additional resources (3 items),
– Implementation team (4 items), and
– Implementation plan (15 items).

23. Overall Readiness Scale and 7 Subscales
Sub-scale Readiness to Engage
Evidence Strength &
Quality of the HPT
Program
Relative Advantage of
the HPT program
Financial Resources Additional Resources Implementation Team Implementation Plan
Brief Description
Indicators of the health
center’s overall
readiness and
commitment
to engage with health
professions training.
Stakeholders’
perceptions of the
quality and validity of
evidence supporting
the belief that the HPT
program will have
desired outcomes at
their health center.
Stakeholders’
perceptions of the
advantage of engaging
with/implementing the
HPT program versus an
alternative solution.
The level of financial
resources dedicated
for implementation
and ongoing
operations.
The level of additional
resources dedicated
for implementation
and on-going
operations, including
appropriate staff and
assistance for staff
(e.g. evaluation
resources, tools,
training, and
coaching).
This subscale is about
the individuals
involved with the HPT
implementation
process who can
formally or informally
influence this process
through their
knowledge, attitudes,
and behaviors. They
are effective in
overcoming
indifference or
resistance that the
implementation of an
HPT program may
provoke in the health
center.
This subscale is
associated with the
implementation
process. Successful
engagement usually
requires an active
change process aimed
to achieve effective
implementation of the
HPT program(s). The
subscale measures the
degree to which a
scheme or method of
behavior and tasks for
implementing an HPT
program are developed
in advance, and the
quality of those
schemes or methods.
Number of Survey
Items
8 4 4 3 3 4 15

24. Summary
The final survey instrument, the Readiness to Train Assessment Tool™
(RTAT™), is a multi-item, multi-subscale, organizational readiness scale
that is both valid and reliable.
• Theoretically-based, valid and reliable 41-item measure with 7 subscales.
• Designed to assess readiness at the health center level
• Demonstrates psychometric and pragmatic strength.
The Readiness to Train Assessment Tool:
• Meets a need at the national level to help health centers address concerns regarding capacity, resources,
organizational abilities when launching any health professions training program(s).
• Can support implementation of HPT programs at health centers
• Can serve as a platform meant to encourage dialogue and action at various levels

25. RTAT Survey Link Distribution
• CHC will send the RTAT survey link to an identified key PCA contact for the RTAT initiative
• The PCA key contact is charged with sending the link to an identified key individual at every
FQHC/FQHC Look-alike in their state/region by email following the webinar
• The identified key individual at every FQHC/FQHC Look-alike is asked to distribute the RTAT survey
to ALL employees.
– Everyone is encouraged to complete the first baseline assessment on general perception of
the health centers readiness to engage with HPT.
– The second portion of the survey on the health centers’ level of readiness for implementing a
specific HPT program. This section can be completed for up to three different types of HPT
programs. “Those ’interested and/or knowledgeable’ about a specific HPT program, or those
‘directly or indirectly involved’ with a specific program are encouraged to continue forward
with the second portion of the survey.” Those individuals will need to select a specific HPT
program. We recommend thinking about the programs that are the most important or of the
greatest interest to YOU or what you feel is most important for your organization.
Year 1: (July 2020-June 2021)

26. Mobile Version Web-Based Version

27. Using the RTAT
Results can be used to inform:
• Determinations of individual health center readiness
to engage with HPT programs
• Determinations of readiness at various levels for the
purposes of evaluation and support such as at the
state/regional level
• Development of effective T/TA at the level of state,
regional and national

28. Results
• The survey allows for three levels of assessment and scoring: at the survey item,
subscale, and overall scale levels by obtaining their mean (average) scores.
• Mean scores may range anywhere from 1 to 5 with 5 indicating highest readiness to
engage with and implement a specific program.
• Scores expressed as a single number to ease interpretation but RTAT permits more
sophisticated disaggregation and analysis
• The scores can be used to assign one of three levels of readiness for each survey item,
subscale, and for the overall scale.
Likert Scale Mean Score READINESS
Strongly Agree 5
Ready
Agree 4.00-4.99
Neutral 3.00-3.99 Approaching Readiness
Disagree 2.00-2.99
Developing Readiness
Strongly Disagree 1.00-1.99

29. Goal for the PCAs
The goal of PCA workforce T/TA activities is to increase the
percentage of health centers at full readiness to engage in health
professions training programs by the end of the PCA project period
(June 30, 2023)
PCAs will use the results to expand targeted workforce training and
technical assistance (TA) capabilities, working with health centers in
their respective states/regions on focused strategic workforce plan
activities intended to increase health center capacity with
recruitment efforts.
https://bphc.hrsa.gov/program-opportunities/pca/workforce-funding-overview

30. BHW/BPHC: PCA Strategic Workforce Initiative
30
Year 2: (July 2021-June 2022)
• Analyze individual and state-level RTAT™ results
and share with health centers
• Facilitate learning sessions to discuss
state/regional implementation
• Develop individual Health Center Workforce
Strategic Plans to advance HP-ET
• Establish strategic partnerships between
community colleges and four-year colleges &
universities to recruit and retain students from
rural and underserved communities
Work will be supported by-
• NTTAPs
• HCCNs
• PCAs
• This initiative will complement and build on
the Education Health Center Guide (PDF 6.4 MB)
developed in partnership with the Northwest Regional
Primary Care Association and Community Health
Association of Mountain/Plains States to assist health
centers with serving as training sites for health
professions students and/or residents.

31. Using the Data
GOAL 2:The HP-ET initiative will enhance health centers’ capabilities to recruit, develop,
and retain their workforce by exposing health and allied health professions students, trainees, and
residents to education and training programs at health centers.
Year 3: (July 2022- June 2023)
– Implement Health Center Workforce Strategic
Plans to advance HP-ET by the end of Year 3
– Collect and disseminate evidence-based
models and promising practices to support HP-
ET at health centers regarding sustainable
model(s) for the implementation of health
professions training programs, including
sharing costs, faculty, university collaborations,
and clinical facilities
31
Work will be supported by-
• NTTAPs
• HCCNs
• PCAs
• This initiative will complement and build on
the Education Health Center Guide (PDF 6.4 MB)
developed in partnership with the Northwest
Regional Primary Care Association and Community
Health Association of Mountain/Plains States to
assist health centers with serving as training sites
for health professions students and/or residents.

32. Expected Impact:
FQHCs/FQHCS Look-Alikes, PCAs and Workforce NTTAPs
32
• Determine the level of readiness (i.e., full readiness,
approaching readiness, or developing readiness) of health
centers to engage in HP-ET.
• Increase in the number of health centers who have
implemented a HP-ET program.
• Identify potential barriers (including lack of initial assessment, accreditation standards,
academic-community partnerships, preceptors, training space, simulation
equipment/technology, paid traineeships, or any other identified barrier) that prevent a health
center from advancing HP-ET.

33. National Webinar to Launch RTAT
• Introducing health centers to the tool
• Introduction to the landing page for the tool
• Introduction of the partnership between PCAs and the Community Health Center,
Inc. Technical Advisory Panel
• Overview of the Call to Action to complete the tool and the project goals/timeline
PCA Collaboration on National Launch-
• Encourage your health centers to join the webinar
• Join the webinar!
October 6th 5pm EST
REGISTRATION LINK

34. RTAT Web Page
www.chc1.com/RTAT
RTAT@chc1.com

35. Questions?
CONTACT INFORMATION:
RTAT@chc1.com