Over the past decade competency has gone from buzzword to requirement in healthcare. The Joint Commission, along with federal and state regulatory agencies, has increasingly focused on staff competency with each coming year. Why this focus on competency?

1. Competency Validation
An Advisory Board Approach
by Tammy Barben, RN, MSN

2. Why Focus on
Competency?
Over the past decade competency has gone from buzzword to
requirement in healthcare. The Joint Commission, along with
federal and state regulatory agencies, has increasingly focused
on staff competency with each coming year. Why this focus on
competency? The clinician and humanitarian in all of us would
love to say it is about raising the standard of care, but the realist
in us knows that it is also about liability and money. Clinical jobs
are complex and require solid, mastery-level knowledge and
skills relevant to the discipline. Competency exams drill deep
into the clinician’s skill-set and knowledge, which leaves little
room for impression when properly developed, administered and
scored (Biddle 2006, p. 84). Ildiko Hock defined competence as the
intrinsic, tacit knowledge that underlies actual performance (Hock
2003, p. 65). Thus, competency exams have become a necessary
standard for healthcare facilities, agencies and employers.

5. Creating a validation program on their own
can be financially draining for employers.
Competency Validation
Program Options
Many companies struggle to meet
regulatory requirements concerning
staff competency. Developing a
competency program is not only labor
intensive, it requires a team of experts
to develop a program with validity. The
use of substandard test development
and validation procedures can greatly
increase the vulnerability of a test to
legal challenges based on inadequate
evidence for validity (McPhail 2007,
p. 307). Content validation generates
two major benefits for employers:
measuring key, relevant job knowledge,
and dissuasion of possible litigation
(Biddle 2006, p. 29). Creating a
validation program on their own can
be financially draining for employers.
Job experts must be clinically
knowledgeable, licensed, educated
and have significant experience in
order to have the skill set for test
development. Employers do not always
have a team of clinical experts readily
available and struggle with having
a diverse ethnic group of clinical
experts to develop a valid competency
program that is free of bias.
Alternatives to developing your own
competency program can prove to be
financially burdensome as well. Hiring
a consultant group, sending employees
to classes or paying monthly or per-
test fees for online testing can quickly
deplete any company’s profits and time.

6. API Healthcare developed a competency program
that has clinical validity and is free of testing bias.
API Healthcare’s Clinical
Validation Program
Healthcare workforce management
technology vendor API Healthcare
developed a competency program that
has clinical validity and is free of testing
bias. They have partnered with a team
of clinical experts that have expertise in
nursing, therapy, respiratory, imaging,
laboratory, counseling, advanced
practice professionals, pharmacy and
the specialties within these disciplines.
The development, validation and
analysis of the program content is
conducted under the direction of an
Advisory Board which includes the
Clinical Advisory Team, software end-
users and API Healthcare staff.
The Clinical Advisory Team is made up of
clinical experts, many of whom taught
at the college level. The advisory team
includes several Master’s prepared
and Nationally Certified Registered
Nurses, a Pharm D, a Master’s
prepared Social Worker, Licensed
Vocational Nurses, a Certified Wound
Care Nurse, a Master’s prepared
Respiratory Therapist, a Radiology
Technician and Master’s prepared
Medical Technologist. Software end-
users consist of organizations that
have licensed and use API Healthcare’s
Recruiting Solution (ARS) and provide
feedback on the content and direction
of the application. API Healthcare staff
includes members of the Executive
team as well as implementation,
training and engineering resources at
API Healthcare.

7. API Healthcare programming experts developed
internal tools to perform testing validation and analysis.
The first step in this process was to
conduct research on test writing,
validation methods and test analysis
strategies. Validating and analyzing
content and exams to ensure a
bias-free, quality product is a multi-
staged and multi-stepped process. To
address the professional standards,
API Healthcare programming experts
developed internal tools to perform
testing validation and analysis.
The Clinical Advisory Team conducted
the validation process in which each
test question and the complete exams
underwent two separate analyses. The
first review, KSAPCs (Knowledge, Skill,
Ability and Personal Characteristics),
is a technical, job-related review that
ensures questions selected for the
written tests cover information that is
1. Needed day one on the job.
2. Important or critical for
performance of the job and linked to
one or more critical job duties.
3. Requires analysis that draws on the
memory of the test taker, not just
information that can be looked up
(Biddle 2006, p. 89).
Each test question has been
categorized according to this analysis:
Basic Knowledge, Critical Thinking,
Medications/Therapeutic Interventions
and Assessment/Patient Care.

8. Mitigating Bias in
Competency Testing
The second step in the exam review covered a seven-point inspection
to ensure the questions and exams were bias free.
1. Is the question clear and understandable?
2. Is the question written at an appropriate difficulty level?
3. Does the question measure the skill set needed the first day of the job?
4. Is the question fair to all groups of people?
5. Is the question job related?
6. What job skill is measured by this question?
7. Will the applicant be able to provide a sufficient
response without employer-specific knowledge?
These court-endorsed ratings allow scoring and analysis of
test results that represent the test taker’s knowledge and skill level.

10. Competency Testing
Analysis Post Validation
Analysis of competency testing results is as important to a bias-free competency
program as the validation of the content by the clinical experts. The Advisory
Board used four primary sources of professional and legal guidance as
guidelines for the ARS software program during the development, validation and
analysis of the competencies.
1. Title VII of the Civil Rights Act of 1964, as amended by the Civil Rights Act of 1991.
2. The Uniform Guidelines on Employee Selection Procedures (Equal Employment
Opportunity Commission, Civil Service Commission, Department of Labor and
Department of Justice, 1978).
3. The Standards for Education and Psychological Testing (American Educational
Research Association, American Psychological Associates and National Council
on Measurement in Education, 1999).
4. The Principles for validation and use of Employee Selection Procedures (Society
for Industrial and Organizational Psychology, 2003) [Alternative pg. 350].
In the next stage of the competency
project, clinical experts shifted their
focus from development and validation
of content to analysis of the program
and testing of content. Test analysis
included review of the content and
each individual question. The passing
score for each exam was set at 80%,
as this is the standard across the
healthcare industry. Two statistical
tests, the 80% test and the two-
tail fisher exact probability statistic,
were used to identify any testing
bias between groups using the TVAP
analysis software. Content reviews
were also completed, looking at the

12. Males
• Total Males taking
exam 30
• Total passing exam 27
• Total who failed exam 3
• Percentage of males
passing the exam 88%
Females
• Total Females taking
exam 181
• Total passing exam 147
• Total who failed exam 34
• Percentage of females
passing the exam 85%
Caucasians
• Total Whites taking
exam 138
• Total passing exam 118
• Total who failed exam 10
• Percentage of Caucasians
passing the exam 86%
African Americans
• Total African Americans
taking exam 41
• Total passing exam 31
• Total who failed exam 10
• Percentage of African
Americans passing the
exam 85%
Hispanics
• Total Hispanics taking
exam 12
• Total passing exam 9
• Total who failed exam 3
• Percentage of Hispanics
passing the exam 83%
Asians
• Total Asians taking
exam 12
• Total passing exam 9
• Total who failed exam 3
• Percentage of Asians
passing the exam 83%
percentage of test takers correctly
answering each question. All questions
with less than 70% of test takers
answering the question correctly
were removed from the test or the
question was rewritten for clarification
of the question and/or answers.
Feedback was also received from ARS
users and clinicians concerning length
of tests, specific questions, overlapping
content between tests and general
questions concerning competencies.
As this sample data shows, the
exams have undergone extensive
analysis on many levels. The
information gained from this analysis
has been used to enhance both
the test format and the content.
Data

13. Native Americans
• Total Native Americans
taking exam: 2
• Total passing exam: 2
• Total who failed exam: 0
• Percentage of Native
Americans passing
the exam: 100%
Others
• Total others
taking exam: 6
• Total passing exam: 5
• Total who failed exam: 1
• Percentage of others
passing the exam: 87%
Total Exam Takers: 211
• Total passing exam: 174
• Total who failed exam: 37
• Percentage of total
taking exam: 82%
The following graph displays the
comparison of group scores as
reported by TVAP for the Nursing
Medication Exam. The analysis is based
on a total of 211 scores. As you can see
in the following graph and bullet points,
the groups are compared using both
gender and ethnicity.
0
50
100
150
200
Caucasian
86%Passed
85%Passed
83%Passed
83%Passed
100%Passed
87%Passed
88%Passed
African
American
Hispanic Asian Native
American
Other Males Females
Pass
Numberofteststaken
Fail
85%Passed

15. Annual Clinical Validation
Hansen (1998) conceptualizes ongoing
monitoring as a feedback loop that
provides important information to
revise if necessary. Therefore, ongoing
test analysis, using the Department
of Labor Statistics for each discipline,
will be conducted by The Clinical
Advisory Team along with the API
Healthcare Validation Committee. To
ensure bias-free and valid content,
the analysis will be used to identify
any questions or content that might
need to be revised or updated.
Using both clinical and technological
expertise, API Healthcare has
developed a library of online reading
material and validated exams
focusing on regulatory requirements,
trends in healthcare, standards of
care and basic skill sets specific
to clinical disciplines. This is in
addition to the discipline-focused
checklists that allow employees
to complete a self-evaluation.
Categorizing of exam questions to
match the checklist substantiates
the skills in which the clinician
has experience and expertise.
Competency Testing
Reading Materials

16. Written April 21, 2008; Amended September 25, 2014
© Copyright 2015 API Healthcare Corporation. All rights reserved.
© 2015 General Electric Company – All Rights Reserved. GE Healthcare, a division of General Electric Company.