1. Is There A Reliable Way of Evaluating Personal
Statements as Indicators of Potential for Patient
Centeredness in Medical School Applications?
Nathan Beucke MD, Rachel Brown MBBS, Alison Martin MEd, Lindsey Gentry BS
University of Missouri School of Medicine
ABSTRACT
Objective: Personal Statements provide a means of
looking at the personal characteristics of applicants.
Need for Innovation: The mission statement for our school
requires the committee to select students who “will excel
as patient centered physicians.” In the absence of a readily
available tool that would enable us to select such
students, our faculty turned to the personal statement as
one such available tool.
Methods: A faculty/administration working group met
and considered behaviors that indicate patient
centeredness as developed through a series of focus
groups. The group read statements written by medical
school applicants. The group then developed a numerical
scale anchored with descriptive statements. Through an
iterative process a scale was developed.
Educational Outcomes: The Patient Centered Personal
Statement Scale (PCPSS) was used as part of the
admissions process for the entering class of 2009. The
PCPSS has been found to be a useful tool for evaluating an
applicant’s potential as a patient centered physician.
Educational Significance: The PCPSS is a new tool to use
for holistic review efforts.
BACKGROUND/NEED FOR INNOVATION
Even though many hours are spent writing, reading and
discussing applicants’ personal statements, very little
research has been done into their potential usefulness in
the selection of medical students. Siu and Reiter (1) lay
out the issues that affect the usefulness of personal
statements. Many applicants receive input from others in
the development of the statement (2). Statements can
differ greatly in content because of their free-form nature
and applicants can have a vested interest in presenting
themselves at “the saintly end of the spectrum” (1).
Ferguson has examined personal statements as predictors
of performance throughout medical school. Their analysis
showed a very modest relationship between information
in the personal statement and clinical performance in
medical school (4). Other means of predicting
performance, such as multiple mini interviews, are
expensive to develop and deliver. The MCAT is largely a
prediction of performance on academic test in the first
two years of medical school.
Our mission is to graduate physicians who will provide
patient centered care for the people of Missouri and
beyond. We ask our admissions committee to select
students who will excel at this. In the absence of a readily
available tool that would enable us to select such
students, our faculty turned to the applicant personal
statement as one such available tool.
METHODS
The University of Missouri School of Medicine developed a
set of observable behavioral statements that describe
patient centered care. These statements were developed
and validated using qualitative methods. Qualitative
thematic analysis included triangulation of patients,
faculty and medical student themes. In the course of
2008-2009 a faculty and administration focus reviewed
these discrete observable behavior statements. The group
also read a number of personal statements written by
successful medical school applicants. An anchored
numerical scale was developed to evaluate personal
statements written by successful medical school
applicants. Through an iterative process, a usable scale,
the Patient Centered Personal Statement Scale (PCPSS)
was developed. In 2010 three raters began using the
scale. The group read ten personal statements and
assigned an individual score. The group then met and
discussed their scores highlighting specific examples of
empathy or service. If there was variation in the rating the
group discussed and arrived at a consensus score.
EDUCATIONAL OUTCOMES
The PCPSS has been found to be a useful tool for -
evaluating an applicant’s potential as a patient centered
physician. The scorers had unanimous agreement on 29%
of personal statements reviewed to date. Two of three
scorers agreed on 88% of the statements prior to group
review. After group discussion, consensus was reached on
100% of personal statements reviewed. Surveys of
admissions committee members revealed that a vast
majority (93%) use personal statements to assess patient
centeredness (table 1). The rating was easy to find on the
AMCAS application and most members agreed that the
PCPSS accurately reflected their view of the personal
statement. However, the data was mixed regarding the
influence that the PCPSS had on individual committee
members.
DISCUSSION & CONCLUSIONS
Our data demonstrate that the PCPSS may be a useful tool
for medical schools to utilize as part of the holistic review
of applicants, and in evaluating potential patient
centeredness. For applicants, the PCPSS requires minimal
investment, since they are already required to submit a
personal statement. Scoring the statements requires some
investment of time, but our data demonstrates that it is
possible to train raters to agree on the appropriate
score. Our interviewers largely agreed with the scores
given, but a majority were not sure or did not feel it
influenced their assessment of an applicant. Our
data suggest the interviewers' reluctance may be,
in part, because they use other aspects of the application
as well as an applicant’s responses during an interview to
assess potential patient centeredness. Our future goals
include: refinement of the scale and of inter rater
reliability; development of an extended scale that will
include evaluation of reported activities and possibly
letters of recommendation; and long term validation
against later tests of clinical aptitude, including our end of
3rd year Patient Centered Objective Structured Clinical
Examinations ( PCC OSCE).
REFERENCES
1. Albanese MA, Snow MH, Skochelak SE, Huggett KN and
Farrell PM (2003). Assessing personal qualities in medical
school admissions. Academic Medicine, 78(3), 313-321
2. Ferguson E, James D, O’Hehir F, Sanders A. Pilot study of
the personality, references, personal statements in relation
to performance over the five years of a medical degree.
BMJ:326:22Feb 2003: 429-431
3. Hanson MD, Dore KL, Reiter HI and Eva KW (2007). Medical
school admissions: Revisiting the veracity and
independence of completion of an autobiographical
screening tool. Academic Medicine, 82, S8-S11.
4. Siu E, Reiter HI. Overview: what’s worked and what hasn’t
as a guide towards predictive admissions tool
development. Adv in Health Sci Educ (2009) 14:759-775
Strongly
Agree
Agree Not Sure Disagree Strongly
Disagree
No Answer
Personal
Statement
54 40 5 2 0 4
Reported
Activities
51 46 4 0 0 4
Letters of
Recommendation
32 55 7 4 0 7
Interview
Responses
59 29 7 0 0 7
Table 1. Assessing Applicants’ Patient Centeredness
0
10
20
30
40
50
60
1 2 3 4 5
NumberofInterviewerResponses(N=75)
AC Response
(1=strongly Agree; 5=Strongly Disagree)
Admissions Committee Survey Responses
PCPSS agreement
PCPSS influence
QUOTES ABOUT SCALE
PCPSS was confirmatory!
-AC Member 1
I enjoy the PCPSS.
-AC Member 3
I was prepared for a self-centered applicant based on
my assessment and the PCPSS, but was very pleased to
be wrong. Applicant was personable, well-balanced
young man.
-AC Member 13