STANDARDIZED/SIMULATED PATIENTS IN MEDICAL EDUCATION
STANDARDIZED/SIMULATED PATIENTS IN MEDICAL EDUCATION
Reed G. Williams, PhD
Professor and Vice Chair for Educational Affairs
Department of Surgery, SIU School of Medicine
Gregory Makoul, PhD
Associate Professor and Director, Program in Communication and Medicine
Northwestern University Medical School
Richard Hawkins, MD
Director, National Capital Area Medical Simulation Center
Uniformed Services University of the Health Sciences
James A. Hallock, MD
President and Chief Executive Officer
Educational Commission for Foreign Medical Graduates
Peter Scoles, MD
Vice President, Assessment Programs
National Board of Medical Examiners
Michael J. Reichgott, MD, Chair-elect
Section on Medical Schools
A 1993 paper by Dr. Howard Barrows1 noted that he had been working in the field of
standardized patients for 30 years. It is impressive how long this technique for teaching and
evaluation has been in place. In medical education, we have a current challenge to develop
validated outcome measures for performance. This is critical to ensure competence.
Standardized patients can be a tool for such evaluation, as well as for education.
Significant Events in the use of Standardized Patients:
The history of standardized patient use is a success story that documents what can be
accomplished through systematic educational research, development, and program evaluation.
Three significant events led to Howard Barrows’ creation of the first standardized patient. As a
chief resident at the New York Neurological Institute, Barrows worked on the service of an
attending physician who observed all medical students work up a patient from beginning to end.
When asked why, the physician replied that no one else was watching students. Barrows noted
that in the absence of observation and feedback, errors could persist. The second major event
occurred as Barrows selected and managed patients for the neurology board examination. When
the patients were debriefed after the exam, one described a physician who was hostile and
performed an uncomfortable examination. When told that the physician would be spoken to, the
patient said that he had “fixed” the examinee by “changing his Babinsky from one side to the
other” and changing his sensory findings. The third triggering event for Barrows came when he
was developing a set of films on the neurological examination using an artist’s model. He noted
that the films did not include the elements of observation and feedback, so important for
learning. He began to think about teaching the model to display a neurological problem, like the
patient who could change his findings at will. In his first case in 1963, he taught the model to
portray the signs, symptoms, and history of a paraplegic patient with MS. He also developed a
checklist for the standardized patient to fill out on what was done by the trainee. This is similar
to the process that is used today.
In a second landmark, Kretzschmar at the University of Iowa taught patients to be Gynecologic
Teaching Associates (GTAs). The GTAs then taught the pelvic and breast examinations to
students. This approach still is prevalent.2 Students liked this format because they were able to
learn from the process.
The Objective Structured Clinical Examination (OSCE) was introduced by Harden and
colleagues in 1975. Previously, encounters with patients for education or evaluation had
included a full examination. The OSCE format focused on the assessment of a specific
competency, with a short encounter.
In 1976, standardized patients first were sent into physician offices in unannounced visits. A
study using this approach by Kopelow and colleagues in Manitoba asked physician faculty
members to determine how many items on a checklist a competent physician should perform.
Then, standardized patients were sent into these same faculty members’ practices. The
physicians who set the standards, on average, performed 60% of the items that they said were
essential for competent performance. This study highlighted the problems with developing
reasonable standards of performance.
The introduction of the ATLS course (in 1976-1978) was an innovative, systematic use of
standardized encounters for training and assessment purposes.
In 1984, Stillman and colleagues made the first report of a multi-institutional standardized
patient examination, when a number of residency programs in the northeast gave the same
examination to their residents.
The first use of standardized patients in a licensure examination (by the Medical Council of
Canada) occurred in 1993. This use of standardized patients in a “high stakes” examination is a
significant landmark. The 1998 addition by the Educational Commission for Foreign Medical
Graduates of an assessment of clinical skills was another such example.
Importance of Standardized Patient Use
• Case specificity – Since clinical performance is case-specific, there is a need to increase the
number of cases and balance cases in order to make a general estimate about clinical
competence. It also is important to give students broad experience, since performance may
be related to experience.
• Convenience – Standardized patients provide the cases that you need at the time and in the
place they are needed.
• Direct comparisons of competence – Standardized patient evaluations allow a direct
comparison of students’ clinical skills. Previously, direct comparisons among students could
only be done in the cognitive domain (for example, through USMLE results).
• Compression/expansion of time – Use of standardized patient simulations allows students to
have a longitudinal experience with patients and to follow a case in a compressed time frame.
• Safe practice – Simulations allow students to be put in clinical situations that they could not
manage alone in a real clinical setting.
• Efficient use of physician faculty time – A physician can train a number of standardized
patients who can then teach/evaluate students. This leaves the physician free to concentrate
on specific areas where his/her expertise is most useful.
• Standardized patient evaluations are responsive to real differences in performance.
What is likely to happen next?
We are likely to see more of the same in the use of standardized patients. Cases have become
stereotypical, and there is a need to build in more creativity. It is likely that there will be more
use of standardized patients to make unannounced visits to physician practices to assess variation
in practice for health care research.
What should happen next?
There should be more use of standardized patients for health care research, for training, and for
clinical performance appraisal. Current methods of performance assessment (such as
observation by faculty members and residents) do not pick up differences in performance among
trainees; standardized patient methods can accomplish this better. Better use should be made of
clinical practice data, health outcomes research data, and patient safety data to identify problem
areas that will guide the development of standardized patient cases. Finally, increased
sophistication would be useful in what data are captured from standardized patient encounters
and how these data are interpreted.
• Clinical competence is highly case specific
• Standardized patients can ensure broad exposure to a variety of cases.
• Current assessment methods do not capture differences in clinical performance.
• Standardized patients allow direct comparison of trainees’ clinical skills.
• Standardized patients can be useful physician extenders, by sparing physician time.
• Scores in standardized patient encounters reflect real differences in clinical performance.
There is a need to broaden the scope of standardized patient use and to link the selection and
design of cases to clinical data.
In the past, students were not routinely observed working with patients. Standardized patients
can help with this problem, but are not the only answer.
As part of the Association of American Medical Colleges Medical School Objectives Project
(MSOP), a survey was conducted about methods used to teach and evaluate communication
skills. Eighty-nine schools responded. Of the responding schools, 79% used interviews with
simulated patients to teach these skills. Feedback by simulated patients was used in 76% of
schools and assessment with simulated patients (e.g., in an OSCE) was used in 70% of schools.
At Northwestern, communication skills are taught throughout the curriculum. There is a
Communication Skills unit for first-year students, an assessment of clinical (including
communication) skills at the end of the first and second years, and tailored communication skills
modules during the clerkships. There are plans to reintroduce a clinical skills assessment at the
end of the third year.
First-year Communication Skills Unit
This unit is part of the two-year “Patient, Physician, and Society” course. The unit begins early
in the first year, to position communication as a fundamental clinical skill, prior to students
learning to take a patient history. In the unit, students practice and obtain feedback on 25
communication tasks. During the unit, groups of about 44 students meet with faculty to discuss
concepts. In groups of 11, students meet with clinicians. Practice groups of four students work
with patient instructors. The patient instructors change their demeanor and personal details from
student to student, allowing practice with a broad range of patient types. They provide feedback
to the students after the sessions. Videotaping of sessions allows students also to do self-
Clinical Skills Assessment
As defined at Northwestern, standardized patients are used in assessment, and therefore must
maintain a constant demeanor and identical story from student to student. It requires lengthy
training to get the standardized patients to present a case in the same way and to record
In the first-year clinical skills examination, students receive an inventory of their strengths and
weaknesses. Students must pass the second-year skills assessment or attend two required
workshops before starting their clerkships.
Specific communication skills modules are inserted into clerkships. These are developed in
collaboration with the clerkship directors and address more advanced communication challenges
(e.g., delivering bad news in the surgery clerkship). This format works if the clerkship takes
ownership of the modules. It is not well received if it is perceived that the modules are taking
students away from the clerkship.
• Standardized patients need to be carefully screened and trained. When recruiting
standardized patients, it is important to ensure that they do not have a personal agenda or
“lecture” students about their opinions.
• The use of standardized patients in assessment requires reliability of performance and
• There are many opportunities for further research, including psychometric studies and
comparisons of global and checklist ratings.
The National Capital Area Medical Simulation Center celebrated its first year of operation in
April of 2001. The Center consists of four units:
• Clinical Skills Area – This includes 12 examination rooms with audiotaping and videotaping
capability. The rooms are built around a central core that allows ongoing monitoring and
• Computer Laboratory – There is a 16-station computer simulation laboratory, which has CD,
DVD and web-based software programs.
• Surgical Simulation Laboratory – There is an operating theater with two computer-driven
anesthesia mannikins. The facility also includes a virtual reality suite, where procedures
such as peritoneal lavage and laparoscopy can be practiced.
• VTC/Distance Learning Facility – This allows distance learning capabilities.
In the Simulation Center, standardized patients are used in four general areas: teaching, learner
evaluation, program evaluation. and outcomes assessment. Standardized patients also are used in
faculty development. Users include the school of medicine, graduate school of nursing, graduate
medical education programs, and other institutions in the region on a space-available basis.
There are a number of educational activities in the school of medicine that use Simulation Center
resources. In the Introduction to Clinical Medicine (ICM) course students are taught
interviewing, including experience with difficult communication tasks. The final examination
for the physical diagnosis section of the course also is held at the Center. Finally, about one-half
of the students take the final segment of ICM, which combines history and physical examination,
at the Center. The ability to videotape these students is an asset, in that faculty members can
review the tapes according to their own schedules and students can use the tapes for self-
assessment. In general, the use of standardized patients allows control over clinical content and
ensures that patients are available on schedule. As part of student evaluation in the ICM course,
there is a six-station OSCE given at the Simulation Center.
The Simulation Center is involved in four of the required clerkships and will add a fifth. For
example, there was concern in the family medicine clerkship about the consistency of students’
clinical experiences across many remote sites. In response, the faculty arranged to bring students
back to campus for a two-week block. On the first day, a multiple-station examination is
administered to identify deficiencies and to serve as a needs assessment for the two-week period.
A similar final examination at the end of the two-week block provides an evaluation of how well
the educational needs of each student were met.
Support for graduate medical education (GME) also is provided by the Simulation Center. For
example, program directors at the Naval Medical Center asked for an examination for interns
that would simulate conditions in remote environments. This would help assess whether the
internship was preparing individuals to deliver care in such settings. To make the OSCE
simulations close to reality, the physical medicine and rehabilitation GME program went to the
community to recruit amputees with ill-fitting prostheses. There also is an assessment focusing
on clinical skills for “problem” residents.
Faculty development involving the Simulation Center takes many forms. There is informal
consultation, for example, with clerkship directors. A course is held annually for GME program
directors. A “direct observation” workshop is offered to train faculty to be consistent in rating
students’ clinical skills. For this session, standardized patients and standardized residents are
trained to portray a specific level of performance.
Within each project carried out by the Simulation Center, specific research questions have been
identified. The Graduate School of Nursing is examining different types of ratings (checklist,
global) and raters (self, peer, faculty, standardized patient). The Department of Pediatrics is
teaching patient advocacy skills, putting students in a simulated managed care environment
where they interact with a case manager and discharge planner, and then with occupational and
physical therapy. At the end of the educational intervention, students present the simulated
patient’s home health care needs to a simulated ward attending.
Among other activities, the Simulation Center developed a scenario for a hospital ship to react to
a disaster (a hurricane). This 4-6 hour mass casualty drill included standardized patients who
presented with accompanying x-rays and other background data.
The cost for construction, information technology, and furnishings for the Simulation Center was
$4.5 million. The Center has an annual operating budget of $1.3 million. Standardized patient
costs are $300,000 and the cost of standardized patient trainers is $125,000.
During the first year of the Simulation Center’s existence, the focus has been on clinical skills
teaching and assessment, directed at learner needs; on teaching communication skills; on faculty
development; and on research. The Simulation Center with its standardized patients offers a safe
environment to learn, for example, communication skills related to difficult or sensitive areas.
For more information, consult the Simulation Center’s web site at http://simcen.usuhs.mil.
Dr. Hallock described the use of standardized patients in a “high stakes” examination, the
Educational Commission for Foreign Medical Graduates (ECFMG) Clinical Skills Assessment.
The purpose of ECFMG certification is to assure residency program directors and the public that
international medical graduates have met the minimum standards to enter residency training in
the United States. ECFMG certification requires the primary source verification of a diploma
and passage of Steps 1 and 2 of the United States Medical Licensing Examination, a test of
English, and the Clinical Skills Assessment (CSA).
The CSA was implemented in July 1998. Over 16,000 assessments have been carried out to date
(about 6,000 per year). The CSA is offered only at the ECFMG facilities in Philadelphia. The
purpose of the CSA is to ensure that International Medical Graduates from non-Liaison
Committee on Medical Education (LCME) accredited medical schools demonstrate the ability to
gather and interpret clinical patient data and communicate effectively at a level comparable to
what would be expected of a graduate of an LCME accredited school.
The CSA is a performance-based examination using standardized patients in an OSCE-type
process. Each examinee completes 10 scored cases. There also may be an 11th case that is being
For the ECFMG, a critical issue is training the standardized patients. Many of the standardized
patient trainers are former standardized patients. The standardized patients are taught to portray
cases consistently and to score accurately. The standardized patients do not assess clinical skills;
they document clinical skills using a checklist. They do evaluate interpersonal skills and spoken
English proficiency (ability to communicate). The standardized patients are trained and
constantly monitored. Examinees also are videotaped.
The cases are designed to simulate a high quality encounter in a physician’s office. Examinees
receive information as they enter. They are expected to take a history and perform a focused
physical examination, discuss their findings with the patient, and answer the patient’s questions.
The examination is balanced by gender, age, and acuity for each examinee. There are no
pediatric patients. The cases cover important and/or common conditions and are appropriate for
an entry-level resident physician.
There are number of characteristics that are required for a high-stakes examination:
• Validity (does the examination measure what you want to measure) – The skills that are
evaluated are those considered important for an individual beginning residency training
(history-taking, physical examination, interpersonal skills, ability to communicate). The
national ambulatory medical care survey and other national data sources are used to select
relevant cases. Every year, the test blueprint is examined to ensure that it matches the
common cases seen in physician offices. There is a multispecialty test committee of
physicians who develop and review the cases. For each form of the test, there is attention to
case mix, acuity, gender, age. All four major content areas are covered: chest, abdomen,
neuropsychiatric, and constitutional. There also is a requirement that a minimum number of
physical findings be present in each examination.
• Fairness – Cases are tested before being used for scoring. The different forms of the test are
equated for difficulty across cases and standardized patients. Because all testing is at one
site, there are uniform testing conditions.
• Security – There is a large “bank” of cases and 90 standardized patients. The forms of the
test change so that the standardized patients do not know beforehand which cases they will
• Reliability (is a score a reasonable reflection of the examinee’s true ability) – Reliability is
ensured through case development and eliminating extraneous sources of error (by
standardized patient training and quality assurance measures). In about 5-10% of cases, a
second standardized patient judges the case. Cases also are videotaped. All cases assess all
skills, so performance across cases can be considered,
• Feasibility – Such a high stakes examination can be done. Use of a single testing center
allows the control of many variables. The great majority of the standardized patients (78 of
90) have been with the program since the beginning. Scheduling of examinees is handled by
Pass rates to date are contained in the following table:
Candidate Group Percent Passing Segment of the Exam
Overall Doctor/Patient Integrated
(Total Communications Clinical
All IMGs 80.4% 89.9% 86.5%
USIMGs 87.1% 98.6% 87.9%
Non-USIMGs 78.7% 87.6% 86.2%
There is consensus that cognitive and affective skills are in different domains, and that mastering
one domain does not ensure mastery of the other. Communication and information gathering
skills are essential for physicians. The Association of American Medical Colleges Medical
School Objectives Project (MSOP) includes the ability to take an accurate history and to
communicate effectively with patients, families, and colleagues among the outcomes that
medical students should achieve. Data from a study related to MSOP by Makoul and colleagues
found that 80% of medical schools use standardized patients for teaching communication skills,
but that fewer used standardized patients in student evaluation. Even rarer is the use of
standardized patients in summative (high stakes) evaluations.
The approaches that medical schools use vary widely. About one-third of schools have no
standardized patient programs for teaching and evaluation. In those schools with programs, what
students learn about communication skills in the first and second years, through use of
standardized patients, may be undermined in years three and four and later.
It would not be necessary to have separate examinations to test for information gathering and
communication skills, if the current exams that test for cognitive achievement also identified
individuals deficient in those areas. Research indicates, however, that this is not the case. In the
Medical Council of Canada examination, students must pass the cognitive portion (Part 1) of the
examination before taking the clinical skills (Part 2) component. There still are 4-12% failure
rates on the clinical skills (Part 2) examination. Projections based on the model standardized
patient examination being developed by the National Board of Medical Examiners (NBME)
suggest that 5-7% of U.S./Canadian graduates will fail and that these individuals will be a
different cohort than fail Step 2 of the USMLE. The correlation between Step 2 and the model
NBME standardized patient examination is modest. Correlation is highest in cases that test
information gathering and lowest in cases that evaluate communication skills.
The NBME began consideration of a clinical skills examination in 1975. The developmental
work has been funded by a wide range of organizations, including the Kellogg, Macy, Robert
Wood Johnson, and Commonwealth Fund foundations. The Pathways for Licensure Task Force
(1985) recommended that a test of clinical skills be incorporated into the examinations for
licensure as soon as it was shown that the test would be valid, reliable, and practical. The
NBME now thinks that these conditions can be met.
The NBME is charged with responsibility to certify that candidates have adequate information to
enter into supervised practice in postgraduate training and unsupervised practice after graduate
medical education. Steps 1, 2, and 3 of the USMLE allow this assurance in the cognitive
domain. The goal of the standardized patient examination would be to certify that the candidate
possesses adequate information gathering and communication skills to enter into supervised
graduate medical education. In this, it would be the counterpart of the Step 2 examination.
The format of the NBME standardized patient examination will be similar to the Clinical Skills
Assessment given by the ECFMG. Each examinee will complete 10 scored cases from a large
pool of cases. Content equivalent forms of the examination will be utilized. Some of the cases
will primarily test information gathering and other cases will focus on communication skills.
The scored elements will include checklists and questionnaires generated by the standardized
patients and post-encounter notes generated by the candidate. Each encounter with a
standardized patient will last about 15 minutes, and then the candidate will have 10 minutes to
generate the post-encounter note. The final scoring mechanism has not yet been set. Options for
scoring include the following: (1) use a numerical score, since this examination will be part of
the licensing examination sequence and some states require a numerical grade; (2) report the
exam as pass-fail; (3) give a successful candidate certification and require that an unsuccessful
candidate repeat the exam.
The NBME considered several delivery models for the standardized patient examination, based
on the balance between convenience/accessibility and reliability/standardization/fairness. The
spectrum ran from using about 60 medical school sites that each delivered the exam for a
window of time to using only a few test sites that delivered a highly standardized examination.
The ultimate delivery mechanism probably will be a group of nationally-distributed, fixed
examination sites that operate year round.
The cost of the exam to students has not yet been set, but a general idea can be gained by adding
the costs that will be incurred in exam development and implementation. There will be costs for
such things as:
• standardized patient salaries,
• test center staff salaries
• test center overhead,
• test scoring,
• candidate scheduling/registration,
• score archiving/reporting,
• NBME central costs.
Including all these factors, the total cost per student would be similar to the cost of the ECFMG
If approved by the NBME Executive Board, large-scale field-testing will begin. This will build
toward delivery of the standardized patient examination beginning in the spring of 2004.
Therefore, the class of 2005 (which will enter in the fall of 2001) will be the first to take the
standardized patient examination as part of USMLE.
The ultimate failure rate in the current USMLE is 1-2% (about 300-600 students do not pass).
The NBME spends about $23 million on these exams. It is anticipated that the standardized
patient examination will cost $22 million per year, and will have an ultimate failure rate of 1-3%.
There is consensus at the NBME that it is essential to certify skills as well as cognitive
1. Barrows H. An overview of the uses of standardized patients for teaching and evaluating
clinical skills. Academic Medicine 1993;68(6):443-453.
2. Data from the 2000-2001 Liaison Committee on Medical Education (LCME) Annual
Medical School Questionnaire are provided below. The questionnaire, which was sent to
the deans of all 125 LCME-accredited medical schools, had a 100% response rate.
Number of medical schools that used standardized patients in instruction of medical
students in the following skills:
- history-taking (106 schools)
- doctor-patient communication skills (104)
- general physical examination skills (93)
- specialized physical exam skills, such as the gynecologic or urologic exam (114)