826 Unertl et al., Describing and Modeling WorkflowResearch .docx

826 Unertl et al., Describing and Modeling Workflow
Research Paper �
Describing and Modeling Workflow and Information Flow in
Chronic Disease Care
KIM M. UNERTL, MS, MATTHEW B. WEINGER, MD, KEVIN
B. JOHNSON, MD, MS,
NANCY M. LORENZI, PHD, MA, MLS
A b s t r a c t Objectives: The goal of the study was to develop
an in-depth understanding of work practices,
workflow, and information flow in chronic disease care, to
facilitate development of context-appropriate
informatics tools.
Design: The study was conducted over a 10-month period in
three ambulatory clinics providing chronic disease
care. The authors iteratively collected data using direct
observation and semi-structured interviews.
Measurements: The authors observed all aspects of care in three
different chronic disease clinics for over 150
hours, including 157 patient-provider interactions. Observation
focused on interactions among people, processes,
and technology. Observation data were analyzed through an
open coding approach. The authors then developed
models of workflow and information flow using Hierarchical
Task Analysis and Soft Systems Methodology. The
authors also conducted nine semi-structured interviews to
confirm and refine the models.

Results: The study had three primary outcomes: models of
workflow for each clinic, models of information flow
for each clinic, and an in-depth description of work practices
and the role of health information technology (HIT)
in the clinics. The authors identified gaps between the existing
HIT functionality and the needs of chronic disease
providers.
Conclusions: In response to the analysis of workflow and
information flow, the authors developed ten guidelines
for design of HIT to support chronic disease care, including
recommendations to pursue modular approaches to
design that would support disease-specific needs. The study
demonstrates the importance of evaluating workflow
and information flow in HIT design and implementation.
� J Am Med Inform Assoc. 2009;16:826 – 836. DOI
10.1197/jamia.M3000.
Introduction
Health information technology (HIT) can enhance efficiency,
increase patient safety, and improve patient outcomes.1,2
However, features of HIT intended to improve patient care
can lead to rejection of HIT,3 or can produce unexpected
negative consequences or unsafe workarounds if poorly
aligned with workflow.4,5
More than 90 million people in the United States, or 30% of
the population, have chronic diseases.6 HIT can assist with
longitudinal management of chronic disease by, for exam-
Affiliations of the authors: Department of Biomedical
Informatics
(KMU, MBW, KBJ, NML), Center for Perioperative Research in
Quality (KMU, MBW, KBJ), Institute of Medicine and Public
Health,
VA Tennessee Valley Healthcare System and the Departments

of
Anesthesiology and Medical Education (MBW), Department of
Pediatrics (KBJ), Vanderbilt University, Nashville, TN.
This research was supported by a National Library of Medicine
Training Grant, Number T15 LM007450-04. The authors thank
clinic
staff and providers for their active participation; this study
would
not have been possible without their assistance. The authors
also
thank Melissa McPheeters, Katherine Hartmann, Laurie Novak,
and
Judith Dexheimer for reading drafts of this manuscript.
Correspondence: Kim M. Unertl, MS, Vanderbilt University,
400
Eskind Biomedical Library, 2209 Garland Avenue, Nashville,
TN
37232-8340.
Received for review: 09/10/08; accepted for publication:
08/16/09.
ple, displaying disease status trends and tracking compli-
ance with recommended care guidelines.7,8 The translation
of theoretical benefits of HIT into actual improvements in
chronic disease care has seen only limited success,9,10 in part
due to insufficient information on how to effectively inte-
grate these tools into existing ambulatory practice.
Researchers have developed rich descriptions of HIT-related
workflow in specific inpatient clinical settings such as criti-
cal care units,11–14 emergency care departments,15 and gen-
eral medicine departments.16 Multiple studies focused on
evaluating the impact of specific HIT implementations, such
as new workflow-related challenges introduced by comput-
erized provider order entry,11 potential patient safety com-

promises caused by implementation of automated drug
dispensing systems,17 changes in workload and communi-
cation after implementation of an electronic whiteboard,15
and a lack of changes in work routines after implementation
of electronic medical records.16 A few open-ended workflow
studies have also been conducted, not linked to evaluation
of specific HIT-implementations.12,18
Several researchers have examined workflow in outpatient
care environments, most frequently focused on primary care
settings and on specific processes such as immunization
delivery in primary care,19,20 diagnostic testing processes,21
and prescribing practices.22 The limited number of outpa-
tient workflow studies focused on chronic disease care have
Journal of the American Medical Informatics Association
Volume 16 Number 6 November / December 2009 827
typically been set in a single chronic disease domain, such as
examining a registry used to organize diabetes mellitus
care,23 developing HIT to deliver patient-centric coordina-
tion of multiple sclerosis care across multiple providers,24
and refining web-based systems to support delivery of home
nursing care to patients with congestive heart failure.25
Previous chronic disease care workflow studies have led to
an understanding of the workflow and information flow of
providers and patients in specific contexts and have sup-
ported development of domain-specific HIT. Our research
approach sought to broaden the scope of existing outpatient

workflow research, by examining multiple specialty care
environments in an open-ended fashion and by determining
commonalities and differences among these environments.
The study’s goal was to evaluate and compare provider
workflow and information flow across three chronic disease
domains in the ambulatory care environment. We studied
the interaction between clinical workflow and an established
electronic health record (EHR) in these clinics to learn about
delivery processes and provider needs. We previously dem-
onstrated the feasibility of our methods in a pilot study in a
multiple sclerosis (MS) clinic.26 The present paper extends
the pilot study to two additional chronic disease clinics,
cystic fibrosis (CF) and diabetes mellitus (DM), discusses the
process we used to develop models of workflow and infor-
mation flow in each of the three clinics, and provides
guidance on the unique informatics needs of the chronic
disease care environment. The three clinics were part of the
same Academic Medical Center and had access to the same
HIT resources, but differed in the chronic disease treated,
the specifics of care provided, the number of patients
managed, and the number and type of providers. The
research questions motivating the study were: what are the
similarities and differences in workflow and information
flow during management of different chronic diseases? How
do these similarities and differences impact the design and
implementation of HIT applications?
Methods
Study Design
Data were collected and analyzed iteratively27 over a 10-month
period using direct observation, semi-structured interviews,
analysis of artifacts,28 and development of workflow and
information flow models (Figure 1, available as an online
data supplement at http://www.jamia.org). Data collection
and analysis continued until data saturation,28 when addi-

tional data did not change analytic results. Vanderbilt Uni-
versity’s Institutional Review Board (IRB) approved study
procedures before data collection.
The study design focused on the use of technology in
complex and dynamic care environments from the perspec-
tive of the study subjects.28,29 A pilot study set in a single
clinic26 confirmed the need for direct observation but also
suggested the need to incorporate semi-structured inter-
views to fill in details and confirm findings.30 Data from the
pilot study were merged into the full study.
Clinic Selection
The study team considered fifteen chronic disease care
clinics at Vanderbilt University Medical Center (VUMC) for
inclusion. A researcher (KMU) conducted open-ended inter-
views with key informants31 from nine clinics to gather
preliminary contextual data. We then evaluated each clinic’s
ability to meet study goals using the Strengths, Weaknesses,
Opportunities, Threats (SWOT) framework.32 Examples of
strengths and weaknesses included clinic accessibility, exist-
ing use of technology, and organizational structure. Exam-
ples of opportunities and threats included trends in care,
existing informatics projects, and departmental policies. We
used the SWOT process to select sites with the highest
ratings in terms of strengths and opportunities and with
manageable weaknesses and threats. The SWOT process
also assisted the research team with developing strategies to
ameliorate potential study barriers at the sites before begin-
ning data collection. The researchers selected three adult
subspeciality clinics for the study based on the SWOT
analysis: multiple sclerosis (MS), cystic fibrosis (CF), and
diabetes mellitus (DM).

Direct Observation
A researcher (KMU) provided a brief project overview and
obtained verbal assent from staff, providers, and patients (as
appropriate) before observation. Initial observation focused
on interactions among people, processes, and technology,
with the focus narrowing in response to data analysis. The
observer remained in an unobtrusive location, watching
computer use whenever possible, and recorded detailed
field notes to assess the function of technology. Notes
included details such as use of EHR functions, comments
from providers about the EHR, how different types of
information were collected and recorded, transfer of infor-
mation among different roles, and descriptions of hand-offs.
As time allowed, the researcher asked open-ended questions
to clarify observations, such as about policies, procedures, or
other potential influences on workflow. For example, while
the researcher was observing a patient-provider interaction,
the patient mentioned calling the clinic’s after-hours number
for assistance and the on-call provider prescribed a new
medication. However, the patient’s regular physician was
not aware of the change and the information for the new
medication was not in the EHR. After the patient visit was
completed, the researcher asked the provider about clinic
policies, procedures, and normal routines related to docu-
menting after-hours calls. Other questions related to clinical
decision-making processes.
Those observed included clinical receptionists, dietitians,
social workers, nurses, nurse practitioners, resident physi-
cians, fellows, and attending physicians. The researcher
observed both routine and non-routine situations in clinic
work areas, private offices, hallways, and in examination
rooms during patient visits. Routine work activities ob-
served included patient check-in, patient intake, patient
examination, diagnostic tests, prescribing, patient education,

patient check-out, handoffs between providers, and commu-
nication processes. Non-routine situations included patient
emergencies, such as an extremely high blood glucose level
in a patient with DM. The researcher also obtained blank
copies of all paper artifacts used during patient care such as
laboratory test order forms, provider data recording tools,
and patient data forms.
Analysis of Observation Data
Data were analyzed inductively, formulating conclusions
based on data rather than seeking to confirm a-priori hy-
http://www.jamia.org
potheses.33 Analysis occurred throughout data collection to
extract recurrent themes and to guide subsequent data
collection. Field notes were transferred to an electronic
notebook34 with functionality to organize and help code
data. An open coding approach was followed, including
cycles of initial and more focused coding.33,35 Initial coding
was open-ended, occurred as close as possible to data
collection, and involved marking recurring observation
events, descriptions, and concepts. Focused coding orga-
nized and synthesized the initial coding results.
Both initial and focused coding assisted in focusing subse-
quent observations by, for example, highlighting areas
where additional information was needed. Analysis pro-
cesses, theme development, and observation refinement
were reviewed and discussed extensively by all manuscript
authors.

Model Development
Data analysis guided development of models31 representing
workflow and information flow processes and relationships
within each clinic. We applied Hierarchical Task Analysis,36
modified for a broader systems-level focus, to develop
workflow models. Analysis of observation data identified
the details and sequences of routine care processes in each
clinic, responses to non-routine events, and elements of
workflow involved in patient care coordination. The details
were then arranged sequentially in graphical workflow
representations. Preliminary workflow model versions fo-
cused on individual roles, such as physician or nurse. With
more data, the models expanded to incorporate multiple
perspectives on workflow throughout the clinic and rela-
tionships among activities.
The information flow models were based on Soft Systems
Methodology37 and focused on the transfer of information
between individual actors (e.g., patient or nurse) including
information required to perform activities and information
generated during activities. The models also depicted both
electronic and paper-based repositories of information. Dur-
ing analysis of field notes, we identified actors and roles
impacting direct patient care and administrative functions.
We identified the information systems used by the actors,
the types of information transferred among actors, and
information-related processes occurring outside of our data
collection frame (i.e., filling a prescription).
Semi-Structured Interviews
We conducted 30 – 45 minute semi-structured interviews
with participants in each clinic to confirm observations and
assess the structure of clinic-specific models. Interview par-
ticipants included three attending physicians, one fellow,
two nurse practitioners, a nurse, and two dietitians. The

interview sampling frame sought to fully represent the
range of users in each clinic, but subject availability and
willingness to participate limited selection. Interviews were
audiotaped and later transcribed.
The interviews used an open-ended conversation guided by
subject responses, with a common set of questions (see data
supplement) providing a high-level structure. After partici-
pant assent, the researcher showed the subject their clinic’s
workflow and information flow diagrams. After familiariz-
ing the participant with the models, initial questions focused
on model sections specific to the subject’s role. The partici-
pant was asked how well the model described their actual
workflow. The interviewer probed subject responses to
clarify difficult to observe workflow details such as decision-
making processes and to obtain additional information
about EHR use.
Analysis of Interview Data and Revision
of Models
The transcribed interview data were transferred to the elec-
tronic notebook. Data analysis focused on differences between
observed and reported actions, which typically related to
elements of decision-making and task order. Both types of
models were revised in response to interview data.
Results
We conducted more than 150 hours of direct observation
and nine interviews across the three clinics. A researcher
(KMU) observed 56 hours of patient care and other activities
in the MS clinic, which included observation of 41 patient-
provider interactions. Observation was conducted in the CF
clinic for 44 hours, during which 48 patient-provider inter-
actions were observed. The researcher spent 52.5 hours in

the DM Clinic, including observation of 68 patient-provider
interactions. Interview subjects included: one participant
from the MS clinic, three participants from the CF Clinic,
and five participants from the DM Clinic.
During data analysis, we compared workflow, information
flow, and relationship with technology to understand cross-
site similarities and differences. We developed models of
workflow and information flow and assessed the role of
technology as a partner within and across the clinics. Based
on common elements and unique attributes of each environ-
ment, we also developed a set of ten guidelines for HIT
design for chronic disease care (Table 1).
Overview of Clinic Characteristics
The three clinics represented a range of disease characteris-
tics, patient populations, and clinic attributes (summarized
Table 1 y Guidelines for HIT Design for Chronic
Disease Care
� Applications should be designed to support shared needs and
behaviors in chronic disease care.
� Applications should be designed to allow for customization
for
disease-specific needs.
� Applications should allow customization to support the needs
of different types of users.
� New approaches for information input into the EHR should be
explored.
� Efficient transfer of data from medical devices into the EHR
should be supported.

� Information scanned into the system should be searchable,
quickly viewable, and more accessible.
� The EHR should be designed so that users are able to search
through the EHR quickly and easily to filter out important
information.
� Alternate methods of displaying the longitudinal data for
individual patients should be investigated to determine if they
assist in the cognitive processing of electronic data.
� New tools and processes should be as efficient as existing
approaches or yield significant benefits to users to promote
adoption.
� The reasons behind organizational and personal resistance to
technology should be addressed to promote adoption.
in Table 2). Of note, those clinics in the main clinic building
were closest to the informatics support center. Clinic pro-
viders and staff ranged from EHR “super users” and tech-
nology champions38 to novice users.
Patterns in Workflow and Information Flow
Workflow Models
Each clinic had an overall pattern of work that we summarized
graphically in workflow diagrams. The workflow diagrams
show sequences and patterns of tasks in the clinics, as well as

information modalities (telephone, paper, computer) used in
specific work tasks. The full workflow diagrams for all three
clinics are available in the data supplement.
Although specific details of workflow differed between
clinics, the clinics shared many common workflow elements,
as shown in Figure 2. Of note, between appointment contact
was a core component of care in each clinic, as were
expectations of regularly scheduled return visits. Providers
in all three clinics used the EHR, although there was
appreciable intra- and interclinic variability in reliance on
the system and the timing of the interaction. Providers in the
MS clinic frequently relied on paper charts maintained by
the clinic. In contrast, the CF and DM clinics did not
Table 2 y Clinic Characteristics
Characteristic Multiple Sclerosis
Location Rehab hospital away from other
clinics and support services
Main clin
Self-described use
of HIT*
“Fully electronic” Enthusia
Involvement in HIT
development
Not involved Some pa
Computer
availability

Each examination room,
charting room, provider
offices
Each exa
chartin
offices
Computer systems
used
Practice management software,
EHR
Practice
disease
Care locations Examination rooms, hallways Examina
Patients managed
by clinic
3000
Attending
physicians
2
Other physicians Occasionally 1–2 residents or
fellows
Typically
Nurses 1 full-time nurse, 2 part-time
nurses

2 intake
Ancillary providers None 1 dietitia
Dedicated
examination
rooms
4
Recommended
minimum visit
frequency
Every 6 mo Every 3 m
Notes: *Initial descriptions of HIT use provided by key
informants during
maintain paper charts. All care was documented in the EHR
as either scanned notes or electronically created documen-
tation. All three clinics used the EHR for record review and
secure messaging. Two specific portions of the care cycle
will be discussed in detail to illustrate the similarities and
differences: the hand-off from nurse to provider and the
postappointment completion activities.
In all clinic encounters, a hand-off from the nurse to the
provider (Figure 3, available as an online data supplement at
http://www.jamia.org) occurred as patients transitioned
from intake to provider management. During the course of
the study, the hand-off processes in all three clinics consisted
of nurses gathering various paper forms, placing them in
clinic-specific locations, and writing the patient name on a
form or Whiteboard. Nurses used paging systems or check
marks on a paper schedule to notify providers when
patients were ready. The purpose of the hand-off process

was to transfer patient information and, regardless of data
type, the information transfer process was primarily pa-
per-based.
Three main differences were identified: visit structure, types
of information transferred, and allocation of physical space.
stic Fibrosis Diabetes Mellitus
ding Main clinic building
T proponents Usage described as variable depending
on provider
ip to customize EHR Limited partnership to develop disease
management tools in EHR
on room, nurses station,
, hallways, provider
Each examination room, nurses station,
charting room, provider offices
ement software, EHR,
ry software
Practice management software, EHR
oms Examination rooms, provider offices
140 6000
2 10
llows Occasionally 1–2 residents or fellows

1 CF nurse-specialist 5 intake nurses, 2 DM nurse educators
cial worker 2 dietitians
4–5 15
Every 3 mo
Cy
ic buil
stic HI
rtnersh
minati
g room
manag
regist
tion ro
2–3 fe
nurses,
n, 1 so
o
preliminary interviews.
and D

The number and type of providers in each clinic directed the
structure of patient visits. Patients in the CF and DM clinics
had access to ancillary providers such as social workers and
dietitians; patients in the MS clinic did not. Variations in
hand-off processes related to providers were most pro-
nounced in the DM Clinic, due to the large number and
different types of providers. Nurses in the DM clinic de-
scribed a binder maintained by the charge nurse to track
provider preferences regarding patient intake and hand-off
processes, such as frequency of point-of-care tests and types
of printed documentation. Appointments in the DM clinic
had a structured schedule; a patient seeing both a dietitian
and a physician on the same day was given a scheduled
appointment time for each provider. In contrast, the CF
clinic followed a more fluid approach to patient visits with
ancillary providers, with one scheduled visit time that
covered all care providers. These scheduling differences
fundamentally altered workflow patterns between clinics.
The types of data transferred in the hand-off process also
varied. The MS clinic primarily used qualitative data such as
assessment of gait and patient reports of disease status. The
DM clinic relied heavily on quantitative data such as blood
glucose and glycosylated hemoglobin values. In the CF
Clinic, both types of data were used. Management of space
also affected workflow. In the MS and CF Clinics, a patient’s
entire appointment (from intake to provider workup and
treatment) took place in one examination room. In the DM
Clinic, moving the patient from the intake area to either an
examination room or a secondary waiting area was part of
the hand-off process.
In contrast to the nurse hand-off, the appointment comple-
tion activities (Figure 4, available as an online data supple-
ment at http://www.jamia.org) often involved the EHR.

Across all clinics and providers, on appointment completion
the patient visit was documented in the EHR. Nevertheless,
F i g u r e 2. Overview workflow diagrams for the MS, CF,
there were multiple differences in workflow related to
appointment completion, organized into three groups: the
role of paper versus the EHR, disease-specific documenta-
tion needs, and variability in EHR usage among clinicians.
Providers and staff entered patient visit documentation into
the EHR through four different routes: scanned handwritten
notes (MS, DM), phone dictation (MS), dictation software
(DM), and typed by the provider (CF, DM). Differences in
disease-specific documentation needs were typically man-
aged through the development of clinic-specific templates in
the EHR. For example, providers in the CF clinic used a
common note template designed by one of the clinic providers.
We previously discussed in detail EHR usage variations
among providers in the DM clinic.39 Similar variations
existed among providers in all clinics and directly impacted
workflow. In addition to variations in documentation mo-
dality, we observed a wide-range of EHR usage patterns
during patient-provider visits, as shown in Table 3. While
some providers started notes in the EHR while with pa-
tients, other providers recorded information on paper and
entered notes in the EHR during appointment completion,
frequently after the patient left.
Information Flow Models
We developed graphical models of information flow for
each of the three clinics (available in the data supplement),
as well as a generalized model of chronic disease care
information flow (Figure 5). A common element across all
the information flow models was the patient as an informa-
tion hub. We found that patients provide and receive
substantial amounts of information related to their care,

but did not interact with the EHR. One provider described
the role that patients play in disease management by
saying:
When you have a chronic disease, it’s not something that a
doctor does to the patient, it’s something that the doctor and
M Clinics.
the patient do together.
Providers and staff used HIT applications in the clinics for
information access, input, and communication. Informa-
tion access involved using the EHR to review existing
data. Information input involved using the EHR to enter
new data or edit existing data. Communication involved
using functions of the EHR to communicate with other
people or entities (e.g., pharmacies). Different types of
users employed subsets of these three main EHR func-
tions (Table 3).
Providers used six types of information to different degrees
depending on location: laboratory results, radiology im-
ages, other test results, external medical records, internal
medical records, and patient-reported status information.
We also identified eight different sources of information:
Table 3 y Variations in HIT use Among Provider Type
EHR, paper records, fax, mail, e-mail, documents brought
by patient, patient verbal reports, and device-generated
data.

Existing HIT resources supported varying degrees of infor-
mation presentation and synthesis. All clinics noted chal-
lenges using the EHR to integrate information from external
providers, most of which was presented as scanned images
in the EHR. The scanned information was also not incorpo-
rated into summaries of patient data over time.
Relationship between Workflow and Information Flow
Workflow and information flow were interconnected, and
were even more significant in integrated practice environ-
ments like the CF and DM Clinic, where patients interacted
with multiple providers during a single visit. For example,
s
provider differences in choice of input modality for patient
ic dise
visit notes and the timing of entering these notes into the
EHR were often explained as being related to efficient
workflow. Provider choices also impacted information flow
and the workflow of others. Handwritten documentation
required other staff members to scan the documents and, in
the case of the MS clinic, to maintain paper charts. More-
over, paper documentation created a time lag before the
information was available to others. Phone dictation re-
quired the provider to review the dictated document after
transcription, again causing in a time delay between the visit
and information availability. Dictation software and notes
typed by the provider resulted in immediate availability of
information and did not involve other staff.
Technology as a Partner

The Role of the Electronic Health Record
Providers and staff throughout the three clinics expressed
satisfaction with many of the EHR features, concurring with
findings of survey-based studies of previous users of the
same EHR system.40 As noted by one provider:
I love the computer. I think it’s fantastic. I love StarPanel. I
love the electronic medical record. I love having access, point-
and-click access, to the problem list, the medication list, all the
labs, all the previous entries from other clinicians, appoint-
ments, oh, I just couldn’t live without it now. I mean, I’m
totally
dependent on it.
However, providers had not fully adopted the EHR system
throughout these clinics. While some functions of the EHR
F i g u r e 5. Generalized information flow model for chron
had become team players41 in providing care, other func-
tions were perceived as barriers to efficient care delivery.
One provider explained his relationship with the EHR by
saying:
The computer is useful, but again, you know, there are layers
you go through the computer so you can’t access that
information, you can’t—it’s not like flipping through a chart.
It’s like looking at a book versus looking at something on a
screen.
Subjects commented that they were unaware of or unfamil-
iar with many of the EHR features. For example, the
provider who earlier described being totally dependent on
the EHR also said:
And I know that I underutilize it [the EHR]. I know that there
are probably a lot of things that I could use it for and I don’t

even know what I don’t know.
Many providers lacked time to learn about additional EHR
features. This was complicated by limited documentation
for end users and multiple feature access options. Many
providers expressed concerns about the amount of time
required to become proficient in EHR use.
Providers found both existing and new functions challeng-
ing to learn. Providers in the CF and DM clinics described a
steep learning curve and difficulty becoming efficient in use
of electronic prescribing functionality implemented during
the course of the study. One provider discussed patients
becoming restless while she was trying to learn how to use
the functionality and switching back to handwritten pre-
ase care.
scriptions as a result. As one provider lamented:
I just spend more and more time here. Every time they roll
out something wonderful, it just takes more of my time.
Another provider, who had decided not to use the prescrip-
tion-writing feature, discussed the balance between the
value of new technologies and the time required for use:
I’d be interested in anything that improves patients’ safety.
At the same time, I’m just very cognizant of anything that
would involve new responsibilities by the physician . . . if you
can show that it saves time, then I think it’s a different issue.
These examples echo comments heard repeatedly across all

three clinics regarding time, effort, and benefits of EHR use.
Despite widespread deployment of computers in all three
clinics, both the location and the number of computers
constrained EHR use. Multiple computers were located
outside of examination rooms in each clinic, but heavy
demand created access problems in some areas. While
computers were available in every exam room in all three
clinics, one provider described her hesitation in using the
computer in the exam room by saying:
It feels very awkward to me to sit there and type while they
are looking over my shoulder and I can’t look at them. I
mean, I just think it impacts the relationship building to do
that.
This concern about the impact of EHR use on the patient-
provider relationship concurs with previous research show-
ing changes in communication and cognition resulting from
EHR use,42,43 although other studies have shown EHR use
may have a positive impact on patient perceptions of
patient-provider interactions.44 Providers not proficient in
typing had difficulty using the EHR, particularly in the
examination room. These providers typically recorded infor-
mation on paper and dictated their notes later, adding to the
amount of time required for documentation work. Providers
who entered notes into the EHR while with patients
described a minimal amount of time required after ap-
pointments to complete clinical documentation. In con-
trast, providers who did not document in the EHR while
with patients complained about excessive documentation
burdens:
I just don’t do that [enter information while with patients]. I
have to do it between patients. That’s what I do during my
lunch breaks, it’s what I do at the end of the day, it’s why I’m

here until 6:30 or 7:00 at night.
The user interface design constrained data entry in some
cases. For example, the EHR did not support a homunculus,
a graphical representation of the human body, used to
document neurological status in the MS clinic. The providers
documented this information on a customized paper form
that was later scanned into the EHR.
Overall, several EHR features were viewed with ambiva-
lence by users. An example of this is the Message Basket, an
EHR feature that enables staff and providers to send and
receive messages about patient care. While providers appre-
ciated the value this feature, they also found it time-
consuming to learn and use. One provider described the
Message Basket feature as useful but also a “time-sucking
pit”.
Communication
We observed the use of multiple communication modalities,
both synchronous and asynchronous, within the clinics
(Table 4) to support communication among physicians,
nurses, ancillary providers, administrative staff, and pa-
tients. Several previous studies demonstrated the complex,
interruption-oriented communication patterns in hospital
settings45,46 and we observed similar complexity and inter-
ruptions in the ambulatory setting. Synchronous modalities,
such as face-to-face conversation, were commonly used to
direct workflow and resolve issues. Synchronous communi-
cation often provided fast answers but also interrupted
others’ work, as discussed in previous studies.47 Asynchro-
nous communication modes also caused interruptions. In
the DM Clinic, intake nurses normally paged providers to
notify them when intake was completed. DM providers

noted that these pages prompted them to wrap up their
current patient visit. Although the page required no direct
action, it interrupted workflow and modified provider
behavior.
Paper artifacts played an integral role in communication in
all three clinics. The clinics used a paper process for ordering
laboratory tests with multiple clinic-specific forms. In the
DM clinic alone, three distinct forms were used depending
on test type. Participants commented that the ordering
process was time-consuming and they would prefer an
electronic order-entry system. Paper artifacts were also used
for quality improvement. For example, the CF clinic used a
checklist to encourage compliance with recommended treat-
ment guidelines. On the day before each patient’s visit, a
technician manually prepared the checklist by compiling the
patient’s test results and treatment recommendations.
Nurses combined this checklist with other paperwork dur-
ing the patient intake process, providers used and updated
the form during the patient visit, and a technician manually
Table 4 y Communication Modalities Used in All
Three Clinics
Communication Modality Used By
Type: Synchronous
Face-to-face conversation All subjects
Telephone Admin ↔ Provider*
Provider ↔ Provider
Patient ↔ Provider
Type: Asynchronous with notification of receipt
Patient portal messages Patient ↔ Provider
StarPanel Message Basket Admin ↔ Provider

Nurse ↔ Provider
Type: Asynchronous without notification of receipt
Email Admin ↔ Provider
Nurse ↔ Provider
Fax External provider ↔ Provider
Pager Admin ↔ Provider
Nurse ↔ Provider
Paper forms Admin ↔ Provider
Nurse ↔ Provider
Voicemail Admin ↔ Provider
*Provider: Physician or Nurse Practitioner.
entered the resulting data into a national disease registry
after the visit. The checklist served both as a prompt for
evidence-based treatment as well as a tool to coordinate
patient care activities among different types of providers
(e.g., physician, dietitian, social worker). Our results concur
with previous research discussing the important role paper
artifacts, such as sign out sheets used during inpatient

rounds,48 play in healthcare communication.
Discussion and Recommendations
Our results demonstrated the strengths and the weaknesses
of an existing HIT infrastructure. Although the EHR used in
the study clinics is a locally developed system, the observed
positive and negative attributes of the HIT systems and the
implementation process are not unique to this environ-
ment.49,50 Clinic staff and providers documented and
tracked patient status over time. Many providers integrated
the EHR into their routine workflow. However, the primary
care orientation of the EHR design did not fully support the
needs of chronic disease care providers. Based on our results
in three diverse ambulatory clinics, we developed a frame-
work of ten guidelines for the design and implementation of
HIT solutions for chronic disease care (Table 1).
The key commonalities and significant differences among
the clinics support a modular approach to HIT design for
chronic disease care. Core functionality could support com-
mon needs across clinic environments including: tracking
patient disease status over time, displaying previous treat-
ments and responses to previous treatments, analyzing
trends, assisting with patient education materials, and facil-
itating communication with patients and throughout each
clinic. The core HIT functionality could be supplemented
with disease-specific modules. For example, the types of
information recorded in notes varied between clinics as a
function of disease-related information requirements.
Specialized templates may be necessary to meet unique
information needs. However, many components of disease-
specific HIT modules could be repurposed for multiple
clinics. For example, a generic nutrition module could be
designed to support the needs of diverse specialty clinics
(e.g., CF, DM, obstetrics, gastroenterology) where tracking

dietary information is an important component of care.
Additional specialized modules should include qualitative
data tracking, tailored order entry and prescription writing,
disease-specific self-management support, and graphical
input elements (e.g., the homunculus would also be useful in
neurology, orthopedics, and physical therapy). In addition
to customizing technology to meet disease-specific needs,
HIT design should support different users’ needs. Clinic
personnel collect and enter different types of data and have
different information needs depending on their role and
responsibilities within the clinic. Interfaces that could be
customized to support individual workflow needs and pref-
erences could improve efficiency, user satisfaction, and data
quality.
As with any “engineering” process, it is vital to have proper
design strategy. Developers and implementers did not have
comprehensive requirements for each chronic disease clinic
during initial EHR implementation; later phases of imple-
mentation did not include time to gather and incorporate
this information into design. Although some modules to
assist with chronic disease care had been developed,51 the
three clinics did not have access to these modules during the
study period. Addressing new requirements as they arise in
implementation is a necessity in user-responsive modular
HIT development.
Entering information into existing HIT, especially during
interaction with patients, was difficult for many providers.
Barriers included lack of typing proficiency, data collection
form design, and hardware placement. New input modali-
ties, such as tablet computers and software-based solutions
like graphical forms, should be developed and evaluated.
Advanced speech recognition software could also facilitate

data entry.
Data are entered into the EHR from multiple sources and in
multiple formats including medical device downloads,
scanned paper forms and handwritten notes, and informa-
tion recorded electronically. As time passes, the volume of
information in the EHR increases tremendously. Chronic
disease care providers are routinely required to search and
filter through this disparate collection of information in an
attempt to find the data they need at a particular time; the
EHR must facilitate this process. Providers also need to
rapidly and accurately synthesize information to formulate a
coherent picture of patient status over time and then to
inform treatment decisions. Providing longitudinal views of
patient disease history is a requirement of HIT for chronic
disease care.
The study also highlighted the challenges of HIT adoption in
chronic disease care. Our results demonstrated that end users
will create inefficient, but policy-compliant, workarounds to
accomplish tasks when the HIT does not meet their needs.52
Perception of system impact on time and workflow can be as
important as the actual impact itself. Barriers to adoption
must be investigated and addressed to bridge implementa-
tion chasms.53 The inadequate transfer of knowledge among
HIT designers, implementation teams, and end users can
inadvertently create barriers to adoption. Multiple providers
stated that they were aware of or proficient in using only a
small fraction of the EHR’s functionality. Finding better
ways to help end users become proficient with the EHR
features they need, including addressing user interface
design challenges, would enhance technology adoption.
Study Limitations
This study provides a picture of workflow, information flow,

and computer use in three chronic disease clinics at one
Academic Medical Center. We selected qualitative methods
based on the research questions. Quantitative methods such
as time-motion studies could supplement the understanding
of aspects of workflow in ambulatory chronic disease care. A
single researcher (KMU) collected the data, introducing the
potential for observer bias. To address this potential limita-
tion, study procedures and data were extensively reviewed
and refined by the entire research team. In addition, inter-
views were conducted with clinic personnel to obtain feed-
back on the validity of observations and conclusions as a
form of member-checking.54 The three clinics all used the
same EHR system. Researchers previously documented high
levels of satisfaction among primary care providers using
this same EHR,55 but before this study little was known
about usage patterns and satisfaction among specialty care
providers. A different EHR system would likely impact
workflow in different ways, underscoring the need for
additional qualitative research to triangulate the needs of
specialty care providers. Chronic disease care was only
studied in disease-specific clinics, yet primary care physi-
cians also provide chronic disease care. Changes to policies,
procedures, staff, and informatics tools were implemented
during the course of the study. While the dynamic nature of
the work environment presented challenges, we collected
data on these changes as part of the study and incorporated
this information into our analysis.
Conclusions

This study examined similarities in workflow and informa-
tion flow between three ambulatory chronic disease clinics.
Differences between the clinics and within each clinic were
also identified.
The results showed that existing technology did not fully
support users’ workflow or information needs. Many users
had difficulty with the available methods for data input. We
identified the challenge of using a relatively flat EHR
structure to provide longitudinal care as a significant barrier
to full adoption of informatics in this environment. Gaps
between how informatics tools are actually used and insti-
tutional expectations of use were identified, as were
workarounds developed by end-users to bridge gaps be-
tween available functionality and their real-world needs.
These methods may be useful in future work examining
process improvement, informatics tool adoption, and design
of HIT applications.
Health Information Technology has the clear potential to
support improved healthcare delivery and better patient
outcomes. The need for effective HIT solutions is especially
pronounced in chronic disease care, due to expanding
patient populations and disease-related complexities. Un-
derstanding workflow, information flow, and provider
needs in chronic disease care environments can contribute in
developing strategies to maximize HIT use, to enable pro-
viders to take full advantage of the capabilities of HIT
systems, and to support patients in managing chronic dis-
eases.
References y
1. Bodenheimer T, Grumbach K. Electronic technology: A spark
to
revitalize primary care? J Am Med Assoc 2003;290(2):259 – 64.

2. Hunt D, Haynes R, Hanna S, Smith K. Effects of computer-
based
clinical decision support systems on physician performance and
patient outcomes: A systematic review. J Am Med Assoc 1998;
280(15):1339 – 46.
3. Lapointe L, Rivard S. Getting physicians to accept new infor-
mation technology: Insights from case studies. CMAJ 2006;
174(11):1573– 8.
4. Ash JS, Sittig D, Poon E, et al. The extent and importance of
unintended consequences related to computerized provider
order entry. J Am Med Inform Assoc 2007;14(4):415–23.
5. Bardram JE. I love the system—I just don’t use it!
Proceedings of
the International ACM SIGGROUP Conference on Supporting
Group Work 1997:251– 60.
6. Hoffman C, Rice D, Sung H. Persons with chronic conditions:
Their prevalence and costs. J Am Med Assoc 1996;276(18):
1473–9.
7. Glasgow RE, Nutting P, King D, et al. Randomized
effectiveness
trial of a computer-assisted intervention to improve diabetes
care. Diabetes Care 2005;28(1):33–9.
8. Montori VM, Dinneen S, Gorman C, et al. The impact of
planned
care and a diabetes electronic management system on commu-
nity-based diabetes care: The Mayo Health System diabetes
translation project. Diabetes Care 2002;25(11):192–7.

9. Dorr D, Bonner L, Cohen A, et al. Informatics systems to
promote improved care for chronic illness: A literature review.
J Am Med Inform Assoc 2007;14(2):156 – 63.
10. Sequist T, Gandhi T, Karson A, et al. A randomized trial of
electronic clinical reminders to improve quality of care for
diabetes and coronary artery disease. J Am Med Inform Assoc
2005;12(4):431–7.
11. Cheng CH, Goldstein M, Geller E, Levitt R. The effects of
CPOE
on ICU workflow: An observational study. AMIA Annu Symp
Proc 2003:150 – 4.
12. Malhotra S, Jordan D, Shortliffe E, Patel V. Workflow
modeling
in critical care: Piecing together your own puzzle. J Biomed
Inform 2007;40(2):81–92.
13. Reddy M, McDonald D, Pratt W, Shabot M. Technology,
work,
and information flows: Lessons from the implementation of a
wireless alert pager system. J Biomed Inform 2005;38(3):229 –
38.
14. Shaw NT, Mador R, Ho S, et al. Understanding the impact
on
intensive care staff workflow due to the introduction of a
critical
care information system: A mixed methods research methodol-
ogy. Stud Health Technol Inform 2009;143:186 –91.
15. France DJ, Levin S, Hemphill R, et al. Emergency
physicians’
behaviors and workload in the presence of an electronic White-
board. Int J Med Inform 2005;74(10):827–37.

16. Lium JT, Tjora A, Faxvaag A. No paper, but the same
routines:
A qualitative exploration of experiences in two Norwegian
hospitals deprived of the paper based medical record. BMC
Med Inform Decis Mak 2008;8:2.
17. Balka E, Kahnamoui N, Nutland K. Who is in charge of
patient
safety? Work practice, work processes and utopian views of
automatic drug dispensing systems. Int J Med Inform 2007;76
(Suppl 1):S48 –57.
18. Symon G, Long K, Ellis J. The coordination of work
activities:
Cooperation and conflict in a hospital context. CSCW
1996;5(1):
1–31.
19. Fontanesi J, De Guire M, Chiang J, Holcomb K, Sawyer M.
Applying workflow analysis tools to assess immunization de-
livery in outpatient primary care settings. Joint Community J
Qual Improv 2000;26(11):654 – 60.
20. Fontanesi J, De Guire M, Chiang J, et al. The forms that
bind:
Multiple data forms result in internal disaggregation of immu-
nization information. J Pub Health Manag Pract 2002;8(2):50 –
5.
21. Hallock M, Alper S, Karsh B. A macro-ergonomic work
system
analysis of the diagnostic testing process in an outpatient health
care facility for process improvement and patient safety. Ergo-
nomics 2006;49(5– 6):544 – 66.

22. Johnson K, Fitzhenry F. Case report: Activity diagrams for
integrating electronic prescribing tools into clinical workflow.
J Am Med Inform Assoc 2006;13(4):391–5.
23. Zai AH, Grant R, Estey G, et al. Lessons from implementing
a
combined workflow-informatics system for diabetes manage-
ment. J Am Med Inform Assoc 2008;15(4):524 –33.
24. Michel-Verkerke MB, Schuring R, Spil T. Workflow
Manage-
ment for Multiple Sclerosis Patients: IT and Organization.
Proceedings of the 37th Hawaii International Conference on
System Sciences, 2004, p 60145a.
25. Casper G, Karsh B, Or CK, et al. Designing a technology
enhanced practice for home nursing care of patients with
congestive heart failure. AMIA Annu Symp Proc 2005:11620.
26. Unertl KM, Weinger M, Johnson K. Applying direct
observation
to model workflow and assess adoption. AMIA Annu Symp
Proc 2006:794 – 8.
27. Huberman A, Miles M. Data Management and Analysis
Meth-
ods in Collecting and Interpreting Qualitative Materials, Thou-
sand Oaks, CA: Sage Publications, 1998, pp 179 –210.
28. Ash JS, Smith A, Stavri P. Performing Subjectivist Studies
in the

Qualitative Traditions Responsive to Users in Evaluation Meth-
ods in Biomedical Informatics, New York, NY: Springer, 2006,
pp 267–300.
29. Corbin J, Strauss A. Introduction in Basics of Qualitative
Re-
search, Thousand Oaks, CA: Sage Publications, 2008, pp 1–18.
30. Rubin H, Rubin I. Why We Do What We Do: Philosophy of
Qualitative Interviewing in Qualitative Interviewing: The Art of
Hearing Data, Thousand Oaks, CA: Sage Publications, 2005, pp
19 –38.
31. Pelto P, Pelto G. Research Designs in Medical Anthropology
in
Medical Anthropology: Contemporary Theory and Method,
Westport, CT: Praeger, 1996, pp 293–324.
32. Casebeer A. Application of SWOT analysis. Br J Hosp Med
1993;49(6):430 –1.
33. Emerson R, Fretz R, Shaw L. Processing Fieldnotes: Coding
and
Memoing in Writing Ethnographic Fieldnotes, Chicago, IL:
University of Chicago Press, 1995, pp 142– 68.
34. Notebook by Circus Ponies Software,
http://www.circusponies.
com/. Accessed: Feb 6, 2009.
35. Charmaz K. Coding in Grounded Theory Practice in
Construct-
ing Grounded Theory: A Practical Guide Through Qualitative
Analysis, Thousand Oaks, CA: Sage Publications, 2006, pp
42–71.

36. Shepherd A. Hierarchical Task Analysis, Danvers, MA:
CRC
Press, 2000.
37. Checkland P, Scholes J. Soft Systems Methodology in
Action: A
30-Year Retrospective, New York: Wiley, 1999.
38. Ash JS, Stavri P, Dykstra R, Fournier L. Implementing
comput-
erized physician order entry: The importance of special people.
Int J Med Inform 2003;69(2–3):235–50.
39. Unertl K, Weinger M, Johnson K. Variation in use of
informatics
tools among providers in a diabetes clinic. AMIA Annu Symp
Proc 2007:756 – 60.
40. Johnson K, Chark D, Chen Q, Broussard A, Rosenbloom S.
Performing without a net: Transitioning away from a health
information technology-rich training environment. Acad Med
2008;83(12):1179 – 86.
41. Nemeth C, Nunnally M, O’Connor M, Klock P, Cook R.
Getting
to the point: Developing IT for the sharp end of healthcare.
J Biomed Inform 2005;38(1):18 –25.
42. Makoul G, Curry R, Tang P. The use of electronic medical
records: Communication patterns in outpatient encounters.
J Am Med Inform Assoc 2001;8(6):610 –5.
43. Ventres W, Kooienga S, Vuckovic N, Marlin R, Nygren P,
Stewart V. Physicians, patients, and the electronic health
record:

An ethnographic analysis. Ann Fam Med 2006;4(2):124 –131.
44. Hsu J, Huang J, Fung V, et al. Health information
technology
and physician–patient interactions: Impact of computers on
communication during outpatient primary care visits. J Am
Med Inform Assoc 2005;12(4):474 – 80.
45. Brixey J, Tang Z, Robinson D, et al. Interruptions in a level
one trauma center: A case study. Int J Med Inform 2008;77(4):
235– 41.
46. Coiera E, Tombs V. Communication behaviours in a hospital
setting: An observational study. BMJ 1998;316(7132):673– 6.
47. Coiera E. When conversation is better than computation. J
Am
Med Inform Assoc 2000;7(3):277– 86.
48. Nemeth C, Nunnally M, O’Connor M, Cook R. Creating
resilient
IT: How the sign-out sheet shows clinicians make healthcare
work. AMIA Annu Symp Proc 2006:584 – 8.
49. Crosson J, Isaacson N, Lancaster D, et al. Variation in
electronic
prescribing implementation among twelve ambulatory prac-
tices. J Gen Intern Med 2008;23(4):364 –71.
50. Computational technologies for effective health care:
Immediate
steps and strategic directions (document on the internet). Avail-
able at: http://www.nlm.nih.gov/pubs/reports/comptech_
prepub.pdf. Accessed Feb 21, 2009.
51. Gregg W, Jirjis J, Garriss G. Experiences with the academic

chronic care collaborative. Clin Diabetes 2007;25(2):72–3.
52. Patterson E, Cook R, Render M. Improving patient safety by
identifying side effects from introducing bar coding in medica-
tion administration. J Am Med Inform Assoc 2002;9(5):540 –53.
53. Lorenzi N, Novak L, Weiss J, Gadd C, Unertl K. Crossing
the
implementation chasm: A proposal for bold action. J Am Med
Inform Assoc 2008;15(3):290 – 6.
54. Lincoln YS, Guba E. Naturalistic Inquiry, Beverly Hills,
CA:
Sage Publications, 1985.
55. Joos D, Chen Q, Jirjis J, Johnson K. An Electronic Medical
Record in Primary Care. Impact on satisfaction, work efficiency
and clinic processes. AMIA Annu Symp Proc 2006; 394 –398.
http://www.circusponies.com/
http://www.circusponies.com/
http://www.nlm.nih.gov/pubs/reports/comptech_prepub.pdf
http://www.nlm.nih.gov/pubs/reports/comptech_prepub.pdfDesc
ribing and Modeling Workflow and Information Flow in
Chronic Disease CareIntroductionMethodsStudy DesignClinic
SelectionDirect ObservationAnalysis of Observation DataModel
DevelopmentSemi-Structured InterviewsAnalysis of Interview
Data and Revision of ModelsResultsOverview of Clinic
CharacteristicsPatterns in Workflow and Information
FlowWorkflow ModelsInformation Flow ModelsRelationship
between Workflow and Information FlowTechnology as a
PartnerThe Role of the Electronic Health
RecordCommunicationDiscussion and RecommendationsStudy
LimitationsConclusionsReferences

JONA
Volume 40, Number 7/8, pp 336-343
Copyright B 2010 Wolters Kluwer Health | Lippincott Williams
& Wilkins
T H E J O U R N A L O F N U R S I N G A D M I N I S T R A
T I O N
Achieving ‘‘Meaningful Use’’ of
Electronic Health Records Through
the Integration of the Nursing
Management Minimum Data Set
Bonnie L. Westra, PhD, RN, FAAN
Amarnath Subramanian, MD, MS
Colleen M. Hart, MS, RN
Susan A. Matney, MS, RN-C
Patricia S. Wilson, RT(R), CPC, PMP
Stanley M. Huff, MD
Diane L. Huber, PhD, RN, FAAN, NEA-BC
Connie W. Delaney, PhD, RN, FAAN, FACMI
Objective: To update the definitions and measures
for the Nursing Management Minimum Data Set
(NMMDS).
Background: Meaningful use of electronic health

records includes reuse of the data for quality im-
provement. Nursing management data are essential
to explain variances in outcomes. The NMMDS is a
research-based minimum set of essential standard-
ized management data useful to support nursing man-
agement and administrative decisions for quality
improvement.
Methods: The NMMDS data elements, definitions,
and measures were updated and normalized to cur-
rent national standards and mapped to LOINC (Logi-
cal Observation Identifier Names and Codes), a
federally recognized standardized data set for pub-
lic dissemination.
Results: The first 3 NMMDS data elements were up-
dated, mapped to LOINC, and publicly disseminated.
Conclusions: Widespread use of the NMMDS could
reduce administrative burden and enhance the mean-
ingful use of healthcare data by ensuring that nurs-
ing relevant contextual data are available to improve
outcomes and safety measurement for research
and quality improvement in and across healthcare
organizations.
The anticipated cost savings associated with health-
care reform are in part predicated on the assump-
tion that meaningful use of electronic health records
(EHRs) can streamline care processes and increase
the reuse of clinical and administrative data to im-
prove patient safety and outcomes and increase ac-
cess to care. Beginning in October 2010, the Centers
for Medicare and Medicaid Services will provide
Medicare incentive payments to hospitals and pro-
viders who meet the criteria for meaningful use of
EHRs, and reimbursement will decrease in 2015
for those who do not meet the criteria.1 The

meaningful-use criteria include electronic documen-
tation of care and exchange of data across orga-
nizations as well as reuse of the data for quality
improvement. The single most important resource in
reforming the healthcare system is the need for ac-
curate, representative, and relevant data regarding
information pertaining to patient needs, care pro-
vided, outcomes realized, and information about the
appropriate use of resources influencing care. Given
that nurses constitute the largest group of healthcare
professionals in the United States,2 it is vital that
336 JONA � Vol. 40, No. 7/8 � July/August 2010
Authors’ Affiliations: Assistant Professor (Dr Westra),
Professor
and Dean (Dr Delaney), Doctoral Student (Ms Hart), School of
Nursing, University of Minnesota, Minneapolis; Medical
Director
(Dr Subramanian), Department of Pathology, Health Partners,
Bloomington, Minnesota; Doctoral Student (Ms Matney), Office
of the Associate VP for Health Sciences Information
Technology,
University of Utah, Salt Lake City; Senior Content Engineer
(Ms
Wilson), 3M Health Information Systems Incorporated, Murray,
Utah; Clinical Professor Biomedical Informatics (Dr Huff),
Inter-
mountain Health Care, Salt Lake City, Utah; Professor (Dr
Huber),
College of Nursing, University of Iowa, Iowa City.
Corresponding author: Dr Westra, University of Minnesota,
School of Nursing, WDH 5-140, 308 Harvard St SE,
Minneapolis,
MN 55455 ([email protected]).

DOI: 10.1097/NNA.0b013e3181e93994
Copyright @ 20 Lippincott Williams & Wilkins. Unauthorized
reproduction of this article is prohibited.10
appropriate nursing clinical and contextual infor-
mation is captured, stored, and linked with other
healthcare data to evaluate and continuously shape
ongoing system changes.
The process of quality improvement is shifting
from review of paper charts to reuse of data from data
warehouses, which contain extracts of data from com-
puterized systems such as billing and claims data and,
more recently, from EHRs. However, management
and administrative data that describe the context
of care and care delivery are missing in these reposi-
tories. Organizational variables, provider and work-
force characteristics, and financial data that represent
the context of nursing care influence the effective-
ness of care delivery and patient outcomes.3 Nursing
management data are collected in every healthcare
setting; however, if the data are not captured in data
warehouses and/or lack consistency in definitions
and coding, it is impossible to reuse these manage-
ment data to compare patient outcomes and nursing
workforce issues within and across settings. These
data need to be standardized and included in data
warehouses along with EHR clinical data to meet
the criteria for ‘‘meaningful use’’ of EHRs.
The Nursing Management Minimum Data Set
(NMMDS) is a research-based minimum set of

essential data elements that can fill the void in data
warehouses to describe the management of nursing
care.4 The NMMDS was developed over 10 years
ago and is available as a paper-based survey upon
request from the developers. With the increased em-
phasis on quality improvement through reuse of
EHR data, it is essential to update and publicly
distribute the standardized NMMDS data elements,
definitions, measures, and codes to complement
EHR data. The distribution of the NMMDS is best
accomplished by linking it to a federally accepted
national terminology that is publicly available. The
Logical Observation Identifier Names and Codes
(LOINC) system is one such standard with a history
of incorporating survey instruments. In this article,
the investigators describe the methods and outcomes
of the initial steps to update the definitions and
measures for 3 of the 18 NMMDS data elements,
normalize these measures to current national stan-
dards, and disseminate the data set by linking the
NMMDS to LOINC.
Background Literature
Nursing Management Minimum Data Set
The NMMDS is a research-based minimum set of
essential data elements for capturing unit- or service-
level nursing management data that are accurate,
reliable, and useful for management decision making.
It is composed of 18 data elements organized in 3 cat-
egories: environment, nursing care, and financial re-
sources, as shown in Table 1. Each NMMDS data
element is operationalized by more specific subcon-
cepts and measures that can be linked with nursing

management data already collected.
The NMMDS development initially began in
1989. Donabedian’s structure, process, and outcome
framework5; the Iowa Model of Nursing Admin-
istration; and the USA Nursing Minimum Data Set
served as conceptual foundations for the data set.
Multiple studies, one of which was supported by the
American Organization of Nurse Executives, were
conducted to develop and establish validity of the
NMMDS data elements and definitions. Validity was
established first in acute-care settings and then in long-
term-care settings, ambulatory clinics, and community
settings.5 In 1998, The American Nurses Association
recognized the NMMDS as 1 of 2 data sets and 10
terminologies for nursing.
The value of the NMMDS is that it identifies
nursing management variables that can be combined
with billing and clinical data to build a better un-
derstanding on how nursing resources and the context
of care influence patient safety and other outcomes.
Moreover, the NMMDS can foster an increased un-
derstanding of the nursing workforce needs in terms of
quantity and level of expertise specific to specialties
and settings of care. The NMMDS has not been
implemented in its entirety within the United States
for comparison of nursing management data across
settings or extensively included in data warehouses.
Consequently, access to these data to support qual-
ity improvement activities or research is minimal.
There is beginning research on incorporating nursing
Table 1. Nursing Management Minimum
Data Set Variables and Definitions

NMMDS: Environment
NMMDS: Nursing
Care Resources
1. Unit/service unique
identifier
11. Management
demographic profile
2. Type of nursing delivery
unit/service
12. Staffing
3. Patient/client population
13. Staff demographic profile
4. Volume of nursing
delivery unit/service
14. Satisfaction
5. Nursing delivery unit/
service accreditation
6. Autonomy
NMMDS: Financial
Resources
7. Environmental complexity
15. Payer type
8. Patient/client accessibility

16. Reimbursement
9. Method of care delivery
17. Nursing delivery unit/
service budget
10. Clinical decision-making
complexity
18. Expenses
JONA � Vol. 40, No. 7/8 � July/August 2010 337
management in data warehouses.6 Specific elements
of the NMMDS have proven valuable for under-
standing costs of care,7 impact of staff turnover,8
adverse events,9 and patient morbidity and mortal-
ity.10,11 Many of the NMMDS variables have been in-
corporated into the Magnet Recognition ProgramA,
the National Database of Nursing Quality Indi-
cators, the National Quality Forum, and The Joint
Commission quality indicators. However, these ef-
forts are focused primarily on acute-care settings.
The NMMDS, on the other hand, has a broader ap-
plication as it is designed to be used in any healthcare
setting. There is a need to update and harmonize the
NMMDS data elements with current national nurs-
ing quality efforts, health data standards, and re-

search as well as to disseminate the results through
a publicly available tool. The LOINC was chosen
as the national data standard to link and dissem-
inate the NMMDS data elements because data struc-
tures are similar, and as mentioned previously, the
LOINC has been specifically used to incorporate sur-
vey instruments.
Logical Observation Identifier Names and Codes
The LOINC terminology is a publicly available, no-
cost database that provides a set of universal names
and codes with a similar structure to the NMMDS.
The structure of LOINC includes a name-value pair,
equivalent to a question or observation requiring the
user to record an answer. LOINC can be used in
computer databases and provides a national struc-
ture for transmitting data in electronic messages.12
LOINC was developed through funding by the Na-
tional Library of Medicine and the Agency for Health-
care Policy and Research beginning in 1994 at the
Regenstrief Institute, a research foundation affiliated
with the Indiana University School of Medicine.13
Major goals of LOINC are to create user-friendly
categories of terms, definitions, and codes that are
universally used by all information systems to fa-
cilitate data exchange and use within and across
healthcare organizations. LOINC is recognized by
the American Nurses Association14 and the US De-
partments of Health and Human Services as a uni-
form standard for the electronic exchange of clinical
health information and adopted by the National Com-
mittee on Vital Statistics for electronic exchange of
laboratory results.15

Methods
The first 3 data elements of the 2005 version of the
NMMDS were evaluated for (1) usefulness, (2) logi-
cal organization, (3) consistency with health data
standards and research, (4) clarity of conceptual and
operational definitions, and (5) the data structure for
linking with LOINC. An iterative process was used
to evaluate each data element. Existing standards and
the literature were reviewed for conceptual and op-
erational definitions. A resulting list of resources was
compiled, and recommendations presented to the re-
search team for consensus on the final definitions.
Each NMMDS data element, subconcept, and mea-
sure was entered in Excel, and a proposed LOINC
coding was developed. A small group from the na-
tional LOINC committee reviewed the definitions and
the proposed LOINC coding before presenting the fi-
nal updated NMMDS data elements, definitions, and
coding to the full national LOINC committee for ap-
proval. Once approved, the revised NMMDS data ele-
ments with LOINC codes were incorporated into the
next release of LOINC for public distribution and the
next version of the NMMDS.
Results
Results are reported separately for each of the
NMMDS data elements with examples of the mea-
sures. The full list of measures is available on the
University of Minnesota School of Nursing’s Interna-
tional Classification of Nursing Practice Center for
Nursing Minimum Data Set Knowledge Discovery
under ‘‘USA NMMDS Updates’’ (http://www.nursing.

umn.edu/ICNP/USANMDS/home.html).
NMMDS 01: Unit/Service Unique Identifier
The Unit/service unique identifier was defined in
2005 as the unique name, identifier, payment and
geographic data for a center of excellence, service
program, cluster by level of care, service/product line,
or service/area where the majority of patient/client
care is delivered; this is the first level of data
aggregation beyond the patient/client care provider
and included 9 subconcepts. The original subcon-
cepts were unique facility identifier, unique service
identifier, unique service name, unique unit identifier,
unique unit name, Medicare payment category, geo-
graphic location, postal location, and country code.
Of the original subconcepts, 3 were retained but up-
dated, 6 were retired, and 3 new subconcepts added
for a total of 6 subconcepts in the updated version.
The unique facility identifier, geographical location,
and postal code were retained and updated. Existing
governmental standards were used to provide mea-
sures for these, and the coding available from the gov-
ernment Web sites is referenced so that as the codes
change, the measures for the NMMDS data also are
updated, supporting consistency in data elements and
coding over time. The place of service also includes
2 for ‘‘stores’’ and ‘‘voluntary health or charitable

agencies’’ that are not in the national governmental
standards; these are important to include as these
are places where nurses practice. Software vendors
and health information technology staff track or
receive notices about changes in government stan-
dards so they can continuously update their soft-
ware; thus, the NMMDS coding also is updated
simultaneously within systems that use these vari-
ables. Two new subconcepts were added: reporting
period and facility name. The reporting period can
be any 2 dates during which data are collected, that
is, monthly, quarterly, or annually. Previously, dates
for reporting NMMDS data were not included, nor
was the name of the facility. A comparison between
the previous 2005 version and the 2009 version is
shown in Table 2.
NMMDS 02: Nursing Delivery Unit or Service
The nursing delivery unit or service can be any ser-
vice program (product line) or physical area where
care is delivered. It is the first level of aggregation
beyond the individual patient/client care provider.16
The subconcepts in the 2005 version contained 37
names for types of units or services; these codes
were retired and replaced with codes included in the
National Database for Nursing Quality Indicators
(https://www.nursingquality.org/Documents/Public/
APPENDIX%20D.pdf). Mapping to an existing
Table 2. Comparison of the Previous and New Versions for
NMMDS 01: Facility Unique identifiers
Previous NMMDS 01: Unit/Service Unique Identifier New

NMMDS 01: Facility Unique Identifiers
A facility is the highest level of an organization for data
aggregation for which unit-level data are reported. In some
cases, a facility is the same as a unit if there is only 1 unit.
A facility is the highest level of an organization for data
aggregation for which unit level data are reported. In some
cases, a facility is the same as a unit if there is only 1 unit.
01.01 Unique facility identifier 01.01 Unique facility
identifierVthe National Provider
Identifier (NPI) is a unique identification number for
healthcare providers specified by HIPAA. For the
NMMDS, the NPI for organizations will be used to
indicate the place that sends the bill. https://nppes.cms.
hhs.gov/NPPES/Welcome.do
01.02 Unique service identifier Moved to 02.01 unique unit
identifier, which includes both
service or unit identifier
01.03 Unique service name Moved to 02.02 unique unit name,
which includes both
service or unit name
01.04 Unique unit identifier Moved to 02.01 unique unit
identifier, which includes both
service or unit identifier
01.05 Unique unit name Moved to 02.02 unique unit name,
which includes both
service or unit name
01.06 Medicare payment category Retired since geographical
location and postal location

capture the essence of this information
01.07 Geographic location (state, province, country) 01.07
Geographic locationVstate or territory of the facility
where the service was provided or originated as defined by
the US Postal Service (http://www.itl.nist.gov/fipspubs/)
01.08 Postal location (mailing code, zip code) 01.08 Postal
location (zip code)Vzip code of the facility
where service was provided or originated as defined by the
US Postal ServiceVuse a 9-digit code if possible (http://zip4.
usps.com/zip4/citytown_zip.jsp)
01.09 Country code RetiredVthis version of the NMMDS will
focus on the
US only at this time
No previous subconcept 01.10 Place of serviceVplace of service
is the location, as
indicated on healthcare professional claims forms, where
the service was provided or originated. It is represented
by 2-digit codes as defined by Centers for Medicare and
Medicaid Services (CMS). (http://www.cms.hhs.gov/
PlaceofServiceCodes/Downloads/placeofservice.pdf). The
NMMDS uses the CMS list plus additional codes that end
with an ‘‘x.’’
1X StoresVthese may include grocery, pharmacy, department,
or other stores where retail goods and merchandise are sold
2X Voluntary health or charitable agencies (eg, National
Cancer Society, National Heart Association, Catholic charities)
No previous subconcept 01.11 Reporting periodVstarting
through end date for
the period in which events occurredVnot when the data

are collected or reported
01.09.01 Start date/time
01.09.02 End date/time
No previous subconcept 01.12 Facility name
national standard used in 1,500 hospitals allows
comparison of data collected by any other nursing
setting that uses the NMMDS. The 4 subconcepts for
the service and unit identifiers and names originally
included in the NMMDS 01 data element were com-
bined into 2 measures for the unique unit/service iden-
tifier and name. Table 3 shows a comparison between
the 2005 and 2009 version for NMMDS 02: nursing
delivery unit or service data element.
NMMDS 03: Patient or Client Population
The NMMDS data element 03: patient or client
population describes the characteristics of the popu-
lation served by a nursing delivery unit or service.
Originally, there were 4 subconcepts for this data
element: specialty, developmental focus, interaction
focus, and population focus. Of these 4 subconcepts,
1 was retired, 2 were retained with new names and
updated measures, and 1 subconcept was moved to
another NMMDS data element. When we compared
the original measures for the population specialty, the
measures were redundant with measures describing

the type of unit in the previous data element; hence,
this subconcept was retired. There were 2 subcon-
cepts that were renamed for clarity. Developmental
focus was renamed chronological age, and popula-
tion focus was renamed catchment area. Develop-
ment stages were used to measure the percentage of
patients served on a unit by the developmental focus;
however, developmental stages have changed over
time, were not sufficiently detailed to describe the
age of the population served, and included over-
lapping groups. Measures for chronological age were
changed to 5-year incremental age categories plus
‘‘fetal’’ and ages ‘‘1-28 days.’’ There is no national
standard for grouping patients by age. The National
Cancer Institute’s age grouping was selected as it
provided the smallest increments for age that would
be applicable across any unit or service (http://www.
seer.cancer.gov/stdpopulations/stdpop.19ages.html).
To prevent redundancy in the NMMDS measures,
the type of client served was moved to the NMMDS
04: volume of nursing care. The volume of nursing
care includes calculations for hours of care by type of
client, type of nurse provider, and type of encounter.
Finally, a new subconcept to capture the total popu-
lation served during a reporting period was added for
comparison of the size of a unit or service and be-
comes the denominator when calculating percentages
of clients served by age or catchment area. The 2005
and 2009 version of the NMMDS 03: patient or cli-
ent population data element, subconcepts, and mea-
sures are shown in Table 4.
Discussion
During the initial phase of this study, the inves-

tigators examined the usefulness, clarity, and con-
sistency of conceptual and operational definitions
with governmental and health data standards and
research, logical organization, and the data structure
requirements for linking the NMMDS with LOINC.
The first 3 NMMDS data elements were reworded,
reorganized, redefined, and harmonized with existing
Table 3. Comparison of the Previous and New Versions for
NMMDS 02: Nursing Delivery
Unit or Service
Previous NMMDS 02: Type of Nursing Delivery Unit/Service
New NMMDS 02: Nursing Delivery Unit or Service
Identify the unique name, identifier, and type of nursing unit
or service for each component of the facility.
The unique name, identifier, and type of nursing unit or
service for each component of the facility
01.02 Unique service identifier and 01.04 unique unit identifier
02.01 Unique unit identifierVan identifier given to a cost
center by the facility for a unit, which only has meaning
within the facility; this is the first level of data aggregation
beyond the individual patient or care provider
01.03 Unique service name and 01.05 unique unit name 02.02
Unique unit nameVthe name assigned to a unit by the
facility, which only has meaning within the facility
02.01-02.37 Type of nursing delivery unit or service
(discontinue and replace)
02.40 Type of nursing delivery unit or serviceVselect all
categories that most accurately describe the unit type or

specialty (This is the National Database for Nursing Quality
Indicators list in Appendix D http://www.nursingquality.
org/Documents/Public/APPENDIX%20D.pdf). There are
2 levels of unit names, with the second unit level providing
more distinct names. There are 36 higher-level-unit names
with varying numbers of more specific unit names under
higher-level names
2.40.01 Adult critical care unit
02.40.01.01 Adult burn critical care unit
02.40.01.02 Adult cardiothoracic critical care unit
governmental and nursing quality improvement stan-
dards and research. Now that the first 3 NMMDS
data elements have been updated and are publicly
available, these data elements can be used to support
multilevel and multiagency analyses of the context of
nursing care on patient safety, outcomes, and the
nursing workforce information requirements. Given
this, there are several implications for nurse manag-
ers in use of the updated NMMDS data elements.
The NMMDS includes 18 essential data ele-
ments; this study presents an update for the first 3
data elements. As identified in Table 1, implemen-
tation of the NMMDS is useful to compare the im-
pact of various nursing care delivery models or types
and amount of staffing on workforce outcomes such
as staff autonomy, retention, turnover, and satisfac-

tion; the effective of changes such as implantation of
EHRs on decision making and patient safety; or the
impact of staff education, certification, and facility
accreditation on patient outcomes and cost savings
at the unit or service level. Imagine the information
nurse managers and administrators would have at
their fingertips if these variables were defined, coded,
and routinely collected in a standardized manner for
nurse managers and administrators to add to a data
warehouse for comparison of clinical and workforce
outcomes. For instance, studies demonstrated that
certified and advanced practice nurses improve out-
comes and reduce costs for specific patient popu-
lations including mental healthcare,17 cardiac,18
neurological,19 and orthopedic conditions.20 How-
ever, these studies are limited primarily to a small
sample size and are costly to conduct. In 2009, the
Wound, Ostomy, and Continence Nursing Society pro-
vided a grant to Westra, Bliss, and Savik (2009) for
$200,000 to evaluate the effect of certified wound,
ostomy, and continence nurses on a national sample
of approximately 1 million patients for outcomes of
urinary and bowel incontinence, urinary tract infec-
tions, and wounds including pressure ulcers, stasis
ulcers, and surgical wound. This study reuses stan-
dardized EHR and administrative data. If new data
were collected with a conservative estimate of $1
per patient, the study would cost $1 million instead
of $200,000. The cost of this study is possible only
because home care agencies collect standardized
assessment data and also track nurse visits with an
associated staff ID, which can be linked with staff-
ing characteristics such as certification. Reuse of
standardized EHR clinical data along with nursing
management data is critical to provide nurse man-

agers with cost-effective information they need for
management decisions.
Nurse managers, administrators, and researchers
must advocate for inclusion and use of the NMMDS
Table 4. Comparison of the Previous and
New Versions for NMMDS 03: Patient
or Client Population
Previous NMMDS 03:
Patient/Client Population
New NMMDS 03: Patient
or Client Population
Characteristics of the
population served by
nursing delivery unit or
service. Identify all
categories that best
describe the actual
patient/client population
served by the nursing
Characteristics of the
population served by nursing
delivery unit or service.
Identify all categories that
best describe the actual
patient/client population
served by the nursing
03.1 Specialty

(03.101-03.139)
RetiredVspecialty is redundant
of new 02.03 type of nursing
delivery unit or serviceExamples
03.101 AIDS/HIV
03.102 Birthing
03.2 Developmental focus 03.02 Chronological ageV
percentage of the population
during the reporting period of
the appropriate age served on
the nursing delivery unit or
service (this is a modification
of age categories listed at
(http://www.seer.cancer.gov/
stdpopulations/stdpop.
19ages.html)
Examples
ExamplesVfetal, 9-28 d, then
5-year increments thereafter
03.201 Fetal
03.02.01 Fetal
03.202 Infant (aged
0-12 mo)
03.02.02 birth to 28 d
03.203 Toddler
(aged 13-23 mo)

03.02.03 Aged 29 d to 1 y
03.204 Early childhood
(aged 2-6 y)
03.02.04 Aged 1-4 y
03.02.05 Aged 5-9 y
03.3 Interaction focus Moved to NMMDS 04
03.31 Individual
03.32 Family
03.33 Group
03.34 Community/
population
03.4 Population focus 03.03 Catchment areaVthis is
an estimate of the percentage
of patients served by this
nursing delivery unit or
service by geographical area.
Select the smallest
geographical unit that best
fits the population served
03.41 City/town
03.03.01 Neighborhood
03.42 District
03.03.02 City or town
03.43 County/parish

03.03.03 District catchment area
03.44 Province
03.03.04 County catchment area
03.45 State
03.03.05 Parish catchment area
03.46 Region
03.03.06 State catchment area
03.47 Nation
03.03.07 Region catchment area
03.48 International
03.03.08 Nation catchment area
03.49 Aerospace
03.03.09 World catchment area
03.03.10 Aerospace
catchment area
03.03.11 Nautical
catchment area
No previous subconcept 03.04 Total patient
populationVa count of
the patient population
during the reporting period

data definitions and coding in health information
systems. The NMMDS variables are a first step in
standardizing nursing management data that can ex-
plain variance in patient outcomes and factors influ-
encing the nursing workforce. Practical steps include
comparing the definitions for data elements reported
in this article with existing data collected by the
healthcare organization. Where data are compara-
ble, no changes are required except to request that
the data be abstracted and linked to clinical and bill-
ing data in data warehouses. If the organizational
data are not comparable, nurse managers need to
reevaluate the way in which they define, capture,
and store the data and request appropriate changes
for future comparison of nursing management data
across organizations.
Once nursing management data are standard-
ized, stored, and linked in data warehouses, new
reports can be requested to understand the relation-
ship between nursing management data with inter-
ventions and patient outcomes. Two of the quality
indicators for meaningful use of EHRs include the
percentage of patients receiving counseling for smok-
ing cessation and of diabetics with adequate long-
term glucose control.21 Nursing management data
can extend an understanding of factors influencing
compliance with these quality indicators by asking

such questions as: Does an increase in compliance
with these quality indicators differ by the nursing
unit? With the consistent definition and coding of
nursing units, comparisons can be made across hospi-
tals affiliated with the same health system or across
health systems. Additional NMMDS variables (which
are in the process of being updated) include certifi-
cation of nurses, staff mix, and staff turnover. How do
these variables influence compliance with quality
indicators, and, hence, reimbursement from Medicare
in the future?
Conclusion
The new criteria for meaningful use of EHRs will
impact financial incentives beginning in October
2010 and disincentives beginning in 2015. Included
in these criteria is reuse of EHR data for quality
improvement. Nursing management data, in addi-
tion to EHR data, are essential to explain variances in
the quality of care. In this article, we described the
process of updating the first 3 NMMDS data ele-
ments. These data elements are now available
publicly through the University of Minnesota School
of Nursing Minimum Data Set Knowledge Discovery
Web site and distributed beginning with release 2.24
of LOINC. We anticipate that this work will help
organizations incorporate management data into
their quality improvement programs.
References
1. Minnesota e-Health Initiative. HIT provisions of the federal
stimulus package. Published May 20, 2009. Available at http://

www.health.state.mn.us/e-health/hitech.html. Accessed January
12, 2010.
2. US Department of Labor Bureau of Labor Statistics. Oc-
cupational Outlook Handbook, 2010-2011 Edition: Regis-
tered Nurses. Updated 2009. Available at http://www.bls.
gov/oco/ocos083.htm. Accessed January 12, 2010.
3. Maas ML, Delaney C, Huber D. Nursing outcomes account-
ability. Contextual variables and assessment of the outcome
effects of nursing interventions. Outcomes Manage Nurs Pract.
1999;3(1):4-6.
4. Huber D, Delaney C. The American Organization of Nurse
Executives (AONE) research column. The Nursing Manage-
ment Minimum Data Set. Appl Nurs Res. 1997;10(3):164-165.
5. Huber D, Schumacher L, Delaney C. Nursing Management
Minimum Data Set (NMMDS). J Nurs Adm. 1997;27(4):
42-48.
6. Junttila K, Meretoja R, Seppal A, Tolppanen E, Ala-Nikkola
T,
Silvennoinen L. Data warehouse approach to nursing manage-
ment. J Nurs Manag. 2007;15(2):155-161.
7. Titler M, Dochterman J, Kim T, et al. Cost of care for
seniors hospitalized for hip fracture and related procedures.

Nurs Outlook. 2007;55(1):5-14.
8. O’Brien-Pallas L, Duffield C, Hayes L. Do we really under-
stand how to retain nurses? J Nurs Manag. 2006;14(4):262-270.
9. Fogarty GJ, McKeon CM. Patient safety during medication
administration: the influence of organizational and individual
variables on unsafe work practices and medication errors.
Ergonomics. 2006;49(5-6):444-456.
10. Aiken LH, Clarke SP, Sloane DM, Sochalski J, Silber JH.
Hospital nurse staffing and patient mortality, nurse burnout,
and job dissatisfaction. JAMA. 2002;288(16):1987-1993.
11. Tourangeau AE, Doran DM, Hall LM, et al. Impact of
hospital nursing care on 30-day mortality for acute medical
patients. J Adv Nurs. 2007;57(1):32-44.
12. Bakken S, Cimino JJ, Haskell R, et al. Evaluation of the
clinical LOINCA (Logical Observation Identifiers, Names,
and Codes) semantic structure as a terminology model for
standardized assessment measures. J Am Med Inform Assoc.
2000;7(6):529-538.
13. Huff SM, Rocha RA, McDonald CJ, et al. Development of
the Logical Observation Identifier Names and Codes (LOINCA)

vocabulary. J Am Med Inform Assoc. 1998;5(3):276-292.
14. Matney S, Bakken S, Huff SM. Representing nursing as-
sessments in clinical information systems using the Logical
Observation Identifiers, Names, and Codes database. J Biomed
Inform. 2003;36(4-5):287-293.
15. Aspden P, Corrigan JM, Wolcott J, Erickson SM, eds.
Patient Safety: Achieving a New Standard. Washington, DC:
The National Academy Press; 2004. Committee on Data
Standards for Patient Safety, Institute of Medicine, ed.
16. Delaney CW, Huber D, eds. A Nursing Management Min-
imum Data Set (NMMDS): A Report of an Invitational Con-
ference. Chicago, IL: The American Organization of Nurse
Executives; 1996.
17. Baradell JG, Bordeaux BR. Outcomes and satisfaction of
patients of psychiatric clinical nurse specialists. J Am Psychiatr
Nurses Assoc. 2001;7(3):77-85.
18. Blue L, Lang E, McMurray JJV, et al. Randomized con-
trolled trial of specialist nurse intervention in heart failure.
BMJ. 2001;323(7315):715-718.

19. King CA. Research reviews. The CNS’s impact on process
and outcomes of patients with total knee replacement. AORN
J. 2001;73(1):243, 245.
20. Russell D, Vor der Bruegge M, Burns SM. Effect of an
outcomes-
managed approach to care of neuroscience patients by acute
care nurse practitioners. Am J Crit Care. 2002;11(4):353-362.
21. Davies MW. The state of U.S. hospitals relative to
achieving
meaningful use measurements. October 5, 2009. Available at
http://www.himssanalytics.org/docs/HA_ARRA_100509.pdf.
Accessed January 12, 2010.
Anniversary Series
Nursing Implementation Science: How Evidence-Based
Nursing Requires Evidence-Based Implementation
Theo van Achterberg, Lisette Schoonhoven, Richard Grol
Purpose: Evidence is not always used in practice, and many
examples of problematic imple-
mentation of research into practice exist. The aim of this paper
is to provide an introduc-

tion and overview of current developments in implementation
science and to apply these
to nursing.
Methods: We discuss a framework for implementation, describe
common implementation
determinants, and provide a rationale for choosing
implementation strategies using the
available evidence from nursing research and general health
services research.
Findings: Common determinants for implementation relate to
knowledge, cognitions, at-
titudes, routines, social influence, organization, and resources.
Determinants are often
specific for innovation, context, and target groups. Strategies
focused on individual profes-
sionals and voluntary approaches currently dominate
implementation research. Strategies
such as reminders, decision support, use of information and
communication technology
(ICT), rewards, and combined strategies are often effective in
encouraging implementation
of evidence and innovations. Linking determinants to theory-
based strategies, however,
can facilitate optimal implementation plans.
Conclusions: An analytical, deliberate process of clarifying
implementation determinants and
choosing strategies is needed to improve situations where
suboptimal care exists. Use of
theory and evidence from implementation science can facilitate
evidence-based implemen-
tation. More research, especially in the area of nursing, is
needed. This research should
be focused on the effectiveness of innovative strategies directed

to patients, individual
professionals, teams, healthcare organizations, and finances.
Clinical Relevance: Implementation of evidence-based
interventions is crucial to professional
nursing and the quality and safety of patient care.
[Key words: diffusion of innovation, professional practice,
healthcare reform, nursing]
JOURNAL OF NURSING SCHOLARSHIP, 2008; 40:4, 302–
310. C©2008 SIGMA THETA TAU INTERNATIONAL.
* * *
O
ptimal use of research evidence in nursing is of-
ten referred to as evidence-based practice. Follow-
ing Sackett’s definition of evidence-based medicine
(Sackett, Rosenberg, Gray, Haynes, & Richardson, 1996),
evidence-based practice can be defined as “the conscien-
tious, explicit, and judicious use of current best evidence
in making decisions about the care for individual patients”
(p.71). This definition indicates the importance of using evi-
dence, while “judicious use” shows the importance of other
factors such as patient preferences and context in the care
for individual patients.
Davis and Taylor-Vaisey (1997) define implementa-
tion as the introduction of an innovation in daily rou-
tines, demanding effective communication, and removing
hindrances. Implementation is different than related terms
such as diffusion (used for a natural process of knowl-
edge spreading), dissemination (used for planned and active

knowledge spreading) and adoption (referring to decisions
on innovations, rather than use in routines).
Regretfully, numerous examples from daily nursing
practice show how the implementation of evidence in prac-
tice is often not accomplished. Studies on hand-hygiene
Theo van Achterberg, RN, PhD, FEANS, Rho Chi , Professor of
Nursing
Science, IQ healthcare; Lisette Schoonhoven, RN, PhD, FEANS,
Rho Chi ,
Associate Professor of Nursing Science, IQ healthcare; Richard
Grol, PhD
Professor of Quality of Care and Implementation Science,
Director of IQ
healthcare; all at Radboud University Nijmegen Medical Centre,
Nijmegen,
The Netherlands. Correspondence to Dr. van Achterberg, IQ
healthcare
114, Radboud University Nijmegen Medical Centre, PO Box
9101, 6500
HB Nijmegen, The Netherlands. E-mail: [email protected]
Accepted for publication August 5, 2008.
302 Fourth Quarter 2008 Journal of Nursing Scholarship
Nursing Implementation Science
practices, for instance, consistently indicate that hospital
workers are compliant to hand-hygiene prescriptions in less
than 50% of all relevant occasions (Pittet et al., 2000). Al-
though nurses tend to be somewhat more compliant than
are physicians, the overall low compliance rates are a serious
threat to patient safety and are truly puzzling considering

the well-established evidence in this area.
Similar difficulties are found in other areas in terms of
changing nurses’ behavior in order to implement evidence.
For example, difficulties in using effective measures for
pressure-ulcer prevention (De Laat, Schoonhoven, Pickkers,
Verbeek, & van Achterberg, 2006) are reported and Segaar
and colleagues (2007) have demonstrated that implementing
effective, nurse-delivered smoking-cessation interventions in
cardiology wards was also difficult.
While these examples show how implementing effec-
tive practices can be problematic, “de-implementation” can
be just as difficult. A recent study by Huizing, Hamers,
Gulpers, and Berger (2006) showed that a program directed
at discouraging ineffective use of restraints for preventing
falls in nursing home residents was largely unsuccessful.
A study by Vermeulen, Meents, and Ubbink (2007) indi-
cated that before surgery, patients are denied nutrition for
approximately four times the duration proposed in current
guidelines. These examples show how a gap between current
knowledge and practice often exists. This gap is not merely
frustrating to academics who hope to see their research re-
sults used but directly threatening to nurses’ professionalism
and the safety and quality of patient care.
Given implementation difficulties, the aim of this paper
is to provide an overview of current developments in imple-
mentation science and to apply these developments to nurs-
ing. We will describe a general framework for implemen-
tation projects, discuss common determinants of successful
and unsuccessful implementation, and describe current evi-
dence for implementation strategies. Finally we will discuss
strategies to facilitate successful implementation in nursing
practice.

A Framework for Implementation
The international literature indicates several models
that refer to implementation. Many were developed con-
cerning nursing. Titler et al. (2001) proposed the Iowa
Model of Evidence-Based Practice to Promote Quality Care.
The Iowa model has a series of steps and decision points,
taking nurses from problem or knowledge-focused trig-
gers to accomplishing an actual change in practice. Imple-
mentation is one of the many steps in this model. Here, the
authors propose the use of Rogers’ theory for diffusion of
innovation (Rogers, 1983, 2003) and that implementation
leaders should consider (a) characteristics of a new guide-
line, (b) users of the guideline, (c) methods of communi-
cating the guideline, and (d) the social system in which a
guideline is to be adopted (Titler & Everett, 2001).
The Stetler model is for applying research findings into
practice (Stetler, 1994) and is a revision of the earlier
Stetler/Marram model (Stetler & Marram, 1976). Stetler
proposes six phases from considering the use of studies
to a final evaluation of actual use in practice. Implemen-
tation occurs after several steps of critical appraisal and
decision making and is addressed in the fifth phase: trans-
lation/application. The discussion of this phase however,
goes into translating findings into practical implications
rather than considerations for choosing implementation
strategies.
In recent publications, Stetler refers to the Pettigrew and
Whipp model for content, context, and process of strategic
change (Pettigrew & Whipp, 1992; Stetler, Richie, Rycroft-
Malone, Schultz, & Charns, 2007), thus emphasizing the
importance of the what, why, and how of strategic change.
Furthermore, she refers to the framework for strategic im-

826 Unertl et al., Describing and Modeling WorkflowResearch .docx

826 Unertl et al., Describing and Modeling WorkflowResearch .docx

Recommended

Recommended

More Related Content

Similar to 826 Unertl et al., Describing and Modeling WorkflowResearch .docx

Similar to 826 Unertl et al., Describing and Modeling WorkflowResearch .docx (20)

More from evonnehoggarth79783

More from evonnehoggarth79783 (20)

Recently uploaded

Recently uploaded (20)

826 Unertl et al., Describing and Modeling WorkflowResearch .docx