TpEEDabcdaARR2.docx

T
p
E
E
D
a
b
c
d
a
A
R
R
2
A
K
E
M

Q
P
U
C
1
h
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i
c s 8 2 ( 2 0 1 3 ) 1037–1045
j o u r n a l h o m e p a g e : w w w . i j m i j o u r n a l . c o m
ransitioning from a computerized provider order entry and
aper documentation system to an electronic health record:
xpectations and experiences of hospital staff
ric S. Kirkendall a,b,c,∗ , Linda M. Goldenhar c, Jodi L. Simon
c,
erek S. Wheeler d, S. Andrew Spooner a,b
Division of Hospital Medicine, Cincinnati Children’s Hospital
Medical Center, Cincinnati, OH, USA
Division of Biomedical Informatics, Cincinnati Children’s
Hospital Medical Center, Cincinnati, OH, USA
James M. Anderson Center for Health Systems Excellence,
Cincinnati Children’s Hospital Medical Center, Cincinnati, OH,
USA
Division of Critical Care Medicine, Cincinnati Children’s
Hospital Medical Center, Cincinnati, OH, USA
r t i c l e i n f o
rticle history:

eceived 6 September 2012
eceived in revised form
1 July 2013
ccepted 7 August 2013
eywords:
lectronic health records
edical informatics
uality of healthcare
atient safety
ser satisfaction
a b s t r a c t
Objectives: To examine healthcare worker’s perceptions,
expectations, and experiences
regarding how work processes, patient-related safety, and care
were affected when a qua-
ternary care center transitioned from one computerized provider
order entry (CPOE) system
to a full electronic health record (EHR).
Methods: The I-SEE survey was administered prior to and 1-
year after transition in sys-

tems. The construct validity and reliability of the survey was
assessed within the current
population and also compared to previously published results.
Pre- and 1-year post-
implementation scale means were compared within and across
time periods.
Results: The majority of respondents were nurses and personnel
working in the acute care
setting. Because a confirmatory factor analysis indicated a lack
of fit of our data to the I-SEE
survey’s 5-factor structure, we conducted an exploratory factor
analysis that resulted in a
7-factor structure which showed better reliability and validity.
Mean scores for each factor
indicated that attitudes and expectations were mostly positive
and score trends over time
were positive or neutral. Nurses generally had less positive
attitudes about the transition
than non-nursing respondents, although the difference
diminished after implementation.
Conclusions: Findings demonstrate that the majority of
responding staff were generally pos-
itive about transitioning from CPOE system to a full electronic
health record (EHR) and

understood the goals of doing so, with overall improved ratings
over time. In addition, the
I-SEE survey, when modified based on our population, was
useful for assessing patient care
and safety related expectations and experiences during the
transition from one CPOE system
to an EHR.
∗ Corresponding author at: Division of Hospital Medicine,
Cincinnati C
incinnati, OH 45229, USA. Tel.: +1 513 636 1260; fax: +1 513
636 7247.
E-mail addresses: [email protected], [email protected]
386-5056/$ – see front matter © 2013 Elsevier Ireland Ltd. All
rights res
ttp://dx.doi.org/10.1016/j.ijmedinf.2013.08.005
© 2013 Elsevier Ireland Ltd. All rights reserved.
hildren’s Hospital Medical Center, 3333 Burnet Avenue ML-
9009,
(E.S. Kirkendall).
erved.
dx.doi.org/10.1016/j.ijmedinf.2013.08.005
www.ijmijournal.com
mailto:[email protected]
mailto:[email protected]
c a l i
1038 i n t e r n a t i o n a l j o u r n a l o f m e d i

1. Introduction
Over the past 20 years, research findings have accelerated our
knowledge of how healthcare providers think about adopting
and using information technology in healthcare [1–5]. Numer-
ous studies have described nurse and physician attitudes,
perceptions, expectations, and experiences around imple-
menting new clinical information systems. While most early
studies focused on physicians, nurses have also reported
favorable attitudes in the last 20 years [1,6]. Studies con-
ducted as early as the 1970s showed that both groups express
positive attitudes and expectations related to health infor-
mation technology (HIT) [7,8]. In more recent studies, most
care providers said they believed technology could improve
healthcare and healthcare delivery, including patient safety
[1,9,10]. A recent study found 93% of physicians agreed or
strongly agreed that using computers in clinical care helps
improve healthcare quality [10]. Overall, factors shown to
positively influence successful EHR implementation include
training and support, mitigating unintended consequences,
minimizing adverse effects on time and efficiency, and man-
aging or limiting the gap between expectations and perception
of outcomes [11–15].
While surveys have shown that healthcare providers are
overall optimistic toward EHRs, they are still concerned with
privacy and security, workflow changes, distraction from
direct patient care, and other unintended consequences of
using an EHR system [9,16–23], which has been shown can
result in lower stakeholder “buy-in” leading to potential
rejection of the system [12,15,24]. Indeed, “buy-in” and user
attitudes may prove to be a more critical variable for success-
ful implementation and adoption than budget, technology,
or sophistication of the vendor [25]. While general attitudes
toward EHRs remain positive, attitudes about routine use are

often negative [26]. A recent study has indicated that attitudes
and the perceived usefulness of computer technology have
shifted over time [6]. Recent changes in the healthcare and
technology landscape including adoption driven by the Mean-
ingful Use incentives may cause user attitudes and acceptance
of IT implementation projects to vary from past reports [27].
Most current studies examining the transition from one
HIT system to another have targeted the ambulatory setting
[28–30]. Only one example was found that examined transi-
tion from one electronic order entry system to a full EHR in
the inpatient setting which showed wide variation in expecta-
tions and experiences for both physicians and nurses [31]. The
overall aim of our study was to administer a previously vali-
dated nursing survey to a wide population of health providers
in the inpatient setting in order to better understand their per-
ceptions of how changing from one CPOE system to a full EHR
would affect them personally and their ability to safely care
for patients.
2. Methods
2.1. Setting
This study was conducted at Cincinnati Children’s Hospital
Medical Center (CCHMC), a 523-bed tertiary care academic
n f o r m a t i c s 8 2 ( 2 0 1 3 ) 1037–1045
medical center. CCHMC is a level 1 trauma center which, in
2010, had 1,078,798 patient encounters, 1498 active medical
staff, and 936 faculty members. In late 2008, implementa-
tion of the new EHR system (Epic SystemsTM; Verona, WI)
began in several ambulatory pilot groups. Additional outpa-
tient divisions were systematically brought on-line until all
units were live by January 2012. The inpatient implementation
go-live date was January 10th, 2010. Prior to implementa-
tion, inpatient care providers were using electronic order

entry through a proprietary vendor product with a highly cus-
tomized user interface. Most patient care documentation was
done on paper, although some aspects of nursing care were
documented electronically to facilitate research studies and
quality improvement activities. Physicians did not document
electronically prior to implementation. All order entry and
documentation has been performed electronically since the
new EHR was implemented.
2.2. Human subjects protection
This study was deemed exempt by the IRB since no patient
data were used and survey responses were anonymous.
2.3. Data collection
We administered the Information Systems Expectations and
Experiences (I-SEE) survey developed by Wakefield and col-
leagues to evaluate the hospital staff’s expectations prior to
implementation and the change in perceptions after the EHR
transition [32]. The I-SEE was selected because of its strong
psychometric properties and direct relevance to our project
in terms of measuring perceived changes in work-process
and patient care/quality resulting from EHR implementation.
Using the I-SEE in a larger and more diverse audience allowed
us to further explore and establish the instrument’s reliability
and external validity. Original survey materials are available
on the Agency for Healthcare Research and Quality’s Health
IT Toolkit website (http://healthit.ahrq.gov); the CCHMC ver-
sion is available as an online supplement. The I-SEE contains
35 questions/items distributed across 7 scales:
• Provider–patient communication (3 items)
• Inter-provider communication (3 items)
• Inter-organizational communication (2 items)
• Work life changes (4 items)

• Improved care (7 items)
• Support and resources (8 items)
• Patient care processes (8 items)
The first five scales use a 7-point response scale ranging
from much worse (−3), to no change (0), to much improved
(+3) and measure perceptions of how the new clinical infor-
mation system would (or did) impact various work processes
in the hospital. The last two scales use a non-neutral 6-
point agree/disagree Likert response scale (from Strongly
Disagree (+1) to Strongly Agree (+6)) to measure perceptions of
the information system’s implementation strategy and qual-
ity. We maintained the same response scales as the I-SEE
and except for modifying the question tense, items were
identically phrased for both pre and post implementation
http://healthit.ahrq.gov/
i n f
a
p
i
t
e
i
t
e
w
i
t
a

a
o
s
d
q
2
T
(
e
h
s
c
t
e
c
m
b
o
r
N
s
2
U
r
a
i
a
p
W
a
m

fi
w
t
T
t
m
time periods. The “other” category included social workers,
nutritionists, unit clerks, and patient attendants. The majority
of respondents had more than 10 years of experience in health
Table 1 – Respondent demographics.
T1 T2
Inpatient work unit n = 377 n = 983
Acute care 149 (40%) 345 (35%)
Critical care 64 (17%) 127 (13%)
Emergency 13 (3%) 87 (9%)
Perioperative 46 (12%) 117 (12%)
Other 105 (28%) 307 (31%)
Staff position n = 374 n = 971
Prescriber (MDs, NPs) 97 (26%) 206 (21%)
Nurse 146 (39%) 358 (37%)
Other 131 (35%) 407 (42%)
Years in healthcare n = 375 n = 981
0–3 years 48 (13%) 188 (19%)
4–10 years 108 (29%) 276 (28%)
>10 years 219 (58%) 517 (53%)
EHR experience (years) n = 298 n = 667
<3 years 62 (21%) 199 (30%)
3–4 years 60 (20%) 141 (21%)
5–7 years 96 (32%) 181 (27%)

>7 years 80 (27%) 146 (22%)
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l
dministrations (time period 1, referred to as T1; and time
eriod 2, referred to as T2). For instance, in the pre-
mplementation survey questions were phrased in the future
ense, e.g., “To what extent do you think the following will be
ither worsened, stay the same, or improved?” In the post-
mplementation survey, the question was phrased in the past
ense; “To what extent do you think the following has been
ither worsened, stayed the same, or improved?” T1 responses
ere considered staff’s perceived expectations regarding EHR
mplementation, while T2 responses were understood to be
he perceived experiences of the actual implementation. We
lso collected demographic data (work unit, staff role, tenure
s a health care provider, experience with healthcare technol-
gy, etc.) and provided a comments box for respondents to
hare additional feedback. We analyzed all available data and
id not discard incomplete data sets (those with unanswered
uestions).
.4. Survey administration timeline and participants
he survey was administered online via Survey MonkeyTM
Palo Alto, CA) at two time periods, 1 year apart:
T1: Five days prior to inpatient implementation until the day
prior to the go-live date (January 5th–9th, 2010)
T2: One year after implementation (January 10th,
2011–February 10th, 2011).
Because our email system would not allow us to send the
mail request to only in-patient staff, we had to send it to all

ospital employees at both T1 and T2. A reminder email was
ent two weeks prior to the conclusion of time T2. The email
ontained a link to the survey as well as a disclaimer in both
he subject and the body of the email requesting that employ-
es who did not work on inpatient units, had no direct patient
are responsibilities, or would not be using the EHR, delete the
essage and not take the survey. Further filtering was done
y manually eliminating responses from non-inpatient staff
r those in non-clinical roles. We were able to do this because
espondents were asked to provide their primary work unit.
o monetary incentives were offered for participating in the
urvey.
.5. Data analysis
sing SASTM statistical software (Cary, NC), we calculated
esponse rates by dividing the number of respondents (T1
nd T2 separately) by the number of eligible employees (i.e.,
npatient staff required to take the EHR training course), and
lso descriptive statistics on participant groups for each time
eriod. Since our respondent pool was different from that in
akefield’s original study, we conducted a confirmatory factor
nalysis for each target population and time period to deter-
ine if the original factor structure fit our data. Based on our
ndings, we then conducted an exploratory factor analysis, as
ell as calculations of Cronbach ˛ coefficients to determine
he construct validity and reliability of the resultant factors.
hose findings allowed us to proceed using the grouped items
hat loaded onto individual factors as the survey scales to
easure staff’s expectations and experiences. Means for each
o r m a t i c s 8 2 ( 2 0 1 3 ) 1037–1045 1039

scale (scale scores, also called factor scores) were calculated
for nurses only and for non-nurse respondents. Student t-tests
were then conducted to examine if there were statistically sig-
nificant changes in attitudes regarding the transition to the
EHR system from T1, to T2 for nurses only, for the organiza-
tion as a whole, and for nurses compared to everyone else in
the organization. A p-value of less than 0.05 was considered
significant.
3. Results
3.1. Response rates and demographics
Response rates were determined by dividing the number of
respondents (T1 and T2) by the number of eligible employees
(i.e., those required to take the EHR training course, n = 7213).
Response rates for T1 and T2 were 5.2% and 13.6%
respectively.
In T1, 97 sets of survey responses from outpatient providers
were removed prior to analysis (based on the staff’s primary
work unit response), while 731 sets of response data were
removed from T2. Many of these sets of data were also incom-
plete, that is, providers did not answer all questions from the
survey.
The sample had similar distribution and representation
across all time periods (Table 1). Most respondents worked in
acute care medical units. The “other” category represents a
heterogeneous group of inpatient locations such as psychi-
atry or short stay units. Nurses and physicians represented
the largest homogeneous groups of staff positions across both
Comparison of respondent demographics across survey time
periods.
T1, time period 1 (pre-implementation); T2, time period 2 (post-
implementation); EHR, electronic health record.

1040 i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r
m a t i c s 8 2 ( 2 0 1 3 ) 1037–1045
Table 2 – Item and scale analysis: factor loadings and reliability
assessment.
Communication* (coefficient ˛ = 0.90) Factor loading
How often families are asked the same question 0.45
Able to share important information with patients and families
0.52
Able to involve patients and families in the care planning
process 0.46
Communication between departments 0.72
Communication at end of shift handoffs 0.64
Communication when patients are transferred to different units
within the hospital 0.77
Communication when patients are transferred to other facilities
0.58
Communication when patients are readmitted or receive follow-
up outpatient care 0.73
Job Satisfaction* (coefficient ˛ = 0.88) Factor loading
The amount of time I spend preparing discharge documents 0.45
The amount of professional satisfaction I get out of my job 0.84
The effect on the hospital to recruit and retain high quality staff
0.63
How much I enjoy my job 0.83
Quality of Patient Data* (coefficient ˛ = 0.91) Factor loading

The consistency with which patient care data are recorded 0.73
The accuracy and validity of the patient care data being
recorded 0.65
Quality and Safety of Patient Care* (coefficient ˛ = 0.89) Factor
loading
The overall safety of patient care 0.53
The timeliness with which patient care services are provided
0.54
The appropriateness of patient care orders 0.66
Legibility and clarity of patient care orders 0.51
Employee Understanding and Support of Implementation**
(coefficient ˛ = 0.86) Factor loading
I support the planned change in current clinical information
systems 0.92
My coworkers support the planned change in clinical
information systems 0.69
My supervisor supports the planned change in clinical
information systems 0.58
I will have no difficulty in adapting to information systems
changes 0.48
I understand the decision to change clinical information systems
0.76
Organizational Support for Implementation** (coefficient ˛ =
0.77) Factor loading
I know who the super users are on my work unit 0.42
Sufficient resources have been provided for me to learn to use
the new system 0.85
Sufficient technical IT support will be/was available to operate
the new system 0.83

The “Rights” of Patient Care*** (coefficient ˛ = 0.97) Factor
loading
The right treatment 0.87
To the right patient 0.89
At the right time 0.88
In the right amount, dose or intensity 0.90
In the right way 0.94
By the right person 0.92
With the right information 0.91
In the right location 0.93
Question items from the survey were loaded onto scales with
other questions that respondents answered similarly. Factor
loading scores
greater than 0.40 are considered significant. Cronbach ˛ values
demonstrating the cohesiveness of the grouped question items
are shown in
parentheses next to each of the 7 scales.
Question stem and scoring:
* (T1) To what extent do you think the following will be worse
(−3, −2, −1), stay the same (0), or improved (1, 2, 3) as a result
of implementation?
* (T2) To what extent do you think the following has been
either worsened (−3, −2, −1), stayed the same (0), or improved
(1, 2, 3) as a result of
implementation?
** To what extent do you agree/disagree with the following?
(1–6 Disagree/Agree Likert scale)
*** (T1) The Epic clinical information system will improve our
ability to give care. . . (1–6 Disagree/Agree Likert scale)
*** (T2) The Epic clinical information system has improved our
ability to give care. . . (1–6 Disagree/Agree Likert scale)

i n f
c
u
3
T
t
e
r
a
r
fi
s
t
s
o
p
‘
t
0
C
s
t
w
I
o
˛
˛

are and was fairly evenly distributed in terms of experience
sing EHRs.
.2. Factor analysis
he initial confirmatory factor analysis yielded fit index values
hat were outside the range for acceptable fit, indicating a large
nough departure from the original author’s model to war-
ant further examination by conducting an exploratory factor
nalysis.
The exploratory factor analysis indicated that a 7-factor
ather than the original 5-factor structure provided the best
t to our data. The chi-square fit statistics were statistically
ignificant, indicating that there was at least one common fac-
or, but that more factors were needed. Since the large sample
izes make the chi-square test extremely sensitive, we based
ur choice of seven factors on the scree test and the inter-
retability of factors.
All eight communications-related items loaded onto one
Communication’ factor (Table 2) compared to Wakefield’s
hree items (coefficient ˛ = 0.90 versus Wakefield ˛ = 0.82, 0.86,
.83). Our Job Satisfaction factor and Wakefield’s Work-Life
hanges scale contained the same four items and had the
ame reliability value (˛ = 0.88). We renamed it ‘Job Satisfac-
ion’ because we believe it better reflected what the items
ere intended to measure. The seven items in Wakefield’s
mproved Care Expectations factor (coefficient ˛ = 0.90) loaded
nto two separate factors: Quality of Patient Data (coefficient
= 0.91) and Quality and Safety of Patient Care (coefficient
= 0.89) in our study. One item on the original survey – access
Table 3 – Scale rating trends, times T1 to mean score trends
from
the entire organization (b). The p values and trends for each sca

(a) Nurse res
Scale name T1 score
Mean (SD)
n = 146
T2 sco
Mean
n = 35
Communicationa 0.82 (1.01) 0.88 (0
Job Satisfactiona 0.03 (0.99) 0.56 (1
Quality of Patient Dataa 0.80 (1.25) 1.04 (1
Quality and Safety of Patient Carea 0.56 (1.09) 1.04 (1
Employee Understanding and Supportb 4.46 (0.92) 4.64 (0
Organizational Supportb 4.58 (1.12) 4.80 (0
The “Rights” of Patient Careb 4.40 (0.90) 4.78 (0
(b) All eligible respondents (org
Scale name T1 score
Mean (SD)
n = 377
T2 scor
Mean (
n = 983
Communicationa 0.99 (0.94) 0.95 (0.8
Job Satisfactiona 0.23 (1.01) 0.60 (1.0
Quality of Patient Dataa 1.01 (1.17) 1.11 (1.2
Quality and Safety of Patient Carea 0.85 (1.09) 1.13 (1.1
Employee Understanding and Supportb 4.67 (0.92) 4.69 (0.8
Organizational Supportb 4.54 (1.16) 4.74 (1.0
The “Rights” of Patient Careb 4.45 (0.91) 4.69 (0.9

SD, standard deviation; CI, confidence interval; T1, time period
1 (pre-impl
a Scale scoring: (−3, −2, −1, 0, +1, +2, +3) [worse ↔
improved].
b Scale scoring: (+1, +2, +3, +4, +5, +6) [disagree ↔ agree].
o r m a t i c s 8 2 ( 2 0 1 3 ) 1037–1045 1041
to information improved my ability to make good patient
care decisions loaded onto both Job Satisfaction and Qual-
ity and Safety of Patient Care and was therefore eliminated.
Items in Wakefield’s Support and Resources scale (coefficient
˛ = 0.88), comprised of 8 items, loaded onto two 2 factors in
our study: Employee Understanding and Support of Imple-
mentation (5 items, coefficient ˛ = 0.86) and Organizational
Support for Implementation (3 items, coefficient ˛ = 0.77). The
final factor in both studies contained items measuring the
nursing tenets of Patients’ “Rights” (coefficient ˛ = 0.97; Wake-
field = 0.99). SAS analytical output from the factor analysis is
available as online supplementary material.
3.3. Trends over time and group comparisons
Nurse scores significantly improved from T1 to T2, except for
Communication, which did not change significantly (Table 3).
All respondents (including nurses) answered more positively
(i.e., had fewer concerns) at T2 in terms of Job Satisfaction,
Quality and Safety of Patient Care, Organizational Support
for the Transition, and the “Rights” of Patient Care. No group
demonstrated a statistically significant decrease in mean
scores over time on any of the measurement scales.
To investigate changes in nursing perceptions compared to
those of all other care providers combined, we examined mean
scale score differences between the 2 subgroups at T1 and

T2. At T1, nurses reflected a less positive perception of the
transition in terms of its potential impact on 5 scales: Com-
munication, Job Satisfaction, Quality of Patient Data, Quality
and Safety of Patient Care, and Employee Understanding and
T1 to T2 for each survey scale for nurses only (a) and for
le are depicted in the last 2 columns.
pondents
re
(SD)
8
Mean
difference 95%
CI
p-Value Trend from
T1 to T2
.84) (−0.113, 0.232) 0.499 –
.03) (0.330, 0.724) <0.001 ↑
.20) (0.007, 0.477) 0.044 ↑
.14) (0.265, 0.699) <0.001 ↑
.84) (0.009, 0.347) 0.039 ↑
.96) (0.027, 0.421) 0.026 ↑
.91) (0.202, 0.556) <0.001 ↑

anization, including nurses)
e
SD)
Mean
difference 95%
CI
p-Value Trend from
T1 to T2
8) (−0.143, 0.071) 0.506 –
9) (0.242, 0.497) <0.001 ↑
7) (−0.055, 0.242) 0.215 ∼
1) (0.154, 0.418) <0.001 ↑
9) (−0.090, 0.126) 0.746 ∼
3) (0.064, 0.321) 0.003 ↑
5) (0.125, 0.352) <0.001 ↑
ementation); T2, time period 2 (post-implementation).
1042 i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r
m a t i c s 8 2 ( 2 0 1 3 ) 1037–1045
Table 4 – Comparing nurse responses to other care providers.
(a) Nurses vs. all other care providers (T1)
Scale name Nurses
T1 score
Mean (SD)
n = 146

Others
T1 score
Mean (SD)
n = 231
Mean
difference 95%
CI
p-Value Highest rating
group
Communicationa 0.82 (1.01) 1.10 (0.87) (0.094, 0.482) 0.004
Others
Job Satisfactiona 0.03 (0.99) 0.34 (1.01) (0.103, 0.522) 0.004
Others
Quality of Patient Dataa 0.80 (1.25) 1.16 (1.10) (0.120, 0.606)
0.004 Others
Quality and Safety of Patient Carea 0.56 (1.09) 1.03 (1.05)
(0.251, 0.697) <0.001 Others
Employee Understanding and Supportb 4.46 (0.92) 4.81 (0.89)
(0.164, 0.543) <0.001 Others
Organizational Supportb 4.58 (1.12) 4.52 (1.19) (−0.305, 0.182)
0.619 –
The “Rights” of Patient Careb 4.40 (0.90) 4.49 (0.92) (−0.102,
0.282) 0.355 –
(b) Nurses vs. all other care providers (T2)
Scale name Nurses
T2 score
Mean (SD)
n = 358
Others

T2 score
Mean (SD)
n = 625
Mean
difference 95%
CI
p-Value Highest rating
group
Communicationa 0.88 (0.84) 1.00 (0.89) (0.013, 0.242) 0.029
Others
Job Satisfactiona 0.56 (1.03) 0.61 (1.12) (−0.087, 0.200) 0.438
–
Quality of Patient Dataa 1.04 (1.20) 1.13 (1.31) (−0.066, 0.268)
0.236 –
Quality and Safety of Patient Carea 1.04 (1.14) 1.18 (1.10)
(−0.005, 0.288) 0.058 –
Employee Understanding and Supportb 4.64 (0.84) 4.72 (0.91)
(−0.031, 0.207) 0.147 –
Organizational Supportb 4.80 (0.96) 4.69 (1.07) (−0.246, 0.030)
0.124 –
“Rights” of Patient Careb 4.78 (0.91) 4.62 (0.97) (−0.283,
−0.027) 0.018 Nurses
Nurse respondent scores compared to all other care providers in
the organization at times T1 (a) and T2 (b). The p values and
highest rating
group for each scale are depicted in the last 2 columns.
SD, standard deviation; CI, confidence interval; T1, time period
1 (pre-implementation); T2, time period 2 (post-
implementation).
a Scale scoring: (−3, −2, −1, 0, +1, +2, +3) [worse ↔
improved].
b Scale scoring: (+1, +2, +3, +4, +5, +6) [disagree ↔ agree].

Support (Table 4a). At T2, nurse respondents rated the “Rights”
of Patient Care higher than other care providers while the
other care providers rated Communication higher than nurses
(Table 4b). All other scale ratings showed no statistically sig-
nificant differences across groups at T2.
4. Discussion
Our study is the first known to the authors that documents
inpatient staff expectations and experiences related to transi-
tioning from a CPOE system to a comprehensive EHR. Previous
studies largely included only nurses or physicians and have
examined the introduction of novel health informatics tech-
nology to clinicians. Our study helped address the knowledge
gap by asking all staff involved in patient care about their
expectations (pre) and experiences (post) related to the Quality
and Safety of Patient Care while transitioning from a semi-
electronic system to a full EHR.
The survey respondents were heterogeneous, represent-
ing many staff roles including patient care assistants, child
life specialists, nutrition specialists, and others (Table 1). Most
had more than 10 years of medical experience, demonstrating
that the sample was not biased toward younger, and perhaps
more computer-facile staff. The majority had at least 5 years
of experience with one form of EHR or another, likely reflect-
ing the fact that CCHMC was already utilizing a more limited
CPOE system prior to implementing an entirely new full EHR.
The I-SEE survey was originally developed and validated
exclusively in a nursing population that had no significant
experience with EMRs. Therefore we believed it was impor-
tant to examine the factor structure to ensure it was a good

fit for our study population. Indeed, the confirmatory factor
analysis revealed a lack of fit to the original factor structure
and the ensuing exploratory analysis revealed the reasons
why. Most notable was that in our study, staff answered all
communication-related items similarly (loading onto one 8-
item factor compared to Wakefield’s 3 factors). They also made
a distinction between employee versus Organizational Sup-
port for the Transition, leading to two factors rather than one
as in the Wakefield study (Table 2). One possible explanation
for the difference is that our respondents were already experi-
enced with EHRs and thus had a more nuanced understanding
of the potential impact of the changes. The inclusion of care
provider roles other than nurses may have also resulted in the
factor structure difference. Regardless, it was critical to use a
factor structure that best fit the data at hand to ensure more
valid results from additional analyses. Our work demonstrates
that the I-SEE survey can be used as a starting point to evalu-
ate the expectations and experiences of inpatient staff in an
organization transitioning to from a hybrid electronic/paper
HIT care delivery system to a full-fledged EHR, but that addi-
tional factor analyses should be conducted to ensure the fit of
the factor structure to the population being examined.
The mean scores for each scale showed a neutral to pos-
itive trend (i.e., improved perceptions) across time periods,
i n f
i
n
m

e
e
z
s
n
(
m
v
c
s
p
t
p
t
t
t
t
t
c
b
t
o
e
a
c
c
t
(
a
s
s
t
a
i
t

o
h
o
p
p
T
s
w
t
O
t
h
m
a
[
h
i
o
J
d
ndicating that expectations of staff were that the immi-
ent implementation would not worsen any of the factors
easured. Nursing responses (Table 3a) indicated that their
xperiences with the transition were more favorable than their
xpectations, more so than was expressed from the organi-
ation as a whole (Table 3b) as six out of seven scale scores
tatistically improved over time. Communication scores did
ot improve with time for the nurses or for all respondents
nurses plus others). This reflects the perception that imple-
enting the EHR did not, in their opinion, influence the

arious aspects of communication (e.g., with patients, with
olleagues, etc.). Future HIT optimization and implementation
trategies at our institution should consider these findings and
roactively demonstrate how EHRs can enhance communica-
ion.
The fact that the scores trended, for the most part, in the
ositive direction from T1 to T2 indicates that the transition
o a full EHR with a new CPOE system was less disruptive
han the staff had anticipated. We were actually surprised
hat the means at T1 were as high as they were given that
he survey was administered only 5 days before implemen-
ation, and many users were stressed (based on numerous
omments offered in the open-ended question) as staff were
eginning to understand the challenges of the HIT transi-
ion. We feel that the high regard from the staff was due to
ur organizational communication and transparency strat-
gy effectively managing expectations. An a priori governance
nd communication framework consisting of a multidis-
iplinary membership (including clinical leaders, frontline
are providers, executives, and vendor representatives) facili-
ated a real-time mechanism for bidirectional communication
before, during, and after implementation) as well as acting
s a conduit for the escalation of safety concerns from any
taff member. As a result, there were positive remarks on our
urvey that reflected staff understanding the reasons behind
he decision to implement a new EHR and, overall, did not
nticipate long-term negative consequences. This contrast is
mportant to note as implementation leaders are often selec-
ively exposed to negative commentary without the benefit
f seeing positive data such as the survey results we present
ere.
In comparing the survey results from nurses against all
ther providers, the nurses tended to be more concerned (less
ositive ratings) about the impact of the transition when com-

ared to all other groups combined (Table 4). At survey time
2, (Table 4b) they reported similar experiences to the other
taff members, demonstrating that the gap had narrowed
ith time and that their experiences with the implementa-
ion were very similar to that of the other care providers.
ne reason that the disparity in expectations may exist is
hat nurses are typically heavy users of EHRs and may have
eightened fears that the implementation would affect them
ore than other care providers perceived themselves being
ffected, especially in the few days preceding implementation
33,34]. In addition, all of the survey questions can be seen as
aving direct relevance to a typical nursing role, whereas some
tems may have had less practical meaning to some of the
ther care providers that were included in the survey (such as
ob Satisfaction question 1 regarding time spent on discharge
ocumentation).
o r m a t i c s 8 2 ( 2 0 1 3 ) 1037–1045 1043
A direct comparison of our results to previous literature
is difficult given the variety of variables to consider; provider
role, sophistication of the technology implemented, inpatient
versus outpatient care, transition from paper-based systems
to EHRs versus transition between EHR systems, etc. Nonethe-
less, there were both similarities and differences noted. Much
of what is known about nursing provider experiences is
based on the Stronge–Brodt questionnaire, which contains
fewer (and different) factors than the I-SEE survey [3,32]. Prior
research has demonstrated mixed results regarding expecta-
tions and attitudes toward EHRs by nursing staff, with Sultana
finding “unfavourable” responses and Simpson et al. shared
more positive results [4,6]. The latter study also highlighted

that nursing responses may be changing over time and under-
scores the importance of studying the effects of transitioning
between EHR systems. Additionally, Kossman et al. found that
nurses thought that patient safety had increased because of an
EHR implementation, but at the cost of quality of care [33,34].
In our study, nurses responded similarly to both quality and
patient safety questions, which resulted in the combination
of questions into one factor that was reliably rated in a pos-
itive manner. Previous literature on physician attitudes and
experiences was more limited than that available for nurs-
ing staff, and often evaluated computer skills and knowledge
[5,10]. Most physicians reported optimism when asked about
the effects of computerized records, which is congruent with
the findings presented in this survey. Finally, even less data is
available about the perceptions of healthcare providers during
EHR transition periods [28–30]. However, our results align well
with previous findings in the ambulatory setting that medical
staff are, on the whole, satisfied with the migration from one
EHR to another.
On the whole, our survey results demonstrated that the
organizational expectations for transitioning from a hybrid
paper/electronic HIT system to a fully functional EHR were
positive and that the organization met and even staff
exceeded expectations (for the items surveyed) 1 year post-
implementation. These findings may be indicative of both
the implementation strategy and our organizational culture.
Specifically, the positive findings may be due to staff having
had experience with the CPOE system and with HIT in general,
therefore mitigating potential disconfirmed expectations and
negative rating trends, as seen in earlier studies [11]. Also, we
adhered to known informatics best practices for implementa-
tion by involving users in the design process, offering strong
support services (including just-in-time, at-the-elbow support
and a responsive call center), contingency planning, close
oversight and monitoring, and robust, accessible channels of

communication from the microsystem to organizational level
[13,15,12].
4.1. Limitations
There are several limitations to this study. The T1 survey
administration was only 5 days prior to the new EHR roll-
out. As such, those responding may have been particularly
motivated to use the survey opportunity to share their con-
cerns. However, even during a time of great stress, ratings
were positive. Unfortunately our response rates were fairly
low; however the distribution across roles and the variation in
c a l i n f o r m a t i c s 8 2 ( 2 0 1 3 ) 1037–1045
Summary points
What was already known on the topic
• Healthcare providers generally have positive attitudes
and perceptions in regards to the potential of elec-
tronic health records to improve the Quality and Safety
of Patient Care.
• Despite this, many providers still have concerns about
the unintended consequences of implementation of
healthcare information technology.
• Little is known about the perceptions and experiences
of inpatient care providers in healthcare organizations
transitioning from a computerized/hybrid system to a
fully electronic health record.

What this study added to our knowledge
• Inpatient care providers at a large pediatric institution
reported positive perceptions prior to and 1 year after
a transition from a hybrid electronic health record to
a fully electronic system, with improvements in most
categories surveyed.
• The I-SEE survey, with some modifications, demon-
strated utility in surveying all care providers about the
Quality and Safety of Patient Care during the transition
r
1044 i n t e r n a t i o n a l j o u r n a l o f m e d i
responses indicates that the data represent a broad spectrum
of opinions. Regardless, the findings and conclusions should
be interpreted with these low response rates in mind, as it
is possible that our findings would not have been as positive
had more staff participated. Future surveys, at our institu-
tion and others, should consider opportunities to incentivize
potential respondents, to encourage more robust participa-
tion. Additionally, another limitation is the possibility that a
small portion of responders had some exposure to the new
EHR in the outpatient environments where it had already been
implemented. This contamination was unavoidable, and likely
of minimal impact given the implementation schedule.
4.2. Future studies
Future surveys about healthcare provider’s attitudes, expec-
tations, and perceptions toward HIT will continue to be
informative as technology advances over time and people
become more computer-savvy. As institutions transition from
one HIT system to another, further data on user expecta-

tions and experience will offer insight into if, and how, best
practices differ from transitioning from paper to an electronic
system. Finally, studies that examine the period immediately
after implementation could shed light on how user opinions
shift during this time and will serve to inform and refine best
practices after a HIT go-live.
5. Conclusions
Surveying healthcare staff expectations and experiences
while transitioning across healthcare delivery information
systems is informative in understanding the organizational
milieu during this time period and in targeting optimization
strategies. This study demonstrates the utility and valid-
ity of the I-SEE survey in measuring the expectations and
experiences of both nursing and non-nursing personnel in a
pediatric tertiary care institution. The resulting factor struc-
ture of the survey was similar to the original factor structure,
but did exhibit some differences, which made it critical for
us to use the new scales with our population. As such, other
institutions applying the instrument should strongly consider
repeating the factor analysis. Baseline expectations at our
institution were positive for all groups and experience scores
indicated that, for the most part, they improved at 1 year post-
implementation.
Author’s contributions
All authors of this manuscript contributed to the (1) concep-
tion and design of the study, or acquisition of data or analysis
and interpretation of data, (2) drafting of the article or revis-
ing it critically for important intellectual content, and (3) final
approval of the version to be submitted.
Conflict of interest statement
No authors or contributors to this manuscript have any

financial or personal relationships with other people or
period.
organizations that could inappropriately influence (bias) this
work, including but not limited to the following: employment,
consultancies, stock ownership, honoraria, paid expert testi-
mony, or patent applications/registrations.
Funding source
This project required no funding, either internal or external to
the organization/site of study.
Acknowledgements
The authors would like to acknowledge Mary Baggett for her
efforts in the statistical analysis of the data and for assistance
in editing the data analysis section of the manuscript.
Appendix A. Supplementary data
Supplementary data associated with this article can be
found, in the online version, at http://dx.doi.org/10.1016/
j.ijmedinf.2013.08.005.
e f e r e n c e s
[1] R. Scarpa, S.C. Smeltzer, B. Jasion, Attitudes of nurses
toward
computerization: a replication, Comput. Nurs. 10 (2) (1992)
72–80.
http://dx.doi.org/10.1016/j.ijmedinf.2013.08.005
http://dx.doi.org/10.1016/j.ijmedinf.2013.08.005

i n f
[2] A.H. Stockton, M.P. Verhey, A psychometric examination of
the Stronge–Brodt Nurses’ Attitudes Toward Computers
Questionnaire, Comput. Nurs. 13 (3) (1995) 109–113.
[3] J.H. Stronge, A. Brodt, Assessment of nurses’ attitudes
toward computerization, Comput. Nurs. 3 (4) (1985) 154–158.
[4] N. Sultana, Nurses’ attitudes towards computerization in
clinical practice, J. Adv. Nurs. 15 (6) (1990) 696–702.
[5] R.D. Cork, W.M. Detmer, C.P. Friedman, Development and
initial validation of an instrument to measure physicians’
use of, knowledge about, and attitudes toward computers, J.
Am. Med. Inform. Assoc. 5 (2) (1998) 164–176.
[6] G. Simpson, M. Kenrick, Nurses’ attitudes toward
computerization in clinical practice in a British general
hospital, Comput. Nurs. 15 (1) (1997) 37–42.
[7] J.M. Melhorn, W.K. Legler, G.M. Clark, Current attitudes of
medical personnel toward computers, Comput. Biomed. Res.
12 (4) (1979) 327–334.
[8] T.S. Startsman, R.E. Robinson, The attitudes of medical and
paramedical personnel toward computers, Comput. Biomed.
Res. 5 (3) (1972) 218–227.
[9] J.G. Anderson, S.J. Jay, H.M. Schweer, M.M. Anderson,
Why
doctors don’t use computers: some empirical findings, J. R.
Soc. Med. 79 (3) (1986) 142–144.

[10] D. Thomas, A. Kushniruk, J. Kannry, Housestaff and
attending physician knowledge of and attitude towards an
EMR on the eve of implementation, in: AMIA Annu. Symp.
Proc., 2007, p. 1133.
[11] R.D. Henderson, F.P. Deane, User expectations and
perceptions of a patient management information system,
Comput. Nurs. 14 (3) (1996) 188–193.
[12] N.M. Lorenzi, R.T. Riley, Managing change: an overview,
J.
Am. Med. Inform. Assoc. 7 (2) (2000) 116–124.
[13] K.G. Adler, How to successfully navigate your EHR
implementation, Fam. Pract. Manag. 14 (2) (2007) 33–39.
[14] Top 10 factors for successful EHR implementation, http://
healthcareitnews.com/news/top-10-factors-successful-ehr-
implementation
[15] N.M. Lorenzi, A. Kouroubali, D.E. Detmer, M.
Bloomrosen,
How to successfully select and implement electronic health
records (EHR) in small ambulatory practice settings, BMC
Med. Inform. Decis. Mak. 9 (2009) 15.
[16] P.A. Beiter, J. Sorscher, C.J. Henderson, M. Talen, Do
electronic medical record (EMR) demonstrations change
attitudes, knowledge, skills or needs? Inform. Prim. Care 16
(3) (2008) 221–227.
[17] E.M. Campbell, D.F. Sittig, J.S. Ash, K.P. Guappone, R.H.
Dykstra, Types of unintended consequences related to
computerized provider order entry, J. Am. Med. Inform.
Assoc. 13 (5) (2006) 547–556.

[18] N.M. Lorenzi, R.T. Riley, N.A. Dewan, Barriers and
resistance
to informatics in behavioral health, Stud. Health Technol.
Inform. 84 (Pt 2) (2001) 1301–1304.
[19] S. McLane, Designing an EMR planning process based on
staff attitudes toward and opinions about computers in
healthcare, Comput. Inform. Nurs. 23 (2) (2005) 85–92.
o r m a t i c s 8 2 ( 2 0 1 3 ) 1037–1045 1045
[20] D.B. Hier, A. Rothschild, A. LeMaistre, J. Keeler,
Differing
faculty and housestaff acceptance of an electronic health
record, Int. J. Med. Inform. 74 (7–8) (2005)
657–662.
[21] W. Ventres, S. Kooienga, N. Vuckovic, R. Marlin, P.
Nygren, V.
Stewart, Physicians, patients, and the electronic health
record: an ethnographic analysis, Ann. Fam. Med. 4 (2) (2006)
124–131.
[22] R. Frankel, A. Altschuler, S. George, J. Kinsman, H.
Jimison,
N.R. Robertson, J. Hsu, Effects of exam-room computing on
clinician–patient communication: a longitudinal qualitative
study, J. Gen. Intern. Med. 20 (8) (2005) 677–682.
[23] E. Toll, A piece of my mind. The cost of technology,
JAMA 307
(23) (2012) 2497–2498.
[24] G. Pare, C. Sicotte, H. Jacques, The effects of creating
psychological ownership on physicians’ acceptance of

clinical information systems, J. Am. Med. Inform. Assoc. 13
(2) (2006) 197–205.
[25] V. Castillo, A. Martínez-García, J. Pulido, A knowledge-
based
taxonomy of critical factors for adopting electronic health
record systems by physicians: a systematic literature review,
BMC Med. Inform. Decis. Mak. 10 (2010) 60.
[26] M.V. Bloom, M.K. Huntington, Faculty, resident, and
clinic
staff’s evaluation of the effects of EHR implementation, Fam.
Med. 42 (8) (2010) 562–566.
[27] Health information technology: initial set of standards,
implementation specifications, and certification criteria for
electronic health record technology. Final rule, Fed. Reg. 75
(144) (2010) 44589–44654.
[28] E.L. Abramson, S. Malhotra, K. Fischer, A. Edwards, E.R.
Pfoh,
S.N. Osorio, A. Cheriff, R. Kaushal, Transitioning between
electronic health records: effects on ambulatory prescribing
safety, J. Gen. Intern. Med. 26 (8) (2011) 868–874.
[29] E.R. Pfoh, E. Abramson, S. Zandieh, A. Edwards, R.
Kaushal,
Satisfaction after the transition between electronic health
record systems at six ambulatory practices, J. Eval. Clin.
Pract. 18 (2011) 1133–1139.
[30] S.O. Zandieh, E.L. Abramson, E.R. Pfoh, K. Yoon-
Flannery, A.
Edwards, R. Kaushal, Transitioning between ambulatory
EHRs: a study of practitioners’ perspectives, J. Am. Med.
Inform. Assoc. 19 (2011) 401–406.

[31] C. Sicotte, G. Pare, M.P. Moreault, A. Lemay, L.
Valiquette, J.
Barkun, Replacing an inpatient electronic medical record.
Lessons learned from user satisfaction with the former
system, Methods Inf. Med. 48 (1) (2009) 92–100.
[32] D.S. Wakefield, J.R. Halbesleben, M.M. Ward, Q. Qiu, J.
Brokel,
D. Crandall, Development of a measure of clinical
information systems expectations and experiences, Med.
Care 45 (9) (2007) 884–890.
[33] S.P. Kossman, S.L. Scheidenhelm, Nurses’ perceptions of
the
impact of electronic health records on work and patient
outcomes, Comput. Inform. Nurs. 26 (2) (2008) 69–77.
[34] S.P. Kossman, Perceptions of impact of electronic health
records on nurses’ work, Stud. Health Technol. Inform. 122
(2006) 337–341.
http://healthcareitnews.com/news/top-10-factors-successful-ehr-
implementation
implementation
implementationTransitioning from a computerized provider
order entry and paper documentation system to an electronic
health record: Expectations and experiences of hospital staff1
Introduction2 Methods2.1 Setting2.2 Human subjects
protection2.3 Data collection2.4 Survey administration timeline
and participants2.5 Data analysis3 Results3.1 Response rates

and demographics3.2 Factor analysis3.3 Trends over time and
group comparisons4 Discussion4.1 Limitations4.2 Future
studies5 ConclusionsAuthors contributionsConflict of interest
statementFunding sourceAcknowledgementsAppendix A
Supplementary dataReferences
Impact of Heath Information Technology on the Quality of
Patient
Care
Amanda Hessels, PhD MPH RN CIC CPHQ,
Postdoctoral Research Fellow at the Center for Interdisciplinary
Research to Prevent Infections
(CIRI), Columbia University, School of Nursing and Nurse
Scientist at Meridian Health in New
Jersey
Linda Flynn, PhD RN FAAN,
Professor and the Associate Dean of Academic Programs at the
University of Colorado College
Of Nursing
Jeannie P. Cimiotti, PhD RN FAAN,
Associate Professor and the Dorothy M. Smith Endowed Chair
at the University of Florida College
Of Nursing
Suzanne Bakken, RN PhD FAAN FACMI, and
Alumni Professor of Nursing and Professor of Biomedical
Informatics at Columbia University
Robyn Gershon, MT (ASCP) MHS DrPH
Professor of Epidemiology and Biostatistics and Core Faculty in
the Philip R. Lee Institute for

Health Policy Studies in the School of Medicine at University
of California, San Francisco
Abstract
Objective—To examine the relationships among Electronic
Health Record (EHR) adoption and
adverse outcomes and satisfaction in hospitalized patients
Materials and Methods—This secondary analysis of cross
sectional data was compiled from
four sources: (1) State Inpatient Database from the Healthcare
Cost Utilization Project; (2)
Healthcare Information and Management Systems Society
(HIMSS) Dorenfest Institute; (3)
Hospital Consumer Assessment of Healthcare Providers and
Systems Survey (HCAHPS) and (4)
New Jersey nurse survey data. The final analytic sample
consisted of data on 854,258 adult
patients discharged from 70 New Jersey hospitals in 2006 and
7,679 nurses working in those same
hospitals. The analytic approach used ordinary least squares and
multiple regression models to
estimate the effects of EHR adoption stage on the delivery of
nursing care and patient outcomes,
controlling for characteristics of patients, nurses, and hospitals.
Results—Advanced EHR adoption was independently associated
with fewer patients with

prolonged length of stay and seven-day readmissions. Advanced
EHR adoption was not associated
with patient satisfaction even when controlling for the strong
relationships between better nursing
practice environments, particularly staffing and resource
adequacy, and missed nursing care and
more patients reporting “Top-Box,” satisfaction ratings.
Conclusions—This innovative study demonstrated that advanced
stages of EHR adoption show
some promise in improving important patient outcomes of
prolonged length of stay and hospital
readmissions. Strongly evident by the relationships among
better nursing work environments,
HHS Public Access
Author manuscript
Online J Nurs Inform. Author manuscript; available in PMC
2016 August 26.
Published in final edited form as:
Online J Nurs Inform. 2015 ; 19: .
A
u
th
o
r M
a
n

u
scrip
t
A
u
th
o
r M
a
n
u
scrip
t
A
u
th
o
r M
a
n
u
scrip
t
A
u
th

o
r M
a
n
u
scrip
t
better quality nursing care, and patient satisfaction is the
importance of supporting the
fundamentals of quality nursing care as technology is integrated
into practice.
Keywords
electronic health records; nursing practice environment; adverse
patient events; patient
satisfaction; patient safety; health information technology
Introduction
The promise of advanced technology to transform healthcare is
underway. We are in an
exciting and dynamic period of discovery, and importantly
generating knowledge that
informs and impacts healthcare organizations, healthcare
workers and ultimately patient

outcomes. Our innovative study adds to this body of knowledge
by examining important and
untested relationships. The purpose of this study was to
examine the relationships among
electronic health record (EHR) adoption stage and hospitalized
patients’ satisfaction and
adverse outcomes (i.e., Patient Safety Indicators [PSIs],
readmissions, length of stay and
prolonged length of stay [PLOS]) while accounting for
important organizational and nurse
factors.
Background and Significance
Adverse events in hospitalized patients increase patient
morbidity and mortality and are
costly to individuals, hospitals, and society. A report by the
Institute of Medicine (IOM)
identified the top 100 healthcare research priorities for the
nation; leading the list is research
aimed at improving patient safety and the quality of care (IOM,
2009). Yet, despite an
increased focus on patient safety since the release of the IOM
report To Err is Human there
has been minimal improvement in patient safety (IOM, 2012;
Leape, et al., 2009; Wachter,

2010a, 2010b). Perhaps most disturbing are findings from a
recent large, landmark study
which indicate that, despite national attention and substantial
resource allocation, there has
been no reduction in the rate of preventable adverse inpatient
events over the last several
years (Landrigan et al., 2010). In fact, the rate of preventable
harm to patients has remained
relatively stable at 40.2 adverse events per 1,000 patient days
(Landrigan et al., 2010). These
sustained rates of inpatient adverse events are detrimental to
individuals, hospitals, and
society, costing our healthcare system more than 4.4 billion
dollars per year (Department of
Health and Human Services (DHHS), 2010a).
Tolerance with this status quo is waning. Payers, regulators,
insurers and consumers are
demanding the delivery of safe healthcare with positive
outcomes. Consumer concern
became evident in a seminal 2006 national survey of public
perspectives on ways to improve
healthcare in which 42 percent of respondents reported
experiencing inefficient, poorly
coordinated or unsafe care in the prior two years (Schoen, How,

Weinbaum, Craig & Davis,
2006). Concern remained evident in a 2011 international survey
in which up to 25 percent of
U.S. respondents reported experiencing an actual error in care
(Schoen et al; 2011).
Importantly, a consequence of low quality healthcare and poor
work environments also
includes decreased patient satisfaction (Kutney-Lee et al., 2009;
Mitchell & Shortell, 1997;
Hessels et al. Page 2
2016 August 26.
A
u
th
o
r M
a
n
u
scrip
t
A
u
th

o
r M
a
n
u
scrip
t
A
u
th
o
r M
a
n
u
scrip
t
A
u
th
o
r M
a
n
u
scrip

t
Schubert et al., 2008). The confluence of these factors has led
to a demand for healthcare
reform.
In response to this demand, the Affordable Care Act (ACA) of
2010 established the Hospital
Value Based Purchasing (VBP) program, a Center for Medicare
and Medicaid Services
(CMS) initiative that rewards acute-care hospitals with
incentive payments for the quality of
care provided (CMS, 2013). VBP places 2 percent of hospital
Medicare reimbursement at
risk by metrics of quality, outcomes, and experiences of care
(CMS, 2013). Reimbursement
associated with patient satisfaction is 30% of the at-risk base
diagnosis-related group (DRG)
operating payment (CMS, 2013). The ACA affects payment for
inpatient stays in 2,985 U.S.
hospitals (CMS, 2013).
To further support healthcare improvement the American
Recovery and Reinvestment Act

(ARRA) of 2009 includes a provision for the Health Information
Technology for Economic
and Clinical Health (HITECH) Act (CMS, 2012a, CMS, 2012b).
The belief that health
information technology (IT) will foster healthcare reform is
supported by a $35 billion
federal investment for HITECH programs, including
demonstration of Meaningful Use
(MU), (DHHS, 2010b, Office of the National Coordinator
(ONC), 2010). MU goals were
designed to occur in stages. The first phase, Stage 1 Meaningful
Use (2011–2012), focuses
on data capture and sharing (ONC, 2012). The second phase,
Stage 2 (2013–2014), advances
stage 1, and includes advanced clinical processes and clinical
decision support, and focuses
on demonstrating health system improvement through wider
adoption and process
improvement. The third phase, Stage 3 (2015), focuses on
transforming health care through
health IT. Finally, beyond 2015, a learning system of
transformed health care will be realized
(ONC, 2010).
Organizations that accept Medicare and Medicaid dollars are

eligible to participate in the
Electronic Health Record (EHR) incentive programs and receive
EHR incentive payments
beginning with a $2 million base payment, with over $5 billion
paid to date (CMS, 2012b).
Eligible hospitals that do not minimally demonstrate MU Stage
1 will be subject to
Medicare penalty payment adjustments in 2015 (CMS, 2012b,
DHHS, 2010a, HIMSS,
2012).
Fully meeting MU Stage 1 objectives includes three of five
stages of EHR adoption (Appari,
Johnson & Anthony, 2013; Garets & Davis 2008; Jha et al.,
2009), (Table 1). Hospitals at
EHR Stage 0 may have some clinical systems in place but are
considered rudimentary and
do not have all three basic ancillary systems installed. Hospitals
at EHR Stage 1 have
adopted all three core ancillary department information systems
(laboratory, radiology,
pharmacy). Hospitals at EHR Stage 2 have adopted all of EHR
Stage 1 applications and
additionally have features such as clinical data and decision
support systems, clinical data

repository and may be health information exchange capable.
Hospitals at EHR Stage 3 have
adopted all of EHR Stage 1 and EHR Stage 2 applications as
well as nursing and clinical
documentation, order entry management and features such as
electronic medication
administration record application and picture archive and
communication systems MU Stage
2 includes hospitals at EHR Stage 4 that achieved all the
preceding stages and have
Computerized Physician Order Entry (CPOE) and advanced
clinical decision support
2016 August 26.
A
u
th
o
r M
a
n
u
scrip

t
A
u
th
o
r M
a
n
u
scrip
t
A
u
th
o
r M
a
n
u
scrip
t
A
u
th
o
r M

a
n
u
scrip
t
(clinical protocols). This classification is based on the HIMSS
Electronic Medical Record
Adoption Model (EMRAM) and the taxonomy developed by an
expert consensus panel
(Garets & Davis 2008; Jha et al., 2009).
Undoubtedly, these Acts have challenged hospital
administrators as they appraise the
evidence and formulate how to direct valuable human and
material resources in efforts to
meet the provisions of both the ARRA and the ACA. The use of
the health IT is one
promising system-level initiative that may improve provider
performance and
interdisciplinary communication, reduce adverse patient events,
and ultimately improve
patient satisfaction with care (Elnahal, Joynt, Bristol & Jha
2011; Himmelstein, Wright &

Woolhandler, 2010; Staggers, Weir & Phansalkar, 2008). Some
evidence suggests that
technology does enhance communication and decision making
and positively impacts
provider performance and a variety of patient outcomes,
including patient satisfaction
(DesRoches, Miralles, Buerhaus, Hess & Donelan, 2011;
Elnahal et al., 2011; Kazley,
Diana, Ford & Menachemi, 2012; Kutney-Lee & Kelly, 2011).
However, an evidence report
published by the Agency for Healthcare Research and Quality
(AHRQ) concluded too few
studies link organizational structures and care processes with
outcomes when examining the
positive effects of EHR (Shekelle, Morton, & Keeler, 2006).
Despite widespread attention and funding, major gaps in the
evidence persist, including
exploring the influence of EHRs across differing organizational
climates, using relatively
small samples of hospitals, and the absence of any multi-site
studies to disentangle the
complex relationships among EHR, the delivery of nursing care,
and patient outcomes. By
leveraging existing databases, this study addressed these

important gaps in the empirical
literature by exploring the relationships among EHR adoption
stage, patient satisfaction, and
adverse patient outcomes while accounting for the important
features of the nursing practice
environment, such as management support, teamwork and
communication, and staffing, in a
sample of 70 New Jersey hospitals.
Objective
The purpose of this study was to examine the relationships
among electronic health record
(EHR) adoption stage and hospitalized patients’ satisfaction and
adverse outcomes (i.e.,
Patient Safety Indicators [PSIs], readmissions, length of stay
and prolonged length of stay
[PLOS]).
Materials and Methods
A secondary analysis of cross sectional data was conducted,
including the following
measures compiled from four sources: (1) adverse patient events
and PSIs using PSI
algorithm (version 3.1) from the Healthcare Cost and Utilization
Project, State Inpatient

Database; (2) patient satisfaction survey data from Hospital
Consumer Assessment of
Healthcare Providers and Systems (HCAHPS), Centers for
Medicare and Medicaid Services
(CMS) data; (3) EHR adoption stage using the EMR Adoption
Model (EMRAM) scale from
the Healthcare Information and Management Systems Society
(HIMSS) Dorenfest Institute,
(Garets & Davis, 2008; HIMSS, 2008); and (4) nurse practice
environment scores using the
Practice Environment Scale-Nursing Work Index (PES-NWI),
(Lake, 2002), and missed
2016 August 26.
A
u
th
o
r M
a
n
u
scrip
t
A

u
th
o
r M
a
n
u
scrip
t
A
u
th
o
r M
a
n
u
scrip
t
A
u
th
o
r M
a

n
u
scrip
t
nursing care scores from the New Jersey nurse survey data. All
study data were from 2006,
with the exception of HCAHPS data with a release date of
March 2008 which captures data
from July 2006 through June 2007. These years were selected so
the data was
contemporaneous with the unique nursing variable dataset
collected only 2006. The
databases were merged using unique hospital level identifiers.
The study design included
adult patients admitted to New Jersey hospitals and nurses who
were employed in those
same hospitals. Individuals under the age of 21 were excluded
from this study as the focus
of the study was adult patients and nurses who are typically
older than 21 years. No gender,
racial or ethnic groups were excluded.
Ethics Approval

The Institutional Review Board of Rutgers, The State University
of New Jersey approved
this study.
Data Sources and Variables
Patients—Patient adverse events were derived from the 2006
New Jersey State Inpatient
Database, which contains inpatient discharge abstracts and more
than 100 clinical and
nonclinical data elements such as facility identification number,
patient demographics,
admission and discharge information, payment source, total
charges, and length of stay. In
addition, International Classification of Diseases, 9th edition,
Clinical Modification (ICD-9-
CM) codes are recorded for both the principal diagnosis and
principal surgical procedures.
An expanded number of diagnosis and procedure codes and
clear demarcation of presenting
and secondary (comorbid) diagnoses are unique and important
features of the discharge data
that permit enhanced risk adjustment (Healthcare Cost and
Utilization Project (HCUP),
2012a).
Nursing-sensitive PSIs that were examined included: (a) PSI 2
death in low-mortality

diagnostic related groups; (b) PSI 4 failure to rescue; (c) PSI 13
postoperative sepsis; (d) PSI
7 central venous catheter-related blood stream infection; and (e)
PSI 8 postoperative hip
fracture. These PSIs were selected following review of
empirical definitions (including
reliability and minimum bias, coding and construct validity,
area level or provider level
metric), empirical performance indicating occurring at rates
sufficient to detect a difference,
and literature review and theoretical rationale they are sensitive
to nursing care (HCUP,
2012a, 2012b). Early hospital readmission was operationally
defined as an all-cause
readmission to the same New Jersey hospital facility from
which the patient was discharged
within seven days (HCUP, 2012a, 2012b). Prolonged length of
stay (PLOS) identifies the
distribution point at which the discharge rate declines after the
daily discharge rates peak
(Silber et al., 2009). The daily patient discharge rate was
calculated as 1/LOS (length of
stay) consistent with previous work (Silber et al., 2003, Silber
et al., 2009). The prolongation

point for hospital discharges, or day of deceleration, was
identified by Kernel-Density plots
constructed for the discharge rates by each Major Diagnostic
Categories and defined as the
day after the prolongation point. In these data, therefore, the
patient’s hospital stay is
considered prolonged if it exceeds the prolongation point (day
of hospitalization), identified
for each Major Diagnostic Categories by the Kernel–Density
plots. All data were examined
2016 August 26.
A
u
th
o
r M
a
n
u
scrip
t
A

u
th
o
r M
a
n
u
scrip
t
A
u
th
o
r M
a
n
u
scrip
t
A
u
th
o
r M
a
n

u
scrip
t
at the hospital level and therefore expressed as PSI rates per
1,000 discharges or percentage
of patient readmissions or PLOS per hospital.
Patient satisfaction was operationally defined as the hospital
level average “Top-Box” score
from the HCAHPS hospital rating measure (CMS, 2012b).
HCAHPS is a national,
standardized database of patients’ hospital experiences in short-
term, acute care hospitals.
The 27-item survey includes categories on communication with
doctors and nurses,
responsiveness of hospital staff, pain management, cleanliness
and quietness of the hospital
environment, and medication and discharge instructions. It is
reported as a set of ten
measures including 6 summary measures, 2 single items, and 2
global ratings. HCAHPS
“Top-Box” is defined as the most positive response to the
HCAHPS survey questions,

including the response “9” or “10” for the overall hospital
rating item (CMS, 2012b).
Individual patient responses are aggregated to the hospital level
by HCAHPS following risk-
adjustment for patient mix and mode of administration (CMS,
2012b).
Nurses—The nursing practice environment was measured using
the Practice Environment
Scale of the Nursing Work Index (PES-NWI), a 5 domain, 31-
item 4-point Likert-type
(ranging from strongly disagree to strongly agree) instrument
that asks nurses to characterize
the presence of features in their work environment. Subscales
from the PES-NWI used in
this study include: nurse participation in hospital affairs,
nursing foundations for quality
care, nurse manager ability, leadership, and support of nurses,
staffing and resource
adequacy and collegial nurse-physician relations (Lake, 2002).
Published internal
consistency coefficients (Cronbach’s alphas) for these subscales
range from .71 to .84 and
validity of this measure is extensively supported in the
literature (Gajewski et al., 2010;
Lake, 2002; Lake, 2007; Lake & Friese, 2006; Liou & Cheng,

2010).
Nurses were also asked to report if any activities, from a set of
12 necessary care activities
were left undone during their last shift due to lack of time. The
activities included: (1)
adequate surveillance (directly observation/monitoring) of
patients, (2) teaching patients or
family, (3) preparing patients and families for discharge, (4)
providing comfort/talk with
patients, (5) adequately document nursing care, (6)
administering medications on time, (7)
skin care, (8) oral hygiene, (9) pain management, (10) treatment
and procedures, (11)
coordinating care and (12) developing or updating nursing care
plans. Construct validity of
this measure has been demonstrated in that scores have been
found to be associated in the
theoretically expected direction with RN staffing, quality of
care, and frequency of adverse
events in hospitals (Sochalski, 2001; 2004). Nurses’ reports of
the work environment and
missed nursing care, although collected at the individual nurse
level, are customarily
aggregated to produce a hospital-level metric as was done in

this study (Aiken, Cimiotti,
Sloane, Neff & Flynn, 2011).
Hospitals—EHR adoption data were obtained from the 2006
HIMSS Analytic Database.
HIMSS annually surveys a sample of U.S. nonfederal acute care
hospitals including
independent hospitals and those within a healthcare delivery
system. Providing data on more
than 5,100 hospitals, the HIMSS database is the most
comprehensive collection of
information technology data and has been used in previous
research on health IT (Kazley &
Ozcan, 2008; McCullough, Casey, Moscovice, & Prasad, 2010).
EHR adoption was
2016 August 26.
A
u
th
o
r M
a
n
u

scrip
t
A
u
th
o
r M
a
n
u
scrip
t
A
u
th
o
r M
a
n
u
scrip
t
A
u
th

o
r M
a
n
u
scrip
t
operationally defined as a hospital’s total cumulative score on
the Electronic Medical
Record Adoption Model scale (EMRAM) ranging between 0–4
(Table 1) where a higher
score indicates more advanced adoption of technology (HIMSS,
2008).
Control variables—The potential confounding variables
hypothesized to affect patient
outcomes included nurse staffing levels, nurse education,
hospital size, teaching status, high
technology status (defined as facilities with open-heart surgery,
major organ transplant
services, or both), and geographic categories (Aiken, Clarke &
Sloane, 2002; Aiken, Clarke,
Sloane, Sochalski & Silber, 2002; Appari, Johnson & Anthony,
2013; Elnahal, Joynt, Bristol
& Jha, 2011; Himmelstein, Wright & Woolhandler, 2010).

These data were derived from the
New Jersey Nurse Survey and originally obtained from the
American Hospital Association
Annual Survey. Patient risk-adjusted covariates were extracted
from the State Inpatient
Databases (SIDS) and include age, sex, race, insurance type and
ICD9-CM primary and
secondary diagnosis codes. The AHRQ risk adjustment method,
based on the Elixhauser
method, was employed and includes a comprehensive set of 30
comorbidities (Elixhauser,
Steiner, Harris & Coffey, 1998).
Data management and analysis—Prior to analysis, all datasets
were aggregated to the
hospital level. The final analytic sample included 854,258
patients and 7,679 nurses in 70
New Jersey hospitals. The relationship between potentially
confounding variables (control
variables) and their respective dependent variables were
examined using bivariate Pearson or
Spearman correlations, as determined by the Shapiro-Wilk test
of normality. Those showing
significant relationships (p < .05) were retained for inclusion in
the multivariable models as
control variables. The presence of multicollinearity was

identified by variance inflation
factor diagnostics (VIF >10). In such cases, only one variable
was included from the set of
correlated variables. Following these steps, the number of
variables retained in all
multivariable models was based on rules for regression
modeling (Harrell, 2001). Because
nurse, patient, and EHR data were clustered in hospitals,
appropriate statistical methods for
analyzing clustered data were employed (Wears, 2002).
Data were assessed for outliers and missing data. Data on the
key variables EMRAM,
nursing practice environment, missed nursing care, PSIs and
PLOS were available for 70
New Jersey hospitals. In 2006, 51 hospitals submitted
readmissions data, and two were
excluded from the readmission models due to incomplete data.
All 41 hospitals that
submitted HCAHPS data were included in the patient
satisfaction models. Ordinary least
squares and multiple regression models were used in the
analytic approach to estimate the
effects of EHR adoption stage on the delivery of nursing care
and patient outcomes and

controlled for characteristics of patients, nurses, and hospitals.
Simple unadjusted OLS
regression models were used, followed by adjusted models
using the retained control
variables identified by the steps previously described. These
models were assessed for
heteroskedasticity, run with robust standard errors (Huber-
White) if indicated, and residuals
were examined. Data were analyzed using STATA/MP 12.1
software. The level of
significance for testing was set at .05 and standardized
coefficients (β) are reported.
2016 August 26.
A
u
th
o
r M
a
n
u
scrip

a
n
u
scrip
t
Results
Patient, Nurse, and Hospital Characteristics
The final analytic sample and unit of analysis was 70 New
Jersey hospitals; data was
available from 854,258 patients and 7,679 nurses. The majority
of study patients were male
(59 percent), white (66 percent), and insured (83 percent) with
an average age of 59 years.
The most common comorbidities were chronic pulmonary
disease (15.5%), uncomplicated
diabetes (15.4 percent) and fluid and electrolyte disorders (15.2
percent). Slightly more than
half of the nurses held specialty certification (52 percent),
nearly half earned a BSN degree
(44%), and they cared for, on average, six patients per shift.
The majority of the 70 hospitals
included in this study were below EMRAM Stage 3 (63

percent), had 250 beds or more (52
percent), were not high technology (75 percent), and were either
non-teaching (46 percent)
or minor teaching hospitals (43 percent).
EHR Adoption Stage and Adverse Outcomes
The unadjusted effect of testing the relationship between EHR
adoption stage and the patient
outcome of readmission within seven days was significant (R2 =
.09, F (1, 47) = 4.70, p = .
03). Bivariate correlations did not significantly identify any
potential confounders that
required additional testing using adjusted models. However, the
Breusch-Pagan test
demonstrated evidence of heteroskedasticity (p < .01); thus, the
model was conservatively
estimated with robust standard errors and was not significant (p
= .06) (Table 2).
The unadjusted effect of testing the relationship between EHR
and PLOS was not significant
(R2 = .003, F (1, 68) = 0.21, p = .65). However, when adjusting
for control correlates of
PLOS (patient comorbidity, patient age, nurse staffing, and
hospital technology status), the
adjusted effect was significant (R2 = .462, F (4, 63) = 6.54, p <
.01), with EHR adoption
stage being a significant contributor (β = −.21, p = .03). For

every standard deviation unit
(SD = 1.39) increase in EHR adoption stage, PLOS decreased by
.21 standard deviation
units (SD = 0.05). EHR adoption stage was not a significant
predictor of other adverse
outcomes.
EHR Adoption Stage and Patient Satisfaction
There was a statistically significant relationship between EHR
adoption level and patient
satisfaction in acute care hospitals in one of the ten patient
satisfaction outcomes: “yes,
given discharge information.” The model included the control
variables of patient race,
being insured, and nurse staffing. Findings indicate that the
overall model was significant
(R2 = .04, F (4, 36) = 7.56, p < .01), with EHR adoption stage
significantly contributing to
this outcome (β = −.31, p = .02). However, this finding was in
the inverse direction and
indicated that higher EHR adoption stages were predictive of
lower percentages of patients
who responded “yes, given discharge information” (Table 3).
As the primary purpose of the study was to examine the effect
of EHR adoption levels on
patient outcomes, and because the evident and strong
relationship among the nursing

practice environment and missed nursing care and outcomes
might confound this
relationship, these variables were controlled to isolate the effect
of EHR. Models were
2016 August 26.
A
u
th
o
r M
a
n
u
scrip
t
A
u
th
o
r M
a
n
u
scrip

t
A
u
th
o
r M
a
n
u
scrip
t
A
u
th
o
r M
a
n
u
scrip
t
constructed to examine with greater precision whether advanced
EHR technology is

positively related to patient satisfaction outcomes by
controlling for the statistically
significant effects of the nursing practice environment and
missed nursing care. The nursing
practice environment dimension of staffing and resource
adequacy was specifically tested
secondary to the evident relationship between this dimension of
the nursing work
environment and patient satisfaction. Results indicate that
higher EHR adoption stages were
predictive of one satisfaction outcome, lower percentages of
patients who respond “yes,
given discharge information” (β = −.27, p < .05) when the
strong relationships among the
nursing practice environment and missed care and satisfaction
were held constant (Table 4).
Discussion
Findings from this study suggest that an inverse relationship
exists between EHR adoption
stage and the patient outcomes of PLOS and readmissions. The
findings of this study did not
suggest that increased EHR adoption stages are related to
decreased adverse outcomes of
PSIs outcomes or increased patient satisfaction. Additional
analysis was conducted to

examine if the strong relationships among the nursing work
environment, missed nursing
care and patient satisfaction were confounding the effect of
EHR on patient satisfaction
outcomes. Notably, in the final adjusted models only one
satisfaction outcome, the patient
response of “being given discharge information,” reached the
level of statistical significance.
This relationship, however, was in the opposite direction of that
theorized.
These important relationships have not been tested in prior
studies, and as such these novel
findings may indicate that at the EMRAM stages 0–4 of EHR
adoption, the patient
satisfaction benefit is tempered by staffing and resource
adequacy. There is little to no extant
theoretical or empirical support for this unexpected finding.
One possible explanation is that
the relationship between EHR and satisfaction may be
moderated by insufficient resources,
which in the presence of new technology has the effect of
reducing workflow and time
efficiencies (Huber, 1990; Poissant, Pereira, Tamblyn &
Kawasumi, 2005).

Methodologically, it is unknown if achievement of these EHR
adoption stages is new in
these settings; consequently the impact on nursing processes of
care and workflow is
unknown. In order to optimize the positive effect of EHR on
patient outcomes,
organizational strategies and resources must be committed to
ease and guide the transition to
this technology (Huber, 1990; Walker et al., 2008). Although
this study accounted for
organizational factors that may serve as indicators of available
resources (teaching status,
hospital size, geographic location, and technology status), the
comprehensive nature and
extent of the organizational strategy to implement EHR
technology was unknown.
Importantly, these findings suggest that sufficient staffing and
resources, as rated by the
nurses, are essential for advanced EHR adoption and patient
reported outcomes of
satisfaction. These findings are consistent with extant literature
and may also suggest that
patient benefits of advanced technology will only be realized in
context of sufficient human

resources (Furukawa, Raghu & Shao, 2011; Jha et al., 2009;
Kazley & Ozcan, 2007; Walker
et al., 2008). These novel findings warrant further investigation.
2016 August 26.
A
u
th
o
r M
a
n
u
scrip
t
A
u
th
o
r M
a
n
u
scrip

t
A
u
th
o
r M
a
n
u
scrip
t
A
u
th
o
r M
a
n
u
scrip
t
Implications
The implications of the knowledge generated by this study are

significant for patients,
nurses, administrators, and policy makers, particularly in
context of the shifting healthcare
delivery and nursing practice landscape. Across the U.S.,
hospitals and nurses have made
significant efforts to achieve higher MU stages. Despite marked
progress, nearly 30 percent
of U.S. hospitals that submitted data to HIMSS in 2015 remain
at or below EMRAM middle
stage (3) of adoption, equivalent to Meaningful Use Stage 1
objectives (Appari et al., 2013;
HIMSS, 2015). In this 2006 study of New Jersey hospitals, 37%
had achieved EMRAM
stage 3; one-third of these achieving the next cumulative level
of EMRAM stage 4. Thus,
findings from this study are highly relevant and timely as we
are in a period of rapidly
accelerating advancement and adoption.
These EMRAM stages correspond to MU Stages 1 and 2; thus,
there are significant and
timely implications of these study findings for both current and
future nursing practice and
hospital payment. Though the data indicate progress as the
majority of hospitals in the New

Jersey 2006 baseline data were below EMRAM stage 3 while
more recent 2015 national
data indicates approximately 30 percent remain at or below
stage 3, a possible critical point
may not be reached to fully demonstrate the potential impact of
EHR, not only in New
Jersey but across the United States.
Importantly, achieving EMRAM stage 3, including nursing
documentation which is the
primary mechanism of electronic communication, is essential
for safe transitions of care. As
such, outcomes that are more sensitive to good communication
and care transitions, such as
readmissions, PLOS, and patient reports of “yes, given
discharge information,” may
conceivably be early indicators of the impact of advanced EHR
adoption. Implications for
hospital administrators and nursing practice are evident and
congruent with best practice
guidelines, suggesting the engagement of nursing staff in the
development, use, and ongoing
feedback of documentation systems and allocation of resources
for ongoing training and

systems evaluation and improvement are necessary to optimize
the system (Blavin, Ramos,
Shah & Devers, 2013). These findings suggest that multi-level
interventions are required for
improving patient care and outcomes. This study demonstrates
that EHR adoption does have
a positive, adjusted effect on PLOS, and it is theoretically
plausible that, as features of
advanced technology become embedded in hospitals and other
healthcare organizations,
positive benefits may extend to additional patient outcomes and
institutional settings across
the continuum of care (Huber, 1990; Powell-Cope et al., 2008).
In summary, the dual demands of the legislative provisions to
implement health IT and
improve quality outcomes may exacerbate the difficult decisions
hospital administrators
need to make regarding allocation of valuable resources. There
is a strong financial incentive
to integrate technology into the healthcare work environment
and sound theoretical rationale
to believe that through enhanced communication, improved data
management and better
transitions of care that EHRs will benefit patients and providers

alike. Broader implications
of these study findings for administrators suggest organizations
that have strong
fundamentals of quality nursing care in place may realize
improved patient satisfaction
outcomes that translate into real dollars through the VBP
program. Additionally,
implications of this study for administrators and policy makers
suggest that meeting the
2016 August 26.
A
u
th
o
r M
a
n
u
scrip
t
A
u
th

t
demands of the ARRA and ACA may not be mutually exclusive.
Rather, in an iterative
manner, a supportive nursing work environment that is
adequately staffed and resourced may
improve patient satisfaction, leading to better organizational
financial health. These fiscal
resources can in turn be used by organizations to continue
advancing EHR adoption, EHR
implementation, and the transformation of health care in the
U.S.
Limitations
Despite these novel findings and important implications, several
limitations of this study
should be acknowledged. This study was cross-sectional and as
such correlations,
relationships, and associations between variables of interest
were examined, but causality
could not be ascertained. The precision of the PSI data was
dependent on the documentation
in the record and coding applied by trained medical coders;
thus, discrepancies in data and

accuracy could have existed at the hospital level (AHRQ, 2004,
2010; Zhan & Miller, 2003).
Methodologically, patient responses cannot be linked
temporally to specific hospital EHR
adoption timelines. This study was designed to mitigate this
possible limitation by including
data from HCAHPS release date of March 2008, which captures
data from July 2006
through June 2007. However, it remains unknown if this
negative finding may in part reflect
early EHR adoption and the attendant human factors and
operational challenges thought to
effect the use of this technology and subsequent proposed
benefits, by nurses and patients
alike (Powell-Cope et al., 2008). EHR data were obtained from
HIMSS and patient
satisfaction from HCAHPS; both are voluntary reporting
systems, and as such these data
were subject to self-selection bias. Finally, analysis at the
hospital level limits sample size,
and though the power analysis indicated the sample size was
sufficient and significant
effects identified, the sample size of hospitals in the patient
satisfaction models may have

been a limitation.
Conclusions
In this study of 70 acute care hospitals, higher levels of EHR
adoption were significantly and
independently associated with fewer incidences of PLOS and
partially associated with lower
rates of seven-day re-hospitalization. This is the first study that
examined this relationship
between EHR adoption stage and PLOS, thus extending this
knowledge. These findings
support the promising role of EHRs in improving patient
outcomes. Importantly, however,
findings also indicate that a supportive nursing practice
environment, including the domain
of adequacy of nursing staff and resources, is significantly and
independently associated
with lower levels of missed nursing care and higher levels of
patient satisfaction, with and
without adjusting for EHR levels. Thus, findings from this study
indicate that a multi-faceted
approach that includes technology, such as EHR, as well as
system-wide nursing supports
are needed to improve patient care outcomes in acute care

hospitals. In summary, these
findings add to a growing body of knowledge in nursing
research that identifies modifiable
technologic and nursing-focused system factors that are critical
to improving patient care
and outcomes.
2016 August 26.
A
u
th
o
r M
a
n
u
scrip
t
A
u
th
o
r M

a
n
u
scrip
t
A
u
th
o
r M
a
n
u
scrip
t
A
u
th
o
r M
a
n
u
scrip
t

Biographies
Amanda Hessels, PhD, MPH, RN, CIC, CPHQ is a Postdoctoral
Research Fellow at the
Center for Interdisciplinary Research to Prevent Infections
(CIRI), Columbia University,
School of Nursing and Nurse Scientist at Meridian Health in
New Jersey. Her program of
funded research examines the relationships among
organizational factors, processes of
nursing care and adverse patient outcomes in acute care
settings. She has used both primary
survey methodology and existing large-scale datasets to advance
knowledge of these
relationships.
Linda Flynn, PhD, RN, FAAN
Dr. Linda Flynn is a Professor and the Associate Dean of
Academic Programs at the
University of Colorado College Of Nursing. Her program of
funded research, using large-
scale survey methodology, focuses on the impact of system
factors on nurse, patient, and
faculty outcomes across a variety of settings. She is the author
of multiple peer-reviewed

publications and her research has influenced policy decisions at
the state and national levels.
Jeannie P. Cimiotti, PhD, RN, FAAN
Dr. Cimiotti is Associate Professor and the Dorothy M. Smith
Endowed Chair at the
University of Florida College Of Nursing. She is internationally
known for the development
and implementation of health care surveys and managing large
health care datasets. Dr.
Cimiotti’s research examines the organizational features of
hospitals that lead to poor patient
care outcomes.
Suzanne Bakken, RN, PhD, FAAN, FACMI is the Alumni
Professor of Nursing and
Professor of Biomedical Informatics at Columbia University.
She directs the Center for
Evidence-based Practice in the Underserved and the Reducing
Health Disparities Through
Informatics pre- and post-doctoral training program. She has
been conducting federally-
funded informatics research for more than 25 years. Dr. Bakken
currently serves as the
President of the American College of Medical Informatics.

Robyn Gershon, MT (ASCP), MHS, DrPH is a Professor of
Epidemiology and Biostatistics
and Core Faculty in the Philip R. Lee Institute for Health Policy
Studies in the School of
Medicine at University of California, San Francisco. She is also
an Adjunct Professor at the
School of Nursing, UCSF, and Adjunct Professor at University
of California, Berkeley. She
is a nationally recognized researcher in the area of occupational
and environmental health,
specializing in public health disaster preparedness and response
and in the health care
workplace and workforce.
References
Agency for Healthcare Research and Quality (AHRQ). AHRQ
Quality Indicators – Guide to Patient
Safety Indicators. Rockville, MD: Agency for Healthcare
Research and Quality; 2004. AHRQ Pub.
03-R2032003. Version 2.1, Revision 2
Agency for Healthcare Research and Quality (AHRQ). AHRQ
quality indicators: Composite measures
user guide for the patient safety indicators (PSI). Department of
Health and Human Services

2016 August 26.
A
u
th
o
r M
a
n
u
scrip
t
A
u
th
o
r M
a
n
u
scrip
t
A
u
th
o

r M
a
n
u
scrip
t
A
u
th
o
r M
a
n
u
scrip
t
Agency for Healthcare Research and Quality; 2010. Version 4.2
(September 2010)Retrieved from
http://www.qualityindicators.ahrq.gov
Aiken LH, Cimiotti JP, Sloane DM, Flynn L, Neff DF. Effects
of nurse staffing and nurse education on
patient deaths in hospitals with different nurse work
environments. Medical Care. 2011; 49(12):
1047–53. [PubMed: 21945978]
Aiken LH, Clarke SP, Sloane DM. Hospital staffing,

organizational, and quality of care: Cross-national
findings. Nursing Outlook. 2002; 50:187–194. DOI:
10.1067/mno.2002.126696 [PubMed:
12386653]
Aiken LH, Clarke SP, Sloane DM, Sochalski J, Silber JH.
Hospital nurse staffing and patient mortality,
nurse burnout and job dissatisfaction. Journal of the American
Medical Association. 2002; 288(16):
1987–1993. [PubMed: 12387650]
Appari A, Johnson E, Anthony DL. Meaningful use of electronic
health record systems and process
quality of care: Evidence from a panel analysis of U.S. acute-
care hospitals. Health Services
Research. 2013; 48(2):354–375. [PubMed: 22816527]
Blavin, F.; Ramos, C.; Shah, A.; Devers, K. Final report:
Lessons from the literature on electronic
health record implementation. 2013. Retrieved from
https://www.healthit.gov/sites/default/files/
hit_lessons_learned_lit_review_final_08-01-2013.pdf
Centers for Medicare & Medicaid Services (CMS). EHR
Incentive program: Active Registrations.
2012a. Retrieved from https://www.cms.gov/Regulations-and-
Guidance/Legislation/
EHRIncentivePrograms/Downloads/
Oct_IncentiveProgramPayment_Registration_SummaryReport.p
df
Centers for Medicare & Medicaid Services (CMS). Hospital care
quality information from the
consumer perspective. 2012b. Retrieved from
http://www.hcahpsonline.org/home.aspx

Centers for Medicare & Medicaid Services (CMS). Hospital
value-based purchasing program. 2013.
Retrieved from http://www.cms.gov/Medicare/Quality-
Initiatives-Patient-Assessment-Instruments/
Hospital-Value-Based-Purchasing/
DesRoches CM, Miralles P, Buerhaus P, Hess R, Donelan K.
Health information technology in the
workplace. The Journal of Nursing Administration. 2011;
41(9):357–364. [PubMed: 21881441]
Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity
measures for use with administrative
data. Medical Care. 1998; 36(1):8–27. [PubMed: 9431328]
Elnahal SM, Joynt KE, Bristol SJ, Jha AK. Electronic health
record functions differ between the best
and worst hospitals. American Journal of Managed Care. 2011;
17(4):e121–e147. [PubMed:
21774097]
Furukawa MF, Raghu TS, Shao BM. Electronic medical records,
nurse staffing, and nurse-sensitive
patient outcomes: evidence from the national database of
Nursing Quality Indicators. Medical Care
Research and Review. 2011; 68(3):311–331. DOI:
10.1177/1077558710384877 [PubMed:
21075750]
Gajewski BJ, Boyle DK, Miller PA, et al. A multilevel
confirmatory factor analysis of the practice
environment scale. Nurs Res. 2010; 59(2):147–153. [PubMed:
20216017]
Garets, D.; Davis, M. Electronic medical records vs. electronic
health records: Yes, there is a

difference. Chicago, IL: HIMSS Analytics; 2006. Retrieved
from https://app.himssanalytics.org/
docs/WP_EMR_EHR.pdf
Harrell, FE. Regression modeling strategies: With applications
to linear models, logistic regression,
and survival analysis. New York, N.Y: Springer; 2001.
HCUP State Inpatient Databases (SID). Healthcare Cost and
Utilization Project (HCUP). New Jersey,
2006–2007. Agency for Healthcare Research and Quality;
Rockville, MD: 2012a. Retrieved from
https://www.hcup-us.ahrq.gov/sidoverview.jsp
HCUP. SID Description of Data Elements All States, Healthcare
Cost and Utilization Project (HCUP).
Agency for Healthcare Research and Quality; Rockville, MD:
2012b. Retrieved from https://
www.hcup-us.ahrq.gov/db/state/siddbdocumentation.jsp
Healthcare Information and Management Systems Society
(HIMSS). EMR Adoption Model. 2015.
Retrieved from http://www.himssanalytics.org/provider-
solutions#block-himss-general-himss-
prov-sol-emram
2016 August 26.
A
u
th
o

r M
a
n
u
scrip
t
A
u
th
o
r M
a
n
u
scrip
t
A
u
th
o
r M
a
n
u
scrip
t

A
u
th
o
r M
a
n
u
scrip
t
http://www.qualityindicators.ahrq.gov
https://www.healthit.gov/sites/default/files/hit_lessons_learned_
lit_review_final_08-01-2013.pdf
https://www.healthit.gov/sites/default/files/hit_lessons_learned_
lit_review_final_08-01-2013.pdf
https://www.cms.gov/Regulations-and-
Guidance/Legislation/EHRIncentivePrograms/Downloads/Oct_I
ncentiveProgramPayment_Registration_SummaryReport.pdf
http://www.hcahpsonline.org/home.aspx
http://www.cms.gov/Medicare/Quality-Initiatives-Patient-
Assessment-Instruments/Hospital-Value-Based-Purchasing/
http://www.cms.gov/Medicare/Quality-Initiatives-Patient-
Assessment-Instruments/Hospital-Value-Based-Purchasing/
https://app.himssanalytics.org/docs/WP_EMR_EHR.pdf
https://app.himssanalytics.org/docs/WP_EMR_EHR.pdf

https://www.hcup-us.ahrq.gov/sidoverview.jsp
https://www.hcup-us.ahrq.gov/db/state/siddbdocumentation.jsp
https://www.hcup-us.ahrq.gov/db/state/siddbdocumentation.jsp
http://www.himssanalytics.org/provider-solutions#block-himss-
general-himss-prov-sol-emram
http://www.himssanalytics.org/provider-solutions#block-himss-
general-himss-prov-sol-emram
Himmelstein DU, Wright A, Woolhandler S. Hospital computing
and the costs and quality of care: A
national study. American Journal of Medicine. 2010; 123:40–46.
DOI: 10.1016/j.amjmed.
2009.09.004 [PubMed: 19939343]
Huber GP. A theory of the effects of advanced information
technologies on organizational design,
intelligence, and decision making. Academy of Management
Review. 1990; 15(1):47–71.
Institute of Medicine (IOM). To Err is Human: Building a Safer
Health System. Washington, DC:
National Academies Press; 2001.
Institute of Medicine (IOM). Comparative Effectiveness
Research. Washington, DC: National
Academies Press; 2009.
Jha AK, DesRoches CM, Campbell EG, Donelan K, Rao SR,
Ferris TG, Blumenthal D. Use of
electronic health records in U.S. hospitals. New England
Journal of Medicine. 2009; 350(16):
1628–1638. [PubMed: 19321858]
Jha AK, Orav J, Zheng J, Epstein AM. Patient’s perceptions of
hospital care in the United States. New

England Journal of Medicine. 2008; 359(18):1921–1931.
[PubMed: 18971493]
Kazley AS, Diana ML, Ford EW, Menachemi N. Is electronic
health record use associated with patient
satisfaction in hospitals? Health Care Management Review.
2012; 37(1):23–30. DOI: 10.1097/
HMR.0b0113e3182307bd3 [PubMed: 21918464]
Kazley AS, Ozcan YA. Does hospital electronic medical record
use increase health care quality? An
examination of three clinical conditions. Medical Care Research
Review. 2008; 65(4):496–513.
[PubMed: 18276963]
Kazley AS, Ozcan YA. Organizational and environmental
determinants of hospital EMR adoption: A
National Study. Journal of Medical Systems. 2007; 31:375–384.
DOI: 10.1007/s10916-007-9079-7
[PubMed: 17918691]
Kutney-Lee A, Kelly D. The effect of hospital electronic health
record adoption on nurse-assessed
quality of care and patient safety. Journal of Nursing
Administration. 2011; 41(11):466–472.
[PubMed: 22033316]
Kutney-Lee A, McHugh MD, Sloane DM, Cimiotti JP, Flynn L,
Neff DF, Aiken AH. Nursing: A key
to patient satisfaction. Health Affairs. 2009; 28(4):w669–w677.
DOI: 10.1377/hltaff.28.4w669
[PubMed: 19525287]
Lake ET. Development of the Practice Environment Scale of the
Nursing Work Index. Research in
Nursing & Health. 2002; 25(3):176–186. [PubMed: 12015780]

TpEEDabcdaARR2.docx

Recommended

Recommended

More Related Content

Similar to TpEEDabcdaARR2.docx

Similar to TpEEDabcdaARR2.docx (20)

More from turveycharlyn

More from turveycharlyn (20)

Recently uploaded

Recently uploaded (20)

TpEEDabcdaARR2.docx