The document outlines an evaluation plan to assess the usability of a hospital's new electronic health record (EHR) system from the nurses' perspective. The evaluation will use qualitative methods, including interviews and observations of nurses, to identify any areas where the EHR's documentation screens are inconsistent with nursing workflow and clinical practice. The goals are to understand usability issues and offer solutions to better align the EHR with nursing work. Ethical considerations around participant consent and data confidentiality will be addressed. The analysis will provide a narrative to illustrate nurses' experiences using the EHR and how it impacts their work.

1. Evaluation Plan
Charles Donovan, BSN, RN
Walden University
NURS-6431-3: System Design, Planning & Evaluation
August 7, 2016

2. Agenda
• Project Overview
• PICO Question
• Evaluation Goals
• Literature Review
• Evaluation Methodology and Research Design
• Evaluation Tool
• Ethical Issues and Mitigation Plan
• Success Criteria
• Study Limitations and Opportunities

3. Project Overview
• Hospital implementation of a unified acute
and ambulatory eHR.
– All patient care documentation
– Interdisciplinary assessment forms
– Clinical decision support
– Medical notes

4. Project Overview (cont.)
• Goal: Improve the hospital’s performance in a
multitude of areas
– reduce the rate of patient safety events
– improve the quality of care
– deter sentinel events
– reduce patient readmissions
– impact spending
– fulfill the “Meaningful Use” requirements

5. PICO Question
“For nurses using the eHR to document patient
care, is the information contained in the data
entry screens presented in a manner consistent
with the order of patient assessment and clinical
workflow?”

6. Evaluation Goals
1. Identify areas of documentation that are
inconsistent with nursing practice/workflow
2. Identify policies that govern those areas of
practice
3. Offer solutions to align documentation
usability with nursing practice

7. Literature Review
1. To be successful, an eHR must be usable and easy to use.
– A poorly designed user interface makes it difficult for nurses to use the system,
hunting for places to document care, and can lead to errors (Usselman, Borycki, &
Kushniruk, 2015).
2. Nursing users must be included in the design of the system.
– Nurses understand clinical practice and the requirements of patient care; this is key to
developing system usability (Ammenwerth, Iller, & Mahler, 2006; Usselman, et al,
2015; Stevenson, & Nilsson, 2011; De Oliveira, & Peres, 2015; Rogers, Sockolow,
Bowles, Hand, & George, 2013).
3. The nurse understands the flow of documentation and the
logical progression of patient care and daily tasks.
– The ability to navigate the eHR, instinctively understand where to enter patient data,
and input data with minimal “clicks” and “page flips” increases compliance and
reduces the chance of error; defining ease of use (Ammenwerth, et al, 2006;
Usselman, et al, 2015; Stevenson, & Nilsson, 2011; De Oliveira, & Peres, 2015; Rogers,
et al, 2013).

8. Evaluation Methodology and Research Design
• Qualitative inquiry using
– One-on-one interview
– On-unit observation
– Nursing point of view
• Task – Technology Fit
• Sample includes:
– Expert and non-expert users
– skeptics

9. Evaluation Tool
• Open ended and unscripted Interview
– Allows openness and comfort
– Freely express their perceptions and experiences
– Interviewee can explore thoughts about how
documentation tool interacts with daily practice
• Three questions
– Workflow
– Tool interaction
– Proposals for change

10. Evaluation Tool (Cont.)
• Direct observation
– On unit
– Observation of workflow and tool interaction
– Interaction with user to record experiences

11. Ethical Issues and Mitigation Plan
• Consent
– Not required if purpose is internal review
– Let IRB make the final decision (Friedman, & Wyatt, 2005).
• Data Confidentiality
– Participant names must be coded
– Details of interviews held in confidence
– Tape recorded data must be secured and later
destroyed

12. Ethical Issues and Mitigation Plan (cont.)
• Include only data pertinent to study
• Clinically trained researchers have
professional obligation
– Must act in cases of suboptimal care
– Suggest a different course of action to prevent
patient harm

13. Success Criteria
• The qualitative analysis will return a story that illustrates how the users
see the resource, how it is used, and how it affects them and their daily
life. (Ash, et al, 2006)
• Presenting the analysis in relation to the nursing documentation workflow
will quickly align the audience with the topic, subjects, setting, and
context of the study.
• Presenting as a narrative that contains user quotes will draw the audience
into the world of the nurse documenting patient care in the eHR.
In this context, success will have the presented argument
accurately portray the nursing perception of the tool within the
work environment; whether it is positive or negative.

14. Study Limitations and Opportunities
• Limitations
– User participation
– Filtered comments
• Opportunities
– Follow-on studies related to documentation of
pain and post-intervention evaluation
– Barriers to home medication documentation
– Barriers to documentation of quality indicators

15. References
Ammenwerth, E., Iller, C., & Mahler, C. (2006). IT-adoption and the interaction of task, technology and individuals: a
fit framework and a case study. BMC Medical Informatics And Decision Making, 63.
Ash, J. S., Smith III, A. C., & Starvi, P. Z. (2006). Performing subjectivist studies in the qualitative traditions
responsive to users. In Evaluation Methods in Biomedical Informatics (pp. 267-300). Springer New York.
De Oliveira, N. B., & Peres, H. H. C. (2015). Evaluation of the functional performance and technical quality of an
Electronic Documentation System of the Nursing Process . Revista Latino-Americana de Enfermagem, 23(2), 242–
249. http://doi.org/10.1590/0104-1169.3562.2548
Ellis, T. J., & Levy, Y. (2008). Framework of problem-based research: A guide for novice researchers on the
development of a research-worthy problem. Informing Science: International Journal of an Emerging
Transdiscipline, 11, 17-33.
Friedman, C. P., & Wyatt, J. (2005). Evaluation methods in biomedical informatics. Springer Science & Business
Media.
Novak, L. L., Anders, S., Gadd, C. S., & Lorenzi, N. M. (2012). Mediation of adoption and use: a key strategy for
mitigating unintended consequences of health IT implementation. Journal of the American Medical Informatics
Association, 19(6), 1043-1049.

16. References (cont.)
QUALITATIVE RESEARCH DESIGNS. (n.d.). Retrieved from http://www.umsl.edu/~lindquists/qualdsgn.html
Rogers, M., Sockolow, P., Bowles, K., Hand, K., & George, J. (2013). Use of a human factors approach to uncover
informatics needs of nurses in documentation of care. International Journal Of Medical Informatics, 82(11), 1068-
1074.
Schardt, C., Adams, M. B., Owens, T., Keitz, S., & Fontelo, P. (2007). Utilization of the PICO framework to improve
searching PubMed for clinical questions. BMC medical informatics and decision making, 7(1), 1.
Stevenson, J., & Nilsson, G. (2011). Nurses' perceptions of an electronic patient record from a patient safety
perspective: a qualitative study. Journal Of Advanced Nursing, 68(3), 667-676.
Thabane, L., PhD., Thomas, T., M.D., Ye, C., B.Sc, & Paul, J., M.D. (2009). Posing the research question: Not so
simple. Canadian Journal of Anesthesia, 56(1), 71-9. doi:http://dx.doi.org/10.1007/s12630-008-9007-4
Usselman, E., Borycki, E. M., & Kushniruk, A. W. (2015). The evaluation of electronic perioperative nursing
documentation using a cognitive walkthrough approach. Studies In Health Technology And Informatics, 208331-
336.

17. Discussion
Please feel free to ask questions and interact
with the presenter and panel.