This document summarizes Karen Monsen's presentation on using electronic health record data to bring the voice of nursing practice into policy and research. Some key points: 1. EHR data from routine nursing documentation and assessments can provide a valuable source of "practice-based evidence" to understand what expert nurses know and do in real-world complex patient situations. 2. "Big data" research using large nursing datasets can examine important healthcare quality questions and identify hidden patterns by letting the data speak. Examples of studies were presented that used nursing EHR data. 3. This practice-based evidence from nurses' daily work with patients is important to complement traditional evidence-based research models and ensure nursing expertise and the patient

1. Big Data and
Practice-based Evidence:
How EHR data is bringing the voice of nursing
practice into policy and research
NNIC 2015
Karen A. Monsen

2. Lovely to be here!

3. Steeped in EBP
• Joanna Briggs – Cochrane – NZ guidelines
Hendry, C. (2011). The New Zealand Institute of Community Health Care: REPORT TO THE MINISTRY OF HEALTH ON
THE IMPROVING NURSING UTILISATION OF EVIDENCE TO INFORM CLINICAL PRACTICE SERVICES PROJECT.
Available at: https://www.health.govt.nz/system/files/documents/publications/cdhb_report.docx
http://www.dilmah.co.nz/wp-content/uploads/2014/06/dimah-ranges.jpg

4. After 20 Years of EBP
• Scholars are concerned.
• Sound clinical judgment is devalued
• Mark Tonelli (1999)
• Evidence is limited
• GoodyearSmith (2012). What is evidence-based practice and how do we get there? The Journal of Primary Healthcare, 4, 2, 90-91.
• Evidence is biased
• Greenhalgh, T, Howick J, Maskrey N.Evidence based medicine: a movement in crisis? 2014 BMJ 348 :g3725

5. The Real World
• The real world with all of its complexities and nuances is not
controlled

6. Practice-based Evidence is Needed
• Beyond algorithms, how do we provide the best personalized care for
individuals in unique situations?

7. Gap in Commentary
• Assumption that there is no data source that underlies and enables
study of practice-based evidence
• This assumption is false
• Nurses know what to do

8. What to do?

9. Learner Objectives
• Describe the role of practice-based evidence as a necessary
component of nursing knowledge
• Discuss possible sources of information that constitute practice-based
evidence
• Provide examples of Big Data research using large nursing data sets

10. Practice
• What expert nurses know and do every day to ensure wellbeing and
safety of patients
• in the real world
• for unique patients and situations

11. Practice-based Evidence
• “How does adding X intervention alter the complex personalized
system of patient Y before me?”
Swisher, A. K. (2010). Practice-Based Evidence. Cardiopulmonary Physical Therapy Journal, 21(2), 4.

12. Practice-based Data
• Data from nursing assessment and documentation that is
part of routine nursing care is an important source of
practice-based evidence

13. Big Data
• Large datasets of structured or unstructured information that may
require new approaches for analysis
• Let the data speak
Garcia, A., L. (2015). How big data can improve health care. American Nurse Today. Available at:
http://www.americannursetoday.com/how-big-data-can-improve-health-care/

14. Big Data Research in Nursing
• Traditional and new methods for big data
• Using large data sets to examine important healthcare quality questions
• Looking for hidden patterns in the data
• Hypotheses generating vs. hypothesis testing
• New voices for nursing and patients: Practice-based evidence

15. Big Data Studies in Nursing
• All of the studies I’m about to share are examples of the rigorous
study of data – from practicing nurses – powerful observational
datasets that speak for nursing and for patients alike.

16. Outcome Variability:
Nurses and Interventions
• Using a logistical mixed-effects model with nursing data
to evaluate outcome variability
This research is partially supported by the National Science Foundation under grant # SES-0851705, and by
the Omaha System Partnership. Monsen, K. A., Chatterjee, S. B., Timm, J. E., Poulsen, J. K., & McNaughton,
D. B. (in review). Public health nurse, client, and intervention factors contribute to variability in health
literacy outcomes for disadvantaged families.
• Client (50%)
• Problem (17%)
• Nurse (17%)
• Intervention (17%)
Age was significantly positively associated with knowledge benchmark attainment

17. Implications
• Research
• We need to incorporate the ‘nurse’ as an important part of the research
model
• Policy
• To ensure optimal outcomes, we need the best nurses
• Best fit with assigned patients
• Support expertise
• Ensure wellbeing

18. Mothers with Mental Health Problems
Monsen, K. A. et al., 2014
Method: Data Visualization
Each image (sunburst) was created in
d3 from public health nursing
assessment data for a single patient.
Data were generated by use of the
Omaha System signs and symptoms
and Problem Rating Scale for
Outcomes
Key:
•Colors = problems
•Shading = risk
•Rings = Knowledge, Behavior, and Status
•Tabs = signs/symptoms
Documentation patterns suggest a
comprehensive, holistic nursing
assessment.
Kim et al. found that the presence of
mental health signs and symptom
tends to be associated with more
diagnostic problems and worse patient
condition
Kim, E., Monsen, K. A., Pieczkiewicz, D. S. (2013). Visualization of Omaha System data enables
data-driven analysis of outcomes. American Medical Informatics Association Annual Meeting,
Washington D. C. Funded by a gift from Jeanne A. and Henry E. Brandt.

19. Implications
• Research
• Visualization methods can help identify individuals with similar patterns in
complex multidimensional data
• Policy
• It is critical to identify and serve the individuals who most need our help

20. • Method: Generalized Estimating Equations for cohort
comparison
• Results: Mothers with intellectual disabilities have
twice as many problems as mothers without
intellectual disabilities
• Receive more public health nursing service
• Twice as many encounters and interventions
• Show improvement in all areas
• Do not reach the desired health literacy benchmark in
Caretaking/parenting
Mothers with Intellectual Disabilities
Monsen, K. A., Sanders, A. N., Yu, F., Radosevich, D. M, & Geppert, J. S. (2011). Family
home visiting outcomes for mothers with and without intellectual disabilities. Journal of
Intellectual Disabilities Research, 55(5), 484-499. doi:10.1111/j.1365-2788.2011.01402.x

21. Implications
• Research
• Large datasets will enable research into situations that are relatively rare and
otherwise difficult to study
• Policy
• Extra time and effort (and therefore funding) is needed to produce the
positive outcomes we desire for mothers with intellectual disabilities

22. Studies of Intervention/Outcome
Patterns
• Problem-specific intervention patterns
• Individual-specific intervention patterns
• Population-specific intervention patterns
• Home care patients
• Mothers with low health literacy

23. Using Kaplan-Meier Curves to Detect
Problem Stabilization
This research was supported by the National Institute of Nursing Research (Grant #P20 NR008992;
Center for Health Trajectory Research). The content is solely the responsibility of the authors and does
not necessarily represent the official views of the National Institute of Nursing Research or the National
Institutes of Health. Monsen, K. A., McNaughton, D. B., Savik, K., & Farri, O. (2011). Problem
stabilization: A metric for problem improvement in home visiting clients. Applied Clinical Informatics, 2,
437-446 http://dx.doi.org/10.4338/ACI-2011-06-RA-0038

24. Using Data Visualization to Detect Nursing
Intervention Patterns
Each image (streamgraph) was
created in d3 from longitudinal
public health nursing intervention
data for a single patient. Data
were generated by use of the
Omaha System in clinical
documentation
Key:
•Colors = problems
•Shading = actions (categories)
•Height = frequency
•Point on x-axis = one month
From 403 images, 29 distinct
patterns were identified and
validated by clinical experts
Documentation patterns suggest
both a unique nurse style and
consistent patient-specific
intervention tailoring
Monsen, K.A., Hattori, K., Kim, E., Pieczkiewicz, D. (In review). Using visualization methods to
discover nurse-specific patterns in nursing intervention data.
Streamgraph development funded by a gift from Jeanne A. and Henry E. Brandt.
Monsen, K. A. et al., 2014

25. COMPREHENSIVE WOUND CARE
BASIC
WOUND
CARE
Treatments &
procedures
Case
management
Surveillance
Monitoring
Teaching, guidance, &
counseling
Informing
Providing Therapy
Using Inductive and Deductive Approaches
to Create Overlapping Intervention Groups
Relationships between
four intervention
grouping/clustering
methods for
wound care.
Monsen, K. A., Westra, B. L., Yu, F., Ramadoss, V. K., & Kerr, M. J. (2009). Data management for intervention effectiveness research: Comparing deductive
and inductive approaches. Research in Nursing and Health, 32(6), 647-656. doi:10.1002/nur.20354

26. Home Care Interventions and Hospitalization
Outcomes
• Method: Logistic regression
• Results: Too little care may result in hospitalization
when patients have more intensive needs
• Frail elders are more likely to be hospitalized if they have
low frequencies of four skilled nursing intervention clusters
Monsen, K. A., Westra, B. L., Oancea, S. C., Yu, F., & Kerr, M. J. (2011). Linking home care
interventions and hospitalization outcomes for frail and non-frail elderly patients.
Research in Nursing and Health, 34(2), 160-168. doi:10.1002/nur.20426. NIHMS274649

27. Knowledge scores across problems over time
•Pre-intervention, patterns by race/ethnicity
•Post-intervention, patterns by problem
Health Literacy Outcomes
Benchmark = 3
Monsen, K. A., Areba, E. M., Radosevich, D. M., Brandt, J. K., Lytton, A. B., Kerr, M. J.,
Johnson, K. E., Farri, O, & Martin, K. S. (2012). Evaluating effects of public health nurse
home visiting on health literacy for immigrants and refugees using standardized nursing
terminology data. Proceedings of NI2012: 11th International Congress on Nursing
Informatics, 614..

28. Implications
• Research
• Nurses address multiple problems in different ways over time
• Future research should take into account and evaluate factors of timing,
specific problem, and individual needs
• Policy
• Encourage personalized interventions tailored to meet individual needs

29. Data Mining for Translation to Practice
(Chih-Lin Chi et al., 2015)

30. Problem: A small percentage of clients consume a
high percentage of service resources (80-20 rule)
20% patients use 70%
of intervention resource

31. Research Question 1:
Predict Intervention
Usage• Regardless of outcome, who will need more interventions?
For 75% threshold
Maximal accuracy ~ 74%
Maximal AUC ~ 77%
Prediction measured using receiver operating curves and area under the curve (AUC).
For 50% threshold
Maximal accuracy ~ 60%
Maximal AUC ~ 75%

32. Research Question 2:
Predict Responsiveness to Interventions
• Within the population, which individuals will be responsive to more
interventions for this problem, compared to those who are less
responsive?
More responsive Less responsive

33. Research Question 3:
Predict Personalized Nursing Intervention
• How to personalize care planning based on an individual’s
characteristics and what intervention patterns can be used to
help personalization?
• Intervention patterns typically used in Oral health
Teaching,
guidance, and
counseling
Treatments and
procedures
Case
management
Surveillance
Number of
clients
A 0.00% 0.00% 0.00% 100.00% 24
B 0.00% 10.00% 0.00% 90.00% 2
C 0.00% 20.00% 0.00% 80.00% 285
D 30.00% 0.00% 30.00% 40.00% 1
E 30.00% 10.00% 10.00% 50.00% 1
F 40.00% 0.00% 10.00% 50.00% 210
G 50.00% 0.00% 10.00% 40.00% 234
H 60.00% 0.00% 10.00% 30.00% 1

34. Research Question 4: Predict Relative
Improvement for Personalized Nursing
Intervention
• Relative improvement is 51% (compared to maximum
possible improvement for all clients)
• Choosing the right pattern can improve care
(efficiency and effectiveness)
51%

35. Next steps
• Nursing Big Data has been shown to enable the identification of
personalized algorithms to improve nursing care quality and efficiency
• Practice-based dissemination and implementation research proposals in
development and review

36. Implications
• Research
• It is becoming feasible to amass large quantities of data and create a pipeline
for research into personalized care
• Policy
• It is critical to support data sharing agreements and collaborations that
support use of clinical data for research

37. Nurses: Let the data speak!
• Thank you!
• Questions?
• mons0122@umn.edu