Centre for Health Systems and Safety
Research
Professor Johanna Westbrook
Centre for Health Systems & Safety Research
How ...
Centre for Health
Systems & Safety
Research
To produce world-class evidence which
informs policy and practice, focusing on...
Effectiveness
Impact of eMMS on medication error rates?
Efficiency
How do eMMS impact work of health
professionals?
Why sh...
Results – Post eMMS
Prescribing errors declined by > 50% from:
406 (95% CI 374-437) 185 (95% CI 160-210)
per 100 admissio...
Serious prescribing errors
Intervention wards – significant 44% (p=0.0002)
reduction in serious prescribing error rate
2...
MAE Post eMMS
Across all clinical error categories - a
significant reduction on the intervention wards
of 4.24 clinical e...
Change in serious medication
administration errors
Significant reduction in serious (ie
potential ADEs) MAEs on the
inter...
This eMMS
is going to
take too
long
Systems are
promoted for their
ability to improve
work efficiency
and safety
- Less time on administrative tasks
- More ti...
Research Evidence
Qualitative accounts - both significantly
hinders and assists work efficiency
Quantitative evidence is...
Use of an eMMS
Aim: To measure changes in how nurses
and doctors distributed their time across
work tasks pre and post eMMS
Changes in t...
Controlled Pre and Post Study
Control
Control
Intervention
Intervention
Control
Control
eMMS
eMMS
4 Wards
Pre Year 1
4 War...
Direct Observations Nurses & Doctors
70 nurses observed for 276.9 hours
59 doctors observed for 356.3 hours
Work Observation
Method By Activity
Timing -
Where?
With what?
With whom?
What task?
Interruptions
Results
Did nurses/doctors on the eMMS wards
spend more or less time on direct care,
medication tasks and professional
com...
No Significant
Differences
Nurses
Comparison of time distribution
control and eMMS Wards
%
Time
P
Value
Direct Care Contro...
Doctors
Comparison of time distribution
control and eMMS Wards
%
time
P
Value
Direct Care Control 19.7 0.08
eMMS 25.7
Medi...
Time Nurses Spent with Others
Baseline
33% of nurses time is spent with patients
50% spent with other nurses
5% with do...
Changes Following eMMS
 Nurses on the eMMS wards spent less time with
doctors (p=0.0001).
 4.2% less time than nurses on...
Time Doctors Spent with Others
Baseline
18% of doctors’ time is spent with patients
63% spent with other doctors
10% wi...
Changes Following eMMS
for Doctors
 Doctors on the eMMS wards spent more time
with other doctors (p=0.003).
 6% more tim...
Available at JAMIA.BMJ.Com
Why should we care about
study design?
Before and after
Versus
Controlled before and after
Controlled Pre and Post Study
Control
Control
Intervention
Intervention
Control
Control
eMMS
eMMS
4 Wards
Pre Year 1
4 War...
Differences in controlled versus
uncontrolled studies
Control
Control
Intervention
Intervention
Control
Control
eMMS
eMMS
...
How did nurses’s work change from
year 1 (before) to year 3 (after)?
Nurses now spending significantly more time on
medic...
Looking for the expected
and unexpected
Selective attention
The Gorilla Strikes Again! –
Drew and colleagues
presented 24 radiologists
with typical lung cancer
screening CT scans
20/24 radiologists (83%) missed the gorilla
25 non-trained reviewers all missed the
gorilla
“It’s important to be willin...
Context Matters
Understanding how systems impact
work will be influenced by context
Example:
Decision support
When, who an...
What impact does decision support
have during ward rounds?
Study – Teaching Hospital
58.5 hours direct observation of 14 teams on
ward rounds
48% of medication orders triggered al...
Senior clinicians during ward-rounds are
the decision-makers but do not receive the
alerts
No junior doctor was observed q...
Junior doctors at night
16:30-22:30
Observational study - 65 hours
78% of those alerts were read ≈ 50% read completely
...
2 hospital wards
Ward A 47 staff
Ward B 54 staff
How often do you seek advice from this
person about medication decisio...
• Each shape is a staff member
• Each line is a medication advice seeking connection
• Arrow indicate the direction of the...
Junior Drs are hubs of
medication information
Senior Drs were
isolates
Prescribing error rates
19.4 / 100 patient days
Sample of 240 admissions
9.0/100 patient days
Sample of 428 admissions
Conclusions regarding impact of eMMS
on work
 eMMS not associated with significant redistribution of time
 Some interact...
Team of researchers and hospital staff who made
this work possible
 Andrew Georgiou
 Ling Li
 Margaret Reckmann
 Melis...
Thank You
J.Westbrook@UNSW.edu.au
Centre for Health Systems & Safety
Research
Australian Institute of Health Innovation
UN...
Prof. Johanna Westbrook, Director, Centre for Health Systems and Safety Research - How Doctors’ and Nurses’ Patterns of Wo...
Prof. Johanna Westbrook, Director, Centre for Health Systems and Safety Research - How Doctors’ and Nurses’ Patterns of Wo...
Prof. Johanna Westbrook, Director, Centre for Health Systems and Safety Research - How Doctors’ and Nurses’ Patterns of Wo...
Prof. Johanna Westbrook, Director, Centre for Health Systems and Safety Research - How Doctors’ and Nurses’ Patterns of Wo...
Upcoming SlideShare
Loading in …5
×

Prof. Johanna Westbrook, Director, Centre for Health Systems and Safety Research - How Doctors’ and Nurses’ Patterns of Work and Communication Change Following the Introduction of an Electronic Medication Management System

703 views

Published on

Johanna Westbrook delivered this presentation at the 3rd Annual Electronic Medication Management Conference 2014. This conference is the nation’s only event to look solely at electronic prescribing and electronic medication management systems.

For more information, please visit http://www.healthcareconferences.com.au/emed14

Published in: Health & Medicine
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
703
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
60
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Prof. Johanna Westbrook, Director, Centre for Health Systems and Safety Research - How Doctors’ and Nurses’ Patterns of Work and Communication Change Following the Introduction of an Electronic Medication Management System

  1. 1. Centre for Health Systems and Safety Research Professor Johanna Westbrook Centre for Health Systems & Safety Research How doctors’ and nurses’ patterns of work and communication change following eMMS
  2. 2. Centre for Health Systems & Safety Research To produce world-class evidence which informs policy and practice, focusing on patient safety and the evaluation of information and communication technologies (ICT) in the health sector
  3. 3. Effectiveness Impact of eMMS on medication error rates? Efficiency How do eMMS impact work of health professionals? Why should we care about study design? What are some of the measurement challenges and how can we design useful evaluation studies? Outline
  4. 4. Results – Post eMMS Prescribing errors declined by > 50% from: 406 (95% CI 374-437) 185 (95% CI 160-210) per 100 admissions; p<0.0001 No significant change in errors on control wards Greatest reduction from orders which were illegible, incomplete or illegal National inpatient medication chart 5-10%
  5. 5. Serious prescribing errors Intervention wards – significant 44% (p=0.0002) reduction in serious prescribing error rate 25/100 admissions 14/100 admissions (95%CI 21-29) (95%CI 10-18) No significant change on the control wards (p=0.4)
  6. 6. MAE Post eMMS Across all clinical error categories - a significant reduction on the intervention wards of 4.24 clinical errors/100 administrations (95%CI: 0.15-8.32, p=0.04) compared to control wards. Wrong timing errors had the greatest decline by 3.35 /100 administrations (95%CI: 0.01-6.69, p<0.05) compared with control wards.
  7. 7. Change in serious medication administration errors Significant reduction in serious (ie potential ADEs) MAEs on the intervention wards compared to the control wards 4.20% 1.83% (95%CI 3.25, 5.15%) (95%CI 1.20, 2.46%) Pre Post
  8. 8. This eMMS is going to take too long
  9. 9. Systems are promoted for their ability to improve work efficiency and safety - Less time on administrative tasks - More time for patient care
  10. 10. Research Evidence Qualitative accounts - both significantly hinders and assists work efficiency Quantitative evidence is sparse Most studies on doctors’ work in ambulatory care and critical care Know little about the impact on nurses on general wards No Australian studies
  11. 11. Use of an eMMS
  12. 12. Aim: To measure changes in how nurses and doctors distributed their time across work tasks pre and post eMMS Changes in time spent on: Medication tasks Direct Care Professional Communication
  13. 13. Controlled Pre and Post Study Control Control Intervention Intervention Control Control eMMS eMMS 4 Wards Pre Year 1 4 Wards Post Year 3
  14. 14. Direct Observations Nurses & Doctors 70 nurses observed for 276.9 hours 59 doctors observed for 356.3 hours
  15. 15. Work Observation Method By Activity Timing - Where? With what? With whom? What task? Interruptions
  16. 16. Results Did nurses/doctors on the eMMS wards spend more or less time on direct care, medication tasks and professional communication compared to colleagues on the control wards?
  17. 17. No Significant Differences Nurses Comparison of time distribution control and eMMS Wards % Time P Value Direct Care Control 22.1 0.23 eMMS 26.1 Medication Control 23.7 0.28 eMMS 22.6 Prof Comm. Control 20.1 0.57 eMMS 22.8
  18. 18. Doctors Comparison of time distribution control and eMMS Wards % time P Value Direct Care Control 19.7 0.08 eMMS 25.7 Medication Control 7.4 0.4 eMMS 8.5 Prof Comm. Control 36.6 0.8 eMMS 37.6 No Significant Differences
  19. 19. Time Nurses Spent with Others Baseline 33% of nurses time is spent with patients 50% spent with other nurses 5% with doctors 4% with Relatives
  20. 20. Changes Following eMMS  Nurses on the eMMS wards spent less time with doctors (p=0.0001).  4.2% less time than nurses on the control wards.  Due to both fewer interactions (tasks per hour) and shorter interactions (mean task time).
  21. 21. Time Doctors Spent with Others Baseline 18% of doctors’ time is spent with patients 63% spent with other doctors 10% with nurses 4% with relatives
  22. 22. Changes Following eMMS for Doctors  Doctors on the eMMS wards spent more time with other doctors (p=0.003).  6% more time than doctors on the control wards.  Doctors spent more time with patients (p=.009)  6% more time than doctors on the control wards.
  23. 23. Available at JAMIA.BMJ.Com
  24. 24. Why should we care about study design? Before and after Versus Controlled before and after
  25. 25. Controlled Pre and Post Study Control Control Intervention Intervention Control Control eMMS eMMS 4 Wards Pre Year 1 4 Wards Post Year 3 Before After
  26. 26. Differences in controlled versus uncontrolled studies Control Control Intervention Intervention Control Control eMMS eMMS 4 Wards Pre Year 1 4 Wards Post Year 3 Comparison over time Comparison with & without eMMS taking account of how both have changed from baseline
  27. 27. How did nurses’s work change from year 1 (before) to year 3 (after)? Nurses now spending significantly more time on medication tasks (p=0.001) 20.2% year 1 23.1% year 3  Nurses now spending more time on direct care (p=0.003) 20.2% year 1 24.2% year 3  These changes were experienced by all nurses regardless of the eMMS
  28. 28. Looking for the expected and unexpected Selective attention
  29. 29. The Gorilla Strikes Again! – Drew and colleagues presented 24 radiologists with typical lung cancer screening CT scans
  30. 30. 20/24 radiologists (83%) missed the gorilla 25 non-trained reviewers all missed the gorilla “It’s important to be willing to look for more than one thing, to set yourself up for success.” Drew et al Psychol Science 2013
  31. 31. Context Matters Understanding how systems impact work will be influenced by context Example: Decision support When, who and how it will impact
  32. 32. What impact does decision support have during ward rounds?
  33. 33. Study – Teaching Hospital 58.5 hours direct observation of 14 teams on ward rounds 48% of medication orders triggered alerts 17% of alerts were read No prescriber read the entire content of an alert. No prescriptions were changed
  34. 34. Senior clinicians during ward-rounds are the decision-makers but do not receive the alerts No junior doctor was observed questioning a senior doctor’s decision following the triggering of an alert JAMIA, 2011
  35. 35. Junior doctors at night 16:30-22:30 Observational study - 65 hours 78% of those alerts were read ≈ 50% read completely 5% resulted in a change in prescribing
  36. 36. 2 hospital wards Ward A 47 staff Ward B 54 staff How often do you seek advice from this person about medication decisions/tasks?
  37. 37. • Each shape is a staff member • Each line is a medication advice seeking connection • Arrow indicate the direction of the advice sought • Networks of medication advice seeking at least weekly ie they sought medication advice weekly or more frequently
  38. 38. Junior Drs are hubs of medication information
  39. 39. Senior Drs were isolates
  40. 40. Prescribing error rates 19.4 / 100 patient days Sample of 240 admissions 9.0/100 patient days Sample of 428 admissions
  41. 41. Conclusions regarding impact of eMMS on work  eMMS not associated with significant redistribution of time  Some interactions change - implications of these for safety and quality should be investigated  On wards with eMMS there were significant reductions in both prescribing and medication administration errors  Only one aspect of work patterns – ie time distribution  eMMS will influence work practice and workflow in a multitude of expected and unexpected ways requiring investigation  Study design may have a significant impact on the results.  What’s next ? – Cost-effectiveness study
  42. 42. Team of researchers and hospital staff who made this work possible  Andrew Georgiou  Ling Li  Margaret Reckmann  Melissa Baysari  Nerida Creswick  Joanne Callen  Ric Day  Jeffrey Braithwaite  William Runciman  Richard Paoloni  Katherine Gibson  John Cullen  Louise Robertson  Rosemary Burke  Connie Lo  Kate Richardson  Maureen Heywood  Fiona McWhinnie  Amanda Woods  Naomi Malouf  Margaret Williamson  Jackie Laurens  Silvia Fazekas  Rosemary Richman  Joanne Villaret  Natasha Smith  Amanda Ampt  Melissa Pignone  David Roffe  Clinical ward staff  Pharmacy staff  ISD Staff
  43. 43. Thank You J.Westbrook@UNSW.edu.au Centre for Health Systems & Safety Research Australian Institute of Health Innovation UNSW Medicine This program of research has been supported by funding from the NHMRC & ARC

×