OPTIMISING DECISION
SUPPORT WITHIN EMMS
AN ONGOING CHALLENGE
MELISSA BAYSARI
+ JOHANNA WESTBROOK, RIC DAY, LING LI, KATE
R...
EMMS EFFECTIVENESS
Error rates fell by 66% at hospital A and 59.7% at hospital B
But most of the improvement was seen in “...
DECISION SUPPORT (DS)
Can mean different things to different people
Computerised alerts
Pre-written orders
Reference mater...
DS EFFECTIVENESS
Literature tells us that alerts can result in substantial
changes in prescribing behaviour
BUT
Most studi...
ALERT FATIGUE
A consequence of too many alerts being presented
Main barrier to prescriber acceptance of computerised alert...
ALERTS @ SVH (MEDCHART)
Allergy
Therapeutic duplication
Dose range
Local messages
Pregnancy
50% alerts are for information...
ALERT FATIGUE A PROBLEM?
1. Observations 2. Interviews 3. Chart audit
OPINIONS OF ALERTS
Registrar: It’s certainly helpful in, like I say, avoiding errors and
mistakes but I don’t think it rea...
OPINIONS OF ALERTS
Registrar: It pops up so often which can be a very bad thing
because you’re dismissing it so often that...
CHART AUDIT
No reporting function in MedChart to allow us to extract alert
information - had to conduct a detailed audit o...
ONLY OVERRIDDEN ALERTS
Limitation: Only alerts that were overridden were visible on
charts for review
But our observationa...
RESULTS – PATIENTS & ORDERS
Mean patient age: 63.7 yrs (20-100 yrs)
58% patients were male
2209 orders were active
Mean: 1...
RESULTS – ALERTS
600/2209 orders had 1 or more computerised alerts
27.2% of orders
934 alerts in total, mean 1.6 alerts/al...
PREGNANCY ALERTS
20 patients met the criteria (female, aged 12-55 yrs)
Of 119 meds ordered for these patients, 43.3% trigg...
LOCAL MESSAGES
¼ alerts we found were local messages
Most offer prescribers advice rather than warning about a
safety crit...
DUPLICATION ALERTS
Most frequent trigger = different drug, same therapeutic class
(40% of duplication alerts)
½ duplicatio...
During observations we noticed that a
number of duplication alerts were
being triggered because prescribers
were not using...
PEOPLE USE SYSTEMS IN
UNEXPECTED WAYS
Most users of applications utilize only a sub-set of system
features
(Think of all t...
STUDY AIM
To examine how the use of eMMS functions by prescribers can
influence alert generation
That is, to identify the ...
SHORT-CUT IN EMMS
THEN, AND, OR = allow similar sequential, concurrent or
alternative orders for the same medication to be...
+
These would save the prescriber up to 11 mouse clicks
During training, all Drs are shown where to find the short-
cuts and...
SHORT-CUT IN EMMS (2)
To make a change to an order on a patient’s chart, a doctor
should click on the order and edit the p...
PROCEDURE
For all orders where at least 1 alert was triggered we asked:
Could the use of a different system function (THEN...
PREVENTABLE ALERTS
189 alerts were technically preventable
= 1/3 of duplication alerts
= 20% of all alerts
Prescribers did...
WHY?
The efficient strategies are not known to users
and/or
The strategies are known but system design features are
poor
a...
CONSISTENCY OR
EFFICIENCY?
There is a tension between designing systems which
replicate paper-based processes and integrat...
REDUCING ALERT FATIGUE
Following the discovery that too many alerts are being
presented, how do we decide what alerts to r...
THE DELPHI TECHNIQUE
Group facilitation technique used to obtain consensus
among experts in a systematic way
Consensus is ...
STUDY AIM
To reach consensus among prescribers of different
specialties and with various levels of experience on
appropria...
SURVEY DEVELOPMENT
10-question web-based survey
Input was sought from prescribers, pharmacists & clinical
information syst...
POTENTIAL STRATEGIES
Identified in our previous work on alert fatigue:
1. Modifying most local messages so that they were
...
PROCEDURE
To recruit prescribers, an ad (with a link to the survey) was
posted in the weekly JMO bulletin sent to all JMOs...
SAMPLE ROUND 2 QUESTION
The percentages beside each option below indicate the proportion
of doctors who selected that opti...
CONSENSUS
Consensus was defined as 80% agreement between
participants on questions requiring a single response
Although co...
RESPONDENTS
Round 1: 47 prescribers, Round 2: 21 prescribers
Various specialties and levels of experience
Round 1
Alcohol ...
AREAS WHERE CONSENSUS
WAS REACHED
Prescribers agreed on what alert type should be retained
81% rated Allergy & intolerance...
AREAS WHERE NO
CONSENSUS WAS REACHED
0
10
20
30
40
50
Proportionofprescribers
Prescriber
responses to the
question ‘If you...
ALERT USEFULNESS
0
10
20
30
40
50
60
Never Rarely Sometimes Often
Proportionofprescribers
Allergy
Pregnancy
Duplication
Lo...
STUDY CONCLUSIONS
We identified some strategies that users viewed as
appropriate for reducing alert numbers
1. Present loc...
RESEARCH TRANSLATION
Based on observations, interviews and chart audit
Pregnancy alerts were removed
Many of the local mes...
OTHER STRATEGIES
(not as easy to implement as they sound)
Tier alerts according to severity
Include only high severity ale...
CONCLUSIONS
Getting alerts right is a challenge
Most sensible approach: include only a few alert types and
provide alterna...
THANK YOU
Contact: m.baysari@unsw.edu.au
This research is supported by NH&MRC Program grant #
568612
Dr Melissa Baysari, Research Fellow, Australian Institute of Health Innovation, University of New South Wales- Optimising ...
Dr Melissa Baysari, Research Fellow, Australian Institute of Health Innovation, University of New South Wales- Optimising ...
Dr Melissa Baysari, Research Fellow, Australian Institute of Health Innovation, University of New South Wales- Optimising ...
Dr Melissa Baysari, Research Fellow, Australian Institute of Health Innovation, University of New South Wales- Optimising ...
Upcoming SlideShare
Loading in …5
×

Dr Melissa Baysari, Research Fellow, Australian Institute of Health Innovation, University of New South Wales- Optimising Decision Support within Electronic Medication Management Systems -An ongoing challenge.

725 views

Published on

Melissa Baysari delivered this presentation at the 3rd Annual Electronic Medication Management Conference 2014. This conference is the nation’s only event to look solely at electronic prescribing and electronic medication management systems.

For more information, please visit http://www.healthcareconferences.com.au/emed14

Published in: Health & Medicine
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
725
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
38
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Dr Melissa Baysari, Research Fellow, Australian Institute of Health Innovation, University of New South Wales- Optimising Decision Support within Electronic Medication Management Systems -An ongoing challenge.

  1. 1. OPTIMISING DECISION SUPPORT WITHIN EMMS AN ONGOING CHALLENGE MELISSA BAYSARI + JOHANNA WESTBROOK, RIC DAY, LING LI, KATE RICHARDSON, ELIN LEHNBOM & MANY MORE
  2. 2. EMMS EFFECTIVENESS Error rates fell by 66% at hospital A and 59.7% at hospital B But most of the improvement was seen in “procedural” errors – e.g. fewer incomplete orders Little change in clinical errors (e.g. wrong doses) – those targeted by decision support
  3. 3. DECISION SUPPORT (DS) Can mean different things to different people Computerised alerts Pre-written orders Reference material What about: Drop down lists? Notes or instructions? Calculators?
  4. 4. DS EFFECTIVENESS Literature tells us that alerts can result in substantial changes in prescribing behaviour BUT Most studies evaluate an alert for a specific condition or problem e.g. alerts designed to reduce the use of contraindicated drugs in patients with renal failure  drop in proportion of patients receiving a contraindicated medication from 89% to 47% Less evidence for the effectiveness of basic decision support alerts within eMMS e.g. few studies showing that DDI alerts lead to reductions in DDIs aJAMIA 2005 12:269-74
  5. 5. ALERT FATIGUE A consequence of too many alerts being presented Main barrier to prescriber acceptance of computerised alerts A significant problem for hospitals because it results in user frustration & annoyance leads to prescribers learning to ignore all alerts, even those that present useful & sometimes safety critical information Alert fatigue affects most doctors in most organisations  most alerts are overridden
  6. 6. ALERTS @ SVH (MEDCHART) Allergy Therapeutic duplication Dose range Local messages Pregnancy 50% alerts are for information only 10% prescribers must enter an override reason 7 alerts do not allow prescriber to continue
  7. 7. ALERT FATIGUE A PROBLEM? 1. Observations 2. Interviews 3. Chart audit
  8. 8. OPINIONS OF ALERTS Registrar: It’s certainly helpful in, like I say, avoiding errors and mistakes but I don’t think it really helps in deciding say what antibiotic or what antihypertensive or whatever because that’s a clinical decision Registrar: The decision to prescribe something is based on your clinical knowledge…by the time you type it in and prescribe it you’ve already made that decision Resident: I guess less words and more point forms would be easier because then we wouldn’t have to scroll through paragraphs and sentences of text
  9. 9. OPINIONS OF ALERTS Registrar: It pops up so often which can be a very bad thing because you’re dismissing it so often that you develop this sort of mechanism so it can be bad in a sense that sometimes you might miss some important things Registrar: I at least scan them and work out what it is that they’re trying to tell me. Often it’s saying you’ve just prescribed, do you want to prescribe it again, and I’m like well yes, I do Resident: I don’t have a problem with all the alerts because I know what they say now before they even come up
  10. 10. CHART AUDIT No reporting function in MedChart to allow us to extract alert information - had to conduct a detailed audit of electronic charts to identify alerts Pharmacist randomly selected patients each day from a list of all inpatients 180 medication charts reviewed (6 weeks) The following info was recorded: Patient info (MRN, age, sex) # total active orders, # orders with 1 or more alerts For orders with an alert: prescriber, med name, med schedule (e.g. PRN), alert type
  11. 11. ONLY OVERRIDDEN ALERTS Limitation: Only alerts that were overridden were visible on charts for review But our observational work showed that the proportion of orders abandoned or changed is small (0-5%) Organisations implementing eMMS should specify the logging and reporting of alert data by vendors
  12. 12. RESULTS – PATIENTS & ORDERS Mean patient age: 63.7 yrs (20-100 yrs) 58% patients were male 2209 orders were active Mean: 12.3 orders/patient 96.8% of orders were initiated by junior doctors
  13. 13. RESULTS – ALERTS 600/2209 orders had 1 or more computerised alerts 27.2% of orders 934 alerts in total, mean 1.6 alerts/alerted order Alert type # (% of total alerts) Duplication 572 (61.2) Local messages 232 (24.8) Pregnancy 100 (10.7) Allergy 21 (2.3) Dose range 9 (1.0) Total 934
  14. 14. PREGNANCY ALERTS 20 patients met the criteria (female, aged 12-55 yrs) Of 119 meds ordered for these patients, 43.3% triggered a pregnancy alert Prescribers received on average 5 pregnancy alerts per eligible patient (range 1-10 alerts) ½ the alerts for these eligible patients were pregnancy alerts
  15. 15. LOCAL MESSAGES ¼ alerts we found were local messages Most offer prescribers advice rather than warning about a safety critical event Could these be removed and presented in a non-interruptive fashion?
  16. 16. DUPLICATION ALERTS Most frequent trigger = different drug, same therapeutic class (40% of duplication alerts) ½ duplication alerts were triggered because a medication was prescribed that was identical to, or in the same class as a drug that had been ceased within the previous 24 h
  17. 17. During observations we noticed that a number of duplication alerts were being triggered because prescribers were not using all the eMMS functions
  18. 18. PEOPLE USE SYSTEMS IN UNEXPECTED WAYS Most users of applications utilize only a sub-set of system features (Think of all the functions you DON’T use in Excel, on your iPhone, on your washing machine…) People use applications in less optimal ways to manage problematic or poorly designed IT E.g. users of an EHR used free-text boxes instead of the appropriate functions because the functions were hard to find
  19. 19. STUDY AIM To examine how the use of eMMS functions by prescribers can influence alert generation That is, to identify the proportion of duplication alerts triggered as a result of prescribers not utilizing eMMS functions
  20. 20. SHORT-CUT IN EMMS THEN, AND, OR = allow similar sequential, concurrent or alternative orders for the same medication to be prescribed together e.g. Frusemide in the morning AND midday
  21. 21. +
  22. 22. These would save the prescriber up to 11 mouse clicks During training, all Drs are shown where to find the short- cuts and complete case scenarios using them Drs are encouraged to use short-cuts as they save time
  23. 23. SHORT-CUT IN EMMS (2) To make a change to an order on a patient’s chart, a doctor should click on the order and edit the parameter (e.g. change the dose), instead of ceasing and re-ordering the medication
  24. 24. PROCEDURE For all orders where at least 1 alert was triggered we asked: Could the use of a different system function (THEN, AND, OR, or MODIFY) have prevented the alert from firing? Yes = technically preventable No = not technically preventable
  25. 25. PREVENTABLE ALERTS 189 alerts were technically preventable = 1/3 of duplication alerts = 20% of all alerts Prescribers did not use the eMMS functions as intended, despite the functions’ potential to improve efficiency of work JAMIA 2012, 19: 1003-1010
  26. 26. WHY? The efficient strategies are not known to users and/or The strategies are known but system design features are poor and/or The strategies are not viewed as beneficial or consistent with preferred prescribing practice
  27. 27. CONSISTENCY OR EFFICIENCY? There is a tension between designing systems which replicate paper-based processes and integrate quickly into clinical practice vs. Harnessing the advantage of technology to allow tasks to be completed in more efficient ways, but which require a change in work & cognitive processes
  28. 28. REDUCING ALERT FATIGUE Following the discovery that too many alerts are being presented, how do we decide what alerts to remove from the system? Previous study1: Interviewed doctors & pharmacists Found no alert types that all clinicians agreed could be turned off Found specialties differed in the number and types of alerts they thought could be safely turned off 1Van der Sijs et al. JAMIA. 2008;15(4):439-48.
  29. 29. THE DELPHI TECHNIQUE Group facilitation technique used to obtain consensus among experts in a systematic way Consensus is reached by allowing participants to consider their responses in light of the overall groups’ responses Delphi previously used to: Identify appropriate information to include in alerts Determine what information about the user and context is helpful in prioritizing and presenting alerts
  30. 30. STUDY AIM To reach consensus among prescribers of different specialties and with various levels of experience on appropriate strategies for reducing alerts within eMMS No previous studies have used Delphi for this purpose Previous Delphi research has included recruitment of experts in CPOE or decision support implementation, not users of the system
  31. 31. SURVEY DEVELOPMENT 10-question web-based survey Input was sought from prescribers, pharmacists & clinical information system staff In the survey, doctors were asked: What alert types they found useful/not useful What alert types, if any, they would remove from the system To rate each alert type on a Likert scale of usefulness Whether or not they believed 2 potential strategies for reducing alerts numbers would compromise patient safety:
  32. 32. POTENTIAL STRATEGIES Identified in our previous work on alert fatigue: 1. Modifying most local messages so that they were presented as hyperlinks on the prescribing screen, rather than interruptive alerts 2. Modifying therapeutic duplication alerts so that they fired only when the initial order was active on a patient’s chart, not when it was ceased within 24 hours
  33. 33. PROCEDURE To recruit prescribers, an ad (with a link to the survey) was posted in the weekly JMO bulletin sent to all JMOs at the site In round 2, doctors were sent a personalized email containing a link to their round 2 survey Feedback about round 1 responses were incorporated into each question in round 2:
  34. 34. SAMPLE ROUND 2 QUESTION The percentages beside each option below indicate the proportion of doctors who selected that option in round 1. Q2. If you could remove only one alert type from the current alert set in MedChart, which type would you remove? In round 1, you selected ‘Pregnancy’. ☐ Allergy & intolerances (2%) ☐ Pregnancy (34%) ☐ Therapeutic duplication (28%) ☐ Local rule (13%) ☐ None, I’d not remove any alert type (23%)
  35. 35. CONSENSUS Consensus was defined as 80% agreement between participants on questions requiring a single response Although consensus was not reached after 2 rounds, response stability was apparent, making it unlikely that participants would change views during a 3rd round
  36. 36. RESPONDENTS Round 1: 47 prescribers, Round 2: 21 prescribers Various specialties and levels of experience Round 1 Alcohol and drug Anesthetics Cardiology Clin Pharm Dermatology ED Gastroenterology Geriatrics Surgery Hematology Immunology ICU Medical oncology Nephrology Neurology Palliative care Psychiatry Rehabilitation Respiratory Urology Night shift/seconded Round 2 Alcohol and drug Cardiology Clin Pharm ED Geriatrics Surgery Hematology Immunology ICU Medical oncology Neurology Palliative care Psychiatry Rehabilitation Night shift/seconded
  37. 37. AREAS WHERE CONSENSUS WAS REACHED Prescribers agreed on what alert type should be retained 81% rated Allergy & intolerance alerts as the most useful alert type No participant believed this alert type should be removed All participants rated this alert type as ‘often’ or ‘sometimes’ useful Prescribers agreed that our suggested strategies would work 95% thought that changing local messages so they appeared as hyperlinks on the prescribing screen would be safe 91% thought that changing duplication warnings so they only fired when the initial order was active would be safe
  38. 38. AREAS WHERE NO CONSENSUS WAS REACHED 0 10 20 30 40 50 Proportionofprescribers Prescriber responses to the question ‘If you could remove one alert type from the current alert set in MedChart, which type would you remove?’
  39. 39. ALERT USEFULNESS 0 10 20 30 40 50 60 Never Rarely Sometimes Often Proportionofprescribers Allergy Pregnancy Duplication Local Prescriber responses to the question ‘How useful is each alert type in warning you about prescribing something potentially dangerous for your patients?’
  40. 40. STUDY CONCLUSIONS We identified some strategies that users viewed as appropriate for reducing alert numbers 1. Present local messages as hyperlinks Not unexpected because many messages provide low priority information 2. Ensure duplication alerts trigger when initial order is active – this would eliminate more than ½ of these alerts 24 h time-frame is only useful for a small number of medications (e.g. colchicine)
  41. 41. RESEARCH TRANSLATION Based on observations, interviews and chart audit Pregnancy alerts were removed Many of the local messages were replaced with corresponding pre-written orders Next step – assess clinical impact of altering duplication alerts so they fire only when the initial order is active
  42. 42. OTHER STRATEGIES (not as easy to implement as they sound) Tier alerts according to severity Include only high severity alerts Apply human factors principles in designing alerts Customize alerts for doctors
  43. 43. CONCLUSIONS Getting alerts right is a challenge Most sensible approach: include only a few alert types and provide alternative forms of DS to prescribers (e.g. pre- written orders) Continuously evaluate DS! Quantitative and qualitative methods allow us to determine if DS is working and why Seeking input and feedback from users is invaluable
  44. 44. THANK YOU Contact: m.baysari@unsw.edu.au This research is supported by NH&MRC Program grant # 568612

×