Drug-Drug Interaction Alerts: Time for a New Paradigm
•Download as PPTX, PDF•
6 likes•1,522 views
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Drug-Drug Interaction Alerts: Time for a New Paradigm
Similar to Drug-Drug Interaction Alerts: Time for a New Paradigm (20)
More from Jon Duke, MD, MS
More from Jon Duke, MD, MS (10)
Recently uploaded
Recently uploaded (20)
Drug-Drug Interaction Alerts: Time for a New Paradigm
- 1. Drug-Drug Interaction Alerts: Time for a New Paradigm Jon D. Duke, MD, MS, Regenstrief Institute
- 2. Reviewed 42,641 orders 11% (4690) produced alerts DDI alert override rate = 88% Allergy override rate = 69% Refinement in order check logic could reduce override rates and may increase practitioner acceptance and effectiveness of order checks.
- 3. Ten Commandments for Effective Clinical Decision Support 1. Speed is everything 2. Anticipate needs and deliver in real time 3. Fit into user’s workflow 4. Little things can make a big difference (usability) 5. Physicians resist stopping 6. Changing direction is fine 7. Simple interventions work best 8. Asking for information is OK − but be sure you really need it 9. Monitor impact, get feedback, and respond 10. Knowledge-based systems must be managed/maintained Bates et al. J Am Med Inform Assoc. 2003;10:523-30. (PMID: 12925543)
- 4. Reviewed 18,354 orders 13% (2455) produced alerts DDI alert override rate = 95% Allergy override rate = 91%
- 5. Conclusions: Despite intensive efforts to improve a commercial drug interaction alert system and to reduce alerting, override rates remain as high as reported over a decade ago. Alert fatigue does not seem to contribute. The results suggest the need To fundamentally question the premises of drug interaction alert systems.
- 6. Why can’t we move the needle?
- 7. Current Approach for Drug-Drug Interaction Decision Support (I) • Typically interruptive pop-up alerts – Computerized provider order entry (CPOE) – Pharmacist verification/dispensing • Required for Meaningful Use Stages 1 and 2 1 • Most organizations use commercially available drug-drug interaction (DDI) knowledgebases – Impractical for most organizations to create/maintain 1) http://www.healthit.gov/providers-professionals/achieve-meaningful-use/core-measures/drug-interaction-check 2) http://www.healthit.gov/providers-professionals/achieve-meaningful-use/core-measures-2/clinical-decision-support-rule
- 8. Current Approach for Drug-Drug Interaction Decision Support (II) • Alerts often perceived excessive or irrelevant – Presentation is suboptimal 1 – Providers dissatisfied 2 – High override rates 3 • Customizing commercial knowledgebases requires substantial resources – Organizations may turn off decision support – Potential unintended consequences 4 1) Russ et al. Int J Med Inform. 2012;81:232-43. 2) Weingart et al.. Arch Intern Med 2009;169:1627-32. 3) van der Sijs et al. J Am Med Inform Assoc 2006;13:138-47. 4) van der Sijs et al. J Am Med Inform Assoc. 2008;15:439-48.
- 9. Drug-Drug Interactions and Harm (I) • Exposure to DDIs is a source of preventable drug-related harm1 Association Between Hospital Admission for Drug Toxicity and Recent Co-Prescription of Interaction Medications (Juurlink et al. 2003) 2 INTERACTING MEDICATIONS TOXICITY OR (95% CI) Glyburide + co-trimoxazole Hypoglycemia 6.6 (4.5-9.7) Digoxin + clarithromycin Digoxin toxicity 11.7 (7.5-18.2) ACE inhibitor + potassium-sparing diuretic Hyperkalemia 20.3 (13.4-30.7) • Estimated to harm 1.9-5 million inpatients3 and cause up to 220,000 ED visits per year4,5 1) IOM. Preventing medication errors. National Academies Press. 2007. 2) Juurlink et al. JAMA. 2003;289:1652-8. 3) Magro et al. Expert Opin Drug Saf. 2012;11:83-94. 4) CDC. FASTSTATS - Emergency Department Visits. 2012; http://www.cdc.gov/nchs/fastats/ervisits.htm; 5) CDC. FASTSTATS - Hospital Utilization. 2010; http://www.cdc.gov/nchs/fastats/hospital.htm.
- 10. Drug-Drug Interactions and Harm (II) • Most potential DDIs are clinically inconsequential • DDIs are responsible for a low proportion of adverse drug events overall (<5%) • But DDIs trigger a high proportion of alerts
- 11. Low Satisfaction with DDI Alerts • Physician survey (N=184) • 53% not satisfied with DDI / allergy alerts • Top complaints – Alerts triggered by discontinued medications – Failure to account for appropriate combinations – Excessive volume of alerts Weingart et al. Arch Intern Med. 2009;169:1627-32.
- 12. Low Adherence to DDI Alerts • Varies study to study but continue to see 60%-95% override rates for interruptive DDI alerts • Non-interruptive alerts generating 1-2% adherence Van der Sijs et al. JAMIA 2006. 13(2):138-147. Seidling et al. J Am Med Inform Assoc. 2011;18:479-84.
- 13. Hard Stops Work But… • RCT including “hard stop” DDI alert – 1981 prescribers, 2 academic medical centers – Warfarin + trimethoprim/sulfamethoxazole Strom et al. Arch Intern Med. 2010;170:1578-83.
- 14. Unintended Consequences Study stopped early due to unintended consequences in intervention group UNINTENDED CONSEQUENCE RELATION TO INTERVENTION 3-day delay in TMP/SMX therapy deemed necessary by infectious disease Probable Failure to prescribe TMP/SMX prophylaxis for critically ill patient Probable 1-day delay in warfarin therapy Definite 3-day delay in warfarin therapy Definite Strom et al. Arch Intern Med. 2010;170:1578-83.
- 15. Alerts with Poor Specificity Study of 279,476 alerts by 2,321 physicians over 6 months in the ambulatory care setting 331 alerts to prevent 1 ADE 10% of alerts accounted for 60% ADEs prevented and 78% of cost benefit Weingart et al. Arch Intern Med. 2009;169:1465-73.
- 16. Lack of Consistency Across Systems • 62 hospitals voluntarily participated for review of simulated DDI orders of varying severity • Detected only 53% of medication orders that would result in fatality • Detected 10-82% of orders that would have caused serious ADEs • Did not correlate with specific vendors Metzger et al. Health Aff. 2010;29:655-63.
- 17. Lack of Consistency Across Systems Scores For Detection of Test Orders That Would Cause an Adverse Event - By Software Vendor Metzger et al. Health Aff. 2010;29:655-63.
- 18. Similar Story in Pharmacies • 64 inpatient and outpatient Arizona pharmacies • Fictitious patient orders to evaluate 19 drug pairs – 13 DDIs and 6 non-DDIs • Median correct responses 89.5% (range 47-100%) Saverno et al. J Am Med Inform Assoc. 2011;18:32-7.
- 19. Lack of Consistency in Pharmacy Alerts 75% 80% 83% 87% 88% 86% 89% 45% 81% 90% 75% 84% 70% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Warfarin + sulfamethoxazole/trimethoprim Warfarin + naproxen Warfarin + gemfibrozil Warfarin + fluconazole Warfarin + amiodarone Simvastatin + gemfibrozil Simvastatin + amiodarone Simvastain + itraconazole Nitroglycerin + sildenafil Digoxin + itraconazole Digoxin + clarithromycin Digoxin + amiodarone Carbamazepine + clarithromycin Saverno et al. J Am Med Inform Assoc. 2011;18:32-7.
- 20. Any Good News?
- 21. Usability Does Help • 50,788 DDI alerts analyzed • Higher quality alert display increased adherence Factors Associated with Interruptive Alert Acceptance PARAMETER OR (95% CI) P-VALUE Quality of alert display 4.75 (3.87-5.84) <0.001 Setting (inpatient vs. outpatient) 2.63 (2.32-2.97) <0.001 Level of the alert 1.74 (1.63-1.86) <0.001 Frequency of the alert 1.30 (1.23-1.38) <0.001 Dose-dependent toxicity 1.13 (1.07-1.21) <0.001 Seidling et al. J Am Med Inform Assoc. 2011;18:479-84.
- 22. Usability Key Factors • Consistent signal words, severity descriptions • Consistent colors and icons • Consistent placement of information • Parsimonious use of text (details on demand) • Directly actionable • Present as early as possible
- 23. Providing Patient Context • Cluster RCT, 81 family physicians, 5,628 elderly patients • Modified alerts with patient-specific estimates of fall risk with psychotropic medications • Reduced risk of injury by 1.7 injuries per 1000 patients (95% CI 0.2 to 3.2; p=0.02) Tamblyn et al. R. J Am Med Inform Assoc. 2012;19:635-43.
- 24. Providing Patient Context • RCT of alert using predictive model to inform risk regarding QT prolongation Tisdale et al. R. J Am Col Cardiology. 2012;59(13):E1799.
- 25. Providing Patient Context • Reduced inappropriate prescribing by 21% • Reduced odds of QT prolongation by 35%
- 26. Getting to Providers Earlier
- 27. Getting to Providers Earlier
- 28. Getting to Providers Earlier
- 29. Improving the Knowledgebase • Identifying high priority alerts • Identifying suppressible alerts • Emerging predictive models around certain adverse outcomes (e.g., DDIs associated with hyperkalemia) • Ideas swirling around the learning healthcare system / feedback loops for improving alert delivery and appropriateness Phansalkar S et al, JAMIA 2012. 19:735-743. Phansalkar S et al, JAMIA 2013 20:489-493. Eschmann E et al, Eur J Clin Pharmacol 2014. 70(2):215-23. McCoy A et al, Ochsner 2014.14:195-202
- 30. So I’d Like to Conclude • We just need to… – Improve alert display and usability –Optimize alert specificity and sensitivity – Increase knowledgebase consistency – Incorporate contextual factors
- 31. But How Much Will It Move the Needle?
- 32. But How Much Will It Move the Needle?
- 33. Our Real Problem Is TRUST
- 34. Why Doctors Still Won’t Trust DDI Alerts • Disregarding alerts has become part of the medical culture • It is inculcated during training, just as medical slang and other aspects of the sub-culture • It is of course reinforced by all the problems we’ve described above • Fixing the problems with our alerts will not fix the trust problem (for a long, long time)
- 35. So How Do We Get Doctors to Listen to DDI Alerts? • First, why do doctors listen to anyone?
- 37. You have received conflicting advice regarding the prescribing of an antibiotic for an inpatient with community acquired PNA. Whose advice are you more likely to trust? To follow?
- 38. Why Do Doctors Take Advice? • “Positive” Drivers – Authority / Hierarchy – Specialty – Perceived Experience / Knowledge – Team-building • “Negative” Drivers – Fear (e.g., of mistakes, lawsuits) – Embarrassment
- 39. Why Do We Adhere?
- 40. Increase Visibility of Adverse Events Note: You have ignored this DDI warning 27 times on 14 unique patients. Of these, 2 patients have developed a bleeding-related condition.
- 41. Increase Visibility of Adverse Events Note: This DDI has been associated with 17 serious adverse events at our hospital in 2014.
- 42. Increase Visibility of Adverse Events Note: 2,585 serious adverse event reports indicating concurrent use of Amiodarone and Warfarin were submitted to FDA in 2014.
- 43. Increase Visibility of Adverse Events Note: There were 12 lawsuits associated with concurrent use of Amiodarone and Warfarin in Indiana between 2010 and 2014.
- 44. Connect with Hospital Hierarchy Steve Nissen, MD Chair of Cardiology Approved Alert
- 45. Allergy Persist and Propagate Override Status Allergy Warning
- 46. Persist and Propagate Override Status
- 47. Persist and Propagate Override Status
- 48. Persist and Propagate Override Status Embed in Chart Addendum: AMOXICILLIN 500MG. Allergy Alert Override by Smith, JD. 11/14/2014 at 8:31am.
- 49. The New Paradigm? It’s People • Recognize and leverage natural human emotions as part of system design • Decisions should be visible to peers and authority figures • DDI warnings should be ‘sponsored’ by specific local experts • Drug safety decisions should be longitudinal rather than instantaneous events
Editor's Notes
- 88% override rate Characteristics and override rates of order checks in a practitioner order entry system. Payne TH1, Nichol WP, Hoey P, Savarino J. Author information Abstract Order checks are important error prevention tools when used in conjunction with practitioner order entry systems. We studied characteristics oforder checks generated in a sample of consecutively entered orders during a 4 week period in an electronic medical record at VA Puget Sound. We found that in the 42,641 orders where an order check could potentially be generated, 11% generated at least one order check and many generated more than one order check. The rates at which the ordering practitioner overrode 'Critical drug interaction' and 'Allergy-drug interaction' alerts in this sample were 88% and 69% respectively. This was in part due to the presence of alerts for interactions between systemic and topical medications and for alerts generated during medication renewals. Refinement in order check logic could lead to lower override rates and increase practitioner acceptance and effectiveness of order checks.
- 88% override rate Characteristics and override rates of order checks in a practitioner order entry system. Payne TH1, Nichol WP, Hoey P, Savarino J. Author information Abstract Order checks are important error prevention tools when used in conjunction with practitioner order entry systems. We studied characteristics oforder checks generated in a sample of consecutively entered orders during a 4 week period in an electronic medical record at VA Puget Sound. We found that in the 42,641 orders where an order check could potentially be generated, 11% generated at least one order check and many generated more than one order check. The rates at which the ordering practitioner overrode 'Critical drug interaction' and 'Allergy-drug interaction' alerts in this sample were 88% and 69% respectively. This was in part due to the presence of alerts for interactions between systemic and topical medications and for alerts generated during medication renewals. Refinement in order check logic could lead to lower override rates and increase practitioner acceptance and effectiveness of order checks.
- Bryant AD, Fletcher GS, Payne TH. Drug interaction alert override rates in the meaningful use era: no evidence of progress. Appl Clin Inform. 2014 Sep 3;5(3):802-13. doi: 10.4338/ACI-2013-12-RA-0103. eCollection 2014. PubMed PMID: 25298818; PubMed Central PMCID: PMC4187095. BACKGROUND: Interruptive drug interaction alerts may reduce adverse drug events and are required for Stage I Meaningful Use attestation. For the last decade override rates have been very high. Despite their widespread use in commercial EHR systems, previously described interventions to improve alert frequency and acceptance have not been well studied. OBJECTIVES: (1) To measure override rates of inpatient medication alerts within a commercial clinical decision support system, and assess the impact of local customization efforts. (2) To compare override rates between drug-drug interaction and drug-allergy interaction alerts, between attending and resident physicians, and between public and academic hospitals. (3) To measure the correlation between physicians' individual alert quantities and override rates as an indicator of potential alert fatigue. METHODS: We retrospectively analyzed physician responses to drug-drug and drug-allergy interaction alerts, as generated by a common decision support product in a large teaching hospital system. RESULTS: (1) Over four days, 461 different physicians entered 18,354 medication orders, resulting in 2,455 visible alerts; 2,280 alerts (93%) were overridden. (2) The drug-drug alert override rate was 95.1%, statistically higher than the rate for drug-allergy alerts (90.9%) (p < 0.001). There was no significant difference in override rates between attendings and residents, or between hospitals. (3) Physicians saw a mean of 1.3 alerts per day, and the number of alerts per physician was not significantly correlated with override rate (R2 = 0.03, p = 0.41). CONCLUSIONS: Despite intensive efforts to improve a commercial drug interaction alert system and to reduce alerting, override rates remain as high as reported over a decade ago. Alert fatigue does not seem to contribute. The results suggest the need to fundamentally question the premises of drug interaction alert systems.
- Before a medication order is completed and acted upon during computerized provider order entry (CPOE), interventions must automatically and electronically indicate to a user drug - drug and drug - allergy contraindications based on a patient’s medication list and medication allergy list
- Exposure to potential drug-drug interactions (DDIs) is a significant source of preventable drug-related harm that requires proper management to avoid medical errors [1]. Studies indicate DDIs harm 1.9 to 5 million inpatients per year and cause 2,600 to 220,000 emergency department visits per year [2]. 1. Aspden P, Institute of Medicine (U.S.). Committee on Identifying and Preventing Medication Errors. Preventing medication errors. Washington, DC: National Academies Press; 2007. 2. Magro L, Moretti U, Leone R. Epidemiology and characteristics of adverse drug reactions caused by drugdrug interactions. Expert Opin Drug Saf. 2012 Jan;11(1):83-94.
- Weingart SN, Simchowitz B, Shiman L, Brouillard D, Cyrulik A, Davis RB, Isaac T, Massagli M, Morway L, Sands DZ, Spencer J, Weissman JS. Clinicians' assessments of electronic medication safety alerts in ambulatory care. Arch Intern Med. 2009 Sep 28;169(17):1627-32. doi: 10.1001/archinternmed.2009.300. PubMed PMID: 19786683. BACKGROUND: While electronic prescribing (e-prescribing) systems with drug interaction and allergy alerts promise to improve medication safety in ambulatory care, clinicians often override these safety features. We undertook a study of respondents' satisfaction with e-prescribing systems, their perceptions of alerts, and their perceptions of behavior changes resulting from alerts. METHODS: Random sample survey of 300 Massachusetts ambulatory care clinicians who used a commercial e-prescribing system. RESULTS: A total of 184 respondents completed the survey (61%). Respondents indicated that e-prescribing improved the quality of care delivered (78%), prevented medical errors (83%), and enhanced patient satisfaction (71%) and clinician efficiency (75%). In addition, 35% of prescribers said that electronic alerts caused them to modify a potentially dangerous prescription in the last 30 days. They suggested that alerts also led to other changes in clinical care: counseling patients about potential reactions (49% of respondents), looking up information in medical references (44%), and changing the way a patient was monitored (33%). Altogether, 63% of clinicians reported taking action other than discontinuing or modifying an alerted prescription in the previous month in response to alerts. Despite these benefits, fewer than half of respondents were satisfied with drug interaction and allergy alerts (47%). Problems included alerts triggered by discontinued medications (58%), alerts that failed to account for appropriate drug combinations (46%), and excessive volume of alerts (37%). CONCLUSION: Although clinicians were critical of the quality of e-prescribing alerts, alerts may lead to clinically significant modifications in patient management not readily apparent based on "acceptance" rates.
- Bryant AD, Fletcher GS, Payne TH. Drug interaction alert override rates in the meaningful use era: no evidence of progress. Appl Clin Inform. 2014 Sep 3;5(3):802-13. doi: 10.4338/ACI-2013-12-RA-0103. eCollection 2014. PubMed PMID: 25298818; PubMed Central PMCID: PMC4187095. BACKGROUND: Interruptive drug interaction alerts may reduce adverse drug events and are required for Stage I Meaningful Use attestation. For the last decade override rates have been very high. Despite their widespread use in commercial EHR systems, previously described interventions to improve alert frequency and acceptance have not been well studied. OBJECTIVES: (1) To measure override rates of inpatient medication alerts within a commercial clinical decision support system, and assess the impact of local customization efforts. (2) To compare override rates between drug-drug interaction and drug-allergy interaction alerts, between attending and resident physicians, and between public and academic hospitals. (3) To measure the correlation between physicians' individual alert quantities and override rates as an indicator of potential alert fatigue. METHODS: We retrospectively analyzed physician responses to drug-drug and drug-allergy interaction alerts, as generated by a common decision support product in a large teaching hospital system. RESULTS: (1) Over four days, 461 different physicians entered 18,354 medication orders, resulting in 2,455 visible alerts; 2,280 alerts (93%) were overridden. (2) The drug-drug alert override rate was 95.1%, statistically higher than the rate for drug-allergy alerts (90.9%) (p < 0.001). There was no significant difference in override rates between attendings and residents, or between hospitals. (3) Physicians saw a mean of 1.3 alerts per day, and the number of alerts per physician was not significantly correlated with override rate (R2 = 0.03, p = 0.41). CONCLUSIONS: Despite intensive efforts to improve a commercial drug interaction alert system and to reduce alerting, override rates remain as high as reported over a decade ago. Alert fatigue does not seem to contribute. The results suggest the need to fundamentally question the premises of drug interaction alert systems.
- Strom BL, Schinnar R, Aberra F, Bilker W, Hennessy S, Leonard CE, Pifer E. Unintended effects of a computerized physician order entry nearly hard-stop alert to prevent a drug interaction: a randomized controlled trial. Arch Intern Med. 2010 Sep 27;170(17):1578-83. doi: 10.1001/archinternmed.2010.324. PubMed PMID: 20876410. BACKGROUND: The effectiveness of computerized physician order entry (CPOE) systems has been modest, largely because clinicians frequently override electronic alerts. METHODS: To evaluate the effectiveness of a nearly "hard stop" CPOE prescribing alert intended to reduce concomitant orders for warfarin and trimethoprim-sulfamethoxazole, a randomized clinical trial was conducted at 2 academic medical centers in Philadelphia, Pennsylvania. A total of 1981 clinicians were assigned to either an intervention group receiving a nearly hard stop alert or a control group receiving the standard practice. The study duration was August 9, 2006, through February 13, 2007. RESULTS: The proportion of desired responses (ie, not reordering the alert-triggering drug within 10 minutes of firing) was 57.2% (111 of 194 hard stop alerts) in the intervention group and 13.5% (20 of 148) in the control group (adjusted odds ratio, 0.12; 95% confidence interval, 0.045-0.33). However, the study was terminated early because of 4 unintended consequences identified among patients in the intervention group: a delay of treatment with trimethoprim-sulfamethoxazole in 2 patients and a delay of treatment with warfarin in another 2 patients. CONCLUSIONS: An electronic hard stop alert as part of an inpatient CPOE system seemed to be extremely effective in changing prescribing. However, this intervention precipitated clinically important treatment delays in 4 patients who needed immediate drug therapy. These results illustrate the importance of formal evaluation and monitoring for unintended consequences of programmatic interventions intended to improve prescribing habits.
- Strom BL, Schinnar R, Aberra F, Bilker W, Hennessy S, Leonard CE, Pifer E. Unintended effects of a computerized physician order entry nearly hard-stop alert to prevent a drug interaction: a randomized controlled trial. Arch Intern Med. 2010 Sep 27;170(17):1578-83. doi: 10.1001/archinternmed.2010.324. PubMed PMID: 20876410. For a synopsis of the monthly monitoring of unintended consequences associated with the intervention—either a delay of treatment with trimethoprim- sulfamethoxazole when determined to be necessary for treatment or inadvertent warfarin discontinuation— see the Table. Review of the electronic records of all 4 individuals involved in these events showed that in no case could we identify specific infectious or thrombotic complications that could have been related to the delays in therapy in the adverse event reporting. METHODS: To evaluate the effectiveness of a nearly "hard stop" CPOE prescribing alert intended to reduce concomitant orders for warfarin and trimethoprim-sulfamethoxazole, a randomized clinical trial was conducted at 2 academic medical centers in Philadelphia, Pennsylvania. A total of 1981 clinicians were assigned to either an intervention group receiving a nearly hard stop alert or a control group receiving the standard practice. The study duration was August 9, 2006, through February 13, 2007. RESULTS: The proportion of desired responses (ie, not reordering the alert-triggering drug within 10 minutes of firing) was 57.2% (111 of 194 hard stop alerts) in the intervention group and 13.5% (20 of 148) in the control group (adjusted odds ratio, 0.12; 95% confidence interval, 0.045-0.33). However, the study was terminated early because of 4 unintended consequences identified among patients in the intervention group: a delay of treatment with trimethoprim-sulfamethoxazole in 2 patients and a delay of treatment with warfarin in another 2 patients. CONCLUSIONS: An electronic hard stop alert as part of an inpatient CPOE system seemed to be extremely effective in changing prescribing. However, this intervention precipitated clinically important treatment delays in 4 patients who needed immediate drug therapy. These results illustrate the importance of formal evaluation and monitoring for unintended consequences of programmatic interventions intended to improve prescribing habits.
- Cumulative number of serious, significant, and minor adverse drug events (ADEs) prevented by safety alerts. Data were obtained from a cohort of electronic prescribers in Massachusetts in 2006. A small percentage of alerts accounted for most of the estimated benefits. Weingart SN, Simchowitz B, Padolsky H, Isaac T, Seger AC, Massagli M, Davis RB, Weissman JS. An empirical model to estimate the potential impact of medication safety alerts on patient safety, health care utilization, and cost in ambulatory care. Arch Intern Med. 2009 Sep 14;169(16):1465-73. doi: 10.1001/archinternmed.2009.252. PubMed PMID: 19752403. METHODS: We studied 279 476 alerted prescriptions written by 2321 Massachusetts ambulatory care clinicians using a single commercial e-prescribing system from January 1 through June 30, 2006. An expert panel reviewed a sample of common drug interaction alerts, estimating the likelihood and severity of adverse drug events (ADEs) associated with each alert, the likely injury to the patient, and the health care utilization required to address each ADE. We estimated the cost savings due to e-prescribing by using third-party-payer and publicly available information. RESULTS: Based on the expert panel's estimates, electronic drug alerts likely prevented 402 (interquartile range [IQR], 133-846) ADEs in 2006, including 49 (14-130) potentially serious, 125 (34-307) significant, and 228 (85-409) minor ADEs. Accepted alerts may have prevented a death in 3 (IQR, 2-13) cases, permanent disability in 14 (3-18), and temporary disability in 31 (10-97). Alerts potentially resulted in 39 (IQR, 14-100) fewer hospitalizations, 34 (6-74) fewer emergency department visits, and 267 (105-541) fewer office visits, for a cost savings of 402,619 USD (IQR, 141,012-1,012,386 USD). Based on the panel's estimates, 331 alerts were required to prevent 1 ADE, and a few alerts (10%) likely accounted for 60% of ADEs and 78% of cost savings. CONCLUSIONS: Electronic prescribing alerts in ambulatory care may prevent a substantial number of injuries and reduce health care costs in Massachusetts. Because a few alerts account for most of the benefit, e-prescribing systems should suppress low-value alerts.
- The median sensitivity to detect well-established interactions was 0.85 (range 0.23e1.0); 58%median specificity was 1.0 (range 0.83e1.0); median positive predictive value was 1.0 (range 0.88e1.0); and median negative predictive value was 0.75 (range 0.38e1.0). Conclusions These study results indicate that many pharmacy clinical decision-support systems perform less than optimally with respect to identifying well-known, clinically relevant interactions. Comprehensive system improvements regarding the manner in which pharmacy information systems identify potential DDIs are warranted.
- Sensitivity of Computer Software to Detect DDIs in Arizona Pharmacies (N=64)
- Russ AL, Zillich AJ, Melton BL, Russell SA, Chen S, Spina JR, Weiner M, Johnson EG, Daggy JK, McManus MS, Hawsey JM, Puleo AG, Doebbeling BN, Saleem JJ. Applying human factors principles to alert design increases efficiency and reduces prescribing errors in a scenario-based simulation. J Am Med Inform Assoc. 2014 Oct;21(e2):e287-96. doi: 10.1136/amiajnl-2013-002045. Epub 2014 Mar 25. PubMed PMID: 24668841; PubMed Central PMCID: PMC4173163. To apply human factors engineering principles to improve alert interface design. We hypothesized that incorporating human factors principles into alerts would improve usability, reduce workload for prescribers, and reduce prescribing errors. MATERIALS AND METHODS: We performed a scenario-based simulation study using a counterbalanced, crossover design with 20 Veterans Affairs prescribers to compare original versus redesigned alerts. We redesigned drug-allergy, drug-drug interaction, and drug-disease alerts based upon human factors principles. We assessed usability (learnability of redesign, efficiency, satisfaction, and usability errors), perceived workload, and prescribing errors. RESULTS: Although prescribers received no training on the design changes, prescribers were able to resolve redesigned alerts more efficiently (median (IQR): 56 (47) s) compared to the original alerts (85 (71) s; p=0.015). In addition, prescribers rated redesigned alerts significantly higher than original alerts across several dimensions of satisfaction. Redesigned alerts led to a modest but significant reduction in workload (p=0.042) and significantly reduced the number of prescribing errors per prescriber (median (range): 2 (1-5) compared to original alerts: 4 (1-7); p=0.024). DISCUSSION: Aspects of the redesigned alerts that likely contributed to better prescribing include design modifications that reduced usability-related errors, providing clinical data closer to the point of decision, and displaying alert text in a tabular format. Displaying alert text in a tabular format may help prescribers extract information quickly and thereby increase responsiveness to alerts. CONCLUSIONS: This simulation study provides evidence that applying human factors design principles to medication alerts can improve usability and prescribing outcomes
- Seidling HM, Phansalkar S, Seger DL, Paterno MD, Shaykevich S, Haefeli WE, Bates DW. Factors influencing alert acceptance: a novel approach for predicting the success of clinical decision support. J Am Med Inform Assoc. 2011 Jul-Aug;18(4):479-84. doi: 10.1136/amiajnl-2010-000039. Epub 2011 May 12. PubMed PMID: 21571746; PubMed Central PMCID: PMC3128393. BACKGROUND: Clinical decision support systems can prevent knowledge-based prescription errors and improve patient outcomes. The clinical effectiveness of these systems, however, is substantially limited by poor user acceptance of presented warnings. To enhance alert acceptance it may be useful to quantify the impact of potential modulators of acceptance. METHODS: We built a logistic regression model to predict alert acceptance of drug-drug interaction (DDI) alerts in three different settings. Ten variables from the clinical and human factors literature were evaluated as potential modulators of provider alert acceptance. ORs were calculated for the impact of knowledge quality, alert display, textual information, prioritization, setting, patient age, dose-dependent toxicity, alert frequency, alert level, and required acknowledgment on acceptance of the DDI alert. RESULTS: 50,788 DDI alerts were analyzed. Providers accepted only 1.4% of non-interruptive alerts. For interruptive alerts, user acceptance positively correlated with frequency of the alert (OR 1.30, 95% CI 1.23 to 1.38), quality of display (4.75, 3.87 to 5.84), and alert level (1.74, 1.63 to 1.86). Alert acceptance was higher in inpatients (2.63, 2.32 to 2.97) and for drugs with dose-dependent toxicity (1.13, 1.07 to 1.21). The textual information influenced the mode of reaction and providers were more likely to modify the prescription if the message contained detailed advice on how to manage the DDI. CONCLUSION: We evaluated potential modulators of alert acceptance by assessing content and human factors issues, and quantified the impact of a number of specific factors which influence alert acceptance. This information may help improve clinical decision support systems design.
- Russ AL, Zillich AJ, Melton BL, Russell SA, Chen S, Spina JR, Weiner M, Johnson EG, Daggy JK, McManus MS, Hawsey JM, Puleo AG, Doebbeling BN, Saleem JJ. Applying human factors principles to alert design increases efficiency and reduces prescribing errors in a scenario-based simulation. J Am Med Inform Assoc. 2014 Oct;21(e2):e287-96. doi: 10.1136/amiajnl-2013-002045. Epub 2014 Mar 25. PubMed PMID: 24668841; PubMed Central PMCID: PMC4173163. To apply human factors engineering principles to improve alert interface design. We hypothesized that incorporating human factors principles into alerts would improve usability, reduce workload for prescribers, and reduce prescribing errors. MATERIALS AND METHODS: We performed a scenario-based simulation study using a counterbalanced, crossover design with 20 Veterans Affairs prescribers to compare original versus redesigned alerts. We redesigned drug-allergy, drug-drug interaction, and drug-disease alerts based upon human factors principles. We assessed usability (learnability of redesign, efficiency, satisfaction, and usability errors), perceived workload, and prescribing errors. RESULTS: Although prescribers received no training on the design changes, prescribers were able to resolve redesigned alerts more efficiently (median (IQR): 56 (47) s) compared to the original alerts (85 (71) s; p=0.015). In addition, prescribers rated redesigned alerts significantly higher than original alerts across several dimensions of satisfaction. Redesigned alerts led to a modest but significant reduction in workload (p=0.042) and significantly reduced the number of prescribing errors per prescriber (median (range): 2 (1-5) compared to original alerts: 4 (1-7); p=0.024). DISCUSSION: Aspects of the redesigned alerts that likely contributed to better prescribing include design modifications that reduced usability-related errors, providing clinical data closer to the point of decision, and displaying alert text in a tabular format. Displaying alert text in a tabular format may help prescribers extract information quickly and thereby increase responsiveness to alerts. CONCLUSIONS: This simulation study provides evidence that applying human factors design principles to medication alerts can improve usability and prescribing outcomes
- . CONTEXT: Computerized drug alerts for psychotropic drugs are expected to reduce fall-related injuries in older adults. However, physicians over-ride most alerts because they believe the benefit of the drugs exceeds the risk. OBJECTIVE: To determine whether computerized prescribing decision support with patient-specific risk estimates would increase physician response to psychotropic drug alerts and reduce injury risk in older people. DESIGN: Cluster randomized controlled trial of 81 family physicians and 5628 of their patients aged 65 and older who were prescribed psychotropic medication. INTERVENTION: Intervention physicians received information about patient-specific risk of injury computed at the time of each visit using statistical models of non-modifiable risk factors and psychotropic drug doses. Risk thermometers presented changes in absolute and relative risk with each change in drug treatment. Control physicians received commercial drug alerts. MAIN OUTCOME MEASURES: Injury risk at the end of follow-up based on psychotropic drug doses and non-modifiable risk factors. Electronic health records and provincial insurance administrative data were used to measure outcomes. RESULTS: Mean patient age was 75.2 years. Baseline risk of injury was 3.94 per 100 patients per year. Intermediate-acting benzodiazepines (56.2%) were the most common psychotropic drug. Intervention physicians reviewed therapy in 83.3% of visits and modified therapy in 24.6%. The intervention reduced the risk of injury by 1.7 injuries per 1000 patients (95% CI 0.2/1000 to 3.2/1000; p=0.02). The effect of the intervention was greater for patients with higher baseline risks of injury (p<0.03). CONCLUSION: Patient-specific risk estimates provide an effective method of reducing the risk of injury for high-risk older people
- Tisdale J, Wroblewski H, Kingery J, et al. EFFECTIVENESS OF A CLINICAL DECISION SUPPORT SYSTEM INCORPORATING A VALIDATED QT INTERVAL PROLONGATION RISK SCORE FOR REDUCING THE RISK OF QT INTERVAL PROLONGATION IN HOSPITALIZED PATIENTS. Journal of the American College of Cardiology. 2012;59(13):E1799.
- Tisdale J, Wroblewski H, Kingery J, et al. EFFECTIVENESS OF A CLINICAL DECISION SUPPORT SYSTEM INCORPORATING A VALIDATED QT INTERVAL PROLONGATION RISK SCORE FOR REDUCING THE RISK OF QT INTERVAL PROLONGATION IN HOSPITALIZED PATIENTS. Journal of the American College of Cardiology. 2012;59(13):E1799.
- Tisdale J, Wroblewski H, Kingery J, et al. EFFECTIVENESS OF A CLINICAL DECISION SUPPORT SYSTEM INCORPORATING A VALIDATED QT INTERVAL PROLONGATION RISK SCORE FOR REDUCING THE RISK OF QT INTERVAL PROLONGATION IN HOSPITALIZED PATIENTS. Journal of the American College of Cardiology. 2012;59(13):E1799.
- Tisdale J, Wroblewski H, Kingery J, et al. EFFECTIVENESS OF A CLINICAL DECISION SUPPORT SYSTEM INCORPORATING A VALIDATED QT INTERVAL PROLONGATION RISK SCORE FOR REDUCING THE RISK OF QT INTERVAL PROLONGATION IN HOSPITALIZED PATIENTS. Journal of the American College of Cardiology. 2012;59(13):E1799.
- Russ AL, Zillich AJ, Melton BL, Russell SA, Chen S, Spina JR, Weiner M, Johnson EG, Daggy JK, McManus MS, Hawsey JM, Puleo AG, Doebbeling BN, Saleem JJ. Applying human factors principles to alert design increases efficiency and reduces prescribing errors in a scenario-based simulation. J Am Med Inform Assoc. 2014 Oct;21(e2):e287-96. doi: 10.1136/amiajnl-2013-002045. Epub 2014 Mar 25. PubMed PMID: 24668841; PubMed Central PMCID: PMC4173163. To apply human factors engineering principles to improve alert interface design. We hypothesized that incorporating human factors principles into alerts would improve usability, reduce workload for prescribers, and reduce prescribing errors. MATERIALS AND METHODS: We performed a scenario-based simulation study using a counterbalanced, crossover design with 20 Veterans Affairs prescribers to compare original versus redesigned alerts. We redesigned drug-allergy, drug-drug interaction, and drug-disease alerts based upon human factors principles. We assessed usability (learnability of redesign, efficiency, satisfaction, and usability errors), perceived workload, and prescribing errors. RESULTS: Although prescribers received no training on the design changes, prescribers were able to resolve redesigned alerts more efficiently (median (IQR): 56 (47) s) compared to the original alerts (85 (71) s; p=0.015). In addition, prescribers rated redesigned alerts significantly higher than original alerts across several dimensions of satisfaction. Redesigned alerts led to a modest but significant reduction in workload (p=0.042) and significantly reduced the number of prescribing errors per prescriber (median (range): 2 (1-5) compared to original alerts: 4 (1-7); p=0.024). DISCUSSION: Aspects of the redesigned alerts that likely contributed to better prescribing include design modifications that reduced usability-related errors, providing clinical data closer to the point of decision, and displaying alert text in a tabular format. Displaying alert text in a tabular format may help prescribers extract information quickly and thereby increase responsiveness to alerts. CONCLUSIONS: This simulation study provides evidence that applying human factors design principles to medication alerts can improve usability and prescribing outcomes
- We are currently studying this question, observing and interviewing doctors to get at what are the drivers of adherence to collegial advice around drug prescribing
- We stop because it’s the law, because we fear getting caught. Do we trust the sign, that there is something we need to stop for? What if your kid put a stop sign in your dirveway? High risk area, lots of kids potentially running around. Would you stop?