Leveraging Advanced Analytics to Help Hospitals Measure Efficacy of Treatment Standards and Improve Detection of Healthcare Associated Infections
•
5 likes•2,936 views
Ontological Framework for EHR Analytics Learn more at http://www.boozallen.com/analytics
Recommended
Recommended
More Related Content
Similar to Leveraging Advanced Analytics to Help Hospitals Measure Efficacy of Treatment Standards and Improve Detection of Healthcare Associated Infections
Similar to Leveraging Advanced Analytics to Help Hospitals Measure Efficacy of Treatment Standards and Improve Detection of Healthcare Associated Infections (20)
More from Booz Allen Hamilton
More from Booz Allen Hamilton (20)
Recently uploaded
Recently uploaded (20)
Leveraging Advanced Analytics to Help Hospitals Measure Efficacy of Treatment Standards and Improve Detection of Healthcare Associated Infections
- 1. Leveraging Advanced Analytics to Help Hospitals Measure Efficacy of Treatment Standards and Improve Detection of Healthcare Associated Infections Ontological Framework for EHR Analytics Yugal Sharma, Ph.D. Reechik Chatterjee, M.A. Lead Associate, Advanced Analytics Lead Associate, Enterprise Integration This document is confidential and is intended solely for the use and information of the client to whom it is addressed.
- 2. Table of contents • The Challenge • Our Solution • Advanced Analytic Techniques • Results & Applications • Helping Booz Allen’s Clients be Ready for What’s Next 2
- 3. Improving the detection and treatment of Severe Sepsis and Septic Shock is a national healthcare priority • As a result of high mortality rates and an estimated annual cost of $28-$33 billion, the US Department of Health and Human Services has designated the reduction of Healthcare Associated Infections (HAIs) as a major priority. • Severe Sepsis and Septic Shock (S4), a specific type of HAI, has become an area of increased focus due to its extremely high mortality rate, ranging from 30-50%, significant prevalence (> 750k cases diagnosed annually), and high burden of cost (estimated $16.7 billion annually) • The Sisters of Mercy Hospital System (Mercy), with over 400 clinic and hospital locations in the Midwest, has a strong interest in reducing mortality rates associated with S4 and increasing compliance to international standards for treatment of S4 within its hospitals • While Mercy has a vast centralized repository of Electronic Health Records (EHRs), they needed sophisticated analytical expertise to tweeze out the inherent correlations and patterns present in the data to answer their questions
- 4. Table of contents • The Challenge • Our Solution • Advanced Analytic Techniques • Results & Applications • Helping Booz Allen’s Clients be Ready for What’s Next 4
- 5. Booz Allen developed a generalizable framework to address the complex analytical challenges inherent to addressing S4 • Booz Allen assembled a team of analytics, bioinformatics, and clinical experts and met with Mercy to identify the areas of greatest priority and interest for addressing S4: • Compliance Analysis: To measure hospital-level compliance with SSC guidelines and evaluate the impact of compliance on patient mortality • Early Detection Analysis: To mine the data for potential clinical indicators that could lead to early detection of S4 • We obtained 27,000 de-identified patient EHRs across 4 Mercy hospitals containing structured and unstructured data for analysis. Based on the volume of data and complexity of these tasks, we identified the need for a generalizable analytic framework to apply to this challenge, but one that could be easily repurposed for other similar analyses. • Because the analysis of relationships among events across time is critical to understanding the evolution of a disease, as well as determining efficacy of time- dependent treatment guidelines, we developed a generalizable framework to represent patient EHR data as a series of temporal events. This approach was later dubbed the “Event-Centric Ontology” (ECO) framework. • With the ECO framework as a basis, Booz Allen brought its deep analytics and informatics expertise, in conjunction with Mercy’s vast clinical and domain experience in S4 to bear on the Compliance and Early Detection analyses.
- 6. The ECO framework was used as the foundation for the Compliance and Early Detection analyses A B
- 7. Table of contents • The Challenge • Our Solution • Advanced Analytic Techniques • Results & Applications • Helping Booz Allen’s Clients be Ready for What’s Next 7
- 8. ECO framework can be populated with both structured and unstructured EHR data by leveraging NLP techniques At the finest level of analysis, our ECO framework captures individual patient- Ontology level event- instances which are documented in the EHR High-level representation of the domain Natural Language Processing (NLP) of interest aims to extract the entities and events from unstructured text fields which can be matched against standard terminologies enabling further analyses, research, decision support, coding, categorization, and understanding NLP The extracted entities/events can also be Extraction and instantiated as new propositions or facts interpretation of individual in our ECO framework data-elements from a domain The source of unstructured text in our study was primarily in the form of doctor and nurse notes
- 9. Compliance with 6-hour treatment protocol is correlated with lower mortality. 100.0% % Compliance Note that Hospital 2 had a 90.0% very small compliance sample (n=3) and was • The overall compliance for the 6-hour 80.0% % Mortality removed from our analyses treatment protocol in our population was 70.0% 16.9%, aggregated across the 4 hospitals in 60.0% our population, corresponding to an overall 50.0% mortality rate of 45.2% 40.0% 30.0% • For Hospitals 1, 3, and 4, initial analysis 20.0% indicates a strong negative correlation 10.0% between compliance and mortality (r=-0.82) 0.0% Hospital 1 Hospital 3 Hospital 4 • This analysis gives credence to the value of 100.0% the 6-hour treatment protocol and will serve 90.0% as the initial basis of an educational 80.0% campaign to increase compliance across 70.0% Hosp 3 % Mortality hospitals 60.0% 50.0% r = -0.82 (p<0.01) • Current analyses in progress include 40.0% compliance and ICU bed days, and potential 30.0% cost benefit for compliance to individual 20.0% Hosp 1 Hosp 4 bundle elements 10.0% 0.0% 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% % Compliance
- 10. The early detection model performs significantly better than chance at identifying septic patients at risk for developing S4 • Expert judgment was used to identify a starting set of three clinical indicators: Temperature, Heart Rate, and y=0 is defined as no risk of Respiratory Rate developing Severe Sepsis • Our initial model used input from these y=1 is defined as high risk of developing Severe Sepsis indicators at t (point of diagnosis), t-1 (one hour before), and t-2 (two hours before) • Performance on the test set gives a ROC plot with AUC=0.68 (note: AUC=0.50 gives 50% correct classification or equal to chance), giving Sensitivity=0.54 and Sens=0.54 Spec=0.73 Specificity=0.73 @ Threshold=0.73 • The model showed the most significant indicators to be [Temp]t- 1, [HR]t, [Resp]t, [Resp]t-2
- 11. Table of contents • The Challenge • Our Solution • Advanced Analytic Techniques • Results & Applications • Helping Booz Allen’s Clients be Ready for What’s Next 11
- 12. Initial results are promising and help to set the stage for implementation of evidence-based, cost-effective approaches to reducing the severity of and mortality of S4 • Compliance Analysis results demonstrate a strong correlation between increased compliance to SSC guidelines and decreased mortality. Mercy intends to use this data as part of a campaign to increase compliance with treatment guidelines in the Mercy hospital system with the goal of reducing mortality and the burden of cost to due to S4. • Early Detection Analysis results indicate there may be a set of clinical indicators that could be used to identify patients at risk for developing S4, allowing their care to be prioritized. Additional validation will be necessary and these analyses are currently underway. If successful, these results could help to lower S4-related costs by prioritizing the care of high-risk patients earlier, possibly avoiding expensive treatment interventions. • The application of the ECO framework proved a valuable strategy for the analyses mentioned above. Additional NLP work could add further value to the framework by fully leveraging information in unstructured data. The ECO framework has been presented at several conferences as a case study for how to effectively analyze temporal data for measurement of compliance, early detection, or any other time-dependent analysis. 12
- 13. Table of contents • The Challenge • Our Solution • Advanced Analytic Techniques • Results & Applications • Helping Booz Allen’s Clients be Ready for What’s Next 13
- 14. Helping Booz Allen’s Clients Be Ready for What’s Next • A 2007 ruling by the Center for Medicaid and Medicare Services (CMS) limited payment to hospitals for certain preventable hospital-acquired infections HAIs. In 2009, CMS added S4 to the list of conditions covered by this ruling. • As a result, the burden of cost of treating S4 is gradually shifting to providers. Further, if private insurance companies, which typically model their guidelines after CMS, come out with similar policies in the future, the burden to providers will be compounded. • The novel, generalizable ECO framework and results from the Compliance and Early Detection Analyses have helped Mercy to begin the process of increasing compliance to SSC guidelines for treatment of S4. Booz Allen will continue to work with Mercy to inform their overarching educational campaign • This work helps Mercy be ready to address specific compliance issues within their system and to be better informed about the efficacy of various components of the SSC treatment protocols, ultimately helping lower mortality and reduce the burden of cost due to non-reimbursement. • This approach can be used for other hospitals similarly burdened the need to increase compliance to treatment standards for HAIs and other diseases
- 15. Learn More about our Advanced Analytic Capabilities www.boozallen.com/analytics Yugal Sharma, PhD Reechik Chatterjee, M.A. Lead Associate/ Advanced Analytics Lead Associate/ Enterprise Integration sharma_yugal@bah.com chatterjee_reechik@bah.com Phone (301/251-7158) Phone (202/346-9525) This document is confidential and is intended solely for the use and information of the client to whom it is addressed.
Editor's Notes
- [Intro from Bill Thoet]: Hello. I’m Bill Thoet, Senior Vice President and Capability Officer for the Advanced Analytics Center of Excellence at Booz Allen Hamilton. As part of our ongoing series on “The Promise and Power of Advanced Analytics,” I’d like to introduce Lead Associates Yugal Sharma and ReechikChatterjee, focused on Health priorities within our Advanced Analytics and Enterprise Integration groups, respectively. When the Sisters of Mercy Health System wanted to know how to leverage advanced data collection and analysis to reduce client risks, Yugal & Reechik’s teams collaborated to develop new systems to track compliance with international standards of treatment, which in turn enables the client to diagnose potential cases more quickly. With results still ongoing, their solutions provided insights and treatment-related cost savings, freeing money for other patient care and other priorities.
- Thanks and hello! As mentioned, my colleague Reechik and I will be presenting a project that our teams are conducting together with the Sisters of Mercy Hospital System to analyze large volumes of patient Electronic Health Records (or EHRs for short). The goal of this effort was two-fold. The first was to measure compliance and efficacy of treatment standards for a specific type of Healthcare Associated Infection (HAI) known as Severe Sepsis and Septic Shock (or S4 for short). The second goal was to improve detection of S4 by attempting to identify a set of clinical indicators from patient data that can predict individuals at risk for developing Severe Sepsis while in the hospital.
- Sepsis is a potentially fatal medical condition arising from an overwhelming immune response to infection. The ensuing widespread inflammation can result in organ damage, a condition called Severe Sepsis. When the heart becomes damages and blood pressure falls, this condition is referred to as Septic Shock. The number of cases of Severe Sepsis and Septic Shock (S4) have been rising, with more than 750k diagnoses and 215k deaths annually in the US alone. This vastly exceeds the number of US deaths from prostrate cancer, breast cancer, and AIDS combined, with an estimated annual treatment cost of $16.7 billion annually. As a result, improving the detection and treatment of S4 has become a national healthcare priority. In order to address this crisis, an initiative known as the Surviving Sepsis Campaign (SSC for short) was created in 2008 to improve the management, diagnosis, and treatment of S4. SSC guidelines involve a set of treatment protocols to be administered within 6 hours and 24 hours of diagnosis. Measuring hospital-level compliance to these guidelines has traditionally been difficult due to their time-dependent nature, but the recent trend toward capturing patient data in Electronic Health Records (EHRs for short) has created unprecedented opportunities to assess compliance with treatment guidelines and its correlation to patient outcomes (e.g. mortality). The Sisters of Mercy Hospital System (or Mercy for short) is a large hospital system in the Midwest with a strong interest in reducing mortality rates from S4 and increasing compliance to SSC guidelines across its hospitals by demonstrating the efficacy of treatment guidelines with respect to patient outcome. Mercy is particularly suited to address these questions as they on the cutting edge of capturing patient data and have created a sophisticated data warehouse to store patient EHR data across its hospitals in a standardized, central location. While exploring ways to fully harness the complex landscape of available patient data in order to meet their goals, they quickly realized the solution would require sophisticated analytic methodology and resources.
- To address the challenge of S4 in their own hospitals, Mercy partnered with Booz Allen to achieve three objectives: 1) educating hospitals within its system by determining the compliance level of its hospitals with SSC guidelines , 2) assessing efficacy of SSC guidelines by evaluating the impact of compliance on patient mortality within its hospitals, and 3) decrease time to diagnosis of S4 by identifying clinical indicators capable of identifying patients at risk for developing S4 from Sepsis. The results from these efforts will serve as the critical foundation for an educational campaign to validate and increase compliance with SSC guidelines in its hospitals, with the ultimate goal of decreasing the mortality and cost due to S4. We worked closely with Mercy to develop a clear understanding of the priority areas and to develop a solid approach based on our deep analytic and informatics expertise, along with Mercy’s subject matter expertise in S4. Our goal was to develop a generalizable framework capable of addressing both the Compliance and Early Detection analyses, but that could later be repurposed for other similar analyses. Since the Compliance Analysis involved measuring compliance with time-dependent treatment guidelines and the Early Detection Analysis involved analysis of patient events linked in time, a framework emphasizing the temporal nature of the data was necessary. We developed an Event Centric Ontology (or ECO for short) to represent the patient’s stay in the hospital as a series of events linked in time. This representation allows for efficient analysis of high-dimensional data and also provides the required standardization of classes and relationships within the data, giving a robust platform for the Compliance and Early Detection Analyses. We obtained 27,000 unique patient EHRs from 4 Mercy hospitals for analysis, containing both structured and unstructured elements (in the form of doctor and nurse notes). This data was then cast into the ECO framework in preparation for analysis.
- This diagram is an overview of the generalizable ECO framework. As mentioned, elements of a patient record are represented in the ontology as a series of events that are linked in time. From this type of representation, we can easily query the ontology to return events specific to our Compliance Analysis (Box A), such as patients who were given a certain type of antibiotic within a relative timeframe, or events required for our Early Detection Analysis (Box B), such as measurement of vitals signs at given intervals. For this application, ECO was joined with a well-known biomedical ontology known as SNOMED, to provide the clinical classes and class-relationships of content in the EHR data. The ECO-SNOMED framework then served as the basis for the Compliance and Early Detection Analyses.
- In order to fully leverage the information inherent in the patient EHR data, both the structured and unstructured information must be considered. To this end, we are focusing on applying Natural Language Processing (or NLP for short) techniques on the unstructured fields, currently in the form of doctor and nurse notes. NLP serves to extract entities and events from unstructured data which then be matched against standard terminologies (such as SNOMED) to enhance analysis or be instantiated in our ECO framework as new propositions or classes. For example, symptomatic information (e.g. pallor, hyperhidrosis) can be used to enhance our knowledge about a patient at particular point in time. This effort is currently underway to further expand the utility of ECO for our analyses.
- For the Compliance Analysis, the 6-hour treatment protocol was evaluated and initial results show low overall aggregate compliance across hospitals (~17%). The elements of the 6-hour treatment protocol are highly time-dependent and nested conditionally, increasing the complexity of the analysis. The overall mortality rate is high at 45.2%. A breakdown of compliance by hospital shows an inverse relationship between compliance and mortality (top graph). The bottom graph takes this one step further and shows a statistically significant negative correlation between compliance and mortality. This correlation is evidence of the efficacy of 6-hour treatment protocol with respect to mortality. These initial results are an important point and the first step towards creating a foundation for an educational campaign to increase compliance with S4 treatment guidelines within the Mercy hospital system and perhaps beyond. Additional analyses are underway to identify correlations between compliance and additional outcomes, such as ICU bed days, which will serve to inform cost-benefit analysis of treatment protocol implementation.
- For the Early Detection Analysis, we created two populations for analysis. The first population consists of patients diagnosed with Sepsis upon hospital admission that later develop S4 before discharge. The second population consist of patients diagnosed with Sepsis upon hospital admission that do not develop S4 before discharge. These populations were then divided into a training (2/3) and test set (1/3) for our predictive model.Due to the complexity of the data and limited timeline of the study, we chose to leverage expert judgment to define an initial set of clinical indicators to differentiate between our two populations. These indicators were chosen to be Temperature, Heart Rate, and Respiratory Rate.Our initial model used these three indicators, each at 3 timepoints: t (point of the diagnosis of S4), t-1 (one hour before diagnosis), and t-2 (two hours before diagnosis) to produce a risk score for development of S4 from Sepsis. The risk score has values of 0 to 1, with 0 defined as no risk of developing S4 and 1 as high risk of developing S4. Training of the model on our training set indicate the most significant indicators to be [Temp]t-1, [HR]t, [Resp]t, [Resp]t-2. Application of the model to the test set show the model’s predictive power to be significantly better than chance as shown by the ROC plot, with an AUC=0.68 (a 50-50 chance of correct prediction is given by AUC=0.5) The model gave an optimal Sensitivity of 0.54 and Specificity of 0.73. These initial results, though preliminary, could help to identify high-risk patients and prioritize their care, potentially decreasing mortality and associated treatment costs.
- Both the Compliance and Early Detection Analyses have shown promising initial results. These results justify additional analyses to shed additional light on the relationship between compliance and patient outcome, as well as early clinical indicators of S4. More generally, our ECO framework proved to be an effective approach to analyzing this large volume of EHR data cast in a representation as a set of linked temporal events. We are currently presenting this approach at various conferences as a case study for how to effectively analyze temporal data with the goal of evaluating efficacy of time-dependent treatment protocols, identifying early clinical indicators, or other similar time-dependent analyses. Further applications of ECO could involve the analysis of temporal data in longitudinal studies or other similar compliance analyses involving domains other than of S4 for hospitals in general.
- Mercy is a case study for our approach to helping clients be ready for impact of policy on their operations. As costs due to HAIs increase as a result of non-reimbursement, it will be necessary for hospitals to focus on ways to improve detection and lower mortality associated with HAIs. With results from this effort, Mercy will be in a position to improve compliance to treatment guidelines for S4 for hospitals in their system. Booz Allen will continue to work with Mercy as needed to inform their overarching educational campaign.Our generalizable advanced analytic approach to helping Mercy address this issue can be applied to other hospitals and disease domains in an effort to reduce their mortality rates and burden of cost. As a result, we can help our clients be ready for what’s next.