My Update of our CHIR MRSA project at Steering Committee

1. CHIR MRSA Brad Doebbeling, MD, MSc; Jennifer Garvin, PhD; Mahesh Merchant, PhD; Mike Rubin, MD; Rick Martinello, MD; Mary Goldstein, MD; Phil Foulis, MD; Steve Luther, PhD; PradeepMutalik, PhD; Matt Scotch, PhD, Samson Tu, MS; Susana Martins, MD; Jeff Friedlin, DO; Katalina Gullans, MA Indianapolis, IN; Salt Lake City, UT; Palo Alto, CA; West Haven, CT; Tampa, FL.

3. VA Pittsburgh Reduced MRSA Significantly!

4. VHA National MRSA Reduction Initiative

5. HA Surveillance Time IntensiveStaff need to Focus on Problems, Intervene Early Need for Automated Classification of MRSA Status Interdisciplinary, multisite collaborative extracting structured and unstructured data from electronic health records (EHR) (VHA, METHICILLIN-RESISTANT STAPHYLOCOCCUS AUREUS (MRSA) INITIATIVE, http://www1.va.gov/vhapublications/ViewPublication.asp?pub_ID=1525. 2007.)

6. CHIR MRSA Project Specific Aims Aim 1:Develop, review and refine an ontology for clinically- and epidemiologically-relevant concepts to enable detection of MRSA for reporting purposes and gather requirements for developing a MRSA surveillance and reporting tool Aim 2:Index MRSA-related Concepts in Clinical Narrative Aim 3:Clinical Inference and Analysis of MRSA-Related Information Contained in the Medical Record Aim 4:Develop and evaluate a prototype surveillance application that uses automatically processed VA electronic health record data Translational Aim:Evaluate algorithms for making multi-type predictions based on heterogeneous data, using MRSA as a clinical domain

7. Research Questions/Hypotheses Developing an ontology for clinically- & epidemiologically-relevant concepts enables detection of MRSA for reporting purposes Concepts from practice guidelines, interviews, clinical narrative, and machine learning help operationalize use cases Developing an automated MRSA surveillance system is as reliable and valid as human reviewers

8. Methods Identify relevant documents; De-identify documents; Determine document quality; Build a clinical and population-based ontology; Annotate documents; Extract and define MRSA relevant concepts; and Determine clinical relevance and perform inference modeling to identify linguistic elements assoc. with MRSA.

9. ADMITS Patients with a MRSA warning/flag within the last YEAR are placed in isolation pending swab results. ER and Psych do NOT do nasal swabs. Ordered for ALL patients admitted to the hospital. Determination of Colonization v. Infection?? “swabbing technique”  molecular tech monitors the occurrence of PCR inconclusive results. Inconclusive results are run again the next day. If still inconclusive, a resubmit order is issued. Patients go to their unit/ward while awaiting swab results. Nasal Nares DNA Swab (2 swabs taken for each patient) Swabs are run by molecular tech once daily. Swabs are processed at 6am – and results are available to unit by 2:30pm. Capacity to run 96 swabs. Indy averages 40-50 day Units that have a high rate of PCR inconclusive results are given training on swabbing techniques. Patients that are swabbed after 8am on a given-day, their tests are not run until the following day. If a swab is ordered at transfer for a patient that tested POSITIVE at admit, the molecular tech will cancel the order. Positive MRSA  phone call to MRSA coordinator and to unit. Results inputted in to DHCP. (if positive, 2nd swab is placed in salt broth and saved for growth. Saved for files) TRANSFERS DISCHARGE Only patients that tested NEGATIVE at admit are swabbed at transfer Tests only detect “colonization.” There are no tests run for “infection” or determinations made regarding colonization v infection in molecular pathology. Microbiology completes a microbiology test for all patients that tested NEGATIVE at admit. Swabs are ordered by the physician in the transferring unit. Receiving unit ‘should’ also order a swab. Indy working to automate this process. Molecular biology might have two swabs for same patient at transfer. Discharge swab is tied to physician orders. Physicians could use a tool that is “physician timed discharge” for ordering swab – this not used frequently. ALL positive MRSA are reported to Diana and sent to BUGS (printer) Positive MRSA  patient kept in isolation until cleared (negative swab). MRSA swabs at discharge are to capture “conversions” only (what patients contracted MRSA during the hospital stay). Molecular swab testing process completed in same fashion as admit process. Conversion count is attributed to the last unit that swabbed.

10. MRSA Pipeline Diagram *Selected Workflow Data Used De-Identification Identify Relevant Documents Data Extraction Ontology Natural Language Processing Use Cases Clinical Relevance & Inference Modeling Information Extraction Annotation of Documents Documents Ready for NLP *Workflow Data Collected (Observation & Interview) Machine Learning Building Ontology Quality of Documents Feedback provided at each step to previous and all other steps affected by changes Relevant Publications

11. MRSA Ontology Annotation-inform schema and link free text to ontology concepts A source of terms (including synonyms) for natural language processing (NLP) MRSR ontology in Protégé MRSA terms & relationships for NLP Protégé API NLP engine Patient 123456789 MRSA has anatomical location Oral cavity structure …Patient 123456789… MRSR…mouth.. Goldstein, Tu, Martin @ Palo Alto

12. Uses of MRSA Ontology A vocabulary for queries and inference MRSA surveillance application NLP engine …Patient 123456789… MRSR…mouth.. Ontology Queries MRSA-related data VISTA data Inference engines Rules/Influence Diagrams…

13. Results MRSA Pipeline Model Ontology Development Sampling Plan & Approvals Use Cases Draft Annotation Schema Development of system-Name: Automatic Classification of MRSA (ACOM): This NLP system will classify patients who have a positive MRSA culture into those who have an infection and those who are colonized using data from the VHA EMR system. This will provide assistance to the Infection Preventionist (actor). The trigger for this system will be the positive culture or PCR for MRSA (note that this will trigger isolation procedures).

14. Interim Conclusions Communication & FTF meetings key to effective progress & collaboration. First MRSA ontology developed with a clinical, healthcare or population emphasis. Ontology will be used for clinical decision support combining concepts identified in NLP process with other structured data to enhance MRSA surveillance. Will help differentiate between patients colonized from those infected and provide timely information for multiple clinical and decision support purposes. Plan create a state-of-the-art surveillance tool that will help reduce MRSA infections in VHA.

16. VA CHIR-MRSA Use Case Interview conducted at six VA facilities

17. Development of annotation timeline

18. Publication and manuscript development

19. Work with document quality on CHIR MRSA research aims

20. Connecting the CHIR MRSA pipeline – how the subgroups inter-relate

21. Provide leadership and direction for subgroups to attain ‘next steps’ and receive deliverables from other teams to accomplish goals and research aims

22. Test annotation schema with data uploaded to VINCI

25. Details of Accomplishments

27. Incorporation of Drug Terms into the ontology. Mapping of terms to NDF-RT.

28. Ontology Team works with NLP group to facilitate integration. (iterative process).

29. Through domain expert meetings, explore possibilities of collaboration with the Clinical Inference and Modeling project.

31. Enhance versions of GATE, cTakes, and YTEX pipelines to support MRSA use cases

32. Explore methods to identify relevant documents in positive cases and develop methods to use them in identifying specific infections

34. Enhance versions of GATE, cTakes, and YTEX pipelines to support MRSA use cases

35. Explore methods to identify relevant documents in positive cases and develop methods to use them in identifying specific infections

38. Project Management Received approval for NDS data access for 51 CHIR MRSA team members Received Indianapolis IRB approval for data extraction of local patient data that will be uploaded to VINCI. Additionally, received local IRB to cover all CHIR MRSA members to access and analyze the Indianapolis patient data that will be stored on VINCI. Led development and refinement of the Use Case Questionnaire. Received Indianapolis IRB approval to conduct all interviews from Indianapolis. Conducted initial Use Case Questionnaire interviews with 4 key informants. Scheduled interviews with MICU physician and SICU nurse. Planned Face-to-Face Meeting held in Salt Lake City, April 18 – 20th, 2010 Participated in the development of the MRSA Working Use Case and Workflow Analysis AMIA Abstracts; review and edits of all abstracts submitted from CHIR MRSA subgroups Lead weekly CHIR MRSA team calls; responsible for agenda development, minutes, and follow-up Interface with CHIR MRSA Annotation subgroup, Ontology subgroup, Machine Learning subgroup, and NLP subgroup Development of patient use cases utilizing de-identified Indianapolis data for annotation schema testing Developed CHIR MRSA pipeline Presenter in the CHIR MRSA WIP

39. Ontology Accomplishments Scientific Literature and MRSA Surveillance Guidelines Identified. Literature and Guidelines discussed with MRSA team and domain experts. Selected Literature posted to Sharepoint. Identification of Formal MRSA Terms from Literature and Guidelines. Terms added to the ontology in Protégé Environment. Ontology team identified standards to use for concepts in ontology in collaboration with standards team. Addition of SNOMED code to ontology concepts. Terms in Sample MRSA reports from Indianapolis integrated into MRSA ontology. Synonyms added to concepts in ontology using ULMS. Meetings with MRSA clinical domain experts to review and refine MRSA Ontology Terms. Experts are provided Protégé Access to MRSA ontology. MRSA Ontology Version 1* handed off to MRSA NLP team. Ontology team provides guidance to NLP team to integrate Protégé output with GATE. Ontology refined to reflect the latest concepts/terms/synonyms as ontology team is informed. Version 2* to Annotation Team for examination and annotation strategy development. Ontology sent to domain experts, refined to include only MRSA relevant concepts (current version has terms related to other health associated infections). Potential MRSA relevant drug list (antibiotics, etc) sent to domain experts for review. Explore possibilities of integration with Clinical Inference and Modeling project.

40. Machine Learning Accomplishments Domain Expertise: Pathology and Laboratory Scientific literature and MRSA surveillance guidelines identified for MRSA Ontology sub-group. Discussed literature and guidelines with other domain experts and MRSA ontology subgroup team members. Emailed literature to group. Sent Tampa VA MRSA surveillance documents to both Ontology sub-group and larger MRSA project group. Documents posted to SharePoint. Identification of formal MRSA terms from literature and guidelines. Meetings held with MRSA Ontology Subgroup to review and refine MRSA Ontology Terms. Reviewed ontology from MRSA Ontology Subgroup to refine scope of ontology. Provided ongoing consultation to MRSA Ontology sub-group on Pathology and Laboratory testing and reporting related to MRSA. MRSA Staff Interviews Assisted with development of interview questionnaire for infection control staff. Piloted MRSA interview questionnaire. Consulted with project staff regarding feasibility and logistics of conducting face-to-face or telephone interviews with infection control staff. Machine Learning Defined use cases for machine learning of Tampa VA MRSA surveillance data. Presented to MRSA Project Team. Identified and selected 1,200 cases from Tampa VA MRSA surveillance database for data analysis. Identified and extracted 855,244 progress notes from selected cases for use in machine learning analysis. Used lab test dates and administrative data to reduce number of progress notes for analysis to 98,889. Identified progress note types likely to contain MRSA-related information; Presented note types to MRSA Project Team. Conducted post-processing of machine learning results to facilitate interpretability. Provided a preliminary report with results of machine learning analyses to MRSA Project Team and MRSA Ontology Sub-group. Shared case selection and data extraction methodologies to assist with local data extraction efforts at other project sites. Emailed to Indianapolis project management staff. Presented to MRSA project team.

41. NLP Accomplishments Successfully imported the MRSA Ontology into GATE NLP. Created a preliminary NLP pipeline for information extraction including the use of processing resources such as logic rules and gazetteers. Published preliminary pipeline as an executable on MRSA SharepointSite. Developed UIMA/cTakes pipeline for NLP of MRSA documents. Created YTEX, a version of cTakes, for machine learning classification of free text documents. Identified and selected 200 cases from West Haven VA MRSA surveillance database for data analysis. Identified and extracted 17,000 progress notes from selected cases for use in machine learning analysis. These notes are within a specified time window of a positive culture. Domain Expertise: Infection Prevention Scientific literature and MRSA surveillance guidelines identified for MRSA Ontology sub-group. Discussed literature and guidelines with other domain experts and MRSA ontology subgroup team members. Identification of formal MRSA terms from literature and guidelines. Meetings held with MRSA Ontology Subgroup to review and refine MRSA Ontology Terms. Reviewed ontology from MRSA Ontology Subgroup to refine scope of ontology. Provided ongoing consultation to MRSA Ontology sub-group on Infection Prevention