Challenges in understanding clinical notes: Why NLP engines fall short and where background knowledge can help.
•Download as PPTX, PDF•
1 like•259 views
This document discusses the challenges of using natural language processing (NLP) to understand unstructured clinical notes. It notes that 80% of patient data is unstructured, and key patient information resides within notes. However, NLP engines fall short when understanding constructs like negation, certainty, and conditionals. The document proposes using a domain knowledge base to help resolve conflicting instances from linguistic constructs and infer implicit relationships between entities like symptoms and medications. It evaluates the approach on 25 documents and shows an accuracy of 71.87% in resolving 32 conflicting instances.
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Challenges in understanding clinical notes: Why NLP engines fall short and where background knowledge can help.
Similar to Challenges in understanding clinical notes: Why NLP engines fall short and where background knowledge can help. (20)
Recently uploaded
Recently uploaded (20)
Challenges in understanding clinical notes: Why NLP engines fall short and where background knowledge can help.
- 1. Challenges in Understanding Clinical Notes: Why NLP Engines Fall Short and Where Background Knowledge Can Help Sujan Perera, Amit Sheth, Krishnaprasad Thirunarayan Ohio Center of Excellence in Knowledge-enabled Computing (Kno.e.sis) Wright State University Suhas Nair, Neil Shah ezDI LLC
- 2. Why is it necessary to understand clinical notes? • 80% of patient data is unstructured1 • Structured data is incomplete and not accurate2,3 1goo.gl/abqYYn 2Strengths and Limitations of CMS Administrative Data in Research 3Comparison of clinical and administrative data sources for hospital coronary artery bypass graft surgery report cards
- 3. Why is it necessary to understand clinical notes? • Key indicators for decision making reside in patient notes • facilities • “Holter monitor was ordered by Lisa. She failed to get this because she did not have transportation” • non-compliance • “Atrial fibrillation with poorly controlled ventricular rate due to noncompliance.” • financial status • “The patient mentioned that Bystolic is expensive and cannot afford it now.” • family history • “his father is hypertensive”
- 4. • ICD10 adaptation – need to understand the relationships E08 - Diabetes mellitus due to underlying condition E08.0 - Diabetes mellitus due to underlying condition with hyperosmolarity E08.00 - without nonketotic hyperglycemic-hyperosmolar coma (NKHHC) E08.01 - with coma E08.1 - Diabetes mellitus due to underlying condition with ketoacidosis E08.10 - without coma E08.11 – with coma • The underlying condition can be congenital rubella, Cushing's syndrome, cystic fibrosis, malignant neoplasm, malnutrition, pancreatitis Why is it necessary to understand clinical notes?
- 5. Patient Data Distribution Structured data Unstructured data
- 6. Understanding Clinical Notes • What conditions patient has? • What symptoms patient has? • What medications with what dosage he/she taking? • What are the relationships between entities? • What is patients medical history? • What is his/her family history? • What is patients behavior? • What are his diet and discharge instructions?
- 7. Understanding Clinical Notes - Tasks • Entity identification • she does not state that the reason for her stopping after two blocks is shortness of breath or chest discomfort. • Annotate with standard vocabulary • Shortness of breath – UMLS concept C0013404 • Chest discomfort – UMLS concept C0235710 • Relationship identification • Temporal information extraction • if her symptoms do not change at all, she will go back on the lipitor after two weeks. • Negation detection • he has not required any nitroglycerin. • Certainty detection • She is not sure if she is just depressed or not. • Conditioning detection • if he experiences any chest pain, shortness of breath with exertion or dizziness or syncopal episodes to let us know.
- 8. Understanding Linguistic Constructs • Rule based algorithms are popular • simplicity • low computational cost • Maintains a dictionary of words indicate particular language construct • Negation – no, not, deny, cannot, don’t etc • Simple rules deciding applicability • <negation phrase> * <entity> • <entity> * <negation phrase> • Does not associate with the correct phrase of the sentence • Lead to incorrect output • he did not make the increase on his metoprolol. • his weight has not changed and that his edema is primarily at the end of the day. • Sometimes even associating to the correct phrase is not enough • “I do not have an explanation for this dyspnea.” • “there was no evidence of ischemia." • Common to other constructs (certainty and conditioning)
- 9. Leads to Conflicting Instances • Failure to identify such linguistic constructs leads to conflicting instances • Document 1: • Coronary artery disease listed in the current diagnosis list • “Send for carotid duplex to rule out carotid artery stenosis given his risk factors and underlying coronary artery dis- ease.“ • Document 2: • “Extremities : Warm and dry. No clubbing or cyanosis. No lower extremity edema.“ • “I have advised the patient on the side effect of potential lower extremity edema.“ • Document 3 • “He is not having any symptoms of chest pain or exertional syncope or dizziness.” • “I advised him that if he experiences chest pain, shortness of breath with exertion or dizziness or syncopal episodes to let us know and we can do appropriate workup.” • 620 instances within 3172 documents
- 10. Solution • Our method attempts to resolve the conflicts by understanding other observations by leveraging coded domain knowledge Atrial FibrillationSyncope Is_symptom_of Warfarin Atenolol AspirinIs_medication_for Symptoms Medication Medication Medication • We used only medication information because extraction algorithms performs well with its list structure.
- 11. Knowledge Base Domains Cardiology Orthopedics Oncology Neurology Etc… Concepts 1008161 Problems(diseases, symptoms) 125778 Procedures 262360 Medicines 298993 Medical Devices 33124 Relationships 77261 is treated with (disease -> medication) 41182 is relevant procedure (procedure -> disease) 3352 is symptom of (symptom -> disease) 8299 contraindicated drug (medication -> disease) 24428
- 12. Knowledge Base Statin simvastatin zocor High Cholesterol vytorin Pravachol Pravastatin • We use medication hierarchy to find the relationships Is_medication_for Type_of
- 13. Evaluation Predicted Class Positive Negative Actual Class Positive 18 6 Negative 3 5 Accuracy = TP + TN TP + TN + FP + FN = 71.87% • 25 Documents • 32 conflicting instances
- 14. Observations and Insights • False Negatives are for common symptoms • headache, obesity, shortness of breath • Doctors may not prescribe medications for these symptoms • False Positives based on common medications • Aspirin • Conflicts on major conditions can resolve accurately • Coronary artery disease, Atrial fibrillation, Peripheral vascular disease, Ischemia, Cardiomyopathy etc • Patient should take medications for these conditions
- 15. Observations and Insights • The evidences should be ranked • Metoprolol is strong evidence for hypertension than aspirin • Insulin is strong evidence for diabetics • More sophisticated evidence aggregation method should be used • Rule-based • Probabilistic method
- 16. Beyond Conflicts • Populate relationships among the entities • Which medications are associated with which condition • Derive implicit information in the patient notes • Patient notes can be incomplete • Domain experts read the note and can understand beyond what is written there • This insights are important for prediction algorithms • Knowledge driven inferencing can be used to fill this gap
- 17. Thank You Perera, Sujan, Amit Sheth, Krishnaprasad Thirunarayan, Suhas Nair, and Neil Shah. "Challenges in understanding clinical notes: Why nlp engines fall short and where background knowledge can help." In Proceedings of the 2013 international workshop on Data management & analytics for healthcare, pp. 21- 26. ACM, 2013. PDF
Editor's Notes
- Measure Set: Heart Failure (HF) Set Measure ID#: HF-3 Performance Measure Name: ACEI or ARB for LVSD Description: Heart failure patients with left ventricular systolic dysfunction (LVSD) who are prescribed an ACEI or ARB at hospital discharge. For purposes of this measure, LVSD is defined as chart documentation of a left ventricular ejection fraction (LVEF) less than 40% or a narrative description of left ventricular systolic (LVS) function consistent with moderate or severe systolic dysfunction. Rationale: ACE inhibitors reduce mortality and morbidity in patients with heart failure and left ventricular systolic dysfunction (The SOLVD Investigators, 1991 and CONSENSUS Trial Study Group, 1987) and are effective in a wide range of patients (Masoudi, 2004). Clinical trials have also established ARB therapy as an acceptable alternative to ACEI, especially in patients who are ACEI intolerant (Granger, 2003 and Pfeffer, 2003). National guidelines strongly recommend ACEIs for patients hospitalized with heart failure (Jessup, 2009 and HFSA, 2010). Guideline committees have also supported the inclusion of ARBs in performance measures for heart failure (Executive Council of the Heart Failure Society of America, 2004).