Successfully reported this slideshow.

Amia06

444 views

Published on

Published in: Health & Medicine, Technology
  • Be the first to comment

  • Be the first to like this

Amia06

  1. 1. A Comparative Study of Supervised Learning as Applied to Acronym Expansion in Clinical Reports Mahesh Joshi, Serguei Pakhomov, Ted Pedersen, Christopher G. Chute University of Minnesota, Duluth Mayo College of Medicine, Rochester
  2. 2. Overview <ul><li>Acronyms are ambiguous </li></ul><ul><ul><li>in general, and in more specialized domains </li></ul></ul><ul><li>Acronyms can be disambiguated by expansion </li></ul><ul><ul><li>expansions act as senses or definitions </li></ul></ul><ul><li>Acronym expansion can be viewed as word sense disambiguation </li></ul><ul><ul><li>supervised learning from annotated examples </li></ul></ul><ul><li>Features trump learning algorithms </li></ul><ul><ul><li>unigrams dominant </li></ul></ul>
  3. 3. AMIA - Top Google Results <ul><li>American Medical Informatics Association </li></ul><ul><li>Association of Moving Image Archivists </li></ul><ul><li>Anglican Mission in America </li></ul><ul><li>Associcion Mutual Israelita Argentina </li></ul>
  4. 4. RN in Wikipedia <ul><li>Registered Nurse </li></ul><ul><li>Royal Navy </li></ul><ul><li>Radio National </li></ul><ul><li>Radio Nederland </li></ul><ul><li>Richard Nixon </li></ul><ul><li>Registered Identification Number </li></ul><ul><li>Renovacion Nacional </li></ul>
  5. 5. Acronym Ambiguity not just a problem for General English… <ul><li>33% of Acronyms in UMLS are ambiguous </li></ul><ul><ul><li>Liu et. al. AMIA-2001 </li></ul></ul><ul><li>81% of Acronyms in MEDLINE abstracts are ambiguous, with an average of 16 expansions </li></ul><ul><ul><li>Liu et. al. AMIA-2002 </li></ul></ul>
  6. 6. We view AE as WSD <ul><li>AE </li></ul><ul><ul><li>sense 1: American Eagle </li></ul></ul><ul><ul><li>sense 2: Arab Emirates </li></ul></ul><ul><ul><li>sense 3: acronym expansion </li></ul></ul><ul><li>WSD </li></ul><ul><ul><li>sense 1: Washington School for the Deaf </li></ul></ul><ul><ul><li>sense 2: web server director </li></ul></ul><ul><ul><li>sense 3: word sense disambiguation </li></ul></ul>
  7. 7. Methodology <ul><li>Identify 16 ambiguous acronyms </li></ul><ul><ul><li>9 from Pakhomov, et. al. AMIA-2005 </li></ul></ul><ul><ul><li>7 newly annotated for this this study </li></ul></ul><ul><li>Manually annotate in clinical notes </li></ul><ul><ul><li>7,738 total instances from Mayo Clinic database of clinical notes </li></ul></ul><ul><li>Use as training data for supervised learning </li></ul>
  8. 8. Acronyms (majority < 50%) <ul><li>AC </li></ul><ul><ul><li>Acromioclavicular </li></ul></ul><ul><ul><li>Antitussive with Codeine </li></ul></ul><ul><ul><li>Acid Controller </li></ul></ul><ul><ul><li>10 more </li></ul></ul><ul><li>APC </li></ul><ul><ul><li>Argon Plasma Coagulation </li></ul></ul><ul><ul><li>Adenomatous Polyposis Coli </li></ul></ul><ul><ul><li>Atrial Premature Contraction </li></ul></ul><ul><ul><li>10 more expansions </li></ul></ul><ul><li>LE </li></ul><ul><ul><li>Limited Exam Lower Extremity </li></ul></ul><ul><ul><li>Initials </li></ul></ul><ul><ul><li>5 more expansions </li></ul></ul><ul><li>PE </li></ul><ul><ul><li>Pulmonary Embolism </li></ul></ul><ul><ul><li>Pressure Equalizing </li></ul></ul><ul><ul><li>Patient Education </li></ul></ul><ul><ul><li>12 more expansions </li></ul></ul>
  9. 9. Acronyms (50% < majority < 80%) <ul><li>CP </li></ul><ul><ul><li>Chest Pain </li></ul></ul><ul><ul><li>Cerebral Palsy </li></ul></ul><ul><ul><li>Cerebellopontine </li></ul></ul><ul><ul><li>19 more expansions </li></ul></ul><ul><li>HD </li></ul><ul><ul><li>Huntington's Disease </li></ul></ul><ul><ul><li>Hemodialysis </li></ul></ul><ul><ul><li>Hospital Day </li></ul></ul><ul><ul><li>9 more expansions </li></ul></ul><ul><li>CF </li></ul><ul><ul><li>Cystic Fibrosis </li></ul></ul><ul><ul><li>Cold Formula </li></ul></ul><ul><ul><li>Complement Fixation </li></ul></ul><ul><ul><li>6 more expansions </li></ul></ul><ul><li>MCI </li></ul><ul><ul><li>Mild Cognitive Impairment </li></ul></ul><ul><ul><li>Methylchloroisothiazolinone </li></ul></ul><ul><ul><li>Microwave Communications, Inc. </li></ul></ul><ul><ul><li>5 more expansions </li></ul></ul><ul><li>ID </li></ul><ul><ul><li>Infectious Disease </li></ul></ul><ul><ul><li>Identification </li></ul></ul><ul><ul><li>Idaho Identified </li></ul></ul><ul><ul><li>4 more expansions </li></ul></ul><ul><li>LA </li></ul><ul><ul><li>Long Acting </li></ul></ul><ul><ul><li>Person </li></ul></ul><ul><ul><li>Left Atrium </li></ul></ul><ul><ul><li>5 more expansions </li></ul></ul>
  10. 10. Acronyms (majority > 80%) <ul><li>MI </li></ul><ul><ul><li>Myocardial Infarction </li></ul></ul><ul><ul><li>Michigan </li></ul></ul><ul><ul><li>Unknown </li></ul></ul><ul><ul><li>2 more expansions </li></ul></ul><ul><li>ACA </li></ul><ul><ul><li>Adenocarcinoma </li></ul></ul><ul><ul><li>Anterior Cerebral Artery </li></ul></ul><ul><ul><li>Anterior Communication Artery </li></ul></ul><ul><ul><li>3 more expansions </li></ul></ul><ul><li>GE </li></ul><ul><ul><li>Gastroesophageal </li></ul></ul><ul><ul><li>General Exam </li></ul></ul><ul><ul><li>Generose </li></ul></ul><ul><ul><li>General Electric </li></ul></ul><ul><li>HA </li></ul><ul><ul><li>Headache </li></ul></ul><ul><ul><li>Hearing Aid </li></ul></ul><ul><ul><li>Hydroxyapatite </li></ul></ul><ul><ul><li>2 more expansions </li></ul></ul><ul><li>FEN </li></ul><ul><ul><li>Fluids, Electrolytes and Nutrition </li></ul></ul><ul><ul><li>Drug Fen Phen </li></ul></ul><ul><ul><li>Unknown </li></ul></ul><ul><li>NSR </li></ul><ul><ul><li>Normal Sinus Rhythm </li></ul></ul><ul><ul><li>Nasoseptal Reconstruction </li></ul></ul>
  11. 11. Experimental Objectives <ul><li>Compare performance of ML methods </li></ul><ul><ul><li>Naïve Bayesian classifier </li></ul></ul><ul><ul><li>J48/C4.5 decision tree learner </li></ul></ul><ul><ul><li>Support vector machine (SMO) </li></ul></ul><ul><li>Compare four different feature sets </li></ul><ul><ul><li>POS tags from Brill-Hepple Tagger </li></ul></ul><ul><ul><li>Unigrams that occur 5 or more times </li></ul></ul><ul><ul><ul><li>Flexible window of size 5 around target </li></ul></ul></ul><ul><ul><li>Bigrams that occur 5 or more times </li></ul></ul><ul><ul><ul><li>Flexible window of size 5 around target </li></ul></ul></ul><ul><ul><li>Unigrams + Bigrams + POS tags </li></ul></ul>
  12. 12. Feature Extraction <ul><li>Horizon : up to 5 content words to left and right of target </li></ul><ul><li>Boundaries : cross sentences, but not clinical notes </li></ul><ul><li>Skip stop words </li></ul><ul><li>Bigrams are pairs of contiguous content words </li></ul><ul><li>Example (CF is target): </li></ul><ul><ul><li>Unigrams: “if she is found to be a carrier , then they will follow with CF carrier testing in her husband .” </li></ul></ul><ul><ul><li>Bigrams: “if she is found to be a carrier, then they will follow with CF carrier testing in her husband.” </li></ul></ul>
  13. 13. Results (majority < 50%)
  14. 14. Results (50% < majority < 80%)
  15. 15. Results (majority > 80%)
  16. 16. Results (flexible window)
  17. 17. Conclusions <ul><li>Overall expansion accuracy at or above 90% regardless of distribution </li></ul><ul><li>Differences in accuracy are largely due to features, not ML algorithms </li></ul><ul><li>Addition of bigrams and POS tags helps performance, but unigrams dominant </li></ul><ul><li>Flexible window improves upon fixed window feature selection </li></ul>
  18. 18. Future Work <ul><li>Expand all acronyms in a text, not just select few </li></ul><ul><ul><li>expand based on prior expansions </li></ul></ul><ul><ul><li>utilize one sense per discourse constraint </li></ul></ul><ul><li>Integrate supervised methods with knowledge based approaches and clustering methods to reduce need for annotated examples </li></ul>
  19. 19. Acknowledgments <ul><li>We would like to thank our annotators Barbara Abbott, Debra Albrecht and Pauline Funk. </li></ul><ul><li>This work was supported in part by the NLM Training Grant (T15 LM07041-19) and the NIH Roadmap Multidisciplinary Clinical Research Career Development Award (K12/NICHD)-HD49078. </li></ul><ul><li>Dr. Pedersen has been partially supported by a National Science Foundation Faculty Early CAREER Development Award (#0092784). </li></ul>
  20. 20. Software Resources <ul><li>NSPGate (from Duluth/Mayo) </li></ul><ul><ul><li>http://nspgate.sourceforge.net/ </li></ul></ul><ul><li>Ngram Statistics Package (from Duluth) </li></ul><ul><ul><li>http://ngram.sourceforge.net/ </li></ul></ul><ul><li>WSDGate (from Duluth/Mayo) </li></ul><ul><ul><li>http://wsdgate.sourceforge.net/ </li></ul></ul><ul><li>WEKA (from Waikato) </li></ul><ul><ul><li>http://www.cs.waikato.ac.nz/ml/weka/ </li></ul></ul><ul><li>GATE (from Sheffield) </li></ul><ul><ul><li>http://gate.ac.uk/ </li></ul></ul>

×