Disambiguation  of Biomedical Text Mark Stevenson Natural Language Processing Group   University of Sheffield, UK http://w...
Outline <ul><li>Ambiguity in biomedical documents </li></ul><ul><li>Disambiguation </li></ul><ul><ul><li>Knowledge sources...
Text in Biomedical Domain <ul><li>The literature on biomedicine and the life sciences is vast and growing rapidly </li></u...
Ambiguity <ul><li>Lexical ambiguity makes text processing more difficult </li></ul><ul><li>Generally believed that ambigui...
cell
culture “ In peripheral blood mononuclear cell  culture  streptococcal erythrogenic toxins are able to stimulate tryptopha...
Extent of Ambiguity Problem <ul><li>Weeber et. al. (2001) </li></ul><ul><ul><li>Estimated that 11.7% of the phrases in abs...
WSD System <ul><li>Supervised learning approach </li></ul><ul><ul><li>Extension of Basque Country University’s Senseval-3 ...
Features <ul><li>General </li></ul><ul><ul><li>Wide range of features which are commonly used by WSD systems </li></ul></u...
Example <ul><li>“ Body surface area  adjustments  of initial heparin dosing …” </li></ul><ul><ul><li>Individual Adjustment...
General Features (1) <ul><li>Local collocations </li></ul><ul><ul><li>Bigrams and trigrams containing ambiguous word const...
General Features (2) <ul><li>Syntactic dependencies </li></ul><ul><ul><li>Five relations: subject, object, noun-modifier, ...
Concept Unique Identifiers (CUIs) <ul><li>CUIs refer to UMLS concepts </li></ul><ul><li>MetaMap segments text and identifi...
Medical Subject Headings (MeSH) <ul><li>Controlled vocabulary for indexing life science publications </li></ul><ul><li>Con...
Learning Algorithms <ul><li>Vector Space Model </li></ul><ul><ul><ul><li>Simple memory-based learning algorithm  </li></ul...
NLM-WSD data set <ul><li>Standard evaluation corpus for WSD in biomedical domain (“Biomedical SemEval”) </li></ul><ul><li>...
Results <ul><li>Combination of linguistic features with MeSH terms significantly better than any features used alone </li>...
cold depression discharge extraction fat implantation japanese lead  mole  pathology  reduction  sex  ultrasound degree gr...
Approach MFS Liu  et. al. (2004) Leroy & Rindflesch (2005) Joshi  et. al. (2005) McInnes  et. al.  (2007) Reported (Genera...
Automatic Example Generation <ul><li>Various approaches to generating sense tagged examples without the need for manual an...
Relevance Feedback <ul><li>Method for improving search results based on analysis of retrieved documents </li></ul>~~~~~~~~...
<ul><li>Common approach to relevance feedback for vector space model (Rocchio, 1971) </li></ul>q m  = modified query vecto...
Acquiring Sense Tagged Examples <ul><li>Treat set of sense tagged examples as retrieved documents </li></ul><ul><ul><li>Ex...
Identifying Query Terms <ul><li>Compute score for each term in the sense-tagged documents against each sense </li></ul>cou...
Terms for two senses of “culture” ‘ anthropological culture’ ‘ laboratory culture’ cultural 26.17 suggest 6.32 recommendat...
Example Collection <ul><li>Identify examples by querying Medline via online interface </li></ul><ul><li>Preserve bias in o...
Experiments <ul><li>10-fold cross validation </li></ul><ul><ul><li>Training portion (90 examples) analysed to generate add...
Performance Basic 87.9 Combined +90 +180 +270 +360 89.6 88.6 88.0 88.0 Additional only 90 180 270 360 88.4 87.9 87.5 87.3
Individual Terms term basic 90 difference blood pressure 53 66 13 reduction 88 96 8 repair 86 92 6 mole 88 94 6 ultrasound...
Conclusion <ul><li>Ambiguity real problem in biomedical domain </li></ul><ul><li>Domain specific knowledge improves WSD pe...
More Information <ul><li>This work has been funded EPSRC grants BioWSD and CASTLE </li></ul><ul><ul><li>http://nlp.shef.ac...
 
Upcoming SlideShare
Loading in …5
×

Slides

195
-1

Published on

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
195
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Cells in culture, stained for keratin (red) and DNA (green)
  • Cells in culture, stained for keratin (red) and DNA (green)
  • Slides

    1. 1. Disambiguation of Biomedical Text Mark Stevenson Natural Language Processing Group University of Sheffield, UK http://www.dcs.shef.ac.uk/~marks Joint work with: Yikun Guo and Robert Gaizauskas (University of Sheffield) and David Martinez (University of Melbourne)
    2. 2. Outline <ul><li>Ambiguity in biomedical documents </li></ul><ul><li>Disambiguation </li></ul><ul><ul><li>Knowledge sources </li></ul></ul><ul><li>Evaluation </li></ul><ul><li>Semi-supervised acquisition of additional training data </li></ul>
    3. 3. Text in Biomedical Domain <ul><li>The literature on biomedicine and the life sciences is vast and growing rapidly </li></ul><ul><li>Promising domain for text processing </li></ul><ul><ul><li>Search engines necessary </li></ul></ul><ul><ul><li>Opportunities for knowledge discovery </li></ul></ul>
    4. 4. Ambiguity <ul><li>Lexical ambiguity makes text processing more difficult </li></ul><ul><li>Generally believed that ambiguities do not occur with domains </li></ul><ul><ul><li>One Sense per Discourse (Gale, Church and Yarowsky, 1992) </li></ul></ul><ul><ul><li>“ there is a very strong tendency (98%) for multiple uses of a word to share the same sense in a well-written discourse” </li></ul></ul>
    5. 5. cell
    6. 6. culture “ In peripheral blood mononuclear cell culture streptococcal erythrogenic toxins are able to stimulate tryptophan degradation in humans.” International Allergy Immunology “ The aim of this paper is to describe the origins, initial steps and strategy, current progress and main accomplishments of introducing a quality management culture within the healthcare system in Poland.” International Journal of Qualitative Health Care
    7. 7. Extent of Ambiguity Problem <ul><li>Weeber et. al. (2001) </li></ul><ul><ul><li>Estimated that 11.7% of the phrases in abstracts added to MEDLINE in 1998 were ambiguous </li></ul></ul><ul><li>Ambiguity is biggest challenge in automation of indexing MEDLINE and a hindrance to automated knowledge discovery (Weeber et. al. 2001)(Nadkarin et. al. 2001)(Aronson 2001) </li></ul>
    8. 8. WSD System <ul><li>Supervised learning approach </li></ul><ul><ul><li>Extension of Basque Country University’s Senseval-3 system (Agirre and Martinez, 2004) </li></ul></ul><ul><li>Combines range of knowledge sources </li></ul><ul><ul><li>Previous work shown that combining knowledge sources is an effective approach to WSD </li></ul></ul>
    9. 9. Features <ul><li>General </li></ul><ul><ul><li>Wide range of features which are commonly used by WSD systems </li></ul></ul><ul><li>Domain specific </li></ul><ul><ul><li>Two knowledge sources specific to biomedical domain </li></ul></ul>
    10. 10. Example <ul><li>“ Body surface area adjustments of initial heparin dosing …” </li></ul><ul><ul><li>Individual Adjustment </li></ul></ul><ul><ul><ul><li>“ By the fast (2.5mph) ambulation trial, both groups were performing equally, suggesting a rapid rate of adjustment to the device.” </li></ul></ul></ul><ul><ul><li>Adjustment Action </li></ul></ul><ul><ul><ul><li>“ Clinically, these four patients had mild symptoms which improved with dietary adjustment .” </li></ul></ul></ul><ul><ul><li>Psychological adjustment </li></ul></ul><ul><ul><ul><li>“ Predictors of patients' mental adjustment to cancer: patient characteristics and social support.” </li></ul></ul></ul>
    11. 11. General Features (1) <ul><li>Local collocations </li></ul><ul><ul><li>Bigrams and trigrams containing ambiguous word constructed from lemmas, word forms and PoS tags </li></ul></ul><ul><ul><ul><li>left-content-word-lemma “ area adjustment ” </li></ul></ul></ul><ul><ul><ul><li>right-function-word-lemma “ adjustment of '' </li></ul></ul></ul><ul><ul><ul><li>left-POS “ NN NNS ” </li></ul></ul></ul><ul><ul><ul><li>right-POS “ NNS IN ” </li></ul></ul></ul><ul><ul><ul><li>left-content-word-form “ area adjustments ” </li></ul></ul></ul><ul><ul><ul><li>right-function-word-form “ adjustment of ” </li></ul></ul></ul><ul><ul><li>First noun, verb, adjective and adverb preceding and following ambiguous word (lemma and word form) </li></ul></ul>
    12. 12. General Features (2) <ul><li>Syntactic dependencies </li></ul><ul><ul><li>Five relations: subject, object, noun-modifier, preposition and sibling </li></ul></ul><ul><li>Salient bigrams </li></ul><ul><ul><li>Salient bigrams in abstract </li></ul></ul><ul><li>Unigrams </li></ul><ul><ul><li>Lemmas of all content words in the abstract and 8 word window around target word </li></ul></ul><ul><ul><li>Lemmas of unigrams which appear frequently in entire corpus </li></ul></ul>
    13. 13. Concept Unique Identifiers (CUIs) <ul><li>CUIs refer to UMLS concepts </li></ul><ul><li>MetaMap segments text and identifies possible CUIs for each phrase </li></ul>&quot;Body surface area adjustments&quot; C0005902:Body Surface Area [Diagnostic Procedure] C1261466:Body surface area [Organism Attribute] C0456081:Adjustments (Adjustment Action) [Health Care Activity] C0376209:Adjustments (Individual Adjustment) [Individual Behavior] &quot;of initial heparin dosing&quot; C0205265:Initial (Initially) [Temporal Concept] C1555582:initial [Idea or Concept] C0019134:Heparin [Biologically Active Substance,Carbohydrate]
    14. 14. Medical Subject Headings (MeSH) <ul><li>Controlled vocabulary for indexing life science publications </li></ul><ul><li>Contains over 24,000 headings organised into an 11 level hierarchy </li></ul><ul><li>Use MeSH terms assigned to abstract containing ambiguous term </li></ul><ul><ul><li>M01.060.116.100: “Aged” </li></ul></ul><ul><ul><li>M01.060.116.100.080: “Aged, 80 and over” </li></ul></ul><ul><ul><li>D27.505.954.502.119: “Anticoagulants” </li></ul></ul><ul><ul><li>G09.188.261.560.150: “Blood Coagulation” </li></ul></ul>
    15. 15. Learning Algorithms <ul><li>Vector Space Model </li></ul><ul><ul><ul><li>Simple memory-based learning algorithm </li></ul></ul></ul><ul><li>Naïve Bayes </li></ul><ul><li>Support Vector Machine </li></ul><ul><ul><ul><li>Weka implementations </li></ul></ul></ul>
    16. 16. NLM-WSD data set <ul><li>Standard evaluation corpus for WSD in biomedical domain (“Biomedical SemEval”) </li></ul><ul><li>Contains highly 50 ambiguous terms frequently found in Medline </li></ul><ul><li>100 instances of each term manually disambiguated with UMLS concepts by a team of annotators </li></ul><ul><li>Baseline (MFS) accuracy of 78% </li></ul><ul><li>Average of 2.64 possible meanings per term </li></ul>
    17. 17. Results <ul><li>Combination of linguistic features with MeSH terms significantly better than any features used alone </li></ul><ul><li>VSM significantly better than other learning algorithms </li></ul>General CUI MeSH CUI+ MeSH Ling + MeSH Ling + CUI All VSM 87.0 85.8 81.9 86.9 87.9 87.3 87.5 NB 86.4 81.2 85.7 81.1 86.4 81.7 81.8 SVM 85.9 83.5 85.3 84.5 86.2 85.3 86.0
    18. 18. cold depression discharge extraction fat implantation japanese lead mole pathology reduction sex ultrasound degree growth man mosaic nutrition repair scale weight white adjustment blood pressure evaluation immunosuppression radiation sensitivity association condition culture determination energy failure fit fluide frequency ganglion glucose inhibition pressure resistance secretion single strains support surgery transient transport variation Liu et. al. (2004) Leroy and Rindflesch (2005) Joshi et. al. (2005) Common Dominant sense < 90% Removed low IAA Dominant sense < 65%
    19. 19. Approach MFS Liu et. al. (2004) Leroy & Rindflesch (2005) Joshi et. al. (2005) McInnes et. al. (2007) Reported (General + MeSH) All words 78.0 85.3 87.9 Joshi 66.9 82.5 80.0 83.3 Leroy 55.3 65.5 77.4 74.5 79.7 Liu 69.9 78.0 84.9 82.0 84.8 Common 54.9 68.8 79.8 75.7 81.1
    20. 20. Automatic Example Generation <ul><li>Various approaches to generating sense tagged examples without the need for manual annotation </li></ul><ul><ul><li>Monosemous relatives (Leacock et. al. 1998) </li></ul></ul><ul><ul><li>Translations as sense definitions (Ng et. al. 2003) </li></ul></ul><ul><li>All unsupervised but require external knowledge sources (e.g. WordNet or parallel text) </li></ul><ul><li>Alternative semi-supervised approach </li></ul>
    21. 21. Relevance Feedback <ul><li>Method for improving search results based on analysis of retrieved documents </li></ul>~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~ <ul><li>Retrieved </li></ul><ul><li>documents </li></ul><ul><li>Relevance </li></ul><ul><li>judgements </li></ul><ul><li>Query </li></ul><ul><li>Modified </li></ul><ul><li>Query </li></ul>~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~ <ul><li>> </li></ul>~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~ ~~~~ <ul><li>> </li></ul><ul><li>> </li></ul>
    22. 22. <ul><li>Common approach to relevance feedback for vector space model (Rocchio, 1971) </li></ul>q m = modified query vector q = original query vector D +q = set of vectors representing known relevant documents D -q = set of vectors representing known irrelevant documents α , β , γ = weights
    23. 23. Acquiring Sense Tagged Examples <ul><li>Treat set of sense tagged examples as retrieved documents </li></ul><ul><ul><li>Examples tagged with sense considered relevant, all other examples considered irrelevant </li></ul></ul><ul><li>For each sense, identify additional query terms which tend to discriminate examples tagged with that sense from those tagged with other senses </li></ul><ul><li>Search for documents matching this extended query </li></ul>
    24. 24. Identifying Query Terms <ul><li>Compute score for each term in the sense-tagged documents against each sense </li></ul>count(t,d) = frequency of term t in document d D +s = set of examples of target sense D -s = set of examples of other senses α , β = weights idf(t) = inverse document frequency of t
    25. 25. Terms for two senses of “culture” ‘ anthropological culture’ ‘ laboratory culture’ cultural 26.17 suggest 6.32 recommendation 14.82 protein 6.13 force 14.80 presence 5.86 ethnic 14.79 demonstrate 5.86 practice 14.76 analysis 5.78 man 14.76 gene 5.58
    26. 26. Example Collection <ul><li>Identify examples by querying Medline via online interface </li></ul><ul><li>Preserve bias in original sense distribution </li></ul><ul><ul><li>For example, if 75% usages are ‘laboratory culture’ and 25% ‘anthropological culture’ then ensure same 75:25 split in retrieved examples </li></ul></ul><ul><li>Use eight highest scoring terms (score(t,s)) for each sense </li></ul><ul><li>Relax queries until enough examples can be retrieved: </li></ul><ul><ul><li>culture AND (suggest AND protein AND presence) </li></ul></ul><ul><ul><li>culture AND ((suggest AND protein) OR (suggest AND presence) OR (protein and presence)) </li></ul></ul><ul><ul><li>culture AND (suggest OR protein OR presence) </li></ul></ul>
    27. 27. Experiments <ul><li>10-fold cross validation </li></ul><ul><ul><li>Training portion (90 examples) analysed to generate additional examples </li></ul></ul><ul><ul><li>Generated three sets for each term: 90, 180, 270 and 360 examples </li></ul></ul><ul><li>Combine automatically generated examples with training portion (+90, +180, +270, +360) </li></ul><ul><li>Automatically generated examples alone (90, 180, 270, 360) </li></ul>
    28. 28. Performance Basic 87.9 Combined +90 +180 +270 +360 89.6 88.6 88.0 88.0 Additional only 90 180 270 360 88.4 87.9 87.5 87.3
    29. 29. Individual Terms term basic 90 difference blood pressure 53 66 13 reduction 88 96 8 repair 86 92 6 mole 88 94 6 ultrasound 88 94 6 white 81 72 -9 weight 82 71 -11 degree 93 81 -12 evaluation 81 69 -12
    30. 30. Conclusion <ul><li>Ambiguity real problem in biomedical domain </li></ul><ul><li>Domain specific knowledge improves WSD performance </li></ul><ul><li>Relevance feedback can be used to acquire additional training examples and further improve performance </li></ul>
    31. 31. More Information <ul><li>This work has been funded EPSRC grants BioWSD and CASTLE </li></ul><ul><ul><li>http://nlp.shef.ac.uk/BioWSD/ </li></ul></ul>
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×