BiTeM / SIBTex @ TREC CDS 2014
•
0 likes•612 views
The document describes strategies used by the BiTeM/SIBtexgroup for the TREC 2014 Clinical Decision Support track. These strategies included using different document representations with text, MeSH terms, and target-specific semantic enrichment. Other strategies were boosting documents based on article type and exploiting the co-citations network. The results showed that the baseline text representation worked best and that most strategies did not improve significantly upon the baseline. Relevant documents were found to discuss tests, diagnoses, and treatments together without distinction.
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to BiTeM / SIBTex @ TREC CDS 2014
Similar to BiTeM / SIBTex @ TREC CDS 2014 (20)
Recently uploaded
Recently uploaded (20)
BiTeM / SIBTex @ TREC CDS 2014
- 1. Full-texts representation with MeSH, co-citations network reranking BiTeM/SIBtexgroup J Gobeill (me), A Gaudinat, E Pascheand P Ruch University of Applied Sciences, SwissInstitute of Bioinformatics, Hospitalsand University of Geneva
- 2. The BiTeM/ SIBtexgroup •TextMining and Bibliomics(P Ruch) Strongfocus on clinicaland biologicaldata heg(training librarians) and SIB (assistingbiocurators) •Long historyof participation in TREC campaigns Genomics, Chemical IR, MedicalRecords… •Translationalmedicineprojects(EU FP7 Programme) Khresmoi: multimodal medicalsearchengine MD-Paedigree: retrievalof similarcases for clinicians
- 3. The CDS Track2014 •ClinicalDecisionSupport : « retrieval of biomedical articles relevant for answering generic clinical questions about medical records » Ex. query: « 25-year-old woman with fatigue,hairloss, weight gain, and cold intolerance for 6 months» Collection: subsetof PubMedCentral
- 4. Strategiesfor TREC CDS 2014 Reranking 4. Boostingbasedon article types 5. Exploitation of the co-citations network Document Representation 1. Classicaldocument representationwithtext 2. Document representationwithMeSH 3. Target-specificsemanticenrichmentwithMeSHIR performedby (Okapi BM25)
- 5. BiTeMofficial results our baseline our baseline
- 6. Creatinga baseline 1. Classicaldocument representationwithtext Textindex Searchengine
- 7. 1. Classicaldocument representationwithtext •Twodifferentindexinglevels: •Document •Section Run2 vs run4 : document > section (+ 65%) •Queryrepresentation(R-Prec): •Numbersremoving(no age) •Onlydescription: 0.169 •Onlysummaries: 0.170 •Both: 0.185 (+10%) Signal/noise ratio: betterwithmore information Document Sections
- 8. Creatinga complementaryview 2. Document representationwithMeSH MeSHindex Searchengine MeSHfor PMC 2649306 D008569 Memory Disorders D001921 Brain D001284 Atrophy D001706 Biopsy D005911 Gliosis
- 9. 2. Document representationwithMeSH •Twopossible sources: •CollectedfromMEDLINE whenthereisa PMID •Extractedfromdocuments witha categorizer(strict mapping) •Twopossible integrationsbetweenoriginal textand MeSH: •Building separateindexes thencombiningruns •Mergingbothrepresentationsintoone unique document
- 10. MeSHconcepts found: D008568 Memory D008569 Memory Disorders D007866 Leg D009068 Movement D001921 Brain D001284 Atrophy D001706 Biopsy D005911 Gliosis <topic number="8"> <summary>62-year-old man withprogressive memory lossand involuntarylegmovements. Brain MRI revealscortical atrophy, and cortical biopsyshows vacuolargray matterchanges withreactiveastrocytosis.</summary> Exampleof MeSHmapping D013035: MuscularSpasm? Somegood (power of synonyms) Somebroad Somemissing(tooambiguous) D002540: CerebralCortex ? D008279: MH = MagneticResonanceImaging ? MedicalResearchInstitute ? ModerateRenalInsufficiency?
- 11. MEDLINE MeSHin docs Humans Animals Female Male Adult Middle Aged Mice Aged Adolescent Molecular Sequence Data Rats Young Adult Time Factors Child Signal Transduction ExtractedMeSHin docs Cells Ficus (becauseof «fig») Patients Time Genes Therapeutics Methods Role Humans Disease Volition Mice Attention DNA Population ExtractedMeSHin topics Women History Pain Blood Physical Examination Female Blood Pressure Pressure Dyspnea Family Thorax Urine Fever Male Emergencies Top 15 MeSHin benchmark
- 12. Resultsfor MeSHrepresentation •Best R-Prec0.143 for MeSHrepresentation(vs 0.211 for text) oMeSHconcepts collectedfromMEDLINE not useful(best R-Prec0.028) oOnly 53%of documents had MeSHterms in MEDLINE •Complementarityfor findingrelevant documents (thanksto qrel) : •Lowcomplementarity •Combination: 0.211 -> 0.213
- 13. Favoringtargettypes MeSHfor PMC 2649306 D008569 Memory Disorders D001921 Brain D001284 Atrophy D005911 Gliosis D001706 Biopsy MeSHtargetDiagnosis MeSHtargetDiagnosis MeSHtargetTest Do relevant documents for diagnosisdeal more withdiagnosis? 3. Target-specific semantic enrichment with MeSH
- 14. 3. Target-specificsemanticenrichmentwithMeSH •In UMLS, eachMeSHtermhas SemanticTypes(ex: T060 Diagnostic Procedure) Focus on targets(diagnosis, treatmentsand tests) •Specificwords(ex: «MeSHtargetDiag») are addedin docs and queries Target % docs thathave at least1 Averagenumberin documents Test 83 % 16 Diagnosis 86 % 41 Treatment 86 % 24 Small improvementonlyfor section indexing
- 15. In the qrel… Set Aver. DiagnosisMeSH Aver. Test MeSH Aver. TreatmentMeSH All collection 41 16 24 Relevant for diagnosis (1|2for queries1..10) 108 41 41 Relevant for test (1|2for queries11..20) 107 41 33 Relevant for treatment (1|2for queries21..30) 114 47 52 All relevant documents: oAre quitesimilar, withno distinction betweentargets oBut have 2/3 times more targetMeSHterms o... but it’salsothe case for documents with0 in the qrel
- 16. 4. Boostingbasedon article types Promotingsomearticle types Are somearticle types more likelyto berelevant ?
- 17. Article type Distribution in docs in qrel in our runs research-article 74.3 % 52.2% 37.9 % case-report 4.0 % 20.4 % 41.5 % review-article 6.9 % 17.9 % 10.9 % Other 2.6 % 3.2 % 3.6 % brief-report 1.1 % 1.5 % 0.9 % 4. Boostingbasedon article types •Strategy: to promotereviewand case-basedarticles (boosting) •Intuition wasgood… •In reality… the IR enginealreadypromotedthesetypes ! but the strategyfailed! Top 5
- 18. 5. Exploitation of the co-citations network Promotingcitations Are citations of retrieveddocuments relevant ?
- 19. 5. Exploitation of the co-citations network •E isthe set of retrieveddocuments •RSVeisthe RetrievalStatusValue of doce •Weboosteachcitation of doceby + αx RSVe •50% of documents cite anotherone in the collection (avg3.8 cits)
- 20. Results •Withα= 0.1, slightimprovement •+ 10% for R-PREC •+ 20% for infNCDG •In TREC Chem2010 Prior Art task, + 150% for MAP
- 21. Conclusions “what is important is to have fought well”
- 22. Conclusions •A lot of strategies, but not muchbetterthanTerrier baseline •Section indexing: neveragain •MeSHnot complementary… Betterwheninferedby a k-NN ? •Relevant docs talk about test, diagand treatmentaltogether. •Maybewehave to startworkingfromthe baselinerun…