Content based image retrieval by metric learning from radiology reports application to interstitial lung diseases
1. CONTENT BASED IMAGE RETRIEVAL BY METRIC LEARNING FROM
RADIOLOGY REPORTS: APPLICATION TO INTERSTITIAL LUNG DISEASES
ABSTRACT
Content Based Image Retrieval (CBIR) is a searchtechnology that could aid medical
diagnosis by retrieving andpresenting earlier reported cases that are related to the one
beingdiagnosed. To retrieve relevant cases, CBIR systems depend onsupervised learning to map
low level image contents to high leveldiagnostic concepts. However, the annotation by medical
doctorsfor training and evaluation purposes is a difficult and timeconsumingtask, which restricts
the supervised learning phaseto specific CBIR problems of well defined clinical
applications.This paper proposes a new technique that automatically learnsthe similarity between
the several exams from textual distancesextracted from radiology reports, thereby successfully
reducing the number of annotations needed. Our method firstly infersthe relation between
patients by using information retrievaltechniques to determine the textual distances between
patientradiology reports. These distances are subsequently used tosupervise a metric learning
algorithm, that transforms the image space accordingly to textual distances. CBIRsystemswith
different image descriptions and different levels of medicalannotations were evaluated, with and
without supervision fromtextual distances, using a database of computer tomographyscans of
patients with interstitial lung diseases. The proposedmethod consistently improves CBIR mean
average precision, withimprovements that can reach 38%, and more marked gains forsmall
annotation sets. Given the overall availability of radiology reports in Picture Archiving and
Communication Systems, theproposed approach can be broadly applied to CBIR systems
indifferent medical problems, and may facilitate the introductionof CBIR in clinical practice.