semantic technologies used to overcome issues with semantic annotation of entities in medical information systems

1. Living with the Semantic Gap
Experiences and Remedies in the Context of Medical Imaging
Yannis Kalfoglou, Srinandan Dasmahapatra, David Dupplaw, Bo Hu, Paul Lewis, Nigel Shadbolt
 Grid based Knowledge Services  Dynamic Registration of Images (MIAS - MIAKT)
domain of Triple Assessment (TA) in medical decision making (MIAKT)  Grid Reasoning Services
 Ontology Services – Breast Cancer Imaging Ontology (BCIO – MIAKT) classify, cluster, retrieve images – (MIAKT)
 Enrichment and Image Annotation Services using BCIO (MIAKT)  Semantic Alignment of Image Representations
Ontology mapping (CROSI)
MIAKT – Medical Imaging and Advanced Knowledge Technologies
• 2 year EPSRC project (2002-2004), Multi-Disciplinary, Multi-Media Knowledge
Decision made about patient’s treatment
formed by 2 large IRCs (Advanced Knowledge Technologies (AKT), Medical Images and Signals (MIAS) Radiologist (X-ray, MRI and UltraSound Mammograms) Diagnosis correct 95% of the time
• 10% of women develop breast cancer (3 million screenings per year in UK) Pathologist (Pathological Slides) But how? No explicit knowledge
Clinician (Patient Records, Patient History) MIAKT provides support to this process
Framework
Knowledge Technologies involved
• Knowledge acquisition and ontology development
• Semantic annotation of images
• Language generation
• Annotation based clustering and classification
• Grid Internet Reasoning Service and Problem
Solving Environments
• Issue tracking and collaborative problem solving
<Mammo-Abnormality rdf:ID="ROI-0001-0001">
<has-depth rdf:resource='d epth-subareolar'/>
<has-morph-feature
<rdf:Description rdf:resource='shape-mammo-irregular'/>
rdf:ID="ROI-0001-0001">
<rdf:type>
<has-morph-feature
<rdfs:Class rdf:about="#Mammo-Abnormality"/>
rdf:resource='margin-mammo-spiculated'/>
<rdf:Description rdf:ID="ROI-0001-0001">
</rdf:type>
<is-finding rdf:resource='mass'/ >
<rdf:type> </RDFNsId2:Mammo-Abnormality>
<RDFNsId2:has-depth rdf:resource='d epth-subareolar'
<rdfs:Class rdf:about="#Mammo-Abnormality"/>
rdf:type='Depth-Descriptor'/>
</rdf:type>
... ...
OWL
<RDFNsId2:has-depth rdf:resource='d epth-subareolar'
</RDFNsId2:Mammo-Abnormality>
rdf:type='Depth-Descriptor'/>
... ...
DAML
</RDFNsId2:Mammo-Abnormality> http://www.aktors.org/miakt/
RDF
Task Invocation Framework <miakt_fw:Instance-Handler rdf:about="MIAKT-System-Preferences_00043"
miakt_fw:has-name="PatientHandler">
<miakt_fw:has-class> domain.miakt.PatientHandler </miakt_fw:has-class>
<miakt_fw:has-description>Handler for instance information </miakt_fw:has-description>
<miakt_fw:has-type>&domain;Patient</miakt_fw:has-type>
</miakt_fw:Instance-Handler>
Nature Language Generation Services
• Mark up patient data and images from the ontology and store as RDF files
• Generate report based on the RDF files
• UMLS provides definitions for medical terms in the reports
<RDFNsId2:Mammo-Abnormality rdf:about='PE-0001-0001'
RDFNsId2:has-branch='false'
RDFNsId2:has-size='10'
RDFNsId2:graphic-region='Abnorm0'>
<RDFNsId2:has-depth rdf:resource='depth-subareolar'
rdf:type='Depth-Descriptor'/ >
EPSRC-LSI
Report The 68 years old patient is
<RDFNsId2:has-morph-feature
rdf:resource='shape-mammo-irregular' involved in a multi-disciplinary
rdf:type='Mammo-Shape'/ >
<RDFNsId2:has-morph-feature
meeting procedure. The multi-
rdf:resource='margin-mammo-spiculated'
rdf:type='Mammo-Margin'/>
disciplinary assessment procedure
<RDFNsId2:is-finding rdf:resource='mass' contains a mammography exam.
rdf:type='Mass'/>
The mammography exam is
Publishing
<RDFNsId2:distri-single-obs
rdf:resource='distri-xray-cluster' carried out on the patient on 22 9
rdf:type='Distribution-Mammo'/>
</RDFNsId2:Mammo-Abnormality>
1995. The mammography exam
produced a right CC image. The
right CC image contains a Region
of Interest… The ROI has a mass,
a probably malignant assessment,
a microlobulated margin, and a
round shape …
System Features
Ontologies and instances •
Graphical navigation •
Retrieval and Annotation: •
<conceptual terms, image
features>
• Image processing algorithms
• Interface to web services using
RDF triple store
• Internet Reasoning Services based on
declarative descriptions
• Language generation – patient reports
• Grid enabled image registration
http://www.aktors.org/miakt
CROSI - Capturing Representing and Operationalising Semantic Integration
• 1 year HP funded project (2004-2005) • Semantic Integration Technologies Survey • Modular architecture for Semantic Integration • Flexible, multi-matcher, multi-strategy aligner
• Investigate Semantic Integration technology • Semantic Intensity Spectrum Systems • Open source, API, documented, evaluated
• Focus on Ontology mapping and merging • Classify Integration Technologies • Exemplar: CMS (CROSI Mapping System) • Linguistic, Structure, Syntax, Semantics
Semantic Intensity Spectrum Exemplar algorithm: structure-based
CMS GUI
f (name similarity, domain similarity, range similarity )
Domain Domain
of P1 H of P1’ H’
C C'
G G’ I’
Range Range
of P1 A of P1’ A’
Modular Architecture
B D’
D B’
E’
E
F
’
http://www.aktors.org/crosi
Associated Researchers
Harith Alani Yannis Kalfoglou
Leslie Carr Yaozhong (David) Liang
Richard Crowder Duncan McRae-Spencer
Srinandan Dasmahapatra Kieron O'Hara
David Dupplaw Monica Schraefel
Nick Gibbins Nigel Shadbolt
Hugh Glaser Paul Smart
Advanced Knowledge Technologies Steve Harris Mischa Tuffield
Chia-Chih Hsu Gary Wills
http://www.aktors.org/ Bo Hu