1
1/30/2015
Automatic Spatial Plausibility Checks for
Medical Object Recognition Results
Using a Spatial-Anatomical Ontolo...
2
1/30/2015
With the shift to the application of digital imaging
techniques for medical diagnosis, such as CT, MRI,
etc., ...
3
1/30/2015
MEDICO, RadSem and RadSpeech: A
Mashup
4
1/30/2015
Automatic object recognition algorithms
5
1/30/2015
Our approach is to augment
medical domain ontologies
and allow for an automatic
detection of anatomically
impl...
6
1/30/2015
Structural Anatomical Knowledge
7
1/30/2015
Motivation
• Automatic object
recognition algorithms
available for several
organs in 3D
• Perform reasonably w...
8
1/30/2015
Proposed Solution:
Integration of automatic object
recognition algorithms with
high-level knowledge about
huma...
9
1/30/2015
Outline
1. Goals and Prerequisites
2. Hierarchical Algorithm for Learning Spatial
Relations
3. Application to ...
10
1/30/2015
Goals and Prerequisites
• Goals:
– bridge semantic gap between low-level and high-level
information
– develop...
11
1/30/2015
Inductive Approach
• Qualitative representation: left/right,…
• Human Anatomy inherently variable  Fuzzy
• D...
12
1/30/2015
Hierarchical Algorithm for
Learning Spatial Relations
Pre-Processing
Automatic
Annotation
Fuzzy Atlas
Qualita...
13
1/30/2015
Pre-Processing
Automatic
Annotation
Fuzzy Atlas
Qualitative
Representation
Spatial
Reasoning
Corpus
• Collect...
14
1/30/2015
• Statistical algorithms for the detection of
various anatomical entities
– Constrained MSL
– Hierarchical Ac...
15
1/30/2015
• Classical Logic:
leftFrom(left kidney, right kidney)  {0,1}
• Fuzzy Logic:
leftFrom(left kidney, right kid...
16
1/30/2015
Patient 5
Patient 4
Patient 3
Patient 2
Patient 1
Natural variability in human anatomy
R Z
α=3°
„right kidney...
17
1/30/2015
A B
(a)
A B
(b) (c)
Right
Left
C
B
A
Relation Types
Pre-Processing
Automatic
Annotation
Fuzzy Atlas
Qualitati...
18
1/30/2015
Spatial Relations in OWL Model
• Extension of the formalism in the
Foundational Model of Anatomy
[0..1]
left|...
19
1/30/2015
• Example:
Learning spatial
relations
• Comparison
between learned
model and new
object recognition
result
• ...
20
1/30/2015
Medico Server
•MEDICO Ontology
•Sesame Triplestore
•>2 Mio. Triples
•Semantic
Annotation Store
•3D Volume Ren...
21
1/30/2015
Retrieval and examination of 2D/3D image series
22
1/30/2015
Conclusion
• Hierarchical abstraction process to learn spatial relations
from annotated volume data sets
• Me...
Upcoming SlideShare
Loading in …5
×

Automatic Spatial Plausibility Checks for Medical Object Recognition Results Using a Spatio-Anatomical Ontology

1,129 views

Published on

We present an approach using medical expert knowledge represented in formal ontologies to check the results of automatic medical object recognition algorithms for spatial plausibility. Our system is based on the comprehensive Foundation Model of Anatomy ontology which we extend with spatial relations between a number of anatomical entities. These relations are learned inductively from an annotated corpus of 3D volume data sets. The induction process is split into two parts. First, we generate a quantitative anatomical atlas using fuzzy sets to represent inherent imprecision. From this atlas we then abstract the information further onto a purely symbolic level to generate a generic qualitative model of the spatial relations in human anatomy. In our evaluation we describe how this model can be used to check the results of a state-of-the-art medical object recognition system for 3D CT volume data sets for spatial plausibility. Our results show that the combination of medical domain knowledge in formal ontologies and sub symbolic object recognition yields improved overall recognition precision.

Published in: Technology, Health & Medicine
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,129
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Automatic Spatial Plausibility Checks for Medical Object Recognition Results Using a Spatio-Anatomical Ontology

  1. 1. 1 1/30/2015 Automatic Spatial Plausibility Checks for Medical Object Recognition Results Using a Spatial-Anatomical Ontology Manuel Möller, Patrick Ernst, Andreas Dengel, Daniel Sonntag German Research Center for Artificial Intelligence University of Kaiserslautern
  2. 2. 2 1/30/2015 With the shift to the application of digital imaging techniques for medical diagnosis, such as CT, MRI, etc., the volume of digital images produced in modern clinics increased tremendously. Our clinical partner,the University Hospital Erlangen in Germany, has a total of about 50 TB of medical images. Currently, they have about 150,000 medical examinations producing 13 TB of data per year.
  3. 3. 3 1/30/2015 MEDICO, RadSem and RadSpeech: A Mashup
  4. 4. 4 1/30/2015 Automatic object recognition algorithms
  5. 5. 5 1/30/2015 Our approach is to augment medical domain ontologies and allow for an automatic detection of anatomically implausible constellations in the results of a state-of-the-art system for automatic object recognition in 3D CT scans. The output of our system also provides feedback which anatomical entities are most likely to have been located incorrectly.
  6. 6. 6 1/30/2015 Structural Anatomical Knowledge
  7. 7. 7 1/30/2015 Motivation • Automatic object recognition algorithms available for several organs in 3D • Perform reasonably well (in many cases) • Integration with anatomical background knowledge: Foundational Model of Anatomy Daniel Sonntag, Daniel.Sonntag@dfki.de
  8. 8. 8 1/30/2015 Proposed Solution: Integration of automatic object recognition algorithms with high-level knowledge about human anatomy. Motivation In some cases the automatic object recognition is grossly wrong. Daniel Sonntag, Daniel.Sonntag@dfki.de
  9. 9. 9 1/30/2015 Outline 1. Goals and Prerequisites 2. Hierarchical Algorithm for Learning Spatial Relations 3. Application to Automatic Object Recognition 4. Conclusion Daniel Sonntag, Daniel.Sonntag@dfki.de
  10. 10. 10 1/30/2015 Goals and Prerequisites • Goals: – bridge semantic gap between low-level and high-level information – develop system integrating information from both sources – perform reasoning to check plausibility of object recognition results • Prerequisites: Automatic object recognition algorithms Structural anatomical knowledge Spatial relations of human anatomy Integration of low-level and high-level information Daniel Sonntag, Daniel.Sonntag@dfki.de
  11. 11. 11 1/30/2015 Inductive Approach • Qualitative representation: left/right,… • Human Anatomy inherently variable  Fuzzy • Data-driven Patient 1 Patient 2 Patient 3 Patient 4 Patient 5 Canonical Anatomy Daniel Sonntag, Daniel.Sonntag@dfki.de
  12. 12. 12 1/30/2015 Hierarchical Algorithm for Learning Spatial Relations Pre-Processing Automatic Annotation Fuzzy Atlas Qualitative Representation Spatial Reasoning Corpus Daniel Sonntag, Daniel.Sonntag@dfki.de
  13. 13. 13 1/30/2015 Pre-Processing Automatic Annotation Fuzzy Atlas Qualitative Representation Spatial Reasoning Corpus • Collected at the University Hospital Erlangen • from 2002 to 2008 • Cancer patients with lymphoma • 3D volume data sets from computer tomography scanners • Available image data Daniel Sonntag, Daniel.Sonntag@dfki.de
  14. 14. 14 1/30/2015 • Statistical algorithms for the detection of various anatomical entities – Constrained MSL – Hierarchical Active Shape Models – Patch-based Deformable Models – Trainable Boundary Detector see: Seifert, Kelm, Möller, Mukherjee, Cavallaro, Huber, Comaniciu: “Semantic Annotation of Medical Images”, SPIE Medical Imaging 2010 Results: • Meshes: 6 different organs left/right kidney, left/right lung, urinary bladder, prostate gland • Landmarks: 22 exposed points: top point of the liver, … • Manually generated gold standard of automatically annotated volume data sets: 1 017 labeled volumes Pre-Processing Automatic Annotation Fuzzy Atlas Qualitative Representation Spatial Reasoning Corpus Daniel Sonntag, Daniel.Sonntag@dfki.de
  15. 15. 15 1/30/2015 • Classical Logic: leftFrom(left kidney, right kidney)  {0,1} • Fuzzy Logic: leftFrom(left kidney, right kidney)  [0,1] • Representation of direction in 2D: 0 ½π +/-π -½π R T right above left below α Pre-Processing Automatic Annotation Fuzzy Atlas Qualitative Representation Spatial Reasoning Corpus 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 unten rechts oben cos^2(x) -½π ½π0 Truthvalue angle below aboveright Daniel Sonntag, Daniel.Sonntag@dfki.de
  16. 16. 16 1/30/2015 Patient 5 Patient 4 Patient 3 Patient 2 Patient 1 Natural variability in human anatomy R Z α=3° „right kidney right from left kidney“: Truth value Absolutefrequency 10 R Z α=0° R Zα=4° R Zα=4° R Z α=0° avg= 0,92… Pre-Processing Automatic Annotation Fuzzy Atlas Qualitative Representation Spatial Reasoning Corpus Daniel Sonntag, Daniel.Sonntag@dfki.de
  17. 17. 17 1/30/2015 A B (a) A B (b) (c) Right Left C B A Relation Types Pre-Processing Automatic Annotation Fuzzy Atlas Qualitative Representation Spatial Reasoning Corpus • Direction: „left kidney left from right kidney“ • Adjacency: „prostate adjacent to urinary bladder“ • Between: „bronchial bifurcation between left and right lung“ • Evaluated with medical experts Daniel Sonntag, Daniel.Sonntag@dfki.de
  18. 18. 18 1/30/2015 Spatial Relations in OWL Model • Extension of the formalism in the Foundational Model of Anatomy [0..1] left|right| above|… term truthValue Instance of SimpleFuzzyRelation Anatomical Entity B Anatomical Entity A location related Object FuzzySpatialAssoc iationRelation type [0..1] left|right| above|… directionalTerm truthValue Instance of SimpleFuzzyRelation [0..1] left|right| above|… directionalTerm truthValue Instance of SimpleFuzzyRelation [0..1] left|right| above|… term truthValue Instance of SimpleFuzzyRelation Pre-Processing Automatic Annotation Fuzzy Atlas Qualitative Representation Spatial Reasoning Corpus Daniel Sonntag, Daniel.Sonntag@dfki.de
  19. 19. 19 1/30/2015 • Example: Learning spatial relations • Comparison between learned model and new object recognition result • Results: true positives 407 true negatives 431 false positives 67 false negatives 213 precision 85,7% recall 65,5% Patient 1Patient 2 Patient 3 Patient 4 Pre-Processing Automatic Annotation Fuzzy Atlas Qualitative Representation Spatial Reasoning Corpus Incorrectly located organ Spatial Consistency Check Daniel Sonntag, Daniel.Sonntag@dfki.de
  20. 20. 20 1/30/2015 Medico Server •MEDICO Ontology •Sesame Triplestore •>2 Mio. Triples •Semantic Annotation Store •3D Volume Renderer •Based on MITK State of the Art Organ and Landmark Detection Ontology-based Visual Navigation Application Central Java-based Data Exchange Application MEDICOServer Volume Parser MITK Semantic Navigation Semantic Search XMLRPC SPARQL CORBA CORBA CORBA Java API CTC-WP4 Triple Store Query Broker RadSpeech
  21. 21. 21 1/30/2015 Retrieval and examination of 2D/3D image series
  22. 22. 22 1/30/2015 Conclusion • Hierarchical abstraction process to learn spatial relations from annotated volume data sets • Method for the generation of a fuzzy anatomical atlas from different patients • Spatial consistency check comparing automatic object recognition results with Daniel Sonntag, Daniel.Sonntag@dfki.de Pre-Processing Automatic Annotation Fuzzy Atlas Qualitative Representation Spatial Reasoning Corpus Patient 5 Patient 4 Patient 3 Patient 2 Patient 1 R Z α=3° R Z α=0° R Zα=4° R Zα=4° R Z α=0° „right kidney right from left kidney“: Truth value Absolutefrequency 10

×