The main objective of this work is to facilitate the identification, sharing, and reasoning about cerebral tumors observations via the formalization of their semantic meanings in order to facilitate their exploitation in both the clinical practice and research. We focused our analysis on the VASARI terminology as a proof of concept, but we are convinced that our work can be useful in other biomedical imaging contexts.

1. Semantic Representation of Neuroimaging
Observations: Proof of Concept Based on the
VASARI Terminology
Emna Amdouni and Bernard Gibaud
E-health department, BCOM Institute of Research and Technology
Rennes, France

2. Plan
1 Introduction
2 State of the art
3 Material and methods
4 Results
5 Conclusion and perspectives
2/29

3. Plan
1 Introduction
2 State of the art
3 Material and methods
4 Results
5 Conclusion and perspectives
3/29

4. Introduction
Denition
An imaging biomarker is a feature that is objectively
measured and evaluated as an indicator of normal biological
processes, or pathological or biological response to a
therapeutic intervention [Downing, 2001]
It studies anatomical, functional and molecular properties of
tissues.
Imaging biomarkers are used to:
Detect pathology;
Predict the degree of risk;
Establish a prognosis;
Evaluate the response to a treatment.
etc.
4/29

5. Introduction
Denition
An imaging biomarker is a feature that is objectively
measured and evaluated as an indicator of normal biological
processes, or pathological or biological response to a
therapeutic intervention [Downing, 2001]
It studies anatomical, functional and molecular properties of
tissues.
Imaging biomarkers are used to:
Detect pathology;
Predict the degree of risk;
Establish a prognosis;
Evaluate the response to a treatment.
etc.
4/29

6. Introduction
tumor size radiodensity in a tumor ROI
main direction of
diffusivity
leakage rate of the contrast product
between the two spaces
intra and extra cellular
radioactivity in tissues
5/29

7. Introduction
Use case:VASARI terminology
Annotation vocabulary of high-grade brain gliomas especially
glioblastoma multiforme in MRI images.
30 image criteria referenced by a number (F 1, F 2, etc.) and a
set of values of scores.
Example F 1: tumor location
F 1 evaluates the location of the geographic epicenter and denes
seven possible score values: frontal, temporal, insular, parietal,
occipital, brainstem, cerebellum.
6/29

8. Introduction
7/29

9. Introduction
Scientic problem
Imaging data are complex to model given that they are rich in
information and they express implicit knowledge.
Our goal
Facilitate the identication, sharing and reasoning of brain tumor
ndings by formalizing their semantic meanings to facilitate their
interpretation by clinicians and their reuse in other contexts.
8/29

10. Introduction
Scientic problem
Imaging data are complex to model given that they are rich in
information and they express implicit knowledge.
Our goal
Facilitate the identication, sharing and reasoning of brain tumor
ndings by formalizing their semantic meanings to facilitate their
interpretation by clinicians and their reuse in other contexts.
8/29

11. Introduction
Our contribution
Semantic representation of
neuroimaging data with a realistic
approach to guide image analysis
systems via the use of a semantic model.
In this work, we propose:
methodology, domain ontology and annotation tool to provide
formal denitions of neuroimaging observations;
an experimental work around the REMBRANDT database to
demonstrate the added value of our work;
9/29

12. Plan
1 Introduction
2 State of the art
3 Material and methods
4 Results
5 Conclusion and perspectives
10/29

13. State of the art
DICOM SR
Data structure of the
DICOM standard which
formalizes the representation
of radiological observations
in radiological reports;
structure adapted to the
exchange, but unt to the
treatment of requests or
inferences;
Source:[Clunie, 2007]
11/29

14. State of the art
Annotation and Image Markup
model
Cancer Biomedical
Informatics Grid Project
Initiative (caBIG);
AIM introduced the main
entities involved in the
annotation of medical
images;
in XML ... lacks formal
semantics
Source:[Buckler et al., 2013]
12/29

15. Plan
1 Introduction
2 State of the art
3 Material and methods
4 Results
5 Conclusion and perspectives
13/29

16. Material and methods
Use of a realistic approach
Need for dierentiation between 3 levels of entities:
1 Observed entity,
2 Representation of the observed entity,
3 Observation data
14/29

17. Material and methods
Use of a realistic approach
An approach based on realism facilitates:
formal and explicit representation of the meanings of
radiological observations [Levy et al., 2012]
Distinction between existing and non existing entities [Ceusters
et al., 2006]
Tracking of entity evolution over time [Ceusters and Smith,
2005]
15/29

18. Material and methods
VASARI ontology design
Our modeling methodology consists of ve main steps:
1 Determine for each VASARI Fi characteristic the meaning of
the studied aspect and sort out its possible congurations
2 Identify and describe the key and real entities involved
3 Connect these entities to existing ontologies or create new
classes by specializing existing ones
4 Specify the axioms characterizing these entities and the
relations between them
5 Check that all possible congurations for each Fi criterion are
formally modeled.
16/29

19. Material and methods
VASARI ontology design
Our modeling methodology consists of ve main steps:
1 Determine for each VASARI Fi characteristic the meaning of
the studied aspect and sort out its possible congurations
2 Identify and describe the key and real entities involved
3 Connect these entities to existing ontologies or create new
classes by specializing existing ones
4 Specify the axioms characterizing these entities and the
relations between them
5 Check that all possible congurations for each Fi criterion are
formally modeled.
16/29

20. Material and methods
VASARI ontology design
Our modeling methodology consists of ve main steps:
1 Determine for each VASARI Fi characteristic the meaning of
the studied aspect and sort out its possible congurations
2 Identify and describe the key and real entities involved
3 Connect these entities to existing ontologies or create new
classes by specializing existing ones
4 Specify the axioms characterizing these entities and the
relations between them
5 Check that all possible congurations for each Fi criterion are
formally modeled.
16/29

21. Material and methods
VASARI ontology design
Our modeling methodology consists of ve main steps:
1 Determine for each VASARI Fi characteristic the meaning of
the studied aspect and sort out its possible congurations
2 Identify and describe the key and real entities involved
3 Connect these entities to existing ontologies or create new
classes by specializing existing ones
4 Specify the axioms characterizing these entities and the
relations between them
5 Check that all possible congurations for each Fi criterion are
formally modeled.
16/29

22. Material and methods
VASARI ontology design
Our modeling methodology consists of ve main steps:
1 Determine for each VASARI Fi characteristic the meaning of
the studied aspect and sort out its possible congurations
2 Identify and describe the key and real entities involved
3 Connect these entities to existing ontologies or create new
classes by specializing existing ones
4 Specify the axioms characterizing these entities and the
relations between them
5 Check that all possible congurations for each Fi criterion are
formally modeled.
16/29

23. Material and methods
VASARI ontology design
We encountered the following modeling issues:
MP1: representation of negative observations
C1 bfo:independent continuant
ex: John's brain tumor is without hemorrhagic region
C2 bfo:dependent continuant
ex: John's tumor is not eroded (bfo:quality)
ex: John's tumor does not take a contrast
(bfo:disposition)
VASARI: negative expressions
none, indeterminate, without, not applicable.
17/29

24. Material and methods
MP2: representation of the location and composition of the
tumor:
ex: the epicenter of John's brain tumor is located in the frontal lobe
ex: the epicenter of John's brain tumor has a border
VASARI: spatial description
Inclusion: within, portion of, comprise.
Overlap and adjacency: surrounding, invasion.
Separation: not contiguous, separated.
18/29

25. Material and methods
Domain ontology Domain Top-level
OBI/IAO measures and information content BFO
FMA anatomic structures BFO
PATO phenotypic traits and qualities BFO
UO measurement units BFO
Integration using Basic Formal Ontology
19/29

26. Material and methods
Experimentation: REMBRANDT
The REMBRANDT database contains 30 VASARI annotations
expressed by 3 radiologists and relating to 34 patients with GBM.
Objective: to facilitate the discovery of signicant correlations
between clinical and genomic information
Implementation details
NetBeans IDE V.8.0.2
JENA-PELLET V.2.3.2
CORESE V.3.2
20/29

27. Plan
1 Introduction
2 State of the art
3 Material and methods
4 Results
5 Conclusion and perspectives
21/29

28. Results
22/29

29. Results
Hierarchy of vasari:cerebral tumor component classes
23/29

30. Results
PM1: conceptualization of negative observations
Introduction of the notion of absence for the two categories C1 and
C2:
Condition 'if and only if' (equivalent class)
No instance should be created when no concrete entity
actually exists
Ex C1: the class vasari:cerebral tumor component not
located in brain cortex ≡ def. isA vasari:cerebral
tumor component and not (locatedInAtSomeTime some
fma:cerebral cortex)
Ex C2: the class vasari:non cystic cerebral tumor
component ≡ def. isA vasari:cerebral tumor
component and not (hasQualityAtSomeTime some
pato:cystic)
24/29

31. Introduction
25/29

32. Results
REMBRANDT : patient #900_00_5316
26/29

33. Plan
1 Introduction
2 State of the art
3 Material and methods
4 Results
5 Conclusion and perspectives
27/29

34. Discussion and conclusion
The developed ontology answers some problematic points
Experimental work demonstrates the feasibility and importance
of making RDF and OWL data available to describe brain
tumor observations
Our work could complement work of:
Semantic query using knowledge represented in Atlases (eg
determination of certain parts impacted by a pathology)
Reasoning on the locations of pathological structures (eg
predicting the consequences of a surgical operation)
Search for images that contain lesions similar to a given lesion
28/29

35. 29/29