Uncertainty of classification is the noisy aspect of the observed data : for a given identified class, the observation is noisy with a given mean and variance. On the other hand, imprecision of data models the fact that a given voxel may contain more than one class only, therefore X may belong to [0,1] instead of {1, 2, …, C}
A Markov chain is a unidimensional vector defined by the initial probabilities that allow choosing the first element, and transition probabilities that allow choosing the next elements. Each element of the chain is associated to a voxel of the image (Y observation vector) through the Hilbert Peano path that fills the 2D image as a fractal. As for the Markov assumption, it allows computation of the probabilities by reducing the dependency to the previous element only instead of the entire chain.
This is the result of fuzzy Hidden Markov chain segmentation combining the three bands (multispectral observation), with two hard classes and 8 fuzzy levels in between. Notice how it makes the structure in the bottom right corner well defined.
In order to apply the FHMC segmentation to PET, the Hilbert Peano path is extended from 2D to 3D. The 1D chain is then segmented as in the previous case, except with monospectral observation. Here, only two fuzzy levels are used because of the limited number of voxels and the small size of the image and the object, that do not allow using more fuzzy levels.
The issue with using FHMC to segment PET tumor image is the fact that the Hilbert Peano path disrupts the spatial correlation between voxels since neighboring voxels in the image may find themselves far from in other in the resulting chain. This is not a significant issue for large 512x512 images, however in PET regions are defined by a small number of voxels and this effect is far more significant, leading to large errors in the estimation of the object dimensions
In order to solve this issue, the spatial modeling has to be replaced: here we solved the spatial correlation problem resulting from the Hilbert Peano path by replacing the hidden Markov chain spatial modeling by a local model estimating the probabilities for each voxel by looking at the immediate neighborhood using a sliding cube of typically 3x3x3 or 5x5x5 voxels. As you can see in the resulting segmentation map, the “block” effect coming from the use of the Hilbert Peano path is replaced by a much more “smooth” segmentation in the case of FLAB
In order to deal with intra-tumor heterogeneities we have to introduce an additional third hard class in the image, hence we need to model two additional fuzzy transitions between the various regions in the image. This is done by modeling interactions between pairs of classes.
Here is illustrated the performances of both the hidden Markov chains (FHMC) and locally adaptive (FLAB) models on the spheres previously described. On left side of the figure are the results for a low contrast of 4 to 1, whereas in the right part of the figure are the results for a higher contrast of 8 to 1. As the sphere diameter goes down (from 37 to 10 mm), the spatial correlation disruption introduced by the Hilbert Peano path of the FHMC model becomes more significant, leading to higher errors, whereas the FLAB model manages to recover the sphere with higher accuracy down to 13 mm.
These results show that the FLAB model can be applied to images acquired in any kind of PET scanner with various textures and noise properties, since the mean error and associated standard deviation computed across the entire range of acquisition is in the 0-10% range down to 13 mm spheres
The figure illustrates the errors (mean +/- standard deviation) of each method ,computed for the entire 20 tumors dataset
Here for these real clinical tumors, the 3d volume is not known, only the maximum diameter was measured in histopathology. This is a restricted parameter but the results are nonetheless interesting since they show that whereas each method leads to small mean error (about half of the dataset are over evaluation while the other half are under evaluation, therefore compensating themselves), the standard deviation associated with FLAB is significantly smaller than for threshold-based approaches.
Transcript
1. From nebulae segmentation in
astronomical imaging to tumor
delineation in 18F-FDG PET imaging:
how can one serve the other?
M. Hatt1
, C. Collet2
, F. Salzenstein3
, C. Roux1
, D. Visvikis1
Speaker: S. David1
1. LaTIM, INSERM U650, Brest, France1. LaTIM, INSERM U650, Brest, France
2. LSIIT, CNRS - UMR 7005, Strasbourg, France2. LSIIT, CNRS - UMR 7005, Strasbourg, France
3. INESS, CNRS - UMR 7163, Strasbourg, France3. INESS, CNRS - UMR 7163, Strasbourg, France
2. Context and objective
Cancer Oncology
 Gold standard for diagnosis
 Other applications of interest:
 Radiotherapy planning
 Prognosis, therapy assessment
PET/CT multimodality imaging
Quantification
 active biological volume
 uptake measurement
 radiotherapy target definition
Requires the definition of a
volume of interest
Computed
tomography (CT)
Positron Emission
Tomography (PET)
Source of image X-ray Positron emitter (18
F)
Nature Anatomic: tissues
and bones density
Functional :
accumulation of
radioactive tracer
Resolution < 1 mm > 5 mm
Imaging for oncology
3. Context and objective Problems of PET images
3
 Noise
(acquisition variability)
 Blur
(spatial resolution)
 Voxels size
(grid spatial sampling)
 uptake heterogeneities
within the tumor
4. Methodologies Existing solutions
 Manual definition of regions of interest in the background
 Parameters optimization for each scanner
 Assume tumors are homogeneous spheres :
Threshold-based methodologies [1-3]
[1] J. A. van Dalen et al, Nuclear Medicine Communications, 2007
[2] U. Nestle et al, Journal of Nuclear Medicine, 2005
[3] J.F. Daisne et al, Radiotherapy Oncology, 2003
Require a lot of a priori information and are system and user dependent
But tumors are often of complex shapes and heterogeneous !
5.  PET images share several characteristics with some astronomical images
Why looking at astronomical images processing for solutions ?
 The segmentation/classification field is more mature for astronomy than PET
Methodologies Astronomical images segmentation
6. Nebulae vs PET tumor ?
Methodologies Astronomical images segmentation
7. Nebulae vs PET tumor ?
Characteristic Nebulae image PET tumor image
Dimensions 2D, multi/hyper spectral 3D, mono spectral
Definition Large (~512x512) Small (~30x30x30)
Encoding 32b real 16b/32b real
Fuzzy yes yes
Noisy yes yes
Band 1 Band 2 Band 3
Slice n+1
Slice n
Slice n-1
Use of statistical image processing to deal with the noise, combined with fuzzy
modeling to deal with blur
Methodologies Astronomical images segmentation
8. Methodology : statistical + fuzzy
 Probabilistic / statistical part models the uncertainty of classification
 Fuzzy part models the imprecision of acquired data
Combining both to model astronomical or PET images characteristics
1 2 ... Cν δ δ δ= + + +
cδ : Discrete Dirac measure on class c
Standard (“hard”) statistical modelling
Ground-truth
0 1ν δ δ ζ= + +
ζ : Continuous Lesbegue measure on ] [0,1
Fuzzy modelling [1] [2]
[1] H. Caillol et al, IEEE Transactions on Geoscience Remote Sensing, 1993
[2] F. Salzenstein and W. Pieczynski, CVGIP : Graphical Models and Image Processing, 1997
Methodologies
9. Methodology: fuzzy Markov chains
Markov assumption:
1 1 1( | ,..., ) ( | )t t t tp x x x p x x− −=
… …1x 2x tx Tx
1( | )t tp x x −
Transition probabilities
1( )p x
Initial probabilities
 Use of the Hilbert-Peano path to transform 2D image into 1D chain
1tx −
1y 2y ty Ty
Observation vector
( | )t tp y x1ty −
in [0,1]tx
Methodologies
10. Result on Nebulae
Fuzzy Hidden Markov Chains (FHMC) multispectral segmentation
F. Salzenstein, C. Collet, S. Lecam, M. Hatt, Pattern Recognition Letters, 2007
Methodologies
11. Apply to PET ?
3D PET
tumor
Iterative stochastic estimation (SEM)
1D chain with discrete values {0,1,F1,F2}
Segmentation (MPM)
1D chain with real values
Hilbert-Peano 3D
Inverse Hilbert-
Peano 3D
Segmentation map
(2 fuzzy levels)
 Extended Hilbert-Peano path to transform 3D image into 1D
M. Hatt et al, Physics in Medicine and Biology, 2007;52(12):3467-3491
Methodologies
12. Problem !
 3D Hilbert-Peano path to transform 3D image into 1D disrupts spatial correlation :
Neighbors voxels in the image may be
far from each other in the chain
 Size of tumors with respect to object and size of voxels leads to large errors for small tumors !
M. Hatt et al, Physics in Medicine and Biology, 2007; 52(12):3467-3491
Methodologies
13. Solution: locally adaptive method
3D PET
tumor
Segmentation
map
Segmentation
map
FHMC
M. Hatt et al, IEEE Transactions on Medical Imaging, 2009;28(6):881-893
Iterative stochastic estimation (SEM)
Segmentation
 Markovian model replaced by sliding estimation cube to
compute probabilities for each voxel regarding its neighbors :
 FLAB (Fuzzy Locally Adaptive Bayesian) method
Methodologies
FLAB
14. 1
2
3
M. Hatt et al, International Journal of Radiation Oncology Biology Physics , in press, 2009
 Modeling fuzzy transitions between pairs of hard classes
to deal with heterogeneities
2 hard classes and
1 fuzzy transition
1
0
Methodologies
3 hard classes and 3
different fuzzy transitions
15. Simple phantom validationResults
Phantom acquisitions with spheres : 37 to 10 mm in diameter
Phantom Computed tomography image (truth)
18
F Positron Emission Tomography image
Axial Coronal Sagital
16. Results FHMC vs FLAB
M. Hatt et al, IEEE Transactions on Medical Imaging, 2009;28(6):881-893
17. Multiple scanners robustness validation
 4 different scanner models and various acquisitions parameters
(contrast, noise, reconstruction algorithms, size of voxels…)
Philips Gemini GE Discovery LS
OSEM
Siemens Biograph
RAMLA 3D
Philips Gemini TF
TF MLEM
A
B
1 2 1 1 21 2
A = 4:1 or 5:1, B = 8:1 or 10:1 1 = 2x2 mm, 2 = 4x4 or 5x5 mm
37 mm28 mm22 mm17 mm13 mm
M. Hatt et al, Society of Nuclear Medicine annual meeting, Toronto, Canada, 2009
Results
18. Real Simulated
Small homogeneous Large heterogeneous
Real Simulated
 20 tumors (NSCLC, H&N, Liver)
 maximum diameter from 12 to 82 mm
 Heterogeneities: from none to high
 Shapes: from almost spherical to complex
 Simulated with Monte Carlo GATE (Geant4 Application for Tomography Emission)
M. Hatt et al, International Journal of Radiation Oncology Biology Physics , in press, 2009
Results Accuracy validation on simulated data
19. FLAB
Ground-truth
Fixed threshold
Classif. error: 6%> 100%Simulated
PET
Adaptive threshold
Classification errors
Grey region 4%
Black region 2%
Volume
error
-62%
Volume
error
+37%
Segmentation
Segmentation
Adaptive threshold
FLAB
Fixed threshold
Ground-truth
Simulated
PET
14%
M. Hatt et al, International Journal of Radiation Oncology Biology Physics , in press, 2009
Results Accuracy validation on simulated data
20. Patients with histology accuracy validation
 18 tumors (NSCLC) with histology study [1]
 maximum diameter from 15 to 90 mm (mean 44, SD 21)
 Heterogeneity : none to high
 Shapes : from almost spherical to complex
CT
PET
[1] A. van Baardwijk et al, International Journal of Radiation Oncology Biology Physics, 2007
Results
21. Patient with NSCLC
FLABAdaptive thresholdFixed threshold (42%)
M. Hatt et al, International Journal of Radiation Oncology Biology Physics , in press, 2009
Results Patients with histology accuracy validation
22. Conclusions and work in progress
 Studies are ongoing to further investigate the clinical impact
of the proposed methodology in radiotherapy or patient
prognosis and therapy assessment
 This work is a good example of know-how transfer from
astronomical to medical imaging
 Once adapted to PET data (2D->3D, spatial modeling),
statistical and fuzzy segmentation developed for astronomical
imaging performed admirably well for tumor delineation
Be the first to comment