présentation de Benoît Lelandais1 , Isabelle Gardin1,2 , Laurent Mouchard1 , Pierre Vera1,2 , Su Ruan1 on belief2012 Compiègne 9-11 mai 2012

1. Using belief function theory to deal with uncertainty and
imprecision in image processing
Benoît Lelandais1 , Isabelle Gardin1,2 , Laurent Mouchard1 , Pierre Vera1,2 , Su Ruan1
1 LITIS EA 4108 - QuantIF, University of Rouen
22 bd Gambetta, 76183 Rouen Cedex, France
2 Department of nuclear medicine, Henri-Becquerel center,
1 rue d’Amiens, 76038 Rouen Cedex 1, France
May 10 2012

2. Introduction Method Application to PET image fusion Discussion - Conclusion References
1 Introduction
PET imaging
Objectives
2 Method for reducing uncertainty and imprecision in image processing
BBA estimation for each image
Information fusion for reducing uncertainty
Information fusion for reducing imprecision
Illustration of the method on two simulated images sources
3 Application to PET image fusion
Frame of discernment
BBA estimation
Fusion with an a priori knowledge
Multi-modal fusion
4 Discussion - Conclusion
- 2/25

3. Introduction Method Application to PET image fusion Discussion - Conclusion References
Introduction
Context
Both 3D anatomical and functional medical images are obtained during the same
acquisition
Anatomical imaging: CT (Computed Tomography ).
Functional imaging: PET (Positron Emission Tomography ) with FDG
(Fluoro-Deoxy-Glucose). The FDG is an indicator of tumor glucose metabolism.
(a) PET/CT tomograph (b) Anatomical image (c) Functional image with
with a tumoral region a high FDG uptake in tu-
mor cells
Transverse slices for a single patient suffering from lung cancer. The area of interest (tumor
lesion) is located in the rectangle.
- 3/25

4. Introduction Method Application to PET image fusion Discussion - Conclusion References
Introduction
Context
Principle of treating tumors by radiation therapy
Segmentation of tumoral volume:
From anatomical imaging: Computed Tomography imaging (Fig. (a)).
From functional imaging: Positron Emission Tomography (PET) with FDG
Fluoro-Deoxy-Glucose (Fig. (b)).
Integration of this information by the radiation oncologist to determine the target
volume to be irradiated preserving organs at risk (Fig. (c)).
(a) CT segmentation (b) PET segmentation (c) Treatment planning.
Irradiation everyday during ∼ 35 days.
- 4/25

5. Introduction Method Application to PET image fusion Discussion - Conclusion References
Introduction
PET imaging
PET imaging using three radiotracers
Radiotracer injection according to the biological function to study.
FDG (Fluoro-Deoxy-Glucose): Indicator of glucose metabolism.
FLT (FLuoro-Thymidine): Indicator of cell proliferation [Yang et al., 2010].
FMiso (Fluoro-Misonidazole): Indicator of lack of oxygen (hypoxia).
⇒ cell radioresistance [Chang et al., 2009].
Complementary images: Relevance of information fusion.
(a) PET FDG (b) PET FLT (c) PET FMiso
Transverse slices for a single patient suffering from lung cancer. The time between each
acquisition is smaller than 72 hours. The area of interest (tumor lesion) is located in the
rectangle.
- 5/25

6. Introduction Method Application to PET image fusion Discussion - Conclusion References
Introduction
PET imaging
Imperfections in PET imaging
Low contrast.
Noise:
Relative to the accuracy of a sensor with respect to reality.
Induced by the statistical nature of the signal and by the reconstruction algorithm.
⇒ Information is uncertain.
Partial Volume Effect (PVE) (contamination of neighboring structures):
Induced by the low spatial resolution of the acquisition system.
⇒ Information is imprecise (lack of knowledge) and is mainly localized at the transition
between regions.
(a) PET phantom (b) Histogram (c) PET phantom (d) Profile
Fig. (b) shows a histogram calculated from the theoretically uniform region of Fig. (a). Fig. (d) shows
the profile selected in Fig. (c) presenting fuzzy transitions between regions.
- 6/25

7. Introduction Method Application to PET image fusion Discussion - Conclusion References
Introduction
Théorie des fonctions de croyance
Belief function theory [Dempster, 1967; Shafer, 1976; Smets, 1990]
Frame of discernment (C classes): Ω = {ω1 , ω2 , . . . , ωC }.
Multiple hypotheses are considered: 2Ω = {∅, {ω1 }, {ω2 }, {ω1 , ω2 }, . . . , Ω}.
Belief masses mΩ , also called BBA (Basic Belief Assignment), defined on these
hypotheses (∑A⊆Ω mΩ (A) = 1).
Uncertainty and imprecision modeling.
Belief revision:
Discounting: manage reliability of a source.
Disjunctive combination: fusion of multiple distinct sources whose one is reliable.
Conjunctive combination: fusion of multiple distinct and reliable sources.
- 7/25

8. Introduction Method Application to PET image fusion Discussion - Conclusion References
Introduction
Objectives
Objectives
Dealing with imperfections in image processing:
Reducing uncertainty and imprecision.
Efficiency in the case of low contrasts.
Construction of parametric images to help the radiation oncologist in the
delineation of lesions for achieving a therapy, with potentially:
An increase of radiation session frequency (from one to two per day) for
high-proliferative lesions.
A radiation dose escalation for hypoxic lesions (due to their radioresistance).
- 8/25

9. Introduction Method Application to PET image fusion Discussion - Conclusion References
Method for reducing uncertainty and imprecision in image processing
proposed approach
Method principle - 3 steps
1 BBA estimation for each image.
Measure the membership degrees of each voxel according to two classes using Fuzzy
C-Means (FCM) [Bezdek, 1981].
Integration of a neighboring information fusion using disjunctive rule in the FCM
iterative process.
Conversion of membership degrees into BBA by transferring a part of belief masses for
imperfect data to disjunctions.
Centroid updating only from perfect data.
2 Fusion of neighboring information using Dempster’s rule [Capelle-Laize et al.,
2004; Zhang et al., 2007].
Uncertainty reduction inside noisy regions.
Highlighting of the boundary between regions (due to PVE problems).
3 Fusion of multiple knowledges using conjunctive rule.
Imprecision reduction.
- 9/25

10. Introduction Method Application to PET image fusion Discussion - Conclusion References
Method for reducing uncertainty and imprecision in image processing
proposed approach
Frame of discernment
Two classes:
Ω = {{ω1 }, {ω2 }}
2Ω = {∅, {ω1 }, {ω2 }, {ω1 , ω2 }}
Neighborhood contribution
Vc : the current voxel.
Vi : a neighbor of Vc .
αi : a weighting coefficient associated to Vi depending on the distance separating
Vi from Vc .
Example of αi on PET images:
1
αi = exp (− (Vc − Vi )2 /σ 2 )
2
with σ = FWHM
√ and FWHM = 6 mm the Full Width at Half Maximum of PET
2 2 log 2
images.
- 10/25

11. Introduction Method Application to PET image fusion Discussion - Conclusion References
Method for reducing uncertainty and imprecision in image processing
1 - BBA estimation for each image
Fuzzy C-Means (FCM) clustering algorithm
+ Unsupervised estimation of membership degrees towards the two classes.
+ Efficiency in the case of low contrast.
– Impossible to deal simultaneously with uncertainty and imprecision at the same
time.
Use of belief function theory to improve FCM algorithm.
Combination using disjunctive rule of each voxel with its weighted neighborhood at
each iteration.
⇒ Transfer the belief masses towards hypothesis {ω1 , ω2 } for voxels spatially
ambiguous (noise, PVE).
⇒ Update the class centroids from only certain and precise data.
- 11/25

12. Introduction Method Application to PET image fusion Discussion - Conclusion References
Method for reducing uncertainty and imprecision in image processing
1 - BBA estimation for each image
Weighting
Each neighbor Vi of Vc is ponderated using:
mVi (A)
′
= αi mVi (A) ∀A ≠ ∅
mVi (∅)
′
= 1 − αi (1 − mVi (∅))
The further away from Vc the voxel Vi is, the lower its contribution to the
computation will be.
⇒ Reduction of the influence of distant voxels before fusing them.
Disjunctive combination
Combination of each voxel with its weighted neighborhood:
MVc (.) = ∪
◯ mVi (.)
′
Vi ∈Φ(Vc )
⇒ The belief for ambiguous data is transfered on hypothesis {ω1 , ω2 }.
⇒ The centroid updating inside FCM algorithm is done using only certain and
precise data.
- 12/25

13. Introduction Method Application to PET image fusion Discussion - Conclusion References
Method for reducing uncertainty and imprecision in image processing
2 - Information fusion for reducing uncertainty
Conjunctive fusion of neighboring information
Objective: Take advantage of neighborhood to remove uncertainty.
Fusion of each voxel with its discounted neighborhood using Dempster’s rule.
Discounting
Each neighbor Vi of Vc is discounted using:
mVi (A)
′
= αi mVi (A) ∀A ≠ Ω
mVi (Ω)
′
= 1 − αi (1 − mVi (Ω))
⇒ Reduction of the influence of distant voxels before fusing them.
Conjunctive combination (Dempster’s rule).
Combination of each voxel with its discounted neighborhood:
M′ c (.) =
V ⊕ MVi (.)
Vi ∈Φ(Vc )
⇒ Removing uncertainty due to noise.
⇒ Highlighting imprecision at the boundary between regions.
- 13/25

14. Introduction Method Application to PET image fusion Discussion - Conclusion References
Method for reducing uncertainty and imprecision in image processing
3 - Information fusion for reducing imprecision
Fusion of multiple knowledge
Reducing the imprecision using multiple knowledge.
Combination of one source with a learned knowledge using the conjunctive rule.
Combination of multiple sources of information using the conjunctive rule.
- 14/25

15. Introduction Method Application to PET image fusion Discussion - Conclusion References
Method for reducing uncertainty and imprecision in image processing
Illustration of the method on two simulated images sources
1 BBA estimation for each image.
2 Information fusion for reducing uncertainty.
3 Information fusion for reducing imprecision.
Source 1 Source 2
Gaussian blur salt and
& Gaussian noise pepper noise
⇒ ⇒
1 1
{ω1 } {ω2 } {ω1 , ω2 } {ω1 , ω2 } {ω2 } {ω1 }
⇒ ⇒
2 2
{ω1 } {ω2 } {ω1 , ω2 } {ω1 , ω2 } {ω2 } {ω1 }
⇒ ⇒
3 3
{ω1 } {ω2 } {ω1 , ω2 }
- 15/25

16. Introduction Method Application to PET image fusion Discussion - Conclusion References
Application to PET image fusion
Frame of discernment
Frame of discernment
Five exclusive classes Ω = {N, M, P, H, F }
N: Normal (without any high-uptake).
M: High-glucose Metabolism.
P: High-glucose Metabolism + High cell Proliferation.
H: High-glucose Metabolism + Hypoxia.
F : F ull (High-Métabolisme glucidique + High cell Proliferation + Hypoxia).
Modality FDG FLT FMiso
Background {N} {N, M, H} {N, M, P}
High-uptake {M, P, H, F } {P, F } {H, F }
(a) PET FDG (b) PET FLT (c) PET FMiso
- 16/25

17. Introduction Method Application to PET image fusion Discussion - Conclusion References
Application to PET image fusion
BBA estimation
BBA estimation of each voxel of each image using FCM and neighborhood fusion:
⇒ ⇒
1 2
FDG {N} {M, P, H, F } Ω {N} {M, P, H, F } Ω
⇒ ⇒
1 2
FLT {N, M, H} {P, F } Ω {N, M, H} {P, F } Ω
⇒ ⇒
1 2
FMiso {N, M, P} {H, F } Ω {N, M, P} {H, F } Ω
Background Uptake Background Uptake
- 17/25

18. Introduction Method Application to PET image fusion Discussion - Conclusion References
Application to PET image fusion
Fusion with an a priori knowledge
Problem in PET imaging
Low spatial resolution.
⇒ Partial Volume Effect:
Spill over effect: contamination of neighboring structures.
Spill out effect: underestimation of the tracer concentration in small structures.
⇒ Fuzzy transition between regions which is particularly important for small structures
and for low contrasts.
Fusion of each voxel with an a priori knowledge
Objective: Take advantage of an a priori knowledge, learned from phantom data,
to remove a part of imprecision.
- 18/25

19. Introduction Method Application to PET image fusion Discussion - Conclusion References
Application to PET image fusion
Fusion with an a priori knowledge
Learning
Estimation of PVE from phantom data.
Learning of a function, β(V , C ) ∈ [0, 1], varying according to both volume and
contrast of spheres and corresponding to the part of imprecision.
PET phantom images
(several contrasts).
- 19/25

20. Introduction Method Application to PET image fusion Discussion - Conclusion References
Application to PET image fusion
Fusion with an a priori knowledge
Inclusion of a priori knowledge
Determination of a value β(V , C ) by measuring the volume corresponding to
High Uptake (HU) and the contrast.
Conversion of β(V , C ) in a simple mass function:
mext (HU) = β(V , C )
mext (Ω) = 1 − β(V , C )
Fusion of each voxel with mext using Dempster’s rule.
⇒ Reduction of imprecision due to partial volume effect.
- 20/25

21. Introduction Method Application to PET image fusion Discussion - Conclusion References
Application to PET image fusion
Fusion with an a priori knowledge
For each modality, application of the fusion with the a priori knowledge:
Imprecision reduction:
⇒ ⇒
1,2 3
FDG {N} {M, P, H, F } Ω {N} {M, P, H, F } Ω
⇒ ⇒
1,2 3
FLT {N, M, H} {P, F } Ω {N, M, H} {P, F } Ω
⇒ ⇒
1,2 3
FMiso {N, M, P} {H, F } Ω {N, M, P} {H, F } Ω
Background Uptake Background Uptake
⇒ Possibility to fuse the three PET images.
- 21/25

22. Introduction Method Application to PET image fusion Discussion - Conclusion References
Application to PET image fusion
Multi-modal fusion
Multi-modal fusion
Conjunctive combination of information contained in the three modalities.
Imprecision reduction.
Identification of conflicting regions:
High-Proliferative ({P}) or Hypoxics ({H}) regions presenting a low glucose
Metabolism ({M}).
Conversion of belief masses into plausibility:
Parametric image creation presenting regions requiring a radiotherapy treatment:
classical ({M}).
with an increased frequency of sessions of radiation ({P}).
with an increased irradiation dose on hypoxic lesions ({H}).
with an increase of both frequency of sessions and irradiation dose ({F }).
- 22/25

23. Introduction Method Application to PET image fusion Discussion - Conclusion References
Application to PET image fusion
Multi-modal fusion
Parametric images obtained helping the radiation-oncologist in the radiotherapy
treatment planning.
⇒
FDG
1,2,3
⇒
Pl({M}) Pl({H})
⇒
⇒
FLT multi-modale
1,2,3
Fusion
Pl({P}) Pl({F })
⇒ ⇒
FMiso
1,2,3
Conflict max Pl
- 23/25

24. Introduction Method Application to PET image fusion Discussion - Conclusion References
Discussion - Conclusion
Discussion
Disjunctive combination (inside FCM algorithm) followed by a conjunctive
combination of voxels for each modality:
Can be applied whatever the distribution of initial belief masses is.
Imprecision due to partial volume effect modeling and noise reduction.
Conjunctive combination of multiple sources (multiple images or learned
knowledges):
Imprecision due to partial volume effect reduction.
Conclusion
Using Belief Function theory and spatial information for taking into account both
imprecision and uncertainty.
It offers a great help for radiation oncologist in order to segment lesions from
multi-tracer functional images (FDG, FLT, FMISO).
Future works
Test our method on a larger database to assess its robustness.
Test our method on other images to confirm its genericity.
- 24/25

25. Introduction Method Application to PET image fusion Discussion - Conclusion References
Bibliography I
J. C. Bezdek. Pattern Recognition with Fuzzy Objective Function Algorithms. Kluwer Academic
Publishers, Norwell, MA, USA, 1981. ISBN 0306406713.
A. S. Capelle-Laize, O. Colot, and C. Femandez-Maloigne. Evidential segmentation scheme of
multi-echo mr images for the detection of brain tumors using neighborhood information. Information
Fusion, 5(3):203–216, 2004.
J. Chang, B. Wen, P. Kazanzides, P. Zanzonico, R. D. Finn, G. Fichtinger, and C. C. Ling. A robotic
system for 18F-FMISO PET-guided intratumoral pO2 measurements. Med Phys, 36:5301–5309, Nov
2009.
A. Dempster. Upper and lower probabilities induced by a multivalued mapping. Annals of Mathematical
Statistics, 38(2):325–339, 1967.
G. Shafer. A mathematical theory of evidence. Princeton university press, 1976.
P. Smets. The combination of evidence in the transferable belief model. IEEE Trans. Pattern Anal.
Mach. Intell., 12(5):447–458, 1990.
W. Yang, Y. Zhang, Z. Fu, J. Yu, X. Sun, D. Mu, and A. Han. Imaging of proliferation with 18F-FLT
PET/CT versus 18F-FDG PET/CT in non-small-cell lung cancer. Eur. J. Nucl. Med. Mol. Imaging,
37:1291–1299, Jul 2010.
P. Zhang, I. Gardin, and P. Vannoorenberghe. Information fusion using evidence theory for segmentation
of medical images. In International Colloquium on Information Fusion, volume 1, pages 265–272,
2007.
- 25/25