Your SlideShare is downloading. ×
Registration & Modeling of Shapes with Uncertainties
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Registration & Modeling of Shapes with Uncertainties


Published on

Contributions and Applications to Knowledge Based Segmentation. …

Contributions and Applications to Knowledge Based Segmentation.
Presentation held on November the 30th.

Published in: Technology, Business

  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide
  • Transcript

    • 1. Registration & Modeling of Shapes with Uncertainties: Contributions and Applications to Knowledge Based Segmentation. PhD Thesis Defense, Presented on November the 15 th of 2007 by Maxime TARON Thesis advisors : Nikos Paragios Marie-Pierre Jolly
    • 2. Medical Imaging : the Modern Challenges
      • Important needs:
        • Hospitals are producing an increasing numbers of acquisitions.  full manual processing is long and demanding.
        • Large number of medical image modalities (US, PET, MRI, CT etc..)
      Introduction Ultrasound Imaging Positron emission tomography Computer Tomography Magnetic Resonance Imaging
    • 3. Medical Imaging : the Modern Challenges
      • Goal medical Image Processing: Assist physicians in their diagnostic task.
        • Provide pre-visualization tools
        • Automated tools for organ segmentation, measurements and comparison with theoretical behavior.
    • 4. Context and Motivations (1)
      • Modeling complex anatomical structures often consists in three steps:
        • recovering a set of anatomical measures from different medical image modalities (landmarks, contours, points clouds),
        • proposing a parametric mathematical model that is consistent with the anatomy and is capable of describing the variations of the organ.
        • estimating the parameters of the model : reproduce the behavior previously observed.
        •  These 3 steps are most often considered as independent…
      Anatomical Landmarks Manual Segmentations Introduction
    • 5. Context and Motivation (2)
      • But they are not:
        • Variations exists among experts manual segmentation of organs
        • Ambiguity exist due to the lack of visual support.
        • Some automatic segmentation methods recover noisy data.
        •  We introduce uncertainty as a tool to introduce variability inherited from the shape in the statistical model building.
      Introduction Animation
    • 6. Overview
      • Shape Alignment:
        • Shape Registration.
        • Computing Uncertainties.
    • 7. Overview
      • Statistical Model Building:
        • Building parametric shape model.
        • Statistical Modeling
          • Linear method (ICA)
          • Non-parametric method
      • Knowledge Based Segmentation.
    • 8. 1 st part: Shape Registration & Uncertainty Estimation
      • Shape Registration
        • Registration Criterion
        • Parametric Transformations
      • Uncertainty Estimation
        • Hessian-based uncertainty.
        • Data-based uncertainty.
      Part 1
    • 9. Shape Registration
      • Retrieve the ‘Best’ diffeomorphism able to deform and align an initial shape (Source) onto an object shape (Target)
        • Consider parametric transform depending on a parameter vector:
        • Global Registration : parameters are acting on the entire shape (Rigid, Affine, Quadratic transform)
        • Local registration : parameters are acting locally on the shape. Build local correspondence between Shape/Surfaces
      Global Local Part 1 – Shape Registration
    • 10. Registration Criterion
      • Consider signed distance transforms and of the shapes and .
        • Multiscale representation properties.
        • Invariant under Similarity transform.
      • According to the definition of the distance transform, the 0 levelset is unaltered by any transformation of the shape.
        • Basic Registration criterion aligns the zero levelset of the source with the zero levelset of the target :
          • Where is a symmetric cost function
        • Extend this criterion to non-zero levels sets. ( Rousson (2002) Huang (2003) )
      Part 1 – Shape Registration
    • 11. Deformable template and distance map deformations
      • Complex transformations alter the Euclidean distance:
        • distance transforms of two shapes produce similar observations also at the other iso-surfaces.
        • A narrow band in the neighborhood of the ‘0’ levelset is considered.
        • is the characteristic function of
      Part 1 – Shape Registration
    • 12. Types of Transformations : Global Transform
      • Similarity transform are considered in medical applications :
        • Used to remove the pose parameters independent of anatomical considerations.
      Part 1 – Shape Registration Affine Identity Rigid + scale
    • 13. Local Transform : Properties of Free Form Deformation and Thin Plate Splines
      • Both parametric transform, linear in the set of parameters : with , a vector depending non linearly on the considered point .
      • TPS
      • Control Points sampled anywhere in space.  Positioned on the surface
      • deformation and on control points in the 3D case.
      • Low degree of freedom.
      Part 1 – Shape Registration
    • 14. Local Transform : Properties of Free Form Deformation and Thin Plate Splines
      • Both parametric transform, linear in the set of parameters : with , a vector depending non linearly on the considered point .
      • FFD
      • Control points located on a regular grid.
      • deformation everywhere.
      • are cubic polynomial expressions, with compact support.
      Part 1 – Shape Registration
    • 15. Deformable template and distance map deformations
      • Distance map contains a multiscale shape representation
        •  Rough registration is performed using « large band » and sparsely sampled points.
      • Updating the transform.
        • Increasing the number of degrees of freedom of the considered transformation.
      • Updating the distance map:
        • the actual distance map of the transformed shape is also re-computed.
      Part 1 – Shape Registration
    • 16. Registration : a review
      • The minimization procedure will lead to a minimum that most probably will be a local one (high dimensional non-convex cost functions)
      • Even if minimum is global, does it really correspond to a meaningful solution? Universal?
      • Consider the registration process of a certain sample:
        • The solutions is not unique and should form a probability density function.
        • Assume that the probability density of a configuration (parameter ) is related to the change in the cost function.
      • It is impossible to evaluate the registration energy for each configuration (Exhaustive Search):
        • The covariance of this pdf is a good indicator on the quality of the obtained solution.
      Part 1 – Uncertainty Estimation
    • 17. Related Work on Uncertainty:
      • (Pennec Thirion 1995) have introduced uncertainty estimates on the registration of corresponding points sets with rigid motion.
      • (Simoncelli 1999) has introduced uncertainty within estimation of dense optical flow. Based on a probabilistic model for the error on the derivative estimates (temporal & spatial)
      • (Rohr 2003) has used uncertainty on landmarks locations for pointwise registration with application to MRI registration. Landmarks are given along with covariance information or a particular direction.
      • (Stewart 2005) with dual bootstrap ICP for points cloud registration is using uncertainty estimate on the transformation to select the area where registration is performed and the complexity of the transformation.
      Part 1 – Uncertainty Estimation
    • 18. Introducing Uncertainty: Hessian Based.
      • Uncertainties are expressed using a covariance matrix on the vector near the optimal registration , denoted by .
      • Consider Registration energy on the surfaces (limit case of the registration process)
      • Compute the expression of the Hessian :
      Part 1 – Uncertainty Estimation
    • 19. Introducing Uncertainty: Hessian Based (2)
      • It can be shown that the Hessian of this energy presents a first order approximation that does not require the use of second derivatives on the distance maps.
      • Expression of the Hessian:
      • Enforce invertible matrix with the use of regularization term.
      Part 1 – Uncertainty Estimation
    • 20. An different approach : Data Based Uncertainty
      • Data corresponds to the location of the Target Shape boundary.
      • Study the variations of under small normal variation of .
      • Assume sampled points of the boundary undergo Gaussian independent displacements with variance , data-based uncertainty propagates local covariance to the vector of parameters .
      Part 1 – Uncertainty Estimation
    • 21. Comparison between Hessian and Data driven
      • Hessian/Data Based lead to different results:
        • Some displacements of cannot be explained with variations of data (data-based uncertainty has a non-empty null space).
        • Hessian-based uncertainty is defined on the entire space. Some displacements of turn irrelevant as the regularization term becomes negligible.
      • Uncertainty will be used to build a prior on the entire parameter space. Relax constraints on to represent the shape deformations:
        • Hessian based uncertainty is retained.
      Part 1 – Uncertainty Estimation
    • 22. Uncertainty Visualization:
      • Although Uncertainty is computed under the form of a matrix, one can only visualize the uncertainty on every control points, considered as independent. This does not account for covariance factors between control points.
      Part 1 – Uncertainty Estimation
    • 23. 2 nd part: Modeling Shape Variations
      • Linear shape modeling applied to cardiac left ventricle.
        • Initial Reference shape.
        • Uncertainty-Driven Independent Component Analysis.
      • Non-parametric shape modeling applied to the Corpus Callosum
        • Initial Reference shape.
        • Kernel density estimation
      Part 2
    • 24. Problem statement and Reference Shape
      • Assume we have a training set of examples onto which registration has been solved.
      • Retreive a set of deformations represented as
      Part 2 – Reference model
    • 25. Problem statement and Reference Shape
      • Assume we have a training set of examples onto which registration has been solved.
        • Retreive a set of deformations represented as
        • Each registration output has associated uncertainty
      Part 2 – Reference model
    • 26. Problem statement and Reference Shape
      • Create a reference model capable of deforming to any sample of the training set:
        • Smooth out any details of the surface smaller than the distance between control points.
      • Construction of an average shape inspired from the Active Shape Models ( Cootes et al. 95 ):
      • Build a prior density which only accounts for the local deformation of the organs.
      Part 2 – Reference model
    • 27. Statistical Modeling of Shapes: a review
      • Modeling should be invariant through similarity transform :
        • Subtract the similarity from the retrieved transformation of the registration step.
        • Statistical shape model only accounts for local variations of shapes.
      • These problem have been well studied when shapes are considered as single points of a high dimensional space:
        • Gaussian models / mixtures
        • Linear Subspace Models (PCA ICA)
        • Non-parametric Density approximations
        • Kernel Principal Component Analysis
      •  All these methods treat modeling independently of the registration problem.
      • It completely ignores errors due to erroneous registration.
      Part 2 – Introduction
    • 28. Linear Model
      • Low dimensional representation of the shape variations.
      • Classical Approaches : PCA vs. ICA
        • PCA : high variance, parts of the ventricle highly correlated.
        • ICA : independent components show localized variations.
      Part 2 – Linear Model ICA 1 st modes using PCA 1 st modes using ICA Animation Animation
    • 29. Introducing ICA
      • Independent component analysis shows a different separation of ventricular deformations.
        • Definition: The « Best » linear transform of the parameter vector that results in statistically independent components. (FastICA)
      Part 2 – Linear Model ICA Distribution in the parameter space Distribution in the ICA space s1 s2
    • 30. Generate new samples
      • Computation of ICA requires a set of points sampled from the distribution to be estimated.
        • For every example of the training set, we draw new samples using the Gaussian law with covariance :
      Part 2 – Linear Model ICA
    • 31. Building Statistical model using ICA
      • Can be estimated with a low number of training samples.
      • Reconstructing the parameter vector density:
        • with a Gaussian mixture estimate of the probability density of the i th independent component .
      Part 2 – Linear Model ICA Projection of the Data on Independent Components
    • 32. Towards non-parametric modeling,
      • Linear Model is not sufficient to model certain variations:
      • Non parametric = no assumptions on the distributions.
      • Create a reference model capable of deforming to any sample of the training set.
      Part 2 – non-linear Model
    • 33. Non-parametric Density approximation with kernels
      • Kernels can do a better job in the approximation of densities lying in non-linear subspace.
      • Using a Gaussian kernel centered at every training sample position.
        • Limitation : estimating the covariance of the kernel.
        • does not account for the shape registration errors.
      Part 2 – non-linear Model
    • 34. Non-parametric Density approximation with variable bandwidth kernels.
      • Every registered shape of the training set with parameter , along with a uncertainty matrix is used to generate a Gaussian Kernel:
      • Gaussian Kernels are cumulated to generate an a priori density in the parameter vector :
      Part 2 – non-linear Model
    • 35. Dimensionality Reduction
      • Kernel-based densities are computationally expensive when the number of examples increases:
        • Retain a subset of kernels that best express the global density.
      • Consider a set of kernels extracted from a training set
        • Each is associated to Gaussian random variable .
      • Compute the probability of a shape with associated Gaussian density with parameters
      • maximize this quantity on the whole training set:
      Part 2 – non-linear Model
    • 36. 3 rd part: Knowledge-Based Segmentation
      • Cardiac left ventricle segmentation from CT-scan.
      • Segmentation of the Corpus Callosum on MR images
      Part 3
    • 37. Segmentation with Model Fitting.
      • Transform a shape according to a deformation model with parameter .
      • Define an objective function which accounts for image support and prior knowledge.
      • : Region based energy term.
        • Intensity distribution of different regions is known a-priori and the model is aligned to maximize the likelihood of each regions.
      • : Shape based Energy term.
        • Assess the quality of the current segmentation with respect the statistical model.
      Part 3 - Introduction
    • 38. Cardiac Left Ventricle Segmentation : Region Based Image Term
      • Consider 4 Regions for segmentation. Left Ventricle, and interfaces with Lung, Blood Pool and Right Ventricle :
        • Where refers to the log-likelihood of the grey level density associated to the region
      • Differentiation lead to an expression on the considered interfaces
      Blood Pool LV Lung RV Setting Set of Points on Interfaces between regions Setting Set of Points on Interfaces between regions Part 3 – Left Ventricle Segmentation Animation
    • 39. Knowledge-based Segmentation.
      • Classical Segmentation Energy, « Image Term » and « Prior Term » :
        • is the prior density of the deformations, estimated from the training set with ICA
      Part 3 – Left Ventricle Segmentation Animation
    • 40. Qualitative Results Part 3 – Left Ventricle Segmentation Animation
    • 41. Qualitative Results Part 3 – Left Ventricle Segmentation
    • 42. Qualitative Results Part 3 – Left Ventricle Segmentation
    • 43. Estimate of Segmentation Uncertainty
      • Computation of cumulative uncertainties along the surface:
      • The amount of local information on the surface accounts for edges and local normal direction.
        • With , , the distance to the transformed shape,
        • a point of the transformed surface.
      Part 3 – Left Ventricle Segmentation
    • 44. Segmentation of the Corpus callosum: Region based image term.
      • Based on the separation of histograms in segmented areas
      • Minimizing through gradient descent can lead to an expression on the interface Corpus Callosum/Exterior.
      Part 3 – Corpus Callosum Segmentation
    • 45. Knowledge based segmentation, introducing shape uncertainty.
      • Classical Segmentation Energy, « Image Term » and « Prior Term » :
        • is the prior density of the deformations, estimated from the training set with kernel density estimation.
      • Due to the linear form of the shape boundary, uncertainty in the segmenting shape exists and can be expressed:
    • 46. Variable Bandwidth Kernel Densities: Hybrid estimator
      • The prior shape based term accounts for the uncertainty:
        • replaced with the hybrid kernel density estimator.
      • Shape term becomes less important in the direction of weak uncertainties.
      • Shape term present a more important smoothing in the directions of high uncertainty and less sensitive to local minimum.
      Part 3 – Corpus Callosum Segmentation
    • 47. Part 3 – Corpus Callosum Segmentation Animation
    • 48. General Conclusion
      • Contributions.
      • Future Directions.
    • 49. Contributions.
      • We have extended shape registration process with distance transform in a coarse to fine framework.
      • We have introduced Uncertainties which assess quality of the registration.
      • We have built a model that propagates the uncertainty of the shape registration step to the statistical modeling.
      • We have used this prior model for organ segmentation.
      • We have proposed a framework that takes advantage of uncertainty estimates during the segmentation process.
      Conclusion Prior shape Model + Posterior Registration Modeling Segmentation Uncertainty Uncertainty
    • 50. Future Directions
      • Consider relations between Uncertainty computation and statistical learning with the use of Riemannian geometry and manifold learning tools.
        • Extend this approach to KPCA/KICA.
      • Exploit the potentials of Image-based segmentation uncertainties.
        • Relate it to the prior shape model.
        • Adapt it to provide the physician with user-friendly diagnosis tool assessing the quality of the segmentation.
      • Comparison with similar results obtained using discrete optimization methods.
    • 51. Related Publications
      • Journal:
        • M. Taron, N. Paragios & M.-P. Jolly. Registration with Uncertainties and Statistical Modeling of Shapes with Variable Metric Kernels. IEEE Transactions on Pattern Analysis and Machine (to appear)
      • Conferences:
        • M. Taron, N. Paragios & M.-P. Jolly. Modelling Shapes with Uncertainties : Higher Order Polynomials, Variable Bandwidth Kernels and Non-Parametric Density Estimation. IEEE International Conference in Computer Vision, 2005.
        • M. Taron, N. Paragios & M.-P. Jolly. Uncertainty-driven Non-parametric Knowledge-based Segmentation:The Corpus Callosum Case. 3rd ICCV workshop on Variational Geometric and Level Set Methods (VLSM), 2005.
        • M. Taron, C. Ghys & N. Paragios. Uncertainties -driven Surface Morphing: The case of Photo-realistic Transitions between Facial Expressions. In 18th International Conference on Pattern Recognition (ICPR), Hong Kong, 2006.
        • M. Taron, N. Paragios & M.-P. Jolly. From Uncertainties to Statistical Model Building and Segmentation of the Left Ventricle. Mathematical Methods in Biomedical Image Analysis (MMBIA), 2007.
    • 52. Thank you for your attention Animation
    • 53.  
    • 54. An application of the 3D Free Form Deformation : 3d morphing:
      • Collaboration with Charlotte Ghys.
      • Acquisition and Reconstruction of 3D Faces using stereo.
      • 3D registration of the mesh for different facial expressions.
      • Texture mapping using a standard 2D morphing algorithm.
      Appendix Animation Animation
    • 55. 3D Shape Registration on Full Body Meshes
      • Initial model Deformed model Target
      Videos Appendix Animation
    • 56. Application to digits Recognition (1)
      • 2DRegistration with uncertainty on a database of 2000 handwritten digits ‘3’ and 2000 ‘9’
      • Estimating density using kernels density estimation, selecting 50 best approximating kernels.
    • 57. Application to digits Recognition (2) Appendix