Download as PDF, PPTX
![Active Shape Model (T. Cootes et al., 1995)
• Representation of Shapes
– Point Distribution Model (PDM)
– Represent a shape instance by a judiciously chosen set of points
(features), each of which is a k-dim vector. N feature points are
stacked into a long vector of length kn:
where
23
q = [p1, . . . , pn]T
pi = (xi, yi), for i = 1, ..., n
landmarks](https://image.slidesharecdn.com/lec12-170330052350/85/Lec12-Shape-Models-and-Medical-Image-Segmentation-23-320.jpg)
![(1) Mean Shape – Model Variation
• We now approximate any instance of the shape, including
the training instances, by projecting onto the first t
eigenvectors:
• The weight vector b is identified as the characteristic of this
instance of the shape:
• Varying the weights bi enables us to explore the allowable
variations in the shape!
32
q = ¯q +
tX
i=1
biui
b = [b1, ..., bt]T](https://image.slidesharecdn.com/lec12-170330052350/85/Lec12-Shape-Models-and-Medical-Image-Segmentation-32-320.jpg)
![Finding the model pose & parameters
• Suppose we have identified a set of points Y in the image.
Evidently, we can seek to minimize the squared distance:
Algorithm
1. Initialize b=0
2. Generate initial model instance
3. Find T that best aligns q to Y (e.g., similarity transform)
4. Invert pose parameters, to project into model frame
5. Update the model parameters:
6. Repeat from step 2 until convergence
40
|Y T(¯q +
tX
i=1
biui)|2
q = (¯q +
tX
i=1
biui)
y = T 1
(Y)
b = UT
(y ¯q) U = [u1|u1|...ut]](https://image.slidesharecdn.com/lec12-170330052350/85/Lec12-Shape-Models-and-Medical-Image-Segmentation-40-320.jpg)
Uploaded byUlaş Bağcı
Lec12: Shape Models and Medical Image Segmentation
This document outlines a lecture on shape models and medical image segmentation, addressing the importance of shape in medical imaging and various shape modeling techniques such as active shape models (ASM) and oriented active shape models (OASM). It details the applications of shape analysis in fields like neuroscience, visual arts, and imaging, while also discussing the methodologies for model construction, landmark selection, and optimization in image segmentation. Additionally, comparisons between ASM and OASM highlight improvements in segmentation accuracy and efficiency.
More Related Content
Similar to Lec12: Shape Models and Medical Image Segmentation
Lec12: Shape Models and Medical Image Segmentation
- 1. MEDICAL IMAGE COMPUTING(CAP 5937) LECTURE 12: Shape Models and Medical Image Segmentation Dr. Ulas Bagci HEC 221, Center for Research in Computer Vision (CRCV), University of Central Florida (UCF), Orlando, FL 32814. bagci@ucf.edu or bagci@crcv.ucf.edu 1SPRING 2017
- 2. Outline • Shape Information –Representation – Simple processing (dilation + erosion) • Shape Modeling – M-reps – Active Shape Models (ASM) – Oriented Active Shape Models (OASM) – Application in anatomy recognition and segmentation – Comparison of ASM and OASM 2
- 3. 3 Motivation Most structures of clinicalinterest have a characteristic shape and anatomical location relative to other structures Brain image shows: • Ventricles • Caudate nucleus • Lentiform nucleus
- 4. 4 Motivation Heart model withlarge vessels
- 5.
- 6. What is Shape? •Shape is any connected set of points! 6
- 7. What is Shape? •Shape is any connected set of points! 7 Shape (S) Points T: boundary P: interior Q: exterior
- 8. Example Applications forShape Analysis 8 NeuroScience • Morphological taxonomy of neural cells • Interplay between form and function • comparisons between cells of different cortical areas • Comparisons between cells of different species • … Internet/Document Analysis • Content-based information retrieval • Watermarking • Graphic design • WWW • Optical character recognition • Multimedia databases • … Visual Arts • Video restoration • Video tracking • Special effects • Games • Computer graphics • Image synthesis • Visualizations • … Imaging/Vision • Pathology detection/classification (spiculated and spherical nodules) • 3D pose estimation • Morphological operations • Anatomy modeling • Segmentation • Registration • Volumetry Other areas: medicine, biology, engineering,physics, agriculture, security…
- 9. Shape Models • Pointdistribution, Active Shape, Active Appearance models (Cootes & Taylor) • Fourier Snakes (Szekely) • Active Contours (Blake) • Parametrically-deformable models (Staib & Duncan) • Medial Representation (m-rep) • … 9
- 10. Shape Models • Pointdistribution, Active Shape, Active Appearance models (Cootes & Taylor) • Fourier Snakes (Szekely) • Active Contours (Blake) • Parametrically-deformable models (Staib & Duncan) • Medial Representation (m-rep) • … 10 Active Shape Models can be used to help interpret new images by finding the parameters which best match an instance of the model to the image.
- 11. The shape ofan organ can be characterized as being a transformed version of some template 11 template Credit: Parmedco.ir Deformations/ transformation
- 12. What shape representationis for • Analysis from images – Extract the kidney-shapedobject – Register based on the pelvic bone shapes Credit: Pizer
- 13. What shape representationis for 13 Snapshots of subcortical shape evolution after geodesic regression on a multi-object complex: putamen,a mygdala,and hippocampus. Credit: Gerig
- 14. What shape representationis for 14 A graphical visualization is presented for the left/right asymmetry of the amygdala–hippocampalcomplex for healthy controls (top row) and patients with schizophrenia (bottom row).
- 15. What shape representationis for • Shape science – Shape and biology – Shape-based diagnosis Brain structures (Gerig)
- 16. Primitives for shaperepresentation: Landmarks • Sets of points of special geometry
- 17. Primitives for shaperepresentation: Boundaries • Boundary points with normals
- 18.
- 19. Representing Boundary Displacements •Along figurally implied boundary normals • Coarse-to-fine • Captures along-boundary covariance • Useful for rendering
- 20. Medial Primitives • x,(b,n) frame, r, θ (object angle) • Imply boundary segments with tolerance • Similarity transform equi-variant – Zoom invariance implies width-proportionality of • tolerance of implied boundary • boundary curvature distribution • spacing along net • interrogation aperture for image θ b rR(- θ)b x rR(θ)b n
- 21. Multiscale Medial Model •From larger scale medial net, interpolate smaller scale medial net and represent medial displacements b.
- 22. Medial Net ShapeModels Medial nets, positions onlyMedial net
- 23. Active Shape Model(T. Cootes et al., 1995) • Representation of Shapes – Point Distribution Model (PDM) – Represent a shape instance by a judiciously chosen set of points (features), each of which is a k-dim vector. N feature points are stacked into a long vector of length kn: where 23 q = [p1, . . . , pn]T pi = (xi, yi), for i = 1, ..., n landmarks
- 24. Active Shape Model– Landmark Representation 24 Landmarks on VB (vertabrae),DXA
- 25. Active Shape Model– How to? (1) Build a statistical boundary shape model that consists of a mean shape and its allowable variations. (2) Use the model to recognize the boundary in the given scene. (3) Use the model to fit to the information in the scene. The resulting model is considered to be the boundary delineation. 25
- 26. (1) Mean Shape •Model is constructed via a set of landmarks or homologous points. • Align the shapes after landmarking corresponding points – The Procrustes Algorithm is used to that the sum of instances to the mean of each shape is minimized! Why? • The mean shape and allowable variations are determined from a set of N training shapes 26 X i=1,...,M (qi ¯q)2
- 27. (1) Mean Shape 27 …… LandmarkSelection
- 28. (1) Mean Shape– Alignment Step 28 Shape Alignment Before After
- 29.
- 30.
- 31.
- 32. (1) Mean Shape– Model Variation • We now approximate any instance of the shape, including the training instances, by projecting onto the first t eigenvectors: • The weight vector b is identified as the characteristic of this instance of the shape: • Varying the weights bi enables us to explore the allowable variations in the shape! 32 q = ¯q + tX i=1 biui b = [b1, ..., bt]T
- 33. Variation in ShapeModel 33 Credit: Inst. of Medical Sciences Univ. of Aberdeen
- 34. (1) Mean Shape– Example Modes 34 1st mode 2nd mode 3rd mode mean -2 std mean mean+2 std Shape instances generated (talus bone of foot):
- 35. (1) Mean Shape– Example Brain Structures 35
- 36. (1) Mean Shape– Example Modes 36 Shape instances generated (talus bone of foot): 3 31 1λ λ− ≤ ≤b 1 3 32 2λ λ− ≤ ≤b 2
- 37. (2) Placing themodel – ASM Search • (0) Specify mean shape location. • (1) Along line segments orthogonal to current shape, determine intensity profile. • (2) Reposition landmark along line to match boundary information. • (3) Subject landmarks to model constraints. If convergence, stop. Else go to (1). 37
- 38. 38 (2) Placing themodel – ASM Search ),( ii YX Need to search for local match For each point: -strongest edge -correlation -statistical model of profile
- 39. Deeper in ASMSearch 39 Model boundary Model point Interpolate at these points ,...2,1,0,1,2... ),(),( −−= + i nsnsiYX ynxn ),( YX xnns ynns ns ),(alonglengthofstepsTake yxn nns )(xg x dx xdg )( ))1()1((5.0 )( −−+= xgxg dx xdg Select point along profile at strongest edge
- 40. Finding the modelpose & parameters • Suppose we have identified a set of points Y in the image. Evidently, we can seek to minimize the squared distance: Algorithm 1. Initialize b=0 2. Generate initial model instance 3. Find T that best aligns q to Y (e.g., similarity transform) 4. Invert pose parameters, to project into model frame 5. Update the model parameters: 6. Repeat from step 2 until convergence 40 |Y T(¯q + tX i=1 biui)|2 q = (¯q + tX i=1 biui) y = T 1 (Y) b = UT (y ¯q) U = [u1|u1|...ut]
- 41. Example Application: KneeCartilage quantification - MRI 41
- 42. Ex: MR Brainstructure segmentation 42
- 43. ASM Summary 43 + Shapeprior helps overcoming segmentation errors + Fast optimization + Can handle interior/exteriordynamics Pros: Cons: Possible improvements: Learn and apply specific motion priors for different actions - Optimization gets trapped in local minima - Re-initialization is problematic
- 44. ASM - Problems 44 ASMresultTrue boundary
- 45. ASM Problems 45 10 points 20points Talus 1 Liver 1 2D landmarking How about 3D? Time consuming! Correspondence Issue!
- 46. Alternative solutions forlandmarking • Based on image registration • Based on MDL (min description length) and optimization techniques • Based on shape characteristics of the objects 46
- 47. ASM Problems –Initialization Sensitivity 47
- 48. Oriented Active ShapeModels (OASM) 48 (1) Selecting landmarks. (2) Building models. (3) Creating boundary cost function. (4) Coarse level recognition. ß (5) Fine level recognition & delineation – 2LDP. ß training Overview of Approach
- 49. OASM (Liu andUdupa) 49 P3 P1 P2 P4P5 P6 Recognition Delineation P1 P2 P6 P1… - Construct a graph to set up synergy between recgn. and deln. - Automate recognition. (5) Fine level recognition & delineation – 2LDP
- 50. 50 Delineation P1 P2 P6P1 Recognition P1 P2 P3 P4 P5 P6 - Construct a graph to set up synergy between recgn. and deln. - Automate recognition. OASM (Liu and Udupa)
- 51. 51 Recognition Delineation P1 P2 P6P1 P1 P2 P3 P4P5 P6 - Construct a graph to set up synergy between recgn. and deln. - Automate recognition. OASM (Liu and Udupa)
- 52. 52 OASM (Liu andUdupa) . . .Recognition Delineation P1 P2 P6 P1 P1 P2 P3 P4P5 P6 - Construct a graph to set up synergy between recgn. and deln. - Automate recognition.
- 53. 53 Recognition Delineation P1 P2 P6P1 P1 P2 P3 P4P5 P6 - Construct a graph to set up synergy between recgn. and deln. - Automate recognition. OASM (Liu and Udupa)
- 54. OASM – Recognition(model placement) Coarse level recognition • At each location in a coarse grid, evaluate the total boundary cost (from Live Wire) via 2LDP by placing the mean shape at that point. • Select that location for which this total cost is the minimum. 54
- 55. OASM – Recognition(model placement) 55
- 56. OASM - Segmentation 56 Automaticrecognition Final segmentation
- 57. Comparison with ASM 57 ASMn=36 OASM n=20
- 58. Comparison with ASM 58 OASMn=28ASM n=28
- 59.
- 60.
- 61.
- 62.
- 63. Active Appearance Model •PLEASE READ THE FOLLOWING PAPER • T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc. European Conference on Computer Vision 1998 Vol. 2, pp. 484- 498, Springer, 1998. (Best paper prize) 63
- 64. Summary • Shape – Shaperepresentation (landmark based) • ASM – Model placement, ASM search (optimization) • OASM – Orientedness – Cost function – Less number of landmarks – Improved results 64
- 65. Slide Credits andReferences • Credits to: Jayaram K. Udupa of Univ. of Penn., MIPG • Bagci’s CV Course 2015 Fall. • TF. Cootes et al. ASM and their training and applications, 1995. • S.Pizer (UNC) presentations • K.D. Toennies, Guide to Medical Image Analysis, • Shape Analysis in Medical Image Analysis, Springer. • Handbook of Medical Imaging, Vol. 2. SPIE Press. • Handbook of Biomedical Imaging, Paragios, Duncan, Ayache. 65