Clustering – K-means – FCM (fuzzyc-means) – SMC (simple membership based clustering) – AP(affinity propagation) – FLAB(fuzzy locally adaptive Bayesian) – Spectral Clustering Methods ShapeModeling – M-reps – Active Shape Models (ASM) – Oriented Active Shape Models (OASM) – Application in anatomy recognition and segmentation – Comparison of ASM and OASM ActiveContour(Snake) • LevelSet • Applications Enhancement, Noise Reduction, and Signal Processing • MedicalImageRegistration • MedicalImageSegmentation • MedicalImageVisualization • Machine Learning in Medical Imaging • Shape Modeling/Analysis of Medical Images Deep Learning in Radiology Fuzzy Connectivity (FC) – Affinity functions • Absolute FC • Relative FC (and Iterative Relative FC) • Successful example applications of FC in medical imaging • Segmentation of Airway and Airway Walls using RFC based method Energy functional – Data and Smoothness terms • GraphCut – Min cut – Max Flow • ApplicationsinRadiologyImages

1. MEDICAL IMAGE COMPUTING (CAP 5937)
LECTURE 13: Clustering Based Medical Image Segmentation
Methods
Dr. Ulas Bagci
HEC 221, Center for Research in Computer
Vision (CRCV), University of Central Florida
(UCF), Orlando, FL 32814.
bagci@ucf.edu or bagci@crcv.ucf.edu
1SPRING 2017

2. Reminder (Project Selection)
• Lung Lobe Segmentation from CT Scans (Use LOLA11 Segmentation Challenge Data Set)
• Segmentation of Knee Images from MRI (Use SKI 2010 Data Set))
• Multimodal Brain Tumor Segmentation (Use BraTS Data Set)
• Automatic Lung Nodule (cancer) Detection (Use LUNA 2016 Data Set, http://luna16.grand-
challenge.org/home/ (Links to an external site.))
• Automatically measure end-systolic and end-diastolic volumes in cardiac MRIs. (Use Kaggle
Cardiac Data Set)
• Head-Neck Auto Segmentation Challenge (Use MICCAI 2015 Segmentation Challange Data
Set)
• CAD of Dementia from Structural MRI (Use MICCAI 2014 Segmentation Challenge Data
Set)
• DTI Tractography Challenge (Use MICCAI 2014 Segmentation Challenge Data Set)
• EMPIRE 2010 - Pulmonary Image Registration Challenge
(http://empire10.isi.uu.nl/index.php, (Links to an external site.) I have the team name and
password for downloading the data set).
• Digital Mammography DREAM challenge
(https://www.synapse.org/#!Synapse:syn4224222/wiki/231837 (Links to an external site.))
• MACHINE LEARNING Challenge in medical imaging
(https://www.nmr.mgh.harvard.edu/lab/laboratory-computational-imaging-biomarkers/miccai-
2014-machine-learning-challenge (Links to an external site.)
2

3. Reminder (Project Selection)
• MICCAI 2017 Challenges (end of march, sites will be fully functional)
3

4. Outline
• Clustering
– K-means
– FCM (fuzzy c-means)
– SMC (simple membership based clustering)
– AP (affinity propagation)
– FLAB (fuzzy locally adaptive Bayesian)
– Spectral Clustering Methods
4

5. What is Clustering?
• Organizing data into classes such that:
– High intra-class similarity
– Low inter-class similarity
• Finding the class labels and the number of classes directly
from the data (as opposed to classification tasks)
5

6. What is a natural grouping?
6

7. What is a natural grouping?
9/29/15
7
Clustering is subjective
School EmployeesSimpson's Family MalesFemales

8. What is similarity ?
8

9. What is similarity ?
9
Cluster by features
• Color
• Intensity
• Location
• Texture
• ….

10. Distance metrics
10
0.23 3 342.7
Peter Piotr

11. Motivation for Clustering in Medical Image
Segmentation
• Assumption: The object of interest can be identified as a cluster in
an appropriate feature space. Delineate the cluster to delineate the
object.
11

12. • Clustering algorithms essentially perform the same function
as classifier methods without the use of training data.
• Thus, they are termed unsupervised methods.
• In order to compensate for the lack of training data, clustering
methods iterate between segmenting the image and
characterizing the properties of the each class.
12
FCM FCM with
Markov PriorPham et al

13. Motivation in Biomedical Image Segmentation
13
Segmenting the image based on threshold implies underlying a posteriori probabilities
for the different classes of which the histogram shows a non-normalized sum

14. K-means Clustering
14

15. K-means Clustering
15

16. K-means Clustering
16

17. K-means Clustering
17

18. K-means Clustering
18

19. K-means Clustering
19

20. K-means Clustering
9/29/15
20

21. 21Credit: Zitao Liu

22. Recap: Core of Clustering
• Similarity metric
• Distance metric
22
Open-box
Spectral clustering
Software for
Medical images
Schultz and Kindlmann
2013, IEEE TVCG

23. Multispectral/Multimodal Image Segmentation
• The segmentation techniques based on integration of
information from several images are called multispectral or
multimodal
23

24. Multispectral/Multimodal Image Segmentation
• The segmentation techniques based on integration of
information from several images are called multispectral or
multimodal
• In multispectral images, each pixel is characterized by a set
of features and the segmentation can be performed in
multidimensional (multichannel) feature space using
clustering algorithms
24

25. Methods:
k-Nearest Neighbor (kNN)
C-Means (CM)
Fuzzy C-Means (FCM)
Simple Membership-based Classification (SMC)
Methods differ based on how clusters are defined,
detected, and delineated.
pI Methods – RpI : Clustering

26. Training:
For each of L object classes, determine a true
classification feature set Fi, i = 1, 2,…., L.
Classification:
(1) Choose and fix a value for K (say K = 7).
(2) For a given scene S = (C, f) to be segmented,
determine the feature vector xc associated with
each voxel c.
RpI : Clustering - kNN

27. (3) To classify voxel c in C, among all feature vectors
in , determine K vectors that are
closest to xc.
(4) Classify c to that among
the L classes which is
represented maximally in
RpI : Clustering - kNN

28. Applications in MR Brain Tissue Segmentation
28

29. MRI T2
Original
MRI PD
Original
GM, WM, CSF
Segmented
RpI : Clustering - kNN

30. It is an unsupervised method. No training needed.
• Requires number of object classes L to be specified.
• Outputs fuzzy membership, at each voxel, in
individual object classes.
• Also outputs class mean (centroid)
• Uses an optimization technique to determine how to
optimally partition the feature space into L classes.
RpI : Clustering - FCM

31. S = (C, f) : a given scene
uck : membership of voxel c in object class k.
q : a weighting exponent (q = 2 often used).
f(c) : vector-valued scene intensity.
µk : centroid (mean) of class k.
|| • || : any inner product norm (e.g., Euclidean
norm)
( )
2
1
L
q
ck k
c C k
J u f c µ
∈ =
= ∑ ∑ −
Minimization of J yields uck and µk .
RpI : Clustering - FCM

32. Applications in T2-MRI Segmentation via K-
means (K=9)
32

33. Quantitative Estimation of Volumetric Breast
Tomosynthesis Images (Pertuz et al, Radiology 2015)
33

34. Quantitative Estimation of Volumetric Breast
Tomosynthesis Images (Pertuz et al, Radiology 2015)
• Breast cancer is the most commonly diagnosed cancer in the
United States and the second leading cause of death from
cancer in women.
34

35. Quantitative Estimation of Volumetric Breast
Tomosynthesis Images (Pertuz et al, Radiology 2015)
• Breast cancer is the most commonly diagnosed cancer in the
United States and the second leading cause of death from
cancer in women.
• Breast density is an independent risk factor for breast
cancer
35

36. Quantitative Estimation of Volumetric Breast
Tomosynthesis Images (Pertuz et al, Radiology 2015)
• Breast cancer is the most commonly diagnosed cancer in the
United States and the second leading cause of death from
cancer in women.
• Breast density is an independent risk factor for breast
cancer
• Objective and accurate methods for the estimation of density
are needed to ensure reliable estimation and, ultimately, yield
quantitative reproducible measures that are clinically useful.
36

37. Quantitative Estimation of Volumetric Breast
Tomosynthesis Images (Pertuz et al, Radiology 2015)
• Breast cancer is the most commonly diagnosed cancer in the
United States and the second leading cause of death from
cancer in women.
• Breast density is an independent risk factor for breast
cancer
• Objective and accurate methods for the estimation of density
are needed to ensure reliable estimation and, ultimately, yield
quantitative reproducible measures that are clinically useful.
• Volume-based quantitative density methods, the aim is to
better estimate the amount of fibroglandular (i.e., dense)
tissue with respect to the total volume of the breast.
37

38. Quantitative Estimation of Volumetric Breast
Tomosynthesis Images (Pertuz et al, Radiology 2015)
38
Red: human annotator,yellow:computer algorithm

39. Let us dig into the theory a bit more…
• Unsupervised learning
– Finding clusters
– Dimensionality reduction (PCA, MDS,…)
– Building topographic maps
– Finding hidden causes or sources of the data
– Modeling the data density
39

40. Let us dig into the theory a bit more…
• Unsupervised learning
– Finding clusters
– Dimensionality reduction (PCA, MDS,…)
– Building topographic maps
– Finding hidden causes or sources of the data
– Modeling the data density
• Uses of Unsupervised learning
– Classification
– Data compression
– Make other learning tasks easier
– A theory of human learning and perception
– ….
40

41. Mixtures of Gaussians
41
x: input, y: output

42. Mixtures of Gaussians
42
You can download GMM train and test C/C++ code from my webpage:
http://www.cs.ucf.edu/~bagci/software.html

43. Constrained GMM for MR Brain Image
Segmentation: Greenspan et al. IEEE TMI 2006
• Each tissue is represented by large
number of Gaussians (to capture
complex tissue spatial layout)
43

44. Constrained GMM for MR Brain Image
Segmentation: Greenspan et al. IEEE TMI 2006
• Each tissue is represented by large
number of Gaussians (to capture
complex tissue spatial layout)
• Intensity is considered as a global
parameter
44

45. Constrained GMM for MR Brain Image
Segmentation: Greenspan et al. IEEE TMI 2006
• Each tissue is represented by large
number of Gaussians (to capture
complex tissue spatial layout)
• Intensity is considered as a global
parameter
• EM is utilized to learn parameter-tied
CGMM.
– Each Gaussian is linked to a single tissue
and all the Gaussians related to the same
tissue share the same intensity patterns.
45

46. Constrained GMM for MR Brain Image
Segmentation: Greenspan et al. IEEE TMI 2006
• Each tissue is represented by large
number of Gaussians (to capture
complex tissue spatial layout)
• Intensity is considered as a global
parameter
• EM is utilized to learn parameter-tied
CGMM.
– Each Gaussian is linked to a single tissue
and all the Gaussians related to the same
tissue share the same intensity patterns.
• Completely unsupervised, no
alignment/atlas registration required.
But note that general properties of T1
is used to include intensity info.
46

47. Constrained GMM for MR Brain Image
Segmentation: Greenspan et al. IEEE TMI 2006
(a) %9 noise, MRI from brain web, slice 95
(b) EM based algorithm
(c) CGMM algorithm results
47

48. Clustering methods for PET Images
• FCM (fuzzy c-means)
• FLAB (fuzzy locally adaptive Bayesian)
• K-means
• K-NN
• Spectral Clustering
• ….
48

49. Fuzzy LocallyAdaptive Bayesian (FLAB) Approach for
PET Segmentation (Hatt et al, TMI)
• Y: observed image (noisy)
• X: (binary) segmented object
• Bayesian setting: given the image Y, what is the segmentation
result X?
• P (Y|X) is the likelihood of the observation Y conditionally with
respect to the hidden ground-truth X
• P(X): prior
• P(X|Y): posterior distribution
49

50. Distribution of X (Prior)
• Two hard classes (Object + background) + finite number of
fuzzy levels.
• Consider Y0 and Y1 distributions, with mean and variations
• Then, mean and standard deviation of each fuzzy levels are
represented as
• Use EM algorithm to find unknowns.
50
Epsilon 1/3, 2/3 used.

51. Clustering methods for PET Images
• FCM (fuzzy c-means)
• FLAB (fuzzy locally adaptive Bayesian)
• K-means
• K-NN
• Spectral Clustering
• ….
• AP Clustering (affinity propagation)
– State of the art for multi-focal uptake segmentation
51
Foster, Bagci, et al., 2014

52. AP – By Frey and Dueck (more than 3.5K
citations, published in Science)
52
Example: The 15 images with highest squared error under either affinity propagation or k-centers
clustering are shown in the top row. The middle and bottom rows show the exemplars assigned by
the two methods,and the boxes show which of the two methods performed betterfor that image,
in terms of squared error. Affinity propagation found higher-quality exemplars.

53. Affinity Propagation (AP) Based Clustering
53
• Clusters data by locating
exemplars (the data point that
best describes a group of data)
based on the similarities
between pairs of points by
passing “messages” between
the data
– Responsibility r(i,k)
• How responsible is point k for
being i’s exemplar
– Availability a(i,k)
• How available is point k to be
point i’s exemplar
– Messages are iteratively sent
and compete until convergence
Responsibility—Message 1
k
i
Possible
Exemplar
Competing
Possible
Exemplar
r(i,k) a(i,k)
Availability—Message 2
k
i
Competing
Possible
Exemplar
r(i,k)
a(i,k)

54. Motivation for AP in PET Images (multi-focal updake)
54
GrosspathologyHisto-
pathology
5 weeks 15 weeks

55. Threshold Selection
55
Intensity
pdf
0 1
Objec
t1
Observed histogram of a PET
image is the summation of the
histogram of hidden objects
Objective for
segmentation: Locate
the optimal thresholding
value(s) for best
separation between these
objects
Objec
t2
Objec
t3

56. Similarity Function Definition
56
Intensity
0 1
pi
pj
ji
s(i,j) = -(|dij
x|n + |dij
G|m)
Probability difference - Distance includes
information of the peaks and valleys between
data points—needed for classification
dij
x
Weightparameters learned for
different image types and for small
animal/human images
Gradient
information
Probability information
dij
G

57. AP is an iterative process
57

58. Convergence of AP Segmentation
58

59. Comparison to Other Methods
59

60. Summary
• K-means
• FCM (fuzzy c-means)
• K-NN (k-nearest neighborhood)
• SMC (simple membership based clustering)
• AP (affinity propagation)
– Useful for PET image segmentation (multi-focal uptake)
60

61. Slide Credits and References
• Credits to: Jayaram K. Udupa of Univ. of Penn., MIPG
• Bagci’s CV Course 2015 Fall.
• K.D. Toennies, Guide to Medical Image Analysis,
• Handbook of Medical Imaging, Vol. 2. SPIE Press.
• Handbook of Biomedical Imaging, Paragios, Duncan, Ayache.
• Seutens,P., Medical Imaging, Cambridge Press.
• Z. Ghahramani, U. Cambridge, UK.
61