srrgf

1. Medical Image Analysis
Image Representation and Analysis
Figures come from the textbook: Medical Image Analysis, by Atam
P. Dhawan, IEEE Press, 2003.

2. Image Representation and
Analysis
Figures come from the textbook: Medical Image Analysis, by Atam
P. Dhawan, IEEE Press, 2003.
 A hierarchical framework of
processing steps representing the
image (data) and knowledge (model)
domains
 Scenes of specific objects
 Surface regions (S-regions)
 Region
 Contours and edges
 Pixels

3. Bottom-Up
Scenario
Scene-1 Scene-I
Object-1 Object-J
S-Region-1 S-Region-
K
Region-1 Region-L
Pixel (i,j)
Edge-M
Edge-1
Pixel (k,l)
Top-Down
Figure 8.1. A hierarchical representation of image features.

4. Image Reconstruction
Image
Segmentation
(Edge and Region)
Feature Extraction
and
Representation
Classification
and
Object Identification
Analysis
of Classified Objects
Multi-Modality/Multi-
Subject/Multi-Dimensional
Registration, Visualization and
Analysis
Raw Data from
Imaging System
Single Image
Understanding
Multi-Modality/ Multi-Subject/Multi-
Dimensional
Image Understanding
Scene
Representation
Models
Object
Representation
Models
Feature
Representation
Models
Edge/Region
Representation
Models
Physical Property/
Constraint
Models
Knowledge Domain
Data
Domain
Figure 8.2. A hierarchical structure of medical image analysis.

5. Feature Extraction and
Representation
Figures come from the textbook: Medical Image Analysis, by Atam
P. Dhawan, IEEE Press, 2003.
 Statistical pixel-level (SPL) features
◦ Mean, variance, histogram, area, contrast
of pixels within the region, edge gradient
of boundary pixels
 Shape feature
◦ Circularity, compactness, moments,
chain-codes and Hough transform,
morphological processing methods

6. Feature Extraction and
Representation
Figures come from the textbook: Medical Image Analysis, by Atam
P. Dhawan, IEEE Press, 2003.
 Texture features
◦ Second-order histogram statistics or co-
occurrence matrices, wavelet processing
methods for spatio-frequency analysis
 Relational features
◦ Relational and hierarchical structure of
the regions associated with a single or a
group of objects

7. Statistical Pixel-Level (SPL)
Features
 Histogram
 Mean
 Variance and central moments
n
r
n
r
p i
i
)
(
)
( 




1
0
)
(
1 L
i
i
i r
p
r
n
m





1
0
)
)(
(
L
i
n
i
i
n m
r
r
p


8. Statistical Pixel-Level (SPL)
Features
◦ The third central moment is a measure of
noncentrality
◦ The fourth central moment is a measure
of flatness of the histogram
 Energy




1
0
2
)]
(
[
L
i
i
r
p
E

9. Statistical Pixel-Level (SPL)
Features
 Entropy
◦ The entropy Ent is a measure of
information represented by the distribution
of gray-values in the region




1
0
2 )
(
log
)
(
L
i
i
i r
r
p
Ent

10. Statistical Pixel-Level (SPL)
Features
 Local contrast
 Maximum, minimum
 The mean, variance, energy and
entropy of contrast values
 Gradient information for the boundary
pixels
 
)
,
(
),
,
(
max
)
,
(
)
,
(
)
,
(
y
x
P
y
x
P
y
x
P
y
x
P
y
x
C
s
c
s
c 


11. Shape Features
 Longest axis GE
 Shortest axis HF
 Perimeter and area of the minimum
bounded rectangle ABCD
 Elongation ratio: GE/HF
 Perimeter and the area of the
segmented region
 Hough transform of the region using
the gradient information of the
boundary pixels of the region
p A

12. Shape Features
 Circularity ( = 1 for a circle) of the
region computed as
 Compactness of the region
computed as
C
2
4
p
A
C


p
C
A
p
Cp
2


13. Shape Features
 Chain code for boundary contour
◦ Obtained using a set of orientation
primitives on the boundary segments
derived from a piecewise linear
approximation
 Fourier descriptor of boundary
contours
◦ Obtained using the Fourier transform of
the sequence of boundary segments
derived from a piecewise linear
approximation

14. Shape Features
 Central moments based shape
features for the segmented region
 Morphological shape descriptors
◦ Obtained through the morphological
processing on the segmented region

15. Boundary Encoding: Chain
Code
 Orientation primitives
◦ 8-connected neighborhood
 Divide-and-conquer
◦ Curve approximation
 Maximum-deviation criterion
◦ Perpendicular distance between any point
on the original curve segment between
the selected vertices and the
corresponding approximated straight-line
segment

16. Figures come from the textbook: Medical Image Analysis, by Atam
P. Dhawan, IEEE Press, 2003.
0
4
2
3 1
5 6 7
xc 0
4
2
3 1
5 6 7
Figure 8.4. The 8-connected neighborhood codes (left) and the
orientation directions (right) with respect to the center pixel xc.

17. F
A D
C
E
B
A D
C
E
B
A D
C
E
B
A D
C
E
B
A
B C
D
Chain Code:
110000554455533
Figure 8.5. A schematic example of developing chain code for a region with
boundary contour ABCDE. From top left to bottom right: the original boundary
contour, two points A and C with maximum vertical distance parameter BF, two
segments AB and BC approximating the contour ABC, five segments approximating
the entire contour ABCDE, contour approximation represented in terms of
orientation primitives, and the respective chain code of the boundary contour.

18. Boundary Encoding: Fourier
Descriptor
 Closed boundary of a region
 Discrete Fourier transform (DFT) of
the sequence
 Rigid geometric transformation of a
boundary
◦ Translation, rotation, scaling
)
(
)
(
)
( n
iy
n
x
n
u 
 1
,...,
2
,
1
,
0 
 N
n





1
0
/
2
)
(
1
]
[
N
n
N
in
d e
n
u
N
n 
F

19. Moments for Shape
Description
 Central moments of a segmented
image
 Invariant moments
◦ Shape matching, pattern recognition

 



L
i
L
j
q
j
p
i
pq y
x
f
y
y
x
x
1 1
)
,
(
)
(
)
(


 

L
i
L
j
i
i
i y
x
f
x
x
1 1
)
,
(

 

L
i
L
j
i
i
i y
x
f
y
y
1 1
)
,
(

20. Figures come from the textbook: Medical Image Analysis, by Atam
P. Dhawan, IEEE Press, 2003.
Set A
Set B
Figure 8.6. A large region with square shape representing the set
A and a small region with rectangular shape representing the
structuring element set B.

21. : Dilation of A by
B
A B: Erosion of A
by B
( A B) B
A A
B
A
B
A A B
Figure 8.7: The dilation of set A by the structuring element set B (top
left), the erosion of set A by the structuring element set B (top right)
and the result of two successive erosions of set A by the structuring
element set B (bottom).

22. A
B
B
A
B
A
Figure 8.8. Dilation and erosion of an arbitrary shape region A
(top left) by a circular structuring element B (top right): dilation of
A by B (bottom left) and erosion of A by B (bottom right).

23. Figure comes from the Wikipedia, www.wikipedia.org.
Dilation

24. Figure comes from the Wikipedia, www.wikipedia.org.
Erosion

25. Morphological Processing for
Shape Description
 Opening
 Closing
B
B
A
B
A 

 )
(

B
B
A
B
A 


 )
(

26. A
B
B
A  B
A 
Figure 8.9. The morphological opening and closing of set A (top left)
by the structuring element set B (top right): opening of A by B (bottom
left) and closing of A by B (bottom right).

27. Figure comes from the Wikipedia, www.wikipedia.org.
Opening

28. Figure comes from the Wikipedia, www.wikipedia.org.
Closing

29. Morphological Processing for
Shape Description
 Skeleton
 Image processing
◦ Erosion can reduce the background noise
◦ Opening can remove the speckle noise
and provide smooth contours

N
n
n A
K
A
K
0
)
(
)
(


B
nB
A
nB
A
A
Kn 
)
(
)
(
)
( 




30. Morphological Processing for
Shape Description
 Image processing
◦ Closing preserves the peaks and reduces
the sharp variations in the signal such as
dark artifacts
◦ Opening followed by closing can reduce
the bright and dark artifacts and noise
◦ The morphological gradient image can be
obtained by subtracting the eroded image
from the dilated image
◦ Edges can also be detected by
subtracting the eroded image from the

31. Figure 8.10. Example of morphological operations on MR brain image using a
structuring element of
(a) the original MR brain image; (b) the thresholded MR brain image for
morphological operations; (c) dilation of the thesholded MR brain image; (d)
resultant image after 5 successive dilations of the thresholded brain image; (e)
erosion of the thresholded MR brain image; (f) closing of the thesholded MR brain
image; (g) opening of the thresholded MR brain image; and (h) morphological
boundary detection on the thresholded MR brain image.






1
0
0
1
(b)
(a)

32. (c) (d)
(f)
(e)

33. Figures come from the textbook: Medical Image Analysis, by Atam
P. Dhawan, IEEE Press, 2003.
(g) (h)

34. Texture Features
 Texture
◦ Statistical
◦ Structural
 A repetitive arrangement of square and
triangular shapes
◦ Spectral
 Fourier and wavelet transforms
 Gray-level co-occurrence matrix
(GLCM)
◦ is the distribution of the number of
occurrences of a pair of gray values
)
,
( j
i
p
i j
]
,
[ dy
dx

d

35. 2 2 2 0 1
0 2 2 1 1
0 1 1 2 0
1 2 2 0 1
2 1 0 1 1
0 3 1 0
2 1 0 1
1 4 3 2 i
0 1 2
j
(a)
(b)
Figure 8.11. (a) A matrix representation of a 5x5 pixel image
with three gray values; (b) the GLCM P(i,j) for d=[1,1].

36. Texture Features

◦ The probability of occurrence of a pair of
gray values and separated by a
distance vector
 ,
◦ The probability that a difference in gray-
levels exists between two distinct pixels
)
,
,
( d
r
q y
y
H
q
y r
y
d
)
,
( d
s
d y
H r
q
s y
y
y 


37. Second-Order Histogram
Statistics
 Entropy of
 Angular second moment of
)
,
,
( d
r
q y
y
H
 
 


t
q
t
r
y
y
y
y
y
y
r
q
r
q
H y
y
H
y
y
H
S
1 1
)]
,
,
(
[
log
)
,
,
( 10 d
d
)
,
,
( d
r
q y
y
H
 
 

t
q
t
r
y
y
y
y
y
y
r
q
H y
y
H
ASM
1 1
2
)]
,
,
(
[ d

38. Second-Order Histogram
Statistics
 Contrast of
 Inverse difference moment of
)
,
,
( d
r
q y
y
H
 
   
 



t
q
t
r
t
q
t
r
y
y
y
y
y
y
r
q
r
q
y
y
y
y
y
y
r
q
r
q y
y
H
y
y
y
y
H
y
y
1 1
1 1
)
,
,
(
)
(
)
,
,
(
)
,
( 2
d
d
)
,
,
( d
r
q y
y
H
 
  


t
q
t
r
y
y
y
y
y
y r
q
r
q
H
y
y
y
y
H
IDM
1 1
)
,
(
1
)
,
,
( d

39. Second-Order Histogram
Statistics
 Correlation of )
,
,
( d
r
q y
y
H
 
 



t
q
t
r
r
q
r
q
y
y
y
y
y
y
r
q
y
r
y
q
y
y
H y
y
H
y
y
Cor
1 1
)
,
,
(
)
)(
(
1
d







t
r
y
y
y
r
q
q
m y
y
H
y
H
1
)
,
,
(
)
,
( d
d



t
q
y
y
y
r
q
r
m y
y
H
y
H
1
)
,
,
(
)
,
( d
d

40. Second-Order Histogram
Statistics
 Mean of
 Deviation of
)
,
,
( d
r
q y
y
H
)
,
( d
q
m y
H



t
r
m
y
y
y
q
m
q
H y
H
y
1
)
,
( d

 
 
 








t
q
t
r
m
y
y
y
q
m
y
y
y
r
m
r
q
H y
H
y
H
y
y
1 1
)
,
(
)
,
(
2
d
d


41. Second-Order Histogram
Statistics
 Entropy of
 Angular second moment of
)
,
( d
s
d y
H




t
s
s
d
y
y
y
s
d
s
d
y
H y
H
y
H
S
1
)]
,
(
[
log
)
,
( 10
)
,
( d
d
d
)
,
( d
s
d y
H



t
s
s
d
y
y
y
s
d
y
H y
H
ASM
1
2
)
,
( )]
,
(
[ d
d

42. Second-Order Histogram
Statistics
 Mean of )
,
( d
s
d y
H



t
s
s
d
y
y
y
s
d
s
y
H y
H
y
1
)
,
(
)
,
( d
d


43. Figures 8.12 (a) A part of a digitized X-ray mammogram showing a region of
benign lesion (b) a part of a digitized X-ray mammogram showing a region of
malignant cancer of the breast (c). A second-order histograms of (a) computed
from the gray-level co-occurrence matrices with a distance vector of [1,1] and (d)
A second-order histogram of (b) computed from the gray-level co-occurrence
matrices with a distance vector of [1,1] .
(a) (b)

44. (c)

45. (d)

46. Relational Features
 Relational features
◦ Information about adjacencies, repetitive
patterns and geometrical relationships
among regions of an object
 Quad-tree representation
 Tree and graph structures

47. Figures come from the textbook: Medical Image Analysis, by Atam
P. Dhawan, IEEE Press, 2003.
R1
R21 R22
R23
R41
R43
R24
R42
R44
R3
Figure 8.13: A block representation of an image with major quad
partitions (top) and its quad-tree representation.

48. Figures come from the textbook: Medical Image Analysis, by Atam
P. Dhawan, IEEE Press, 2003.
R
R
4
R
3
R
2
R
1
R
24
R
23
R
22
R
21
R
44
R
43
R
42
R
41
R
14
R
13
R
12
R
11
R
34
R
33
R
32
R
31

49. A
C
B
D
F
I
E
B
C
A
I
E
D
F
Figure 8.14. A 2-D brain ventricles and skull model (top) and region-
based tree representation.

50. Feature and Image
Classification
 Statistical classification methods
◦ Unsupervised: k-means, fuzzy clustering
◦ Supervised
 Nearest neighbor classifier
◦ Assigned to the class if
j
j
D u
f
f 

)
( 


j
c
f
j
j
j
N
f
u
1
C
j ,...,
2
,
1

i
c
)
(
min
)
( 1 f
f j
C
j
i D
D 


51. Feature and Image
Classification
 Bayes classifier
◦ Risk of wrong classification for assigning
the feature vector to the class
◦ Assigned to the class if



C
k
k
kj
j c
p
Z
r
1
)
|
(
)
( f
f
C
j ,...,
2
,
1

j
c
i
c

 


C
k
k
kj
C
k
k
ki c
p
Z
c
p
Z
1
1
)
|
(
)
|
( f
f

52. Feature and Image
Classification
 Rule-based systems
◦ Analyze the feature vector using multiple
sets of rules that are designed to check
specific conditions in the database of
feature vectors to initiate an action

53. Strategy Rules
A priori
knowledge
or models
Focus of
Attention Rules
Knowledge
Rules
Activity
Center
Input
Database
Output
Database
Figure 8.15. A schematic diagram of a rule-based system for image analysis.

54. Feature and Image
Classification
 Image and feature classification:
neural networks
◦ Backpropagation
◦ Radial basis function
◦ Associative memories
◦ Self-organizing
 Neuro-fuzzy pattern classification

55. Figures come from the textbook: Medical Image Analysis, by Atam
P. Dhawan, IEEE Press, 2003.
X f()

1
w2

w0
w1
wd
f():
Y
Figure 8.16. A computational neuron model with linear synapses.

56. Figures come from the textbook: Medical Image Analysis, by Atam
P. Dhawan, IEEE Press, 2003.
M1
winner-take-all
output layer
L
1
fuzzy membership
function layer
x1
xi
xd
hyperplane
layer
input
layer
max
M2
MK
C
Figure 8.17. The architecture of the Neuro-Fuzzy Pattern Classifier.

57. Figures come from the textbook: Medical Image Analysis, by Atam
P. Dhawan, IEEE Press, 2003.
L
input from
hyperplane
layer
2
1
scaling
1f
2f
Lf
f
multiplication
Mf
output
fuzzy
function
Figure 8.18. The structure of the fuzzy membership function.

58. Figures come from the textbook: Medical Image Analysis, by Atam
P. Dhawan, IEEE Press, 2003.
1 2 3 4 5 6 7 8
1
2
3
4
5
6
7
8
Figure 8.19. Convex set-based separation of two categories.

59. Figure 8.20. (a). Fuzzy membership function M1(x) for the subset #1 of the
black category.
(b). Fuzzy membership function M2(x) for the subset #2 of the black category.
(a)

60. Figures come from the textbook: Medical Image Analysis, by Atam
P. Dhawan, IEEE Press, 2003.
(b)

61. Figure 8.21. Fuzzy membership function M3(x) (decision surface)
for the white category membership.

62. Figure 8.22. Resulting decision surface Mblack(x) for the black
category membership function.

63. Image Analysis Example: Analysis
of Difficult-to-Diagnose
Mammographic Microcalcification
 Features
◦ Number of microcalcification
◦ Average number of pixels per
microcalcification
◦ …
◦ Entropy of
◦ …
◦ Energy fro the wavelet packet at Level
0
◦ …
)
,
,
( d
r
q y
y
H
6
D