Tissue Segmentation Methods Using 2D Histogram Matching in a Sequence of MR Brain Images_part1

Tissue Segmentation Methods using
2D Histogram Matching in a Sequence
of MR Brain Images (Part 1)
Vladimir Kanchev, PhD
Radiocommunications and
Videotechnologies Department
TU Sofia, Sofia, Bulgaria
July 2017

Page  2
This Research is Reported in:
Kanchev, Vladimir and
Roumen Kountchev.
"Tissue Segmentation Methods Using
2D Histogram Matching in a Sequence
of MR Brain Images."
New Approaches in Intelligent Image
Analysis. Springer International
Publishing, 2016. 183-222.
(Chapter 6)

Page  3
Contents
1. Main idea and contributions
2. Introduction
3. Method description
4. Experimental results
5. Conclusions and future work

Page  4
Main Objective
The volume of brain tissues depends on the age,
health condition and sex of the patient. Additionally,
some diseases and health conditions lead to a
change and a decrease in the tissue volumes.
Therefore, their precise measurement is important
for the patient diagnosis.

Page  5
MR Image Segmentation
Def: Image segmentation aims to divide an
image into non-overlapping, constituent
regions, homogeneous to some feature as
intensity and texture*. Classification aims
to assign to each image pixel a tissue class
or air. Both operations are interconnected.
Purpose: Assist radiologists in the detection of brain
diseases based on MR images – provide “a
second opinion”.
* Pham et al. "Current methods in medical image segmentation”. Annual
review of biomedical engineering 2.1 (2000): 315-337.

Page  6
MR Brain Image Segmentation
* Goossens, B., & Philips, W. (2015). MRI segmentation of the human
brain: challenges, methods, and applications. Computational and
mathematical methods in medicine, 2015.
*

Page  7
Main Idea
In the chapter we suggest an MR image
segmentation method using a 2D histogram
matching.
We apply the 2D histogram matching to
(1) calculate train matrices for each tissue of the
test MR image. Then we (2) classify a test
2D histogram of each test MR image using them.

Page  8
Other Challenges
 Tissue intensity distributions vary depending on
the patient, scanner and scanning conditions
 Tissue properties (area, texture, intensity
distribution, etc.) change along an MR image
sequence
 Artefacts modify the intensity distributions of
brain tissues upon few neighboring MR images

Page  9
Main Contributions
Our method includes algorithms for:
 MR image sequence division using a normalized
wave hedges distance
 2D histogram matching using a vector
specification
 MR image segmentation using a back projection
of a classified 2D histogram

Page  10
Contents
2. Introduction

Page  11
Introduction
2.1 MRI data properties
2.2 Transductive learning
2.3 Histogram matching/specification
2.4 Our 2D histogram
2.5 2D histogram matching
2.6 Literature review

Page  12
MRI Data Properties
*
* http://images.wisegeek.com/mri-machine-with-screens.jpg

Page  13
MRI Data Properties
 MR imaging is a leading medical technology for
visualization of internal human organs and
tissues
 The intensity levels of tissues in MR images
depend on the quantity of hydrogen atoms
 Presence of artefacts – bias field, chemical shift,
partial volume, ringing, etc.
 Ever increasing quantity of available MRI data –
introduction of data-driven methods

Page  14
Different tissues in MR brain images
Main tissues:
 CSF – CerebroSpinal Fluid
 GM – Gray Matter
 WM – White Matter
Other (redundant) tissues:
sculp, skull, dura, muscle, etc.

Page  15
Different tissues in MR brain images
*
*Ramon, Ceon et al. "Influence of head models on EEG simulations and inverse
source localizations." BioMedical Engineering Online 5.1 (2006): 10.

Page  16
MRI Data Properties
 MR imaging (T1 - weighted) provides very good
resolution on soft brain tissues (CSF - the lowest
intensity values, GM - middle, WM - the highest)
 Neighboring pixel pairs within tissues have equal
or close intensity values (piecewise smooth or
constant, slowly varying values)
 Neighboring pixel pairs on the border between
tissues have far different intensity values
(piecewise discontinuous)

Page  17
MRI Data Properties
 Variability of intensity values of separate tissues
within a patient/MRI scanning (intra-scan) or
between patients/MRI scanning (inter-scan)
 Intra-scan variability necessitates intensity
standardization of MRI data
 MR image noises and artefacts require filtering and
correction

Page  18
MR Image Sequence
*
* Anum Masood ‘Detection and Classification of Tumor in Magnetic
Resonance (MRI) Brain Images’ [Retrieved 13.07.2017]

Page  19
Bias Field
Bias field or Intensity inhomogeneity (INH):
 represents a low-frequency and a very smooth
signal which distorts the MR image (in our case)
 alters tissue intensity distribution in one or more
neighbouring MR images
 blurs the MR image and reduces high frequency
content of the MR image as edges and contours

Page  20
Bias Field
*
* Vovk, Uros et al. “A Review of Methods for Correction of Intensity
Inhomogeneity in MRI.” IEEE transactions on medical imaging 26.3
(2007): 405-21.

Page  21
Partial Volume Effect
 Makes border pixels/voxels with intensity
values of two neighboring tissues, which
depend on the proportions of each tissue
 Occurs due to the limited resolution of an MR
image and image sampling
 Causes blurring of the image and spill-out effect

Page  22
Partial Volume Effect
*
* Tohka Jussi: “Partial volume effect modeling for segmentation and tissue
classification of brain magnetic resonance images: A review”. World Journal of
Radiology 6.11 (2014):855

Page  23
MRI Artefacts
 Bias field leads to:
 a change in the intensity distribution of a single tissue
along the whole MR image
 a change in the intensity range of the MR image
 introduction of low-frequency noise in the MR image
 Partial volume effect leads to:
 blurring of neighboring tissue edges
 the introduction of new intensity values and a change in
high frequencies
 the introduction of additional transitive tissue classes
CSF-GM, CSF-WM, GM-WM into segmentation methods

Page  24
Properties
Presence of both artefacts:
 breaks the assumption of piecewise intra-tissue
smoothness and inter-tissue discontinuity of
MRI data
 leads to worse results of the segmentation
algorithms
 is not easily recognizable from humans

Page  25
Introduction

Page  26
Transductive Learning
“When solving a problem of interest, do not solve a
more general problem as an intermediate step. Try
to get the answer that you really need but not a
more general one.”
Vladimir Vapnik (author of the idea for Transductive learning, SVM
classifier, Vapnik-Chervonenkis theory, etc.)

Page  27
Depending on whether we build a general model
for the classification of a test set of instances, we use:
 inductive learning (build a model)
 transductive learning (do not build a model)

Page  28
We have a*:
 Training set of labeled examples
where is an input vector and –
a corresponding output label
 Test set of unlabeled examples
We look for:
 Expected labels for all instances in
),x( i i
yU
ix i
y
V
)x( '
i
'
i
y
V
* https://codesachin.wordpress.com/2016/07/03/a-small-and-easy-
introduction-to-transductive-learning/

Page  29
We build labels only for instances from :
Input for supervised learning (and , if we
go for semi-supervised learning).
Output The set of expected labels for all
instances in .
Approach We leverage the information contained
in the instances of (with respect to
those in ) to make better prediction
for specifically.
U
V
V
V
U
V
V

Page  30
Inductive Learning
We build a general model to classify unseen
instances in the future besides those in :
Input Train set for supervised learning (and
if we go for semi-supervised learning).
Output The set of expected labels for all
instances in test set .
Approach Compute a function , using information
in such that is as close to as
possible over all instances in . Using
the function, compute each as much
as .
U
V
V
f
U
)x( if i
y
'
i
y
)x( '
i
'
fyi

V
U

Page  31
Properties
Transductive learning:
 is slower but more precise method than the
inductive learning method
 cannot label new instances except those in
 requires integrity of test set instances – no
additional noise
V

Page  32
How it relates to our segmentation method:
 we build an ad-hoc model (model 2D histograms)
for each MR image subsequence
 we adapt the model for each test 2D histogram,
using matching, thus we find out the train matrices
of each test 2D histogram (partial classification)

Page  33
Introduction

Page  34
Histogram Matching
Histogram matching or specification *:
Def: A method used to generate a processed
image that has a specified histogram. We
use a reference (target) histogram to
provide a form for the specified histogram.
Histogram equalization is a special case of
histogram specification with a uniform reference
histogram.
* Gonzalez & Woods: Digital image processing. 3rd edn. (2007)

Page  35
Image Histogram
Def: Intensity histogram shows how many times a
particular intensity level appears in the image.
Mathematical description:
1. Non-normalized image histogram
2. Normalized image histogram – int. range levels
– row dimension
– column dimension
– the -th intensity value of the image
– number of pixel in the image with intensity
kk nrh )(
  MNrrp kk 
kr k
kn
kr
1,...,1,0  Lk
M
N
L

Page  36
Operations (Gonzalez & Woods, 2007):
1. Compute PDF (or a histogram) of an input image and
find the histogram equalization transformation. Round the
resulting values to the integer range .
Histogram Matching Algorithm
)( jr rp
k
s  1,0 L


k
0j
)()1()( jrkk rpLrTs

Page  37
2. Compute all values of the transformation function for
, where are the values of the
reference (target) histogram. Round the values of to
integers in the range . Store the values of in a
table.
G
1,...,2,1,0  Lq )( iz
zp
 1,0 L


q
i
izq zpLzG
0
)()1()(
G
G

Page  38
3. For each value of , use the stored values
of from step 2 above to find the corresponding values of
so that is closest to and store this mapping
from to :
,
.
k
s 1...,,1,0  Lk
G
q
z )( qzG k
s
s z
kq
szG )(
)(1
kq
sGz 


Page  39
4. Form the histogram-specified image in the following
manner:
4.1 First form histogram-equalized image of the input
image.
4.2 Then map every equalized pixel value of this image
to the corresponding value in the histogram-
specified image using the mapping from step 3 above.
Equalization step 4.1 above might be skipped by
combining two transformation functions and :
. .
q
z
k
s
T 1
G
)())(( 11
kkq sGrTGz 


Page  40 Gonzalez & Woods: Digital image processing.
3rd edn. (2007).* Gonzalez & Woods: Digital image processing. 3rd edn. (2007)
*

Page  41
Image before Histogram Matching
*
Input image of the Mars moon and its non-
normalized histogram

Page  42
Image after Histogram Matching
b *

Page  43
Properties
Properties of the histogram matching:
 a point operation
 a “trial-and-error” process
 considers the shape of the input and target
histograms
 makes the actual histogram of the transformed
image to approximate the target histogram

Page  44
Properties
Properties of both histogram equalization and
matching:
 point operations – no spatial information included
 non-linear operations
 the input, target and matched histograms have
equal intensity ranges (or length)
 do not introduce new intensity values

Page  45
Histogram Matching
How it relates to our segmentation method:
 a well-known algorithm for image enhancement
 works well with images of slightly different intensity
ranges
 adapts well to images with different types of
histograms

Page  46
Introduction

Page  47
Gray Level Co-occurrence Matrices
*
Gray Level Co-occurrence matrix (GLCM) of
horizontal neighboring pixel pairs in a matrix by
with an intensity scale .
* https://www.mathworks.com/help/images/ref/referenceetoh38.gif
4
 8,1
5

Page  48
Gray Level Co-occurrence Matrix
GLCM is used in computer vision for:
 texture analysis
(image classification)
 feature extraction
 image retrieval systems
 object recognition
GLCM provides a useful estimation of similarity
between neighboring elements in a matrix/image.

Page  49
Our 2D Histogram
We use a 2D histogram as a sum of eight GLCMs,
corresponding to eight directions in a by window.
We apply it into two ways:
 non-preprocessed, normalized (MR image
sequence division)
 preprocessed, non-normalized (2D histogram
matching)
3 3

Page  50
Benchmark 2D Histogram
Another popular benchmark 2D histogram (Zhang
& Hu, 2008) has a bin with the following coordinates
and :
 the first coordinate is an intensity value of a
central pixel, and
 the second coordinate is an averaged intensity
value of eight pixels around the central one in the
local window by
i
j
i
j
3 3

Page  51
2D Histograms
our 2D histogram (preprocessed,
non-normalized) (set to zero the
first row and column bins)
“benchmark”
2D histogram
(Zhang & Hu, 2008)

Page  52
Properties
Our 2D histogram has:
 comparatively good separability between the bins
(pixel pairs) of separate tissues and edges
 good compactness and reflectional symmetry
along the diagonal
 the bins of the tissue-background edges stay on
the first row and column in the 2D histogram
 the pixel pairs of the background stay on bin
predominantly
)1,1(

Page  53
Properties
The second benchmark 2D histogram gained
popularity in the past because of:
 its compactness and fewer outliers
 the good discrimination of objects in the test
images from the s – few low-resolution images
 its intuitive and fast computation
 its less memory consumption
1980

Page  54
Properties
How it facilitates the matching and image
segmentation algorithms in our method:
 compact distribution (no gaps in the areas of tissue
pixel pairs)
 good discrimination between the pixel pairs of
tissues, edges and background
 correspondence between the image pixel pairs and
the bins of the 2D histogram

Page  55
Introduction

Page  56
2D Histogram Matching
The position of the algorithm for 2D histogram
matching in our method:
 compute a train matrix for each tissue of the test
MR image (partial classification) (using the 2D
histogram matching!)
 kNN classify the test 2D histogram using the train
matrices (full classification)
 segment the corresponding test MR image with the
classified test 2D histogram

Page  57
2D Histogram Matching
Important points of our implementation:
 conversion of 2D histograms into vectors using a
zig-zag ordering
 close but not equal length of the corresponding
test and model (target) vectors/histograms
 truncation of the cumulative vectors within a
percentile interval
 then truncation of the corresponding elements from
the normalized vectors

Page  58
2D Histogram Matching Algorithm
Operations of our implementation:
1. Compute test and model 2D histograms as a sum
of eight GLCMs within a 3x3 local window (eight
directions).
2. Convert 2D histograms into vectors using zig-zag
ordering.
3. Truncate the cumulative vectors within a percentile
interval. Truncate the corresponding elements
from the normalized vectors before matching.

Page  59
2D Histogram Matching Algorithm
4. Perform vector specification:
4.1 Calculate LUT between two cumul. vectors.
4.2 Calculate new values of the resulting vector
using the LUT between two normalized vectors.
5. Calculate the difference vector between the
resulting and initial normalized vectors.
6. Perform k-means clustering on the difference
vector, keep elements of the greatest cluster and
set others to zero.
7. Go back to the 2D histogram space and thus we
have a train matrix.

Page  60
Properties
Our 2D histogram matching algorithm:
 uses position of pixel pairs distribution of different
tissues in the 2D histograms
 considers correlation between the neighboring bins
in the 2D histogram by zig-zag ordering
 localizes segments of bins of different tissues in
the 2D histogram as a final result

Page  61
Introduction

Page  62
Literature Review
https://i.ytimg.com/vi/FFcrQweRAp0/hqdefault.jpg
* https://i.ytimg.com/vi/FFcrQweRAp0/hqdefault.jpg
*

Page  63
Literature Review
Segmentation approaches of interest based on:
 histograms, 2D histograms and related thresholds
 models, built with different amount of information
 different image spatial information

Page  64
Literature Review
Type of histograms used for image segmentation
 1D histogram and related algorithms:
 thresholding (Wu et al., 2007), find peaks (Morin et al., 2012)
and relative positions (Solanas et al.,2000), fit probability
distributions to а histogram form (Lee et al., 2009)
 tissue intensity distributions as blind source separation
problem (BSS) (Zagorodnov & Ciptadi , 2011)
 extraction of a feature vector from the histogram and
training a SOM classifier (Ortiz et al., 2013)

Page  65
Literature Review
 2D histogram:
 types of 2D histograms - 2D histogram (Kirby &
Rosenfeld, 1979), 2D-D histogram (Yimit et al., 2013), GLSC
histogram (Xiao et al., 2008), GLGM histogram (Xiao et al.,
2014)
 kinds of thresholds – Otsu threshold (Zhang & Hu, 2008),
curvilinear threshold (Zhang & Hu,2009), entropic threshold
(Xiao, Cao, & Yuan, 2014), 2-phase threshold (Chen et al,
2010)

Page  66
Literature Review
Advantages of the suggested 2D histograms:
 distinctive bins of tissues and edges
 more information on edges in recent types of
2D histograms
 lower memory consumption and higher speed of
computation
 more accurate thresholds, easier and faster to
calculate

Page  67
Literature Review
Methods with different amount of training data:
 unsupervised methods:
 parametric models - GMM (Dong & Peng, 2014), RMM
(Roy et al., 2012)
 non-parametric models - mean-shift (Mahmood, 2012),
kNN classification (Vrooman et al., 2013)
 fuzzy models – FLGMM (Ji et al., 2012), FCM algorithm
(Ji et al., 2014)

Page  68
Literature Review
 semi-supervised methods
 ensemble framework (Azmi et al., 2013), Bayesian
transductive learning (Lee et al., 2013), spectral clustering
(Zhang et al.,2010)
 supervised methods
 atlas-based (Cabezas et al., 2011)

Page  69
Literature Review
Discussion:
 the supervised methods are more accurate but
require more data that is labeled – a hurdle in the
current BIG Data era
 the unsupervised methods are less accurate but
do not require labeled data – a great advantage
 the semi-supervised methods are in-between …
–

Page  70
Literature Review
Methods which include image spatial information:
 Markov random fields (MRF)
 HMRF (Zhang et al., 2001), MWMAP (Monaco & Madabhushi,
2012), HMRF-MCMC ( Zhang et al., 2014)
 variational methods
 shape prior constraints (Liu et al., 2011), intensity
distributions (Chen & Radke, 2009)
 level set theory methods
 deformable models (Lee et al., 2012), INH artifact (Liu et al.,
2011)

Page  71
Literature Review
The methods use the following image properties:
 piecewise smoothness between neighboring
pixels within tissues
 piecewise discontinuity of neighboring pixels
between tissues
 presence of noise and artefacts, which should be
included in the segmentation model

Page  72
Literature Review
How the above approaches are related to our
method:
 Image spatial information, as it is included in the
test and model 2D histograms
 Amount of preliminary information, as it selects
some MR images to build model 2D histograms
 Transductive learning, as it performs 2D
histogram matching to find out the train matrices
 Histogram thresholding, as the 2D histogram is
classified using the train matrices

Page  73
Summary
Points to remember:
 MRI data – what are their characteristics,
artefacts, etc.
 Transductive learning framework – how we
compute and apply our segmentation model
 2D histogram – how we construct it
 2D histogram matching – how we perform it

Page  74
Next - Part 2
2. Introduction

Page  75
Next - Part 2
3.1 Preprocess an MR image sequence
3.2 Divide into MR image subsequences
3.3 Compute test and model 2D histograms
3.4 Match a 2D histogram
3.5 Classify a 2D histogram
3.6 Segment using a back projection

Page  76
Next - Part 2
How will we merge 2D histograms and a 2D
histogram matching algorithm in a transductive
learning framework for a MR image segmentation?

Tissue Segmentation Methods Using 2D Histogram Matching in a Sequence of MR Brain Images_part1

Recommended

Recommended

More Related Content

Similar to Tissue Segmentation Methods Using 2D Histogram Matching in a Sequence of MR Brain Images_part1

Similar to Tissue Segmentation Methods Using 2D Histogram Matching in a Sequence of MR Brain Images_part1 (20)

Recently uploaded

Recently uploaded (20)

Tissue Segmentation Methods Using 2D Histogram Matching in a Sequence of MR Brain Images_part1