Tissue segmentation methods using 2D histogram matching in a sequence of mr brain images part 3

Tissue Segmentation Methods using
2D Histogram Matching in a Sequence
of MR Brain Images (Part 3)
Vladimir Kanchev, PhD
Radiocommunications and
Videotechnologies Department
TU Sofia, Sofia, Bulgaria
July 2017

Page  2
This Research is Reported in:
Kanchev, Vladimir and
Roumen Kountchev.
"Tissue Segmentation Methods Using
2D Histogram Matching in a Sequence
of MR Brain Images."
New Approaches in Intelligent Image
Analysis. Springer International
Publishing, 2016. 183-222.
(Chapter 6)

Page  3
Contents – Part 2
3.1 Preprocess an MR image sequence
3.2 Divide into MR image subsequences
3.3 Compute test and model 2D histograms
3.4 Match a 2D histogram
3.5 Classify a 2D histogram
3.6 Segment using back projection

Page  4
Summary – Part 2
Points to remember:
 what is new – 2D histogram, 2D histogram
matching, back projection algorithms
 separate algorithms – their sequence, separate
parameters values, input and output data, etc.
 motivation for each algorithm – problem,
challenge and solution
 analysis of each algorithm – properties,
advantages and disadvantages

Page  5
Contents
1. Main idea and contributions
2. Introduction
3. Method description
4. Experimental results
5. Conclusions and future work

Page  6
Experimental Results
We will show here test results of the suggested
segmentation method on public test data sets and
we will discuss them afterwards.

Page  7
4.1 Benchmark algorithms and metrics
4.2 Parameters of the segmentation method
4.3 Test MRI data
4.4 Segmentation results. Analysis
4.5 PC hardware, software environment and
execution time

Page  8
Benchmark Algorithms and Metrics
Where do we stand compared with the results
of other recent segmentation (benchmark)
algorithms and methods published in the scientific
literature?

Page  9
Benchmark Algorithms
We compare our results with benchmark segmen-
tation methods and algorithms results* (from the last
5 years, 2015):
 IBSR metrics (IBSR site) – benchmark basic
algorithms as adaptive MAP (amap), biased MAP
(bmap), MAP, fuzzy c-means (fuzzy), tree-structu-
re k-means (tskmeans), maximum likelihood (MLC)
 software (segmentation) packages – SPM8,
FSL, Brainsuite
* The results were taken from publications since no code was
publicly available

Page  10
 mixture models – GMM-CSA (Zhang et al., 2014),
FLGMM ( Ji et al., 2012), Rice (Roy et al., 2012)
 fuzzy models – FLGMM (Ji et al., 2012)
 hidden Markov models – MLHMM (Foruzan et
al.,2013)
 Markov random fields – APRS (Lin et al., 2011)

Page  11
Characteristics of the benchmark algorithms and
methods:
 benchmark basic algorithms rely mostly on basic
statistic measures and ML algorithms
 software packages use improved basic
segmentation algorithms
 advanced methods compute complex features (of
spatial and intensity MR image information) and
use elaborated classifiers

Page  12
 advanced methods adapt their segmentation
model to artefacts and image noises
 advanced methods use selected MR images and
brain atlases for training
 all methods are evaluated on publicly available
test datasets of MR images
 deep learning methods are around the corner in
the field (2015-17)

Page  13
Benchmark Metrics
Benchmark metrics to evaluate segmentation
results*:
 Jaccard similarity coefficient (JSC)
 Dice similarity coefficient (DSC)
, – segmented from our method and ground truth
segmented region in a test MR image
– the number of pixels in a segmented region
* As metrics to benchmark data given in publications
%100.
21
21
SS
SS
JSC



%100.
2
21
21
SS
SS
DSC



1S 2S
.

Page  14
Characteristics of the evaluation metrics:
 widely used metrics in the literature
 dependence on tissue size is avoided
 JSC is a little bit more conservative than DSC
 JSC and DSC can be derived from one another

Page  15
Benchmark results were taken directly from the
published papers from the test data sets we also use.
But ….
 no public code is available
 so direct comparison in terms of execution time is
not possible

Page  16
4.3 Test MRI data
4.4 Segmentation results. Analysis.
execution time

Page  17
Parameters of the Method
The segmentation method comprises six
algorithmic steps and each of them is determined
by a few parameters. This is an important aspect
that must not be ignored!

Page  18
1. Preprocess an MR image sequence
 extraction of CSF, GM and WM tissues (Brainweb) (with
labeled masks)
 conversion to the coronal plane (IBSR18 ( )
and Brainweb ( ) )
 no gamma correction ( =1)
2. Divide into MR image subsequences
 interval of similarity values (for IBSR20)
 max length of MR image subsequence – 10 MR images
(mostly for IBSR18 and Brainweb)
 a normalized and non-preprocessed 2D histogram of
each MR image

128256256 
256256181 
]1.1,9.0[

Page  19
3. Compute model and test 2D histograms
 min 20 entries for a 2D histogram model
 min 50 pixels for a model MR image
 sum of eight co-occurrence matrices (K=8), of adjacent
pixel pairs; non-normalized and preprocessed
 three tissue classes (R=3)
 256 intensity levels (B=256) – intensity range
 finally, model 2D histograms of IBSR20 dataset are
thresholded with 2 (all bins with values, smaller or equal
to 2, are set to 0)

Page  20
4. Match a 2D histogram
 direction of (JPEG) zig-zag ordering – regardless of the
direction – it should be consistent
 percentile interval (IBSR18 and 20),
(Brainweb)
 k-means vector clustering – 2 clusters (CSF and WM)
and 3 (2 - Brainweb) clusters (GM); min 20 elements in
a cluster
5. Classify a 2D histogram
 k-Nearest Neighbor – 1 neighbor, l2 distance metric
 LMNN distance metric learning (default parameter
values)
 95,5  5.97,5.2

Page  21
6. Segment using back projection
 window, eight pairs of adjacent pixels (K=8)
 priority of CSF, WM, GM and Bckgr in case of equality
during majority vote
 morphological opening (for smaller noisy elements) with
an area of few pixels (8) (IBSR20)
33

Page  22
Some comments on the parameters:
 their values slightly differ for each test data set
 their values should be applied strictly to achieve
a stable classification of the test 2D histogram
 the method is quite sensitive to parameter values,
related to 2D histogram computation and matching

Page  23
4.3 Test MRI data
execution time

Page  24
Test MRI Data
We need to test our developed algorithms on
public MRI data sets of different properties to make
an extensive analysis.

Page  25
Test MRI Data
We experiment with three test sets of MRI data –
Brainweb, IBSR18, and IBSR20. They are all:
 publicly available on the Internet
 of diverse properties
 widely used in scientific literature – important for
benchmarking our results
 provided with ground-truth labeled masks –
important for the construction of model 2D
histograms and for the evaluation of the results

Page  26
Parameters of the Test MRI Data

Page  27
Test MRI Data
A different type of MRI data leads to a different
distribution and shape of their 2D histograms:
 artificially generated MRI data (Brainweb) – very
compact distribution with very high peaks along the
diagonal, no outliers
 filtered real MRI data (IBSR 18) – good
concentration of the distribution along the diagonal,
very few outliers
 unfiltered real MRI data (IBSR 20) – quite wide
distribution and presence of many outliers

Page  28
Types of Test MRI Data
Brainweb IBSR18
IBSR20

Page  29
Brainweb IBSR18
IBSR20

Page  30
We compute 2D histograms of:
 shorter elongated distribution and sharp peaks
along the diagonal (Brainweb)
 narrower distribution and distinctive line (IBSR18)
along the diagonal
 extended distribution and lower, rounded line
(IBSR 20) along the diagonal

Page  31
4.3 Test MRI data
execution time

Page  32
Segmentation Results
Different types of MRI datasets lead to different
types of 2D histograms.
Which MRI dataset will give the best results?
Which properties of MR images and 2D histograms
influence the final results the most?

Page  33
Let’s start with the test dataset Brainweb of
artificially generated MRI data.

Page  34
Segmentation Results – Brainweb
input 2D histogram classified 2D histogram

Page  35
(6.12 g) – input MRI data slice
(6.12 h) – ground truth segmented MRI data slice – CSF, GM and
WM tissues
(6.12 i) – segmented MRI data slice (our algorithm) – CSF, GM
and WM tissues

Page  36

Page  37
Let’s go on with the test dataset IBSR 18 of
filtered real MR images.

Page  38

Page  39
Segmentation Results – IBSR18
(6.12 d) – input MR image
(6.12 e) – ground truth segmented MR image – CSF, GM and WM
tissues
(6.12 f) – segmented MR image (our algorithm) – CSF, GM and
WM tissues

Page  40

Page  41
Let’s finish with the test dataset IBSR20 of
unfiltered real MR images.

Page  42

Page  43
(6.12 a) – input MR image
(6.12 b) – ground truth segmented MR image – CSF, GM and WM
tissues
(6.12 c) – segmented MR image (our algorithm) – CSF, GM and
WM tissues

Page  44

Page  45
GM
WM

Page  46
(a)
(c)
(b)
(d)
(a) original 2D histogram
(b), (c), (d) classified 2D histograms at the following percentile intervals:
(b) [0.005,0.995] (c) [0.05,0.95] (d) [0.20,0.60]

Page  47
(a) (b)
(c) (d)
(а) ground truth segmented MR image
(b), (c), (d) segmented MR image at the following percentile intervals:
b) [0.005,0.995] c) [0.05,0.95] d) [0.20,0.60]

Page  48
Analysis of the Segmentation Results
Our results:
 are comparable with those of the other advanced
state-of-the-art segmentation methods
 are better compared with those of the basic
segmentation methods
 vary depending on the type of MRI data – real vs
artificially generated, filtered vs unfiltered
 are the best with Brainweb, then IBSR18 and
finally IBSR20, respectively (like the other
benchmark methods)

Page  49
 depend on the compactness of the 2D histogram
along the diagonal, peaks (Brainweb) and the
presence of outliers (IBSR20)
 are lower in the case of tubular structures (WM)
with interrupted or/and trimmed ends (IBSR 20)
 are better in the case of more compact tissues
(GM) (IBSR20)
 are better as we add edges bins to classified bins
of main tissues in the test 2D histogram

Page  50
Our method unlike the other methods:
 selects automatically MR images from the
sequence to build a segmentation model
 uses only one basic structure – a 2D histogram,
not a long list of features; has an opportunity for
elaboration
 sets a minimum number of 2D histogram bins and
MR image pixels for each tissue

Page  51
4.3 Test MRI data
execution time

Page  52
Time Performance
We apply analysis of the time performance of
our segmentation method in the following
hardware and software environment (a bit
outdated ):

Page  53
PC Hardware and Software Environ.
 PC hardware
 64 bits, Intel Core i5-440/ 3.1 GHz with RAM 8 GB
DDR3L at 1600 MHz
 Software environment
 Matlab 2013a, Image Processing, Statistics, Parallel
computing toolboxes
 R Studio, R packages – ggplot, gridExtra, hexbin,
R.matlab, knitr, gdata, data.table – to load and visualize
the final results
 additional libraries as mLMNN2.4 (distance metric
learning), FieldTrip (Oostenveld et al., 2011), MoMinc,
Nifti, k-Wave1.0 toolboxes

Page  54
Execution Time
 The developed method segments a single MRI
sequence for about:
 5-7 minutes for unfiltered real MRI (IBSR20) data with
LMNN distance metric learning and w/o parallelization
 2-3 minutes for unfiltered real MRI (IBSR20) data, with
LMNN distance metric learning and with parallelization
 1 minute for unfiltered real MRI data (IBSR20), w/o

Page  55
Execution Time
 18 minutes for filtered real MRI data (IBSR18), with
 22 minutes for artificially generated (Brainweb) MRI
data with LMNN distance metric learning and with
parallelization

Page  56
Analysis of Execution Time
Time performance of our method:
 the slowest execution time steps are distance
metric learning, 2D histogram matching and
dimension rescaling (Brainweb preprocessing)
 middle execution time steps are segmentation
using back projection, 2D histogram kNN
classification (w/o LMNN distance learning)
 the other steps have negligible execution time

Page  57
Speeding it up
 We parallelize the following algorithms, as we:
 divide into MR image subsequences
 match model and segment 2D histograms
 calculate LUT with the model and test vectors
 We apply strict memory management, as we:
 remove from the memory all matrices after use
 vectorize all possible operations

Page  58
Speeding it up
Some additional comments:
 execution time would depend significantly on the
type of the test platform
 use of more processor cores (for parallel work) will
definitely speed the method up
 quantization of the 2D histogram will improve the
speed but might worsen the final results

Page  59
Contents
1. Main idea and contributions
2. Introduction
3. Method description
4. Experimental results
5. Conclusions and future work

Page  60
Conclusions
The main points of the segmentation method:
 the method adapts to the tissue characteristics
along the MR image sequences
 our 2D histogram provides moderate distinction
between the classes of separate tissues and
edges
 the application of 2D histogram matching within a
percentile interval decreases the outlier influence

Page  61
Conclusions
 the addition of edges bins to the classified test 2D
histogram during back projection improves the
segmentation results
 parameter values of each algorithm of the
segmentation method should be followed strictly

Page  62
Future Work
The current work can be extended in the
following directions:
 increase the speed and decrease the memory
consumption as we modify a few of the algorithms
 improve the results considering contours with a
specific form between the neighboring tissues
 apply it to other types of test images with compact
2D histograms

Page  63
References
Azmi et al.: Ensemble semi-supervised framework for brain magnetic resonance imaging
tissue segmentation. J Med Signals Sens 3(2), 94–106 (2013).
Cabezas et al.: A review of atlas-based segmentation for magnetic resonance brain images.
Comput. Meth Prog Bio 104(3), 158–177 (2011).
Chen, S., Radke, R.J.: Level set segmentation with both shape and intensity priors. In:
ICCV, 2009.
Chen et al.: A 2-phase 2-D thresholding algorithm. Digit Signal Proc 20(6), 1637–1644
(2010).
Dong, F., Peng, J.: Brain MR image segmentation based on local Gaussian mixture model
and nonlocal spatial regularization. J Vis Commun Image Represent 25(5), 827–839 (2014).
Ekin, A.: Pathology-robustmr intensity normalization with global and local constraints. In:
ISBI, 2011.
Foruzan, et al.: Segmentation of brain tissues using a 3-D multi-layer Hidden Markov Model.
Comput Biol Med 43(2), 121–130 (2013).
Gonzalez & Woods: Digital image processing. 3rd edn (2007).
Hedges, T.: Technical note. an empirical modification to linear wave theory. In: ICE Proceed
ings, Thomas Telford, vol. 61, pp. 575–579 (1976).
Ji et al.: Robust spatially constrained fuzzy c-means algorithm for brain MR image
segmentation. Pattern Recogn 47(7), 2454–2466 (2014).
Ji et al.: Fuzzy local Gaussian mixture model for brain MR image segmentation. IEEE TITB
16(3), 339–347 (2012).

Page  64
References
Kirby & Rosenfeld. A Note on the Use of (Gray Level, Local Average Gray Level) Space as
an Aid in Threshold Selection. No. TR-722. MARYLAND UNIV COLLEGE PARK COMPUTER
SCIENCE CENTER, 1979.
Lee et al.: Segmentation of interest region in medical volume mages using geometric
deformable model. Comput Biol Med 42(5), 523–537 (2012).
Lee et al.: MR image segmentation using a power transformation approach. IEEE TMI 28(6),
894–905 (2009).
Li et al.: A level set method for image segmentation in the presence of intensity
inhomogeneities with application to MRI. IEEE TIP 20(7), 2007–2016 (2011).
Lin et al.: Adaptive pixon represented segmentation (APRS) for 3D MR brain images based
on mean shift and Markov random fields. Pattern Recogn Lett 32(7), 1036–1043 (2011).
Mahmood et al.: A novel Bayesian approach to adaptive mean shift segmentation of brain
images. In: CBMS, 2012.
Monaco & Madabhushi: Class-specific weighting for Markov random field estimation:
Application to medical image segmentation. Med Image Anal 16(8), 1477–1489 (2012).
Morin et al.: Image segmentation using random-walks on the histogram. In: Proceedings of
SPIE, 2012.
Ortiz et al.: Two fully-unsupervised methods for MR brain image segmentation using SOM-
based strategies. Appl Soft Comput13(5), 2668–2682 (2013).
Robitaille et al.: Tissue-based MRI intensity standardization: application to multicentric
datasets. Int. J. Biomed Imaging 2012 (347), 120 (2012).

Page  65
References
Robitaille et al.: Tissue-based MRI intensity standardization: application to multicentric
datasets. Int J Biomed Imaging 2012 (347), 120 (2012).
Roy et al..: Consistent segmentation using a Rician classifier. Med Image Anal 16(2), 524–
535 (2012).
Shapira et al., “Multiple Histogram Matching”, In: ICIP, 2013.
Shen, D. Image registration by local histogram matching. Pattern Recognit 40(4),1161–1172
(2007).
Solanas et al.: Relative anatomical location for statistical non-parametric brain tissue
classification in MR images. In: ICIP, 2001.
Tohka J., Partial volume effect modeling for segmentation and issue classification of brain
Magnetic resonance images: A review. World journal of radiology 6.11(2014):855 2014.
Valverde, S. et al.: "Comparison of 10 brain tissue segmentation methods using revisited
IBSR annotations." J Magn Reson Imaging 41.1 (2015): 93-101 2015.
Vrooman et al. : Auto-kNN: Brain Tissue Segmentation using Automatically Trained k-
Nearest-Neighbor Classification (2013).
Wang et al.: Fast histogram equalization for medical image enhancement. In: EMBS, 2008.
Wu et al: Medical image thresholding algorithm based on fuzzy sets theory. In: ICIEA, 2007.
Xiao et al.: "Entropic image thresholding based on GLGM histogram." Pattern Recogn Lett
40 (2014): 47-55.
Xiao et al., Entropic thresholding based on gray-level spatial correlation histogram, in: ICPR,
2008.

Page  66
References
Yimit, et al., 2-D direction histogram based entropic thresholding, Neurocomput 120 (23)
(2013) 287–297.
Zhang & Hu: Image segmentation based on 2D Otsu method with histogram analysis. In:
ICCSSE, 2008.
Zhang & Hu: Curvilinear thresholding method for noisy images based on 2D histogram. In:
ROBIO, 2009.
Zhang, et al.: Semi-supervised tissue segmentation of 3D brain MR images. In: iV, 2010.
Zhang et al.: Segmentation of brain MR images through a hidden Markov random field
model and the expectation-maximization algorithm. IEEE TMI 20(1), 45–57:2 (2001).
Zhang et al.: Hidden Markov random field model based brain MR image segmentation using
clonal selection algorithm and Markov chain Monte Carlo method. Biomed Signal Proces 12,
10–18 (2014).
Zhang et al.: A clonal selection based approach to statistical brain voxel classification in
magnetic resonance images. Neurocomput 134, 122–131 (2014).

Tissue segmentation methods using 2D histogram matching in a sequence of mr brain images part 3

Recommended

Recommended

More Related Content

What's hot

What's hot (16)

Similar to Tissue segmentation methods using 2D histogram matching in a sequence of mr brain images part 3

Similar to Tissue segmentation methods using 2D histogram matching in a sequence of mr brain images part 3 (20)

Recently uploaded

Recently uploaded (20)

Tissue segmentation methods using 2D histogram matching in a sequence of mr brain images part 3