This document provides a summary of a PhD presentation on automated kidney segmentation in 3D ultrasound imagery and its application in computer-assisted trauma diagnosis. The key points are: 1) Rapid diagnosis of abdominal trauma is critical for survival but ultrasound exams can be improved with computer assistance. The presentation focuses on automatically detecting and segmenting the kidney in ultrasound images of the right upper quadrant view, which is most relevant for diagnosing bleeding. 2) A complex-valued implicit shape model (CVISM) is used to mathematically represent the multi-regional structure of the kidney. This shape prior aids in detecting the kidney alignment within ultrasound volumes. 3) For segmentation, a complex-valued regional level-set method with

1. Automated Kidney Segmentation In 3D Ultrasound Imagery,
and its Application in Computer-assisted Trauma Diagnosis
Final PhD Oral Examination - Presentation Slides
September 1st , 2016
Mahdi Marsousi
Supervisor: Prof. Konstantinos N. Plataniotis

2. Introduction
 Abdominal trauma: injuries to abdomen, either blunt or penetrating, resulting
in severe blood loss.
 Importance: massive internal bleeding quickly threats patient’s life.
 Cure: rapid diagnosis and surgery
[1]
[1] http://www.radiologyassistant.nl/en/p466181ff61073/acute-abdomen-role-of-ct-in-trauma.html
[2] http://www.wikidoc.org/index.php/Abdominal_trauma
[2]
Slide #1

3. Diagnosis by 3D ultrasound (3DUS)
 Ultrasound imagery is the preferred imaging modality for diagnosing
hemodynamically unstable patients, because it is portable.
 It is also called: focused assessment with sonography for trauma “FAST”.
 2D ultrasound is the popular diagnostic tool.
 3DUS imagery is key to design computer-assisted diagnosis, because:
a) detecting and localizing internal organs in 3D space is possible by 3DUS;
b) 3DUS facilitates measuring the volume of internal bleeding.
Slide #2

4. :Focus of this presentation
FAST exam
 FAST exam: rapid bedside ultrasound scan to find free fluids, as
indication of internal bleeding, around abdominal organs [1].
 Six major abdominal views associated with FAST exam.
 Right Upper Quadrant view (2), also called the Morison’s pouch
view, is the most relevant one, because:
 It is the most sensitive view to abdominal bleeding
 It entirely shows the right kidney, and a portion of liver and diaphragm[1].
 Abdominal bleeding usually places between the right kidney and liver [1].
 Therefore, detecting and segmenting the kidney shape is the key
for computer-assisted trauma diagnosis.
[1] Heller et al. Ultrasound use in trauma: the fast exam. Academic Emergency Medicine, 14(6):525–525, 2007.
[2] Ingeman et al. Emergency physician use of ultrasonography in blunt abdominal trauma. Academic Emergency
Medicine, 3(10):931–937, 1996.
Slide #3

5. Motivation, problem definition, & challenges
 Motivation:
• In emergency, only paramedics provide medical service for trauma patients.
• Usually paramedics are not capable to do FAST exam.
• Thus, computer-assisted solution can help paramedics to conduct trauma diagnosis.
 Problem definition:
Slide #4

6. 3DUS dataset
Slide #5
 Training set: 6 with-kidney and 6 without-kidney images.
 Evaluation set: 15 with-kidney and 15 without-kidney from healthy volunteers,
and 8 with-kidney images from abnormal patients.
 Images of abnormal patients represent morphological changes to the kidney shape.
With-kidney image:
taken from Morison’s pouch view
Without-kidney image:
taken from other views.

7. Contributions
Slide #6

8. Contribution 1:
Kidney detection

9. Overview on kidney detection
Definition: searching within 3D image domain to find kidney shape to answer:
• whether kidney shape exists in a 3DUS image?
• if yes, what is its alignment?
State-of-the-art:
Proposed solutions:
Name Methodology Advantage Disadvantage
Noll et al.
[1]
- applies volume enhancement
- uses redial rays to find kidney shape mass-center
robust against speckle &
low-contrast intensity
may wrongly detect other
structures instead of kidney
[1] Noll et al. Automated kidney detection and segmentation in 3d ultrasound, in Clinical Image-Based
Proc. Translational Research in Medical Imag. Springer, 2014, pp. 83–90.
Slide #7

10.  Definition: mathematical representation of kidney shape to add shape prior in kidney detection
 Proposed shape modeling: complex-valued implicit shape model (CVISM)
Ψ 𝑋 = 𝜓 𝑃𝑆 𝑋 + 𝜓 𝐾𝐶 𝑋 − 𝑖 ⋅ 𝜓 𝑅𝑀 𝑋 ,
 𝜓 𝑝𝑠, 𝜓 𝐾𝐶, and 𝜓 𝑅𝑀 are real-positive functions, defining voxel’s membership to Pyelocalyceal System,
Kidney Capsule, and Renal Medulla, respectively.
 Generating regions: using ground truth data of training set of with-kidney images.
Kidney shape model
Slide #8
Kidney
Capsule
Renal
Medulla
Pyelocalyceal
System
Real values are used to
model bright regions:
𝐾𝐶: Kidney Capsule
𝑃𝑆: Pyelocalyceal System
Imaginary values are used
to model dark region:
𝑅𝑀: Renal Medulla

11.  Objective: registering kidney CVISM on 3DUS images, to detect kidney shape and
estimate its alignment.
 Challenges:
• Partial occlusion of kidney shape
• Non-kidney structures with similar appearance to the kidney shape
 Components:
• Global deformation based on similarity transformation,
• Similarity metric,
• optimization algorithm.
 Global deformation: similarity transformation (shape-preserving)
𝑌 = 𝑆𝑇 𝑝𝑠𝑡,1
𝑋 + 𝑝𝑠𝑡,2
Shape-to-volume registration
Slide #9
Orientations & scaling
𝑝𝑠𝑡,1 = 𝜃𝑥,𝜃𝑦,𝜃𝑧,𝑠
𝑇
Translations
𝑝𝑠𝑡,2 = 𝑡𝑥,𝑡𝑦,𝑡𝑧
𝑇
CVISM domain Image domain

12.  Definition: quantifies similarity of deformed kidney CVISM and image data.
 Proposed similarity metric: Regularized complex normalized cross-correlation (RCNCC)
 Formulation:
Γ 𝑝𝑠𝑡,1, 𝑝𝑠𝑡,2
∗
= max
𝑝𝑠𝑡,2
∗ ∈ΩV
𝑠𝑢𝑏
1
Λ 𝑝𝑠𝑡,1, 𝑝𝑠𝑡,2
⋅
max 0,ℜ Σ𝐼 𝑝𝑠𝑡,1, 𝑝𝑠𝑡,2
Σ𝐼𝐼
ℜ
𝑝𝑠𝑡,1
⋅
max 0,ℑ Σ𝐼 𝑝, 𝑋, 𝑉
Σ𝐼𝐼
ℑ
𝑝𝑠𝑡,1
Λ 𝑝𝑠𝑡,1, 𝑝𝑠𝑡,2 increases as kidney CVISM aligns out of the ultrasound field of view.
RCNCC similarity metric
Regularization
Term
Checking similarity of image data
with Kidney Capsule and
Pyelocalyceal System
Checking similarity of image
data with Renal Medulla
Robustness against
Partial kidney shape occlusion
Considering multi-regional structure of the kidney shape,
max 𝑎, 𝑏 : selects maximum of 𝑎 and 𝑏.
“max”s are used to avoid multiplication of two negative values in RCNCC.
ℜ{𝑎} and ℑ{𝑏} extract real part of 𝑎 and imaginary part of 𝑏.
Slide #10

13. Registration problem
 Objective: Finding 𝑝𝑠𝑡,1
∗
and 𝑝𝑠𝑡,2
∗
which provides a maximum RCNCC for input image.
Optimization problem:
Γ∗, 𝑝𝑠𝑡,1
∗
, 𝑝𝑠𝑡,2
∗
= max
𝑝𝑠𝑡,1
Γ 𝑝𝑠𝑡,1, 𝑝𝑠𝑡,2
∗
 Optimization algorithm:
• initialization: searching for best seed point from a
set of seed points to initialize registration.
• iterative improvement: iteratively updating
registration parameters using Gradient Descent.
Figure shows RCNCC metric versus 𝜃 𝑥 and 𝜃𝑧.
Yellow regions correspond to desirable registration solutions.
The gray stars are seed point.
Red star is selected seed point.
Black arrows show iterations of Gradient descent.
Slide #11

14. Processing pipeline of kidney detection
• Γ 𝑡ℎ is threshold on similarity metric to decide whether the kidney shape exists or not.
• It is obtained using training set, as Γ 𝑡ℎ = 3.5.
Slide #12

15. Experimentation
 Objective: to evaluate shape-based and atlas-based kidney detection, compared to Noll et al. [1].
 Dataset:
• 15 with-kidney and 15 without-kidney images of healthy volunteers
• 8 with-kidney images abnormal patients
• 3DUS image simulator of Morison’s pouch
 Comparison metrics:
• accuracy: 𝐴𝑐𝑐 𝐾𝐷 = 100% ⋅
𝑁 𝑇𝑃+𝑁 𝑇𝑁
𝑁 𝑇𝑃+𝑁 𝐹𝑁+𝑁 𝑇𝑁+𝑁 𝐹𝑃
,
• sensitivity: 𝑆𝑒𝑛𝑠 𝐾𝐷 =
NTP
𝑁 𝑇𝑃+𝑁 𝐹𝑁
,
• specificity: 𝑆𝑝𝑒𝑐 𝐾𝐷 =
𝑁 𝑇𝑁
𝑁 𝑇𝑁+𝑁 𝐹𝑃
.
𝑁 𝑇𝑃, 𝑁 𝑇𝑁, 𝑁𝐹𝑃, and 𝑁𝐹𝑁 are numbers of true positive, true negative, false positive, false negative detections.
[1] Noll et al. Automated kidney detection and segmentation in 3d ultrasound, in Clinical Image-Based
Proc. Translational Research in Medical Imag. Springer, 2014, pp. 83–90.
Slide #13

16. Visibility portion: 100 100 100 100 100 100 100 100 100 50 70 95 100 100 100 95 70 45 55 75 90 100 100 100 95 75 55
Sensitivity analysis
 Ultrasound volume simulator is used to analyze sensitivity to kidney shape’s deformation and occlusion.
 Both methods provides high robustness toward kidney shape translation.
 Atlas-based method provides more accurate estimation of deformation (except for Δ𝜃 𝑥)
Slide #14
40 70 95 100 100 100 85 65 45 50 60 85 100 100 100 90 60 4560 80 95 100 100 100 95 85 65Visibility portion:

17. Kidney detection accuracy
 Using with-kidney and without-kidney images of healthy volunteers:
• Shape-based and atlas-based provide higher accuracy, compared to Noll et al., because:
1) higher specificity: involving multi-structural regions of kidney shape,
2) higher sensitivity: regularization factor provides robustness against kidney shape’s occlusion.
• Atlas-based shows higher accuracy than shape-based because of using texture information.
 Using actual ultrasound volumes of abnormal patients:
Accuracy is reduced, because:
 different device settings from healthy volunteers,
 Morphological changes of RUQ view in abnormal patients.
Method 𝑵 𝑻𝑷 𝑵 𝑻𝑵 𝑵 𝑭𝑷 𝑵 𝑭𝑵 Accuracy (%) Sensitivity Specificity
Noll et al. 9 11 4 6 66.67 0.60 0.73
Shape-based kidney detection 14 12 3 1 86.67 0.93 0.80
Atlas-based kidney detection 15 13 2 0 93.33 1 0.87
Slide #15
Method 𝑵 𝑻𝑷 𝑵 𝑭𝑵 Accuracy (%)
Noll et al. [1] 5 3 62.5
Shape-based 5 3 62.5
Atlas-based 6 2 75.0

18. Examples of kidney detection
Blue region: Renal Medulla
Red region: Kidney Capsule/Pyelocalyceal System
Slide #16

19. Contribution 2:
Automated kidney segmentation

20. Overview on automated kidney segmentation
 Problem definition: automatically segmenting the kidney shape in 3DUS images.
 State-of-the-art:
 Proposed method: complex-valued regional level-set with shape prior (CVRLS-SP)
• Complex-valued structure is used to add multi-regional structure into segmentation.
• Adding multi-regional structure is important because it improves segmentation specificity.
Name Methodology Advantage Disadvantage
Noll
et al. [1]
Radial ray trace + fast
marching + edge-based level-
set
- Robust against speckle and low-contrast
intensity profile
- Automated initialization
- Not robust against kidney deformation
- High-computational cost
MRF-AC
[2]
2D active contour + Markov
random field (MRF) + 3D
reconstruction
- Low computational cost - Manual initialization
- Discontinuity along z-axis
[1] Noll et al. Automated kidney detection and segmentation in 3d ultrasound, in Clinical Image-Based Proc.
Translational Research in Medical Imag. Springer, 2014, pp. 83–90.
[2] Martn-Fernndez et al., An approach for contour detection of human kidneys from ultrasound images
using markov random fields and active contours, Med. Image Anal., 1–23, 2005.
Slide #17

21. CVRLS-SP representation
 Mathematical representation of regions in CVRLS-SP:
𝑏𝑟 : 𝑋 ∈ 𝑝𝑦𝑒𝑙𝑜𝑐𝑎𝑙𝑦𝑐𝑒𝑎𝑙 𝑠𝑦𝑠𝑡𝑒𝑚 𝑜𝑟 𝑘𝑖𝑑𝑛𝑒𝑦 𝑐𝑎𝑝𝑠𝑢𝑙𝑒, 𝑖𝑓 ℜ 𝜙 𝑋 > 0,
𝑑𝑟 : 𝑋 ∈ 𝑟𝑒𝑛𝑎𝑙 𝑚𝑒𝑑𝑢𝑙𝑙𝑎, 𝑖𝑓 ℑ 𝜙 𝑋 > 0,
𝑏𝑔 : 𝑋 ∈ 𝑏𝑎𝑐𝑘𝑔𝑟𝑜𝑢𝑛𝑑, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒.
where 𝜙 is level-set function.
 Prior shape: (𝜙𝑠) is generated by aligning CVISM on detected kidney shape:
𝜙𝑠 𝑋 = 2 Ψ 𝑆𝑇 𝑝 𝑠𝑡,1
∗ 𝑋 + 𝑝𝑠𝑡,2
∗
> 0 − 1 + 𝑖 , ∀𝑋 ∈ Ω 𝑉
where 𝑆𝑇 𝑝 𝑠𝑡,1
∗ 𝑋 + 𝑝𝑠𝑡,2
∗
places kidney CVISM, Ψ, on detected kidney shape.
 level-set function, 𝜙, is initialized by prior shape: 𝜙 𝑋; 𝑡 = 0 = 𝜙𝑠 𝑋 , 𝑡 is iteration.
Slide #18

22. CVRLS-SP evolution
[1] Chan and Zhu. Level set based shape prior segmentation. IEEE CVPR, vol 2, pp 1164–1170, 2005..
Slide #19
 Conventional regional level-set with shape prior defines energy function as [1],
𝐹 𝑐, 𝜙, 𝜙𝑠 𝐴𝑇 𝑝 𝑎𝑓
, 𝑉 = 𝜆 ⋅ 𝐹𝑆𝑃 𝜙, 𝜙𝑠 𝐴𝑇 𝑝 𝑎𝑓
+ 𝜇 ⋅ 𝐹𝑖𝑛𝑡 𝜙 + 𝛾 ⋅ 𝐹𝑒𝑥𝑡(𝑐, 𝜙, 𝑉)
𝐹𝑆𝑃 𝜙, 𝜙𝑠 𝐴𝑇 𝑝 𝑎𝑓
: shape prior energy,
𝐹𝑖𝑛𝑡 𝜙 : internal energy, controlling smoothness.
𝐹𝑒𝑥𝑡(𝑐, 𝜙, 𝑉): external energy, pushing level-set toward region of interest.
𝑐: average intensity level of region of interest, 𝐴𝑇 𝑝 𝑎𝑓
: affine transformation.
𝜆, 𝜇, 𝛾: Lagrange multipliers
 Contribution: CVRLS-SP’s energy functional is defined as follows,
𝐹 𝑐 𝑏𝑟, 𝑐 𝑑𝑟, 𝜙, 𝜙𝑠 𝐴𝑇 𝑝 𝑎𝑓
, 𝑉 = 𝜆 ⋅ 𝐹𝑆𝑃
𝑏𝑟
ℜ{𝜙}, ℜ{𝜙𝑠 𝐴𝑇 𝑝 𝑎𝑓
} + 𝜆 ⋅ 𝐹𝑆𝑃
𝑑𝑟
ℑ{𝜙}, ℑ{𝜙𝑠 𝐴𝑇 𝑝 𝑎𝑓
}
+𝜇 ⋅ 𝐹𝑖𝑛𝑡 𝜙 + 𝛾 ⋅ 𝐹𝑒𝑥𝑡
𝑏𝑟
(𝑐 𝑏𝑟, ℜ{𝜙}, 𝑉) + 𝛾 ⋅ 𝐹𝑒𝑥𝑡
𝑑𝑟
(𝑐 𝑑𝑟, ℑ{𝜙}, 𝑉)
• Multi-regional segmentation is added By dividing both FSP and Fext into br and dr terms.

23. CVRLS-SP segmentation procedure
 Initialization: Creating prior shape and level-set initialization
 Iterative evolution:
 𝜙 is updated by reducing 𝐹 𝑐 𝑏𝑟, 𝑐 𝑑𝑟, 𝜙, 𝜙𝑠 𝐴𝑇 𝑝 𝑎𝑓
, 𝑉 , using Euler-Lagrange equation.
 convergence criteria: ΣΔ𝜙 𝑡 = ∭ 𝜙 𝑋, 𝑡 − 𝜙 𝑋, 𝑡 − 1
2
𝑑𝑥𝑑𝑦𝑑𝑧
Slide #20

24. Experimentation
 Objective: to evaluate proposed kidney segmentation compared to Noll et al. [1] and MRF-AC. [2]
 Dataset: evaluation set of actual ultrasound volumes
 Comparison metrics:
• Dice’s similarity coefficient (DSC)=
2𝑇𝑃
2𝑇𝑃+𝐹𝑁+𝐹𝑃
,
• Accuracy metric (ACC)=
𝑇𝑃+𝑇𝑁
𝑇𝑃+𝑇𝑁+𝐹𝑃+𝐹𝑁
× 100% ,
• Sensitivity measure (SENS)=
𝑇𝑃
𝑇𝑃+𝐹𝑁
,
• Specificity measure (SPEC)=
𝑇𝑁
𝑇𝑁+𝐹𝑃
.
 Parameter setting:
[1] Noll et al. Automated kidney detection and segmentation in 3d ultrasound, in Clinical Image-Based
Proc. Translational Research in Medical Imaging, Springer, 2014, pp. 83–90.
[2] Martin-Fernandez et al., An approach for contour detection of human kidneys from ultrasound images
using Markov random fields and active contours, Medical Image Analysis, 1–23, 2005.
Slide #21
Name 𝜆 𝜇 𝛾 𝑁𝑖𝑡𝑟 𝑡 𝑚𝑎𝑥 𝜖
Value 1 0.1 0.05 5 50 10

25.  Specificity of CVRLS-SP is higher because:
• capability of segmenting multi-regional structure,
• using shape prior.
 Box-plot: shows statistical variation of DSC of the methods:
minimum accuracy of CVRLS-SP is higher that maximum accuracy of both MRF-AC and Noll et al.
Kidney segmentation accuracy
Method
𝑫𝑺𝑪 𝑨𝑪𝑪 𝑺𝑬𝑵𝑺 𝑺𝑷𝑬𝑪
𝜇 𝜎 𝜇 𝜎 𝜇 𝜎 𝜇 𝜎
CVRLS-SP 0.8143 0.0408 97.48 0.72 0.7863 0.0814 0.9890 0.0060
Noll et al. 0.4207 0.0795 85.48 2.77 0.7024 0.0912 0.8678 0.0326
MRF-AC 0.5921 0.1457 93.10 2.74 0.6947 0.2132 0.9490 0.0246
𝜇: mean, 𝜎:standard deviation
Slide #22
CVRLS-SP MRFAC Noll et al.
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Dice'sCoefficient(DSC)

26. Examples of kidney segmentation
Fourth column:
Yellow, red, and green colors are TP, FP, and FN regions. Slide #23
Original image Detected kidney Segmented
kidney regions
Comparing
automated segmentation
vs. ground-truth
3D view
Second and third columns:
Blue and red regions indicate Renal Medulla and
Pyelocalyceal System/Kidney Capsule, respectively.

27. Conclusions & Future Work
In this presentation, importance of computer-assisted solution for diagnosing
trauma patients by paramedics in emergency situation was discussed.
Shape-based and atlas-based kidney detection methods were introduced.
Automated kidney segmentation was introduced, which used kidney detection
module to automatically initialize segmentation process.
Kidney detection and segmentation accuracies of proposed methods were compared
to state-of-the-art, and evaluations and analysis confirmed improved accuracy using
proposed methods.
Future work:
Using the automated kidney segmentation to automatically detect free fluids.
Slide #24

28. Publications
 Journal Papers:
[1] M. Marsousi, K. Plataniotis, and S. Stergiopoulos, “An Automated Approach for Kidney Segmentation in Three-Dimensional Ultrasound Images”,
IEEE Journal of Biomedical and Health Informatics, DOI: 10.1109/JBHI.2016.2580040, (Date of publication: June 13th 2016)
[2] M. Marsousi, K. Plataniotis, and S. Stergiopoulos, “Computer-Assisted 3D Ultrasound Probe Placement for Emergency Healthcare Applications”,
IEEE Transactions on Industrial Informatics, DOI: 10.1109/TII.2016.2569522, (Date of publication: May 18th 2016)
[3] M. Marsousi, K. Plataniotis, and S. Stergiopoulos, “Kidney Detection in 3D Ultrasound Imagery Based on Shape and Texture Priors”, to be
submitted at IEEE Transaction on Biomedical Engineering (T-BME) on September 2016.
 Conference Papers:
[4] M. Marsousi, X. Lee, and K. Plataniotis, “Shape-included Label-Consistent Discriminative Dictionary Learning: An Approach to Detect and
Segment Multi-Class Objects in Image”, IEEE international conference on Image Processing (ICIP), 2016, accepted.
[5] M. Marsousi, K. Plataniotis, and S. Stergiopoulos. “Atlas-based segmentation of abdominal organs in 3D ultrasound, and its application in
automated kidney segmentation.”, IEEE International conference in Engineering in Medicine and Biology Society (EMBC), pp. 2001-2005, 2015.
[6] M. Marsousi and K. Plataniotis, “Binomial classification based on DLENE features in sparse representation: Application in kidney detection in
3D ultrasound.” IEEE International Conference in Acoustics, Speech and Signal Processing (ICASSP), pp. 1017-1021, 2015.
[7] M. Marsousi, K. Plataniotis, and S. Stergiopoulos, “Shape-based kidney detection and segmentation in three-dimensional abdominal ultrasound
images”. IEEE International conference in Engineering in Medicine and Biology Society (EMBC), pp. 2890-2894, 2014.
[8] M. Marsousi, K. Plataniotis, and S. Stergiopoulos. “A multi-steps segmentation approach for 3D ultrasound images using the combination of 3D-
Snake and Level-Set.” IEEE International Conference in Digital Signal Processing (DSP), pp. 1-4, 2013.
 US patent:
[9] S. Stergiopoulos, P. Shek, K. Plataniotis, and M. Marsousi. “Computer aided diagnosis for detecting abdominal bleeding with 3D ultrasound
imaging.” U.S. Patent Application 14/159,744, filed January 21, 2014.