Presentation for Ultrasound Shadowing Artifact Removal (IUS, 2021)

1. Jaeyoung Huh,
Shujaat Khan, and Jong Chul Ye
BISPL - BioImaging, Signal Processing, and Learning lab.
KAIST, Korea
Unsupervised Learning for
Acoustic Shadowing Artifact Removal
in Ultrasound Imaging

2. Introduction
 Acoustic Shadowing
 It happens usually with solid
structure like bone.
 The signal is highly reflected or
strongly absorb, therefore, the
received signal is highly attenuated.
 The structure in dark region cannot
be shown correctly, so it hinders
accurate diagnosis .
High Attenuation

3. Strategy
 Acoustic Shadowing
 Applying deep learning technique for fast and accurate performance.
 There is no paired-dataset, therefore, we applied unsupervised learning.
 There is no shadowing-free dataset, therefore, we crop the patches where does not
contain any shadowing region.

4. Background
 One of the representative unsupervised method.
 It minimizes the distance between 𝜇, 𝜇𝜃 and 𝜈, 𝜈𝜙.
 Geometry of CycleGAN  CycleGAN structure
 CycleGAN

5. Our contribution
 𝐦𝐢𝐧
𝜽,𝝓
𝐦𝐚𝐱
𝜻,𝜼
𝑳𝒕𝒐𝒕𝒂𝒍 = 𝑳𝑫𝒊𝒔𝒄 + 𝝀cycle𝑳𝒄𝒚𝒄𝒍𝒆 + 𝝀GP𝑳GP + 𝝀iden𝑳𝒊𝒅𝒆𝒏
 Proposed method
 LDisc = 𝐸𝑦~𝑃𝑦
[𝐷𝜁 𝑦 ] − 𝐸𝑦~𝑃𝑦
[𝐷𝜂 𝐺𝜃 𝑦 ] + 𝐸𝑥~𝑃𝑥
[𝐷𝜂 𝑥 ] − 𝐸𝑥~𝑃𝑥
[𝐷𝜁 𝐹𝜙 𝑥 ]
 Lcycle = 𝐸𝑦~𝑃𝑦
𝑦 − 𝐹𝜙 𝐺𝜃 𝑦
1
+ 𝐸𝑥~𝑃𝑥
𝑥 − 𝐺𝜃 𝐹𝜙 𝑥
1
 LGP = −𝐸𝑦~𝑃𝑦
∇yD𝜁 y
2
− 1
2
− 𝐸𝑥~𝑃𝑥
∇xD𝜂 x
2
− 1
2
(y ← 𝜖y + 1 − 𝜖 F𝜙, x ← 𝜖x + 1 − 𝜖 G𝜃 y )
 L𝑖𝑑𝑒𝑛 = 𝐸𝑦~𝑃𝑦
𝑦 − 𝐹𝜙 𝑦
1
+ 𝐸𝑥~𝑃𝑥
𝑥 − 𝐺𝜃 𝑥 1

6. Our contribution
 Generator, Discriminator Structure
 The generator is simple ResNet structure.
 The discriminator is simple Patch-GAN.
(b) 4x4 Convolution with stride 2 + Leaky ReLU
4x4 Convolution with stride 2 + InstNorm + Leaky ReLU
4x4 Convolution with stride 1 + InstNorm + Leaky ReLU
4x4 Convolution with stride 1
3x3 Convolution with stride=1 + InstNorm
1x1 Convolution with stride=1
3x3 Convolution with stride=1 + InstNorm + ReLU
3x3 Convolution with stride=2 + InstNorm + ReLU
(Bilinear Up-sampling + 3x3 Convoluton with stride=1) + InstNorm +
ReLU
(a)

7. Training Details
Implementation Details
Total Epoch 100
Learning Rate 1e-4
Optimizer Adam Optimizer
Batch size 4
Parameter (𝜆GP, 𝜆rec, 𝜆iden, 𝜆adv)= (1,10,1,1)
Optimization
Formulation
WGAN with Gradient-Penalty
Dataset Details
Training set
672 images (9 subjects) - input
723 patches ( 7 subjects) - target
Validation set 72 images (4 subjects)
Test set
141 images ( 3 subjects)
13 images (hyper-echoic phantom)
Data
Augmentation
Flipping, Rotating, Random Scaling
Normalization Normalized all images to 0~1
Patch size 96

8. Results
Fig. 1. The results of B-mode in vivo images
Input
(Shadowing)
Output
(Shadowing-free)
Difference Map
Invivo 1 Invivo 2 Invivo 3 Invivo 4
60dB
0dB
30dB
60dB
0dB
30dB
0
255
127

9. Conclusion
 The shadowing is one of the prominent artifact in US image.
 We propose the shadowing removal method using deep learning.
 To overcome the un-paired dataset problem, we used un-supervised method based on
the CycleGAN.

10. Thank You