Recent proposals of deep learning-based beamformers for ultrasound imaging (US) have attracted significant attention as computational efficient alternatives to adaptive and compressive beamformers. Moreover, deep beamformers are versatile in that image post-processing algorithms can be readily combined. Unfortunately, with the existing technology, a large number of beamformers need to be trained and stored for different probes, organs, depth ranges, operating frequency, and desired target ‘styles’, demanding significant resources such as training data, etc. To address this problem, here we propose a switchable and tunable deep beamformer that can switch between various types of outputs such as DAS, MVBF, DMAS, GCF, etc., and also adjust noise removal levels at the inference phase, by using a simple switch or tunable nozzle. This novel mechanism is implemented through Adaptive Instance Normalization (AdaIN) layers, so that distinct outputs can be generated using a single generator by merely changing the AdaIN codes. Experimental results using B-mode focused ultrasound confirm the flexibility and efficacy of the proposed method for various applications.

1. Switchable and Tunable Deep Beamformer using
Adaptive Instance Normalization for Medical
Ultrasound
Bio-Imaging, Signal Processing and Learning,
Bio and Brain Engineering, KAIST
Shujaat Khan, Jaeyoung Huh, Jong Chul Ye
Published in: IEEE Transactions on Medical Imaging ( Volume: 41, Issue: 2, Feb. 2022)

2. Contents
I. Ultrasound Imaging (overview)
II. Universal Deep beamformer for Medical Ultrasound Imaging
III. Switchable and Tunable Beamformer
IV. Conclusion
2

3. I. Ultrasound Imaging (overview)
basics of ultrasound data acquisition and delay and sum (DAS) beamformer

4. Application Needs
 Better Image quality
 Reduce the number of measurement
- Ultra-fast US
- Portable US
- 3 dimensional US
Ultrasound Imaging Physics (overview) 4
Introduction

5. Acquisition Modes
Focused Imaging
Planewave Imaging
Ultrasound Imaging Physics (overview) 5

6. Ultrasound Imaging Physics (overview) 6
delay-and-sum (DAS) beamforming
Time-of-Flight Correction
Depth
Scanlines
Log compressed
B-Mode Image
Envelope
RF-sum Signal
Envelope
detection
depth
Scanlines
Dynamic
Range
selection

7. Ultrasound Imaging Physics (overview) 7
Spatio-temporal redundancy in RF-data
Rx-Xmit data from depth-Rx-Xmit data cube and its Fourier spectrum
Skew redundancy with in the SC/RX plane
(spatial redundancy)
Redundancy in neighboring planes
(temporal redundancy)
depth
Rx
TE
Focused Rx data cube
(TE)
Input
Rx
TE

8. II. Universal Deep beamformer for
Medical Ultrasound Imaging
Adaptive beamforming using convolutional neural network
Khan et al, IEEE TUFFC, 2020

9. Conventional approach
Conventional delay-and-sum (DAS) Beamforming Pipeline
Rx
probe
Beam
focusing
Signal
Adder
Envelope
detection
Log
compression
B-mode image
Hilbert
Transform
IQ data
(signal before envelope detection)
Universal Deep beamformer for Medical Ultrasound Imaging 9

10. Full-sampling
pattern
Our contribution
Beam
focusing
Envelope
detection
Log
compression
B-mode image
IQ data
(signal before envelope detection)
Convolutional neural network (CNN)
Universal Deep Beamformer (DeepBF)
probe
Rx
Sub-sampling
pattern 2x
Sub-sampling
pattern 4x
Universal Deep beamformer for Medical Ultrasound Imaging 10

11. III. Switchable and Tunable Beamformer
Learning multiple tasks using single model
Khan et al, IEEE TMI, (in review)

12. Motivation
Unsupervised method for multi domain image generation in medical ultrasound 12
MVBF
RF
…………
GCF
RF
Deconvolution
RF
Despeckle
RF
DeepBF
Separate models for each task
RF
MVBF
GCF
Deconvolution
……
Despeckle
Switchable beamformer
target-code
One model for multiple tasks

13. AdaIN simply scale the normalized content input wit
h 𝜎𝜎(𝑦𝑦), and shift it with µ(𝑦𝑦).
content input
𝑥𝑥
style input
𝑦𝑦
output
AdaIN code generator network
ℱ
Style-codes
𝑐𝑐 ∈ {−1, −0.5, 0.5, 1}
Target-style statistics
𝜎𝜎, µ ≔ wc ≔ ℱ(𝑐𝑐)
𝜎𝜎
µ
𝑐𝑐
Switchable and Tunable Beamformer 13
Theory
How to get style mean and variance vectors ?
Huang, Xun et al, ICCV, 2017 Khan et al, IEEE ISBI, 2021
How to get style mean and variance vectors ?

14. Switchable and Tunable Beamformer 14
Switchable Compressive Beamformer
Multiple targets act as regularizers
to improve the reconstruction performance
Input is 2.5x down-sampled RF data, and target is the fully-sampled image generated using specified beamforming methods

15. Switchable and Tunable Beamformer 15
Despeckle beamformer
- 0 dB
- 50 dB
- 12 dB
- 24 dB
- 36 dB
0 dB
-60 dB
-15 dB
-30 dB
-45 dB
Label
Proposed
Separate model
DESPECKLE
MVBF
LABEL
Proposed
Separate model
DESPECKLE
MVBF
45
36
27
18
9
0
38.4
0 Lateral length(mm)
20
10 30
75
60
45
30
15
0
140
0
Lateral length(mm) 70
35 105
Depth(mm)
Depth(mm)
RF-domain processing for beamformer
independent post-filtration
Zhu, Lei, et al, IEEE CVPR 2017

16. Switchable and Tunable Beamformer 16
Tunable beamformer Phased-array |Target-Output|
1
0
0.5
MVBF
DESPECKLE
Linear-array

17. Switchable and Tunable Deep
Beamformer
Data Processing Pipeline, Network Architecture,
Experimental Results

18. Switchable and Tunable Beamformer 18
Proposed beamforming pipelines
Switchable compressive beamformer
Tunable beamformer
full-image
at
once
One
depth-plane
at
a
time

19. Switchable and Tunable Beamformer 19
Here 𝒁𝒁𝑛𝑛
(𝑡𝑡)
is input cube and 𝒐𝒐𝑛𝑛
𝑡𝑡
(𝑐𝑐) is the target image for a given style 𝑐𝑐 at the depth 𝑛𝑛.
min
𝒢𝒢,ℱ
�
𝑐𝑐,𝑡𝑡,𝑛𝑛
𝒐𝒐𝑛𝑛
𝑡𝑡
𝑐𝑐 − 𝒢𝒢 𝒁𝒁𝑛𝑛
𝑡𝑡
; ℱ(𝑐𝑐)
2
2
𝑙𝑙2 norm loss function
Network Architecture
Switchable compressive beamformer Tunable beamformer
full-image at once
One depth-plane at a time

20. Switchable and Tunable Beamformer 20
Performance Stats of Switchable BF
For DAS, MVBF, DMAS and GCF style conversion, the statistical significance (p-values) of our results
are 0:0064, 0:0151, 0:0001, and 0:0352, respectively.

21. Switchable and Tunable Beamformer 21
Performance Stats of Switchable BF

22. Switchable and Tunable Beamformer 22
Performance Stats of Tunable BF
For MVBF, and DESPECKLE styles, the statistical significance (p-values) are 0:0317, and 0:0442,
respectively

23. END