The acquisition conditions for low-dose and high-dose CT images are usually different, so that the shifts in the CT numbers of-ten occur. Accordingly, unsupervised deep learning-based approaches, which learn the target image distribution, often introduce CT number distortions and result in detrimental effects in diagnostic performance. To address this, here we propose a novel unsupervised learning approach for lowdose CT reconstruction using patch-wise deep metric learning. The key idea is to learn embedding space by pulling the positive pairs of image patches which shares the same anatomical structure, and pushing the negative pairs which have same noise level each other. Thereby, the network is trained to suppress the noise level, while retaining the original global CT number distributions even after the image translation. Experimental results confirm that our deep metric learning plays a critical role in producing high quality denoised images without CT number shift.

1. Patch-wise Deep Metric Learning
for Unsupervised Low-Dose CT Denoising
Chanyong Jung1, Joonhyung Lee1, Sunkyoung You2 and Jong Chul Ye1
1 Korea Advanced Institute of Science and Technology (KAIST), Republic of Korea
2 Chungnam National University College of Medicine and Chungnam National University Hospital, Republic of Korea
MICCAI 2022

2. Introduction
 Temporal CT scan data
 HU value contains useful information for pathologies of ear
 HU value shift problem between Low-/High-dose data due to
different acquisition condition
 Unrealistic to obtain paired Low-/High-dose data
We present the method:
 Unsupervised denoising method by unpaired CT data
 Prevent CT number shift during the denoising process
X-ray tube
(Low Dose)
X-ray tube
(High Dose)
Different
Acquisition
Condition
 Low Dose
 High Dose
Different average HU value for same media
 HU value distortion during denoising
(Example: HU value for cholesteatoma diagnosis[1])
Hounsfield Unit
Cholesteatoma 24.68 ± 24.42
Non-cholesteatoma 86.07 ± 26.50
[1] Park M.H. et al, Usefulness of computed tomography hounsefield unit density in preoperative detection
of cholesteatoma in mastoid ad antrum, American Journal of Otolaryngology, 2011
 HU value supports the evaluation for pathologies of ear

3. Background
Deep metric learning & Contrastive learning Contrastive learning for Image Translation
 Deep metric learning obtains embedding space
by the learnable metric 𝒅
 Contrastive loss maximizes Mutual Information (MI)
Embedding space with metric 𝒅
: Positive Pair
: Negative Pair
: Decrease 𝑑 (Pull)
: Increase 𝑑 (Push)
 Contrastive learning also pull/push pairs using the similarity.
For positive pair x, y+
~ 𝑃𝑋𝑌
negative pair x, 𝑦′ ~ 𝑃𝑋𝑃𝑌
𝐼 𝑋; 𝑌 = 𝐾𝐿(𝑃𝑋𝑌| 𝑃𝑋𝑃𝑌
≥ max
f
𝔼 f x, y+
−
1
𝐾
log ∑ exp(𝑓(𝑥, 𝑦′))
Using Donsker-Varadhan (DV) lower bound:
 InfoNCE: Contrastive Loss
 MI maximization is not appropriate for denoising,
since the preservation of the noise also increases MI.
 We present the pull/push mechanism for image denoising
𝐺
𝑃𝑌 𝑃𝑋
Input Y Output X
: Positive Pair
: Negative Pair
𝑦+
𝑦1
′
𝑦2
′
𝑥
Pull/Push by
similarity manipulation
 Related work [2] presents patch-wise contrastive learning
 Maximizes MI between input and output patches
[2] Park et al., Contrastive learning for unpaired image-to-image translation, ECCV (2020)

4. Method
 Deep metric learning for denoising
 Proposed framework Discriminator
Noisy Image Wavelet
Decomposition
(5 Level)
High frequency
Image
RRDBnet
Block
Conv
Conv
Conv
RRDBnet
Block
Conv
Conv
Conv
2 hidden layer features for
patch-wise metric learning
Generator Generator (Shared)
𝑧1:𝐾 𝑤1:𝐾
𝑷𝟏 𝑷𝟐 𝑷𝟏 𝑷𝟐
Deep metric learning
Embedding space learned by deep metric learning
: Input patch feature
: Output patch feature
: Decrease metric 𝑑
: Increase metric 𝑑
𝑥𝑖
𝑥𝑗
𝑥𝑙
𝑥𝑘 𝑦𝑘
𝑦𝑖
𝑦𝑗
𝑦𝑙 𝑑(𝑥𝑘, 𝑦𝑘) =
𝑧𝑘−𝑤𝑘 2
2
2𝜏
Metric:
 We push patches with same noise level,
hence the noise features are suppressed.
 We pull patches with same anatomic structure,
hence structure features are enhanced.

5. Method
 Deep metric loss
𝐿𝑚 = −𝔼𝑝 exp(𝑧𝑘
⊤
𝑤𝑘/𝜏 ) +𝔼𝑞𝑛
exp(𝑧𝑘
⊤
𝑧𝑗/𝜏 ) +𝔼𝑞𝑜
exp(𝑤𝑘
⊤
𝑤𝑗/𝜏 )
 Total loss
GAN Loss
Identity Loss
Metric Loss
𝐿 = 𝐿𝐺𝐴𝑁 + 𝜆𝑖𝑑𝑡𝐿𝑖𝑑𝑡 + 𝜆𝑚𝐿𝑚
𝐿𝐺𝐴𝑁(𝐺) = 𝔼[ 𝐷 𝐺(𝑥 − 1 2
2
]
𝐿𝐺𝐴𝑁 𝐷 = 𝔼 𝐷 𝑦 − 1 2
2
+ 𝔼 𝐷 𝐺(𝑥) 2
2
𝐿𝑖𝑑𝑡 = 𝔼 𝐺 𝑦 − 𝑦 1
𝐿𝑚 𝐸, 𝐶1, 𝑃𝑖 = 𝐿𝑚,1 𝐸, 𝑃1 + 𝐿𝑚,2(𝐸, 𝐶1, 𝑃2)
Conv
ReLU
SE
Layer
× 𝟑
Real
/ Fake
Prediction
Conv
Real
/ Fake
Image
RRDBnet
Block
𝑷𝟏
𝑬 𝑪𝟏 𝑪𝟐
Noisy
Input (𝑥)
Denoised
Output
(𝑦)
Conv
Conv
Conv
𝑷𝟐
𝑧 / 𝑤
[ Generator 𝑮 ] [ Discriminator 𝑫 ]
𝑦 = 𝑮 𝑥 = (𝑪𝟐 ∘ 𝑪𝟏 ∘ 𝑬)(𝑥)
𝑧 / 𝑤
Embedding space learned by deep metric learning
: Input patch feature
: Output patch feature
: Decrease metric 𝑑
: Increase metric 𝑑
𝑥𝑖
𝑥𝑗
𝑥𝑙
𝑥𝑘 𝑦𝑘
𝑦𝑖
𝑦𝑗
𝑦𝑙 𝑑(𝑥𝑘, 𝑦𝑘) =
𝑧𝑘−𝑤𝑘 2
2
2𝜏
Metric:
𝐿𝑚 = 𝔼𝑝 exp{𝑑 𝑥𝑘, 𝑦𝑘 } − 𝔼𝑞𝑛
exp{𝑑 𝑥𝑘, 𝑥𝑗 } − 𝔼𝑞𝑜
exp{𝑑 𝑦𝑘, 𝑦𝑗 }
Since 𝑧, 𝑤 are L2 normalized,
𝑧𝑘 − 𝑤𝑘 2
2
= 2 − 2𝑧𝑘
⊤
𝑤𝑘
 Pull patches with same anatomic structure: (𝑥𝑘, 𝑦𝑘)
 Push patches with same noise level: (𝑥𝑘, 𝑥𝑖), (𝑦𝑘, 𝑦𝑖)
Then, the metric loss is defined as :

6. Supervised
Results
 AAPM dataset
wavCycleGAN cycleGAN
GAN-Circle
Invertible
cycleGAN
InfoNCE
Proposed
LDCT HDCT
50
0
-50
25
-25
Input
Input
(zoomed)
Difference

7. wavCycleGAN cycleGAN
GAN-Circle
Invertible
cycleGAN
Proposed
Low Dose
200
0
-200
50
-50
 Temporal CT scans
Results
200
0
-200
50
-50
Input
Input
(zoomed)
Difference
Difference
(zoomed)
 CT number statistics for selected temporal region
(Colored circles of the Figure)
 CT number statistics for 300 patches

8. Thank You