Presentation document for "Diffusion adversarial representation learning for self-supervised vessel segmentation " presented in ICLR 2023.

1. Diffusion adversarial representation learning
for self-supervised vessel segmentation
ICLR 2023
Boah Kim*, Yujin Oh*, and Jong Chul Ye

2. Research objectives
2
Vessel segmentation
• To analyze vessel structures and therapy planning
X-ray coronary angiography
• To enhance vessel visualization by injecting a contrast
agent into the blood vessels
• To have many tiny branches, confusing structures, etc
→ Hard to extract vessels
Background Angiography
• To segment vascular structures without ground-truth labels
• To adopt the diffusion model in learning semantic information
Research goal

3. Related works: Diffusion model for segmentation
3
[1] Tomer Amit et al., arXiv 2022. [2] Dmitry Baranchuk, et al., arXiv 2022.
Develop a fast image segmentation method leveraging DDPM without labeled data
Amit et al.
• To train the diffusion model using (image, label) pairs.
• Estimating segmentation maps through the stochastic
generative process → Requiring a long time
Dmitry Baranchuk et al.
• Describing the diffusion model in terms of image
representations
• Learning semantic image segmentation with labeled data

4. Theory: Diffusion adversarial representation learning
4
Training framework
• Using unpaired angiography images 𝒙𝑎
& background images 𝒙𝑏
• Diffusion module 𝝐𝜃: To estimate latent features
• Generation module 𝑮: To estimate both vessel masks & synthetic angiograms
• Discriminators (𝑫𝑠 𝑫𝑎): To distinguish real and fake images

5. Theory: Diffusion adversarial representation learning
5
• Utilizing SPADE (spatially adaptive normalization) [1] to perform
semantic image synthesis
• Switchable SPADE (S-SPADE):
[1] Taesung, Park et al., CVPR 2019.
Generation module with switchable SPADE layers
𝒗 =
SPADE(𝒗, 𝒔), if mask 𝒔 is given,
IN(𝒗), otherwise
Generation module, 𝑮
Conv
I
ReLU
I
Conv
ReLU
S
S
×N
Switchable SPADE
ResnetBlock
Ma et al. ICCV 2021
(B) path:
Background fractal mask Synthetic
angiography
Input Output
(A) path:
Angiography Vessel map
Input Output

6. Theory: Diffusion adversarial representation learning
6
[1] Xudong Mao et al., ICCV 2017.
Loss function
• Training the diffusion model with adversarial learning
• Optimization problem employing LSGAN[1] framework:

7. Theory: Diffusion adversarial representation learning
7
Loss function
• Diffusion loss 𝓛𝑑𝑖𝑓𝑓: To intensively learn the
distribution of background image signal
• Adversarial loss 𝓛𝑎𝑑𝑣: To generate realistic images in
the both (A) and (B) paths
• Cyclic reconstruction loss 𝓛𝑐𝑦𝑐: For the generator to
capture the vessel semantic information

8. Theory: Diffusion adversarial representation learning
8
[1] Xudong Mao et al., ICCV 2017.
Inference of vessel segmentation
• Can obtain the vessel segmentation masks of angiography from the (A) path through one step
• The closer 𝑡𝑎 is to zero, the better the vessel segmentation performance
Noisy level of (A) path

9. Dataset and experiment setting
9
X-ray coronary angiography disease (XCAD) dataset
• Images obtained during stent placement surgery
• Train: 1621 images / Test: 114 images
- Subsampled to 256x256
- Normalized to [-1, 1]
Programming language Python, PyTorch
GPU Nvidia Quadro RTX 6000
Augmentation Horizontal/vertical flipping, rotation with 90
Batch size 1
Baseline network DDPM1(diffusion module) + SPADE2 (generation module)
Training epochs 150
Learning rate 5 × 10−6
Optimization Adam algorithm
Baseline methods SGC3, Redrawing4, DS5, STEGO6, DA7, SSVS8
[1] Jonathan Ho et al., NeurIPS 2020. [2] Taesung, Park et al., CVPR 2019. [3] Ahn et al., 2021. [4] Chen et al., 2019.
[5] Melas-Kyriazi et al., 2022. [6] Hamilton et al., 2022. [7] Mahmood et al., 2019. [8] Ma et al., 2021.

10. 10
Experimental results
Comparison of ours to baselines
• To demonstrate the vessel segmentation performance of our method
Data Metric SGC Redrawing DS STEGO DA SSVS Ours
XCAD
IoU 0.060 0.059 0.366 0.146 0.375 0.410 0.471
Dice 0.111 0.109 0.526 0.249 0.542 0.575 0.636
Precision 0.062 0.139 0.469 0.152 0.557 0.590 0.701

11. 11
Experimental results
Generalization capability
• To verify that our model can be generally used for various images from other machines or other organs
• Only test without training a new model

12. 12
Experimental results
Robustness to noises
• To evaluate the usefulness of our model on noisy angiography images (Simulation with Gaussian noise)

13. 13
Experimental results
Ablation study
• To study on the effect of each component of our model
Module Loss function Metric
Methods Diffusion Generation 𝓛𝑑𝑖𝑓𝑓 𝓛𝑎𝑑𝑣 𝓛𝑐𝑦𝑐 IoU Dice Precision
Ours √ √ √ √ √ 0.471 0.636 0.701
(a) √ √ √ 0.449 0.616 0.646
(b) w/o S-SPADE √ √ 0.439 0.606 0.620
(c) √ √ √ √ 0.322 0.485 0.580
(d) √ √ √ √ L1→CE 0.346 0.508 0.672
(a-b): Ablation models of our diffusion/generation modules
(c-d): Ablation models of the proposed loss function

14. 14
Experimental results
Diffusion module in latent feature estimation
• To show the effectiveness of diffusion module in latent feature estimation
Latent feature estimation IoU Dice Precision
Autoencoder 0.399 0.566 0.621
Diffusion (Ours) 0.471 0.636 0.701

15. Summary
15
• Proposed method learns vessel representation without labels via adversarial learning.
• The generation with switchable SPADE layers allows the model to generate synthetic angiograms as well as
vessel segmentation masks. → Learning semantic information effectively
• Experimental results using various vessel datasets verified high performance of the proposed method.
• Robustness to image diversity and noise
Propose a novel diffusion-based self-supervised vessel segmentation model by employing DDPM