This document summarizes two papers on semantic segmentation: DenseASPP for semantic segmentation of street scenes using DenseNet and ASPP, and Cycle-Shape-GAN for translating and segmenting multimodal medical volumes using a Cycle-GAN with added shape consistency losses. It provides code links and descriptions of the key techniques used in each paper, including DenseASPP, ASPP, Cycle-GAN, and the added shape losses in Cycle-Shape-GAN. Results are shown applying these methods to semantic segmentation of CityScapes street scenes and multimodal medical image segmentation.

1. 46
@
2018/7/1

2. ◾ ea466 :
◾
// - - /
◾ 4 b
◾
f
. /

3. semantic segmentation
semantic segmentation
l

4. DenseASPP
Cycle-Shape-GAN
M. Yang, et. al. ” DenseASPP for Semantic Segmentation in Street
Scenes
Z. Zhang, et. al. ”Translating and Segmenting Multimodal Medical
Volumes with Cycle- and Shape-Consistency Generative
Adversarial Network”

5. DenseASPP
M. Yang, et. al. ” DenseASPP for Semantic Segmentation in
Street Scenes”
CVF URL
http://openaccess.thecvf.com/content_cvpr_2018/papers/Yang
_DenseASPP_for_Semantic_CVPR_2018_paper.pdf
https://github.com/masataka46/DenseASPP-chainer

6. DenseASPP
DenseASPP
DenseASPP DenseNet ASPP

7. DenseNet
DenseASPP DenseNet ASPP
G. Huang, et. al. “Densely Connected Convolutional Networks”
CVPR2017
l

8. ASPP
DenseASPP DenseNet ASPP
F. Yu, et. al. “Multi-scale context aggregation by dilated
convolutions ” ICLR2016
l Atrous convolution dilated
convolution
l
http://deeplearning.net/software/the
ano/tutorial/conv_arithmetic.html

9. ASPP
DenseASPP DenseNet ASPP
l dilation rate
L. Chen, et. al. “Rethinking Atrous
Convolution for Semantic Image Segmentation”

10. ASPP
DenseASPP DenseNet ASPP
L. Chen, et. al. “Rethinking Atrous Convolution for Semantic
Image Segmentation”
l Atrous
convolution

11. ASPP
DenseASPP DenseNet ASPP

12. ASPP
DenseASPP DenseNet ASPP
M. Yang, et. al. ” DenseASPP for Semantic Segmentation in
Street Scenes” CVPR2018
l Atrous
convolution Densely

13. ASPP
DenseASPP DenseNet ASPP
M. Yang, et. al. ” DenseASPP for Semantic Segmentation in
Street Scenes” CVPR2018
l Atrous convolution
Densely

14. DenseASPP
atrous-conv
dilation rate 3 dilation rate6
3×2 + 6×2 + 1 = 19 (pixel)

15. DenseASPP
atrous conv
dilation rate 3, 6, 12, 18, 19 atrous conv
3×2 + 6×2 + 12×2 + 18×2 + 24×2 + 1 = 127 (pixel)

16. DenseASPP
densely scale

17. DenseASPP
l
l segmentation

18. CityScape datasets state-of-the-art
s
→

19. Chainer
https://github.com/masataka46/DenseASPP-chainer
Atrous convolution Dilated convolution L.DilatedConvolution2D
DilatedConv concat

20. CityScape dataset
Dilated Conv conv

21. DenseASPP
1 epoch 50 epoch
50 epoch
ground
truth
ground
truth

22. DenseASPP
l DenseASPP DenseNet ASPP
l

23. Cycle-Shape-GAN
Z. Zhang, et. al. ”Translating and Segmenting Multimodal
Medical Volumes with Cycle- and Shape-Consistency
Generative Adversarial Network”
arXiv URL
https://arxiv.org/abs/1802.09655
Chainer
https://github.com/masataka46/Cycle-Shape-GAN-chainer

24. Cycle-Shape-GAN
Cycle-Shape-GAN
Cycle-Shape-GAN Cycle-GAN Shape

25. Cycle-GAN
GAN
J. Y. Zhu, et. al. Unpaired Image-to-Image Translation using Cycle-
Consistent Adversarial Networks
• Generator X
• Generator Y

26. Cycle-GAN
→

27. Cycle-GAN
Generator Y Generator X
→

28. Cycle-GAN
MRI CT
→
CT
MRI

29. Cycle-Shape-GAN
Cycle-Shape-GAN Segmentor
Segmentor segment
→
Segmentor
X
Segmentor
Y
CTMRI

30. Chainer
https://github.com/masataka46/Cycle-Shape-GAN-chainer
Cycle-GAN Cycle-Shape-GAN
Generator X
Generator Y
Discriminator X
Discriminator Y
Generator X
Generator Y
Discriminator X
Discriminator Y
Segmentor X
Segmentor Y
Segmentor
Class
Class
Class
Class
Class
Class
Class
Class
Class
Class

31. Chainer
Cycle-GAN Cycle-Shape-GAN
Segmentor loss
!"#"$% = !$'( + !*+*%, + !-,.!"#"$% = !$'( + !*+*%,
https://github.com/masataka46/Cycle-Shape-GAN-chainer

32. Chainer
Cycle-GAN Cycle-Shape-GAN
Segmentor
https://github.com/masataka46/Cycle-Shape-GAN-chainer

33. Cycle-GAN
CityScape annotation
300
300
U-Net ResNet
Cycle-GAN

34. Cycle-GAN
Cycle-GAN Cycle-Shape-GAN10 epoch 10 epoch
→
→ → →
→ →
Cycle-GAN Cycle-Shape-GAN
→
→ → →
→ →

35. Cycle-Shape-GAN segmentation
Cycle-Shape-GAN Cycle-Shape-GAN1 epoch 60 epoch
(1 epoch) 60 epoch segment
ground
truth
ground
truth
ground
truth
ground
truth

36. Cycle-Shape-GAN
l Cycle-Shape-GAN Cycle-GAN segmentor
l