Review : Segmenting Medical MRI via Recurrent Decoding Cell - by Seunghyun Hwang (Yonsei University, Severance Hospital, Center for Clinical Data Science)

1. Segmenting Medical MRI via Recurrent Decoding Cell
East China Normal University, China
Hwang seung hyun
Yonsei University Severance Hospital CCIDS
University of Notre Dame | AAAI 2020
2020.09.13

2. Introduction Related Work Methods and
Experiments
01 02 03
Conclusion
04
Yonsei Unversity Severance Hospital CCIDS
Contents

3. CRDN
Introduction – Background
• Encoder-decoder networks are commonly
used in medical image segmentation
• Three main challenges for medical image
segmentation
(1) Importance of hierarchical feature fusion
→ semantic information form deep layers +
spatial information from shallow layers
(2) Use of multi-modality information (T1, T2, PD, ..)
(3) Robustness of networks
→ Deficient data leads to overfitting
Introduction / Related Work / Methods and Experiments / Conclusion
01
[SegNet]
[U-Net]
[FCN]

4. CRDN
Introduction – Background
Introduction / Related Work / Methods and Experiments / Conclusion
02
• Many decoders only use concatenation or element-wise summation for the fusion of
feature information across layers
→ Neglect the long-term memory of the former layers
→ The operation for hierarchical feature fusion are not cable enough in memory to
carry all information from the early fusion stage

5. CRDN
Introduction – Proposal
• Propose Recurrent Decoding Cell (RDC) for better hierarchical feature fusion with its
ability to memorize long-term context information through decoding pathway.
• RDC combines the current score map of low resolution with the high resolution feature
map.
• Propose Convolutional Recurrent Decoding Network(CRDN) with RDC-based decoder
Introduction / Related Work / Methods and Experiments / Conclusion
03
[Overview of proposed framework]

6. CRDN
Introduction – Contribution
• Proposed a new feature fusion unit called Recurrent Decoding Cell (RDC) which
leverages the ability of convolutional RNN in memorizing long-term context
information.
• Each RDC unit shares parameter; RDC can be added into any encoder-decoder
segmentation network to help reduce model size
• Proposed Convolutional Recurrent Decoding Network(CRDN) increased segmentation
accuracy and showed robustness in image noise and intensity non-uniformity.
Introduction / Related Work / Methods and Experiments / Conclusion
04

7. Related Work
Introduction / Related Work / Methods and Experiments / Conclusion
05
Convolutional Recurrent Neural Networks
[1] Shi, X.; Chen, Z.; Wang, H.; Yeung, D.-Y.;Wong, W.-K.;and Woo, W.-C. 2015. Convolutional lstm network: A machine learning approach for precipitationnowcasting.In
Advances in neural information processing systems, 802–810.
• Recurrent neural networks, like LSTM and GRU have advantages in memorizing
long-term context information.
• Convolutional version of RNN extends this ability to 2D image sequence.
precipitation nowcasting [1]
• Convolutional RNN has not been applied to feature fusion in medical image
segmentation yet.
[RNN] [LSTM] [GRU]

8. Methods and Experiments
Proposed Framework - CRDN
Introduction / Related Work / Methodsand Experiments / Conclusion
06
• End-to-end pipeline that receives multi-modality image as input.
• CNN backbone encoder + RDC-based decoder
• {F1,…FL} are further squeezed into C channels(number of segmentation classes) through 5x5
convolution filter
• Decoder consists of L-stage RDC / Current score map is twice the size as the previous one

9. Methods and Experiments
Recurrent Decoding Cell
Introduction / Related Work / Methodsand Experiments / Conclusion
07
• RDC is a feature fusion unit that can
memorize the long-term context
information to refine the current score
map.
• Since the number of channels of score
maps from each stage remain the same,
RDC can share its parameters.
• In each RDC unit, previous score map Si-1 (hidden state of an RNN cell) is refined with the current
input Xi, generating the current new score map Si as the input of the next RDC
• Score map Si-1 is upsampled to the same spatial dimension as Xi
• There are three types of RDC
ConvRNN ConvLSTM ConvGRU

10. Methods and Experiments
Recurrent Decoding Cell
Introduction / Related Work / Methodsand Experiments / Conclusion
08[RNN]
[LSTM]
[GRU]

11. Methods and Experiments
Experiments - Dataset
Introduction / Related Work / Methodsand Experiments / Conclusion
09
• Two brain datasets and one cardiovascular MRI dataset
- BrainWeb datset (T1, T2, PD)
- MICCAI 2013 MR BrainS Challenge dataset (T1, T2, FLAIR)
- HVSMR 2016 Challenge dataset
(segment blood pool and myocardium)
• Concatenate multiple modalities of MR slices as input
• Comparison Models
- FCN, SegNet, U-Net
• Test on different encoding backbones

12. Methods and Experiments
Experiments
Introduction / Related Work / Methodsand Experiments / Conclusion
10

13. Methods and Experiments
Experiments
Introduction / Related Work / Methodsand Experiments / Conclusion
11

14. Methods and Experiments
Experiments
Introduction / Related Work / Methodsand Experiments / Conclusion
12

15. Methods and Experiments
Experiments
Introduction / Related Work / Methodsand Experiments / Conclusion
13

16. Conclusion
Introduction / Related Work / Methods and Experiments / Conclusion
• Proposed Recurrent Decoding Cell (RDC) for hierarchical feature fusion
in encoder-decoder segmentation networks
• Proposed Convolutional Recurrent Decoding Network (CRDN) based on
RDC for multi-modality medical image segmentation
• RDC helps to achieve better boundary adherence and reduces model
size
• CRDN shows robustness to image noise and intensity non-uniformity in
MRI
14