This document summarizes a method for semi-supervised 3D left atrium segmentation using hierarchical consistency regularized mean teacher. The proposed method uses a multi-scale deeply supervised network with hierarchical supervision and adds hierarchical consistency losses between student and teacher networks at multiple scales. On a dataset of 80 3D MRIs with labels for 16, the method achieves state-of-the-art performance compared to other semi-supervised segmentation methods, obtaining results comparable to full supervision using only 1/5 labeled data. Hierarchical supervision and consistency losses both contribute to improved performance over baseline methods.

1. Hierarchical Consistency Regularized Mean Teacher for
Semi-supervised 3D Left Atrium Segmentation
Shumeng Li*1,2, Ziyuan Zhao*1,2 , Kaixin Xu2, Zeng Zeng2, Cuntai Guan1
Presenter: Zhao Ziyuan
1 Nanyang Technological University 2 Institute for Infocomm Research, A*STAR

2. Background
Natural Images Medical Images
Need different pre-processing methods Not many classes
To segment certain specific structures or tissues
● Image segmentation -> assign a label to every pixel

3. Problem Statement
Challenge for medical images:
● Requires manual identification by
experts
● Annotations are difficult to obtain
Semi-supervised
learning
Full-supervised
learning
Data
Data
Labeled
data
Labeled
data
Unlabeled
data
Improve the segmentation
performance with the
unlabeled data

4. Proposed Method
- Overview
16
128
64
32
16
32
64
128
256
Lc0 Lc1 Lc2 Lc3
Lunsu
p
Ls0
Ls1
Ls2
Ls3
Lsu
p
Noise 𝞷
Noise 𝞷’
Teacher
Network
concatenate
downsample conv
upsample deconv
upsample, conv 1×1, softmax
Ground
Truth
Upsample
× 2
Upsample
× 4
Upsample
× 8
Prediction
Input
16
128
64
32
16
32
64
128
256
Student
Network
Upsample
× 2
Upsample
× 4
Upsample
× 8
EMA
(a) Multi-scale deeply supervised network
(b) Hierarchical consistency regularized mean teacher

5. Proposed Method (a)
- Multi-scale deeply supervised network
Ls0
Ls1
Ls2
Ls3
Lsu
p
concatenate
downsample conv
upsample deconv
upsample, conv 1×1, softmax
Ground
Truth
Prediction
16
128
64
32
16
32
64
128
256
V-Net
Upsample
× 2
Upsample
× 4
Upsample
× 8
● Hierarchical supervised (HS) component
○ add the auxiliary layers after each block in the decoding stage of V-Net
■ an upsampling layer, a 1x1x1 convolution, and a softmax layer
○ multiple prediction results form multi-scale deep supervision
● Supervised loss
○ dice loss + cross-entropy loss

6. Lsu
p
Lunsu
p
Proposed Method (b)
- Basic mean teacher (MT)
16
128
64
32
16
32
64
128
256
Noise 𝞷
Noise 𝞷’
Teacher
Network
Ground
Truth
Prediction
Input
16
128
64
32
16
32
64
128
256
Student
Network
EMA
concatenate
downsample conv
upsample deconv
upsample, conv 1×1, softmax
● Semi-supervised learning
○ objective function: supervised loss Lsup + unsupervised consistency loss Lunsup
the teacher network weights are
updated with the exponential
moving average (EMA) of the
student network weights
Labeled &
Unlabeled
Labeled
Tarvainen, A., & Valpola, H. (2017). Mean teachers are better role
models: Weight-averaged consistency targets improve semi-
supervised deep learning results. arXiv preprint arXiv:1703.01780.

7. Proposed Method (b)
- Hierarchical consistency regularized MT
16
128
64
32
16
32
64
128
256
Lc0 Lc1 Lc2 Lc3
Lunsu
p
Noise 𝞷
Noise 𝞷’
Teacher
Network
concatenate
downsample conv
upsample deconv
upsample, conv 1×1, softmax
Upsample
× 2
Upsample
× 4
Upsample
× 8
Prediction
Input
16
128
64
32
16
32
64
128
256
Student
Network
Upsample
× 2
Upsample
× 4
Upsample
× 8
EMA
● Hierarchical unsupervised (HU) component
○ add the auxiliary layers to both student and teacher networks
○ further encourage the prediction consistent in the hidden feature space

8. Proposed Method (b)
- Hierarchical consistency regularized MT
16
128
64
32
16
32
64
128
256
Lc0 Lc1 Lc2 Lc3
Lunsu
p
Noise 𝞷
Noise 𝞷’
Teacher
Network
concatenate
downsample conv
upsample deconv
upsample, conv 1×1, softmax
Upsample
× 2
Upsample
× 4
Upsample
× 8
Prediction
Input
16
128
64
32
16
32
64
128
256
Student
Network
Upsample
× 2
Upsample
× 4
Upsample
× 8
EMA
● Consistency loss
○ mean squared error (MSE) loss of multi-scale predictions between the student and
the teacher networks

9. Proposed Method
- Overview
16
128
64
32
16
32
64
128
256
Lc0 Lc1 Lc2 Lc3
Lunsu
p
Ls0
Ls1
Ls2
Ls3
Lsu
p
Noise 𝞷
Noise 𝞷’
Teacher
Network
concatenate
downsample conv
upsample deconv
upsample, conv 1×1, softmax
Ground
Truth
Upsample
× 2
Upsample
× 4
Upsample
× 8
Prediction
Input
16
128
64
32
16
32
64
128
256
Student
Network
Upsample
× 2
Upsample
× 4
Upsample
× 8
EMA
Labeled &
Unlabeled
Labeled

10. Dataset
● Data source:
○ provided by 2018 Atrial Segmentation Challenge
● Data size:
○ training Set – 80 3D gadolinium-enhanced MRIs and left atrium cavity labels
■ 16 samples - labeled dataset
■ 64 samples - unlabeled dataset
○ test Set – 20 3D MRIs and left atrium cavity labels
● Pre-processing:
○ all the scans were cropped centering at the heart region and normalized as zero
mean and unit variance

11. Results
● Our approach achieves the best performance
○ over the state-of-the-art semi-supervised methods in dice, jaccard and 95HD
● Our approach effectively utilizes the unlabeled data
○ compared with full annotation, it obtains comparable segmentation results using only
1/5 labeled data
Table 1: Segmentation results of different approaches

12. Results
UA-MT SASSNet LG-ER-MT DTC Ours GT
● Example of 2D and 3D results of five semi-supervised segmentation methods, where GT
denotes corresponding ground truth labels

13. Results
● To evaluate the effectiveness of HS and HU
○ HS = hierarchical supervised component
○ HU = hierarchical unsupervised component
● Introduce HS to V-Net
○ better utilize the labeled data
● Introduce HU + HS to MT
○ both are effective and combining together can achieve better performance
Table 2: Effectiveness of core component in the proposed framework

14. Conclusion
● The proposed framework based on the mean teacher architecture
○ it encourages the consistency of predictions between the student and the teacher
networks with the same input under different perturbations
● In each iteration, the student network is optimized by multi-scale deep supervision and
hierarchical consistency regularization, concurrently
● Experimental results demonstrate the superiority over the other state-of-the-art SSL
methods
● The proposed method is simple and versatile
○ can be widely applied to other segmentation tasks

15. Thanks
Zhao_Ziyuan@i2r.a-star.edu.sg