Review : Deep Generative model-based quality control for cardiac MRI segmentation - by Seunghyun Hwang (Yonsei University, Severance Hospital, Center for Clinical Data Science)

1. Deep Generative Model-based Quality Control
for Cardiac MRI Segmentation
Hwang seung hyun
Yonsei University Severance Hospital CCIDS
BioMedIA, Imperial College London, UK| MICCAI 2020
2020.09.27

2. Introduction Related Work Methods and
Experiments
01 02 03
Conclusion
04
Yonsei Unversity Severance Hospital CCIDS
Contents

3. Quality Control
Introduction – Background
• When a trained segmentation model is
deployed into the real clinical world, the
model may not perform optimally.
- Degraded Image Quality
- Domain Shift Issues
• Need to develop an Automated quality control
method that can detect poor segmentations
and feedback to clinicians.
• Reliable quality control (QC) of cardiac MRI
segmentation is highly desired.
Introduction / Related Work / Methods and Experiments / Conclusion
01

4. Quality Control
Introduction – Proposal
• Novel deep generative mode-based framework for quality control of cardiac MRI
segmentation
• First learns a manifold of good-quality image-segmentation pairs using a generative
model.
• The quality of a given test segmentation is then assessed by evaluating the difference
from its projection onto the good-quality manifold.
Introduction / Related Work / Methods and Experiments / Conclusion
02
[Overview of proposed framework]

5. Quality Control
Introduction – Contribution
• Propose a generic deep generative model-based framework which learns the manifold
of good-quality segmentations for quality control on a per-case basis.
• Implement the framework with a VAE and propose an iterative search strategy in the
latent space.
• Compare the performance of proposed method with regression-based methods on
two different datasets.
Introduction / Related Work / Methods and Experiments / Conclusion
03

6. Related Work
Introduction / Related Work / Methods and Experiments / Conclusion
04
Learning-based quality control
[1] Kohlberger, T., Singh, V., Alvino, C., Bahlmann, C., Grady, L.: Evaluating segmentation error without ground truth. In: International Conference on Medical Image Computing
and Computer-Assisted Intervention, Springer (2012) 528–536
[2] Robinson, R., Oktay, O., Bai, W., Valindria, V.V., Sanghvi, M.M., Aung, N., Paiva, J.M., Zemrak, F., Fung, K., Lukaschuk, E., et al.: Real-time prediction of segmentation quality. In:
International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer (2018) 578–585
[3] Liu, F., Xia, Y., Yang, D., Yuille, A.L., Xu, D.: An alarm system for segmentation algorithm based on shape model. In: Proceedings of the IEEE International Conference on
Computer Vision. (2019) 10652–10661
• [1] Proposed 42 hand-crafted features
based on intensity and appearance and
achieved an accuracy of 85% in
detecting segmentation failure.
• [2] developed a CNN-based method for
real-time regression of the Dice
similarity metric from image-
segmentation pairs.
• [3] used a variational auto-encoder
(VAE) for learning the shape features of
segmentation in an unsupervised
manner.

7. Related Work
Introduction / Related Work / Methods and Experiments / Conclusion
05
Registration-based quality control
[4] Valindria, V.V., Lavdas, I., Bai, W., Kamnitsas, K., Aboagye, E.O., Rockall, A.G., Rueckert, D., Glocker, B.: Reverse classification accuracy: predicting segmentation performance
in the absence of ground truth. IEEE transactions on medical imaging 36(8) (2017) 1597–1606
• [4] proposed the concept of reverse
classification accuracy (RCA) to predict
segmentation quality and achieved
good performance on a large-scale
cardiac MRI dataset.

8. Methods and Experiments
Proposed Framework
Introduction / Related Work / Methods and Experiments / Conclusion
06
• The proposed framework aims to find a good-quality segmentation s as a surrogate for GT.
• The generative model G is trained to learn a mapping from the low-dimensional latent space
to the good-quality manifold
• The input image-segmentation pair I is projected to on the manifold through
iterative search.
• is the initial guess in the latent space and it converges to

9. Methods and Experiments
Iterative Search in the latent space
Introduction / Related Work / Methods and Experiments / Conclusion
07
• Develop an iterative search scheme in the latent space to find a surrogate segmentation for a
given image-segmentation pair as input.
• Find a closest surrogate segmentation on the good-quality manifold as an optimization problem.

10. Methods and Experiments
Generative Model using VAE
Introduction / Related Work / Methods and Experiments / Conclusion
08
• Employ the VAE. Image-segmentation pair (I,S) is encoded by E to follow a Gaussian
distribution in the latent space.
• At the training stage, ground-truth image-segmentation pairs are used to train VAE.
• In the application stage, the VAE decoder is used as the generator for iterative
search of the surrogate segmentation on the good-quality manifold.
• Initial guess: from the encoder / Final Guess: , is calculated.

11. Methods and Experiments
Generative Model using VAE
Introduction / Related Work / Methods and Experiments / Conclusion
09

12. Methods and Experiments
Experiments - Dataset
Introduction / Related Work / Methods and Experiments / Conclusion
10
• UK Biobank dataset
- Short-axis cardiac images at the end-diastolic (ED) frame
of 1,500 subjects were obtained.
• ACDC dataset
- 100 subjects including a normal group and four pathology
groups were obtained.
• Comparison Methods:
- Support vector regression(SVR) with 42 hand-crafted features
about shape and appearance.
- CNN regression network (ResNet-18 back-bone) with the
image-segmentation pair as input.

13. Methods and Experiments
Experiments Settings
Introduction / Related Work / Methods and Experiments / Conclusion
11
• Experiment 1: UK Biobank
- Besides the GT segmentations, generated poor-quality segmentations by
attacking the segmentation model (White noise with different variance levels).
Quality prediction was performed on the test set of the attacked segmentations
• Experiment 2: ACDC
- Deployed a UK Biobank trained segmentation model on ACDC dataset without
fine-tuning. This reflects a real-world clinical setting, where segmentation failures
would occur due to domain shift issues.
* Focused on myocardium segmentation which is a challenging cardiac structure to
segment and of high clinical relevance.

14. Methods and Experiments
Experiments
Introduction / Related Work / Methods and Experiments / Conclusion
12

15. Methods and Experiments
Experiments
Introduction / Related Work / Methods and Experiments / Conclusion
13

16. Methods and Experiments
Experiments
Introduction / Related Work / Methods and Experiments / Conclusion
14

17. Conclusion
Introduction / Related Work / Methods and Experiments / Conclusion
• Regression based QC methods are easily overfitted. The ACDC dataset
consists of more pathological cases, whereas the UK Biobank comes
from a general healthy population.
• Proposed method maintained a high prediction accuracy against
domain shift → Advantage of a generative model-based framework
• Proposed method does not depend on specific segmentation models or
types of segmentation failures.
• Potential to be extended for quality control in different anatomical
structures.
15