Presentation file for "Mumford-Shah Loss Functional for Image Segmentation With Deep Learning" published in IEEE Transactions on Image Processing.

1. Mumford-Shah Loss Functional for Image
Segmentation with Deep Learning
IEEE Transactions on Image Processing
Boah Kim, Jong Chul Ye

2.  Fully supervised segmentation : 100% pixel-wise labels
 Semi-supervised segmentation: <100% of pixel-wise labels
 Weakly-supervised segmentation: No pixel-wise labels, rather Image-level / bounding box labels
 Unsupervised segmentation: No labels
Introduction
Semi-, Un- supervised Image Segmentation
No label
Image Segmentation

3.  Fully pixel-wise labels : time-consuming, difficult to obtain in certain domains (ex. medical images)
 How to use unlabeled images without any (image-level, bounding box) labels?
⇒ Revisiting the classical image segmentation method, “Mumford-Shah functional”
Introduction
Semi-, Un- supervised Image Segmentation
No label
Image Segmentation

4. Related works
Semi-, Un- supervised Image Segmentation
Mumford-Shah Functional
- 1st term : Distance between the model
and the input image
- 2nd term : Smoothness of the model
within the sub-regions
 Optimality criterion for segmenting an image into sub-regions
 Chan-Vese, multiphase level-set framework
- Make the characteristic function be differentiable with Heaviside function

5. Main Theory
Semi-, Un- supervised Image Segmentation
Mumford-Shah Loss Functional
 Similarity between the characteristic function and softmax layer in CNN
 Proposed “Mumford-Shah loss functional”

6. Main Theory
Semi-, Un- supervised Image Segmentation
Application of Mumford-Shah Loss
 In the presence of semantic labels,

7. Main Theory
Semi-, Un- supervised Image Segmentation
Application of Mumford-Shah Loss
 In the absence of semantic labels,

8. Main Theory
Semi-, Un- supervised Image Segmentation
Application of Mumford-Shah Loss
 In the absence of semantic labels,

9. Main Theory
Semi-, Un- supervised Image Segmentation
Application of Mumford-Shah Loss
 In the absence of semantic labels,
- For image with intensity inhomogeneities, impose a term of bias field estimation

10. Experiments
Semi-, Un- supervised Image Segmentation
 Semi-supervised object segmentation in Natural Images
- PASCAL VOC 2012 dataset
 Semi-supervised tumor segmentation in Medical Images
- LiTS 2017 dataset
- BRATS 2015 dataset
 Unsupervised segmentation in Natural images
- BSDS500 dataset
- PASCAL VOC 2012 dataset

11. Experiments
Semi-, Un- supervised Image Segmentation
Semi-supervised Object Segmentation in Natural Images
 Qualitative evaluation on PASCAL VOC 2012 dataset

12. Experiments
Semi-, Un- supervised Image Segmentation
Semi-supervised Object Segmentation in Natural Images
 Quantitative evaluation on PASCAL VOC 2012 dataset (Training with ¼ labeled data)

13. Experiments
Semi-, Un- supervised Image Segmentation
Semi-supervised Tumor Segmentation in Medical Images
 Qualitative evaluation on LiTS and BRATS datasets
1/10 labeled data on LiTS dataset 1/4 labeled data on BRATS dataset

14. Experiments
Semi-, Un- supervised Image Segmentation
Semi-supervised Tumor Segmentation in Medical Images
 Quantitative evaluation on LiTS and BRATS datasets
LiTS dataset
BRATS dataset
(1/4 labeled data)

15. Experiments
Semi-, Un- supervised Image Segmentation
Unsupervised Segmentation in Natural Images
 Evaluation on BSDS500 dataset

16. Experiments
Semi-, Un- supervised Image Segmentation
Unsupervised Segmentation in Natural Images
 Evaluation on PASCAL VOC 2012 dataset

17. Thank you.