[EMBC 2021] Multi Slice Dense Sparse Learning for Efficient Liver and Tumor Segmentation
•Download as PPTX, PDF•
0 likes•28 views
Slides presented at 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC 2021). DOI: 10.1109/EMBC46164.2021.9629698
Recommended
More Related Content
Similar to [EMBC 2021] Multi Slice Dense Sparse Learning for Efficient Liver and Tumor Segmentation
Similar to [EMBC 2021] Multi Slice Dense Sparse Learning for Efficient Liver and Tumor Segmentation (20)
More from Ziyuan Zhao
More from Ziyuan Zhao (14)
Recently uploaded
Recently uploaded (20)
[EMBC 2021] Multi Slice Dense Sparse Learning for Efficient Liver and Tumor Segmentation
- 1. Multi-Slice Dense-Sparse Learning for Efficient Liver and Tumor Segmentation Ziyuan Zhao1 , Zeyu Ma1,2, Yanjie Liu1,2, Zeng Zeng1, Pierce KH Chow3,4 Presenter: Zhao Ziyuan 1 Institute for Infocomm Research, A*STAR 2 National University of Singapore 3 National Cancer Center and Singapore General Hospital, Singapore 4 Duke-NUS Medical School Singapore
- 2. Introduction – Liver & Tumor Segmentation - Liver tumor is one of the main causes of human death. - Accurate automatic liver and tumor segmentation plays a vital role in treatment planning and disease monitoring. - Deep learning has been widely used for medical image segmentation.
- 3. Challenge – Trade off between Accuracy and Complexity - 2D DCNNs cannot fully leverage the inter-slice information in 3D volumetric CT scans - 3D DCNNs are capable to explore the inter-slice correlations and learn deep 3D representations with volumetric inputs, but they are computationally expensive and memory intensive. Prediction Accuracy Model Complexity
- 4. Method – Multi-slice Sampling + Efficient nnU-Net - Consider both data and network perspectives in our framework for more efficient liver and tumor segmentation. - We propose a novel dense-sparse training flow from a data perspective. - We design a 2.5D light-weight nnU-Net from a network perspective.
- 5. Method (1) –Dense-Sparse Sampling - 2.5D network uses a stack of T continuous slices during training and generates the segmentation mask for the central slice at the inference stage. - We propose a novel dense-sparse sampling method to generate densely adjacent slices and sparsely adjacent slices - Consider different strides of sampling for a more comprehensive view. s is the stride of sampling When s = 1, densely adjacent slices When s > 1, sparsely adjacent slices
- 6. Method (2) – Depthwise Separable nnU-Net - Select 2D nnU-Net as our backbone and design a depthwise separable nnU-Net (DS nnU-Net). - Employ depthwise separable convolutions instead of standard convolution layers to further ease the computational burden. - DS nnU-Net has only 7.7 million (M) parameters, while 2D nnU-Net has more than 40 M parameter.
- 7. Experiments - Dataset - Liver Tumor Segmentation (LiTS) dataset - 201 CT scans (131 for training and 70 for testing) - 105 volumes for training and the remaining 26 for testing in our experiments - Data preprocessing - CT volumes are resized to multiple slices of size 512 × 512 after resampling and normalization - Implementation details - Supervised loss: Dice + Cross-entropy - We implement DS sampling with thickness T = 7. - For DSD training, we set 400 and 600 epochs for DS step and D step
- 8. Results (1) – Quantitative Comparison - Compare our method with 2D nnU-Net and 3D nnUNet with full resolution - Compare with different variants - nnU-Net-DS: 2D nnU-Net with the proposed densesparse sampling. - nnU-Net-DSD: 2D nnU-Net with the proposed dense-sparse-dense training strategy. - DS nnU-Net-DSD: Depthwise Separable nnU-Net with DSD training strategy.
- 9. Results (2) – Qualitative Comparison - The masks of our method are close to the ground truth labels, which further shows the feasibility of the proposed method for efficient liver and lesion segmentation
- 10. Conclusions - In this work, we design a novel end-to-end deep learning framework from both perspectives of data and network for liver and tumor segmentation. - Extensive experiments show that the proposed approach can obtain accurate segmentation results, as well as speed up the training and inference process with only 7 M parameters. - Ablation studies demonstrate the effectiveness of different components in our framework.
Editor's Notes
- Hello everyone, welcome to my presentation. My name is Zhao Ziyuan. Here I am presenting our work titled “Multi-Slice Dense-Sparse Learning for Efficient Liver and Tumor Segmentation”
- The liver is a common site of primary or secondary tumor development. Liver cancer is life-threatening and one of the most dangerous tumors to human health Computed tomography (CT) is one of the most effective non-invasive diagnostic imaging procedures to help doctors detect and characterize liver lesions Accurate automatic liver and tumor segmentation plays a vital role in treatment planning and disease monitoring. Deep convolutional neural network (DCNNs) has obtained success in 2D and 3D medical image segmentation.
- However, 2D models ignore the inter-slice features in 3D volumetric CT scans, which limits the segmentation performance. On the other hand, replacing 2D convolutions with 3D ones, 3D networks are capable to explore the inter-slice correlations and learn deep 3D representations with volumetric inputs. But high computational complexity and cost of 3D models impede the broader clinical use.
- To address these issues, in our work, we propose an end-to-end framework for efficient liver and tumor segmentation from data and network perspectives. First, from data perspective, we propose a novel dense-sparse training workflow. Then, from network perspective, we design a 2.5 D light-weight nnU-Net.
- To probe the inter-slice information while reducing the computational complexity, many methods from different perspectives have been proposed, including 2.5D models and hybrid 2D–3D models. These methods alleviate the problems of 2D and 3D models to a certain extent. For a 2D network, only one slice is used to generate the segmentation mask, lacking the context information for volumetric medical image segmentation. In common, a 2.5D network uses a stack of T continuous slices during training and generates the segmentation mask for the central slice at the inference stage. In our dense-sparse sampling, we generate not only densely adjacent slices, but also sparsely adjacent slices. In this manner, more comprehensive view can be obtained with different strides of sampling. To facilitate dense-sparse sampling, we propose a two stage progressive learning strategy, namely Dense-Sparse-Dense training, in which, we first randomly input two types of adjacent slices to train and regularize the network for fast convergence, while in the second step, we retrain the model with densely adjacent slices to avoid overfitting.
- On the other hand, we adopt 2D nnU-Net as our segmentation architecture, which is a self-adapting framework. Differently, we employed depthwise separable convolutions instead of standard convolution layers to ease the computational burden, which help decrease over ¾ parameters. include one
- Our experiments were done on liver tumor segmentation dataset, which includes 201 CT scans, 131 for training ,and 70 for testing. Since the ground truths of testing data are not publicly available, for a fair comparison, we randomly select 105 volumes for training and the remaining 26 for testing in our experiments. CT volumes are resized to multiple slices of size 512 × 512 after resampling and normalization. Following the settings in nnU-Net, we train the network with the combination of cross-entropy loss and dice loss We implement DS sampling with thickness T = 7. For DSD training, we set 400 and 600 epochs for DS step and D step, respectively.
- We compare our method with 2D nnU-Net and 3D nnUNet with full resolution. Besides, we validate the effectiveness of different components of our pipeli The quantitative results highlight the effectiveness of the proposed method. It is well noted that the proposed dense-sparse sampling can help improve the performance of 2D nnU-Net on liver and tumor segmentation. With DS sampling and DSD training strategy together, 2D nnU-Net achieved comparable performance with 3D nnU-Net. The results have demonstrated the effectiveness of the proposed DS sampling and DSD training strategy for improving segmentation performance without carefully modified architecture
- The figure shows the visualization results of our method DS nnU-Net-DSD. We can see that the masks of our method are close to the ground truth labels, which further shows the feasibility of the proposed method for efficient liver and lesion segmentation.
- In this work, we design a novel end-to-end deep learning framework from both perspectives of data and network from both perspectives of data and network for liver and tumor segmentation. Extensive experiments show that the proposed approach can obtain accurate segmentation results, as well as speed up the training and inference process with only 7 M parameters. Ablation studies demonstrate the effectiveness of different components in our framework.
- Thanks for listening to my presentation.