Deep Learning-based Fully Automated
Detection and Quantification of Acute Infarcts
Seung Hyun Hwang , Hwi Young Kim , Ji Hoon Cha , Hyug Gi Kim ,
Kyung Mi Lee , Hyun Suk Choi
Yonsei University , Kyung Hee University
Severance Hospital
Yonsei University | RSNA 2020
November 29, 2020
1 1 1 2
2 1
21
Materials &
Methods
Introduction Results
01 02 03 04
Discussion &
Conclusion
Contents
Introduction
Ischemic Stroke
• Ischemic Stroke
- Stroke caused by a blockage in an artery
that supplies blood to the brain. MRI
sequences are dominantly used in
detecting infarcted tissue.
• Commonly used MRI Sequence
- Diffusion weighted imaging (DWI)
- FLAIR
- T2
Introduction/ Materials & Methods / Results / Discussions & Conclusion
01
Introduction
Imaging
• DWI MR
- DWI (a combination of T2 and diffusion
weighting) is commonly used for evaluation
of “acute ischemic stroke” for its sensitivity
in the detection of small and early infarcts
• Apparent Diffusion Coefficient Map
(ADC map)
- ADC values can be used as reference data
in acute ischemic stroke populations.
However, its reliability is continuously
questioned [1] .
[1] Fiehler, Jens, et al. "Severe ADC decreases do not predict irreversible tissue damage in humans." Stroke 33.1 (2002): 79-86.
[1]
02
Introduction/ Materials & Methods / Results / Discussions & Conclusion
Introduction
Deep Learning
• Deep learning approaches
- U-shaped architecture consisted of
encoder and decoder [2] is dominant in
deep learning models for detecting
ischemic stroke lesions.
- One of the main issues of deep-learning
based detection algorithms are highly
unbalanced class extent, small size of lesion,
overfitting to the train set, and lack of
annotated datasets [3].
[2] Ronneberger, Olaf, Philipp Fischer, and Thomas Brox. "U-net: Convolutional networks for biomedical image segmentation." International Conference on Medical
image computing and computer-assisted intervention. Springer, Cham, 2015.
[3] Clèrigues, Albert, et al. "Acute ischemic stroke lesion core segmentation in CT perfusion images using fully convolutional neural networks." Computers in
Biology and Medicine 115 (2019): 103487.
[2]
03
Introduction/ Materials & Methods / Results / Discussions & Conclusion
Introduction
• Proposal
- A deep-learning based automated infarct segmentation model on DWI was
developed.
- Infarct severity with respect to ADC map was measured.
- Deep learning algorithm is expected to detect lesion areas missed by
predictions based on ADC values.
Ensemble
Results
Prediction ADC mapping
Large lesion
Small lesion
Modified U-Net
[An overview of proposed method]
04
Proposed Method
Introduction/ Materials & Methods / Results / Discussions & Conclusion
Materials and Methods
Dataset and Preprocessing Steps
• DWIs and ADC maps from 394 patients with acute
infarct (from a single institution)
- Train 216 / Validation 24 / Test 154
• Preprocessing
- N4 Bias Correction
- Pixel Respacing (to 1x1)
- Z-normalization
Pixel Respacing
Pre-processed DataOriginal Data
06
Introduction/ Materials & Methods / Results / Discussions & Conclusion
Materials and Methods
Model
• Use Modified 3D U-Net [4] as our baseline model.
• Modified 3D U-Net adds additional deep supervision in the decoder by integrating
segmentation layers at different levels of the network
• Replace conventional 2D Convolution with Partial Convolution [5].
• Our model performs 2D segmentation.
Modified U-Net
[4] Isensee, Fabian, et al. "Brain tumor segmentation and radiomics survival prediction: Contribution to the brats 2017 challenge." International MICCAI Brainlesion
Workshop. Springer, Cham, 2017.
[5] Liu, Guilin, et al. "Image inpainting for irregular holes using partial convolutions." Proceedings of the European Conference on Computer Vision (ECCV). 2018.
07
Introduction/ Materials & Methods / Results / Discussions & Conclusion
Materials and Methods
Training Details
Large-size Lesion
Small-size Lesion
• Divide train dataset into two subsets w.r.t lesion size.
• Train two separate models for each subset of train
dataset and ensemble the results to get final
prediction.
• Use data augmentation, including rescaling,
rotation, horizontal flip, and x/y translation.
• Use Generalized Dice Loss with different class
weights to cope with imbalanced data (background
is much more dominant than the stroke lesion)
• Train for 100 epochs using cosine-annealing, with
minimum learning rate set to 2e-4. Adam optimizer
with weight decay of 1e-5 is used.
08
Generalized Dice Loss
* r - reference foreground segmentation
* p - predicted probabilistic map
[6] Sudre, Carole H., et al. "Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations." Deep learning in medical image
analysis and multimodal learning for clinical decision support. Springer, Cham, 2017. 240-248.
[6]
Introduction/ Materials & Methods / Results / Discussions & Conclusion
Result
- Average Dice Coefficient : 0.85 ± 0.06
- Dice Coefficient after excluding outlier (extremely small lesions) : 0.89 ± 0.17
- 83% Sensitivity with 99% Specificity
- Average volume difference : 0.25ml
Original Data PredictionGround Truth
09
Introduction/ Materials & Methods / Results / Discussions & Conclusion
Result
Ablation Studies
Original Data Modified-Unet +
Partial Convolution +
Ensemble
U-NetModified-Unet +
Partial Convolution
Modified-UnetGround Truth
0.86 0.820.830.84Dice Coefficient :
10
Introduction/ Materials & Methods / Results / Discussions & Conclusion
Results
Infarct Severity
Four different levels
of ADC values
for Infarct Severity
ADC > 620
- No Stroke
520 < ADC <= 620
- Minor Stroke
420 < ADC <= 520
- Moderate Stroke
ADC <= 420
- Severe Stroke
11
Introduction/ Materials & Methods / Results / Discussions & Conclusion
Discussion
Conclusion
• Showed feasibility for detecting and quantifying acute infarcts.
• Showed superior performance to conventional U-Net.
• Proposed an end-to-end segmentation model that can be easily
employed to different domains of imaging.
• Needs External Validation to further validate our model.
• Future studies needed on improving prediction sensitivity.
12
Introduction/ Materials & Methods / Results / Discussions & Conclusion
Thank You
Introduction/ Materials & Methods / Results / Discussions & Conclusion

Deep Learning-based Fully Automated Detection and Quantification of Acute Infarcts

  • 1.
    Deep Learning-based FullyAutomated Detection and Quantification of Acute Infarcts Seung Hyun Hwang , Hwi Young Kim , Ji Hoon Cha , Hyug Gi Kim , Kyung Mi Lee , Hyun Suk Choi Yonsei University , Kyung Hee University Severance Hospital Yonsei University | RSNA 2020 November 29, 2020 1 1 1 2 2 1 21
  • 2.
    Materials & Methods Introduction Results 0102 03 04 Discussion & Conclusion Contents
  • 3.
    Introduction Ischemic Stroke • IschemicStroke - Stroke caused by a blockage in an artery that supplies blood to the brain. MRI sequences are dominantly used in detecting infarcted tissue. • Commonly used MRI Sequence - Diffusion weighted imaging (DWI) - FLAIR - T2 Introduction/ Materials & Methods / Results / Discussions & Conclusion 01
  • 4.
    Introduction Imaging • DWI MR -DWI (a combination of T2 and diffusion weighting) is commonly used for evaluation of “acute ischemic stroke” for its sensitivity in the detection of small and early infarcts • Apparent Diffusion Coefficient Map (ADC map) - ADC values can be used as reference data in acute ischemic stroke populations. However, its reliability is continuously questioned [1] . [1] Fiehler, Jens, et al. "Severe ADC decreases do not predict irreversible tissue damage in humans." Stroke 33.1 (2002): 79-86. [1] 02 Introduction/ Materials & Methods / Results / Discussions & Conclusion
  • 5.
    Introduction Deep Learning • Deeplearning approaches - U-shaped architecture consisted of encoder and decoder [2] is dominant in deep learning models for detecting ischemic stroke lesions. - One of the main issues of deep-learning based detection algorithms are highly unbalanced class extent, small size of lesion, overfitting to the train set, and lack of annotated datasets [3]. [2] Ronneberger, Olaf, Philipp Fischer, and Thomas Brox. "U-net: Convolutional networks for biomedical image segmentation." International Conference on Medical image computing and computer-assisted intervention. Springer, Cham, 2015. [3] Clèrigues, Albert, et al. "Acute ischemic stroke lesion core segmentation in CT perfusion images using fully convolutional neural networks." Computers in Biology and Medicine 115 (2019): 103487. [2] 03 Introduction/ Materials & Methods / Results / Discussions & Conclusion
  • 6.
    Introduction • Proposal - Adeep-learning based automated infarct segmentation model on DWI was developed. - Infarct severity with respect to ADC map was measured. - Deep learning algorithm is expected to detect lesion areas missed by predictions based on ADC values. Ensemble Results Prediction ADC mapping Large lesion Small lesion Modified U-Net [An overview of proposed method] 04 Proposed Method Introduction/ Materials & Methods / Results / Discussions & Conclusion
  • 7.
    Materials and Methods Datasetand Preprocessing Steps • DWIs and ADC maps from 394 patients with acute infarct (from a single institution) - Train 216 / Validation 24 / Test 154 • Preprocessing - N4 Bias Correction - Pixel Respacing (to 1x1) - Z-normalization Pixel Respacing Pre-processed DataOriginal Data 06 Introduction/ Materials & Methods / Results / Discussions & Conclusion
  • 8.
    Materials and Methods Model •Use Modified 3D U-Net [4] as our baseline model. • Modified 3D U-Net adds additional deep supervision in the decoder by integrating segmentation layers at different levels of the network • Replace conventional 2D Convolution with Partial Convolution [5]. • Our model performs 2D segmentation. Modified U-Net [4] Isensee, Fabian, et al. "Brain tumor segmentation and radiomics survival prediction: Contribution to the brats 2017 challenge." International MICCAI Brainlesion Workshop. Springer, Cham, 2017. [5] Liu, Guilin, et al. "Image inpainting for irregular holes using partial convolutions." Proceedings of the European Conference on Computer Vision (ECCV). 2018. 07 Introduction/ Materials & Methods / Results / Discussions & Conclusion
  • 9.
    Materials and Methods TrainingDetails Large-size Lesion Small-size Lesion • Divide train dataset into two subsets w.r.t lesion size. • Train two separate models for each subset of train dataset and ensemble the results to get final prediction. • Use data augmentation, including rescaling, rotation, horizontal flip, and x/y translation. • Use Generalized Dice Loss with different class weights to cope with imbalanced data (background is much more dominant than the stroke lesion) • Train for 100 epochs using cosine-annealing, with minimum learning rate set to 2e-4. Adam optimizer with weight decay of 1e-5 is used. 08 Generalized Dice Loss * r - reference foreground segmentation * p - predicted probabilistic map [6] Sudre, Carole H., et al. "Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations." Deep learning in medical image analysis and multimodal learning for clinical decision support. Springer, Cham, 2017. 240-248. [6] Introduction/ Materials & Methods / Results / Discussions & Conclusion
  • 10.
    Result - Average DiceCoefficient : 0.85 ± 0.06 - Dice Coefficient after excluding outlier (extremely small lesions) : 0.89 ± 0.17 - 83% Sensitivity with 99% Specificity - Average volume difference : 0.25ml Original Data PredictionGround Truth 09 Introduction/ Materials & Methods / Results / Discussions & Conclusion
  • 11.
    Result Ablation Studies Original DataModified-Unet + Partial Convolution + Ensemble U-NetModified-Unet + Partial Convolution Modified-UnetGround Truth 0.86 0.820.830.84Dice Coefficient : 10 Introduction/ Materials & Methods / Results / Discussions & Conclusion
  • 12.
    Results Infarct Severity Four differentlevels of ADC values for Infarct Severity ADC > 620 - No Stroke 520 < ADC <= 620 - Minor Stroke 420 < ADC <= 520 - Moderate Stroke ADC <= 420 - Severe Stroke 11 Introduction/ Materials & Methods / Results / Discussions & Conclusion
  • 13.
    Discussion Conclusion • Showed feasibilityfor detecting and quantifying acute infarcts. • Showed superior performance to conventional U-Net. • Proposed an end-to-end segmentation model that can be easily employed to different domains of imaging. • Needs External Validation to further validate our model. • Future studies needed on improving prediction sensitivity. 12 Introduction/ Materials & Methods / Results / Discussions & Conclusion
  • 14.
    Thank You Introduction/ Materials& Methods / Results / Discussions & Conclusion

Editor's Notes

  • #2 We studied and developed algorithm based on deep learning to perform image segmentation on MR imaging of acute infarct.
  • #4 A stroke is a serious life-threatening medical condition that happens when blood supply to the part of the brain is blocked. In this work, we deal with ischemic stroke.
  • #5 Two types of imaging are used in our study. DWIs and Apparent Diffusion Coefficient Map, denoted as ADC map. In terms of ADC map, ADC values can be used as a reference for infarction. However, it is dangerous to solely depend on ADC values since ADC values are usually acquired at a single time point after stroke in a current clinical practice.
  • #6 There have been many trials based on deep learning, in the field of medical image segmenetion. Majority of deep learning models follow U-shaped architecture, consisted of encoder and decoder. U-shaped architecture is first presented in U-net, in 2015. From since, lots of modifications of conventional u-net are developed to address for better performance. Undeniably, deep learning is one of the most powerful and accurate segmentation method. But there are some major issues around it. First, it is hard to cope with highly unbalanced class extent. Also, it is difficult to detect small size of lesion. The last one is a chronical issue in medical field, which is the lack of annotated datasets, which can easily lead to overfitting in model
  • #7 Here, we present a brief overview of our proposed method. We first developed a deep-learning based automated infarct segmentation model on DWI. After acquiring a predicted segmentation map, we measure the infarct severity of the predicted region, with respect to ADC map. We set few cut off values in ADC values to express the severity in several intensities.
  • #8 For dataset, we collected DWIs and ADC maps from a single institution. Imagings of total 394 patients are used First corrected the DWI values using N4 bias correction. Next, we standardized the pixel spacing by equalizing pixel spacing to 1mm in both x and y axis Last, we used z-score normalization. Check out the slight difference between the original data and pre-processed data on the screen .
  • #9 We selected modified 3d unet as a baseline model for our deep learning algorithm. It follows the general architecture of unet. Modified 3D U-Net adds additional deep supervision in the decoder <Press> by integrating segmentation layers at different levels of the network -We replaced the conventional 2D Convolution with Partial Convolution Partial conv is known to outperform the standard zero padding, by improving segmentation on boundary regions. -In this task, we performed 2d segmentation instead of 3D, for more precise prediction of infarctions.
  • #10 For training, we divided train datset into two subsets. One with Large-size lesion, and other with small size. -We Trained two separate models for each subset of train dataset and ensemble the results to get better predictions on small size lesions -Due to sporadic characteristic of acute infarct, the background is much more dominant than the stroke lesion. To cope with such imbalance, we employed Generalized Dice loss. Additional training details are written in the slide.
  • #11 Our model scored average dice coefficient of 0.85 with ~ We expect that abnormaly high specificity is due to dominant backgrounds. We also calculated the total volume of infarctions by reconstructing the 2D prediction.
  • #12 For further results, we tested and compared the performance of our method with conventional U-net, and Several other variants of our baseline model. From the utmost left, we show results of U-net, ~~~ Our final model with partial convolution and ensemble scored highest in dice coefficient. <Press> The qualitative results also showed that predicetion of our final model is closest to the ground truth. Consequently, we validated the positive effect and contribution of partial convolution and ensemble in image segmentaion.
  • #13 The following plot shows different levels of adc values in the regions of predicted infarct area. Four different levels of ADC values for Infarct Severity are depicted in color map.<Press> Each level represents the infarct severity. Infarct severity are classified into four categories, which are no stroke, minor stroke, moderate stroke, and severe stroke.
  • #14 In conclusion, The quantitative and qualititative results of our study showed feasibility for detecting and quantifying acute infarcts. Our model showed superior results to conventional U-net. Our algorithm, which is an end-to end semgnetion model, can be easily deployed and applied to other semantic segmentaion tasks as well. For further studies, we plan to perform external validation test for we have only validated our results using internal dataset so far. Moreover, we plan to optimize our model in order to be more sensitive to small lesions. Thank you for listening.
  • #15 The quantitative and qualititative results of our study Our algorithm, which is an end-to end semgnetion model, can be easily deployed and applied to other s드무샻 segmentaion tasks as well. For further studies, we plan to perform external validation test using datas collected from other organizations, for we have only validated our results using internal dataset so far. Moreover, we plan to optimize our model in order to be more sensitive to small lesions. Thank you for listening.