This document discusses a deep learning approach for predicting prostate cancer Gleason Grade Group from multiparametric MRI data. The authors developed a model called SummerNet that uses a multi-level approach, combining deep learning and conventional machine learning. SummerNet achieved a Cohen's kappa of 0.26 on the test data of the PROSTATEx-2 challenge, outperforming models that only used deep learning or conventional machine learning. The authors conclude that combining deep and conventional machine learning can help reduce overfitting, and suggest directions for future improvement including using multi-channel 3D convolutional layers.
OSVC_Meta-Data based Simulation Automation to overcome Verification Challenge...
Deep Learning Predicts Prostate Cancer Gleason Grade from MRI
1. Prostate Cancer Diagnosis using Deep Learning
with 3D Multiparametric MRI:
Predicting the Gleason Grade Group
Saifeng Liu1, Huaixiu Zheng2, Yesu Feng3
1 The MRI Institute for Biomedical Research, Detroit, MI, USA
2 Uber Technologies Inc., San Francisco, CA, USA
3 LinkedIn, San Francisco, CA, USA
PROSTATEx-2 Challenge, AAPM 2017
Aug. 1st, 2017, Denver
2. Prostate Cancer
Estimated New Cancer Cases in Men in US (2017)2
Prostate Cancer
Other Cancers
19%
The second most common type of cancer in men1
Risk of prostate cancer: 1 in 7
3. Prostate Cancer
The second most common type of cancer in men1
Deaths caused by prostate cancer: 1 in 39
Estimated New Cancer Cases in Men in US (2017)2
Prostate Cancer
Other Cancers
19%
4. Prostate Cancer
• Limited diagnostic accuracy
• Prostate-specific antigen (PSA) test, digital rectal exam (DRE)
• Transrectal ultrasound (TRUS)/MRI guided biopsy
• Multiparametric MRI (mpMRI) with PI-RADS v2 (sensitivity & specificity ~ 0.8)3
• Few studies on the prediction of Gleason grade group using mpMRI4,5
5. PROSTATEx-2 Challenge
• Data: 162 MRI cases
• 182 lesions, 40% are used as test set.
• Multiple types of MRI images (DWI, ADC, Ktrans, T2WI).
• Each case contains at least one prostate lesion with provided location.
• Goal: Prediction of Gleason Grade Group (1 to 5)
• Evaluation metric: Quadratic-weighted Cohen’s kappa
6. • Challenges
• Ordinal classification
• Heterogenous data
• Very limited and unbalanced samples
PROSTATEx-2 Challenge
36
41
20
8 7
7. Data Preprocessing
Registration
• Interpolation
• Rigid-body registration
Region Growing
• Region growing using
the provided lesion
location
• Cropping the ROI
Train/Validation
Sample Preparation
• Train (85%)/Validation
(15%)
• Channel composition
DWI
8. Data Preprocessing
Registration
• Interpolation
• Rigid-body registration
Region Growing
• Region growing using
the provided lesion
location
• Cropping the ROI
Train/Validation
Sample Preparation
• Train (85%)/Validation
(15%)
• Channel composition
DWI
ADC
9. Data Preprocessing
Registration
• Interpolation
• Rigid-body registration
Region Growing
• Region growing using
the provided lesion
location
• Cropping the ROI
Train/Validation
Sample Preparation
• Train (85%)/Validation
(15%)
• Channel composition
DWI
ADC
Ktrans
10. Data Preprocessing
Registration
• Interpolation
• Rigid-body registration
Region Growing
• Region growing using
the provided lesion
location
• Cropping the ROI
Train/Validation
Sample Preparation
• Train (85%)/Validation
(15%)
• Channel composition
DWI
ADC
Ktrans
T2WI
11. Data Preprocessing
Registration
• Interpolation
• Rigid-body registration
Region Growing
• Region growing using
the provided lesion
location
• Cropping the ROI
Train/Validation
Sample Preparation
• Train (85%)/Validation
(15%)
• Channel composition
DWI
ADC
Ktrans
T2WI
12. Data Preprocessing
Registration
• Interpolation
• Rigid-body registration
Region Growing
• Region growing using
the provided lesion
location
• Cropping the ROI
Train/Validation
Sample Preparation
• Train (85%)/Validation
(15%)
• Channel composition
DWI
ADC
Ktrans
T2WI
13. Data Preprocessing
Registration
• Interpolation
• Rigid-body registration
Region Growing
• Region growing using
the provided lesion
location
• Cropping the ROI
Train/Validation
Sample Preparation
• Train (85%)/Validation
(15%)
• Channel composition
DWI
ADC
Ktrans
T2WI
14. Data Preprocessing
Registration
• Interpolation
• Rigid-body registration
Region Growing
• Region growing using
the provided lesion
location
• Cropping the ROI
Train/Validation
Sample Preparation
• Train (85%)/Validation
(15%)
• Channel composition
DWI
ADC
Ktrans
T2WI
DWI(D), ADC(A), Ktrans(K), T2 (T)
AKTDAK
DAT DKT
23. PROSTATEx-1:
Deep Learning vs. Conventional Machine Learning
Train Data ROC Curve
0.80
0.84
XGBoost XmasNet
Test Data AUC
24. SummerNet:
Deep Learning + Conventional Machine Learning
Level 1
Model
Level 2
Model
Level 3
Model
Level 4
Model
1
2
3
4
5
Trained using conventional
machine learning
Trained using deep learning
P(ggg>1)
P(ggg>2)
P(ggg>3)
P(ggg>4)
Multi-class
Model
25. Dealing with Unbalanced Data
Training level-3 model: train sample selection
ggg=1
ggg=2
ggg=3
ggg=4
ggg=5
26. Dealing with Unbalanced Data
Training level-3 model: train sample selection ggg>3
ggg=1
ggg=2
ggg=3
ggg=4
ggg=5
27. Dealing with Unbalanced Data
Training level-3 model: train sample selection ggg≤3ggg>3
ggg=1
ggg=2
ggg=3
ggg=4
ggg=5
28. Deep Learning + Conventional Machine Learning:
Model Importance
29. Deep Learning + Conventional Machine Learning:
Model Importance
Trained using conventional
machine learning
30. Deep Learning + Conventional Machine Learning:
Model Importance
Trained using conventional
machine learning
Trained using deep learning
32. Conclusions and Future Directions
• Multi-level model for predicting Gleason Grade Group with mpMRI data
• Data augmentation with 3D rotation and slicing
• Deep learning + conventional machine learning reduced over-fitting
• Single model with multiple outputs
• Multi-channel input with 3D convolution layers
34. References
• 1. Key Statistics for Prostate Cancer. Retrieved February 10, 2017, from
https://www.cancer.org/cancer/prostate-cancer/about/key-statistics.html
• 2. American Cancer Society. Cancer facts and figures 2017.
• 3. Kasel-Seibert M et al. European journal of radiology. 2016 ;85(4):726-31.
• 4. Fehr et al. PNAS. 2015;112(46):E6265-73.
• 5. Gondo et al. European Radiology. 2014 Dec;24(12):3161-70.
• 6. Ehrenberg HR et al. In SPIE Medical Imaging 2016 (pp. 97851J-97851J).
• 7. Chen T, Guestrin C. InProceedings of the 22Nd ACM SIGKDD International Conference on Knowledge
Discovery and Data Mining 2016 (pp. 785-794). ACM.
• 8. Simonyan, K.., Zisserman, A., ArXiv14091556 Cs (2014).
• 9. Vallières, M., et al. Phys. Med. Biol. 60(14), 5471 (2015).
• 10. Saifeng Liu, Huaixiu Zheng, Yesu Feng and Wei Li. Proc. SPIE 10134, Medical Imaging 2017:
Computer-Aided Diagnosis, 1013428; doi:10.1117/12.2277121;