1.Lung Cancer Screening 1.1.Deep learning (feasible but not interpretable) 1.2.Radiomics (concise model) 1.3.Spiculation quantification (interpretable feature) 2.PET/CT Tumor Response 2.1.Aggressive Lung ADC subtype prediction (helpful for surgeons) 2.2.Pathologic response prediction (accurate but not concise) 2.3.Local tumor morphological changes (accurate and interpretable)

1. Sep 17, 2018
Quantitative Image Analysis for
Cancer Diagnosis and Radiation Therapy
Wookjin Choi, PhD, et al.
Department of Medical Physics
choiw@mskcc.org

2. Acknowledgements
Memorial Sloan Kettering Cancer Center
– Wei Lu PhD
– Sadegh Riyahi, PhD
– Jung Hun Oh, PhD
– Saad Nadeem, PhD
– George Li, PhD
– James G. Mechalakos, PhD
– Joseph O. Deasy, PhD
– Andreas Rimner, MD
– Prasad Adusumilli, MD
– Chia-ju Liu, MD
– Wolfgang Weber, MD
University of Maryland School of Medicine
– Howard Zhang, PhD
– Feng Jiang, MD, PhD
– Wengen Chen, MD, PhD
– Charles White, MD
– Steven Feigenberg, MD
– Warren D. D'Souza, PhD
– William Regine, MD
– Seth Kligerman, MD
– Shan Tan, PhD
– Jiahui Wang, PhD
Stony Brook University
– Allen Tannenbaum, PhD
2
NIH/NCI Grant R01 CA172638 and NIH/NCI Cancer Center Support Grant P30 CA008748

3. Outline
1. Lung Cancer Screening
1. Deep learning
2. Radiomics
3. Spiculation quantification
2. PET/CT Tumor Response
1. Aggressive Lung ADC subtype prediction
2. Pathologic response prediction
3. Local tumor morphological changes
3

4. Lung Cancer Screening
4
 Early detection of lung cancer by LDCT can reduce mortality
 Known features correlated with PN malignancy
 Size, growth rate (Lung-RADS)
 Calcification, enhancement, solidity → texture features
 Boundary margins (spiculation, lobulation), attachment → shape and
appearance features
Malignant nodules Benign nodules
Size Total Malignancy
< 4mm 2038 0%
4-7 mm 1034 1%
8-20 mm 268 15%
> 20 mm 16 75%
1

5. Deep Learning (Motivation)
• Data Science Bowl 2017 presented by
Booz|Allen|Hamilton & Kaggle
• Many deep learning methods were
proposed
– Top 10 teams: log loss 0.39~0.44
– Detection and Classification
– Top 99th: log loss 0.60
5
1.1

6. Deep Learning
(Methodology: 3D Fully Convolutional Neural Network)
6
1.1

7. Deep Learning (Results)
• 3-fold cross-validation
– Nodule Detection: Sensitivity 95.1% with 5 false positives per scan
– Nodule Classification: Accuracy 67.4%
• Deep Learning: Feasible but not interpretable
Ranked 99th out of 1972 teams (Top 6%, Bronze medal)
7
Log loss
1.1

8. Outline
1. Lung Cancer Screening
1. Deep learning (feasible but not interpretable)
2. Radiomics
3. Spiculation quantification
2. PET/CT Tumor Response
1. Aggressive Lung ADC subtype prediction
2. Pathologic response prediction
3. Local tumor morphological changes
8

9. Radiomics (Motivation)
9
 Controllable Feature Analysis
 More Interpretable
Lambin, et al. Eur J Cancer 2012
Aerts et al., Nature Communications, 2014
1.2

10. Radiomics Framework
• Automated Workflow (Python)
– Integrate all the radiomics components
– 3D Slicer, ITK (C++), Matlab, R, and Python
– Scalable: support multicore & grid computing
10
1.2
Image
Registration
• Multi-level rigid
• Deformable
• Pre/Post-CT
• MSE, MI
Tumor
Segmentation
• Adaptive region growing
• Level set
• Grow cut
• Morphology filter
• Multi-modality
image segmentation
Feature
Extraction
• Intensity distribution
• Spatial variations
(texture)
• Geometric properties
• Jacobian feature
from DVF
• Feature selection
Predicting
Response
• ROC analyses
• Prediction models
• Validation
• Tumor response
• Recurrence
• Survival
Source codes: https://github.com/taznux/radiomics-tools

11. Radiomics (Methodology)
• TCIA LIDC-IDRI public data set (n=1,010)
– Multi-institutional data
– 72 cases evaluated (31 benign and 41 malignant cases)
• Consensus contour
11
GLCM GLRM
Texture features Intensity features
2D
Shape features
3D
1.2

12. Radiomics (Methodology: SVM-LASSO Model )
12
SVM classification
Distinctive feature identification
Malignant?
Predicted malignancy
Feature extraction
Yes
10x10-foldCV
10-foldCV
LASSO feature selection
1.2

13. Radiomics (Results)
• Size (BB_AP) : Highly correlated with the axial longest diameter and its perpendi
cular diameter (r = 0.96, larger – more malignant)
• Texture (SD_IDM) : Tumor heterogeneity (smaller – more malignant)
13
1.2

14. Radiomics (Results: Comparison)
Sensitivity Specificity Accuracy AUC
Lung-RADS
Clinical guideline
73.3% 70.4% 72.2% 0.74
Hawkins et al. (2016)
Radiomics – 23 features
51.7 % 92.9% 80.0% 0.83
Ma et al. (2016)
Radiomics – 583 features
80.0% 85.5% 82.7%
Buty et al. (2016)
DL – 400 SH and 4096 AlexNet features
82.4%
Kumar et al. (2015)
DL: 5000 features
79.1% 76.1% 77.5%
Proposed
Radiomics: two features (Size and Texture)
87.2% 81.2% 84.6% 0.89
14
DL: Deep Learning, SH: Spherical Harmonics
1.2
Choi et al., Medical Physics, 2018.

15. Outline
1. Lung Cancer Screening
1. Deep learning (feasible but not interpretable)
2. Radiomics (concise model)
3. Spiculation quantification
2. PET/CT Tumor Response
1. Aggressive Lung ADC subtype prediction
2. Pathologic response prediction
3. Local tumor morphological changes
15

16. Spiculation Quantification (Motivation)
• Blind Radiomics
• Semantic Features
16
1.3
Radiologist's spiculation score 𝑠𝑟 for different pulmonary nodules

17. Spiculation Quantification (Methodology)
17
𝜖𝑖: = log
𝑗,𝑘 𝐴 ([𝜙(𝑣𝑖), 𝜙(𝑣𝑗), 𝜙(𝑣 𝑘)])
𝑗,𝑘 𝐴 ([𝑣𝑖, 𝑣𝑗, 𝑣 𝑘])
Area Distortion Map
Nadeem et al., IEEE TVCG, 2017
Spherical Mapping
1.3
Eigenfunction

18. Spicule Quantification: Height, Area Distortion
• Area distortion better than solid angle for non-conic spicules
• Height is medial axis length, can be perpendicular distance
• Width is measured at base, can be FWHM
𝑠1 =
𝑖 mean 𝜖 𝑝 𝑖 ∗ ℎ 𝑝 𝑖
𝑖 ℎ 𝑝 𝑖
𝑠2 =
𝑖 min 𝜖 𝑝 𝑖 ∗ ℎ 𝑝 𝑖
𝑖 ℎ 𝑝 𝑖
1.3

19. Spiculation Quantification (Results)
• Higher correlation with the radiologist’s score (𝑠1: 0.67 AUC, P=4.29E-08, 𝜌=-0.33)
• Mean curvature method (𝑠 𝑎: 0.59 AUC, P=0.068, 𝜌=0.22 and 𝑠 𝑏: 0.66 AUC, P=1.47E-07, 𝜌=0.29)
19
𝑠 𝑟:2, 𝑁𝑝:5, 𝑠1:-0.6 𝑠 𝑟:3 , 𝑁𝑝:6, 𝑠1:-0.9 𝑠 𝑟:4 , 𝑁𝑝:30, 𝑠1:-1.2 𝑠 𝑟:5 , 𝑁𝑝:15, 𝑠1:-1.6
𝑠 𝑟: radiologist's spiculation score, 𝑠1: proposed spiculation score
1.3

20. Spiculation Quantification (Results: Comparison)
20
Sensitivity Specificity Accuracy AUC
Size+Texture (previous model) 87.20% 81.20% 84.60% 0.89
Size+Texture+Radiologist’s score 87.80% 87.10% 87.50% 0.91
Size+Texture+Our measures 92.68% 83.87% 88.89% 0.92
Performance of Malignancy Prediction
• TCIA LIDC-IDRI public dataset
– 811 cases: only annotations
– 72 cases: with malignancy information
Choi et al. Accepted in ShapeMI @ MICCAI 2018
1.3

21. Outline
1. Lung Cancer Screening
1. Deep learning (feasible but not interpretable)
2. Radiomics (concise model)
3. Spiculation quantification (interpretable feature)
2. PET/CT Tumor Response
1. Aggressive Lung ADC subtype prediction
2. Pathologic response prediction
3. Local tumor morphological changes
21

22. PET/CT Tumor Response
• RECIST – Size
• PERCIST – Metabolic Response
• PET/CT Radiomics Response Prediction
– Pre and/or Post
– Complementary information
22
2
RECIST: stable disease. CT Morphologic Criteria: optimal response –
decreased attenuation , homogeneous, and sharp tumor-liver
interface.
Tumor size: minimal change. Structure: marked decreased
attenuation centrally (marked central necrosis), indicating favorable
response.

23. Aggressive Lung ADC Subtype Prediction (Motivation)
23
2.1
CT
MIP
PET/CT
Soild
CT PET/CT
Five classifications of lung ADC Travis et
al. JTO 2011
 Solid and MIP components: poor surgery/SBRT prognosis factor
 Benefit from lobectomy rather than limited resection
 Core biopsy (Leeman et al. IJROBP 2017)
 Minimally invasive, not routinely performed, sampling error (about 60%
agreement with pathology)
 Preoperative diagnostic CT and FDG PET/CT radiomics
 Non-invasive and routinely performed

24. Aggressive Lung ADC Subtype Prediction (Method & Results)
• Retrospectively enrolled 120 patients
– Stage I lung ADC, ≤2cm
– Preoperative diagnostic CT and FDG PET/CT
• Histopathologic endpoint
– Aggressiveness (Solid : 18 cases, MIP : 5 cases)
• 206 radiomic features & 14 clinical parameters
• SVM-LASSO model
24
Type Feature Sensitivity Specificity Accuracy AUC P
1 PET Mean of Cluster Shade 67.0±4.3% 84.7±0.9% 81.3±1.2% 0.76±0.02 0.00083
2 Clinical Age 18.1±6.6% 82.6±3.0% 70.0±3.1% 0.50±0.04 0.27
Performance of the SVM-LASSO model to predict aggressive lung ADC
Choi et al. Manuscript In Preparation
2.1
Box plots of SUVmax (FDR q=0.0042) and PET Mean of
Cluster Shade (q=0.0021)

25. Outline
1. Lung Cancer Screening
1. Deep learning (feasible but not interpretable)
2. Radiomics (concise model)
3. Spiculation quantification (interpretable feature)
2. PET/CT Tumor Response
1. Aggressive Lung ADC subtype prediction (helpful for surgeons)
2. Pathologic response prediction
3. Local tumor morphological changes
25

26. Pathologic Response Prediction (Motivation)
26
Tan et al. IJROBP 2013
2.2 ResponderNon-responder

27. Pathologic Response Prediction (Method & Results)
• 20 esophageal cancer patients
– Underwent trimodality therapy (CRT + surgery) and FDG PET/CT
– 9 responders, 11 non-responders
• SVM model with 17 selected radiomic features: AUC = 1.0
• Conventional response measures: AUCs < 0.75
27
Zhang et al. IJROBP 2014
2.2

28. Outline
1. Lung Cancer Screening
1. Deep learning (feasible but not interpretable)
2. Radiomics (concise model)
3. Spiculation quantification (interpretable feature)
2. PET/CT Tumor Response
1. Aggressive Lung ADC subtype prediction (helpful for surgeons)
2. Pathologic response prediction (accurate but not concise)
3. Local tumor morphological changes
28

29. Local tumor morphological changes (Motivation)
29
2.3

30. Local tumor morphological changes (Methodology)
• Jacobian Map
– Jacobian matrix: calculates rate of displacement change in each direction.
– Determinant indicates volumetric ratio of shrinkage/expansion.
30
𝐷𝑒𝑡 𝐽 =
𝐷𝑒𝑡 𝐽 > 1 volume expansion
𝐷𝑒𝑡 𝐽 = 1 no volume change
𝐷𝑒𝑡 𝐽 < 1 volume shrinkage
𝐷𝑒𝑡 𝐽 = 1.2 = 20% expansion
𝐷𝑒𝑡 𝐽 = 0.8 = 20% shrinkage (-20%)
2.3

31. Local tumor morphological changes (Results)
31
2.3

32. Local tumor morphological changes (Results)
32
Features P-value AUC Correlation to responders
Minimum Jacobian 0.009 0.98 -0.79
Median Jacobian 0.046 0.95 -0.72
The P-value, AUC and correlation to responders for all significant features in univariate analysis
Riyahi, Choi et al., PMB SVM-LASSO: AUC 0.91
2.3

33. Local tumor morphological changes (Methodology)
33
2.3
• 𝐵𝑙𝑒𝑛𝑑𝑒𝑑 𝑃𝐸𝑇 − 𝐶𝑇 = 𝛼 𝑛𝐶𝑇 + 1 − 𝛼 𝑛𝑃𝐸𝑇
• 𝛼 empiracally chosen to be 0.2, given more weight to PET
• 𝑐 𝜑 𝑥, 𝑡 , 𝐼 𝑏° 𝐼𝑓 = 𝐸𝑠𝑖𝑚𝑖𝑙𝑎𝑟𝑖𝑡𝑦
𝑀𝐼
𝜑 𝑥, 1 , 𝐼 𝑏, 𝐼𝑓 +
𝐸𝑔𝑒𝑜𝑑𝑒𝑠𝑖𝑐
2
𝜑 𝑥, 0 , 𝜑(𝑥, 1) +
𝜌 𝐵𝑠𝑝𝑙𝑖𝑛𝑒(𝑣 𝜑 𝑥, 𝑡 , 𝐵 𝑘)

34. • Blended PET-CT: large MTV s
hrinkage towards the center
• PET: smaller MTV shrinkage
• CT: no change
Riyahi, Choi et al. Accepted in DATRA @ MICCAI 2018
Local tumor morphological changes (Results- pCR)
2.3

35. Local tumor morphological changes (Results- Non-pCR)
• Blended PET-CT: small MTV s
hrinkage
• PET: smaller MTV shrinkage
• CT: no change
Riyahi, Choi et al. Accepted in DATRA @ MICCAI 2018
2.3

36. ∆MTV by Jacobian vs. Segmentation2.3

37. Summary
1. Lung Cancer Screening
1. Deep learning (feasible but not interpretable)
2. Radiomics (concise model)
3. Spiculation quantification (interpretable feature)
2. PET/CT Tumor Response
1. Aggressive Lung ADC subtype prediction (helpful for surgeons)
2. Pathologic response prediction (accurate but not concise)
3. Local tumor morphological changes (accurate and interpretable)
37

38. Short-term Future Works
• Develop interpretable radiomic features
– Semi-automatic segmentation
– Multi-institution validation
• Integrate the radiomics framework into TPS
– Eclipse (C#), MIM (Python), Raystation (Python)
38

39. Long-term Future Works
• Radiomics Framework for Radiation Therapy
– Multi-modal imaging
– Response prediction (Pre, Post)
– Longitudinal analysis of tumor change
• Automation of Clinical Workflow
– Big Data Analytics: EMR, PACS, ROIS, Genomics, etc.
– Provide an informatics platform for comprehensive cancer therapy
39

40. Selected Publications
1. Wookjin Choi et al., “Radiomics analysis of pulmonary nodules in low-dose CT for early detection of lung cancer”, Medical Physi
cs, 2018
2. Wookjin Choi et al. “Technical Note: Identification of Normal Lung CT Texture Features Robust to Tumor Size for the Prediction
of Radiation-Induced Lung Disease”, International Journal of Medical Physics, Clinical Engineering and Radiation Oncology,
2018
3. Sadegh Riyahi, Wookjin Choi, et al., “Quantifying local tumor morphological changes with Jacobian map for prediction of
pathologic tumor response to chemo-radiotherapy in locally advanced esophageal cancer”, Physics in Medicine and Biology,
2018
4. Shan Tan, Laquan Li, Wookjin Choi, et al., “Adaptive region-growing with maximum curvature strategy for tumor segmentation i
n 18F-FDG PET”, Physics in Medicine and Biology, 2017
5. Wookjin Choi et al., “Individually Optimized Contrast-Enhanced 4D-CT for Radiotherapy Simulation in Pancreatic Ductal Adenoca
rcinoma”, Medical Physics, 2016
6. Wookjin Choi, Tae-Sun Choi, “Automated Pulmonary Nodule Detection based on Three-dimensional Shape-based Feature Descri
ptor”, Computer Methods and Programs in Biomedicine, 2014
7. Wookjin Choi, Tae-Sun Choi, “Automated Pulmonary Nodule Detection System in Computed Tomography Images: A Hierarchical
Block Classification Approach”, Entropy, Vol. 15, No. 2, pp. 507-523, February 2013
8. Wookjin Choi, Tae-Sun Choi, “Genetic Programming-based feature transform and classification for the automatic detection of pu
lmonary nodules on computed tomography images”, Information Sciences, 2012
40
Complete list of publications: https://scholar.google.com/citations?user=iHgsGLUAAAAJ

41. 41
Thank You!
Q & A
https://qradiomics.wordpress.com
E-mail: choiw@mskcc.org

42. ACR Lung-RADS
Category Baseline Screening Malignancy
1 No PNs; PNs with calcification
Negative
<1% chance of malignancy
2
Solid/part-solid: <6 mm
GGN: <20 mm
Benign appearance
<1% chance of malignancy
3
Solid: ≥6 to <8 mm
Part-solid: ≥6 mm with solid component <6 mm
GGN: ≥20 mm
Probably benign
1-2% chance of malignancy
4A
Solid: ≥8 to <15 mm
Part-solid: ≥8 mm with solid component ≥6 and <8 mm
Suspicious
5-15% chance of malignancy
4B
Solid: ≥15 mm
Part-solid: Solid component ≥8 mm
>15% chance of malignancy
4X
Category 3 or 4 PNs with suspicious features (e.g. enlarged lymph
nodes) or suspicious imaging findings (e.g. spiculation)
>15% chance of malignancy
42
ACR: American College of Radiology
Lung-RADS: Lung CT Screening Reporting and Data System

43. Nodule Classification
43

44. Spicule Quantification: Height, Angle
𝑠 𝑎 =
𝑖
𝑒−𝜔 )𝑝(𝑖 ℎ )𝑝(𝑖
𝑠 𝑏 =
𝑖 ℎ )𝑝(𝑖 cos𝜔 )𝑝(𝑖
𝑖 ℎ )𝑝(𝑖
• Model spicules as cones: height of the cone and solid angle subtended
at peak by the base
• The shaper or the higher a spicule, the larger Sa and Sb
Dhara, et al. 2016. Int J Comput Assit Radiol Surg 11: 337-349.