1.Lung Cancer Screening 1.1.Deep learning (feasible but not interpretable) 1.2.Radiomics (concise model) 1.3.Spiculation quantification (interpretable feature) 2.PET/CT Tumor Response 2.1.Aggressive Lung ADC subtype prediction (helpful for surgeons) 2.2.Pathologic response prediction (accurate but not concise) 2.3.Local tumor morphological changes (accurate and interpretable)

1. Medical Physics, Memorial Sloan Kettering Cancer Center
Wookjin Choi, PhD
May 21, 2018
Quantitative Image Analysis for
Cancer Diagnosis and
Radiation Therapy

2. Outline
1. Lung Cancer Screening
1. Deep learning
2. Radiomics
3. Spiculation quantification
2. PET/CT Tumor Response
1. Aggressive Lung ADC subtype prediction
2. Pathologic response prediction
3. Local tumor morphological changes
2

3. Lung Cancer Screening
3
 Early detection of lung cancer by LDCT can reduce mortality
 Known features correlated with PN malignancy
 Size, growth rate (Lung-RADS)
 Calcification, enhancement, solidity → texture features
 Boundary margins (spiculation, lobulation), attachment → shape and
appearance features
Malignant nodules Benign nodules
Size Total Malignancy
< 4mm 2038 0%
4-7 mm 1034 1%
8-20 mm 268 15%
> 20 mm 16 75%
1

4. Deep Learning (Motivation)
4
 Data Science Bowl 2017 presented
by Booz|Allen|Hamilton & Kaggle
 Many deep learning methods were
proposed
 Top 10 teams: log loss 0.39~0.44
 Detection and Classification
 Top 99th: log loss 0.60
1.1

5. Deep Learning
(Methodology: 3D Fully Convolutional Neural Network)
5
1.1

6. Deep Learning (Results)
6
 3-fold cross-validation
 Nodule Detection: Sensitivity 95.1% with 5 false positives per scan
 Nodule Classification: Accuracy 67.4%
 Deep Learning: Feasible but not interpretable
Ranked 99th out of 1972 teams (Top 6%, Bronze medal)
Log loss
1.1

7. Outline
1. Lung Cancer Screening
1. Deep learning (feasible but not interpretable)
2. Radiomics
3. Spiculation quantification
2. PET/CT Tumor Response
1. Aggressive Lung ADC subtype prediction
2. Pathologic response prediction
3. Local tumor morphological changes
7

8. Radiomics (Motivation)
8
 Controllable Feature Analysis
 More Interpretable
Lambin, et al. Eur J Cancer 2012
Aerts et al., Nature Communications, 2014
1.2

9. Radiomics Framework
9
 Automated Workflow (Python)
 Integrate all the radiomics components
 3D Slicer, ITK (C++), Matlab, R, and Python
 Scalable: support multicore & grid computing
1.2
Image
Registration
• Multi-level rigid
• Deformable
• Pre/Post-CT
• MSE, MI
Tumor
Segmentation
• Adaptive region growing
• Level set
• Grow cut
• Morphology filter
• Multi-modality
image segmentation
Feature
Extraction
• Intensity distribution
• Spatial variations
(texture)
• Geometric properties
• Jacobian feature
from DVF
• Feature selection
Predicting
Response
• ROC analyses
• Prediction models
• Validation
• Tumor response
• Recurrence
• Survival
Source codes: https://github.com/taznux/radiomics-tools

10. Radiomics (Methodology)
 TCIA LIDC-IDRI public data set (n=1,010)
 Multi-institutional data
 72 cases evaluated (31 benign and 41 malignant cases)
 Consensus contour
GLCM GLRM
Texture features Intensity features
2D
Shape features
3D
10
1.2

11. Radiomics (Methodology: SVM-LASSO Model )
11
SVM classification
Distinctive feature identification
Malignant?
Predicted malignancy
Feature extraction
Yes
10x10-foldCV
10-foldCV
LASSO feature selection
1.2

12. Radiomics (Results)
 Size (BB_AP) : Highly correlated with the axial longest diameter and its
perpendicular diameter (r = 0.96, larger – more malignant)
 Texture (SD_IDM) : Tumor heterogeneity (smaller – more malignant)
12
1.2

13. Radiomics (Results: Comparison)
Sensitivity Specificity Accuracy AUC
Lung-RADS
Clinical guideline
73.3% 70.4% 72.2% 0.74
Hawkins et al. (2016)
Radiomics – 23 features
51.7 % 92.9% 80.0% 0.83
Ma et al. (2016)
Radiomics – 583 features
80.0% 85.5% 82.7%
Buty et al. (2016)
DL – 400 SH and 4096 AlexNet features
82.4%
Kumar et al. (2015)
DL: 5000 features
79.1% 76.1% 77.5%
Proposed
Radiomics: two features (Size and Texture)
87.2% 81.2% 84.6% 0.89
DL: Deep Learning, SH: Spherical Harmonics
13
1.2
Choi et al., Medical Physics, 2018.

14. Outline
1. Lung Cancer Screening
1. Deep learning (feasible but not interpretable)
2. Radiomics (concise model)
3. Spiculation quantification
2. PET/CT Tumor Response
1. Aggressive Lung ADC subtype prediction
2. Pathologic response prediction
3. Local tumor morphological changes
14

15. Spiculation Quantification (Motivation)
15
 Blind Radiomics
 Semantic Features
1.3
Radiologist's spiculation score 𝑠𝑟 for different pulmonary nodules

16. Spiculation Quantification (Methodology)
16
𝜖𝑖: = log
𝑗,𝑘 𝐴 ([𝜙(𝑣𝑖), 𝜙(𝑣𝑗), 𝜙(𝑣 𝑘)])
𝑗,𝑘 𝐴 ([𝑣𝑖, 𝑣𝑗, 𝑣 𝑘])
Area Distortion Map
Nadeem et al., IEEE TVCG, 2017
Spherical Mapping
1.3

17. Spiculation Quantification (Results)
 Higher correlation with the radiologist’s score (𝑠1: 0.67 AUC, P=4.29E-08, 𝜌=-0.33)
 Mean curvature method (𝑠 𝑎: 0.59 AUC, P=0.068, 𝜌=0.22 and 𝑠 𝑏: 0.66 AUC, P=1.47E-07,
𝜌=0.29)
𝑠 𝑟:2, 𝑁𝑝:5, 𝑠1:-0.6 𝑠 𝑟:3 , 𝑁𝑝:6, 𝑠1:-0.9 𝑠 𝑟:4 , 𝑁𝑝:30, 𝑠1:-1.2 𝑠 𝑟:5 , 𝑁𝑝:15, 𝑠1:-1.6
𝑠 𝑟: radiologist's spiculation score, 𝑠1: proposed spiculation score
17
1.3

18. Spiculation Quantification (Results: Comparison)
18
Sensitivity Specificity Accuracy AUC
Size+Texture (previous model) 87.20% 81.20% 84.60% 0.89
Size+Texture+Radiologist’s score 87.80% 87.10% 87.50% 0.91
Size+Texture+Our measures 92.68% 83.87% 88.89% 0.92
Performance of Malignancy Prediction
• TCIA LIDC-IDRI public dataset
– 811 cases: only annotations
– 72 cases: with malignancy information
Choi et al. Submitted to MICCAI 2018
1.3

19. Outline
1. Lung Cancer Screening
1. Deep learning (feasible but not interpretable)
2. Radiomics (concise model)
3. Spiculation quantification (interpretable feature)
2. PET/CT Tumor Response
1. Aggressive Lung ADC subtype prediction
2. Pathologic response prediction
3. Local tumor morphological changes
19

20. PET/CT Tumor Response
20
 RECIST – Size
 PERCIST – Metabolic Response
 PET/CT Radiomics Response Prediction
 Pre and/or Post
 Complementary information
2
RECIST: stable disease. CT Morphologic Criteria: optimal
response – decreased attenuation , homogeneous, and sharp
tumor-liver interface.
Tumor size: minimal change. Structure: marked decreased
attenuation centrally (marked central necrosis), indicating
favorable response.

21. Aggressive Lung ADC Subtype Prediction (Motivation)
21
2.1
CT
MIP
PET/CT
Soild
CT PET/CT
Five classifications of lung ADC
Travis et al. JTO 2011
 Solid and MIP components: poor surgery/SBRT prognosis
factor
 Benefit from lobectomy rather than limited resection
 Core biopsy
 minimally invasive, not routinely performed, sampling error
 Preoperative diagnostic CT and FDG PET/CT radiomics
 Non-invasive and routinely performed
Leeman et al. IJROBP 2017

22. Aggressive Lung ADC Subtype Prediction (Method & Results)
22
 Retrospectively enrolled 120 patients
 Stage I lung ADC, ≤2cm
 Preoperative diagnostic CT and FDG PET/CT
 Histopathologic endpoint
 Aggressiveness (Solid : 18 cases, MIP : 5 cases)
 206 radiomic features & 14 clinical parameters
 SVM-LASSO model
Type Feature Sensitivity Specificity Accuracy AUC P
1 PET Mean of Cluster Shade 67.0±6.1% 82.7±3.4% 79.6±3.0% 0.75±0.03 0.0030
2 PET Mean of Haralick Correlation 24.7±5.3% 85.5±2.9% 73.6±2.2% 0.55±0.02 0.062
Performance of the SVM-LASSO model to predict aggressive lung ADC
Choi et al. Manuscript In Preparation
2.1

23. Outline
1. Lung Cancer Screening
1. Deep learning (feasible but not interpretable)
2. Radiomics (concise model)
3. Spiculation quantification (interpretable feature)
2. PET/CT Tumor Response
1. Aggressive Lung ADC subtype prediction (helpful for surgeons)
2. Pathologic response prediction
3. Local tumor morphological changes
23

24. Pathologic Response Prediction (Motivation)
24 Tan et al. IJROBP 2013
2.2 ResponderNon-responder

25. Pathologic Response Prediction (Method & Results)
25
 20 esophageal cancer patients
 Underwent trimodality therapy (CRT + surgery) and FDG PET/CT
 9 responders, 11 non-responders
 SVM model with 17 selected radiomic features: AUC = 1.0
 Conventional response measures: AUCs < 0.75
Zhang et al. IJROBP 2014
2.2

26. Outline
1. Lung Cancer Screening
1. Deep learning (feasible but not interpretable)
2. Radiomics (concise model)
3. Spiculation quantification (interpretable feature)
2. PET/CT Tumor Response
1. Aggressive Lung ADC subtype prediction (helpful for surgeons)
2. Pathologic response prediction (accurate but not concise)
3. Local tumor morphological changes
26

27. Local tumor morphological changes (Motivation)
27
2.3

28. Local tumor morphological changes (Methodology)
28
 Jacobian Map
 Jacobian matrix: calculates rate of displacement change in each direction.
 Determinant indicates volumetric ratio of shrinkage/expansion.
𝐷𝑒𝑡 𝐽 =
𝐷𝑒𝑡 𝐽 > 1 volume expansion
𝐷𝑒𝑡 𝐽 = 1 no volume change
𝐷𝑒𝑡 𝐽 < 1 volume shrinkage
𝐷𝑒𝑡 𝐽 = 1.2 = 20% expansion
𝐷𝑒𝑡 𝐽 = 0.8 = 20% shrinkage (-20%)
2.3

29. Local tumor morphological changes (Results)
29
2.3

30. Local tumor morphological changes (Results)
Features P-value AUC Correlation to responders
Minimum Jacobian 0.009 0.98 -0.79
Median Jacobian 0.046 0.95 -0.72
The P-value, AUC and correlation to responders for all significant features in univariate analysis
30 Riyahi, Choi et al., PMB, Under Revision SVM-LASSO: AUC 0.91
2.3

31. Local tumor morphological changes (Methodology)
31
 Blended PET/CT Jacobian Map
(a) Main workflow of our method. (b) Converging DVF represents a volume loss and generates a Jacobian map (c)
that illustrates local shrinkage (blue).
2.3

32. Local tumor morphological changes (Results)
Baseline Follow-up Blended PET/CT PET CT
32 Riyahi, Choi et al. Submitted to MICCAI 2018 AUC 0.85
2.3

33. Summary
1. Lung Cancer Screening
1. Deep learning (feasible but not interpretable)
2. Radiomics (concise model)
3. Spiculation quantification (interpretable feature)
2. PET/CT Tumor Response
1. Aggressive Lung ADC subtype prediction (helpful for surgeons)
2. Pathologic response prediction (accurate but not concise)
3. Local tumor morphological changes (accurate and interpretable)
33

34. Short-term Future Works
 Develop interpretable radiomic features
 Semi-automatic segmentation
 Multi-institution validation
 Breast Cancer
 Integrate the radiomics framework into TPS
 Eclipse (C#), MIM (Python), Raystation (Python)
34

35. Long-term Future Works
 Radiomics Framework for Radiation Therapy
 Multi-modal imaging
 Response prediction (Pre, Post)
 Longitudinal analysis of tumor change
 Automation of Clinical Workflow
 Big Data Analytics: EMR, PACS, ROIS, Genomics, etc.
 Provide an informatics platform for comprehensive cancer therapy
35

36. Selected Publications
1. Wookjin Choi et al., “Radiomics analysis of pulmonary nodules in low-dose CT for early detection of lung
cancer”, Medical Physics, 2018.
2. Shan Tan, Laquan Li, Wookjin Choi, et al., “Adaptive region-growing with maximum curvature strategy for
tumor segmentation in 18F-FDG PET”, Physics in Medicine and Biology, 2017.
3. Wookjin Choi et al., “Individually Optimized Contrast-Enhanced 4D-CT for Radiotherapy Simulation in
Pancreatic Ductal Adenocarcinoma”, Medical Physics, 2016
4. Wookjin Choi, Tae-Sun Choi, “Automated Pulmonary Nodule Detection based on Three-dimensional Shape-
based Feature Descriptor”, Computer Methods and Programs in Biomedicine, 2014
5. Wookjin Choi, Tae-Sun Choi, “Automated Pulmonary Nodule Detection System in Computed Tomography
Images: A Hierarchical Block Classification Approach”, Entropy, Vol. 15, No. 2, pp. 507-523, February 2013
6. Wookjin Choi, Tae-Sun Choi, “Genetic Programming-based feature transform and classification for the
automatic detection of pulmonary nodules on computed tomography images”, Information Sciences, 2012
7. M.T. Mahmood, Wookjin Choi, Tae-Sun Choi, “PCA-Based Method for 3-D Shape Recovery of Microscopic
Objects From Image Focus Using Discrete Cosine Transform”, Microscopy Research and Technique, 2008
36
Complete list of publications: https://scholar.google.com/citations?user=iHgsGLUAAAAJ

37. Acknowledgements
Memorial Sloan Kettering Cancer Center
 Wei Lu PhD
 Sadegh Riyahi, PhD
 Jung Hun Oh, PhD
 Saad Nadeem, PhD
 George Li, PhD
 James G. Mechalakos, PhD
 Joseph O. Deasy, PhD
 Andreas Rimner, MD
 Prasad Adusumilli, MD
 Chia-ju Liu, MD
 Wolfgang Weber, MD
University of Maryland School of Medicine
 Howard Zhang, PhD
 Feng Jiang, MD, PhD
 Wengen Chen, MD, PhD
 Charles White, MD
 Steven Feigenberg, MD
 Warren D. D'Souza, PhD
 William Regine, MD
 Seth Kligerman, MD
 Shan Tan, PhD
 Jiahui Wang, PhD
Stony Brook University
 Allen Tannenbaum, PhD
NIH/NCI Grant R01 CA172638 and NIH/NCI Cancer Center Support Grant P30 CA008748
37

38. Thank You!
Q & A
https://qradiomics.wordpress.com
E-mail: choiw@mskcc.org
38

39. ACR Lung-RADS
Category Baseline Screening Malignancy
1 No PNs; PNs with calcification
Negative
<1% chance of malignancy
2
Solid/part-solid: <6 mm
GGN: <20 mm
Benign appearance
<1% chance of malignancy
3
Solid: ≥6 to <8 mm
Part-solid: ≥6 mm with solid component <6 mm
GGN: ≥20 mm
Probably benign
1-2% chance of malignancy
4A
Solid: ≥8 to <15 mm
Part-solid: ≥8 mm with solid component ≥6 and <8 mm
Suspicious
5-15% chance of malignancy
4B
Solid: ≥15 mm
Part-solid: Solid component ≥8 mm
>15% chance of malignancy
4X
Category 3 or 4 PNs with suspicious features (e.g. enlarged lymph
nodes) or suspicious imaging findings (e.g. spiculation)
>15% chance of malignancy
ACR: American College of Radiology
Lung-RADS: Lung CT Screening Reporting and Data System39

40. Nodule Classification
40

41. Comparisons
Study
Patient
number
Follow-up
CT
Change in tumor volume
Sensitivity Specificity AUC p
Responders
Non-
responders
Jones et al. 50
4-5 w post-
Chemo
-11.6% (tumor length)
-33.3% (esophageal wall
thickness)
65% 33% - 0.22
van Heijl et al. 39
14 d during
CRT
12% 22% 19% 92% 0.63 0.18
Beer et al. 21
14 d during
Chemo
-24% -16% 100% 53% 0.73 0.04
Griffith et al. 45
6 d (1-17d)
post-CRT
~-55% ~-35% - - - 0.58
Conventional
volume change 20
4-6 w post-
CRT
-33% -36 64% 67% 0.58 0.6
Jacobian map -20% 5% 92% 89% 0.91 0.0002
Comparison with studies using CT for esophageal cancer response evaluation. Negative sign (-) indicates shrinkage
Riyahi, Choi et al., PMB, Under Revision41

42. Blended PET/CT Jacobian Results
Study Features AUC p-value
Yip et al. Run length matrix 0.71∼0.81 <0.02
Current study
∆MTV
∆SUVmax
0.62
0.53
0.33
0.81
Blended PET-CT(BSD) Co-occurrence matrix 0.85 0.006
PET (BSD) Co-occurrence matrix 0.81 0.014
CT (BSD) Run length matrix 0.76 0.038
The p-value and AUC for Jacobian and clinical features in univariate analysis for prediction of pathologic tumor
response
BSD: B-spline regularized Symmetric Diffeomorphic
42 Riyahi, Choi et al. Submitted to MICCAI 2018