The document discusses image processing techniques for lung cancer screening and treatment. It covers topics like lung segmentation, nodule detection, computer-aided diagnosis, image-guided radiotherapy, and quantitative assessment of tumor response. Lung segmentation is used to isolate the lungs from other organs in CT images. Nodule detection algorithms then aim to find potential cancerous nodules. Computer-aided diagnosis systems analyze extracted features of nodules to determine if they are malignant or benign. Image-guided radiotherapy utilizes 4D CT and gating to account for tumor motion during treatment. Quantitative metrics like standardized uptake value are used to assess tumor response in PET imaging.

1. Image Processing in Lung Cancer
Screening and Treatment
Wookjin Choi, PhD

2. Introduction
Lung cancer is the leading cause of cancer deaths.
Most patients diagnosed with lung cancer already have advanced
disease
40% are stage IV and 30% are III
The current five-year survival rate is only 16%
(a) male (b) female
Trends in death rates for selected cancers, United States, 1930-2008 2

3. Lung cancer screening and treatment
Lung Cancer
Screening
• Nodule detection
and diagnosis
• Biopsy
• …
Lung Cancer
Treatment
• Surgery
• Chemotherapy
• Radiotherapy
• Tumor Response
• …
Computer-Aided De
tection (CADe) and
Diagnosis (CADx)
Image-guided Radiot
herapy and Quantitat
ive Assessment
3

4. Image processing in Lung cancer screening and
treatment
Feature Extraction and Analysis
Nodule (tumor) segmentation
Lung segmentation
Image Registration
4

5. Lung Segmentation
• Thresholding
– Fixed threshold
– Optimal threshold
– 3-D adaptive fuzzy thresholding
• Lung region extraction
– 3-D connectivity with seed point
– 3-D connected component
labeling
• Contour correction
– Morphological dilation
– Rolling ball algorithm
– Chain code representation
5

6. Lung Segmentation
6

7. Lung CAD
COMPUTER-AIDED
DETECTION AND DIAGNOSIS
7

8. Detecting Lung Cancer is hard
8

9. Where is t
he nodule
?
9

10. Where is t
he lung ca
ncer?
10

11. Where is t
he lung ca
ncer?
11

12. Computer Aided Detection
12

13. Pulmonary Nodule Detection CAD system
CAD systems Lung segmentation Nodule Candidate Detection False Positive Reduction
Suzuki et al.(2003)[3] Thresholding Multiple thresholding MTANN
Rubin et al.(2005)[4] Thresholding Surface normal overlap
Lantern transform and rule-
based classifier
Dehmeshki et al.(2007)[5] Adaptive thresholding Shape-based GATM Rule-based filtering
Suarez-Cuenca et al.(2009)[6]
Thresholding and 3-D
connected component
labeling
3-D iris filtering
Multiple rule-based LDA
classifier
Golosio et al.(2009)[7] Isosurface-triangulation Multiple thresholding Neural network
Ye et al.(2009)[8]
3-D adaptive fuzzy
segmentation
Shape based detection
Rule-based filtering and
weighted SVM classifier
Sousa et al.(2010)[9] Region growing Structure extraction SVM classifier
Messay et al.(2010)[10]
Thresholding and 3-D
connected component
labeling
Multiple thresholding and
morphological opening
Fisher linear discriminant and
quadratic classifier
Riccardi et al.(2011)[11] Iterative thresholding
3-D fast radial filtering and
scale space analysis
Zernike MIP classification
based on SVM
Cascio et al.(2012)[12] Region growing Mass-spring model
Double-threshold cut and
neural network 13

14. Genetic Programming based Classifier
Wook-Jin Choi, Tae-Sun Choi, “Genetic programming-based feature transform and classification for the automatic detection of pulmonary nodules on
computed tomography images”, Information Sciences, Vol. 212, pp. 57-78, December 2012, doi: http://dx.doi.org/10.1016/j.ins.2012.05.008
Feature spaces for four types of features
2-D geometric feature 3-D geometric feature
2-D intensity-based statistical feature 3-D intensity-based statistical feature
Genetic programming classifier learning
Classification space
GP based classification expression in tree shape
Optimal multi-thresholding based Nodule candidates de
tection.
14

15. Hierarchical Block-image Analysis
Wook-Jin Choi, Tae-Sun Choi, “Automated Pulmonary Nodule Detection System in Computed Tomography Images: A Hierarchical Block
Classification Approach”, Entropy, Vol. 15, No. 2, pp. 507-523, February 2013, doi:http://dx.doi.org/10.3390/e15020507
ROC curves of the SVM classifiers with respect to three different kernel functions,
SVM-r: radial basis function, SVM-p: polynomial function, and SVM-m:
Minkowski distance function; (a) p = 0:25 and (b) p = 1.
Result images after block splitting with respect to various block sizes
The entropy histograms of block-images for five different block sizes
(x-axis : entropy value, y-axis : number of blocks, (a) 32, (b) 24, (c) 16, (d) 12, and (e) 8)
15

16. θ φ
θ φ
Three-dimensional Shape-based Feature Descriptor
Wook-Jin Choi, Tae-Sun Choi, “Automated Pulmonary Nodule Detection based on Three-dimensional Shape-based Feature Descriptor”, Computer
Methods and Programs in Biomedicine, Vol. 113, No. 1, January 2014, pp. 37–54, doi: http://dx.doi.org/10.1016/j.cmpb.2013.08.015
Surface saliency weighted surface
normal vectors
Two angular histograms of the
surface normal vectors
θ φ
ROC curves of the SVM classifiers with respect to three different kernel
functions, SVM-r: radial basis function, SVM-p: polynomial function,
and SVM-m: Minkowski distance function; (a) p = 0:25 and (b) p = 1.
FROC curves of the proposed CAD system with
respect to three different dimensions of AHSN
features
θ φ
θ φ
Feature optimization with wall detection a
nd elimination algorithm
3D shape-based feature descriptor
High surface saliency
value
Low surface saliency
value
16

17. Comparative Analysis
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 2 4 6 8 10 12 14 16
Overallsensitivity
False positives per scan
Suzuki et al. (2003)
Dehmeshki et al. (2007)
Suarez-Cuenca et al.
(2009)
Golosio et al. (2009)
Ye et al. (2009)
Messay et al. (2010)
Riccardi et al. (2011)
Cascio et al. (2012)
Choi et al. (2012)
Choi et al. (2013)
Choi et al. (2014)
17

18. Computer Aided Diagnosis
Once the lung nodules are detected and segmented from the
corresponding chest images
The next task is to determine whether the detected nodules
are malignant or benign
The malignancy of lung nodules correlates highly with
Geometrical size
Shape
Appearance descriptors
Ayman El-Baz, Garth M. Beache, Georgy Gimel'farb, et al., “Computer-Aided Diagnosis Systems for Lung Cancer: Challenges and Metho
dologies,” International Journal of Biomedical Imaging, vol. 2013, Article ID 942353, 46 pages, 2013. doi:10.1155/2013/942353 18

19. Computer Aided Diagnosis
Typical computer-aided diagnosis (CAD) system for lung cancer. The input of a CAD system i
s the medical images obtained using an appropriate modality. A lung segmentation step is used
to reduce the search space for lung nodules. Nodule detection is used to identify the locations
of lung nodules. The detected nodules are segmented. Then, a candidate set of features, such a
s volume, shape, and/or appearance features, are extracted and used for diagnosis.
Ayman El-Baz, Garth M. Beache, Georgy Gimel'farb, et al., “Computer-Aided Diagnosis Systems for Lung Cancer: Challenges and Meth
odologies,” International Journal of Biomedical Imaging, vol. 2013, Article ID 942353, 46 pages, 2013. doi:10.1155/2013/942353 19

20. Computer Aided Diagnosis
Ayman El-Baz, Garth M. Beache, Georgy Gimel'farb, et al., “Computer-Aided Diagnosis Systems for Lung Cancer: Challenges and Meth
odologies,” International Journal of Biomedical Imaging, vol. 2013, Article ID 942353, 46 pages, 2013. doi:10.1155/2013/942353
Study Purpose Method Database Observations
Jennings
et al.
To retrospectively
determine the distribution
of stage I lung cancer
growth rates with serial
volumetric CT
Volumetric
measurement
149
patients
At serial volumetric CT measurements, there was wide
variability in growth rates. Some biopsy-proved cancers
decreased in volume between examinations
Zheng et
al.
To simultaneously segment
and register lung and
tumor in serial CT data
Nonrigid
transformation
on lung
deformation and
rigid structure
on the tumor
6 volumes
of 3
patients,
83 nodules
The mean error of boundary distances between automatic
segmented boundaries of lung tumors and manual
segmentation is 3.50 pixels. The mean and variance of
percentages of the nodule volume variations caused by
errors in segmentation are 0.8 and 0.6
Marchian
ò et al.
To assess in vivo
volumetric repeatability of
an automated algorithm for
volume estimation.
Semiautomatic
volumetric
measurement
101
subjects,
233
nodules
The 95% confidence interval for difference in measured
volumes was in the range of ±27%. About 70% of
measurements had a relative difference in nodule volume
of less than 10%
El-Baz et
al.
To monitor the
development of lung
nodules
Global and local
registration, GR
volumetric
measurement
135 LDCT
from 27
subjects,
27 nodules
All the 27 nodules are correctly classified based on GR
measurements over 12 months
Growth-rate-based methodologies for following up pulmonary nodules.
20

21. Computer Aided Diagnosis
Ayman El-Baz, Garth M. Beache, Georgy Gimel'farb, et al., “Computer-Aided Diagnosis Systems for Lung Cancer: Challenges and Meth
odologies,” International Journal of Biomedical Imaging, vol. 2013, Article ID 942353, 46 pages, 2013. doi:10.1155/2013/942353
Study Purpose Method Database Observations
Suzuki et al.
To classify nodules into
Benign or Malignant
Multiple MTANNs using
pixel values in
a subregion
Thick-slice (10 mm) screening
LDCT scans of
76 M and 413 Bnodules
= 0.88 in a leave-one-
out test
Iwano et al.
To classify nodules into
benign or malignant
LDA based on nodule's
circularity and second
moment features
HRCT (0.5–1 mm slice) scans of
52 Mand 55 B nodules
Sensitivity of 76.9% and
a specificity of 80%
Way et al.
To classify nodules into
benign or malignant
LDA or SVM with
stepwise feature selection
CT scans of 124 Mand
132 B nodules in 152 patients
= 0.857 in a leave-one-
out test
Chen et al.
To classify nodules into
benign or malignant
ANN ensemble
CT scans (slice thickness of 2.5
or 5 mm) of 19 M and
13B nodules
= 0.915 in a leave-one-
out test
Lee et al.
To classify nodules into
benign or malignant
GA-based feature
selection and a random
subspace method
Thick-slice (5 mm) CT scans of
62 M and 63 B nodules
= 0.889 in a leave-one-
out test
El-Baz et al.
To classify nodules into
benign or malignant
Analysis of the spatial
distribution of the nodule
Hounsfield values
CT scans (2 mm slice) of
51 M and 58 B nodules
Sensitivity of 92.3% and
a specificity of 96.6%
Classification between malignant (M) and benign (B) nodules based on shape and appearance features.
21

22. Lung CAD
IMAGE-GUIDED RADIATION
THERAPY
22

23. Image-Guided Radiotherapy
Gupta T, Narayan C A, Image-guided radiation therapy: Physician's perspectives, J Med Phys
Tumor Segmen
tation
Registration
Feature Extracti
on and Analysis
Registration
Registration
Tumor Segmen
tation
23

24. Radiotherapy and lung cancer
The efficacy and safety of RT reflect the interplay
between
• Total dose delivered to the malignant tumor
• The rate of dose delivery (daily fractionation)
• The volume (and type) of tumor-bearing organ irradiated.
• The intrinsic tolerance of the tissue irradiated
24

25. 4D CT
Excellent for taking into account
respiratory motion
Takes a set of CT images and
sorts them to represent each
phase of the breathing cycle
Box with infrared reflectors on
abdomen, set up infrared camera
to capture movement

26. Why is 4D CT important?
Same slice in different
phases of the breathing
cycle showing tumor
movement
26

27. Is 4DCT Worthwhile?
Underberg, R.W.M., Lagerwaard, F.J., Cuijpers, J.P., Slotman, B.J., Van Sornsen de Koste, J.R.,Senan, S. (2004). Four-Dimensional CT Scan
s for Treatment Planning in Stereotactic Radiotherapy for Stage 1 Lung Cancer, International Journal of Radiation Oncology Biology Physics 27

28. Gating
• Utilize 4DCT scan to get brea
thing pattern
• Determine a phase of the bre
athing cycle to treat during, p
lan on that scan Only
• Monitor treatment with respi
ratory motion
• when patient’s breathing
enters the selected part o
f the breathing cycle, tre
atment is delivered
Varian RPM system
28

29. Stereotactic body radiotherapy (SBRT)
 Modeled after brain radiosurgery principles
• Multiple convergent beams
• Rigid patient immobilization
• Precise localization via stereotactic coordinate system
• Single fraction treatment
• Size-restriction for target
29

30. Anatomic Tumor Response Assessment in CT or MRI
Imaging as Surrogate for
Survival or progression-free survival
Response rate, time to tumor progression
RECIST criteria based on longest diameter
Complete response (CR): disappear
Partial response (PR): ≥ 50% decrease
Stable disease (SD): others
Progressive disease (PD): ≥ 25% increase or new
Tumor size change does not occur or does not occur early in
some effective treatments
30

31. Metabolic Tumor Response Assessment in FDG-PET
Strong correlation between FDG uptake and cancer cell
number
Metabolic (functional) change may occur earlier and more
markedly than tumor size change
Qualitative evaluation plus semi-quantitative assessment
with SUV or SUL
31

32. Wahl, J Nucl Med. 50(Suppl 1): 122S–150S.
Large decline in SUL (-41%) despite stable pancreatic mass anatomically (a
rrows)  Partial metabolic response.
PET/CT for Tumor Response: An Example in Pancre
atic Tumor
32

33. Qualitative (Visual) PET Response Evaluation
Distribution and intensity of FDG uptake in tumor are
compared with uptake in normal tissues
Changes are visually evaluated
Requires clinical experience, disease patterns
Performs well in conversion of markedly positive PET scan to totally
negative scan
Moderate inter-observer variation
Difficult to detect small changes
33

34. Semi-Quantitative PET Response Assessment
SUV is most widely used
SUL is more consistent across patients
ROI selection
Maximal pixel: SUVmax, not as reproducible
Manual contour
Small fixed region ~1 cm3: SUVpeak
Fixed percentage isocontour: 40%, 50%
Fixed threshold: SUV = 2.5
3×SD above background (typically liver)
34

35. PERCIST Criteria
SULpeak of the hottest tumor
PERCIST criteria
CMR: normalize to background level
PMR: ≥ 30% decrease and ≥ 0.8 unit in SUL
SMR: others
PMD: ≥ 30% increase and ≥ 0.8 unit in SUL or visible increase in extent
of uptake (75% in TLG) with no decline in SUL, or new FDG-avid lesion
35

36. FDG Uptake Shows Spatial Variation
Belhassen and Zaidi 2010. Med Phys
Zhao, et al. 2005. J Nucl Med
36

37. Quantitative PET/CT analysis framework
Extract spatial-temporal image features:
Intensity distribution (histogram)
Spatial variations (texture)
Geometric properties (shape, structure)
Temporal changes due to therapy
Construct response models using machine learning
techniques with multiple features
Feature selection
Support vector machine
Cross-validation
37

38. • Region growing
• Morphology filter
• Multi-modality im
age segmentation
• ITK
• Intensity distribution
• Spatial variations
• Geometric properties
• > 100 features for each
tumor
• ITK
• ROC analyses
• Tumor response
• Survival
• Matlab
Tumor
Segmentation
Image
Registration
Feature
Extraction
Predicting
Ability
• Multi-level rigid
• Pre/Post-CT
• ITK
Extracting Spatial-Temporal FDG-PET Features for
Tumor Response Evaluation
38

39. Registration
Article Type of
Registration Abnormality Treatme
nt Scanning Time
Aristophanous
et al. Rigid NSCLC RT Before and after
treatment
Necib et al. Rigid Metastatic
colorectal cancer CTx Before and after
treatment
Tan et al. Rigid Esophageal Cancer CRT Before and after
treatment
Vera et al. Rigid Esophageal cancer CRT Before and during
treatment
Cannon et al. Deformable
(Demons)
Head and neck
cancer
RT or
CRT
Before and after
treatment
Roels et al. Deformable
(B-Spline) Rectal Cancer CRT Before, during and
after treatment
van Velden et al. Deformable
Advanced
colorectal
carcinoma
CTx Before and after
treatment
PET/CT based tumor response assessment studies using rigid registration, deformable registration or the combinatio
n of rigid and deformable registration algorithms. 39

40. Tumor Segmentation
Tumor segmentation can be performed either manually by
physicians or (semi-)automatically using image analysis
tools.
The accuracy of a tumor segmentation method is often hard
to evaluate in patients due to the lack of ground truth.
In response evaluation that involves two or more serial image
studies
The reproducibility of a segmentation method is as important as its
accuracy
40

41. Multi-modality adaptive region-growing (MARG)
A sharp volume increase occurred at an f where the region
just grows into the background (normal tissue)
f was identified by fitting the curve and calculating curvature
Tumor A
rea
Background Area
f
41

42. MARG: Results on a NSCLC Dataset from AAPM
TG211
Pathologic tumor v
olume
MARG results50% threshold res
ult
• For 10 patients, MARG (Dice = 0.69), slightly higher accuracy than thresh
olding methods (Dice = 0.67)
• Accuracy limited by the reliability of 3D pathologic tumor volume reconstr
uction and its alignment with PET/CT images 42

43. Spatial-Temporal FDG-PET Features for Predicting
Pathologic Tumor Response
A new SUV intensity feature - Skewness
pre-CRT
• Top: responder, more skewed (fewer hig
her SUVs)
• Bottom: non-responder, less skewed (mo
re higher SUVs)
Three texture features post-CRT – Inertia, Co
rrelation, and Cluster Prominence
• Top: responder, homogeneous FDG uptake p
ost-CRT
• Bottom: non-responder, heterogeneous FDG
uptake post-CRT
43

44. Accuracy of Individual Spatial-Temporal FDG-PET
Features
44

45. FDG-PET Histogram Distances for Predicting
Pathologic Tumor Response
• A responder shows larger histogram dista
nce from pre-CRT to post-CRT
• A non-responder shows smaller histogra
m distance
45

46. Accuracy of Individual Histogram Distances
7 bin-to-bin and 7 cross-bin histogram distances have high
er AUCs than conventional PET/CT response measures
46

47. Modeling Tumor Response – Model Construction
and Evaluation with Cross-Validation
47

48. Results
20 patients with esophageal cancer. Model predicts
pathologic response to chemoradiotherapy (CRT)
SVM model with 17 selected features from all feature groups:
AUC = 1.0, sensitivity = 100%, specificity = 100%
Models with conventional PET/CT response measures or
clinical parameters: AUCs < 0.75
48

49. Conclusions
Image processing in Lung cancer screening and
treatment
Computer aided detection and diagnosis
• Lung segmentation
• Nodule detection and segmentation
• Feature extraction and analysis
Image-guided radiotherapy
• Registration
– CT/CT, PET/CT
• Tumor segmentation
• Feature extraction and analysis
49

50. Future works
Validate the accuracy of image registration and tumor segmentation
methods
The usefulness of image features, and the generalizability of
response models
often developed on small retrospective datasets in large retrospective and
prospective datasets.
Clinic and biologic interpretation of the advanced PET/CT image
features
For physicians and biologists
Challenges for implementing the quantitative PET/CT image
analysis for tumor response evaluation
Delineating the tumor volume in multi-modality (PET/CT) images
Identifying a few features that truly capture biological changes correlated with
tumor response for a specific disease and therapy
Validating the results in large, multi-center patient datasets, vendor
implementation and ultimately clinic acceptance

51. Thank you