In this paper we present a novel technique for characterizing and classifying 3D textured volumes belonging to different lung tissues in 3D CT images.We build a volume based 3D descripton, robust to changes of size, rigid spatial transformations and texture variability, thanks to the integration of Riesz-wavelet features within a Covariance-based descriptor formulation. 3D Riesz features characterize the morphology of tissue density thanks to their response to changes in intensity in CT images. These features are encoded in a Covariancebased descripton formulation: this provides a compact and flexible representation thanks to the use of feature variations rather than dense features themselves, and adds robustness to spatial changes. Furthermore, the particular symmetric definite positive matrix form of these descriptors causes them to lay in a Riemannian manifold. Thus, descriptors can be compared with analytical measures, and accurate techniques from Machine Learning and clustering can be adapted to their spatial domain. Additionally we present a classification model following a “Bag of Covariance Descriptors” paradigm in order to distinguish three different nodule tissue types in CT: solid, ground-glass opacity (GGO), and healthy. Classification accuracy is estimated based on an acquired dataset of 95 patients with manually delineated ground truth by radiology specialists in 3D. The promising outcomes of the presented method support a future aim for automated lung nodule detection and computerized diagnosis assistance applications.

1. Introduction
Methodology
Experimental evaluation
Conclusions
3D Riesz–wavelet Based Covariance Descriptors
for Texture Classification of
Lung Nodule Tissue in CT
Pol Cirujeda , Henning M¨uller†, Daniel Rubin ,
Todd A. Aguilera , Billy W. Loo Jr. ,
Maximilian Diehn , Xavier Binefa , Adrien Depeursinge†,‡
† ‡
SaAT6.6
August 29th, 2015
1 / 18

2. Introduction
Methodology
Experimental evaluation
Conclusions
Motivation / Contribution
Related Work
Motivation + Contribution
(Statistical) feature extraction and representation
Dictionary modelling of lung/nodule tissue areas
Classification of lung nodule areas
(solid / ground glass opacity -GGO / healthy)
2 / 18

3. Introduction
Methodology
Experimental evaluation
Conclusions
Motivation / Contribution
Related Work
Motivation + Contribution
(Statistical) feature extraction and representation
Dictionary modelling of lung/nodule tissue areas
Classification of lung nodule areas
(solid / ground glass opacity -GGO / healthy)
Goal and application
Supervised learning of region classes from texture
Robustness to size and shape variations
Applications: tissue modelling, classification, segmentation
3 / 18

4. Introduction
Methodology
Experimental evaluation
Conclusions
Motivation / Contribution
Related Work
Related work
Clinical domain
3D visualization and annotation software
Manual delineation with expertise of clinicians
4 / 18

5. Introduction
Methodology
Experimental evaluation
Conclusions
Motivation / Contribution
Related Work
Related work
Clinical domain
3D visualization and annotation software
Manual delineation with expertise of clinicians
Computer vision + Machine Learning domain
3D features: Riesz transform, 1st and 2nd order visual cues
3D descriptors: MCOV, 3D-SIFT, SHOT, THRIFT...
Linear/non-linear (un)supervised classification methods:
CNN, Kernel-SVMs, Sparse coding, Bag-of-visual features...
5 / 18

6. Introduction
Methodology
Experimental evaluation
Conclusions
Intuition
Riesz-Covariance descriptors
Data and patients
Classification
Intuition - features
Riesz-wavelet transform as texture features 1
Figure 1: Lung nodule CT slice with corresponding Riesz filter responses
1A. Depeursinge et al., ”Lung Texture Classification Using Locally–Oriented Riesz
Components”, in MICCAI 2011
6 / 18

7. Introduction
Methodology
Experimental evaluation
Conclusions
Intuition
Riesz-Covariance descriptors
Data and patients
Classification
Intuition - feature representation (in 3D)
3D Riesz-covariance models
Figure 2: 3D CT volume and associated Riesz-covariance descriptor
7 / 18

8. Introduction
Methodology
Experimental evaluation
Conclusions
Intuition
Riesz-Covariance descriptors
Data and patients
Classification
3D Riesz-Covariance descriptors
x y
z
v
Φ(ct, v) = {φx,y,z, ∀x, y, z ∈ v} (1)
φx,y,z = R
(n1,n2,n3)
x,y,z , R x,y,z, ctx,y,z (2)
RieszCov (Φ(ct, v)) =
1
N − 1
N
i=1
(φx,y,z − µφ)) (φx,y,z − µφ))T
,
(3)
8 / 18

9. Introduction
Methodology
Experimental evaluation
Conclusions
Intuition
Riesz-Covariance descriptors
Data and patients
Classification
Covariance model benefits
Common framework for statistical data modelling.
Features ≡ samples of n−dim joint distributions.
Second order moment statistics (n × n covariance matrices).
Covariances manifold (Sym+
d ) ⇒ analytical modelling
Riemannian space ported machine learning techniques.
9 / 18

10. Introduction
Methodology
Experimental evaluation
Conclusions
Intuition
Riesz-Covariance descriptors
Data and patients
Classification
Covariance descriptors - Sym+
d Riemannian space
logId
TId
x = logY (X) = Y
1
2 log Y −1
2 XY −1
2 Y
1
2 (4)
ˆx = vect(x) = (x1,1, x1,2, ..., x1,d , x2,2, x2,3, ..., xd,d ) (5)
δ(X, Y ) = Trace log X−1
2 YX−1
2 (6)
10 / 18

11. Introduction
Methodology
Experimental evaluation
Conclusions
Intuition
Riesz-Covariance descriptors
Data and patients
Classification
Data gathering
Ground-truth:
95 patients (from Stanford Hospital and Clinics)
Biopsy-proven early stage non-small cell lung carcinoma
Nodule regions delineated in CTs by clinicians
Processing with MATLAB software:
isotropic voxels of 0.8 mm3
11 / 18

12. Introduction
Methodology
Experimental evaluation
Conclusions
Intuition
Riesz-Covariance descriptors
Data and patients
Classification
Data samples
GGO vs. Solid lung nodule tissue components (vs. healthy lung)
Figure 3: Lung nodule Riesz filter
responses - GGO component
Figure 4: Lung nodule Riesz filter
responses - solid component
12 / 18

13. Introduction
Methodology
Experimental evaluation
Conclusions
Intuition
Riesz-Covariance descriptors
Data and patients
Classification
Bag-of-covariances
Standard bag-of-visual features paradigm.
Sub-sampling of partial class regions for a complete dictionary
modelling
13 / 18

14. Introduction
Methodology
Experimental evaluation
Conclusions
Intuition
Riesz-Covariance descriptors
Data and patients
Classification
Bag-of-covariances
Dictionary D ≡ ˆx c
v,p = vect(logId
(RieszCovC
V ,P))
Training set: modelling frequencies of words in D
Classification decision: class(ct) = argmini D(hct, hi )
14 / 18

15. Introduction
Methodology
Experimental evaluation
Conclusions
Experimental setup
Experimental setup
Data sets:
35 patients for model learning, 60 for test set.
60 words per class (dictionary size, 3 × 60 × 35 = 3600)
10-fold cross-validation.
Quantitative evaluation w.r.t. ground-truth:
Avg. sensitivity (TP / TP+FN) = 82.2%
Avg. specificity (TN / TN+FP) = 86.2%
15 / 18

16. Introduction
Methodology
Experimental evaluation
Conclusions
Remarks
Future work
Conclusions
Computer vision and Machine Learning to the service of
medical knowledge
Statistical design for a robust solution
Easily extendible framework
16 / 18

17. Introduction
Methodology
Experimental evaluation
Conclusions
Remarks
Future work
Future work
More patients and bigger data collection
Different modelling contexts for concrete problems
Exploit the covariance-based descriptor space for different
Machine Learning techniques (clustering, classification,
regression...)
17 / 18

18. Introduction
Methodology
Experimental evaluation
Conclusions
Remarks
Future work
Thanks for your attention
Questions?
3D Riesz–wavelet Based Covariance Descriptors
for Texture Classification of
Lung Nodule Tissue in CT
SaAT6.6
Pol Cirujeda, UPF
EMBC 2015, August 29th 2015
18 / 18