Colloque IMT - 15/10/2019 - Healthcare 4.0 – « Quantification of idiopathic interstitial pneumonia: a unified Computer Vision and AI approach » - Présentation par Catalin Fetita

1. QUANTIFICATION OF
IDIOPATHIC INTERSTITIAL
PNEUMONIA:
A UNIFIED COMPUTER
VISION AND AI APPROACH
Catalin FETITA
ARTEMIS Department, TELECOM SudParis
Institut Mines-Telecom, Evry, France

2. Background: social impact of lung diseases
Lung diseases
Interstitial Lung
Disease (ILD)
Pneumonia /
Infections
COPD Cancer
UK deaths from IPF compared with other lung diseases
(2012). Source: https://statistics.blf.org.uk/pulmonary-fibrosis
 In 2012, 115043 people died
from lung disease (20% of all
deaths), among which 5292
from IIP (4.6% of deaths from
lung disease)
 New drugs become available
and need evaluation
 Very rare
 More than 300 different ILDs
• IIP (~55% of ILDs, >60 years)
• Sarcoidosis
• PICE
• HCL
• …
QUANTIFICATIONOFIIPS:AUNIFIED
COMPUTERVISIONANDAIAPPROACH
2
CLINICAL CONTEXT

3. • IIP CT scans biomarkers
investigation (consensus ATS
2011)
 Reticulation
 Traction Bronchiectasis
 Honeycombing
 Ground Glass
 Emphysema (tobacco)
CLINICAL CONTEXT
ILDs diagnosis and follow-up
 CT as reference modality for ILDs diagnosis and follow-up:
• High spatial resolution
• Contrast healthy vs. pathological tissue
QUANTIFICATIONOFIIPS:AUNIFIED
COMPUTERVISIONANDAIAPPROACH
3

4. CLINICAL CONTEXT
fibrosis (IIP) honeycombing (IIP)
CT examples
 Examples of CT biomarkers for infiltrative lung diseases (ILDs)
QUANTIFICATIONOFIIPS:AUNIFIED
COMPUTERVISIONANDAIAPPROACH
4

5. CLINICAL CONTEXT
CT examples
 Examples of CT biomarkers for infiltrative lung diseases (ILDs)
ground glass (IIP) emphysema
QUANTIFICATIONOFIIPS:AUNIFIED
COMPUTERVISIONANDAIAPPROACH
5

6. OBJECTIVE
Automated computer-aided diagnosis system
Develop an automated system for a quantitative follow-up of
idiopathic interstitial pneumonia (IIP) and emphysema using
volumetric CT
Input set analysis model biomarker output
• process the entire CT lung volume
• provide the lung region partitioning into defined phenotypes (classes)
in association with severity markers
• accurate and robust for inclusion as second reader in radiological workflow
QUANTIFICATIONOFIIPS:AUNIFIED
COMPUTERVISIONANDAIAPPROACH
6

7. IDIOPATHIC INTERSTITIAL PNEUMONIA (IIP)
 Materials and methods:
 IIP regions detection: a CNN-based approach
 vascular remodeling markers within IIP
 traction bronchiectasis (TB) markers as
confirmation of fibrosis
 Results and discussion
 Conclusion and future work
 Presentation overview
Image biomarkers with CT
QUANTIFICATIONOFIIPS:AUNIFIED
COMPUTERVISIONANDAIAPPROACH
7

8. IDIOPATHIC INTERSTITIAL PNEUMONIA (IIP)
 Materials and methods:
 IIP regions detection: a CNN-based approach
 vascular remodeling markers within IIP
 traction bronchiectasis (TB) markers as
confirmation of fibrosis
 Results and discussion
 Conclusion and future work
 Presentation overview
Image biomarkers with CT
QUANTIFICATIONOFIIPS:AUNIFIED
COMPUTERVISIONANDAIAPPROACH
8

9. I. CNN: choice of the architecture
Materials and Methods
(Anthimopoulos et al, IEEE TMI 2016)Extension (deeper) of T-CNN
Texture analysis
Input
pattern
16x16 8 Convolutional layers ( 2x2 kernels) + Leaky RELU
Average
Pooling
8x8
Fully connected
layers (DropOut +
RELU)
3 cls -
SoftMax
16@
15x15
36@
14x14
64@
13x13
100@
12x12
144@
11x11
196@
10x10
256@
9x9
324@
8x8
324@
1x1
864
288
3
16x16 patch to
classify central
pixel
x
N
E
IIP
DT-
CNN
QUANTIFICATIONOFIIPS:AUNIFIED
COMPUTERVISIONANDAIAPPROACH
9
 Patch-based CNN (sliding window)

10. I. CNN: choice of the architecture
Materials and Methods
(Anthimopoulos et al, IEEE TMI 2016)Extension (deeper) of T-CNN
Texture analysis
Input
pattern
16x16 8 Convolutional layers ( 2x2 kernels) + Leaky RELU
Average
Pooling
8x8
Fully connected
layers (DropOut +
RELU)
3 cls -
SoftMax
16@
15x15
36@
14x14
64@
13x13
100@
12x12
144@
11x11
196@
10x10
256@
9x9
324@
8x8
324@
1x1
864
288
3
QUANTIFICATIONOFIIPS:AUNIFIED
COMPUTERVISIONANDAIAPPROACH
10
 End-to-end CNN
 Patch-based CNN (sliding window)
U-Net

11.  Controlled acquisition protocol (Avicenne Hospital)
• similar convolution kernel (same image quality)
• pixel size in the range of 0.4 – 0.9 mm
• image resolution 512x512 or 768x768
• slice thickness in the range of 0.625 – 1.25 mm
• different manufacturers (Philips, Siemens, GE)
 1 expert radiologist and 3 junior radiologists to perform/validate
annotations (~10 images per case with GT annotation)
 130 cases with GT available, split at patient level in training (100),
validation (10), testing (20) sets
II. Experimental setup – Construction of the database
Materials and Methods
QUANTIFICATIONOFIIPS:AUNIFIED
COMPUTERVISIONANDAIAPPROACH
11

12.  Examples of annotated GT in the database
Emphysema-GGlass-Fibrosis Ground Glass Fibrosis
II. Experimental setup – Construction of the database
Materials and Methods
N E F IIPGG
QUANTIFICATIONOFIIPS:AUNIFIED
COMPUTERVISIONANDAIAPPROACH
12

13. III. Data preprocessing
 Bias introduced by normal high densities of vascular tree
Remove ambiguities normal vs. pathological tissue
 Removal of vascular tree opacities (local connected filtering)
Tarando et al, SPIE MI 2018
Materials and Methods
 Images rescaled to match 0.4 mm/pixel spacing (patch-based CNN)
QUANTIFICATIONOFIIPS:AUNIFIED
COMPUTERVISIONANDAIAPPROACH
13

14. III. Data preprocessing
 Bias introduced by normal high densities of vascular tree
Materials and Methods
Remove ambiguities normal vs. pathological tissue
 Removal of vascular tree opacities (local connected filtering)
QUANTIFICATIONOFIIPS:AUNIFIED
COMPUTERVISIONANDAIAPPROACH
14
 Images rescaled to match 0.4 mm/pixel spacing (patch-based CNN)
Tarando et al, SPIE MI 2018

15.  Creating training/validation datasets:
IV. Training and optimization
• From the training CT dataset 16x16 pixels half-overlapping patches were
extracted (falling 80% inside the marked ROIs) – for DT-CNN
Materials and Methods
QUANTIFICATIONOFIIPS:AUNIFIED
COMPUTERVISIONANDAIAPPROACH
15
• Data augmentation: horizontal flipping, rotations, random change
of the overall luminosity
• Class balance ensured during training
 Several loss functions tested: (weighted) cross-entropy vs Dice loss

16. IDIOPATHIC INTERSTITIAL PNEUMONIA (IIP)
 Materials and methods:
 IIP regions detection: a CNN-based approach
 vascular remodeling markers within IIP
 traction bronchiectasis (TB) markers as
confirmation of fibrosis
 Results and discussion
 Conclusion and future work
 Presentation overview
Image biomarkers with CT
QUANTIFICATIONOFIIPS:AUNIFIED
COMPUTERVISIONANDAIAPPROACH
16

17. 70 patients with fibrosing IIP from the database
Materials and Methods
Vascular remodeling assessment
Vascular segmentation exploits the multiresolution LCF used in CNN
preprocessing input
QUANTIFICATIONOFIIPS:AUNIFIED
COMPUTERVISIONANDAIAPPROACH
17
Kouvahe et al, SPIE MI 2019

18. 70 patients with fibrosing IIP from the database
Materials and Methods
Vascular remodeling assessment
QUANTIFICATIONOFIIPS:AUNIFIED
COMPUTERVISIONANDAIAPPROACH
18

19. Vascular remodeling estimated in terms of vessel caliber distribution
in the lung reported on medial axis of the structure
Materials and Methods
Vascular remodeling assessment
Large
calibers
Small
calibers
Caliber = granulometric definition
[Crosnier et al, SPIE MI’10]
cumulated distributions per range of 0.4 mm
(5 subjects)
vascular calibers medial axis
QUANTIFICATIONOFIIPS:AUNIFIED
COMPUTERVISIONANDAIAPPROACH
19

20. Materials and Methods
Vascular remodeling assessment
 Different scores computed from normalized caliber distribution:
 Normalization inter-patient with respect to the cube root of the lung volume
• area under the curve (AUC) for radii in different ranges: [0 - 2], [0 - 4],
[0 - 6], [2 - 4] , [2 - 6] mm
• variance, skewness, kurtosis and
the exponential decay constant
(EDC) from interpolated values
 Image scores confronted to most widely used prognostic markers:
• pulmonary function tests (DLCO, FVC, FEV1, TLC)
• GAP (Gender-Age-Physiology) and CPI (composite physiologic) index
Simple and multiple linear regression analyses also considered
Pearson correlation coefficient between quantitative variables
 Volume percent of pulmonary vessels (PPV)
QUANTIFICATIONOFIIPS:AUNIFIED
COMPUTERVISIONANDAIAPPROACH
20

21. IDIOPATHIC INTERSTITIAL PNEUMONIA (IIP)
 Materials and methods:
 IIP regions detection: a CNN-based approach
 vascular remodeling markers within IIP
 traction bronchiectasis (TB) markers as
confirmation of fibrosis
 Results and discussion
 Conclusion and future work
 Presentation overview
Image biomarkers with CT
QUANTIFICATIONOFIIPS:AUNIFIED
COMPUTERVISIONANDAIAPPROACH
21

22. TB are associated with the presence of fibrosis in IIP and their extent
reflects also the extent of fibrosis and a severity degree
Materials and Methods
Traction bronchiectasis assessment
 Distal bronchi have increased calibers and become visible at CT
 Lack of tapering for distal segments – potentially detectable by
monitoring airway caliber change from proximal to distal sites
[Fetita et al. 2009]
QUANTIFICATIONOFIIPS:AUNIFIED
COMPUTERVISIONANDAIAPPROACH
22

23. Exploit central axis to analyze caliber change along all paths in each lung lobe
Materials and Methods
Traction bronchiectasis assessment
Requires axis partitioning in lobes: robust algorithms considering
segments calibers and orientation, including anatomical variants
Overcomes the presence of spurious segments due to shape distortions
QUANTIFICATIONOFIIPS:AUNIFIED
COMPUTERVISIONANDAIAPPROACH
23

24. Exploit central axis to analyze caliber change along all paths in each lung lobe
Materials and Methods
Traction bronchiectasis assessment
Requires axis partitioning in lobes: robust algorithms considering
segments calibers and orientation, including anatomical variants
Overcomes the presence of spurious segments due to shape distortions
QUANTIFICATIONOFIIPS:AUNIFIED
COMPUTERVISIONANDAIAPPROACH
24

25. Exploit central axis to analyze caliber change along all paths in each lung lobe
Materials and Methods
Traction bronchiectasis assessment
Requires axis partitioning in lobes: robust algorithms considering
segments calibers and orientation, including anatomical variants
Overcomes the presence of spurious segments due to shape distortions
QUANTIFICATIONOFIIPS:AUNIFIED
COMPUTERVISIONANDAIAPPROACH
25

26. Airway (granulometric) calibers monitored for all possible paths in
each lobe and max-caliber curve extracted
Materials and Methods
Traction bronchiectasis assessment
Biomarkers evaluated:
 positive deviation between the max-caliber curve and theoretical 2-power-law
decrease evaluated at different starting points from lobar root
e.g. RLL
CalDev = Dev(0)
2/3CalDev = Dev(L/3)
1/2CalDev = Dev(L/2)
= -0.015 (heuristic setup)
𝐷𝑒𝑣 𝑥0 = 𝑑 𝑥 𝑑 𝑥 > 0, 𝑤𝑖𝑡ℎ 𝑑 𝑥 =
𝑟 𝑥
𝑟 𝑥0
− 2 𝛼(𝑥−𝑥0)
𝐿
𝑥=𝑥0
QUANTIFICATIONOFIIPS:AUNIFIED
COMPUTERVISIONANDAIAPPROACH
26

27. Study conducted on 18 subjects from the initial database
Materials and Methods
Traction bronchiectasis assessment
Biomarkers evaluated:
 positive deviation between the max-caliber curve and theoretical 2-power-law
decrease evaluated at different starting points from lobar root
 the lobar tree axis length, Nlen, normalized by the cube root of the half lung
volume to allow inter-patient analysis
 the lobar tree volume, Nvol, normalized by the half lung volume
Pearson correlation coefficient
between the computed scores and
the severity degree assigned by a
radiologist (-1= airway obstructions,
0=normal, 1=moderate TB,
2=severe TB, 2.5=very severe TB)
RMLB excluded from analysis
QUANTIFICATIONOFIIPS:AUNIFIED
COMPUTERVISIONANDAIAPPROACH
27

28. IDIOPATHIC INTERSTITIAL PNEUMONIA (IIP)
 Materials and methods:
 IIP regions detection: a CNN-based approach
 vascular remodeling markers within IIP
 traction bronchiectasis (TB) markers as
confirmation of fibrosis
 Results and discussion
 Conclusion and future work
 Presentation overview
Image biomarkers with CT
QUANTIFICATIONOFIIPS:AUNIFIED
COMPUTERVISIONANDAIAPPROACH
28

29. QUANTIFICATIONOFIIPS:AUNIFIED
COMPUTERVISIONANDAIAPPROACH
29Original DICOM slice Ground truth mask U-Net prediction Patch prediction
F1-
score
Patch U-Net
N 0.83 0.91
E 0.4 0.6
IIP 0.63 0.72
NEIIP
recall =
[𝑇𝑃]
𝑇𝑃 +[𝐹𝑁]
precision =
[𝑇𝑃]
𝑇𝑃 +[𝐹𝑃]
F1 = 2
𝑝𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 ∗ 𝑟𝑒𝑐𝑎𝑙𝑙
𝑝𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛+𝑟𝑒𝑐𝑎𝑙𝑙
Results
Pattern classification (N, E, IIP)
Quantitative evaluation on test dataset (20 patients)

30. Results
Vascular remodeling assessment
Two significant parameters reflecting the degree of vascular tree remodeling:
 PPV was moderately correlated to CPI significantly (r = 0.380; p = 0.003)
 AUC:2-6 was moderately correlated to CPI significantly (r = 0.395; p = 0.002)
AUC:2-6 was associated with CPI
in a multiple linear regression
model considering the age and the
extent of lesions in CT-scan as
potential confounders (p = 0.009)
AUC:2-6 could be a promising
biomarker of the severity of the
disease, independently of the
extent of lesions

31. Results
Traction bronchiectasis assessment
Several TB biomarkers showed moderate significant correlation with the TB
severity defined by the radiologist (72 samples -> 18 subjects x 4 lobes)
 Nlen (r= 0.575, p= 1.54E-7),
 Nvol (r= 0.523, p= 2,80E-6)
 2/3CalDev (r= 0.474, p= 2.89E-5)
2/3CalDev

32. Conclusion
 Severity diagnosis and management of IIP require concurrent
information from the lung structures affected, based on their
assessment with CT imaging
 Several biomarkers for IIP quantification proposed, associating:
• severity measures in relation with vascular and bronchial
remodeling, which show correlation with clinical parameters
• localization of the disease using lung texture classification
reaching on average for all three classes (N, E, IIP) 70.86%
sensitivity, 86.3% specificity, 86.4% accuracy
 Future work: extending the biomarker validation (mainly for TB) and
conduct a follow-up and a mortality study (vascular remodeling)
Quantitative assessment of IIPs

33. Quantification of idiopathic interstitial
pneumonia:
a unified Computer Vision and AI
approach
Catalin FETITA 1
Sebastian TARANDO 1
Simon Rennotte 1
Young-Wouk KIM 1,3
Pierre-Yves BRILLET 2,3
1 ARTEMIS Department, TELECOM SudParis
Institut Mines-Telecom, Evry, France
2 Université Paris13, Paris, France
3 Avicenne Hospital, AP-HP, Bobigny, France