My master's degree thesis concerned the use of deep learning tecniques for lesion detection and cancer classification in Breast DCE-MRI

1. Does Registration improve DCE-MRI analisys in the Deep Learning Era?
Scuola Politecnica e delle Scienze di Base
Corso di Laurea Magistrale in Ingegneria Informatica
tesi di laurea magistrale
Relatore
Ch.mo Prof. Carlo Sansone
Correlatori
Ing. Gabriele Piantadosi, Ph.D.
Ing. Stefano Marrone
Candidato
Maurizio Gentile
Matr. M63/648
Does Registration improve DCE-MRI analisys in the Deep Learning Era?
Anno Accademico 2017/18

2. Does Registration improve DCE-MRI analisys in the Deep Learning Era?
Scuola Politecnica e delle Scienze di Base
Corso di Laurea Magistrale in Ingegneria Informatica
Context
Breast Cancer Analisys via Dynamic Contrast-Enhanced Magnetic Resonance (DCE-MRI)
Computer Aided Detection and Diagnosis (CAD) systems
Deep Convolution Neural Networks (CNNs)
Contribution
Analisys of the Image Registration impact on deep learning based CAD systems
Comparison of eight Image Registration techniques
Evaluation over four deep learning approaches
One model for lesion Segmentation
Three models for lesion Classification

3. Does Registration improve DCE-MRI analisys in the Deep Learning Era?
Scuola Politecnica e delle Scienze di Base
Corso di Laurea Magistrale in Ingegneria Informatica
Neoplasia (tumor) indicates irregular
growth of cells due to neo-angiogenesis
Malignant tumors are called cancers
Breast cancer is one of the most
common type of cancer among women,
the second after skin cancer. 1
Breast Cancer
266,120 new cases of women breast
cancer are estimated for 2018 only in USA1
Of these, the estimated deaths
are 41.400
1. JEMAL, Ahmedin, et al. Cancer statistics, 2008. CA: a cancer journal for clinicians, 2008, 58.2: 71-96

4. Does Registration improve DCE-MRI analisys in the Deep Learning Era?
Scuola Politecnica e delle Scienze di Base
Corso di Laurea Magistrale in Ingegneria Informatica
Dynamic Contrast-Enhanced Magnetic Risonance
DCE-MRI
Uses magnetic fields and radio waves for
body organs image acquisition
Able to analyze the temporal enhancement
pattern of a tissue due to the flowing of a
paramagnetic contrast agent (CA)
Allows to obtain information that can not be
obtained by other imaging techniques:
Time intensity Curve (TIC)
Healty tissue/lesion discrimination
Non invasive and painless, safe to use
and strongly operator indipendent.
4D volumes:
A tridimensional volume for each
acquisition time (one pre-contrast and
others post-contrast)

5. Does Registration improve DCE-MRI analisys in the Deep Learning Era?
Scuola Politecnica e delle Scienze di Base
Corso di Laurea Magistrale in Ingegneria Informatica
Computer Aided Detection and Diagnosis systems
Medical image analysis is generally difficult
Noisy images
Often textured in complex ways
Object of interest have complex shape
Signs of clinical interest are subtle
Support physicians in the diagnostic task
Two of the most important phases are:
Lesion Detection (Segmentation): identifying a
tumor lesion
Malignancy Diagnosis (Classification): assesting
the benignity or malignancy of a tumor lesion
Volume
Extraction
Pre-Processing
Lesion
Detection
Lesion
Diagnosis
Therapy
Assessment

6. Does Registration improve DCE-MRI analisys in the Deep Learning Era?
Scuola Politecnica e delle Scienze di Base
Corso di Laurea Magistrale in Ingegneria Informatica
DCE-MR Image Registration
Image Registration
Allines two or more images of the same subject
taken at different times, from different viewpoints,
and/or by different sensors.
Used in DCE-MRI to reduce motion artefacts (such
as those due to breathing)
Several studies have shown registration effectivness on classical approaches:
Impacts on enhancement curve extimation 2
Improvements on ROC curve of a logistic regression based CAD 3
2 A. Hill, et al., Engineering in medicine and biology society, 2006. Dynamic Breast MRI: Image Registration and its impact on enhancement curve extimation
3 C.Tanner, et al., Biomedical Imaging: Nano to Macro, 2006. Does registration improve performance of a CAD system for DCE MR Mammography?
It’s aim is to realline each voxel of each post
contrast acquisition time with the
correspondent pre-contrast one.
It works trying to optimize a similarity index
between the images

7. Does Registration improve DCE-MRI analisys in the Deep Learning Era?
Scuola Politecnica e delle Scienze di Base
Corso di Laurea Magistrale in Ingegneria Informatica
Deep learning
Cascade of multiple layers of non
linear processing units for feature
extraction and transformation
Hierarchy of concepts: multiple levels
of representations learning
Automatic feature engineering
Deep Learning & CNN
Is still Registration
useful in this case?

8. Does Registration improve DCE-MRI analisys in the Deep Learning Era?
Scuola Politecnica e delle Scienze di Base
Corso di Laurea Magistrale in Ingegneria Informatica
Lesion Detection
Segmentation via U-Net
Deep CNN 4
5 Layers Depth
Batch Normalization
Validation Set for best
weights selection
Evaluation:
Sensitivity, specificity
Dice Similarity Index
4 G. Piantadosi et al, Breast Segmentation in mri via u-net deep convolution neural network, IN 2018 ICPR

9. Does Registration improve DCE-MRI analisys in the Deep Learning Era?
Scuola Politecnica e delle Scienze di Base
Corso di Laurea Magistrale in Ingegneria Informatica
Diagnosis approaches
Lesion Classification
Mixture Ensemble of CNN 5
Evaluation:
Slice Voting
ROC AUC
5 R. Rasti et al, Breast Cancer diagnosis in dce mri usign mixture ensemble of CNN, Pattern Recognition 2017
6 C. Haarburger et al, Transfer Learning for Breast Cancer Malignancy on DCE-MRI
7 S. Marrone et al, An investigation of deeep learning for lesion malignancy classification in breast dce-mri, Springer 2017
Fine Tuning via ResNet 6
Transfer Learning via AlexNet 7

10. Does Registration improve DCE-MRI analisys in the Deep Learning Era?
Scuola Politecnica e delle Scienze di Base
Corso di Laurea Magistrale in Ingegneria Informatica
Experimental Setup
Volumes Registration in Matlab
Medx3, Medx5 (simple filtering approaches)
Rueckert (Both affine and non rigid, FFD, B-spline,
Mutual Information)
Elastix (B-spline cubic, Mutual Information)
MIRT (Non rigid, B-spline, Mutual Information)
Matlab image registration Tool
(Affine, Intensity-based, multi-modal
config., Mutual Information)
Kim et al.8 (Hierarchical alignment
by group-wise, registration using
Reuckert)
PASCALE DATASET
Patients 35 women (age range 16-69)
Weighting T1
Mode
1.5 (Magneton Symphony Siemens
Medical System)
TR/TE 8.9/4.76
Flip Angle 25 deg
Acquisition Time 56 s
Dose 72,11%±16,33%
Injection Flow Rate 2 ml/s
Time Point 1 pre + 9 post
Gold Standard: ROI defined by an experienced radiologist;
histopatologically proved lesions (14 benign - 21 malignant)
8. Kim et al., 2012, Hierarchical alignment of breast dce-mr images by group-wise registration and robust features matching, Medical physics, 39:353-366

11. Does Registration improve DCE-MRI analisys in the Deep Learning Era?
Scuola Politecnica e delle Scienze di Base
Corso di Laurea Magistrale in Ingegneria Informatica
Detection Results
Original Ground
Truth
No Reg
Medx5
Kim et al.
Medx3
Rueckert Elastix MIRT Matlab
No Reg Medx3 Medx5 Rueckert Elastix MIRT Matlab Kim et. al
SENS
MEAN 50,63% 55,49% 44,57% 52,19% 51,86% 48,84% 53,03% 37,83%
MEDIAN 58,39% 65,22% 45,45% 66,71% 65,26% 48,23% 59,89% 30,19%
SPEC
MEAN 99,96% 99,98% 99,96% 99,97% 99,97% 99,97% 99,95% 99,96%
MEDIAN 100,00% 100,00% 100,00% 100,00% 100,00% 100,00% 100,00% 99,98%
DICE
MEAN 46,53% 51,78% 42,17% 49,59% 47,53% 47,42% 49,24% 33,77%
MEDIAN 47,34% 55,26% 48,18% 55,72% 54,51% 58,00% 49,95% 29,45%

12. Does Registration improve DCE-MRI analisys in the Deep Learning Era?
Scuola Politecnica e delle Scienze di Base
Corso di Laurea Magistrale in Ingegneria Informatica
Diagnosis Results
No Reg Medx3 Medx5 Rueckert Elastix MIRT Matlab Kim et. al
Rasti et al.* 71,43% 68,48% 60,88% 70,07% 70,41% 67,69% 72,11% 70,75%
Haarburger et al* 77,89% 71,77% 69,05% 79,93% 79,93% 76,87% 76,19% 76,19%
Marrone et al.* 88,78% 80,1% 79,59% 85,71% 88,44% 84,18% 85,54% 86,74%
Piantadosi et al. 9 76,35% 79,7% - - - - - -
Lavasani et al. 10 65,31% 72,11% 68,37% 72,45% 72,79% 70,07% - -
*Best voting performances reported for
each technique
9 G. Piantadosi et al, LBP-TOP for volume lesion classification in breast DCE-MRI, IN 2015 ICIAP
10 S. Lavasani et al, Discrimination of malignant suspicious breast tumors based on semi-quantitative DCE-MRI parameters employing SVM, FBT 2015

13. Does Registration improve DCE-MRI analisys in the Deep Learning Era?
Scuola Politecnica e delle Scienze di Base
Corso di Laurea Magistrale in Ingegneria Informatica
Conclusions
Future works
Involve more patients and different datasets in order to achieve a more reliable
statistical evaluation
Consider other deep learning approaches and registration techniques to wider
demonstrate whether registration affect the DCE-MRI analisys
Our results show that Image Registration can still bring some noticeable effects to
lesion detection task by CNNs
On the other hand, tumor diagnosis with CNNs seems to be more invariant to
Registration
A possible interpretation is that CNNs are able to learn Motion invariant features
However, simple registration approaches (such as Medx3 and Medx5) are not
proved as effective as advanced ones in lesion diagnosis by CNNs