Industrial pollution resulting in ozone layer depletion has influenced increased UV radiation in recent years which is a major environmental risk factor for invasive skin cancer Melanoma and other keratinocyte cancers. The incidence of deaths from Melanoma has risen worldwide in past two decades. Deep learning has been employed successfully for dermatologic diagnosis. In this work, we present a deep learning based scheme to automatically segment skin lesions and detect melanoma from dermoscopy images. U-Net was used for segmenting out the lesion from surrounding skin. The limitation of utilizing deep neural networks with limited medical data was solved with data augmentation and transfer learning. In our experiments, U-Net was used with spatial dropout to solve the problem of overfitting and different augmentation effects were applied on the training images to increase data samples. The model was evaluated on two different datasets. It achieved a mean dice score of 0.87 and a mean jaccard index of 0.80 on ISIC 2018 dataset. The trained model was assessed on PH² dataset where it achieved a mean dice score of 0.93 and a mean jaccard index of 0.87 with transfer learning. For classification of malignant melanoma, a DCNN-SVM model was used where we compared state of the art deep nets as feature extractors to find the applicability of transfer learning in dermatologic diagnosis domain. Our best model achieved a mean accuracy of 92% on PH² dataset. The findings of this study is expected to be useful in cancer diagnosis research. Published at IJCCI 2018. Source code available at https://github.com/zabir-nabil/lesion-segmentation-melanoma-tl

1. Zabir Al Nazi, Tasnim Azad Abir
Department of Electronics and Communication Engineering
Khulna University of Engineering and Technology
Khulna-9203, Bangladesh
International Joint Conference on Computational Intelligence (IJCCI 2018)
1

2. Agenda
 Introduction
 Objectives/Research Questions
 Methodology
 Results
 Conclusion and Future Work
2

3. Introduction
3
 Skin cancer is the most common form of cancer (US) fifth in World [1]
 2285 deaths from melanoma (UK)
 Mortality rate in skin cancer has increased +156% since 1970 [2]
 One person dies from melanoma each hour [3]
 Melanoma survival rate : 99% (early state) 63% (spreading to lymph
nodes) 20% (spreading to more parts of body) [4]
[1] American Academy of Dermatology
[2] Cancer Research UK
[3] skincancer.org
[4] cancer.net

4. Objectives/ Research Question (1/3)
4
Common Nevus Melanoma
[5] PH2 skin cancer dataset
 Can AI based CADx assist a dermatologist?

5. Objectives/Research Questions (2/3)
5
 Automated Skin Lesion Analysis Based on Color and Shape Geometry Feature Set for Melanoma Early
Detection and Prevention – Abuzaghleh et. al, 2014
 Automatic Lesion Detection System (ALDS) for Skin Cancer Classification Using SVM and Neural Classifiers –
Farooq et. al, 2016
 Advanced Skin Lesion Discrimination Pipeline for Early Melanoma Cancer Diagnosis towards PoC Devices –
Conoci et. al, 2017
 Evaluation Methodology between Globalization and Localization Features Approaches for Skin Cancer Lesions
Classification – Ahmed et. al, 2016
 An Overview of Melanoma Detection in Dermoscopy Images Using Image Processing and Machine Learning –
Mishra et. al, 2016
 A Novel Approach to Segment Skin Lesions in Dermoscopic Images Based on a Deformable Model – Zhen et. al,
IEEE journal of biomedical and health informatics, 2016
Approaches focused on Image Processing and Feature Engineering

6. Objectives/Research Questions (3/3)
6
 Using Deep Learning for Melanoma Detection in Dermoscopy Images – Salido et. al, International Journal of
Machine Learning and Computing 2018
Approaches focused on Deep Learning and Automatic Detection
AlexNet + Transfer Learning No contribution in Segmentation
Hair removal using inpainting PH2 dataset Accuracy: 85% (3 class)
 Deep CNN and Data Augmentation for Skin Lesion Classification – TC Pham et. al, Asian Conference on
Intelligent Information and Database Systems 2018
Inception V4 as feature extractor No contribution in Segmentation
Merging datasets + Augmentation SVM, RF, NN as classifier
 Automatic skin lesion analysis towards melanoma detection – Thao et. al, 2017 21st Asia Pacific Symposium on
Intelligent and Evolutionary Systems, IEEE
VGG 16 + Transfer Learning Conv-Deconv Net for Segmentation
ISBI dataset Dice score: 0.63 ISBI dataset Accuracy: 86.9% (2 class)

7. Methodology (1/3)
 PH2 - A dermoscopic image database for research and benchmarking [6]
[6] https://www.fc.up.pt/addi/ph2%20database.html
CommonNevus
AtypicalNevus
Melanoma
 767 X 576 X 3
 200 samples with binary segmentation mask
 3 classes
80 samples
40 samples
80 samples
7

8. Methodology (2/3)
 ISIC 2018: Skin Lesion Analysis Towards Melanoma Detection [7]
[7] https://challenge2018.isic-archive.com/
 1022 X 767 X 3
 2594 samples with binary segmentation mask
8

9. Methodology (3/3)
Figure: Proposed classification scheme for melanoma detection
9

10. Semantic Segmentation : Introduction
10
 Study on semantic segmentation
 Given an input image X with dimensions m by n. Classify each of the
m*n pixels into two classes (0/1)
Self driving car : SegNet Sample image and mask from PH2

11. Semantic Segmentation : Architecture (1/2)
11
Figure: Proposed architecture for end-to-end semantic lesion segmentation
Hyper-parameters
Performance metric [DCS, JC]
Loss function: dice loss (negative dice)
Activation: ReLU (sigmoid in last layer)
Epochs: 100
Batch size: 16
Optimizer: SGD
Learning rate: 0.001

12. Semantic Segmentation : Architecture (2/2)
12
Table: Architecture of proposed U-Net for lesion segmentation
Layer Feature
size No. of filters Layer Feature size No. of filters
input 224×224 conv_block 7 14×14 512
conv_block 1 224×224 32 upsampling 2 28×28 512
pooling 1 112×112 32 concatenate 2 28×28 768
conv_block 2 112×112 64 conv_block 8 28×28 256
pooling 2 56×56 64 upsampling 3 56×56 256
conv_block 3 56×56 128 concatenate 3 56×56 384
pooling 3 28×28 128 conv_block 9 56×56 128
conv_block 4 28×28 256 upsampling 4 112×112 128
pooling 4 14×14 256 concatenate 4 112×112 192
conv_block 5 14×14 512 conv_block 10 112×112 64
pooling 5 7×7 512 upsampling 5 224×224 64
conv_block 6 7×7 1024 concatenate 5 224×224 96
upsampling 1 14×14 1024 conv_block 11 224×224 32
concatenate 1 14×14 1536 conv 1×1 224×224 1

13. Semantic Segmentation : U-Net
13
 U-Net: Transfer Learning + Augmentation
𝐷𝐿 = −
2 ∗ 𝑛=1
𝑁
𝑝 𝑛 ∗ 𝑟𝑛 + 𝜀
𝑛=1
𝑁
𝑟𝑛 + 𝑝 𝑛 + 𝜀
Figure: Training of U-Net for semantic segmentation with augmented data

14. Semantic Segmentation : Result (1/5)
14
Figure: Sample augmented images from ISIC 2018 dataset
 Additional 2500 augmented images
 random rotation, random flip, random zoom, Gaussian distortion, random brightness, random color,
random contrast, random elastic distortion and histogram equalization

15. Semantic Segmentation : Result (2/5)
15
Figure: U-Net, Dropout U-Net (augmentation) training with dice loss

16. Semantic Segmentation : Result (3/5)
16
Figure: Jaccard index comparison for U-Net, Dropout U-Net (augmentation) training

17. Semantic Segmentation : Result (4/5)
17Table: Segmentation result, Figure: Binary masks predicted by U-Net, Dropout U-Net (augmentation)
Dataset Dice-coefficient
score Jaccard index
ISIC 2018 0.87±0.31 0.80±0.36
PH² 0.93±0.13 0.87±0.19

18. Semantic Segmentation : Result (5/5)
18Figure: Histogram analysis of Dice Coefficient Score and Jaccard’s Coefficient

19. Melanoma Detection : Architecture
19
Figure: Feature extraction with DenseNet
 Feature reuse
 For a dense block with 𝐿 layers, there are
𝐿(𝐿+1)
2
direct connections
 Transfer learning : DCNN as feature extractor

20. Melanoma Detection : Result (1/3)
20Table: Classification result
Feature extractor No. of features Classifier Accuracy
VGG-16 25088 SVM
Random Forest
Decision Tree
AdaBoost
Gradient boosting
0.89±0.11
0.86±0.10
0.82±0.12
0.89±0.13
0.91±0.13
VGG-19 25088 SVM
Random Forest
Decision Tree
AdaBoost
Gradient boosting
0.90±0.12
0.86±0.10
0.76±0.21
0.91±0.13
0.86±0.11
InceptionResNetV2 38400 SVM
Random Forest
Decision Tree
AdaBoost
Gradient boosting
0.90±0.11
0.82±0.10
0.82±0.17
0.88±0.10
0.88±0.09

21. Melanoma Detection : Result (2/3)
21Table: Classification result
Feature extractor No. of features Classifier Accuracy
ResNet50 2048 SVM
Random Forest
Decision Tree
AdaBoost
Gradient boosting
0.91±0.11
0.85±0.11
0.85±0.15
0.90±0.15
0.88±0.17
Xception 100352 SVM
Random Forest
Decision Tree
AdaBoost
Gradient boosting
0.89±0.10
0.86±0.12
0.79±0.12
0.86±0.12
0.82±0.17
InceptionV3 51200 SVM
Random Forest
Decision Tree
AdaBoost
Gradient boosting
0.89±0.12
0.81±0.10
0.78±0.18
0.86±0.15
0.83±0.12

22. Melanoma Detection : Result (3/3)
22
Table: Classification result
Feature extractor No. of features Classifier Accuracy
DenseNet201 94080 SVM
Random Forest
Decision Tree
AdaBoost
Gradient boosting
0.92±0.14
0.84±0.10
0.83±0.13
0.91±0.13
0.84±0.17

23. Conclusion and Future Work
 Deep learning based segmentation and melanoma detection with
high accuracy
 Generalizing with small amount of data : Transfer learning, data
augmentation, dropout
 New domains to explore transfer learning
 New augmentation techniques for regularization
 ONE-SHOT learning – Can we train our model with a single
example to detect melanoma?
23

24. 24
Source code available at: https://github.com/zabir-nabil/lesion-segmentation-melanoma-tl

25. 25