Transfer Learning for the Detection and Classification of traditional pneumonia and pneumonia induced by the COVID-19 from Chest X-ray Images

Problem Objectives Learning Methodology Dataset Simulation Results Conclusion References
Transfer Learning for the Detection and
Classification of traditional pneumonia and
pneumonia induced by the COVID-19 from Chest
X-ray Images
Yusuf Brima
Supervised by
Dr. Marcellin Atemkeng
Dr. Stive Roussel Tankio Djiokap
August 9, 2021

Outline
1 Problem
2 Objectives
3 Learning
4 Methodology
5 Dataset
6 Simulation
7 Results
8 Conclusion
9 References

Research Problem
Figure 1: Map for Coronavirus-related incidence rate across the globe reported
to Johns Hopkins University on June 17, 2021 (source: Johns Hopkins
University).

Research Problem
I There are various molecular and serologic assays to test Severe
Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2)

Research Problem
I There are various molecular and serologic assays to test the Severe
I Reverse Transcriptase-Polymerase Chain Reaction (RT–PCR) is the
laboratory standard for the SARS-CoV-2 testing.

Research Problem
I RT-PCR has very high falsely negative rate.

Research Problem
I RT-PCR has very high falsely negative rate.
I RT-PCR testing is very time-consuming. and presents a slew of
laboratory logistical challenges.

Research Problem
Figure 2: Map of Coronavirus-related confirmed deaths per 100,000 population
across the globe with a total of 3,861,121 as reported to Johns Hopkins
University on June 17, 2021 (source: Wikipedia).

Research Objectives
I To detect and classify traditional pneumonia and pneumonia induced
by the SARS-CoV-2 virus using Chest X-ray scans.

Research Objectives
I For safe, accurate, less cumbersome and timely diagnosis.

Research Objectives
I For safe, accurate, less cumbersome and timely diagnosis.
I Using Deep Transfer Learning

Learning Background
Figure 3: Hierarchy of Learning in Intelligent Machines.

Learning Background
UNKNOWN TARGET FUNCTION
TRAINING SAMPLES
LEARNING
ALGORITHM
HYPOTHESIS SET
FINAL HYPOTHESIS
Figure 4: A framework for supervised learning [1].

Learning Background
General Learning Setting
I A function estimation model such that we have a generator of
random vector x from a probability distribution P(x) which is
unknown.
I A process that maps the vector x to the output vector y according
to an unknown conditional probability distribution P(y|x)
P(y, x) = P(y|x)P(x). (1)
I Given a learning setting T ,
T := {H, P(Z), L} , (2)
where H ⊂ YX
, the hypotheses space of learnable models; P(Z) is
the probability measure of examples, that is:
Z := {(x1, y1), (x2, y2), . . . , (xm, ym)}

Learning Background
I L is the loss function such that:
L : H × Z → R
LCE = −
n
X
i=1
yi log(fθ(xi )),
Lmse =
1
2
n
X
i=1
(yi − fθ(xi ))
2
,
I Risk Functional R:
R(θ) =
Z
L(y, fθ(x))dP(x, y), ∀θ ∈ Θ. (3)

Learning Background
T is a learnable setting if the corresponding hypotheses space H is
learnable, if H is a VC dimension n, we therefore say T has a VC
dimension n.
f ∗
= min
fθ∈H
E[L(y, f (x))]
R̂m(θ) =
1
m
m
X
i=1
L(yi , fθ(xi )) (4)
The Empirical Risk Minimization (ERM) Induction Principle posits that
as m, the number of training samples gets larger,
R̂m(θ)
m→∞
= R(θ)

Learning Background
Generalization Error bound
The non-convexity of the loss objective makes deep learning a Hadamard
ill-posed problem. From a statistical learning theory standpoint, these
networks has a Generalization Error bound GE(θ) as stated thus:
GE(θ) = |R(θ) − R̂m(θ)|

Learning Background
Learning Representations and Task
Given K layers and an input vector x ∈ Rd
Let k = 1 . . . K and a non-linear activation function φ(.)
Thus, the transformation at layer k is: xk = φk (xk−1W k
) where
W k
= φk−1(xk−1W k−1
).
Generally, a deep neural network is:
Φ(x, W 1
, . . . , W K
) = φK (φK−1(. . . φ2(φ1(x, W 1
)W 2
) . . .)W K−1
)W K
),
φ(.) can be:
φ(x) = tanh(x),
φ(x) = max{0, x},
φ(x) =
1
1 + e−x
,
and many more.

Learning Background
Learning Representations and Task
Given {(xi , yi )}
N
i=1, xi ∈ Rd
and yi ∈ {0, 1} for a classification and
yi ∈ R for regression.
Therefore,
Φ∗
(W) = argmin
{W k }K
k=1
L(Y , Φ(x, W 1
, . . . , W K
)) + λΘ(W 1
, . . . , W K
),
where λ > 0, and
Θ(W) =
K
X
k=1
||W k
||2
.

Learning Background
Inbalance class-weighting
Wj =
m
knj
, (5)
Time-based learning rate decay
ηt+1 =
ηt
1 + ρet
, (6)
Stochastic Gradient Descent with Momentum
vt+1 ← ρvt + ∇θL(θ)
θj ← θj − ηvt+1
(7)

Transfer Learning
Formal Definition
D := {X, P(X)}
X = {x1, x2, x3, . . . , xn}, ∀xi ∈ X
Formal Definition
For domain D, a task is defined as:
T := {Y, P(Y |X)}
Y = {y1, y2, y3, . . . , yn}, ∀yi ∈ Y

Transfer Learning
Formal Definition
Therefore
DS := {XS , P(XS )}
XS = {xS1
, xS2
, xS3
, . . . , xSn
}, ∀xSi
∈ XS
YS = {yS1
, yS2
, yS3
, . . . , ySn
}, ∀ySi
∈ YS
DT := {XT , P(XT )}
XT = {xT1
, xT2
, xT3
, . . . , xTn
}, ∀xTi
∈ XT
YT = {yT1
, yT2
, yT3
, . . . , yTn
}, ∀yTi
∈ YT

Transfer Learning
The goal of Transfer Learning
Given
f : X 7→ Y where f ∈ H
∼
X = argmin
fθ∈H
{L(f (XSi
) 6= YSi
)}
And
RDT
:= P(η(XT ) 6= yT |
∼
X)
f ∗
= argmin
f ∈H
{RDT
(fθ(XT ), YT ,
∼
X)}

Transfer Learning
Input Layer ℝ¹²
∈ Hidden Layer ℝ¹²
∈ Output Layer ℝ⁴
∈
Output
Input
Conv-1
Conv-2
Conv-3
Conv-
4
...
Conv-n
Standard Convolutional Neural Network Architecture
1
2
...
2
n
Figure 5: Standard Convolutional Neural Network.

Transfer Learning
Convolution Operation for 2D
s(i, j) = (K ∗ I)(i, j) =
X
m
X
n
I(i + m, j + n)K(m, n),
Convolution Dimension
O =
W − K + 2P
S
+ 1.

Transfer Learning
Source
domain
ImageNet
Input Layer ℝ¹²
∈ Hidden Layer ℝ¹²
∈ Output Layer ℝ⁴
∈
conv-1
conv-2
conv-3
...
conv-n
conv-1
conv-2
conv-3
...
conv-n
Target Domain
Chest X-ray
Output
1
2
3
4
Figure 6: Proposed network architecture.

Transfer Learning
Data Augmentation
zooming/flipping
/rotation etc
CNN feature
extraction layers
of ResNet50
Training the
Dense layers 1
2
3
4
Output
Input Covid-19
Lung
Opacity
Normal
(Healthy)
Viral
Pneumonia
Figure 7: The schematic represents the proposed system model.

Transfer Learning
Data Augmentation
zooming/flipping
/rotation etc
CNN feature
extraction layers
of ResNet50
Training the
Dense layers 1
2
3
4
Output
Input Covid-19
Lung
Opacity
Normal
(Healthy)
Viral
Pneumonia
Figure 8: Deep Transfer Learning Stages

Dataset
Normal Lung_Opacity COVID Viral Pneumonia
Class Type - Diagnosis
48.2%
28.4%
17.1%
6.4%
Number of Sample X-Ray Images per Class
Figure 9: A histogram of the distribution of the X-Ray Images per Class. The
total dataset is 18,865.

Dataset
0 1 2 3 4 5 6
Min
0.0
0.2
0.4
0.6
0.8
Density
Images Colour Min Value Distribution by Class
Class
Lung_Opacity
Viral Pneumonia
Normal
COVID
(a)
0 50 100 150 200 250
Mean
0.000
0.002
0.004
0.006
0.008
0.010
Density
Image Color Mean Value Distribution by Class
Class
Lung_Opacity
Viral Pneumonia
Normal
COVID
(b)
160 180 200 220 240 260
Max
0.000
0.005
0.010
0.015
0.020
0.025
0.030
0.035
0.040
Density
Images Colour Max Value Distribution by Class
Class
Lung_Opacity
Viral Pneumonia
Normal
COVID
(c)
Figure 10: We present the min, mean and max RGB color intensity
distributions for the four X-ray image classes.

Dataset
Figure 11: X-ray image format where the upper right zoomed illustration
indicates the RGB color channels.

Dataset
Formal Definition
x =
1
Ic IhIw
Ic
X
i
Ih
X
j
Iw
X
k
xijk (8)
where Ic is the number of color channels, Ih is the height of the image
and Iw is the width of the image.
σ =
v
u
u
u
t
1
Ic IhIw
Ic IhIw
X
i


Ic
X
j
Ih
X
k
Iw
X
l
xjkl − x


2
(9)

Dataset
COVID
Lung_Opacity
Normal
Viral Pneumonia
(a)
COVID
Lung_Opacity
Normal
Viral Pneumonia
(b)
Figure 12: A comparison illustrating a plot of the 3 colors channels (Left plot)
and single channel in (Right plot).

Dataset
25 50 75 100 125 150 175 200 225
ImageChannelColourMean
20
40
60
80
100
Image
Channel
Colour
Standard
Deviation
MeanandStandardDeviationofImageSamples
Class
Lung_Opacity
ViralPneumonia
Normal
COVID
(a)
25 50 75 100 125 150 175 200 225
ImageChannelColourMean
20
40
60
80
100
Image
Channel
Colour
Standard
Deviation
MeanandStandardDeviationofImageSamples-10%ofData
(b)
Figure 13: A side-by-side comparison of the dataset clusters using image mean
and standard deviation.

Dataset
25 50 75 100 125 150 175 200 225
Mean
20
40
60
80
100
Standard
Deviation
Lung_Opacity
25 50 75 100 125 150 175 200 225
Mean
ViralPneumonia
25 50 75 100 125 150 175 200 225
Mean
Normal
25 50 75 100 125 150 175 200 225
Mean
COVID
Figure 14: Individual class distributions for COVID-19 (far Left) to Healthy
(normal case, far Right). From the graph, Normal (healthy) and Lung Opacity
images have a similar cluster formation and pixel intensity distribution.

Simulation Environment
I NVIDIA 537K80s, T4s, P4s and P100s Graphic Processing Unit
(GPU)
I Keras API (TensorFlow)
I Google Colaboratory (Colab) (Python 3.8x kernel)

Results
Metrics
Accuracy =
TP
TP + FP + TN + FN
Sensitivity (r/sn) =
TP
TP + FN
Specificity (sp) =
TN
TN + FP
Precision (p) =
TP
TP + FP
F1 Score =
2
1
r + 1
p
= 2

rp
r + p

FPR =
FP
FP + TN
= 1 − sp

Results
0 20 40 60 80 100
Epoch
0.0
0.2
0.4
0.6
0.8
loss
VGG19 Loss
Train loss
Validation loss
(a)
0 20 40 60 80 100
Epoch
0.70
0.75
0.80
0.85
0.90
0.95
1.00
Accuracy
VGG19 Accuracy
Train accuracy
Validation accuracy
(b)
Figure 15: VGG-19 model was trained for 100 epochs.

Results
Model Correct classification Incorrect classification
VGG-19 1988 127
DenseNet-121 1972 143
ResNet-50 1985 130
Table 1: A summary of total images classified correctly and incorrectly by
VGG-19, DenseNet-121, and ResNet-50 using a total test dataset of 2,115
images. Amongst the three models, VGG-19 demonstrated high accuracy of
XCR image classification with only 127 misclassifications.

Results
COVID Lung Opacity Normal Viral Pneumonia
Predicted Label
COVID
Lung Opacity
Normal
Viral Pneumonia
True
Label
333 10 18 0
0 548 53 0
2 32 983 2
0 6 10 124
0
200
400
600
800
(a)
0.0 0.2 0.4 0.6 0.8 1.0
False Positive Rate
0.0
0.2
0.4
0.6
0.8
1.0
True
Positive
Rate
ROC curve
ROC curve of Viral Pneumonia (area = 0.998)
ROC curve of Lung_Opacity (area = 0.986)
ROC curve of Normal (area = 0.982)
ROC curve of COVID (area = 1.000)
random guessing
(b)
Figure 16: VGG-19 Test Confusion Matrix and Receiver Operator Characteristic
Curve.

Results
0 20 40 60 80 100
Epoch
0.0
0.2
0.4
0.6
0.8
loss
Densenet121 Loss
Train loss
Validation loss
(a)
0 20 40 60 80 100
Epoch
0.70
0.75
0.80
0.85
0.90
0.95
1.00
Accuracy
Densenet121 Accuracy
Train accuracy
Validation accuracy
(b)
Figure 17: A DenseNet-121 model was trained for 100 epochs.

Results
Predicted Label
COVID
Lung Opacity
Normal
Viral Pneumonia
True
Label
321 15 24 1
1 539 61 0
0 32 987 0
1 0 8 125
0
200
400
600
800
(a)
0.0 0.2 0.4 0.6 0.8 1.0
False Positive Rate
0.0
0.2
0.4
0.6
0.8
1.0
True
Positive
Rate
ROC curve
random guessing
(b)
Figure 18: (DenseNet-121 Test Confusion Matrix and ROC curve.

Results
0 20 40 60 80 100
Epoch
0
1
2
3
4
5
Loss
ResNet50 Loss
Train loss
Validation loss
(a)
0 20 40 60 80 100
Epoch
0.2
0.4
0.6
0.8
1.0
Accuracy
ResNet50 Accuracy
Train accuracy
Validation accuracy
(b)
Figure 19: ResNet50 model was trained for 100 epochs.

Results
Predicted Label
COVID
Lung Opacity
Normal
Viral Pneumonia
True
Label
337 9 15 0
0 533 67 1
2 28 988 1
0 0 7 127
0
200
400
600
800
(a)
0.0 0.2 0.4 0.6 0.8 1.0
False Positive Rate
0.0
0.2
0.4
0.6
0.8
1.0
True
Positive
Rate
ROC curve
random guessing
(b)
Figure 20: ResNet-50 Confusion Matrix and ROC curve.

Results
References Dataset description Method Accuracy
[2]
5,184 chest X-ray images
that comprised 184 COVID-19 cases and 5000 normal cases
ResNet18 + ResNet50 +SqueezeNet
+ DenseNet-121
98%
[3]
18,567 X-ray images (COVID-19 = 140, normal = 8851
and Pneumonia = 9576)
ResNet-101 + ResNet-152 96.1%
[4] 320 images (COVID-19 = 160 and normal = 160)
Transfer learning with CNN
networks (Inceptionv3 and ResNet50)
99.01%
[5] 6926 images (COVID-19 = 2589 and normal = 4337) CNN 94.43%
[6] 5090 chest X-ray images (COVID-19 = 1979 and normal = 3111)
Fusion features (CNN+HOG)
+ VGG19 pre-train model
99.43%
Proposed
COVID-19 = 3616, Normal= 10192 ,
Lung Opacity = 6012, and Viral Pneumonia = 1345 images
ResNet-50V2
DenseNet-121
VGG-19
93.80%
93.24%
94.0%
Table 2: Comparative survey of literature results.

Results
Figure 21: Activation map of ResNet-50 layer 48 before fine-tuning.

Results
Figure 22: Activation map of ResNet-50 layer 48 after fine-tuning.

Conclusion
I RT-PCR is error-prone and less accurate
I COVID-19 detection from Chest X-ray Images is a promising
diagnostic method.
I It is a fast, accurate and feasible solution especially for
asymptomatic carriers.

References
[1] Y. S. Abu-Mostafa, M. Magdon-Ismail, and H.-T. Lin, Learning
from data. AMLBook New York, NY, USA: 2012, vol. 4.
[2] S. Minaee, R. Kafieh, M. Sonka, S. Yazdani, and G. J. Soufi,
“Deep-covid: Predicting covid-19 from chest x-ray images using deep
transfer learning,” Medical image analysis, vol. 65, p. 101 794, 2020.
[3] N. Wang, H. Liu, and C. Xu, “Deep learning for the detection of
covid-19 using transfer learning and model integration,” in 2020
IEEE 10th International Conference on Electronics Information and
Emergency Communication (ICEIEC), IEEE, 2020, pp. 281–284.
[4] H. Benbrahim, H. Hachimi, and A. Amine, “Deep transfer learning
with apache spark to detect covid-19 in chest x-ray images,”
Romanian Journal of Information Science and Technology, vol. 23,
S117–S129, 2020.
[5] L. Duran-Lopez, J. P. Dominguez-Morales, J. Corral-Jaime,
S. Vicente-Diaz, and A. Linares-Barranco, “Covid-xnet: A custom
deep learning system to diagnose and locate covid-19 in chest x-ray
images,” Applied Sciences, vol. 10, no. 16, p. 5683, 2020.

References
[6] M. Ahsan, M. Based, J. Haider, M. Kowalski, et al., “Covid-19
detection from chest x-ray images using feature fusion and deep
learning,” Sensors, vol. 21, no. 4, p. 1480, 2021.

End
Thank You!

Transfer Learning for the Detection and Classification of traditional pneumonia and pneumonia induced by the COVID-19 from Chest X-ray Images

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Transfer Learning for the Detection and Classification of traditional pneumonia and pneumonia induced by the COVID-19 from Chest X-ray Images

Similar to Transfer Learning for the Detection and Classification of traditional pneumonia and pneumonia induced by the COVID-19 from Chest X-ray Images (20)

More from Yusuf Brima

More from Yusuf Brima (7)

Recently uploaded

Recently uploaded (20)

Transfer Learning for the Detection and Classification of traditional pneumonia and pneumonia induced by the COVID-19 from Chest X-ray Images