Deep Learning-Based Face Mask State Detection for COVID-19

1. Deep Learning Based Face Mask State Detection for
COVID-19
Yagya Raj Pandey, Ph.D. and Bhuwan Bhattarai
2021-03-08 1

2. Why it is important?
• World Health Organization (WHO) suggest to wear a face mask
in public places to minimize the spread of COVID-19.
• Local government made a rule of wearing mask in public
presence.
• Monitoring to rule is essential
– Human monitoring: Costly and tedious
– Machine: Economic and effective
2021-03-08 2

3. System design
• Detect and classify the person
– with_mask,
– with_out_mask and
– Wrong_wore_wask
• Alert if
– with_out_mask and
– Wrong_wore_wask
2021-03-08 3
Semi-automatic annotation tool

4. Deep neural networks
• Faster region-based convolutional neural network (FRCNN) [1]
• Feature pyramid network (FPN) [2]
• Cascade R-CNN (CRCNN) [3]
• Cascade FPN
2021-03-08 4
[1] Ren S, He K, Girshick R, Sun J. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 2017. vol. 39, no. 6, pp. 1137–1149. 10.1109/TPA
MI.2016.2577031. PMID: 27295650
[2] Lin TY, Dollar P, Girshick R, He K, Hariharan B, Belongie S. Feature pyramid networks for object detection. In CVPR. 2016. pp. 936–944. arXiv:1612.03144
[3] Cai Z, Vasconcelos N. Cascade R-CNN: Delving into high quality object detection. In CVPR. 2018, pp. 6154–6162. arXiv:1712.00726
Method Without mask With mask Mask weared incorr
ect
mean
AP 𝐹𝐹1 AP 𝐹𝐹1 AP 𝐹𝐹1 AP 𝐹𝐹1
FRCNN 0.861 0.854 0.906 0.911 0.976 0.968 0.914 0.911
FPN 0.836 0.856 0.925 0.919 0.968 0.973 0.910 0.916
CRCNN 0.837 0.851 0.859 0.886 0.972 0.975 0.889 0.904
CFPN 0.806 0.814 0.912 0.896 0.970 0.971 0.896 0.894
Method Without mask With mask Mask weared incor
rect
mean
AP 𝐹𝐹1 AP 𝐹𝐹1 AP 𝐹𝐹1 AP 𝐹𝐹1
FRCNN 0.818 0.845 0.899 0.886 0.965 0.963 0.891 0.898
FPN 0.834 0.839 0.923 0.917 0.985 0.973 0.914 0.910
CRCNN 0.859 0.866 0.863 0.876 0.967 0.968 0.896 0.904
CFPN 0.809 0.841 0.910 0.910 0.966 0.973 0.895 0.908
Experiments using
ResNet101 as a ba
ckbone
Experiments using
ResNet50 as a bac
kbone
Resnet101
Base network
ROI
Pooling
mask pro
posals
If cascade:
IoUnew > IoU0.5
Single/Multi-Layer Features
C5
C4
C3
C2
C1
Region proposal network
Softmax
Regressor
Fully
connected
Detected face mask

5. System attention
2021-03-08 5
Mask detection on input image Heatmap using P2 feature of FPN Heatmap using P3 feature of FPN Heatmap using P4 feature of FPN Heatmap using P5 feature of FPN
The heatmap visualization of detected area in pyramidal representation of FPN

6. Application
• Transportations:
– Airport, Bus station, train station
• Public places:
– School, Office, Crowded street
• Many more:
– Industrial device
– Self-control system
– Monitoring drone, Roberts
2021-03-08 6

7. THANKS