In order to raise awareness towards pandemics such as the COVID-19, where mask-wearing is necessary, a mask-recognition program using artificial intelligence and Python libraries such as NumPy and pandas was created to tackle goals such as being used as an analyzing tool for calculating the percentage of masks and implemented in security cameras in public buildings such as offices.
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Are You Social Distancing? A Computational Approach
1. By Aparnaa Senthilnathan, High School
Senior, Bronx Science
AreYouSocial
Distancing?A
Computational
Approach
2. “We’ve actually invested very little in a
system to stop an epidemic...The problem
wasn’t that there was a system that didn’t
work well enough. The problem was that we
didn’t have a system at all”
—BillGates
”
4. Research Goals
Successfully apply mask
recognition to videos
Use ML algorithms to influence effective
measures to stopping the pandemic
Use as an analytical tool for calculating
the percentage of masks
Implement the program as a
security measure in buildings
5. If CNN models are trained on a few mask and no mask
datasets and then tested on separate images and
videos… then:
• It will correctly identify masks and an alarm will
sound that will remind non-mask wearers.
• This program will correctly predict whether people
are six feet or more apart.
• This method will be more effective in raising
awareness towards pandemics and will be ready
for scientists and researchers to use the program
as both an analytical tool and a security measure
in buildings.
Hypothesis
7. ● Neural Networks
● Used to train the CNN model to
identify items in images
● Consists of four layers: convolutional,
pooling, fully connected and softmax.
● These layers are ordered by how they
learn filters of increasing complexity.
CNNModels
8. • Use machine learning concepts to create a program to help encourage more people
to follow the guidelines necessary for Coronavirus prevention
• Attempt to implement mask recognition algorithms into video recordings.
Firsttimes?
9. Materials
The program was coded using
Python and its libraries. A few
mask and no mask datasets
from Kaggle and Github were
download.
10. TSC is the number of test
samples the model was able to
correctly identify and the TS is
the total number of test samples
received by the model.
The loss is just calculated by
subtracting the accuracy from 1,
where 1 is the maximum
percentage of accuracy
Accuracy Loss
Equations
A(TSC,TS)
= TSC/TS
L(A) = 1 - A
11. P is the percentage of people
wearing masks, M is the number
of masks identified, and NP is the
number of people in the frame.
Percentageofmasks
Equations(cont.)
P = M/NP
12. TheProgram’s mainMenuModule
Take a Picture
using the
Computer
Open a
Webcam Video
on your
Computer
Upload an
Image
Upload a Video
01 02 03 04
Exit
05
13. Demo
The photo sent by the user
Percentage of mask-
wearers over a total
number of people
A green border indicates
the person is wearing a
mask
If the person isn’t wearing
the mask, then the border
should be red.
15. Table1:TakeaPhoto
Trials(MaskPercentage/#ofPeoplerecorded/TotalFacesshown)
Photo Type Trial 1 Trial 2 Trial 3
Single Person (Mask) 0%/1(plastic mask) 100%/1 (pink
mask)
100%/1 (black
mask)
Single Person (No-Mask) 0%/1 0%/1 0%/1
Two People (Both Masks) 50%/2 100%/2 50%/2 (-wrong
position)
Two People (One Mask) 50%/2 (wrong face) 100%/2 (person
without a mask
counted)
50%/2
Two People (No Masks) 0%/2/2 0%/2/2 0%/2/2
Group of People (All Masks) 66.7%/2/3 100%/3/3 33.3%/1/3
Group of People (Some Masks) 66.7%/3/3 66.7%/3/3 33.3%/1/3
Group of People (No Masks) 33.3%/3/3 (one non-mask wearer counted) 33.3%/3/3 (one
non-mask wearer
counted)
33.3%/3/3 (one
non-mask wearer
counted)
16. Table2:UploadaPhoto
Trials(MaskPercentage/DistanceAccuracy/#ofPeoplerecorded/Total
Facesshown)
Photo Type Trial 1 Trial 2 Trial 3
Single Person (Mask) 100%/100%/2 (pillow counted/1) 100%/100%/1/1 100%/100%/1/1
Single Person (No-Mask) 0%/100%/1/2 0%/100%/1/2 0%/100%/1/2
Two People (Both Masks) 100%/100%/2/2 100%/100%/2/2 100%/100%/2/2
Two People (One Mask) 100%/100%/1/2 0%/0%/0/2 50%/100%/2/2
Two People (No Masks) 100%/100%/2/2 50%/100%/2/2 50%/100%/2/2
Group of People (All Masks) 0%/100%/2/4 75%/50%/8/13
(girl’s shoulder
counted)
86.66%/100%/15
/50
Group of People (Some Masks) 100%/0%/5/20 (a non-mask wearer counted) 53.85%/100%/13
/13 (phone
counted)
70%/100%/28/30
(few non-mask
wearers counted)
Group of People (No Masks) 0%/0%/10/10 0%/0%/9/9 0%/0%/6/7
17. Table3:Webcam
Trials(MaskPercentage/PeopleIdentified/Numberof
People)
Photo Type Trial 1
Single Person (Mask) 0%/1/1
Single Person (No-Mask) 100%/1/1
Two People (Both Masks) 50%/2/2
Two People (One Mask) 50%/2/2
Two People (No Masks) 0%/2/2
Group of People (All Masks) 33.3%/1/3
Group of People (Some Masks) 66.7%/2/3
Group of People (No Masks) 0%/3/3
18. Table4:Upload aVideo
VideoName Observations
How to wear your single use face mask? (short version) Before the girl puts on her mask, the program identifies that
her face has a mask on. As the girl was putting on her mask,
the camera turned 360 degrees in slow motion, which the
program wasn’t able to identify her until her face was shown
in front view. The program was able to identify her with the
mask on correctly, but without the mask, the program still
identifies her as her wearing the mask.
Group of People Putting on Masks Three people are recorded in front view by the camera, and the
program was able to classify which of the three people are
wearing masks as well as the distance between them for the
most part.
20. Observations
Photos
● There were mixed results
● The program was able to identify and classify masks correctly for the most part.
● The program also calculated the percentage of masks within the image correctly.
● In some cases, the program can’t identify other faces due to lighting conditions or the person is wearing
the mask
Videos
● There also were mixed results
● The program can identify and classify the face properly when the subject is facing the camera.
Otherwise, it is hard for the program to identify the face.
● The program also calculated the percentage of masks within the video correctly.
● The alarm is correctly initiated when the program identified a non-mask wearer in the webcam video.
21. Mask recognition can be used
successfully in videos to an
extent
For the most part, the program
can serve as an analytical tool
because it calculates the
percentage of masks correctly.
VideoRecognition AnalyticalTool
Discoveries
22. The program allowed the pre-
trained model to identify an
object as a face, despite a low
confidence score.
If the video size is large, the
video playback would slow down
a lot due to the excessively large
amount of data the model has to
read.
Wrongface Videoprocessing
Limitations
No mask
23. • Find ways to improve the machine learning
algorithms such that the program can process
videos in original sizes and allow the program to
identify the right faces in the same video.
• Can be expanded further by training the model to
identify whether or not the person is wearing the
mask correctly and is it the right type of mask.
• Introduce future methodology that allow
scientists to jump further into raising awareness
towards the coronavirus pandemic.
Future RESEarch
24. • All icons and images are created by Flaticon and Freepick. The screenshots of
code/demo are from my laptop. The presentation template was created by
Slidesgo.
• Special thanks to Dr. Sarah Ostadabbas of Northeastern University and members
from the MIT Beaver Works Summer Institute for mentoring and helping me with
CNN Networks and resources necessary for the project.
• Also, a big thanks to Dr. Vladimir Shapovalov for guiding me through the research
process
Acknowledgements
25. 1. The University of Texas at Austin. "Coronavirus spreads quickly and sometimes before people have symptoms, study finds." ScienceDaily. ScienceDaily, 16 March 2020.
<www.sciencedaily.com/releases/2020/03/200316143313.htm>.
2. Bai, Nina. “How the New Coronavirus Spreads and Progresses – And Why One Test May Not Be Enough.” How the New Coronavirus Spreads and Progresses – And Why One Test May Not Be
Enough | UC San Francisco, 10 Nov. 2020, www.ucsf.edu/news/2020/02/416671/how-new-coronavirus-spreads-and-progresses-and-why-one-test-may-not-be-enough.
3. The New York Times. “Covid World Map: Tracking the Global Outbreak.” The New York Times, The New York Times, 28 Jan. 2020, www.nytimes.com/interactive/2020/world/coronavirus-
maps.html?pageType=LegacyCollection.
4. Rabin, Roni Caryn. “Will Protests Set Off a Second Viral Wave?” The New York Times, The New York Times, 31 May 2020, www.nytimes.com/2020/05/31/health/protests-coronavirus.html.
5. Sault, Samantha, Writer. “'We're Not Ready for the next Epidemic,' Bill Gates Warned – Five Years Ago.” World Economic Forum, www.weforum.org/agenda/2020/03/bill-gates-epidemic-
pandemic-preparedness-ebola-covid-19/.
6. Stewart, Matthew, PhD Researcher. “Simple Introduction to Convolutional Neural Networks.” Medium, Towards Data Science, 29 July 2020, towardsdatascience.com/simple-introduction-
to-convolutional-neural-networks-cdf8d3077bac.
7. Mueller, John Paul. “Distinguishing Classification Tasks with Convolutional Neural Networks.” Dummies, 2 January 2019, www.dummies.com/programming/big-data/data-
science/distinguishing-classification-tasks-with-convolutional-neural-networks/.
8. American Enterprise Institute. “COVID-19: Documenting Changes in Public Opinion.” American Enterprise Institute, 2020, www.jstor.org/stable/resrep25364. Accessed 18 Oct. 2020.
9. Padan, Carmit. “Coronavirus: On Crisis, Emergency, and the Power of Words." Institute for National Security Studies, 2020, www.jstor.org/stable/resrep25527. Accessed 18 Oct. 2020.
10. Dennis, Brady, et al. “With Coronavirus Cases Spiking Nationwide, All Signs Point to a Harrowing Autumn.” The Washington Post, WP Company, 11 Nov. 2020,
www.washingtonpost.com/health/with-coronavirus-cases-spiking-nationwide-all-signs-point-to-a-harrowing-autumn/2020/11/10/d61fa050-238b-11eb-a688-5298ad5d580a_story.html.
RESOURCES
