The document describes a project that aims to use facial recognition as a means of contact discovery and metadata retrieval. The project seeks to optimize machine learning models for facial detection and verification in order to provide fast and accurate contact matching based on facial encodings. It outlines the objectives, scope, literature review, proposed system architecture and implementation details. The system would take facial landmarks and encodings to compare and rank the top 10 most similar encodings to identify matches from a database. The optimized model aims to reduce latency and improve accuracy for contact matching based on facial scans.

1. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 09 Issue: 07 | July 2022 www.irjet.net p-ISSN: 2395-0072
© 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 633
Virtual Contact Discovery using Facial Recognition
Aryan Thakur1, Sharon Mary Thomas2, Kshitij Tapre3, Vijay Jumb4
1-3 Student, Dept. of Computer Engineering, Xavier Institute of Engineering, Mumbai, India
4 Professor, Dept. of Computer Engineering, Xavier Institute of Engineering, Mumbai, India
---------------------------------------------------------------------***---------------------------------------------------------------------
Abstract - We are surrounded by data. We produce a
significant amount of data ourselves too. One major
challenge we face today is storing and handling this data.
Even though we have abstractions such as metadata to help
us out, current components that are employed give us a lot
to desire for. We are working in this niche to implement a
metadata retrieval system using a handy component - our
face. We are implementing an example of contact discovery
using our facial encodings as the identifying component. We
are relying on some Machine Learning models in our stack,
so we will also be working on setting up the machine
learning pipeline efficiently to give us fast and accurate
results.
Key Words: Contact Discovery, Facial Recognition,
DeepFace, Face encodings, Machine Learning
1. INTRODUCTION
Carrying meta-data in real life is dependent on using an
auxiliary component (url links, systems, rfid/nfc tags etc).
The aim of this project is to come up with a solution to use
facial as an auxiliary component and associate some form
of metadata with it (contact removing the need to carry
another physical component and having a better user
experience.
1.1 Objectives
The objectives are but not limited to:
i) Although the project has its main components as facial
detection and verification, the main goal of the project is
to find, explore and if possible to set guidelines and ways
to optimize the deployment of machine learning models
as this particular area has lesser resources to explore as
compared to the minor but important phases of this
project that are facial detection and verification.
ii) Optimize the backend logic for real-time performance
and accuracy.
iii) Create a user friendly and intuitive flow in the
frontend.
1.2 Scope
The applications and the scope of the project would
include but is not limited to:
i) This can be implemented in a B2B or B2C environment
to provide additional features to the end-user.
ii) In forensic science to identify a suspect.
iii) Face-scanning and verification can be used by
companies employing drivers. A selfie could be a mode of
authentication to verify the identity of the driver and the
passenger before the journey begins and also to give
access to the driver to operate the vehicle assigned. Some
companies in this domain are currently testing this
concept in their trial phase.
iv) To implement a cashless system where the face is used
as the identifying element
2. LITERATURE SURVEY
In order to authenticate users through ID verification
services, facial recognition systems are often used to
match a human face from a digital image or video frame
against a database of faces. These systems operate by
identifying and quantifying facial features from an image.
Development began on similar systems in the 1960s,
beginning as a form of computer application. Since their
inception, facial recognition systems have seen wider uses
in recent times on smartphones and in other forms of
technology. There are APIs already in place in the domain
of facial recognition and detection like Kairos (Finding
faces and features), Animetrics (Deep-learning powered
face recognition ), etc. Our Goal through this project is to
optimize and deploy so the latency and accuracy are
satisfactory in a resource-restricted environment.
A. Intelligent detection and recognition system for
mask-wearing based on improved RetinaFace
algorithm [7]: In this paper, Bin Xue et al. have
designed a smart detection and system for mask-
wearing. The system is mainly composed of face
mask detection and face recognition algorithm.
The main functions of the system can be divided
into parts: face mask detection, face recognition,
and voice prompts. The paper implements

2. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 09 Issue: 07 | July 2022 www.irjet.net p-ISSN: 2395-0072
© 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 634
RetinaFace which is a cutting-edge deep learning
algorithm that uses facial landmarks.
B. Deep Face Recognition [6]: The authors of this
work investigate the possibility of applying deep
learning to facial recognition. The DeepFace
procedure consists of two phases. First, it adjusts
an image's angles such that the subject's face is
facing forward. It makes use of a 3-D
representation of a face to achieve this. After that,
a numerical description of the face is produced via
deep learning. DeepFace implies that two photos
that have sufficiently similar DeepFace
descriptions have the same face.
C. ArcFace: Additive Angular Margin Loss for Deep
Face Recognition [3]: In this paper, Jiankang Deng
et al. implement ArcFace to obtain highly
discriminative features for face recognition. The
ArcFace has a clear geometric interpretation due
to the exact correspondence to the geodesic
distance on the hypersphere. It factors in concepts
like Inter-Loss and Intra -Loss
D. Neural Networks (CNNs) and VGG on Real-Time
Face Recognition System [2]: In this paper, the
authors have suggested a system where the three
main components of the suggested system are
database collecting, and facial recognition to
identify specific people, and performance
evaluation. The system gathers the faces in real-
time as the initial phase. 24 unique people are
represented in the database as 1056 photos with
a 112*92 resolution. For improving recognition
accuracy, Convolutional Neural Networks (CNN)
and [2]VGG-16 Deep Convolutional Neural
Networks (DCNN) is introduced in the second
step. Finally, precision, recall, F1-score, and
accuracy are used to evaluate the performance of
these two classifiers. These classifiers accurately
identify faces in real-time.
3. IMPLEMENTATION
Fig. 1 Proposed Architecture
Data Acquisition: Data set which was provided by our
framework over which our model has been pre-trained.
Although we were given a pre-trained model, some of the
requirements did not match up to our expectations. We
also are creating our own dataset by using user input.
Two Use Cases:
The two use cases of our application that we take under
consideration:-
• Case 1: Registering a new user (Reference image upload)
• Case 2: Scan and Identify the user (Identification)
Working:
Case 1: When a user sends a registered query, The image is
uploaded to the backend wherein first the encodes are
calculated and these encodings are then uploaded to an
RDBMS. The encodings are of the dimension 3x224x224.
These encodings represent the model’s attempt to
fingerprint your face and as such are entered into a table
row-wise. Another table is also created which stores the
user data related to the encodings. If the user uploads
valid data and the backend is able to process the facial
data and store it in the RDBMS a successful response is
sent back.
Case 2: The user uses the scan route to take a picture of a
target user. This image is then sent to the backend
wherein the encodes are derived in a similar manner as
that of the registration process. A query is created to find a
specific neighborhood of faces that would return the least
amount of cosine distance. Each face is compared against

3. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 09 Issue: 07 | July 2022 www.irjet.net p-ISSN: 2395-0072
© 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 635
the target face and their cosine distances are then
calculated. Generally, the neighborhood consists of 10
faces. From these faces, the face with the least cosine
distance is selected and the corresponding user data is
queried from the metadata table. This user data and the
corresponding distance is sent back to the client as a
response. The client can then check the cosine value
against the threshold value and show and show the
appropriate output.
Backend: Face identification and face verification are two
subcategories of facial recognition (FR). In either case, the
system enrolls a group of known subjects initially (the
gallery), and during testing, a different subject is shown
(the probe). Each of the test images is fed through the
networks to produce a deep feature representation once
the deep networks have been trained on huge amounts of
data under the guidance of a suitable loss function.
Face identification computes one-to-many similarity to
pinpoint the precise identity of a probe face, while face
verification computes the one-to-one similarity between
the gallery and probe to establish whether the two photos
are of the same subject.
The proposed system takes the facial landmarks and facial
encodings into consideration to compare and rank the top
10 facial encodings that match the encoding detected
when a subject is fed to the probe.
The neighborhood of the closest 10 records having
records that have the values of facial encodings the most
similar or close to are compared and ranked. The record
with the top most rank is identified as the result of the
identification phase of the application.
The optimized comparison and storage would make it
easier for getting the output quicker where the latency will
be as low as possible provided the limitations of our
resources. The face will be scanned, the image gets
compared with the reference database clusters and the
details connected to the particular face will be presented
as the output.
4. CONCLUSIONS
Although you may be able to opt-out of or disable face
recognition when it comes to a device or piece of software
you own or use, it is becoming more and more difficult to
do so in public due to the prevalence of cameras. Major
firms, including Amazon, IBM, and Microsoft, have
imposed a hold on selling their software to police
enforcement because of worries about that prevalence,
which have been exacerbated by evidence of racial
profiling and protester identification. However, when
moratoria end and facial recognition technology advances,
society will need to decide how facial recognition should
be controlled. It will also need to decide which services we
are all willing to use and what privacy compromises we
are all prepared to accept.
REFERENCES
[1] Zhiqiang Cao. “Feature Extraction of Single Face Image
Based Active Appearance Model”. In: Jan. 2016. doi:
10.2991/amsm-16.2016.56.
[2] Showkat Dar and S Palanivel. “Neural Networks
(CNNs) and Vgg on Real Time Face Recognition
System”. In: (Jan. 2021), pp. 1809–1822.
[3] Jiankang Deng, Jia Guo, and Stefanos Zafeiriou.
“ArcFace: Additive Angular Margin Loss for Deep Face
Recognition”. In: CoRR abs/1801.07698 (2018). arXiv:
1801.07698. url:http://arxiv.org/abs/1801.07698.
[4] Mostafa Mehdipour-Ghazi and Hazim Kemal Ekenel.
“A Comprehensive Analysis of Deep Learning Based
Representation for Face Recognition”. In: CoRR
abs/1606.02894 (2016). arXiv: 1606.02894. url:
http://arxiv.org/abs/1606.02894.
[5] Florian Schroff, Dmitry Kalenichenko, and James
Philbin. “FaceNet: A Unified Embedding for Face
Recognition and Clustering”. In: CoRR
abs/1503.03832
(2015).arXiv:1503.03832.url:http://arxiv.org/abs/15
03.03832
[6] Mei Wang and Weihong Deng. “Deep Face
Recognition: A Survey”. In: CoRR abs/1804.06655
(2018). arXiv: 1804.06655. url:
http://arxiv.org/abs/1804.06655.
[7] Bin Xue, Jianpeng Hu, and Pengming Zhang.
“Intelligent detection and recognition system for mask
wearing based on improved RetinaFace algorithm”. In:
2020 2nd International Conference on Machine
Learning, Big Data and Business Intelligence
(MLBDBI).2020,pp.474–
479.doi:10.1109/MLBDBI51377.2020.00100.
[8] Chenchen Zhu et al. “Seeing Small Faces from Robust
Anchor’s Perspective”. In: CoRR abs/1802.09058
(2018). arXiv: 1802.09058. url:
http://arxiv.org/abs/1802.09058
[9] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual
learning for image recognition. In CVPR, 2016
[10] http://data.mxnet.io/models/.

4. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 09 Issue: 07 | July 2022 www.irjet.net p-ISSN: 2395-0072
© 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 636
[11] Kak, Shakir & Mustafa, Firas & Valente, Pedro. (2018).
A Review of Person Recognition Based on Face Model.
4. 157-168. 10.23918/eajse.v4i1sip157.
[12] M. Turk and A. Pentland, “Eigenfaces for recognition,”
Journal of cognitive neuroscience, vol. 3, no. 1, pp. 71–
86, 1991.