Application Specific Hardware Accelerator Edge Vision for Secured Driving License Validation

1. Application Specific
Hardware Accelerated
Edge Vision for Secured
Driving License Validation
Jyotsna V J (23MES0018)
Lakshmi V (23MES0005)
Guide : Dr. Sujatha R

2. Introduction
• Driving licenses are official permissions allowing individuals to drive on
public roads. With the growing population and increasing vehicle traffic
on the road, there's an increase in unauthorized driving.
• A considerable number of individuals engage in driving without
possessing a legitimate driving license. A 2018 study by the World
Health Organization (WHO) estimated that 25% of drivers worldwide
lack a valid license, with the highest rates in low- and middle-income
countries.
• A novel concept has been implemented in vehicles, connecting them to
a worldwide license database. When a person takes the driver's seat,
their facial features are recognized and compared with a valid driving
license through IoT technology.
• If no match is found, the ignition remains disabled, allowing vehicle
operation only with a verified driving license. The validation mechanism
is facilitated by integrating face recognition algorithms with a
prototyping FPGA board.

3. Literature Survey
No Reference Paper Year Methodology Limitations
1 AlBdairi, A.J.A., Xiao, Z., Alkhayyat, A.,
Humaidi, A.J., Fadhel, M.A., Taher, B.H.,
Alzubaidi, L., Santamaría, J. and Al-Shamma,
O., 2022. Face recognition based on deep
learning and FPGA for ethnicity identification.
Applied Sciences, 12(5), p.2605.
2022 The study proposes a Deep Learning (DL)
approach based on a Field-Programmable Gate
Array (FPGA) hardware accelerator for
ethnicity identification. A dataset of 3141
images from three countries was collected, and
the approach was compared to four different
pre-trained CNN models.
• The dataset used in the study is
relatively small, which may limit the
generalizability of the results.
• The study compared the proposed
approach only with four pre-trained
CNN models, which may not be an
exhaustive evaluation of the
approach's performance against
other methods.
• The proposed approach has not yet
been tested in real-world
applications.
2 Baobaid, A., Meribout, M., Tiwari, V.K. and
Pena, J.P., 2022. Hardware accelerators for
real-time face recognition: A survey. IEEE
Access.
2022 The paper is a survey on the state-of-the-art
hardware accelerators used for real-time face
recognition. It provides a comprehensive
review of the most recent face recognition
algorithms along with their embedded
hardware systems targeting real-time
performance. It also discusses the suitability of
these algorithms to be implemented into
parallel hardware architectures.
• The paper does not provide a
comparison of the energy efficiency
of the various hardware accelerators
used for face recognition.
• Moreover, the paper does not give
an indication of the maximum
number of faces that can be
recognized by the reviewed
hardware accelerators, and it does
not offer any suggestions for future
directions or research gaps in this
field.

4. No Reference Paper Year Methodology Limitations
3 Günay, B., Okcu, S.B. and Bilge, H.Ş., 2022.
LPYOLO: low precision YOLO for face
detection on FPGA. arXiv preprint
arXiv:2207.10482.
2022 An optimized neural network model based on
the TinyYolov3 architecture for face detection
on FPGA. The model was trained in quantized
structure using the WiderFace dataset, and
high degree of parallelism was applied to
logical resources of the FPGA. The method
achieved low power consumption, acceptable
accuracy rate, and sufficient throughput for
edge computing.
• The paper does not discuss the
deployment cost of the model or
the overall system.
• The work has been validated on
only one target board, the PYNQ-
Z2, which is a low-end Zynq SoC.
• The paper does not compare the
results of the proposed approach
with other state-of-the-art
methods in facial recognition or
object detection
4 Selvi, S.S., Bharanidharan, D., Qadir, A. and
Pavan, K.R., 2021, July. FPGA Implementation
of a Face Recognition System. In 2021 IEEE
International Conference on Electronics,
Computing and Communication Technologies
(CONECCT) (pp. 1-5). IEEE.
(Base Paper)
2021 The paper proposes FPGA implementation for
face recognition, addressing computational
complexity. It outlines a methodology on Xilinx
PYNQ Z2, incorporating Haar cascade for face
detection, Local Binary Pattern Histogram for
feature vectors. Achieving real-time face
recognition, the system attains 91.6% precision
and 92% overall accuracy
• Implementing complex algorithms
on FPGAs from scratch is a difficult
task, and it can be challenging to
scale this algorithm for large
datasets.

5. No Reference Paper Year Methodology Limitations
5 Devi, S.S., Prakash, T.S., Vignesh, G.
and Venkatesan, P.V., 2021, August.
Ignition system-based licensing
using PIC microcontroller. In 2021
Second International Conference
on Electronics and Sustainable
Communication Systems (ICESC)
(pp. 252-256). IEEE.
2021 The proposed fingerprint-based license
verification system involves scanning
fingerprints, using a microcontroller to check
and compare them with a database, and
enabling vehicle operation only if there is a
match. The procedure requires selecting an ID
for finger storage, checking FP identification and
using a seat belt to activate the motor.
• The limitations of the paper are that the
hardware can only store up to 250 fingerprint
data, and its hardware maintainability can be
difficult as it can be easily damaged.
6 Saadouli, G., Elburdani, M.I., Al-
Qatouni, R.M., Kunhoth, S. and Al-
Maadeed, S., 2020, February.
Automatic and secure electronic
gate system using fusion of license
plate, car make recognition and
face detection. In 2020 IEEE
International Conference on
Informatics, IoT, and Enabling
Technologies (ICIoT) (pp. 79-84).
IEEE.
2020 The proposed methodology is an automatic gate
system that uses license plate, car make and
model recognition, and face detection for
authenticity confirmation. The system includes
image acquisition, pre-processing, feature
extraction, and multi-level authentication
schema.
The paper does not provide detailed information
on the specific implementation of the system
and its suitability for large-scale deployment. The
system was tested using a toy car database,
which may not reflect the actual recognition
performance in a real-world scenario. Moreover,
the accuracy of the system for license plate and
face detection was not perfect.

6. No Reference Paper Year Methodology Limitations
7 Rakshith, B.S., Nagabhushan, N. and
Madhavi, T., 2021, April. Fingerprint-based
licensing for driving. In 2021 6th
International Conference for Convergence
in Technology (I2CT) (pp. 1-6). IEEE.
2019 The methodology involves fingerprint and
driving license verification as prerequisites
for vehicle ignition. It includes an RFID
system and fingerprint recognition
algorithm, with Raspberry Pi acting as an
intermediate between all components. A
centralized database further cross-verifies
the data, ensuring optimum security.
• The paper lacks a real-world validation of the
proposed system, and the testing has been
done on self-made databases.
• The performance of the fingerprint
verification algorithm used in the system has
not been evaluated in-depth, and further
research in this area is required.
• This system has been designed only for four-
wheelers and may not be feasible for two-
wheelers.
8 Alsayaydeh, J.A.J., Indra, W.A., Khang,
W.A.Y., Shkarupylo, V. and Jkatisan,
D.A.P.P., 2019. Development of vehicle
ignition using fingerprint. ARPN Journal of
Engineering and Applied Sciences, 14(23),
pp.4045-4053.
2019 The document outlines a prototype vehicle
ignition system with enhanced security
using a fingerprint sensor, Arduino
microcontroller, and GSM SIM 900.
Authorized user fingerprints allow ignition,
triggering a theft alarm and owner
notification. The methodology details the
system's design and implementation.
• It did not address the challenges or
drawbacks of the fingerprint sensor-based
vehicle ignition in terms of robustness,
reliability, or usability.
• The document did not report the results of
user acceptance testing or verification of the
efficiency, accuracy, and effectiveness of the
proposed system.

7. No Reference Paper Year Methodology Limitations
9 Jabeen, F., Rupanagudi, S.R. and Bhat, V.G.,
2019, December. IoT based smart vehicle
ignition and monitoring system. In 2019
International Conference on Advances in
Computing, Communication and Control
(ICAC3) (pp. 1-7). IEEE.
2019 The methodology utilized a combination of
Raspberry Pi and Arduino boards to implement
a smart vehicle ignition and monitoring system
that incorporates biometric checks, an
emergency alert system, and wireless
communication between sensors and a tablet.
The system includes algorithms for face
recognition, fingerprint authentication, and
fuel level detection.
• The paper doesn't provide a detailed
explanation of how the alcohol
sensor works, and the accuracy of
the alcohol sensor was not tested.
• The paper acknowledges that the
face recognition algorithm may fail in
cases where the user's face is only
partially illuminated.
• The sample size of the experiment
for obtaining fuel level accuracy was
very small, with only data from seven
cars being used.
10 Austria, Y.D., Lacatan, L.L., Funtera, J.G.D.,
Garcia, S.C., Montenegro, J.H. and Santilleces,
L.T., 2019. Face recognition for motorcycle
engine ignition with messaging system. arXiv
preprint arXiv:1907.10385.
2017 Integrating a face recognition system with a
Raspberry Pi, Arduino, and various modules for
motorcycle security. The system utilizes a
Camera for user authentication, a GPS Module
for location tracking, and GSM for
communication. The relay mechanism ensures
engine control. Acceptance testing validates
face recognition and anti-theft features,
affirming system efficacy.
• It may not have the same level of
performance as an FPGA-based
system.
• FPGA-based systems can offer
advantages such as faster
computation, higher throughput, and
better power efficiency.

8. Research Gaps
• Most of the works don’t provide any real-life
scenarios.
• Physical implementation is missing.
• Fingerprint technology used will not always be an
advantage when considering the limitations of
that technique.
• Less secure systems.

9. Importance of Current Study
• Enhanced Security: Face detection technology adds an extra layer of security to driving license
validation processes.
• Reduced Impersonation: The technology ensures that the person claiming to be the license holder is
indeed the legitimate owner.
• Improved Road Safety: Accurate validation of driving licenses ensures that only qualified and authorized
individuals operate vehicles, contributing to overall road safety.
• Technological Advancements: The study contributes to the ongoing advancements in facial recognition
technology and its applications, fostering innovation in the field of identity verification.

10. Problem
Statement
&
Objectives
• To design and develop a solution for a valid and secure
driving system using driving license validation.
• To develop a system where driver validity is maintained
using hardware accelerators.

11. Proposed Methodology
• The proposed system uses a camera attached to the Pynq Z2 FPGA, which captures the face of the driver.
• The facial detection technique using various deep learning algorithms is to recognize the face of the driver in
front of the driver seat.
• The faces detected will be compared with the data set of the valid registered users.
• If the detected face is in the valid license system, then start the ignition and show the results on the display.
• Otherwise, do not start the ignition and show the results on the display.
Hardware:
• Xilinx Pynq Z2 FPGA
• USB Camera
• Display
Software:
• Jupyter

12. Block
Diagram

13. PYNQ Z2 FPGA

14. Flowchart

15. February March April
Literature Review
& Abstract Submission
First Project Review
Phase 1:
Algorithm
Development
Phase 2:
Software
Implementation
Phase 3: Hardware
Software
Integration
Phase 4:
Draft Paper
Submission
Phase 5:
Final Project Submission &
Presentation
Feb
18
Mar
29
Feb
28
Apr
3
Mar
8
Mar
12
Apr
25
Timeline

16. References
1. Selvi, S.S., Bharanidharan, D., Qadir, A. and Pavan, K.R., 2021, July. FPGA Implementation of a Face Recognition System. In
2021 IEEE International Conference on Electronics, Computing and Communication Technologies (CONECCT) (pp. 1-5). IEEE.
(Base Paper)
2. Rakshith, B.S., Nagabhushan, N. and Madhavi, T., 2021, April. Fingerprint-based licensing for driving. In 2021 6th
International Conference for Convergence in Technology (I2CT) (pp. 1-6). IEEE.
3. Devi, S.S., Prakash, T.S., Vignesh, G. and Venkatesan, P.V., 2021, August. Ignition system based licensing using PIC
microcontroller. In 2021 Second International Conference on Electronics and Sustainable Communication Systems (ICESC)
(pp. 252-256). IEEE.
4. Alsayaydeh, J.A.J., Indra, W.A., Khang, W.A.Y., Shkarupylo, V. and Jkatisan, D.A.P.P., 2019. Development of vehicle ignition
using fingerprint. ARPN Journal of Engineering and Applied Sciences, 14(23), pp.4045-4053.
5. Jabeen, F., Rupanagudi, S.R. and Bhat, V.G., 2019, December. IoT based smart vehicle ignition and monitoring system. In
2019 International Conference on Advances in Computing, Communication and Control (ICAC3) (pp. 1-7). IEEE.
6. Austria, Y.D., Lacatan, L.L., Funtera, J.G.D., Garcia, S.C., Montenegro, J.H. and Santilleces, L.T., 2019. Face recognition for
motorcycle engine ignition with messaging system. arXiv preprint arXiv:1907.10385.
7. Ashwin, S., Loganathan, S., Kumar, S.S. and Sivakumar, P., 2013, February. Prototype of a fingerprint based licensing system
for driving. In 2013 International Conference on Information Communication and Embedded Systems (ICICES) (pp. 974-
987). IEEE.
8. Omidiora, E.O., Fakolujo, O.A., Arulogun, O.T. and Aborisade, D.O., 2011. A prototype of a fingerprint based ignition systems
in vehicles.

17. 9. Cho, J., Mirzaei, S., Oberg, J. and Kastner, R., 2009, February. Fpga-based face detection system using haar classifiers. In
Proceedings of the ACM/SIGDA international symposium on Field programmable gate arrays (pp. 103-112).
10. Sajid, I., Ahmed, M.M., Taj, I., Humayun, M. and Hameed, F., 2008, July. Design of high performance fpga based face
recognition system. In Progress in Electromagnetic Research Symposium Proceeding (pp. 504-510).
11. VENKATESWARLU, C., RAO, D.S. and KUMARI, S., 2015. Prototype of A Fingerprint Based Licensing System for Driving.
12. Jayalakshmi, B., Vijayan, A. and Gopikrishnan, V., 2017, July. Driving license test automation. In 2017 International
Conference on Intelligent Computing, Instrumentation and Control Technologies (ICICICT) (pp. 748-751). IEEE.
13. Fu, C. and Yu, Y., 2019, December. FPGA-based power efficient face detection for mobile robots. In 2019 IEEE International
Conference on Robotics and Biomimetics (ROBIO) (pp. 467-473). IEEE.
14. Bhojane, K.J. and Thorat, S.S., 2018. Face Recognition Based Car Ignition and Security System. International Research Journal
of Engineering and Technology (IRJET), 5(05), pp.3565-3668.
15. Thomas, M.J., 2015. Combining facial recognition, automatic license plate readers and closed-circuit television to create an
interstate identification system for wanted subjects (Doctoral dissertation, Monterey, California: Naval Postgraduate School).
16. Li, S.Z., Jain, A.K., Huang, T., Xiong, Z. and Zhang, Z., 2005. Face recognition applications. Handbook of Face Recognition,
pp.371-390.
17. Günay, B., Okcu, S.B. and Bilge, H.Ş., 2022. LPYOLO: low precision YOLO for face detection on FPGA. arXiv preprint
arXiv:2207.10482.
18. AlBdairi, A.J.A., Xiao, Z., Alkhayyat, A., Humaidi, A.J., Fadhel, M.A., Taher, B.H., Alzubaidi, L., Santamaría, J. and Al-Shamma,
O., 2022. Face recognition based on deep learning and FPGA for ethnicity identification. Applied Sciences, 12(5), p.2605.
19. Baobaid, A., Meribout, M., Tiwari, V.K. and Pena, J.P., 2022. Hardware accelerators for real-time face recognition: A survey.
IEEE Access.
20. Saadouli, G., Elburdani, M.I., Al-Qatouni, R.M., Kunhoth, S. and Al-Maadeed, S., 2020, February. Automatic and secure
electronic gate system using fusion of license plate, car make recognition and face detection. In 2020 IEEE International
Conference on Informatics, IoT, and Enabling Technologies (ICIoT) (pp. 79-84). IEEE.