The document proposes a smart glass system that utilizes computer vision and deep learning to help visually impaired people navigate their environment. The system uses a camera to capture images, detect faces and objects using models like Faster R-CNN, and provides audio descriptions to the user, such as identifying known people and alerting about obstacles. It aims to make movement safer and easier for the visually impaired by recognizing surroundings through real-time voice messages. The system is designed to be low-cost, fast, and user-friendly. It evaluates the performance of its detection and recognition abilities through metrics like accuracy, sensitivity and specificity.

1. Introduction
 Artificial intelligence, commonly referred to as AI, is the process of imparting data, information,
and human intelligence to machines.
 The main goal of Artificial Intelligence is to develop self-reliant machines that can think and act
like humans.
 Deep learning’s core concept lies in artificial neural networks, which enable machines to make
decisions.

2. Abstract
Vision is one of the most important human senses, and it plays a critical role in understanding the
surrounding environment.
However, millions of people in the world are experiencing visual impairment.
This project has proposed a novel framework by utilizing AI, which makes the framework more
straightforward to use specifically for the individuals with visual impedances and to help the society.
 The proposed system will be able to properly recognize humans in complex environments with
multiple moving targets, thus providing to the user a complete set of information, namely presence,
position and nature of the available targets.
 Furthermore, a voice message alerts the visually impaired person about the obstacle or known or
unknown person

3. Relevance and Need of the Project in the Present
Context
India is home to one third of the world’s blind population and these numbers are expected to go up
further. This project mainly focuses on providing a safer and recognizable environment to the visually
impaired.
To develop a smart eye glass system that employs computer vision techniques and deep learning
models for Visually Impaired people.
To provides users with information regarding surrounding objects through real-time audio output.
To recognize the face and identity information of relatives and friends.

4. Literature Survey
Title & Year Author Methods Merits Demerits
A Hybrid Algorithm for Face
Detection to Avoid Racial
Inequity Due to Dark Skin
Year:2021
Muhammad, Syed Sarmad
Abbas, Adnan Abid, Saim
Rasheed
Link:
https://ieeexplore.ieee.org
/document/9585604
There has been significant development in the facial
recognition technology during past few decades. .
Furthermore, it has now been incorporated into our
daily usages, such as consumer applications, personal
data protection, or cyber-security, particularly while
using smartphones.
Its accuracy is high.
It is efficient and time consuming is
low.
The accuracy is very low to detect
people with dark skin.
To detect the variation among the
skin tone within races has been
considered as major challenge for all
skin modelling techniques.
Exposing Fake Faces Through
Deep Neural Networks
Combining Content and
Trace Feature Extractors
Year:2021
Eunji Kim, Sungzoon Cho
Link:
https://ieeexplore.ieee.org
/document/9531572
With the breakthrough of computer vision and deep
learning, there has been a surge of realistic looking fake
face media manipulated by AI such as DeepFake or
Face2Face that manipulate facial identities or
expressions.
Highest accuracy at various video
compression levels
It effectively learns the different
characteristics of fake manipulation
methods.
It is less effective.
Accuracy is low.
Object Detection in Thermal
Spectrum for Advanced
Driver-Assistance Systems
Year:2021
Muhammad Ali
Farooq, Peter
Corcoran, Cosmin
Rotariu, Waseem Shariff
Link:
https://ieeexplore.ieee.org
/document/9618926
Thermal cameras can be used for object detection in
both day and night-time environmental conditions.
Since it is invariant to illumination changes, occlusions,
and shadows it provides improved situational
awareness.
Highest mean average precision. Data has yet not achieved vigorous
result.
Little variability in the scene.

5. Title & Year Author Methods Merits Demerits
YOLO-FIRI: Improved YOLOv5
for Infrared Image Object
Detection
Year:2021
Shasha Li; Yongjun Li; Yao
Li; Mengjun Li; Xiaorong Xu
Link:
https://ieeexplore.ieee.org/
document/9576741
To infrared image object detection using a one-stage
region-free object detector YOLO-FIR. The designed
feature extraction network extends and iterates the
shallow CSP module, which uses an improved attention
module,
The detection accuracy of infrared
small objects is high and images
low recognition rate.
false alarm rate.
Learning Domain-Invariant
Discriminative Features for
Heterogeneous Face
Recognition
Year:2020
Shanmin Yang, Keren
Fu, Xiao Yang, Ye
Lin, Jianwei Zhang, Cheng
Peng
Link:
https://ieeexplore.ieee.org/
document/9262951
To a novel framework for heterogeneous face recognition
(HFR), integrating domain-level and class-level alignment
in one unified network using domain-invariant
discriminative features (DIDF) method.
The effectiveness and superiority of
the proposed DIDF framework
Replaced/improved for other
problems such as pose/lighting/
expression
Matching face images across different
domains, is a challenging problem.
Lacking in sufficient pairwise cross-
modality training data.
Accuracy is low.

6. Problem Identified
Individuals suffering from visual impairments and blindness encounter difficulties in moving
independently and overcoming various problems in their routine lives.
They are facing difficulties in their daily navigations since they cannot see the obstacles or persons in
their surroundings.
Lot of companies try to develop electric equipment and gadgets for visually challenged people in
order to overcome their mobility problem without any sort of dependence.

7. Objective
To develop a smart glass system that employs computer vision techniques and deep learning models
for Visually Impaired people.
To provides users with information regarding surrounding objects through real-time audio output.
To recognize the face and identity information of relatives and friends.

8. Proposed System
 The proposed system of the project is to design and fabricate a Smart Camera that designed to make
recognizing faces and objects easier for visually impaired people.
 The comprised system comprises of data collection, Face/Object Identification, Classification of the
identified face/object, Prediction of the data. A voice alert message is sent to the user.

9. Unique Features of the Proposed System
 Smart Camera is designed to recognizing faces and objects for visually impaired
people.
 The Faster R-CNN model uses perform the object detection and classification.
 CNN is used to detect known person.
 A trained assistant who provides spoken feedback about what you are looking at.

10. Advantages
 An effective way to help the visually impaired people recognize and locate objects.
 The proposed smart camera is a low-cost device with faster response, user-friendly.
 It interacts with its user by means of audio messages.
 Obtain high accuracy in detection of object/face.
 Visually Impaired people feel the environment safe, convenient and comfortable.

11. Modules List (Functional Requirements)
1. Object Detection and Face Recognition Module
2.1. Face Enrollment
2. Object and Face Identification
3. Prediction
4. Performance Analysis

12. Object Detection and Face Recognition
2.1. Face Enrollment
 This module begins by registering a few frontal face of Blind persons friends,
family or other know person.
 These templates then become the reference for evaluating and registering the
templates for the other poses: tilting up/down, moving closer/further, and turning
left/right.
 Frames are extracted from video input.

13. Object and Face Identification
 Capturing the object or face image from the Smart Glass Camera, the image is
given to face detection module.
 This module detects the image regions which are likely to be human.
 Face image is given as input to the feature extraction module to find the key
features that will be used for classification.
 The face image is then classified as either known or unknown.

14. Prediction
 In this module the matching process is done with trained classified result and test
Live Camera Captured Classified file.
 Hamming Distance is used to calculate the difference according to the result the
prediction accuracy will be displayed.
 Audio output.
 If a data is triggered during processing, voice is used to alert the user, generating,
for example, “stop,” if there is an obstacle in the way. Saying that Hi Suramya.

15. Performance Analysis
 In this module we able to find the performance of our system using
SENSITIVITY, SPECIFICITY AND ACCURACY of Data in the datasets are
divided into two classes not pedestrian (the negative class) and pedestrian (the
positive class).
 Sensitivity, specificity, and accuracy are calculated using the True positive (TP),
true negative (TN), false negative (FN), and false positive (FP).
 TP is the number of positive cases that are classified as positive.

16. SNO Description Stimulus Response Dependencies &
constrains
1 Face Enrollment This module begins by
registering a few frontal face of
Visually Impaired persons
friends, family or other know
person.
The system accepts and
register the individuals
N/A
2 Object or Face Image
Acquisition
Cameras should be deployed in
Smart Glass to capture relevant
video. Computer and camera are
interfaced and here webcam is
used.
Camera that is used
capture the images of
the individual/object
N/A
3 Frame Extraction Frames are extracted from video
input. The video must be divided
into sequence of images which
are further processed.
The sequenced image
is further processed.
N/A
4 Pre -Processing Object or Face Image pre-
processing are the steps taken to
format images before they are
used by model training and
inference
• Read image
• RGB to Grey Scale
conversion
• Resize image
• Noise Removal
N/A

17. SNO Description Stimulus Response Dependencie
s &
constrains
5 Face Detection Face detection and segmentation
method based on improved RPN.
For Storing the data, Face
is detected
N/A
6 Object Detection Object detection is an important
computer vision task used to detect
instances of visual objects of
certain classes (for example,
humans, animals, cars, or
buildings) in digital images such as
photos or video frames
Similar to Face Detection.
Objects are also detected
and stored.
N/A
7 Feature Extraction Face image is given as input to the
feature extraction module to find the
key features that will be used for
classification.
The facial information including eyes,
nose and mouth is automatically
extracted and is then used to calculate
the effects of the variation using its
relation to the frontal face templates.
N/A
8 Object and Face
Classification
Classification of collected images based
on faster R-CNN
Faster R-CNN runs the neural network
once on the whole image. At the end of
the network is a novel method known as
Region of Interest (ROI) Pooling, which
slices out each Region of Interest from
the network’s output tensor, reshapes,
and classifies it.
N/A

18. SNO Description Stimulus Response Dependencies
& constrains
8 Object and Face
Classification
Classification of collected images
based on faster R-CNN
Faster R-CNN runs the
neural network once on the
whole image. At the end of
the network is a novel
method known as Region of
Interest (ROI) Pooling,
which slices out each
Region of Interest from the
network’s output tensor,
reshapes, and classifies it.
N/A
9 Object and Face
Identification
This module detects the image regions
which are likely to be human. After the
face detection using Region Proposal
Network (RPN), face image is given as
input to the feature extraction module
to find the key features that will be
used for classification.
After storing the data,
Each time the face or
object is identified
using RPN.
N/A
10 Performance Analysis Sensitivity, specificity, and accuracy are
calculated using the True positive (TP),
true negative (TN), false negative (FN),
and false positive (FP).
.
The performance and
accuracy is tested.
N/A

19. NON-FUNCTIONAL REQUIREMENTS
System should be able to accommodate large data items.
System should have user-friendly interface.
Responses should appear within seconds on the screen.
The system should be available 24*7

20. Hardware specification
 Processors: Intel® Core™ i5 processor 4300M at 2.60 GHz or 2.59 GHz (1 socket,
2 cores, 2 threads per core), 8 GB of DRAM.
 Disk space: 320 GB.
 Operating systems: Windows® 10, macOS*, and Linux*.

21. Software Specification
Server Side : Python 3.7.4(64-bit) or (32-bit)
Client Side : HTML, CSS, Bootstrap
IDE : Flask 1.1.1
Back end: MySQL 5.
Server : Wampserver 2i
DL DLL: TensorFlow, Pandas, SiKit Learn

22. System Architecture
Live Video
Dataset Annotation
Data Preparation
Object or Face Detection
Feature Extraction
Classification
Online Classifier Loader
Prediction
Person Name
Object Name
Face Enrollment Object Dataset
Validation
Training Phase
FRCNN

23. Data Flow Diagram
Level 0

24. Level 1

25. Usecase

29. Screen Interface Designs