The document outlines a project focused on end-to-end sign language detection using deep learning techniques to recognize and translate gestures into written or spoken language. The objective is to bridge communication gaps between hearing and non-hearing individuals, utilizing methods such as object detection, classification, and localization with the YOLO algorithm. Essential resources include Python, OpenCV, and TensorFlow, aimed at improving communication accessibility in various settings like hospitals and offices.

1. End-to-end
OBJECT DETECTION
(Sign language detection)
Submitted By : Vijay Sharma
(2021pgcaca083)
Faculty-In-Charge : Dr. Koushlendra Kumar Singh

2. TABLE OF CONTENTS
 Introduction to Sign Language
Detection
 Project Overview
 Objective
 Problem Domain
 Solution Domain
 Scope of the Application
 Required Resources
 Methodology
o Use Case Diagram
o Process of Object Detection
Image Classification
Object Localization
Object Detection

3. Sign language Detection
Sign language detection is a technology-driven process that
involves recognizing and interpreting the gestures, movements, and
expressions of individuals who use sign language as their primary
means of communication. Using techniques such as computer
vision, machine learning, and sensor-based systems, sign language
detection translates these visual cues into written or spoken
language or other forms of communication. This technology aims to
facilitate communication access for individuals who are deaf or
hard of hearing, enhancing their ability to interact with others and
access information in various contexts, including education, public
spaces, and online content

4. Project Overview
• In this project, we use a completely deep learning based approach to solve the problem of
object detection.
• The input to the system will be a real time image, and the output will be a bounding box
corresponding to all the objects in the image, along with the class of object in each box.
• the Main Aim of the project is to create an innovative solution that enhances communication
accessibility and inclusivity by effectively recognizing and translating hand signs for seamless
interaction between individuals with and without sign language proficiency

5. OBJECTIVE
 The objective of this project is to identify the symbolic
expression through images so that the communication gap
between a normal and hearing impaired person can be easily
bridged by:
 Creating data with respect to American sign language &pre-
process it.
 ii. Training the pre-processed data with Deep Learning based
models to perform sign language recognition & speech
conversion in real time.
 Testing the model in the real world scenario

6. PROBLEM DOMAIN
• Dumb people use hand signs to communicate, normal people face problem in
recognizing their language by signs made. Hence there is a need of the systems
which recognizes the different signs and conveys the information to the normal
people.
• Humans can easily detect and identify objects present in an image but for the
computer or machine a classifying and finding an unknown number of individual
objects within an image is extremely a difficult problem.

7. SOLUTION DOMAIN
• The main point of this application is to use camera to recognize gestures from
the sign language to offer a new means of communication. The program will be
able to transcribe gestures done by dumb people into written words printed on
the screen
• We use object classification and localization it provides accuracy, speed for real
time detection and also improves detection tasks are optimized using one multi-
task function and an object is compared to the image’s true objects.

8. Scope Of This Application
 Sign language recognition system could be used at reception
desks or during video conferences to allow signing people to
speak with people who don't know Sign Language.
 -Hotels
 -Hospitals
 -Clinic
 -Office

9. REQUIRED RESOURCES
 Software Requirements:
 An integrated development environment (IDE) Microsoft Visual Studio code.
 OpenCV understanding
 Python programming language
 Tensor flow
 YOLO Framework.
 Anaconda’s setup
 Hardware Requirements:
 To run Visual Studio code we need-
• Operating system of Microsoft Windows 10 (32-or 64-bit)
• RAM- 4 GB
• Hard Disk- Minimum 1GB disk space

10. METHODS
 The system uses a vision-based method. Since all of the signs are portrayed
with bare hands, there is no need for any artificial gadgets for interaction.
 We looked for pre-made datasets for the project but couldn't find any in
the form of raw images that met the software specifications. As a result, we
decided to develop our own data set. To create the dataset, Open
Computer Vision (OpenCV) library of python is used

11. YOLO ALGORITHM
 YOLO is an extremely fast real time multi object detection algorithm. YOLO stands
for “You Only Look Once”.
 The algorithm applies a neural network to an entire image. The network divides the
image into an S x S grid and comes up with bounding boxes, which are boxes
drawn around images and predicted probabilities for each of these regions.
 The method used to come up with these probabilities is logistic regression. The
bounding boxes are weighted by the associated probabilities. For class prediction,
independent logistic classifiers are used.
 YOLO uses a totally different approach. YOLO is a clever convolutional neural
network (CNN) for doing object detection in real-time. The algorithm applies a
single neural network to the full image, and then divides the image into regions
and predicts bounding boxes and probabilities for each region. These bounding
boxes are weighted by the predicted probabilities.

13. Object Classification: Predict the type or class of an object in an image.
 Input: An image with a single object, such as a photograph.
 Output: A class label (e.g. one or more integers that are mapped to class labels).
Object Localization: Locate the presence of objects in an image and indicate their
location with a bounding box.
 Input: An image with one or more objects, such as a photograph.
 Output: One or more bounding boxes (e.g. defined by a point, width, and height).
Object Detection: Locate the presence of objects with a bounding box and types or
classes of the located objects in an image.
 Input: An image with one or more objects, such as a photograph.
 Output: One or more bounding boxes (e.g. defined by a point, width, and height),
and a class label for each bounding box.
PROCESS OF OBJECT
DETECTION