This document summarizes a research paper on using artificial neural networks and a sensory glove for American Sign Language word recognition. It outlines the problem of communication barriers for deaf people, describes sign language and why a data glove was used for input. The system structure and components are explained, including data collection from the glove, feature extraction, and an artificial neural network for training and recognition. Test results showed the system could successfully recognize signed words and unknown words. Backpropagation training performed better than Levenberg-Marquardt, with faster training time, higher accuracy, and better performance. The goal is to further improve sentence recognition capabilities to help human-machine interaction for disabled users.

1. Khulna University ofEngineering & Technology
Department of Electrical and Electronic Engineering
Seminar On EE-4130
Presented by–
S.M.Kamrul Hasan
Roll No. 1003079
Department of
Electrical & Electronic Engineering 1

2. American Sign Languagewordrecognitionwith
a sensoryglove usingartificial neural networks
Published in: Elsevier Journal of Engineering Application of
Artificial Intelligence, Volume 24, Issue 7.
Date of Publication: October 2011
Pages: 1204-1213
Authors:
Cemil Oz & Ming C.
Leu
2

3. Outline :-
 Problem definition
 Motivation
 What is Sign Language
 Why Data glove
 System Structure
 Data Collection
 Feature Extraction
 Artificial Neural Network(ANN)
 Training Algorithm
 Test Results
 Conclusion
3

4. Problem definition:
 Humans have been endowed by nature with the voice
& hearing capability...
Kids speaking
4 /40
Problem definition:-
4

5. Problem definition:
 Humans have been endowed by nature with the voice
& hearing capability...
 ...but not everybody possesses this capability←deaf people
Deaf kid
4 /40
Problem definition:-
5

6. Motivation:-
A Real time Sign Language Recognition System
Combines Adaptive Filtering & Artificial Neural Network
 Interpret Sign language into English word.
 Fully Flexible
 Intelligent online learning
 Training time so faster
 Better performance
 Adaptable
6

7. What is Sign Language ?
 Visual gestural communicating
language used by deaf.
 Movement not just with hands
 Varies from country to country,
language to language
English alphabet in Sign Language
7

8. System Structure:-
8
System Architecture

9. Why Data Glove?
 Cyber-Glove
 measure hand shape accurately
 light in weight
 high resolution data
 data acquiring is more difficult
 complicated data processing
 slower recognition rate
Vision based system
 Flock of Birds(3D Motion tracker)
 tracks hand orientation & position
Cyber glove

10. 1.Data Collection:
10
Data collection Block
 Indicate hand moving or static
Cyber-glove
& motion
Tracker
Velocity
Network
X
Y
Z
Data
store
Velocity Network
Velocity

11. 2.Feature Extraction:-
 to determine exactly which features are Important
 Part of the data reduction process
11
 Seven feature vectors

12. Artificial Neural Network (ANN):
 A Genetic Algorithm, resembles human brain
12
 acquires knowledge through learning.
 it involves human like thinking.
 they handle noisy or missing data.

13.  BackPropagation
Training Algorithm
 Successful approach to construct ANN
 A supervised learning
Predicted output != actual output
Weight is adjusted until no error
13
Error Adjust
N.N
Compare
Actual
output
Desired output
Input
output
Weight

14. 14
Test Results
 50 different American signs each with 6 samples total of 50x6=300
samples for training.
 Successfully recognize sign language to English Word
Sign Recognition by system

15. Test Results
15
ANN Test results for Known words

16. 16
Output
Output
Decoding
Does it cross
threshold?
Yes
Training
by N.N
No
Unknown
word
Do you want to
add the word?
Yes
Unknown Words Recognition
Test Results

17. Test Results
Levenberg-Marquardt Vs. Backpropagation
 Backpropagation gives Better performance with
 less train time
17
Number of hidden layer nodes

18. 18
Trained by 3 users Trained by 7 users
Algorithm Same users New users Same users New users
LM 87.96% 72.22% 92.09% 82.17%
BP 93.51% 80.09% 95.72% 85.41%
Levenberg-Marquardt Vs. Backpropagation
 Better accuracy
 Backpropagation gives Better performance with
Test Results

19. Conclusion
 An advanced initiative to recognize American Sign Language with
faster training, better accuracy & better recognition performance.
 The ultimate goal of this paper is to further improve the proposed
sign language recognition system that can use sentence recognition
and eliminate the limitations and use it successfully for Human to
Machine Interface for disable people.
19

20. THANK YOU
ALL
20