2. Dedication
First of all we would like to remember the
deaf and dumb people of the world for
whom we tried to develop a Sign language
Recognizer (SLR).
3. Outline
• Sign language
• SLR & its necessity
• Helping process of SLR
• Working procedure of SLR
• Block Diagram of SLR
• BP training time & graph
• Recognition accuracy
• Limitations
• Future plan
• Papers
4. What is Sign Language ??
Communicating language
used primarily by deaf people.
Uses different medium such
as hands, face, or eyes rather
than vocal tract or ears for
communication purpose.
Communication using sign language
5. What is SLR ??
Sign language recognizer (SLR) is a tool for
recognizing sign language of deaf and dumb
people of the world.
6. Why we need SLR ??
Problems:
• About 2 million people are deaf in our world
• They are deprived from various social
activities
• They are under-estimated to our society
• Communication problem
8. How SLR help ?? An Example.....
Suppose a deaf customer
went to a shop. She is ??
trying to express her
demands to the
shopkeeper using sign
language but the
shopkeeper can not
understand her demands. shopkeeper Deaf customer
9. Continued..
SLR brings the solution for this problem>>
• SLR capture signs shown by deaf man
• Convert the signs to text
• This text is shown to shopkeeper
Now the shopkeeper can understand the deaf man’s demands
11. Continued..
Text to sign conversion
When shopkeeper replied to the deaf customer SLR
• Convert text to sign
• This sign is shown to the deaf customer
Now the deaf man can understand the shopkeeper’s speeches
18. How SLR works ??
Image processing &
sign detection
Normalization
Sign recognition
Sign to text conversion
19. Continued..
Image processing &
sign detection
• Image capture
• Skin color detection
Normalization
Sign recognition
Sign to text conversion
20. Continued..
Image processing &
sign detection
• Hand gesture detection
Normalization • Sign detection
Sign recognition
Sign to text conversion
21. Continued..
Image processing &
sign detection
• Reducing image
size
Normalization
Sign recognition
Sign to text conversion
200x200 30x33
22. Continued..
Image processing &
sign detection
• Backpropagation
implementation
Normalization
Sign recognition
Sign to text conversion
23. Continued..
Image processing &
sign detection • Converting sign
language to Bengali
or English text
Normalization
Sign recognition
v
Sign to text conversion
25. BP Training
Figure: Training error versus number of iteration
26. Training time for BP
Training
Input size of pixel Time
(min)
30*33 1.5
45*48 2.8
60*63 3.7
We have used 50 signs as training input where each
sign has 5 samples that make 50 x 5 = 250 samples.
28. Limitations
• Due to brightness and contrast
sometimes webcam can hardly detect
the expected skin color.
• Because of the similarity of tracking
environment background color and skin
color the SLR gets unexpected pixels.
31. Future Plan
• Real time word recognition of ASL & BSL
• Implementing neural network Ensembles
• Implementing Genetic algorithm for sign
recognition
32. Required Tools
• Visual studio 2008
• XML
• Avro Keyboard installed
• Aforge .Net
• Open CV
• Webcam