Sign language recognizer


Published on

describe a sign language recognizer which can interpret deaf sign language to English or Bengali text.

Published in: Technology, Business
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Sign language recognizer

  1. 1. An intelligent approach to recognize sign language for deaf and dumb people of the world
  2. 2. Dedication First of all we would like to remember the deaf and dumb people of the world for whom we tried to develop a Sign language Recognizer (SLR).
  3. 3. Outline• Sign language• SLR & its necessity• Helping process of SLR• Working procedure of SLR• Block Diagram of SLR• BP training time & graph• Recognition accuracy• Limitations• Future plan• Papers
  4. 4. What is Sign Language ?? Communicating language used primarily by deaf people. Uses different medium such as hands, face, or eyes rather than vocal tract or ears for communication purpose. Communication using sign language
  5. 5. What is SLR ?? Sign language recognizer (SLR) is a tool for recognizing sign language of deaf and dumb people of the world.
  6. 6. Why we need SLR ??Problems:• About 2 million people are deaf in our world• They are deprived from various social activities• They are under-estimated to our society• Communication problem
  7. 7. Continued..Proposed Solution: SLRSLR can be a desirable interpreter which can helpboth the community general and deaf.
  8. 8. How SLR help ?? An Example.....Suppose a deaf customerwent to a shop. She is ??trying to express herdemands to theshopkeeper using signlanguage but theshopkeeper can notunderstand her demands. shopkeeper Deaf customer
  9. 9. Continued.. SLR brings the solution for this problem>> • SLR capture signs shown by deaf man • Convert the signs to text • This text is shown to shopkeeper Now the shopkeeper can understand the deaf man’s demands
  10. 10. Continued..Sign to text conversion using SLR Sign Converted text
  11. 11. Continued..Text to sign conversion When shopkeeper replied to the deaf customer SLR • Convert text to sign • This sign is shown to the deaf customer Now the deaf man can understand the shopkeeper’s speeches
  12. 12. Continued..Text to sign conversion using SLR Shopkeeper speech/text Sign
  13. 13. Text to Sign Conversion
  14. 14. Process Collecting Text • Text from the writing place are collected Separate each letters Showing sign
  15. 15. Continued.. Collecting Text • From the sentences each letter are Separate each letter separated and put into an array. Showing sign
  16. 16. Continued.. Collecting Text • For each letter a predefined sign image Separate each letters are shown. Showing sign
  17. 17. Sign to Text Conversion
  18. 18. How SLR works ?? Image processing & sign detection Normalization Sign recognitionSign to text conversion
  19. 19. Continued.. Image processing & sign detection • Image capture • Skin color detection Normalization Sign recognitionSign to text conversion
  20. 20. Continued.. Image processing & sign detection • Hand gesture detection Normalization • Sign detection Sign recognitionSign to text conversion
  21. 21. Continued.. Image processing & sign detection • Reducing image size Normalization Sign recognitionSign to text conversion 200x200 30x33
  22. 22. Continued.. Image processing & sign detection • Backpropagation implementation Normalization Sign recognitionSign to text conversion
  23. 23. Continued.. Image processing & sign detection • Converting sign language to Bengali or English text Normalization Sign recognition vSign to text conversion
  24. 24. Block diagram of the SLR
  25. 25. BP Training Figure: Training error versus number of iteration
  26. 26. Training time for BP Training Input size of pixel Time (min) 30*33 1.5 45*48 2.8 60*63 3.7 We have used 50 signs as training input where each sign has 5 samples that make 50 x 5 = 250 samples.
  27. 27. Recognition Accuracy Avg. No. of input Accuracy (%) 10 74 20 65 30 60
  28. 28. Limitations• Due to brightness and contrast sometimes webcam can hardly detect the expected skin color.• Because of the similarity of tracking environment background color and skin color the SLR gets unexpected pixels.
  29. 29. Continued..• Due to almost similar pattern its becomehard to take decision.
  30. 30. Continued..
  31. 31. Future Plan • Real time word recognition of ASL & BSL • Implementing neural network Ensembles • Implementing Genetic algorithm for sign recognition
  32. 32. Required Tools • Visual studio 2008 • XML • Avro Keyboard installed • Aforge .Net • Open CV • Webcam
  33. 33. References
  34. 34. Published papers1. Bikash Chandra Karmokar, Kazi Md. RokibulAlam, Md.KibriaSiddiquee, “An intelligent approach to recognize touchlesswritten Bengali characters”, International Conference onInformatics, Electronics & Vision (ICIEV), ISSN: 2226-2105, 2012,Dhaka, Bangladesh2. Kazi Md. RokibulAlam, Bikash Chandra Karmokar, Md.KibriaSiddiquee, “A comparison of constructive and pruningalgorithms to design neural networks”, Indian Journal ofComputer Science and Engineering (IJCSE), ISSN : 0976-5166 Vol.2 No. 3 Jun-Jul 2011