Sign language translator ieee power point

5,653 views

Published on

Sign language translator ieee power point

  1. 1. Dept of Biomedical EngineeringAUTOMATIC LANGUAGE TRANSLATION SOFTWAREFOR AIDING COMMUNICATION BETWEEN INDIANSIGN LANGUAGE AND SPOKEN ENGLISH USINGLABVIEWByYELLAPU MADHURI,Reg.No.1651110002,MTECH II YEAR,SRM University.Guided byMs.G.ANITHAAssistant professor (O.G) /BME
  2. 2. Dept of Biomedical EngineeringINTRODUCTION SIGN LANGUAGE (SL)Natural way of communication of speech and/or hearing-impaired people. SIGNMovement of one or both hands, accompanied with facial expression, whichcorresponds to a specific meaning. TRANSLATORCommunication between speech and/or sound impaired person and person that donot understand sign language, avoiding by this way the intervention of anintermediate person. And allow communication using their natural way of speaking.
  3. 3. Dept of Biomedical EngineeringANATOMY OF HUMAN EAR
  4. 4. Dept of Biomedical EngineeringEVENTS INVOLVED IN HEARING
  5. 5. Dept of Biomedical EngineeringSPEECH CHAIN
  6. 6. Dept of Biomedical EngineeringAIM To develop a mobile interactive application software for automatic translation ofIndian sign language into speech in English and vice-versa to assist thecommunication between speech and/or hearing impaired people with normalpeople. This language translator should be able to translate one handed fingerspelling input of Indian Sign language alphabets A-Z and numbers 1-9 into spokenEnglish audio output and 165 spoken English words input to Indian Sign languagepicture display output.
  7. 7. Dept of Biomedical EngineeringGRAPHICAL ABSTRACT
  8. 8. Dept of Biomedical EngineeringOBJECTIVE For Sign to Speech conversion1. Acquire images using the inbuilt camera of the device.2. Perform vision analysis functions in the operating system and provide speech outputthrough the inbuilt audio device. For Speech to Sign conversion1. Acquire speech input using the inbuilt microphone of the device.2. Perform speech analysis functions in the operating system and provide visual signoutput through the inbuilt display device. Minimize hardware requirements and expense.
  9. 9. Dept of Biomedical EngineeringLITERATURE REVIEW1. Jose l. Hernandez-rebollar et al [2004]Discusses a novel approach for capturing and translating isolatedgestures of ASL into spoken and written words using combinedacceleglove and a two-link arm skeleton.2. Paschaloudi N.Vassilia et al [may2006]Extensible system to recognize GSL modules for signs or finger-spelled words, using isolation or combined neural networks3. Beifang yi [ may 2006] Explorations in the areas of computer graphics, interface design,and human-computer interactions with emphasis on softwaredevelopment and implementation in ASLT4. Andreas domingo et al[2007]ASLT using pattern-matching algorithm.5. Rini akmeliawatil et al[may 2007]ASLT for real-time english translation of the malaysia SL usingneural networks.6. Abang irfan halil et al[ 2007]Extent of development details on recognition system by usingstate-of-the-art graphical programming software
  10. 10. Dept of Biomedical EngineeringALGORITHM CRITERION1. REAL-TIME2. VISION-BASED3. AUTOMATIC AND CONTINUOUS OPERATION4. EFFICIENT TRANSLATION
  11. 11. Dept of Biomedical EngineeringMATERIALSSoftware Tools used: National Instruments LabVIEW and toolkits LABVIEW 2012 version Vision Development Module Vision acquisition ModuleHardware tools used: Laptop inbuilt webcamera- Acer Crystal Eye Laptop inbuilt speaker-Acer eAudio
  12. 12. Dept of Biomedical EngineeringGUI OF SOFTWARE
  13. 13. Dept of Biomedical EngineeringPAGE 2- SPEECH TO SIGN LANGUAGE TRANSLATOR
  14. 14. Dept of Biomedical EngineeringBLOCK DIAGRAM OF SPEECH TO SIGN LANGUAGETRANSLATOR
  15. 15. Dept of Biomedical EngineeringFLOW CHART OF SPEECH TO SIGN LANGUAGE TRANSLATION
  16. 16. Dept of Biomedical EngineeringWINDOWS SPEECH RECOGNITION TUTORIAL
  17. 17. Dept of Biomedical EngineeringWINDOWS SPEECH RECOGNITION SOFTWARE GUI
  18. 18. Dept of Biomedical EngineeringUSER INTRFACE OF SPEECH TO SIGN LANGUAGE TRANSLATOR
  19. 19. Dept of Biomedical EngineeringPAGE 3- TEMPLATE PREPARATION
  20. 20. Dept of Biomedical EngineeringIMAGE ACQUISITION SEQUENCE OF FRAMES
  21. 21. Dept of Biomedical EngineeringUSER INTERFACE OF TEMPLATE PREPARATION FOR SIGNLANGUAGE TO ENGLISH TRANSLATION
  22. 22. Dept of Biomedical EngineeringFLOW CHART OF TEMPLATE PREPARATION FOR SIGN LANGUAGE TOENGLISH TRANSLATION
  23. 23. Dept of Biomedical EngineeringPAGE 4- PATTERN MATCHING
  24. 24. Dept of Biomedical EngineeringUSER INTERFACE OF PATTERN MATCHING FOR SIGN LANGUAGETO ENGLISH TRANSLATION
  25. 25. Dept of Biomedical EngineeringBLOCK DIAGRAM OF SIGN LANGUAGE TO SPEECHTRANSLATOR
  26. 26. Dept of Biomedical EngineeringDATABASE OF ONE HANDED ALPHABETS AND NUMBERS OF SIGN LANGUAGE
  27. 27. Dept of Biomedical EngineeringADVANTAGES Eliminates the need for an interpreter for communication between sign languageand speech language. Easy to incorporate and execute in any supporting operating system. Real time translation. Does not require any additional hardware.
  28. 28. Dept of Biomedical EngineeringFUTURE APPLICATIONSWeb conferenceCOMPUTER AND VIDEO GAMESPRECISION SURGERYDOMESTIC APPLICATIONSWEARABLE COMPUTERS
  29. 29. Dept of Biomedical EngineeringCHALLENGESBackground subtraction for robust usage.Making the system user independent.Pattern matching training.
  30. 30. Dept of Biomedical EngineeringLIMITATIONS System is trained on a limited database.. Possibility of misinterpretation for closely related gestures. Translates only static signs. Not trained to translate dynamic signs. Facial expressions are not considered. Possibility of misinterpretation for words of similar pronunciation.
  31. 31. Dept of Biomedical EngineeringCONCLUSION The feature vectors which include whole image frames containing all the aspects ofthe sign are considered. The geometric features which are extracted from the signers’ dominant hand, improvethe accuracy of the system to a great degree. Training the speech recognition for shorter phrases is difficult than longer phrases.
  32. 32. Dept of Biomedical EngineeringFUTURE WORK To increase the performance and accuracy of the ASLT, the quality of the trainingdatabase used should be enhanced to ensure that the ASLT picks up correct andsignificant characteristics in each individual sign and further improve theperformance more efficiently. Current collaboration with Assistive Technology researchers and members of theDeaf community for continued design work should be considered for continuedprogress. This project did not focus on facial expressions although it is well known that facialexpressions convey important part of sign-languages. This system can be implemented in many application areas examples includeaccessing government websites whereby no video clip for deaf and mute is availableor filling out forms online whereby no interpreter may be present to help.
  33. 33. Dept of Biomedical EngineeringREFERENCES Andreas Domingo, Rini Akmeliawati, Kuang Ye Chow ‘Pattern Matching for AutomaticSign Language Translation System using LabVIEW’, International Conference onIntelligent and Advanced Systems 2007. Beifang Yi Dr. Frederick C. Harris ‘A Framework for a Sign Language InterfacingSystem’, A dissertation submitted in partial fulllment of the requirements for the degreeof Doctor of Philosophy in Computer Science and Engineering May 2006 University ofNevada, Reno. Helene Brashear & Thad Starner ‘Using Multiple Sensors for Mobile Sign LanguageRecognition’, ETH - Swiss Federal Institute of Technology Wearable ComputingLaboratory 8092 Zurich, Switzerland flukowicz, junker g@ife.ee.ethz.ch
  34. 34. Dept of Biomedical Engineering Jose L. Hernandez-Rebollar1, Nicholas Kyriakopoulos1, Robert W. Lindeman2 ‘ANew Instrumented Approach For Translating American Sign Language Into SoundAnd Text’, Proceedings of the Sixth IEEE International Conference on AutomaticFace and Gesture Recognition (FGR’04) 0-7695-2122-3/04 $ 20.00 © 2004 IEEE. K. Abe, H. Saito, S. Ozawa: Virtual 3D Interface System via Hand MotionRecognition From Two Cameras. IEEE Trans. Systems, Man, and Cybernetics, Vol.32, No. 4, pp. 536–540, July 2002.
  35. 35. Dept of Biomedical EngineeringTHANK YOU

×