Kerie2006 Poster Template 01


Published on


Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • hhhh d
  • Kerie2006 Poster Template 01

    1. 1. <ul><li>Main Capabilities of SSLIS: </li></ul><ul><li>Real-time speech to text to on-screen video sign language translation (in English word order). </li></ul><ul><li>Real-time speech to text translation. </li></ul><ul><li>Text to on-screen video sign language. </li></ul><ul><li>Text to computer generated voice. </li></ul><ul><li>Recognition’s accuracy calculations in terms of Word recognition Error Rate (WER) with supplementary examples. </li></ul><ul><li>List of the sign language database constituents and definitions of non-basic words that our software is capable to recognize. </li></ul><ul><li>A simple drag and drop tool allows end users to have email, web pages, and documents created in other applications signed. </li></ul>Speech to Sign Language Interpreter System (SSLIS) Khalid K. El-Darymli, Othman O. Khalifa and Hassan Enmsoah Department of Computer Engineering, Faculty of Engineering International Islamic University Malaysia, PO BOX 10, Kuala Lumpur, 50728, Malaysia Phone: 03-6196-4433, Fax: 03-6196-4433, E-mail: Figure 1: Flowchart depicts basic structure of SSLIS Figure 2: Flowchart depicts simplified structure of Sphinx 3.5 speech engine Figure 3: Signed English markers Figure 4: ASL alphabets Figure 5: Demonstration of ASL in SSLIS <ul><li>1. Research Goal and Objectives: </li></ul><ul><li>Design and Manipulation of Speech to Sign Language Interpreter System . </li></ul><ul><li>To fill the gap between deaf and nondeaf people in two senses. Firstly, by using this SW for educational purposes for deaf people and secondly, by facilitating the communication between deaf and nondeaf people. </li></ul><ul><li>To increase independence and self-confidence of the deaf person. </li></ul><ul><li>To increase opportunities for advancement and success in education, employment, personal relationships, and public access venues. </li></ul><ul><li>To improve quality of life. </li></ul><ul><li>2. SSLIS Capabilities: </li></ul><ul><li>Real time speech to text to video sign language. </li></ul><ul><li>Text to video sign language. </li></ul><ul><li>Automatic WER calculation. </li></ul><ul><li>Text to computer generated voice with synchronized lips. </li></ul><ul><li>Speed control of ASL movies in play. </li></ul><ul><li>“ Minimize to Auto” allows “drag and drop” from any text editor to be signed. </li></ul><ul><li>Demonstration of SE manual as parallel to English. </li></ul><ul><li>Demonstration of decoding process of speech. </li></ul><ul><li>“ Live Decode” Program allows real time speech recognition while “Live Pretend” and “Decode” allows speech recognition in batch mode. </li></ul><ul><li>3. Conclusions </li></ul><ul><li>The research aim of offering freely available and open source SSLIS is fulfilled. </li></ul><ul><li>Sphinx 3.5 was manipulated as the SR engine. </li></ul><ul><li>Signed English manual was followed for translation. </li></ul><ul><li>4. Shortcomings and Further Work: </li></ul><ul><li>Degradation in the speech recognition accuracy [an acoustic model generated from dictation tasks DB needs to be generated and employed], </li></ul><ul><li>To highly improve the speech recognition accuracy, speech recognition engine can be adapted to different users through introducing enrollment session. </li></ul>Figure 4: ASL alphabets Figure 5: Demonstration of ASL in SSLIS Figure 6: Some Snapshots of SSLIS Regular past verbs:-ed talk ed , want ed , learn ed Regular plural nouns: s bears, houses 3 rd person singular : -s walks, eats, sings Irregular past verbs: (sweep RH open B, tips out, to the right) saw, heard, blew Irregular plural nouns: (sign the word twice) children, sheep, mice mice Possessive: -'s cat's, daddy's, chair's verb form: -ing climb ing , play ing, running Adjective: -y Sleepy, sunny, cloudy Adverb: -ly Beautifully, happ ily , nice ly Participle: Fallen, gone, grown Comparative: -er small er , fast er , long er Superlative: -est Small est , fast est , long est Opposite of: un-, im-, in-, etc. (made before the sign word, as a prefix) un happy, im patient, in considerate Agent (person): (sign made near the body) teach er , act or , art ist Agent (thing) (sign made away from the body) wash er , dry er , plant er