1. By: Khalid El-Darymli G0327887 Speech to Sign Language Interpreter System Supervisor: Dr. Othman O. Khalifa International Islamic University Malaysia Kulliyyah of Engineering, ECE Dept.
2.
3.
4.
5. Main Parts of Speech to Sign Language Interpreter System Speech-Recognition Engine ASL pre-recorded Video-clips Database Recognized Text ASL Translation Continuous Input Speech Recognized Text
6.
7. The Structure of SR Engine (LVCSR) Signal Processing AM P ( A 1 , …, A T | P 1 ,… , P k ) Dictionary P ( P 1 , P 2 , …, P k | W ) LM P ( W n | W 1 , …, W n-1 ) X={x 1 ,x 2 , …, x T } Hypothesis Evaluation Decoder P(X | W)*P(W) TRAINING DECODING Best Hypotheses H = {W 1 , W 2 , …, W k } W BEST Input Audio
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20. ASL vs. SE (an Example) It is alright if you have a lot ASL Translation SE Translation IT I S ALL RIGHT IF YOU HAVE A LOT
21. DEMONSTRATION OF THE ASL IN OUR SW: A number of 2,600 ASL prerecorded video clips In case of nonbasic word, extract the basic word out of it Recognized Word (SR engine’s output) Is the basic word within the ASL database vocabulary? The American Manual Alphabet Only in case of a nonbasic input word, append some suitable marker Final Output None of the database contents matched the input basic word No Yes Fingerspelling of the original input word The equivalent ASL video clip of the input word, some marker could be appended
22. Speech to Sign Language Interpreter System - MILESTONE Thesis Writing Outline & Progress SW Development & Progress % Drafted Chapter 2: State-of-the-Art of SR Chapter 3: Sphinx SR Chapter 4: Sphinx Decoder Chapter 5: Sign Language Chapter 6: SW Demo ., Conclusions & Further Work Appendices SR Engine ASL Database Overall Integrated SW Chapter 1: Introduction % Completed
![OUTLINE ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)