Final year project presentation

1. Supervisor
Dr. Basanta Joshi
Aavaas Gajurel (068/BCT/501)
Anup Pokhrel (068/BCT/505)
Manish K. Sharma (068/BCT/523)

2. System Overview

3. System Block Diagram - Training
Noise
Reduction
Split Module
(VAD Based) Training Set
MFCC
Features Train HMM

4. System Block Diagram - Recognition
Audio Input
Noise
Reduction
Split Module
(VAD Based)
MFCC
Computation
HMM
Audio
ClassifierLanguage Model Output

5. SYSTEM DESIGN METHODOLOGY

6. NOISE REDUCTION

7. Creating Noise Profile
BUILD NOISE PROFILE
Update the computed Noise Profile
AVERAGE OVER TIME
1
𝑁
[𝑆𝑢𝑚 𝑜𝑓 𝐹𝐹𝑇 𝐶𝑜𝑚𝑝𝑜𝑛𝑒𝑛𝑡 > 10 𝑓𝑟𝑎𝑚𝑒𝑠]
FOURIER TRANSFORM
FFT of 32ms Audio Samples

8. Spectral Subtraction
INVERSE FOURIER TRANSFORM
Rebuild the Signal
SUBTRACT NOISE PROFILE (STATIC AND MUSICAL)
Over Subtraction Short Segment Removal
FOURIER TRANSFORM OF SIGNAL
FFT of 32ms Audio Samples

9. After Spectral Subtraction After Musical Noise Removal
Before Noise Removal Spectral Subtraction output

10. VOICE ACTIVITY DETECTION

11. Voice Activity Detection

12. Voice Activity Detection Process I
CALCULATE THE TRIGGER
𝑡 𝑤 = 𝜇 + 𝛼𝛿 𝑤
COMPUTE mean AND variance
SAMPLE
10 Frame Sampling

13. Voice Activity Detection Process II
CLASSIFY
If greater than trigger then voice
COMPUTE CLASSIFICATION MEASURE
READ THE SAMPLE
Read the frame
𝑊𝑠1 𝑚 = 𝑃𝑠1(𝑚) 1 − 𝑍 𝑠1 𝑚 𝑆𝑐

14. Feature Extraction

15. Audio Feature Extraction

16. Feature Extraction Process I
APPLY MEL FILTERBANK
Multiply Filterbank(20-40) by Periodogram Estimate
CALCULATE PERIODOGRAM ESTIMATE
𝑃𝑖 𝑘 =
1
𝑁
| 𝑆𝑖(𝑘)|2
FRAMING
Divide Audio into Sections of 20ms-40ms

17. Feature Extraction Process II
KEEP REQUIRED COEFFICIENTS
Keep Required Number of Coefficients
DISCRETE COSINE TRANSFORM OF ENERGIES
Take DCT of Coefficients of Above Step
SCALING
Take Logarithm of Filterbank Energies

18. Language Model

19. Using Language Model

20. Language Model Training
UPDATE DICTIONARY
Add to existing Dictionary
CREATE LANGUAGE MODELS
Derive NGram Models
FETCH RAW DATA
Currently From News Websites

21. Language Model Based Classification
SELECT BEST
𝑃 𝑊𝑖 𝑊𝑖−1 = 𝜆1 𝑃 𝑊𝑛 𝑊𝑛−1 + 𝜆2 𝑃(𝑊𝑛)
GET POSSIBLE CANDIDATES
From Acoustic Model
READ PREVIOUS WORD

22. ACOUSTIC MODEL

23. HMM Based Classification

24. Training the Acoustic Model
TRAIN USING BAUM WELCH ALGORITHM
SELECT HMM MODEL
READ MFCC COEFFICIENTS AND WORD

25. Using the Acoustic Model
OUTPUT WORD CORRESPONDING TO MODEL
SELECT MODEL WITH MAXIMUM PROBABILITY
FIND LOG PROBABILITY OF WORD FOR EACH MODEL
READ MFCC COEFFICIENTS OF WORD

26. RESULTS

27. Trained vs. Untrained Input
• 3 Speakers
• 5X10 Words Each
• 5 Testing Set Each
86.67
66.67
0
10
20
30
40
50
60
70
80
90
100
Accuracy of System
Using Trained and Untrained Input
Trained Set Untrained Set

28. Noise Reduced vs. Not Noise Reduced
• 3 Speakers
• 5X10 Words Each
• Untrained Input Files for Testing
• 5 Testing Set Each
46.67
66.67
0
10
20
30
40
50
60
70
80
Accuracy of System
Effect of Noise Reduction
Noise Not Reduced Noise Reduced

29. Gender Based Results
• 7 Speakers
• 3 Females, 4 Males
• Animal Names as Test
• Untrained Input Files for Testing
36
64
59
66
44
56
51
54
58
0
10
20
30
40
50
60
70
Female Voice Training Male Voice Training Female and Male Voice
Training
Gender Based Result
Male Female Male + Female

30. LIMITATIONS AND
RECOMMENDATIONS

31. Limitations
Limited Vocabulary
User Specific Noise Profiles
Static MFCC Coefficients Only
Training Data Storage Absent
Non-Continuous Recognition

32. Recommendations
Using Dynamic Coefficients
Continuous HMM Model
Extensive Training
Better Phonemic Modeling
Dynamic Noise Modeling

33. USAGE SCENARIO

34. Usage Scenario I
Easy Nepali Input
Automated Telecom Assistance
Speech Controlled Interface
Automated Transcribing

35. Usage Scenario II
Military Sector for Automated Wire Tapping
Public Guidance System
Automated User Support (banks, corporate houses,etc.)

36. Thank You !