The document discusses the architecture and advantages of deep neural network-hidden Markov model (DNN-HMM) hybrid systems for speech recognition, highlighting performance improvements over traditional Gaussian mixture model-HMM systems. Key points include the effective training and decoding process utilizing the discriminative nature of DNNs and the importance of model depth and context-dependent state modeling. The findings indicate that DNN-HMM systems enable significant advancements in automatic speech recognition technology.

1. Deep Neural
Network
Hidden Markov
Model
Hybrid Systems

2. Agenda
01
DNN-HMM Architecture
02
03 Advantages of DNN-HMM
04
Introduction
Training Procedure
Overview and advantages of DNN-HMM hybrid systems
Key components, illustration, and key equations
Discriminative nature, efficient training and decoding,
performance benefits
Step-by-step process, embedded Viterbi training algorithm, key
equations
05
Depth of Neural Networks
07
08 Use of Neighboring Frames
Context Dependent vs Independent Models
Comparison, Performance improvement
Importance, empirical results, key equations related to
performance metrics
Steps, Training Algorithms
09 Key Findings and Conclusion
Key Findings and conclusion of DNN-HMM hybrid-system

3. • Combining the representation learning power of
Deep Neural Networks (DNNs) with the sequential
modeling capability of Hidden Markov Models
(HMMs).
• Significant improvements over traditional Gaussian
Mixture Model-HMM (GMM-HMM) systems in speech
recognition.
Introduction
Overview

4. • HMMs: Model the sequential nature of speech signals.
• DNNs: Estimate the observation probabilities (posterior probabilities)
for HMM states.
where p(xt qt=s)
∣ is the likelihood, p(qt=s xt)
∣ is the posterior
probability estimated by the DNN, and p(s) is the prior probability of
state s.
DNN-HMM Architecture

5. Advantages of DNN-HMM
DNNs are inherently discriminative,
providing better classification.
Discriminative Nature
Uses the embedded Viterbi algorithm
for training and efficient decoding.
Efficient Training and Decoding
Outperforms GMM-HMM systems in
large vocabulary continuous speech
recognition (LVCSR).
Superior Performance

6. Training Procedure
Algorithm:
• Input Preparation: Convert speech into frames.
• IOU-Based Sampling: Create positive and negative
samples.
• DNN Processing: Compute state probabilities from
sampled windows.
• HMM Decoding: Decode probabilities to find likely
state sequences and generate detection scores.
• Model Training: Iteratively train the DNN and HMM.
• Evaluation: Use the trained model to detect target
speech events in new utterances.
Embedded Viterbi training for CD-DNN-HMMs (Context-
Dependent DNN-HMMs).
Steps:

7. Context-Dependant vs Independent Models
• Use context-independent phone
states.
• Simpler model but less accurate.
Monophone State Models
• Use context-dependent triphone
states (senones).
• More complex but significantly
more accurate.
Senone Models
• Directly modeling senones captures more detailed acoustic
variations.
• Leads to significant error rate reduction.
Performance
Improvement

8. Depth of Neural
Networks
Importance of Depth
• Deeper networks significantly outperform
shallow ones.
• Performance improvements stop increasing
after a certain number of layers.
Empirical Results
• WER (Word Error Rate) and SER (Sentence
Error Rate) reductions with increasing layers.

9. Use of Neighboring Frames
Benefit
• Including a window of neighboring frames improves
accuracy.
Comparison
• DNNs can exploit temporal correlations, unlike
GMMs which assume frame independence.

10. Key Findings and
Conclusion
Deep neural networks with sufficient depth.
Using a long window of input frames.
Directly modeling context-dependent states (senones).
✓
✓
✓
DNN-HMM hybrid systems represent a significant advancement in automatic
speech recognition technology, using both DNN and HMM strengths.
Conclution

11. Thank
you