Efficient Viterbi algorithms for lexical tree based models

1.
Introduction Algorithms for expanded tree models Error-Correcting Viterbi for DAGs Experimental results and conclusions Efﬁcient Viterbi algorithms for lexical tree based models S. España Boquera, M.J. Castro Bleda, F. Zamora Martínez, J. Gorbe Moya Departamento de Sistemas Informáticos y Computación Universidad Politécnica de Valencia, Spain 22-25 May 2007, Paris, France S. España, M.J. Castro, F. Zamora, J. Gorbe NOLISP 2007

2.
Introduction Algorithms for expanded tree models Error-Correcting Viterbi for DAGs Experimental results and conclusions Index 1 Introduction 2 Algorithms for expanded tree models Left-to-right algorithm Extension to across-word context dependent units Left-to-right with skips algorithm 3 Error-Correcting Viterbi for DAGs VERTEX - STEP algorithm EDGE - STEP algorithm 4 Experimental results and conclusions Experimental results Conclusions Conclusions S. España, M.J. Castro, F. Zamora, J. Gorbe NOLISP 2007

3.
Introduction Algorithms for expanded tree models Error-Correcting Viterbi for DAGs Experimental results and conclusions Viterbi algorithms for Automatic Speech Recognition Hidden Markov Models (HMM) and Finite State Automata (FSA) are the most widespread used models for Automatic Speech Recognition (ASR). The Viterbi algorithm is used for decoding. Most of large vocabulary Viterbi based decoders make use of a lexicon tree organization together with pruning techniques such as beam search which only maintain active the best hypothesis. S. España, M.J. Castro, F. Zamora, J. Gorbe NOLISP 2007

4.
Introduction Algorithms for expanded tree models Error-Correcting Viterbi for DAGs Experimental results and conclusions Why to use a tree lexicon? The lexical tree organization is a very good tradeoff between compact space representation and adequacy for decoding: A lexicon tree organization (which has many advantages over a linear lexicon representation) reduces the size of the model. More compact representations are possible (lexicon network), but the gain in space is accompanied with a more complex Viterbi decoder. S. España, M.J. Castro, F. Zamora, J. Gorbe NOLISP 2007

5.
Introduction Algorithms for expanded tree models Error-Correcting Viterbi for DAGs Experimental results and conclusions How to use a tree lexicon? Large vocabulary one-step decoders usually keep a set of lexical tree based Viterbi parsers in parallel. Two common approaches are: time-start copies All hypothesis competing in a tree parsing share the same word start time. language model history copies When a trigram language model is used, the language model history copies approach maintains a tree parsing for every bigram history (w1, w2). The second approach has a loss of optimality which is known as word-pair approximation. S. España, M.J. Castro, F. Zamora, J. Gorbe NOLISP 2007

6.
Introduction Algorithms for expanded tree models Error-Correcting Viterbi for DAGs Experimental results and conclusions Contributions of this work In both time-start and language model history copies, faster Viterbi algorithms for the lexical tree model have a signiﬁcant impact in the overall performance of the ASR system. This work proposes Viterbi algorithms specialized for lexical tree based FSA and HMM acoustic models in order to: Decode a tree lexicon with left-to-right models VITERBI - M Decode a tree lexicon with left-to-right models with skips VITERBI - MS Parse a directed acyclic graph (DAG) (e.g. a phone graph or a word lattice) with error correcting edition operations (insertion, substitution, deletion) VITERBI - MEC - DAG S. España, M.J. Castro, F. Zamora, J. Gorbe NOLISP 2007

7.
Introduction Left-to-right algorithm Algorithms for expanded tree models Extension to across-word context dependent units Error-Correcting Viterbi for DAGs Left-to-right with skips algorithm Experimental results and conclusions Index 1 Introduction 2 Algorithms for expanded tree models Left-to-right algorithm Extension to across-word context dependent units Left-to-right with skips algorithm 3 Error-Correcting Viterbi for DAGs VERTEX - STEP algorithm EDGE - STEP algorithm 4 Experimental results and conclusions Experimental results Conclusions Conclusions S. España, M.J. Castro, F. Zamora, J. Gorbe NOLISP 2007

8.
Introduction Left-to-right algorithm Algorithms for expanded tree models Extension to across-word context dependent units Error-Correcting Viterbi for DAGs Left-to-right with skips algorithm Experimental results and conclusions Observations about the Expanded Tree Models S. España, M.J. Castro, F. Zamora, J. Gorbe NOLISP 2007

9.
Introduction Left-to-right algorithm Algorithms for expanded tree models Extension to across-word context dependent units Error-Correcting Viterbi for DAGs Left-to-right with skips algorithm Experimental results and conclusions Observations about the Expanded Tree Models If a lexicon tree is expanded with left-to-right acoustic HMM models without skips, we can observe that Every state has at most two predecessors: itself and possibly his parent. If we ignore the loops, the expanded model is acyclic. Therefore, a topological order is possible in general. A level traversal of the tree provides a topological order with some additional features: Children occupy contiguous positions. If a subset of states is stored in topological order and we generate the children of every active state following that order, the resulting list is also ordered with respect to the topological order. S. España, M.J. Castro, F. Zamora, J. Gorbe NOLISP 2007

10.
Introduction Left-to-right algorithm Algorithms for expanded tree models Extension to across-word context dependent units Error-Correcting Viterbi for DAGs Left-to-right with skips algorithm Experimental results and conclusions Model representation A tree model T of n states is represented with three vectors of size n and one of size n + 1 as follows: loop_prob stores the loop transition probabilities. from_prob stores the parent incoming transition probabilities. e_index stores the index of the associated emission probability class associated to the acoustic frame to be observed. ﬁrst_child stores the index of the ﬁrst child. S. España, M.J. Castro, F. Zamora, J. Gorbe NOLISP 2007

11.
Introduction Left-to-right algorithm Algorithms for expanded tree models Extension to across-word context dependent units Error-Correcting Viterbi for DAGs Left-to-right with skips algorithm Experimental results and conclusions VITERBI - M algorithm S. España, M.J. Castro, F. Zamora, J. Gorbe NOLISP 2007

12.

13.

14.

15.

16.

17.

18.

19.

20.

21.

22.

23.

24.

25.

26.

27.

28.
Introduction Left-to-right algorithm Algorithms for expanded tree models Extension to across-word context dependent units Error-Correcting Viterbi for DAGs Left-to-right with skips algorithm Experimental results and conclusions Extension to across-word context dependent units Coarticulated word transitions: First phone(s) of words depends on the last phone(s) of the previous words. A model which resembles a tree lexicon excepting the root is needed. Different Tree lexicon HMM models phonetic contexts General topology HMM model Coarticulated word transition The VITERBI - M can be easily adapted to this structure by treating specially the general topology at the root. S. España, M.J. Castro, F. Zamora, J. Gorbe NOLISP 2007

29.
Introduction Left-to-right algorithm Algorithms for expanded tree models Extension to across-word context dependent units Error-Correcting Viterbi for DAGs Left-to-right with skips algorithm Experimental results and conclusions Left-to-right with skips algorithm (VITERBI - MS) Left-to-right units with skips are known as Bakis topology. The states of the expanded tree lexicon can have one, two or three predecessors. The model representation is similar to that of VITERBI - M algorithm but another vector skip_prob is needed to store, for every state, the incoming skip transition probabilities. The algorithm uses another auxiliary queue to store the active states with scores computed by means of the skip transitions. S. España, M.J. Castro, F. Zamora, J. Gorbe NOLISP 2007

30.
Introduction Left-to-right algorithm Algorithms for expanded tree models Extension to across-word context dependent units Error-Correcting Viterbi for DAGs Left-to-right with skips algorithm Experimental results and conclusions Left-to-right with skips algorithm (VITERBI - MS) α (t) >= beam threshold? *loop probability aux_child α (t+1) merge generate children aux_gchild *emission probability update best probability generate grandchildren S. España, M.J. Castro, F. Zamora, J. Gorbe NOLISP 2007

31.
Introduction Algorithms for expanded tree models VERTEX - STEP algorithm Error-Correcting Viterbi for DAGs EDGE - STEP algorithm Experimental results and conclusions Index 1 Introduction 2 Algorithms for expanded tree models Left-to-right algorithm Extension to across-word context dependent units Left-to-right with skips algorithm 3 Error-Correcting Viterbi for DAGs VERTEX - STEP algorithm EDGE - STEP algorithm 4 Experimental results and conclusions Experimental results Conclusions Conclusions S. España, M.J. Castro, F. Zamora, J. Gorbe NOLISP 2007

32.
Introduction Algorithms for expanded tree models VERTEX - STEP algorithm Error-Correcting Viterbi for DAGs EDGE - STEP algorithm Experimental results and conclusions Error-Correcting Viterbi for DAGs The input is a directed acyclic graph (phone graphs, word lattices, etc.). Uses error correcting decoding with the following edition operations: insertions, substitutions and deletions. The VITERBI - MEC - DAG algorithm is a combination of: VERTEX - STEP algorithm, EDGE - STEP algorithm. following the input DAG topologic order: S. España, M.J. Castro, F. Zamora, J. Gorbe NOLISP 2007

33.
Introduction Algorithms for expanded tree models VERTEX - STEP algorithm Error-Correcting Viterbi for DAGs EDGE - STEP algorithm Experimental results and conclusions VERTEX - STEP algorithm Is applied for every vertex when all edges arriving to it have been processed This procedure only considers the cost of insertions. Several consecutive insertions are possible. A sole active state at the root could, in principle, activate all the states of the model. Beam search can be used to control this growth. S. España, M.J. Castro, F. Zamora, J. Gorbe NOLISP 2007

34.
Introduction Algorithms for expanded tree models VERTEX - STEP algorithm Error-Correcting Viterbi for DAGs EDGE - STEP algorithm Experimental results and conclusions VERTEX - STEP algorithm S. España, M.J. Castro, F. Zamora, J. Gorbe NOLISP 2007

35.

36.

37.

38.

39.

40.

41.

42.

43.

44.

45.
Introduction Algorithms for expanded tree models VERTEX - STEP algorithm Error-Correcting Viterbi for DAGs EDGE - STEP algorithm Experimental results and conclusions EDGE - STEP algorithm Is applied for every edge after the origin vertex has been updated with VERTEX - STEP algorithm. Destination vertex is updated This procedure only considers the cost of Deletions (the edge of input DAG is traversed but edge label is deleted). Substitutions (including a symbol by itself or a correct transition). S. España, M.J. Castro, F. Zamora, J. Gorbe NOLISP 2007

46.
Introduction Algorithms for expanded tree models VERTEX - STEP algorithm Error-Correcting Viterbi for DAGs EDGE - STEP algorithm Experimental results and conclusions EDGE - STEP algorithm S. España, M.J. Castro, F. Zamora, J. Gorbe NOLISP 2007

47.

48.

49.

50.

51.

52.

53.

54.

55.

56.

57.

58.

59.

60.

61.

62.
Introduction Experimental results Algorithms for expanded tree models Conclusions Error-Correcting Viterbi for DAGs Conclusions Experimental results and conclusions Index 1 Introduction 2 Algorithms for expanded tree models Left-to-right algorithm Extension to across-word context dependent units Left-to-right with skips algorithm 3 Error-Correcting Viterbi for DAGs VERTEX - STEP algorithm EDGE - STEP algorithm 4 Experimental results and conclusions Experimental results Conclusions Conclusions S. España, M.J. Castro, F. Zamora, J. Gorbe NOLISP 2007

63.
Introduction Experimental results Algorithms for expanded tree models Conclusions Error-Correcting Viterbi for DAGs Conclusions Experimental results and conclusions Experimental results Viterbi-M In order to compare the performance of our Viterbi-M algorithm, three algorithms have been implemented in C++ and tested with the same data structures to represent tree based HMM models: VITERBI - M described in this work. A conventional Viterbi. algorithm based on two hash tables with chaining to store and to look up the active states. The active envelope algorithm: uses a single linked list to store the set of states in topologic order. Since this algorithm modiﬁes the original set of active states to perform a Viterbi step, it is restricted to sequential input data and cannot be used when the input data is a DAG S. España, M.J. Castro, F. Zamora, J. Gorbe NOLISP 2007

64.
Introduction Experimental results Algorithms for expanded tree models Conclusions Error-Correcting Viterbi for DAGs Conclusions Experimental results and conclusions Experimental results The lexical trees used in the experiments were obtained by expanding 3-state left-to-right without skips hybrid neural/HMM acoustic models the tree lexicon. Only the Viterbi decoding time has been measured (the emission scores calculation and other preprocessing steps were not taken into account). The results are shown in millions of active states or hypothesis updated per second Num. states Hash swap A. Envelope Viterbi-M 9 571 3.001 16.082 29.350 76 189 2.761 12.924 28.036 310 888 1.922 6.442 24.534 S. España, M.J. Castro, F. Zamora, J. Gorbe NOLISP 2007

65.
Introduction Experimental results Algorithms for expanded tree models Conclusions Error-Correcting Viterbi for DAGs Conclusions Experimental results and conclusions Conclusions The proposed algorithms are based on contiguous memory queues which contain the set of active states topologically sorted. Less operations in the internal loops and more locality of data (cache friendly algorithm) are expected. Although the asymptotic cost of these algorithms is the same as any reasonable implementation, VITERBI - M is approximately 10 times faster than the hash-table swapping implementation and from 2 to 4 times faster than the active envelope algorithm. S. España, M.J. Castro, F. Zamora, J. Gorbe NOLISP 2007

66.
Introduction Experimental results Algorithms for expanded tree models Conclusions Error-Correcting Viterbi for DAGs Conclusions Experimental results and conclusions Conclusions More experimentation is needed in order to better understand a decrease in speed with the size of the models and the effect of other parameters such as the beam width of the pruning during the search. We are preparing experiments using HMM models trained with HTK toolkit and the the Wall Street Journal corpus. Two word graphs algorithms are being implemented: Using a tree lexicon with the time-start and the VITERBI - M algorithm. Using a phone graph generator algorithm and the VITERBI - MEC - DAG algorithm afterwards to obtain the desired word graph. S. España, M.J. Castro, F. Zamora, J. Gorbe NOLISP 2007

Efficient Viterbi algorithms for lexical tree based models

More Related Content

More from Francisco Zamora-Martinez

Recently uploaded

Efficient Viterbi algorithms for lexical tree based models