Hidden Markov models (HMMs) are a set of rules used to model probabilistic systems where an underlying process with hidden states influences a series of observable outputs. The document discusses: 1. HMMs involve hidden states that influence observable outputs according to transition and emission probabilities. 2. Two common tasks for HMMs are calculating the probability of a sequence of outputs given a model, and training a model to maximize the probability of an observed sequence. 3. The forward and backward algorithms use dynamic programming to efficiently solve these tasks, with the forward algorithm calculating probabilities and the backward algorithm supporting training via the Baum-Welch algorithm.

1. Hidden Markov
Model
Nghia Bui
Nov 2016
Andrei Markov (1856-1922)

2. The weather problem
• I talked to Jane for 𝐿 days through telephone.
Everyday she told me what she does, either
“walk” or “shop” or “clean”, only one!
• I know, on a day, the weather in her city can
be either “sunny” or “rainy”, only one!
• But she didn’t tell me exactly the weather on
the 𝐿 days, and how it affected her actions.
• Then I have to figure out by myself!  HMM
2

3. HMM is just a set of 3 rules
• If today weather is 𝑺𝒊
then tmrw it will be 𝑺𝒋
with probability 𝒂𝒊𝒋
• When weather is 𝑺𝒊
Jane will do action 𝑶 𝒌
with probability 𝒃𝒊(𝒌)
• In the 1st day, the
weather is 𝑺𝒊 with
probability 𝝅𝒊
3
https://en.wikipedia.org/wiki/Hidden_Markov_model

4. What are hidden?
• The states of weather 𝑆𝑖(𝑖 = 1 … 𝑁) {“sunny”,
“rainy”} are not observable  they are hidden
• The actions 𝑂 𝑘(𝑘 = 1 … 𝑀) {“walk”, “shop”,
“clean”} are observed in an index sequence
𝑜ℎ ℎ = 1 … 𝐿 where 𝑜ℎ = 𝑘 1 ≤ 𝑘 ≤ 𝑀
4

5. Two common tasks
1. Given a model 𝜆(𝑎, 𝑏, 𝜋) and a sequence of
action indexes 𝑜 = 𝑜1, 𝑜2 … 𝑜 𝐿 please
calculate the probability 𝑃(𝑜|𝜆) the model
generates the sequence.
 The forward algorithm
2. Given a sequence 𝑜, build a model 𝜆 so that
𝑃(𝑜|𝜆) is maximum.
 The Baum-Welch algorithm
5

6. The forward algorithm
• Let 𝛼𝑖 ℎ be the probability of generating the
sequence 𝑜1 … 𝑜ℎ(ℎ = 1 … 𝐿) and ending up
at state 𝑆𝑖
• Using dynamic programming we have:
 𝛼𝑖 ℎ = 𝑗=1
𝑁
𝛼𝑗 ℎ − 1 𝑎𝑗𝑖 𝑏𝑖(𝑜ℎ)
 𝛼𝑖 1 = 𝜋𝑖 𝑏𝑖(𝑜1)
• And result: 𝑃 𝑜 𝜆 = 𝑖=1
𝑁
𝛼𝑖(𝐿)
6

7. The Baum-Welch algorithm
• Given a model 𝜆 𝑎, 𝑏, 𝜋 , we use it to generate many
sequences, but consider only the ones that emit
𝑜1, 𝑜2 … 𝑜 𝐿:
Main idea: init with a random model
and make it better incrementally
𝑆1 𝑆2 … 𝑆2 𝑆1
𝑆2 𝑆2 … 𝑆1 𝑆1
… … … … …
𝑆1 𝑆1 … 𝑆1 𝑆2
𝑜1 𝑜2 … 𝑜 𝐿−1 𝑜 𝐿
• Nothing is hidden in these sequences! Now we simply
base on them to estimate 𝑎′, 𝑏′, 𝜋′
7

8. Estimate 𝑎′, 𝑏′, 𝜋′
• To estimate 𝑎′𝑖𝑗, count the transitions from 𝑆𝑖 to
𝑆𝑗 and to other states
• To estimate 𝑏′
𝑖(𝑘), count the appearances of 𝑆𝑖
that have action index 𝑜ℎ = 𝑘, also count all the
appearances of 𝑆𝑖
• To estimate 𝜋′𝑖, count the appearances of 𝑆𝑖 at
the first element of all sequences, and count the
number of all sequences too
• But, to count all of things above, we need …
8

9. Forward and backward variables
• Using the forward algorithm we have 𝛼𝑖 ℎ
• Using the backward algorithm we have 𝛽𝑖 ℎ
the probability of generating the sequence
𝑜ℎ+1 … 𝑜 𝐿(ℎ = 1 … 𝐿) starting from tmrw,
given the state 𝑆𝑖 of today. Dynamic
programming is used again:
 𝛽𝑖 ℎ = 𝑗=1
𝑁
𝑎𝑖𝑗 𝑏𝑗(𝑜ℎ+1)𝛽𝑗 ℎ + 1
 𝛽𝑖 𝐿 = 1
9

10. Estimate 𝑎′
• Count transitions from 𝑆𝑖 to 𝑆𝑗:
𝜉𝑖𝑗 =
ℎ=1
𝐿−1
𝛼𝑖(ℎ)𝑎𝑖𝑗 𝑏𝑗(𝑜ℎ+1)𝛽𝑗(ℎ + 1)
• Thus:
𝑎′𝑖𝑗 =
𝜉𝑖𝑗
𝑘=1
𝑁
𝜉𝑖𝑘
10

11. Estimate 𝑏′
• Count the appearances of state 𝑆𝑖:
ℎ=1
𝐿
𝛼𝑖 ℎ 𝛽𝑖 ℎ
• Thus:
𝑏′
𝑖 𝑘 =
ℎ=1,𝑜ℎ=𝑘
𝐿
𝛼𝑖 ℎ 𝛽𝑖 ℎ
ℎ=1
𝐿
𝛼𝑖 ℎ 𝛽𝑖 ℎ
11

12. Estimate 𝜋′
• Count the appearances of 𝑆𝑖 at the first element:
𝛼𝑖 1 𝛽𝑖 1
• Count the number of all sequences:
𝑃 𝑜 𝜆 =
𝑖=1
𝑁
𝛼𝑖 𝐿
• Thus:
𝜋′𝑖 =
𝛼𝑖 1 𝛽𝑖 1
𝑃 𝑜 𝜆
12

13. Thank you!
• Contact: katatunix@gmail.com
13