1. Hidden Markov Models By Marc Sobel

6. Andrei Andreyevich Markov Born: 14 June 1856 in Ryazan, Russia Died: 20 July 1922 in Petrograd (now St Petersburg), Russia Markov is particularly remembered for his study of Markov chains, sequences of random variables in which the future variable is determined by the present variable but is independent of the way in which the present state arose from its predecessors. This work launched the theory of stochastic processes .

8. Chain Rule & Markov Property Bayes rule Markov property

9. s 1 s 3 s 2 Has N states, called s 1 , s 2 .. s N There are discrete timesteps, t=0, t=1, … N = 3 t=0 A Markov System

11. A Plot of 100 observed numbers for the stochastic automoton

12. Histogram for the stochastic automaton: the proportions reflect the stationary distribution of the chain

18. The Coin Toss Example – 2 coins

19. From Markov to Hidden Markov Model: The Coin Toss Example – 3 coins

22. Doubly Stochastic System The Urn-and-Ball Model O = {green, blue, green, yellow, red, ..., blue} How can we determine the appropriate model for the observation sequence given the system above?

27. Observed colored balls in the hmm model

31. Particle Results: based on 50 observations

33. Viterbi learning versus the actual state (estimate =3; 62% accuracy)

38. Coin tossing model: results