Applied Stochastic process MTH 412 IIT Kanpur

1. Home Work 4
MTH 412
Applied Stochastic Process
1. Prove that for an Ergodic Markov chain the stationary distribution is unique.
2. Consider the following transition probability matrix with the state space S = {0, 1, 2, 3}
P =





1/6 1/3 1/2 0
1/2 1/2 0 0
1/6 1/3 1/2 0
0 1/6 1/3 1/2





Find the recurrent and transient classes. Find the probability that starting from the
transient state i, it is going to get absorbed in the given recurrent state j. Find the
expected time to get absorbed in a recurrent class.
3. For a finite absorbing chain suppose the transition probability matrix P is written as
follows:
P =
P1 0
R1 Q
Find Pn
.
4. Prove or give a counter example that for an infinite recurrent Markov chain stationary
probability distribution does not exist.
5. Prove or give a counter example that for a finite Markov chain with transient states
stationary probability distribution does not exist.
6. Prove the following:
TNn ≤ n < TNn+1.
1