A stochastic process is a sequence of random variables indexed by time. A discrete-time Markov chain is a stochastic process where the probability of the next random variable only depends on the current variable, not on previous variables. This conditional independence property is represented as Pr(Xt+1 = j | Xt = a) = Pij.