Markov Assumption, also known as the Markov property, is a fundamental concept in probability theory and stochastic processes. It refers to the assumption that the future state of a system depends solely on its current state and is independent of its past states. In other words, given the present state, the probability distribution of future states is not influenced by the sequence of events that led to the current state.

1. Markov Assumptions and its application 1
Markov Assumption and its Application
Presented By:
Shahriar Ahsan Taisiq (201002396)
Shagor Kumar Das (201002403)
Fakir Tohidul Islam (201002402)
Presented To:
Dr. Muhammad Abul Hasan
Professor
Department of CSE
Green University of Bangladesh

2. Markov Assumptions and its application 2
Contents
✔ Introduction
✔ Markov Chain
✔ Application
✔ Advantage
✔ Limitation
✔ Conclusion

3. Markov Assumptions and its application 3
Introduction
The Markov Assumption states that the future state of a system depends only on its
current state.
𝑋𝑡−1 𝑋𝑡 𝑋𝑡+1
yesterday today tomorrow
State:
×
Mathematically, the Markov assumption
can be expressed as follows:
𝑃 𝑋𝑡+1 𝑋𝑡,𝑋𝑡−1,………,𝑋1) = 𝑃 𝑋𝑡+1 𝑋𝑡)
𝑋𝑡 = state of the process time t

4. Markov Assumptions and its application 4
Markov Chain
up down
up down
0.7 0.6
0.3
0.4
𝑃 𝑋𝑡+1 = 𝑢𝑝 𝑋𝑡 = 𝑢𝑝)
= 𝑃 𝑢𝑝 → 𝑢𝑝 = 0.7
𝑃 𝑋𝑡+1 = 𝑑𝑜𝑤𝑛 𝑋𝑡 = 𝑢𝑝)
= 𝑃 𝑢𝑝 → 𝑑𝑜𝑤𝑛 = 0.3
Suppose we are interested in predicting the market state two days ahead. We can
continue applying the Markov assumption recursively.
𝑃 𝑋𝑡+2 = 𝑢𝑝 𝑋𝑡+1 = 𝑢𝑝, 𝑋𝑡 = 𝑢𝑝) = 𝑃 𝑋𝑡+1 = 𝑢𝑝|𝑋𝑡 = 𝑢𝑝 ∗ 𝑃 𝑢𝑝 → 𝑢𝑝
= 0.7 ∗ 0.7 = 0.49
𝑃 𝑋𝑡+2 = 𝑢𝑝 𝑋𝑡+1 = 𝑑𝑜𝑤𝑛, 𝑋𝑡 = 𝑢𝑝) = 𝑃 𝑋𝑡+1 = 𝑑𝑜𝑤𝑛|𝑋𝑡 = 𝑢𝑝 ∗ 𝑃 𝑑𝑜𝑤𝑛 → 𝑢𝑝
= 0.3 ∗ 0.4 = 0.12

5. Markov Assumptions and its application 5
Application
• Markov Chains: Markov chains are mathematical models that exhibit the Markov
assumption.
𝑃 𝑋𝑡+1 𝑋𝑡,𝑋𝑡−1,………,𝑋1) = 𝑃 𝑋𝑡+1 𝑋𝑡)
• Natural Language Processing: In language modeling and text generation tasks,
the Markov assumption is applied to create n-gram models.

6. Markov Assumptions and its application 6
Application (Cont..)
• Hidden Markov Models (HMMs): HMMs are statistical models that
incorporate both observed and hidden states.

7. Markov Assumptions and its application 7
Application (Cont..)
• Reinforcement Learning: Markov decision processes (MDPs) form the
basis for many reinforcement learning algorithms.
• Genetics and Bioinformatics: Markov models are employed in genome
analysis, protein structure prediction, and sequence alignment.

8. Markov Assumptions and its application 8
Advantages
• Simplifies complex systems
• Memoryless property
• Mathematical tractability
• Predictive power

9. Markov Assumptions and its application 9
Limitations
• Lack of long-term dependencies
• Independence assumption
• Independence assumption
• Fixed transition probabilities
• Model order selection

10. Markov Assumptions and its application 10
Conclusion
Markov chains provide a useful framework for representing systems that adhere
to these assumptions. While there are limitations to the Markov assumptions,
such as:
• the assumption of independence
• lack of flexibility
Higher-order Markov models can be employed to address these limitations by
considering a certain number of previous states.

11. Markov Assumptions and its application 11
References
[1] https://www.igi-global.com/dictionary/markov-assumption/37576
[2] https://en.wikipedia.org/wiki/Markov_chain
[3] https://singhharsh246.medium.com/basics-of-nlp-text-classification-
with-markov-assumption-415ce51ca62e
[4] https://ai.stackexchange.com/questions/16667/what-does-the-markov-
assumption-say-about-the-history-of-state-sequences

12. Does anyone have
any questions?
Markov Assumptions and its application 12