YouTube Link: https://youtu.be/Gs2xtNzogSY ** Python Data Science Training: https://www.edureka.co/data-science-python-certification-course ** This Edureka session on Introduction To Markov Chains will help you understand the basic idea behind Markov chains and how they can be modeled as a solution to real-world problems. Here’s a list of topics that will be covered in this session: 1. What Is A Markov Chain? 2. What Is The Markov Property? 3. Understanding Markov Chains With An Example 4. What Is A Transition Matrix? 5. Markov Chain In Python 6. Markov Chain Applications Follow us to never miss an update in the future. YouTube: https://www.youtube.com/user/edurekaIN Instagram: https://www.instagram.com/edureka_learning/ Facebook: https://www.facebook.com/edurekaIN/ Twitter: https://twitter.com/edurekain LinkedIn: https://www.linkedin.com/company/edureka Castbox: https://castbox.fm/networks/505?country=in

1. Copyright © 2017, edureka and/or its affiliates. All rights reserved.

2. WHAT IS A MARKOV CHAIN?
UNDERSTANDING MARKOV CHAINS WITH AN EXAMPLE
MARKOV CHAIN IN PYTHON
MARKOV CHAIN APPLICATIONS
TRANSITION MATRIX & TRANSISTION STATE DIAGRAM
www.edureka.co

3. WHAT IS A MARKOV CHAIN?
www.edureka.co

4. A stochastic process containing random variables, transitioning from one state to another
depending on certain assumptions and definite probabilistic rules.
What Is A Markov chain?
Markov Property states that the calculated probability of a random process transitioning to the
next possible state is only dependent on the current state and time and it is independent of the
series of states that preceded it.
Let the random process be, { 𝑋 𝑚 , m=0,1,2 … }
This process is a Markov chain only if,
𝑃 𝑋 𝑚+1 = ȁ𝑗 𝑋 𝑚 = ⅈ, 𝑋 𝑚−1 = ⅈ 𝑚−1, … , 𝑋0 = ⅈ0 = 𝑃 𝑋 𝑚+1 = ȁ𝑗 𝑋 𝑚 = ⅈ
𝑃𝑖𝑗 = 𝑃 𝑋 𝑚+1 = ȁ𝑗 𝑋 𝑚 = ⅈ
www.edureka.co

5. UNDERSTANDING MARKOV CHAINS
www.edureka.co

6. A Markov Model is a stochastic model that models random variables in such a manner that the variables
follow the Markov property.
• Keys denote the unique words in the sentence, i.e., 5 keys (one, two, hail, happy, edureka)
• Tokens denote the total number of words, i.e. 8 tokens.
one edureka two edureka hail edureka happy edureka
www.edureka.co

7. A Markov Model is a stochastic model that models random variables in such a manner that the variables
follow the Markov property.
one
edureka
two
hail
happy
1
4
1
1
1
Keys Frequencies
www.edureka.co

8. A Markov Model is a stochastic model that models random variables in such a manner that the variables
follow the Markov property.
one
edureka
two
hail
happy
1
4
1
1
1
Weighted distributions:
1. 'edureka' is 50% (4/8)
2. (one, two, hail, happy)
is ≈ 13% (1/8)
www.edureka.co

9. Start one edureka two edureka hail edureka happy edureka end
one
edureka
two
hail
happy
1
4
1
1
1
start
end
1
1
Keys Frequencies
www.edureka.co

10. (Start, one)
(one, edureka)
(edureka, two)
(two, edureka)
(edureka, hail)
(hail, edureka)
(edureka, happy)
(happy, edureka)
(edureka, end)
(end, none)
(Start, one)
(one, edureka)
(edureka, two) (edureka, hail) (edureka, happy) (edureka, end)
(two, edureka)
(hail, edureka)
(happy, edureka)
(end, none)
www.edureka.co

11. (Start, one)
(one, edureka)
(edureka, two) (edureka, hail) (edureka, happy) (edureka, end)
(two, edureka)
(hail, edureka)
(happy, edureka)
(end, none)
Start [one]
One [edureka]
Edureka [two, hail, happy, end]
Two [edureka]
Hail [edureka]
Happy [edureka]
End [none]
www.edureka.co

12. Start
One
hailtwo
happy
end
edureka
1
1
1
1
1
1/4
1/4
1/4
1/4
Start [one]
One [edureka]
Edureka [two, hail, happy, end]
Two [edureka]
Hail [edureka]
Happy [edureka]
End [none]
www.edureka.co

13. TRANSITION MATRIX & STATE DAIGRAM
www.edureka.co

14. In a Markov Process, we use a matrix to represent the transition probabilities from one state to
another. This matrix is called the Transition or probability matrix. It is usually denoted by P.
What Is A Transition Matrix?
𝑝11 𝑝12 𝑝1𝑟
𝑝21 𝑝22 𝑝2𝑟
𝑝21 𝑝22 𝑝2𝑟
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
P =
෍
𝑘=1
𝑟
𝑝𝑖𝑘 = ෍
𝑘=1
𝑟
𝑃 𝑋 𝑚+1 = ȁ𝑘 𝑋 𝑚 = ⅈ
Note, 𝑝ij ≥ 0, and 'i' for all values is,
www.edureka.co

15. Consider a Markov chain with three states 1, 2, and 3.
Transition Matrix & State Diagram Example
Transition matrix
1/4 0 3/4
P = 1/2 0 1/2
1/4 1/2 1/4
1 2
3
1/4
3/4
1/2
1/2
1/4
1/2
1/4
State Transition Diagram
www.edureka.co

16. DEMO
www.edureka.co

17. To apply Markov Property and create a Markov Model that can generate text simulations by
studying Donald Trump speech data set.
Problem Statement
Apply Markov property
Generate Text Simulations
Donald Trump speeches
www.edureka.co

18. MARKOV CHAIN APPLICATIONS
www.edureka.co

19. Markov Chain Applications
Google PageRank Autocomplete
www.edureka.co

20. Markov Chain Applications
Subreddit simulator Text generator
www.edureka.co

21. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
www.edureka.co

22. www.edureka.co