An introduction to Markov chains via Monopoly

1. How to Win at Monopoly:
Markov Chains for Fun and Profit
Derek Bruff, PhD
Director, Center for Teaching
Senior Lecturer, Mathematics

3. Which properties
are landed on most
often?

4. Which properties are
landed on most
frequently?

5. Which properties
are most profitable
to own?

6. Monopoly Rules
•
•
•
•
•
40 spaces (Go through Boardwalk)
Roll two six-sided dice to move.
“Go to Jail” sends you to Jail.
Rolling three doubles in a row sends you to Jail.
Get out of jail by…
–
–
–
–
Paying $50,
Using a “Get out of Jail, Free” card,
Rolling doubles, or
Spending three turns in Jail.
• Chance and Community Chest cards have various
effects.

7. Monopoly Rules
•
•
•
•
•
40 spaces (Go through Boardwalk)
Roll two six-sided dice to move.
“Go to Jail” sends you to Jail.
Rolling three doubles in a row sends you to Jail.
Get out of jail by…
–
–
–
–
Paying $50,
Using a “Get out of Jail, Free” card,
Rolling doubles, or
Spending three turns in Jail.
• Chance and Community Chest cards have various
effects.

8. Monopoly Rules
•
•
•
•
•
40 spaces (Go through Boardwalk)
Roll two six-sided dice to move.
“Go to Jail” sends you to Jail.
Rolling three doubles in a row sends you to Jail.
Get out of jail by…
–
–
–
–
Paying $50,
Using a “Get out of Jail, Free” card,
Rolling doubles, or
Spending three turns in Jail.
• Chance and Community Chest cards have various
effects.

9. Monopoly Rules
•
•
•
•
•
40 spaces (Go through Boardwalk)
Roll two six-sided dice to move.
“Go to Jail” sends you to Jail.
Rolling three doubles in a row sends you to Jail.
Get out of jail by…
–
–
–
–
Paying $50,
Using a “Get out of Jail, Free” card,
Rolling doubles, or
Spending three turns in Jail.
• Chance and Community Chest cards have various
effects.

10. Monopoly Rules
•
•
•
•
•
40 spaces (Go through Boardwalk)
Roll two six-sided dice to move.
“Go to Jail” sends you to Jail.
Rolling three doubles in a row sends you to Jail.
Get out of jail by…
–
–
–
–
Paying $50,
Using a “Get out of Jail, Free” card,
Rolling doubles, or
Spending three turns in Jail.
• Chance and Community Chest cards have various
effects.

11. Monopoly Rules
•
•
•
•
•
40 spaces (Go through Boardwalk)
Roll two six-sided dice to move.
“Go to Jail” sends you to Jail.
Rolling three doubles in a row sends you to Jail.
Get out of jail by…
–
–
–
–
Paying $50,
Using a “Get out of Jail, Free” card,
Rolling doubles, or
Spending three turns in Jail.
• Chance and Community Chest cards have various
effects.

12. Monopoly Rules
•
•
•
•
•
40 spaces (Go through Boardwalk)
Roll two six-sided dice to move.
“Go to Jail” sends you to Jail.
Rolling three doubles in a row sends you to Jail.
Get out of jail by…
–
–
–
–
Paying $50,
Using a “Get out of Jail, Free” card,
Rolling doubles, or
Spending three turns in Jail.
• Chance and Community Chest cards have various
effects.

13. Monopoly Rules
•
•
•
•
•
40 4 spaces (Go through Boardwalk)
Roll two six-sided dice Flip a coin to move.
“Go to Jail” sends you to Jail.
Rolling three doubles in a row sends you to Jail.
Get out of jail by…
–
–
–
–
Paying $50,
Using a “Get out of Jail, Free” card,
Rolling doubles, or
Spending three turns in Jail.
• Chance and Community Chest cards have various
effects.

14. Suppose we only have four
spaces (A, B, C, and D) and
that a move consists of
flipping a coin.
• Heads = Move two
spaces
• Tails = Move one space

15. Monopoly: Terminally Boring Edition
x0 =
x1 =
What is x2?

16. Monopoly: Terminally Boring Edition
x0 =
x1 =
x2 =

18. If x2=Px1, then what is P?

19. If x2=Px1, then what is P?
=
x2
P
x1

20. Monopoly: Terminally Boring Edition
Model: xk+1=Pxk
P=

21. Monopoly: Terminally Boring Edition
Model: xk+1=Pxk
x0 = (1, 0, 0, 0)

22. Monopoly: Terminally Boring Edition
Model: xk+1=Pxk
x0 = (1, 0, 0, 0)
x1 = (0, .5, .5, 0)

23. Monopoly: Terminally Boring Edition
Model: xk+1=Pxk
x0 = (1, 0, 0, 0)
x1 = (0, .5, .5, 0)
x2 = (.25, 0, .25, .5)

24. Monopoly: Terminally Boring Edition
Model: xk+1=Pxk
x0 = (1, 0, 0, 0)
x1 = (0, .5, .5, 0)
x2 = (.25, 0, .25, .5)
x3 = (.375, .375, .125, .125)

25. Monopoly: Terminally Boring Edition
Model: xk+1=Pxk

26. Monopoly: Terminally Boring Edition
•
•
•
•
•
40 4 spaces (Go through Boardwalk)
Roll two six-sided dice Flip a coin to move.
“Go to Jail” sends you to Jail.
Rolling three doubles in a row sends you to Jail.
Get out of jail by…
–
–
–
–
Paying $50,
Using a “Get out of Jail, Free” card,
Rolling doubles, or
Spending three turns in Jail.
• Chance and Community Chest cards have various
effects.

27. Monopoly: Simple Model
•
•
•
•
•
40 spaces (Go through Boardwalk)
Roll two six-sided dice to move.
“Go to Jail” sends you to Jail.
Rolling three doubles in a row sends you to Jail.
Get out of jail by…
–
–
–
–
Paying $50,
Using a “Get out of Jail, Free” card,
Rolling doubles, or
Spending three turns in Jail.
• Chance and Community Chest cards have various
effects.

28. Rolling Two Six-Sided Dice
Spaces Moved
Probability
2
1/36
3
2/36
4
3/36
5
4/36
6
5/36
7
6/36
8
5/36
9
4/36
10
3/36
11
2/36
12
1/36

29. P=

30. Monopoly: Simple Model

31. Monopoly: Simple Model

32. Monopoly: Simple Model

33. Monopoly: Simple Model

34. Monopoly: Simple Model

35. Monopoly: Simple Model

36. Monopoly: Simple Model

37. Monopoly: Simple Model

38. Monopoly: Simple Model

39. Monopoly: Simple Model

40. Monopoly: Simple Model

41. Monopoly: Simple Model

42. Monopoly: Simple Model

43. Markov Chains
Definition: A vector with the
property that the sum of its
entries is 1 is called a probability
vector.
Definition: A square matrix with
the property that the sum of the
entries in each of its columns is 1
is called a stochastic matrix.
Andrey Markov, 1856 – 1922

44. Markov Chains
Definition: A Markov chain is a dynamical system for
which
• the probability vector xk describes the state of the
system at time k and
• successive state vectors are related by the following
equation, where P is a stochastic matrix called the
transition matrix for the system.
xk+1=Pxk

45. Markov Chains
Theorem: If P is the transition matrix for a Markov
chain (and P is regular), then…
• There is a unique probability vector q such that
Pq=q.
• For any initial state vector x0,
xk q as k 
Finding q means solving the equation
Pq=q

46. Monopoly: Simple Model
Finding q means solving the equation
Pq=q

47. Monopoly: Model #2
•
•
•
•
•
40 spaces (Go through Boardwalk)
Roll two six-sided dice to move.
“Go to Jail” sends you to Jail.
Rolling three doubles in a row sends you to Jail.
Get out of jail by…
–
–
–
–
Paying $50,
Using a “Get out of Jail, Free” card,
Rolling doubles, or
Spending three turns in Jail.
• Chance and Community Chest cards have various
effects.

48. P=

49. Monopoly: Model #2
Finding q means solving the equation
Pq=q

50. Monopoly: Model #3
•
•
•
•
•
40 spaces (Go through Boardwalk)
Roll two six-sided dice to move.
“Go to Jail” sends you to Jail.
Rolling three doubles in a row sends you to Jail.
Get out of jail by…
–
–
–
–
Paying $50,
Using a “Get out of Jail, Free” card,
Rolling doubles, or
Spending three turns in Jail.
• Chance and Community Chest cards have various
effects.

51. Monopoly: Model #3
Finding q means solving the equation
Pq=q

52. Monopoly: Model #4
•
•
•
•
•
40 spaces (Go through Boardwalk)
Roll two six-sided dice to move.
“Go to Jail” sends you to Jail.
Rolling three doubles in a row sends you to Jail.
Get out of jail by…
–
–
–
–
Paying $50,
Using a “Get out of Jail, Free” card,
Rolling doubles, or
Spending three turns in Jail.
• Chance and Community Chest cards have various
effects.

53. Monopoly: Model #4
Finding q means solving the equation
Pq=q

54. What’s Left?
• Rolling three doubles in a row sends you to
Jail.
• Chance and Community Chest cards have
various effects.
You still have two underlying models—leave jail
quickly or stay as long as you can.

55. Short Jail Stay
Probabilities by Truman Collins

56. Short Jail Stay
Probabilities by Truman Collins

57. Short Jail Stay
Probabilities by Truman Collins

58. Short Jail Stay
Probabilities by Truman Collins

59. Long Jail Stay
Probabilities by Truman Collins

60. Long Jail Stay
Probabilities by Truman Collins

61. Long Jail Stay
Probabilities by Truman Collins

62. Long Jail Stay
Probabilities by Truman Collins

63. OTHER APPLICATIONS OF MARKOV
CHAINS

64. RISK—and other board games

65. Baseball,
tennis,
jai alai,…

66. Migration Models

67. Google’s PageRank Algorithm

68. Flickr Credits
• “monopoly,” foreverdigital
• “Black Dice,” Mariano Kamp
• “Monopoly,” unloveablesteve
• “last man standing,” Robert Terrell
• “Racing for Home,” Scott Ableman
• “Nomads (brog pa) crossing Lha chu at Kailash
Kora,” reurinkjan