This document provides an overview of reinforcement learning in a grid world problem. It introduces reinforcement learning and compares it to supervised learning. Reinforcement learning involves an agent interacting with an environment and receiving rewards or penalties for its actions without being explicitly told which actions to take. The document discusses Markov decision processes and uses a grid world as an example. It presents the iterative policy evaluation and policy iteration algorithms for solving grid world problems and includes example output and state-action mappings generated by these algorithms.

1. 1
Reinforcement Learning
in The Grid World problem
Author
Alireza Andalib
Learning Machine

2. 2

3. 3

4. 4

5. 5

6. 6
Supervised Learning:
Example Class
Reinforcement Learning:
Situation Reward Situation Reward
…
RL

7. Supervised Learning
SystemInputs Outputs
Training Info = desired (target) outputs
Error = (target output – actual output)
7

8. RL
SystemInputs Outputs (“actions”)
Training Info = evaluations (“rewards” / “penalties”)
8

9. 9

10. 10}|Pr{),( ssaaas ttt ===π

11. 11

13. 13

14. 14

15. 15

16. 16

17. 17
Markov Decision Processes

18. 18
Grid World
A B
B’
A’

19. 19

20. 20

21. Bellman

22. 22
2525
1.7120 9.7461 3.1311 5.4209 1.0036
0.7994 2.9233 2.3299 1.9586 0.4665
0.0023 0.7899 07355 0.4364 0.2287-
0.7664- 0.8488- 0.0076 0.1855- 0.9621-
0.9949- 1.3554- 1.0946- 1.4766- 2.0021-

23. 23
IPE

24. 24
IPE
K100i,j
1.4008 9.5698 3.1841 5.4309 0.8827
0.6503 2.9231 1.9576 1.8581 0.3910
0.0303- 0.8137 0.7354 0.4787 0.2830-
0.4062- 0.0118- 0.0183 0.1828- 0.7333-
0.6535- 0.4780- 0.4594- 0.5763- 0.9488-

25. 25
PI

26. 26
PI
State
Go Right Jump Go Left Jump Go Left
Go Up Go Up Go Left Go Up Go Left
Go Up Go Up Go Up Go Up Go Left
Go Up Go Up Go Up Go Up Go Left
Go Up Go Up Go Up Go Up Go Left

27. 27

28. 28
 Horstmann, Cay. "GridWorld". horstmann.com.
Accessed September 15, 2008
 www.inf.ed.ac.uk/teaching/courses/rl
 www.math-info.univ-paris5.fr/~bouzy/Doc/AA2/Reinforcement
 www.cs.berkeley.edu/~pabbeel/cs287-fa12
 courses.cs.washington.edu/courses/cse473/12sp/s
lides/16-mdp.pdf

29. 29
THANKS FORYOURATTENTION