This slide is a part of Introduction to Machine Learning course by Code Heroku. Here is the recorded version of our Reinforcement Learning with OpenAI Gym tutorial: https://www.youtube.com/watch?v=3begG_s9lzg Here is the link to Introduction to Machine Learning Course: http://www.codeheroku.com/course?course_id=1 You can watch all our upcoming and past workshops here: http://www.codeheroku.com Subscribe to our YouTube channel: https://www.youtube.com/channel/UCL-_0RrZ3084Ea8Yavtcd9g Follow our publication on Medium: https://medium.com/code-heroku Visit our Facebook page: https://www.facebook.com/codeheroku

1. Pleaseturn off your webcam
If you arejoining from a mobile phone
besureto click on
Join via Device Audio
Weare waiting for other participants to join
Wewill begin at 4:30 PM IST

2. Mihir Thakkar
Founderand Instructor
hello@codeheroku.com
Reinforcement Learning with
OpenAIGym

3. SESSION OBJECTIVES
• Quick Recap
• Bellman’sEquations
• Value Iterationin OpenAI Gym

4. www.codeheroku.com Introduction toMachine Learning –Reinforcement Learning

5. www.codeheroku.com Introduction toMachine Learning –Reinforcement Learning
RL Problem

6. www.codeheroku.com Introduction toMachine Learning –Reinforcement Learning
Q Function

7. www.codeheroku.com Introduction toMachine Learning –Reinforcement Learning

8. www.codeheroku.com Introduction toMachine Learning –Reinforcement Learning

9. www.codeheroku.com Introduction toMachine Learning –Reinforcement Learning
Quiz
Given the following Reward Table,estimatethe value of Q(A3,East)

10. www.codeheroku.com Introduction toMachine Learning –Reinforcement Learning
Quiz
Given the following Reward Table,estimatethe value of Q(B3,North)

11. www.codeheroku.com Introduction toMachine Learning –Reinforcement Learning
Given a Value Function
Extract Policy

12. Value Iteration
OpenAI Gym
https://drive.google.com/file/d/16xMyG7bKrtT_6SId1kqLpR2vL1Km_us8/view?usp=sharing
https://github.com/codeheroku/Introduction-to-Machine-Learning/tree/master/Reinforcement%20Learning/RL2%20Value%20Iteration

13. www.codeheroku.com Introduction toMachine Learning –Reinforcement Learning
Value Iteration Algorithm

14. Reinforcement Learning
Challenges
• Access to the Environment
• Delayed Reward (Temporal Credit RiskAssignment)
• High Cost Actions
• Distribution of data changes by the choice of actions you
take
• Efficient state representations?
• Good Rewards functions?

15. Thanks

17. www.codeheroku.com Introduction toMachine Learning –Reinforcement Learning
Markov Decision Process (MDP)

18. www.codeheroku.com Introduction toMachine Learning –Reinforcement Learning
Multi Arm Bandit
• Unknown Reward Distribution
• Deterministic Actions
• Objective:FindSequence of actions
whichwillmaximizetotal reward

19. www.codeheroku.com Introduction toMachine Learning –Reinforcement Learning
Exploration Vs Exploitation
To approximatevaluesof actionsAgent must choose actionsthatare non-
optimalto start with.
Once an agent has approximatedthe values, it can greedily pick the
highest value action.

20. www.codeheroku.com Introduction toMachine Learning –Reinforcement Learning
Iterative Averaging