The document appears to be slides from a presentation on reinforcement learning. It introduces reinforcement learning and discusses key concepts like states, actions, rewards, and the reinforcement learning model. It also covers the multi-armed bandit problem as an introduction to reinforcement learning techniques, discussing exploration versus exploitation and iterative averaging to solve multi-armed bandit problems. Links are provided to examples and code.

1. Please turn off your webcam
If you are joining from a mobile phone
be sure to click on
Join via Device Audio
We are waiting for other participants to join
We will begin at 4:30 PM IST

2. Mihir Thakkar
Founder and Instructor
hello@codeheroku.com
Introduction to
Reinforcement
Learning

3. SESSION
OBJECTIVES
• Introduction to RL
• Use Cases
• Formalization
• Multi Arm Bandit in
Python

4. www.codeheroku.com Introduction to Machine Learning – Reinforcement Learning

5. www.codeheroku.com Introduction to Machine Learning – Reinforcement Learning
Quiz
Which is the next best move to make for our
robot?

6. Why
Reinforcement
Learning?
• Closer to Real World
• Deals with Stochastic
Nature of Environment
• Understand Intelligence
• Replicate Intelligence

7. www.codeheroku.com Introduction to Machine Learning – Reinforcement Learning
https://drive.google.com/file/d/1gOrJu2svliyEnlIIeItHeBEd1rT3Vhob/view?usp=sharing

8. www.codeheroku.com Introduction to Machine Learning – Reinforcement Learning
RL Model

9. www.codeheroku.com Introduction to Machine Learning – Reinforcement Learning
Quiz:
Define States, Actions and Rewards

10. www.codeheroku.com Introduction to Machine Learning – Reinforcement Learning
Quiz:
Define States, Actions and Rewards

11. www.codeheroku.com Introduction to Machine Learning – Reinforcement Learning
Quiz
Define States, Actions and Rewards

12. www.codeheroku.com Introduction to Machine Learning – Reinforcement Learning
Quiz
Define States, Actions and Rewards

13. www.codeheroku.com Introduction to Machine Learning – Reinforcement Learning
Markov Decision Process (MDP)

14. www.codeheroku.com Introduction to Machine Learning – Reinforcement Learning
Our Goal

15. www.codeheroku.com Introduction to Machine Learning – Reinforcement Learning

16. www.codeheroku.com Introduction to Machine Learning – Reinforcement Learning
Multi Arm Bandit
•Unknown Reward
Distribution
•Deterministic Actions
•Objective: Find Sequence
of actions which will
maximize total reward

17. www.codeheroku.com Introduction to Machine Learning – Reinforcement Learning
Iterative Averaging

18. www.codeheroku.com Introduction to Machine Learning – Reinforcement Learning
Exploration Vs Exploitation
To approximate values of actions Agent must
choose actions that are non-optimal to start
with.
Once an agent has approximated the values, it
can greedily pick the highest value action.

19. Reinforcement
Learning
Challenges
• Access to the environment
• Delayed Reward (Temporal Credit
Risk Assignment)
• High Cost Actions
• Distribution of data changes by the
choice of actions you take
• Efficient state representations? What
constitutes a good state
• Good Rewards functions?

20. Thanks

21. Multi Arm Bandit
https://drive.google.com/file/d/1gql13NuNpRyEnpJJOUnIAsp
m2k4krQ9b/view?usp=sharing
https://github.com/codeheroku/Introduction-to-Machine-
Learning/tree/master/Reinforcement%20Learning/RL1%20Multiarm%20Bandit