Thompson Learning Q-Learning

1. Reinforcement Learning

2. Reinforcement Learning
2
Topics
7.1 Upper Confidence Bound
7.2 Thompson Sampling
7.3 Q-Learning

3. Reinforcement Learning
• Reinforcement Learning is used to solve interacting problems
where the data observed up to time t is considered to decide
which action to take at time t + 1.
• It is also used for Artificial Intelligence when training
machines to perform tasks such as walking. Desired
outcomes provide the AI with reward, undesired with
punishment. Machines learn through trial and error.
• Reinforcement Learning models:
Upper Confidence Bound (UCB)
Thompson Sampling

4. Reinforcement Learning

5. What is RL?

6. RL Process

7. RL Process

8. RL Definitions:

9. RL Definitions:

10. RL Concept- Reward Maximization

11. Exploration and Exploitation

12. Upper Confidence Bound (UCB)

13. Thompson Sampling Algorithm
13

14. Difference between UCB & Thompson Sampling Algorithm
14

15. Q-Learning Algorithm
15

16. Q-Learning Algorithm
16

17. Reinforcement Learning
•Reinforcement Learning is used to solve interacting
problems where the data observed up to time t is
considered to decide which action to take at time t + 1.
•It is also used for Artificial Intelligence when training
machines to perform tasks such as walking. Desired
outcomes provide the AI with reward, undesired with
punishment. Machines learn through trial and error.
•Reinforcement Learning models:
Upper Confidence Bound (UCB)
Thompson Sampling
https://www.amazon.com/gp/mpc/A18C1YWPOVXRKS

18. Multi-Armed Bandit Problem using Upper
Bound confidence Algorithm

19. Upper Confidence Bound (UCB)

20. Upper Confidence Bound (UCB)

21. Upper Confidence Bound (UCB)

22. Upper Confidence Bound (UCB)

23. Upper Confidence Bound (UCB)

24. Upper Confidence Bound (UCB)

25. Upper Confidence Bound (UCB)

26. Upper Confidence Bound (UCB)

27. Upper Confidence Bound (UCB)

28. Upper Confidence Bound (UCB)

29. Upper Confidence Bound (UCB)

30. Upper Confidence Bound (UCB)

31. Upper Confidence Bound (UCB)

32. Upper Confidence Bound (UCB)

33. Eclat Output

34. Upper Confidence Bound (UCB)

35. Upper Confidence Bound (UCB)

36. Upper Confidence Bound (UCB)

37. Upper Confidence Bound (UCB)

38. Upper Confidence Bound (UCB)

39. UCB Python Code
# Importing the libraries
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
# Importing the dataset
dataset = pd.read_csv('Ads_CTR_Optimisation.csv')
# Implementing UCB
import math
N = 10000
d = 10
ads_selected = []
numbers_of_selections = [0] * d
sums_of_rewards = [0] * d
total_reward = 0
for n in range(0, N):
ad = 0
max_upper_bound = 0

40. UCB Python Code
for i in range(0, d):
if (numbers_of_selections[i] > 0):
average_reward = sums_of_rewards[i] / numbers_of_selections[i]
delta_i = math.sqrt(3/2 * math.log(n + 1) / numbers_of_selections[i])
upper_bound = average_reward + delta_i
else:
upper_bound = 1e400
if upper_bound > max_upper_bound:
max_upper_bound = upper_bound
ad = i
ads_selected.append(ad)
numbers_of_selections[ad] = numbers_of_selections[ad] + 1
reward = dataset.values[n, ad]
sums_of_rewards[ad] = sums_of_rewards[ad] + reward
total_reward = total_reward + reward
# Visualising the results
plt.hist(ads_selected)
plt.title('Histogram of ads selections')
plt.xlabel('Ads')
plt.ylabel('Number of times each ad was selected')
plt.show()

41. UCB Output

42. What is RL?

43. RL Process

44. RL Process

45. RL Definitions:

46. RL Definitions:

47. RL Concept- Reward Maximization

48. Exploration and Exploitation

49. Markov Decision Process

50. Markov Decision Process-Shortest Path Problem
This is an example of EXPLOITATION