Downloaded 11 times
SB
Uploaded bySlobodan Blazeski
PPTX, PDF856 views
Deep deterministic policy gradient
This document discusses Deep Deterministic Policy Gradient (DDPG), a reinforcement learning algorithm for problems with continuous state and action spaces. DDPG uses an actor-critic method with experience replay and soft target updates to learn a policy in an off-policy manner. It demonstrates how DDPG can be used to train an agent to drive a vehicle in a simulator by designing a reward function, but notes that designing effective rewards, avoiding local optima, instability, and data requirements are challenges for DDPG.
Related topics:
More Related Content
What's hot
Recently uploaded
In this document
Powered by AI
Introduces Deep Deterministic Policy Gradient (DDPG), a method in reinforcement learning.
Discusses the historical background of reinforcement learning methods.
Covers various machine learning methods relevant to DDPG and its application.
Differentiates between supervised and unsupervised learning approaches in machine learning.
Explores different types of neural networks and their functions in learning models.
Discusses the gradient descent optimization method used in training models.
Introduces key concepts of reinforcement learning, setting the stage for advanced discussions.
Discusses the concept of grid worlds as environments used in reinforcement learning.
Explains the distinction and relationship between value functions and policies in RL.
Describes the actor-critic method, a key concept in reinforcement learning involving two components.
Outlines the characteristics of DDPG including continuous spaces, replay buffers, and exploration.
Identifies major challenges in DDPG such as reward function design and training requirements.
Details an application of DDPG in simulating driving scenarios.
Deep deterministic policy gradient
- 1. Deep Deterministic PolicyGradient DDPG
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14. Value function vsPolicy
- 15.
- 16.
- 17. DDPG - Continuous stateand action space - Replay buffer - Soft updates - Exploration noise
- 18. Pitfalls - Designing rewardfunction is very hard - Tends to get stuck into local optima - Unstable - Needs lots of training samples
- 19.