1. Deep Reinforcement Learning for Self
driving Car using Unreal engine
environment
- Priyanshu Jain(SC17B113)
- Ashish Bansal (SC17B078)
2. Introduction
● Self driving car has become one of the most prominent topic in today's world.
● We are trying to implement self driving car using Deep-Q network which has one front camera
with field of view of 110 degree to get glance of real life driver ,we are then using Deep neural
network architecture to classify image into 5 classes which is then used by deep Q network to get
the Q table.
● The objective of this project is to allow the car to identify safe way of driving using a Deep Neural
Network.
3. Introduction to Carla
● CARLA is an unreal engine based environment aimed at assisting Deep learning studies
● The CARLA simulator consists of a scalable client-server architecture.
● The server is responsible for everything related with the simulation itself - sensor rendering,
computation of physics, updates on the world-state and its actors and much more. As it aims for
realistic result.
● Control over the simulation is granted through an API handled in Python and C++
4. Deep-Q learning
● In deep Q-learning, we use a neural network to approximate the Q-value function. The state is
given as the input and the Q-value of all possible actions is generated as the output.
● The DeepMind system used a deep convolutional neural network, with layers of tiled
convolutional filters to mimic the effects of receptive fields.
● We are comparing two architecture of deep-q learning first with simple convnet and second with
Xception model for CNN
7. Policy
Actions
1)Car stops (break).
2)Car accelerates forward(full throttle)
3)Car accelerates forward (partial forward with left steer)
4)Car accelerates forward (partial forward with right steer)
5)Car remains in its previous state (no brake , no throttle, no steer)
Awards
Based on speed ( to avoid any loophole that network may find)
e.g:
elif
KPH == 40:
done = False (continue episode if false)
reward = 1
8. Simulations
● We started training the deep neural network from scratch using random parameter from uniform
distribution but as no. of episodes increased model learned to train itself to drive a car.
● Simulation environment consist of Carla unreal simulator where the actor(Car) is placed with front
camera (RGB) only .
● We made the environment to have collision sensor attached to car and using it as parameter to
detect collision and update necessary weight.
9. Results
● Final results after training are very interesting as car taught itself to drive starting from random
action.
● We can also see with only 500 episodes we were able to get some exciting turns to protect itself
from collision.
● We started from random samples in start but we can see learning to drive and take turn is really
interesting.
10. Comparative study between different
architecture
● We can observe that for Xception the car started to rotate continuously and not moving straight ,
this can be because of complexity of model and the parameter is not changing much due to very
complex model.
● We decided to work with 3 layer conv network described above and results were better..After
training with conv network,car was not only moving straight perfectly ,sometime it was able to
avoid obstacles and go ahead.
● Deep-Q is training as well as GPU intensive task ,we trained for 500 episodes and as we can see in
next slide the trend of rewards ,the car is learning to drive itself better with every episode.
