This presentation was made by Bhanu Prakash (ADAS, Continental AG.,) as part of AI Dev Days conference held in Bangalore on 9th March 2018. URL: www.aidevdays.com --- * Intro to AD and Levels of AD (Autonomous Driving) * Building blocks of AD ( ADAS Sensors) * Different AI/Deep Learning Techniques used in ADAS and AD * Challenges and Pitfalls of AI and AD

1. Artificial Intelligence in Autonomous Driving
Bhanu Prakash P.
ADAS, Continental AG.,
Chassis & Safety | Advanced Driver Assistance Systems

2. Public
BU ADAS 08.03.2018
2Robert Thiel
Agenda
SAE Levels of Driving Automation2
Building Blocks of AD3
AI in Autonomous Driving4
AI/AD Challenges5
Continental Product Line1

3. Public
BU ADAS 08.03.2018
3Bhanu Prakash P.
ADAS
Product Portfolio
SHORT RANGE RADAR
SURROUND VIEW
3D HIGH – RESOLUTION
FLASH LIDAR
CAMERA (opt. LIDAR) ASS./AUT. DRIVING
CONTROL UNIT
LONG RANGE RADAR
STEREO CAMERA
eHorizon
cloud
services

4. Public
BU ADAS 08.03.2018
4Robert Thiel
Agenda
Continental Product Line1
Building Blocks of AD3
AI in Autonomous Driving4
AI/AD Challenges5
SAE Levels of Driving Automation2

5. Public
BU ADAS 08.03.2018
5Bhanu Prakash P.
Levels of Driving Automation
Level 0
Level 1
Level 2
Level 3
Level 4*
Level 5
Driver Only
Assisted
Partial
Automation
Conditional
Automation
High
Automation
Full
Automation
Chauffeur
Assistant
Robot
Driver Only
Assistant
Chauffeur
‘Vehicle supports the driver.
Driver must monitor the system at all times.’
‘Vehicle performs driving functions
partially or fully.’
* machine is fallback

6. Public
BU ADAS
Automated
Parking
2
NCAP
New Car Assessment
Programme
1
Partly / Highly
Automated Driving
3
Self Driving Car
4
6Bhanu Prakash P.
08.03.2018
On the Way to Automated Driving

7. Public
BU ADAS 08.03.2018
7Robert Thiel
Agenda
Continental Product Line1
SAE Levels of Driving Automation2
AI in Autonomous Driving4
AI/AD Challenges5
Building Blocks of AD3

8. Public
BU ADAS 08.03.2018
8Bhanu Prakash P.
Building Blocks of Automated Driving
IMUHFLRADARCamera
Features :
› High Resolution
› Wider FOVs
› Driving path Geometry
› Static Scene Semantics
› 360° View with Fish eye
lens
Features :
› Cheap
› Velocity Accuracy
› Weather Robustness
› Depth Estimation
› Object Detection
› Tracking
Features :
› Low Light
› 3D point cloud
› Range/Depth Accuracy
› Angular Resolution
› Road Surface
Features:
› Where am I?
› vehicle’s dynamics in space
and time
› Odometry

9. Public
BU ADAS 08.03.2018
9
CEMGPSHD MapV2X
Features:
› Situational awareness
› predictive insights,
› around-corner View
› Real time updates
› DSRC/5G
› Not affected by weather
conditions
Features:
› Path sensing
› foresight purposes
› Crowd sourced
› Localization
Features:
› For Localization
› “Cm” level accuracy
› GNSS/DR
Bhanu Prakash P.
Building Blocks of Automated Driving
Features:
› Understand Environment
› Trajectory planning
› Traffic Participants
› Static Objects
› Kerbs, lanes
› Road Infrastructure - Signs,
Traffic lights

10. Public
BU ADAS
10Bhanu Prakash P.
Together Solves the problem of
• Where am I ?
• Where is everyone else?
• How do I get from A to B
08.03.2018
Building Blocks of Automated Driving

11. Public
BU ADAS 08.03.2018
11Robert Thiel
Agenda
Continental Product Line1
SAE Levels of Driving Automation2
Building Blocks of AD3
AI/AD Challenges5
AI in Autonomous Driving4

12. Public
BU ADAS
AI is better than classical SW
2005
Source: http://www.vision.caltech.edu
Example Pedestrian Detection
(caltech pedestrian dataset)
x5
x10
08.03.2018
12Bhanu Prakash P.
Artificial Intelligence in AD
› Deep Learning can be a solution for AD if
› There is no correct „physical“ model
› There is enough and the right data
available
› The architecture fits to the problem
› We optimize for the right criterions
› We can bring it to embedded hardware
Examples

13. Public
BU ADAS 08.03.2018
13
› Results based on state of the art Deep Learning
architecture. (based on ResNet-50)
› Improved by Factor of 5
Ped@Night
Bhanu Prakash P.
Deep Learning improves performance significantly
Pedestrian Pose Estimation
› Using Adverserial PoseNets

14. Public
BU ADAS 08.03.2018
14
• Object Detection
• TSR/TLR
• Pedestrian Detection
• Semantic Segmentation
• Road Boundary Detection
• Free space detection
• Debris Detection
• Driving policy & Path Planning
• Driver Status Monitoring
Bhanu Prakash P.
Other AI Applications

15. Public
BU ADAS
15Bhanu Prakash P.
08.03.2018
Deep Learning In Action

16. Public
BU ADAS 08.03.2018
16Robert Thiel
Agenda
Continental Product Line1
SAE Levels of Driving Automation2
Building Blocks of AD3
AI in Autonomous Driving4
AI/AD Challenges5

17. Public
BU ADAS
17
Challenges: Complexity / Portability
Complexity Reduction Embedded Hardware
0
10
20
30
40
50
60
70
80
mAP (%) Parameters
(Million)
GMAC/frame
Example: Object Detection
@Kitti dataset „cars hard“
original (yolo v2) reduced
x15
x10
08.03.2018
Bhanu Prakash P.
• High performing Hardware
• Lack of flexibility and multitasking
• Need Lots and Lots of Data
• Need Millions of miles of Labelled data
• Data Augmentation
© Carla Simulator

18. Public
BU ADAS
18
+
2017 – Metzen et al - Universal Adversarial Perturbations
Against Semantic Image Segmentation
“Adversarial Examples for Semantic Image Segmentation” Fischer et al.,
Bosch Center for AI
Bhanu Prakash P.
Challenges:Adversarial Attacks
08.03.2018

19. Public
BU ADAS 08.03.2018
19
Camouflage graffiti and art stickers or
Perturbations cause NN to misclassify
• stop signs -> speed limit 45 signs
• Right turn -> stop signs
Bhanu Prakash P.
Challenges:Adversarial Attacks

20. Public
BU ADAS
20
• AI cannot interpret data on its own.
• Neural networks are essentially Blackboxes
• Need networks that can be explained / Interpretable
• Risk: Hundreds of signals and their combinations to
plan ahead. Every time there is a mishap, should
understand why a certain decision?
Challenges: Explainability/Interpretability of DNN
08.03.2018
Bhanu Prakash P.
David Gunning DARPA/I2O

21. Public
BU ADAS
21Bhanu Prakash P.
08.03.2018
Summary
Performance ScalabilityPerformance Av. Hardware
Compute Power
Code Complexity Data Engineering
Efforts Predictability
Featurebased
MachineLearning
Deep
MachineLearning

22. Public
BU ADAS
22Bhanu Prakash P.
08.03.2018
› Deep Learning Architecture
› Design, Training, Validation
› Deep Machine learning Solutions for Real world Problems
› Computer Vision, Radar, Lidar, Sensor Fusion, Planning and Action
› Deep Learning Frameworks
› Tensorflow, Keras, Theano, nvidia Digits, CuDNN
› Scheduling and distribution of compute jobs
› Amazon AWS, HPC Cluster
› Compute complexity analysis
› nVidia Drive PX, Renesas HW
What are we working on at Continental ADAS

23. Public
BU ADAS 08.03.2018
23Bhanu Prakash P.
From Research to Industry
Project CUbE

24. SensePlanAct