This document discusses requirements and technologies for autonomous vehicle systems. It outlines: 1) Requirements for automotive system-on-chips including high performance processors, CNN acceleration, memory, sensor interfaces, and safety/reliability standards. 2) Levels of vehicle automation from driver assistance to full autonomy, describing example scenarios and hardware/software needs at each level. 3) An architecture for autonomous driving including sensor fusion, localization, safety checking, planning and actuation with varying Automotive Safety Integrity Level (ASIL) requirements.

1. The Smarter Car for Autonomous Driving
Heiko Joerg Schick
Chief Architect
Huawei Technologies

2. Requirements of automotive SoCs
• High-performance application processor with 64-bit which execute multiple vision based applications.
• Acceleration for algorithms such as Convolutional Neural Networks (CNN) are a must.
• Support for multimedia interfaces (e.g. HDMI and MIPI D-PHY).
• Support for multiple cameras and radar sensors (e.g. 77 GHz long range radar, infrared, video and ultrasonic).
• Up to 16 GB automotive-grade LPDDR4 memory.
• Extended system connectivity with Automotive Ethernet (including Audio Video Bridging, Time Sensitive Networking for multimedia traffic).
• Additional interfaces (e.g. PCI Express, SATA, UARTs, SPI, QSPI, CAN or FlexRay).
• Support for cloud connectivity (Bluetooth Smart, WiFi, and 5G radio IC.
• Support for secure boot secure identification and authentication.
• Support for encryption and decryption.
• ISO26262 and AEC Q100 reliability qualification, and TS16949 quality management standard:
- Temperature range: -40 °C - 85/155 °C
- Operation time: up to 15 years
- Humidity 0% up to 100%
- Tolerated filed failure rates: zero failure
- Documentation “True” and in English / German
- Supply: up to 30 years

3. L0 L1 L2 L3 L4 L5
No Automation Driver Assistance
SOP 2013-2015
Partial
Automation SOP 2016-2017
Conditional
Automation. SOP 2018-2019
High
Automation. SOP 2020-2023
Full
Automation SOP >2025
Scenario • Autonomous Emergency Braking
• Lane Keep Assist
• Auto High Beam
• Adaptive Cruise Control
• Cross Traffic Alert
• Surround View
• Lane Change Assist
• Lane Centering
• Advanced Parking Assist
• Traffic Jam / Highway Assist
• Advanced Emergency Braking
• Adaptive Cruise Control with LKA
• Rush Hour Pilot
• Automated Lane Change
• Traffic Jam Pilot
• Automated Parking
• Highway Pilot
• Piloted Highway Driving
• Geo-fenced City Pilot
• Unattended Valet Parking
• Mobility on Demand
Auto Pilot / Driverless Driving
Hardware 1-6 Sensors
+ Optional Control Unit
• Mono-Vison
• Mid Range Rader
• 2 Front Corner Radars
• 2 Rear Corner Radars
• ADAS ECU
1,500 DMIPS
6 MB RAM
2-10 Sensors + Control Unit
• Mono-Vision
• 2 Front Corner Radars
• Mid Range Radar
• 2 Rear corner Radars
• Far Infrared Camera
• ADAS ECU
• E-Horizon
• Rear/Surround View
• Driver Monitoring
• HD Map
3,000 DMIPS 16 MB RAM 25W
>15 Sensors + Control Unit (incl.
AI) and Driver Monitoring
• 4 Corner Radars
• Stereo Vision
• Mid Range Radar
• Far Infrared Camera
• ADAS ECU
• Mono-Vision Rear
• V2X
• LiDAR
• HD Map
• E-Horizon
• Driver Monitoring System
• Rear/Surround View
>40,000 DMIPS >25 TOPS
512 MB – 3 GB RAM 200W
25 Sensors + Control Unit (incl. AI) and Driver Monitoring
• Stereo-Vison, Long Range Radar
• 4 Corner Radars, Satellites
• LiDAR Front, Mono-Vision Rear
• Driver and / or Passenger Monitoring System
• HD Map
• V2X
• Surround View
• Far Infrared Camera
• Mid-Range Side Sensors
• AD ECU
260,000 – 845,000 DMIPS
>300 TOPS 32 GB RAM 600W
Software 40+ Features
• Classic AUTOSAR
50+ Features
• Classic + Adaptive AUTOSAR
• POSIX Operating System
55+ Features
• Classic + Adaptive AUTOSAR
• POSIX Operating System
60+ Features
• Adaptive AUTOSAR
• POSIX Operating System
Safety concept Fail-safe Fail-operational
No intervening vehicle
system active.
Vehicle assisted longitudinal and
lateral control.
Vehicle assisted longitudinal and
lateral control (for a period of time
and/or in specific use cases).
System as longitudinal and lateral
control in specific use cases (for a
period of time).
System has longitudinal and lateral
control in a specific use case.
Recognize its performance limits
and requests driver to resume
control with sufficient time margin
System can cope with all situations
automatically during the entire
journey. Driver does not monitor
the system.
E/E architecture • Each function has his
ECU with functional
integration
• Central Domain ECUs
• Central cross domain
ECUs
• Central Domain ECUs
• Central cross domain ECUs
• Central Domain ECUs
• Central cross domain ECUs
• Central cross domain ECUs
• Zone oriented architecture and
vehicle control computer
• Zone oriented architecture and
vehicle control computer
• Zone oriented architecture and
vehicle control computer
Global adoption
rate
2017 :
2020 :
1%
8% 2020 : 2%

4. Different sensor locations
High-level sensor fusion Low-level sensor fusion
• High-level fusion systems draw man decisions at early stages on
an incomplete knowledge basis
• Some of the incorrect decisions cannot be correct at alter steps
• Decisions are drawn at the very last processing step; no
feedback loops
• Need for central high performance unit
Source: Autonomous Driving – A mobility revolution, Helge Neuner, Head of Automated Driving for Group Research, Volkswagen AG
Radar
Camera
Lidar
Radar
Camera
Lidar
Seg
Seg
Seg
Fusion Fusion Seg

5. Strongest specialized cores, hundreds of cores, stacked memory with
highest bandwidth, strongest IO interface à 16 PCIe 3.0 lanes, special
interconnect for accelerator to accelerator communication,
acceleration for artificial intelligence à training and inference,
acceleration for linear algebra, strongest vector units, highest energy
consumption à 300W.
Stronger general cores, tens of cores à 12 – 24 cores, standard
memory interfaces with high memory bandwidth, strongest IO
interfaces à 48-128 PCIe 3.0 lanes, strong vector units, high energy
consumption à 120-170W.
Medium general cores à 8 – 16 cores, standard memory interface
with medium bandwidth, strong IO interface à 16 PCIe 3.0 lanes, low
energy consumption à10 – 30W.
Strong general cores, tens of cores à 48 - 64, standard memory
interfaces with high memory bandwidth, strongest IO interfaces à 48-
128 PCIe 3.0 lanes, strong vector units in high-end version, integrated
controllers à USB, SATA, cryptography, integrated network à
Standard Ethernet and RoCEE, medium energy consumption à 50-
60W.
Medium general cores à 12 – 24, specialized engines à [GPU,
image, vision, signal processing, artificial intelligence, security and
compression], integrated network à CAN/CAN-FD and Automotive
Ethernet, integrated camera interfaces à 8-12, integrated display
interface à 3 – 6, automotive functional safety à [dual execution
including lockstep operation, functional diversity, build-in self test] à
ASIL B – ASIL D, lower power mode à [adaptive voltage scaling,
dynamic voltage and frequency scaling, power and power gating],
high energy efficiency à 15W – 60W.
Weak to medium general cores à 8, very specialized engines à
[GPU, image, vision, signal processing and artificial intelligence], high
memory bandwidth, strongest IO interface à 16 PCIe 3.0 lanes,
integrated network à CAN/CAN-FD and Automotive Ethernet,
integrated camera interfaces à 12-18, automotive functional
safety à [dual execution including lockstep operation, functional
diversity, build-in self test] à ASIL B – ASIL D, highest energy
efficiency à 5W – 30W.
IT & Cloud Embedded & Edge Automotive
Accelerators
CPUs
Integrated
Systems
Smart
Devices
Cloud Embedded
Specialization in vertical industry HighLow

6. Scalable across devices
Device Edge Cloud
Earphone Always-on Smartphone Laptop IPC Edge Server Data Centre
Compute 20 MOPS 100 GOPS 1-10 TOPS 10-20 TOPS 10-20 TOPS 10-100 TOPS 200+ TOPS
Power budget 1 mW 10 mW 1-2 W 3-10 W 3-10 W 10-100 W 200+ W
Model size 10 KB 100 KB 10 MB 10-100 MB 10-100 MB 100+ MB 300+ MB
Latency < 10 ms ~10 ms 10-100 ms 10-500 ms 10-500 ms ms ~ s Ms ~ s
Inference? Y Y Y Y Y Y Y
Training? N N Y Y Y Y Y
Ascend-SKU Nano Tiny >Lite Mini Mini Multi-Mini Max

7. Unified and scalable HW architecture
Scalable compute:
• Scalable cube: 16x16xN， N=16/8/4/2/1
• Multiple precision: int8/int32/FP16/FP32
• Multiple Compute units: Tensor/Vector/Scalar
• Current control in picoseconds
• Hardware-assisted task scheduler
Scalable architecture:
• Dedicated & distributed, tiling-friendly, explicit memory design
Scalable on-chip interconnection
• Ultra-high bandwidth mesh network
FHD Video Image
Codec
Peripherals/IO DDR/HBM
System
Cache/
Buffer
ARM
Cube
LSU
Cache/Buffer
Vector
Scalar
N

8. Unified and versatile SW architecture
CANN
(Compute Architecture for Neural Networks)
Ascend
All Scenarios
Full Stack
AI Applications
Application Enablement
Framework
Chip Enablement
IP & Chip
Application enablement:
• Full-pipeline services (ModelArts), hierarchical APIs,
and pre-integrated solutions
MindSpore:
• Unified training and inference framework for device,
edge, and cloud (both standalone and cooperative)
CANN:
• Chip operators library and highly automated
operators development toolkit
Ascend:
• AI IP and chip series based on unified scalable
architecture
Industrial IoT
Device
Edge ComputingEdge ComputingPrivate CloudPublic CloudConsumer Device
Ascend-MaxAscend-MiniAscend-LiteAscend-TinyAscend-Nano
PaddlePaddlePyTorchTensorFlowMindSpore …
ModelArts
General APIs Advanced APIs Pre-integrated SolutionsHiAI Service
HiAI Engine

9. Next steps: Deep sematic scene
understanding

10. Challenges: Neural hacking

11. Call to action
Algorithmic pattern view
Scenarios Key technologies
Hardware | Software | Connectivity
Safety concept E/E architecture Business
Runtime / OS / middleware
view
Performance Cores Safety Cores
Safety OSPOSIX OS
Adaptive
AUTOSAR
Classic AUTOSAR
Source: Self-Driving Vehicles That (Fore) See, Dariu M. Gavrila, Intelligent Vehicle, TU Delft
Functional view
Sense
Think
Act
Safety view
ISO 26262 ASIL level A – D
Fusion
ASIL B
Localization
ASIL B
Check
ASIL D
Trajectory planning
ASIL B
Actuation
ASIL D