MIPI DevCon 2020 | MIPI DevCon 2020 | How MIPI Interfaces Solve Challenges in ADAS/AD Sensor Validation
•
0 likes•201 views
This presentation, from Gregor Sievers, Ph.D., of dSPACE GmbH, addresses how the MIPI CSI-2℠, D-PHY℠, CCS, and A-PHY℠ specifications simplify validation and testing and help bring autonomous driving to the streets.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to MIPI DevCon 2020 | MIPI DevCon 2020 | How MIPI Interfaces Solve Challenges in ADAS/AD Sensor Validation
Similar to MIPI DevCon 2020 | MIPI DevCon 2020 | How MIPI Interfaces Solve Challenges in ADAS/AD Sensor Validation (20)
More from MIPI Alliance
More from MIPI Alliance (20)
Recently uploaded
Recently uploaded (20)
MIPI DevCon 2020 | MIPI DevCon 2020 | How MIPI Interfaces Solve Challenges in ADAS/AD Sensor Validation
- 1. Gregor Sievers, Ph.D. dSPACE GmbH How MIPI® Interfaces Solve Challenges in ADAS/AD Sensor Validation
- 2. © 2020 MIPI Alliance, Inc. 2 Vision „Autonomy“ From simplicity
- 3. © 2020 MIPI Alliance, Inc. 3 Vision „Autonomy“ Hard real-time processing § Functional safety § … Artificial intelligence Dozens of Sensors § Camera § Radar § Lidar High Data Rates • Dozens of Gbit/s Towards complexity
- 4. © 2020 MIPI Alliance, Inc. 4 Vision „Autonomy“ Validation of sensors today §Real test drives §Expensive §Lot of data §Critical situations §Sensor hardware upgrade
- 5. © 2020 MIPI Alliance, Inc. 5 dSPACE AD validation solution Data-driven development and validation
- 6. © 2020 MIPI Alliance, Inc. 6 AD software stack – Validation and release testing Sensor front end tests Data replay tests Closed-loop SIL and HIL simulation Real test drives SIL: Software in the Loop HIL: Hardware in the Loop
- 7. © 2020 MIPI Alliance, Inc. 7 Automotive Front Camera Architecture Camera Image sensor (e.g. CMOS) Camera and processing module on one PCB Lens Image processing unit (SoC, FPGA, DSP, … e.g. EyeQ™) MIPI CSI-2℠ Data + Clock Ctrl µC CAN FlexRay ETH ECUconnector RAM/ flash Power Monochrome, RGBBayeror RCCCfilter § Short range sensor interface: MIPI CSI-2℠ and MIPI D-PHY℠ § MIPI CSI-2℠ replaced proprietary interfaces § Parallel § LVDS § Also used for Radar and Lidar applications
- 8. © 2020 MIPI Alliance, Inc. 8 Automotive Surround View Camera Architecture § Long range sensor interfaces § 1 cable for data, control, and power § Serializer and Deserializer (SerDes) § MIPI A-PHY℠ § Proprietary: TI FPD-Link, Maxim GMSL SerDes Links (e.g. MIPI A-PHY℠) processing module CAN FlexRay ETH ECUconnector RAM/ flash Power Serializer Deserializer Camera module Camera Image sensor (e.g. CMOS) Lens Monochrome, RGBBayeror RCCCfilter Image processing unit (SoC, FPGA, DSP, … e.g. EyeQ™) µC
- 9. © 2020 MIPI Alliance, Inc. 9 Data Logging of ADAS/AD Sensors
- 10. © 2020 MIPI Alliance, Inc. 10 Automotive Surround View Camera Architecture § Long range sensor interfaces § 1 cable for data, control, and power § Serializer and Deserializer (SerDes) SerDes Links (e.g. MIPI A-PHY℠) processing module CAN FlexRay ETH ECUconnector RAM/ flash Power Serializer Deserializer Camera module Camera Image sensor (e.g. CMOS) Lens Monochrome, RGBBayeror RCCCfilter Image processing unit (SoC, FPGA, DSP, … e.g. EyeQ™) µC
- 11. © 2020 MIPI Alliance, Inc. 11 Surround View Camera: Data Logging with ECU High-EndSSD processing module CAN FlexRay ETH ECUconnector RAM/ flash Power Deserializer Camera module Serializer Camera Image sensor (e.g. CMOS) Lens Monochrome, RGBBayeror RCCCfilter Image processing unit (SoC, FPGA, DSP, … e.g. EyeQ™) µC Logging platform Deserializer Serializer SerDes Links (e.g. MIPI A-PHY℠)
- 12. © 2020 MIPI Alliance, Inc. 12 Data Logging and Prototyping for ADAS/AD dSPACE AUTERA - The power of a Server – in the vehicle! § Intel Xeon CPU with 12 Cores (@2GHz) § Support for CAN, Automotive Ethernet, etc. § Up to 6x PCIe slots for extensions § High End Data Logging Capability § Up to 64 TB Storage § Up to 50 Gbit/s logging bandwidth § PCIe Grabber Card for Sensor Interfaces § Maxim GMSL and TI FPD-Link § Up to 6 Gbit/s per sensor § Additional Interfaces under development § MIPI A-PHY℠ once available Mainboard I/O-Rack 10GbE, USB 3.0, and 1GbE General Purpose I/O, USB 3.0, SIM-Slot and User-LEDs For additional sensor connections, Wireless Connectivity, Status-Info
- 13. © 2020 MIPI Alliance, Inc. 13 Virtual test drives
- 14. © 2020 MIPI Alliance, Inc. 14 Video: Recorded and Simulated Test Data
- 15. © 2020 MIPI Alliance, Inc. 15 AD software stack – Validation and release testing Virtualtestdrives– SILsim ulation Virtualtestdrives– HILsim ulation Realtest drives Virtual ECUs Real ECUs PC cluster SIL: Software in the Loop HIL: Hardware in the Loop ECU: Electronic Control Unit
- 16. © 2020 MIPI Alliance, Inc. 16 Front Camera: Hardware in the Loop testing Camera Image sensor (e.g. CMOS) Camera and processing module on one PCB Lens Image processing unit (SoC, FPGA, DSP, … e.g. EyeQ™) MIPI CSI-2℠ Data + Clock Ctrl µC CAN FlexRay ETH ECUconnector RAM/ flash Power Monochrome, RGBBayeror RCCCfilter § Short range sensor interface: MIPI CSI-2℠ and MIPI D-PHY℠
- 17. © 2020 MIPI Alliance, Inc. 17 Front Camera: Hardware in the Loop testing Camera Image sensor (e.g. CMOS) Camera and processing module on one PCB Lens Image processing unit (SoC, FPGA, DSP, … e.g. EyeQ™) µC CAN FlexRay ETH ECUconnector RAM/ flash Power Monochrome, RGBBayeror RCCCfilter Development andtest interface MIPI CSI-2℠ Data + Clock Ctrl § Short range sensor interface: MIPI CSI-2℠ and MIPI D-PHY℠
- 18. © 2020 MIPI Alliance, Inc. 18 Front Camera: Hardware in the Loop testing Raw data (Recorded or simulated) Camera Image sensor (e.g. CMOS) Camera and processing module on one PCB Lens Image processing unit (SoC, FPGA, DSP, … e.g. EyeQ™) µC CAN FlexRay ETH ECUconnector RAM/ flash Power Monochrome, RGBBayeror RCCCfilter Development andtest interface Camera Image sensor (e.g. CMOS) Camera and processing module on one PCB Lens Image processing unit (SoC, FPGA, DSP, … e.g. EyeQ™) µC CAN FlexRay ETH ECUconnector RAM/ flash Power Monochrome, RGBBayeror RCCCfilter Development andtest interface MIPI CSI-2℠ Data + Clock Ctrl § Short range sensor interface: MIPI CSI-2℠ and MIPI D-PHY℠ § Test system simulates the sensor § MIPI CSI-2℠ data stream incl. timing § Sensor control interface (e.g. I2C)
- 19. © 2020 MIPI Alliance, Inc. 19 Surround View Camera: HIL testing § Long range sensor interfaces § 1 cable for data, control, and power § Serializer and Deserializer (SerDes) SerDes Links (e.g. MIPI A-PHY℠) processing module CAN FlexRay ETH ECUconnector RAM/ flash Power Serializer Deserializer Camera module Camera Image sensor (e.g. CMOS) Lens Monochrome, RGBBayeror RCCCfilter Image processing unit (SoC, FPGA, DSP, … e.g. EyeQ™) µC
- 20. © 2020 MIPI Alliance, Inc. 20 Surround View Camera: HIL testing Serializer Raw data (Recorded or simulated) processing module CAN FlexRay ETH ECUconnector RAM/ flash Power Serializer Deserializer Camera module Camera Image sensor (e.g. CMOS) Lens Monochrome, RGBBayeror RCCCfilter Image processing unit (SoC, FPGA, DSP, … e.g. EyeQ™) µC LVDS Link (e.g. MIPI A-PHY℠)
- 21. © 2020 MIPI Alliance, Inc. 21 Virtual Test Drives – Recorded Sensor Data
- 22. © 2020 MIPI Alliance, Inc. 22 Virtual Test Drives – Recorded Sensor Data Serializer Raw data (Recorded or simulated) processing module CAN FlexRay ETH ECUconnector RAM/ flash Power Serializer Deserializer Camera module Camera Image sensor (e.g. CMOS) Lens Monochrome, RGBBayeror RCCCfilter Image processing unit (SoC, FPGA, DSP, … e.g. EyeQ™) µC LVDS Link (e.g. MIPI A-PHY℠)
- 23. © 2020 MIPI Alliance, Inc. 23 Sensor Data Replay HIL TestManagement Cloud Data center System under Test Data Replay solution Playback Recorded data Compare “KPI” Ground truth Test results Results Annotated recorded data Raw sensor data Camera Radar LiDAR GPS V2X Ultrasonic Network/Bus data Ethernet Some/IP CAN(FD) FlexRay
- 24. © 2020 MIPI Alliance, Inc. 24 gPTP: generalized precision time protocol Sensor Data Replay HIL § Test sensor components and ADAS/AD platforms § High quality data synchronization through gPTP § Modular solution to fit variant bandwidth § Flexible adaption of sensor interfaces via plug-in modules e.g. MIPI CSI-2℠, LVDS, … Key takeawaysSensor fusion and environment perception tests dSPACE SCALEXIO real-time system dSPACE ESI Unit
- 25. © 2020 MIPI Alliance, Inc. 25 Virtual Test Drives – Simulated Sensor Data
- 26. © 2020 MIPI Alliance, Inc. 26 Virtual Test Drives – Simulated Sensor Data processing module CAN FlexRay ETH ECUconnector RAM/ flash Power Serializer Deserializer Camera module Camera Image sensor (e.g. CMOS) Lens Monochrome, RGBBayeror RCCCfilter Image processing unit (SoC, FPGA, DSP, … e.g. EyeQ™) µC LVDS Link (e.g. MIPI A-PHY℠) § Simulated traffic scenarios § Testing relevant corner-cases § Thousands of different scenarios based on one test-drive § Change, e.g., weather, # pedestrians, etc. § Closed-loop simulation
- 27. © 2020 MIPI Alliance, Inc. 27 § Vehicle & Environment Models § Running in hard real-time § Camera Sensor Model on GPU § Lens simulation § Bayer Pattern Simulation § FPGA provides SERDES sensor interfaces § Camera Sensor Model on FPGA Key takeawaysdSPACE Sensor Simulation HIL Sensor Models Vehicle & Environment Models AD controllerInterfaces Ethernet Displayport SERDESinterface SCALEXIO (ASM) Sensor Simulation (powerful GPUs) ESI Unit (powerful FPGAs) Simulation of ADAS/AD Sensors
- 28. © 2020 MIPI Alliance, Inc. 28 Simulation of ADAS/AD Sensors - GPU Challenges § Verification of sophisticated AI algorithms for AD § ISO 26262 ASIL checks Solution § Accurate and verified sensor models for Camera, Radar, Lidar § Ray tracing § Weather simulation § Lens simulation § Open standard for sensor simulation § OpenSimulationInterface (OSI) § OpenDrive, OpenScenario Key takeawaysGPU-based Camera Simulation Sensor Models Vehicle & Environment Models AD controllerInterfaces Ethernet Displayport SERDESinterface SCALEXIO (ASM) Sensor Simulation (powerful GPUs) ESI Unit (powerful FPGAs)
- 29. © 2020 MIPI Alliance, Inc. 29 Simulation of ADAS/AD Sensors – ESI Unit § Support for all relevant sensor interfaces (Up to 10 Gbit/s) § Raw data simulation for Camera, Radar, and Lidar § FPGA-based sensor models § E.g., I2C simulation § Up to 50 Gbit/s aggregated data rate per ESI Unit § Synchronous simulate of up to 12 sensors per ESI Unit § Synchronization of multiple ESI Units and SCALEXIO Key takeawaysdSPACE Sensor Simulation HIL Sensor Models Vehicle & Environment Models AD controllerInterfaces Ethernet Displayport SERDESinterface SCALEXIO (ASM) Sensor Simulation (powerful GPUs) ESI Unit (powerful FPGAs)
- 30. © 2020 MIPI Alliance, Inc. 30 Conclusion