Downloaded 18 times
Uploaded byMIPI Alliance
MIPI DevCon 2020 | MIPI DevCon 2020 | How MIPI Interfaces Solve Challenges in ADAS/AD Sensor Validation
The document discusses how MIPI interfaces address challenges in validation of advanced driver-assistance systems (ADAS) and automated driving (AD) sensors. It outlines the validation solutions provided by dSPACE, such as data-driven development and testing methodologies, including hardware in the loop (HIL) and software in the loop (SIL) simulations. The presentation also highlights the importance of high data rates and real-time processing in ensuring the effectiveness of automotive sensor systems.
More Related Content
Similar to MIPI DevCon 2020 | MIPI DevCon 2020 | How MIPI Interfaces Solve Challenges in ADAS/AD Sensor Validation
MIPI DevCon 2020 | MIPI DevCon 2020 | How MIPI Interfaces Solve Challenges in ADAS/AD Sensor Validation
- 1. Gregor Sievers, Ph.D. dSPACEGmbH How MIPI® Interfaces Solve Challenges in ADAS/AD Sensor Validation
- 2. © 2020 MIPIAlliance, Inc. 2 Vision „Autonomy“ From simplicity
- 3. © 2020 MIPIAlliance, Inc. 3 Vision „Autonomy“ Hard real-time processing § Functional safety § … Artificial intelligence Dozens of Sensors § Camera § Radar § Lidar High Data Rates • Dozens of Gbit/s Towards complexity
- 4. © 2020 MIPIAlliance, Inc. 4 Vision „Autonomy“ Validation of sensors today §Real test drives §Expensive §Lot of data §Critical situations §Sensor hardware upgrade
- 5. © 2020 MIPIAlliance, Inc. 5 dSPACE AD validation solution Data-driven development and validation
- 6. © 2020 MIPIAlliance, Inc. 6 AD software stack – Validation and release testing Sensor front end tests Data replay tests Closed-loop SIL and HIL simulation Real test drives SIL: Software in the Loop HIL: Hardware in the Loop
- 7. © 2020 MIPIAlliance, Inc. 7 Automotive Front Camera Architecture Camera Image sensor (e.g. CMOS) Camera and processing module on one PCB Lens Image processing unit (SoC, FPGA, DSP, … e.g. EyeQ™) MIPI CSI-2℠ Data + Clock Ctrl µC CAN FlexRay ETH ECUconnector RAM/ flash Power Monochrome, RGBBayeror RCCCfilter § Short range sensor interface: MIPI CSI-2℠ and MIPI D-PHY℠ § MIPI CSI-2℠ replaced proprietary interfaces § Parallel § LVDS § Also used for Radar and Lidar applications
- 8. © 2020 MIPIAlliance, Inc. 8 Automotive Surround View Camera Architecture § Long range sensor interfaces § 1 cable for data, control, and power § Serializer and Deserializer (SerDes) § MIPI A-PHY℠ § Proprietary: TI FPD-Link, Maxim GMSL SerDes Links (e.g. MIPI A-PHY℠) processing module CAN FlexRay ETH ECUconnector RAM/ flash Power Serializer Deserializer Camera module Camera Image sensor (e.g. CMOS) Lens Monochrome, RGBBayeror RCCCfilter Image processing unit (SoC, FPGA, DSP, … e.g. EyeQ™) µC
- 9. © 2020 MIPIAlliance, Inc. 9 Data Logging of ADAS/AD Sensors
- 10. © 2020 MIPIAlliance, Inc. 10 Automotive Surround View Camera Architecture § Long range sensor interfaces § 1 cable for data, control, and power § Serializer and Deserializer (SerDes) SerDes Links (e.g. MIPI A-PHY℠) processing module CAN FlexRay ETH ECUconnector RAM/ flash Power Serializer Deserializer Camera module Camera Image sensor (e.g. CMOS) Lens Monochrome, RGBBayeror RCCCfilter Image processing unit (SoC, FPGA, DSP, … e.g. EyeQ™) µC
- 11. © 2020 MIPIAlliance, Inc. 11 Surround View Camera: Data Logging with ECU High-EndSSD processing module CAN FlexRay ETH ECUconnector RAM/ flash Power Deserializer Camera module Serializer Camera Image sensor (e.g. CMOS) Lens Monochrome, RGBBayeror RCCCfilter Image processing unit (SoC, FPGA, DSP, … e.g. EyeQ™) µC Logging platform Deserializer Serializer SerDes Links (e.g. MIPI A-PHY℠)
- 12. © 2020 MIPIAlliance, Inc. 12 Data Logging and Prototyping for ADAS/AD dSPACE AUTERA - The power of a Server – in the vehicle! § Intel Xeon CPU with 12 Cores (@2GHz) § Support for CAN, Automotive Ethernet, etc. § Up to 6x PCIe slots for extensions § High End Data Logging Capability § Up to 64 TB Storage § Up to 50 Gbit/s logging bandwidth § PCIe Grabber Card for Sensor Interfaces § Maxim GMSL and TI FPD-Link § Up to 6 Gbit/s per sensor § Additional Interfaces under development § MIPI A-PHY℠ once available Mainboard I/O-Rack 10GbE, USB 3.0, and 1GbE General Purpose I/O, USB 3.0, SIM-Slot and User-LEDs For additional sensor connections, Wireless Connectivity, Status-Info
- 13. © 2020 MIPIAlliance, Inc. 13 Virtual test drives
- 14. © 2020 MIPIAlliance, Inc. 14 Video: Recorded and Simulated Test Data
- 15. © 2020 MIPIAlliance, Inc. 15 AD software stack – Validation and release testing Virtualtestdrives– SILsim ulation Virtualtestdrives– HILsim ulation Realtest drives Virtual ECUs Real ECUs PC cluster SIL: Software in the Loop HIL: Hardware in the Loop ECU: Electronic Control Unit
- 16. © 2020 MIPIAlliance, Inc. 16 Front Camera: Hardware in the Loop testing Camera Image sensor (e.g. CMOS) Camera and processing module on one PCB Lens Image processing unit (SoC, FPGA, DSP, … e.g. EyeQ™) MIPI CSI-2℠ Data + Clock Ctrl µC CAN FlexRay ETH ECUconnector RAM/ flash Power Monochrome, RGBBayeror RCCCfilter § Short range sensor interface: MIPI CSI-2℠ and MIPI D-PHY℠
- 17. © 2020 MIPIAlliance, Inc. 17 Front Camera: Hardware in the Loop testing Camera Image sensor (e.g. CMOS) Camera and processing module on one PCB Lens Image processing unit (SoC, FPGA, DSP, … e.g. EyeQ™) µC CAN FlexRay ETH ECUconnector RAM/ flash Power Monochrome, RGBBayeror RCCCfilter Development andtest interface MIPI CSI-2℠ Data + Clock Ctrl § Short range sensor interface: MIPI CSI-2℠ and MIPI D-PHY℠
- 18. © 2020 MIPIAlliance, Inc. 18 Front Camera: Hardware in the Loop testing Raw data (Recorded or simulated) Camera Image sensor (e.g. CMOS) Camera and processing module on one PCB Lens Image processing unit (SoC, FPGA, DSP, … e.g. EyeQ™) µC CAN FlexRay ETH ECUconnector RAM/ flash Power Monochrome, RGBBayeror RCCCfilter Development andtest interface Camera Image sensor (e.g. CMOS) Camera and processing module on one PCB Lens Image processing unit (SoC, FPGA, DSP, … e.g. EyeQ™) µC CAN FlexRay ETH ECUconnector RAM/ flash Power Monochrome, RGBBayeror RCCCfilter Development andtest interface MIPI CSI-2℠ Data + Clock Ctrl § Short range sensor interface: MIPI CSI-2℠ and MIPI D-PHY℠ § Test system simulates the sensor § MIPI CSI-2℠ data stream incl. timing § Sensor control interface (e.g. I2C)
- 19. © 2020 MIPIAlliance, Inc. 19 Surround View Camera: HIL testing § Long range sensor interfaces § 1 cable for data, control, and power § Serializer and Deserializer (SerDes) SerDes Links (e.g. MIPI A-PHY℠) processing module CAN FlexRay ETH ECUconnector RAM/ flash Power Serializer Deserializer Camera module Camera Image sensor (e.g. CMOS) Lens Monochrome, RGBBayeror RCCCfilter Image processing unit (SoC, FPGA, DSP, … e.g. EyeQ™) µC
- 20. © 2020 MIPIAlliance, Inc. 20 Surround View Camera: HIL testing Serializer Raw data (Recorded or simulated) processing module CAN FlexRay ETH ECUconnector RAM/ flash Power Serializer Deserializer Camera module Camera Image sensor (e.g. CMOS) Lens Monochrome, RGBBayeror RCCCfilter Image processing unit (SoC, FPGA, DSP, … e.g. EyeQ™) µC LVDS Link (e.g. MIPI A-PHY℠)
- 21. © 2020 MIPIAlliance, Inc. 21 Virtual Test Drives – Recorded Sensor Data
- 22. © 2020 MIPIAlliance, Inc. 22 Virtual Test Drives – Recorded Sensor Data Serializer Raw data (Recorded or simulated) processing module CAN FlexRay ETH ECUconnector RAM/ flash Power Serializer Deserializer Camera module Camera Image sensor (e.g. CMOS) Lens Monochrome, RGBBayeror RCCCfilter Image processing unit (SoC, FPGA, DSP, … e.g. EyeQ™) µC LVDS Link (e.g. MIPI A-PHY℠)
- 23. © 2020 MIPIAlliance, Inc. 23 Sensor Data Replay HIL TestManagement Cloud Data center System under Test Data Replay solution Playback Recorded data Compare “KPI” Ground truth Test results Results Annotated recorded data Raw sensor data Camera Radar LiDAR GPS V2X Ultrasonic Network/Bus data Ethernet Some/IP CAN(FD) FlexRay
- 24. © 2020 MIPIAlliance, Inc. 24 gPTP: generalized precision time protocol Sensor Data Replay HIL § Test sensor components and ADAS/AD platforms § High quality data synchronization through gPTP § Modular solution to fit variant bandwidth § Flexible adaption of sensor interfaces via plug-in modules e.g. MIPI CSI-2℠, LVDS, … Key takeawaysSensor fusion and environment perception tests dSPACE SCALEXIO real-time system dSPACE ESI Unit
- 25. © 2020 MIPIAlliance, Inc. 25 Virtual Test Drives – Simulated Sensor Data
- 26. © 2020 MIPIAlliance, Inc. 26 Virtual Test Drives – Simulated Sensor Data processing module CAN FlexRay ETH ECUconnector RAM/ flash Power Serializer Deserializer Camera module Camera Image sensor (e.g. CMOS) Lens Monochrome, RGBBayeror RCCCfilter Image processing unit (SoC, FPGA, DSP, … e.g. EyeQ™) µC LVDS Link (e.g. MIPI A-PHY℠) § Simulated traffic scenarios § Testing relevant corner-cases § Thousands of different scenarios based on one test-drive § Change, e.g., weather, # pedestrians, etc. § Closed-loop simulation
- 27. © 2020 MIPIAlliance, Inc. 27 § Vehicle & Environment Models § Running in hard real-time § Camera Sensor Model on GPU § Lens simulation § Bayer Pattern Simulation § FPGA provides SERDES sensor interfaces § Camera Sensor Model on FPGA Key takeawaysdSPACE Sensor Simulation HIL Sensor Models Vehicle & Environment Models AD controllerInterfaces Ethernet Displayport SERDESinterface SCALEXIO (ASM) Sensor Simulation (powerful GPUs) ESI Unit (powerful FPGAs) Simulation of ADAS/AD Sensors
- 28. © 2020 MIPIAlliance, Inc. 28 Simulation of ADAS/AD Sensors - GPU Challenges § Verification of sophisticated AI algorithms for AD § ISO 26262 ASIL checks Solution § Accurate and verified sensor models for Camera, Radar, Lidar § Ray tracing § Weather simulation § Lens simulation § Open standard for sensor simulation § OpenSimulationInterface (OSI) § OpenDrive, OpenScenario Key takeawaysGPU-based Camera Simulation Sensor Models Vehicle & Environment Models AD controllerInterfaces Ethernet Displayport SERDESinterface SCALEXIO (ASM) Sensor Simulation (powerful GPUs) ESI Unit (powerful FPGAs)
- 29. © 2020 MIPIAlliance, Inc. 29 Simulation of ADAS/AD Sensors – ESI Unit § Support for all relevant sensor interfaces (Up to 10 Gbit/s) § Raw data simulation for Camera, Radar, and Lidar § FPGA-based sensor models § E.g., I2C simulation § Up to 50 Gbit/s aggregated data rate per ESI Unit § Synchronous simulate of up to 12 sensors per ESI Unit § Synchronization of multiple ESI Units and SCALEXIO Key takeawaysdSPACE Sensor Simulation HIL Sensor Models Vehicle & Environment Models AD controllerInterfaces Ethernet Displayport SERDESinterface SCALEXIO (ASM) Sensor Simulation (powerful GPUs) ESI Unit (powerful FPGAs)
- 30. © 2020 MIPIAlliance, Inc. 30 Conclusion