Successfully reported this slideshow.
Your SlideShare is downloading. ×



Check these out next

1 of 25 Ad

More Related Content


More from iQHub (20)


Recently uploaded (20)


  1. 1. Assessing ADS real-life performance: Uniting real data, AI and optimization for scenario-driven design validation and optimization Alexandre Mugnai Business Development Manager
  2. 2. ESTECO The Challenge ADAS & AD Validation Approach Conclusions What have we learned? Agenda The Proposition
  3. 3. ESTECO
  4. 4. © 2021 ESTECO SpA ESTECO is an independent software company, highly specialized in numerical optimization and simulation process and data management.
  5. 5. © 2021 ESTECO SpA Our products The leading software solution for simulation process automation and design optimization The innovative enterprise platform for Simulation Process and Data Management (SPDM) and design optimization
  6. 6. Find the optimal design Handle your design parameters and balance conflicting objectives. Maximize IT resources Exploit all computational resources and engineering solvers. Deliver results on time Accelerate the engineering process and run multiple simulations.
  7. 7. Make simulation data accessible Expand the usage of engineering simulation across teams. Reduce time-to-market Fast deliver the best product by applying intelligent algorithms to the simulation process. Lower costs Maximize the investment in engineering solvers and IT resources.
  8. 8. © 2021 ESTECO SpA Our customers and industries Embraer Leonardo Lockheed Martin Bombardier FCA Ford Honda PSA Group Toyota Volvo Cars Corporation Mahindra TAFE Volvo Trucks ABB Bajaj BASF Cummins FAW Whirlpool Sony Automotive and Ground Transportation Aerospace Architecture, Engineering and Construction Manufacturing and Industrial Equipment Marine Energy Healthcare Consumer Goods Electronics
  9. 9. The challenge
  10. 10. © 2021 ESTECO SpA Autonomous Vehicles: The verification challenge ”Autonomous vehicle shall safely manage every possible situation on the road” Safety proof by driving Identify all possible situations Test AV in these situations 1-10 billion km Accidents may happen Scenario types Parameter ranges Edge cases Realistic interactions between actors Performance evaluation Assuring test coverage Finding & solving performance gaps
  11. 11. © 2021 ESTECO SpA • Data, AI & Optimization to Aid Scenario-Based Validation REAL DATA AI analysis and modelling Test Coverage & Opt. Evaluation Test coverage Find & solve issues Scenario types Parameter ranges Edge cases Realistic interactions Understand complex scenarios Test only relevant conditions Sample complexity of real world Data-driven scenario-based development and validation process REAL DATA Data Collection Test vehicles High cost & time Complex infrastructure Driver bias Customer vehicles Low accuracy Sensor bias Huge investment Complex infrastructure Infrastructure sensors Anywhere Anytime Cost-efficient
  12. 12. The proposition
  13. 13. © 2021 ESTECO SpA Vehicle data Sensor data # RAW DATA STORAGE - vehicles - sensors, GPS - annotations Scenario category definition Scenario Mining Run identification and classification algorithms SCENARIO DATABASE incl. all characteristic parameter distributions Specify Operational Design Domain Statistical based sampling Simulations Residual Risk & Uncertainty Vehicle Validation Behavioural model (Inv. Reinforc. Learning) Critical scenarios Algorithms Sensors Actuators DB Completeness Edge cases Severity Exposure Optimisation Propose a methodology for the assessment of ADAS/AD systems based on scenario mining and scenario based testing (Streetwise – ) The offering OEM / Tier
  14. 14. Validation Approach Applied To ACC Function
  15. 15. © 2021 ESTECO SpA • Scenario: ACC • First impression • Very basic, yet relatively complicated case when considering function parameters and various vehicle conditions (i.e. ego vehicle & target vehicle) • Simple enough to confirm method is functioning properly • Partners • TNO providing scenario database, measurables (i.e. severity – exposure – risk) • ESTECO developing process automation and optimization tools (i.e. specialized algorithms) • Scope: provide customers a methodology to optimize and validate a given function operating in a specific ODD The Use Case: ACC
  16. 16. © 2021 ESTECO SpA Lead vehicle decelerating scenario The Use Case: ACC (Cont.) ACC parameters • k1: distance gain • k2: speed gain ve,0 Scenario parameters • ve,0 • vl,0 Identify the ACC parameters such that: • The driving style is similar to human drive style • The risk of a collision is minimized Optimisation
  17. 17. © 2021 ESTECO SpA The Use Case: ACC (Cont.) ACC parameter optimisation
  18. 18. © 2021 ESTECO SpA The Use Case: ACC (Cont.) k1 = 1.0 k2 = 0.4 Pareto Front ACC parameter optimisation
  19. 19. © 2021 ESTECO SpA The Use Case: ACC (Cont.) Edge case identification
  20. 20. © 2021 ESTECO SpA The Use Case: ACC (Cont.) Edge case identification Pareto Front Safe and Probable Scenario Edge cases Collisions
  21. 21. Conclusion
  22. 22. © 2021 ESTECO SpA • By using cutting-edge AI to understand and model the real-world data, and by bringing these AI models into simulation and optimization, the approach optimizes and validates the function and its operational design domain (ODD) • Gather real-life unbiased scenario data in relevant locations and time • Produce a catalogue of parametrized scenarios with parameter distributions • Identify most critical cases for the validation of a function • Objectively quantify a system function and allow studies on: • Sensor positions • Sensor selection • Algorithm parameter tuning for function optimization Conclusion
  23. 23. What have we learned?
  24. 24. © 2021 ESTECO SpA • The proposed approach has proven to work on an ACC study and can be extended to various ADS functions that need to be developed, optimized and validated. • The method can be applied for any domain in which vehicle requires best performance to be identified in any real-life condition • Automate process to simplify system assessment and optimization • Algorithms play a significant role • Building statistical understanding on the environment in which system needs to operate • Identifying edge cases which challenge system performance • Optimizing system performance • Identify possible challenges ground truth system is facing • Design a system that is not over spec-ed (i.e. excessive sensors) understanding its strengths and weaknesses • Provide safety argumentation, based on right real-life data Take Away Ideas
  25. 25. Read the ESTECO Copyright Policy Thank you! Alexandre Mugnai +31 6 27392381