The progress in the development of articial intelligence engines has been driving the autonomous vehicle technology, which is projected to be a signicant market disruptor for various industries. For the public acceptance though, the autonomous vehicles must be proven to be reliable and their functionalities must be thoroughly validated. This is essential for improving the public trust for these vehicles and creating a communication medium between the manufacturers and the regulation authorities. Existing testing methods fall short of this goal and provide no clear certication scheme for autonomous vehicles. In this paper, we present a simulation scenario generation methodology with pseudo-random test generation and edge scenario discovery capabilities for testing autonomous vehicles. The validation framework separates the validation concerns and divides the testing scheme into several phases accordingly. The method uses a semantic language to generate scenarios with a particular focus on the validation of autonomous vehicle decisions, independent of environmental factors.

1. Scenario Generation for Validating AI
Based Autonomous Vehicles
03/23/2020
RKA 2016-06-22
Christopher Medrano-Berumen˚, M. Ilhan Akbas*
*Embry-Riddle Aeronautical University
˚Florida Polytechnic University

2. Outline
• Application Domain
• Problem Definition and Objective
• Approach
• Implementation
• Summary and Future Work

3. Application Domain
• Autonomous Vehicle Validation is Critical
– AVs will make a significant impact on various industries
– ADAS systems have been deployed commercially
– There are AV companies testing on streets
– More rigorous testing is required

4. Problem Definition
• Problem
– No fundamental structure exists to demonstrate the robustness of an AV
– The shadow driving testing is not sufficient and simulation testing is required
– Given the operation domain, there is no guidance on the exact type and
number of scenarios to test an AV in this domain
• Objective
– A simulation scenario generation methodology with pseudo-random test
generation capability for AV testing

5. Approach
• Part of a Larger Ongoing Research Study:
– Validation system is aimed for:
− Modeling of relevant driving scenario environment
− Analysis of sections along the pilot route for safety
− Simulation of critical scenarios and identification of improvement options
− Software-in-the-loop (SiL) testing of the vehicle using the generated scenarios
− Use of collected data through a physical AV test vehicle
– The validation approach will include people and vehicle movement
– A solution with lowest risk level is the goal

6. Approach
• Methodology
– Focus on simulation development for AV validation
– Simulation model development for low fidelity testing
– Details used to describe roads are parameterized, allowing us to both
randomly and deterministically generate scenarios
– The created scenarios can be used as input to high fidelity simulators

7. Approach
• Modeling and Simulation of
Abstract Scenarios
– We had developed a semantic
language to generate validation
scenarios*
– The study in this paper focuses
on the implementation for
simulation
– MATLAB AD Toolbox is used
*C. Medrano-Berumen and M. I. Akbas, “Abstract Simulation
Scenario Generation for Autonomous Vehicle Verification,” in
Proceedings of the IEEE SoutheastCon, IEEE, April 2019.

8. Implementation
• Road Segments
- Geometric centers are used for placing the roads in simulation models
- The width of the road at each center, the banking angles, and lane details
- As each road piece is generated, it is stitched to the previous piece
- The road pieces can also be created for specific street networks

9. Implementation
• Multilane Road
- Variable length and width
- Can take any geometry with G2 continuity
- Uses lines, clothoids, and arcs and parameters for curvature

10. Implementation
• Four Way Intersection
– Each connecting road has its
own attributes
– Possible paths are determined
according to attributes of all four
connecting roads
• Side Entrance
– Added to database for a specific
project
– Splits the road with a median
and has an entrance in the
middle

11. Implementation
• Actors
– Can be randomly generated
– Vehicles are generated along the path with variable size, path and velocity
– Pedestrians are generated on the sides of the road and move across

12. Example
• Example Scenario
– Curving multilane road with a vehicle and a pedestrian
– Pedestrian crossing the road

13. Example
• Example Scenario
– The simulation data is collected during and after each run
– Current collected data include legality of each decision and scenario definition

14. Summary and Future Work
• Validation of AVs is critical for their deployment in real life
• We build a framework to create a validation regime for a given AV
operation domain
• This presentation demonstrated the implementation of the approach
for the generation of abstract scenarios
• The next step will be applying the approach in specific operational
domains
• We also work on the identification of edge case scenarios

15. Thank You!
For more information about our research:
Dr. M. Ilhan Akbas
Assistant Professor of Electrical and Computer Engineering
Embry-Riddle Aeronautical University
• akbasm@erau.edu
• https://sites.google.com/site/miakbas/