1) The document discusses using simulation to address the "long tail problem" of validating automated vehicles through testing alone. 2) It proposes using simulation with techniques like unsupervised learning on scenario clustering, reinforcement learning, and crowdsourcing data to recreate and test uncommon scenarios. 3) The future of validation is predicted to involve open standards and sharing of models and data through public scenario databases and blockchain technology.

1. Safety Argumentation and Using
Simulation to Solve Long Tail
Problem of Automated Vehicle
Feng Liu
Verification and Simulation Consultant
Providing Safety Argumentation as a Service

2. 10+ years experience
ØProducts
• Sensor: Camera
• ADAS system testing (MIL/SIL)
• AD system verification strategy
• Simulator development
• HPC cluster & toolchain
• Control by wire chassis
• AD truck (logistic as a service)
• ALKS regulation (VMAD)
ØCompanies
• Volvo Cars
• Zenuity
• CEVT
• Einride

3. Safety
How to explain to
authorities?
1
How to explain to
insurance
companies?
2
How to explain to
the public if an
accident
happened?
3

4. Safety Argumentation
A safety report
1. Safe by design ( ISO26262, SOTIF, UL4600, ISO21434)
• Model requirements (set requirements with models)
• Automatic test case generation
• Model check using Formal Methods
2. Safe by testing
• MIL/SIL/Test track
• Scenario database
• Synthetic data for training and validation
3. Safe by field montior / fleet monitor
• Abnormal detection
• Scenario annotation
Some examples:
- Start-of-the-art technology
- Data collection mileage
- Road test mileage
- Average human driver
- Statistical estimation
- Responsibility-Sensitive Safety

5. VIL
HIL
Simulation
Environment
Camera
Radar
Lidar
GNSS
IMU
Driver
ADAS/AD
Chassis
• Open-loop v.s. Close-loop
• Environment modeling
• Senaor modeling
• Driver modeling
• Chassis modeling
• MIL, SIL, virtual ECU
• Agent based simulation
AEB LKA
TSI
SF
Mode Manager
BSW
Planning
SIL
MIL

6. The long tail challenge

7. The long tail challenge
1 Million
Is 1-million-test-scenario equivalent to 11-billion-mile driving?

8. Solve the long tail problem by simulation

9. Solve the long tail problem by simulation
ØAI
• Unsupervised learning on scenario clustering
• Scenario recreation
• Reinforcement learning
ØCrowdsourcing
• Own fleet data
• 3rd party scenario database
• Public data
• Synthetic data

10. The future is open and standardization
ØASAM and C-ASAM
• OpenScenario
• OpenLabel
• OpenODD
ØPublic Scenario Database
• CATARC
• Safety Pool
• TNO
ØShare models and data with blockchain technology