Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Autonomous Vehicle Benchmarking Panel

995 views

Published on

Automated and Connected Vehicle Systems Testing Symposium panel slides: heavy tail ceiling and other issues

Published in: Automotive
  • Be the first to comment

Autonomous Vehicle Benchmarking Panel

  1. 1. SAE INTERNATIONAL
  2. 2. SAE INTERNATIONALSAE INTERNATIONAL Copyright © SAE International. Further use or distribution is not permitted without permission from SAE SAFE AUTOMATED VEHICLES BENCHMARKING PANEL: THE HEAVY TAIL SAFETY CEILING Phil Koopman, Safe Automated Vehicles Benchmarking Panel Prof. Phil Koopman http://ece.cmu.edu/~koopman/
  3. 3. SAE INTERNATIONALSAE INTERNATIONAL Copyright © SAE International. Further use or distribution is not permitted without permission from SAE Phil Koopman, Safe Automated Vehicles Benchmarking Panel What if you have 1 million miles between safety violation? • Situation #1: 100 “surprises” @ 100M miles/surprise • Situation #2: 100,000 “surprises” @ 100,000M miles/surprise Where will you be after 1 billion miles of validation testing? 1. High Confidence: Heavy Tail Surprises
  4. 4. SAE INTERNATIONALSAE INTERNATIONAL Copyright © SAE International. Further use or distribution is not permitted without permission from SAE The Heavy Tail Ceiling Phil Koopman, Safe Automated Vehicles Benchmarking Panel
  5. 5. SAE INTERNATIONALSAE INTERNATIONAL Copyright © SAE International. Further use or distribution is not permitted without permission from SAE Phil Koopman, Safe Automated Vehicles Benchmarking Panel •First get low hanging fruit – Drive/fail/fix for easy stuff •Try to genericize migitations – The less brittle the system, the better •Use noise & fault injection to inoculate against brittleness – For example, inject sensor noise Mitigating Heavy Tail Scenario Problems Pedestrian Missed: Gaussian Noise + Black Car Pedestrian Missed: Gaussian Blur
  6. 6. SAE INTERNATIONALSAE INTERNATIONAL Copyright © SAE International. Further use or distribution is not permitted without permission from SAE Phil Koopman, Safe Automated Vehicles Benchmarking Panel Need to cover edge cases in simulations • Simulation needs scenario coverage Synthetic environments for training • “Realistic” is in the eye of the beholder All simulations make assumptions • Use road testing to validate assumptions Pass tests for the right reason • Design system for observability 2. Real vs. Synthetic Environments
  7. 7. SAE INTERNATIONALSAE INTERNATIONAL Copyright © SAE International. Further use or distribution is not permitted without permission from SAE Phil Koopman, Safe Automated Vehicles Benchmarking Panel Problem: ML has no conventional requirements Use ISO 26262 for things that make sense • Conventional functions, vehicle control Safety Checker when feasible • E.g., check that path is clear Perception requirement proxies • Zoo of scenarios and obstacles • Brittleness metric for perception • Explicitly manage uncertainty (requirements; runtime) 3. Machine Learning vs. ISO 26262 ?
  8. 8. SAE INTERNATIONALSAE INTERNATIONAL Copyright © SAE International. Further use or distribution is not permitted without permission from SAE Phil Koopman, Safe Automated Vehicles Benchmarking Panel The more pressing issue is ethical deployment • Do we have transparency in safety arguments? • Is on-road testing adequately safe? • Are there safety losers in deployed systems? Will full AV vehicles be deployed before they are safe enough? • Do we know they are safe enough, or are we just hoping? • How can we balance market pressure, safety risk, and FOMO? 4 & 5. Forensic Analysis, Blame, Insurance

×