Machine Learning-based Test Selection for Simulation-based Testing of Self-driving Cars Software
•
0 likes•170 views
This document discusses using machine learning to select self-driving car test cases for simulation-based testing. It presents an approach called SDC-Scissor that uses ML models and road features extracted from test cases to predict if they will be safe or unsafe. This allows prioritizing potentially unsafe tests to improve testing efficiency. The approach is evaluated on its ability to identify safe and unsafe tests cases, and whether it can reduce testing time and costs. SDC-Scissor was able to select relevant test cases and showed potential to improve the cost-effectiveness of simulation-based self-driving car testing.
Recommended
More Related Content
Similar to Machine Learning-based Test Selection for Simulation-based Testing of Self-driving Cars Software
Similar to Machine Learning-based Test Selection for Simulation-based Testing of Self-driving Cars Software (20)
Recently uploaded
Recently uploaded (20)
Machine Learning-based Test Selection for Simulation-based Testing of Self-driving Cars Software
- 1. Machine Learning-based Test Selection for Simulation-based Testing of Self-driving Cars Software Journal-First Track Empirical Software Engineering 28, 71 (2023) Christian Birchler, Sajad Khatiri, Bill Bosshard, Alessio Gambi, Sebastiano Panichella
- 2. Motivation 392 incidents with ADAS vehicles, NHTSA 2022. ASE'23 Journal-First 2 14.09.23
- 3. SDC Testing in Simulation ASE'23 Journal-First 3 14.09.23
- 4. SDC-Scissor Approach ASE'23 Journal-First 4 14.09.23 Test Generation Test Execution Labeled Dataset Feature Extraction ML Assessment Test Generation Feature Extraction Test Outcome Prediction
- 5. SDC Test Definition ASE'23 Journal-First 5 14.09.23
- 6. Dataset Generation ASE'23 Journal-First 6 14.09.23
- 7. Generic ML-based Workflow Test Outcome Prediction Road Features ASE'23 Journal-First 7 14.09.23
- 8. Road Features Direct Distance Road Length # Left/Right Turns Turn Angles ASE'23 Journal-First 8 14.09.23
- 9. Evaluation RQ1: To what extent is it possible to identify safe and unsafe SDC test cases before executing them? RQ2: Does SDC-Scissor improve the cost-effectiveness of simulation-based testing of SDCs? RQ3: What is the actual upper bound on the precision and recall of ML techniques in identifying SDC safe and unsafe test cases when using static SDC features? ASE'23 Journal-First 9 14.09.23
- 10. RQ1: To what extent is it possible to identify safe and unsafe SDC test cases before executing them? J48 Decision Tree Naïve Bayes Logistic Random Forest ASE'23 Journal-First 10 14.09.23
- 11. RQ2: Does SDC-Scissor improve the cost-effectiveness of simulation-based testing of SDCs? # Test cases is fixed Select N=10 test cases ASE'23 Journal-First 11 14.09.23
- 12. RQ2: Does SDC-Scissor improve the cost-effectiveness of simulation-based testing of SDCs? # Test cases selected until N=10 failing test cases are identified SDC-Scissor spends ca. 50% less time on executing passing tests! ASE'23 Journal-First 12 14.09.23
- 13. RQ3: What is the actual upper bound on the precision and recall of ML techniques in identifying SDC safe and unsafe test cases when using static SDC features? Improving the ML models Hyperparameter optimization Grid Search ASE'23 Journal-First 13 14.09.23
- 14. Practical Relevance ASE'23 Journal-First 14 14.09.23
- 15. COSMOS: DevOps for Complex Cyber-physical Systems Sebastiano Panichella Sajad Khatiri Christian Birchler https://www.cosmos-devops.org/ https://twitter.com/COSMOS_DEVOPS ASE'23 Journal-First 15 14.09.23
- 16. CAN Bus Use Case Lighting Brakes Transmission Engine ASE'23 Journal-First 16 14.09.23
- 17. CAN Bus Use Case CAN Bus Lighting Brakes Transmission Engine Brakes ASE'23 Journal-First 17 14.09.23
- 18. Problem CAN Bus Our test cases do not reflect real world scenarios! ASE'23 Journal-First 18 14.09.23
- 19. Our Intuition CAN Bus I simulate real world scenarios! ASE'23 Journal-First 19 14.09.23
- 20. System Overview ASE'23 Journal-First 20 14.09.23
- 21. Use Case Scenario Windows Server Simulator SDC-Scissor CAN Driver ECU (Raspberry Pi) CAN Driver Filter Cloud App CAN Bus ASE'23 Journal-First 21 14.09.23
- 22. SDC-Scissor selects relevant test cases Summary SDC-Scissor uses ML models and road features SDC-Scissor has a practical relevance for AICAS ASE'23 Journal-First 22 14.09.23