SlideShare a Scribd company logo
Automated Repair of Feature Interaction
Failures in Automated Driving Systems
Raja Ben Abdessalem, Annibale Panichella, Shiva Nejati,
Lionel C. Briand, and Thomas Stifter
!1
Automated Driving Systems
Traffic Sign Recognition (TSR)
Pedestrian Protection (PP) Lane Departure Warning (LDW)
!2
Automated Emergency Braking (AEB)
Feature Interactions
Sensors /
Camera
Autonomous
Feature
Actuator
Braking (over time)
!3
Sensors /
Camera
Autonomous
Feature
Actuator
Sensors /
Camera
Autonomous
Feature
Actuator
.
.
.
30 % 20 % … 80 %
Acceleration (over time)
60 % 10 % … 20 %
Steering (over time)
30 % 20 % … 80 %
(Deep Learning)
(Neural Net.)
(K-means)
Integration Components
!4
Pedestrian
Protection
(PP)
Autom. Emerg.
Braking
(AEB)
Lane Dep.
Warning
(LDW)
The integration is a rule set:
each condition checks a
specific feature interaction
situation and resolves
potential conflicts that may
arise under that condition
Testing Automated Driving Systems
!5
Testing on-the-road
!
Simulation-based Testing
Simulation-Based Test Case
Simulator
(Matlab/Simulink)
Test Input
Test Output
!6
Software
Under Test
(SUT)
Case Study
• Two case study systems from IEE (industrial partner)
• Designed by experts
• Manually tested for more than six months
• Different rules to integrated feature actuator commands
• 700K eLOC
• Two system-level test suites (≈30 min) with failing tests
• Both systems consist of four self-driving features
• ACC, AEB, TSR, PP
!7
Feature Interactions Failures
!8
Stop
Program Repair
!9
C. Le Goues et al. TSE 2012 Martinez and Monperrus, ISSTA 2016
Genetic Programming
!10
Patch
Selection
Faulty
Program
GP
Patch
Evaluation
Variants
Generation
Test Suite
Potential patches
are generated
using crossover
(AST cuts) and
mutation (AST
changes)
Run the entire test
suite against each
generated patch
The patches with a
lower number of failing
test cases survive
Genetic Programming
!11
Implicit Assumptions:
• One-defect assumption
• The patches require file line
changes
• Inexpensive test suites (a few
seconds)
• No guiding heuristics (a test
either fails or passes)
Automated Driving Systems:
• Multiple defects in different
locations
• Up to 100 lines to changes
• Each test suite requires 30 min
• Not all failures are equal (the
intensity of the violation changes)
ARIEL
Automated Repair of IntEgration
ruLes for ADS
12
ARIEL
ARIEL is a (1+1) Evolutionary Algorithm with an Archive
!13
ICSE ’20, May 23-29, 2020, Seoul, South Korea
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
e then
(1)
t have
failed
passed
assing
to the
failing
):
(2)
some
is the
t fails.
signed
s they
Algorithm 1: ARIEL
Input:
(f1, . . . , fn, ): Faulty self-driving system
TS: Test suite
Result: ⇤: a repaired rule-set satisfying all tc 2 TS
1 begin
2 Archive
3 RUN-EVALUATE( , TS)
4 while not(|Archive|==1 & Archive satises all tc 2 TS) do
5 p SELECT-A-PARENT(Archive) // Random selection
6 o GENERATE-PATCH( p, TS, )
7 RUN-EVALUATE( o, TS)
8 Archive UPDATE-ARCHIVE(Archive , o, )
9 return Archive
localization (Equation 1) and (2) mutating the rule set in p. The
routine GENERATE-PATCH is presented in subsection 3.2.1.
Then, the ospring o is evaluated (line 7) by running the test
suite TS, extracting the remaining failures, and computing their
corresponding objective scores ( ). Note that the severities of the
failures are our search objectives to optimize and are discussed in
Section 3.2.3. The ospring o is added to the archive (line 8 of Al-
gorithm 1) if it decreases the severity of the failures compared to the
patches currently stored in the archive. The archive and its updating
Archive
Run the faulty program and
computes the failures
intensities (search-objectives)
Generate only one patch
through customized fault
localization and mutation
Add the offspring to the archive
if it is better than the archive
for at least one failure
(1 parent + 1 offspring)
Customized Fault Localization
!14
FL formulae measures the suspicious (likely faulty) statements
in the production code based on the number of failing tests
te by wtc the weight (severity) of the failure of tc. We then
pute the suspiciousness of each statement s as follows:
Susp(s) =
Õ
tc2T Sf
[wtc ·co (tc,s)]
Õ
tc2T Sf
wtc
passed(s)
total_passed +
f ailed(s)
total_f ailed
(1)
e passed(s) counts the number of passed test cases that have
uted s at some time step; f ailed(s) counts the number of failed
ases that have executed s at some time step; and total_passed
otal_f ailed denote the total numbers of failing and passing
cases, respectively. Note that Equation 1 is equivalent to the
dard Tarantula formula if we let the weight (severity) for failing
cases be equal to one (i.e., if wtc = 1 for every tc 2 TSf ):
Susp(s) =
f ailed(s)
total_f ailed
passed(s)
total_passed +
f ailed(s)
total_f ailed
(2)
r each test case tc that fails at time step u and violates some
irement r, we dene wtc = |O(tc(u),r)|. That is, wtc is the
ee of violation caused by tc at the time step u when it fails.
ce, test cases that lead to more severe violations are assigned
r weights. Note that since we stop test cases as soon as they
each test case can violate at most one requirement.
Program Repair
Algorithm 1: ARIEL
Input:
(f1, . . . , fn, ): Faulty self-driving system
TS: Test suite
Result: ⇤: a repaired rule-set satisfying all tc 2
1 begin
2 Archive
3 RUN-EVALUATE( , TS)
4 while not(|Archive|==1  Archive satises al
5 p SELECT-A-PARENT(Archive)
6 o GENERATE-PATCH( p, TS,
7 RUN-EVALUATE( o, TS)
8 Archive UPDATE-ARCHIVE(Archi
9 return Archive
localization (Equation 1) and (2) mutatin
routine GENERATE-PATCH is presented
Then, the ospring o is evaluated (lin
suite TS, extracting the remaining failur
corresponding objective scores ( ). Note
failures are our search objectives to optim
Section 3.2.3. The ospring o is added to
gorithm 1) if it decreases the severity of the
patches currently stored in the archive. Th
routine are described in details in subsecti
when the termination criteria are met (se
Tarantula [Jones et al. 2002]
Suspicious statements are covered
by failing tests mostly
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
Automated Repair of Integration Rules in Automated Driving Systems
denote by wtc the weight (severity) of the failure of tc. We then
compute the suspiciousness of each statement s as follows:
Susp(s) =
Õ
tc2T Sf
[wtc ·co (tc,s)]
Õ
tc2T Sf
wtc
passed(s)
total_passed +
f ailed(s)
total_f ailed
(1)
where passed(s) counts the number of passed test cases that have
executed s at some time step; f ailed(s) counts the number of failed
test cases that have executed s at some time step; and total_passed
and total_f ailed denote the total numbers of failing and passing
test cases, respectively. Note that Equation 1 is equivalent to the
standard Tarantula formula if we let the weight (severity) for failing
test cases be equal to one (i.e., if wtc = 1 for every tc 2 TSf ):
f ailed(s)
Our formula
Failing tests have weights that are
proportional to the severity of the failures
Customized Mutation
!15
Potential patches are generated using only two operators:
• Changing the thresholds in the rules (e.g., minimum distance between cars)
• Shifting conditions within rule sets (changing the priorities of the checks/rules)
• No deletion (legal and ethical constraints) Anon.
727
728
729
730
731
732
733
734
735
736
737
738
739
Figure 5: Illustrating the shift operator: (a) selecting bs and
path , and (b) applying the shift operator.
Empirical Evaluation
16
Setting
!17
Benchmark:
• SafeDrive1 and SafeDrive2 from our industrial partner
Baselines:
• Genetic Programming (GP)
• Random Search (RS)
Parameters:
• GP with population size of 10 patches
• Search time = 16h
• 50 repetitions
Results
!18
SelfDrive1
#FailingTests
0
1
2
3
4
Time(h)
0 2 4 6 8 10 12 14 16
GP
ARIEL
Random
SelfDrive2
#FailingTests
0
0,5
1
1,5
2
Time(h)
0 2 4 6 8 10 12 14 16
GP
ARIEL
Random
Feedback From Domain Experts
!19
• We interviewed software engineers involved in the development of
AutoDrive1 and AutoDrive2
• ARIEL produces patches that differ from patches developers would
write manually (developers would add more integration rules)
• According to the developers, the patches generated by ARIEL are
valid, understandable, useful and optimal. Besides, they cannot be
produced by engineers
Synthesized patches are superior to manually-written
patches based on expert judgements
!20
In Summary

More Related Content

What's hot

Fault simulation – application and methods
Fault simulation – application and methodsFault simulation – application and methods
Fault simulation – application and methods
Subash John
 
Spyglass dft
Spyglass dftSpyglass dft
Spyglass dft
kumar gavanurmath
 
Search-driven String Constraint Solving for Vulnerability Detection
Search-driven String Constraint Solving for Vulnerability DetectionSearch-driven String Constraint Solving for Vulnerability Detection
Search-driven String Constraint Solving for Vulnerability Detection
Lionel Briand
 
Dissertation Defense
Dissertation DefenseDissertation Defense
Dissertation Defense
Sung Kim
 
Design for Testability
Design for TestabilityDesign for Testability
Design for Testability
Stanislav Tiurikov
 
Symbolic Execution And KLEE
Symbolic Execution And KLEESymbolic Execution And KLEE
Symbolic Execution And KLEE
Shauvik Roy Choudhary, Ph.D.
 
Ch 6 randomization
Ch 6 randomizationCh 6 randomization
Ch 6 randomization
Team-VLSI-ITMU
 
Coverage and Introduction to UVM
Coverage and Introduction to UVMCoverage and Introduction to UVM
Coverage and Introduction to UVM
Dr. Shivananda Koteshwar
 
Session 6 sv_randomization
Session 6 sv_randomizationSession 6 sv_randomization
Session 6 sv_randomization
Nirav Desai
 
Introduction to System verilog
Introduction to System verilog Introduction to System verilog
Introduction to System verilog
Pushpa Yakkala
 
Known XML Vulnerabilities Are Still a Threat to Popular Parsers ! & Open Sour...
Known XML Vulnerabilities Are Still a Threat to Popular Parsers ! & Open Sour...Known XML Vulnerabilities Are Still a Threat to Popular Parsers ! & Open Sour...
Known XML Vulnerabilities Are Still a Threat to Popular Parsers ! & Open Sour...
Lionel Briand
 
01 Transition Fault Detection methods by Swetha
01 Transition Fault Detection methods by Swetha01 Transition Fault Detection methods by Swetha
01 Transition Fault Detection methods by Swetha
swethamg18
 
Semaphores OS Basics
Semaphores OS BasicsSemaphores OS Basics
Semaphores OS Basics
Shijin Raj P
 
C++ Unit Test with Google Testing Framework
C++ Unit Test with Google Testing FrameworkC++ Unit Test with Google Testing Framework
C++ Unit Test with Google Testing Framework
Humberto Marchezi
 
When assertthat(you).understandUnitTesting() fails
When assertthat(you).understandUnitTesting() failsWhen assertthat(you).understandUnitTesting() fails
When assertthat(you).understandUnitTesting() fails
Martin Skurla
 
UVM TUTORIAL;
UVM TUTORIAL;UVM TUTORIAL;
UVM TUTORIAL;
Azad Mishra
 
API Performance Testing
API Performance TestingAPI Performance Testing
API Performance Testing
rsg00usa
 
System Verilog 2009 & 2012 enhancements
System Verilog 2009 & 2012 enhancementsSystem Verilog 2009 & 2012 enhancements
System Verilog 2009 & 2012 enhancements
Subash John
 
Semaphores
SemaphoresSemaphores
Semaphores
Mohd Arif
 
Session 8 assertion_based_verification_and_interfaces
Session 8 assertion_based_verification_and_interfacesSession 8 assertion_based_verification_and_interfaces
Session 8 assertion_based_verification_and_interfaces
Nirav Desai
 

What's hot (20)

Fault simulation – application and methods
Fault simulation – application and methodsFault simulation – application and methods
Fault simulation – application and methods
 
Spyglass dft
Spyglass dftSpyglass dft
Spyglass dft
 
Search-driven String Constraint Solving for Vulnerability Detection
Search-driven String Constraint Solving for Vulnerability DetectionSearch-driven String Constraint Solving for Vulnerability Detection
Search-driven String Constraint Solving for Vulnerability Detection
 
Dissertation Defense
Dissertation DefenseDissertation Defense
Dissertation Defense
 
Design for Testability
Design for TestabilityDesign for Testability
Design for Testability
 
Symbolic Execution And KLEE
Symbolic Execution And KLEESymbolic Execution And KLEE
Symbolic Execution And KLEE
 
Ch 6 randomization
Ch 6 randomizationCh 6 randomization
Ch 6 randomization
 
Coverage and Introduction to UVM
Coverage and Introduction to UVMCoverage and Introduction to UVM
Coverage and Introduction to UVM
 
Session 6 sv_randomization
Session 6 sv_randomizationSession 6 sv_randomization
Session 6 sv_randomization
 
Introduction to System verilog
Introduction to System verilog Introduction to System verilog
Introduction to System verilog
 
Known XML Vulnerabilities Are Still a Threat to Popular Parsers ! & Open Sour...
Known XML Vulnerabilities Are Still a Threat to Popular Parsers ! & Open Sour...Known XML Vulnerabilities Are Still a Threat to Popular Parsers ! & Open Sour...
Known XML Vulnerabilities Are Still a Threat to Popular Parsers ! & Open Sour...
 
01 Transition Fault Detection methods by Swetha
01 Transition Fault Detection methods by Swetha01 Transition Fault Detection methods by Swetha
01 Transition Fault Detection methods by Swetha
 
Semaphores OS Basics
Semaphores OS BasicsSemaphores OS Basics
Semaphores OS Basics
 
C++ Unit Test with Google Testing Framework
C++ Unit Test with Google Testing FrameworkC++ Unit Test with Google Testing Framework
C++ Unit Test with Google Testing Framework
 
When assertthat(you).understandUnitTesting() fails
When assertthat(you).understandUnitTesting() failsWhen assertthat(you).understandUnitTesting() fails
When assertthat(you).understandUnitTesting() fails
 
UVM TUTORIAL;
UVM TUTORIAL;UVM TUTORIAL;
UVM TUTORIAL;
 
API Performance Testing
API Performance TestingAPI Performance Testing
API Performance Testing
 
System Verilog 2009 & 2012 enhancements
System Verilog 2009 & 2012 enhancementsSystem Verilog 2009 & 2012 enhancements
System Verilog 2009 & 2012 enhancements
 
Semaphores
SemaphoresSemaphores
Semaphores
 
Session 8 assertion_based_verification_and_interfaces
Session 8 assertion_based_verification_and_interfacesSession 8 assertion_based_verification_and_interfaces
Session 8 assertion_based_verification_and_interfaces
 

Similar to Automated Repair of Feature Interaction Failures in Automated Driving Systems

(SAC2020 SVT-2) Constrained Detecting Arrays for Fault Localization in Combin...
(SAC2020 SVT-2) Constrained Detecting Arrays for Fault Localization in Combin...(SAC2020 SVT-2) Constrained Detecting Arrays for Fault Localization in Combin...
(SAC2020 SVT-2) Constrained Detecting Arrays for Fault Localization in Combin...
Hao Jin
 
Multiple Sensors Soft-Failure Diagnosis Based on Kalman Filter
Multiple Sensors Soft-Failure Diagnosis Based on Kalman FilterMultiple Sensors Soft-Failure Diagnosis Based on Kalman Filter
Multiple Sensors Soft-Failure Diagnosis Based on Kalman Filter
sipij
 
A Novel Extended Adaptive Thresholding for Industrial Alarm Systems
A Novel Extended Adaptive Thresholding for Industrial Alarm SystemsA Novel Extended Adaptive Thresholding for Industrial Alarm Systems
A Novel Extended Adaptive Thresholding for Industrial Alarm Systems
Koorosh Aslansefat
 
reliability workshop
reliability workshopreliability workshop
reliability workshop
Gaurav Dixit
 
Atc On An Simd Cots System Wmpp05
Atc On An Simd Cots System   Wmpp05Atc On An Simd Cots System   Wmpp05
Atc On An Simd Cots System Wmpp05
Ülger Ahmet
 
Combinational & Sequential ATPG.pdf
Combinational & Sequential ATPG.pdfCombinational & Sequential ATPG.pdf
Combinational & Sequential ATPG.pdf
MoinPasha12
 
Margin Parameter Variation for an Adaptive Observer to a Class of Systems
Margin Parameter Variation for an Adaptive Observer to a Class of SystemsMargin Parameter Variation for an Adaptive Observer to a Class of Systems
Margin Parameter Variation for an Adaptive Observer to a Class of Systems
CSCJournals
 
Argumentation Extensions Enumeration as a Constraint Satisfaction Problem: a ...
Argumentation Extensions Enumeration as a Constraint Satisfaction Problem: a ...Argumentation Extensions Enumeration as a Constraint Satisfaction Problem: a ...
Argumentation Extensions Enumeration as a Constraint Satisfaction Problem: a ...
Federico Cerutti
 
680report final
680report final680report final
680report final
Rajesh M
 
Efficient Implementation of Self-Organizing Map for Sparse Input Data
Efficient Implementation of Self-Organizing Map for Sparse Input DataEfficient Implementation of Self-Organizing Map for Sparse Input Data
Efficient Implementation of Self-Organizing Map for Sparse Input Data
ymelka
 
LeastSquaresParameterEstimation.ppt
LeastSquaresParameterEstimation.pptLeastSquaresParameterEstimation.ppt
LeastSquaresParameterEstimation.ppt
StavrovDule2
 
2014-mo444-practical-assignment-04-paulo_faria
2014-mo444-practical-assignment-04-paulo_faria2014-mo444-practical-assignment-04-paulo_faria
2014-mo444-practical-assignment-04-paulo_faria
Paulo Faria
 
Rbd best
Rbd bestRbd best
Rbd best
Khalid Aslam
 
08 pid.controller
08 pid.controller08 pid.controller
08 pid.controller
Mahmoud Hussein
 
Basic reliability models
Basic reliability modelsBasic reliability models
Basic reliability models
Ana Zuliastuti
 
CHAC Algorithm ECAL'07 Presentation
CHAC Algorithm ECAL'07 PresentationCHAC Algorithm ECAL'07 Presentation
CHAC Algorithm ECAL'07 Presentation
Antonio Mora
 
ELEG 421 Control Systems Transient and Steady State .docx
ELEG 421 Control Systems  Transient and Steady State .docxELEG 421 Control Systems  Transient and Steady State .docx
ELEG 421 Control Systems Transient and Steady State .docx
toltonkendal
 
An Exponential Observer Design for a Class of Chaotic Systems with Exponentia...
An Exponential Observer Design for a Class of Chaotic Systems with Exponentia...An Exponential Observer Design for a Class of Chaotic Systems with Exponentia...
An Exponential Observer Design for a Class of Chaotic Systems with Exponentia...
ijtsrd
 
9-High-Level-Fault-Grading.ppt
9-High-Level-Fault-Grading.ppt9-High-Level-Fault-Grading.ppt
9-High-Level-Fault-Grading.ppt
ZeroOne70
 
Actuator Constrained Optimal Control of Formations Near the Libration Points
Actuator Constrained Optimal Control of Formations Near the Libration PointsActuator Constrained Optimal Control of Formations Near the Libration Points
Actuator Constrained Optimal Control of Formations Near the Libration Points
Belinda Marchand
 

Similar to Automated Repair of Feature Interaction Failures in Automated Driving Systems (20)

(SAC2020 SVT-2) Constrained Detecting Arrays for Fault Localization in Combin...
(SAC2020 SVT-2) Constrained Detecting Arrays for Fault Localization in Combin...(SAC2020 SVT-2) Constrained Detecting Arrays for Fault Localization in Combin...
(SAC2020 SVT-2) Constrained Detecting Arrays for Fault Localization in Combin...
 
Multiple Sensors Soft-Failure Diagnosis Based on Kalman Filter
Multiple Sensors Soft-Failure Diagnosis Based on Kalman FilterMultiple Sensors Soft-Failure Diagnosis Based on Kalman Filter
Multiple Sensors Soft-Failure Diagnosis Based on Kalman Filter
 
A Novel Extended Adaptive Thresholding for Industrial Alarm Systems
A Novel Extended Adaptive Thresholding for Industrial Alarm SystemsA Novel Extended Adaptive Thresholding for Industrial Alarm Systems
A Novel Extended Adaptive Thresholding for Industrial Alarm Systems
 
reliability workshop
reliability workshopreliability workshop
reliability workshop
 
Atc On An Simd Cots System Wmpp05
Atc On An Simd Cots System   Wmpp05Atc On An Simd Cots System   Wmpp05
Atc On An Simd Cots System Wmpp05
 
Combinational & Sequential ATPG.pdf
Combinational & Sequential ATPG.pdfCombinational & Sequential ATPG.pdf
Combinational & Sequential ATPG.pdf
 
Margin Parameter Variation for an Adaptive Observer to a Class of Systems
Margin Parameter Variation for an Adaptive Observer to a Class of SystemsMargin Parameter Variation for an Adaptive Observer to a Class of Systems
Margin Parameter Variation for an Adaptive Observer to a Class of Systems
 
Argumentation Extensions Enumeration as a Constraint Satisfaction Problem: a ...
Argumentation Extensions Enumeration as a Constraint Satisfaction Problem: a ...Argumentation Extensions Enumeration as a Constraint Satisfaction Problem: a ...
Argumentation Extensions Enumeration as a Constraint Satisfaction Problem: a ...
 
680report final
680report final680report final
680report final
 
Efficient Implementation of Self-Organizing Map for Sparse Input Data
Efficient Implementation of Self-Organizing Map for Sparse Input DataEfficient Implementation of Self-Organizing Map for Sparse Input Data
Efficient Implementation of Self-Organizing Map for Sparse Input Data
 
LeastSquaresParameterEstimation.ppt
LeastSquaresParameterEstimation.pptLeastSquaresParameterEstimation.ppt
LeastSquaresParameterEstimation.ppt
 
2014-mo444-practical-assignment-04-paulo_faria
2014-mo444-practical-assignment-04-paulo_faria2014-mo444-practical-assignment-04-paulo_faria
2014-mo444-practical-assignment-04-paulo_faria
 
Rbd best
Rbd bestRbd best
Rbd best
 
08 pid.controller
08 pid.controller08 pid.controller
08 pid.controller
 
Basic reliability models
Basic reliability modelsBasic reliability models
Basic reliability models
 
CHAC Algorithm ECAL'07 Presentation
CHAC Algorithm ECAL'07 PresentationCHAC Algorithm ECAL'07 Presentation
CHAC Algorithm ECAL'07 Presentation
 
ELEG 421 Control Systems Transient and Steady State .docx
ELEG 421 Control Systems  Transient and Steady State .docxELEG 421 Control Systems  Transient and Steady State .docx
ELEG 421 Control Systems Transient and Steady State .docx
 
An Exponential Observer Design for a Class of Chaotic Systems with Exponentia...
An Exponential Observer Design for a Class of Chaotic Systems with Exponentia...An Exponential Observer Design for a Class of Chaotic Systems with Exponentia...
An Exponential Observer Design for a Class of Chaotic Systems with Exponentia...
 
9-High-Level-Fault-Grading.ppt
9-High-Level-Fault-Grading.ppt9-High-Level-Fault-Grading.ppt
9-High-Level-Fault-Grading.ppt
 
Actuator Constrained Optimal Control of Formations Near the Libration Points
Actuator Constrained Optimal Control of Formations Near the Libration PointsActuator Constrained Optimal Control of Formations Near the Libration Points
Actuator Constrained Optimal Control of Formations Near the Libration Points
 

More from Lionel Briand

Precise and Complete Requirements? An Elusive Goal
Precise and Complete Requirements? An Elusive GoalPrecise and Complete Requirements? An Elusive Goal
Precise and Complete Requirements? An Elusive Goal
Lionel Briand
 
Large Language Models for Test Case Evolution and Repair
Large Language Models for Test Case Evolution and RepairLarge Language Models for Test Case Evolution and Repair
Large Language Models for Test Case Evolution and Repair
Lionel Briand
 
Metamorphic Testing for Web System Security
Metamorphic Testing for Web System SecurityMetamorphic Testing for Web System Security
Metamorphic Testing for Web System Security
Lionel Briand
 
Simulator-based Explanation and Debugging of Hazard-triggering Events in DNN-...
Simulator-based Explanation and Debugging of Hazard-triggering Events in DNN-...Simulator-based Explanation and Debugging of Hazard-triggering Events in DNN-...
Simulator-based Explanation and Debugging of Hazard-triggering Events in DNN-...
Lionel Briand
 
Fuzzing for CPS Mutation Testing
Fuzzing for CPS Mutation TestingFuzzing for CPS Mutation Testing
Fuzzing for CPS Mutation Testing
Lionel Briand
 
Data-driven Mutation Analysis for Cyber-Physical Systems
Data-driven Mutation Analysis for Cyber-Physical SystemsData-driven Mutation Analysis for Cyber-Physical Systems
Data-driven Mutation Analysis for Cyber-Physical Systems
Lionel Briand
 
Many-Objective Reinforcement Learning for Online Testing of DNN-Enabled Systems
Many-Objective Reinforcement Learning for Online Testing of DNN-Enabled SystemsMany-Objective Reinforcement Learning for Online Testing of DNN-Enabled Systems
Many-Objective Reinforcement Learning for Online Testing of DNN-Enabled Systems
Lionel Briand
 
ATM: Black-box Test Case Minimization based on Test Code Similarity and Evolu...
ATM: Black-box Test Case Minimization based on Test Code Similarity and Evolu...ATM: Black-box Test Case Minimization based on Test Code Similarity and Evolu...
ATM: Black-box Test Case Minimization based on Test Code Similarity and Evolu...
Lionel Briand
 
Black-box Safety Analysis and Retraining of DNNs based on Feature Extraction ...
Black-box Safety Analysis and Retraining of DNNs based on Feature Extraction ...Black-box Safety Analysis and Retraining of DNNs based on Feature Extraction ...
Black-box Safety Analysis and Retraining of DNNs based on Feature Extraction ...
Lionel Briand
 
PRINS: Scalable Model Inference for Component-based System Logs
PRINS: Scalable Model Inference for Component-based System LogsPRINS: Scalable Model Inference for Component-based System Logs
PRINS: Scalable Model Inference for Component-based System Logs
Lionel Briand
 
Revisiting the Notion of Diversity in Software Testing
Revisiting the Notion of Diversity in Software TestingRevisiting the Notion of Diversity in Software Testing
Revisiting the Notion of Diversity in Software Testing
Lionel Briand
 
Applications of Search-based Software Testing to Trustworthy Artificial Intel...
Applications of Search-based Software Testing to Trustworthy Artificial Intel...Applications of Search-based Software Testing to Trustworthy Artificial Intel...
Applications of Search-based Software Testing to Trustworthy Artificial Intel...
Lionel Briand
 
Autonomous Systems: How to Address the Dilemma between Autonomy and Safety
Autonomous Systems: How to Address the Dilemma between Autonomy and SafetyAutonomous Systems: How to Address the Dilemma between Autonomy and Safety
Autonomous Systems: How to Address the Dilemma between Autonomy and Safety
Lionel Briand
 
Mathematicians, Social Scientists, or Engineers? The Split Minds of Software ...
Mathematicians, Social Scientists, or Engineers? The Split Minds of Software ...Mathematicians, Social Scientists, or Engineers? The Split Minds of Software ...
Mathematicians, Social Scientists, or Engineers? The Split Minds of Software ...
Lionel Briand
 
Reinforcement Learning for Test Case Prioritization
Reinforcement Learning for Test Case PrioritizationReinforcement Learning for Test Case Prioritization
Reinforcement Learning for Test Case Prioritization
Lionel Briand
 
Mutation Analysis for Cyber-Physical Systems: Scalable Solutions and Results ...
Mutation Analysis for Cyber-Physical Systems: Scalable Solutions and Results ...Mutation Analysis for Cyber-Physical Systems: Scalable Solutions and Results ...
Mutation Analysis for Cyber-Physical Systems: Scalable Solutions and Results ...
Lionel Briand
 
On Systematically Building a Controlled Natural Language for Functional Requi...
On Systematically Building a Controlled Natural Language for Functional Requi...On Systematically Building a Controlled Natural Language for Functional Requi...
On Systematically Building a Controlled Natural Language for Functional Requi...
Lionel Briand
 
Efficient Online Testing for DNN-Enabled Systems using Surrogate-Assisted and...
Efficient Online Testing for DNN-Enabled Systems using Surrogate-Assisted and...Efficient Online Testing for DNN-Enabled Systems using Surrogate-Assisted and...
Efficient Online Testing for DNN-Enabled Systems using Surrogate-Assisted and...
Lionel Briand
 
Guidelines for Assessing the Accuracy of Log Message Template Identification ...
Guidelines for Assessing the Accuracy of Log Message Template Identification ...Guidelines for Assessing the Accuracy of Log Message Template Identification ...
Guidelines for Assessing the Accuracy of Log Message Template Identification ...
Lionel Briand
 
A Theoretical Framework for Understanding the Relationship between Log Parsin...
A Theoretical Framework for Understanding the Relationship between Log Parsin...A Theoretical Framework for Understanding the Relationship between Log Parsin...
A Theoretical Framework for Understanding the Relationship between Log Parsin...
Lionel Briand
 

More from Lionel Briand (20)

Precise and Complete Requirements? An Elusive Goal
Precise and Complete Requirements? An Elusive GoalPrecise and Complete Requirements? An Elusive Goal
Precise and Complete Requirements? An Elusive Goal
 
Large Language Models for Test Case Evolution and Repair
Large Language Models for Test Case Evolution and RepairLarge Language Models for Test Case Evolution and Repair
Large Language Models for Test Case Evolution and Repair
 
Metamorphic Testing for Web System Security
Metamorphic Testing for Web System SecurityMetamorphic Testing for Web System Security
Metamorphic Testing for Web System Security
 
Simulator-based Explanation and Debugging of Hazard-triggering Events in DNN-...
Simulator-based Explanation and Debugging of Hazard-triggering Events in DNN-...Simulator-based Explanation and Debugging of Hazard-triggering Events in DNN-...
Simulator-based Explanation and Debugging of Hazard-triggering Events in DNN-...
 
Fuzzing for CPS Mutation Testing
Fuzzing for CPS Mutation TestingFuzzing for CPS Mutation Testing
Fuzzing for CPS Mutation Testing
 
Data-driven Mutation Analysis for Cyber-Physical Systems
Data-driven Mutation Analysis for Cyber-Physical SystemsData-driven Mutation Analysis for Cyber-Physical Systems
Data-driven Mutation Analysis for Cyber-Physical Systems
 
Many-Objective Reinforcement Learning for Online Testing of DNN-Enabled Systems
Many-Objective Reinforcement Learning for Online Testing of DNN-Enabled SystemsMany-Objective Reinforcement Learning for Online Testing of DNN-Enabled Systems
Many-Objective Reinforcement Learning for Online Testing of DNN-Enabled Systems
 
ATM: Black-box Test Case Minimization based on Test Code Similarity and Evolu...
ATM: Black-box Test Case Minimization based on Test Code Similarity and Evolu...ATM: Black-box Test Case Minimization based on Test Code Similarity and Evolu...
ATM: Black-box Test Case Minimization based on Test Code Similarity and Evolu...
 
Black-box Safety Analysis and Retraining of DNNs based on Feature Extraction ...
Black-box Safety Analysis and Retraining of DNNs based on Feature Extraction ...Black-box Safety Analysis and Retraining of DNNs based on Feature Extraction ...
Black-box Safety Analysis and Retraining of DNNs based on Feature Extraction ...
 
PRINS: Scalable Model Inference for Component-based System Logs
PRINS: Scalable Model Inference for Component-based System LogsPRINS: Scalable Model Inference for Component-based System Logs
PRINS: Scalable Model Inference for Component-based System Logs
 
Revisiting the Notion of Diversity in Software Testing
Revisiting the Notion of Diversity in Software TestingRevisiting the Notion of Diversity in Software Testing
Revisiting the Notion of Diversity in Software Testing
 
Applications of Search-based Software Testing to Trustworthy Artificial Intel...
Applications of Search-based Software Testing to Trustworthy Artificial Intel...Applications of Search-based Software Testing to Trustworthy Artificial Intel...
Applications of Search-based Software Testing to Trustworthy Artificial Intel...
 
Autonomous Systems: How to Address the Dilemma between Autonomy and Safety
Autonomous Systems: How to Address the Dilemma between Autonomy and SafetyAutonomous Systems: How to Address the Dilemma between Autonomy and Safety
Autonomous Systems: How to Address the Dilemma between Autonomy and Safety
 
Mathematicians, Social Scientists, or Engineers? The Split Minds of Software ...
Mathematicians, Social Scientists, or Engineers? The Split Minds of Software ...Mathematicians, Social Scientists, or Engineers? The Split Minds of Software ...
Mathematicians, Social Scientists, or Engineers? The Split Minds of Software ...
 
Reinforcement Learning for Test Case Prioritization
Reinforcement Learning for Test Case PrioritizationReinforcement Learning for Test Case Prioritization
Reinforcement Learning for Test Case Prioritization
 
Mutation Analysis for Cyber-Physical Systems: Scalable Solutions and Results ...
Mutation Analysis for Cyber-Physical Systems: Scalable Solutions and Results ...Mutation Analysis for Cyber-Physical Systems: Scalable Solutions and Results ...
Mutation Analysis for Cyber-Physical Systems: Scalable Solutions and Results ...
 
On Systematically Building a Controlled Natural Language for Functional Requi...
On Systematically Building a Controlled Natural Language for Functional Requi...On Systematically Building a Controlled Natural Language for Functional Requi...
On Systematically Building a Controlled Natural Language for Functional Requi...
 
Efficient Online Testing for DNN-Enabled Systems using Surrogate-Assisted and...
Efficient Online Testing for DNN-Enabled Systems using Surrogate-Assisted and...Efficient Online Testing for DNN-Enabled Systems using Surrogate-Assisted and...
Efficient Online Testing for DNN-Enabled Systems using Surrogate-Assisted and...
 
Guidelines for Assessing the Accuracy of Log Message Template Identification ...
Guidelines for Assessing the Accuracy of Log Message Template Identification ...Guidelines for Assessing the Accuracy of Log Message Template Identification ...
Guidelines for Assessing the Accuracy of Log Message Template Identification ...
 
A Theoretical Framework for Understanding the Relationship between Log Parsin...
A Theoretical Framework for Understanding the Relationship between Log Parsin...A Theoretical Framework for Understanding the Relationship between Log Parsin...
A Theoretical Framework for Understanding the Relationship between Log Parsin...
 

Recently uploaded

A Step-by-Step Guide to Selecting the Right Automated Software Testing Tools.pdf
A Step-by-Step Guide to Selecting the Right Automated Software Testing Tools.pdfA Step-by-Step Guide to Selecting the Right Automated Software Testing Tools.pdf
A Step-by-Step Guide to Selecting the Right Automated Software Testing Tools.pdf
kalichargn70th171
 
B.Sc. Computer Science Department PPT 2024
B.Sc. Computer Science Department PPT 2024B.Sc. Computer Science Department PPT 2024
B.Sc. Computer Science Department PPT 2024
vmsdeptcom
 
Old Tools, New Tricks: Unleashing the Power of Time-Tested Testing Tools
Old Tools, New Tricks: Unleashing the Power of Time-Tested Testing ToolsOld Tools, New Tricks: Unleashing the Power of Time-Tested Testing Tools
Old Tools, New Tricks: Unleashing the Power of Time-Tested Testing Tools
Benjamin Bischoff
 
Authentication Review-June -2024 AP & TS.pptx
Authentication Review-June -2024 AP & TS.pptxAuthentication Review-June -2024 AP & TS.pptx
Authentication Review-June -2024 AP & TS.pptx
DEMONDUOS
 
TEQnation 2024: Sustainable Software: May the Green Code Be with You
TEQnation 2024: Sustainable Software: May the Green Code Be with YouTEQnation 2024: Sustainable Software: May the Green Code Be with You
TEQnation 2024: Sustainable Software: May the Green Code Be with You
marcofolio
 
Hotel Management Software Development Company
Hotel Management Software Development CompanyHotel Management Software Development Company
Hotel Management Software Development Company
XongoLab Technologies LLP
 
bangalore Girls call 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
bangalore Girls call  👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Deliverybangalore Girls call  👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
bangalore Girls call 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
sunilverma7884
 
Verified Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeli...
Verified Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeli...Verified Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeli...
Verified Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeli...
87tomato
 
Mumbai Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service A...
Mumbai Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service A...Mumbai Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service A...
Mumbai Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service A...
3610stuck
 
🚂🚘 Premium Girls Call Ranchi 🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
🚂🚘 Premium Girls Call Ranchi  🛵🚡000XX00000 💃 Choose Best And Top Girl Service...🚂🚘 Premium Girls Call Ranchi  🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
🚂🚘 Premium Girls Call Ranchi 🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
bahubalikumar09988
 
當測試開始左移
當測試開始左移當測試開始左移
當測試開始左移
Jersey (CHE-PING) Su
 
Test Polarity: Detecting Positive and Negative Tests (FSE 2024)
Test Polarity: Detecting Positive and Negative Tests (FSE 2024)Test Polarity: Detecting Positive and Negative Tests (FSE 2024)
Test Polarity: Detecting Positive and Negative Tests (FSE 2024)
andrehoraa
 
How To Fill Timesheet in TaskSprint: Quick Guide 2024
How To Fill Timesheet in TaskSprint: Quick Guide 2024How To Fill Timesheet in TaskSprint: Quick Guide 2024
How To Fill Timesheet in TaskSprint: Quick Guide 2024
TaskSprint | Employee Efficiency Software
 
09. Ruby Object Oriented Programming - Ruby Core Teaching
09. Ruby Object Oriented Programming - Ruby Core Teaching09. Ruby Object Oriented Programming - Ruby Core Teaching
09. Ruby Object Oriented Programming - Ruby Core Teaching
quanhoangd129
 
VVIP Girls Call Mumbai 9910780858 Provide Best And Top Girl Service And No1 i...
VVIP Girls Call Mumbai 9910780858 Provide Best And Top Girl Service And No1 i...VVIP Girls Call Mumbai 9910780858 Provide Best And Top Girl Service And No1 i...
VVIP Girls Call Mumbai 9910780858 Provide Best And Top Girl Service And No1 i...
jealousviolet
 
Agra Girls Call Agra 0X0000000X Unlimited Short Providing Girls Service Avail...
Agra Girls Call Agra 0X0000000X Unlimited Short Providing Girls Service Avail...Agra Girls Call Agra 0X0000000X Unlimited Short Providing Girls Service Avail...
Agra Girls Call Agra 0X0000000X Unlimited Short Providing Girls Service Avail...
rachitkumar09887
 
04. Ruby Operators Slides - Ruby Core Teaching
04. Ruby Operators Slides - Ruby Core Teaching04. Ruby Operators Slides - Ruby Core Teaching
04. Ruby Operators Slides - Ruby Core Teaching
quanhoangd129
 
08. Ruby Enumerable - Ruby Core Teaching
08. Ruby Enumerable - Ruby Core Teaching08. Ruby Enumerable - Ruby Core Teaching
08. Ruby Enumerable - Ruby Core Teaching
quanhoangd129
 
SAP implementation steps PDF - Zyple Software
SAP implementation steps PDF - Zyple SoftwareSAP implementation steps PDF - Zyple Software
SAP implementation steps PDF - Zyple Software
Zyple Software
 
Busty Girls Call Mumbai 9930245274 Unlimited Short Providing Girls Service Av...
Busty Girls Call Mumbai 9930245274 Unlimited Short Providing Girls Service Av...Busty Girls Call Mumbai 9930245274 Unlimited Short Providing Girls Service Av...
Busty Girls Call Mumbai 9930245274 Unlimited Short Providing Girls Service Av...
revolutionary575
 

Recently uploaded (20)

A Step-by-Step Guide to Selecting the Right Automated Software Testing Tools.pdf
A Step-by-Step Guide to Selecting the Right Automated Software Testing Tools.pdfA Step-by-Step Guide to Selecting the Right Automated Software Testing Tools.pdf
A Step-by-Step Guide to Selecting the Right Automated Software Testing Tools.pdf
 
B.Sc. Computer Science Department PPT 2024
B.Sc. Computer Science Department PPT 2024B.Sc. Computer Science Department PPT 2024
B.Sc. Computer Science Department PPT 2024
 
Old Tools, New Tricks: Unleashing the Power of Time-Tested Testing Tools
Old Tools, New Tricks: Unleashing the Power of Time-Tested Testing ToolsOld Tools, New Tricks: Unleashing the Power of Time-Tested Testing Tools
Old Tools, New Tricks: Unleashing the Power of Time-Tested Testing Tools
 
Authentication Review-June -2024 AP & TS.pptx
Authentication Review-June -2024 AP & TS.pptxAuthentication Review-June -2024 AP & TS.pptx
Authentication Review-June -2024 AP & TS.pptx
 
TEQnation 2024: Sustainable Software: May the Green Code Be with You
TEQnation 2024: Sustainable Software: May the Green Code Be with YouTEQnation 2024: Sustainable Software: May the Green Code Be with You
TEQnation 2024: Sustainable Software: May the Green Code Be with You
 
Hotel Management Software Development Company
Hotel Management Software Development CompanyHotel Management Software Development Company
Hotel Management Software Development Company
 
bangalore Girls call 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
bangalore Girls call  👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Deliverybangalore Girls call  👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
bangalore Girls call 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
 
Verified Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeli...
Verified Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeli...Verified Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeli...
Verified Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeli...
 
Mumbai Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service A...
Mumbai Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service A...Mumbai Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service A...
Mumbai Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service A...
 
🚂🚘 Premium Girls Call Ranchi 🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
🚂🚘 Premium Girls Call Ranchi  🛵🚡000XX00000 💃 Choose Best And Top Girl Service...🚂🚘 Premium Girls Call Ranchi  🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
🚂🚘 Premium Girls Call Ranchi 🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
 
當測試開始左移
當測試開始左移當測試開始左移
當測試開始左移
 
Test Polarity: Detecting Positive and Negative Tests (FSE 2024)
Test Polarity: Detecting Positive and Negative Tests (FSE 2024)Test Polarity: Detecting Positive and Negative Tests (FSE 2024)
Test Polarity: Detecting Positive and Negative Tests (FSE 2024)
 
How To Fill Timesheet in TaskSprint: Quick Guide 2024
How To Fill Timesheet in TaskSprint: Quick Guide 2024How To Fill Timesheet in TaskSprint: Quick Guide 2024
How To Fill Timesheet in TaskSprint: Quick Guide 2024
 
09. Ruby Object Oriented Programming - Ruby Core Teaching
09. Ruby Object Oriented Programming - Ruby Core Teaching09. Ruby Object Oriented Programming - Ruby Core Teaching
09. Ruby Object Oriented Programming - Ruby Core Teaching
 
VVIP Girls Call Mumbai 9910780858 Provide Best And Top Girl Service And No1 i...
VVIP Girls Call Mumbai 9910780858 Provide Best And Top Girl Service And No1 i...VVIP Girls Call Mumbai 9910780858 Provide Best And Top Girl Service And No1 i...
VVIP Girls Call Mumbai 9910780858 Provide Best And Top Girl Service And No1 i...
 
Agra Girls Call Agra 0X0000000X Unlimited Short Providing Girls Service Avail...
Agra Girls Call Agra 0X0000000X Unlimited Short Providing Girls Service Avail...Agra Girls Call Agra 0X0000000X Unlimited Short Providing Girls Service Avail...
Agra Girls Call Agra 0X0000000X Unlimited Short Providing Girls Service Avail...
 
04. Ruby Operators Slides - Ruby Core Teaching
04. Ruby Operators Slides - Ruby Core Teaching04. Ruby Operators Slides - Ruby Core Teaching
04. Ruby Operators Slides - Ruby Core Teaching
 
08. Ruby Enumerable - Ruby Core Teaching
08. Ruby Enumerable - Ruby Core Teaching08. Ruby Enumerable - Ruby Core Teaching
08. Ruby Enumerable - Ruby Core Teaching
 
SAP implementation steps PDF - Zyple Software
SAP implementation steps PDF - Zyple SoftwareSAP implementation steps PDF - Zyple Software
SAP implementation steps PDF - Zyple Software
 
Busty Girls Call Mumbai 9930245274 Unlimited Short Providing Girls Service Av...
Busty Girls Call Mumbai 9930245274 Unlimited Short Providing Girls Service Av...Busty Girls Call Mumbai 9930245274 Unlimited Short Providing Girls Service Av...
Busty Girls Call Mumbai 9930245274 Unlimited Short Providing Girls Service Av...
 

Automated Repair of Feature Interaction Failures in Automated Driving Systems

  • 1. Automated Repair of Feature Interaction Failures in Automated Driving Systems Raja Ben Abdessalem, Annibale Panichella, Shiva Nejati, Lionel C. Briand, and Thomas Stifter !1
  • 2. Automated Driving Systems Traffic Sign Recognition (TSR) Pedestrian Protection (PP) Lane Departure Warning (LDW) !2 Automated Emergency Braking (AEB)
  • 3. Feature Interactions Sensors / Camera Autonomous Feature Actuator Braking (over time) !3 Sensors / Camera Autonomous Feature Actuator Sensors / Camera Autonomous Feature Actuator . . . 30 % 20 % … 80 % Acceleration (over time) 60 % 10 % … 20 % Steering (over time) 30 % 20 % … 80 % (Deep Learning) (Neural Net.) (K-means)
  • 4. Integration Components !4 Pedestrian Protection (PP) Autom. Emerg. Braking (AEB) Lane Dep. Warning (LDW) The integration is a rule set: each condition checks a specific feature interaction situation and resolves potential conflicts that may arise under that condition
  • 5. Testing Automated Driving Systems !5 Testing on-the-road ! Simulation-based Testing
  • 6. Simulation-Based Test Case Simulator (Matlab/Simulink) Test Input Test Output !6 Software Under Test (SUT)
  • 7. Case Study • Two case study systems from IEE (industrial partner) • Designed by experts • Manually tested for more than six months • Different rules to integrated feature actuator commands • 700K eLOC • Two system-level test suites (≈30 min) with failing tests • Both systems consist of four self-driving features • ACC, AEB, TSR, PP !7
  • 9. Program Repair !9 C. Le Goues et al. TSE 2012 Martinez and Monperrus, ISSTA 2016
  • 10. Genetic Programming !10 Patch Selection Faulty Program GP Patch Evaluation Variants Generation Test Suite Potential patches are generated using crossover (AST cuts) and mutation (AST changes) Run the entire test suite against each generated patch The patches with a lower number of failing test cases survive
  • 11. Genetic Programming !11 Implicit Assumptions: • One-defect assumption • The patches require file line changes • Inexpensive test suites (a few seconds) • No guiding heuristics (a test either fails or passes) Automated Driving Systems: • Multiple defects in different locations • Up to 100 lines to changes • Each test suite requires 30 min • Not all failures are equal (the intensity of the violation changes)
  • 12. ARIEL Automated Repair of IntEgration ruLes for ADS 12
  • 13. ARIEL ARIEL is a (1+1) Evolutionary Algorithm with an Archive !13 ICSE ’20, May 23-29, 2020, Seoul, South Korea 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 e then (1) t have failed passed assing to the failing ): (2) some is the t fails. signed s they Algorithm 1: ARIEL Input: (f1, . . . , fn, ): Faulty self-driving system TS: Test suite Result: ⇤: a repaired rule-set satisfying all tc 2 TS 1 begin 2 Archive 3 RUN-EVALUATE( , TS) 4 while not(|Archive|==1 & Archive satises all tc 2 TS) do 5 p SELECT-A-PARENT(Archive) // Random selection 6 o GENERATE-PATCH( p, TS, ) 7 RUN-EVALUATE( o, TS) 8 Archive UPDATE-ARCHIVE(Archive , o, ) 9 return Archive localization (Equation 1) and (2) mutating the rule set in p. The routine GENERATE-PATCH is presented in subsection 3.2.1. Then, the ospring o is evaluated (line 7) by running the test suite TS, extracting the remaining failures, and computing their corresponding objective scores ( ). Note that the severities of the failures are our search objectives to optimize and are discussed in Section 3.2.3. The ospring o is added to the archive (line 8 of Al- gorithm 1) if it decreases the severity of the failures compared to the patches currently stored in the archive. The archive and its updating Archive Run the faulty program and computes the failures intensities (search-objectives) Generate only one patch through customized fault localization and mutation Add the offspring to the archive if it is better than the archive for at least one failure (1 parent + 1 offspring)
  • 14. Customized Fault Localization !14 FL formulae measures the suspicious (likely faulty) statements in the production code based on the number of failing tests te by wtc the weight (severity) of the failure of tc. We then pute the suspiciousness of each statement s as follows: Susp(s) = Õ tc2T Sf [wtc ·co (tc,s)] Õ tc2T Sf wtc passed(s) total_passed + f ailed(s) total_f ailed (1) e passed(s) counts the number of passed test cases that have uted s at some time step; f ailed(s) counts the number of failed ases that have executed s at some time step; and total_passed otal_f ailed denote the total numbers of failing and passing cases, respectively. Note that Equation 1 is equivalent to the dard Tarantula formula if we let the weight (severity) for failing cases be equal to one (i.e., if wtc = 1 for every tc 2 TSf ): Susp(s) = f ailed(s) total_f ailed passed(s) total_passed + f ailed(s) total_f ailed (2) r each test case tc that fails at time step u and violates some irement r, we dene wtc = |O(tc(u),r)|. That is, wtc is the ee of violation caused by tc at the time step u when it fails. ce, test cases that lead to more severe violations are assigned r weights. Note that since we stop test cases as soon as they each test case can violate at most one requirement. Program Repair Algorithm 1: ARIEL Input: (f1, . . . , fn, ): Faulty self-driving system TS: Test suite Result: ⇤: a repaired rule-set satisfying all tc 2 1 begin 2 Archive 3 RUN-EVALUATE( , TS) 4 while not(|Archive|==1 Archive satises al 5 p SELECT-A-PARENT(Archive) 6 o GENERATE-PATCH( p, TS, 7 RUN-EVALUATE( o, TS) 8 Archive UPDATE-ARCHIVE(Archi 9 return Archive localization (Equation 1) and (2) mutatin routine GENERATE-PATCH is presented Then, the ospring o is evaluated (lin suite TS, extracting the remaining failur corresponding objective scores ( ). Note failures are our search objectives to optim Section 3.2.3. The ospring o is added to gorithm 1) if it decreases the severity of the patches currently stored in the archive. Th routine are described in details in subsecti when the termination criteria are met (se Tarantula [Jones et al. 2002] Suspicious statements are covered by failing tests mostly 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 Automated Repair of Integration Rules in Automated Driving Systems denote by wtc the weight (severity) of the failure of tc. We then compute the suspiciousness of each statement s as follows: Susp(s) = Õ tc2T Sf [wtc ·co (tc,s)] Õ tc2T Sf wtc passed(s) total_passed + f ailed(s) total_f ailed (1) where passed(s) counts the number of passed test cases that have executed s at some time step; f ailed(s) counts the number of failed test cases that have executed s at some time step; and total_passed and total_f ailed denote the total numbers of failing and passing test cases, respectively. Note that Equation 1 is equivalent to the standard Tarantula formula if we let the weight (severity) for failing test cases be equal to one (i.e., if wtc = 1 for every tc 2 TSf ): f ailed(s) Our formula Failing tests have weights that are proportional to the severity of the failures
  • 15. Customized Mutation !15 Potential patches are generated using only two operators: • Changing the thresholds in the rules (e.g., minimum distance between cars) • Shifting conditions within rule sets (changing the priorities of the checks/rules) • No deletion (legal and ethical constraints) Anon. 727 728 729 730 731 732 733 734 735 736 737 738 739 Figure 5: Illustrating the shift operator: (a) selecting bs and path , and (b) applying the shift operator.
  • 17. Setting !17 Benchmark: • SafeDrive1 and SafeDrive2 from our industrial partner Baselines: • Genetic Programming (GP) • Random Search (RS) Parameters: • GP with population size of 10 patches • Search time = 16h • 50 repetitions
  • 18. Results !18 SelfDrive1 #FailingTests 0 1 2 3 4 Time(h) 0 2 4 6 8 10 12 14 16 GP ARIEL Random SelfDrive2 #FailingTests 0 0,5 1 1,5 2 Time(h) 0 2 4 6 8 10 12 14 16 GP ARIEL Random
  • 19. Feedback From Domain Experts !19 • We interviewed software engineers involved in the development of AutoDrive1 and AutoDrive2 • ARIEL produces patches that differ from patches developers would write manually (developers would add more integration rules) • According to the developers, the patches generated by ARIEL are valid, understandable, useful and optimal. Besides, they cannot be produced by engineers Synthesized patches are superior to manually-written patches based on expert judgements