SlideShare a Scribd company logo

Incremental Control Dependency Frontier Exploration for Many-Criteria Test Case Generation

The presentation was given at the 10th International Symposium on Seach-Based Software Engineering (SSBSE 2018) Abstract: Several criteria have been proposed over the years for measuring test suite adequacy. Each criterion can be converted into a specific objective function to optimize with search-based techniques in an attempt to generate test suites achieving the highest possible coverage for that criterion. Recent work has tried to optimize for multiple-criteria at once by constructing a single objective function obtained as a weighted sum of the objective functions of the respective criteria. However, this solution suffers the problem of sum scalarization, i.e., differences along the various dimensions being optimized get lost when such dimensions are projected into a single value. Recent advances in SBST formulated coverage as a many-objective optimization problem rather than applying sum scalarization. Starting from this formulation, in this work, we apply many-objective test generation that handles multiple adequacy criteria simultaneously. To scale the approach to the big number of objectives to be optimized at the same time, we adopt an incremental strategy, where only coverage targets in the control dependency frontier are considered until the frontier is expanded by covering a previously uncovered target.

1 of 29
Download to read offline
Incremental Control Dependency Frontier
Exploration for Many-Criteria
Test Case Generation
Annibale Panichella, Fitsum M. Kifetew, Paolo Tonella
September 11, 2017
!1
Background
Rojas et al. SSBSE 2015
"Optimising for several criteria at
the same time is feasible without
increasing computational costs"
!2
Classical Approach (WSA)
1
2 3
4 5
8
Class Under Test
7
9
6
Suite-level Fitness Function
3
Classical Approach (WSA)
1
2 3
4 5
8
7
9
6
Suite-level Fitness Function
Branch
Coverage
b2b1
b3 b4
b5
b6 b7
b8
b9
b10
b11 b12
F = f(b1) + f(b2) + … +f (b12)
Class Under Test
4
Classical Approach (WSA)
Suite-level Fitness Function
Statement
Coverage
F = f(b1) + f(b2) + … +f (b12)
+
f(s1) + f(s2) + … +f (s9)
1
2 3
5 4
8
7
9
6
Class Under Test
5
Classical Approach (WSA)
Suite-level Fitness Function
Weak Mutation
Coverage
F = f(b1) + f(b2) + … +f (b12)
+
f(s1) + f(s2) + … +f (s9)
f(m1) + f(m2) + … +f (m3)
+
1
2 m1
m2 m3
8
7
9
6
Class Under Test
5

Recommended

9. 8085 instruction set v
9. 8085 instruction set v9. 8085 instruction set v
9. 8085 instruction set vsandip das
 
TMPA-2017: Modeling of PLC-programs by High-level Coloured Petri Nets
TMPA-2017: Modeling of PLC-programs by High-level Coloured Petri NetsTMPA-2017: Modeling of PLC-programs by High-level Coloured Petri Nets
TMPA-2017: Modeling of PLC-programs by High-level Coloured Petri NetsIosif Itkin
 
Compiler Construction | Lecture 11 | Monotone Frameworks
Compiler Construction | Lecture 11 | Monotone FrameworksCompiler Construction | Lecture 11 | Monotone Frameworks
Compiler Construction | Lecture 11 | Monotone FrameworksEelco Visser
 
Gilbert: Declarative Sparse Linear Algebra on Massively Parallel Dataflow Sys...
Gilbert: Declarative Sparse Linear Algebra on Massively Parallel Dataflow Sys...Gilbert: Declarative Sparse Linear Algebra on Massively Parallel Dataflow Sys...
Gilbert: Declarative Sparse Linear Algebra on Massively Parallel Dataflow Sys...Till Rohrmann
 
Reformulating Branch Coverage as a Many-Objective Optimization Problem
Reformulating Branch Coverage as a Many-Objective Optimization ProblemReformulating Branch Coverage as a Many-Objective Optimization Problem
Reformulating Branch Coverage as a Many-Objective Optimization ProblemAnnibale Panichella
 
Results for EvoSuite-MOSA at the Third Unit Testing Tool Competition
Results for EvoSuite-MOSA at the Third Unit Testing Tool CompetitionResults for EvoSuite-MOSA at the Third Unit Testing Tool Competition
Results for EvoSuite-MOSA at the Third Unit Testing Tool CompetitionAnnibale Panichella
 
Speeding Up Distributed Machine Learning Using Codes
Speeding Up Distributed Machine Learning Using CodesSpeeding Up Distributed Machine Learning Using Codes
Speeding Up Distributed Machine Learning Using CodesNAVER Engineering
 

More Related Content

Similar to Incremental Control Dependency Frontier Exploration for Many-Criteria Test Case Generation

Optimizing the Catalyst Optimizer for Complex Plans
Optimizing the Catalyst Optimizer for Complex PlansOptimizing the Catalyst Optimizer for Complex Plans
Optimizing the Catalyst Optimizer for Complex PlansDatabricks
 
Sample Exam Questions on Python for revision
Sample Exam Questions on Python for revisionSample Exam Questions on Python for revision
Sample Exam Questions on Python for revisionafsheenfaiq2
 
Study on Application of Ensemble learning on Credit Scoring
Study on Application of Ensemble learning on Credit ScoringStudy on Application of Ensemble learning on Credit Scoring
Study on Application of Ensemble learning on Credit Scoringharmonylab
 
CS540-2-lecture11 - Copy.ppt
CS540-2-lecture11 - Copy.pptCS540-2-lecture11 - Copy.ppt
CS540-2-lecture11 - Copy.pptssuser0be977
 
Quality By Design
Quality By DesignQuality By Design
Quality By Designrealmayank
 
Iterative Multilevel Empirical Bayes (IMEB): An efficient flexible and robust...
Iterative Multilevel Empirical Bayes (IMEB): An efficient flexible and robust...Iterative Multilevel Empirical Bayes (IMEB): An efficient flexible and robust...
Iterative Multilevel Empirical Bayes (IMEB): An efficient flexible and robust...SKIM
 
Control Flow Graphs
Control Flow GraphsControl Flow Graphs
Control Flow Graphsdaimk2020
 
Control Flow Graphs
Control Flow GraphsControl Flow Graphs
Control Flow Graphsdaimk2020
 
A Performance Study of BDD-Based Model Checking
A Performance Study of BDD-Based Model CheckingA Performance Study of BDD-Based Model Checking
A Performance Study of BDD-Based Model CheckingOlivier Coudert
 
Kaggle boschコンペ振り返り
Kaggle boschコンペ振り返りKaggle boschコンペ振り返り
Kaggle boschコンペ振り返りKeisuke Hosaka
 
Code optimisation presnted
Code optimisation presntedCode optimisation presnted
Code optimisation presntedbhavanatmithun
 
Principal Sources of Optimization in compiler design
Principal Sources of Optimization in compiler design Principal Sources of Optimization in compiler design
Principal Sources of Optimization in compiler design LogsAk
 
Dat 305 dat305 dat 305 education for service uopstudy.com
Dat 305 dat305 dat 305 education for service   uopstudy.comDat 305 dat305 dat 305 education for service   uopstudy.com
Dat 305 dat305 dat 305 education for service uopstudy.comULLPTT
 
Fm G As
Fm G AsFm G As
Fm G Asysemet
 

Similar to Incremental Control Dependency Frontier Exploration for Many-Criteria Test Case Generation (20)

GCC
GCCGCC
GCC
 
Compiler Design Unit 5
Compiler Design Unit 5Compiler Design Unit 5
Compiler Design Unit 5
 
Optimizing the Catalyst Optimizer for Complex Plans
Optimizing the Catalyst Optimizer for Complex PlansOptimizing the Catalyst Optimizer for Complex Plans
Optimizing the Catalyst Optimizer for Complex Plans
 
Sample Exam Questions on Python for revision
Sample Exam Questions on Python for revisionSample Exam Questions on Python for revision
Sample Exam Questions on Python for revision
 
Study on Application of Ensemble learning on Credit Scoring
Study on Application of Ensemble learning on Credit ScoringStudy on Application of Ensemble learning on Credit Scoring
Study on Application of Ensemble learning on Credit Scoring
 
Code Optimization.ppt
Code Optimization.pptCode Optimization.ppt
Code Optimization.ppt
 
CS540-2-lecture11 - Copy.ppt
CS540-2-lecture11 - Copy.pptCS540-2-lecture11 - Copy.ppt
CS540-2-lecture11 - Copy.ppt
 
Quality By Design
Quality By DesignQuality By Design
Quality By Design
 
Iterative Multilevel Empirical Bayes (IMEB): An efficient flexible and robust...
Iterative Multilevel Empirical Bayes (IMEB): An efficient flexible and robust...Iterative Multilevel Empirical Bayes (IMEB): An efficient flexible and robust...
Iterative Multilevel Empirical Bayes (IMEB): An efficient flexible and robust...
 
Control Flow Graphs
Control Flow GraphsControl Flow Graphs
Control Flow Graphs
 
Control Flow Graphs
Control Flow GraphsControl Flow Graphs
Control Flow Graphs
 
A Performance Study of BDD-Based Model Checking
A Performance Study of BDD-Based Model CheckingA Performance Study of BDD-Based Model Checking
A Performance Study of BDD-Based Model Checking
 
Kaggle boschコンペ振り返り
Kaggle boschコンペ振り返りKaggle boschコンペ振り返り
Kaggle boschコンペ振り返り
 
Code optimisation presnted
Code optimisation presntedCode optimisation presnted
Code optimisation presnted
 
Lp (2)
Lp (2)Lp (2)
Lp (2)
 
Principal Sources of Optimization in compiler design
Principal Sources of Optimization in compiler design Principal Sources of Optimization in compiler design
Principal Sources of Optimization in compiler design
 
Code optimization
Code optimizationCode optimization
Code optimization
 
Code optimization
Code optimizationCode optimization
Code optimization
 
Dat 305 dat305 dat 305 education for service uopstudy.com
Dat 305 dat305 dat 305 education for service   uopstudy.comDat 305 dat305 dat 305 education for service   uopstudy.com
Dat 305 dat305 dat 305 education for service uopstudy.com
 
Fm G As
Fm G AsFm G As
Fm G As
 

More from Annibale Panichella

Searching for Quality: Genetic Algorithms and Metamorphic Testing for Softwar...
Searching for Quality: Genetic Algorithms and Metamorphic Testing for Softwar...Searching for Quality: Genetic Algorithms and Metamorphic Testing for Softwar...
Searching for Quality: Genetic Algorithms and Metamorphic Testing for Softwar...Annibale Panichella
 
A Fast Multi-objective Evolutionary Approach for Designing Large-Scale Optica...
A Fast Multi-objective Evolutionary Approach for Designing Large-Scale Optica...A Fast Multi-objective Evolutionary Approach for Designing Large-Scale Optica...
A Fast Multi-objective Evolutionary Approach for Designing Large-Scale Optica...Annibale Panichella
 
An Improved Pareto Front Modeling Algorithm for Large-scale Many-Objective Op...
An Improved Pareto Front Modeling Algorithm for Large-scale Many-Objective Op...An Improved Pareto Front Modeling Algorithm for Large-scale Many-Objective Op...
An Improved Pareto Front Modeling Algorithm for Large-scale Many-Objective Op...Annibale Panichella
 
An Adaptive Evolutionary Algorithm based on Non-Euclidean Geometry for Many-O...
An Adaptive Evolutionary Algorithm based on Non-Euclidean Geometry for Many-O...An Adaptive Evolutionary Algorithm based on Non-Euclidean Geometry for Many-O...
An Adaptive Evolutionary Algorithm based on Non-Euclidean Geometry for Many-O...Annibale Panichella
 
Speeding-up Software Testing With Computational Intelligence
Speeding-up Software Testing With Computational IntelligenceSpeeding-up Software Testing With Computational Intelligence
Speeding-up Software Testing With Computational IntelligenceAnnibale Panichella
 
Java Unit Testing Tool Competition — Fifth Round
Java Unit Testing Tool Competition — Fifth RoundJava Unit Testing Tool Competition — Fifth Round
Java Unit Testing Tool Competition — Fifth RoundAnnibale Panichella
 
Evolutionary Testing for Crash Reproduction
Evolutionary Testing for Crash ReproductionEvolutionary Testing for Crash Reproduction
Evolutionary Testing for Crash ReproductionAnnibale Panichella
 
Parameterizing and Assembling IR-based Solutions for SE Tasks using Genetic A...
Parameterizing and Assembling IR-based Solutions for SE Tasks using Genetic A...Parameterizing and Assembling IR-based Solutions for SE Tasks using Genetic A...
Parameterizing and Assembling IR-based Solutions for SE Tasks using Genetic A...Annibale Panichella
 
Security Threat Identification and Testing
Security Threat Identification and TestingSecurity Threat Identification and Testing
Security Threat Identification and TestingAnnibale Panichella
 
Adaptive User Feedback for IR-based Traceability Recovery
Adaptive User Feedback for IR-based Traceability RecoveryAdaptive User Feedback for IR-based Traceability Recovery
Adaptive User Feedback for IR-based Traceability RecoveryAnnibale Panichella
 
Diversity mechanisms for evolutionary populations in Search-Based Software En...
Diversity mechanisms for evolutionary populations in Search-Based Software En...Diversity mechanisms for evolutionary populations in Search-Based Software En...
Diversity mechanisms for evolutionary populations in Search-Based Software En...Annibale Panichella
 
Estimating the Evolution Direction of Populations to Improve Genetic Algorithms
Estimating the Evolution Direction of Populations to Improve Genetic AlgorithmsEstimating the Evolution Direction of Populations to Improve Genetic Algorithms
Estimating the Evolution Direction of Populations to Improve Genetic AlgorithmsAnnibale Panichella
 
When and How Using Structural Information to Improve IR-Based Traceability Re...
When and How Using Structural Information to Improve IR-Based Traceability Re...When and How Using Structural Information to Improve IR-Based Traceability Re...
When and How Using Structural Information to Improve IR-Based Traceability Re...Annibale Panichella
 
Multi-Objective Cross-Project Defect Prediction
Multi-Objective Cross-Project Defect PredictionMulti-Objective Cross-Project Defect Prediction
Multi-Objective Cross-Project Defect PredictionAnnibale Panichella
 

More from Annibale Panichella (18)

Searching for Quality: Genetic Algorithms and Metamorphic Testing for Softwar...
Searching for Quality: Genetic Algorithms and Metamorphic Testing for Softwar...Searching for Quality: Genetic Algorithms and Metamorphic Testing for Softwar...
Searching for Quality: Genetic Algorithms and Metamorphic Testing for Softwar...
 
A Fast Multi-objective Evolutionary Approach for Designing Large-Scale Optica...
A Fast Multi-objective Evolutionary Approach for Designing Large-Scale Optica...A Fast Multi-objective Evolutionary Approach for Designing Large-Scale Optica...
A Fast Multi-objective Evolutionary Approach for Designing Large-Scale Optica...
 
An Improved Pareto Front Modeling Algorithm for Large-scale Many-Objective Op...
An Improved Pareto Front Modeling Algorithm for Large-scale Many-Objective Op...An Improved Pareto Front Modeling Algorithm for Large-scale Many-Objective Op...
An Improved Pareto Front Modeling Algorithm for Large-scale Many-Objective Op...
 
VST2022.pdf
VST2022.pdfVST2022.pdf
VST2022.pdf
 
IPA Fall Days 2019
 IPA Fall Days 2019 IPA Fall Days 2019
IPA Fall Days 2019
 
An Adaptive Evolutionary Algorithm based on Non-Euclidean Geometry for Many-O...
An Adaptive Evolutionary Algorithm based on Non-Euclidean Geometry for Many-O...An Adaptive Evolutionary Algorithm based on Non-Euclidean Geometry for Many-O...
An Adaptive Evolutionary Algorithm based on Non-Euclidean Geometry for Many-O...
 
Speeding-up Software Testing With Computational Intelligence
Speeding-up Software Testing With Computational IntelligenceSpeeding-up Software Testing With Computational Intelligence
Speeding-up Software Testing With Computational Intelligence
 
Sbst2018 contest2018
Sbst2018 contest2018Sbst2018 contest2018
Sbst2018 contest2018
 
Java Unit Testing Tool Competition — Fifth Round
Java Unit Testing Tool Competition — Fifth RoundJava Unit Testing Tool Competition — Fifth Round
Java Unit Testing Tool Competition — Fifth Round
 
ICSE 2017 - Evocrash
ICSE 2017 - EvocrashICSE 2017 - Evocrash
ICSE 2017 - Evocrash
 
Evolutionary Testing for Crash Reproduction
Evolutionary Testing for Crash ReproductionEvolutionary Testing for Crash Reproduction
Evolutionary Testing for Crash Reproduction
 
Parameterizing and Assembling IR-based Solutions for SE Tasks using Genetic A...
Parameterizing and Assembling IR-based Solutions for SE Tasks using Genetic A...Parameterizing and Assembling IR-based Solutions for SE Tasks using Genetic A...
Parameterizing and Assembling IR-based Solutions for SE Tasks using Genetic A...
 
Security Threat Identification and Testing
Security Threat Identification and TestingSecurity Threat Identification and Testing
Security Threat Identification and Testing
 
Adaptive User Feedback for IR-based Traceability Recovery
Adaptive User Feedback for IR-based Traceability RecoveryAdaptive User Feedback for IR-based Traceability Recovery
Adaptive User Feedback for IR-based Traceability Recovery
 
Diversity mechanisms for evolutionary populations in Search-Based Software En...
Diversity mechanisms for evolutionary populations in Search-Based Software En...Diversity mechanisms for evolutionary populations in Search-Based Software En...
Diversity mechanisms for evolutionary populations in Search-Based Software En...
 
Estimating the Evolution Direction of Populations to Improve Genetic Algorithms
Estimating the Evolution Direction of Populations to Improve Genetic AlgorithmsEstimating the Evolution Direction of Populations to Improve Genetic Algorithms
Estimating the Evolution Direction of Populations to Improve Genetic Algorithms
 
When and How Using Structural Information to Improve IR-Based Traceability Re...
When and How Using Structural Information to Improve IR-Based Traceability Re...When and How Using Structural Information to Improve IR-Based Traceability Re...
When and How Using Structural Information to Improve IR-Based Traceability Re...
 
Multi-Objective Cross-Project Defect Prediction
Multi-Objective Cross-Project Defect PredictionMulti-Objective Cross-Project Defect Prediction
Multi-Objective Cross-Project Defect Prediction
 

Recently uploaded

Monthly HSE Report March for overall HSE
Monthly HSE Report March for overall HSEMonthly HSE Report March for overall HSE
Monthly HSE Report March for overall HSEOlgaOliveaJohn
 
Relationships in Teaching By Dr. Cherinet Aytenfsu Weldearegay (2023).pdf
Relationships in Teaching By Dr. Cherinet Aytenfsu Weldearegay (2023).pdfRelationships in Teaching By Dr. Cherinet Aytenfsu Weldearegay (2023).pdf
Relationships in Teaching By Dr. Cherinet Aytenfsu Weldearegay (2023).pdfDr. Cherinet Aytenfsu Weldearegay
 
Freeman_Abigail Personal Brand Exploration
Freeman_Abigail Personal Brand ExplorationFreeman_Abigail Personal Brand Exploration
Freeman_Abigail Personal Brand Explorationabbytoliver
 
Issues affecting LGBT as they grow older.pptx
Issues affecting LGBT as they grow older.pptxIssues affecting LGBT as they grow older.pptx
Issues affecting LGBT as they grow older.pptxbill846304
 
Vazquez_Gabriel_SPCASTNG_PB1_2024-02.pdf
Vazquez_Gabriel_SPCASTNG_PB1_2024-02.pdfVazquez_Gabriel_SPCASTNG_PB1_2024-02.pdf
Vazquez_Gabriel_SPCASTNG_PB1_2024-02.pdfgabev3104
 
Exploring Wisdom-Based Leadership.pptx
Exploring Wisdom-Based Leadership.pptxExploring Wisdom-Based Leadership.pptx
Exploring Wisdom-Based Leadership.pptxAkash Das
 
Reflection Vs Technical Rationality - By Dr. Cherinet Aytenfsu Weldearegay (2...
Reflection Vs Technical Rationality - By Dr. Cherinet Aytenfsu Weldearegay (2...Reflection Vs Technical Rationality - By Dr. Cherinet Aytenfsu Weldearegay (2...
Reflection Vs Technical Rationality - By Dr. Cherinet Aytenfsu Weldearegay (2...Dr. Cherinet Aytenfsu Weldearegay
 
TheSimpsons_Fandom_Assignment_4.5pc.pptx
TheSimpsons_Fandom_Assignment_4.5pc.pptxTheSimpsons_Fandom_Assignment_4.5pc.pptx
TheSimpsons_Fandom_Assignment_4.5pc.pptxStevenLuker3
 
Garcia_RobertDaniel_SPCSTA_PB1_2024-02.pptx
Garcia_RobertDaniel_SPCSTA_PB1_2024-02.pptxGarcia_RobertDaniel_SPCSTA_PB1_2024-02.pptx
Garcia_RobertDaniel_SPCSTA_PB1_2024-02.pptx0461620
 

Recently uploaded (10)

Monthly HSE Report March for overall HSE
Monthly HSE Report March for overall HSEMonthly HSE Report March for overall HSE
Monthly HSE Report March for overall HSE
 
Relationships in Teaching By Dr. Cherinet Aytenfsu Weldearegay (2023).pdf
Relationships in Teaching By Dr. Cherinet Aytenfsu Weldearegay (2023).pdfRelationships in Teaching By Dr. Cherinet Aytenfsu Weldearegay (2023).pdf
Relationships in Teaching By Dr. Cherinet Aytenfsu Weldearegay (2023).pdf
 
Freeman_Abigail Personal Brand Exploration
Freeman_Abigail Personal Brand ExplorationFreeman_Abigail Personal Brand Exploration
Freeman_Abigail Personal Brand Exploration
 
Tethex Cards - Crypto VISA Card - Invest 2 ENGLISH
Tethex Cards - Crypto VISA Card - Invest 2 ENGLISHTethex Cards - Crypto VISA Card - Invest 2 ENGLISH
Tethex Cards - Crypto VISA Card - Invest 2 ENGLISH
 
Issues affecting LGBT as they grow older.pptx
Issues affecting LGBT as they grow older.pptxIssues affecting LGBT as they grow older.pptx
Issues affecting LGBT as they grow older.pptx
 
Vazquez_Gabriel_SPCASTNG_PB1_2024-02.pdf
Vazquez_Gabriel_SPCASTNG_PB1_2024-02.pdfVazquez_Gabriel_SPCASTNG_PB1_2024-02.pdf
Vazquez_Gabriel_SPCASTNG_PB1_2024-02.pdf
 
Exploring Wisdom-Based Leadership.pptx
Exploring Wisdom-Based Leadership.pptxExploring Wisdom-Based Leadership.pptx
Exploring Wisdom-Based Leadership.pptx
 
Reflection Vs Technical Rationality - By Dr. Cherinet Aytenfsu Weldearegay (2...
Reflection Vs Technical Rationality - By Dr. Cherinet Aytenfsu Weldearegay (2...Reflection Vs Technical Rationality - By Dr. Cherinet Aytenfsu Weldearegay (2...
Reflection Vs Technical Rationality - By Dr. Cherinet Aytenfsu Weldearegay (2...
 
TheSimpsons_Fandom_Assignment_4.5pc.pptx
TheSimpsons_Fandom_Assignment_4.5pc.pptxTheSimpsons_Fandom_Assignment_4.5pc.pptx
TheSimpsons_Fandom_Assignment_4.5pc.pptx
 
Garcia_RobertDaniel_SPCSTA_PB1_2024-02.pptx
Garcia_RobertDaniel_SPCSTA_PB1_2024-02.pptxGarcia_RobertDaniel_SPCSTA_PB1_2024-02.pptx
Garcia_RobertDaniel_SPCSTA_PB1_2024-02.pptx
 

Incremental Control Dependency Frontier Exploration for Many-Criteria Test Case Generation

  • 1. Incremental Control Dependency Frontier Exploration for Many-Criteria Test Case Generation Annibale Panichella, Fitsum M. Kifetew, Paolo Tonella September 11, 2017 !1
  • 2. Background Rojas et al. SSBSE 2015 "Optimising for several criteria at the same time is feasible without increasing computational costs" !2
  • 3. Classical Approach (WSA) 1 2 3 4 5 8 Class Under Test 7 9 6 Suite-level Fitness Function 3
  • 4. Classical Approach (WSA) 1 2 3 4 5 8 7 9 6 Suite-level Fitness Function Branch Coverage b2b1 b3 b4 b5 b6 b7 b8 b9 b10 b11 b12 F = f(b1) + f(b2) + … +f (b12) Class Under Test 4
  • 5. Classical Approach (WSA) Suite-level Fitness Function Statement Coverage F = f(b1) + f(b2) + … +f (b12) + f(s1) + f(s2) + … +f (s9) 1 2 3 5 4 8 7 9 6 Class Under Test 5
  • 6. Classical Approach (WSA) Suite-level Fitness Function Weak Mutation Coverage F = f(b1) + f(b2) + … +f (b12) + f(s1) + f(s2) + … +f (s9) f(m1) + f(m2) + … +f (m3) + 1 2 m1 m2 m3 8 7 9 6 Class Under Test 5
  • 8. Enhanced Control Flow Graph 6 Annibale Panichella1 , Fitsum Meshesha Kifetew2 , and Paolo Tonella3 (a) Example program (b) Single criterion (c) Multiple criteria Fig. 1. Code (left), CDG (middle), and ECDG (right) of an example program Algorithm 1: MC-DynaMOSA Input: B = {⌧1, . . . , ⌧m} the set of coverage targets of a program. CDG = hN, E, si: control dependency graph of the program Result: A test suite T L1 L2 L5 L3 b1 b4 b2 b3 L6 L7 L8 b5 b6 b0 Control Flow Graph Branches + Statements 7
  • 9. Enhanced Control Flow Graph 6 Annibale Panichella1 , Fitsum Meshesha Kifetew2 , and Paolo Tonella3 (a) Example program (b) Single criterion (c) Multiple criteria Fig. 1. Code (left), CDG (middle), and ECDG (right) of an example program Algorithm 1: MC-DynaMOSA Input: B = {⌧1, . . . , ⌧m} the set of coverage targets of a program. CDG = hN, E, si: control dependency graph of the program Result: A test suite T L1 L2 L5 b1 b4 b2 L7 L8 b5 b6 b0 Enhanced CFG L3, μ1 L6, μ2 Branches + Statements +
 + Weak Mutation b3 8
  • 10. Enhanced Control Flow Graph 6 Annibale Panichella1 , Fitsum Meshesha Kifetew2 , and Paolo Tonella3 (a) Example program (b) Single criterion (c) Multiple criteria Fig. 1. Code (left), CDG (middle), and ECDG (right) of an example program Algorithm 1: MC-DynaMOSA Input: B = {⌧1, . . . , ⌧m} the set of coverage targets of a program. CDG = hN, E, si: control dependency graph of the program Result: A test suite T L1 L2 L5 b1 b4 b2 L7 b5 b6 b0 Enhanced CFG L3, μ1 L6, μ2// o2: x<0, o3: x>0,… L8, o1, o2 Branches + Statements
 + Weak Mutation + Output Diversity b3 9
  • 11. Dynamic Many-Objective Sorting Algorithm for Multi-Criteria Coverage L1 L2 L5 b1 b4 b2 L7 b5 b6 b0 L3, μ1 L6, μ2 L8, o1, o2 b3 Status Current Objectives Covered Targets Init. {b0} 10
  • 12. Dynamic Many-Objective Sorting Algorithm for Multi-Criteria Coverage L1 L2 L5 b1 b4 b2 L7 b5 b6 b0 L3, μ1 L6, μ2 L8, o1, o2 b3 Status Current Objectives Covered Targets Init. {b0} b0 covered {b1, b5} {b0, L1} 11
  • 13. Dynamic Many-Objective Sorting Algorithm for Multi-Criteria Coverage L1 L2 L5 b1 b4 b2 L7 b5 b6 b0 L3, μ1 L6, μ2 L8, o1, o2 b3 11 f(b1) f(b5)
  • 14. Dynamic Many-Objective Sorting Algorithm for Multi-Criteria Coverage L1 L2 L5 b1 b4 b2 L7 b5 b6 b0 L3, μ1 L6, μ2 L8, o1, o2 b3 11 f(b1) f(b5) T1 T2 T3 T4
  • 15. Dynamic Many-Objective Sorting Algorithm for Multi-Criteria Coverage L1 L2 L5 b1 b4 b2 L7 b5 b6 b0 L3, μ1 L6, μ2 L8, o1, o2 b3 Status Current Objectives Covered Targets Init. {b0} b0 covered {b1, b5} {b0, L1} b1 covered {b2, b3, b5} {b0, L1, b1, L2} 12
  • 16. Dynamic Many-Objective Sorting Algorithm for Multi-Criteria Coverage L1 L2 L5 b1 b4 b2 L7 b5 b6 b0 L3, μ1 L6, μ2 L8, o1, o2 b3 Status Current Objectives Covered Targets Init. {b0} b0 covered {b1, b5} {b0, L1} b1 covered {b2, b3, b5} {b0, L1, b1, L2} b5 covered {b2, b3, µ2, b4, b6} {b0, L1, b1, L2, b5, L6} 13
  • 17. Dynamic Many-Objective Sorting Algorithm for Multi-Criteria Coverage L1 L2 L5 b1 b4 b2 L7 b5 b6 b0 L3, μ1 L6, μ2 L8, o1, o2 b3 Status Current Objectives Covered Targets Init. {b0} b0 covered {b1, b5} {b0, L1} b1 covered {b2, b3, b5} {b0, L1, b1, L2} b5 covered {b2, b3, µ2, b4, b6} {b0, L1, b1, L2, b5, L6} b2 covered {b3, b4, b6} {b0, L1, b1, L2, b5, L6, b2, L3, µ1, µ2} b3 covered {b4, b6} {b0, L1, b1, L2, b5, L6, b2, L3, µ1, µ2, b3, L5} 14
  • 19. Study Context Benchmark: 180 non-trivial classes sampled from SF110 Selection Procedure: • Computing the McCabe’s cyclomatic complexity (MCC) • Filtering out all trivial classes (having only methods with MCC <5) • Random sampling from the pruned projects Search Budget: Three minutes 16
  • 20. RQ1: DynaMOSA vs. WSAFrequency 0 35 70 105 140 Coverage Criterion Branch Line WeakMutation Output Equivalent Better (p-value<0.05) Worse (p-value<0.05) 17
  • 21. DynaMOSA is better than WSA WSA is better than DynaMOSA RQ1: DynaMOSA vs. WSA %DifferenceinBranchCoverage -50 -38 -25 -13 0 13 25 38 50 Classes Under Test 18
  • 22. RQ1: DynaMOSA vs. WSA %DifferenceinBranchCoverage -10 1 13 24 35 Classes Under Test Larger difference CUT = TableMeta Project = schemaspy N.Branches = 68 % Cov. Diff = +31.71% Larger Class CUT = JVCParserTokenMan. Project = JVC N.Branches = 4252 % Cov. Diff = +13.09% 19
  • 23. RQ2: DynaMOSA vs. MOSAFrequency 0 35 70 105 140 Coverage Criterion Branch Line WeakMutation Output Equivalent Better (p-value<0.05) Worse (p-value<0.05) 20
  • 24. RQ2: DynaMOSA vs. MOSA %DifferenceinBranchCoverage -10 -1 8 16 25 Classes Under Test Larger difference CUT = CoordSystemUtilities Project = schemaspy N.Branches = 882 % Cov. Diff = +23.68% Larger Class CUT = JVCParserTokenMan. Project = JVC N.Branches = 4252 % Cov. Diff = +13.79% 21
  • 25. RQ3: DynaMOSA Multi-Criteria vs. Branch OnlyFrequency 0 30 60 90 120 Coverage Criterion Branch Line WeakMutation Output Equivalent Better (p-value<0.05) Worse (p-value<0.05) 22
  • 26. RQ4: What Are the Benefits in Terms of Fault Detection Capability?Frequency 0 25 50 75 100 DynaMOSA vs WSA DynaMOSA vs MOSA Multi-Criteria vs Branch Only Equivalent Better (p-value<0.05) Worse (p-value<0.05) 23
  • 27. RQ4: What Are the Benefits in Terms of Fault Detection Capability? DynaMOSA vs MOSA DynaMOSA vs WSA Multi-Criteria vs Branch Only DifferenceinMutationScore 60% 40% 20% 0% -20% 24
  • 28. What EAs to Choose? Even Larger
  • 29. Incremental Control Dependency Frontier Exploration for Many-Criteria Test Case Generation Annibale Panichella, Fitsum M. Kifetew, Paolo Tonella September 11, 2017