Constraint-based Fitness Function for Search-based Software Testing

1. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
Constraint-based Fitness Function
for Search-based Software Testing
Abdelilah Sakti Yann-Ga¨el Gu´eh´eneuc Gilles Pesant
Department of Computer and Software Engineering
´Ecole Polytechnique de Montr´eal, Qu´ebec, Canada
May 21, 2013
CPAIOR, Yorktown Heights
Pattern Trace Identification, Detection, and Enhancement in Java
SOftware Cost-effective Change and Evolution Research Lab

2. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
What is the cost of a
programming error ?
Software testing
Search Based Software
Testing (SBST)
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
What is the cost of a programming error ?
2 / 14

3. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
What is the cost of a
programming error ?
Software testing
Search Based Software
Testing (SBST)
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
What is the cost of a programming error ?
Every year inadequate infrastructure software costs the U.S.
economy around 60 billion $.
2 / 14

4. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
What is the cost of a
programming error ?
Software testing
Search Based Software
Testing (SBST)
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
What is the cost of a programming error ?
Every year inadequate infrastructure software costs the U.S.
economy around 60 billion $.
Between 1985 and 1987 : the radiotherapy
machine Therac-25 sent to patients a X-ray dose
100 times greater than expected.
At least five deaths ;
Several other patients were severely affected
by radiation.
2 / 14

5. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
What is the cost of a
programming error ?
Software testing
Search Based Software
Testing (SBST)
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
What is the cost of a programming error ?
Every year inadequate infrastructure software costs the U.S.
economy around 60 billion $.
Between 1985 and 1987 : the radiotherapy
machine Therac-25 sent to patients a X-ray dose
100 times greater than expected.
At least five deaths ;
Several other patients were severely affected
by radiation.
Only a high quality software can reduce errors cost.
Testing is an important technique for validating and
checking the correctness of software.
Despite that the testing may cost more than 50% of the
budget of critical software, it remains an inevitable
phase in the life cycle of software.
2 / 14

6. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
What is the cost of a
programming error ?
Software testing
Search Based Software
Testing (SBST)
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
Software testing
Formal Methods
Is a way of modelling and verifying a software by using
mathematics or logic techniques ;
Aims to prove and argue that a software respect some
properties (e.g., an undesired behaviour will never
occur).
Some complex programmes are difficult or impossible to
model.
3 / 14

7. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
What is the cost of a
programming error ?
Software testing
Search Based Software
Testing (SBST)
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
Software testing
Formal Methods
Is a way of modelling and verifying a software by using
mathematics or logic techniques ;
Aims to prove and argue that a software respect some
properties (e.g., an undesired behaviour will never
occur).
Some complex programmes are difficult or impossible to
model.
Informal or Pragmatic Techniques
Performed by executing a real implementation on a
small subset of all possible inputs data that satisfies
some criteria (e.g., all statements, all branches...) ;
Aims to show the presence of errors.
3 / 14

8. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
What is the cost of a
programming error ?
Software testing
Search Based Software
Testing (SBST)
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
Software testing
Formal Methods
Is a way of modelling and verifying a software by using
mathematics or logic techniques ;
Aims to prove and argue that a software respect some
properties (e.g., an undesired behaviour will never
occur).
Some complex programmes are difficult or impossible to
model.
Informal or Pragmatic Techniques
Performed by executing a real implementation on a
small subset of all possible inputs data that satisfies
some criteria (e.g., all statements, all branches...) ;
Aims to show the presence of errors.
One main challenge is automating inputs data generation.
3 / 14

9. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
What is the cost of a
programming error ?
Software testing
Search Based Software
Testing (SBST)
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
Search Based Software Testing (SBST)
4 / 14

10. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
What is the cost of a
programming error ?
Software testing
Search Based Software
Testing (SBST)
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
Search Based Software Testing (SBST)
Fitness Evaluation
The Fitness Evaluation is the core of SBST.
4 / 14

11. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
What is the cost of a
programming error ?
Software testing
Search Based Software
Testing (SBST)
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
Search Based Software Testing (SBST)
Fitness Evaluation
The Fitness Evaluation is the core of SBST.
An evolution in the fitness evaluation may bring a significant
enhancement to SBST
4 / 14

12. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
Motivating Example
1 int sample(int x,int y,int z){
2 if(y==z)
3 if(y>0)
4 if(x==10)
5 ...//Target
6 }
5 / 14

13. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
Motivating Example
1 int sample(int x,int y,int z){
2 if(y==z)
3 if(y>0)
4 if(x==10)
5 ...//Target
6 }
tinyx, y, z
tinyy == z
tinyEND
tinyy > 0
tinyx == 10
tinyTarget
5 / 14

14. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
Motivating Example
1 int sample(int x,int y,int z){
2 if(y==z)
3 if(y>0)
4 if(x==10)
5 ...//Target
6 }
tinyx, y, z
tinyy == z
tinyEND
tinyy > 0
tinyx == 10
tinyTarget
i1 : (10,−30,60)
i2 : (30, −20, −20)
5 / 14

15. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
Motivating Example
1 int sample(int x,int y,int z){
2 if(y==z)
3 if(y>0)
4 if(x==10)
5 ...//Target
6 }
tinyx, y, z
tinyy == z
tinyEND
tinyy > 0
tinyx == 10
tinyTarget
i1 : (10,−30,60)
i1
Critical Branch of i1
i2 : (30, −20, −20)
5 / 14

16. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
Motivating Example
1 int sample(int x,int y,int z){
2 if(y==z)
3 if(y>0)
4 if(x==10)
5 ...//Target
6 }
tinyy == z
tinyx, y, z
tinyy == z
tinyEND
tinyy > 0
tinyx == 10
tinyTarget
i1 : (10,−30,60)
i1
Critical Branch of i1
i2 : (30, −20, −20)
Test candidate fAL fSE
i1 3
i2
5 / 14

17. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
Motivating Example
1 int sample(int x,int y,int z){
2 if(y==z)
3 if(y>0)
4 if(x==10)
5 ...//Target
6 }
tinyy == z
tinyx, y, z
tinyy == z
tinyEND
tinyy > 0
tinyx == 10
tinyTarget
i1 : (10,−30,60)
i1
Critical Branch of i1
i2 : (30, −20, −20)
Test candidate fAL fSE
i1 3+90
91
i2
5 / 14

18. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
Motivating Example
1 int sample(int x,int y,int z){
2 if(y==z)
3 if(y>0)
4 if(x==10)
5 ...//Target
6 }
tinyy == z
tinyx, y, z
tinyy == z
tinyEND
tinyy > 0
tinyx == 10
tinyTarget
i1 : (10,−30,60)
i1
Critical Branch of i1
i2 : (30, −20, −20)
Test candidate fAL fSE
i1 3+90
91
90
91
i2
5 / 14

19. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
Motivating Example
1 int sample(int x,int y,int z){
2 if(y==z)
3 if(y>0)
4 if(x==10)
5 ...//Target
6 }
tinyy > 0
tinyx, y, z
tinyy == z
tinyEND
tinyy > 0
tinyx == 10
tinyTarget
i1 : (10,−30,60)
i1
Critical Branch of i1
i2 : (30, −20, −20)
Test candidate fAL fSE
i1 3+90
91
90
91 +31
32
i2
5 / 14

20. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
Motivating Example
1 int sample(int x,int y,int z){
2 if(y==z)
3 if(y>0)
4 if(x==10)
5 ...//Target
6 }
tinyx == 10
tinyx, y, z
tinyy == z
tinyEND
tinyy > 0
tinyx == 10
tinyTarget
i1 : (10,−30,60)
i1
Critical Branch of i1
i2 : (30, −20, −20)
Test candidate fAL fSE
i1 3+90
91
90
91 +31
32 +0
i2
5 / 14

21. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
Motivating Example
1 int sample(int x,int y,int z){
2 if(y==z)
3 if(y>0)
4 if(x==10)
5 ...//Target
6 }
tinyx, y, z
tinyy == z
tinyEND
tinyy > 0
tinyx == 10
tinyTarget
i1 : (10,−30,60)
i1
Critical Branch of i1
i2 : (30, −20, −20)
i2
Test candidate fAL fSE
i1 3+90
91 = 3.9890 90
91 +31
32 +0= 1.9577
i2 2 + 21
22 =2.9545 0 + 21
22 + 20
21 =1.9068
5 / 14

22. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
Motivating Example
1 int sample(int x,int y,int z){
2 if(y==z)
3 if(y>0)
4 if(x==10)
5 ...//Target
6 }
tinyx, y, z
tinyy == z
tinyEND
tinyy > 0
tinyx == 10
tinyTarget
i1 : (10,−30,60)
i1
Critical Branch of i1
i2 : (30, −20, −20)
i2
Test candidate fAL fSE
i1 3+90
91 = 3.9890 90
91 +31
32 +0= 1.9577
i2 2 + 21
22 =2.9545 0 + 21
22 + 20
21 =1.9068
Both fitness functions favour i2 over i1 because either they
do not consider non-executed branches or they ignore some
branches’ future.5 / 14

23. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Overview
Branch-Hardness Metrics
Branch-Hardness Fitness
Functions
Applying fDC on our
motivating example
Evaluation
Conclusions
Fitness Function Based on Branch Hardness
Combines SBST with static analysis of the
non-executed branches.
Prioritizes branches according to how hard it is to
satisfy them.
6 / 14

24. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Overview
Branch-Hardness Metrics
Branch-Hardness Fitness
Functions
Applying fDC on our
motivating example
Evaluation
Conclusions
Fitness Function Based on Branch Hardness
Combines SBST with static analysis of the
non-executed branches.
Prioritizes branches according to how hard it is to
satisfy them.
Defines the difficulty to satisfy a constraint in terms of
its arity and its projection tightness (the ratio of the
approximate number of solutions to the size of its
search space).
6 / 14

25. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Overview
Branch-Hardness Metrics
Branch-Hardness Fitness
Functions
Applying fDC on our
motivating example
Evaluation
Conclusions
Fitness Function Based on Branch Hardness
Combines SBST with static analysis of the
non-executed branches.
Prioritizes branches according to how hard it is to
satisfy them.
Defines the difficulty to satisfy a constraint in terms of
its arity and its projection tightness (the ratio of the
approximate number of solutions to the size of its
search space).
1. The lower the arity of the constraint, the less freedom
we have to choose some of its variables in order to
evolve the test candidate.
6 / 14

26. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Overview
Branch-Hardness Metrics
Branch-Hardness Fitness
Functions
Applying fDC on our
motivating example
Evaluation
Conclusions
Fitness Function Based on Branch Hardness
Combines SBST with static analysis of the
non-executed branches.
Prioritizes branches according to how hard it is to
satisfy them.
Defines the difficulty to satisfy a constraint in terms of
its arity and its projection tightness (the ratio of the
approximate number of solutions to the size of its
search space).
1. The lower the arity of the constraint, the less freedom
we have to choose some of its variables in order to
evolve the test candidate.
2. A projection tightness close to 0 will indicate high
constrainedness and hardness to satisfy a constraint.
6 / 14

27. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Overview
Branch-Hardness Metrics
Branch-Hardness Fitness
Functions
Applying fDC on our
motivating example
Evaluation
Conclusions
Branch-Hardness Metrics
Difficulty Coefficient (DC)
DC is a possible representation of the hardness of a
branch ;
Each constraint has its own DC that is determined
according to its arity and tightness ;
DC(c) = B2 · 1
arityc
+ B · (1 − tightness) + 1 .
7 / 14

28. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Overview
Branch-Hardness Metrics
Branch-Hardness Fitness
Functions
Applying fDC on our
motivating example
Evaluation
Conclusions
Branch-Hardness Metrics
Difficulty Coefficient (DC)
DC is a possible representation of the hardness of a
branch ;
Each constraint has its own DC that is determined
according to its arity and tightness ;
DC(c) = B2 · 1
arityc
+ B · (1 − tightness) + 1 .
Difficulty Level (DL)
DL is based on DC ranking (r) ;
DL is a representation of a relative hardness level of a
constraint in a set of constraints ;
DL(c, C) =
|C|, if r = 0
2r−1 · (|C| + 1), if r > 0
.
7 / 14

29. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Overview
Branch-Hardness Metrics
Branch-Hardness Fitness
Functions
Applying fDC on our
motivating example
Evaluation
Conclusions
Branch-Hardness Fitness Functions
DC Fitness Function (fDC)
DC is used as a penalty coefficient for breaking a
constraint ;
The target of this fitness function is determining a
standard-branch-distance.
fDC(i, C) = c∈C DC(c) · η(i, c) .
8 / 14

30. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Overview
Branch-Hardness Metrics
Branch-Hardness Fitness
Functions
Applying fDC on our
motivating example
Evaluation
Conclusions
Branch-Hardness Fitness Functions
DC Fitness Function (fDC)
DC is used as a penalty coefficient for breaking a
constraint ;
The target of this fitness function is determining a
standard-branch-distance.
fDC(i, C) = c∈C DC(c) · η(i, c) .
DL Fitness Function (fDL)
DL is used as a constant penalty for breaking a
constraint in a set of constraints to satisfy ;
fDL(i, C) = c∈C (i, c) + η(i, c),
where (i, c) =
0, if η(i, c) = 0
DL(c, C), if η(i, c) = 0
.
8 / 14

31. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Overview
Branch-Hardness Metrics
Branch-Hardness Fitness
Functions
Applying fDC on our
motivating example
Evaluation
Conclusions
Applying fDC on our motivating example
Assume that all domains are equal to [−99, 100].
DC for each branch
1. DC(”y == z”) = 102 · 0.5 + 10 · 0.995 + 1 = 60.95 ;
2. DC(”y > 0”) = 102 · 1 + 10 · 0.5 + 1 = 106 ;
3. DC(”x == 10”) = 102 · 1 + 10 · 0.995 + 1 = 110.95.
9 / 14

32. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Overview
Branch-Hardness Metrics
Branch-Hardness Fitness
Functions
Applying fDC on our
motivating example
Evaluation
Conclusions
Applying fDC on our motivating example
Assume that all domains are equal to [−99, 100].
DC for each branch
1. DC(”y == z”) = 102 · 0.5 + 10 · 0.995 + 1 = 60.95 ;
2. DC(”y > 0”) = 102 · 1 + 10 · 0.5 + 1 = 106 ;
3. DC(”x == 10”) = 102 · 1 + 10 · 0.995 + 1 = 110.95.
(fDC) for each test candidate
fDC(i1, C) = 60.95· 90
91 +106· 31
32 +110.95· 0
1 = 162.9677 ;
fDC(i2, C) = 60.95· 0
1 +106· 21
22 +110.95· 20
21 = 206.8485.
Contrary to fAL and fSE this fitness function makes the
adequate choice by choosing the test candidate i1 instead of
i2.
9 / 14

33. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Subjects and configuration
EA Results
SA Results
Conclusions
Evaluation
Subjects and configuration
analysed meta-heuristics and fitness functions
Two widely used meta-heuristic algorithms are
analysed : Simulated Annealing (SA) and Evolutionary
Algorithm (EA).
fAL and fSE from the literature ;
fSEL a natural combination of fAL and fSE ;
Our fitness functions fDC and fDL.
10 / 14

34. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Subjects and configuration
EA Results
SA Results
Conclusions
Evaluation
Subjects and configuration
analysed meta-heuristics and fitness functions
Two widely used meta-heuristic algorithms are
analysed : Simulated Annealing (SA) and Evolutionary
Algorithm (EA).
fAL and fSE from the literature ;
fSEL a natural combination of fAL and fSE ;
Our fitness functions fDC and fDL.
Subjects
440 synthetic test targets that were randomly generated.
20 executions for every combination of fitness function
and meta-heuristic algorithm.
If test data was not found after 25000 (respectively
100000) fitness evaluations for EA (respectively SA),
the search was terminated.
10 / 14

35. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Subjects and configuration
EA Results
SA Results
Conclusions
Evaluation
EA Results
0 5000 10000 15000 20000 25000
0
20
40
60
80
EA: Comparing all fitnesses on 440 test targets
Evaluations
Approach Level
Symbolic Enhanced
Symbolic Enhanced with levels
Difficulty Coefficient
Difficulty Level
11 / 14

36. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Subjects and configuration
EA Results
SA Results
Conclusions
Evaluation
SA Results
0e+00 2e+04 4e+04 6e+04 8e+04 1e+05
0
20
40
60
80
SA: Comparing all fitnesses on 440 test targets
Evaluations
BranchCoverage% Approach Level
Symbolic Enhanced
Symbolic Enhanced with levels
Difficulty Coefficient
Difficulty Level
12 / 14

37. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
Conclusions
Two new metrics to measure the difficulty to satisfy a
constraint in the context of test case generation for
software testing are defined ;
Two new fitness functions for SBST are defined ;
Our new fitness functions are significantly more effective
and efficient than with the largely used fitness functions
from the literature.
The obtained results are promising but more
experiments must be performed.
13 / 14

38. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
Conclusions
Two new metrics to measure the difficulty to satisfy a
constraint in the context of test case generation for
software testing are defined ;
Two new fitness functions for SBST are defined ;
Our new fitness functions are significantly more effective
and efficient than with the largely used fitness functions
from the literature.
The obtained results are promising but more
experiments must be performed.
Future work
Performing more experiments on real world programs ;
Extending our approach by defining new ways to
compute projection tightness for constraints involving
other data types than integer.
13 / 14

39. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨el
Gu´eh´eneuc, Gilles
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
Thank you
Questions ?
14 / 14