Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...
CEC'10: Improving Evolutionary Testing by Means of Efficiency Enhancement Techniques
1. Improving Evolutionary Testing by Means of
Efficiency Enhancement Techniques
Matteo Miraz, Pier Luca Lanzi, and Luciano Baresi
(presented by Daniele Loiacono)
Politecnico di Milano, Italy
2. 2
Table of Contents
1. TestFul: an Evolutionary Test Approach for Java
2. Improving the Approach
1. Seeding
2. Hybridization
3. Fitness Inheritance
3. Conclusions and Future Work
DEI
3. Testful 3
• Leverages Search-Based Techniques to generate Tests
• We are not alone:
• Search Based Software Engineering is a vibrant community!
• Targets stateful systems (e.g., Java classes)
• A sequence of operations
is required to exercise
the features of a class
enables
DEI
4. Test Representation 4
• A test is a sequence of operations
• Assign a primitive value to a variable
• char_0 = „c‟
• Create an object
• cut_1 = new CUT(char_0)
• Invoke a method on an object
• char_1 = cut_1.fooBar(int_1)
• Working on variables
• Includes all the parameters‟ types
• Enables polymorphism
• Calculate compatibilities among types
DEI
5. Our Evolutionary Testing Approach 5
Goal: Find tests that put objects in interesting states
• We use a NSGA-II Evolutionary Algorithm, guided by:
Number of statements and branches covered
• Tests with a high branch coverage stress more deeply the class
under test and put objects in more interesting states
Compactness of tests
• Unnecessary long tests waste computational resources during their
evaluation
• Recombination:
• variable-length one-point crossover, replacement mutation
crossover mutation
DEI
6. Need for Efficiency Enhancement Techniques 6
• Testful achieves good results
• generates good tests for complex classes (GECCO ‟09 poster)
• It requires huge computational requirements, because:
• It starts the evolution from a population with a poor quality.
• Most of the initial tests do not exercise any feature
• The most difficult elements in the class are not exercised
• It does not provide any guidance to exercise uncovered elements
• The fitness function only counts the number of branch exercised
• The fitness function is expensive
• It requires to execute a test to measure its branch coverage
DEI
7. Seeding 7
• Problem:
• Testful starts the evolution from a population with a poor quality.
• Solution:
• Run an inexpensive technique (random search) for a short time,
and use its result as initial population
Branch Coverage with a Random Search
DEI
8. Seeding 8
Simple Problem
(Fraction)
Complex Problem
(Red-Black Tree)
DEI
9. Hybridization 9
• Problem:
• Testful does not provide any guidance to exercise uncovered
branches
• Solution:
• Hybridize the evolutionary search
with a local search
• It focuses on the easiest
branch not yet executed
(works at method level)
• It adopts a simpler search
strategy (hill climbing)
• Which element to improve?
• One of the best elements
• 5% of the population
• 10% of the population
DEI
10. Hybridization 10
Simple Problem
(Fraction)
Complex Problem
(Disjoint Set Fast)
DEI
11. Fitness Inheritance 11
• Problem:
• The fitness function is expensive
• Solution:
• Use Fitness Inheritance
• Evaluate the “real” fitness only on a part of the population
• Other individuals inherit the fitness of their parents
• Which policy to select individuals to evaluate?
• Uniform selection
• Fronteer selection: evaluate more often individuals on the frontier
• How to evaluate the improvements?
• Number of generation evaluated w.r.t. elapsed time
• Number of branch covered
• (to evaluate the impact of a noisier fitness function)
DEI
12. Fitness Inheritance: # of generations 12
Complex Problem
(Stack Array)
More complex Problem
(Red-Black Tree)
DEI
13. Fitness Inheritance: branch executed 13
Simple Problem
(Fraction)
Computationally-intensive Problem
(Disjoint Set Fast)
DEI
14. Conclusions and Future Work 14
• Efficiency enhancement techniques allow Testful to reach better
results using less computational resources
• The improvement is even higher if the three efficiency
enhancement techniques are combined
• We plan to leverage even more on the Seeding technique
• To test newer versions of a class
• To test similar classes of a class
DEI
15. Improving Evolutionary Testing by Means of
Efficiency Enhancement Techniques
M. Miraz, P. L. Lanzi, L. Baresi
{miraz,lanzi,baresi}@elet.polimi.it
http://code.google.com/p/testful/