Niels van Harten Radboud University Nijmegen Nijmegen, The Netherlands CDN (Diego) Damasceno Radboud University Nijmegen Nijmegen, The Netherlands Daniel Strüber Chalmers | University of Gothenburg (SE) Radboud University Nijmegen (NL)

1. Model-Driven Optimization:
Generating Smart Mutation Operators for
Multi-Objective Problems
Niels van Harten
Radboud University Nijmegen
Nijmegen, The Netherlands
niels@vharten.com
CDN (Diego) Damasceno
Radboud University Nijmegen
Nijmegen, The Netherlands
https://damascenodiego.github.io/
d.damasceno@cs.ru.nl
Daniel Strüber
Chalmers | University of Gothenburg (SE)
Radboud University Nijmegen (NL)
https://www.danielstrueber.de/
danstru@chalmers.se
Euromicro DSD/SEAA 2022
Aug. 31 – Sep. 2, 2022

2. Optimization problems are at the heart of
many software engineering tasks
26/08/2022 2
van Harten, Damasceno and Strüber (2022)
Solutions
Optimality
Architecture
refactoring
Assignment
Classes → Packages
max. Cohesion
min. Coupling
Release
planning
Assignment
Next Release → Artifacts
min. Development Cost
max. Customer Satisfaction
Components
deployment
Assignment
Components → Hosts
min. Price
min. Overhead

3. Search-based software engineering(SBSE)
• Problem: Search space usually too large to enumerate all solutions
• Approach: Search guided by principles from the evolutionary theory
• Challenge: Search algorithms need to be custom-tailored to the problem at hand
Mutation Crossover Selection
Fitness
criteria
based on
Genetic algorithm
Domain
expert
26/08/2022 van Harten, Damasceno and Strüber (2022) 3

4. Model-Driven Optimization
4
26/08/2022
Domain expert
Solution
meta-model
Constraints
Fitness
functions Optimization tool
Optimal
solution
models
has domain
knowledge
encodes
optimisation
knowledge
uses model
transformation
rules

5. FitnessStudio: A two-tier framework of nested genetic algorithms
Crossover
LOWER
TIER
Solution
candidate
UPPER TIER
Mutation
Crossover
Fitness
Function
Fitness
Function
modifies assesses
combines
modifies
assesses
combines
informed by
Constraints
informed by
Input
model
is first
Mutation
26/08/2022 van Harten, Damasceno and Strüber (2022) 5
Limitations:
1. Relies on a single-objective
genetic algorithm on both tiers
2. Mutation operators are
generated: from scratch and
without additional user input

6. Our Contribution
A model-driven optimization technique to automatically
generate efficient mutation operators with support for:
1. Multiple optimization criteria
2. User-provided domain knowledge
26/08/2022 van Harten, Damasceno and Strüber (2022) 6

7. Model Driven-Optimization

8. Model-Driven Optimization
• Model-driven principles to bridge the abstraction gap between:
• The low-level SBSE implementations
• Declarative specifications of optimization problems (e.g., models)
• Graph-transformations languages (e.g., Henshin)
26/08/2022 8
Henshin: Japanese for Transformation
https://www.eclipse.org/henshin/
Vector-based encoding
Chromosome 0 1 1 0
Cost 5 1 9 7
Model-based encoding
Artifact 1
Cost:5
Artifact 2
Cost:1
Artifact 3
Cost:9
Artifact 4
Cost:7
Solution

9. The Next Release Problem
26/08/2022 9
Customers
Software
artifacts
Artifact 1
Cost:5
Artifact 2
Cost:1
Artifact 3
Cost: 7
Artifact 4
Cost:9
Customer b
Importance:2
Customer a
Importance:1
Customer c
Importance:3
Find the “optimal” selection of artifacts w.r.t.
Requirements Requirement #2
Requirement #1 Requirement #3

10. The Next Release Problem
26/08/2022 10
Example of solution: Release artifacts 1, 2
Cost = 5+1
Satisfaction = 1*100% + 2*50% + 3*0%
Customers
Software
artifacts
Artifact 1
Cost:5
Artifact 2
Cost:1
Artifact 3
Cost: 7
Artifact 4
Cost:9
Customer b
Importance:2
Customer a
Importance:1
Customer c
Importance:3
Requirements Requirement #2
Requirement #1 Requirement #3
= 6
= 2

11. The Next Release Problem
26/08/2022 11
Example of solution: Release artifacts 2, 3, 4
Cost = 1+7+9
Satisfaction = 1*0% + 2*100% + 3*100%
Customers
Software
artifacts
Artifact 1
Cost:5
Artifact 2
Cost:1
Artifact 3
Cost: 7
Artifact 4
Cost:9
Customer b
Importance:2
Customer a
Importance:1
Customer c
Importance:3
Requirements Requirement #2
Requirement #1 Requirement #3
= 17
= 5

12. The Next Release Problem
26/08/2022 12
Example of solution: Release artifacts 2, 3
Cost = 1+7
Satisfaction = 1*0% + 2*100% + 3*100%
Customers
Software
artifacts
Artifact 1
Cost:5
Artifact 2
Cost:1
Artifact 3
Cost: 7
Artifact 4
Cost:9
Customer b
Importance:2
Customer a
Importance:1
Customer c
Importance:3
Requirements Requirement #2
Requirement #1 Requirement #3
= 8
= 5
Can we do
better?

13. Transformation Rules
Metamodel for the next release problem (NRP)
The Next Release Problem
26/08/2022 13
van Harten, Damasceno and Strüber (2022)

14. The Next Release Problem
26/08/2022 14
A1
Cost:5
A2
Cost:7
A3
Cost:9
A4
Cost:1
Inconsistent Solution
?
A1
Cost:5
A2
Cost:7
A3
Cost:9
A4
Cost:1
Solution 4
1x addSingleSa(A1) ●
2x addSingleSa(A4 then A1) ●
A1
Cost:5
A2
Cost:7
A3
Cost:9
A4
Cost:1
Solution 2
1x addSingleSa ●
1x removeSingleSa ●
A1
Cost:5
A2
Cost:7
A3
Cost:9
A4
Cost:1
Solution 3
1x removeSingleSa ●
van Harten, Damasceno and Strüber (2022)
A1
Cost:5
A2
Cost:7
A3
Cost:9
A4
Cost:1
Solution 1

15. Our Contribution
26/08/2022 15
Solution
candidate
Crossover
Fitness
Function
modifies
assesses
combines
Constraints
LOWER
TIER
Crossover
UPPER TIER
Mutation
Fitness
Function
modifies assesses
combines informed by informed by
Input
model
is first
Mutation
Fitness
Function
Fitness
Function
A
A
Configuration
Mutation
B
B

16. (A) Support for multi-objective problems
• FitnessStudio aims at a single objective
• Inapplicable to problems with multiple objectives (e.g., NRP)
• Our contribution #1: Improved fitness functions
• Lower tier: Relies on an arbitrary number of functions
• Upper tier: Hypervolume for lower-tier solutions
26/08/2022 van Harten, Damasceno and Strüber (2022) 16
https://doi.org/10.1109/ACCESS.2019.2927418

17. (B) Support for user-defined domain knowledge
• FitnessStudio generates mutation operators only from scratch (a.k.a. full automation)
• Our contribution #2: User-specified rule set
• At large scale problems, we need "the best of both worlds“:
• Useful, but not necessarily optimal mutation operators
• Continuous improvement of mutation operators
• Our contribution #3: Configuration of upper-tier algorithm
• Assign a weight to each higher-order mutation rule
26/08/2022 van Harten, Damasceno and Strüber (2022) 17
fixedXORgen: Combining generated
with user specified fixed rules

18. Evaluation and Results

19. Evaluation
26/08/2022
Table: Instances of the next release problem
• Subject: The Next Release Problem (NRP)
• RQ1: How does the mutation operator generated by our framework impact performance,
compared to a manually specified operator?
• Performance = Result quality and Execution time
• Focus: multi-objective NRP
• RQ2: To which extent does the customization with user-provided domain knowledge impact the
performance?
• Isolated user-provided domain knowledge
• Focus: single-objective NRP
https://jmetal.github.io/jMetal/
19
van Harten, Damasceno and Strüber (2022)

20. Results for RQ1 (Performance Compared to Manually Specified Operators)
26/08/2022 van Harten, Damasceno and Strüber (2022) 20

21. Results for RQ1 (Performance Compared to Manually Specified Operators)
26/08/2022 van Harten, Damasceno and Strüber (2022) 21
Creation/Deletion of a single
selected edge
�
Creation/Deletion of multiple selected edges
(aka. larger "steps" in the search space)
MDO

22. Results for RQ2 (Impact of Domain Knowledge on Performance)
26/08/2022 van Harten, Damasceno and Strüber (2022) 22

23. Final Remarks

24. Final Remarks
26/08/2022 van Harten, Damasceno and Strüber (2022) 24
DOI: 10.5281/zenodo.6645808 https://github.com/nielsvharten/fitnessstudio-nrp

25. Niels van Harten
Radboud University Nijmegen
Nijmegen, The Netherlands
niels@vharten.com
CDN (Diego) Damasceno
Radboud University Nijmegen
Nijmegen, The Netherlands
https://damascenodiego.github.io/
d.damasceno@cs.ru.nl
Daniel Strüber
Chalmers | University of Gothenburg (SE)
Radboud University Nijmegen (NL)
https://www.danielstrueber.de/
danstru@chalmers.se
van Harten, Damasceno and Strüber (2022)
Euromicro DSD/SEAA 2022
Aug. 31 – Sep. 2, 2022
Questions?
Thank you!
26/08/2022 25
Images from https://www.flaticon.com/