A parallelized approach for MOO.

1. Search-based SE: without search, you won’t find a thing.
“Engineering is optimization and optimization is search.”
ai4se.net
Evolutionary Multi-Objective
Optimization : A Distributed
Computing Approach
- Rahul Krishna (rkrish11)
- George Mathew (george2)
1

2. Search-based SE: without search, you won’t find a thing.
“Engineering is optimization and optimization is search.”
ai4se.net
Multi-Objective Problem
Pareto Frontier : State of solutions in
which it is impossible to make any one
individual better off without making at
least one individual worse off.
Pareto Point : A point that lies on the
pareto frontier.
Feasible Point : A satisifiable solution
for the problem but not necessarily the
optimum one.
Infeasible Point: A solution outside the
pareto frontier
Utopia Point: The ideal theoretical
solution we would love to reach but
practically its not possible. At this point
all the objectives are optimal without
any conflicts amongst them.
Image from Search Based SE class NCSU Fall - 2014 https://github.com/timm/sbse14/wiki/DiffEvol
2

3. Search-based SE: without search, you won’t find a thing.
“Engineering is optimization and optimization is search.”
ai4se.net
DTLZ2
● Set of multiobjective mathematical test problems created by Kalyanmoy Deb, Lothar
Thiele, Marco Laumans and Eckhart Zitzler.
● Decisions : DTLZ2 has 30 decisions where each decision ranges between 0 and 1
● Objectives : 3 objectives defined as follows
● Optimal solutions :
Ideal decisions xi
= 0.5.
Ideal objectives should satisfy the equation
Reference: Kalyanmoy Deb et.al, Scalable multi-objective optimization test problems. CEC ’02. Proceedings of the 2002 Congress
3

4. Search-based SE: without search, you won’t find a thing.
“Engineering is optimization and optimization is search.”
ai4se.net
Evolutionary Algorithm
4

5. Search-based SE: without search, you won’t find a thing.
“Engineering is optimization and optimization is search.”
ai4se.net
Differential Evolution (DE)
● Stochastic evolutionary optimization technique.
● Iteratively approximates the shape of the Pareto
Frontier
● Advantages:
○ Simple and computationally inexpensive
○ High dimensional problems can be handled easily
○ Solutions are very stable
5

6. Search-based SE: without search, you won’t find a thing.
“Engineering is optimization and optimization is search.”
ai4se.net
DE (Algorithm)
6

7. Search-based SE: without search, you won’t find a thing.
“Engineering is optimization and optimization is search.”
ai4se.net
Geometric Active Learner(GALE)
● Near linear time MOEA
● Builds piecewise approximation to the best solutions of the
pareto frontier
● Based on WHERE which is a recursive clustering based on
dimensionality reduction.
● Advantages:
○ Less number of computations
○ Adept at handling objective functions that are non-
differentiable, non-linear, multidimensional or multi-
constraint problems
○ Concise representation of problem space
7

8. Search-based SE: without search, you won’t find a thing.
“Engineering is optimization and optimization is search.”
ai4se.net
GALE(Algorithm)
● Repeat for n generations
● Cluster data based on WHERE
○ Pick point X from cluster. Then pick point East furthest from X and point West furthest
from East. Let c be distance between East and West.
○ For every other point in cluster, compute a and b as distance of point from East and West
respectively.
○ Compute the projection x as follows
x = (a2
+ c2
- b2
) / 2c
○ Split the cluster at median value of x and repeat for each sub cluster
● Select the best point(East/West) from the non-dominated leaf cluster. and mutate towards it.
● The best points after each generation represents the non dominated solutions
8

9. Search-based SE: without search, you won’t find a thing.
“Engineering is optimization and optimization is search.”
ai4se.net
Why Parallelization?
• Multi-objective problems are complex and usually resource
intensive.
• Exhaustive search consumes prohibitively large amounts of
memory and time
• Meta-heuristics:
• Approximate solutions fast.
• Reasonably accurate
• However, scaling to solve real world is still hard
• Distributed computing approaches offer significant speed up.
• Distribute the evaluation over several “nodes”.
• In theory:
• The solutions must be robust, and
• Be of better quality. 9

10. Search-based SE: without search, you won’t find a thing.
“Engineering is optimization and optimization is search.”
ai4se.net
Evaluation
Runtime — Time taken to run the algorithm in the parallelized version versus the serial
version. This can be measured using a profiler.
Image courtesy : Kalyanmoy Deb et.al., A Fast and Elitist Multiobjective Genetic Algorithm: NSGA-II
Solution Quality
● Convergence
○ Accuracy of the obtained
solutions.
○ It represents the hypervolume
between the obtained solutions
and Pareto frontier.
● Diversity
○ Spread of the proposed solutions.
○ Ideally the solutions should be
well distributed across the Pareto
frontier.
10

11. Search-based SE: without search, you won’t find a thing.
“Engineering is optimization and optimization is search.”
ai4se.net
Convergence:
● Find a set of H optimal solutions.
● For each solution, compute the minimum
euclidean distance from each of the
solutions to a point on the Pareto Frontier.
● The average of these distances represent
convergence.
Diversity:
● di
represents the distance between
consecutive solutions.
● represents the mean of all di
.
● df
and dl
are the distances between the
extreme solutions and the boundary
solutions
Measures
11

12. Search-based SE: without search, you won’t find a thing.
“Engineering is optimization and optimization is search.”
ai4se.net
Framework
● Python
○ Better support for efficient computation frameworks: numpy, scipy, etc.
○ Quick prototyping and benchmarking.
● Open-MPI
○ An open source Message Passing Interface with a python wrapper.
○ The Open MPI Project is actively developed and maintained by a
consortium of academic, research, and industry partners.
● HPC
○ The henry2 shared memory linux cluster at NCSU.
○ Up to 16 shared memory processor cores and up to 128GB of memory
accessible through a dedicated queue.
12

13. Search-based SE: without search, you won’t find a thing.
“Engineering is optimization and optimization is search.”
ai4se.net
Challenges
● Identifying dependencies: Since the meta-heuristics were designed to work
synchronously, a key challenge would be to identify the modules that are
independent, so that they may be executed concurrently.
● Communication: In order to reduce runtime, it is pertinent that the
communication between the slaves and the master be limited. In addition, any
data transfer needs to be minimized.
● Quality and Runtime Trade-off: The increased speed of parallel algorithms
comes at the cost of reduced quality when compared to the serial version of
the algorithm.
13

14. Search-based SE: without search, you won’t find a thing.
“Engineering is optimization and optimization is search.”
ai4se.net
Thank You
14