Multiobjective Design of Micro- and Macrostructures. "To craft and analyze algorithms that search for optimal structures is the subject of the research in the multiobjective optimization and decision analysis group, and in the talk, we will discuss approaches, their theoretical limits, as well as applications to challenging design problems across multiple scales."

1. Multiobjective Design of
Micro- and Macrostructures
Michael T. M. Emmerich
LIACS, Leiden University
@ESTEC Noordwijk, Science Coffee
December 14th 2018
http://moda.liacs.nl

2. Perfect Structures
Min Surface, Volume = const
Given points
Connect with min path
(Steiner tree)

3. Perfect Designs, 𝑐 𝑤 = min
Norman Bel Geddes, "Motor Car No. 9 (without tail fin)"

4. Perfect design of a ligand, free energy → min

5. Multicriteria Problems: Car Example
𝑆𝑝𝑒𝑒𝑑 𝑥 → 𝑚𝑖𝑛
Your objectives: Cost → min, Speed → max
Pareto optimal cars (no-dominated points)
Cost 𝑥 → 𝑚𝑖𝑛
𝑥 ∈ 𝑋 = {𝑆𝑡𝑟𝑒𝑡𝑐ℎ𝐿𝑖𝑚𝑜, 𝐵𝑀𝑊, 𝐵𝑒𝑒𝑡𝑙𝑒, 𝐹𝑒𝑟𝑟𝑎𝑟𝑖}
Dominated point

6. Perfect Multiobjective Designs?
𝑂𝑟𝑎𝑛𝑔𝑒 𝑀𝑜𝑢𝑛𝑡𝑎𝑖𝑛𝑠 𝑓1 𝑥1, 𝑥2 → max
𝐵𝑙𝑢𝑒 𝑀𝑜𝑢𝑛𝑡𝑎𝑖𝑛𝑠 𝑓1 𝑥1, 𝑥2 → max

7. Multi-sphere Testproblem with local Pareto fronts
Kerschke, Pascal, Hao Wang, Mike Preuss, Christian Grimme, André Deutz, Heike Trautmann,
and Michael Emmerich. "Towards Analyzing Multimodality of Continuous Multiobjective
Landscapes." In International Conference on Parallel Problem Solving from Nature, pp. 962-
972. Springer International Publishing, 2016. (Best paper award)

8. Pareto optimality
x1
x2
x3
Search space
(decision space)
𝑓2 𝑥1, 𝑥2, 𝑥3 → 𝑚𝑖𝑛
Objective space
(solution space)
Image
under f = all possible solutions
Pareto front =
Set of interesting solutions
f
𝑓1 𝑥1, 𝑥2, 𝑥3 → 𝑚𝑖𝑛

9. Research questions
• Find ‘Pareto perfect’ structures:
• Micro-
• Macro-
• Efficiency
• Precision & Coverage Niek de Kruijf, Shiwei Zhou, Qing Li, Yiu-Wing
Mai,Topological design of structures and
composite materials with multiobjectives,
International Journal of Solids and Structures,
Volume 44, Issues 22-23, 2007, Pages 7092-7109

10. Visualization of a 3-D Pareto front

11. Visualization of a 4-D Pareto Front

12. Finding the Pareto front

13. Finding the efficient set in LP: Example
Efficient set
Infeasible region

14. Single objective, unconstrained

15. Unconstrained Multiobjective Optimization
In 2-dimensional spaces this
criterion reduces to the
observation, that either one of
the objectives has a zero
gradient ( neccesary condition
for ideal points) or the gradients
are parallel.
X*

16. Level Set Continuation

17. Set-oriented Optimization
• Hypervolume Indicator
1. Size of dominated space
2. Bound from above by a
reference point
3. Maximization yields Pareto front

18. Complexity of
Hypervolume Indicator
• #P-completeness in 𝑚, 𝑛 ≫ 𝑚: (Bringmann,
Friedrichs (2011))
• Θ 𝑛 log 𝑛 Fonseca et al. ′08 for 2D and 3D;
Incremental Θ log 𝑛 (Hupkens, Emmerich ’13),
4D: O(𝑛2
); Θ 𝑛 update; N-D:
O(𝑛
𝑑
3 log 𝑛)(Chan’13),
• Optimal 𝑘 subset Selection: Polynomial time in
2D, but NP hard in 3D => reduction to 3-degree
planar graph independent set. [2]; Fixed
parameter tractable algorithm(FPTA) O(2 𝑘
𝑛)
• Single Point Contributions 𝚯(𝒏 𝒍𝒐𝒈 𝒏) in 2D and in
3D, (Emmerich & Fonseca 2011); [Guerreira and
Fonseca, 2014], optimal time algorithm. Used in
SMS-EMOA and archivers [Knowles, Fleischer 2013]
• Integrated in PygMO library (Marcus Maertens)
[1] Emmerich, M. T., & Fonseca, C. M. (2011, April). Computing hypervolume contributions in low dimensions: Asymptotically optimal algorithm and complexity results. EMO’11.
[2] Bringmann, K., & Friedrich, T. (2009, April). Approximating the least hypervolume contributor: NP-hard in general, but fast in practice. In International Conference on EMO’09 (pp. 6-20).
Springer Berlin Heidelberg.
[3] K. Bringman, S. Cabello, M. Emmerich (G. Rote): Maximum Volume Subset Selection for Anchored Boxes: accepted: Symposium on Computational Geometry, (SoCG) July 2017

19. Theorem: Find optimal 𝑘 subset from union of 𝑛 (anchored)
boxes[1], convex hull [2] NP hard
for 𝑑 ≥ 3 but can be solved in subexponential time 𝑂(2√𝑛).
Karl Bringmann, Sergio Cabello, and Michael T.M. Emmerich: Maximum Volume Subset Selection for Anchored Boxes, Symposium on Computational
Geometry, Brisbane, Australia, July 2017
Günter Rote, Kevin Buchin, Karl Bringmann, Sergio Cabello, and Michael Emmerich. Selecting K Points that Maximize the Convex Hull Volume.
JCDCG3 2016; The 19th Japan Conference on Discrete and Computational Geometry, Graphs, and Games. September 2 - 4, 2016

20. Bayesian Optimization:
Expected Hypervolume Improvement
• EHVI Definition and
Monte Carlo Integration (Emmerich
2005,Emmerich Giannakoglou Naujoks IEEE TEVC
2006)
• Best method in many applications of multicriteria
Bayesian optimization (e.g. Simoyama et al.):
airfoils, quantum physics, robotics, biofuel
plants)
• Exact computation (Emmerich, Klinkenberg,
Deutz 2011), but computationally expensive
(already 𝑂(𝑛3
log 𝑛) in 2D)
• Faster Exact Algorithms (average case): Couckuyt,
Deschrijver, Dhaene 2015, J. Global Optimization
• Asymptotically optimal algorithm in 2D and 3D:
Θ(n log n). Emmerich, Yang, Fonseca EMO’2017Gaussian
Process (kriging)
Model assisted op-
timization
𝑓1 → min 𝑓2 → min
Leiden University 20
Emmerich, M., Yang, K., Deutz, A., Wang, H., & Fonseca, C. M. (2016).
A Multicriteria Generalization of Bayesian Global Optimization. In: Pardalos et al. Advances in Stochastic and Deterministic Global
Optimization: Dedicated to 70ties Birthday of Antanas Zilinskas (pp. 229-242). Springer International Publishing.

21. Box-partitioning of hypervolume in
Θ(𝑛) time in 3-D case
21
Kaifeng Yang, Michael Emmerich, André Deutz and Carlos Fonseca. (2017) Computing 3-D Expected Hypervolume Improvement
and Related Integrals in Asymptotically Optimal Time, Accepted for EMO, March 2017
‘ Dominated space can be partitioned into n+1 boxes in 2-D,
and into 2n+1 boxes in 3-D’

22. Bayesian Global Optimization using EHVI
• Initial design 10 points
• 2-sphere problem
• 15 updates of archive
based on maximal EHVI
infill
• Infill happens in
underexplored but
promising regions
• Variance monotonicity (2-
D), see Emmerich, Deutz,
Klinkenberg (2011)
• Mean value monotonicity
Wagner, Deutz, Emmerich
(2010)
Emmerich, M. T., Deutz, A. H., & Klinkenberg, J. W. (2011, June). Hypervolume-based expected improvement: Monotonicity properties and exact
computation. In Evolutionary Computation (CEC), 2011 IEEE Congress on (pp. 2147-2154). IEEE.
T Wagner, M Emmerich, A Deutz, W Ponweiser: On expected-improvement criteria for model-based multi-objective optimization
Parallel Problem Solving from Nature, PPSN XI, 718-727

23. Set Gradient Methods: Fast Numerical
Computation of Pareto Front
• Pareto front gradients proposed by
Emmerich, Beume, Deutz 2007 for 2-
D
• Generalization to N-D and efficient
computation (compute visible facets)
in Θ(𝑛𝑑 + 𝑛 log 𝑛 ) optimal time by
Emmerich & Deutz 2013
• Gradient Methods: Linear
convergence speed (Emmerich,
Beume 2007), (Wang, Emmerich,
Bäck 2017)
• Multicriteria Newton’s Method:
Quadratic Convergence speed (Sosa,
Schütze, Emmerich 2014);
Multicriteria Hessian (Sosa, Wang,
Schütze, Deutz, Emmerich, 2017)
Emmerich, Michael, and André Deutz. "Time complexity and zeros of the hypervolume indicator gradient field." EVOLVE-A Bridge between
Probability, Set Oriented Numerics, and Evolutionary Computation III. Springer International Publishing, 2014. 169-193.
𝛻𝐻 𝐹( Ԧ𝑥1 ∘ ⋯ ∘ Ԧ𝑥 𝜇)
Leiden University 23

24. Trade-off bounding/Uniformity:
Cone based hypervolume indicator
Cone-base hypervolume for trade-off bounded optimization
Shear transformation: Linear-time reduction to HI
Shukla, P. K., Emmerich, M., & Deutz, A. (2013, March). A theoretical analysis of curvature based preference models. In International
Conference on Evolutionary Multi-Criterion Optimization (pp. 367-382). Springer Berlin Heidelberg.
Emmerich, Michael, et al. "Cone-based hypervolume indicators: Construction, properties, and efficient computation." International
Conference on Evolutionary Multi-Criterion Optimization. Springer Berlin Heidelberg, 2013.
Ԧ𝑐(𝑖)
= (Ԧ𝑒 𝑖
∧ Ԧ𝑎) / sin(𝛾);
(here ∧ is the geometric product in the Clifford-Grassmann Algebra)

25. CHI-EMOA on
Lamé
Superspheres
Similar to linear weighting Pareto based selection Normal boundary
intersection
Ԧ𝑐(𝑖) = ( Ԧ𝑒 𝑖 ∧ Ԧ𝑎) / sin(𝛾);
(here ∧ is the geometric product)

26. 26
Questions in Indicator Based Algorithm Design
1. Fast computation of performance indicator
2. Optimal subset selection
3. Contributions of single points
4. Fast incremental updates, archiving.
5. Distribution of optimal sets?
6. Computation of integrals/gradients
7. More than 3 objectives? 1,2,3, many
SIMCO
Lorentz Center Workshop
June 2013
Leiden University26
SIMCO open problem page:
http://simco.gforge.inria.fr/doku.php?id=
openproblems

27. 27
Approach
Optimization Algorithms
e.g. gradients, orders, global search
Computational Geometry
e.g. volumes, metrics, data-structures
Leiden University27

28. Example Projects

29. Example 1: Building Design (with TU/e)
Hérm Hofmeyer (TU Eindhoven), Koen
v.d.Blom, M. Emmerich - Excellent
Buildings via Forefront MDO-STW Open
Technology Call (2015)
Leiden University
• Multidisciplinary Optimization
• Energy design vs. structural design efficiency
Hopfe, C. J., Emmerich, M. T., Marijt, R., & Hensen, J. (2012, September). Robust multi-criteria design optimisation in building
design. In First Building Simulation and Optimization Conference (pp. 10-11). ISO 690
Boonstra, S., van der Blom, K., Hofmeyer, H., Emmerich, M. T. M., van Schijndel, A. W. M., & de Wilde, P. (2018). Toolbox for
super-structured and super-structure free multi-disciplinary building spatial design optimisation. Advanced Engineering
Informatics Advanced Engineering Informatics, 36, 86.Boonstra, S
Vd Blom, K.

30. 𝑖 ∈ {1,2, … , 𝑁 𝑤} 𝑤𝑖 ∈ ℝ ≥ 0
𝑗 ∈ {1,2, … , 𝑁𝑑} 𝑑𝑗 ∈ ℝ ≥ 0
𝑘 ∈ {1,2, … , 𝑁ℎ} ℎ 𝑘 ∈ ℝ ≥ 0
ℓ ∈ {1,2, … , 𝑁𝑟𝑜𝑜𝑚𝑠}
𝑏𝑖,𝑗,𝑘
ℓ
= ቊ
1 if cell (𝑖, 𝑗, 𝑘) belongs to room ℓ
0 otherwise
Superstructure representation

31. Pareto Front
• Gradient only is not sufficient
• In this example: Memetic ≻ Evolutionary before gradient search starts
(in discrete space)

32. Visualisation
Structural Kneepoint Thermal
• Trade-off between objectives

33. Multiobjective optimization to define wall structur?
14-12-201833
Variables
Variable types
Beams
Trusses
Flat Shells
Or No Structure!

34. LIACS, Leiden University
lightweightstiffness

35. Example 2: In-Silico
Drug Discovery
(with LACDR)
1060
Potency → max
Side-effects → min
Cost ≤ Budget
Diversity → max
Leiden University
van der Horst, E., Marqués-Gallego, P., Mulder-Krieger, T., van Veldhoven, J.,
Kruisselbrink, J., Aleman, A., M. Emmerich & IJzerman, A. P. (2012). Multi-
objective evolutionary design of adenosine receptor ligands. Journal of chemical
information and modeling, 52(7), 1713-1721.

36. Design of a 14-3-3𝛾 ligand
• Objectives
• Max. binding affinity to
𝛾 −isoform
• Min. binding affinity to other 14-3-
3 isoforms
• Peptides = Sequences of amino
acids 2024 possibilities
• Micro-structures with elaborate
3-D structure; MOE simulation

37. 37
Evolution with SMS-EMOA

38. 38
MODA
2.0
Future vision: From Theory to Optimal Design
Human-centric design;
domain expert knowledge
Computational Challenges:
Precision, Coverage & Time Efficiency
Ω(𝑛 log 𝑛 𝑑−1
)
Applications Visual Analytics/Innovization
Sustainable
Design
Computational
Pharmacology

39. 39
MODA Research Team
Leiden University39
*MODA = Multiobjective Optimization and Decision Analysis
http://moda.liacs.ml

40. 40
References
• Emmerich, M. T., & Deutz, A. H. (2018). A tutorial on multiobjective optimization: fundamentals and evolutionary
methods. Natural computing, 17(3), 585-609.
• Emmerich, M., & Deutz, A. (2014). Time complexity and zeros of the hypervolume indicator gradient field. In
EVOLVE-A Bridge between Probability, Set Oriented Numerics, and Evolutionary Computation III (pp. 169-193).
Springer, Heidelberg.
• Boonstra, S., van der Blom, K., Hofmeyer, H., Emmerich, M. T. M., van Schijndel, A. W. M., & de Wilde, P. (2018).
Toolbox for super-structured and super-structure free multi-disciplinary building spatial design optimisation.
Advanced Engineering Informatics Advanced Engineering Informatics, 36, 86.
• van der Horst, E., Marqués-Gallego, P., Mulder-Krieger, T., van Veldhoven, J., Kruisselbrink, J., Aleman, A., ... &
IJzerman, A. P. (2012). Multi-objective evolutionary design of adenosine receptor ligands. Journal of chemical
information and modeling, 52(7), 1713-1721.
• Chen, Yuhang, Shiwei Zhou, and Qing Li. "Multiobjective topology optimization for finite periodic structures."
Computers & Structures 88.11-12 (2010): 806-811.
Leiden University40
https://esa.github.io/pagmo2/index.html
SOFTWARE IMPLEMENTING MODA IN PYTHON: