Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

MOMDPSO_IDETC_2014_Weiyang

245 views

Published on

Complex system design problems tend to be high dimen- sional and nonlinear, and also often involve multiple objectives and mixed-integer variables. Heuristic optimization algorithms have the potential to address the typical (if not most) charac- teristics of such complex problems. Among them, the Particle Swarm Optimization (PSO) algorithm has gained significant popularity due to its maturity and fast convergence abilities. This paper seeks to translate the unique benefits of PSO from solving typical continuous single-objective optimization problems to solving multi-objective mixed-discrete problems, which is a relatively new ground for PSO application. The previously de- veloped Mixed-Discrete Particle Swarm Optimization (MDPSO) algorithm, which includes an exclusive diversity preservation technique to prevent premature particle clustering, has been shown to be a powerful single-objective solver for highly con- strained MINLP problems. In this paper, we make fundamental advancements to the MDPSO algorithm, enabling it to solve challenging multi-objective problems with mixed-discrete design variables. In the velocity update equation, the explorative term is modified to point towards the non-dominated solution that is the closest to the corresponding particle (at any iteration). The fractional domain in the diversity preservation technique, which was previously defined in terms of a single global leader, is now applied to multiple global leaders in the intermediate Pareto front. The multi-objective MDPSO (MO-MDPSO) algorithm is tested using a suite of diverse benchmark problems and a disc-brake design problem. To illustrate the advantages of the new MO-MDPSO algorithm, the results are compared with those given by the popular Elitist Non-dominated Sorting Genetic Algorithm-II (NSGA-II).

Published in: Education
  • Be the first to comment

MOMDPSO_IDETC_2014_Weiyang

  1. 1. A New Multi-Objective Mixed-Discrete Particle Swarm Optimization Algorithm (MO-MDPSO) Weiyang Tong*, Souma Chowdhury#, and Achille Messac# * Syracuse University, Department of Mechanical and Aerospace Engineering # Mississippi State University, Department of Aerospace Engineering ASME 2014 International Design Engineering Technical Conference August 18-20, 2014 Buffalo, NY
  2. 2. Particle Swarm Optimization 2 • Particle Swarm Optimization (PSO) • was introduced by Eberhart and Kennedy in 1995 • was inspired by the swarm behavior observed in nature • is a population based stochastic algorithm • Advantages of basic PSO: • Fast convergence • Easy implementation • Few parameters to adjust PSO suffers from pre-stagnation, which is mainly attributed to the loss of diversity during the fast convergence Powerful optimizer for single objective unconstrained continuous problems
  3. 3. Single Objective Mixed-Discrete PSO* Position update: 𝒙𝑖 𝑡 + 1 = 𝒙𝑖 𝑡 + 𝒗𝑖 𝑡 + 1 Velocity update: 𝒗𝑖 𝑡 + 1 = 𝑤𝒗𝑖 𝑡 + 𝑟1 𝐶1 𝑃𝑖 𝑙 (𝑡) − 𝒙𝑖 + 𝑟2 𝐶2 𝑃 𝑔(𝑡) − 𝒙𝑖 + 𝑟3 𝛾𝑐 𝒗𝑖(𝑡) 𝑤 – inertia weight 𝒙𝑖 – position of a particle 𝒗𝑖 – velocity of a particle 𝐶1 – cognitive parameter 𝐶2 – social parameter 𝑡 – generation 𝑟 – random number between 0-1 𝑃𝑖 𝑙 – pbest of a particle 𝑃 𝑔 – gbest of the swarm 3 Diverging velocity Diversity preservation coefficient Inertia Local search Global search *: Chowdhury et al (2013)
  4. 4. Outline • Research Motivation and Objectives • Search Strategy in MO-MDPSO • Dynamics of MO-MDPSO • Multi-domain Diversity Preservation • Numerical Experiments • Continuous Unconstrained Test Problems • Continuous Constrained Test Problems • Mixed-Discrete Test Problems • Concluding Remarks 4
  5. 5. Research Motivation 5 • Major attributes involved in complex engineering optimization problems: • Nonlinearity • Multimodality • Constraints • Mixed-discrete • Multi-objective Constrained Multiobjective MultimodalNonlinear Mixed types of variables MDPSO addressed by
  6. 6. Research Objective • Develop a Multi-Objective MDPSO (MO-MDPSO) • Introduce the Multi-objective capability to MDPSO • Allow the algorithm to search for Pareto solutions • Make important advancements on the diversity preservation technique – primary feature of MDPSO • Avoid stagnation and capture the complete Pareto frontier • Keep a desirably even distribution of Pareto solutions 6
  7. 7. Search Strategies in Multi-Objective PSO 7 • MOPSO by Parsoupulos and Vrahatis (2002) • DNPSO by Hu and Eberhart (2002) • NSPSO by Li (2003) • MOPSO by Coello (2004) • To best retain the original dynamics, the Pareto based strategy is selected Aggregating function based Single objective based Hybrid with other techniques Pareto dominance based Basic PSO MOPSO Local leader The best solution is based on a particle’s own history Local set with non-dominated historical solutions are found within the local neighborhood Global leader The best solution among all local best solutions The global Pareto solutions are obtained using all local ones
  8. 8. Solutions Comparison if both solu-x and solu-y are infeasible choose the one with the smaller constraint violation; else if solu-x is feasible and solu-y is infeasible choose solu-x; else if solu-x is infeasible and solu-y is feasible choose solu-y; else both solu-x and solu-y are feasible apply non-dominance comparison:  solu-x strongly dominates solu-y if and only if 𝑓𝑘 𝑥 < 𝑓𝑘 𝑦 , ∀k = 1,2,…, N  solu-x weakly dominates solu-y if 𝑓𝑘 𝑥 ≤ 𝑓𝑘 𝑦 for at least one k  solu-x is non-dominated with solu-y when 𝑓𝑜𝑟 𝑎𝑡 𝑙𝑒𝑎𝑠𝑡 𝑜𝑛𝑒 𝑘: 𝑓𝑘 𝑥 > 𝑓𝑘 𝑦 , for the rest: 𝑓𝑘 𝑥 < 𝑓𝑘 𝑦 8
  9. 9. 1 2 3 4 5 1 2 3 4 5 f1 f2 0 0.5 1 1.5 2 2.5 3 0 0.5 1 1.5 2 2.5 3 Infeasible R egion Actual Boundary Local Pareto set Dynamics of MO-MDPSO Position update: 𝒙𝑖 𝑡 + 1 = 𝒙𝑖 𝑡 + 𝒗𝑖 𝑡 + 1 Velocity update: 𝒗𝑖 𝑡 + 1 = 𝑤𝒗𝑖 𝑡 + 𝑟1 𝐶1 𝑷𝑖 𝑙 (𝑡) − 𝒙𝑖 + 𝑟2 𝐶2 𝑷𝑖 𝑔 (𝑡) − 𝒙𝑖 + 𝑟3 𝛾 𝑐,𝑖 𝒗𝑖(𝑡) 9 𝑷𝑖 𝑙 – local leader of particle-i, which is the selected from the local Pareto set 𝑷𝑖 𝑔 – global leader of particle-i that is determined by a stochastic process Crowding Distance – to manage the size of local/global Pareto set Multiple global leaders Inertia Local search Global search Applied w.r.t. each particle’s global leader Current particle Stored particle
  10. 10. Diversity Metrics The diversity is measured based on the spread in design space for continuous variables: 𝐷𝑐 = 𝑗=𝑚+1 𝑛 𝑥 𝑚𝑎𝑥,𝑗(𝑡) − 𝑥 𝑚𝑖𝑛,𝑗(𝑡) 𝑋 𝑚𝑎𝑥,𝑗 − 𝑋 𝑚𝑖𝑛,𝑗 1 𝑛−𝑚 for discrete variables: 𝐷 𝑑 𝑗 = 𝑥 𝑚𝑎𝑥,𝑗(𝑡) − 𝑥 𝑚𝑖𝑛,𝑗(𝑡) 𝑋 𝑚𝑎𝑥,𝑗 − 𝑋 𝑚𝑖𝑛,𝑗 10 Considering the impact of outlier solutions, the diversity metrics are modified as 𝐷𝑐,𝑖 = 𝐹𝜆𝑖 𝐷𝑐, and 𝐷 𝑑,𝑖 𝑗 = 𝐹𝜆𝑖 𝐷 𝑑 𝑗 where 𝐹𝜆𝑖 = 𝜆 𝑁 𝑝 + 1 𝑁𝑖 + 1 1 𝑛 𝜆 is used to define the fractional domain w.r.t. the global leader of particle-i The spread along the jth dimension The upper & lower bounds along the jth dimension Number of candidate solutions per global leader Number of candidate solutions enclosed by the 𝜆-fractional domain
  11. 11. Hypercube enclosing 72 candidate solutions X1 X2 0 1 2 3 0 1 2 3 Particles Global leaders Multiple λ-Fractional Domains 11 • 7 global leaders are observed • Ideally, each fractional domain should enclose 10 particles • Particles located in the overlapping regions are uniformly re-allocated between domains 13 10 14 𝜆 = 0.25 Design Variable Space
  12. 12. Multi-domain Diversity Preservation The boundaries of the 𝜆𝑖 - fractional domain are defined as 𝑥𝑖 𝑚𝑎𝑥,𝑗 = 𝑚𝑎𝑥 𝑥 𝑚𝑖𝑛,𝑗 + 𝜆𝑖 𝛿𝑥 𝑗, 𝑚𝑖𝑛 𝑷𝑖 𝑔,𝑗 + 1 2 𝜆𝑖 𝛿𝑥 𝑗, 𝑥 𝑚𝑎𝑥,𝑗 𝑥𝑖 𝑚𝑖𝑛,𝑗 = 𝑚𝑖𝑛 𝑥 𝑚𝑎𝑥,𝑗 − 𝜆𝑖 𝛿𝑥 𝑗, 𝑚𝑎𝑥 𝑷𝑖 𝑔,𝑗 − 1 2 𝜆𝑖 𝛿𝑥 𝑗, 𝑥 𝑚𝑖𝑛,𝑗 12 • Continuous variables 𝛾 𝑐,𝑖 = 𝛾 𝑐0 𝑒𝑥𝑝 − 1 2 𝐷𝑐,𝑖 𝜎𝑐 2 , 𝜎𝑐 = 1 2 ln 1 𝛾 𝑚𝑖𝑛 • Discrete variables 𝛾𝑑,𝑖 𝑗 = 𝛾 𝑑0 𝑒𝑥𝑝 − 1 2 𝐷 𝑑,𝑖 𝑗 𝜎 𝑑 2 , 𝜎 𝑑 = 1 2 ln 1 𝑀 𝑗
  13. 13. • Three classes of test problems are used to evaluate the performance of MO-MDPSO (two performance metrics and comparison with NSGA-II) • Each of the test problems is run 30 times to compensate for the impact of random parameters • Sobol’s quasirandom sequence generator is used for the initial population User-defined parameters in MO-MDPSO Numerical Experiment 13 Parameter Continuous uncon- strained problems Continuous con- strained problems Mixed-discrete constrained problems 𝑤 0.5 0.5 0.5 𝐶1 1.5 1.5 1.5 𝐶𝑐0 1.5 1.5 1.5 𝛾 𝑐0 1.0 1.0 1.0 𝛾 𝑚𝑖𝑛 1e-4 1e-6 1e-8 𝛾 𝑑0 NA NA 0.5,1.0 𝜆 0.2 0.1 0.1 Local set size 5 6 10 Global set size 50 50 Up to 100 Population size 100 100 min(5n, 100)
  14. 14. Performance Metric for Continuous Unconstrained Problems Results: Continuous Unconstrained Problems 14 Function name Accuracy 𝜸 Diversity ∆ Mean 𝜇 𝛾 Std. deviation 𝜎𝛾 Mean 𝜇∆ Std. deviation 𝜎∆ MO-MDPSO NSGA-II MO-MDPSO NSGA-II MO-MDPSO NSGA-II MO-MDPSO NSGA-II Fonseca 2 0.0043 0.0004 0.7900 0.0062 Coello 0.0069 0.0009 0.7101 0.0137 Schaffer 1 0.0086 0.0010 0.7131 0.0066 Schaffer 2 0.0116 0.0008 0.9655 0.0015 ZDT 1 0.0009 0.0009 0.0001 0 0.7265 0.4633 0.0045 0.0416 ZDT 2 0.0007 0.0008 7.3e-5 0 0.7248 0.4351 0.0049 0.0246 ZDT 3 0.0042 0.0434 0.0004 4.2e-5 0.7457 0.5756 0.0052 0.0051 ZDT 4 1.5922 0.5130 2.1411 0.1184 0.8306 0.7026 0.1040 0.06465 ZDT 6 0.0337 0.2965 0.0525 0.0131 0.8860 0.6680 0.2072 0.0099
  15. 15. Comparison of Performance Indicators 15 Performance of Accuracy Performance of Diversity ZDT 4 0 1 2 3 ZDT 3 0 0.01 0.02 0.03 0.04 0.05 ZDT 1 0 0.0002 0.0004 0.0006 0.0008 0.001 0.0012 0.0014 ZDT 2 0 0.0002 0.0004 0.0006 0.0008 0.001 ZDT 6 0 0.1 0.2 0.3 0.4MO- MDPSO NSGA2 MO- MDPSO NSGA2 MO- MDPSO NSGA2 MO- MDPSO NSGA2 MO- MDPSO NSGA2 ZDT 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 ZDT 2 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 ZDT 3 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 ZDT 4 0 5 10 ZDT 6 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 MO- MDPSO NSGA2 MO- MDPSO NSGA2 MO- MDPSO NSGA2 MO- MDPSO NSGA2 MO- MDPSO NSGA2 The lower the better
  16. 16. Plots: Continuous Unconstrained Problems 16 Number of function evaluations for these problems is 10,000 (25,000 was used by NSGA-II) f1 f2 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 Pareto solution by MO-MDPSO Actual Pareto solution f1 f2 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 Pareto solution by MO-MDPSO Actual Pareto solution f1 f2 0 0.2 0.4 0.6 0.8 1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 Pareto solution by MO-MDPSO Actual Pareto solution ZDT 1 30 design variables ZDT 2 30 design variables ZDT 3 30 design variables f1 f2 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 Pareto solution by MO-MDPSO Actual Pareto solution f1 f2 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 Pareto solution by MO-MDPSO Actual Pareto solution ZDT 4 10 design variables ZDT 6 10 design variables
  17. 17. Plots: Continuous Constrained Problems 17f1 f2 0 50 100 150 200 -200 -150 -100 -50 0 Pareto solution by MO-MDPSO Actual Pareto solution f1 f2 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 Pareto solution by MO-MDPSO Actual Pareto solution f1 f2 0 0.2 0.4 0.6 0.8 1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 Pareto solution by MO-MDPSO Actual Pareto solution f1 f2 0.4 0.6 0.8 1 0 1 2 3 4 5 6 7 8 9 Pareto solution by MO-MDPSO Actual Pareto solution f1 f2 0 20 40 60 80 100 120 10 20 30 40 50 Pareto solution by MO-MDPSO Actual Pareto solution BNH 2 design variables CONSTR 2 design variables KITA 2 design variables SRN 2 design variables TNK 2 design variables Number of function evaluations for these problems is 10,000 (20,000 was used by NSGA-II)
  18. 18. • The MINLP problem adopted from Dimkou and Papalexandri* 18 No. of design variables 6 No. of discrete variables 3 (binary) Function evaluations 10,000 Population size 100 Elite size of NSGA-II 78 Elite size of MO-MDPSO 100 *: Dimkou and Papalexandri (1998) Mixed-Discrete Constrained Test Problem 1 f1 f2 -60 -50 -40 -30 -20 -10 0 -20 0 20 40 60 80 100 Pareto solution by NSGA-II Pareto solution by MO-MDPSO
  19. 19. 19 • The design of disc brake problem adopted from Osyczka and Kundu* No. of design variables 4 No. of discrete variables 1 (integer) Function evaluations 10,000 Population size 100 Elite size of NSGA-II 87 Elite size of MO-MDPSO 100 *: Osyczka and Kundu (1998) Mixed-Discrete Constrained Test Problem 2
  20. 20. Multi-objective Wind Farm Optimization 20 • Two objectives: • Maximize the wind farm capacity factor • Minimize the unit land usage • 150 design variables: • 100 continuous variables: Location of turbines • 50 discrete variables: Type of turbines • 1225 constraints: • Inter-turbine spacing 50 turbines with 10 turbine selections
  21. 21. Concluding Remarks • We developed a new MO-MDPSO algorithm that is capable of handling all the major attributes in complex engineering optimization problems • The original dynamics of basic PSO is best retained by introducing the Pareto dominance strategy to MDPSO • The multi-domain diversity preservation technique was developed to maintain a desirably even distribution of Pareto solutions • MO-MDPSO showed favorable results in solving continuous bi-objective optimization problems; the performance in solving mixed-discrete problems is comparable with or better than NSGA-II • Future work should test MO-MDPSO to high dimensional engineering problems, and include problems with more than two objectives. 21
  22. 22. Acknowledgement • I would like to acknowledge my research adviser Prof. Achille Messac, and my co-adviser Dr. Souma Chowdhury for their immense help and support in this research. • Support from the NSF Awards is also acknowledged. 22
  23. 23. 23 Thank you Test function
  24. 24. BNH 0 0.05 0.1 0.15 0.2 CONSTR 0 0.002 0.004 0.006 0.008 0.01 KITA 0 0.002 0.004 0.006 0.008 0.01 0.012 SRN 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 TNK 0 0.002 0.004 0.006 0.008 0.01 0.012 Results: Continuous Constrained Problems 24 Function name Accuracy 𝜸 Diversity ∆ Mean 𝜇 𝛾 Std. deviation 𝜎𝛾 Mean 𝜇∆ Std. deviation 𝜎∆ BNH 0.1342 0.0204 0.7778 0.0088 CONSTR 0.0070 0.0007 0.8336 0.0101 KITA 0.0091 0.0008 0.7656 0.0086 SRN 0.4909 0.0567 0.7165 0.0046 TNK 0.0055 0.0006 0.9010 0.0216 Performance Metric for Continuous Constrained Problems CONSTR 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 BNH 0 0.1 0.2 0.3 0.4 0.5 KITA 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 SRN 0 0.2 0.4 0.6 0.8 1 TNK 0 0.2 0.4 0.6 0.8 1 1.2 Performance of Accuracy Performance of Diversity
  25. 25. Illustration of MO-MDPSO 1. Both solu-x and solu-y are infeasible – choose the one with the smaller constraint violation; 2. solu-x is feasible and solu-y is infeasible – choose solu-x; 3. solu-x is infeasible and solu-y is feasible – choose solu-y; 4. Both solu-x and solu-y are feasible – apply non-dominance comparison:  solu-x strongly dominates solu-y if and only if: 𝑓𝑘 𝑥 < 𝑓𝑘 𝑦 , ∀k = 1,2,…, N  solu-x weakly dominates solu-y if: 𝑓𝑘 𝑥 ≤ 𝑓𝑘 𝑦 for at least one k 25 1 2 3 4 5 f1 f2 0 0.5 1 1.5 2 2.5 3 0 0.5 1 1.5 2 2.5 3 Infeasible R egion Actual Boundary 1 2 3 4 5 1 2 3 4 5 f1 f2 0 0.5 1 1.5 2 2.5 3 0 0.5 1 1.5 2 2.5 3 Infeasible R egion Actual Boundary Current particle Stored particle Local Pareto set A solution is non-dominated when 𝑓𝑜𝑟 𝑎𝑡 𝑙𝑒𝑎𝑠𝑡 𝑜𝑛𝑒 𝑘: 𝑓𝑘 𝑥 > 𝑓𝑘 𝑦 , for the rest: 𝑓𝑘 𝑥 < 𝑓𝑘 𝑦 k = 1,2,…, N The crowding distance is used to manage the size of Pareto set
  26. 26. Multi-directional Diversity Preservation (cont’d) • Continuous variables 𝛾 𝑐,𝑖 = 𝛾 𝑐0 𝑒𝑥𝑝 − 1 2 𝐷𝑐,𝑖 𝜎𝑐 2 , 𝜎𝑐 = 1 2 ln 1 𝛾 𝑚𝑖𝑛 • The diversity coefficient for continuous variables is to apply a repulsion away from each of the global leaders • Discrete variables 𝛾𝑑,𝑖 𝑗 = 𝛾 𝑑0 𝑒𝑥𝑝 − 1 2 𝐷 𝑑,𝑖 𝑗 𝜎 𝑑 2 , 𝜎 𝑑 = 1 2 ln 1 𝑀 𝑗 • A stochastic update process is applied to help particles jump out of the local hypercude:  if 𝑟 > 𝛾𝑑,𝑖 𝑗 , use the nearest vertex approach (NVA)  else, update randomly to the upper or lower bound of the local hypercube 26 The diversity coefficient is expressed as a monotonically decreasing function of the current diversity metric Size of feasible values of the jth variable
  27. 27. Discrete Variables and Constraints Handling • The Nearest Vertex Approach (NVA) is used to deal with discrete variables, where a local hypercube is defined as 𝑯 = 𝑥 𝐿1, 𝑥 𝑈1 , 𝑥 𝐿2, 𝑥 𝑈2 , … , 𝑥 𝐿 𝑚, 𝑥 𝑈 𝑚 and 𝑥 𝐿 𝑗 < 𝑥 𝑗 < 𝑥 𝑈 𝑗, ∀𝑗 = 1,2, … , 𝑚 • The net constraint is used to handle constraints, as given by 𝑓𝑐 = 𝑝=1 𝑃 max 𝑔 𝑞, 0 + 𝑞=1 𝑄 max ℎ 𝑞 − 𝜖, 0 27 Nearest Vertex Approach Normalized inequality constraints Normalized equality constraints Tolerance
  28. 28. Coello Fonseca 2 f1 f2 0 0.2 0.4 0.6 0.8 1 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 1.2 Pareto solution by MO-MDPSO Actual Pareto solution f1 f2 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 Pareto solution by MO-MDPSO Actual Pareto solution Continuous Unconstrained Test Problems 28 Schaffer 1 Schaffer 2 f1 f2 0 1 2 3 4 0 1 2 3 4 Pareto solution by MO-MDPSO Actual Pareto solution f1 f2 -1 -0.5 0 0.5 1 0 5 10 15 20 Pareto solution by MO-MDPSO Actual Pareto solution Number of function evaluations for these problems is 2,000

×