Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Effects of population initialization on differential evolution for large scale optimization

1,000 views

Published on

This work provides an in-depth investigation of the effects of population initialization on Differential Evolution (DE) for dealing with large scale optimization problems. Firstly, we conduct a statistical parameter sensitive analysis to study the effects of DE’s control parameters on its performance of solving large scale problems. This study reveals the optimal parame- ter configurations which can lead to the statistically superior performance over the CEC-2013 large-scale test problems. Thus identified optimal parameter configurations interestingly favour much larger population sizes while agreeing with the other parameter settings compared to the most commonly employed parameter configuration. Based on one of the identified optimal configurations and the most commonly used configuration, which only differ in the population size, we investigate the influence of various population initialization techniques on DE’s performance. This study indicates that initialization plays a more crucial role in DE with a smaller population size. However, this observation might be the result of insufficient convergence due to the use of a large population size under the limited computational budget, which deserve more investigations.

Published in: Science
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Effects of population initialization on differential evolution for large scale optimization

  1. 1. Effects of Population Initialization on Differential Evolution for Large-Scale Optimization Borhan Kazimipour Xiaodong Li A.K. Qin
  2. 2. Outlines 1. Introduction 2. Background 3. Experiments 4. Future Work 5. Questions CEC 2014, Beijing, China 2Population Initialization on DE for LSO
  3. 3. Outlines 1. Introduction 2. Background 3. Experiments 4. Future Work 5. Questions CEC 2014, Beijing, China 3Population Initialization on DE for LSO
  4. 4. What is it all about? CEC 2014, Beijing, China 4Population Initialization on DE for LSO Investigating the effect of advanced population initialization techniques on DE for large scale Large scale is not discovered Initialization is important DE is powerful
  5. 5. What is it all about? • What is missing in previous works? – Lack: Large scale optimization has received little attention on this topic. – Ambiguity: Advanced statistical tools have not been employed to validate the significance of improvement. – Contradiction: Some claims appose others CEC 2014, Beijing, China 5Population Initialization on DE for LSO
  6. 6. What is it all about? CEC 2014, Beijing, China 6Population Initialization on DE for LSO Contribution Dimensionality Advanced Statistical Tests Parameter Configuration
  7. 7. What is it all about? • Basic Facts – Differential Evolution (DE) is one the most promising optimizers and winner of many optimization competitions. – Many researches have claimed that adopting advanced population initialization techniques improve Evolutionary Algorithms (EAs), including DE. – Those claims have not been deeply studied in Large Scale Optimization (LSO) domain. • The Goal – This research aims to study the effect of advanced population initialization techniques on a widely used DE variant when it comes to deal with large scale problems considering the effects of DE parameter setting. CEC 2014, Beijing, China 7Population Initialization on DE for LSO
  8. 8. Outlines 1. Introduction 2. Background 3. Experiments 4. Future Work 5. Questions CEC 2014, Beijing, China 8Population Initialization on DE for LSO
  9. 9. Definitions • CEC 2014, Beijing, China 9Population Initialization on DE for LSO
  10. 10. Differential Evolution (DE) CEC 2014, Beijing, China 10Population Initialization on DE for LSO + Effective: the winner of many competitions + Popular: with many publications and applications - Population-based algorithm: Sensitive to initial population - Has parameters to control exploration-exploitation balance: Sensitive to parameter setting - Suffers from Curse of Dimensionality
  11. 11. Process and Operators • DE Operators 0. Initialization 1. Mutation 2. Recombination 3. Selection CEC 2014, Beijing, China 11Population Initialization on DE for LSO
  12. 12. 0- Population Initialization • Common Parameters – Number of decision variables or problem dimensionality (given) – Variable ranges (given) – Population size • Technique-Specific Parameters – Chaotic Number Generators: map type and number of iterations – Uniform Design: number of levels – Opposition-Based Learning: original population initializer – … CEC 2014, Beijing, China 12Population Initialization on DE for LSO
  13. 13. Categorize of Population Initialization CEC 2014, Beijing, China 13Population Initialization on DE for LSO Population Initialization Randomness Stochastic Pseudo-Random Number Generators Chaotic Number Generator Deterministic Quasi-Random Sequence Uniform Experimental Design Composition ality Non-Composite Composite Hybrid Multi-Step Generality Generic Application Specific
  14. 14. Categorize of Population Initialization CEC 2014, Beijing, China 14Population Initialization on DE for LSO Population Initialization Randomness Stochastic Pseudo-Random Number Generators Chaotic Number Generator Deterministic Quasi-Random Sequence Uniform Experimental Design Composition ality Non-Composite Composite Hybrid Multi-Step Generality Generic Application Specific
  15. 15. 1- Mutation • A DE Mutation Strategy (rand/1) – r1 and r2 are randomly chosen from population – F is scaling factor • Scaling Factor (F) – Controls exploration-exploitation balance – Too small F values increase the risk of premature convergence – Too large F values decrease the convergence speed, degrades efficiency and may result in early termination CEC 2014, Beijing, China 15Population Initialization on DE for LSO
  16. 16. 2- Recombination • Binomial Crossover • Crossover Rate (CR) – CR determines the number of variables of target vector which must be interchanged with the corresponding variables of mutant vector – Small CR values can boost convergence speed when a few decision variables are interacting with each others (separable functions) – Large CR values are more effective when lots of decision variables are interacting (non-separable functions). In dealing with black-box problems, we have no idea about the separability of the objective function. CEC 2014, Beijing, China 16Population Initialization on DE for LSO
  17. 17. 3- Selection • Elite Selection CEC 2014, Beijing, China 17Population Initialization on DE for LSO
  18. 18. 3- Selection • Elite Selection CEC 2014, Beijing, China 18Population Initialization on DE for LSO Yes! No more parameters!
  19. 19. Differential Evolution (DE) • Important Parameters – NP: Population Size – CR: Crossover Rate – F: Scale Factor CEC 2014, Beijing, China 19Population Initialization on DE for LSO
  20. 20. Differential Evolution (DE) • Important Parameters – NP: Population Size – CR: Crossover Rate – F: Scale Factor CEC 2014, Beijing, China 20Population Initialization on DE for LSO Population Initialization Technique
  21. 21. Outlines 1. Introduction 2. Background 3. Experiments 4. Future Work 5. Questions CEC 2014, Beijing, China 21Population Initialization on DE for LSO
  22. 22. Experiments • Two- parts Experiment: A.Parameter Calibration – Aim: to find the most effective parameter configuration for DE/rand/1/bin to deal with large scale problems B.Population Initialization – Aim: to investigate whether advanced population initialization techniques can improve common techniques using the most effective parameter setting. CEC 2014, Beijing, China 22Population Initialization on DE for LSO
  23. 23. Experiments Setup Parts A & B • Benchmark – CEC 2013 LSGO Benchmarks – 15 functions – 1000 dimensions – Categories 1. fully separable functions (f1 - f3), 2. partially separable functions with a separable subcomponent (f4 - f7), 3. partially separable functions with no separable subcomponents (f8 - f11), 4. overlapping functions (f12 - f14), 5. fully non-separable function (f15). • Statistical Tests – Iman and Davenport (a.k.a. Friedman rank) test is used for ranking – Li post-hoc procedure is used as significance test CEC 2014, Beijing, China 23Population Initialization on DE for LSO
  24. 24. Experiments Part A A. Parameter Calibration – PRNG as population initializer – 14 population sizes [10,20,30,40,50,60,70,80,90,100,150,200,250,300] – 3 CR values [0.1, 0.5, 0.9] – 2 F values [0.5, 0.8] – 84 configurations X 15 functions X 51 runs CEC 2014, Beijing, China 24Population Initialization on DE for LSO
  25. 25. Experiments Results Part A • Range of parameter values which perform significantly better than the others based on Li post-hoc procedure – NP = [80,90,100,150,200,250] – CR = 0.9 – F = 0.5 CEC 2014, Beijing, China 25Population Initialization on DE for LSO
  26. 26. Experiments Results Part A • Among 84 configurations (on 15 functions in 51 runs), the best configuration based on Iman and Davenport test is found to be: – NP = 150 – CR = 0.9 – F = 0.5 CEC 2014, Beijing, China 26Population Initialization on DE for LSO
  27. 27. Experiments Results Part A CC: 3,000,000 FE = 50 NP X 60,000 iterations CS: 3,000,000 FE = 150 NP X 20,000 iterations CEC 2014, Beijing, China 27Population Initialization on DE for LSO
  28. 28. Experiments Results Part A • What we learn from Part A? – NP, CR an F must be set carefully. – NP is more relaxed than CR and F values. – Most effective values for CR and F are the same in low and high dimensional problems – Higher dimension problems demand larger populations (even if computational budget is fixed.) – Note: The findings are based on the dedicated computational budget; Large increment or decrement of this limit may affect the results. CEC 2014, Beijing, China 28Population Initialization on DE for LSO
  29. 29. Experiments Part B A. Parameter Calibration – PRNG as population initializer – 14 population sizes [10,20,30,40,50,60,70,80,90,100,150,200,250,300] – 3 CR values [0.1, 0.5, 0.9] – 2 F values [0.5, 0.8] – 84 configurations X 15 functions X 51 runs B. Population Initialization Techniques – Pseudo-Random Number Generators (PRNG) – Chaotic Number Generator (CNG) – Sobol Ste (SBL) – Good Lattice Point (GLP) – Opposition-Based Learning (OBL) – Quasi-Opposition-Based Learning (QOBL) CEC 2014, Beijing, China 29Population Initialization on DE for LSO Using the best configuration found in Part A
  30. 30. Experiments Results Part B No significant improvement is visible CEC 2014, Beijing, China 30Population Initialization on DE for LSO
  31. 31. Experiments Results Part B • What we learn from Part B? – Advanced population initializers may improve DE/rand/1/bin, but not significantly. – When proper values for the control parameters are used, population initialization has only a minor effect. – Size of population plays more important role than the way it is initialized. – Note: The findings are based on the dedicated computational budget; Large increment or decrement of this limit may affect the results. CEC 2014, Beijing, China 31Population Initialization on DE for LSO
  32. 32. Discussions • Important finding: – Obtained results challenges the general belief of significant advantages of advanced techniques in high dimensional spaces. • How we discuss the contradiction? 1. Significant effect of parameters: None of the previous studies has tried to compare population initialization techniques on the well-tuned optimizers. 2. Importance of advanced statistical tools: Some statistically minor improvements of using advanced initializers may wrongly considered as significant contributions. Note: This study is well-conducted based on a systematic framework and the findings are statistically validated. However, the we are well aware of the need of further investigations to generalise the findings from DE/rand/1/bin to other EAs. CEC 2014, Beijing, China 32Population Initialization on DE for LSO
  33. 33. Outlines 1. Introduction 2. Background 3. Experiments 4. Future Work 5. Questions CEC 2014, Beijing, China 33Population Initialization on DE for LSO
  34. 34. Future Work • Expansion to other EAs: – Repeating this study on other popular EAs for further generalization of the findings. • Involving other metrics: – Considering other performance metrics besides final objective values can help us to investigate whether advanced initializers are able to significantly improve EAs according to other aspects • Effect of budget – computational budget has significant effects on the performance of EAs when armed with different population initialization techniques. CEC 2014, Beijing, China 34Population Initialization on DE for LSO
  35. 35. Outlines 1. Introduction 2. Background 3. Experiments 4. Future Work 5. Questions CEC 2014, Beijing, China 35Population Initialization on DE for LSO
  36. 36. Thank you ☺☺☺☺ Any question or comment? 36CEC 2014, Beijing, China Population Initialization on DE for LSO

×