This document analyzes metaheuristic optimization algorithms and discusses open problems in their analysis. It reviews convergence analyses that have been done for simulated annealing and particle swarm optimization. It also provides a novel convergence analysis for the firefly algorithm, showing that it can converge for certain parameter values but also exhibit chaos which can be advantageous for exploration. The document outlines the need for further mathematical analysis of convergence and efficiency in metaheuristics.
COMPUTATIONAL COMPLEXITY COMPARISON OF MULTI-SENSOR SINGLE TARGET DATA FUSION...ijccmsjournal
Target tracking using observations from multiple sensors can achieve better estimation performance than a single sensor. The most famous estimation tool in target tracking is Kalman filter. There are several mathematical approaches to combine the observations of multiple sensors by use of Kalman filter. An
important issue in applying a proper approach is computational complexity. In this paper, four data fusion algorithms based on Kalman filter are considered including three centralized and one decentralized methods. Using MATLAB, computational loads of these methods are compared while number of sensors
increases. The results show that inverse covariance method has the best computational performance if the number of sensors is above 20. For a smaller number of sensors, other methods, especially group sensors, are more appropriate..
Recent research in finding the optimal path by ant colony optimizationjournalBEEI
The computation of the optimal path is one of the critical problems in graph theory. It has been utilized in various practical ranges of real world applications including image processing, file carving and classification problem. Numerous techniques have been proposed in finding optimal path solutions including using ant colony optimization (ACO). This is a nature-inspired metaheuristic algorithm, which is inspired by the foraging behavior of ants in nature. Thus, this paper study the improvement made by many researchers on ACO in finding optimal path solution. Finally, this paper also identifies the recent trends and explores potential future research directions in file carving.
Chaos Suppression and Stabilization of Generalized Liu Chaotic Control Systemijtsrd
In this paper, the concept of generalized stabilization for nonlinear systems is introduced and the stabilization of the generalized Liu chaotic control system is explored. Based on the time-domain approach with differential inequalities, a suitable control is presented such that the generalized stabilization for a class of Liu chaotic system can be achieved. Meanwhile, not only the guaranteed exponential convergence rate can be arbitrarily pre-specified but also the critical time can be correctly estimated. Finally, some numerical simulations are given to demonstrate the feasibility and effectiveness of the obtained results. Yeong-Jeu Sun | Jer-Guang Hsieh "Chaos Suppression and Stabilization of Generalized Liu Chaotic Control System" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-3 | Issue-1 , December 2018, URL: http://www.ijtsrd.com/papers/ijtsrd20195.pdf
http://www.ijtsrd.com/engineering/electrical-engineering/20195/chaos-suppression-and-stabilization-of-generalized-liu-chaotic-control-system/yeong-jeu-sun
Parameter Estimation for the Exponential distribution model Using Least-Squar...IJMERJOURNAL
Abstract: We find parameter estimates of the Exponential distribution models using leastsquares estimation method for the case when partial derivatives were not available, the Nelder and Meads, and Hooke and Jeeves optimization methodswere used and for the case when first partial derivatives are available, the Quasi – Newton Method (Davidon-Fletcher-Powel (DFP) and the Broyden-Fletcher-Goldfarb-Shanno (BFGS) optimization methods)were applied. The medical data sets of 21Leukemiacancer patients with time span of 35 weeks ([3],[6]) were used.
COMPUTATIONAL COMPLEXITY COMPARISON OF MULTI-SENSOR SINGLE TARGET DATA FUSION...ijccmsjournal
Target tracking using observations from multiple sensors can achieve better estimation performance than a single sensor. The most famous estimation tool in target tracking is Kalman filter. There are several mathematical approaches to combine the observations of multiple sensors by use of Kalman filter. An
important issue in applying a proper approach is computational complexity. In this paper, four data fusion algorithms based on Kalman filter are considered including three centralized and one decentralized methods. Using MATLAB, computational loads of these methods are compared while number of sensors
increases. The results show that inverse covariance method has the best computational performance if the number of sensors is above 20. For a smaller number of sensors, other methods, especially group sensors, are more appropriate..
Recent research in finding the optimal path by ant colony optimizationjournalBEEI
The computation of the optimal path is one of the critical problems in graph theory. It has been utilized in various practical ranges of real world applications including image processing, file carving and classification problem. Numerous techniques have been proposed in finding optimal path solutions including using ant colony optimization (ACO). This is a nature-inspired metaheuristic algorithm, which is inspired by the foraging behavior of ants in nature. Thus, this paper study the improvement made by many researchers on ACO in finding optimal path solution. Finally, this paper also identifies the recent trends and explores potential future research directions in file carving.
Chaos Suppression and Stabilization of Generalized Liu Chaotic Control Systemijtsrd
In this paper, the concept of generalized stabilization for nonlinear systems is introduced and the stabilization of the generalized Liu chaotic control system is explored. Based on the time-domain approach with differential inequalities, a suitable control is presented such that the generalized stabilization for a class of Liu chaotic system can be achieved. Meanwhile, not only the guaranteed exponential convergence rate can be arbitrarily pre-specified but also the critical time can be correctly estimated. Finally, some numerical simulations are given to demonstrate the feasibility and effectiveness of the obtained results. Yeong-Jeu Sun | Jer-Guang Hsieh "Chaos Suppression and Stabilization of Generalized Liu Chaotic Control System" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-3 | Issue-1 , December 2018, URL: http://www.ijtsrd.com/papers/ijtsrd20195.pdf
http://www.ijtsrd.com/engineering/electrical-engineering/20195/chaos-suppression-and-stabilization-of-generalized-liu-chaotic-control-system/yeong-jeu-sun
Parameter Estimation for the Exponential distribution model Using Least-Squar...IJMERJOURNAL
Abstract: We find parameter estimates of the Exponential distribution models using leastsquares estimation method for the case when partial derivatives were not available, the Nelder and Meads, and Hooke and Jeeves optimization methodswere used and for the case when first partial derivatives are available, the Quasi – Newton Method (Davidon-Fletcher-Powel (DFP) and the Broyden-Fletcher-Goldfarb-Shanno (BFGS) optimization methods)were applied. The medical data sets of 21Leukemiacancer patients with time span of 35 weeks ([3],[6]) were used.
This research paper is a statistical comparative study of a few average case asymptotically optimal sorting algorithms namely, Quick sort, Heap sort and K- sort. The three sorting algorithms all with the same average case complexity have been compared by obtaining the corresponding statistical bounds while subjecting these procedures over the randomly generated data from some standard discrete and continuous
probability distributions such as Binomial distribution, Uniform discrete and continuous distribution and Poisson distribution. The statistical analysis is well supplemented by the parameterized complexity analysis.
COMPUTATIONAL PERFORMANCE OF QUANTUM PHASE ESTIMATION ALGORITHMcsitconf
A quantum computation problem is discussed in this paper. Many new features that make
quantum computation superior to classical computation can be attributed to quantum coherence
effect, which depends on the phase of quantum coherent state. Quantum Fourier transform
algorithm, the most commonly used algorithm, is introduced. And one of its most important
applications, phase estimation of quantum state based on quantum Fourier transform, is
presented in details. The flow of phase estimation algorithm and the quantum circuit model are
shown. And the error of the output phase value, as well as the probability of measurement, is
analysed. The probability distribution of the measuring result of phase value is presented and
the computational efficiency is discussed.
COMPUTATIONAL PERFORMANCE OF QUANTUM PHASE ESTIMATION ALGORITHMcscpconf
A quantum computation problem is discussed in this paper. Many new features that make quantum computation superior to classical computation can be attributed to quantum coherence
effect, which depends on the phase of quantum coherent state. Quantum Fourier transform algorithm, the most commonly used algorithm, is introduced. And one of its most important
applications, phase estimation of quantum state based on quantum Fourier transform, is presented in details. The flow of phase estimation algorithm and the quantum circuit model are
shown. And the error of the output phase value, as well as the probability of measurement, is analysed. The probability distribution of the measuring result of phase value is presented and the computational efficiency is discussed.
Adaptive Projective Lag Synchronization of T and Lu Chaotic Systems IJECEIAES
In this paper, the synchronization problem of T chaotic system and Lu chaotic system is studied. The parameter of the drive T chaotic system is considered unknown. An adaptive projective lag control method and also parameter estimation law are designed to achieve chaos synchronization problem between two chaotic systems. Then Lyapunov stability theorem is utilized to prove the validity of the proposed control method. After that, some numerical simulations are performed to assess the performance of the proposed method. The results show high accuracy of the proposed method in control and synchronization of chaotic systems.
Applied Mathematics and Sciences: An International Journal (MathSJ)mathsjournal
The main goal of this research is to give the complete conception about numerical integration including
Newton-Cotes formulas and aimed at comparing the rate of performance or the rate of accuracy of
Trapezoidal, Simpson’s 1/3, and Simpson’s 3/8. To verify the accuracy, we compare each rules
demonstrating the smallest error values among them. The software package MATLAB R2013a is applied to
determine the best method, as well as the results, are compared. It includes graphical comparisons
mentioning these methods graphically. After all, it is then emphasized that the among methods considered,
Simpson’s 1/3 is more effective and accurate when the condition of the subdivision is only even for solving
a definite integral.
A Mathematical Programming Approach for Selection of Variables in Cluster Ana...IJRES Journal
Data clustering is a common technique for statistical data analysis; it is defined as a class of
statistical techniques for classifying a set of observations into completely different groups. Cluster analysis
seeks to minimize group variance and maximize between group variance. In this study we formulate a
mathematical programming model that chooses the most important variables in cluster analysis. A nonlinear
binary model is suggested to select the most important variables in clustering a set of data. The idea of the
suggested model depends on clustering data by minimizing the distance between observations within groups.
Indicator variables are used to select the most important variables in the cluster analysis.
THE ACTIVE CONTROLLER DESIGN FOR ACHIEVING GENERALIZED PROJECTIVE SYNCHRONIZA...ijait
This paper discusses the design of active controllers for achieving generalized projective synchronization (GPS) of identical hyperchaotic Lü systems (Chen, Lu, Lü and Yu, 2006), identical hyperchaotic Cai systems (Wang and Cai, 2009) and non-identical hyperchaotic Lü and hyperchaotic Cai systems. The synchronization results (GPS) for the hyperchaotic systems have been derived using active control method and established using Lyapunov stability theory. Since the Lyapunov exponents are not required for these calculations, the active control method is very effective and convenient for achieving the GPS of the
hyperchaotic systems addressed in this paper. Numerical simulations are provided to illustrate the effectiveness of the GPS synchronization results derived in this paper.
A brief talk on reservoir computing from the perspective of dynamical system. Mostly based on these 2 papers:
1. Pathak, J., Hunt, B., Girvan, M., Lu, Z., & Ott, E. (2018). Model-free prediction of large spatiotemporally chaotic systems from data: A reservoir computing approach. Physical review letters, 120(2), 024102.
2. A Parsimonious Dynamical Model for Structural Learning in the Human Brain. arXiv preprint arXiv:1807.05214.
LINEAR SEARCH VERSUS BINARY SEARCH: A STATISTICAL COMPARISON FOR BINOMIAL INPUTSIJCSEA Journal
For certain algorithms such as sorting and searching, the parameters of the input probability distribution,in addition to the size of the input, have been found to influence the complexity of the underlying algorithm.The present paper makes a statistical comparative study on parameterized complexity between linear and binary search algorithms for binomial inputs.
Design of State Estimator for a Class of Generalized Chaotic Systemsijtsrd
In this paper, a class of generalized chaotic systems is considered and the state observation problem of such a system is investigated. Based on the time domain approach with differential inequality, a simple state estimator for such generalized chaotic systems is developed to guarantee the global exponential stability of the resulting error system. Besides, the guaranteed exponential decay rate can be correctly estimated. Finally, several numerical simulations are given to show the effectiveness of the obtained result. Yeong-Jeu Sun "Design of State Estimator for a Class of Generalized Chaotic Systems" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-3 | Issue-6 , October 2019, URL: https://www.ijtsrd.com/papers/ijtsrd29270.pdf Paper URL: https://www.ijtsrd.com/engineering/electrical-engineering/29270/design-of-state-estimator-for-a-class-of-generalized-chaotic-systems/yeong-jeu-sun
The International Journal of Engineering and Science (The IJES)theijes
The International Journal of Engineering & Science is aimed at providing a platform for researchers, engineers, scientists, or educators to publish their original research results, to exchange new ideas, to disseminate information in innovative designs, engineering experiences and technological skills. It is also the Journal's objective to promote engineering and technology education. All papers submitted to the Journal will be blind peer-reviewed. Only original articles will be published.
Proposing a New Job Scheduling Algorithm in Grid Environment Using a Combinat...Editor IJCATR
Grid computing is a hardware and software infrastructure and provides affordable, sustainable, and reliable access. Its aim is
to create a supercomputer using free resources. One of the challenges to the Grid computing is scheduling problem which is regarded
as a tough issue. Since scheduling problem is a non-deterministic issue in the Grid, deterministic algorithms cannot be used to improve
scheduling. In this paper, a combination of imperialist competition algorithm (ICA) and gravitational attraction is used for to address the
problem of independent task scheduling in a grid environment, with the aim of reducing the makespan and energy. Experimental results
compare ICA with other algorithms and illustrate that ICA finds a shorter makespan and energy relative to the others. Moreover, it
converges quickly, finding its optimum solution in less time than the other algorithms.
Foundation and Synchronization of the Dynamic Output Dual Systemsijtsrd
In this paper, the synchronization problem of the dynamic output dual systems is firstly introduced and investigated. Based on the time domain approach, the state variables synchronization of such dual systems can be verified. Meanwhile, the guaranteed exponential convergence rate can be accurately estimated. Finally, some numerical simulations are provided to illustrate the feasibility and effectiveness of the obtained result. Yeong-Jeu Sun "Foundation and Synchronization of the Dynamic Output Dual Systems" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-3 | Issue-6 , October 2019, URL: https://www.ijtsrd.com/papers/ijtsrd29256.pdf Paper URL: https://www.ijtsrd.com/engineering/electrical-engineering/29256/foundation-and-synchronization-of-the-dynamic-output-dual-systems/yeong-jeu-sun
Modified Projective Synchronization of Chaotic Systems with Noise Disturbance,...IJECEIAES
The synchronization problem of chaotic systems using active modified projective non- linear control method is rarely addressed. Thus the concentration of this study is to derive a modified projective controller to synchronize the two chaotic systems. Since, the parameter of the master and follower systems are considered known, so active methods are employed instead of adaptive methods. The validity of the proposed controller is studied by means of the Lyapunov stability theorem. Furthermore, some numerical simulations are shown to verify the validity of the theoretical discussions. The results demonstrate the effectiveness of the proposed method in both speed and accuracy points of views.
This research paper is a statistical comparative study of a few average case asymptotically optimal sorting algorithms namely, Quick sort, Heap sort and K- sort. The three sorting algorithms all with the same average case complexity have been compared by obtaining the corresponding statistical bounds while subjecting these procedures over the randomly generated data from some standard discrete and continuous
probability distributions such as Binomial distribution, Uniform discrete and continuous distribution and Poisson distribution. The statistical analysis is well supplemented by the parameterized complexity analysis.
COMPUTATIONAL PERFORMANCE OF QUANTUM PHASE ESTIMATION ALGORITHMcsitconf
A quantum computation problem is discussed in this paper. Many new features that make
quantum computation superior to classical computation can be attributed to quantum coherence
effect, which depends on the phase of quantum coherent state. Quantum Fourier transform
algorithm, the most commonly used algorithm, is introduced. And one of its most important
applications, phase estimation of quantum state based on quantum Fourier transform, is
presented in details. The flow of phase estimation algorithm and the quantum circuit model are
shown. And the error of the output phase value, as well as the probability of measurement, is
analysed. The probability distribution of the measuring result of phase value is presented and
the computational efficiency is discussed.
COMPUTATIONAL PERFORMANCE OF QUANTUM PHASE ESTIMATION ALGORITHMcscpconf
A quantum computation problem is discussed in this paper. Many new features that make quantum computation superior to classical computation can be attributed to quantum coherence
effect, which depends on the phase of quantum coherent state. Quantum Fourier transform algorithm, the most commonly used algorithm, is introduced. And one of its most important
applications, phase estimation of quantum state based on quantum Fourier transform, is presented in details. The flow of phase estimation algorithm and the quantum circuit model are
shown. And the error of the output phase value, as well as the probability of measurement, is analysed. The probability distribution of the measuring result of phase value is presented and the computational efficiency is discussed.
Adaptive Projective Lag Synchronization of T and Lu Chaotic Systems IJECEIAES
In this paper, the synchronization problem of T chaotic system and Lu chaotic system is studied. The parameter of the drive T chaotic system is considered unknown. An adaptive projective lag control method and also parameter estimation law are designed to achieve chaos synchronization problem between two chaotic systems. Then Lyapunov stability theorem is utilized to prove the validity of the proposed control method. After that, some numerical simulations are performed to assess the performance of the proposed method. The results show high accuracy of the proposed method in control and synchronization of chaotic systems.
Applied Mathematics and Sciences: An International Journal (MathSJ)mathsjournal
The main goal of this research is to give the complete conception about numerical integration including
Newton-Cotes formulas and aimed at comparing the rate of performance or the rate of accuracy of
Trapezoidal, Simpson’s 1/3, and Simpson’s 3/8. To verify the accuracy, we compare each rules
demonstrating the smallest error values among them. The software package MATLAB R2013a is applied to
determine the best method, as well as the results, are compared. It includes graphical comparisons
mentioning these methods graphically. After all, it is then emphasized that the among methods considered,
Simpson’s 1/3 is more effective and accurate when the condition of the subdivision is only even for solving
a definite integral.
A Mathematical Programming Approach for Selection of Variables in Cluster Ana...IJRES Journal
Data clustering is a common technique for statistical data analysis; it is defined as a class of
statistical techniques for classifying a set of observations into completely different groups. Cluster analysis
seeks to minimize group variance and maximize between group variance. In this study we formulate a
mathematical programming model that chooses the most important variables in cluster analysis. A nonlinear
binary model is suggested to select the most important variables in clustering a set of data. The idea of the
suggested model depends on clustering data by minimizing the distance between observations within groups.
Indicator variables are used to select the most important variables in the cluster analysis.
THE ACTIVE CONTROLLER DESIGN FOR ACHIEVING GENERALIZED PROJECTIVE SYNCHRONIZA...ijait
This paper discusses the design of active controllers for achieving generalized projective synchronization (GPS) of identical hyperchaotic Lü systems (Chen, Lu, Lü and Yu, 2006), identical hyperchaotic Cai systems (Wang and Cai, 2009) and non-identical hyperchaotic Lü and hyperchaotic Cai systems. The synchronization results (GPS) for the hyperchaotic systems have been derived using active control method and established using Lyapunov stability theory. Since the Lyapunov exponents are not required for these calculations, the active control method is very effective and convenient for achieving the GPS of the
hyperchaotic systems addressed in this paper. Numerical simulations are provided to illustrate the effectiveness of the GPS synchronization results derived in this paper.
A brief talk on reservoir computing from the perspective of dynamical system. Mostly based on these 2 papers:
1. Pathak, J., Hunt, B., Girvan, M., Lu, Z., & Ott, E. (2018). Model-free prediction of large spatiotemporally chaotic systems from data: A reservoir computing approach. Physical review letters, 120(2), 024102.
2. A Parsimonious Dynamical Model for Structural Learning in the Human Brain. arXiv preprint arXiv:1807.05214.
LINEAR SEARCH VERSUS BINARY SEARCH: A STATISTICAL COMPARISON FOR BINOMIAL INPUTSIJCSEA Journal
For certain algorithms such as sorting and searching, the parameters of the input probability distribution,in addition to the size of the input, have been found to influence the complexity of the underlying algorithm.The present paper makes a statistical comparative study on parameterized complexity between linear and binary search algorithms for binomial inputs.
Design of State Estimator for a Class of Generalized Chaotic Systemsijtsrd
In this paper, a class of generalized chaotic systems is considered and the state observation problem of such a system is investigated. Based on the time domain approach with differential inequality, a simple state estimator for such generalized chaotic systems is developed to guarantee the global exponential stability of the resulting error system. Besides, the guaranteed exponential decay rate can be correctly estimated. Finally, several numerical simulations are given to show the effectiveness of the obtained result. Yeong-Jeu Sun "Design of State Estimator for a Class of Generalized Chaotic Systems" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-3 | Issue-6 , October 2019, URL: https://www.ijtsrd.com/papers/ijtsrd29270.pdf Paper URL: https://www.ijtsrd.com/engineering/electrical-engineering/29270/design-of-state-estimator-for-a-class-of-generalized-chaotic-systems/yeong-jeu-sun
The International Journal of Engineering and Science (The IJES)theijes
The International Journal of Engineering & Science is aimed at providing a platform for researchers, engineers, scientists, or educators to publish their original research results, to exchange new ideas, to disseminate information in innovative designs, engineering experiences and technological skills. It is also the Journal's objective to promote engineering and technology education. All papers submitted to the Journal will be blind peer-reviewed. Only original articles will be published.
Proposing a New Job Scheduling Algorithm in Grid Environment Using a Combinat...Editor IJCATR
Grid computing is a hardware and software infrastructure and provides affordable, sustainable, and reliable access. Its aim is
to create a supercomputer using free resources. One of the challenges to the Grid computing is scheduling problem which is regarded
as a tough issue. Since scheduling problem is a non-deterministic issue in the Grid, deterministic algorithms cannot be used to improve
scheduling. In this paper, a combination of imperialist competition algorithm (ICA) and gravitational attraction is used for to address the
problem of independent task scheduling in a grid environment, with the aim of reducing the makespan and energy. Experimental results
compare ICA with other algorithms and illustrate that ICA finds a shorter makespan and energy relative to the others. Moreover, it
converges quickly, finding its optimum solution in less time than the other algorithms.
Foundation and Synchronization of the Dynamic Output Dual Systemsijtsrd
In this paper, the synchronization problem of the dynamic output dual systems is firstly introduced and investigated. Based on the time domain approach, the state variables synchronization of such dual systems can be verified. Meanwhile, the guaranteed exponential convergence rate can be accurately estimated. Finally, some numerical simulations are provided to illustrate the feasibility and effectiveness of the obtained result. Yeong-Jeu Sun "Foundation and Synchronization of the Dynamic Output Dual Systems" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-3 | Issue-6 , October 2019, URL: https://www.ijtsrd.com/papers/ijtsrd29256.pdf Paper URL: https://www.ijtsrd.com/engineering/electrical-engineering/29256/foundation-and-synchronization-of-the-dynamic-output-dual-systems/yeong-jeu-sun
Modified Projective Synchronization of Chaotic Systems with Noise Disturbance,...IJECEIAES
The synchronization problem of chaotic systems using active modified projective non- linear control method is rarely addressed. Thus the concentration of this study is to derive a modified projective controller to synchronize the two chaotic systems. Since, the parameter of the master and follower systems are considered known, so active methods are employed instead of adaptive methods. The validity of the proposed controller is studied by means of the Lyapunov stability theorem. Furthermore, some numerical simulations are shown to verify the validity of the theoretical discussions. The results demonstrate the effectiveness of the proposed method in both speed and accuracy points of views.
Multi objective predictive control a solution using metaheuristicsijcsit
The application of multi objective model predictive control approaches is significantly limited with
computation time associated with optimization algorithms. Metaheuristics are general purpose heuristics
that have been successfully used in solving difficult optimization problems in a reasonable computation
time. In this work , we use and compare two multi objective metaheuristics, Multi-Objective Particle
swarm Optimization, MOPSO, and Multi-Objective Gravitational Search Algorithm, MOGSA, to generate
a set of approximately Pareto-optimal solutions in a single run. Two examples are studied, a nonlinear
system consisting of two mobile robots tracking trajectories and avoiding obstacles and a linear multi
variable system. The computation times and the quality of the solution in terms of the smoothness of the
control signals and precision of tracking show that MOPSO can be an alternative for real time
applications.
A COMPREHENSIVE ANALYSIS OF QUANTUM CLUSTERING : FINDING ALL THE POTENTIAL MI...IJDKP
Quantum clustering (QC), is a data clustering algorithm based on quantum mechanics which is
accomplished by substituting each point in a given dataset with a Gaussian. The width of the Gaussian is a
σ value, a hyper-parameter which can be manually defined and manipulated to suit the application.
Numerical methods are used to find all the minima of the quantum potential as they correspond to cluster
centers. Herein, we investigate the mathematical task of expressing and finding all the roots of the
exponential polynomial corresponding to the minima of a two-dimensional quantum potential. This is an
outstanding task because normally such expressions are impossible to solve analytically. However, we
prove that if the points are all included in a square region of size σ, there is only one minimum. This bound
is not only useful in the number of solutions to look for, by numerical means, it allows to to propose a new
numerical approach “per block”. This technique decreases the number of particles by approximating some
groups of particles to weighted particles. These findings are not only useful to the quantum clustering
problem but also for the exponential polynomials encountered in quantum chemistry, Solid-state Physics
and other applications.
On the Fractional Optimal Control Problems With Singular and Non–Singular ...Scientific Review SR
The aim of this paper is to design an efficient numerical method to solve a class of time fractional optimal control
problems. In this problem formulation, the fractional derivative operator is consid- ered in three cases with both
singular and non–singular kernels. The necessary conditions are derived for the optimality of these problems and the
proposed method is evaluated for different choices of derivative operators. Simulation results indicate that the
suggested technique works well and pro- vides satisfactory results with considerably less computational time than
the other existing methods. Comparative results also verify that the fractional operator with Mittag –Leffler kernel in
the Caputo sense improves the performance of the controlled system in terms of the transient response compared to
the other fractional and integer derivative operators.
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
Sinc collocation linked with finite differences for Korteweg-de Vries Fraction...IJECEIAES
A novel numerical method is proposed for Korteweg-de Vries Fractional Equation. The fractional derivatives are described based on the Caputo sense. We construct the solution using different approach, that is based on using collocation techniques. The method combining a finite difference approach in the time-fractional direction, and the Sinc-Collocation in the space direction, where the derivatives are replaced by the necessary matrices, and a system of algebraic equations is obtained to approximate solution of the problem. The numerical results are shown to demonstrate the efficiency of the newly proposed method. Easy and economical implementation is the strength of this method.
Similar to Metaheuristic Optimization: Algorithm Analysis and Open Problems (20)
Cuckoo Search Algorithm: An IntroductionXin-She Yang
This presentation explains the fundamental ideas of the standard Cuckoo Search (CS) algorithm, which also contains the links to the free Matlab codes at Mathswork file exchanges and the animations of numerical simulations (video at Youtube). An example of multi-objective cuckoo search (MOCS) is also given with link to the Matlab code.
Welcome to WIPAC Monthly the magazine brought to you by the LinkedIn Group Water Industry Process Automation & Control.
In this month's edition, along with this month's industry news to celebrate the 13 years since the group was created we have articles including
A case study of the used of Advanced Process Control at the Wastewater Treatment works at Lleida in Spain
A look back on an article on smart wastewater networks in order to see how the industry has measured up in the interim around the adoption of Digital Transformation in the Water Industry.
Event Management System Vb Net Project Report.pdfKamal Acharya
In present era, the scopes of information technology growing with a very fast .We do not see any are untouched from this industry. The scope of information technology has become wider includes: Business and industry. Household Business, Communication, Education, Entertainment, Science, Medicine, Engineering, Distance Learning, Weather Forecasting. Carrier Searching and so on.
My project named “Event Management System” is software that store and maintained all events coordinated in college. It also helpful to print related reports. My project will help to record the events coordinated by faculties with their Name, Event subject, date & details in an efficient & effective ways.
In my system we have to make a system by which a user can record all events coordinated by a particular faculty. In our proposed system some more featured are added which differs it from the existing system such as security.
TECHNICAL TRAINING MANUAL GENERAL FAMILIARIZATION COURSEDuvanRamosGarzon1
AIRCRAFT GENERAL
The Single Aisle is the most advanced family aircraft in service today, with fly-by-wire flight controls.
The A318, A319, A320 and A321 are twin-engine subsonic medium range aircraft.
The family offers a choice of engines
Cosmetic shop management system project report.pdfKamal Acharya
Buying new cosmetic products is difficult. It can even be scary for those who have sensitive skin and are prone to skin trouble. The information needed to alleviate this problem is on the back of each product, but it's thought to interpret those ingredient lists unless you have a background in chemistry.
Instead of buying and hoping for the best, we can use data science to help us predict which products may be good fits for us. It includes various function programs to do the above mentioned tasks.
Data file handling has been effectively used in the program.
The automated cosmetic shop management system should deal with the automation of general workflow and administration process of the shop. The main processes of the system focus on customer's request where the system is able to search the most appropriate products and deliver it to the customers. It should help the employees to quickly identify the list of cosmetic product that have reached the minimum quantity and also keep a track of expired date for each cosmetic product. It should help the employees to find the rack number in which the product is placed.It is also Faster and more efficient way.
Vaccine management system project report documentation..pdfKamal Acharya
The Division of Vaccine and Immunization is facing increasing difficulty monitoring vaccines and other commodities distribution once they have been distributed from the national stores. With the introduction of new vaccines, more challenges have been anticipated with this additions posing serious threat to the already over strained vaccine supply chain system in Kenya.
Overview of the fundamental roles in Hydropower generation and the components involved in wider Electrical Engineering.
This paper presents the design and construction of hydroelectric dams from the hydrologist’s survey of the valley before construction, all aspects and involved disciplines, fluid dynamics, structural engineering, generation and mains frequency regulation to the very transmission of power through the network in the United Kingdom.
Author: Robbie Edward Sayers
Collaborators and co editors: Charlie Sims and Connor Healey.
(C) 2024 Robbie E. Sayers
Water scarcity is the lack of fresh water resources to meet the standard water demand. There are two type of water scarcity. One is physical. The other is economic water scarcity.
Immunizing Image Classifiers Against Localized Adversary Attacksgerogepatton
This paper addresses the vulnerability of deep learning models, particularly convolutional neural networks
(CNN)s, to adversarial attacks and presents a proactive training technique designed to counter them. We
introduce a novel volumization algorithm, which transforms 2D images into 3D volumetric representations.
When combined with 3D convolution and deep curriculum learning optimization (CLO), itsignificantly improves
the immunity of models against localized universal attacks by up to 40%. We evaluate our proposed approach
using contemporary CNN architectures and the modified Canadian Institute for Advanced Research (CIFAR-10
and CIFAR-100) and ImageNet Large Scale Visual Recognition Challenge (ILSVRC12) datasets, showcasing
accuracy improvements over previous techniques. The results indicate that the combination of the volumetric
input and curriculum learning holds significant promise for mitigating adversarial attacks without necessitating
adversary training.
Metaheuristic Optimization: Algorithm Analysis and Open Problems
1. arXiv:1212.0220v1 [math.OC] 2 Dec 2012
Metaheuristic Optimization: Algorithm Analysis and Open
Problems
Xin-She Yang
Mathematics and Scientific Computing, National Physical Laboratory,
Teddington, Middlesex TW11 0LW, UK.
Abstract
Metaheuristic algorithms are becoming an important part of modern optimization.
A wide range of metaheuristic algorithms have emerged over the last two decades, and
many metaheuristics such as particle swarm optimization are becoming increasingly
popular. Despite their popularity, mathematical analysis of these algorithms lacks be-hind.
Convergence analysis still remains unsolved for the majority of metaheuristic
algorithms, while efficiency analysis is equally challenging. In this paper, we intend to
provide an overview of convergence and efficiency studies of metaheuristics, and try to
provide a framework for analyzing metaheuristics in terms of convergence and efficiency.
This can form a basis for analyzing other algorithms. We also outline some open ques-tions
as further research topics.
Citation Details: Yang, X. S., (2011). Metaheuristic optimization: algorithm
analysis and open problems, in: Proceedings of 10th International Symposium on Ex-perimental
Algorithms (SEA 2011) (Eds. P. M. Pardalos and S. Rebennack), Kolimpari,
Chania, Greece, May 5-7 (2011), Lecture Notes in Computer Sciences, Vol. 6630, pp.
21-32.
1 Introduction
Optimization is an important subject with many important application, and algorithms
for optimization are diverse with a wide range of successful applications [10, 11]. Among
these optimization algorithms, modern metaheuristics are becoming increasingly popular,
leading to a new branch of optimization, called metaheuristic optimization. Most meta-heuristic
algorithms are nature-inspired [8, 29, 32], from simulated annealing [20] to ant
colony optimization [8], and from particle swarm optimization [17] to cuckoo search [35].
Since the appearance of swarm intelligence algorithms such as PSO in the 1990s, more than
a dozen new metaheuristic algorithms have been developed and these algorithms have been
2. applied to almost all areas of optimization, design, scheduling and planning, data mining,
machine intelligence, and many others. Thousands of research papers and dozens of books
have been published [8, 9, 11, 19, 29, 32, 33].
Despite the rapid development of metaheuristics, their mathematical analysis remains
partly unsolved, and many open problems need urgent attention. This difficulty is largely
due to the fact the interaction of various components in metaheuristic algorithms are highly
nonlinear, complex, and stochastic. Studies have attempted to carry out convergence anal-ysis
[1, 22], and some important results concerning PSO were obtained [7]. However, for
other metaheuristics such as firefly algorithms and ant colony optimization, it remains an
active, challenging topic. On the other hand, even we have not proved or cannot prove their
convergence, we still can compare the performance of various algorithms. This has indeed
formed a majority of current research in algorithm development in the research community
of optimization and machine intelligence [9, 29, 33].
In combinatorial optimization, many important developments exist on complexity anal-ysis,
run time and convergence analysis [25, 22]. For continuous optimization, no-free-lunch-theorems
do not hold [1, 2]. As a relatively young field, many open problems still remain
in the field of randomized search heuristics [1]. In practice, most assume that metaheuris-tic
algorithms tend to be less complex for implementation, and in many cases, problem
sizes are not directly linked with the algorithm complexity. However, metaheuristics can
often solve very tough NP-hard optimization, while our understanding of the efficiency and
convergence of metaheuristics lacks far behind.
Apart from the complex interactions among multiple search agents (making the math-ematical
analysis intractable), another important issue is the various randomization tech-niques
used for modern metaheuristics, from simple randomization such as uniform distri-bution
to random walks, and to more elaborate L´evy flights [5, 24, 33]. There is no unified
approach to analyze these mathematically. In this paper, we intend to review the conver-gence
of two metaheuristic algorithms including simulated annealing and PSO, followed by
the new convergence analysis of the firefly algorithm. Then, we try to formulate a framework
for algorithm analysis in terms of Markov chain Monte Carlo. We also try to analyze the
mathematical and statistical foundations for randomization techniques from simple random
walks to L´evy flights. Finally, we will discuss some of important open questions as further
research topics.
2 Convergence Analysis of Metaheuristics
The formulation and numerical studies of various metaheuristics have been the main focus
of most research studies. Many successful applications have demonstrated the efficiency of
metaheuristics in various context, either through comparison with other algorithms and/or
3. applications to well-known problems. In contrast, the mathematical analysis lacks behind,
and convergence analysis has been carried out for only a minority few algorithms such as
simulated annealing and particle swarm optimization [7, 22]. The main approach is often
for very simplified systems using dynamical theory and other ad hoc approaches. Here in
this section, we first review the simulated annealing and its convergence, and we move onto
the population-based algorithms such as PSO. We then take the recently developed firefly
algorithm as a further example to carry out its convergence analysis.
2.1 Simulated Annealing
Simulated annealing (SA) is one of the widely used metaheuristics, and is also one of the
most studies in terms of convergence analysis [4, 20]. The essence of simulated annealing
is a trajectory-based random walk of a single agent, starting from an initial guess x0. The
next move only depends on the current state or location and the acceptance probability p.
This is essentially a Markov chain whose transition probability from the current state to
the next state is given by
p = exp h −
E
kBT i, (1)
where kB is Boltzmann’s constant, and T is the temperature. Here the energy change
E can be linked with the change of objective values. A few studies on the convergence
of simulated annealing have paved the way for analysis for all simulated annealing-based
algorithms [4, 15, 27]. Bertsimas and Tsitsiklis provided an excellent review of the conver-gence
of SA under various assumptions [4, 15]. By using the assumptions that SA forms
an inhomogeneous Markov chain with finite states, they proved a probabilistic convergence
function P, rather than almost sure convergence, that
max P[xi(t) 2 S
4.
5.
6. x0]
A
t , (2)
where S is the optimal set, and A and α are positive constants [4]. This is for the cooling
schedule T(t) = d/ ln(t), where t is the iteration counter or pseudo time. These studies
largely used Markov chains as the main tool. We will come back later to a more general
framework of Markov chain Monte Carlo (MCMC) in this paper [12, 14].
2.2 PSO and Convergence
Particle swarm optimization (PSO) was developed by Kennedy and Eberhart in 1995 [17,
18], based on the swarm behaviour such as fish and bird schooling in nature. Since then, PSO
has generated much wider interests, and forms an exciting, ever-expanding research subject,
called swarm intelligence. PSO has been applied to almost every area in optimization,
computational intelligence, and design/scheduling applications.
7. The movement of a swarming particle consists of two major components: a stochastic
component and a deterministic component. Each particle is attracted toward the position
of the current global best g
and its own best location x
i in history, while at the same time
it has a tendency to move randomly.
Let xi and vi be the position vector and velocity for particle i, respectively. The new
velocity and location updating formulas are determined by
v
t+1
i = v
ti
+ αǫ1[g
− x
ti
] + βǫ2[x
i − x
ti
]. (3)
x
t+1
i = x
ti
+ v
t+1
i , (4)
where ǫ1 and ǫ2 are two random vectors, and each entry taking the values between 0 and 1.
The parameters α and β are the learning parameters or acceleration constants, which can
typically be taken as, say, α β 2.
There are at least two dozen PSO variants which extend the standard PSO algorithm,
and the most noticeable improvement is probably to use inertia function θ(t) so that v
ti
is
replaced by θ(t)v
ti
where θ 2 [0, 1] [6]. This is equivalent to introducing a virtual mass to
stabilize the motion of the particles, and thus the algorithm is expected to converge more
quickly.
The first convergence analysis of PSO was carried out by Clerc and Kennedy in 2002 [7]
using the theory of dynamical systems. Mathematically, if we ignore the random factors,
we can view the system formed by (3) and (4) as a dynamical system. If we focus on a
single particle i and imagine that there is only one particle in this system, then the global
best g
is the same as its current best x
i . In this case, we have
v
t+1
i = v
ti
+ γ(g
− x
ti
), γ = α + β, (5)
and
x
t+1
i = x
ti
+ v
t+1
i . (6)
Considering the 1D dynamical system for particle swarm optimization, we can replace g
by a parameter constant p so that we can see if or not the particle of interest will converge
towards p. By setting ut = p − x(t + 1) and using the notations for dynamical systems, we
have a simple dynamical system
vt+1 = vt + γut, ut+1 = −vt + (1 − γ)ut, (7)
or
Yt+1 = AYt, A =
1 γ
−1 1 − γ
, Yt =
vt
ut
. (8)
The general solution of this dynamical system can be written as Yt = Y0 exp[At]. The
system behaviour can be characterized by the eigenvalues λ of A, and we have λ1,2 =
8. 1 − γ/2 ± pγ2 − 4γ/2. It can be seen clearly that γ = 4 leads to a bifurcation. Following
a straightforward analysis of this dynamical system, we can have three cases. For 0
γ 4, cyclic and/or quasi-cyclic trajectories exist. In this case, when randomness is
gradually reduced, some convergence can be observed. For γ 4, non-cyclic behaviour
can be expected and the distance from Yt to the center (0, 0) is monotonically increasing
with t. In a special case γ = 4, some convergence behaviour can be observed. For detailed
analysis, please refer to Clerc and Kennedy [7]. Since p is linked with the global best, as
the iterations continue, it can be expected that all particles will aggregate towards the the
global best.
2.3 Firefly algorithm, Convergence and Chaos
Firefly Algorithm (FA) was developed by Yang [32, 34], which was based on the flashing
patterns and behaviour of fireflies. In essence, each firefly will be attracted to brighter
ones, while at the same time, it explores and searches for prey randomly. In addition, the
brightness of a firefly is determined by the landscape of the objective function.
The movement of a firefly i is attracted to another more attractive (brighter) firefly j is
determined by
x
t+1
i = x
ti
+ β0e−
r2
ij (x
t
j − x
ti
) + α ǫ
ti
, (9)
where the second term is due to the attraction. The third term is randomization with α
being the randomization parameter, and ǫ
ti
is a vector of random numbers drawn from a
Gaussian distribution or other distributions. Obviously, for a given firefly, there are often
many more attractive fireflies, then we can either go through all of them via a loop or use the
most attractive one. For multiple modal problems, using a loop while moving toward each
brighter one is usually more effective, though this will lead to a slight increase of algorithm
complexity.
Here is β0 2 [0, 1] is the attractiveness at r = 0, and rij = ||xi − xj ||2 is the ℓ2-
norm or Cartesian distance. For other problems such as scheduling, any measure that can
effectively characterize the quantities of interest in the optimization problem can be used
as the ‘distance’ r.
For most implementations, we can take β0 = 1, α = O(1) and γ = O(1). It is worth
pointing out that (9) is essentially a random walk biased towards the brighter fireflies. If
β0 = 0, it becomes a simple random walk. Furthermore, the randomization term can easily
be extended to other distributions such as L´evy flights [16, 24].
We now can carry out the convergence analysis for the firefly algorithm in a frame-work
similar to Clerc and Kennedy’s dynamical analysis. For simplicity, we start from the
equation for firefly motion without the randomness term
x
t+1
i = x
ti
+ β0e−
r2
ij (x
t
j − x
ti
). (10)
9. Figure 1: The chaotic map of the iteration formula (13) in the firefly algorithm and the
transition between from periodic/multiple states to chaos.
If we focus on a single agent, we can replace x
t
j by the global best g found so far, and we
have
x
t+1
i = x
ti
+ β0e−
r2
i (g − x
ti
), (11)
where the distance ri can be given by the ℓ2-norm r2
i = ||g − x
ti ||2
2. In an even simpler 1-D
case, we can set yt = g − x
ti
, and we have
yt+1 = yt − β0e−
y2
t yt. (12)
We can see that γ is a scaling parameter which only affects the scales/size of the firefly
movement. In fact, we can let ut = pγyt and we have
ut+1 = ut[1 − β0e−u2
t ]. (13)
These equations can be analyzed easily using the same methodology for studying the well-known
logistic map
ut+1 = λut(1 − ut). (14)
The chaotic map is shown in Fig. 1, and the focus on the transition from periodic multiple
states to chaotic behaviour is shown in the same figure.
As we can see from Fig. 1 that convergence can be achieved for β0 2. There is
a transition from periodic to chaos at β0 4. This may be surprising, as the aim of
designing a metaheuristic algorithm is to try to find the optimal solution efficiently and
accurately. However, chaotic behaviour is not necessarily a nuisance; in fact, we can use it
to the advantage of the firefly algorithm. Simple chaotic characteristics from (14) can often
be used as an efficient mixing technique for generating diverse solutions. Statistically, the
logistic mapping (14) with λ = 4 for the initial states in (0,1) corresponds a beta distribution
B(u, p, q) =
(p + q)
(p)(q)
up−1(1 − u)q−1, (15)
10. when p = q = 1/2. Here (z) is the Gamma function
(z) = Z 1
0
tz−1e−tdt. (16)
In the case when z = n is an integer, we have (n) = (n − 1)!. In addition, (1/2) = pπ.
From the algorithm implementation point of view, we can use higher attractiveness β0
during the early stage of iterations so that the fireflies can explore, even chaotically, the
search space more effectively. As the search continues and convergence approaches, we
can reduce the attractiveness β0 gradually, which may increase the overall efficiency of the
algorithm. Obviously, more studies are highly needed to confirm this.
2.4 Markov Chain Monte Carlo
From the above convergence analysis, we know that there is no mathematical framework in
general to provide insights into the working mechanisms, the stability and convergence of a
give algorithm. Despite the increasing popularity of metaheuristics, mathematical analysis
remains fragmental, and many open problems need urgent attention.
Monte Carlo methods have been applied in many applications [28], including almost
all areas of sciences and engineering. For example, Monte Carlo methods are widely used
in uncertainty and sensitivity analysis [21]. From the statistical point of view, most meta-heuristic
algorithms can be viewed in the framework of Markov chains [14, 28]. For example,
simulated annealing [20] is a Markov chain, as the next state or new solution in SA only
depends on the current state/solution and the transition probability. For a given Markov
chain with certain ergodicity, a stability probability distribution and convergence can be
achieved.
Now if look at the PSO closely using the framework of Markov chain Monte Carlo
[12, 13, 14], each particle in PSO essentially forms a Markov chain, though this Markov
chain is biased towards to the current best, as the transition probability often leads to the
acceptance of the move towards the current global best. Other population-based algorithms
can also be viewed in this framework. In essence, all metaheuristic algorithms with piece-wise,
interacting paths can be analyzed in the general framework of Markov chain Monte
Carlo. The main challenge is to realize this and to use the appropriate Markov chain theory
to study metaheuristic algorithms. More fruitful studies will surely emerge in the future.
3 Search Efficiency and Randomization
Metaheuristics can be considered as an efficient way to produce acceptable solutions by
trial and error to a complex problem in a reasonably practical time. The complexity of the
problem of interest makes it impossible to search every possible solution or combination, the
aim is to find good feasible solutions in an acceptable timescale. There is no guarantee that
11. the best solutions can be found, and we even do not know whether an algorithm will work
and why if it does work. The idea is to have an efficient but practical algorithm that will
work most the time and is able to produce good quality solutions. Among the found quality
solutions, it is expected some of them are nearly optimal, though there is no guarantee for
such optimality.
The main components of any metaheuristic algorithms are: intensification and diversi-fication,
or exploitation and exploration [5, 33]. Diversification means to generate diverse
solutions so as to explore the search space on the global scale, while intensification means
to focus on the search in a local region by exploiting the information that a current good
solution is found in this region. This is in combination with with the selection of the best
solutions.
As discussed earlier, an important component in swarm intelligence and modern meta-heuristics
is randomization, which enables an algorithm to have the ability to jump out
of any local optimum so as to search globally. Randomization can also be used for local
search around the current best if steps are limited to a local region. Fine-tuning the ran-domness
and balance of local search and global search is crucially important in controlling
the performance of any metaheuristic algorithm.
Randomization techniques can be a very simple method using uniform distributions, or
more complex methods as those used in Monte Carlo simulations [28]. They can also be
more elaborate, from Brownian random walks to L´evy flights.
3.1 Gaussian Random Walks
A random walk is a random process which consists of taking a series of consecutive random
steps. Mathematically speaking, let uN denotes the sum of each consecutive random step
si, then uN forms a random walk
uN =
N
Xi
=1
si = s1 + ... + sN = uN−1 + sN, (17)
where si is a random step drawn from a random distribution. This suggests that the next
state uN will only depend the current existing state uN−1 and the motion or transition uN
from the existing state to the next state. In theory, as the number of steps N increases,
the central limit theorem implies that the random walk (17) should approaches a Gaussian
distribution. In addition, there is no reason why each step length should be fixed. In fact,
the step size can also vary according to a known distribution. If the step length obeys the
Gaussian distribution, the random walk becomes the standard Brownian motion [16, 33].
From metaheuristic point of view, all paths of search agents form a random walk, in-cluding
a particle’s trajectory in simulated annealing, a zig-zag path of a particle in PSO,
or the piecewise path of a firefly in FA. The only difference is that transition probabilities
are different, and change with time and locations.
12. Under simplest assumptions, we know that a Gaussian distribution is stable. For a
particle starts with an initial location x0, its final location xN after N time steps is
xN = x0 +
N
Xi
=1
αisi, (18)
where αi 0 is a parameters controlling the step sizes or scalings. If si is drawn from a
normal distribution N(μi, σ2
i ), then the conditions of stable distributions lead to a combined
Gaussian distribution
xN N(μ, σ2
), μ =
N
Xi
=1
αiμi, σ2
=
N
Xi
=1
αi[σ2
i + (μ − μi)2]. (19)
We can see that that the mean location changes with N and the variances increases as N
increases, this makes it possible to reach any areas in the search space if N is large enough.
A diffusion process can be viewed as a series of Brownian motion, and the motion obeys
the Gaussian distribution. For this reason, standard diffusion is often referred to as the
Gaussian diffusion. As the mean of particle locations is obviously zero if μi = 0, their
variance will increase linearly with t. In general, in the d-dimensional space, the variance
of Brownian random walks can be written as
σ2(t) = |v0|2t2 + (2dD)t, (20)
where v0 is the drift velocity of the system. Here D = s2/(2τ ) is the effective diffusion
coefficient which is related to the step length s over a short time interval τ during each
jump. If the motion at each step is not Gaussian, then the diffusion is called non-Gaussian
diffusion. If the step length obeys other distribution, we have to deal with more generalized
random walks. A very special case is when the step length obeys the L´evy distribution,
such a random walk is called L´evy flight or L´evy walk.
3.2 Randomization via L´evy Flights
In nature, animals search for food in a random or quasi-random manner. In general, the
foraging path of an animal is effectively a random walk because the next move is based on the
current location/state and the transition probability to the next location. Which direction it
chooses depends implicitly on a probability which can be modelled mathematically [3, 24].
For example, various studies have shown that the flight behaviour of many animals and
insects has demonstrated the typical characteristics of L´evy flights [24, 26]. Subsequently,
such behaviour has been applied to optimization and optimal search, and preliminary results
show its promising capability [30, 24].
In general, L´evy distribution is stable, and can be defined in terms of a characteristic
function or the following Fourier transform
F(k) = exp[−α|k|
14. where α is a scale parameter. The inverse of this integral is not easy, as it does not have
nay analytical form, except for a few special cases [16, 23]. For the case of β = 2, we have
F(k) = exp[−αk2], whose inverse Fourier transform corresponds to a Gaussian distribution.
Another special case is β = 1, which corresponds to a Cauchy distribution
For the general case, the inverse integral
L(s) =
1
Z 1
π 0
cos(ks) exp[−α|k|
15. ]dk, (22)
can be estimated only when s is large. We have
L(s) !
α β (β) sin(πβ/2)
π|s|1+
16. , s ! 1. (23)
L´evy flights are more efficient than Brownian random walks in exploring unknown, large-scale
search space. There are many reasons to explain this efficiency, and one of them is
due to the fact that the variance of L´evy flights takes the following form
σ2(t) t3−
17. , 1 β 2, (24)
which increases much faster than the linear relationship (i.e., σ2(t) t) of Brownian random
walks.
Studies show that L´evy flights can maximize the efficiency of resource searches in un-certain
environments. In fact, L´evy flights have been observed among foraging patterns of
albatrosses and fruit flies [24, 26, 30]. In addition, L´evy flights have many applications.
Many physical phenomena such as the diffusion of fluorenscent molecules, cooling behavior
and noise could show L´evy-flight characteristics under the right conditions [26].
4 Open Problems
It is no exaggeration to say that metahueristic algorithms have been a great success in
solving various tough optimization problems. Despite this huge success, there are many
important questions which remain unanswered. We know how these heuristic algorithms
work, and we also partly understand why these algorithms work. However, it is difficult to
analyze mathematically why these algorithms are so successful, though significant progress
has been made in the last few years [1, 22]. However, many open problems still remain.
For all population-based metaheuristics, multiple search agents form multiple interact-ing
Markov chains. At the moment, theoretical development in these areas are still at
early stage. Therefore, the mathematical analysis concerning of the rate of convergence is
very difficult, if not impossible. Apart from the mathematical analysis on a limited few
metaheuristics, convergence of all other algorithms has not been proved mathematically,
at least up to now. Any mathematical analysis will thus provide important insight into
these algorithms. It will also be valuable for providing new directions for further important
18. modifications on these algorithms or even pointing out innovative ways of developing new
algorithms.
For almost all metaheuristics including future new algorithms, an important issue to
be addresses is to provide a balanced trade-off between local intensification and global
diversification [5]. At present, different algorithm uses different techniques and mechanism
with various parameters to control this, they are far from optimal. Important questions
are: Is there any optimal way to achieve this balance? If yes, how? If not, what is the best
we can achieve?
Furthermore, it is still only partly understood why different components of heuristics
and metaheuristics interact in a coherent and balanced way so that they produce efficient
algorithms which converge under the given conditions. For example, why does a balanced
combination of randomization and a deterministic component lead to a much more efficient
algorithm (than a purely deterministic and/or a purely random algorithm)? How to measure
or test if a balance is reached? How to prove that the use of memory can significantly
increase the search efficiency of an algorithm? Under what conditions?
In addition, from the well-known No-Free-Lunch theorems [31], we know that they have
been proved for single objective optimization for finite search domains, but they do not hold
for continuous infinite domains [1, 2]. In addition, they remain unproved for multiobjective
optimization. If they are proved to be true (or not) for multiobjective optimization, what
are the implications for algorithm development? Another important question is about
the performance comparison. At the moment, there is no agreed measure for comparing
performance of different algorithms, though the absolute objective value and the number of
function evaluations are two widely used measures. However, a formal theoretical analysis
is yet to be developed.
Nature provides almost unlimited ways for problem-solving. If we can observe carefully,
we are surely inspired to develop more powerful and efficient new generation algorithms.
Intelligence is a product of biological evolution in nature. Ultimately some intelligent algo-rithms
(or systems) may appear in the future, so that they can evolve and optimally adapt
to solve NP-hard optimization problems efficiently and intelligently.
Finally, a current trend is to use simplified metaheuristic algorithms to deal with complex
optimization problems. Possibly, there is a need to develop more complex metaheuristic
algorithms which can truly mimic the exact working mechanism of some natural or biological
systems, leading to more powerful next-generation, self-regulating, self-evolving, and truly
intelligent metaheuristics.
19. References
[1] A. Auger and B. Doerr, Theory of Randomized Search Heuristics: Foundations and
Recent Developments, World Scientific, (2010).
[2] A. Auger and O. Teytaud, Continuous lunches are free plus the design of optimal
optimization algorithms, Algorithmica, 57(1), 121-146 (2010).
[3] W. J. Bell, Searching Behaviour: The Behavioural Ecology of Finding Resources,
Chapman Hall, London, (1991).
[4] D. Bertsimas and J. Tsitsiklis, Simulated annealing, Stat. Science, 8, 10-15 (1993).
[5] C. Blum and A. Roli, Metaheuristics in combinatorial optimization: Overview and
conceptural comparison, ACM Comput. Surv., 35, 268-308 (2003).
[6] A. Chatterjee and P. Siarry, Nonlinear inertia variation for dynamic adapation in
particle swarm optimization, Comp. Oper. Research, 33, 859-871 (2006).
[7] M. Clerc, J. Kennedy, The particle swarm - explosion, stability, and convergence in
a multidimensional complex space, IEEE Trans. Evolutionary Computation, 6, 58-73
(2002).
[8] Dorigo, M., Optimization, Learning and Natural Algorithms, PhD thesis, Politecnico
di Milano, Italy, (1992).
[9] A. P. Engelbrecht, Fundamentals of Computational Swarm Intelligence, Wiley, (2005).
[10] C. A. Floudas and P. M. Parlalos, Collection of Test Problems for Constrained Global
Optimization Algorithms, Springer-Verlag, Lecture Notes in Computation Science,
455, (1990).
[11] C. A. Floudas and P. M. Pardolos, Encyclopedia of Optimization, 2nd Edition, Springer
(2009).
[12] D. Gamerman, Markov Chain Monte Carlo, Chapman Hall/CRC, (1997).
[13] C. J. Geyer, Practical Markov Chain Monte Carlo, Statistical Science, 7, 473-511
(1992).
[14] A. Ghate and R. Smith, Adaptive search with stochastic acceptance probabilities for
global optimization, Operations Research Lett., 36, 285-290 (2008).
[15] V. Granville, M. Krivanek and J. P. Rasson, Simulated annealing: A proof of conver-gence,
IEEE Trans. Pattern Anal. Mach. Intel., 16, 652-656 (1994).
20. [16] M. Gutowski, L´evy flights as an underlying mechanism for global optimization algo-rithms,
ArXiv Mathematical Physics e-Prints, June, (2001).
[17] J. Kennedy and R. C. Eberhart, Particle swarm optimization, in: Proc. of IEEE
International Conference on Neural Networks, Piscataway, NJ. pp. 1942-1948 (1995).
[18] J. Kennedy, R. C. Eberhart, Swarm intelligence, Academic Press, 2001.
[19] J. Holland, Adaptation in Natural and Artificial systems, University of Michigan Press,
Ann Anbor, (1975).
[20] S. Kirkpatrick, C. D. Gellat and M. P. Vecchi, Optimization by simulated annealing,
Science, 220, 670-680 (1983).
[21] C. Matthews, L.Wright, and X. S. Yang, Sensitivity Analysis, Optimization, and Sam-pling
Methodds Applied to Continous Models, National Physical Laboratory Report,
UK, (2009).
[22] F. Neumann and C. Witt, Bioinspired Computation in Combinatorial Optimization:
Algorithms and Their Computational Complexity, Springer, (2010).
[23] J. P. Nolan, Stable distributions: models for heavy-tailed data, American University,
(2009).
[24] I. Pavlyukevich, L´evy flights, non-local search and simulated annealing, J. Computa-tional
Physics, 226, 1830-1844 (2007).
[25] S. Rebennack, A. Arulselvan, L. Elefteriadou and P. M. Pardalos, Complexity analysis
for maximum flow problems with arc reversals, J. Combin. Optimization, 19(2), 200-
216 (2010).
[26] A. M. Reynolds and C. J. Rhodes, The L´evy fligth paradigm: random search patterns
and mechanisms, Ecology, 90, 877-887 (2009).
[27] L. Steinhofel, A. Albrecht and C. K. Wong, Convergence analysis of simulated
annealing-based algorithms solving flow shop scheduling problems, Lecture Notes in
Computer Science, 1767, 277-290 (2000).
[28] I. M. Sobol, A Primer for the Monte Carlo Method, CRC Press, (1994).
[29] E. G. Talbi, Metaheuristics: From Design to Implementation, Wiley, (2009).
[30] G. M. Viswanathan, S. V. Buldyrev, S. Havlin, M. G. E. da Luz, E. P. Raposo, and H.
E. Stanley, L´evy flight search patterns of wandering albatrosses, Nature, 381, 413-415
(1996).
21. [31] D. H. Wolpert and W. G. Macready, No free lunch theorems for optimisation, IEEE
Transaction on Evolutionary Computation, 1, 67-82 (1997).
[32] X. S. Yang, Nature-Inspired Metaheuristic Algorithms, Luniver Press, (2008).
[33] X. S. Yang, X. Engineering Optimization: An Introduction with Metaheuristic Appli-cations,
John Wiley Sons.
[34] X. S. Yang, Firefly algorithm, stochastic test functions and design optimisation, Int.
J. Bio-Inspired Computation, 2, 78–84 (2010).
[35] X. S. Yang and S. Deb, Engineering optimization by cuckoo search, Int. J. Math.
Modelling Num. Optimization, 1, 330-343 (2010).