This document provides an overview of a survey of multi-objective evolutionary algorithms for data mining tasks. It discusses key concepts in multi-objective optimization and evolutionary algorithms. It also reviews common data mining tasks like feature selection, classification, clustering, and association rule mining that are often formulated as multi-objective problems and solved using multi-objective evolutionary algorithms. The survey focuses on reviewing applications of multi-objective evolutionary algorithms for feature selection and classification in part 1, and applications for clustering, association rule mining and other tasks in part 2.
Cuckoo Search: Recent Advances and ApplicationsXin-She Yang
This document summarizes recent advances and applications of the cuckoo search algorithm, a nature-inspired metaheuristic optimization algorithm developed in 2009. Cuckoo search mimics the brood parasitism breeding behavior of some cuckoo species. It uses a combination of local and global search achieved through random walks and Levy flights to efficiently explore the search space. Studies show cuckoo search often finds optimal solutions faster than genetic algorithms and particle swarm optimization. The algorithm has been applied to diverse optimization problems and continues to be improved and extended to multi-objective optimization.
The document provides an overview of quantitative analysis. It discusses that quantitative analysis is the systematic study of an organization's structure, characteristics, functions, and relationships to provide executives with a quantitative basis for decision making. The characteristics of quantitative analysis include a focus on decision making, applying a scientific approach, using an interdisciplinary team, and applying formal mathematical models. The quantitative analysis process involves defining the problem, developing a model, acquiring data, developing a solution, testing the solution, and validating the model. Common tools used in quantitative analysis include linear programming, statistical techniques, decision tables, decision trees, game theory, forecasting, and mathematical programming.
This document provides an overview of algorithmic issues in computational intelligence optimization from design to implementation. It discusses key concepts in optimization problems including analytical approaches, exact methods, approximate iterative methods, and metaheuristics. It also examines challenges in optimizing real-world problems that are highly non-linear, multi-modal, computationally expensive, and have memory/time constraints. The document concludes by discussing the need for algorithms to balance exploration and exploitation and to adapt to problem landscapes.
Lecture 2 Basic Concepts of Optimal Design and Optimization Techniques final1...Khalil Alhatab
This document provides an overview of optimization techniques and the process of formulating an optimal design problem. It begins with introducing optimization and defining it as finding the best solution under given circumstances by optimizing an objective function. It then contrasts conventional versus optimal design processes and discusses classifying optimization problems based on constraints, variables, objectives, and other factors. The document concludes by outlining the five steps to formulate an optimal design problem: project description, data collection, defining design variables, establishing an optimization criterion or objective function, and specifying constraints.
Operation research history and overview application limitationBalaji P
This document provides an overview of operation research (OR). It discusses OR topics like quantitative approaches to decision making, the history and definition of OR, common OR models like linear programming and network flow programming, and applications of OR. It also explains problem solving, decision making, and quantitative analysis approaches. OR aims to apply analytical methods to help make optimal decisions for complex systems and problems.
This document contains answers to assignment questions on operations research. It defines operations research and describes types of operations research models including physical and mathematical models. It also outlines the phases of operations research including the judgment, research, and action phases. Additionally, it provides explanations and examples of linear programming problems and their graphical solution method, as well as addressing how to solve degeneracies in transportation problems and explaining the MODI optimality test procedure.
This document provides an introduction to multi-objective optimization using evolutionary algorithms. It discusses how evolutionary algorithms are well-suited for multi-objective optimization problems as they use a population approach to find multiple non-dominated solutions simultaneously. The document outlines the basic principles of evolutionary optimization for single-objective problems and then describes the key concepts and operating principles of evolutionary multi-objective optimization.
This document provides an overview of a survey of multi-objective evolutionary algorithms for data mining tasks. It discusses key concepts in multi-objective optimization and evolutionary algorithms. It also reviews common data mining tasks like feature selection, classification, clustering, and association rule mining that are often formulated as multi-objective problems and solved using multi-objective evolutionary algorithms. The survey focuses on reviewing applications of multi-objective evolutionary algorithms for feature selection and classification in part 1, and applications for clustering, association rule mining and other tasks in part 2.
Cuckoo Search: Recent Advances and ApplicationsXin-She Yang
This document summarizes recent advances and applications of the cuckoo search algorithm, a nature-inspired metaheuristic optimization algorithm developed in 2009. Cuckoo search mimics the brood parasitism breeding behavior of some cuckoo species. It uses a combination of local and global search achieved through random walks and Levy flights to efficiently explore the search space. Studies show cuckoo search often finds optimal solutions faster than genetic algorithms and particle swarm optimization. The algorithm has been applied to diverse optimization problems and continues to be improved and extended to multi-objective optimization.
The document provides an overview of quantitative analysis. It discusses that quantitative analysis is the systematic study of an organization's structure, characteristics, functions, and relationships to provide executives with a quantitative basis for decision making. The characteristics of quantitative analysis include a focus on decision making, applying a scientific approach, using an interdisciplinary team, and applying formal mathematical models. The quantitative analysis process involves defining the problem, developing a model, acquiring data, developing a solution, testing the solution, and validating the model. Common tools used in quantitative analysis include linear programming, statistical techniques, decision tables, decision trees, game theory, forecasting, and mathematical programming.
This document provides an overview of algorithmic issues in computational intelligence optimization from design to implementation. It discusses key concepts in optimization problems including analytical approaches, exact methods, approximate iterative methods, and metaheuristics. It also examines challenges in optimizing real-world problems that are highly non-linear, multi-modal, computationally expensive, and have memory/time constraints. The document concludes by discussing the need for algorithms to balance exploration and exploitation and to adapt to problem landscapes.
Lecture 2 Basic Concepts of Optimal Design and Optimization Techniques final1...Khalil Alhatab
This document provides an overview of optimization techniques and the process of formulating an optimal design problem. It begins with introducing optimization and defining it as finding the best solution under given circumstances by optimizing an objective function. It then contrasts conventional versus optimal design processes and discusses classifying optimization problems based on constraints, variables, objectives, and other factors. The document concludes by outlining the five steps to formulate an optimal design problem: project description, data collection, defining design variables, establishing an optimization criterion or objective function, and specifying constraints.
Operation research history and overview application limitationBalaji P
This document provides an overview of operation research (OR). It discusses OR topics like quantitative approaches to decision making, the history and definition of OR, common OR models like linear programming and network flow programming, and applications of OR. It also explains problem solving, decision making, and quantitative analysis approaches. OR aims to apply analytical methods to help make optimal decisions for complex systems and problems.
This document contains answers to assignment questions on operations research. It defines operations research and describes types of operations research models including physical and mathematical models. It also outlines the phases of operations research including the judgment, research, and action phases. Additionally, it provides explanations and examples of linear programming problems and their graphical solution method, as well as addressing how to solve degeneracies in transportation problems and explaining the MODI optimality test procedure.
This document provides an introduction to multi-objective optimization using evolutionary algorithms. It discusses how evolutionary algorithms are well-suited for multi-objective optimization problems as they use a population approach to find multiple non-dominated solutions simultaneously. The document outlines the basic principles of evolutionary optimization for single-objective problems and then describes the key concepts and operating principles of evolutionary multi-objective optimization.
This document analyzes and compares three different drying systems - pneumatic dryers, spiral dryers, and rotation dryers with a drum - based on five criteria: heat transfer coefficient, price, drying energy, thermal usefulness, and specific energy use. The Promethee-Gaia methodology is applied to rank the alternatives. Based on the Promethee I, II, and Gaia analyses in the Decision Lab software, the pneumatic dryer is ranked as the best system as it provides the most benefits in terms of cost savings on investment and energy. The spiral dryer is next best, followed by the rotation dryer. Sensitivity analyses show the pneumatic dryer remains the top ranked system even when the criteria
This document discusses optimization problems in engineering applications. It begins by defining optimization and describing how it can be applied to engineering problems to minimize costs or maximize benefits. Some examples of engineering applications that can be optimized are described, such as designing structures for minimum cost or maximum efficiency. The document then discusses procedures for solving optimization problems, including recognizing and defining the problem, constructing a model, and implementing solutions. It also describes different types of optimization problems and methods for solving linear programming problems, including the graphical and simplex methods.
1PPA 670 Public Policy AnalysisBasic Policy Terms an.docxfelicidaddinwoodie
This document provides an overview of key concepts in public policy analysis. It defines public policy as purposive actions by government institutions to address problems and create change. The eight-fold path and process model are introduced as approaches to policy analysis. Key institutions and individuals in the policy environment are discussed. Problem recognition is described as the essential first step, including defining the problem type, scale, location, intensity, extensiveness, and timeline. Cost-benefit analysis is introduced as a tool to evaluate policy alternatives.
This document provides information about obtaining fully solved assignments from an assignment help service. Students are instructed to send their semester, specialization, and contact details to the provided email address or call the phone number to receive help with their assignments. The document includes sample assignments covering topics in quantitative management, with questions regarding linear programming, inventory management, queuing theory, simulation, game theory, and dynamic programming.
LNCS 5050 - Bilevel Optimization and Machine Learningbutest
This document discusses using bilevel optimization and machine learning techniques to improve model selection in machine learning problems. It proposes framing machine learning model selection as a bilevel optimization problem, where the inner level problems involve optimizing models on training data and the outer level problem selects hyperparameters to minimize error on test data. This bilevel framing allows for systematic optimization of hyperparameters and enables novel machine learning approaches. The document illustrates the approach for support vector regression, formulating model selection as a Stackelberg game and solving the resulting mathematical program with equilibrium constraints.
The document discusses reinforcement learning, including Q-learning. It provides an overview of reinforcement learning, describing what it is, important machine learning algorithms for it like Q-learning, and how Q-learning works in theory and practice. It also discusses challenges of reinforcement learning, potential applications, and links between reinforcement learning algorithms and human psychology.
CHAPTER Modeling and Analysis Heuristic Search Methods .docxtiffanyd4
CHAPTER
Modeling and Analysis: Heuristic
Search Methods and Simulation
LEARNING OBJECTIVES
• Explain the basic concepts of simulation
and heuristics, and when to use them
• Understand how search methods are
used to solve some decision support
models
• Know the concepts behind and
applications of genetic algorithms
• Explain the differences among
algorithms, blind search, and heuristics
• Understand the concepts and
applications of different types of
simulation
• Explain what is meant by system
dynamics, agent-based modeling, Monte
Carlo, and discrete event simulation
• Describe the key issues of model
management
I n this chapter, we continue to explore some additional concepts related to the model base, one of the major components of decision support systems (DSS). As pointed out in the last chapter, we present this material with a note of caution: The purpose
of this chapter is not necessarily for you to master the topics of modeling and analysis.
Rather, the material is geared toward gaining familiarity with the important concepts
as they relate to DSS and their use in decision making. We discuss the structure and
application of some successful time-proven models and methodologies: search methods,
heuristic programming, and simulation. Genetic algorithms mimic the natural process of
evolution to help find solutions to complex problems. The concepts and motivating appli-
cations of these advanced techniques are described in this chapter, which is organized
into the following sections:
10.1 Opening Vignette: System Dynamics Allows Fluor Corporation to Better Plan
for Project and Change Management 436
10.2 Problem-Solving Search Methods 437
10.3 Genetic Algorithms and Developing GA Applications 441
10.4 Simulation 446
435
436 Pan IV • Prescriptive Analytics
10.5 Visu al Interactive Simulatio n 453
10.6 System Dynamics Modeling 458
10.7 Agents-Based Mode ling 461
10.1 OPENING VIGNETTE: System Dynamics Allows Fluor
Corporation to Better Plan for Project and Change
Management
INTRODUCTION
Fluor is an engineering and construction company with over 36,000 employers spread
over several countries worldwide . The company's net income in 2009 amounted to
about $680 million based on total revenue o f $22 b illion. As part of its operations, Fluor
manages varying sizes of projects that are subject to scope changes, design changes, and
schedule changes.
PRESENTATION OF PROBLEM
Fluor estimated that changes accounted for about 20 to 30 percent of revenue . Most
changes were due to secondary impacts like ripple effects, disruptions, and p roductivity
loss. Previously, the changes were collated and reported at a later period and the burden
of cost allocated to the stakeholder responsible. In certain instances when late su rprises
abou t cost and project schedule are attributed to clients, it causes friction between
clients and Fluor, w hich eventually affect future business dealings. .
CHAPTER Modeling and Analysis Heuristic Search Methods .docxmccormicknadine86
CHAPTER
Modeling and Analysis: Heuristic
Search Methods and Simulation
LEARNING OBJECTIVES
• Explain the basic concepts of simulation
and heuristics, and when to use them
• Understand how search methods are
used to solve some decision support
models
• Know the concepts behind and
applications of genetic algorithms
• Explain the differences among
algorithms, blind search, and heuristics
• Understand the concepts and
applications of different types of
simulation
• Explain what is meant by system
dynamics, agent-based modeling, Monte
Carlo, and discrete event simulation
• Describe the key issues of model
management
I n this chapter, we continue to explore some additional concepts related to the model base, one of the major components of decision support systems (DSS). As pointed out in the last chapter, we present this material with a note of caution: The purpose
of this chapter is not necessarily for you to master the topics of modeling and analysis.
Rather, the material is geared toward gaining familiarity with the important concepts
as they relate to DSS and their use in decision making. We discuss the structure and
application of some successful time-proven models and methodologies: search methods,
heuristic programming, and simulation. Genetic algorithms mimic the natural process of
evolution to help find solutions to complex problems. The concepts and motivating appli-
cations of these advanced techniques are described in this chapter, which is organized
into the following sections:
10.1 Opening Vignette: System Dynamics Allows Fluor Corporation to Better Plan
for Project and Change Management 436
10.2 Problem-Solving Search Methods 437
10.3 Genetic Algorithms and Developing GA Applications 441
10.4 Simulation 446
435
436 Pan IV • Prescriptive Analytics
10.5 Visu al Interactive Simulatio n 453
10.6 System Dynamics Modeling 458
10.7 Agents-Based Mode ling 461
10.1 OPENING VIGNETTE: System Dynamics Allows Fluor
Corporation to Better Plan for Project and Change
Management
INTRODUCTION
Fluor is an engineering and construction company with over 36,000 employers spread
over several countries worldwide . The company's net income in 2009 amounted to
about $680 million based on total revenue o f $22 b illion. As part of its operations, Fluor
manages varying sizes of projects that are subject to scope changes, design changes, and
schedule changes.
PRESENTATION OF PROBLEM
Fluor estimated that changes accounted for about 20 to 30 percent of revenue . Most
changes were due to secondary impacts like ripple effects, disruptions, and p roductivity
loss. Previously, the changes were collated and reported at a later period and the burden
of cost allocated to the stakeholder responsible. In certain instances when late su rprises
abou t cost and project schedule are attributed to clients, it causes friction between
clients and Fluor, w hich eventually affect future business dealings. ...
To investigate the stability of materials by using optimization methodUCP
The document describes an experiment to investigate material stability using optimization methods. The introduction discusses how optimization is used to solve quantitative problems by finding optimal solutions. The aim is to design composite material structures with certain qualities using optimization theory and methods. Monte Carlo tree search and other algorithms are discussed as ways to optimize problems in fields like physics, chemistry, and biology to find the best possible results. The conclusion outlines advantages of process optimization like increased efficiency, better customer service, more clarity in procedures, and easier compliance and improvement.
COMPARISON BETWEEN THE GENETIC ALGORITHMS OPTIMIZATION AND PARTICLE SWARM OPT...IAEME Publication
Close range photogrammetry network design is referred to the process of placing a set of
cameras in order to achieve photogrammetric tasks. The main objective of this paper is tried to find
the best location of two/three camera stations. The genetic algorithm optimization and Particle
Swarm Optimization are developed to determine the optimal camera stations for computing the three
dimensional coordinates. In this research, a mathematical model representing the genetic algorithm
optimization and Particle Swarm Optimization for the close range photogrammetry network is
developed. This paper gives also the sequence of the field operations and computational steps for this
task. A test field is included to reinforce the theoretical aspects.
Comparison between the genetic algorithms optimization and particle swarm opt...IAEME Publication
The document compares the genetic algorithms optimization and particle swarm optimization methods for designing close range photogrammetry networks. It presents the genetic algorithm and particle swarm optimization as two popular meta-heuristic algorithms inspired by natural evolution and collective animal behavior, respectively. The document develops mathematical models representing the genetic algorithm and particle swarm optimization for close range photogrammetry network design and evaluates them in a test field to reinforce the theoretical aspects.
The document provides an overview of operations research (OR), including its history, methodology, tools and techniques, and applications. It discusses how OR began during World War II to analyze military operations and optimize resource allocation. The seven main steps of the OR methodology are described. Common OR tools include linear programming, game theory, decision theory, queuing theory, inventory models, simulation, and dynamic programming. Finally, the document outlines some example applications of OR in fields like accounting, construction, and facilities planning.
Chapter 6 - Learning data and analytics coursegideymichael
The document discusses machine learning and concept learning. It introduces concept learning as learning a function that maps examples into categories. An example of concept learning is classifying mushrooms as poisonous or not based on their attributes. The key aspects of concept learning covered are:
- Representing hypotheses as conjunctions of attributes and values
- Defining a general to specific ordering of hypotheses
- Searching the hypothesis space using an algorithm that starts with the most specific hypothesis and generalizes it when it fails to cover positive examples
The goal is to find the maximally specific hypothesis that is consistent with all training examples.
Nature-Inspired Metaheuristic AlgorithmsXin-She Yang
This chapter introduces optimization problems and nature-inspired metaheuristics. Optimization problems involve minimizing or maximizing objective functions subject to constraints. Nature-inspired metaheuristics are computational algorithms inspired by natural phenomena, such as simulated annealing, genetic algorithms, particle swarm optimization, and ant colony optimization. They provide near-optimal solutions to complex optimization problems.
Nonlinear Programming: Theories and Algorithms of Some Unconstrained Optimiza...Dr. Amarjeet Singh
Nonlinear programming problem (NPP) had become an important branch of operations research, and it was the mathematical programming with the objective function or constraints being nonlinear functions. There were a variety of traditional methods to solve nonlinear programming problems such as bisection method, gradient projection method, the penalty function method, feasible direction method, the multiplier method. But these methods had their specific scope and limitations, the objective function and constraint conditions generally had continuous and differentiable request. The traditional optimization methods were difficult to adopt as the optimized object being more complicated. However, in this paper, mathematical programming techniques that are commonly used to extremize nonlinear functions of single and multiple (n) design variables subject to no constraints are been used to overcome the above challenge. Although most structural optimization problems involve constraints that bound the design space, study of the methods of unconstrained optimization is important for several reasons. Steepest Descent and Newton’s methods are employed in this paper to solve an optimization problem.
The document presents a modification to the Jaya optimization algorithm. The standard Jaya algorithm seeks guidance from only the best and worst solutions in each iteration. The modification proposes that Jaya should also seek guidance from the top and bottom 10% of solutions, in addition to the best and worst. This allows information to flow more continuously from the extremities.
The proposed algorithm is tested on the sphere function optimization problem. Initial candidate solutions are generated and ranked. The top and bottom 10% solutions near the best and worst are identified. Each candidate is then modified based on these neighboring solutions, moving toward the top 10% and away from the bottom 10%. Finally, candidates are refined using the standard Jaya equations seeking guidance from the
The document presents a modification to the Jaya optimization algorithm. The standard Jaya algorithm seeks guidance only from the best and worst solutions in each iteration. The modification proposes that Jaya should also seek guidance from the top and bottom 10% of solutions, in addition to the best and worst. This allows information to flow more continuously from the extremities.
The proposed algorithm is tested on the sphere function optimization problem. Initial candidate solutions are generated and ranked. The top and bottom 10% solutions near the best and worst are identified. Each candidate is then modified based on these neighboring solutions, moving toward the top 10% and away from the bottom 10%. Finally, candidates are refined using the standard Jaya equations seeking guidance from the
This document provides an overview of operations research (OR). It defines OR as the scientific approach to problem solving and decision making through mathematical modeling and analysis. The document outlines the history, terminology, problem solving process, and applications of OR. Key points include that OR uses scientific methods to help organizations make better decisions, solve complex problems, and optimize performance across various industries and applications such as production, marketing, finance, and research.
Sca a sine cosine algorithm for solving optimization problemslaxmanLaxman03209
The document proposes a new population-based optimization algorithm called the Sine Cosine Algorithm (SCA) for solving optimization problems. SCA creates multiple random initial solutions and uses sine and cosine functions to fluctuate the solutions outward or toward the best solution, emphasizing exploration and exploitation. The performance of SCA is evaluated on test functions, qualitative metrics, and by optimizing the cross-section of an aircraft wing, showing it can effectively explore, avoid local optima, converge to the global optimum, and solve real problems with constraints.
QA or the Highway - Component Testing: Bridging the gap between frontend appl...zjhamm304
These are the slides for the presentation, "Component Testing: Bridging the gap between frontend applications" that was presented at QA or the Highway 2024 in Columbus, OH by Zachary Hamm.
More Related Content
Similar to Optimization- Non derivative approach_pt-2010-v2.ppt
This document analyzes and compares three different drying systems - pneumatic dryers, spiral dryers, and rotation dryers with a drum - based on five criteria: heat transfer coefficient, price, drying energy, thermal usefulness, and specific energy use. The Promethee-Gaia methodology is applied to rank the alternatives. Based on the Promethee I, II, and Gaia analyses in the Decision Lab software, the pneumatic dryer is ranked as the best system as it provides the most benefits in terms of cost savings on investment and energy. The spiral dryer is next best, followed by the rotation dryer. Sensitivity analyses show the pneumatic dryer remains the top ranked system even when the criteria
This document discusses optimization problems in engineering applications. It begins by defining optimization and describing how it can be applied to engineering problems to minimize costs or maximize benefits. Some examples of engineering applications that can be optimized are described, such as designing structures for minimum cost or maximum efficiency. The document then discusses procedures for solving optimization problems, including recognizing and defining the problem, constructing a model, and implementing solutions. It also describes different types of optimization problems and methods for solving linear programming problems, including the graphical and simplex methods.
1PPA 670 Public Policy AnalysisBasic Policy Terms an.docxfelicidaddinwoodie
This document provides an overview of key concepts in public policy analysis. It defines public policy as purposive actions by government institutions to address problems and create change. The eight-fold path and process model are introduced as approaches to policy analysis. Key institutions and individuals in the policy environment are discussed. Problem recognition is described as the essential first step, including defining the problem type, scale, location, intensity, extensiveness, and timeline. Cost-benefit analysis is introduced as a tool to evaluate policy alternatives.
This document provides information about obtaining fully solved assignments from an assignment help service. Students are instructed to send their semester, specialization, and contact details to the provided email address or call the phone number to receive help with their assignments. The document includes sample assignments covering topics in quantitative management, with questions regarding linear programming, inventory management, queuing theory, simulation, game theory, and dynamic programming.
LNCS 5050 - Bilevel Optimization and Machine Learningbutest
This document discusses using bilevel optimization and machine learning techniques to improve model selection in machine learning problems. It proposes framing machine learning model selection as a bilevel optimization problem, where the inner level problems involve optimizing models on training data and the outer level problem selects hyperparameters to minimize error on test data. This bilevel framing allows for systematic optimization of hyperparameters and enables novel machine learning approaches. The document illustrates the approach for support vector regression, formulating model selection as a Stackelberg game and solving the resulting mathematical program with equilibrium constraints.
The document discusses reinforcement learning, including Q-learning. It provides an overview of reinforcement learning, describing what it is, important machine learning algorithms for it like Q-learning, and how Q-learning works in theory and practice. It also discusses challenges of reinforcement learning, potential applications, and links between reinforcement learning algorithms and human psychology.
CHAPTER Modeling and Analysis Heuristic Search Methods .docxtiffanyd4
CHAPTER
Modeling and Analysis: Heuristic
Search Methods and Simulation
LEARNING OBJECTIVES
• Explain the basic concepts of simulation
and heuristics, and when to use them
• Understand how search methods are
used to solve some decision support
models
• Know the concepts behind and
applications of genetic algorithms
• Explain the differences among
algorithms, blind search, and heuristics
• Understand the concepts and
applications of different types of
simulation
• Explain what is meant by system
dynamics, agent-based modeling, Monte
Carlo, and discrete event simulation
• Describe the key issues of model
management
I n this chapter, we continue to explore some additional concepts related to the model base, one of the major components of decision support systems (DSS). As pointed out in the last chapter, we present this material with a note of caution: The purpose
of this chapter is not necessarily for you to master the topics of modeling and analysis.
Rather, the material is geared toward gaining familiarity with the important concepts
as they relate to DSS and their use in decision making. We discuss the structure and
application of some successful time-proven models and methodologies: search methods,
heuristic programming, and simulation. Genetic algorithms mimic the natural process of
evolution to help find solutions to complex problems. The concepts and motivating appli-
cations of these advanced techniques are described in this chapter, which is organized
into the following sections:
10.1 Opening Vignette: System Dynamics Allows Fluor Corporation to Better Plan
for Project and Change Management 436
10.2 Problem-Solving Search Methods 437
10.3 Genetic Algorithms and Developing GA Applications 441
10.4 Simulation 446
435
436 Pan IV • Prescriptive Analytics
10.5 Visu al Interactive Simulatio n 453
10.6 System Dynamics Modeling 458
10.7 Agents-Based Mode ling 461
10.1 OPENING VIGNETTE: System Dynamics Allows Fluor
Corporation to Better Plan for Project and Change
Management
INTRODUCTION
Fluor is an engineering and construction company with over 36,000 employers spread
over several countries worldwide . The company's net income in 2009 amounted to
about $680 million based on total revenue o f $22 b illion. As part of its operations, Fluor
manages varying sizes of projects that are subject to scope changes, design changes, and
schedule changes.
PRESENTATION OF PROBLEM
Fluor estimated that changes accounted for about 20 to 30 percent of revenue . Most
changes were due to secondary impacts like ripple effects, disruptions, and p roductivity
loss. Previously, the changes were collated and reported at a later period and the burden
of cost allocated to the stakeholder responsible. In certain instances when late su rprises
abou t cost and project schedule are attributed to clients, it causes friction between
clients and Fluor, w hich eventually affect future business dealings. .
CHAPTER Modeling and Analysis Heuristic Search Methods .docxmccormicknadine86
CHAPTER
Modeling and Analysis: Heuristic
Search Methods and Simulation
LEARNING OBJECTIVES
• Explain the basic concepts of simulation
and heuristics, and when to use them
• Understand how search methods are
used to solve some decision support
models
• Know the concepts behind and
applications of genetic algorithms
• Explain the differences among
algorithms, blind search, and heuristics
• Understand the concepts and
applications of different types of
simulation
• Explain what is meant by system
dynamics, agent-based modeling, Monte
Carlo, and discrete event simulation
• Describe the key issues of model
management
I n this chapter, we continue to explore some additional concepts related to the model base, one of the major components of decision support systems (DSS). As pointed out in the last chapter, we present this material with a note of caution: The purpose
of this chapter is not necessarily for you to master the topics of modeling and analysis.
Rather, the material is geared toward gaining familiarity with the important concepts
as they relate to DSS and their use in decision making. We discuss the structure and
application of some successful time-proven models and methodologies: search methods,
heuristic programming, and simulation. Genetic algorithms mimic the natural process of
evolution to help find solutions to complex problems. The concepts and motivating appli-
cations of these advanced techniques are described in this chapter, which is organized
into the following sections:
10.1 Opening Vignette: System Dynamics Allows Fluor Corporation to Better Plan
for Project and Change Management 436
10.2 Problem-Solving Search Methods 437
10.3 Genetic Algorithms and Developing GA Applications 441
10.4 Simulation 446
435
436 Pan IV • Prescriptive Analytics
10.5 Visu al Interactive Simulatio n 453
10.6 System Dynamics Modeling 458
10.7 Agents-Based Mode ling 461
10.1 OPENING VIGNETTE: System Dynamics Allows Fluor
Corporation to Better Plan for Project and Change
Management
INTRODUCTION
Fluor is an engineering and construction company with over 36,000 employers spread
over several countries worldwide . The company's net income in 2009 amounted to
about $680 million based on total revenue o f $22 b illion. As part of its operations, Fluor
manages varying sizes of projects that are subject to scope changes, design changes, and
schedule changes.
PRESENTATION OF PROBLEM
Fluor estimated that changes accounted for about 20 to 30 percent of revenue . Most
changes were due to secondary impacts like ripple effects, disruptions, and p roductivity
loss. Previously, the changes were collated and reported at a later period and the burden
of cost allocated to the stakeholder responsible. In certain instances when late su rprises
abou t cost and project schedule are attributed to clients, it causes friction between
clients and Fluor, w hich eventually affect future business dealings. ...
To investigate the stability of materials by using optimization methodUCP
The document describes an experiment to investigate material stability using optimization methods. The introduction discusses how optimization is used to solve quantitative problems by finding optimal solutions. The aim is to design composite material structures with certain qualities using optimization theory and methods. Monte Carlo tree search and other algorithms are discussed as ways to optimize problems in fields like physics, chemistry, and biology to find the best possible results. The conclusion outlines advantages of process optimization like increased efficiency, better customer service, more clarity in procedures, and easier compliance and improvement.
COMPARISON BETWEEN THE GENETIC ALGORITHMS OPTIMIZATION AND PARTICLE SWARM OPT...IAEME Publication
Close range photogrammetry network design is referred to the process of placing a set of
cameras in order to achieve photogrammetric tasks. The main objective of this paper is tried to find
the best location of two/three camera stations. The genetic algorithm optimization and Particle
Swarm Optimization are developed to determine the optimal camera stations for computing the three
dimensional coordinates. In this research, a mathematical model representing the genetic algorithm
optimization and Particle Swarm Optimization for the close range photogrammetry network is
developed. This paper gives also the sequence of the field operations and computational steps for this
task. A test field is included to reinforce the theoretical aspects.
Comparison between the genetic algorithms optimization and particle swarm opt...IAEME Publication
The document compares the genetic algorithms optimization and particle swarm optimization methods for designing close range photogrammetry networks. It presents the genetic algorithm and particle swarm optimization as two popular meta-heuristic algorithms inspired by natural evolution and collective animal behavior, respectively. The document develops mathematical models representing the genetic algorithm and particle swarm optimization for close range photogrammetry network design and evaluates them in a test field to reinforce the theoretical aspects.
The document provides an overview of operations research (OR), including its history, methodology, tools and techniques, and applications. It discusses how OR began during World War II to analyze military operations and optimize resource allocation. The seven main steps of the OR methodology are described. Common OR tools include linear programming, game theory, decision theory, queuing theory, inventory models, simulation, and dynamic programming. Finally, the document outlines some example applications of OR in fields like accounting, construction, and facilities planning.
Chapter 6 - Learning data and analytics coursegideymichael
The document discusses machine learning and concept learning. It introduces concept learning as learning a function that maps examples into categories. An example of concept learning is classifying mushrooms as poisonous or not based on their attributes. The key aspects of concept learning covered are:
- Representing hypotheses as conjunctions of attributes and values
- Defining a general to specific ordering of hypotheses
- Searching the hypothesis space using an algorithm that starts with the most specific hypothesis and generalizes it when it fails to cover positive examples
The goal is to find the maximally specific hypothesis that is consistent with all training examples.
Nature-Inspired Metaheuristic AlgorithmsXin-She Yang
This chapter introduces optimization problems and nature-inspired metaheuristics. Optimization problems involve minimizing or maximizing objective functions subject to constraints. Nature-inspired metaheuristics are computational algorithms inspired by natural phenomena, such as simulated annealing, genetic algorithms, particle swarm optimization, and ant colony optimization. They provide near-optimal solutions to complex optimization problems.
Nonlinear Programming: Theories and Algorithms of Some Unconstrained Optimiza...Dr. Amarjeet Singh
Nonlinear programming problem (NPP) had become an important branch of operations research, and it was the mathematical programming with the objective function or constraints being nonlinear functions. There were a variety of traditional methods to solve nonlinear programming problems such as bisection method, gradient projection method, the penalty function method, feasible direction method, the multiplier method. But these methods had their specific scope and limitations, the objective function and constraint conditions generally had continuous and differentiable request. The traditional optimization methods were difficult to adopt as the optimized object being more complicated. However, in this paper, mathematical programming techniques that are commonly used to extremize nonlinear functions of single and multiple (n) design variables subject to no constraints are been used to overcome the above challenge. Although most structural optimization problems involve constraints that bound the design space, study of the methods of unconstrained optimization is important for several reasons. Steepest Descent and Newton’s methods are employed in this paper to solve an optimization problem.
The document presents a modification to the Jaya optimization algorithm. The standard Jaya algorithm seeks guidance from only the best and worst solutions in each iteration. The modification proposes that Jaya should also seek guidance from the top and bottom 10% of solutions, in addition to the best and worst. This allows information to flow more continuously from the extremities.
The proposed algorithm is tested on the sphere function optimization problem. Initial candidate solutions are generated and ranked. The top and bottom 10% solutions near the best and worst are identified. Each candidate is then modified based on these neighboring solutions, moving toward the top 10% and away from the bottom 10%. Finally, candidates are refined using the standard Jaya equations seeking guidance from the
The document presents a modification to the Jaya optimization algorithm. The standard Jaya algorithm seeks guidance only from the best and worst solutions in each iteration. The modification proposes that Jaya should also seek guidance from the top and bottom 10% of solutions, in addition to the best and worst. This allows information to flow more continuously from the extremities.
The proposed algorithm is tested on the sphere function optimization problem. Initial candidate solutions are generated and ranked. The top and bottom 10% solutions near the best and worst are identified. Each candidate is then modified based on these neighboring solutions, moving toward the top 10% and away from the bottom 10%. Finally, candidates are refined using the standard Jaya equations seeking guidance from the
This document provides an overview of operations research (OR). It defines OR as the scientific approach to problem solving and decision making through mathematical modeling and analysis. The document outlines the history, terminology, problem solving process, and applications of OR. Key points include that OR uses scientific methods to help organizations make better decisions, solve complex problems, and optimize performance across various industries and applications such as production, marketing, finance, and research.
Sca a sine cosine algorithm for solving optimization problemslaxmanLaxman03209
The document proposes a new population-based optimization algorithm called the Sine Cosine Algorithm (SCA) for solving optimization problems. SCA creates multiple random initial solutions and uses sine and cosine functions to fluctuate the solutions outward or toward the best solution, emphasizing exploration and exploitation. The performance of SCA is evaluated on test functions, qualitative metrics, and by optimizing the cross-section of an aircraft wing, showing it can effectively explore, avoid local optima, converge to the global optimum, and solve real problems with constraints.
Similar to Optimization- Non derivative approach_pt-2010-v2.ppt (20)
QA or the Highway - Component Testing: Bridging the gap between frontend appl...zjhamm304
These are the slides for the presentation, "Component Testing: Bridging the gap between frontend applications" that was presented at QA or the Highway 2024 in Columbus, OH by Zachary Hamm.
This talk will cover ScyllaDB Architecture from the cluster-level view and zoom in on data distribution and internal node architecture. In the process, we will learn the secret sauce used to get ScyllaDB's high availability and superior performance. We will also touch on the upcoming changes to ScyllaDB architecture, moving to strongly consistent metadata and tablets.
Northern Engraving | Nameplate Manufacturing Process - 2024Northern Engraving
Manufacturing custom quality metal nameplates and badges involves several standard operations. Processes include sheet prep, lithography, screening, coating, punch press and inspection. All decoration is completed in the flat sheet with adhesive and tooling operations following. The possibilities for creating unique durable nameplates are endless. How will you create your brand identity? We can help!
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfChart Kalyan
A Mix Chart displays historical data of numbers in a graphical or tabular form. The Kalyan Rajdhani Mix Chart specifically shows the results of a sequence of numbers over different periods.
Session 1 - Intro to Robotic Process Automation.pdfUiPathCommunity
👉 Check out our full 'Africa Series - Automation Student Developers (EN)' page to register for the full program:
https://bit.ly/Automation_Student_Kickstart
In this session, we shall introduce you to the world of automation, the UiPath Platform, and guide you on how to install and setup UiPath Studio on your Windows PC.
📕 Detailed agenda:
What is RPA? Benefits of RPA?
RPA Applications
The UiPath End-to-End Automation Platform
UiPath Studio CE Installation and Setup
💻 Extra training through UiPath Academy:
Introduction to Automation
UiPath Business Automation Platform
Explore automation development with UiPath Studio
👉 Register here for our upcoming Session 2 on June 20: Introduction to UiPath Studio Fundamentals: https://community.uipath.com/events/details/uipath-lagos-presents-session-2-introduction-to-uipath-studio-fundamentals/
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsDianaGray10
Join us to learn how UiPath Apps can directly and easily interact with prebuilt connectors via Integration Service--including Salesforce, ServiceNow, Open GenAI, and more.
The best part is you can achieve this without building a custom workflow! Say goodbye to the hassle of using separate automations to call APIs. By seamlessly integrating within App Studio, you can now easily streamline your workflow, while gaining direct access to our Connector Catalog of popular applications.
We’ll discuss and demo the benefits of UiPath Apps and connectors including:
Creating a compelling user experience for any software, without the limitations of APIs.
Accelerating the app creation process, saving time and effort
Enjoying high-performance CRUD (create, read, update, delete) operations, for
seamless data management.
Speakers:
Russell Alfeche, Technology Leader, RPA at qBotic and UiPath MVP
Charlie Greenberg, host
Introduction of Cybersecurity with OSS at Code Europe 2024Hiroshi SHIBATA
I develop the Ruby programming language, RubyGems, and Bundler, which are package managers for Ruby. Today, I will introduce how to enhance the security of your application using open-source software (OSS) examples from Ruby and RubyGems.
The first topic is CVE (Common Vulnerabilities and Exposures). I have published CVEs many times. But what exactly is a CVE? I'll provide a basic understanding of CVEs and explain how to detect and handle vulnerabilities in OSS.
Next, let's discuss package managers. Package managers play a critical role in the OSS ecosystem. I'll explain how to manage library dependencies in your application.
I'll share insights into how the Ruby and RubyGems core team works to keep our ecosystem safe. By the end of this talk, you'll have a better understanding of how to safeguard your code.
Must Know Postgres Extension for DBA and Developer during MigrationMydbops
Mydbops Opensource Database Meetup 16
Topic: Must-Know PostgreSQL Extensions for Developers and DBAs During Migration
Speaker: Deepak Mahto, Founder of DataCloudGaze Consulting
Date & Time: 8th June | 10 AM - 1 PM IST
Venue: Bangalore International Centre, Bangalore
Abstract: Discover how PostgreSQL extensions can be your secret weapon! This talk explores how key extensions enhance database capabilities and streamline the migration process for users moving from other relational databases like Oracle.
Key Takeaways:
* Learn about crucial extensions like oracle_fdw, pgtt, and pg_audit that ease migration complexities.
* Gain valuable strategies for implementing these extensions in PostgreSQL to achieve license freedom.
* Discover how these key extensions can empower both developers and DBAs during the migration process.
* Don't miss this chance to gain practical knowledge from an industry expert and stay updated on the latest open-source database trends.
Mydbops Managed Services specializes in taking the pain out of database management while optimizing performance. Since 2015, we have been providing top-notch support and assistance for the top three open-source databases: MySQL, MongoDB, and PostgreSQL.
Our team offers a wide range of services, including assistance, support, consulting, 24/7 operations, and expertise in all relevant technologies. We help organizations improve their database's performance, scalability, efficiency, and availability.
Contact us: info@mydbops.com
Visit: https://www.mydbops.com/
Follow us on LinkedIn: https://in.linkedin.com/company/mydbops
For more details and updates, please follow up the below links.
Meetup Page : https://www.meetup.com/mydbops-databa...
Twitter: https://twitter.com/mydbopsofficial
Blogs: https://www.mydbops.com/blog/
Facebook(Meta): https://www.facebook.com/mydbops/
ScyllaDB is making a major architecture shift. We’re moving from vNode replication to tablets – fragments of tables that are distributed independently, enabling dynamic data distribution and extreme elasticity. In this keynote, ScyllaDB co-founder and CTO Avi Kivity explains the reason for this shift, provides a look at the implementation and roadmap, and shares how this shift benefits ScyllaDB users.
inQuba Webinar Mastering Customer Journey Management with Dr Graham HillLizaNolte
HERE IS YOUR WEBINAR CONTENT! 'Mastering Customer Journey Management with Dr. Graham Hill'. We hope you find the webinar recording both insightful and enjoyable.
In this webinar, we explored essential aspects of Customer Journey Management and personalization. Here’s a summary of the key insights and topics discussed:
Key Takeaways:
Understanding the Customer Journey: Dr. Hill emphasized the importance of mapping and understanding the complete customer journey to identify touchpoints and opportunities for improvement.
Personalization Strategies: We discussed how to leverage data and insights to create personalized experiences that resonate with customers.
Technology Integration: Insights were shared on how inQuba’s advanced technology can streamline customer interactions and drive operational efficiency.
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...Alex Pruden
Folding is a recent technique for building efficient recursive SNARKs. Several elegant folding protocols have been proposed, such as Nova, Supernova, Hypernova, Protostar, and others. However, all of them rely on an additively homomorphic commitment scheme based on discrete log, and are therefore not post-quantum secure. In this work we present LatticeFold, the first lattice-based folding protocol based on the Module SIS problem. This folding protocol naturally leads to an efficient recursive lattice-based SNARK and an efficient PCD scheme. LatticeFold supports folding low-degree relations, such as R1CS, as well as high-degree relations, such as CCS. The key challenge is to construct a secure folding protocol that works with the Ajtai commitment scheme. The difficulty, is ensuring that extracted witnesses are low norm through many rounds of folding. We present a novel technique using the sumcheck protocol to ensure that extracted witnesses are always low norm no matter how many rounds of folding are used. Our evaluation of the final proof system suggests that it is as performant as Hypernova, while providing post-quantum security.
Paper Link: https://eprint.iacr.org/2024/257
What is an RPA CoE? Session 2 – CoE RolesDianaGray10
In this session, we will review the players involved in the CoE and how each role impacts opportunities.
Topics covered:
• What roles are essential?
• What place in the automation journey does each role play?
Speaker:
Chris Bolin, Senior Intelligent Automation Architect Anika Systems
High performance Serverless Java on AWS- GoTo Amsterdam 2024Vadym Kazulkin
Java is for many years one of the most popular programming languages, but it used to have hard times in the Serverless community. Java is known for its high cold start times and high memory footprint, comparing to other programming languages like Node.js and Python. In this talk I'll look at the general best practices and techniques we can use to decrease memory consumption, cold start times for Java Serverless development on AWS including GraalVM (Native Image) and AWS own offering SnapStart based on Firecracker microVM snapshot and restore and CRaC (Coordinated Restore at Checkpoint) runtime hooks. I'll also provide a lot of benchmarking on Lambda functions trying out various deployment package sizes, Lambda memory settings, Java compilation options and HTTP (a)synchronous clients and measure their impact on cold and warm start times.
How information systems are built or acquired puts information, which is what they should be about, in a secondary place. Our language adapted accordingly, and we no longer talk about information systems but applications. Applications evolved in a way to break data into diverse fragments, tightly coupled with applications and expensive to integrate. The result is technical debt, which is re-paid by taking even bigger "loans", resulting in an ever-increasing technical debt. Software engineering and procurement practices work in sync with market forces to maintain this trend. This talk demonstrates how natural this situation is. The question is: can something be done to reverse the trend?
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...Jason Yip
The typical problem in product engineering is not bad strategy, so much as “no strategy”. This leads to confusion, lack of motivation, and incoherent action. The next time you look for a strategy and find an empty space, instead of waiting for it to be filled, I will show you how to fill it in yourself. If you’re wrong, it forces a correction. If you’re right, it helps create focus. I’ll share how I’ve approached this in the past, both what works and lessons for what didn’t work so well.
Dandelion Hashtable: beyond billion requests per second on a commodity serverAntonios Katsarakis
This slide deck presents DLHT, a concurrent in-memory hashtable. Despite efforts to optimize hashtables, that go as far as sacrificing core functionality, state-of-the-art designs still incur multiple memory accesses per request and block request processing in three cases. First, most hashtables block while waiting for data to be retrieved from memory. Second, open-addressing designs, which represent the current state-of-the-art, either cannot free index slots on deletes or must block all requests to do so. Third, index resizes block every request until all objects are copied to the new index. Defying folklore wisdom, DLHT forgoes open-addressing and adopts a fully-featured and memory-aware closed-addressing design based on bounded cache-line-chaining. This design offers lock-free index operations and deletes that free slots instantly, (2) completes most requests with a single memory access, (3) utilizes software prefetching to hide memory latencies, and (4) employs a novel non-blocking and parallel resizing. In a commodity server and a memory-resident workload, DLHT surpasses 1.6B requests per second and provides 3.5x (12x) the throughput of the state-of-the-art closed-addressing (open-addressing) resizable hashtable on Gets (Deletes).
Dandelion Hashtable: beyond billion requests per second on a commodity server
Optimization- Non derivative approach_pt-2010-v2.ppt
1. Sept, 2010
®Copyright of Shun-Feng Su
1
Optimization:
Non-Derivative Approaches
非微分型最佳化
Offered by 蘇順豐
Shun-Feng Su,
E-mail: su@orion.ee.ntust.edu.tw
Department of Electrical Engineering,
National Taiwan University of Science and Technology
2. Sept, 2010
®Copyright of Shun-Feng Su
2
Preface
Optimization is central to many occasions
involving decision or finding good solutions in
various research problems.
In this talk, I shall provide some fundamental
concepts and ideas about optimization.
This talk will also introduce one group of
optimization techniques – non-derivative
optimization, like genetic algorithms, ant
systems, and particular swarm optimization.
3. Sept, 2010
®Copyright of Shun-Feng Su
3
Preface
Fundamentals of Optimization
Traditional Optimization
Non-derivative Approaches
Genetic Algorithms
Particle Swarm Optimization
Ant colony optimization
Epilogue
Outline
4. Sept, 2010
®Copyright of Shun-Feng Su
4
Fundamentals of Optimization
Optimization is to find the best one among all
possible alternatives.
It is easy to see that optimization is always a
good means in demonstrating your research
results.
But, the trick is what you mean “better”?
5. Sept, 2010
®Copyright of Shun-Feng Su
5
Fundamentals of Optimization
Optimization is to find the best one among all
possible alternatives.
It is easy to see that optimization is always a
good means in demonstrating your research
results.
But, the trick is what you mean “better”?
Why the optimal one is better than the others?
In other words, based on which criterion the
evaluation is conducted?
6. Sept, 2010
®Copyright of Shun-Feng Su
6
Fundamentals of Optimization
The measure of goodness of alternatives is
described by an so-called objective function or
performance index.
Thus, it is desired that when you see “optimal”, you
should first check what is the objective function used.
Optimization then is to maximized or minimized the
objective function considered.
Other terms used are cost function (maximized),
fitness function (minimized), etc.
7. Sept, 2010
®Copyright of Shun-Feng Su
7
Fundamentals of Optimization
Consider an intelligent system, usually
optimization methodology is required due to :
Better selection of applicable knowledge or
strategies can result in better performance;
In the learning process, an optimal way of
defining the updating rule is required.
8. Sept, 2010
®Copyright of Shun-Feng Su
8
Fundamentals of Optimization
Consider an intelligent system, usually
optimization methodology is required due to:
Better selection of applicable knowledge or
strategies can result in better performance;
In the learning process, an optimal way of
defining the updating rule is required.
To act as a leaning mechanism is the most popular
approach currently employed in the literature.
9. Sept, 2010
®Copyright of Shun-Feng Su
9
Fundamentals of Optimization
In general, an optimization problem requires
finding a setting of variable vector (or
parameters) of the system such that an
objective function is optimized. Sometimes, the
variable vector may have to satisfy some
constraints.
• Alternatives are to choose among values
Numerical approach.
• This is why optimization is considered as one part of
computational intelligence.
10. Sept, 2010
®Copyright of Shun-Feng Su
10
Preface
Fundamentals of Optimization
Traditional Optimization
Non-derivative Approaches
Genetic Algorithms
Particle Swarm Optimization
Ant colony optimization
Epilogue
Outline
11. Sept, 2010
®Copyright of Shun-Feng Su
11
Traditional Optimization
A traditional optimization problem can be
expressed as
Min (or Max) f(x)
subject to x
f( ) is the objective function to be optimized.
If some constraint like x is specified, it is
referred to as a constrained optimization
problem; otherwise it is called unconstrained
optimization problem.
12. Sept, 2010
®Copyright of Shun-Feng Su
12
Traditional approaches for
unconstrained optimization
If the objective function can be explicitly
expressed as a function of parameters,
traditional mathematic approaches can be
employed to solve the optimization:
Traditional optimization approaches can be
classified into two categories; direct approach
and incremental approach.
13. Sept, 2010
®Copyright of Shun-Feng Su
13
Traditional approaches for
unconstrained optimization
Direct approaches can be said to find the
solution mathematically (to find the solution
with certain properties).
In a direct approach, the idea is to directly find x
such that df(x)/dx=0 or f(x)=0.
This kind of approaches is Newton kind of
approaches.
In optimization, it is
f(x)=0
Newton’s method is to find a way of solving
f(x)=0 and the used approach can also be
iterative.
14. Sept, 2010
®Copyright of Shun-Feng Su
14
Traditional approaches for
unconstrained optimization
Increment approach is to find which way can
improve the current situation based on the
current error. (back forward approach)
Usually, an incremental approach is to update
the parameter vector as x(k+1)=x(k)+x.
In fact, such an approach is usually fulfilled as a
gradient approach; that is x=f(x)/x.
Need to find a relationship between the current error
and the change of the variable considered; that is
why x=f(x)/x is employed.
15. Sept, 2010
®Copyright of Shun-Feng Su
15
Traditional approaches for
constrained optimization
In general, the constraint can be written as h(x)=0.
When the constraint is not expressed as such an
equality (e.g., h(x)0),
either the constraint is not effective (not used) or
the minimier is located on the boundary (i.e.,
h(x)=0)
16. Sept, 2010
®Copyright of Shun-Feng Su
16
Traditional approaches for
constrained optimization
In general, the constraint can be written as h(x)=0.
A commonly-used approach is the Lagrange
Theorem, which is to find x and such that
f(x)+ h(x)= 0
where is called the Lagrange multiplier.
Then, traditional unconstrained optimization
approaches can be employed.
17. Sept, 2010
®Copyright of Shun-Feng Su
17
Traditional Optimization
Traditional optimization approaches are to
develop a formal model (objective function and
constraints) that resembles the original
problem and then to solve it by means of
traditional mathematical methods.
In other words, in order to find f(x)=0 or
f(x)/x, the objective function f( ) must be
explicitly expressed as a function of the
parameter vector x
18. Sept, 2010
®Copyright of Shun-Feng Su
18
Traditional Optimization
Traditional optimization techniques even have
some problems (like being trapped in local
optima), those approaches have been shown to
be very successful in many applications.
However, other drawbacks are found in those
traditional optimization applications.
19. Sept, 2010
®Copyright of Shun-Feng Su
19
Traditional Optimization
In real-world problems, the objective function
and/or the constraints imposed on the variables
may not be analytically treatable or even cannot
be expressed in a closed form.
Thus, either there is no way of representing the
problem considered in a form so that the
derivative of the form can be performed or
simplifications of the original problem formulation
are required.
20. Sept, 2010
®Copyright of Shun-Feng Su
20
Traditional Optimization
When there is no way of representing the problem
in a closed form, backward kinds of approaches
cannot be implemented.
When simplification is conducted, it is more than
often that the found solutions do not solve the
original problem but the simplified problem.
Thus, some approaches of use only forward are
needed. Then it is impossible to modify
candidates based on the current output.
21. Sept, 2010
®Copyright of Shun-Feng Su
21
Traditional Optimization
If only forward evaluation is used, the approach is
to find the objective function value for the current
candidate and then try another candidate.
It is more like a search algorithm.
The issue may be how to define the next
candidates.
Randomly select or
select with some guidance?
22. Sept, 2010
®Copyright of Shun-Feng Su
22
Preface
Fundamentals of Optimization
Traditional Optimization
Non-derivative Approaches
Genetic Algorithms
Particle Swarm Optimization
Ant colony optimization
Epilogue
Outline
23. Sept, 2010
®Copyright of Shun-Feng Su
23
Non-Derivative Optimization
An important property of such algorithms is that in
the process, auxiliary forms of the objective
function, such as derivations, are not required.
non-derivative optimization
Non-derivative optimization does not define the
relationship between the current situation (error)
and the variable considered. Thus, another way
of defining the finding the optimal solution need
to be employed.
24. Sept, 2010
®Copyright of Shun-Feng Su
24
Non-Derivative Optimization
A search algorithm is to find the solution based
on trying possible candidates.
Since it is impossible to try all possibilities, how
to define the next one usually is the key issue
in search algorithms.
Non-derivative optimization are also called
Evolutionary Computation, Nature Inspired
Algorithm or meta-heuristic Algorithms.
29. Sept, 2010
®Copyright of Shun-Feng Su
29
Non-Derivative Optimization
Non-Derivative Optimization approaches are to
mimic various natural phenomena, like natural
selection process or animal behaviors so as to
find the best candidate for the problem.
Those search processes are to find the next
candidates by using experience obtained from
previous search together with some random
search mechanisms.
That is to define the next candidates with
some guidance and randomness.
30. Sept, 2010
®Copyright of Shun-Feng Su
30
Non-Derivative Optimization
1. It works with a coding of solution set, not the
solutions itself. need to code solutions
2. It searches from a population of solutions, not a
single solution. parallel search
3. It uses payoff information (fitness function), not
derivatives or other auxiliary knowledge.
non-derivative optimization
4. It uses probabilistic transition rules, not
deterministic rules. random search with
guidance (stochastic search)
31. Sept, 2010
®Copyright of Shun-Feng Su
31
Genetic Algorithms
Genetic Algorithms (GAs) simulate the natural
evolutionary process in searching for the best
solution based on the mechanism of natural
selection and natural genetic operation.
John Holland, from the University of Michigan
began his work on GAs in the early 60s.
A first achievement was the publication of
Adaptation in Natural and Artificial System in
1975.
32. Sept, 2010
®Copyright of Shun-Feng Su
32
Genetic Algorithms
GA encodes solutions to the problem in a structure
that can be stored in the computer.
This object is a genome (or chromosome). GA
creates a population of genomes then applies
genetic operators (crossover and mutation) to the
candidates in the population to generate new
candidates.
It uses various selection criteria so that it picks the
best candidates for mating (and subsequent
crossover). The objective function determines how
'good' each individual is.
33. Sept, 2010
®Copyright of Shun-Feng Su
33
Genetic Algorithms
To represent solutions in terms of genes --
Representation of Candidate Solutions (CS):
– Binary encoding
– Real number encoding
– Integer or literal permutation encoding
– General data structure encoding : array, tree,
matrix, . . . , etc.
34. Sept, 2010
®Copyright of Shun-Feng Su
34
Genetic Algorithms
A Genetic Algorithm (GA) emulates biological
evolution to solve a complex problem.
GAs rely heavily on randomness. Instead of trying
to solve the problem directly, they create
random solutions and randomly mix them up
until a good solution is found.
35. Sept, 2010
®Copyright of Shun-Feng Su
35
Genetic Algorithms
The evolution starts from a population of
completely random candidates and searches
for the best generation by generation.
In each generation, multiple candidates are
stochastically selected from the current
population, modified (mutated or recombined)
to form a new population, which is used in the
next generation (iteration).
36. Sept, 2010
®Copyright of Shun-Feng Su
36
Genetic Algorithms
Use of the encoding of the parameters, not the
parameters themselves.
Work on a population of points, not a unique one.
Use the only values of the function to optimize,
not their derived function or other auxiliary
knowledge.
Use probabilistic transition function not
determinist ones.
37. Sept, 2010
®Copyright of Shun-Feng Su
37
Apply reproduction and crossover on P(t)
to yield C(t)
Apply mutation on C(t) to yield
and then evaluate D(t)
Select P(t+1) from P(t) and D(t)
based on the fitness
Initialize population P(t)
Evaluate P(t)
Stop criterion
satisfied ?
Stop
Flow chart of a
simple genetic
algorithm
38. Sept, 2010
®Copyright of Shun-Feng Su
38
Genetic Algorithms
GA uses three basic operators to manipulate the genetic
composition (chromosomes) of a population:
Reproduction is a process of selecting parents for
generating offspring. The most highly rated
chromosomes in the current generation are most likely
copied in the new generation.
Crossover provides a mechanism for chromosomes to
mix and match attributes through random processes.
Mutation is to changed attributes (genes) in the new
generation to bring new possibility. Mutation is a very
important mechanism in avoiding local minimum in
optimization search.
39. Sept, 2010
®Copyright of Shun-Feng Su
39
Reproduction
Reproduction can be divided into two kinds of
processes:
to select parents from the population and
to determine who will survive in the next generation.
Both processes need to select among all candidates.
Selection methodology can be considered based
on the following foundations:
Sampling space
Sampling Mechanism
Probability Selection
40. Sept, 2010
®Copyright of Shun-Feng Su
40
Reproduction
To select parents from the population:
GA researchers have used a number of parent
selection methods. Some of the more popular
methods are:
Proportionate Selection
Linear Rank Selection
Tournament Selection
How many parents will be selected is also an
issue for designing GA.
41. Sept, 2010
®Copyright of Shun-Feng Su
41
Proportionate Selection
In Proportionate Selection, candidates are
assigned a probability of being selected based
on their fitness: pi = fi / fj,
where pi is the probability that candidate i will be
selected and fi is the fitness of candidate i.
This type of selection is also referred to as the
roulette wheel selection.
Fitness maximum problem.
If a minimum problem is consider,
some modifications are needed.
42. Sept, 2010
®Copyright of Shun-Feng Su
42
Proportionate Selection
There are a number of disadvantages associated
with using proportionate selection:
– Cannot be used on minimization problems,
– Loss of selection pressure (search direction)
as population converges,
– Susceptible to Super Individuals
– Scaling issue for fitness values
Local optima
issue
43. Sept, 2010
®Copyright of Shun-Feng Su
43
Linear Rank Selection
In Linear Rank selection, candidates are assigned
subjective fitness based on the rank within the
population: sfi = (P-ri)(max-min)/(P-1) + min
where ri is the rank of individual i,
P is the population size,
Max represents the fitness to assign to the
best candidate,
Min represents the fitness to assign to the
worst candidate.
44. Sept, 2010
®Copyright of Shun-Feng Su
44
Linear Rank Selection
pi = sfi / sfj Roulette Wheel Selection can be
performed using the subjective fitness values.
One disadvantage associated with linear rank
selection is that the population must be sorted
on each cycle.
45. Sept, 2010
®Copyright of Shun-Feng Su
45
Tournament Selection
In Tournament Selection, q candidates are
randomly selected from the population and the
best of the q candidates is returned as a parent.
Selection pressure increases as q is increased
and decreases as q is decreased.
46. Sept, 2010
®Copyright of Shun-Feng Su
46
Selecting Who Survives
An Example Genetic Algorithm
Procedure GA{
t = 0;
Initialize P(t);
Evaluate P(t);
While (Not Done)
{
Parents(t) = Select_Parents(P(t));
Offspring(t) = Procreate(Parents(t));
Evaluate(Offspring(t));
P(t+1)= Select_Survivors(P(t),Offspring(t));
t = t + 1;
}
Genetic operations:
crossover and mutation
Select who
survive
Parent
selection
47. Sept, 2010
®Copyright of Shun-Feng Su
47
Selecting Who Survives
By itself, pick best.
Darwinian survival of the fittest.
Give more copies to better guys.
Ways to do:
–truncation
–roulette wheel
–tournament
48. Sept, 2010
®Copyright of Shun-Feng Su
48
Selection Who Survives
Basically, there are two types of selections in GAs:
Let = # of parents, = # of offspring,
( +) selection: select best out of offspring and
old parents as parents of the next generation.
(, ) selection: select best offspring as parents
of the next generation. ( <)
49. Sept, 2010
®Copyright of Shun-Feng Su
49
Genetic Operators
Genetic Algorithms typically use two types of
operators: Crossover and Mutation.
Crossover is usually the primary operator for
inheriting properties from parents with mutation
serving only as a mechanism to introduce
diversity in the population.
However, when designing a GA, it is possible to
develop unique crossover and mutation
operators that take advantage of the structure
of the problem.
50. Sept, 2010
®Copyright of Shun-Feng Su
50
Crossover Operator
There are a number of crossover operators that
have been used on binary and real-coded GAs:
Single-point Crossover,
Two-point Crossover,
Uniform Crossover
How many offspring will be generated is also an
issue in designing GA.
51. Sept, 2010
®Copyright of Shun-Feng Su
51
Crossover Operator
Given two parents, single-point crossover will
generate a cut-point and recombines the first
part of first parent with the second part of the
second parent to create one offspring.
Example:
Parent 1: X X | X X X X X
Parent 2: Y Y | Y Y Y Y Y
Offspring 1: X X Y Y Y Y Y
Offspring 2: Y Y X X X X X
52. Sept, 2010
®Copyright of Shun-Feng Su
52
Two-Point Crossover
Two-Point crossover is very similar to single-point
crossover except that two cut-points are
generated instead of one.
Example:
Parent 1: X X | X X X | X X
Parent 2: Y Y | Y Y Y | Y Y
Offspring 1: X X Y Y Y X X
Offspring 2: Y Y X X X Y Y
53. Sept, 2010
®Copyright of Shun-Feng Su
53
Uniform Crossover
In Uniform Crossover, a value of the first parent’s
gene is assigned to the first offspring and the
value of the second parent’s gene is to the
second offspring with probability 0.5.
With probability 0.5 the value of the first parent’s
gene is assigned to the second offspring and
the value of the second parent’s gene is
assigned to the first offspring.
54. Sept, 2010
®Copyright of Shun-Feng Su
54
Uniform Crossover
Example:
Parent 1: X X X X X X X
Parent 2: Y Y Y Y Y Y Y
Offspring 1: X Y X Y Y X Y
Offspring 2: Y X Y X X Y X
55. Sept, 2010
®Copyright of Shun-Feng Su
55
Real-Coded Crossover Operators
For Real-Coded representations there exist a
number of other crossover operators:
Mid-Point Crossover,
Flat Crossover (BLX-0.0),
BLX-0.5
56. Sept, 2010
®Copyright of Shun-Feng Su
56
Mid-Point Crossover
Given two parents where X and Y represent a
floating point number:
Parent 1: X
Parent 2: Y
Offspring: (X+Y)/2
If a chromosome contains more than one gene,
then this operator can be applied to each gene
with a probability of Pmp.
57. Sept, 2010
®Copyright of Shun-Feng Su
57
Flat Crossover (BLX-0.0)
Flat crossover was developed by Radcliffe (1991)
Given two parents where X and Y represent a
floating point number:
Parent 1: X
Parent 2: Y
Offspring: rnd(X,Y)
Of course, if a chromosome contains more than
one gene then this operator can be applied to
each gene with a probability of Pblx-0.0.
58. Sept, 2010
®Copyright of Shun-Feng Su
58
BLX-
Developed by Eshelman & Schaffer (1992)
Given two parents where X and Y represent a floating
point number, and where X < Y:
Parent 1: X
Parent 2: Y
Let = (Y-X), where = 0.5
Offspring: rnd(X-, Y+ )
Of course, if a chromosome contains more than one gene
then this operator can be applied to each gene with a
probability of Pblx-.
59. Sept, 2010
®Copyright of Shun-Feng Su
59
Mutation (Binary-Coded)
In Binary-Coded GAs, each bit in the chromosome is
mutated with probability pbm known as the mutation rate.
Parent1 1 0 0 0 0 1 0
Parent2 1 1 1 0 0 0 1
Child1 1 0 0 1 0 0 1
Child2 0 1 1 0 1 1 0
An Example of Single-point Crossover Between the
Third and Fourth Genes with a Mutation Rate of
0.01 Applied to Binary Coded Chromosomes
60. Sept, 2010
®Copyright of Shun-Feng Su
60
Mutation (Real-Coded)
In real-coded GAs, Gaussian mutation can be
used.
For example, BLX-0.0 Crossover with Gaussian
mutation.
Given two parents where X and Y represent a
floating point number:
Parent 1: X
Parent 2: Y
Offspring: rnd(X,Y) + N(0,1)
61. Sept, 2010
®Copyright of Shun-Feng Su
61
Advanced GA techniques
• Elitism – Carry over some portion of the best
solutions to the next generation.
• Variable operators – Create multiple types of
crossovers and mutations. Track the health of
the offspring they produce, and adjust their
usage accordingly.
• Tribes – Create separate populations that only
occasionally mix. This may help avoid
converging on local maxima.
62. Sept, 2010
®Copyright of Shun-Feng Su
62
Advanced GA techniques
"General Structure of Hybrid Genetic Algorithms"
Begin
t←0;
initialize P(t);
evaluate P(t);
while (not termination condition) do
recombine P(t) to yield C(t);
locally climb C(t);
evaluate C(t);
selecte P(t+1) from P(t) and C(t);
t← t + 1;
end
end
Local search
mechanism: to
provide greedy
advance in candidate
in this step.
There are various approaches and
variants for this local search
mechanism.
63. Sept, 2010
®Copyright of Shun-Feng Su
63
Genetic Algorithms
Genetic operations play a role of generating the
new chromosomes for evolution. Hopefully,
the best-fitted solution can be generated.
In the algorithm, randomness plays essential
roles in all operations.
One attractive property of GA is that the
performance of the solution is always getting
better.
64. Sept, 2010
®Copyright of Shun-Feng Su
64
Genetic Algorithms
In fact, GA should be understood as a general
adaptable concept for problem solving rather
than a collection of related and ready-to-use
algorithms.
However, due to the nature of adaptation to the
problems, the operations of GAs must be
designed by the users.
Moreover, if the optimization is constrained, the
initial population and the generations of new
chromosomes must be carefully selected.
65. Sept, 2010
®Copyright of Shun-Feng Su
65
Genetic Algorithms
Illegal chromosome can not be decoded to a
solution. It can not be evaluated.
To use a penalty function is usually a bad
approach to this situation.
Some repairing techniques have been proposed
to convert an illegal or infeasible chromosome
to an acceptable one.
66. Sept, 2010
®Copyright of Shun-Feng Su
66
Genetic Algorithms -- websites
IlliGAL (http://www-illigal.ge.uiuc.edu/) - Illinois Genetic
Algorithms Laboratory - Download technical reports
and code
Golem Project (http://demo.cs.brandeis.edu/golem/) -
Automatic Design and Manufacture of Robotic
Lifeforms
Introduction to Genetic Algorithms Using
RPL2 (http://www.epcc.ed.ac.uk/computing/training/doc
ument_archive/GAs-course/main.html)
Talk.Origins FAQ on the uses of genetic algorithms, by
AdamMarczyk (http://www.talkorigins.org/faqs/genalg/g
enalg.html)
Genetic algorithm in search and optimization, by Richard
Baker (http://www.fenews.com/fen5/ga.html)
67. Sept, 2010
®Copyright of Shun-Feng Su
67
Genetic Algorithms -- websites
Genetic Algorithm and Markov chain Monte Carlo: Differential
Evolution Markov chain makes Bayesian Computing easy
(http://www.biometris.nl/Markov%20Chain.pdf)
Differential Evolution using Genetic Algorithm
(http://www.icsi.berkeley.edu/~storn/code.html#hist)
Introduction to Genetic Algorithms and Neural Networks
(http://www.ai-junkie.com/) including an example windows program
Genetic Algorithm Solves the Toads and Frogs Puzzle (http://www.cut-
the-knot.org/SimpleGames/evolutions.shtml) (requires Java)
Not-So-Mad Science: Genetic Algorithms and Web Page Design for
Marketers (http://www.marketingprofs.com/4/syrett6.asp) by
Matthew Syrett
68. Sept, 2010
®Copyright of Shun-Feng Su
68
Genetic Algorithms -- websites
http://www.aic.nrl.navy.mil/galist/The Genetic Algorithms Archives
(maintained by Alan C Schultz at The Navy Center for Applied
Research in Artificial Intelligence)
http://www.genetic-programming.org/(A source of information about
the field of genetic programming)
http://www.genetic-programming.com/(the home page of Genetic
Programming Inc.)
http://www.genetic-programming.com/johnkoza.html(Home Page of
Professor John R. Koza)
http://www-illigal.ge.uiuc.edu:8080/(International Society for Genetic
and Evolutionary Computation)
http://www-illigal.ge.uiuc.edu/index.php3(Illinois Genetic Algorithms
Laboratory ILLiGAL)
http://cs.felk.cvut.cz/~xobitko/ga/Introduction to Genetic Algorithms
http://ww.lalena.com/ai/tsp/Travelling Salesman Problem Using
Genetic Algorithms
http://www4.ncsu.edu/eos/users/d/dhloughl/public/stable.htmGenetic
Algorithms Online
69. Sept, 2010
®Copyright of Shun-Feng Su
69
Genetic Algorithms -- websites
http://cs.gmn.edu/research/gag/George Mason University GA Group
(GAG)
http://garage.cse.msu.edu/Michigan State University - Genetic
Algorithms Research and Application Groups (GARAGe)
http://gaslab.cs.unr.edu/Genetic Adaptive Systems LAB (GASLAB)
Evoluationary Computation (Journal)
http://www.densis.fee.unicamp.br/~moscatoMemetic Algorithms – Prof
Pablo Moscato
http://www.cs.newcastle.edu.au/~mendesMemetic Algorithms –
Softwares
http://groups.yahoo.com/group/MALL/Memetic Algorithms Discussion
Group
http://www.cs.newcastle.edu.au/~nbiNewcastle Bioinformatics Group
http://webhost.ua.ac.be/eume/welcome.htm?eume.sidebar.html&0Eur
opean Chapter on Metaheuristics
70. Sept, 2010
®Copyright of Shun-Feng Su
70
Other Non-derivation Optimization
Other often mentioned approaches are Particle
Swarm Optimization (PSO) and Ants (ACS,
ACO, etc).
The overall ideas are all similar in that they all
use fitness values to guide the search with
some random mechanisms associated with
the search process.
Usually, these approaches can have better
search performance than that of genetic
algorithms.
71. Sept, 2010
®Copyright of Shun-Feng Su
71
Particle Swarm Optimization
Particle Swarm Optimization is an optimization
technique which provides an evolutionary based
search. This search algorithm was introduced
by R. Eberhart and J. Kennedy in Proc. 1995
IEEE Int'l. Conf. on Neural Networks IV, pp.
1942-1948.
PSO shares many similarities with evolutionary
computation techniques such as Genetic
Algorithms (GA). The system is initialized
with a population of random solutions and
searches for optima by updating generations.
However, unlike GA, PSO has no evolution
operators such as crossover and mutation.
72. Sept, 2010
®Copyright of Shun-Feng Su
72
Particle Swarm Optimization
PSO algorithms are especially useful for
parameter optimization in continuous, multi-
dimensional search spaces.
PSO is mainly inspired by social behavior
patterns of organisms that live and interact
within large groups.
In PSO, the potential solutions, called particles,
fly through the problem space by following
the current optimum particles.
73. Sept, 2010
®Copyright of Shun-Feng Su
73
Particle Swarm Optimization
The connection to search and optimization
problems is made by assigning direction vectors
and velocities to each particle in a multi-
dimensional search space.
Each particle then 'moves' or 'flies' through the
search space following its velocity vector, which
is influenced by the directions and velocities of
other particles in its neighborhood.
These localized interactions with neighboring
particles propagate through the entire 'swarm' of
potential solutions.
74. Sept, 2010
®Copyright of Shun-Feng Su
74
Particle Swarm Optimization
How much influence a particular point has on
other points is determined by its 'fitness‘;
that is, a measure assigned to a potential
solution, which captures how good it is
compared to all other solution points.
Hence, an evolutionary idea of 'survival of the
fittest' comes into play, as well as a social
behavior component through a 'follow the
local leader' effect and emergent pattern
formation.
75. Sept, 2010
®Copyright of Shun-Feng Su
75
Particle Swarm Optimization
Each particle keeps track of its coordinates in
the problem space which are associated
with the best fitnes) it has achieved so far.
This value is called pbest.
Another "best" value is obtained so far by any
particle in the neighbors of the particle. This
location is called lbest.
When a particle takes all the population as its
topological neighbors, the best value is a
global best and is called gbest.
76. Sept, 2010
®Copyright of Shun-Feng Su
76
Particle Swarm Optimization
The particle swarm optimization concept
consists of, at each time step, changing the
velocity of (accelerating) each particle toward
its pbest and gbest (global version of PSO) or
lbest locations (local version of PSO).
Acceleration is weighted by a random term, with
separate random numbers being generated
for acceleration toward pbest and gbest (or
lbest) locations.
77. Sept, 2010
®Copyright of Shun-Feng Su
77
PSO Algorithm
After finding the two best values, the particle
updates its velocity and positions with
following equation (a) and (b).
(a) v[ ] = v[ ] + c1 * rand() * (pbest[ ] - present[ ])
+ c2 * rand() * (gbest[ ] - present[ ])
(b) present[ ] = present[ ] + v[ ]
78. Sept, 2010
®Copyright of Shun-Feng Su
78
PSO Algorithm
1) Initialize the population - locations and velocities
2) Evaluate the fitness of the individual particle
(pBest)
3) Keep track of the individuals highest fitness
(gBest)
4) Modify velocities based on pBest and gBest
position
5) Update the particles position
6) Terminate if the condition is met
7) Go to Step 2
80. Sept, 2010
®Copyright of Shun-Feng Su
80
Particle Swarm Optimization
PSO shares many common points with GA.
However, PSO does not have genetic
operators like crossover and mutation.
Particles update themselves with the internal
velocity. They also have memory, which is
important to the algorithm.
Compared with genetic algorithms (GAs), the
information sharing mechanism in PSO is
significantly different.
81. Sept, 2010
®Copyright of Shun-Feng Su
81
Particle Swarm Optimization
In GAs, chromosomes share information with
each other. So the whole population moves
like a one group towards an optimal area. In
PSO, only gbest (or lbest) gives out the
information to others. It is a one -way
information sharing mechanism.
The evolution only looks for the best solution.
Compared with GA, all the particles tend to
converge to the best solution quickly even in
the local version in most cases.
82. Sept, 2010
®Copyright of Shun-Feng Su
82
Ant Colony Optimization
Ant colony optimization (ACO) is a population-
based metaheuristic that can be used to find
approximate solutions to difficult optimization
problems.
An analogy with the way ant colonies function has
suggested the definition of a new
computational paradigm.
M. Dorigo, V. Maniezzo and A. Colorni, “Ant System: Optimization by
a Colony of Cooperating Agents,” IEEE Trans. Systems, Man,
and Cybernetics, Part B: Cybernetics, vol. 26, no. 1, pp. 29-41,
1996.
83. Sept, 2010
®Copyright of Shun-Feng Su
83
Ant Colony Optimization
In ACO, a set of agents called artificial ants search
for good solutions to a given optimization
problem.
In ACO, the optimization problem is transformed
into the problem of finding the best path on a
weighted graph.
The artificial ants incrementally build solutions by
moving on the graph.
The solution construction process is stochastic
and is biased by a pheromone model.
84. Sept, 2010
®Copyright of Shun-Feng Su
84
What is “pheromone”
A moving ant lays some pheromone on paths on
which it traverses, thus marking the path by a
trail of this substance.
While an isolated ant moves essentially at
random,
an ant encountering a previously laid trail can
detect it and follow it with a high probability.
85. Sept, 2010
®Copyright of Shun-Feng Su
85
An example with Real Ants
(a) Ants follow a path between A and E.
(b) An obstacle is interposed.
(c) On the shorter path more pheromone is laid down.
86. Sept, 2010
®Copyright of Shun-Feng Su
86
An Example with Artificial Ants
(a) The initial graph with distances.
(b) At time t = 0 there is no trail on the graph edges.
(c) At time t = 1 trail is stronger on shorter edges.
87. Sept, 2010
®Copyright of Shun-Feng Su
87
Ant System
Each ant is a simple agent with the following
characteristics:
it chooses a path to go to with a probability that is
a function of heuristics (distance) and the
amount of trail (pheromone) present on the
connecting edge.
to force an ant to make legal tours, transitions to
already visited towns are disallowed until a tour
is completed.
when it completes a tour, it lays a substance called
trail (pheromone) on each edge visited.
88. Sept, 2010
®Copyright of Shun-Feng Su
88
Pseudo Code of Ant System
The ACO metaheuristic is:
Set parameters, initialize pheromone trails
SCHEDULE_ACTIVITIES
ConstructAntSolutions
DaemonActions {optional}
UpdatePheromones
END_SCHEDULE_ACTIVITIES
89. Sept, 2010
®Copyright of Shun-Feng Su
89
Schedule_Activities
The Schedule_Activities does not specify how the
three algorithmic components are scheduled
and synchronized.
In most applications of ACO to NP-hard problems
however, the three algorithmic components
undergo a loop that consists in (i) the
construction of solutions by all ants, (ii) the
(optional) improvement of these solution via the
use of a local search algorithm, and (iii) the
update of the pheromones.
90. Sept, 2010
®Copyright of Shun-Feng Su
90
ConstructAntSolutions
At each construction step, the current partial solution
is extended by adding a feasible solution
component from the set of feasible neighbors .
The process of constructing solutions can be
regarded as a path on the construction graph
GC(V,E).
The allowed paths in GC are implicitly defined by the
solution construction mechanism that defines the
set with respect to a partial solution .
91. Sept, 2010
®Copyright of Shun-Feng Su
91
ConstructAntSolutions
The choice of a solution component is done
probabilistically. The rules for the probabilistic
choice of solution components vary across
different ACO variants. The best known rule is
the one of ant system (AS):
where and are the pheromone value and the
heuristic value associated with the component.
and are parameters used to representing
the importance of pheromone and heuristics.
roulette wheel
selection
92. Sept, 2010
®Copyright of Shun-Feng Su
92
DaemonActions
Once solutions have been constructed, and before
updating the pheromone values, often some
problem specific actions may be required. These
are often called daemon actions, and can be used
to implement problem specific and/or centralized
actions, which cannot be performed by single ants.
The most used daemon action is the use of local
search to the constructed solutions: the locally
optimized solutions are then used to decide which
pheromone values to update.
93. Sept, 2010
®Copyright of Shun-Feng Su
93
UpdatePheromones
The aim of the pheromone update is to increase the
pheromone values associated with good solutions,
and to decrease those that are associated with bad
ones (not used).
Usually, this is achieved (i) by decreasing all the
pheromone values through pheromone evaporation,
and (ii) by increasing the pheromone levels
associated with a chosen set of good solutions
Evaporation for
all edges Add those who are good.
94. Sept, 2010
®Copyright of Shun-Feng Su
94
Main ACO Algorithms
Several special cases of the ACO metaheuristic
have been proposed in the literature.
Ant System (Dorigo 1992, Dorigo et al. 1991,
1996),
Ant Colony System (ACS) (Dorigo & Gambardella
1997), and
MAX-MIN Ant System (MMAS) (Stützle & Hoos
2000).
95. Sept, 2010
®Copyright of Shun-Feng Su
95
Ant Systems
Ant system (AS) was the first ACO algorithm
proposed in the literature .
Its main characteristic is that the pheromone
values are updated by all ants that have
completed the tour.
When constructing solutions, ants in AS traverse
the construction graph and make a probabilistic
decision at each vertex. The transition
probability is
96. Sept, 2010
®Copyright of Shun-Feng Su
96
Ant Colony Systems
The first major improvement over the original ant system
was ant colony system (ACS), introduced by Dorigo and
Gambardella (1997).
The main difference between ACS and AS is the decision
rule used by the ants during the construction process.
Ants in ACS use the so-called pseudorandom proportional
rule: the probability of selecting next edges depends on
a random variable q uniformly distributed over [0, 1],
and a parameter q0; if q<q0, then, among the feasible
components, the component with maximal pheromone
heurestic is chosen, otherwise the same equation as in
AS is used.
97. Sept, 2010
®Copyright of Shun-Feng Su
97
MAX-MIN Ant Systems
MAX-MIN ant system (MMAS) is another improvement,
proposed by Stützle and Hoos (2000), over the original
ant system idea.
MMAS differs from AS in that (i) only the best ant adds
pheromone trails, and (ii) the minimum and maximum
values of the pheromone are explicitly limited (in AS
and ACS these values are limited implicitly, that is, the
value of the limits is a result of the algorithm working
rather than a value set explicitly by the algorithm
designer).
98. Sept, 2010
®Copyright of Shun-Feng Su
98
Other Non-derivation Optimization
It is because GAs and PSO are solution-wise
search and swarm search algorithms are
component-wise search.
Also, it can be found that solution-wise search
algorithms are easier to be trapped into a
local minimum if the initial population has
some local optimum properties.
Component-wise search algorithms can easily
escape from such an initial local optimum
phenomena.
99. Sept, 2010
®Copyright of Shun-Feng Su
99
Epilogue
Traditional optimization approaches are good but
only for the mathematical form is true and can
be manipulated.
Non derivate optimization is one nice kind of
optimization techniques, but you need to adapt
the methodology to the problem you face.
An often used idea is to adapt your problem to
those traditional NP problems, like Travel
Salesman Problem (TSP), Quadratic
Assignment Problem (QAP), etc.
100. Sept, 2010
®Copyright of Shun-Feng Su
10
0
Epilogue
Non-derivative optimization cannot guarantee the
success of the search.
Most of unsuccessful cases are either the search
converges too slow or the search gets stuck in
local optima.
The guadiance is
not strong
enough
Randomness is
not sufficently
used.
101. Sept, 2010
®Copyright of Shun-Feng Su
10
1
Epilogue
Non-derivative optimization cannot guarantee the
success of the search.
Most of unsuccessful cases are either the search
converges too slow or the search gets stuck in
local optima.
How to strengthen and to balance those two
factors are important issues in the design of
those search approaches.
102. Sept, 2010
®Copyright of Shun-Feng Su
10
2
Thankyoufor yourattention!
Any Questions ?!
Shun-Feng Su,
Professor of Department of Electrical Engineering,
National Taiwan University of Science and Technology
E-mail: su@orion.ee.ntust.edu.tw,