This document outlines a talk on the use of 1 in mathematical programming and summarizes several topics to be covered, including degeneracy and the simplex method, nonlinear optimization, equality constraints, inequality constraints, and data envelopment analysis. It provides details on linear programming and the simplex method, including preliminary results, basic feasible solutions, the single iteration process, and the lexicographic rule for selecting leaving variables. The document contains mathematical notation and definitions to explain these concepts.
This document outlines a talk on using grossone in optimization. It discusses single and multi-objective linear programming and nonlinear optimization. It covers linear programming and the simplex method, including preliminary results, basic feasible solutions, associated bases, and convergence. It also discusses the lexicographic rule and recent results.
This document describes a new implementation of the finite collocation method for solving time-dependent partial differential equations of parabolic type. The method discretizes the time variable using finite differences, resulting in elliptic PDEs for the spatial variables. It then uses a combination of finite collocation and local radial basis function methods for spatial discretization, dividing the domain into local regions to improve stability compared to global RBF methods. The method is computationally efficient due to using strong forms, collocation, and inverting small matrices. The document tests the method on several linear and nonlinear PDEs.
Semantic Web technologies are a set of languages standardized by the World Wide Web Consortium (W3C) and designed to create a web of data that can be processed by machines. One of the core languages of the Semantic Web is Web Ontology Language (OWL), a family of knowledge representation languages for authoring ontologies or knowledge bases. The newest OWL is based on Description Logics (DL), a family of logics that are decidable fragments of first-order logic. leanCoR is a new description logic reasoner designed for experimenting with the new connection method algorithms and optimization techniques for DL. leanCoR is an extension of leanCoP, a compact automated theorem prover for classical first-order logic.
This document provides an overview of Chapter 6 from an introductory mathematical analysis textbook. The chapter covers matrix algebra, including:
- Concepts of matrices, matrix size, and special types of matrices
- Operations of matrix addition, scalar multiplication, and subtraction
- Matrix multiplication and how it relates to the sizes of matrices
- Using matrix operations to represent and solve systems of linear equations
- Applications of matrices including analyzing production sectors of an economy and modeling costs
This document discusses relation matrices and graphs. It begins by defining a relation matrix as a way to represent a relation between two finite sets A and B using a matrix with 1s and 0s. An example is provided to demonstrate how to construct a relation matrix. The document then discusses how relations can be represented using graphs by connecting elements with edges. Properties of relations like reflexive, symmetric, and anti-symmetric are explained through examples using relation matrices. Finally, the conclusion restates that relation matrices and graphs can be used to represent relations between sets.
The document discusses various topics related to analytics including:
1. It defines analytics as transforming data into insights for better decision making and describes the Deming cycle of plan, do, check, act.
2. It provides definitions and descriptions of different types of innovation - product, process, marketing, and organizational innovation.
3. It discusses how analytics can drive innovation and describes descriptive, predictive, and prescriptive analytics categories and common analytics tools.
4. Supply chain management and inventory optimization are provided as examples of analytics applications.
The document discusses the simplex method for solving linear programming problems (LPP). The simplex method is an optimization technique that distributes limited resources between competing programs to obtain the optimal solution. It provides a method for solving LPPs of any size involving two or more decision variables. The simplex method assumes the optimal solution exists at a corner point of the feasible region.
This document outlines a talk on using grossone in optimization. It discusses single and multi-objective linear programming and nonlinear optimization. It covers linear programming and the simplex method, including preliminary results, basic feasible solutions, associated bases, and convergence. It also discusses the lexicographic rule and recent results.
This document describes a new implementation of the finite collocation method for solving time-dependent partial differential equations of parabolic type. The method discretizes the time variable using finite differences, resulting in elliptic PDEs for the spatial variables. It then uses a combination of finite collocation and local radial basis function methods for spatial discretization, dividing the domain into local regions to improve stability compared to global RBF methods. The method is computationally efficient due to using strong forms, collocation, and inverting small matrices. The document tests the method on several linear and nonlinear PDEs.
Semantic Web technologies are a set of languages standardized by the World Wide Web Consortium (W3C) and designed to create a web of data that can be processed by machines. One of the core languages of the Semantic Web is Web Ontology Language (OWL), a family of knowledge representation languages for authoring ontologies or knowledge bases. The newest OWL is based on Description Logics (DL), a family of logics that are decidable fragments of first-order logic. leanCoR is a new description logic reasoner designed for experimenting with the new connection method algorithms and optimization techniques for DL. leanCoR is an extension of leanCoP, a compact automated theorem prover for classical first-order logic.
This document provides an overview of Chapter 6 from an introductory mathematical analysis textbook. The chapter covers matrix algebra, including:
- Concepts of matrices, matrix size, and special types of matrices
- Operations of matrix addition, scalar multiplication, and subtraction
- Matrix multiplication and how it relates to the sizes of matrices
- Using matrix operations to represent and solve systems of linear equations
- Applications of matrices including analyzing production sectors of an economy and modeling costs
This document discusses relation matrices and graphs. It begins by defining a relation matrix as a way to represent a relation between two finite sets A and B using a matrix with 1s and 0s. An example is provided to demonstrate how to construct a relation matrix. The document then discusses how relations can be represented using graphs by connecting elements with edges. Properties of relations like reflexive, symmetric, and anti-symmetric are explained through examples using relation matrices. Finally, the conclusion restates that relation matrices and graphs can be used to represent relations between sets.
The document discusses various topics related to analytics including:
1. It defines analytics as transforming data into insights for better decision making and describes the Deming cycle of plan, do, check, act.
2. It provides definitions and descriptions of different types of innovation - product, process, marketing, and organizational innovation.
3. It discusses how analytics can drive innovation and describes descriptive, predictive, and prescriptive analytics categories and common analytics tools.
4. Supply chain management and inventory optimization are provided as examples of analytics applications.
The document discusses the simplex method for solving linear programming problems (LPP). The simplex method is an optimization technique that distributes limited resources between competing programs to obtain the optimal solution. It provides a method for solving LPPs of any size involving two or more decision variables. The simplex method assumes the optimal solution exists at a corner point of the feasible region.
The document discusses post-optimal analysis in linear optimization problems. It describes how changes can affect feasibility or optimality, including changes to right-hand sides, adding new constraints, or changing objective coefficients. It also discusses adding a new activity/variable and using the dual simplex method to find the new optimal solution.
LINEAR PROGRAMMING Assignment help services at Globalwebtutors are available 24/ by online LINEAR PROGRAMMING experts , LINEAR PROGRAMMING tutors are available for instant LINEAR PROGRAMMING questions help , LINEAR PROGRAMMING writers can help you with complex LINEAR PROGRAMMING dissertation requirements.
This document provides an overview of sensitivity analysis in linear programming, which determines how changes to coefficients or right-hand sides of constraints impact the optimal solution. It discusses standard output from solving linear programs using software, including objective value ranges, variable values and reduced costs, and constraint slack and shadow prices. An example problem is presented to illustrate sensitivity analysis for a company optimizing frame production under material constraints.
This chapter discusses computer solutions and sensitivity analysis for linear programming problems. It covers using software packages like Excel and QM for Windows to solve problems instead of manual methods. Examples are provided to demonstrate solving a problem in Excel and QM as well as conducting sensitivity analysis on objective function coefficients and right-hand side constraint values. Sensitivity analysis determines how changes to parameters impact the optimal solution.
This webinar will provide pesticides residue analysts with valuable information on the development and optimization of gas chromatographic separations and mass spectrometry methods for the analysis of pesticide residues in food. The expert speakers will share their knowledge in understanding the critical points of the method, assisting analysts in modifying existing methods, and understanding instrumental and software technologies with the goal of improving laboratory productivity and reducing the overall cost per sample. The results of experiments for both screening and quantification workflows, using the latest technology, will be presented.
The document discusses sensitivity analysis for a linear programming problem. It provides an example of a manufacturing company that produces two types of grates. The optimal solution from solving the linear program is to produce 120 model A grates and 160 model B grates per day for a maximum profit of Rs. 480. Sensitivity analysis is then performed to determine how changes to the objective function coefficients and right-hand side constants of the constraints impact the optimal solution. The ranges that each coefficient can change without affecting optimality are identified.
Linear programming using the simplex methodShivek Khurana
This document discusses linear programming and provides an example of using the simplex method to solve a crop plantation optimization problem. It begins with a brief history of linear programming and describes how the simplex method works by introducing slack variables and performing row operations to iteratively find an optimal solution. It then presents a crop plantation problem involving allocating land between potatoes and ladyfingers to maximize profit while satisfying pesticide, manure and land constraints. The simplex method is applied step-by-step to arrive at the optimal solution that allocates 1.42 acres to ladyfingers and yields a maximum profit of 8.57.
The document discusses the simplex method for solving linear programming problems. It introduces some key terminology used in the simplex method like slack variables, surplus variables, and artificial variables. It then provides an overview of how the simplex method works for maximization problems, including forming the initial simplex table, testing for optimality and feasibility, pivoting to find an optimal solution. Finally, it provides an example application of the simplex method to a sample maximization problem.
This document discusses several types of complications that can occur when solving linear programming problems (LPP), including degeneracy, unbounded problems, multiple optimal solutions, infeasible problems, and redundant or unrestricted variables. It provides examples and explanations of how to identify each type of complication and the appropriate steps to resolve it such as introducing slack or artificial variables, breaking ties, or setting unrestricted variables equal to the difference of two non-negative variables.
The document discusses iterative improvement algorithms and provides examples such as the simplex method for solving linear programming problems. It explains the standard form of a linear programming problem and gives an outline of the simplex method, which generates a sequence of feasible solutions with improving objective values until an optimal solution is found. Some notes on limitations of the simplex method and improvements like the ellipsoid and interior-point methods are also mentioned.
The document discusses duality theory in linear programming (LP). It explains that for every LP primal problem, there exists an associated dual problem. The primal problem aims to optimize resource allocation, while the dual problem aims to determine the appropriate valuation of resources. The relationship between primal and dual problems is fundamental to duality theory. The document provides examples of primal and dual problems and their formulations. It also outlines some general rules for constructing the dual problem from the primal, as well as relations between optimal solutions of primal and dual problems.
Numerical analysis dual, primal, revised simplexSHAMJITH KM
The document discusses linear programming optimization methods including the revised simplex method, duality of LP problems, dual simplex method, and sensitivity analysis. It provides details on the iterative steps of the revised and dual simplex methods. Examples are provided to illustrate finding the dual of a primal LP problem and solving the dual using the dual simplex method. Sensitivity analysis is introduced as examining how changes to coefficients or right-hand sides can impact optimality and feasibility.
The document provides an overview of the simplex method for solving linear programming problems with more than two decision variables. It describes key concepts like slack variables, surplus variables, basic feasible solutions, degenerate and non-degenerate solutions, and using tableau steps to arrive at an optimal solution. Examples are provided to illustrate setting up and solving problems using the simplex method.
This document provides an overview of techniques for formalized sensitivity analysis and expected value decisions. It discusses determining sensitivity to parameter variation, using three estimates to analyze sensitivity, calculating expected values of cash flows and alternatives, and using decision trees to model staged evaluations under uncertainty. The techniques aim to account for variability in parameters, quantify uncertainty through probabilities, and identify best decisions considering risk.
-What is Sensitivity Analysis in Project Risk Management?
-Example on Sensitivity Analysis….
-Types of Sensitivity Analysis……
-Advantages & Disadvantages
Sensitivity analysis determines how sensitive the optimal solution is to changes made to the original linear programming model after obtaining the optimal solution. It is important because it allows analysts to check how changes to the data in the model, such as the coefficients, constraints, or variables, would affect the optimal solution and gives the model dynamic characteristics to handle potential future changes.
This document presents information on sensitivity analysis techniques. It discusses how sensitivity analysis is used to determine how changes to independent variables impact dependent variables given certain assumptions. It also describes how sensitivity analysis can predict outcomes if a situation differs from key predictions. Various sensitivity analysis methods are outlined, including correlation and screening techniques, regression analysis, and analyzing oscillations through measuring behavior patterns like period and amplitude. An example of applying sensitivity analysis to a simple supply chain model is also provided.
The document discusses sensitivity analysis, which quantifies how changes in design parameters or analysis errors affect predicted performance. As an example, it analyzes how changes in lift coefficient (C Lmax ) and weight (W) would affect the predicted take-off distance (S LO ) of an aircraft. Using both linear and nonlinear sensitivity analysis, it finds:
1) A ±0.1 change in C Lmax would result in a ±0.1S LO change in take-off distance according to linear analysis. Nonlinear analysis estimates changes of ±353 feet, in close agreement.
2) A 10% increase in weight would lead to a 0.1S LO or 636 feet increase in take
This document outlines a talk on nonlinear programming and grossone theory and algorithms. It discusses equality constraints, inequality constraints, quadratic problems, and algorithms. For equality constraints, it presents the Lagrangian function and Karush-Kuhn-Tucker (KKT) first-order optimality conditions. It then discusses penalty functions and the sequential penalty method. Two examples applying the theory to problems with equality constraints are provided. Inequality constraints and first-order optimality conditions for problems with equality and inequality constraints are also covered.
The document discusses post-optimal analysis in linear optimization problems. It describes how changes can affect feasibility or optimality, including changes to right-hand sides, adding new constraints, or changing objective coefficients. It also discusses adding a new activity/variable and using the dual simplex method to find the new optimal solution.
LINEAR PROGRAMMING Assignment help services at Globalwebtutors are available 24/ by online LINEAR PROGRAMMING experts , LINEAR PROGRAMMING tutors are available for instant LINEAR PROGRAMMING questions help , LINEAR PROGRAMMING writers can help you with complex LINEAR PROGRAMMING dissertation requirements.
This document provides an overview of sensitivity analysis in linear programming, which determines how changes to coefficients or right-hand sides of constraints impact the optimal solution. It discusses standard output from solving linear programs using software, including objective value ranges, variable values and reduced costs, and constraint slack and shadow prices. An example problem is presented to illustrate sensitivity analysis for a company optimizing frame production under material constraints.
This chapter discusses computer solutions and sensitivity analysis for linear programming problems. It covers using software packages like Excel and QM for Windows to solve problems instead of manual methods. Examples are provided to demonstrate solving a problem in Excel and QM as well as conducting sensitivity analysis on objective function coefficients and right-hand side constraint values. Sensitivity analysis determines how changes to parameters impact the optimal solution.
This webinar will provide pesticides residue analysts with valuable information on the development and optimization of gas chromatographic separations and mass spectrometry methods for the analysis of pesticide residues in food. The expert speakers will share their knowledge in understanding the critical points of the method, assisting analysts in modifying existing methods, and understanding instrumental and software technologies with the goal of improving laboratory productivity and reducing the overall cost per sample. The results of experiments for both screening and quantification workflows, using the latest technology, will be presented.
The document discusses sensitivity analysis for a linear programming problem. It provides an example of a manufacturing company that produces two types of grates. The optimal solution from solving the linear program is to produce 120 model A grates and 160 model B grates per day for a maximum profit of Rs. 480. Sensitivity analysis is then performed to determine how changes to the objective function coefficients and right-hand side constants of the constraints impact the optimal solution. The ranges that each coefficient can change without affecting optimality are identified.
Linear programming using the simplex methodShivek Khurana
This document discusses linear programming and provides an example of using the simplex method to solve a crop plantation optimization problem. It begins with a brief history of linear programming and describes how the simplex method works by introducing slack variables and performing row operations to iteratively find an optimal solution. It then presents a crop plantation problem involving allocating land between potatoes and ladyfingers to maximize profit while satisfying pesticide, manure and land constraints. The simplex method is applied step-by-step to arrive at the optimal solution that allocates 1.42 acres to ladyfingers and yields a maximum profit of 8.57.
The document discusses the simplex method for solving linear programming problems. It introduces some key terminology used in the simplex method like slack variables, surplus variables, and artificial variables. It then provides an overview of how the simplex method works for maximization problems, including forming the initial simplex table, testing for optimality and feasibility, pivoting to find an optimal solution. Finally, it provides an example application of the simplex method to a sample maximization problem.
This document discusses several types of complications that can occur when solving linear programming problems (LPP), including degeneracy, unbounded problems, multiple optimal solutions, infeasible problems, and redundant or unrestricted variables. It provides examples and explanations of how to identify each type of complication and the appropriate steps to resolve it such as introducing slack or artificial variables, breaking ties, or setting unrestricted variables equal to the difference of two non-negative variables.
The document discusses iterative improvement algorithms and provides examples such as the simplex method for solving linear programming problems. It explains the standard form of a linear programming problem and gives an outline of the simplex method, which generates a sequence of feasible solutions with improving objective values until an optimal solution is found. Some notes on limitations of the simplex method and improvements like the ellipsoid and interior-point methods are also mentioned.
The document discusses duality theory in linear programming (LP). It explains that for every LP primal problem, there exists an associated dual problem. The primal problem aims to optimize resource allocation, while the dual problem aims to determine the appropriate valuation of resources. The relationship between primal and dual problems is fundamental to duality theory. The document provides examples of primal and dual problems and their formulations. It also outlines some general rules for constructing the dual problem from the primal, as well as relations between optimal solutions of primal and dual problems.
Numerical analysis dual, primal, revised simplexSHAMJITH KM
The document discusses linear programming optimization methods including the revised simplex method, duality of LP problems, dual simplex method, and sensitivity analysis. It provides details on the iterative steps of the revised and dual simplex methods. Examples are provided to illustrate finding the dual of a primal LP problem and solving the dual using the dual simplex method. Sensitivity analysis is introduced as examining how changes to coefficients or right-hand sides can impact optimality and feasibility.
The document provides an overview of the simplex method for solving linear programming problems with more than two decision variables. It describes key concepts like slack variables, surplus variables, basic feasible solutions, degenerate and non-degenerate solutions, and using tableau steps to arrive at an optimal solution. Examples are provided to illustrate setting up and solving problems using the simplex method.
This document provides an overview of techniques for formalized sensitivity analysis and expected value decisions. It discusses determining sensitivity to parameter variation, using three estimates to analyze sensitivity, calculating expected values of cash flows and alternatives, and using decision trees to model staged evaluations under uncertainty. The techniques aim to account for variability in parameters, quantify uncertainty through probabilities, and identify best decisions considering risk.
-What is Sensitivity Analysis in Project Risk Management?
-Example on Sensitivity Analysis….
-Types of Sensitivity Analysis……
-Advantages & Disadvantages
Sensitivity analysis determines how sensitive the optimal solution is to changes made to the original linear programming model after obtaining the optimal solution. It is important because it allows analysts to check how changes to the data in the model, such as the coefficients, constraints, or variables, would affect the optimal solution and gives the model dynamic characteristics to handle potential future changes.
This document presents information on sensitivity analysis techniques. It discusses how sensitivity analysis is used to determine how changes to independent variables impact dependent variables given certain assumptions. It also describes how sensitivity analysis can predict outcomes if a situation differs from key predictions. Various sensitivity analysis methods are outlined, including correlation and screening techniques, regression analysis, and analyzing oscillations through measuring behavior patterns like period and amplitude. An example of applying sensitivity analysis to a simple supply chain model is also provided.
The document discusses sensitivity analysis, which quantifies how changes in design parameters or analysis errors affect predicted performance. As an example, it analyzes how changes in lift coefficient (C Lmax ) and weight (W) would affect the predicted take-off distance (S LO ) of an aircraft. Using both linear and nonlinear sensitivity analysis, it finds:
1) A ±0.1 change in C Lmax would result in a ±0.1S LO change in take-off distance according to linear analysis. Nonlinear analysis estimates changes of ±353 feet, in close agreement.
2) A 10% increase in weight would lead to a 0.1S LO or 636 feet increase in take
This document outlines a talk on nonlinear programming and grossone theory and algorithms. It discusses equality constraints, inequality constraints, quadratic problems, and algorithms. For equality constraints, it presents the Lagrangian function and Karush-Kuhn-Tucker (KKT) first-order optimality conditions. It then discusses penalty functions and the sequential penalty method. Two examples applying the theory to problems with equality constraints are provided. Inequality constraints and first-order optimality conditions for problems with equality and inequality constraints are also covered.
The document discusses power production and storage in microgrids. It presents a case study of optimizing the Leaf Community microgrid in Italy, which contains a photovoltaic plant, hydroelectric plant, battery storage, and loads from an office building and industrial facility. The goal is to minimize energy costs by determining the optimal strategy for buying and selling power to the grid and charging/discharging the battery storage. The optimization problem is formulated as a mixed-integer linear program to minimize costs while meeting loads based on forecasts of renewable production and demand over multiple days. The results show that renewable energy is used first to meet loads and the battery charges from low-cost power and discharges during high-cost periods.
The document describes a system dynamics approach to modeling the airplane boarding process. Key points:
1. A system dynamics model was developed to better understand the boarding system's behavior and provide strategic help to airlines in simulating different boarding policies.
2. The model considers passengers as stocks that flow through the boarding process. Interactions and delays caused by passengers are modeled to capture feedback loops.
3. Simulations tested different boarding strategies like random boarding and back-to-front boarding to analyze their effects on reducing boarding time. The model provided insights into optimizing the boarding process.
Nucleophilic Addition of carbonyl compounds.pptxSSR02
Nucleophilic addition is the most important reaction of carbonyls. Not just aldehydes and ketones, but also carboxylic acid derivatives in general.
Carbonyls undergo addition reactions with a large range of nucleophiles.
Comparing the relative basicity of the nucleophile and the product is extremely helpful in determining how reversible the addition reaction is. Reactions with Grignards and hydrides are irreversible. Reactions with weak bases like halides and carboxylates generally don’t happen.
Electronic effects (inductive effects, electron donation) have a large impact on reactivity.
Large groups adjacent to the carbonyl will slow the rate of reaction.
Neutral nucleophiles can also add to carbonyls, although their additions are generally slower and more reversible. Acid catalysis is sometimes employed to increase the rate of addition.
ESPP presentation to EU Waste Water Network, 4th June 2024 “EU policies driving nutrient removal and recycling
and the revised UWWTD (Urban Waste Water Treatment Directive)”
What is greenhouse gasses and how many gasses are there to affect the Earth.moosaasad1975
What are greenhouse gasses how they affect the earth and its environment what is the future of the environment and earth how the weather and the climate effects.
The debris of the ‘last major merger’ is dynamically youngSérgio Sacani
The Milky Way’s (MW) inner stellar halo contains an [Fe/H]-rich component with highly eccentric orbits, often referred to as the
‘last major merger.’ Hypotheses for the origin of this component include Gaia-Sausage/Enceladus (GSE), where the progenitor
collided with the MW proto-disc 8–11 Gyr ago, and the Virgo Radial Merger (VRM), where the progenitor collided with the
MW disc within the last 3 Gyr. These two scenarios make different predictions about observable structure in local phase space,
because the morphology of debris depends on how long it has had to phase mix. The recently identified phase-space folds in Gaia
DR3 have positive caustic velocities, making them fundamentally different than the phase-mixed chevrons found in simulations
at late times. Roughly 20 per cent of the stars in the prograde local stellar halo are associated with the observed caustics. Based
on a simple phase-mixing model, the observed number of caustics are consistent with a merger that occurred 1–2 Gyr ago.
We also compare the observed phase-space distribution to FIRE-2 Latte simulations of GSE-like mergers, using a quantitative
measurement of phase mixing (2D causticality). The observed local phase-space distribution best matches the simulated data
1–2 Gyr after collision, and certainly not later than 3 Gyr. This is further evidence that the progenitor of the ‘last major merger’
did not collide with the MW proto-disc at early times, as is thought for the GSE, but instead collided with the MW disc within
the last few Gyr, consistent with the body of work surrounding the VRM.
ANAMOLOUS SECONDARY GROWTH IN DICOT ROOTS.pptxRASHMI M G
Abnormal or anomalous secondary growth in plants. It defines secondary growth as an increase in plant girth due to vascular cambium or cork cambium. Anomalous secondary growth does not follow the normal pattern of a single vascular cambium producing xylem internally and phloem externally.
This presentation explores a brief idea about the structural and functional attributes of nucleotides, the structure and function of genetic materials along with the impact of UV rays and pH upon them.
Current Ms word generated power point presentation covers major details about the micronuclei test. It's significance and assays to conduct it. It is used to detect the micronuclei formation inside the cells of nearly every multicellular organism. It's formation takes place during chromosomal sepration at metaphase.
Or: Beyond linear.
Abstract: Equivariant neural networks are neural networks that incorporate symmetries. The nonlinear activation functions in these networks result in interesting nonlinear equivariant maps between simple representations, and motivate the key player of this talk: piecewise linear representation theory.
Disclaimer: No one is perfect, so please mind that there might be mistakes and typos.
dtubbenhauer@gmail.com
Corrected slides: dtubbenhauer.com/talks.html
When I was asked to give a companion lecture in support of ‘The Philosophy of Science’ (https://shorturl.at/4pUXz) I decided not to walk through the detail of the many methodologies in order of use. Instead, I chose to employ a long standing, and ongoing, scientific development as an exemplar. And so, I chose the ever evolving story of Thermodynamics as a scientific investigation at its best.
Conducted over a period of >200 years, Thermodynamics R&D, and application, benefitted from the highest levels of professionalism, collaboration, and technical thoroughness. New layers of application, methodology, and practice were made possible by the progressive advance of technology. In turn, this has seen measurement and modelling accuracy continually improved at a micro and macro level.
Perhaps most importantly, Thermodynamics rapidly became a primary tool in the advance of applied science/engineering/technology, spanning micro-tech, to aerospace and cosmology. I can think of no better a story to illustrate the breadth of scientific methodologies and applications at their best.
Remote Sensing and Computational, Evolutionary, Supercomputing, and Intellige...University of Maribor
Slides from talk:
Aleš Zamuda: Remote Sensing and Computational, Evolutionary, Supercomputing, and Intelligent Systems.
11th International Conference on Electrical, Electronics and Computer Engineering (IcETRAN), Niš, 3-6 June 2024
Inter-Society Networking Panel GRSS/MTT-S/CIS Panel Session: Promoting Connection and Cooperation
https://www.etran.rs/2024/en/home-english/
mô tả các thí nghiệm về đánh giá tác động dòng khí hóa sau đốt
Main
1. The use of ① in Mathematical Programming
R. De Leone
School of Science and Tecnology
Universit `a di Camerino
June 2013
NUMTA2013 1 / 46
2. Outline of the talk
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
Degeneracy and the Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment Analysis
NUMTA2013 2 / 46
3. Degeneracy and the Simplex
Method
• Outline of the talk
Degeneracy and the
Simplex Method
• Linear Programming
and the Simplex Method
• Preliminary results
and notations
• BFS and associated
basis
• Single iteration of the
simplex method
• Lexicographic Rule
• Lexicographic rule
and RHS perturbation
• Lexicographic rule
and RHS perturbation
and ①
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
NUMTA2013 3 / 46
4. Linear Programming and the Simplex Method
• Outline of the talk
Degeneracy and the
Simplex Method
• Linear Programming
and the Simplex Method
• Preliminary results
and notations
• BFS and associated
basis
• Single iteration of the
simplex method
• Lexicographic Rule
• Lexicographic rule
and RHS perturbation
• Lexicographic rule
and RHS perturbation
and ①
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
min
x
cT x
subject to Ax = b
x ≥ 0
The simplex method proposed by George Dantzig in 1947
• start at a corner point (a Basic Feasible Solution, BFS)
• verify if the current point is optimal
• if not, moves along an edge to a new corner point
until the optimal corner point is identified or it discovers that the
problem has no solution.
NUMTA2013 4 / 46
5. Preliminary results and notations
• Outline of the talk
Degeneracy and the
Simplex Method
• Linear Programming
and the Simplex Method
• Preliminary results
and notations
• BFS and associated
basis
• Single iteration of the
simplex method
• Lexicographic Rule
• Lexicographic rule
and RHS perturbation
• Lexicographic rule
and RHS perturbation
and ①
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
Let
X = {x ∈ IRn : Ax = b, x ≥ 0}
where A ∈ IRm×n, b ∈ IRm, m ≤ n.
A point ¯x ∈ X is a Basic Feasible Solution (BFS) iff the columns of
A corresponding to positive components of ¯x are linearly
independent.
NUMTA2013 5 / 46
6. Preliminary results and notations
• Outline of the talk
Degeneracy and the
Simplex Method
• Linear Programming
and the Simplex Method
• Preliminary results
and notations
• BFS and associated
basis
• Single iteration of the
simplex method
• Lexicographic Rule
• Lexicographic rule
and RHS perturbation
• Lexicographic rule
and RHS perturbation
and ①
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
Let
X = {x ∈ IRn : Ax = b, x ≥ 0}
where A ∈ IRm×n, b ∈ IRm, m ≤ n.
A point ¯x ∈ X is a Basic Feasible Solution (BFS) iff the columns of
A corresponding to positive components of ¯x are linearly
independent.
Let ¯x be a BFS and define ¯I = I(¯x) := {j : ¯xj > 0} then
rank(A.¯I ) = |¯I|. Note: |¯I| ≤ m
NUMTA2013 5 / 46
7. Preliminary results and notations
• Outline of the talk
Degeneracy and the
Simplex Method
• Linear Programming
and the Simplex Method
• Preliminary results
and notations
• BFS and associated
basis
• Single iteration of the
simplex method
• Lexicographic Rule
• Lexicographic rule
and RHS perturbation
• Lexicographic rule
and RHS perturbation
and ①
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
Let
X = {x ∈ IRn : Ax = b, x ≥ 0}
where A ∈ IRm×n, b ∈ IRm, m ≤ n.
A point ¯x ∈ X is a Basic Feasible Solution (BFS) iff the columns of
A corresponding to positive components of ¯x are linearly
independent.
Let ¯x be a BFS and define ¯I = I(¯x) := {j : ¯xj > 0} then
rank(A.¯I ) = |¯I|. Note: |¯I| ≤ m
BFS ≡ Vertex ≡ Extreme Point
Vertex Point, Extreme Points and Basic Fea-sible
Solution Point coincide
NUMTA2013 5 / 46
8. BFS and associated basis
• Outline of the talk
Degeneracy and the
Simplex Method
• Linear Programming
and the Simplex Method
• Preliminary results
and notations
• BFS and associated
basis
• Single iteration of the
simplex method
• Lexicographic Rule
• Lexicographic rule
and RHS perturbation
• Lexicographic rule
and RHS perturbation
and ①
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
Assume now
rank(A) = m ≤ n
A base B is a subset of m linearly independent columns of A.
B ⊆ {1, . . . , n} , det(A.B)6= 0
N = {1, . . . , n} − B
Let ¯x be a BFS. .
NUMTA2013 6 / 46
9. BFS and associated basis
• Outline of the talk
Degeneracy and the
Simplex Method
• Linear Programming
and the Simplex Method
• Preliminary results
and notations
• BFS and associated
basis
• Single iteration of the
simplex method
• Lexicographic Rule
• Lexicographic rule
and RHS perturbation
• Lexicographic rule
and RHS perturbation
and ①
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
Assume now
rank(A) = m ≤ n
A base B is a subset of m linearly independent columns of A.
B ⊆ {1, . . . , n} , det(A.B)6= 0
N = {1, . . . , n} − B
Let ¯x be a BFS. .
non-degenerate BFS
If |{j : ¯xj > 0}| = m the BFS is said to be non–
degenerate and there is only a single base B :=
{j : ¯xj > 0} associated to ¯x
NUMTA2013 6 / 46
10. BFS and associated basis
• Outline of the talk
Degeneracy and the
Simplex Method
• Linear Programming
and the Simplex Method
• Preliminary results
and notations
• BFS and associated
basis
• Single iteration of the
simplex method
• Lexicographic Rule
• Lexicographic rule
and RHS perturbation
• Lexicographic rule
and RHS perturbation
and ①
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
Assume now
rank(A) = m ≤ n
A base B is a subset of m linearly independent columns of A.
B ⊆ {1, . . . , n} , det(A.B)6= 0
N = {1, . . . , n} − B
Let ¯x be a BFS. .
degenerate BFS
If |{j : ¯xj > 0}| < m the BFS is said to be degener-ate
and there are more than one base B1,B2, . . . ,Bl
associated to ¯x with {j : ¯xj > 0} ⊆ Bi
NUMTA2013 6 / 46
11. BFS and associated basis
• Outline of the talk
Degeneracy and the
Simplex Method
• Linear Programming
and the Simplex Method
• Preliminary results
and notations
• BFS and associated
basis
• Single iteration of the
simplex method
• Lexicographic Rule
• Lexicographic rule
and RHS perturbation
• Lexicographic rule
and RHS perturbation
and ①
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
Assume now
rank(A) = m ≤ n
A base B is a subset of m linearly independent columns of A.
B ⊆ {1, . . . , n} , det(A.B)6= 0
N = {1, . . . , n} − B
Let ¯x be a BFS. Let B a base associated to ¯x.
Then
¯xN = 0, ¯xB = A−1
.B b ≥ 0
NUMTA2013 6 / 46
12. BFS and associated basis
• Outline of the talk
Degeneracy and the
Simplex Method
• Linear Programming
and the Simplex Method
• Preliminary results
and notations
• BFS and associated
basis
• Single iteration of the
simplex method
• Lexicographic Rule
• Lexicographic rule
and RHS perturbation
• Lexicographic rule
and RHS perturbation
and ①
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
Assume now
rank(A) = m ≤ n
A base B is a subset of m linearly independent columns of A.
B ⊆ {1, . . . , n} , det(A.B)6= 0
N = {1, . . . , n} − B
Let ¯x be a BFS. Let B a base associated to ¯x.
Any feasible point x in X can be expressed in term of the base B as
follows:
xB = A−1
.B b + A−1
.B A.NxN
with xN ≥ 0 (and xB ≥ 0)
NUMTA2013 6 / 46
13. Single iteration of the simplex method
• Outline of the talk
Degeneracy and the
Simplex Method
• Linear Programming
and the Simplex Method
• Preliminary results
and notations
• BFS and associated
basis
• Single iteration of the
simplex method
• Lexicographic Rule
• Lexicographic rule
and RHS perturbation
• Lexicographic rule
and RHS perturbation
and ①
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
Step 0 Let B ⊆ {1, . . . , n} be the current base and let x ∈ X the
current BFS
xB = A−1
.B b ≥ 0, xN = 0, |B| = m.
Assume
B = {j1, j2, . . . , jm}
and
N = {1, . . . , n} − B = {jm+1, . . . , jn} .
NUMTA2013 7 / 46
14. Single iteration of the simplex method
• Outline of the talk
Degeneracy and the
Simplex Method
• Linear Programming
and the Simplex Method
• Preliminary results
and notations
• BFS and associated
basis
• Single iteration of the
simplex method
• Lexicographic Rule
• Lexicographic rule
and RHS perturbation
• Lexicographic rule
and RHS perturbation
and ①
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
Step 1 Compute
π = A−T
.B cB
and the reduced cost vector
¯cjk = cjk − A.jk
T π, k = m + 1, . . . , n.
Step 2 If
¯cjk ≥ 0, ∀k = m + 1, . . . , n
the currect point is an optimal BFS and the algorithm stops.
Instead if ¯cN6≥ 0 choose jr with r ∈ {m + 1, . . . , n}) with
¯cjr < 0. This is the variable candidate to enter the base.
NUMTA2013 7 / 46
15. Single iteration of the simplex method
• Outline of the talk
Degeneracy and the
Simplex Method
• Linear Programming
and the Simplex Method
• Preliminary results
and notations
• BFS and associated
basis
• Single iteration of the
simplex method
• Lexicographic Rule
• Lexicographic rule
and RHS perturbation
• Lexicographic rule
and RHS perturbation
and ①
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
Step 3 Compute
A¯.jr = A−1
.B A.jr
Step 4 If
A¯.jr ≤ 0
the problem is unbounded below and the algorithm stops.
Otherwise, compute
¯ρ = min
i=1,...m
¯A
ijr
>0
(
A−1
i
.B b
A¯ijr
)
and let s ∈ {1, . . . ,m} such that
(A−1
.B b)s
A¯sjr
= ¯ρ
js is the leaving variable.
NUMTA2013 7 / 46
16. Single iteration of the simplex method
• Outline of the talk
Degeneracy and the
Simplex Method
• Linear Programming
and the Simplex Method
• Preliminary results
and notations
• BFS and associated
basis
• Single iteration of the
simplex method
• Lexicographic Rule
• Lexicographic rule
and RHS perturbation
• Lexicographic rule
and RHS perturbation
and ①
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
Step 5 Define
¯xjk = 0, k = m + 1, . . . , n, k6= r
¯xjr = ¯ρ
¯xB(ρ) = A−1
.B b − ρ¯A¯.jr .
and
¯B
= B − {js} ∪ {jr} = {j1, j2, . . . , js−1, jr, js+1, . . . , jm}
NUMTA2013 7 / 46
17. Lexicographic Rule
• Outline of the talk
Degeneracy and the
Simplex Method
• Linear Programming
and the Simplex Method
• Preliminary results
and notations
• BFS and associated
basis
• Single iteration of the
simplex method
• Lexicographic Rule
• Lexicographic rule
and RHS perturbation
• Lexicographic rule
and RHS perturbation
and ①
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
At each iteration of the simplex method we choose the leaving
variable using the lexicographic rule
NUMTA2013 8 / 46
18. Lexicographic Rule
• Outline of the talk
Degeneracy and the
Simplex Method
• Linear Programming
and the Simplex Method
• Preliminary results
and notations
• BFS and associated
basis
• Single iteration of the
simplex method
• Lexicographic Rule
• Lexicographic rule
and RHS perturbation
• Lexicographic rule
and RHS perturbation
and ①
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
Let B0 be the initial base and N0 = {1, . . . , n} − B0.
We can always assume, after columns reordering, that A has the
form
A =
A.Bo
...
A.No
NUMTA2013 8 / 46
19. Lexicographic Rule
• Outline of the talk
Degeneracy and the
Simplex Method
• Linear Programming
and the Simplex Method
• Preliminary results
and notations
• BFS and associated
basis
• Single iteration of the
simplex method
• Lexicographic Rule
• Lexicographic rule
and RHS perturbation
• Lexicographic rule
and RHS perturbation
and ①
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
Let
¯ρ = min
i:A¯ijr0
(A.−1
B b)i
A¯ijr
if such minimum value is reached in only one index this is the leaving
variable.
OTHERWISE
NUMTA2013 8 / 46
20. Lexicographic Rule
• Outline of the talk
Degeneracy and the
Simplex Method
• Linear Programming
and the Simplex Method
• Preliminary results
and notations
• BFS and associated
basis
• Single iteration of the
simplex method
• Lexicographic Rule
• Lexicographic rule
and RHS perturbation
• Lexicographic rule
and RHS perturbation
and ①
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
Among the indices i for which
min
i:A¯ijr0
(A.−1
B b)i
A¯ijr
= ¯ρ
we choose the index for which
min
i:A¯ijr0
(A.−1
B A.Bo)i1
A¯ijr
If the minimum is reached by only one index this is the leaving
variable.
OTHERWISE
NUMTA2013 8 / 46
21. Lexicographic Rule
• Outline of the talk
Degeneracy and the
Simplex Method
• Linear Programming
and the Simplex Method
• Preliminary results
and notations
• BFS and associated
basis
• Single iteration of the
simplex method
• Lexicographic Rule
• Lexicographic rule
and RHS perturbation
• Lexicographic rule
and RHS perturbation
and ①
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
Among the indices reaching the minimum value, choose the index for
which
min
i:A¯ijr0
(A.−1
B A.Bo)i2
A¯ijr
Proceed in the same way.
This procedure will terminate providing a single index: the rows of
the matrix (A.−1
B A.Bo) are linearly independent.
NUMTA2013 8 / 46
22. Lexicographic rule and RHS perturbation
• Outline of the talk
Degeneracy and the
Simplex Method
• Linear Programming
and the Simplex Method
• Preliminary results
and notations
• BFS and associated
basis
• Single iteration of the
simplex method
• Lexicographic Rule
• Lexicographic rule
and RHS perturbation
• Lexicographic rule
and RHS perturbation
and ①
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
The procedure outlined in the previous slides is equivalent to perturb
each component of the RHS vector b by a very small quantity.
If this perturbation is small enough, the new linear programming
problem is nondegerate and the simplex method produces exactly
the same pivot sequence as the lexicographic pivot rule
However, is very difficult to determine how small this perturbation
must be. More often a symbolic perturbation is used (with higher
computational costs)
NUMTA2013 9 / 46
23. Lexicographic rule and RHS perturbation and ①
• Outline of the talk
Degeneracy and the
Simplex Method
• Linear Programming
and the Simplex Method
• Preliminary results
and notations
• BFS and associated
basis
• Single iteration of the
simplex method
• Lexicographic Rule
• Lexicographic rule
and RHS perturbation
• Lexicographic rule
and RHS perturbation
and ①
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
Replace bi witheb
i with
bi +
X
j∈Bo
Aij①−j .
NUMTA2013 10 / 46
24. Lexicographic rule and RHS perturbation and ①
• Outline of the talk
Degeneracy and the
Simplex Method
• Linear Programming
and the Simplex Method
• Preliminary results
and notations
• BFS and associated
basis
• Single iteration of the
simplex method
• Lexicographic Rule
• Lexicographic rule
and RHS perturbation
• Lexicographic rule
and RHS perturbation
and ①
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
Replace bi witheb
i with
bi +
X
j∈Bo
Aij①−j .
Let
e =
①−1
①−2
...
①−m
and
eb
= A.−1
B (b + A.Boe) = A.−1
B b + A.−1
B A.Boe.
NUMTA2013 10 / 46
25. Lexicographic rule and RHS perturbation and ①
• Outline of the talk
Degeneracy and the
Simplex Method
• Linear Programming
and the Simplex Method
• Preliminary results
and notations
• BFS and associated
basis
• Single iteration of the
simplex method
• Lexicographic Rule
• Lexicographic rule
and RHS perturbation
• Lexicographic rule
and RHS perturbation
and ①
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
Replace bi witheb
i with
bi +
X
j∈Bo
Aij①−j .
Thereforeeb= (A.−1
B b)i +
Xm
k=1
(A.−1
B A.Bo)ik①−k
and
min
i:A¯ijr0
(A.−1
B b)i +
Xm
k=1
(A.−1
B A.Bo)ik①−k
A¯ijr
=
min
i:A¯ijr0
(A.−1
B b)i
A¯ijr
+
(A.−1
B A.Bo)i1
A¯ijr
①−1+. . .+
(A.−1
B A.Bo)im
A¯ijr
①−m
NUMTA2013 10 / 46
26. Nonlinear Optimization
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
NUMTA2013 11 / 46
27. Equality Constraint
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
NUMTA2013 12 / 46
28. The Equality Constraint Nonlinear Problem
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
min
x
f(x)
subject to h(x) = 0
where f : IRn → IR and h : IRn → IRk
L(x, π) := f(x) +
Xk
j=1
πjhj(x) = f(x) + πT h(x)
NUMTA2013 13 / 46
29. First Order Optimality Conditions
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
Let
F(x∗) :=
n
d ∈ Rn : ∇hi(x∗)T d = 0
o
d ∈ TX(x∗) ⇐⇒ ∃{xl}l feasible points with {xl}l → x∗ and
{tl}l real positive number with {tl}l → 0 such that
liml
xl − x∗
tl
= d
NUMTA2013 14 / 46
30. First Order Optimality Conditions
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
Constraints Qualifications
The set of tangent directions TX(x∗) coincides with the
set of “linearized” feasible directions F(x∗).
Regularity Conditions
NUMTA2013 14 / 46
31. First Order Optimality Conditions
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
Let x∗ ∈ IRn and assume that rank(∇hi(x∗)) = k (i.e., the
columns are linearly independent)
If x∗ is a local minimizer then
NUMTA2013 14 / 46
32. First Order Optimality Conditions
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
Let x∗ ∈ IRn and assume that rank(∇hi(x∗)) = k (i.e., the
columns are linearly independent)
If x∗ is a local minimizer then
there exists π∗ ∈ IRk such that
∇xL(x∗, π∗) = ∇f(x∗) +
Xk
j=1
∇hj(x∗)π∗j = 0
∇L(x∗, π∗) = h(x∗) = 0
NUMTA2013 14 / 46
33. First Order Optimality Conditions
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
Let x∗ ∈ IRn and assume that rank(∇hi(x∗)) = k (i.e., the
columns are linearly independent)
If x∗ is a local minimizer then
there exists π∗ ∈ IRk such that
∇xL(x∗, π∗) = ∇f(x∗) + ∇h(x∗)T π∗ = 0
∇L(x∗, π∗) = h(x∗) = 0
NUMTA2013 14 / 46
34. First Order Optimality Conditions
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
Let x∗ ∈ IRn and assume that rank(∇hi(x∗)) = k (i.e., the
columns are linearly independent)
If x∗ is a local minimizer then
there exists π∗ ∈ IRk such that
∇xL(x∗, π∗) = ∇f(x∗) + ∇h(x∗)T π∗ = 0
∇L(x∗, π∗) = h(x∗) = 0
KKT (Karush–Kuhn–Tucker) Conditions
NUMTA2013 14 / 46
35. Penalty and barrier functions
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
A penalty function P : IRn → IR satisfies the following condition
P(x)
= 0 if x belongs to the feasible region
0 otherwise
NUMTA2013 15 / 46
36. Penalty and barrier functions
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
A penalty function P : IRn → IR satisfies the following condition
P(x)
= 0 if x belongs to the feasible region
0 otherwise
Examples of penalty functions
P(x) =
Xk
j=1
|hj(x)|
P(x) =
Xk
j=1
h2j
(x)
NUMTA2013 15 / 46
37. Exactness of a Penalty Function
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
The optimal solution of the constrained problem
min
x
f(x)
subject to h(x) = 0
can be obtained by solving the following unconstrained minimization
problem
min f(x) +
1
σ
P(x)
for sufficiently small but fixed σ 0.
NUMTA2013 16 / 46
38. Exactness of a Penalty Function
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
The optimal solution of the constrained problem
min
x
f(x)
subject to h(x) = 0
can be obtained by solving the following unconstrained minimization
problem
min f(x) +
1
σ
P(x)
for sufficiently small but fixed σ 0.
P(x) =
Xk
j=1
|hj(x)|
NUMTA2013 16 / 46
39. Exactness of a Penalty Function
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
The optimal solution of the constrained problem
min
x
f(x)
subject to h(x) = 0
can be obtained by solving the following unconstrained minimization
problem
min f(x) +
1
σ
P(x)
for sufficiently small but fixed σ 0.
P(x) =
Xk
j=1
|hj(x)|
Non–smooth function!
NUMTA2013 16 / 46
40. Sequential Penalty method
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
min
x −x1 − x2
subject to x11
+ x22
− 1 = 0
The unique solution is x∗ =
1/√2
1/√2
and π∗ = 1/√2.
NUMTA2013 17 / 46
41. Sequential Penalty method
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
min
x −x1 − x2
subject to x11
+ x22
− 1 = 0
The unique solution is x∗ =
1/√2
1/√2
and π∗ = 1/√2.
In fact
∇f(x) =
−1
−1
, ∇h1(x)
2x1
2x2
∇f(x∗) + π∗∇h1(x∗) =
−1
−1
+
1
√2
2/√2
2/√2
= 0
NUMTA2013 17 / 46
42. Sequential Penalty method
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
min
x −x1 − x2
subject to x11
+ x22
− 1 = 0
The unique solution is x∗ =
1/√2
1/√2
and π∗ = 1/√2.
For no finite values of σ the solution of
min−x1 − x2 +
1
2σ
x11
+ x22
− 1
2
is also the solution of the constrained problem.
NUMTA2013 17 / 46
43. Sequential Penalty method
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
Let {σl} ↓ 0 and P(x) =
Xk
j=1
h2j
(x)
Step 0 Set l = 0
Step 1 Let x(σl) be an optimal solution of the unconstrained
differentiable problem
min f(x) +
1
σl
P(x)
Step 2 Set l = l + 1 and return to Step 1
NUMTA2013 18 / 46
44. Convergence Results
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
Assume that
• f(x) be bounded below in the (nonempty) feasible region,
• {σl} be a monotonic non-increasing sequence such that
{σlk} ↓ 0,
• for each l there exists a global minimum x(σl) of
f(x) + 1
l
P(x) =: φ(x, σl).
NUMTA2013 19 / 46
45. Convergence Results
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
Assume that
• f(x) be bounded below in the (nonempty) feasible region,
• {σl} be a monotonic non-increasing sequence such that
{σlk} ↓ 0,
• for each l there exists a global minimum x(σl) of
f(x) + 1
l
P(x) =: φ(x, σl).
Then
n
φ(x(σl); σl)
o
is monotonically non–decreasing
n
P(x(σl))
o
is monotonically non–increasing
f (x (σl)) is monotonically non–decreasing
NUMTA2013 19 / 46
46. Convergence Results
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
Assume that
• f(x) be bounded below in the (nonempty) feasible region,
• {σl} be a monotonic non-increasing sequence such that
{σlk} ↓ 0,
• for each l there exists a global minimum x(σl) of
f(x) + 1
l
P(x) =: φ(x, σl).
Moreover, liml h (x (σl)) = 0 and each limit point x∗ of the
sequence
n
(x (σl))
o
l
solves the nonlinear constrained problem.
NUMTA2013 19 / 46
47. Convergence Results
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
Assume that
• f(x) be bounded below in the (nonempty) feasible region,
• {σl} be a monotonic non-increasing sequence such that
{σlk} ↓ 0,
• for each l there exists a global minimum x(σl) of
f(x) + 1
l
P(x) =: φ(x, σl).
Moreover, liml h (x (σl)) = 0 and each limit point x∗ of the
sequence
n
(x (σl))
o
l
solves the nonlinear constrained problem.
If x∗ is regular limit point of
n
(x (σl))
o
l
then (x∗, π∗) satisfy
KKT–conditions where 1
l
h (x (σl)) → π∗.
NUMTA2013 19 / 46
48. Introducing ①
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
Let
P(x) =
Xk
j=1
h2j
(x)
Solve
min f(x) + ①P(x)φ (x,①)
NUMTA2013 20 / 46
49. Assumptions
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
In the sequel we assume that
x = x0 + ①−1x1 + ①−2x2 + . . .
with xi ∈ IRn
NUMTA2013 21 / 46
50. Assumptions
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
In the sequel we assume that
f(x) = f(x0) + ①−1f(1)(x) + ①−2f(2)(x) + . . .
h(x) = h(x0) + ①−1h(1)(x) + ①−2h(2)(x) + . . .
where f(i) : IRn → IR, h(i) : IRn → IRk are all finite–value functions.
NUMTA2013 21 / 46
51. Assumptions
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
In the sequel we assume that
∇f(x) = ∇f(x0) + ①−1F(1)(x) + ①−2F(2)(x) + . . .
∇h(x) = ∇h(x0) + ①−1H(1)(x) + ①−2H(2)(x) + . . .
where F(i) : IRn → IRn, H(i) : IRn → IRk×n are all finite–value
functions.
NUMTA2013 21 / 46
52. Assumptions
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
Previous conditions are satisfied (for example) by functions that are
product of polynomial functions in a single variable, i.e.,
p(x) = p1(x1)p2(x2) · · · pn(xn)
where pi(xi) is a polynomial function.
NUMTA2013 21 / 46
53. Convergence Results for Equality Constrained Problems
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
min
x
f(x)
subject to h(x) = 0
(1)
NUMTA2013 22 / 46
54. Convergence Results for Equality Constrained Problems
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
min
x
f(x)
subject to h(x) = 0
(1)
min
x
f(x) +
1
2
①kh(x)k2 (2)
NUMTA2013 22 / 46
55. Convergence Results for Equality Constrained Problems
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
min
x
f(x)
subject to h(x) = 0
(1)
min
x
f(x) +
1
2
①kh(x)k2 (2)
Let
x∗ = x∗0 + ①−1x∗1 + ①−2x∗2 + . . .
be a stationary point for (2) and assume that the LICQ condition
holds true at x∗0.
NUMTA2013 22 / 46
56. Convergence Results for Equality Constrained Problems
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
min
x
f(x)
subject to h(x) = 0
(1)
min
x
f(x) +
1
2
①kh(x)k2 (2)
Let
x∗ = x∗0 + ①−1x∗1 + ①−2x∗2 + . . .
be a stationary point for (2) and assume that the LICQ condition
holds true at x∗0.
Then, the pair
x∗0, π∗ = h(1)(x∗)
is a KKT point of (1).
NUMTA2013 22 / 46
57. Proof
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
Since x∗ = x∗0 +①−1x∗1 +①−2x∗2 +. . . is a stationary point we
have
∇f(x∗) + ①
Xk
j=1
∇hj(x∗)hj(x∗) = 0
NUMTA2013 23 / 46
58. Proof
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
Since x∗ = x∗0 +①−1x∗1 +①−2x∗2 +. . . is a stationary point we
have
∇f(x∗) + ①
Xk
j=1
∇hj(x∗)hj(x∗) = 0
∇f(x0) + ①−1F(1)(x) + ①−2F(2)(x) + . . .+
+①
Xk
j=1
∇hj(x0) + ①−1H(1)
j (x) + ①−2H(2)
j (x) + . . .
hj(x0) + ①−1h(1)
j (x) + ①−2h(2)
j (x) + . . .
= 0
NUMTA2013 23 / 46
59. Proof
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
Xk
①
j=1
!
∇hj(x∗0)hj(x∗0)+
∇f(x∗0) +
+
Xk
j=1
∇hj(x∗0)h(1)
j (x∗) +
Xk
j=1
H(1)
j (x∗)hj(x∗0)
!
+①1
. . .
!
+ ①2
. . .
!
+ ....
NUMTA2013 23 / 46
60. Proof
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
Xk
①
j=1
∇hj(x∗0)hj(x∗0)
!
Assuming LICQ we obtain
h(x∗0) = 0
NUMTA2013 23 / 46
61. Proof
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
∇f(x∗0) +
Xk
j=1
∇hj(x∗0)h(1)
j (x∗) +
Xk
j=1
H(1)
j (x∗)hj(x∗0) = 0
⇓
∇f(x∗0) +
Xk
j=1
∇hj(x∗0)h(1)
j (x∗) = 0
NUMTA2013 23 / 46
62. Example 1
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
min
x
1
2x21
+ 1
6x22
subject to x1 + x2 = 1
The pair (x∗, π∗) with x∗ =
1
4
3
4
, π∗ = −1
4 is a KKT point.
NUMTA2013 24 / 46
63. Example 1
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
min
x
1
2x21
+ 1
6x22
subject to x1 + x2 = 1
The pair (x∗, π∗) with x∗ =
1
4
3
4
, π∗ = −1
4 is a KKT point.
f(x) + ①P(x) =
1
2
x21
+
1
6
x22
−
1
2
①(1 − x1 − x2)2
NUMTA2013 24 / 46
64. Example 1
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
f(x) + ①P(x) =
1
2
x21
+
1
6
x22
−
1
2
①(1 − x1 − x2)2
First Order Optimality Condition
x1 − ①(1 − x1 − x2) = 0
1
3x2 − ①(1 − x1 − x2) = 0
NUMTA2013 24 / 46
65. Example 1
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
f(x) + ①P(x) =
1
2
x21
+
1
6
x22
−
1
2
①(1 − x1 − x2)2
x∗1
=
1①
1 + 4①
, x∗2
=
3①
1 + 4①
NUMTA2013 24 / 46
66. Example 1
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
f(x) + ①P(x) =
1
2
x21
+
1
6
x22
−
1
2
①(1 − x1 − x2)2
x∗1
=
1①
1 + 4①
, x∗2
=
3①
1 + 4①
x∗1
=
1
4 − ①−1(
1
16 −
1
64
①−1 . . .)
x∗2=
3
4 − ①−1(
3
16 −
3
64
①−1 . . .)
NUMTA2013 24 / 46
67. Example 1
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
f(x) + ①P(x) =
1
2
x21
+
1
6
x22
−
1
2
①(1 − x1 − x2)2
x∗1
=
1①
1 + 4①
, x∗2
=
3①
1 + 4①
−①(1 − x
2) = −①
1 − x
1 −
1
4
+ ①−1 1
16 −
1
64
①−2 . . .
−
3
4
+ ①−1 3
16 −
3
64
①−2 . . .
= −①
①−1 1
16
+ ①−1 3
16 − ①−2 4
64
. . .
= −
1
4
+
4
64
①−1 . . .
and h(1)(x∗) = −1
4 = π∗
NUMTA2013 24 / 46
68. Example 2
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
min x1 + x2
subject to x21
+ x22
− 2 = 0
L(x, π) = x1 + x2 + π
x21
+ x22
− 2
The optimal solution is x∗ =
−1
−1
and
the pair
x∗, π∗ = 1
2
satisfies the KKT conditions.
NUMTA2013 25 / 46
69. Example 2
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
φ (x,①) = x1 + x2 +
①
2
x21
+ x22
− 2
2
First–Order Optimality Conditions
x1 + 2①x1
x21
+ x22
− 2
2 = 0
x2 + 2①x2
x21
+ x22
− 2
2 = 0
The solution is given by
8 + ①−2C
x1 = −1 − ①−1 1
x2 = −1 − ①−1 1
8 + ①−2C
NUMTA2013 25 / 46
70. Example 2
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
−1−
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
In fact
2①x1 = −2① −
1
4
+ 2①−1C
x21
+ x22
− 2 = 1 +
1
64
①−2 + ①−4C2 1
4
①−1 − 2①−2 −
1
4
①−3C +
1 +
1
64
①−2 + ①−4C2 1
4
①−1 − 2①−2 −
1
4
①−3C
=
1
2
①−1 +
1
32 − 4C
①−2 +
−
1
2
C
①−3 +
−2C2
①−4
−1 −
h
2①x1
ih
x21
+ x22
− 2
i
=
h
−2①−
1
4
+2①−1C
ih 1
2
①−1+
1
32 − 4C
①−2+
−
1
2
C
①−3+
−2C2
①−4
i
=
−1 + 2
1
2
+ (. . .)①−1 + (. . .)①−2 + . . .
NUMTA2013 25 / 46
71. Example 2
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
−1−
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
and
2①x2 = −2① −
1
4
+ 2①−1C
x21
+ x22
− 2 = 1 +
1
64
①−2 + ①−4C2 1
4
①−1 − 2①−2 −
1
4
①−3C +
1 +
1
64
①−2 + ①−4C2 1
4
①−1 − 2①−2 −
1
4
①−3C
=
1
2
①−1 +
1
32 − 4C
①−2 +
−
1
2
C
①−3 +
−2C2
①−4
−1 −
h
2①x2
ih
x21
+ x22
− 2
i
=
h
−2①−
1
4
+2①−1C
ih 1
2
①−1+
1
32 − 4C
①−2+
−
1
2
C
①−3+
−2C2
①−4
i
=
−1 + 2
1
2
+ (. . .)①−1 + (. . .)①−2 + . . .
NUMTA2013 25 / 46
72. Example 2
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
• The Equality
Constraint Nonlinear
Problem
• First Order Optimality
Conditions
• Penalty and barrier
functions
• Exactness of a
Penalty Function
• Sequential Penalty
method
• Sequential Penalty
method
• Convergence Results
• Introducing ①
• Assumptions
• Convergence Results
for Equality Constrained
Problems
• Proof
• Example 1
• Example 2
Inequality Constraint
Data Envelopment
Analysis
Finally,
h
x21
①
+x22
−2
i
= ①
ih1
2
①−1+
1
32 − 4C
①−2+
−
1
2
C
①−3+
NUMTA2013 25 / 46
−2
1
2
+ (. . .)①−1 + (. . .)①−2 + . . .
73. Inequality Constraint
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
NUMTA2013 26 / 46
74. Inequality Constraints
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
min
x
f(x)
subject to g(x) ≤ 0
h(x) = 0
where f : IRn → IR, g : IRn → IRm h : IRn → IRk.
L(x, π, μ) := f(x) +
Xm
i=1
μigi(x) +
Xk
j=1
πjhj(x)
= f(x) + μT g(x) + πT h(x)
NUMTA2013 27 / 46
75. First Order Optimality Conditions
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
Let x∗ ∈ IRn with
n
∇gi(x∗), i : gi(x∗) = 0,∇hj(x∗), j = 1, . . . , k
o
linearly independent
NUMTA2013 28 / 46
76. First Order Optimality Conditions
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
Let x∗ ∈ IRn with
n
∇gi(x∗), i : gi(x∗) = 0,∇hj(x∗), j = 1, . . . , k
o
linearly independent
If x∗ is a local minimizer then
NUMTA2013 28 / 46
77. First Order Optimality Conditions
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
Let x∗ ∈ IRn with
n
∇gi(x∗), i : gi(x∗) = 0,∇hj(x∗), j = 1, . . . , k
o
linearly independent
If x∗ is a local minimizer then
there exists μ∗ ∈ IRm+
, π∗ ∈ IRk such that
∇xL(x∗, μ∗, π∗) = ∇f(x∗) +
Xk
j=1
∇hj(x∗)π∗j = 0
∇μL(x∗, μ∗, π∗) = g(x∗) ≤ 0
∇L(x∗, μ∗, π∗) = h(x∗) = 0
μ∗ ≥ 0
μ∗T∇L(x∗, μ∗, π∗) = 0
NUMTA2013 28 / 46
78. Additional Assumptions
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
In addition to the conditions stated before we similarly require that
the following conditions hold:
g(x) = g(x0) + ①−1g(1)(x) + ①−2g(2)(x) + . . .
∇g(x) = ∇g(x0) + ①−1G(1)(x) + ①−2G(2)(x) + . . .
where g(i) : IRn → IRm and G(i) : IRn →→ IRm×n are all
finite–value functions
NUMTA2013 29 / 46
79. Modified LICQ condition
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
Let x0 ∈ IRn. The Modified LICQ (MLICQ) condition is said to hold
true at x0 if the vectors
n
∇gi(x0), i : gi(x0) ≥ 0,∇hj(x0), j = 1, . . . , k
o
are linearly independent.
NUMTA2013 30 / 46
80. Convergence Results for Inequality Constrained Problems
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
min
x
f(x)
subject to g(x) ≤ 0
h(x) = 0
min
x
f(x) +
①
2 kmax{0, gi(x)}k2 +
①
2 kh(x)k2
x∗ = x∗0 + ①−1x∗1 + ①−2x∗2 + . . .
⇓ (MLICQ)
NUMTA2013 31 / 46
81. Convergence Results for Inequality Constrained Problems
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
min
x
f(x)
subject to g(x) ≤ 0
h(x) = 0
min
x
f(x) +
①
2 kmax{0, gi(x)}k2 +
①
2 kh(x)k2
x∗ = x∗0 + ①−1x∗1 + ①−2x∗2 + . . .
⇓ (MLICQ)
KKT–point
x0, μ = g(1)(x), π = h(1)(x)
NUMTA2013 31 / 46
82. Proof
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
∇f(x) + ①
mX
i=1
∇gi(x)max {0, gi(x)} + ①
Xp
j=1
∇hj (x)hj (x) = 0.
−−∇f(x0) + ①1F(1)(x) + ①2F(2)(x) + . . .+
mX
+①
i=1
∇gi(x0) + ①−1G
(1)
i (x) + ①−2G
(2)
i (x) + . . .
max
n
0, gi(x0) + ①−1g
(1)
i (x) + ①−2g
o#
(2)
i (x) + . . .
+
+①
Xp
j=1
∇hj (x0) + ①−1H
(1)
j (x) + ①−2H
(2)
j (x) + . . .
hj (x0) + ①−1h
(1)
j (x) + ①−2h
#
(2)
j (x) + . . .
= 0
NUMTA2013 32 / 46
83. Proof, cont.
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
(1)
i (x) + ①−2g
gi(x0) 0 ⇒ max {0, gi(x)} = gi(x0) + ①−1g
(2)
i (x) + . . .
gi(x0) 0 ⇒ max {0, gi(x)} = 0
gi(x0) = 0 ⇒ max {0, gi(x)} = ①−1 max
n
0, g
(1)
i (x) + ①−1g
o
(2)
i (x) + . . .
NUMTA2013 33 / 46
84. Proof, cont.
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
−−∇f(x0) + ①1F(1)(x) + ①2F(2)(x) + . . .+
mX
+①
i=1
gi(x0)0
∇gi(x0) + ①−1G
(1)
i (x) + ①−2G
(2)
i (x) + . . .
gi(x0) + ①−1g
(1)
i (x) + ①−2g
#
(2)
i (x) + . . .
+
+①
mX
i=1
gi(x0)=0
∇gi(x0) + ①−1G
(1)
i (x) + ①−2G
(2)
i (x) + . . .
①−1 max
n
0, g
(1)
i (x) + ①−1g
o#
(2)
i (x) + . . .
+
+①
Xp
j=1
∇hj (x0) + ①−1H
(1)
j (x) + ①−2H
(2)
j (x) + . . .
hj (x0) + ①−1h
(1)
j (x) + ①−2h
(2)
j (x) + . . .
#
= 0
NUMTA2013 33 / 46
85. Proof, cont.
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
Xm
i=1
gi(x0)≥0
∇gi(x∗0)gi(x∗0) +
Xp
j=1
∇hj(x∗0)hj(x∗0) = 0
NUMTA2013 33 / 46
86. Proof, cont.
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
Xm
i=1
gi(x0)≥0
∇gi(x∗0)gi(x∗0) +
Xp
j=1
∇hj(x∗0)hj(x∗0) = 0
and hence from MLICQ
g(x∗0) ≤ 0 and h(x∗0) = 0
NUMTA2013 33 / 46
87. Example 3
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
min
x∈IR
x
subject to x ≥ 1
The solution is ¯x = 1 with associated multiplier μ∗ = 1.
NUMTA2013 34 / 46
88. Example 3
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
min
x∈IR
x
subject to x ≥ 1
The solution is ¯x = 1 with associated multiplier μ∗ = 1.
min
x∈IR
x +
①
2
max {0, 1 − x}2
NUMTA2013 34 / 46
89. Example 3
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
min
x∈IR
x
subject to x ≥ 1
The solution is ¯x = 1 with associated multiplier μ∗ = 1.
min
x∈IR
x +
①
2
max {0, 1 − x}2
1 − ①max {0, 1 − x} = 0
For x 1 the only solution is
x∗ =
① − 1
①
= 1 − ①−1
NUMTA2013 34 / 46
90. Example 3
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
min
x∈IR
x
subject to x ≥ 1
The solution is ¯x = 1 with associated multiplier μ∗ = 1.
min
x∈IR
x +
①
2
max {0, 1 − x}2
1 − ①max {0, 1 − x} = 0
For x 1 the only solution is
x∗ =
① − 1
①
= 1 − ①−1
Therefore x∗0 = 1.
Moreover, g(x∗) = 1 −
1 − ①−1
= ①−1 and μ∗ = 1
NUMTA2013 34 / 46
91. Example 4
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
min x1 + x2
subject to x21
+ x22
− 2 ≤ 0
−x2 ≤ 0
NUMTA2013 35 / 46
92. Example 4
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
min x1 + x2
subject to x21
+ x22
− 2 ≤ 0
−x2 ≤ 0
L(x, π) = x1 + x2 + μ1
x21
+ x22
− 2
− μ2x2
The solution is x∗ =
−√2
0
and (x+, μ∗) with
μ∗ =
1/2√2
0
satisfies KKT conditions.
∇f(x) =
1
1
, ∇g1(x) =
2x1
2x2
, ∇g2(x) =
0
−1
1
1
+
1
2√2
−2√2
0
+ 1
0
−1
= 0
NUMTA2013 35 / 46
93. Example 4
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
min x1 + x2
subject to x21
+ x22
− 2 ≤ 0
−x2 ≤ 0
φ(x,①) = x1+x2+
①
2
max
0, x21
+ x22
− 2
2
+
①
2
max {0,−x2}2
NUMTA2013 35 / 46
94. Example 4
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
φ(x,①) = x1+x2+
①
2
max
0, x21
+ x22
− 2
2
+
①
2
max {0,−x2}2
NUMTA2013 35 / 46
95. Example 4
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
φ(x,①) = x1+x2+
①
2
max
0, x21
+ x22
− 2
2
+
①
2
max {0,−x2}2
1 + 2x1①max
0, x21
+ x22
− 2
= 0
1 + 2x2①max
0, x21
+ x22
− 2
− ①max {0,−x2} = 0
NUMTA2013 35 / 46
96. Example 4
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
φ(x,①) = x1+x2+
①
2
max
0, x21
+ x22
− 2
2
+
①
2
max {0,−x2}2
1 + 2x1①max
0, x21
+ x22
− 2
= 0
1 + 2x2①max
0, x21
+ x22
− 2
− ①max {0,−x2} = 0
x∗1
= −√2 + A①−1 + B①−2 + . . .
x∗2
= 0 + C①−1 + D①−2 + . . .
NUMTA2013 35 / 46
97. Example 4
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
2x∗1
① = −2√2① + 2A + 2B①−1 + · · ·
(x∗1
)2 + (x∗2
)2 − 2 =
h
−
√2+A①−1+B①−2+· · ·
i2
+
h
C①−1+D①−2+· · ·
i2
−2 =
2+A2①−2+B2①−4−2√2A①−1−2√2B①−2+2AB①−3+· · ·+
C2①−2 + D2①−4 + 2CD①−3 + h
· · · − 2 =
−2√2A①−1 +
A2 2√− 2B + C2
i
①−2 + · · ·
NUMTA2013 35 / 46
98. Example 4
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
1 + 2x∗1
①
(x∗1
)2 + (x∗2
)2 − 2
=
1 +
h
−2√2① + 2A + 2B①−1 + · · ·
i
−2√2A①−1 +
h
A2 − 2√2B + C2
i
①−2 + · · ·
#
=
1 +
−2√2
−2√2
A +
h
· · ·
i
①−1 +
h
· · ·
i
①−2 + · · ·
A = −
1
8
NUMTA2013 35 / 46
99. Example 4
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
2x∗2
① = 2C + 2D①−1 + · · ·
1 + 2x∗2
①
(x∗1
)2 + (x∗2
)2 − 2
− x∗2
① =
1 +
h
2C + 2D①−1 + · · ·
i
−2√2A①−1+
h
A2 − 2√2B + C2
i
①−2+· · ·
i
−
h
−C①−1−D①−2+· · ·
#
① =
1 + C +
h
· · ·
i
①−1 +
h
· · ·
i
①−2 + · · ·
C = −1
NUMTA2013 35 / 46
100. Example 4
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
x∗1
= −√2 1
− 8
①−1 + B①−2 + · · ·
x∗2
= 0 − ①−1 + D①−2 + · · ·
NUMTA2013 35 / 46
101. Example 4
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
x∗1
= −√2 1
− 8
①−1 + B①−2 + · · ·
x∗2
= 0 − ①−1 + D①−2 + · · ·
①h1(x∗) = ①((x∗1
)2 + (x∗2
)2 − 2) =
+2√2
①
1
8
①−1 +
1
64 − 2√2B + C2
①−2 + · · ·
#
μ∗1= 2√2
1
8
=
1
2√2
NUMTA2013 35 / 46
102. Example 4
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
x∗1
= −√2 1
− 8
①−1 + B①−2 + · · ·
x∗2
= 0 − ①−1 + D①−2 + · · ·
①h2(x∗) = ①(−x∗2
)
①
−①−1 − D①−2 + · · ·
#
μ∗2
= 1
NUMTA2013 35 / 46
103. Example 5
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
min 1
2
3
x1 − 2
2
+ 1
2
1
x2 − 2
4
subject to x1 + x2 − 1 ≤ 0
x1 − x2 − 1 ≤ 0
−x1 + x2 − 1 ≤ 0
−x1 − x2 − 1 ≤ 0
The solution is x∗ =
1
0
and (x+, μ∗) with μ∗ =
3/8
1/8
0
0
satisfies KKT conditions.
NUMTA2013 36 / 46
104. Example 5
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
(x,①) =
1
2
x1 −
3
2
2
+
1
2
x2 −
1
2
4
+
①
2
(
max{0, x1 + x2 − 1}2+
max{0, x1 − x2 − 1}2 + max{0,−x1 + x2 − 1}2 + max{0,−x1 − x2 − 1}2
)
NUMTA2013 36 / 46
110. Example 6
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
min x1 + x2
subject to
x21
+ x22
− 2
2 = 0
L(x, π) = x1 + x2 + π
x21
+ x22
− 2
2
The optimal solution is x∗ =
−1
−1
and
the pair
x∗, π∗ = 1
2
satisfies the KKT conditions.
NUMTA2013 37 / 46
111. Example 6
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
φ (x,①) = x1 + x2 +
①
2
x21
+ x22
− 2
4
First–Order Optimality Conditions
x1 + 4①x1
x21
+ x22
− 2
3 = 0
x2 + 4①x2
x21
+ x22
− 2
3 = 0
NUMTA2013 37 / 46
112. Example 6
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
x1 = −1 1
− 8
①−1 + ①−2C
x2 = −1 1
− 8
①−1 + ①−2C
1 + 4①x∗1
h
x21
+ x22
− 2
i3
=
1 +
h
−4① −
1
2
① + 2①−1C
i
1
2
①−1 +
h
· · ·
i
①−2 + · · ·
#3
=
1 +
h
−4① −
1
2
① + 2①−1C
i
①−3
h
· · ·
i
6= 0
NUMTA2013 37 / 46
113. Example 6
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
• Inequality Constraints
• First Order Optimality
Conditions
• Additional
Assumptions
• Modified LICQ
condition
• Convergence Results
for Inequality
Constrained Problems
• Proof
• Proof, cont.
• Example 3
• Example 4
• Example 5
• Example 6
Data Envelopment
Analysis
x1 = A + B①−1 + C①−2
x2 = D + E①−1 + F①−2
1 + 4①x∗1
h
x21
+ x22
− 2
i3
=
1 +
h
4A① + 4B + 4C①−1
i
R +
h
· · ·
i
①−1 + · · · + · · ·
#3
=
where R = A2 + B2 − 2. If R = 0 there is still a term multiplying
①. If R = 0, a term ①−3 can be factored out. The only possibility to
eliminate the term multiplying ① is A = 0. Spurious solution!
NUMTA2013 37 / 46
114. Data Envelopment Analysis
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
• Problem Data
• First DEA model
• CCR primal model
• CCR dual model
• Assurance interval for
ǫ
• CCR dual model
• Conclusions (?)
•
NUMTA2013 38 / 46
115. Problem Data
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
• Problem Data
• First DEA model
• CCR primal model
• CCR dual model
• Assurance interval for
ǫ
• CCR dual model
• Conclusions (?)
•
n Decision Making Unit (DMUs)
j = 1, . . . , n
(
Inputj = {xj
i , i = 1, . . . ,m}
Outputj = {yj
r, r = 1, . . . , s}
Effk(π, σ) =
Xs
r=1
σryk
r
Xm
i=1
πixk
i
NUMTA2013 39 / 46
116. First DEA model
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
• Problem Data
• First DEA model
• CCR primal model
• CCR dual model
• Assurance interval for
ǫ
• CCR dual model
• Conclusions (?)
•
DMUk
max
,
σT yk
πT xk
subject to
σT yj
πT xj ≤ 1 j = 1, . . . , n
π ≥ 0, σ ≥ 0
NUMTA2013 40 / 46
117. First DEA model
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
• Problem Data
• First DEA model
• CCR primal model
• CCR dual model
• Assurance interval for
ǫ
• CCR dual model
• Conclusions (?)
•
DMUk
max
,
σT yk
πT xk
subject to
σT yj
πT xj ≤ 1 j = 1, . . . , n
π ≥ 0, σ ≥ 0
Input Oriented: reduce input as much as possible while keeping at
least the present level of outputs
NUMTA2013 40 / 46
118. First DEA model
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
• Problem Data
• First DEA model
• CCR primal model
• CCR dual model
• Assurance interval for
ǫ
• CCR dual model
• Conclusions (?)
•
DMUk
max
,
σT yk
πT xk
subject to
σT yj
πT xj ≤ 1 j = 1, . . . , n
π ≥ 0, σ ≥ 0
Output Oriented: increase output level as much as possible under at
most the present level of input consumption
NUMTA2013 40 / 46
119. CCR primal model
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
• Problem Data
• First DEA model
• CCR primal model
• CCR dual model
• Assurance interval for
ǫ
• CCR dual model
• Conclusions (?)
•
Constant Return to Scale
Input Oriented
max
u,v
vT yk
subject to −uT xj + vT yj ≤ 0
j = 1, . . . , n
uT xk = 1
π ≥ ǫ, σ ≥ ǫ
where ǫ is an non-Archimedean infinitesimal.
Charnes, Cooper, Rhodes (1978)
NUMTA2013 41 / 46
120. CCR dual model
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
• Problem Data
• First DEA model
• CCR primal model
• CCR dual model
• Assurance interval for
ǫ
• CCR dual model
• Conclusions (?)
•
Constant Return to Scale
Input Oriented
min
,,s,s−
θ − ǫ
eT s∗ + eT s−
subject to
Xn
j=1
xjλj + s∗ = θxk
Xn
j=1
yjλj − s− = yk
λ ≥ 0, s∗ ≥ 0, s− ≥ 0
NUMTA2013 42 / 46
121. Assurance interval for ǫ
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
• Problem Data
• First DEA model
• CCR primal model
• CCR dual model
• Assurance interval for
ǫ
• CCR dual model
• Conclusions (?)
•
Acceptability intervals for ǫ can be obtained by solving n linear
programs (n is the number of DMUs).
B. Daneshian, G. R. Jahanshahloo et al, Mathematical and
Computational Applications, 2005
A polynomial-time algorithm for finding in DEA models has been
proposed by Gholam R. Amin and Mehdi Toloo (Computers
Operations Research,2004)
NUMTA2013 43 / 46
122. CCR dual model
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
• Problem Data
• First DEA model
• CCR primal model
• CCR dual model
• Assurance interval for
ǫ
• CCR dual model
• Conclusions (?)
•
Constant Return to Scale
Input Oriented
min
,,s,s−
θ − ①−1
eT s∗ + eT s−
subject to
Xn
j=1
xjλj + s∗ = θxk
Xn
j=1
yjλj − s− = yk
λ ≥ 0, s∗ ≥ 0, s− ≥ 0
NUMTA2013 44 / 46
123. Conclusions (?)
• Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
• Problem Data
• First DEA model
• CCR primal model
• CCR dual model
• Assurance interval for
ǫ
• CCR dual model
• Conclusions (?)
•
• The use of ① is extremely beneficial in various aspects in Linear
and Nonlinear Optimization
• Difficult problems in NLP can be approached in a simpler way
using ①
• A new convergence theory for standard algorithms (gradient,
Newton’s, Quasi-Newton) needs to be developed in theis new
framework
NUMTA2013 45 / 46
124. • Outline of the talk
Degeneracy and the
Simplex Method
Nonlinear Optimization
Equality Constraint
Inequality Constraint
Data Envelopment
Analysis
• Problem Data
• First DEA model
• CCR primal model
• CCR dual model
• Assurance interval for
ǫ
• CCR dual model
• Conclusions (?)
•
Thanks for your attention
NUMTA2013 46 / 46