Combinatorial optimization M.Tech subject

1. Combinatorial Optimization
CS-724
Lec-4: Date: 26-02-2021
Dr. Parikshit Saikia
Assistant Professor
Department of Computer Science and Engineering
NIT Hamirpur (HP)
India

2.  Local and global optima
 Convex Set
 Convex Functions

5. Convex programming problem (CPP)
 The optimization problem of the form:
Min f(x)
subject to: gi (x) ≤ 0, i = 1, 2, 3,……..,m
is called a CPP if f and gi (i = 1, 2, 3,…., m) are convex functions.
 Theorem: Let gi for each i = 1, 2, 3, …m be a convex function. Then
S = {x Rn | gi (x) ≤ 0, i = 1, 2, 3, …m}
is a convex set.
i.e., the feasible set S in a CPP is convex.

7.  How to determine if a functions is convex?
 Prove by definition
 Use properties
 Sum of convex functions is convex
 If 𝑓 𝑥 = σ𝑖 𝑤𝑖𝑓𝑖 𝑥 , 𝑤𝑖 ≥ 0, 𝑓𝑖 𝑥 convex, then 𝑓(𝑥) is convex
 Convexity is preserved under a linear transformation
 If 𝑓 𝑥 = 𝑔(𝐴𝑥 + 𝑏), 𝑔 convex, then 𝑓(𝑥) is convex
 If 𝑓 is a twice differentiable function of one variable, 𝑓 is convex on an interval
𝑎, 𝑏 ⊂ ℝ iff (if and only if) its second derivative 𝑓′′ 𝑥 ≥ 0 in 𝑎, 𝑏

8. If 𝑓 is a twice continuously differentiable function of 𝑛 variables, 𝑓 is convex on ℱ iff its
Hessian matrix of second partial derivatives is positive semidefinite on the interior of ℱ
2/26/2021
Fei Fang 8
𝐻 is positive semidefinite in 𝑆 if ∀𝑥 ∈ 𝑆, ∀𝑧 ∈
ℝ𝑛, 𝑧𝑇𝐻(𝑥)𝑧 ≥ 0
𝐻 is positive semidefinite in ℝ𝑛 iff all
eigenvalues of 𝐻 are non-negative
Alternatively, prove 𝑧𝑇𝐻 𝑥 𝑧 = σ𝑖 𝑔𝑖 𝑥, 𝑧
2

13. Different formats of CPP
Optimization Problem Condition for CPP
Min f(x) subject to
gi (x) ≤ 0, i = 1, 2, 3,...,m
f and gi for all i are convex
Max f(x) subject to
gi (x) ≤ 0, i = 1, 2, 3,...,m
f is concave and gi for all i are convex
Min f(x) subject to
gi (x) ≥ 0, i = 1, 2, 3,...,m
f is convex and gi for all i are concave
Max f(x) subject to
gi (x) ≥ 0, i = 1, 2, 3,...,m
f and gi for all i are concave

14.  Theorem: In a CPP every point locally optimal with respect to the
Euclidean distance neighborhood N is also globally optimal.
Proof: Home work

15.  A convex function c(x) defined on defined on [0, 1] ⊆ Rn can have many local optima,
but all must be global. This is illustrated in the following figure.