凸最適化（boyd本）輪読会の資料です。 ~Chapters~ 4.6 Generalized inequality constraints 4.7 Vector optimization

1. Nov 19, 2020
Kitsuregawa Lab. (M2)
Koki Isokawa
4.6 Generalized inequality constraints
4.7 Vector optimization
Reading circle on Convex Optimization - Boyd & Vandenberghe

2. Generalized inequality constraints
Standard form convex optimization problem (review)
Standard form convex optimization problem with
generalized inequality constraints
2
where are proper cones, are -convex
f0 : Rn
→ R, Ki ⊆ Rki fi : Rn
→ Rki Ki

3. Relationship with convex optimization
• Convex optimization problem is a special case with
• Some results in convex opt. problem can be diverted
• The feasible set, any sublevel set, and the optimal set are
convex
• Any point that is locally optimal for the problem is globally
optimal
• The optimality condition for differentiable (see 4.2.3) holds
without any change
Ki = R+, i = 1,…, m
f0
3

4. Basic example1: Conic form problem
One of the simplest convex optimization problems with
inequality constraints
A generalization of linear programs in which componentwise
inequality is replaced with a generalized linear inequality
4
• A linear objective
• One inequality constraint

5. Standard and inequality form conic form problem
Conic form problem in standard form
Conic form problem in inequality form
5
Both forms are derived using the analogy of linear programming

6. Basic example2: Semidefinite programming
(SDP): conic form problem when is
(the cone of positive semidefinite matrices)
If are all diagonal, the SDP reduces to a linear
program
Semidefinite program K
Sk
+ k × k
G, F1, …, Fn
6
where , and
G, F1, …, Fn ∈ Sk
A ∈ Rp×n

7. Standard and inequality form SDP
A standard form SDP
An inequality form SDP
7
where C, A1, …, Ap ∈ Sn
where B, A1, …, An ∈ Sk

8. Multiple LMIs and linear inequalities
Following problem is common to be referred as an SDP
These problems can be transformed to an SDP
8
linear objective several LMI constraints
linear equality and inequality

9. Examples: Second order cone programming 9
in which,
⇔ (Aix + bi, cT
i x + di) ∈ Ki
SOCP can be expressed as a conic form problem

10. Examples: matrix norm minimization
Suppose an unconstrained convex problem
Let , where
where denotes the spectral norm (maximum singular value)
Problem with matrix inequality constraint
A(x) = A0 + x1A1 + ⋯ + xnAn Ai ∈ Rp×q
∥ ⋅ ∥2
10
⇔
SDP semidefinite matrix
(see A.5.5)
特異値

11. Examples: moment problems 1/3
The (power) of the distribution of :
The expected value ( be a random variable in )
Moments satisfy following constraint:
(Proof) Let
moments t
xk = Etk
t R
y = (y0, y1, …, yn) ∈ Rn+1
11
Hankel matrix

12. Examples: moment problems 2/3
Let : a given polynomial in
Suppose is a random variable on
not knowing the distribution
knowing some bounds on the moment as follow
The expected value of :
p(t) = c0 + c1t + ⋯ + c2nt2n
t
t R
p(t)
12

13. Examples: moment problems 3/3
Upper and lower bound for :
Rewritten as following SDP by using moments
Ep(t)
13

14. 4.7 Vector optimization
A general :
:
• is -convex
• are convex
• are affine
Here, the two objective values need not be comparable:
we can have neither
vector optimization problem
Convex vector optimization problem
f0 K
f1, …, fm
h1, …, hp
f0(x), f0(y)
14
f0 : Rn
→ Rq
fi : Rn
→ R, hi : Rn
→ R
K ⊆ Rp

15. Optimal points and values
Here we consider the set of :
If it has a minimum element (a feasible such that
for all feasible ), is called and
- A point is optimal iff it is feasible and
- Most vector optimization problems do not
have an optimal point and an optimal value
achievable objective values
x
y x optimal f0(x) optimal value
x⋆
15
with K = R2
+

16. Pareto optimal points and values
A feasible point is if is a minimal
element and is called a
- A point is Pareto optimal iff it is feasible and
- The set of Pareto optimal values satisfies
x Pareto optimal f0(x)
f0(x) pareto optimal value
x
16
f0(x) − K
with K = R2
+

17. Scalarization
A standard technique for finding Pareto optimal points
Choose any , and consider the opt. problem and
let be an optimal point
(proof)
• If were not Pareto optimal, then there is a feasible point
which satisfies and
• Since and is nonzero, we have
scalar
x
x y
17
Contradict the assumption that is optimal for the scalar problem
x

18. Properties on scalarization
• The vector is called
• By varying , we obtain different Pareto optimal
solutions
• Scalarization cannot find every Pareto optimal point
weight vector
18
is Pareto optimal but cannot
be found by scalarization
x3

19. Scalarization of convex vector opt. problems
• When is , any solution is Pareto optimal
• For every Pareto optimal point , there is some
nonzero , such that is a solution of the
scalarization problem
e.g.,
λ
xPO
xPO
19

20. Multicriterion optimization
When a vector optimization problem involves the cone
, it is called - optimization problem
• are interpreted as different scalar objectives
• a multicriterion opt. problem is convex iff. : convex,
: affine, and : convex
K = Rq
+ multi objective
f0 = (F1, …Fq) q
f1, …, fm
h1, …, hp F1, …, Fq
20

21. An optimal point in a multicriterion problem
An optimal point satisfies for
every feasible
In other words, is optimal for each of the -th scalar
problems
If there is an optimal point, the objectives are said
x*
y
x* j
noncompeting
21

22. Trade-off analysis(1/4)
Suppose and are Pareto optimal points, say,
Here and must be both empty or nonempty
We want to compare to
x y
A C
x y
22
where A ∪ B ∪ C = {1,…, q}

23. Trade-off analysis(2/4) 23
the set of PO values
= - ( )
optimal trade off surface curve

24. Trade-off analysis(3/4) 24
by small amount of increase in F1
we obtain small large reduction in F2

25. Trade-off analysis(4/4) 25
by small amount of increase in F1
we obtain small large reduction in F2
by large amount of increase in F2
we obtain small reduction in F1
• A point of large curvature in one objective is called -
• In many applications represent a good compromise solution
knee of the trade off curve

26. Scalarizing multicriterion problems
Scalarizing multicriterion problem is shown in the form
of weighted sum objective
• we can interpret as the attached to the -th objective
• when we want to be small, we should take large
• : , or or relative importance of -th
objective compared to -th
λi weight i
Fi λi
λi/λj excahnge rate relative weight i
j
26

27. Examples: risk-return trade-off in portfolio optimization
Objectives: negative mean return and the variance of
the return
scalarize with :
λ1 = 1, λ2 = μ > 0
27
: the amount of asset
: price change of asset
xi i
pi i
(QP)

28. Examples: risk-return trade-off in portfolio optimization
e.g.,
28
opt. problem Optimal risk-return trade-off curve
x
scalarized opt. problem
known asset properties

29. Examples: risk-return trade-off in portfolio optimization
e.g.,
29
opt. problem Optimal risk-return trade-off curve
x
scalarized opt. problem
known asset properties