凸最適化のBoyd本の輪読会の発表資料です。 [References] Stephen Boyd and Lieven Vandenberghe. Convex Optimization. Cambridge University Press, 2004.

1. 2.3 Operations that preserve convexity
& 2.4 Generalized inequalities
Reading circle on Convex Optimization - Boyd & Vandenberghe
Presenter Ryotaro Tsukada
Nov. 13, 2019
1

2. 2.3 Operations that preserve convexity

3. How to establish convexity of a set C ?
1. Apply definition
x1, x2 ∈ C, 0 ≤ θ ≤ 1 ⇒ θx1 + (1 − θ)x2 ∈ C
2. Show that C can be built from simple convex
sets by operations that preserve convexity
• Intersection
• Affine function
• Perspective function
• Linear-fractional function
2

4. Intersection
The intersection of convex sets is convex
Example 2.8 (for m = 2)
S = {x ∈ R2
| |p(t)| ≤ 1 for |t| ≤ 3/π}
where p(t) = x1 cos t + x2 cos 2t
is convex because:
S =
∩
|t|≤3/π
{x| − 1 ≤ x1 cos t + x2 cos 2t ≤ 1}
halfspace
3

5. Affine function (1/3)
Suppose
• S ⊆ Rn
, C ⊆ Rm
are convex
• f : Rn
→ Rm
is affine (f(x) = Ax + b)
Then
• the image of S under f is convex
f(S) = {f(x) | x ∈ S}
• the inverse image of C under f is convex
f−1
(C) = {f−1
(x) | x ∈ C}
4

6. Affine function (2/3)
Example 2.10
The solution set of a linear matrix inequality
{x | A(x) = x1A1 + · · · + xnAn ⪯ B}
where B, Ai ∈ Sm
is convex because it is the inverse image of
{f(x) | f(x) = B − A(x) ⪰ 0} (positive semidefinite cone)
under the affine function f : Rn
→ Sm
5

7. Affine function (3/3)
Example 2.11
The set
{x | xT
Px ≤ (cT
x)2
, cT
x ≥ 0} (hyperbolic cone)
where P ∈ Sn
+ and c ∈ Rn
is convex because it is the inverse image of
{(z, t) | zT
z ≤ t2
, t ≥ 0} (second-order cone)
under the affine function f(x) = (P1/2
x, cT
x)
6

8. Perspective function
Define the perspective function P : Rn+1
→ Rn
as
P(z, t) = z/t, dom P = {(z, t) | t > 0}
Then the image of a convex set C ⊆ dom P is convex
P(C) = {P(x) | x ∈ C}
7

9. Pin-hole camera interpretation of P
P : R3
→ R2
is just like a pin-hole camera:
• an opaque horizontal plane x3 = 0,
a horizontal image plane x3 = −1
• an object at x (x3 > 0) forms an image at
−(x1/x3, x2/x3, 1) = −(P(x), 1)
8

10. P(line segments) = line segments
Suppose
x = (˜x, xn+1), y = (˜y, yn+1) ∈ Rn+1
, xn+1 > 0, yn+1 > 0
Then
P(
line segments [x,y]
θx + (1 − θ)y) =
θ˜x + (1 − θ)˜y
θxn+1 + (1 − θ)yn+1
= µP(x) + (1 − µ)P(y)
line segments [P(x),P(y)]
where
µ =
θxn+1
θxn+1 + (1 − θ)yn+1
∈ [0, 1]
9

11. Linear-fractional function (1/3)
Suppose g : Rn
→ Rm+1
is affine, i.e.,
g(x) =
[
A
cT
]
x +
[
b
d
]
where A ∈ Rm×n
, b ∈ Rm
, c ∈ Rn
, d ∈ R
10

12. Linear-fractional function (1/3)
Suppose g : Rn
→ Rm+1
is affine, i.e.,
g(x) =
[
A
cT
]
x +
[
b
d
]
where A ∈ Rm×n
, b ∈ Rm
, c ∈ Rn
, d ∈ R
Then the linear-fractional function f : Rn
→ Rm
is
given by f = P ◦ g, i.e,
f(x) =
Ax + b
cT + d
, dom f = {x | cT
x + d > 0}
10

13. Linear-fractional function (2/3)
Example 2.13
Suppose
• u and v are random variables that take on
values in {1, · · · , n} and {1, · · · , m}, respectively
• pij := prob(u = i, v = j)
• fij := prob(u = i | v = j)
• C := {p | p ∈ all possible discrete distributions}
11

14. Linear-fractional function (3/3)
Example 2.13
Then the associated set of conditional probabilities
{f(p) | p ∈ C} is convex because:
• C is a convex set (by definition)
• f is a linear-fractional mapping from p
(fij =
pij
∑n
k=1 pkj
)
12

15. 2.4 Generalized inequalities

16. Preliminary: proper cone
A cone K ⊆ Rn
is a proper cone if
• K is convex
• K is closed
• K is solid (int K ̸= ∅)
• K is pointed (x ∈ K, −x ∈ K ⇒ x = 0)
13

17. Generalized inequality
Associate with the proper cone K:
• the partial ordering on Rn
defined by
x ⪯K y ⇐⇒ y − x ∈ K
• the strict partial ordering on Rn
defined by
x ≺K y ⇐⇒ y − x ∈ int K
14

18. Why ‘generalized’?
When K = R+:
• the partial ordering ⪯K is the usual ≤ on R
x ⪯R+
y ⇐⇒ y − x ∈ R+ ⇐⇒ x ≤ y
• the strict partial ordering ≺K is the usual < on R
x ≺R+
y ⇐⇒ y − x ∈ int R+ ⇐⇒ x < y
15

19. Componentwise inequality
Example 2.14
When K = Rn
+ (nonnegative orthant):
x ⪯Rn
+
y ⇐⇒ xi ≤ yi, i = 1, · · · , n
x ≺Rn
+
y ⇐⇒ xi < yi, i = 1, · · · , n
This partial ordering is so common that we drop the
subscript: x ⪯ y, x ≺ y
16

20. Matrix inequality
Example 2.15
When K = Sn
+ (positive semidefinite cone):
X ⪯Sn
+
Y ⇐⇒ Y − X is positive semidefinite
X ≺Sn
+
Y ⇐⇒ Y − X is positive definite
This partial ordering is so common that we drop the
subscript: X ⪯ Y, X ≺ Y
17

21. Properties of generalized inequalities
Many properties of ⪯K are similar to ≤ on R:
• if x ⪯K y and u ⪯K v, then x + u ⪯K y + v
• if x ⪯K y and y ⪯K z, then x ⪯K z
• if x ⪯K y and α ≥ 0, then αx ⪯K αy
• x ⪯K x
• if x ⪯K y and y ⪯K x, then x = y
• · · ·
18

22. ⪯K is not in general a linear ordering
When K = R2
+:
x1
x2
x3

23. ⪯K is not in general a linear ordering
When K = R2
+:
x1
x2
x3
x1 ⪯R2
+
x2

24. ⪯K is not in general a linear ordering
When K = R2
+:
x1
x2
x3
x1 ⪯R2
+
x2
?
19

25. Minimum and minimal elements (1/2)
With respect to ⪯K:
• x ∈ S is the minimum element of S if
y ∈ S ⇒ x ⪯K y
(the element that is smaller than everything else)
• x ∈ S is a minimal element of S if
y ∈ S, y ⪯K x ⇒ y = x
(an element that has nothing smaller than it)
20

26. Minimum and minimal elements (2/2)
Example 2.17
When K = R2
+ (componentwise inequality in R2
):
x ⪯R2
+
y ⇐⇒ y is above and to the right of x
Then x1 is the minimum of S1, x2 is a minimal of S2
21