凸最適化（boyd本）輪読会の資料です。 Chapter3: Convex functions 3.1 Basic properties and examples

1. Presenter Koki Isokawa
Oct. 29, 2020
3 Convex functions
3.1 Basic properties and examples
Reading circle on Convex Optimization - Boyd & Vandenberghe

2. Convex function
❖ A function is if
• is a convex set
• for all , and with
❖ Notes
• Strictly convex:
• Concave: is convex
f convex
dom f
x, y ∈ dom f θ 0 ≤ θ ≤ 1
x ≠ y, 0 < θ < 1
−f
2
f(θx + (1 − θ)y) ≤ θ˜
f(x) + (1 − θ)˜
f(y)

3. Properties on convex function
❖ Affine function is both convex and concave
any convex and concave function is affine
❖ is convex iff
• it is convex when restricted to any line that intersects
its domain
•
f
3

4. Extended-value extensions
❖ Extend a convex function to all
• - : is defined by
• It can simplify notation by reducing description like
for all
Rn
Extended value extension ˜
f Rn
→ R ∪ {∞}
x ∈ dom f
4

5. Examples of extended-value extensions
❖ Definition of convex function
➡ for , for any and ,
❖ Pointwise sum: with
• , : convex functions on
•
➡ for any ,
0 < θ < 1 x y
f(x) = f1(x) + f2(x) x ∈ dom f
f1 f2 Rn
dom f = dom f1 ∩ dom f2
x ˜
f(x) = ˜
f1(x) + ˜
f2(x)
5
˜
f(θx + (1 − θ)y) ≤ θx + (1 − θ)y

6. Example 3.1: Indicator function
Assume 𝐶 is a 𝑐𝑜𝑛𝑣𝑒𝑥 set
• Convex function : for any
• Extended-value extension of :
• The convex function is called the
• Use case: minimizing on the set is equivalent to
minimizing over all of
Rn
IC x ∈ C IC(x) = 0
IC
ĨC indicator function
f C
f + ĨC Rn
6

7. First-order conditions
Suppose is differentiable
is convex iff
• is convex
• for all
f
f
dom f
f(y) ≥ f(x) + ∇f(x)T
(y − x) x, y ∈ dom f
7

8. First-order conditions
Suppose is differentiable
is convex iff
• is convex
• for all
f
f
dom f
f(y) ≥ f(x) + ∇f(x)T
(y − x) x, y ∈ dom f
8
the first order Taylor approximation of near
f x
local information
global underestimator

9. First-order conditions
Suppose is differentiable
is convex iff
• is convex
• for all
f
f
dom f
f(y) ≥ f(x) + ∇f(x)T
(y − x) x, y ∈ dom f
9

10. Second-order conditions
Suppose is twice differentiable
is convex iff
• is convex
• Hessian is positive semidefinite: for all
f
f
dom f
x ∈ dom f
10

11. Examples of convex functions on R
❖ : on (for any )
❖ : on (when or )
❖ : on (for )
❖ : is concave on
❖ : on
Exponential eax
R a ∈ R
Powers xa
R++ a ≥ 1 a ≤ 0
Powers of absolute value |x|p
R p ≥ 1
Logarithm log x R
Negative entropy x log x R++
11

12. Examples of convex functions on Rn
❖ : Every norm on
❖ : on
❖ - - :
with
❖ - - : on
❖ : is concave on
❖ - : is concave on
Norms Rn
Max function f(x) = max{x1, . . . , xn} Rn
Quadratic over linear function
f(x, y) = x2
/y dom f = R × R++ = {(x, y) ∈ R2
|y > 0}
Log sum exp f(x) = log(ex1 + ⋯ + exn) Rn
Geometric mean f(x) = (Πn
i=1xi)1/n
dom f = Rn
++
Log determinant f(X) = log det X dom f = Sn
++
12

13. Sublevel sets
- set of a function :
• is convex sublevel set of is convex for any
α sublevel f Rn
→ R
f ⇒ f
α
13
Cα = {x ∈ dom f |f(x) ≤ α}

14. Epigraph
❖ The graph of a function of :
❖ The epigraph of a function :
f Rn
→ R
f Rn
→ R
14
{(x, f(x))|x ∈ dom f}

15. Convexity via epigraph
❖ The epigraph links convex sets and convex
functions
• A function is convex iff its epigraph is a convex set
• A function is concave iff its is a convex set
hypograph
15

16. Utility of epigraph
❖ Many results for convex functions can be
proved (or interpreted) geometrically
e.g., First-order condition for convexity
If , then .
It is expressed as:
(y, t) ∈ epi f
16
f(y) ≥ f(x) + ∇f(x)T
(y − x)
t ≥ f(y) ≥ f(x) + ∇f(x)T
(y − x)
(y, t)

17. Jensen s inequality and extensions
❖ The basic inequality is called
❖ Extension to convex combinations:
❖ Extension to infinite sums and expectations:
Jensen′s inequality
17
, and with
x1, …, xk ∈ dom f θ1, . . . θk ≥ 0 θ1 + ⋯ + θk = 1
f(θ1x1 + ⋯ + θkxk) ≤ θ1f(x1) + ⋯θk f(xk)
on ,
p(x) ≥ 0 S ⊆ dom f
∫S
p(x) dx = 1
f
(∫S
p(x)x dx
)
≤
∫S
f(x)p(x) dx

18. Jensen s inequality and expectations
❖ If is random variable and is convex
❖ Remark 3.2: Randomization can t decrease the
value of convex function on average
Suppose and is any zero mean random
vector in
x ∈ dom f f
x ∈ dom f ⊆ Rn
z
Rn
18
f(Ex) ≤ Ef(x)
Ef(x + z) ≥ f(E(x + z)) = f(x)

19. Inequalities (1/2)
Jensen s inequality derives famous inequalities
• The arithmetic-geometric mean inequality:
• Here, take exponential of both side:
19
( )
ab ≤ (a + b)/2 a, b ≥ 0
−log(
a + b
2
) ≤
−log a − log b
2
It is equal to Jensen s inequality with and (convex)
θ = 1/2 f(x) = − log(x)

20. Inequalities (2/2)
Jensen s inequality derives famous inequalities
• Hölder s inequality:
• Jensen s inequality with general and
• Hölder s inequality is derived by apply the following
θ f(x) = − log x
20
( , and )
n
∑
i=1
xiyi ≤
(
n
∑
i=1
|xi |p
)
1/p
(
n
∑
i=1
|yi |q
)
1/q
p > 1, 1/p + 1/q = 1 x, y ∈ Rn
( )
aθ
b1−θ
≤ θa + (1 − θb) a, b ≥ 0, 0 ≤ θ ≤ 1