The document discusses extending the barrier method to optimization problems with generalized inequalities. It introduces logarithmic barrier functions for generalized inequalities and defines the central path. Points on the central path give dual feasible solutions. The barrier method can be applied to find an epsilon-optimal solution in O(log(1/epsilon)) iterations, each requiring O(sqrt(n)) Newton iterations under self-concordance assumptions. Complexity is analyzed using self-concordant functions and properties of generalized logarithms.

1. Presenter Koki Isokawa
Mar. 17, 2021
11.6 Problems with generalized inequalities
Reading circle on Convex Optimization - Boyd & Vandenberghe

2. Overview
Introduce how the barrier method can be extended to problems with
generalized inequalities
11.6.1 Logarithmic barrier and central path
11.6.2 Barrier method
11.6.3 Examples
11.6.4 Complexity analysis via self-concordance
2

3. Preparation: problem setting
As in section 11.1, we assume that
the function are twice continuously differentiable
with
the problem is solvable
fi
A ∈ Rp×n
rank A = p
3
where
• is convex
• are -convex
• are proper cones
f0 : Rn
→ R
fi : Rn
→ Rki, i = 1,...,m Ki
Ki ⊆ Rki
(11.38)

4. Preparation: KKT conditions for the problem
We assume that the problem (11.38) is strictly feasible, so that KKT
conditions are necessary and sufficient conditions for optimality of x⋆
4
where is the derivative of at
Dfi(x⋆
) ∈ Rki×n
fi x⋆

5. 11.6.1 Logarithmic barrier and central path

6. Generalized logarithm for a proper cone
We first define the analog of the logarithm, , for a proper cone
We say that is a for if
is concave, closed, twice continuously differentiable, , and
for
There is a constant such that for all , and all , .
Intuitively, behaves like a logarithm along any ray in the cone
log x K ⊆ Rq
ψ : Rq
→ R generalized logarithm K
ψ dom ψ = int K
∇2
ψ(y) ≺ 0 y ∈ int K
θ > 0 y ≻K 0 s > 0 ψ(sy) = ψ(y) + θ log s
ψ K
6
* We call the constant the degree of
θ ψ

7. Properties of generalized logarithm
A generalized logarithm is only defined up to an additive constant;
if is a generalized logarithm for , then so is , where
The ordinary logarithm is a generalized logarithm for
If , then
(11.40), which implies is -increasing (see 3.6.1). (exercise 11.15)
. (directly proved by differentiating with )
ψ K ψ + a a ∈ R
R+
y ≻K 0
∇ψ(y) ≻K* 0 ψ K
yT
∇ψ(y) = θ ψ(sy) = ψ(y) + θ log s s
7

8. Logarithmic barrier functions for generalized inequalities
Let be generalized logarithms for the cones , with
degrees .
We define the for problem (11.38) as
Convexity of follows since the functions are -increasing, and the
functions are -convex (see 3.6.2)
ψ1, …, ψm K1, …, Km
θ1, …, θm
logarithmic barrier function
ϕ ψi Ki
fi Ki
8

9. The central path
Next, we define the central path for problem (11.38)
We define the central point , for , as the minimizer of ,
subject to , i.e.,
Central points are characterized by the optimality condition
x⋆
(t) t ≥ 0 tf0 + ϕ
Ax = b
9
for some , where is the derivative of at
ν ∈ Rp
Dfi(x) fi x

10. Dual points on central path
As in the scalar case, points on the central path give dual feasible points
for the problem (11.38)
For , we define (11.42), and ,
where is the optimal dual variable in (11.41)
We will show that , together with , are dual feasible for
the original problem (11.38) in the next slide
i = 1,…, m λ⋆
i (t) =
1
t
∇ψi (−fi(x⋆
(t))) ν⋆
(t) = ν/t
ν
λ⋆
1 (t), …, λ⋆
m(t) ν⋆
(t)
10

11. Dual points on central path
The Lagrangian is minimized over by from (11.41)
The dual function evaluated at is therefore equal to
The duality gap with primal feasible point and the dual feasible
point is
x x = x⋆
(t)
g (λ⋆
(t), ν⋆
(t))
x⋆
(t)
(λ⋆
(t), ν⋆
(t)) (1/t)
m
∑
i=1
θi = θ̄/t
11
for and therefore
.
yT
∇ψi(y) = θi y ≻Ki
0
λ⋆
i (y)T
fi(x⋆
(t)) = − θi/t, i = 1,…, m
This is just like a scalar case,
except that the sum of the degrees,
appears in place of , the number of inequalities.
m

12. 11.6.2 Barrier method
We have seen that the key properties of the central path generalized to
problems with generalized inequalities
Computing a point on the central path needs minimizing a twice differentiable
convex function with equality constraints (it can be done with Newton s method)
With the central point we can associate a dual feasible point with
duality gap . In particular, is no more than -suboptimal
So, we can apply the barrier method in 11.3 to the problem (11.38)
The desired accuracy is achieved after , plus one initial
centering step
x⋆
(t) (λ⋆
(t), ν⋆
(t))
θ̄/t x⋆
(t) θ̄/t
ϵ
[
log(θ̄/(t(0)
ϵ))
log μ ]
12

13. Phase 1 and feasibility problems
The phase 1 methods described in 11.4 are readily extended to problems
with generalized inequalities
Let be some given, -positive vectors, for .
To determine feasibility of the equalities and generalized inequalities
we solve the problem
The optimal value determines the feasibility of the equalities and
generalized inequalities, exactly as in 11.4
ei ≻Ki
0 Ki i = 1,…, m
p̄⋆
13

14. 11.6.3 Examples
A small SOCP
A small SDP
A family of SDPs
14

15. A small SOCP with barrier method
Similar to the results for linear and geometric programming in 11.3
Linear convergence
A reasonable choice of is in the range 10-100 with around 30 Newton iterations
μ
15

16. A small SDP with barrier method 16

17. A family of SDPs
Examine the performance of the barrier method as a function of the
problem dimensions with variable and parameter
The number of Newton steps required grows very slowly with the
problem dimensions
x ∈ Rn
A ∈ Sn
17

18. 11.6.4 Complexity analysis via self-concordance
Extend the complexity analysis of the barrier method for problems with
ordinary inequalities ( 11.5), to problems with generalized inequalities
(Preview) The number of outer iterations is given by , (plus
one initial centering step)
We want to bound the number of Newton steps required in each
centering step
We use the complexity theory of Newton s method for self-concordant functions
[
log(θ̄/(t(0)
ϵ))
log μ ]
18

19. Self-concordance assumptions
We make the same assumptions as in 11.5
The function is closed and self-concordant for all
The sublevel sets of (11.38) are bounded
Exactly as in the scalar case, we have (proved in next 3 slides)
Therefore when self-concordance and bounded sublebel set conditions hold,
the number of Newton steps per centering step in no more than
Almost equal to the analysis for scalar case, with one exception: is
instead of the number of inequalities
tf0 + ϕ t ≥ t(0)
θ̄
19

20. Generalized logarithm for dual cone
We will use conjugates to prove the bound (11.48)
Let be a generalized logarithm for the proper cone , with degree
The conjugate of the function is , which is convex, and has
domain .
We define by (11.49)
is concave, and equal to a generalized logarithm with the same parameter (exercise 11.17)
It is called the
From (11.49) we obtain the inequality (11.50), which holds for any
, with equality holding if and only
ψ K θ
−ψ (−ψ)*(v) = sup
u
(vT
u + ψ(u))
−K* = {v|v ≺K* 0}
ψ̄ ψ̄(v) = − (−ψ)*(−v) = inf
u
(vT
u + ψ(u)) domψ̄ = intK* .
ψ̄ θ
dual logarithm
ψ̄(v) + ψ(u) ≤ uT
v
u ≻K 0, v ≻K* 0 ∇ψ(u) = v
20

21. Derivation of the basic bound (1/2)
We denote as , as , as , and as
From (in 11.42) and property (11.43), we conclude that
(10.50) for the pair gives
which becomes, using logarithmic homogeneity of ,
Subtracting the equality (11.51), we get
Summing over :
x⋆
(t) x x⋆
(μt) x+
λ⋆
i (t) λi ν⋆
(t) ν
tλi = ∇ψi(−fi(x))
u = − fi(x+
), v = μtλi
ψ̄i
i
21
(11.52)

22. Derivation of the basic bound (2/2)
We also have, from the definition of the dual function,
Multiplying this inequality by and adding to the inequality (11.52), we
get
We get the desired inequality (11.48) by re-arranging
μt
22