Thesis_Proposal2

Deﬁnitions Applications and Motivation SDP Representation of NPCO and MCO Drawbacks and Remedies Chebyshev Change of
Polynomial and Moment Conic Optimizations:
Theory and Applications
Mohammad M. Ranjbar,
Advisor: Farid Alizadeh
Rutgers, The State University Of New Jersey
Dec/01/2016
Mohammad M. Ranjbar, Advisor: Farid Alizadeh Rutgers
Polynomial and Moment Conic Optimizations: Theory and Applications

Outline
1 Deﬁnitions
2 Applications and Motivation
3 SDP Representation of NPCO and MCO
4 Drawbacks and Remedies
5 Chebyshev Change of Basis
6 Non-Symmetric Interior Point Method
7 Homogeneous Self-Dual Embedding
8 Numerical Results
9 Discretization
10 Conclusion and Future Works

Non-Negative Polynomial Cone:
Deﬁnition: Standard Basis
un
t = (1, t, t2
, . . . , tn
)T
,
Deﬁnition: Non-Negative (Positive) Polynomial Cone in standard
basis on interval [a, b]
Pn+
[a,b] = {p ∈ Rn+1
: p, un
t 0, ∀t ∈ [a, b]}.

Non-Negative Polynomial Cone vs Semideﬁnite Positive
Cone:
Non-Negative Polynomial Cone Semideﬁnite Positive Cone
{p :
n
i=0
pi ti ≥ 0, ∀t ∈ [a, b]} {X : yT Xy ≥ 0, ∀y}
p =
n
i=1
λi conv(pi , pi ) X =
n
i=1
λi qi qT
i

Non-Negative Polynomial Conic Optimization:
Non-Negative Polynomial Conic Optimization (NPCO):
(NPCO) min cT
1 s1 + · · · + cT
k sk
s.t. A1s1 + · · · + Aksk = b,
si
P
n+
i
[a,b]
0, i = 1, ..., k,
where ci ∈ Rni +1, Ai ∈ Rm×ni +1, b ∈ Rm and si ∈ Rni +1.

Moment Cone
Deﬁnition: Moment Cone in standard basis on interval [a, b]
Mn
[a,b] = cl(cone{un
t | ∀t ∈ [a, b]}).

Moment Conic Optimization:
Moment Conic Optimization (MCO):
(MCO) min cT
1 x1 + · · · + cT
k xk
s.t. A1x1 + · · · + Akxk = b,
xi M
ni
[a,b]
0, i = 1, ..., k,
where ci ∈ Rni +1, Ai ∈ Rm×ni +1, b ∈ Rm and xi ∈ Rni +1.

Notations: For simplicity we will show
c = (c1; · · · ; ck),
A = [A1, · · · , Ak],
x = (x1; · · · ; xk),
s = (s1; · · · ; sk),
M[a,b] = Mn1
[a,b] ⊗ · · · ⊗ Mnk
[a,b],
P+
[a,b] = P
n+
1
[a,b] ⊗ · · · ⊗ P
n+
k
[a,b].
Conic Optimization:
min cT
x
s.t. A x = b,
x K 0,
where K is either M[a,b] or P+
[a,b].

Deﬁnition: Hankel Operator:
H(.) : R2n+1
→ R(n+1)×(n+1)
H(x) =






x0 x1 . . . xn
x1 ...
...
xn+1
... ...
... ...
xn xn+1 . . . x2n






Deﬁnition: Dehankel Operator:
H∗
(.) : R(n+1)×(n+1)
→ R2n+1
[H∗
(Y )]2n
i=0 =





y0,0
y0,1 + y1,0
...
yn,n






Deﬁnition: Toeplitz Operator:
T(.) : Rn+1
→ R(n+1)×(n+1)
T(x) =






x0 x1 . . . xn
x1
...
... xn−1
...
...
...
...
xn xn−1 . . . x0






Deﬁnition: Detoeplitz Operator:
T∗
(.) : R(n+1)×(n+1)
→ Rn+1
[T∗
(Y )]n
i=0 =





y0,0 + ... + yn,n
y0,1 + ... + y1,0 + ...
...
y0,n + yn,0






Applications:
Applications of NPCO and MCO
1 Non-Negative Polynomial Conic Optimization
Statistics: Nonparametric Estimation With Shape Constraint
Finance: Nonparametric Cost Function, Production Function,
Utility Function and Option Pricing Under Shape Restriction
Approximation
Time Variant Network Flows: Maximum Flows Problem
with Time Variant Capacities, Min-cost Flows Problem with
Time Variant Costs
Engineering: Envelope Signal
2 Moment Cone Optimization
Statistics: Diﬀerent orders of moment of a distribution with
desired properties.

Statistics:
Consider a set of points in two dimension, (xi , yi ) for i = 1, ..., p.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0
0.2
0.4
0.6
0.8
1
1.2
Data Points

Statistics:
What is the Best Fitting Polynomial with the given degree?
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0.8
1
1.2
1.4
1.6
1.8
2
2.2
2.4
2.6
data1
Best Fitting Poly(deg=40)
Best Fitting Poly (deg=100)

Statistics:
Now what if the Best Fitting Polynomial has to be Positive
Polynomial?
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
-1
-0.5
0
0.5
1
1.5
2
2.5
3
3.5
4

Statistics:
Now what if the Best Fitting Polynomial has to be Positive
Polynomial?
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
-1
-0.5
0
0.5
1
1.5
2
2.5
3
3.5
4
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
-1
-0.5
0
0.5
1
1.5
2
2.5
3
3.5
4
Data Points
Best Fitting Positive Poly(deg=40)

Statistics:
What is the Best Envelope Convex Polynomial with the given
degree Below all data points?
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0
2
4
6
8
10
12
Data Points
Best Envelope Convex Poly(deg=60)
How about the Best Fitting Convex Polynomial with the given
degree?

Statistics: Nonparametric Estimation under Shape
Constraints I
In general, all of these Nonparametric Estimation under shape
constraints can be formulated as:
min
s
p
i=1
(s(ti ) − yi )2
s.t s(t) ≥ 0, ∀ t ∈ [a, b],
s (t) ≥ 0, ∀ t ∈ [a, b], ←− increasing or decreasing
s (t) ≥ 0, ∀t ∈ [a, b], ←− convexity or concavity
where s and s are the ﬁrst and the second derivative of s(t)
respectively.

Statistics: Nonparametric Estimation under Shape
Constraints II
This problem can be cast as a NPCO and SOCP.
min z
s.t
z
Vs − y socp 0,
s Pn+
[a,b]
0,
s
P
(n−1)+
[a,b]
0,
s
P
(n−2)+
[a,b]
0,
where V is the Vandermonde matrix (Vi,j = tj
i ).

Finance:
Usually, in ﬁnance and economy:
Cost function must be Increasing and Convex
Production function must be Increasing and Concave
Utility function must be Increasing and Concave
Call option pricing function must be Decreasing and Convex
All of these cases can be formulated as:
min T(s)
s.t. A s = b,
s(t) Pn+
[a,b]
0,
s (t) Pn−1+
[a,b]
0,
s (t) Pn−2+
[a,b]
0.
where T(.) is a linear operator.

Approximation or Envelope Polynomial:
What is the Best Polynomial (coeﬃcients of the polynomial) that
approximates all other polynomials and is below (or above) them?
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
2.5
3
3.5
4
4.5
5
5.5

Approximation or Envelope Polynomial:
What is the Best Polynomial (coeﬃcients of the polynomial) that
approximates all other polynomials and is below (or above) them?
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
2.5
3
3.5
4
4.5
5
5.5
-0.7 -0.65 -0.6 -0.55 -0.5 -0.45 -0.4 -0.35 -0.3 -0.25
3
3.1
3.2
3.3
3.4
3.5
3.6
Approx Poly(deg=80)

Increase Degree of Polynomial
As the degree of polynomial increases, we get better optimal
solution.
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
2.5
3
3.5
4
4.5
5
Degree, n=20,
Degree, n=200,

Approximation:
This class of problem can be formulated as:
max
b
a
x(t)dt = u[a,b], x
s.t. x(t) Pn+
[a,b]
pi (t), i = 1, ..., k,
x(t) Pn+
[a,b]
0.
What if we have arbitrary functions instead of polynomials?
max
b
a
x(t)dt = u[a,b], x
s.t. x(t) ≤ fi (t), i = 1, ..., k, ∀t ∈ [a, b],
x(t) Pn+
[a,b]
0.

Time Dependent Network Flows: I
Conventional Network Flows problems: Network flows with
Constant Input Data.
For example, in Maximum Flows problem “One seeks to push
maximum flows from source to sink with respect to the
Constant Capacities of the edges”.
Time Dependent Network Flows Problems: Network flows with
Time-Variant Input Data.
For example in Time Variant Maximum Flows, “One seeks
to push maximum flows from source to sink with respect to
the Time Dependent Capacities of the edges”.

Time Dependent Network Flows: II
1Source
2
3
i
j
k
r
m Sink
p1,2(t)
p1,3(t)
pk,m(t)
pr,m(t)
Time Dependent Maximum Flows problem can be formulated as:
max
i|(s,i)∈E
b
a
Xs,i (t) dt
s.t.
j|(j,i)∈E
Xj,i (t) −
j|(i,j)∈E
Xi,j (t) = 0, ∀i, ∀t ∈ [a, b],
Xi,j (t) Pn+
[a,b]
Pi,j (t), (i, j) ∈ E, ∀t ∈ [a, b],
Xi,j (t) Pn+
[a,b]
0, (i, j) ∈ E, ∀t ∈ [a, b].

Time Dependent Network Flows: III
How about when the input data are arbitrary functions?
max
i|(s,i)∈E
b
a
Xs,i (t) dt
s.t.
j|(j,i)∈E
Xj,i (t) −
j|(i,j)∈E
Xi,j (t) = 0, ∀i, ∀t ∈ [a, b],
Xi,j (t) Pn+
[a,b]
fi,j (t), (i, j) ∈ E, ∀t ∈ [a, b],
Xi,j (t) Pn+
[a,b]
0, (i, j) ∈ E, ∀t ∈ [a, b].

Engineering:
Envelope Signal: In Electrical Engineering and Signal Processing
we need to have the Envelope Signal of given signal or signals

NPCO+SOCP+LP:
All of preceding problems can be formulated as Non-Negative
Polynomial Conic Optimization with the combination of the
other well-known cones such as LP, SOCP.
min cL
xL
+
i
cS
j xS
j +
l
cP+
l xP+
l
s.t. AL
xL
+
i
AS
j xS
j +
l
AP+
l xP+
l = b,
xL
≥ 0, xS
j SOCP 0, xP+
l Pn+
[a,b]
0.

SDP Representation of NPC:
Theorem (Nesterov 97)
a) p ∈ P2n+
R if and only if ∃Y 0 s.t. p = H∗(Y ),
b) p ∈ P2n+
[a,b] if and only if
∃Y1, Y2 0, s.t. p = H∗
(Y1) + AH∗
(Y2),
c) p ∈ P
(2n+1)+
[a,b] if and only if
∃Y1, Y2 0, s.t. p = BH∗
(Y1) + CH∗
(Y2),
where A, B, C are three matrices dependent on the interval shift
the vectors H∗(Y1), H∗(Y2).

SDP Representation of NPCO:
Using this theorem, NPCO can be cast as SDP. For instance, when
K = P2n+
R :
min cT
x =⇒ min H(c)•Y
s.t. Ax = b, s.t. H(ai )•Y = bi , i = 1, .., k,
x P2n+
R
0, Y 0,
where s = H∗(Y ).

SDP Representation of MC:
Theorem (Karlin-Studden 66, Nesterov 97)
a) x ∈ M2n
R ⇔ H(x) 0.
b) x ∈ M2n
[a,b] ⇔
H(x) 0,
H((−abxi + (b + a)xi+1 − xi+2)2n−2
i=0 ) 0
c) x ∈ M2n+1
[a,b] ⇔
H((−axi + xi+1)2n
i=0) 0,
H(bxi − xi+1)2n
i=0) 0.

SDP Representation of MCO:
Using this theorem, MCO can be cast as SDP. For instance, when
K = M2n
R :
min cT
x =⇒ min cT
x
s.t. Ax = b, s.t. Ax = bi , i = 1, .., k,
x M2n
R
0, H(x) 0.

Drawbacks and Remedies:
However, casting these conic optimizations as SDP have two major
drawbacks:
(a) It is extremely ill-conditioned.
(b) It quadratically increases the dimension of the problem.
We have proposed two remedies for these drawbacks, this is:
(a) Orthogonal change of basis.
(b) Non-symmetric interior point methods.

Chebyshev Change of Basis
Chebyshev Basis
T0(t) = 1,
T1(t) = t,
Tn+1(t) = 2tTn(t) − Tn−1(t).
Deﬁnition: Moment and Non-negative polynomial cone in
Chebyshev basis on [−1, 1] are deﬁned as:
Mn
Ch = cl(Cone{(T0(t), T1(t), ..., Tn(t))T
| ∀t ∈ [−1, 1]}),
Pn+
Ch = {s ∈ Rn+1
|
n
j=0
sj Tj (t) 0, ∀t ∈ [−1, 1]}.

Representation of Moment Cone in Chebyshev Basis
Theorem (Papp 2011, Ranjbar 2016)
a) x ∈ M2n
Ch over R if and only if H(x) + T(x) 0.
b) x ∈ M2n
Ch over [−1, 1] if and only if



H(x) + T(x) 0,
H([xi − 1
2xi+2]2n−2
i=0 − 1
2[x2; x1; [xi ]2n−4
i=0 ])
+T([xi − 1
2xi+2]n−1
i=0 ] − 1
2[x2; x1; [xi ]n−3
i=0 ]) 0.
c) x ∈ M2n+1
Ch over [−1, 1] if and only if
H([xi + xi+1]2n
i=0) + T([xi + xi+1]n
i=0) 0,
H([xi − xi+1]2n
i=0) + T([xi − xi+1]n
i=0) 0.

Change of Basis Result
By using Chebyshev change of basis, SDP representation of MCO
and NPCO of much higher sizes can be solved.
Our experiences showed problem with block dimension of 3000 can
be solved.
But casting MCO or NPCO as SDP requires squaring the number of
decision variables and increases the number of constraints.
For example, if NPCO has 103
blocks and the degree of each block
is of size 103
, then it needs O(109
) ﬂoating point memory storage
to only save the hessian.
The situation is even worse in term of running time, for example, a
problem of 200 blocks and each block of dimension 1200 needs
more than 4 days to be solved by a server with 32 GB RAM and
3GHZ processor.
The so-called non-symmetric interior point method will be the remedy.

Primal-Dual Path-Following Algorithm (IPM): I
To use Primal-Dual Path-Following IPM, we need to have
followings:
An efficiently computable LHSCB function for the cones.
The gradient and Hessian of barrier function should be
efficiently computable
The pair of primal-dual problem.
An efficient algorithm that in the case of feasible problem
converges to optimal solution and in the case of infeasibility
detects it.
The algorithm should not require any information about the
dual barrier function.
Following has been offered

Theorem (3) gives a suitable barrier function for moment cone
in Chebyshev basis.
It has been shown that the gradient and the hessian of the
barrier are eﬃciently computable.
Karlin-Studden solved the primal-dual setting.
Homogeneous Self-Dual embedding (HSD) is used which
either converges to optimal solution or detects infeasibility.
Skajaa-Ye (2015) proposed a HSD primal-dual
predictor-corrector IPM which does not need any information
of the dual barrier.

Barrier Function for Moment Cone in Chebyshev Basis
logarithmic homogeneous self-concordant barrier (LHSCB)
function for moment cone in Chebyshev basis are deﬁned:
F(x) = − ln det((H + T)(x)), ∀x ∈ M2n
ChR
,
F(x) = − ln det((H + T)(x))−
ln det((H + T)([xi −
1
2
xi+2]2n−2
i=0 −
1
2
[x2; x1; [xi ]2n−4
i=0 ])),
∀x ∈ M2n
Ch,
F(x) = − ln det((H + T)([xi + xi+1]2n
i=0))−
ln det((H + T)([xi − xi+1]2n
i=0)), ∀x ∈ M2n+1
Ch ,

Gradient of Barrier Function
x (F(x)) = −(H∗
+ T∗
)((H + T)−1
(x)), ∀x ∈ int(M2n
ChR
),
x (F(x)) = −(H∗
+ T∗
)((H + T)−1
(x))−
(H∗
+ T∗
)((H + T)−1
([xi −
1
2
xi+2]2n−2
i=0 −
1
2
[x2; x1; [xi ]2n−4
i=0 ])),
∀x ∈ int(M2n
Ch),
x (F(x)) = −(H∗
+ T∗
)((H + T)−1
([xi + xi+1]2n
i=0))−
(H∗
+ T∗
)((H + T)−1
([xi − xi+1]2n
i=0)), ∀x ∈ int(M2n+1
Ch ).
Let us show gradient by gx .
gx ∈ R2n+1
in contrast to (n + 1) × (n + 1) matrix in SDP
formulation.

Hessian of Barrier Function
2
x (F(x)) =conv2T((H + T)−1
(x)), ∀x ∈ int(M2n
ChR
),
2
x (F(x)) =conv2T((H + T)−1
(x))+
conve2T((H + T)−1
([xi −
1
2
xi+2]2n−2
i=0 −
1
2
[x2; x1; [xi ]2n−4
i=0 ])),
∀x ∈ int(M2n
Ch),
2
x (F(x)) =conv2T((H + T)−1
([xi + xi+1]2n
i=0))+
conv2T((H + T)−1
([xi − xi+1]2n
i=0)), ∀x ∈ int(M2n+1
Ch ).
conv2T(.) is the convolution of two bivariate polynomials in
Chebyshev basis.
Let us show hessian by Hx .
Hessian is a (2n + 1) × (2n + 1) matrix in contrast to
(n + 1)2
× (n + 1)2
in SDP formulation.

Computation of Gradient and Hessian I
Solving a linear systems of equations with (hankel+toeplitz)
coeﬃcient matrices can be done in O(rn2
) and O(rn) memory.
In implementation, we have used “drsolve” package written by Arico
and Rodriguez [2] to computing the inverse of (hankel+toeplitz)
matrix.
The convolution of two polynomials of degree n in Chebyshev basis
can be computed, in O((12n + 3) log 2n − 14n + 15) via Discrete
Cosine Transform (DCT) .
The convolution of two bivariate polynomial in Chebyshev basis of
degree n × n can be done by convolution of two single variable
polynomials of degree n2
.

Computation of Gradient and Hessian II
In implementation, we have used a version of DCT method based
on the work of Baszenski and Tasche [3]. This is
conv2T(A, B) =
4
N2
CN ((CN ACT
N ) ◦ (CN BCT
N ))CT
N
where A and B ∈ R(N+1)×(N+1)
, ◦ is the Hadamard product, and
CN ∈ R(N+1)×(N+1)
is
CN = (εN,j cos
ijπ
N
)N
i,j=0,
with εN,0 = 1
2 and εN,j = 1, j = 1, ..., N − 1.
The computation work of this method is O(n2
log n).

Barrier Function for Non-Negative Polynomial Cone I
We have investigated two LHSCB function for non-negative
polynomial cone.
Faybusovich Barrier Function [4]: Faybusovich has
proposed following LHSCB function for P2n+
R
F∗
(s) =
1
2
ln det D(s),
Di,j (s) =
1
−1
ui (t)uj (t) − uj (t)ui (t)
s2(t)
dt, i, j = 0, ..., n.
This barrier is not eﬃciently and practically computable.

Barrier Function for Non-Negative Polynomial Cone II
Convex conjugate of the moment barrier function: Using
the definition of convex conjugate function the LHSCB
function of s ∈ P2n+
R can be defined as:
F∗
(s) = − inf
x∈int(M2n
R )
{ x, s − ln det H(x)},
This is a convex optimization which does not have any closed
form solution.
Therefore, even finding the value of the barrier function of
non-negative polynomial cone, is a challenge.
Hence, we need an algorithm which does not need the information
of the dual barrier function.

Duality:
Theorem (Karlin-Studden 66)
Moment cone and Non-Negative Polynomial cone are dual.
(M[a,b])∗
= P+
[a,b].
Using this duality, MCO and NPCO can be put in a non-symmetric
primal-dual setting.
(MCO) min cT
x
dual
⇐=⇒ (NPCO)max bT
y
s.t. A x = b, s.t. AT
y+s = c
x M[a,b]
0, s P+
[a,b]
0.

Homogeneous Self-Dual (HSD) Embedding
min 0
s.t. Ax−bτ = 0
AT
y +cτ−s = 0
bT
y−cT
x −κ= 0
(x, τ) ∈ M[a,b] × R+
, y ∈ Rm
, (s, κ) ∈ P+
[a,b] × R+
Path-following algorithm on HSD model:
does not need initial feasible solution.
does not increase the size of the problem.
In the case of feasibility gives the optimal solution.
In the case of infeasibility detects the source.

HSD Solution
Theorem (Ye-Todd-Mizuno)
Assume (x∗, τ∗, y∗, s∗, κ∗) solves HSD model. Then
a) (x∗, τ∗, s∗, κ∗) is complementary, this is, x∗T s∗ + τ∗κ∗ = 0.
b) If τ∗ > 0 then (x∗, y∗, s∗)/τ∗ is optimal for HSD.
c) If κ∗ > 0 then either primal or dual is infeasible.

More Notations:
¯x =
x
τ
¯s =
s
κ
¯F(¯x) = F(x) − log τ, ¯F∗
(¯s) = F∗
(s) − log κ,
¯K = K × R+
, ¯K∗
= K+
× R+
,
¯ν = ν + 1
µ(z) = ¯xT
¯s/¯v,
ψ(¯x, ¯s, µ(z)) = ¯s − µ(z)g¯x .
where z = (¯x, y, ¯s)
G =


0 A −b
−AT
0 c
bT
−cT
0

 ,


rp
rd
rc

 = G
y
¯x
−
0
¯s
.

Path Following IPM:
The idea of Path Following IPM is to Approximately Follow
the Central Path and Stay Close to it.
Central Path
Approximately
Close to central path

Central Path:
Central Path
Central Path is deﬁned as the unique
solution of the parametrized equations.
G
y
¯x
−
0
¯s
= γ


rp
rd
rc

 (CP)
¯s − γµg¯x = 0,
where 0 ≤ γ ≤ 1.
z=(x,,y,s,𝜿)
Central Path
Opt Sol
Feasibility

Approximately:
Approximately
Approximately means to ﬁnd the solution
of central path by using First Order
Newton Method, this is,
G
dy
d ¯x
−
0
d¯s
= (1 − γ)


rp
rd
rc

 (Dir)
ds + µHx dx = −s − γµgx
τdκ + κdτ = −τκ + γµ.
z=(x,𝞃,y,s,𝜿)
Central Path
Opt Sol
Feasibility
dz=(dx,d𝞃,dy,ds,d𝜿)

Close to Central Path:
Hessian Neighborhood:
N(η) = {z ∈ F :
s − µ(z)gx
κ + µ(z)/τ
∗
(x,τ)
≤ ηµ(z)}
where v ∗
x = H
−1/2
x v .
In implementation computing Hessian is expensive instead inﬁnite
norm will be used which is related to hessian.
|| F(x)||−1
∞ (s + µ F(x))
τκ − µ ∞
≤ ηµ(z)

Predictor-Corrector:
The idea of Predictor-Corrector IPM is to start from an initial
point near the central path and then alternate between the
Prediction Phase and the Correction Phase.

Prediction Phase (γ = 0)
Find direction that Reduces the
Residuals and Duality gap and is not
Concerned about the Centrality
G
dyp
d ¯xp
−
0
d¯sp
=


rp
rd
rc

 (PD)
dsp + µHx dxp = −s
τdκp + κdτp = −τκ.
z=(x,𝞃,y,s,𝜿)
Larg 𝓝
Small 𝓝
zp=(xp,𝞃p,yp,sp,𝜿p) dzp=(dxp,d𝞃p,dyp,dsp,d𝜿p)
Central Path

Correction Phase (γ = 1)
Find direction that Reduce the
Centrality but Keeps the Residuals
and the Duality Gap Unchanged
G
dyc
d ¯xc
−
0
d¯sc
=


0
0
0

 (CD)
dsc + µHxp dxc = −sp − µpgxp
τpdκc + κpdτc = −τpκp + µp.
z=(x,𝞃,y,s,𝜿)
Larg 𝓝
Small 𝓝
Central Path
Zc=(xc,𝞃c,yc,sc,𝜿c)
dzp=(dxp,d𝞃p,dyp,dsp,d𝜿p)

Next Prediction and Correction Iteration:
z=(x,𝞃,y,s,𝜿)
Larg 𝓝
Small 𝓝
Central Path

Next Prediction and Correction Iteration:
z=(x,𝞃,y,s,𝜿)
Larg 𝓝
Small 𝓝
Central Path
Larg 𝓝
Small 𝓝
Central Path
z=(x,𝞃,y,s,𝜿)

Convergency:
Skajaa-Ye [7] showed if η ≤ 1/6 then by taking a ﬁxed
step-length, αp = Ω(1/
√
ν), z+
p ∈ N(2η).
In practical implementation a larger step-length can be taken.
Skajaa-Ye (see lemma 6 in [7]) showed if z+
p ∈ N(2η) then by
applying the correction phase at most two times z+
c ∈ N(η).
They also showed that by using the speciﬁed step-length the
algorithm terminates with a -solution in no more than
O( (ν) log(1/ )).

Symmetric vs Non-Symmetric HSD P-D P-C for LP
Non-symmetry comes from duality gap
LP has both formats
Symmetric
Prediction
Sdxp + Xdsp = −Xs
Correction
Spdxc + Xpdsc = −Xpsp + µpe
Non-Symmetric
Prediction
µX−2
dxp + dsp = −s
Correction
µpX−2
p dxc + dsc = −sp + µpX−1
p e

Two experiments have been done:
1- Number of iterations in correction phase are ﬁxed to 1 in
symmetric and to 2 in non-symmetric algorithm.
Algorithm m n Iter. Avg. Corr.
Symmetric 20 2e2 11 1
Non-Symmetric 20 2e2 14 2
Table: Symmetric vs Non-Symmetric HSD P-D P-C IPM for LP

2- Number of iterations in correction phase is decided by
algorithm.
Algorithm m n Iter. Avg. Corr.
Symmetric 20 2e2 11 1.54
Non-Symmetric 20 2e2 11 1.81
Table: Symmetric vs Non-Symmetric HSD P-D P-C IPM for LP

Non-Symmetric HSD P-D P-C for MCO-NPCO
To show the numerical results of non-symmetric HSD P-D P-C for
MCO-NPCO, we have considered the approximation problem.
Algorithm blk m n Iter. Avg. Corr.
MCO-NPCO 3 41 123 16 1.5
SDP 3 121 1362 12 1
MCO-NPCO 3 201 603 20 1.5
SDP 3 601 30802 22 1
MCO-NPCO 3 401 1203 23 1.65
SDP 3 1201 1216022 20 1
Table: Symmetric vs Non-Symmetric HSD P-D P-C IPM for MCO-NPCO

Discretization of MCO and NPCO I
Considered a reﬁned grid of points in [a, b].
S[a b] := {a ≤ t1 < t2 < ... < tp ≤ b}
Therefore S[a,b] is going to be an approximation of [a, b] when
p is large enough.
Then, the discrete moment and non-negative polynomial
cones over S[a,b] are deﬁned:
Mn
S[a,b]
= cl(Cone{un
tj
, j = 1, .., p})
Pn+
S[a,b]
= {s ∈ Rn+1
: s, un
tj
≥ 0, j = 1, ..., p}

Discretization of MCO and NPCO II
It has been shown that
Mn
S[a,b]
⊂ Mn
[a,b],
Pn+
[a,b] ⊂ Pn+
S[a,b]
.
As p −→ ∞,
Mn
S[a,b]
−→ Mn
[a,b],
Pn+
S[a,b]
−→ Pn+
[a,b].

Discretization of MCO and NPCO III
Using this approximation, discrete MCO and NPCO can be
reformulated as a linear programming. This is:
min cT
x min cT
s
s.t. Ax = b, s.t. As = b,
x =
j=1
p
αj un
tj
, un
tj
,s ≥ 0, j = 1, ..., p,
αj ≥ 0, j = 1, ..., p,

LP vs SDP vs Non-Symmetric for NPCO:
Consider NPCO:
min cT
s
s.t. A s = b (NPCO)
s Pn+
[a,b]
0.
The number of variables and constraints in LP, SDP and
NPCO formulations:
Problem # of Const. # of Var.
LP p n
SDP kn O(kn2)
Non-Symmetric n kn
Table: Number of Const. & Var. in NPCO with k Blocks

Numerical Results: SDP vs Discrete (LP)
Approximation Problem:
max
b
a
s(t)dt = e[a,b], s
s.t. s(t) Pn+
[a,b]
pi (t), i = 1, ..., m,
s(t) Pn+
[a,b]
0.
Time and Feasibility of SDP and LP:
Form m n p time(Sec.) feasibility optval
SDP 26 100 −− 3.66E + 01 Yes 6.0000
LP 26 100 102
3.33E + 00 infeas. prob. −−
LP 26 100 103
3.49E + 00 infeas. sol. 6.0026
LP 26 100 104
5.84E + 00 Yes 6.0015
Table: NPCO: SDP vs LP as p Increases

Figure: SDP vs Discrete (LP)
Time and Feasibility:
0.97 0.975 0.98 0.985 0.99 0.995 1
2.985
2.99
2.995
3
3.005
3.01
3.015
3.02
max flow LP 26-100-1000
max flow LP 26-100-10000
max flow SDP 26-100

Conclusion: I
Non-negative polynomial and Moment conic optimizations
have been defined.
Different applications in statistics, finance, network flows etc
have been mentioned.
SDP representation of NPCO and MCO have been shown.
The drawbacks of SDP have been mentioned.
The remedies have been proposed.
Orthogonal change of basis has been shown.
A tailor-made non-symmetric HSD P-D P-C based on
Skajaa-Ye algorithm has been proposed.

Conclusion: II
Numerical results for non-symmetric vs symmetric HSD for LP
have been shown.
Numerical results of implementation of non-symmetric HSD
P-D P-C have been shown.
Discretization of NPCO and MCO have been shown.

Future: I
Many real world problems can be formulated as an optimization
problem over the combination of diﬀerent cones (LP,SOCP, NPCO)
Write a software package that can solve following problem
min cL
xL
+
i
cS
j xS
j +
l
cP+
l xP+
l
s.t. AL
xL
+
i
AS
j xS
j +
l
AP+
l xP+
l = b,
xL
≥ 0, xS
j SOCP 0, xP+
l Pn+
[a,b]
0.

Future: II
Write a software package that can solve following problem
min f ( xL
, xS
j , xP+
l )
s.t. AL
xL
+
i
AS
j xS
j +
l
AP+
l xP+
l = b,
xL
≥ 0, xS
j SOCP 0, xP+
l Pn+
[a,b]
0.
where f (x) is a well-known convex function, like entropy
f (x) =
n
i=1
xi log xi .

Question!?

Akle Serrano S., Algorithms for unsymmetric cone
optimization and an implementation for problems with the
exponential cone, PhD Thesis, Stanford university (2015)
Arico, A., Rodriguez, G., A fast solver for linear systems with
displacement structure, NUMERICAL ALGORITHMS, 55,
529-556 (2010)
Baszenski, G., Tasche, M., Fast Polynomial Multiplication and
Convolution Related to the Discrete Cosine Transform, Linear
Algebra and Its Applications, 252, 1-25 (1997)
Faybusovich, L., Self-Concordant Barriers for Cones Generated
by Chebyshev Systems, SIAM J. Optim., 12(3), 770-781
(2002)

Nesterov, Y., Squared Functional Systems and Optimization
Problems, High Performance Optimization, 33, 405-440,
Kluwer Academic Press, (2000)
Nesterov, Y. E., Towards Nonsymmetric Conic Optimization.
Optim. Method. Softw., 27, 893-917 (2012)
Skajaa A., Ye Y., A homogeneous interior-point algorithm for
nonsymmetric convex conic optimization, Math. Program.,
Ser. A, 150, 391-422 (2015)

Thesis_Proposal2

Recommended

Recommended

More Related Content

Similar to Thesis_Proposal2

Similar to Thesis_Proposal2 (11)

Thesis_Proposal2