Duality of a Generalized Absolute Value Optimization Problem

Duality of a Generalized Absolute Value
Optimization Problem
Shota Yamanaka* Nobuo Yamashita
Graduate School of Informatics, Kyoto University
2016/8/9
S. Yamanaka*, N. Yamashita Generalized AVP 2016/8/9 1 / 33

1 Absolute Value Programming Problem (AVP)
2 A Generalized Absolute Value Optimization Problem (GAVP)
3 Examples with linear and nonlinear terms
4 Examples with only nonlinear terms
5 The Lagrangean duality and GAVP duality
6 Conclusion

6 Conclusion

Absolute Value Programming (Mangasarian, 2007)
The absolute value programming (AVP) problem is written as
min ˜cT x ` ˜dT |x|
s.t. ˜Ax ` ˜B|x| “ ˜b,
˜Hx ` ˜K|x| ě ˜p,
(P0)
where ˜c, ˜d P Rn,˜b P Rk, ˜p P Rℓ, ˜A, ˜B P Rkˆn, ˜H, ˜K P Rℓˆn, and
|x| :“ p|x1|, . . . , |xn|qT
.
It is generally nonconvex and nondifferentiable.
It includes the integer optimization problem:
x P t0, 1u ô |x ´ 1| “ 0.

Absolute Value Programming (Mangasarian, 2007)
The absolute value programming (AVP) problem is written as
min ˜cT x ` ˜dT |x|
s.t. Ãx ` ˜B|x| “ ˜b,
˜Hx ` ˜K|x| ě ˜p,
(P0)
where ˜c, ˜d P Rn,˜b P Rk, ˜p P Rℓ, Ã, ˜B P Rkˆn, ˜H, ˜K P Rℓˆn, and
|x| :“ p|x1|, . . . , |xn|qT
.
The equation Ãx ` ˜B|x| “ ˜b, called the absolute value equation, is
equivalent to the linear complementarity problem.
The AVP includes the mathematical problem with linear
complementarity constraints.

The dual problem of the AVP
The dual problem of pP0q is given by
max ˜bT u ` ˜pT v
s.t. | ˜AT u ` ˜H T v ´ ˜c| ` ˜B T u ` ˜K T v ď ˜d,
v ě 0.
(D0)
The above problem is always a linear programming (LP):
|x| ď 1 ô ´1 ď x ď 1.
The weak duality theorem holds between pP0q and pD0q.

The weak duality theorem of the AVP
Theorem 1
If x P Rn and pu, vq P Rk ˆ Rℓ are feasible solutions of pP0q and pD0q,
respectively, then the following inequality holds:
˜cT
x ` ˜dT
|x| ě ˜bT
u ` ˜pT
v.
The lower bound of pP0q is obtained by solving pD0q.
pP0q is nonconvex and nondifferentiable, pD0q is always an LP.
pP0q can handle only linear and the absolute value terms.

6 Conclusion

A Generalized AVP (GAVP)
A Generalized absolute value optimization problem (GAVP) is written as
min cT x ` dT Ψpxq
s.t. Ax ` BΨpxq “ b,
Hx ` KΨpxq ě p,
(P)
where c P Rn, d P Rm, b P Rk, p P Rℓ, A P Rkˆn, B P Rkˆm,
H P Rℓˆn, K P Rℓˆm, and Ψ: Rn Ñ Rm is nonlinear.
It is generally nonconvex and nondifferentiable.
The function Ψ is the generalization of the absolute value function.

The GAVP dual problem and the assumption of Ψ
The dual problem of pPq is given by
max bT u ` pT v
s.t. Ψ˚pAT u ` HT v ´ cq ` BT u ` KT v ď d,
v ě 0.
(D)
Assumption 1
The function Ψ: Rn Ñ Rm, Ψ˚ : Rn Ñ Rm satisﬁes the following
conditions:
ΨpxqT Ψ˚pyq ě xT y @x P Rn, @y P Rn,
Ψpxq ě 0 @x P Rn,
Ψ˚pxq ě 0 @x P Rn.
We call this function Ψ a norm-like function.

The weak duality theorem of the GAVP problem
Proposition 2
Suppose that Ψ is a norm-like function. Then, the following inequality
holds:
cT
x ` dT
Ψpxq ě bT
u ` pT
v
for all x P Rn and pu, vq P Rk ˆ Rℓ feasible points of (P) and (D),
respectively.

The property of the function Ψ˚
If Ψpxq “ |x|, then Ψpxq “ Ψ˚pxq.
Let Ψ: Rn Ñ R, Ψ˚ : Rn Ñ R.
If Ψpxq “ }x}p, then Ψ˚pxq “ }x}q, where 1
p ` 1
q “ 1.
The function Ψ can be nonconvex.
We introduce some examples of norm-like function Ψ
that is equivalent to Ψ˚.

Assumption for norm-like functions
Assumption 2
The function Ψ: Rn Ñ Rm is decomposed as follows:
Ψpxq “
»
—
–
ψ1pxI1 q
...
ψmpxIm q
fi
ffi
fl , ψi : Rni
Ñ R, i “ 1, . . . , m,
where Ii Ď t1, . . . , mu is a set of indices satisfying
Ii X Ij “ H, i ‰ j,
mÿ
i“1
|Ii| “ n,
and xIi P Rni is a disjoint subvector of x.

6 Conclusion

The generalized AVP
We consider the primal and dual problems that have linear and
nonlinear terms.
min cT x ` dT Ψpxq
s.t. Ax ` BΨpxq “ b,
Hx ` KΨpxq ě p,
(P)
max bT u ` pT v
s.t. ΨpAT u ` HT v ´ cq ` BT u ` KT v ď d,
v ě 0.
(D)

Example of norm-like function
Ψpxq “
»
—
–
ψ1pxI1 q
...
ψmpxIm q
fi
ffi
fl , ψi : Rni
Ñ R, i “ 1, . . . , m
Proposition 3
Suppose that the function Ψ is described above and Gi P Sni
`` satisfies
minjtλjpGiqu ě 1, where λjpGiq is the j-th eigenvalue of Gi.
Then, the function Ψ given above with
ψipxIi q “
b
xIi
T GixIi , i “ 1, . . . , m,
is a norm-like function.

Example: Conic linear programming
Let x “ px1, x2q P R ˆ Rn´1. We consider the linear second-order cone
programming (SOCP) problem as
min cT x
s.t. Ax “ b,
x1 ´ }x2}2 ě 0.
The above problem is rewritten in GAVP form as
min cT x ` 0T Ψpxq
s.t. Ax ` 0 Ψpxq “ b,
p1, 0, . . . , 0qx ´ p0, 1qΨpxq ě 0,
where Ψ: Rn Ñ R2, Ψpxq “ p|x1|, }x2}2qT .

Example: Conic linear programming
The dual of the previous problem is
max bT u
s.t. |pAT uq1 ` v ´ c1| ď 0,
}pAT uq2 ´ c2}2 ď v,
v ě 0.
The ﬁrst constraint of the above problem is v “ c1 ´ pAT uq1. Then,
max bT u
s.t. }pAT uq2 ´ c2}2 ď c1 ´ pAT uq1,
which is the standard form of the SOCP dual problem.

Example of norm-like function
Ψpxq “
»
—
–
ψ1pxI1 q
...
ψmpxIm q
fi
ffi
fl , ψi : Rni
Ñ R, i “ 1, . . . , m
Proposition 4
Suppose that Ψ is described above. For any θi : Rni Ñ R satisfying
θipxIi q ě }xIi }2
2, i “ 1, . . . , m,
and a positive constant αi ě 1
2 , the function Ψ with
ψipxIi q “ θipxIi q ` αi, i “ 1, . . . , m,
is a norm-like function.

Example: Group Lasso type problem
We consider a primal problem written by
min }b ´ Ax}2
2 ` λ
mÿ
i“1
}xIi }2
where λ ě 0. If we denote z :“ b ´ Ax, then the above problem can be
rewritten as:
min }z}2
2 ` λ
mÿ
i“1
}xIi }2
s.t. rA Iksˆx “ b,
where ˆx “ px1, . . . , xn, z1, . . . , zkq P Rn`k and Ik P Rkˆk is the identity.

Example: Group Lasso type problem
The previous problem is described in GAVP form as:
min pλ, . . . , λ, 1qΨpˆxq ´ 1
2
s.t. rA Iksˆx “ b,
where
Ψpˆxq :“
ˆ
}xI1 }2, . . . , }xIm }2, }z}2
2 `
1
2
˙T
.
Then, the dual problem is written by
max bT u ´ 1
2
s.t. ΨprA IksT uq ď pλ, . . . , λ, 1qT .

Example : Group Lasso type problem
The dual problem is rewritten as
max bT u ´ 1
2
s.t. }pAT qIi u}2 ď λ, i “ 1, . . . , m,
}u}2
2 ď 1
2 .
where pAT qIi is a submatrix of AT with pAT qj, j P Ii as its rows.
The primal problem is written by
min }b ´ Ax}2
2 ` λ
mÿ
i“1
}xIi }2
where λ ě 0.

6 Conclusion

GAVP problem with only nonlinear terms
We consider the GAVP primal problem (P) that has no linear terms in its
objective and constraint functions.
min dT Ψpxq
s.t. BΨpxq “ b,
KΨpxq ě p.
(P1)
And, the dual problem of pP1q is given by
max bT u ` pT v
s.t. BT u ` KT v ď d,
v ě 0.
(D1)

Assumption for the function Ψ
Assumption 3
The function Ψ: Rn Ñ Rm satisﬁes the following conditions:
ΨpxqT Ψpyq ě xT y @x P F, @y P F,
Ψpxq ě 0 @x P F,
where F is a closed set. We also call this function Ψ a norm-like function.
If we set F :“ tx | x ě 0u, then Ψpxq :“ px1, x2, x2
1, x2
2, x1x2qT is a
norm-like function.

Example: The quadratic optimization problem
We consider the nonconvex quadratic optimization problem:
min x2
1 ´ x2
2
s.t. x2
1 ` x2
2 ď 4
x2
1 ` px2 ´ 1q2 ě 1
x ě 0.
The optimal value is ´4 at x˚ “ p0, 2q.

Example: The quadratic optimization problem
We obtain the GAVP dual problem as
max ´4v1
s.t. v3 ď 0
´2v2 ` v4 ď 0
´v1 ` v2 ď 1
´v1 ` v2 ď ´1
v ě 0,
which is just a linear programming.
The optimal value is ´4 at v˚ “ p1, 0, 0, 0q.

6 Conclusion

The dual optimal value
Theorem 5
Suppose that the GAVP dual problem (D) has an optimal solution. Then,
we have
f˚
DL
ě f˚
D
where f˚
DL
and f˚
D are the optimal values of the Lagrangean dual problem
(DL) and the GAVP dual problem (D), respectively.
The GAVP dual gives the lower bound of the Lagrangean dual.
However, there exists a closed form for GAVP dual problem.
We consider conditions under which the GAVP and Lagrangean dual
problems are equivalent.

Assumption for the equivalence
Ψpxq “
»
—
–
ψ1pxI1 q
...
ψmpxIm q
fi
ffi
fl , ψi : Rni
Ñ R, i “ 1, . . . , m
Assumption 4
For any x P Rn, the function ψi : Rni Ñ R satisfies the following
conditions:
1 ψipαxIi q ď αψipxIi q, α ą 0, i “ 1, . . . , m,
2 }xIi }2
2 ě ψipxIi q2, i “ 1, . . . , m,
3 xIi ‰ 0 ñ ψipxIi q ą 0, i “ 1, . . . , m.
For example, the absolute value and the ℓ2 norm functions satisfy the
above conditions.

Sufﬁcient conditions for the equivalence
Theorem 6
Suppose that the Lagrangean dual problem (DL) has a feasible solution,
which is p¯u, ¯vq P Rk ˆ Rℓ, and there exist x˚ P Rn satisfying
pd ´ BT
¯u ´ KT
¯vqT
Ψpx˚
q ´ pAT
¯u ` HT
¯v ´ cqT
x˚
“ 0.
Then, the GAVP dual problem (D) is equivalent to the Lagrangean dual
problem pDLq.
For example, the absolute value function and ℓ2 norm function satisfy
the above equation at x˚ “ 0.
If Ψ and Ψ˚ are a norm function and its dual norm function
respectively, then the GAVP dual problem is equivalent to the
Lagrangean dual problem.

6 Conclusion

Conclusion
We proposed the generalized absolute value optimization problem
(GAVP), and proved the weak duality theorem for GAVP.
We presented some examples of GAVP problems.
The relation between GAVP duality and the Lagrangean duality were
discussed.
We showed sufﬁcient conditions under which the Lagrangean dual
and GAVP dual problems are equivalent.
As future works, we will investigate norm-like functions Ψ (Ψ˚), and
the relation between GAVP and the Lagrangean dualities.

Example : The quadratic function
Let Mi P Sn
``, and λminpMiq ě 1, where λminpMiq is the minimum
eigenvalue of Mi. Then, the quadratic function given by
ψipxIi q “ xT
Ii
MixIi `
1
2
satisﬁes the conditions of the previous proposition.

Example: The quadratic function
Let m “ 1, and denote xI1 , Mi as x, M respectively. The primal and dual
problems can be written as
min cT x ` dpxT Mx ` 1
2 q
s.t. AT x ` BpxT Mx ` 1
2 q “ b,
HT x ` KpxT Mx ` 1
2 q ě p,
max bu ` pv
s.t. pAu ` Hv ´ cqT MpAu ` Hv ´ cq ` 1
2 ` Bu ` Kv ď d,
v ě 0.

Example: The sine function
Suppose that Ii “ tiu. Then, the nonconvex function given by
ψipxIi q “ x2
Ii
` sinpxIi q `
3
2
satisﬁes the conditions of Proposition 4. Notice that we set
θipxIi q “ x2
Ii
` sinpxIi q ` 1,
αi “
1
2
,
respectively.

Example: The sine function
For n “ m “ 1, we denote xIi “ x P R, and we have the primal and dual
problems as
min cx ` dpx2 ` sinpxq ` 3
2 q
s.t. Ax ` Bpx2 ` sinpxq ` 3
2 q “ b,
Hx ` Kpx2 ` sinpxq ` 3
2 q ě p,
max bu ` pv
s.t. pAu ` Hv ´ cq2 ` sinpAu ` Hv ´ cq ` 3
2 ` Bu ` Kv ď d,
v ě 0.

Duality of a Generalized Absolute Value Optimization Problem

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Duality of a Generalized Absolute Value Optimization Problem

Similar to Duality of a Generalized Absolute Value Optimization Problem (20)

Recently uploaded

Recently uploaded (14)

Duality of a Generalized Absolute Value Optimization Problem