YaPingPresentation

Duality Theory for Composite
Geometric Programming
Ya-Ping Wang
Department of Industrial Engineering
University of Pittsburgh
December 14, 2012

Duality Theory for Composite Geometric Programming
Page 1 of 27
Geometric Program (GP)
Primal posynomial program (GP)
(GP) 0inf ( ) . . ( ) 1, : {1,..., }m k
R
G s t G k K p

  
0<t
t t
Note that there are
 m design variables, 1
( , , ) , :{1,..., }mt t J m  t 0 ,
 each Gk(t) is a sum of terms indexed by the set [k]:
[ ]
( ): ( ), 0,1, , ,k ii k
G U k p
  t t
Across the p+1 different functions there are a total of
 n terms 1
, , , {1, , };whereU U I nn  
, ( ) : , 0,ija
i i j i ijj J
i I U C t C a R
    t ,
These terms are sequentially distributed into the (p+1)
problem functions as follows:
0 1[0] [1] [ ]; , 0< |[ ]|p kI p n n n n n k       
The “exponent matrix” is given by ,
, 1:[ ]n m
ij i ja A

Page 2 of 27
Equivalent Formulations of (GP)
In the variables = ln , ,j jz t j J
 ln ( ) ln ln , , lni i ij j i i ij J
U C a t c i I c C
       i
t a z
  [ ]
ˆln ( ) ln exp[ ] : ( ), .i
k i ki k
G c g k K
   t a z z
A convex formulation of GP:
(GP)z: 0inf ( ) . . ( ) 0,m k
R
g s t g k K

  
z
z z

Page 3 of 27
Equivalent Formulations of (GP) (cont’d)
In the variables =:ix i Ii
a z,
 [ ]
( ) ln ( ) ln exp( ) : ( )k k
k k i ii k
g G x c geo
    z t x c
where [ ]
( ) : ln[ exp ] : knk
ii k
geo x R R
 x is called a
geometric function (also called logexp(x)), [ ][ ]k
i i kx x
and [ ][ ]k
i i kc c .
A GGP (Generalized Geometric Programming)
formulation of GP: (GP)x
0 0
inf ( ) . . ( ) 0, ,k k
geo s t geo k K     x x c x c x P
where  | m n
R R  Az zP is the column space of A.

Page 4 of 27
Dual Posynomial Program (GD)
 ˆ [ ]
0
1
[ ]
sup ( ) : (Dual function)
. . 0, , 1 (normality condition)
0, (orthogonality conditions)
where : ,
i
n
i k ik K i k
R
i
n
ij i
i
k i
i k
V C
s t i I
a j J

 
 

 
 




   
  
 
 




ˆ.k K







 

Each dual variable i corresponds to a primal term iU .
Degree of difficulty is defined as ( 1)d n m  
The log-dual function
 
0 [ ]
1 1
( ) :ln ( ) ln /
( ln ) ln
p
i i k i
k i k
pn
i i i k k
i k
v V C
c
  
   
 
 
    
  

 
 
is concave on its domain of definition, and differentiable
in the interior of its domain.

Page 5 of 27
Main Lemma of GP
Lemma: If t is feasible for primal program (GP) and  is
feasible for dual program (GD), then
0 ( ) ( )G Vt  .
Moreover, under the same conditions 0 ( ) ( )G Vt  if, and
only if, one of the following two sets of equivalent
extremality conditions holds:
I.
( ) 1, , (1)
ˆ( ) ( ), [ ], (2)
k
k
i k k i
G k K
G U i k k K

 
   

    
t
t t
II.
0( ) ( ), [0] (1)
( ), [ ], (2)
i
i
k i
U G i
U i k k K



 
  
t t
t
in which case t is optimal for primal program (GP) and 
is optimal for dual program (GD).

Page 6 of 27
Dual to Primal Conversion at Optimality
When 0 ( ) ( )G Vt  ,
we get from condition II on p.5 the equations
( ), [0]
( )
, [ ], , and 0
i
i
i k k
V i
U
i k k K

  
 
 
  
t

,
which are log-linear:
 1
ln( ( )), [0]
ln , [ ], for which 0
m
i
ij j i
j i k k
V i
a z c
i k k K

  

  
  


So, knowing the optimal dual solution  an optimal
primal solution can in general be easily recovered by
solving this log-linear system for z and hence for t.

Page 7 of 27
EXAMPLE:
(Maximum Likelihood Estimator of a Bernoulli Parameter)
Suppose that n independent trials, each of which is a
success with probability p, are performed. What is the
maximum likelihood estimator (MLE) of p?
The likelihood function to be maximized is
1, 1
( , | ) (1 ) , with , 0 or 1,n n
ns n s
n n i if x x p p p s x x i
    
Example: n=3, m=2: 1
( , ) 0
inf s.t. 1n ns s n
p q
f p q p q 

  
The exponent matrix and a unique solution from
extremality condition II (2) on p.5:
* *
1
1 *
*2
2 *
1
3
=
1
ˆ1 0 , so
0 1
n n
n n
n
n
p q n
s s n
s S
s p p X
n n
n s
 





  
      
   

Page 8 of 27
Prior Extensions to GP
 Signomial GP which allow some <0iC  non-
convex programs: duality results are not strong
 Peterson’s Generalized GP (GGP)More general
separable convex programs
Extensions to GP investigated in this thesis
 Composite GP (CGP): itself a special case of
Peterson’s GGP. It includes as special cases:
 Exponential GP (EGP)
 Quadratic GP (QGP)
 (lpGP)
We start with Exponential GP (EGP) using a motivating
example…

Page 9 of 27
A Motivational Example
(Maximum Likelihood Estimator of a Poisson Parameter)
Suppose 1, , nX X are independent Poisson random
variables each having mean λ. Determine the MLE of λ.
The likelihood function to be maximized is
1, 1 1
( , | ) / ( ! !), where :n
ns n
n n n ii
f x x e x x s x
  

    
Equivalently, one can 0min ns n
e 
 
  . Although this is
not a posynomial, it can be solved as an EGP:
1
1 1
1
From condition (e) on p.14, we get
1 ˆ/ = , so /
1
n
n n
n
y s
y s n S n X
s

  

 
   
 
This same method also works for Exponential and Normal
parameters…

Page 10 of 27
Exponential Geometric Program (EGP)
In the previous example:
 a posynomial term is multiplied by an exponential
factor of another posynomial term.
This gives rise to an EGP problem where some
posynomial term ( )iU t is multiplied by an exponential
factor of another posynomial ( ): ( )i ll i
E V 
 t t .
Primal EGP problem (EGP): (cf (GP) on p.1)
0inf ( ) . . ( ) 1, ,m kR
G s t G k K
 0
 
<t
t t
where
[ ]
1
ˆ( ) : ( ) exp ( ) , : {0}
( ) , ,and ( ) ,
{1, , } 1 ,| | 0,
ij lj
k i li k l i
a b
i i j l l jj J j J
i n
G U V k K K
U C t i I V D t l L
L r n i r r r r
  
 
      
     
            
 
 
t t t
t t

 
[ ]:ljB b r m 

Page 11 of 27
Equivalent formulations of (EGP)
In the variables
A convex form of (EGP): (EGP)z
0inf ( ) . . ( ) 0,m k
R
g s t g k K

   
z
z z , where
  [ ]
ˆ( ) :ln ( ) ln exp exp ,i
k k i li k l i
g G c d k K  
        l
z t a z b z +
In the variables : , : ,i lx i I l L   i l
a z, b z,
[ ] [ ]
( ) ( , ) with exp,k k k k
k lg geo h l L      z x c ξ d
where
 [ ]
[ ]
( , ): ln exp exp( )k k
i li k l i
geo x   
    x ξ
:
k
kn r
R R R  is called an exponential geometric
function, [ ]
[ ] [ ]
[ ] , [ ] , :irk i i k
i k l l i ii k
R r r   
    ξ ξ ξ ,
and [ ]k
d is similarly defined.
= ln , ,j jz t j J

Page 12 of 27
Equivalent formulations of (EGP) (cont’d)
A GGP formulation of (EGP):
(EGP)x,ξ
0 0 [0] [0]
( )
[ ] [ ]
inf ( , )
. .
( , ) 0, ,( )
n r
R R
k k k k
geo
s t
geo k K
 
  


      
x,
x c ξ d
x c ξ d x,ξ



P
where  | m n r
M R R 
  z zP is the column space of the
composite exponent matrix
A
B
M
 
  
 
.

Page 13 of 27
Dual of EGP program (EGD) (cf (GD) on p.4)
ˆ( ) ( , ) [ ]
0
1 1
sup ( , ) :
. . 0, , 0, , 1,
0, (Orthogonality Conditions)
0
i l
n r
y
i k l i
R R i k l ik K i Ii l
i l
n r
ij i lj l
i l
i
C D y
V e
y
s t y i I l L
a y b j J
y



 


    
 
   
     
   
      
  
 
   
 
,
y
y 

[ ]
0, , (*) p.16
ˆwhere : , , : =
l
k i l
i k l L
l i i I
y k K  

 










       

   

 
Note that ( ), ( )i i l ly U V  t t
and degree of difficulty ( 1)d n r m    →p.4
The log-dual function (cf p.4)
   
ˆ [ ]
( , ) ln / ln /i i k i l l i l
i k l ik K i I
v y C y D y   
   
       y 
is usc proper concave in( , )y  , and differentiable in the
interior of its domain.

Page 14 of 27
Main Lemma of EGP
Lemma If t is feasible for primal program (EGP) and
( , )y  is feasible for dual program (EGD), then 0 ( ) ( , )G V y t 
Moreover, under the same conditions, 0 ( ) ( , )G V y t  if
and only if one of the following two sets of equivalent
extremality conditions hold: (cf I & II on p.5)
I' Condition (e) & I
( ) 1, ,
ˆ( ) ( ), [ ],
k
k
i k k i
G k K
y G U i k k K


  

  

 
t
t t
II' Condition (e) & II
0( ) ( ), [0]
( ), [ ],
i
i
k i
U G i
y
U i k k K
 
 
 


t t
t
where (e) ( ), ,l i lyV l i i I 
   t
in which case t is optimal for primal program (EGP) and
( , )y  is optimal for dual program (EGD).
 p.9 Solve the previous motivational Example.

Page 15 of 27
Composite Geometric Program (CGP)
Primal CGP Program (CGP) (cf (EGP) on p.10)
0inf ( ) . . ( ) 1,m kR
G s t G k K
 0
 
<t
t t , where
[ ]
ˆ( ) ( ) exp (ln ( )) ,k i l li k l i
G U h V k K  
       t t t ,
:lh R R is a differentiable and strictly convex function.
Note that:
(QGP): if 21
2( ) ,lh l L   
(lpGP): if ( ) | | / , where 1,lp
l l lh p p l L    

Page 16 of 27
Composite Geometric Program (CGP) (cont’d)
A Convex Form of (CGP): (CGP)z
0inf ( ) . . ( ) 0,m kR
g s t g k K
   z
z z , where
  [ ]
( ): ln ( ) ln exp i l
k k i l li k l i
g G c h d  
        z t a z b z
A GGP Form of (CGP): (CGP)x,
0 [0] [ ]
0
( )
inf ( , ) . . ( , ) 0, , ( )n r
k k
k
R R
f s t f k K
 
   ξ ξ ξ
x,
x x x, P

with   [ ]
( , ) ( , ) : ln expk i l li k l i
f geo x h   
          

Page 17 of 27
Dual program (CGD) (cf (EGD) on p.13)
*
ˆ( ) [ ]
0
1 1
sup ( , ): exp ( / )
. . 0, , 1, (Normality Condition)
0, (Ortho
i
n r
y
i k
l l i l l i
R R l ii kk K i Ii
i
n r
ij i lj l
i l
C
V d y h y
y
s t y i I
a y b j J

 


    
 
   
       
  
   
  
 
 
y,η
y 
[ ]
gonality Conditions)
, , (**)
ˆwhere : , .
l i l
k i
i k
y J l i i I
y k K














     

   


where *
lh , the conjugate of lh , has domain interval lJ .
(**) 0l  in (EGD) on p. 13 and that in any (CGD)
(*) 0 0, ,i ly l i i I 
       
The log-dual function (cf p.13)
  *
ˆ [ ]
( , ) ln / ( / )i i k i l l i l l i
i k l ik K i I
v y C y d y h y  
   
          y 
is usc proper concave in( , )y  , and differentiable in the
interior of its domain.

Page 18 of 27
Main Lemma of CGP
Lemma: If t is feasible for primal program (CGP) and
( , )y  is feasible for dual program (CGD), then
0 ( ) ( , )G V t y  .
Moreover, under the same conditions, 0 ( ) ( , )G V t y  if
and only if one of the following two sets of equivalent
extremality conditions hold: (cf I’ & II’ on p.14)
I'' Condition (c) & I
( ) 1, ,
ˆ( ) ( ), [ ],
k
k
i k k i
G k K
y G U i k k K


  

  

 
t
t t
II'' Condition (c) & II
0( ) ( ), [0]
( ), [ ],
i
i
k i
U G i
y
U i k k K
 
 
 


t t
t
where (c) '
(ln ( )), ,ll i ly h V l i i I 
   t
in which case t is optimal for primal program (CGP) and
( , )y  is optimal for dual program (CGD).
(lpGP): (c) 1
| ln ( ) | sgn(ln ( )), ,lp
l i l ly V V l i i I  
   t t

Page 19 of 27
First Duality Theorem of CGP
NOTE: The statements for EGP, QGP, lpGP cases are
almost the same.
Theorem: Suppose that primal program (CGP) is super-
consistent. Then the following three conditions are
equivalent:
1) t’ is a minimal solution to (CGP).
2) There exists a vector ' p
R for z′ (where z′= ln t′)
such that ( ', ')z  forms a saddle point of ( , )l z  .
where ( , )l z  is the Lagrangian of (CGP)z.

Page 20 of 27
3) There exists a vector ' p
R for z′ (where z′= ln t′)
such that ( ', ')z  satisfies the KKT conditions for
(CGP)z:
in which case the set of all such vectors 'λ is a non-empty
compact convex subset of p
R , and the dual program
(CGD) also has a maximum solution ( ', ')y  such that
0min(CGP) ( ') ( ', ') max(CGD)G V   t y 
and they satisfy the extremality conditions I''& II'' on
p.18.
(Perfect Duality)
' '
' '
0ˆ
( ) 0, ( ) 0, ( ) 0,
( ) ( , ') ( ) , 1
k k k k
z k kk K
a g g k K
b l g where
 
 
    
     0
 
 
z' z'
z' z'

Page 21 of 27
Second Duality Theorem of EGP
From linear programming duality theory:
Of the following two linear systems, exactly one has a
solution (where  :
A
B
M n r m
 
  
 
):
(I) Find z with 0 0
 
  
 
A
B
z
(II) Find >0 with T T
A B
 
 
 
 
y
y 0

We say that program (EGD) is canonical if system (II)
has a solution when M is the program’s composite
exponent matrix.
Theorem: Suppose that primal program (EGP) is
consistent. Then the minimum set of program (EGP)z is
non-empty and bounded if, and only if, dual program
(EGD) is canonical, in which case program (EGP) has a
minimum solution t′.

Page 22 of 27
Second Duality Theorem of lpGP*
* This theorem also applies to its special case QGP.
Again, from linear programming duality theory
Of the following two linear systems exactly one has a
solution (where  :
A
B
M n r m
 
   
 
):
(I) Find z with ,  0 Az 0 Bz 0
(II) Find with , and T T
A B
 
 
 
  
y
y 0 y 0


We say that program (lpGD) is canonical if system (II)
has a solution when M is the program’s composite
exponent matrix.
Theorem 7.3.2 Suppose that primal program (lpGP) is
consistent. Then the minimum set of program (lpGP)z is
non-empty and bounded if and only if its dual program
(lpGD) is canonical.

Page 23 of 27
SUMMARY OF CONTRIBUTIONS
 Extension of traditional GP models to
o to the more general Exponential GP models,
o to the even more general Composite GP models,
which include as important special cases:
 (EGP)
 (lpGP)
 (QGP)
 Showing that all of these are special cases of
Peterson’s GGP models (for which he has given a
Main Lemma but not a First or a Second duality
theorem).

Page 24 of 27
CONTRIBUTIONS (cont’d)
 For each extension:
o multiple direct proofs of the Main Lemma, with a
2nd set of equivalent Extremality Conditions.
 For the First Duality Theorem, proof that
o a superconsistent primal program (CGP) has a
minimal solution 't if, and only if, there exists a
vector ' p
R such that ( ', ')z  forms a saddle
point of the Lagrangian ( , )l z  and if, and only
if, it satisfies the KKT conditions for (CGP)z,
o in which case the set of all such Lagrange
multiplier vectors λ is a non-empty compact
convex subset of p
R , whereas the original
theorem is a “If…then” statement and only
showed the existence ofλ for the (GP) case only.

Page 25 of 27
 For the Second Duality Theorem, proof that
o for the special (and more important) cases of
(QGP), (lpGP) and (EGP),
 the minimum set of a consistent primal
program is non-empty and bounded if and
only if its dual program is canonical
 the original theorem showed the existence of
a primal solution for the (GP) case only,
when its dual program is canonical.
 Sensitivity Analysis: If *
z is optimal for (CGP)z:
* * *
* * *0 0 0
* *
* * * *0 0
( ) ( ) ( )
, , ,
( ) ( )
, , where ln
i l k
i l k
i j l j k k
ij lj
g g g
y
c d b
g g
y z z b B
a b
 

  
   
  
 
  
 
  
 
z z z
z z
 Dual to primal conversion of optimal solutions
when there is no duality gap: 0 ( ) ( , )G V y t  .

Page 26 of 27
Directions for Future Research
1.Development of proofs of a strong version of the
First Duality Theorem of (CGP): If a primal program
(CGP) is superconsistent and has a finite infimum,
then the dual program (CGD) has a maximum
solution ( ', ')y  such that
inf(CGP) max(CGD) ( ', ')V  y 
2.Development of computational algorithms.

Page 27 of 27
QUESTIONS

YaPingPresentation

More Related Content

What's hot

Viewers also liked

YaPingPresentation