Download Presentation

Reformulations, Relaxations and Cutting
Planes for Linear Generalized Disjunctive
Programming
Ignacio E. Grossmann
Nicolas Sawaya, Juan Pablo Ruiz
Department of Chemical Engineering
Center for Advanced Process Decision-making
Carnegie Mellon University
Pittsburgh, PA 15213, U.S.A.
MIP-2008
Columbia University, New York, NY
Carnegie Mellon
1
Discrete/Continuous Optimization Models
Mixed Integer Program (MIP)
- Most common non-linear discrete/continuous optimization model.
- Purely equation-based.
- If all functions in MIP are linear → MILP (nonlinear → MINLP).
Disjunctive Programming (DP)
- Developed by Balas E. (1979, 1985, 1998 (1974))
- Linear programming (LP) with disjunctive constraints.
Generalized Disjunctive Program (GDP)
- Linear: Raman, Grossmann (1994); Nonlinear: Turkay, Grossmann (1996)
- Combination of algebraic equations, disjunctions and logic propositions.
- Natural representation of engineering problems.
Mixed-Logic Linear Programming
Hooker and Osorio (1999)
2
Carnegie Mellon
Goal:
To unify GDP with DP in order to develop MILP
reformulations with improved relaxations for linear GDP
• To unify linear GDP with DP in order to develop
- A hierarchy of LP relaxations for linear GDP
- A family of disjunctive cutting planes for linear GDP
• Brief extension to Nonlinear and Bilinear GDPs
- Approximation of convex hull and cutting plane algorithm
- Tightening bounds of bilinear GDPs through basic steps
3
Carnegie Mellon
Linear Generalized Disjunctive Programming
LGDP Model
Raman R. and Grossmann I.E. (1994)
Min Z 

ck  d T x
Objective function
kK
s.t.
Bx  b
 Y jk

 jk

jk
A x  a 
jJ k 
ck   j k 


 Y jk

jJ k
Common constraints
kK
Logic constraints
x L  x  xU
Y jk True, False j  J k , k  K
Logical OR operator
Carnegie Mellon
Disjunctive constraints
kK
(Y )  True
ck  R1
(LGDP)
Continuous variables
Boolean variables
kK
4
Process Network with fixed charges
GDP model
Min Z  c1  c2  c3  d T x
s.t.
x1  x2  x4
x6  x3  x5
Y21
 Y11  

 x  p x    x  x  0
1 2
2
 3
 3

 c1   1   c1  0 
Y22
 Y12  

 x  p x    x  x  0
2 4
4
 5
 5

 c2   2   c2  0 
Y23
 Y13  

 x  p x    x  x  0
3 6
6
 7
 7

 c3   3   c3  0 
Y11  Y21
Y12  Y22
Y13  Y23
Y11  Y12  Y13
Y13  Y11  Y12
Y21  Y22
0  x  xU
Y11 , Y21 , Y12 , Y22 , Y13 , Y23 True, False
c1 , c2 , c3  R1
LOGMIP-GAMS
5
Carnegie Mellon
LGDP to MILP Reformulations
Big-M Reformulation
Min Z =  jkjk + dTx
kK jJk
s.t.
Big-M parameters
Bx  b
Ajk x  ajk -Mjk (1-λjk)

jk
j  Jk , k K
(BM)
k K
=1
jJk
Hλ  h
x Rn, λjk {0,1}
j  Jk , k K
Note: (Mjk = maxLB  x  UB (Ajk x - ajk), j Jk and k K ).
Relaxation : λjk  {0,1} becomes 0  λjk  1 in (BM)
Carnegie Mellon
6
LGDP to MILP Reformulations
Convex Hull Reformulation
Min Z =
 
jk jk
+ d Tx
kK jJk
s.t.
Lee S. and Grossmann I.E. (2000)
Bx  b
Ajk νjk  ajk λjk
Disaggregated variables
j  Jk , k K
 νjk
x=
k K
(CH)
jJk
0  νjk  λjk Ujk

jk
j  Jk , k K
k K
=1
jJk
Hλ  h
x Rn, νjk Rn+ , λjk {0,1}
j  Jk , k K
Relaxation : λjk  {0,1} becomes 0  λjk  1 in (CH)
Carnegie Mellon
7
Proposition:
The Convex Hull of a set of disjunctions is the smallest convex set that includes that set of
disjunctions. Furthermore, the projected relaxation of (CH) onto the space of (BM) is always
as tight or tighter than that of (BM) (Grossmann & Lee , 2002)
1. Tighter feasible region/lower bound  less nodes  decrease in computational solution time.
2. More variables and constraints  more iterations  increase in computational solution time.
x2
Big-M
Relaxed Feasible Region
Convex Hull
Relaxed Projected Feasible Region
Carnegie Mellon
 Y1 
 g ( x )  0


 Y2 
 h( x )  0


x1
Is Convex Hull best relaxation?
8
Disjunctive Programming
Disjunction: A set of constraints connected to one another
through the logical OR operator 
Conjunction: A set of constraints connected to one another
through the logical AND operator 
Constraint set of a DP can be expressed in two equivalent
extreme forms
- Disjunctive Normal Form (DNF)
. A disjunction whose terms do not contain further disjunctions


F  x  R n :  ( Ai x  ai )
iQ
- Conjunctive Normal Form (CNF)
. A conjunction whose terms do not contain further conjunctions

F  x  Rn : Ax  a,  (d h x  d0h ), j  1,..., t
Carnegie Mellon
hQ j

9
Linear Generalized Disjunctive Programming
LGDP Model
(LGDP)
Raman R. and Grossmann I.E. (1994)
Min Z 
c
k
 dT x
Objective function
kK
s.t.
Bx  b
 Y jk

 jk

jk
A x  a 
jJ k 
ck   j k 


 Y jk

jJ k
Common constraints
kK
kK
Disjunctive constraints
Logic constraints
(Y )  True
x L  x  xU
Y jk True, False j  J k , k  K
ck  R1
kK
Boolean variables
How to deal with Boolean and logic constraints in Disjunctive Programming?
Carnegie Mellon
10
Reformulating LGDP into Disjunctive
Programming Formulation
Sawaya N.W. and Grossmann I.E. (2007)
Min Z 

ck  d T x
Min Z 
kK
s.t.

jJ k
k
 dT x
kK
Bx  b
 Y jk

 jk

jk
A
x

a


jJ k 
ck   j k 


 Y jk
c
Bx  b
s.t.
  jk  1 
 jk

jk
A x  a 
jJ k 
ck   j k 


kK
1
kK
kK

kK

jk
jJ k
(Y )  True
H  h
x L  x  xU
Y jk True, False j  J k , k  K
x L  x  xU
0   jk  1
ck  R1
ck  R1
kK
LGDP
LDP
j  Jk , k  K
k  K 
=> Integrality  guaranteed
11
Proposition. LGDP and LDP have equivalent solutions.
Carnegie Mellon
Equivalent Forms in DP Through Basic Steps
There are many forms between CNF and DNF that are equivalent
Regular Form (RF): form represented by intersection of unions of polyhedra
Thus the RF is:
F   St
tT
where for t  T ,
St   Pi ,
iQt
Pi a polyhedron, i  Qt .
Proposition 1 (Theorem 2.1 in Balas (1979)). Let F be a disjunctive set in RF. Then F
can be brought to DNF by | T | 1 recursive applications of the following basic steps,
which preserve regularity:
For some r , s  T , r  s, bring Sr  Ss to DNF, by replacing it with:
Srs   ( Pi  Pt ).
iQr
tQs
12
Carnegie Mellon
Illustrative Example: Basic Steps
F  S1  S2  S3
S1  ( P11  P21 )
S2  ( P12  P22 )
S3  ( P13  P23 )
Then F can be brought to DNF through 2 basic steps.
Apply Basic Step to:
S1  S2  ( P11  P21 )  ( P12  P22 )
S12  ( P11  P12 )  ( P11  P22 )  ( P21  P12 )  ( P21  P22 )
We can then rewrite
F  S1  S2  S3
as F  S12  S3
Apply Basic Step to:
S12  S3  (( P11  P12 )  ( P11  P22 )  ( P21  P12 )  ( P21  P22 ))  ( P13  P23 )
 ( P11  P12  P13 )  ( P11  P22  P13 )  ( P21  P12  P13 )  ( P21  P22  P13 ) 
S123  

 ( P11  P12  P23 )  ( P11  P22  P23 )  ( P21  P12  P23 )  ( P21  P22  P23 ) 
We can then rewrite
F  S12  S3
Carnegie Mellon
as F  S123
which is its equivalent DNF
13
Equivalent Forms for GDP
Min Z 
c
k
Min Z 
 dT x

jJ k
 dT x
s.t.
Bx  b
kK

  jk  1 
 jk

jk
A x  a 
 c 

jk
 k

kK

Bx  b
Y jk


 jk

jk
A x  a 
jJ k 
c   jk 
 k

 Y jk
k
kK
kK
s.t.
c
jJ k
kK
1
jk
kK
jJ k
(Y )  True
H  h
x xx
Y jk True, False j  J k , k  K
x L  x  xU
0   jk  1
ck  R
ck  R
L
U
1
LGDP
kK
1
j  Jk , k  K
k  K 
LDP
n   | J k | | K |


F   z : ( x,  , c)  R kK
:  b i z  b0i   ( A jk z  a jk ) 
iT
kK jJ k


All possible equivalent forms for GDP, obtained through any number of basic steps,
are represented by:
LDP’
n   | J k | | K |

i
i
jk
jk
mn
mn 
kK
ˆ
ˆ
F   z : ( x,  , c)  R
:  b z  b0   ( A z  a ) ˆ  ( A z  a )  14
iT
kK jJ k
nK mJ n


Carnegie Mellon
Converting LDP to MIP reformulations
Proposition 2 (Theorem 3.3 combined with Corollary 3.5 in Balas (1979)). Let


F   Pi , Pi  x  Rn : Ai x  a0i , i  Q , where Q is an arbitrary set and each
iQ
( Ai , a0i ) is an mi  (n  1) matrix such that every Pi is a bounded non-empty polyhedron.
Furthermore, let  (Q ) be the set of all those x  R n such that there exist vectors
(vi , yi )  R n 1 , i  Q , satisfying
x

i
vi  0
disaggregated variables
iQ
Ai vi  a0i yi  0
i Q
yi  0
i Q
 y 1
i
=> Convex Hull
i Q
iQ
Then cl conv F   (Q).
Proposition 3 (Corollary 3.7 in Balas (1979).
Let  I (Q) : x  (Q) : yi {0,1}, i  Q .
Then  I (Q)  F .
Carnegie Mellon
=> MIP representation
15
Family of MIP Reformulations For GDP
n   | J k | | K |


F   z : ( x,  , c)  R kK
:  b i z  b0i   ( A jk z  a jk ) ˆ  ( Aˆ mn z  aˆ mn ) 
iT
kK jJ k
nK mJ n


LDP’
General template for any MILP reformulation
 
Min Z 
jk
y jk  d T x
kK jJ k
s.t.
bi x  b0i
i  I B1
A jk vˆ mn  a jk yˆ mn
h y  h0
i  I H1
x L yˆ mn  vˆ mn  xU yˆ mn
i  I X1
0  uˆ jk mn  yˆ mn
i
i
xi L  xi  xiU
y jk 
 uˆ
mn
jk
( j , k )  L2n  K S2  I H 2 , n  N
n
mJ n
 vˆ
x
n
b vˆ

 b0 yˆ mn
i
 yˆ
n N
i  I B2 , m  J n , n  N
n
 yˆ mn
k  K S2 , m  J n ,n  N
 h0 yˆ mn
i  I H2 , m  J n , n  N
uˆ jk
mn
n
jJ k
h uˆ
i
mn
mn
1
m  J n ,n  N
( j , k )  L3n , m  J n , n  N
n N
mJ n
mn
mJ n
i mn
 yˆ
( j , k )  M mn , m  J n , n  N
i
uˆ jk mn  yˆ mn
n
mn jk
 y jk
n jk  N , j  J k , k  K
MIP’
mQn jk
y
jk
1
kK
jJ k
yˆ mn  0
m  Jn, n  N
y jk  {0,1}
j  Jk , k  K
( j , k )  M mn , m  J n , n  N
16
Carnegie Mellon
Particular case: Convex Hull Reformulation of LGDP
Lee S. and Grossmann I.E. (2000)
Min Z 

 j k y jk  d T x
(CH)
Disaggregated variables
kK jJ k
Bx  b
s.t.
x

v jk
kK
jJ k
A jk v jk  a jk y jk
j  Jk , k  K
x L y jk  v jk  xU y jk
j  Jk , k  K
y
kK
jk
1
jJ k
Hy  h
y jk  {0,1}
j  Jk , k  K
While this MILP formulation has stronger relaxation than big-M, it is not strongest!!
Carnegie Mellon
17
A Hierarchy of Relaxations for DP
Hull Relaxation (Balas, 1985):
Let us take the following disjunctive set:
F   Sj
jT
Then the hull-relaxation corresponds to:
h  rel F :  clconv S j .
jT
Proposition 3 (Theorem 4.3 in Balas (1979)): For i  0,1,
, t , let Fi   S j be a
jTi
sequence of regular forms of a disjunctive set, such that
i) F0 is in CNF, with P0   S j ;
jT0
ii) Ft is in DNF;
iii) for i  1,
, t , Fi is obtained from Fi 1 by a basic step.
Then,
P0  h  rel F0  h  rel F1 
Carnegie Mellon
 h  rel Ft  clconv Ft . (true convex hull)
18
A Hierarchy of Relaxations for GDP
Proposition 4. For i  T |  | K | 1 let FGDPi be a sequence of regular forms of the disjunctive set:
n   | J k | | K |


F   z : ( x,  , c)  R kK
:  b i z  b0i   ( A jk z  a jk ) ˆ  ( Aˆ mn z  aˆ mn )  , such that
iT
kK jJ k
nK mJ n


i) FGDP0 corresponds to the disjunctive form:
n   | J k | | K |


F   z : ( x,  , c)  R kK
:  b i z  b0i   ( A jk z  a jk )  ;
iT
kK jJ k


ii) FGDP|T ||K|1 : Ft is in DNF;
iii) for i  1,
, t , FGDPi is obtained from FGDPi1 by a basic step.
Then,
h  rel FGDP0  h  rel FGDP1 
 h  rel FGDP|T ||K|1  clconv FGDP|T ||K|1  clconv Ft . (true convex hull)
19
Carnegie Mellon
Illustrative Example:
Hierarchy of Relaxations
x1  x2  0.5  0
Convex Hull of disjunction
 x1  x2  1  0
 x1  0   x1  1 
0  x  1  0  x  1
2
2

 

x2
LP Relaxation
Application of
2 Basic Steps
 x1  x2  0.5  0  x1  x2  0.5  0
 x  x  1  0   x  x  1  0 
 1 2
 1 2

 x1  0
  x1  1


 

0

x

1
0

x

1
2
2

 

x1
Convex Hull of disjunction
Tighter
Relaxation!
20
Carnegie Mellon
Numerical Example:
Strip-packing problem
Problem statement: Hifi (1998)
- Given a set of small rectangles with width Hi and length Li.
- Large rectangular strip of fixed width W and unknown length L.
- Objective is to fit small rectangles onto strip without overlap and
rotation while minimizing length L of the strip.
y
W
(0,0)
j
(xi,yi)
ij
j
ji
L=?
x
Set of small rectangles
21
Carnegie Mellon
GDP/DP Model for
Strip-packing problem
Min
lt
s.t.
lt  xi  Li
i  N
 Y 1ij   Y 2ij   Y 3ij   Y 4ij





 
 
 
 i, j  N , i  j
 xi  Li  x j   x j  L j  xi   yi  H i  y j   y j  H j  yi 
xi  UBi  Li
i  N
H i  yi  W
i  N
Objective function
Minimize length
Disjunctive constraints
No overlap between rectangles
Bounds on variables
lt , xi , yi  R1 , Y 1ij , Y 2ij , Y 3ij , Y 4ij True, Falsei, j  N , i  j
Min
lt
s.t.
lt  xi  Li
i  N
 1ij  1    2ij  1    3ij  1    4ij  1 

  
  
  
 i, j  N , i  j
 xi  Li  x j   x j  L j  xi   yi  H i  y j   y j  H j  yi 
1ij   2ij   3ij   4ij 1 i, j  N , i  j
xi  UBi  Li
i  N
H i  yi  W
i  N
lt , xi , yi  R1 , 0  1ij ,  2ij ,  3ij ,  4ij 1i, j  N , i  j
Carnegie Mellon
22
DP Model For
4 Rectangle Strip-packing Problem
Min
s.t.
lt
lt  xi  Li
 112  1 

 
 x1  L1  x2 
 113  1 

 
 x1  L1  x3 
 114  1 

 
 x1  L1  x4 
 123  1 

 
 x2  L2  x3 
 124  1 

 
 x2  L2  x4 
 134  1 

 
 x3  L3  x4 
H1  h1
i  N
  212  1    312  1    412  1 

  
  

 x2  L2  x1   y1  H1  y2   y2  H 2  y1 
  213  1    313  1 
  413  1 

  
  

 x3  L3  x1   y1  H1  y3 
 y3  H 3  y1 
  214  1    314  1    414  1 

  
  

 x4  L4  x1   y1  H1  y4   y4  H 4  y1 
  2 23  1    323  1    4 23  1 

  
  

 x3  L3  x2   y2  H 2  y3   y3  H 3  y2 
  2 24  1    324  1    4 24  1 

  
  

 x4  L4  x2   y2  H 2  y4   y4  H 4  y2 
  234  1    334  1    434  1 

  
  

 x4  L4  x3   y3  H 3  y4   y4  H 4  y3 
xi  UBi  Li
i  N
H i  yi  W
i  N
lt , xi , yi  R , 0   ,  ij ,  ij ,  ij i, j  N , i  j
1

1
ij
2
3
4
Original CH formulation
102 variables (24 0-1), 143 constraints
LP relaxation = 4, No.nodes=45
Optimum min L=8
Carnegie Mellon
Min
lt
s.t.
lt  xi  Li
 112  1 

 1
  13  1 
 1  1 
 
 23
 x1  L1  x2 


 x1  L1  x3 
x  L  x 
 2 2 3
i  N
  412  1 

 4
  13  1 
 4 1 
23

 
 y2  H 2  y1 


 y3  H 3  y1 
y  H  y 
 3 3 2
 114  1    214  1    314  1    414  1 

  
  
  

 x1  L1  x4   x4  L4  x1   y1  H1  y4   y4  H 4  y1 
 124  1    2 24  1    324  1    4 24  1 

  
  
  

 x2  L2  x4   x4  L4  x2   y2  H 2  y4   y4  H 4  y2 
 134  1    234  1    334  1    434  1 

  
  
  

 x3  L3  x4   x4  L4  x3   y3  H 3  y4   y4  H 4  y3 
H 2  h2
xi  UBi  Li
i  N
H i  yi  W
i  N
lt , xi , yi  R1 , 0  1ij ,  2ij ,  3ij ,  4ij i, j  N , i  j
Strengthened formulation
170 variables (60 0-1), 347 constraints
LP relaxation = 8, No.nodes=0
23
Optimum min L = 8
25 Rectangle Problem Optimal solution= 31
Original CH
Strengthened
1,112 0-1 variables
4,940 cont vars
7,526 constraints
1,112 0-1 variables
5,783 cont vars
8,232 constraints
=>
LP relaxation = 9
LP relaxation = 27!
31 Rectangle Problem Optimal solution= 38
Original CH
Strengthened
2,256 0-1 variables
9,716 cont vars
14,911 constraints
2,256 0-1 variables
11,452 cont vars
15,624 constraints
LP relaxation = 10.64
=>
LP relaxation = 33!
24
Carnegie Mellon
Motivation for Cutting Plane Method
1. Tighter feasible region/lower bound  less nodes  decrease in computational solution time.
2. More variables and constraints  more iterations  increase in computational solution time.
Feasible region as tight as full space AND fewer variables and constraints
25
Carnegie Mellon
DERIVATION OF CUTTING PLANES:
Dual perspective
Proposition
5.
The
inequality
 z  0
is
a
consequence
of
n   | J k | | K |


F   z : ( x,  , c)  R kK
:  b i z  b0i   ( A jk z  a jk ) ˆ  ( Aˆ mn z  aˆ mn )  ,
iT
kK jJ k
nK mJ n


if
and
only
if
there
exists
a
set
of
 jk  0, j  J k , k  K ; ˆmn  0, m  J n , n  Kˆ ; and i , i T , satisfying
  b   A  ˆ
   b   a  ˆ

i
i
jk
iT
0
i
mn
jk
jk
0
kK
Aˆ mn , j  J k , m  J n
nKˆ
kK
i
iT
Carnegie Mellon
jk
nKˆ
mn
aˆ mn , j  J k , m  J n
Reverse Polar Cone
26
DERIVATION OF CUTTING PLANES:
Dual perspective
Proposition 7.  z   0 with  0  0 is a facet of the h  rel FGDPi , i  0,1,...,| T |  | K | 1
if and only if   0 is a vertex of the polyhedron





i b i 
 jk A jk  ˆ mn Aˆ mn , j  J k , m  J n 

iT
kK


nKˆ


i
i
jk
jk
mn
mn
n   | J k | | K |  
 b0 
 a  ˆ aˆ , j  J k , m  J n 

0
kK
  R
.
iT
kK
nKˆ


 jk  0, j  J k , k  K




mn
ˆ  0, m  J n , n  Kˆ


i



,
i

T




Carnegie Mellon




Reverse Polar Cone
27
CUT GENERATION PROBLEM:
Dual perspective
Max   z LP   0
s.t.
     
n
Normalization set
Balas, Ceria, Cornuejols
(1993)
 | J k | | K |

    b   A   ˆ
    b    a   ˆ
kK
( i    i  )  1
i 1
i
i
jk
iT
jk
0
i
jk
jk
0
iT
Aˆ mn , j  J k , m  J n
nKˆ
kK
i
mn
kK
mn
aˆ , j  J k , m  J n
Reverse Polar Cone
mn
nKˆ
 jk  0, j  J k , k  K
ˆ mn  0, m  J n , n  Kˆ
i , i  T
  ,   0
28
Carnegie Mellon
CUT GENERATION PROBLEM:
Primal perspective

Min
  z  z LP
Infinity Norm
  z LP  z
b i z  b0i
z
i T

jk
 ˆ
mn
0
kK
0
n  Kˆ
jJ k
z
Hull-relaxation
mJ n
A jk jk  a jk y jk  0
j  Jk , k  K
Aˆ mnˆ mn  aˆ mn yˆ mn  0
m  J n , n  Kˆ
y
jk
1
kK
1
n  Kˆ
jJ k
 yˆ
mn
mJ n
Carnegie Mellon
y jk ,  jk  0
j  Jk , k  K
yˆ mn , ˆ mn  0
m  J n , n  Kˆ
29
Cut Generation Problem for Lee & Grossmann
(Separation Problem)
Primal perspective separation problem Sawaya, Grossmann (2006)
Min  ( z )  || z  z bm ||
(SEP)
s.t.
Bx  b
Ajk jk  a jk y jk
x

j  J k , k  K
k  K
jk
jJ k
 jk  y jkU jk
j  J k , k  K
y
k  K
jk
1
Hull Relaxation
for Lee &
Grossmann
jJ k
Dy  d
0  y jk  j  J k , k  K
 |Jk |
x,  R  , z  [ x, y ]  R   R kK
n
n

Note that  ( z ) || z  z bm ||1 or  ( z ) || z  z bm ||2 or  ( z ) || z  z bm || can be used.
Carnegie Mellon
30
Derivation of Cutting Planes:
Primal perspective
Proposition 10: Let (FR-SEP) be the feasible region of the separation problem (SEP), and
let (FRP-SEP) represent the projection of (FR-SEP) onto the z-space. Then,(FRP-SEP)
 (FR-BM), where (FR-BM) represents the feasible region of (BM). Furthermore, (FRPSEP) is a convex set.
Proposition 11: Let z bm be the optimal solution of (BM) and z sep be an optimal solution to
(SEP). If z bm  (FRP-SEP), then   such that  T ( z  z sep )  0 is a valid linear inequality
in z that cuts away z bm , and such that  is a subgradient of  ( z ) at z sep .
31
Carnegie Mellon
Cutting Plane Method
Derivation of Cutting Planes
Propositions 12, 13, 14:
(1) Let  (z)  ║z - zbm║2  (z – zbm)T(z – zbm). Then,
    = (z – zbm)
(2) Let  (z)  ║z - zbm║inf  maxizi - zibm. Then,
  (+- -)
Min u
s.t
u  zi - zibm i  I
u  zibm - zi i  I
Feasible region of (SEP)
(3) Let  (z)  ║z - zbm║1  zi - zibm. Then,
  (+- -)
Min u
s.t
ui  zi - zibm i  I
ui  zibm - zi i  I
Feasible region of (SEP)
Lagrange Multipliers
+
-
Lagrange Multipliers
+
-
32
Carnegie Mellon
CUTTING PLANE METHOD
1. Solve relaxed Big-M MILP. This yields zBM.
2. Solve separation problem.
Feasible region corresponds to relaxed hull relaxation.
Objective corresponds to finding point in hull relaxation closest to zBM.
3. Cutting plane is generated and added to relaxed big-M MILP.
4. Solve strengthened relaxed Big-M MILP. Go to 2.
z1
This yields zSEP.
zBM
z
Strengthened Big-M
Relaxed Feasible Region
Big-M
Relaxed Feasible Region
Hull relaxation
Projected Feasible Region
Carnegie Mellon
zSEP
Cutting Plane
z2
33
NUMERICAL RESULTS
21-RECTANGLE STRIP PACKING PROBLEM
Table 3
Results for twenty one-rectangle strip-packing problem (CPLEX v. 8.1, default MIP options turned on)
Relaxation
Optimal
Solution
Gap
(%)
Total Nodes in
MIP
Solution Time for
Cut Generation
(sec)
*Total
Solution
Time (sec)
Number of
Nodes per
sec
Convex Hull
9.1786
---
---
968 652
0
>10 800
89.69
Big-M
9
24
62.5
1 416 137
0
4 093.39
345.95
Big-M + 20 cuts
9.1786
24
61.75
306 029
3.74
917.79
334.80
Big-M + 40 cuts
9.1786
24
61.75
547 828
7.48
1 063.51
518.76
Big-M + 60 cuts
9.1786
24
61.75
28 611
11.22
79.44
419.32
Big-M + 62 cuts
9.1786
24
61.75
32 185
11.59
91.4
403.27
* Total solution time includes times for relaxed MIP(s) + LP(s) from separation problem + MIP
34
Carnegie Mellon
NUMERICAL RESULTS
21-RECTANGLE STRIP PACKING PROBLEM
Table 3
Results for twenty one-rectangle strip-packing problem (CPLEX v. 8.1, default MIP options turned on)
Relaxation
Optimal
Solution
Gap
(%)
Total Nodes in
MIP
Solution Time for
Cut Generation
(sec)
*Total
Solution
Time (sec)
Number of
Nodes per
sec
Convex Hull
9.1786
---
---
968 652
0
>10 800
89.69
Big-M
9
24
62.5
1 416 137
0
4 093.39
345.95
Big-M + 20 cuts
9.1786
24
61.75
306 029
3.74
917.79
334.80
Big-M + 40 cuts
9.1786
24
61.75
547 828
7.48
1 063.51
518.76
Big-M + 60 cuts
9.1786
24
61.75
28 611
11.22
79.44
419.32
Big-M + 62 cuts
9.1786
24
61.75
32 185
11.59
91.4
403.27
* Total solution time includes times for relaxed MIP(s) + LP(s) from separation problem + MIP
35
Carnegie Mellon
Non-linear Discrete/Continuous Optimization:
GDP Model
Lee & Grossmann I.E. (2000)
Min Z 
c
k
 f ( x)
Objective function
kK
s.t.
r ( x)  0
Common constraints
 Y jk



 g jk ( x)  0 
jJ k 

c


k
j
k



kK
 Y jk
kK
jJ k
(GDP)
Disjunctive constraints
Logic constraints
(Y )  True
x L  x  xU
Y jk True, False j  J k , k  K
ck  R1
Logical OR operator
Carnegie Mellon
f ( x), r ( x), g jk ( x)
convex functions
kK
36
Boolean variables
Cutting Plane Method
1. Solve relaxed (BM) problem. This yields zbm ≡ [x,y]bm.
2. Solve separation problem.
Feasible region corresponds to relaxed (CH) problem.
Objective corresponds to finding point in relaxed projected (CH) problem closest to zbm.
This yields zsep.
3. Cutting plane is generated and added to relaxed (BM) problem.
4. Solve strengthened relaxed (BM) problem. Go to 2.
(CH)
Relaxed Projected
Feasible Region
(BM)
Strengthened Relaxed
Feasible Region
(BM)
Relaxed
Feasible Region
Carnegie Mellon
zsep
Cutting Plane
zbm
37
Convex Hull Formulation
• Consider Disjunction k  K

jJ
k


Y jk


 g jk ( x)  0 


 c   jk 
 Theorem: Convex Hull of Disjunction k (Lee, Grossmann, 2000)
 Disaggregated variables  j
c    ,
{( x , c ) | x   v ,
jk
jJk
jJ
jk
jk
0 v   U , jJ
jk
jk
   1,
jJk
k
jk
=> Convex Constraints
0    1,
jk
jk
 g (v /  )  0 , j  J }
jk
jk
jk
jk
k
 j - weights for linear combination
- Generalization of Stubbs and Mehrotra (1999)
38
Remarks
1. h(v ,  )   g (v /  )
If g(x) is a bounded convex function,
h(v ,  ) is a bounded convex function
Hiriart-Urruty and Lemaréchal (1993)
2. h( ,0)  0 for bounded g(x)
3. For linear constraints convex hull reduces to result by Balas (1985)
39
Cutting Plane Method: Separation Problem
Min  ( z )  || z  z bm ||
(SEP)
s.t.
r ( x)  0
y jk g jk ( jk / y jk )  0
How do we
Implement this?
x

j  J k , k  K
k  K
jk
jJ k
 jk  y jkU jk
j  J k , k  K
y
k  K
jk
CONVEX NLP
1
(CH) Relaxed
Feasible Region
jJ k
Dy  d
0  y jk  j  J k , k  K
x,  R  , z  [ x, y ]  R   R
n
n
 |Jk |
kK

For  ( z ) || z  z bm ||2 , we get ( z sep  z bm )( z  z sep )  0
Different Norms
Different Cuts
For  ( z ) || z  z bm ||1 , we get (      )( z  z sep )  0
For  ( z ) || z  z bm || , we get (      )( z  z sep )  0
Carnegie Mellon
40
Computational Implementation
of Separation Problem
Furman, Sawaya & Grossmann (2007)
Replace y jk g jk ( jk / y jk )  0 by:
where
0  jk U y jk
((1   ) y jk   )( g jk ( jk /((1   ) y jk   )))   g jk (0)(1  y jk )  0
1. The divisibility by 0 problem is avoided.
2. The new constraints are an exact approximation of the original constraints as ε → 0.
3. The new constraints are an exact approximation of the original constraints at yjk = 0
and at yjk = 1 regardless of value of ε.
if y jk  0,  ( )( g jk (0))   g jk (0)  0  0
if y jk  1,  ((1)( g jk ( jk /(1))   g jk (0)(0) (1) g jk ( jk /(1)  0
4. The LHS of the new constraints are convex.
Carnegie Mellon
41
Numerical Example
Design of Multi-product Batch Plant
Problem statement: Ravemark (1995)
– Design of batch plant with multiple units in parallel and intermediate
storage tanks.
– Problem consists of determining volume of equipment, number of units
in parallel and volume and location of intermediate storage tanks while
minimizing investment cost.
How many units in parallel per stage?
Stage 1
Stage 1
A
Stage 2
B
C
Stage 3
How large should
my tanks be?
Carnegie Mellon
D
Stage 1
E
How many intermediate
storage tanks?
How big?
42
Numerical Results
Design of 10 Unit/Product Batch Plant
Problem Size
# of constraints
# of variables
# of discrete variables
Convex Hull
1296
637
89
Big-M
800
239
89
Table 1: Results for design of 10 stage/product batch plant using traditional B&B (SBB)
Relaxation
Optimal
Solution
Gap
(%)
Total Nodes in
MINLP
Solution Time for
Cut Generation
(sec)
*Total
Solution
Time (sec)
Number of
Nodes per
sec
improvement
Convex Hull
650 401. 14
729 948.49
10.9
5 359
0
711.76
7.53
Big-M
641 763.19
729 948.49
12.1
12 449
0
787.98
15.80
610.00
12.52
40%
Big-M + 58 cuts 650 401. 14
729 948.49
10.9
7 528
23%
8.7
43
Carnegie Mellon
Global Optimization of Bilinear
Generalized Disjunctive Programs
Juan Ruiz
Min
Z  f ( x )   ck
Objective Function
kK
s.t. g ( x)  0

iDk
Bilinearities
 Yik 
 r ( x )  0
 ik

 ck   ik 
Ω(Y)= True
x  R n ,ck  R,Yik  {True,Fals e}
Global Constraints
Disjunctions
k K
Logic Propositions
i  Dk
,
kK
Bilinearities may lead to multiple local minima → Global Optimization techniques
are required
Relaxation of Bilinear terms using McCormick envelopes leads to a LGDP → Improved
relaxations for Linear GDP has recently been obtained (Sawaya & Grossmann, 2007)
44
Carnegie Mellon
Guidelines for applying basic steps in Bilinear GDP
• Replace bilinear terms in GDP by McCormick convex envelopes (LGDP)
• Apply basic steps between those disjunctions with at least one variable in
common.
• The more variables in common two disjunctions have the more the
tightening can be expected
• If bilinearities are outside the disjunctions apply basic steps by introducing
them in the disjunctions previous to the relaxation.
• If bilinearities are inside the disjunctions a smaller tightening effect is
expected.
• A smaller increase in the size of the formulation is expected when basic
steps are applied between improper disjunctions and proper disjunctions.
45
Carnegie Mellon
Case Study I: Water treatment network design
Process superstructure
Generalized Disjunctive Program
 CP
Min Z =
M1
S1
S4
A/B/C
s.t.
fk 
f
j
S2
M2
S5
D/E/F
M4
f
i

i
iS k
M3
S3
S6
G/H/I
iS k
iM k
 fk
k
1
A
k
S4
M2
D
S5
Carnegie Mellon
Z* = 1.214
k  SU
j
j
i  Sk
j, k
M4
0  fi , f k
i, j , k
j
0  CPk
S3
j
0  i 1
k
S2
k  MU
k  SU
h


YPk
 j

jh
j
 f i   k f i ' , i  OPU k , i ' IPU k , j 

j
 k  PU
hDk 
Fk   f i , i  OPU k


j


CPk   ik Fk


Optimal structure
M1
j
j
k  SU
fi   i f k
N of cont. vars. : 114
N of disc. vars. : 9
N of bilinear terms: 36
j
i
j
j
S1
k
k PU
j
k
h
YPk {true, false} h  Dk
k  PU
46
Case Study II: Pooling network design
Process superstructure
Stream i
Pool j
Product k
Generalized Disjunctive Program
Min Z =
 CP   CST   c  f   d   f
j J
j
i I
i
j J iI
ij
wW
ijw
k K
k
j J wW
jkw
s.t.
S1
P1
1
S2
P2
2
S3
P3
S4
P4
S1
P1
Product k
1
S2
2
P3
3
S5
Carnegie Mellon
Z* = -4.640
 f
jkw
 Sk  0
jJ wW
k K wW
f
jkw
j  J
k  K
fijw  iw  fijw'
i  I , j  J , w  W
 Z kw   f jkw'  0
k  K , w  W
j J w 'W


YSTi


 f lo    f ijw  


jJ wW
 CSTi   i 
Optimal structure
Pool j
   f jkw
j J
N of cont. vars. : 76
N of disc. vars. : 9
N of bilinear terms: 24
Stream i
ijw
iI wW
w'W
3
S5
 f
 YSTi 
 f 0 
 ijw

CSTi  0
i  I
YPj




lo
f    f ijw
YPj

 

iI wW

  f  0, i  I , w  W 
 j  J
  f jkw   f ijw , w  W
  ijw
i I
 k K k
   f jkw  0, k  K , w  W 

 f jkw   j  f ijw , w  W , k  K  
CPj  0
iI




k


1




 j

k K




CPj   j


0   j  1;0  f jkw , fijw  f up
k
0  CSTi , CPj ;YSTi , YPj  {true, false}
47
Performance
Example 1
Initial Lower Bound
Global Optimization
Technique using Lee &
Grossmann relaxation
Global Optimization
Technique using
proposed relaxation
Relative
Improvement
400.66
499.86
24.90%
Bound contraction
Nodes
Example 2
Initial Lower Bound
99.7%
399
204
Global Optimization
Technique using Lee &
Grossmann relaxation
Global Optimization
Technique using
proposed relaxation
Relative
Improvement
-5515
-5468
0.90%
Bound contraction
Nodes
51%
8%
748
683
9%
48
Carnegie Mellon
Conclusions
Unified GDP with Disjunctive Programming
- Developed DP equivalent formulation for GDP
- Developed a family of MIP reformulations for GDP
- Developed a hierarchy of relaxations for GDP
Developed framework for obtaining improved LP relaxations
- Demonstrated improved relaxations can be obtained compared
to convex hull formulation Lee & Grossmann (2000)
- Numerical results have shown great improvement in lower bound
for strip packing problem
Cutting Planes
- Showed equivalence dual and primal cut-generation problems.
- Developed a primal cut-and-branch algorithm where cutting planes
were generated from the primal separation problem
Nonlinear GDPs
- Cutting planes can be readily extended
- Concept basic steps improves relaxation in bilinear GDPs
Carnegie Mellon
49
50
Carnegie Mellon
CONCLUSIONS AND CONTRIBUTIONS
1-) We established novel connections between disjunctive programming and linear GDP.
2-) We extended Balas’ theory of equivalent forms to linear GDP.
3-) We developed a family of MIP reformulations that encompasses all possible MIP formulations
for linear GDP, including the Lee and Grossmann formulation.
4-) We developed a hierarchy of relaxations for linear GDP that mirror those developed by
Balas for disjunctive programs
5-) We described the facets of the hull-relaxation in the dual space, and showed that every facet
of the hull-relaxation can be obtained from the convex hull of some individual disjunction.
6-) We showed that the equivalence between the dual and primal cut-generation problems.
7-) We developed a primal cut-and-branch algorithm where cutting planes were generated from
the primal separation problem and were added to the root node of a B&B tree.
8-) We proceeded to derive different families of cutting planes in the primal space (from different
norms) that obtain from the aforementioned cut-generation problem,
9-) We established rigorous approximations for reformulations of nonlinear GDP problems
10-) We extended the primal cut-and-branch algorithm developed for the linear case to the
nonlinear case.
Carnegie Mellon
51
52
Carnegie Mellon
NUMERICAL RESULTS
21-RECTANGLE STRIP PACKING PROBLEM
Table 3
Results for twenty one-rectangle strip-packing problem (CPLEX v. 8.1, default MIP options turned on)
Relaxation
Optimal
Solution
Gap
(%)
Total Nodes in
MIP
Solution Time for
Cut Generation
(sec)
*Total
Solution
Time (sec)
Number of
Nodes per
sec
Convex Hull
9.1786
---
---
968 652
0
>10 800
89.69
Big-M
9
24
62.5
1 416 137
0
4 093.39
345.95
Big-M + 20 cuts
Big-M + 40 cuts
Big-M + 60 cuts
Big-M + 62 cuts
9.1786
9.1786
9.1786
9.1786
24
24
24
24
61.75
61.75
61.75
61.75
306 029
547 828
28 611
32 185
3.74
7.48
11.22
11.59
917.79
1 063.51
79.44
91.4
334.80
518.76
419.32
403.27
* Total solution time includes times for relaxed MIP(s) + LP(s) from separation problem + MIP
53
Carnegie Mellon
Convex Hull
The Convex Hull of a set of disjunctions is the smallest convex set that
includes that set of disjunctions.
 Y1 
 g ( x )  0


x1
z1
xA
λ2 z2 =ν2
λ1 z1=ν1
Convex Hull
Relaxed Projected Feasible Region
Carnegie Mellon
z2
 Y2 
 h( x )  0


x2
54
DERIVATION OF CUTTING PLANES:
Dual perspective
Proposition 8. Every facet of h  rel FGDPi , i  0,1,...,| T |  | K | 1 as described in (84) is
n   | J k | | K |


a facet either of Fk   z : ( x,  , c)  R kK
:  b i z  b0i  clconv  ( A jk z  a jk ) 
jJ k
iT


for some k  K , or of
n   | J k | | K |


Fn   z : ( x,  , c)  R kK
:  b i z  b0i  clconv  ( Aˆ mn z  aˆ mn )  for some n  Kˆ .
mJ n
iT


55
Carnegie Mellon