Survey
* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project
Cutting Plane Algorithm for Convex Generalized Disjunctive Programming Francisco Trespalacios and Ignacio E. Grossmann May 23, 2017 Center for Advanced Process Decision-making Department of Chemical Engineering Carnegie Mellon University Pittsburgh, PA 15213 Motivation and goals Discrete-continuous optimization problems can be formulated in many ways as mixed-integer programming problems Motivation • Some formulations are good, some not • How to derive “good” formulations?1 Use Generalized Disjunctive Programming (GDP)2 as a general modeling framework to derive mixed-integer programming formulations Goals Derive cutting planes for a weak formulation using a stronger one • Stronger formulation obtained through the application of basic steps 2 1. Hooker,J., Logic-Based Methods for Optimization: Combining Optimization and Constraint Satisfaction, John Wiley & Sons (2000) 1. Wlliams H.P. Model Building in Mathematical Programming, John Wiley (1999) 1. Jeroslow, R.G. anf J.K. Lowe, Modelling with integer variables, Mathematical Programming StudiesVolume 22, 1984, pp 167-184 2. Raman R., Grossmann I.E., Modelling and Computational Techniques for Logic-Based Integer Programming, Computers and Chemical Engineering, 18, 563, 1994 Modeling mixed-integer programs The quality of the formulation depends on the modeler’s skills Classic Modeler MILP/ MINLP More control on the model: (BM) is smaller, while (HR) has a tighter relaxation Higher level of modeling – Using not only algebraic formulations, but also logic arguments Classic GDP Approach Modeler Big M (BM) MILP/ MINLP GDP model Hull Ref. (HR) MILP/ MINLP Modeler GDP model + cuts Proposed Improve the “weak” (BM) formulation by adding cuts, derived from a very strong formulation 3 Strengthened r-(HR) region min distance1 Solve r-(BM) (BM) + cuts MILP/ MINLP (HR) strengthened through basic steps 1. Stubbs, Robert A. and Mehrotra, Sanjay: “A branch-and-cut method for 0-1 mixed convex Programming”, Math. Program., Ser. A 86: 515–532 (1999) Modeling mixed-integer programs Classic Classic GDP Approach Modeler MILP/ MINLP Big M (BM) MILP/ MINLP Modeler GDP model Hull Ref. (HR) MILP/ MINLP Modeler GDP model + cuts Proposed Strengthened r-(HR) region min distance1 Solve r-(BM) The performance of the algorithms strongly depend on the size of the problem and tightness of relaxation 4 (BM) + cuts MILP/ MINLP Modeling mixed-integer programs Classic Classic GDP Approach Modeler MILP/ MINLP Big M (BM) MILP/ MINLP Modeler GDP model Hull Ref. (HR) MILP/ MINLP 5 Modeler GDP model + cuts Proposed Strengthened r-(HR) model min distance1 Solve r-(BM) (BM) + cuts MILP/ MINLP GDP is a higher level of representation for MILP/MINLP Optimization problem with algebraic expressions, disjunctions & logic propositions General Convex formGDP of GDP1 Illustration – Process network F2 F1 min 𝑧 = 𝑓 𝑥 F7 Objective Function F3 Convex Convex s.t. 𝑔 𝑥 ≤ 0 ⋁ 𝑖 ∈ 𝐷𝑘 𝑟𝑘𝑖 F6 R1 𝑌𝑘𝑖 𝑘∈𝐾 𝑥 ≤0 𝛺 𝑌 = 𝑇𝑟𝑢𝑒 F4 S2 F5 S1 Global Constraints ⊻ 𝑖 ∈ 𝐷𝑘 𝑌𝑘𝑖 R2 Disjunctions 𝑘∈𝐾 Logic Propositions max 𝑧 = 𝑃7 𝐹7 − 𝑃1 𝐹1 − 𝑐𝑅 − 𝑐𝑆 Objective function 𝐹1 = 𝐹2 + 𝐹3 𝐹7 = 𝐹5 + 𝐹6 Global Constraints 𝑌𝑅1 𝑌𝑅2 𝐹 = 𝐹2 = 0 𝐹6 = 𝛽𝑅1 𝐹2 ⋁ 6 𝐹3 = 𝐹4 = 0 𝐹4 = 𝛽𝑅2 𝐹3 𝑐𝑅 = 𝛾𝑟1 𝑐𝑅 = 𝛾𝑟2 𝑌𝑆_𝑁𝑂 𝑌𝑆1 𝑌𝑆2 𝐹5 = 𝛽𝑆1 𝐹4 ⋁ 𝐹5 = 𝛽𝑆2 𝐹4 ⋁ 𝐹5 = 0 𝑐𝑆 = 𝛾𝑠2 𝑐𝑆 = 𝛾𝑠2 𝑐𝑆 = 0 Disjunctions 𝑌𝑅1 ⊻ 𝑌𝑅2 𝑥 ∈ ℝ𝑛 𝑌𝑘𝑖 ∈ 𝑇𝑟𝑢𝑒, 𝐹𝑎𝑙𝑠𝑒 6 𝑌𝑆1 ⊻ 𝑌𝑆2 ⊻ 𝑌𝑆_𝑁𝑂 𝑘 ∈ 𝐾, 𝑖 ∈ 𝐷𝑘 𝑌𝑅1 ⇔ 𝑌𝑆_𝑁𝑂 1. Raman R. and Grossmann I.E., “Modelling and Computational Techniques for Logic-Based Integer Programming”, Computers and Chemical Engineering, 18, 563, 1994. Logic Modeling mixed-integer programs Classic Classic GDP Approach Modeler MILP/ MINLP Big M (BM) MILP/ MINLP Modeler GDP model Hull Ref. (HR) MILP/ MINLP 7 Modeler GDP model + cuts Proposed Strengthened r-(HR) model min distance1 Solve r-(BM) (BM) + cuts MILP/ MINLP GDPs can be reformulated using either the Big-M (BM) or the Hull Reformulation (HR) GDP (BM) 𝑓 𝑥 min 𝑧 = 𝑓 𝑥 Large, positive M. Constraint relaxes when yik = 0 s.t. 𝑔 𝑥 ≤ 0 ˅ 𝑖 ∈ 𝐷𝑘 (HR) 𝑟𝑘𝑖 𝑌𝑘𝑖 𝑥 ≤0 𝑥 𝑙𝑜 ≤ 𝑥 ≤ 𝑥 𝑢𝑝 𝑔 𝑥 ≤0 𝑥= 𝑖𝑘 𝑟𝑘𝑖 𝑥 ≤ 𝑀 (1 − 𝑦𝑖𝑘 ) 𝑖 ∈𝐷𝑘 𝑦𝑖𝑘 ⊻ 𝑖 ∈ 𝐷𝑘 𝑌𝑘𝑖 𝛺 𝑌 = 𝑇𝑟𝑢𝑒 x is disaggregated. Constraints transformed with perspective function2 If 𝑦𝑘𝑖 = 0, 𝑦𝑘𝑖 𝑟𝑘𝑖 𝜈 𝑘𝑖 /𝑦𝑘𝑖 = 0 Linear case: 𝐴𝑘𝑖 𝜈 𝑘𝑖 ≤ 𝑎𝑘𝑖 𝑦𝑘𝑖 Reformulation of the logic constraints in the discrete space1 𝑌𝑘𝑖 ∈ 𝑇𝑟𝑢𝑒, 𝐹𝑎𝑙𝑠𝑒 8 2. 𝑘𝑖 𝑦𝑘𝑖 𝑟𝑘𝑖 𝜈 𝑘𝑖 /𝑦𝑘𝑖 ≤ 0 =1 𝑖 ∈𝐷𝑘 𝑦𝑘𝑖 =1 𝐻𝑦 ≥ ℎ 𝑥 𝑙𝑜 ≤ 𝑥 ≤ 𝑥 𝑢𝑝 𝑥 𝑙𝑜 𝑦𝑘𝑖 ≤ 𝜈 𝑘𝑖 ≤ 𝑥 𝑢𝑝 𝑦𝑘𝑖 𝑦𝑘𝑖 ∈ 0,1 Logic variables into binary vars 1. 𝑖 ∈𝐷𝑘 𝜈 Easily accomplished as described in: Williams H.P., Mathematical Building in Mathematical Programming, John Wiley, 1985 𝑦𝑘𝑖 𝑟𝑘𝑖 𝜈 𝑘𝑖 /𝑦𝑘𝑖 is convex if if 𝑟𝑘𝑖 𝑥 ≤ 0 is convex. (HR) provides a tighter relaxation than (BM) Illustration of (BM) and (HR) relaxations (BM) x2 (HR) Variables: 6 (4 binary) Constraints: 18 F: BM( 𝑨𝟏 ⊻ [𝑨𝟐]) BM( 𝑩𝟏 ⊻ [𝑩𝟐]) x2 Variables: 14 (4 binary) Constraints: 36 F: HR( 𝑨𝟏 ⊻ [𝑨𝟐]) HR( 𝑩𝟏 ⊻ [𝑩𝟐]) (HR) is the intersection of the convex hull of each disjunction 𝑨𝟏 𝑨𝟏 𝑩𝟏 𝑩𝟏 𝑩𝟐 𝑩𝟐 (HR) is tighter than (BM), but does not yield convex hull 𝑨𝟐 𝑨𝟐 x1 Can we further improve the tightness of the (HR)? 9 x1 Modeling mixed-integer programs Classic Classic GDP Approach Modeler MILP/ MINLP Big M (BM) MILP/ MINLP Modeler GDP model Hull Ref. (HR) MILP/ MINLP Modeler GDP model + cuts Proposed Strengthened r-(HR) model Using basic steps 10 min distance1 Solve r-(BM) (BM) + cuts MILP/ MINLP The HR after the application of a Basic Step (BS), is at least as tight as the previous Hull Reformulation F: F: x2 Application of a Basic Step Variables: 14 (4 binary) Constraints: 36 𝑨𝟏 ⊻ [𝑨𝟐] 𝑩𝟏 ⊻ [𝑩𝟐] x2 𝑨𝟏 𝑨𝟏 𝑨𝟐 𝑨𝟐 ⊻ ⊻ ⊻ 𝑩𝟏 𝑩𝟐 𝑩𝟏 𝑩𝟐 Variables: 14 (4 binary) Constraints: 51 Tighter feasible region! (Better Bounds) 𝑨𝟏 𝑨𝟏 𝑩𝟏 𝑩𝟏 𝑩𝟐 𝑩𝟐 𝑨𝟐 𝑨𝟐 x1 11 x1 Basic Steps improve tightness but increase problem size Improper disjunction: No OR statement (i.e. global constraints) 𝐴 Original Problem 𝐵1 ∨ [𝐵2 ] # of disj. |K| Improper Basic Step: Between proper and improper disjunction 𝐶1 ∨ [𝐶2 ] The (HR) after each Basic Step is as tight as the previous formulation Proper disjunction 𝐷1 ∨ 𝐷2 ∨ [𝐷3 ] BS 1: A, B 𝐴 𝐴 ∨ 𝐵1 𝐵2 # ofBasic termsStep: does Between not increase, Proper but terms increase proper disjunctions in size 𝐶1 ∨ [𝐶2 ] 𝐷1 ∨ 𝐷2 ∨ [𝐷3 ] BS 2: A,B,C 4 𝐴 𝐴 𝐴 𝐴 𝐵1 ∨ 𝐵1 ∨ 𝐵 2 ∨ 𝐵 2 𝐶1 𝐶2 𝐶1 𝐶2 # of disj. terms increases exponentially, and terms increase in size (HR) of DNF is a perfect formulation! 3 2 𝐷1 ∨ 𝐷2 ∨ [𝐷3 ] BS 3: DNF 12 𝐴 𝐴 𝐴 𝐴 𝐴 𝐴 𝐴 𝐴 𝐴 𝐴 𝐴 𝐴 𝐵1 𝐵1 𝐵1 𝐵1 𝐵1 𝐵1 𝐵2 𝐵2 𝐵2 𝐵2 𝐵2 𝐵2 ⊻ ⊻ ⊻ ⊻ ⊻ ⊻ ⊻ ⊻ ⊻ ⊻ ⊻ 𝐶1 𝐶1 𝐶1 𝐶2 𝐶2 𝐶2 𝐶1 𝐶1 𝐶1 𝐶2 𝐶2 𝐶2 𝐷1 𝐷2 𝐷3 𝐷1 𝐷2 𝐷3 𝐷1 𝐷2 𝐷3 𝐷1 𝐷2 𝐷3 Source: Balas E., Disjunctive Programming and a hierarchy of relaxations for discrete optimization problems, SIAM J. Alg. Disc. Meth., 6, 466-486, 1985 Juan P. Ruiz, Ignacio E. Grossmann, A hierarchy of relaxations for nonlinear convex generalized disjunctive programming 1 Modeling mixed-integer programs Classic Classic GDP Approach Modeler MILP/ MINLP Big M (BM) MILP/ MINLP Modeler GDP model Hull Ref. (HR) MILP/ MINLP 13 Modeler GDP model + cuts Proposed Strengthened r-(HR) model min distance Solve r-(BM) (BM) + cuts MILP/ MINLP Deriving cuts to (BM) using stronger formulations Generation of cuts in (BM) formulation using (HR) Max Z x2 s.t. 𝑨𝟏 ∨ [𝑨𝟐 ] 𝑩𝟏 ∨ [𝑩𝟐 ] 𝑪𝟏 ∨ [𝑪𝟐 ] Z* (BM) x1 14 Deriving cuts to (BM) using stronger formulations Generation of cuts in (BM) formulation using (HR) x2 𝒙𝑩𝑴 Z* 𝒙𝒔𝒆𝒑 (BM) + 1 (HR) cut (HR) Solve: min 𝑓(𝑥) 𝑥∈(𝐵𝑀) Yields: 𝑥 𝐵𝑀 Solve: min 𝜑 𝑥 = 𝑥 − 𝑥 𝐵𝑀 𝑥∈(𝐻𝑅) 𝑝 (BM) Yields: 𝑥 𝑠𝑒𝑝 Obtaining a valid cut: 𝜉 𝑇 (𝑥 − 𝑥 𝑠𝑒𝑝 ) ≥ 0 𝜉 is a subgradient of 𝜑(𝑥) at 𝑥 𝑠𝑒𝑝 x1 15 Deriving cuts to (BM) using stronger formulations Generation of cuts in (BM) formulation using basic steps x2 𝒙𝑩𝑴 Z* Stronger cutting plane (HR) after BS A and B Solve: min 𝑓(𝑥) 𝑥∈(𝐵𝑀) 𝒙𝒔𝒆𝒑 (BM) + 1 basic step cut Yields: 𝑥 𝐵𝑀 Solve: min 𝜑 𝑥 = 𝑥 − 𝑥 𝐵𝑀 𝑥∈(𝑆𝐸𝑃) 𝑝 (BM) Yields: 𝑥 𝑠𝑒𝑝 Obtaining a valid cut: 𝜉 𝑇 (𝑥 − 𝑥 ∗ ) ≥ 0 𝜉 is a subgradient of 𝜑(𝑥) at 𝑥 𝑠𝑒𝑝 x1 16 Deriving cuts to (BM) using stronger formulations Generation of cuts in (BM) formulation using (HR) vs. basic steps Max Z x2 𝒛𝑩𝑴𝟑 𝒛𝑩𝑴𝟐 𝒛𝑩𝑴 Z* (BM) + 1 (HR) cut s.t. 𝑨𝟏 ∨ [𝑨𝟐 ] 𝑩𝟏 ∨ [𝑩𝟐 ] 𝑪𝟏 ∨ [𝑪𝟐 ] (BM) (BM) + 1 basic step cut x1 17 Deriving cuts to (BM) using stronger formulations Generation of each cut requires solution of two NLP NLP : 𝑥 − 𝑥 𝐵𝑀 NLP: r-(BM) min 𝑥𝑛 Solve relaxed (BM) s.t. Constraints where no basic step was applied Solution 𝑥 𝐵𝑀 𝑔 𝑥 ≤0 𝑖𝑘 𝑟𝑘𝑖 𝑥 ≤ 𝑀 (1 − 𝑦𝑖𝑘 ) 𝑘 ∈ 𝐾, 𝑖 ∈ 𝐷𝑘 𝑖∈𝐷𝑘 𝑦𝑖𝑘 =1 0 ≤ 𝑦𝑘𝑖 ≤ 1 min 𝑗(𝑥𝑗 − 𝑥𝑗𝐵𝑀 )2 2 2; 𝑥 ∈ (𝑆𝐸𝑃) 2 2 s.t. is convex and gradient simple. For linear cases 1 or ∞ 𝑔𝑒 𝑥 ≤ 0 𝑥 = 𝑖∈𝐷𝑘 𝜈 𝑘𝑖 𝑒 ∈ 𝐸\𝐸 𝑘 ∈ 𝐾\𝐾 𝑦𝑘𝑖 𝑟𝑘𝑖 𝜈 𝑘𝑖 /𝑦𝑘𝑖 ≤ 0 𝑘 ∈ 𝐾\𝐾, 𝑖 ∈ 𝐷𝑘 𝑖∈𝐷𝑘 𝑦𝑘𝑖 =1 𝑘 ∈ 𝐾\𝐾 𝑘∈𝐾 𝑥 𝑙𝑜 𝑦𝑘𝑖 ≤ 𝜈 𝑘𝑖 ≤ 𝑥 𝑢𝑝 𝑦𝑘𝑖 𝑘 ∈ 𝐾\𝐾, 𝑖 ∈ 𝐷𝑘 𝑘 ∈ 𝐾, 𝑖 ∈ 𝐷𝑘 𝐻𝑦 ≥ ℎ Constraints where basic steps were applied 𝜉 𝑇 (𝑥 − 𝑥 𝑠𝑒𝑝 ) ≥ 0 Add cuts 𝜉 𝑇 (𝑥 − 𝑥 𝑠𝑒𝑝 ) ≥ 0, where 𝜉 = 2(𝑥 𝑠𝑒𝑝 − 𝑥 𝐵𝑀 ) is the subgradient 0 ≤ 𝑦𝑘𝑖 ≤ 1 𝑥= Solution 𝑥 𝑠𝑒𝑝 𝑖∈𝐷𝑘 𝜈 𝑘 ∈ 𝐾\𝐾, 𝑖 ∈ 𝐷𝑘 𝑘𝑖 𝑘∈𝐾 𝑦𝑘𝑖 𝑔𝑒 𝜈 𝑘𝑖 /𝑦𝑘𝑖 ≤ 0 𝑘 ∈ 𝐾, 𝑖 ∈ 𝐷𝑘 ; 𝑒 ∈ 𝐸𝑘 𝑦𝑘𝑖 𝑟𝑘𝑖 𝜈 𝑘𝑖 /𝑦𝑘𝑖 ≤ 0 𝑘 ∈ 𝐾, 𝑖 ∈ 𝐷𝑘 ; 𝑘𝑖 ∈ 𝐾𝐼𝑘𝑖 𝑖∈𝐷𝑘 𝑦𝑘𝑖 =1 𝑘∈𝐾 𝑥 𝑙𝑜 𝑦𝑘𝑖 ≤ 𝜈 𝑘𝑖 ≤ 𝑥 𝑢𝑝 𝑦𝑘𝑖 𝑘 ∈ 𝐾, 𝑖 ∈ 𝐷𝑘 0 ≤ 𝑦𝑘 𝑖 ≤ 1 18 𝑘 ∈ 𝐾, 𝑖 ∈ 𝐷𝑘 Algorithm generates cutting planes for (BM) There are many alternative heuristics and strategies Add cut 𝜉 𝑇 (𝑥 − 𝑥 𝑠𝑒𝑝 ) ≥ 0, where 𝜉 = 2(𝑥 𝑠𝑒𝑝 − 𝑥 𝐵𝑀 ) Algorithm Decisions in the algorithm Continue adding cuts? • Number of cuts • Value of minimum distance problem Solve r-(BM) to obtain 𝑥 𝐵𝑀 No Add cut? Yes Solve: min 𝑥∈(𝑆𝐸𝑃) 𝑥 − 𝑥 𝐵𝑀 and obtain 𝑥 𝑠𝑒𝑝 2 2 Solve (BM) + cuts Proper basic steps • Number of “key disjunctions” (one, five, as most as possible?) • Number of basic steps in each “key” • Which disjunctions to intersect in each “key” Improper basic steps • Which global constraints intersect with which disjunctions? • Repeat some global constraints to increase the number of intersections? Minimization norm 2 • 2 is convex and its gradient simple. For linear cases 1 or ∞ provide LP min problem GDP presolve1 • Eliminate infeasible terms, find better bounds, and provide a value that characterizes each disjunction 1. 19 Francisco Trespalacios and Ignacio E. Grossmann, Algorithmic approach for improved mixed-integer reformulations of convex Generalized Disjunctive Programs, submitted for publication Informs JOC Examples and results 20 Different strategies were tested for the algorithm Heuristic for selection of intersecting disjunctions is based on three key concepts Global constraints “key disjunctions” Tested strategies 21 Strategy # of key disjunctions K0 0 K1 1 K5 5 KK Max Strategy # of key disjunctions I1 No-redundant I2 Max Selection of intersecting disjunctions Only with disjunctions that share variables in common For improper Basic steps First priority: “key disjunctions” No-redundant Second: order of disjunction Between disjunctions that share variables in common For proper Basic steps Consider # of terms in resulting disjunction = (# of terms in D1)*(# of terms in D2) The disjunction with highest char. value from the presolve (when presolve is active) Algorithm was tested with convex GDP problems Constrained Layout (Clay) Process Flowsheet Type A (Proc-1) 8 instances 4 instances y x19 6 x1 1 x12 1 3 x3 x2 4 2 2 x4 x11 5 x6 3 x16 x25 x17 𝐽 𝑗=1 4 instances x22 x18 8 Non linear term 𝐽 𝑑𝑖𝑗 𝑒 Farm Layout (Flay) 7 x9 Unit function is nonlinear: x 𝑥𝑗 𝑡𝑖𝑗 −1 − 𝑠𝑖𝑗 𝑥𝑗 ≤ 0 𝑗=1 Type A: Select at most one unit 𝐿 A2 Process Flowsheet Type B (Proc-2) A3 𝐻 3 instances A4 Type B: Possible to select more than one unit A1 x 22 x23 x21 x8 y x13 x15 6 ∆𝒙𝟐𝟓 x24 4 x5 ∆𝒚𝟒𝟔 5 x20 Algorithm solves problems faster than (BM) and (HR) 19 test instances Algorithm with 5 “key disjunctions” nonredundant improper basic steps and 3 cuts Percentage of problem solved vs. time % of problems 100 90 (BM) 80 70 (HR) 60 K5-I1-3cuts (BM) 50 (HR) 40 30 20 10 0 0 500 1,000 1,500 2,000 2,500 3,000 3,500 4,000 4,500 5,000 5,500 6,000 6,500 7,000 Time (s) Implemented in GAMS 24.1.3. SBB solver 23 The algorithm improves the relaxation gap Relaxation does not improve much after the first cut Average relaxation gap vs. number of cuts Relaxation gap 1.0 (BM) 0.9 0.8 (HR) 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 0 1 2 3 4 5 6 Number of cuts 24 K0-I1 K1-I1 K5-I1 KK-I1 (HR) K0-I2 K1-I2 K5-I2 KK-I2 (BM) 7 8 9 10 Algorithm improves solution times using any strategy Using ~3 cuts seems to provide the best solution times Accumulated solution time vs. number of cuts Acc. Solution time (s) 40,000 (HR) 35,000 30,000 (BM) 25,000 20,000 15,000 10,000 5,000 0 0 1 2 3 4 5 6 Number of cuts 25 K0-I1 K1-I1 K5-I1 KK-I1 (HR) K0-I2 K1-I2 K5-I2 KK-I2 (BM) Implemented in GAMS 24.1.3. SBB solver 7 8 9 10 Conclusions and future work Proposed new cutting plane algorithm for convex GDP that relies on concept of basic steps and GDP-to-MINLP reformulations Results show that the proposed algorithm improves formulations in convex GDPs • Solved larger problems in less time using the algorithm • Obtained stronger relaxations than BM and HR Future work will explore improvements for the algorithm and test additional problems 26