Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Lecture 10: The Geometry of Linear Programs Murty Ch 3 1 Linear Combinations Let S = {v 1, . . . , v n} be a set of vectors in ℜm. A linear combination of S is any vector α1 v 1 + . . . + αn v n where α1, . . . , αn are scalars. Equivalently • If v 1, . . . , v n are column vectors, and A = [v 1, . . . , v n], then a linear combination of S is any vector Aα, α a column n-vector. 1 v . 1 n . • If v , . . . , v are row vectors, and A = , then vn a linear combination of S is any vector αA, α a row n-vector. The vectors v 1, . . . , v n are linearly independent if none of the vi’s can be written as a linear combination of the others. Equivalently, v 1, . . . , v n are linearly independent if there are no scalars α1, . . . , αn, not all zero, such that α1v 1 + . . . + αnv n = 0. (Proof: Exercise) 2 Examples v 1 = (8, 18) v 2 = (24, 18) v 3 = (16, 12) v 4 = (24, 6) (24, −10) = −2v 1 + 1v 2 + 12 v 3 + 13 v 4 8 24 (24, −10) = [−2, 1, 1/2, 1/3] 16 24 1 1 v 3 = v 1 + v 4, or 2 2 8 24 (0, 0) = [1/2, 0, −1, 1/2] 16 24 3 18 18 12 6 18 18 12 6 Linear Spaces A (linear) subspace is any subset V of vectors that contains every linear combination of its elements. Some important subspaces are The span or linear hull of the set S, span(S), is the set of all linear combinations of S. The null space of S, null(S), is the set of α = (α1, . . . , αn) i for which α1v1i + . . . + αmvm = 0, i=1,. . . ,n. (Proofs: exercise) For m × n matrix A, the row space of A, RA, is the span of the row vectors of A, and the column space of A, CA, is the span of the column vectors of A. 4 Examples span{v 2, v 3} null{v 2, v 3} v 2 = (24, 18) v 3 = (16, 12) span({v , v }) = row space of 2 3 null({v 2, v 3}) = x : 5 24 18 16 12 24 18 T 0 = x 0 16 12 Example: Linear systems Facts: Consider the system of equations (E) Ax = b 1. (E) has a solution if and only if b is in CA. 2. Any row in a row-equivalent system to (E) is an element of RA. 3. null(CA) is simply the set of solutions to the system Ax = 0. These are called homogeneous solutions associated with (E). 4. Let [Ā | b̄ ] be a basic tableau associated with the system (E) and basis B. Then the change vector v̂ B,j associated with any nonbasic variable j defined v̂kB,j 1 k=j = −āij k = Bi 0 otherwise is an element of null(CA). 6 Geometry and Rank The dimension of a subspace V , dim(V ), is the maximum number of linearly independent vectors in that space. A basis for V is any set of dim(V ) linearly independent elements of V Example: The rank of an m × n matrix A is equal to dim(RA), and a basis for RA is the set of nonzero rows left after applying the GJ reduction. Fact from linear algebra: rank(A)=rank(AT )=the size of the largest nonsingular submatrix of A. Thus the dimension of RA and CA are the same, and equal to the size of the largest nonsingular submatrix of A. 7 Examples x3 x3 A·,2 A2,· A3,· A·,3 A·,1 x1 x2 x1 (2,0,0) A1,· x2 (1,1,-1) 1 2 0 0 1 2 A = 1 3 2 dim({(1,2,0),(0,1,2),(1,3,2)})=dim({(1,0,1),(2,1,3),(0,2,2)})=2 (All 2 × 2 submatrices are nonsingular) (1, 1, −1) is perpendicular to (1, 0, 1), (2, 1, 3), (0, 2, 2), but not to b = (2, 0, 0) 8 Dot Products, Norms, and Farkas’ Lemma for Equality Systems The dot product of m-vectors u and v is ⟨u, v⟩ = m ∑ i=1 uivi √ The norm of v is ∥v∥ = ⟨v, v⟩. Facts from linear algebra: • ∥v∥ is the Euclidean length of the vector v. • ⟨u, v⟩ = ∥u∥∥v∥ cos θ, where θ is the angle between u and v. • In particular, if u and v are nonzero, then ⟨u, v⟩ = 0 ≡ u is perpendicular to v. Theorem of the Alternative for Equality Systems (geometric interpretation): Let v 1, . . . , v n and b be vectors in ℜm. Then either b ∈ span({v 1, . . . , v n}) or there exists a vector y that is perpendicular to every v j but not perpendicular to b. Proof: Let A = [v 1 . . . v n]. 9 Nonnegative Combinations and Cones A nonnegative combination of a set S = {v 1, . . . , v n} is any linear combination of S for which the multipliers α are nonnegative. A (convex) cone is any set K of vectors that contains every nonnegative combination of its elements. The (positive) cone of S, Pos(S), is the set of nonnegative combinations of S. Example: The positive orthant ℜm + is a cone, and in 1 m i th fact, ℜm + =Pos(e , . . . , e ), where e is the i unit vector. 10 Example Pos({v 1, v 2, v 3, v 4}) v 2 = (24, 18) v 1 = (8, 18) b = (4, 16) v 3 = (16, 12) v 4 = (24, 6) y=(9,-4) Farkas’ Lemma: Either the system 8 24 16 24 4 x = , 18 18 12 6 16 has a solution or the system y 8 24 16 24 ≥ (0, 0, 0, 0), 18 18 12 6 has a solution, but not both. 11 x≥0 y 4 < 0 16 Cones and Linear Programs Fact: The linear program max z = cx Ax = b x ≥ 0 has a feasible solution if and only if b is in the positive cone generated by the columns of A. Farkas’ Lemma for LPs (geometric interpretation): Let v 1, . . . , v n and b be vectors in ℜm. Then either b is in Pos(v 1, . . . , v n) or there exists a vector y that forms an angle of at most 90◦ with every v j but forms an angle of strictly greater than 90◦ with b. Proof: Let A = [v 1 . . . v n]. 12 Affine Sets An affine combination of a set S = {p1, . . . , pr } is any linear combination of S for which the multipliers sum to 1. An affine set is any set Γ of vectors that contains every affine combination of its elements. Example: The hyperplane in ℜn defined by n-vector α and scalar β is the set H = Hα,β = {x ∈ ℜn | n ∑ j=1 αj xj = β} is an affine set. The affine hull of S, Aff(S), is the set of affine combinations of S. Example: The affine hull of two distinct point u and v is the line going through u and v. A set of points p1, . . . , pr are affinely independent if none of them can be written as an affine combination of the others. Equivalently, p1, . . . , pr are affinely independent if there is no set of scalars α1, . . . , αr , not all zero, such that α1 + . . . + αr = 0, and α1p1 + . . . + αr pr = 0 (Proof: exercise) Examples: Any two distinct points are affinely independent. Any three points are affinely independent iff they are not on the same line segment. 13 Examples Aff({p1, p3, p4}) p1 = (8, 18) p2 = (24, 18) p3 = (16, 12) p4 = (24, 6) Aff({p1, p3, p4}) − p4 p4 = 2p3 − 1p1 ∈ Aff({p1, p3} = Aff({p1, p3, p4}) or equivalently: 1p4 − 2p3 + 1p1 = 0. p2 ∈ / Aff({p1, p3, p4}) 14 The Relationship Between Linear and Affine Independence Lemma 6.1 A set of points p1, . . . , pr is affinely independent if and only if the set p2 − p1, . . . , pr − p1 of vectors is linearly independent. Proof: First, suppose that p1, . . . , pr are affinely independent. Suppose that p2 − p1, . . . , pr − p1 are not linearly independent, that is, there exist multipliers α2, . . . , αr , not all zero, such that α2(p2 − p1) + . . . + αr (pr − p1) = 0. We have α2(p2 − p1) + . . . + αr (pr − p1) = −(α2 + . . . + αr )p1 + α2p2 + . . . + αr pr . so that if we let α1 = −(α2 + . . . + αr ), then we will have α1p1 + . . . + αr pr = 0 with α1 +. . .+αr = 0 and at least one αi nonzero. Thus p1, . . . , pr could not have been affinely independent. 15 Conversely, suppose that p2 − p1, . . . , pr − p1 are linearly independent, but p1, . . . , pr are not affinely independent, that is, there exist multipliers α1, . . . , αr , with α1 + . . . + αr = 0 and at least one αi is nonzero, such that α1p1 + . . . + αr pr = 0. Then we have α2(p2 − p1) + . . . + αr (pr − p1) = −(α2 + . . . + αr )p1 + α2p2 + . . . + αr pr = α1p1 + α2p2 + . . . + αr pr = 0. Further, since α1 + . . . + αr = 0 and at least one αi is nonzero, then at least two αi are nonzero, that is, at least one of α2, . . . , αr is nonzero. This implies that p2 − p1, . . . , pr −p1 could not have been linearly independent, and the lemma follows. 16 The Dimension of an Affine Space Notation: For any set S ⊂ ℜn and any point u ∈ ℜn S + u = {x + u| x ∈ S} Fact: An affine set is the translation of a linear set. In particular, if Γ is an affine set, and u is any point in Γ, then Γ = V + u, where V = Γ − u is a linear set. (Proof: Exercise) Definition: The dimension of any affine set Γ is the dimension of its associated linear set, that is, dim(Γ)=dim(Γ − u) for any u ∈ Γ. Lemma 6.2 The dimension d of any affine set Γ is equal to r − 1, where r is the maximum number of affinely independent points in Γ. Proof: Let V = Γ − u for any u ∈ Γ — so that d=dim(V ) — and let v 1, . . . , v d be a basis for V . Then from Lemma 6.1 the points u, u+v 1, . . . , u+v d are a set of d + 1 affinely independent points in Γ, so r ≥ d + 1. On the other hand, if p1, . . . , pr is any set of affinely independent points in Γ, then from Lemma 6.1 p2 − p1, . . . , pr − p1 is a set of r − 1 linearly independent points in V , and so r − 1 ≤ d. Thus r − 1 = d. 17 Affine Sets, Dimension, and Linear Systems We can talk about an affine set Γ ⊂ ℜn in one of two ways: • as Aff(p1, . . . , pr ) for points p1, . . . , pr in ℜn, • as the solution to a linear system (E) Ax = b where A is an m × n matrix and b is an m-vector. (Proof: Exercise.) We are interested in describing systems in either form. In particular, Given points p1, . . . , pr , can we produce equality system (E) whose feasible solutions are precisely Aff(p1, . . . , pr )? Given an equality system (E), can we find points p1, . . . , pr such that Aff(p1, . . . , pr ) is precisely the set of points satisfying (E)? Further, what is the minimal representation of Γ in each case, that is, the minimum number of equations/points that represent Γ? 18 Facts from Linear Algebra Lemma 6.3 Let A be an m × n matrix and b a column m-vector, and let Γ be the set of solutions for the system (E) Ax = b Suppose that (E) is feasible, and let x̂ be an element of L. (a) The linear space Γ − x̂ is exactly the set of solutions to the system Ax = 0. That is, Γ − x̂ = null(CA). [ ] (b) For any system Ā | b̄ row equivalent to (E), the rows of Ā are elements of span(RA). (c) For any set S ⊂ ℜn, dim(span(S)) + dim(null(S)) = n, in particular, dim(span(RA)) + dim(null(RA)) = n, 19 Lemma 6.4 The dimension d of a nonempty affine set Γ is equal to n−m where m the minimum number of rows of a full row-rank equality system containing Γ. Proof: Let u ∈ Γ, and let V = Γ − u be the associated linear space. Thus d=dim(V ), and let v 1, . . . , v d be a basis for V . Then from Lemma 6.3(c) there exist m = n−d linearly independent points w1, . . . , wm in null(V ). Let A be the m×n matrix whose rows are the wi vectors, and let b = Au. Then A is full row rank. Further, for every x ∈ Γ, we have x = u + v with v ∈ V , and so Ax = Au + Av = b + 0 = b. Thus every element of Γ satisfies the set of equalities. Finally suppose there were a system Āx = b̄ where Ā has full row rank m̄ < m. By performing Gauss-Jordan reduction on this system, we obtain tableau [I N̂ | b̂ ] by choosing change vectors v̂ B,1, . . . , v̂ B,n−m̄ for this system, we obtain n − m̄ + 1 > d + 1 affinely independent points u, u + v̂ B,1, . . . , u + v̂ B,n−m̄ in Γ, contradicting the dimension of Γ. Thus m = n − d is the maximum number of such equalities. 20 Examples p3 = (4, 0, 4) x3 p2 = (2, 1, 3) p1 = (0, 2, 2) x1 x2 The dimension of A= Aff({p1, p3, p4} is d = 1: • A is in n = 3-space. • p1 and p2 are r = d + 1 affinely independent points in A. • All of the points in A satisfy the n − d = 2 nonredundant equations x1 − 2x3 = −4 x2 + x3 = 4 21 Convex Sets An convex combination of a set S = {p1, . . . , pr } any linear combination of S for which the multipliers are nonnegative and sum to 1. That is, a convex combination is any linear combination that is both a nonnegative combination and an affine combination. Example: The line segment uv between two distinct points u and v is the set of convex combinations of u and v: uv = {λu + (1 − λ)v | 0 ≤ λ ≤ 1} A convex set is any set C of vectors that contains every convex combination of its elements. Lemma 6.5 A set C is convex if and only if it contains the line segments between every two of its points. Proof: Suppose first that C is a convex set. Since it contains any convex combination of any of its points, then clearly it contains the convex combination of any two of its points, that is, it contains the line segment between every two of its points. 22 Conversely, suppose that C contains the the line segment between every two of its points. Let p1, . . . , pr be a set of points in S and let α1, . . . , αr be a set of nonnegative scalars that sum to 1. We need to show that p̂ = α1p1 + . . . + αr pr ∈ S If r = 1 then α1 = 1 and clearly so clearly α1p1 = p1 ∈ S. Now proceed by induction on r ≥ 2. Since the αi sum to 1 and there are at least two of them, then we must have at least one αi < 1, say α1. Write α2 2 αr r p̂ = α1p1 +. . .+αr pr = α1p1 +(1−α1) p + ... + p 1 − α1 1 − α1 Now the point v= α2 2 αr r p + ... + p 1 − α1 1 − α1 is a convex combination of p2, . . . , pr , since negative and α2 1−α1 αr + . . . + 1−α = 1 α2 +...+αr 1−α1 = 1−α1 1−α1 αi 1−α1 is non- = 1. Thus by induction v ∈ S, and letting u = p1 and λ = α1 we have p̂ = λu + (1 − λ)v. Thus p̂ must be in S. 23 Examples p1 = (8, 18) p2 = (24, 18) Conv({p1, p2, p3, p4}) p3 = (16, 12) (20, 11) = 16 p2 + 12 p3 + 13 p4 p4 = (24, 6) p1 p4 p2 p3 A 3-simplex 24 More Examples of Convex Sets A half space is one of the two regions separated by a hyperplane Ha,β (and including Hα,β ). These are specified as + H = + Hα,β = {x ∈ ℜ | n − H − = Hα,β = {x ∈ ℜn | n ∑ j=1 n ∑ j=1 αj xj ≥ β} αj xj ≤ β} The convex hull of a set S of points in ℜn, Conv(S), is the set of convex combinations of S. Example: An r-simplex is the convex hull of any affinely independent set of r + 1 points. The standard n-simplex in ℜn+1 is the simplex made up of the unit vectors e1, . . . , en+1, and can be represented by the system x1 + . . . + xn+1 = 1 x ≥ 0. 25 LP Feasible Regions Lemma 6.6 The set of feasible solutions to the LP max z = cx Ax = b (P ) x ≥ 0 is a convex set. Proof: Let x1 and x2 be two feasible points of (P ), that is, Axi = b and xi ≥ 0, i = 1, 2, and let 0 ≤ λ ≤ 1 be a scalar. Then clearly λx1 + (1 − λ)x2 is nonnegative, since all of the factors are, and further A(λx1 + (1 − λ)x2) = λAx1 + (1 − λ)Ax2 = λb + (1 − λ)b = b So that λx1 + (1 − λ)x2 is also feasible to (P ). 26 Historical Note The name “simplex method” originated because the type of LP originally solved had the form max z = cx Ax = b (P ) ∑n j=1 xj = 1 x ≥ 0 A = m+1. Then the Where A is an m×n matrix with rank 1 ··· 1 feasible bases all have cardinality m + 1. Further, if B1 , . . . , Bm+1 is a feasible basis for (P ), then the basic variable values xB1 , . . . , xBm+1 must satisfy b = A.B1 xB1 + . . . + A.Bm+1 xBm+1 z = cB1 xB1 + . . . + cBm+1 xBm+1 xB1 + ... + xBm+1 = 1 xB1 ≥ 0, . . . , xBm+1 ≥ 0 Now consider the m-simplex ∆B created by the set of points A.B1 cB1 ,..., A.Bm+1 cBm+1 . From above, we havethat B is a feasible basis exactly when there is b a point of the form in ∆B , in which case the final coordinate z z will always be equal the objective function value of the associated basic solution. The simplex method, therefore, involves finding that simplex ∆B which intersects the vertical line b L = { | ζ ∈ ℜ} ζ at the highest possible point. It does this by iteratively replacing one point at a time in the simplex, each time raising the point of intersection of the simplex with L, until the simplex with the highest intersection with L is reached. 27 Convex Functions and Convex Sets convex function: real-valued function f on ℜn such that for each u, v ∈ ℜn and 0 ≤ λ ≤ 1, f (λu + (1 − λ)v) ≤ λf (u) + (1 − λ)f (v), that is, the f -value of any point on uv lies below the corresponding value of the linear approximation of f by line segment [u, f (u)][v, f (v)]. Example: Any linear function is a convex function, since for any function of the form f (x) = cx, and any u, v ∈ ℜn and 0 ≤ λ ≤ 1, f (λu + (1 − λ)v) = c(λu + (1 − λ)v) = λcu + (1 − λ)cv = λf (u) + (1 − λ)f (v). 28 Lemma 6.7 If g is a convex function, then the set of points satisfying the inequality g(x) ≤ b is a convex set. Proof: If u and v satisfy g(x) ≤ b and 0 ≤ λ ≤ 1, then g(λu + (1 − λ)v) ≤ λg(u) + (1 − λ)g(v) ≤ λb + (1 − λ)b = b and so λu + (1 − λ)v likewise satisfies the inequality. Lemma 6.8 The intersection of a collection of convex sets is also convex. Proof: If S1, . . . , Sr are convex sets, and u, v ∈ ∩ri=1Si, then u and v are in each Si, and so uv will be a subset of each Si and hence uv ⊆ ∩ri=1Si. 29 Convexity and Optimization Problems General description of an optimization problem: (O) min f (x) x∈S where f is a real-valued objective function on ℜn and S is the feasible region of points in ℜn. The feasible region S can often be more precisely described in functional form by those x ∈ ℜn satisfying gi(x) ≤ bi, i = 1, . . . r (∗) for set g1, . . . , gr of real-valued functions on ℜn. Lemma 6.9 If g1, . . . , gr are convex functions, then the region defined by (∗) is a convex region. Proof: By Lemma 6.7, the set of points satisfying any one of the inequalities gi(x) ≤ bi is convex, so by Lemma 6.8 the set of points satisfying all of the constraints will likewise be convex. Example: The feasible region of an LP is convex, since it can be defined by a set of linear inequalities. 30 Convexity and Local Minima local minimum: A point x̂ is a local minimum for (O) if x̂ ∈ S and there is a δ > 0 such that no point in S within distance δ of x̂ has smaller objective function than x̂. Theorem 6.1 If S is a convex set and f is a convex function, then every local optimal minimum for (O) is also a global minimum. Proof: Let x̂ be local minimum with objective function value ẑ = f (x̂), and suppose x̂ is not a global minimum. Let x∗ be a global minimum, with objective function value z∗ = f (x∗) < ẑ. Since S is convex, then every point on the line x∗x̂ is also in S. Further, since f is convex, then each point λx∗ + (1 − λ)x̂ ̸= x̂ on this line (λ > 0) has objective function value f (λx∗ + (1 − λ)x̂) ≤ λf (x∗) + (1 − λ)f (x̂) = λz∗ + (1 − λ)ẑ = ẑ + λ(z∗ − ẑ) < ẑ so x̂ cannot be a local minimum. This justifies “local improvement” methods, such as the simplex method, since when no local improvement is possible, the current solution is optimal. 31 Concavity and Maximization Problems concave function: real-valued function f on ℜn such that −f is convex, that is, for each u, v ∈ ℜn and 0 ≤ λ ≤ 1, f (λu + (1 − λ)v) ≥ λf (u) + (1 − λ)f (v) Facts: • If g is convex, then the set of points satisfying g(x) ≥ b is a convex set, and any set of inequalities of this sort define a convex set. • Linear functions are concave as well as convex, and so for example canonical minimization problems also have convex feasible regions. • If f is concave and S is convex, then any local maximum for the optimization problem max f (x) x∈S will be a global maximum. 32 The Dimension of a Convex Set The dimension of a convex set C is its affine dimension, that is, dim(C) equals one less than the maximum number of affinely independent points in C. Examples: • A singleton point has dimension 0 • A line segment has dimension 1 • An m-simplex has dimension m • The dimension the convex hull of the six points p1 p2 p3 p4 p5 p6 1 2 3 4 5 6 4 3 2 2 1 0 15 15 15 14 14 14 1 2 3 1 2 3 is equal to 2, since the maximum number of affinely independent points in this set is 3. 33 Lemma 6.10 Suppose the m × n LP min z = cx Ax = b (P ) x ≥ 0 has at least one basic nondegenerate tableau. Then the feasible region to (P ) has dimension n − m. Proof: Let (B, N ) be the basic/nonbasic partition of the components of x in a nondegenerate tableau for (P ), and let x̂ be the associated basic solution. Since this tableau is nondegenerate all of the RHS values are positive, and thus for any entering variable all of the minimum ratios ∆∗ will be positive. Let δ > 0 be chosen to be smaller than any of these minimum ratios. Then for each nonbasic column Nj we have that the solution xδ = x̂ + δv B,N1 will be feasible. It follows that x̂, x̂ + δv B,N1 , . . . , x̂ + δv B,Nn−m is a set of n − m + 1 affinely independent feasible points, and so the feasible region for (P ) has dimension n − m. 34 Example Consider the (nondegenerate) optimal vertex (12, 2, 0) of Woody’s LP. The associated tableau is basis x5 x2 x1 z z 0 0 0 1 x1 0 0 1 0 x2 x3 x4 x5 x6 rhs 0 −10 15/4 1 −10 30 1 2 −1/4 0 2/3 2 0 −1 1/2 0 −1 12 0 10 5/2 0 5 540 The 3 change vectors associated with the three nonbasic variables x3, x4, and x6 are v1 = 1 −2 1 0 10 0 v2 = −1/2 1/4 0 1 −15/4 0 v3 = 1 −2/3 0 0 10 1 and so the 4 affinely independent points that determine the dimension of the polytope (using, say, δ = 1) are x̂ = 12 2 0 1 , x̂ + v = 0 30 0 13 11 21 1 2 0 4 0 1 3 2 , x̂ + v = , x̂ + v = 1 0 1 26 40 4 0 0 35 13 1 31 0 1 40 1 Polyhedra and Polytopes polyhedron: the intersection of a finite set of linear equalities and inequalities. polytope: bounded polyhedron, that is, for which the component values are bounded. Example: The feasible region of any LP is a polyhedron. extreme point of polyhedron P : point x ∈ P such that every line segment uv in P containing x will have either x = u or x = v. Theorem 6.2 The set of basic feasible solutions for any linear program is exactly the set of extreme points for the associated polyhedron. Proof: Exercise. 36 Polytopes and Convex Hulls Theorem 6.3 (“Minkowski”) A polytope is the convex hull of its extreme points. Theorem 6.4 (“Weyl”): The convex hull of any set of points in ℜn is a polytope in ℜn. (The proofs of these results are technical, and will not be presented here.) Theorem 6.3 can be generalized to unbounded polyhedron, but it involves describing the “linear space” and “unbounded cone” for P , and so will also not be presented here. Murty Section 3.7 has a complete treatment of this. Theorem 6.4 is a key result in the study and solution of combinatorial problems that can be represented by linear programs. 37 Faces of Polyhedra Let P be a polyhedron. supporting hyperplane for P : Hyperplane H such that every point P is contained in the half space H +. face of P : subset of points of P that lie on some supporting hyperplane for P . Some special faces: P itself and the empty set: P is a face with supporting hyperplane H0,0, and ∅ is a face with any Hα,β not intersecting P . They are sometimes called improper faces to distinguish them from the other faces. vertices: 0-dimensional (singleton) faces. edges: 1-dimensional faces. facets: faces of dimension dim(P )-1 (maximum dimensional proper faces). 38 Example Consider the feasible region P for Woody’s 3-variable LP, defined by the inequalities 8x1 + 12x2 + 16x3 ≤ 120 15x2 + 20x3 ≤ 60 3x1 + 6x2 + 9x3 ≤ 48 x1 ≥ 0 x2 ≥ 0 x3 ≥ 0 The following equalities define supporting hyperplanes for P (P ⊂ Hi−): H0 : 35x1 + 60x2 + 75x3 = 540 H1 : 11x1 + 18x2 + 25x3 = 168 H2 : 2x1 + 3x2 + 4x3 = 30 and the associated faces of (P ) are F0 = the vertex (12,2,0) F1 = the edge between (12,2,0) and (13,0,1) F2 = the facet which is the convex hull of the points (12,2,0), (13,0,1), and (15,0,0) Fact: Any face of polyhedron P will itself be a polyhedron. 39 4 (0,0,3) 3 x2 5 2 4 x3 3 1 (0,4,0) 2 1 (0,0,0)0 0 (7,0,3) 5 (8,4,0) x1 10 (12,2,0) (13,0,1) 15 40 (15,0,0) Theorem 6.5 The set of optimal solutions for a linear program (L) is a face of the feasible region for (L). Proof: Let z∗ be the maximum objective function value for (L). Then every feasible point satisfies cx ≤ z∗, and so Hc,z∗ will be a supporting hyperplane for this set − (P ⊆ Hc,z ). ∗ Theorem 6.6 Any BFS for a linear program (L) is a vertex of P . Proof: Let x̂ = [x̂B , x̂N ] = [b̄, 0̄] be a BFS for (L). Consider the linear inequality ∑ xj ∈xN xj ≥ 0. (∗) Since xj must be nonnegative in any feasible solution, then every point of P satisfies (∗), so that (∗) is a supporting inequality. Further, the only solutions that satisfy (∗) at equality must have every element of xN equal to 0, and it follows from the equivalent linear representation xB = b̄ − N̄ xN that x̂ is the only such solution. Theorem 6.7 For adjacent BFS u and v, the line segment uv is an edge of P . Proof: Exercise. 41 Solving Combinatorial Problems Using the Convex Hull Combinatorial optimization problems often involve maximizing a linear function over a set S: max f (x) = cx (CO) x∈S where S is a finite set of points in ℜn representing the set of solutions that are allowed. In particular, since S if finite, the assumption of continuity for S does not hold. Question: Can we use linear programming to solve this problem? Answer: Yes, if we can describe S by a system (LP ) of linear equalities and inequalities in such a way that the optimal solution for the associated LP is the same as that for (CO). 42 Key Tool: Weyl’s Theorem (6.4), which states that conv(S) can be represented as a polytope, that is, as the feasible region for a system of linear equalities and inequalities. Lemma 6.11 The extreme points of conv(S) are all contained in S. Proof Let p = ∑ri=1 λipi be a convex combination of points pi ∈ S that is an extreme point of conv(S), and let r be the smallest number of points for which p can be expressed as a convex combination. Then 0 < λi < 1 for all λi. If r = 1 then p ∈ S. Otherwise let λi i p = p i=1 1 − λr ′ r−1 ∑ Then p′ ∈conv(S), since it is a combination of points in S whose multipliers are positive and sum to 1. But now p = λr pr + (1 − λr )p′. This means p is a convex combination of pr and p′, so that by the definition of extreme point p must be equal to pr or p′. But this means that p can be represented by a smaller number of points of S, contradicting the choice of points given above. Therefore p is in S. 43 Theorem 6.8 The basic feasible optimal solutions to the linear program (CP ) max cx x ∈ conv(S) are optimal to (CO). Proof By Theorem 6.2 the basic optimal solutions to (CP ) are extreme points of convS, and by Lemma 6.11 these are in turn in S. But since S ⊂ conv(S), they then must be optimal to (CO) Solving combinatorial optimization problem by linear programming, therefore, requires that you obtain the description of conv(S) by means of linear equalities and inequalities. For hard combinatorial problems this is generally also a difficult problem. For special classes of combinatorial problems, such as the Traveling Salesperson Problem, however, collections of inequalities have been found that describe conv(S) fairly accurately. This allows these problems to be solved much more quickly than with other methods. 44