Download Lecture 10: The Geometry of Linear Programs

Lecture 10:
The Geometry of Linear Programs
Murty Ch 3
1
Linear Combinations
Let S = {v 1, . . . , v n} be a set of vectors in ℜm. A
linear combination of S is any vector
α1 v 1 + . . . + αn v n
where α1, . . . , αn are scalars. Equivalently
• If v 1, . . . , v n are column vectors, and A = [v 1, . . . , v n],
then a linear combination of S is any vector Aα, α
a column n-vector.


1
 v 


 . 
1
n
 . 
• If v , . . . , v are row vectors, and A =  , then


vn
a linear combination of S is any vector αA, α a row
n-vector.
The vectors v 1, . . . , v n are linearly independent if
none of the vi’s can be written as a linear combination
of the others. Equivalently, v 1, . . . , v n are linearly independent if there are no scalars α1, . . . , αn, not all zero,
such that
α1v 1 + . . . + αnv n = 0.
(Proof: Exercise)
2
Examples
v 1 = (8, 18)
v 2 = (24, 18)
v 3 = (16, 12)
v 4 = (24, 6)
(24, −10) = −2v 1 + 1v 2 + 12 v 3 + 13 v 4

8
24
(24, −10) = [−2, 1, 1/2, 1/3]
16
24
1
1
v 3 = v 1 + v 4, or
2
2

 8


 24

(0, 0) = [1/2, 0, −1, 1/2] 
 16


24











3

18 

18 

12 

6

18 

18 

12 

6
Linear Spaces
A (linear) subspace is any subset V of vectors that
contains every linear combination of its elements. Some
important subspaces are
The span or linear hull of the set S, span(S), is the
set of all linear combinations of S.
The null space of S, null(S), is the set of α = (α1, . . . , αn)
i
for which α1v1i + . . . + αmvm
= 0, i=1,. . . ,n.
(Proofs: exercise)
For m × n matrix A, the row space of A, RA, is the
span of the row vectors of A, and the column
space of A, CA, is the span of the column vectors
of A.
4
Examples
span{v 2, v 3}
null{v 2, v 3}
v 2 = (24, 18)
v 3 = (16, 12)

span({v , v }) = row space of
2
3








null({v 2, v 3}) = x :
5





24 18 

16 12

24 18  T  0
=
x
0
16 12








Example: Linear systems
Facts: Consider the system of equations
(E)
Ax = b
1. (E) has a solution if and only if b is in CA.
2. Any row in a row-equivalent system to (E) is an
element of RA.
3. null(CA) is simply the set of solutions to the system Ax = 0. These are called homogeneous
solutions associated with (E).
4. Let [Ā | b̄ ] be a basic tableau associated with
the system (E) and basis B. Then the change
vector v̂ B,j associated with any nonbasic variable j defined
v̂kB,j








1 k=j
=  −āij k = Bi




0 otherwise
is an element of null(CA).
6
Geometry and Rank
The dimension of a subspace V , dim(V ), is the maximum number of linearly independent vectors in that
space. A basis for V is any set of dim(V ) linearly
independent elements of V
Example: The rank of an m × n matrix A is equal to
dim(RA), and a basis for RA is the set of nonzero
rows left after applying the GJ reduction.
Fact from linear algebra: rank(A)=rank(AT )=the
size of the largest nonsingular submatrix of A.
Thus the dimension of RA and CA are the same, and
equal to the size of the largest nonsingular submatrix of A.
7
Examples
x3
x3
A·,2
A2,·
A3,·
A·,3
A·,1
x1
x2
x1
(2,0,0)
A1,·
x2
(1,1,-1)


 1 2 0 




 0 1 2 
A = 




1 3 2
dim({(1,2,0),(0,1,2),(1,3,2)})=dim({(1,0,1),(2,1,3),(0,2,2)})=2
(All 2 × 2 submatrices are nonsingular)
(1, 1, −1) is perpendicular to (1, 0, 1), (2, 1, 3), (0, 2, 2),
but not to b = (2, 0, 0)
8
Dot Products, Norms, and
Farkas’ Lemma for Equality Systems
The dot product of m-vectors u and v is
⟨u, v⟩ =
m
∑
i=1
uivi
√
The norm of v is ∥v∥ = ⟨v, v⟩.
Facts from linear algebra:
• ∥v∥ is the Euclidean length of the vector v.
• ⟨u, v⟩ = ∥u∥∥v∥ cos θ, where θ is the angle between u and v.
• In particular, if u and v are nonzero, then
⟨u, v⟩ = 0 ≡ u is perpendicular to v.
Theorem of the Alternative for Equality Systems (geometric interpretation): Let v 1, . . . , v n
and b be vectors in ℜm. Then either b ∈ span({v 1, . . . , v n})
or there exists a vector y that is perpendicular to
every v j but not perpendicular to b.
Proof: Let A = [v 1 . . . v n].
9
Nonnegative Combinations and Cones
A nonnegative combination of a set S = {v 1, . . . , v n}
is any linear combination of S for which the multipliers α are nonnegative.
A (convex) cone is any set K of vectors that contains every nonnegative combination of its elements.
The (positive) cone of S, Pos(S), is the set of nonnegative combinations of S.
Example: The positive orthant ℜm
+ is a cone, and in
1
m
i
th
fact, ℜm
+ =Pos(e , . . . , e ), where e is the i unit
vector.
10
Example
Pos({v 1, v 2, v 3, v 4})
v 2 = (24, 18)
v 1 = (8, 18)
b = (4, 16)
v 3 = (16, 12)
v 4 = (24, 6)
y=(9,-4)
Farkas’ Lemma: Either the system




8 24 16 24 
 4 


x
=


,
18 18 12 6
16
has a solution or the system





y 

8 24 16 24 
 ≥ (0, 0, 0, 0),
18 18 12 6
has a solution, but not both.
11
x≥0


y 

4 
 < 0
16
Cones and Linear Programs
Fact: The linear program
max z = cx
Ax = b
x ≥ 0
has a feasible solution if and only if b is in the
positive cone generated by the columns of A.
Farkas’ Lemma for LPs (geometric interpretation): Let v 1, . . . , v n and b be vectors in ℜm.
Then either b is in Pos(v 1, . . . , v n) or there exists
a vector y that forms an angle of at most 90◦ with
every v j but forms an angle of strictly greater than
90◦ with b.
Proof: Let A = [v 1 . . . v n].
12
Affine Sets
An affine combination of a set S = {p1, . . . , pr } is any linear
combination of S for which the multipliers sum to 1.
An affine set is any set Γ of vectors that contains every affine
combination of its elements.
Example: The hyperplane in ℜn defined by n-vector α and
scalar β is the set
H = Hα,β = {x ∈ ℜn |
n
∑
j=1
αj xj = β}
is an affine set.
The affine hull of S, Aff(S), is the set of affine combinations of
S.
Example: The affine hull of two distinct point u and v is the line
going through u and v.
A set of points p1, . . . , pr are affinely independent if none of
them can be written as an affine combination of the others.
Equivalently, p1, . . . , pr are affinely independent if there is no
set of scalars α1, . . . , αr , not all zero, such that
α1 + . . . + αr = 0, and
α1p1 + . . . + αr pr = 0
(Proof: exercise)
Examples: Any two distinct points are affinely independent. Any
three points are affinely independent iff they are not on the
same line segment.
13
Examples
Aff({p1, p3, p4})
p1 = (8, 18)
p2 = (24, 18)
p3 = (16, 12)
p4 = (24, 6)
Aff({p1, p3, p4}) − p4
p4 = 2p3 − 1p1 ∈ Aff({p1, p3} = Aff({p1, p3, p4})
or equivalently: 1p4 − 2p3 + 1p1 = 0.
p2 ∈
/ Aff({p1, p3, p4})
14
The Relationship Between Linear and Affine
Independence
Lemma 6.1 A set of points p1, . . . , pr is affinely independent if and only if the set p2 − p1, . . . , pr − p1 of
vectors is linearly independent.
Proof: First, suppose that p1, . . . , pr are affinely independent. Suppose that p2 − p1, . . . , pr − p1 are not linearly independent, that is, there exist multipliers α2, . . . , αr ,
not all zero, such that
α2(p2 − p1) + . . . + αr (pr − p1) = 0.
We have
α2(p2 − p1) + . . . + αr (pr − p1)
= −(α2 + . . . + αr )p1 + α2p2 + . . . + αr pr .
so that if we let α1 = −(α2 + . . . + αr ), then we will
have
α1p1 + . . . + αr pr = 0
with α1 +. . .+αr = 0 and at least one αi nonzero. Thus
p1, . . . , pr could not have been affinely independent.
15
Conversely, suppose that p2 − p1, . . . , pr − p1 are linearly independent, but p1, . . . , pr are not affinely independent, that is, there exist multipliers α1, . . . , αr , with
α1 + . . . + αr = 0 and at least one αi is nonzero, such
that
α1p1 + . . . + αr pr = 0.
Then we have
α2(p2 − p1) + . . . + αr (pr − p1)
= −(α2 + . . . + αr )p1 + α2p2 + . . . + αr pr
= α1p1 + α2p2 + . . . + αr pr = 0.
Further, since α1 + . . . + αr = 0 and at least one αi
is nonzero, then at least two αi are nonzero, that is, at
least one of α2, . . . , αr is nonzero. This implies that p2 −
p1, . . . , pr −p1 could not have been linearly independent,
and the lemma follows.
16
The Dimension of an Affine Space
Notation: For any set S ⊂ ℜn and any point u ∈ ℜn
S + u = {x + u| x ∈ S}
Fact: An affine set is the translation of a linear set. In
particular, if Γ is an affine set, and u is any point
in Γ, then Γ = V + u, where V = Γ − u is a linear
set. (Proof: Exercise)
Definition: The dimension of any affine set Γ is
the dimension of its associated linear set, that is,
dim(Γ)=dim(Γ − u) for any u ∈ Γ.
Lemma 6.2 The dimension d of any affine set Γ is
equal to r − 1, where r is the maximum number of
affinely independent points in Γ.
Proof: Let V = Γ − u for any u ∈ Γ — so that
d=dim(V ) — and let v 1, . . . , v d be a basis for V . Then
from Lemma 6.1 the points u, u+v 1, . . . , u+v d are a set
of d + 1 affinely independent points in Γ, so r ≥ d + 1.
On the other hand, if p1, . . . , pr is any set of affinely
independent points in Γ, then from Lemma 6.1
p2 − p1, . . . , pr − p1 is a set of r − 1 linearly independent
points in V , and so r − 1 ≤ d. Thus r − 1 = d.
17
Affine Sets, Dimension, and Linear Systems
We can talk about an affine set Γ ⊂ ℜn in one of two ways:
• as Aff(p1, . . . , pr ) for points p1, . . . , pr in ℜn,
• as the solution to a linear system
(E) Ax = b
where A is an m × n matrix and b is an m-vector.
(Proof: Exercise.) We are interested in describing systems in either form. In particular,
Given points p1, . . . , pr , can we produce equality
system (E) whose feasible solutions are precisely Aff(p1, . . . , pr )?
Given an equality system (E), can we find points
p1, . . . , pr such that Aff(p1, . . . , pr ) is precisely
the set of points satisfying (E)?
Further, what is the minimal representation of Γ
in each case, that is, the minimum number of equations/points that represent Γ?
18
Facts from Linear Algebra
Lemma 6.3 Let A be an m × n matrix and b a column m-vector, and let Γ be the set of solutions for
the system
(E) Ax = b
Suppose that (E) is feasible, and let x̂ be an element
of L.
(a) The linear space Γ − x̂ is exactly the set of solutions to the system
Ax = 0.
That is, Γ − x̂ = null(CA).
[
]
(b) For any system Ā | b̄ row equivalent to (E),
the rows of Ā are elements of span(RA).
(c) For any set S ⊂ ℜn,
dim(span(S)) + dim(null(S)) = n,
in particular,
dim(span(RA)) + dim(null(RA)) = n,
19
Lemma 6.4 The dimension d of a nonempty affine
set Γ is equal to n−m where m the minimum number
of rows of a full row-rank equality system containing
Γ.
Proof: Let u ∈ Γ, and let V = Γ − u be the associated
linear space. Thus d=dim(V ), and let v 1, . . . , v d be a
basis for V . Then from Lemma 6.3(c) there exist m =
n−d linearly independent points w1, . . . , wm in null(V ).
Let A be the m×n matrix whose rows are the wi vectors,
and let b = Au. Then A is full row rank. Further, for
every x ∈ Γ, we have x = u + v with v ∈ V , and so
Ax = Au + Av = b + 0 = b.
Thus every element of Γ satisfies the set of equalities.
Finally suppose there were a system
Āx = b̄
where Ā has full row rank m̄ < m. By performing
Gauss-Jordan reduction on this system, we obtain tableau
[I N̂ | b̂ ]
by choosing change vectors v̂ B,1, . . . , v̂ B,n−m̄ for this system, we obtain n − m̄ + 1 > d + 1 affinely independent
points
u, u + v̂ B,1, . . . , u + v̂ B,n−m̄
in Γ, contradicting the dimension of Γ. Thus m = n − d
is the maximum number of such equalities.
20
Examples
p3 = (4, 0, 4)
x3
p2 = (2, 1, 3)
p1 = (0, 2, 2)
x1
x2
The dimension of A= Aff({p1, p3, p4} is d = 1:
• A is in n = 3-space.
• p1 and p2 are r = d + 1 affinely independent points
in A.
• All of the points in A satisfy the n − d = 2 nonredundant equations
x1
− 2x3 = −4
x2 + x3 = 4
21
Convex Sets
An convex combination of a set S = {p1, . . . , pr }
any linear combination of S for which the multipliers are nonnegative and sum to 1.
That is, a convex combination is any linear combination that is both a nonnegative combination and
an affine combination.
Example: The line segment uv between two distinct points u and v is the set of convex combinations of u and v:
uv = {λu + (1 − λ)v | 0 ≤ λ ≤ 1}
A convex set is any set C of vectors that contains
every convex combination of its elements.
Lemma 6.5 A set C is convex if and only if it contains the line segments between every two of its points.
Proof: Suppose first that C is a convex set. Since it
contains any convex combination of any of its points,
then clearly it contains the convex combination of any
two of its points, that is, it contains the line segment
between every two of its points.
22
Conversely, suppose that C contains the the line segment between every two of its points. Let p1, . . . , pr
be a set of points in S and let α1, . . . , αr be a set of
nonnegative scalars that sum to 1. We need to show
that
p̂ = α1p1 + . . . + αr pr ∈ S
If r = 1 then α1 = 1 and clearly so clearly
α1p1 = p1 ∈ S. Now proceed by induction on r ≥ 2.
Since the αi sum to 1 and there are at least two of them,
then we must have at least one αi < 1, say α1. Write


α2 2
αr r 
p̂ = α1p1 +. . .+αr pr = α1p1 +(1−α1) 
p + ... +
p
1 − α1
1 − α1
Now the point
v=
α2 2
αr r
p + ... +
p
1 − α1
1 − α1
is a convex combination of p2, . . . , pr , since
negative and
α2
1−α1
αr
+ . . . + 1−α
=
1
α2 +...+αr
1−α1
=
1−α1
1−α1
αi
1−α1
is non-
= 1.
Thus by induction v ∈ S, and letting u = p1 and λ = α1
we have
p̂ = λu + (1 − λ)v.
Thus p̂ must be in S.
23
Examples
p1 = (8, 18)
p2 = (24, 18)
Conv({p1, p2, p3, p4})
p3 = (16, 12)
(20, 11) = 16 p2 + 12 p3 + 13 p4
p4 = (24, 6)
p1
p4
p2
p3
A 3-simplex
24
More Examples of Convex Sets
A half space is one of the two regions separated by
a hyperplane Ha,β (and including Hα,β ). These are
specified as
+
H =
+
Hα,β
= {x ∈ ℜ |
n
−
H − = Hα,β
= {x ∈ ℜn |
n
∑
j=1
n
∑
j=1
αj xj ≥ β}
αj xj ≤ β}
The convex hull of a set S of points in ℜn, Conv(S),
is the set of convex combinations of S.
Example: An r-simplex is the convex hull of any
affinely independent set of r + 1 points. The standard n-simplex in ℜn+1 is the simplex made up
of the unit vectors e1, . . . , en+1, and can be represented by the system
x1 + . . . + xn+1 = 1
x ≥ 0.
25
LP Feasible Regions
Lemma 6.6 The set of feasible solutions to the LP
max z = cx
Ax = b
(P )
x ≥ 0
is a convex set.
Proof: Let x1 and x2 be two feasible points of (P ), that
is, Axi = b and xi ≥ 0, i = 1, 2, and let 0 ≤ λ ≤ 1 be
a scalar. Then clearly λx1 + (1 − λ)x2 is nonnegative,
since all of the factors are, and further
A(λx1 + (1 − λ)x2)
= λAx1 + (1 − λ)Ax2
= λb + (1 − λ)b = b
So that λx1 + (1 − λ)x2 is also feasible to (P ).
26
Historical Note
The name “simplex method” originated because the type of LP originally solved had the form
max z = cx
Ax = b
(P )
∑n
j=1 xj = 1
x ≥ 0


A
 = m+1. Then the
Where A is an m×n matrix with rank
1 ··· 1
feasible bases all have cardinality m + 1. Further, if B1 , . . . , Bm+1 is
a feasible basis for (P ), then the basic variable values xB1 , . . . , xBm+1
must satisfy
b = A.B1 xB1 + . . . + A.Bm+1 xBm+1
z = cB1 xB1 + . . . + cBm+1 xBm+1
xB1
+ ... +
xBm+1
= 1
xB1 ≥ 0, . . . ,
xBm+1 ≥ 0
Now consider the m-simplex ∆B created by the set of points


A.B1
cB1


,...,
A.Bm+1
cBm+1

.
From above, we havethat
 B is a feasible basis exactly when there is
b
a point of the form   in ∆B , in which case the final coordinate
z
z will always be equal the objective function value of the associated
basic solution.
The simplex method, therefore, involves finding that simplex ∆B
which intersects the vertical line


b
L = {  | ζ ∈ ℜ}
ζ
at the highest possible point. It does this by iteratively replacing
one point at a time in the simplex, each time raising the point of
intersection of the simplex with L, until the simplex with the highest
intersection with L is reached.
27
Convex Functions and Convex Sets
convex function: real-valued function f on ℜn such
that for each u, v ∈ ℜn and 0 ≤ λ ≤ 1,
f (λu + (1 − λ)v) ≤ λf (u) + (1 − λ)f (v),
that is, the f -value of any point on uv lies below
the corresponding value of the linear approximation of f by line segment [u, f (u)][v, f (v)].
Example: Any linear function is a convex function,
since for any function of the form f (x) = cx, and
any u, v ∈ ℜn and 0 ≤ λ ≤ 1,
f (λu + (1 − λ)v) = c(λu + (1 − λ)v)
= λcu + (1 − λ)cv = λf (u) + (1 − λ)f (v).
28
Lemma 6.7 If g is a convex function, then the set of
points satisfying the inequality g(x) ≤ b is a convex
set.
Proof: If u and v satisfy g(x) ≤ b and 0 ≤ λ ≤ 1,
then
g(λu + (1 − λ)v) ≤ λg(u) + (1 − λ)g(v)
≤ λb + (1 − λ)b = b
and so λu + (1 − λ)v likewise satisfies the inequality.
Lemma 6.8 The intersection of a collection of convex sets is also convex.
Proof: If S1, . . . , Sr are convex sets, and u, v ∈ ∩ri=1Si,
then u and v are in each Si, and so uv will be a subset
of each Si and hence uv ⊆ ∩ri=1Si.
29
Convexity and Optimization Problems
General description of an optimization problem:
(O)
min f (x)
x∈S
where f is a real-valued objective function on
ℜn and S is the feasible region of points in ℜn.
The feasible region S can often be more precisely
described in functional form by those x ∈ ℜn satisfying
gi(x) ≤ bi, i = 1, . . . r
(∗)
for set g1, . . . , gr of real-valued functions on ℜn.
Lemma 6.9 If g1, . . . , gr are convex functions, then
the region defined by (∗) is a convex region.
Proof: By Lemma 6.7, the set of points satisfying any
one of the inequalities gi(x) ≤ bi is convex, so by Lemma
6.8 the set of points satisfying all of the constraints will
likewise be convex.
Example: The feasible region of an LP is convex, since
it can be defined by a set of linear inequalities.
30
Convexity and Local Minima
local minimum: A point x̂ is a local minimum
for (O) if x̂ ∈ S and there is a δ > 0 such that
no point in S within distance δ of x̂ has smaller
objective function than x̂.
Theorem 6.1 If S is a convex set and f is a convex
function, then every local optimal minimum for (O)
is also a global minimum.
Proof: Let x̂ be local minimum with objective function
value ẑ = f (x̂), and suppose x̂ is not a global minimum.
Let x∗ be a global minimum, with objective function
value z∗ = f (x∗) < ẑ. Since S is convex, then every
point on the line x∗x̂ is also in S. Further, since f is
convex, then each point λx∗ + (1 − λ)x̂ ̸= x̂ on this line
(λ > 0) has objective function value
f (λx∗ + (1 − λ)x̂) ≤ λf (x∗) + (1 − λ)f (x̂)
= λz∗ + (1 − λ)ẑ
= ẑ + λ(z∗ − ẑ) < ẑ
so x̂ cannot be a local minimum.
This justifies “local improvement” methods, such as the
simplex method, since when no local improvement is
possible, the current solution is optimal.
31
Concavity and Maximization Problems
concave function: real-valued function f on ℜn such
that −f is convex, that is, for each u, v ∈ ℜn and
0 ≤ λ ≤ 1,
f (λu + (1 − λ)v) ≥ λf (u) + (1 − λ)f (v)
Facts:
• If g is convex, then the set of points satisfying
g(x) ≥ b is a convex set, and any set of inequalities of this sort define a convex set.
• Linear functions are concave as well as convex,
and so for example canonical minimization problems also have convex feasible regions.
• If f is concave and S is convex, then any local
maximum for the optimization problem
max f (x)
x∈S
will be a global maximum.
32
The Dimension of a Convex Set
The dimension of a convex set C is its affine dimension, that is, dim(C) equals one less than the maximum number of affinely independent points in C.
Examples:
• A singleton point has dimension 0
• A line segment has dimension 1
• An m-simplex has dimension m
• The dimension the convex hull of the six points
p1
p2
p3
p4
p5
p6
1
2
3
4
5
6
4
3
2
2
1
0
15
15
15
14
14
14
1
2
3
1
2
3
is equal to 2, since the maximum number of
affinely independent points in this set is 3.
33
Lemma 6.10 Suppose the m × n LP
min z = cx
Ax = b
(P )
x ≥ 0
has at least one basic nondegenerate tableau. Then
the feasible region to (P ) has dimension n − m.
Proof: Let (B, N ) be the basic/nonbasic partition of
the components of x in a nondegenerate tableau for (P ),
and let x̂ be the associated basic solution. Since this
tableau is nondegenerate all of the RHS values are positive, and thus for any entering variable all of the minimum ratios ∆∗ will be positive. Let δ > 0 be chosen
to be smaller than any of these minimum ratios. Then
for each nonbasic column Nj we have that the solution
xδ = x̂ + δv B,N1 will be feasible. It follows that
x̂, x̂ + δv B,N1 , . . . , x̂ + δv B,Nn−m
is a set of n − m + 1 affinely independent feasible points,
and so the feasible region for (P ) has dimension n − m.
34
Example
Consider the (nondegenerate) optimal vertex (12, 2, 0)
of Woody’s LP. The associated tableau is
basis
x5
x2
x1
z
z
0
0
0
1
x1
0
0
1
0
x2 x3
x4 x5 x6 rhs
0 −10 15/4 1 −10 30
1
2 −1/4 0 2/3 2
0 −1 1/2 0 −1 12
0 10 5/2 0
5 540
The 3 change vectors associated with the three nonbasic
variables x3, x4, and x6 are

v1 =













1
−2
1
0
10
0




























v2 =
−1/2
1/4
0
1
−15/4
0




























v3 =
1
−2/3
0
0
10
1














and so the 4 affinely independent points that determine
the dimension of the polytope (using, say, δ = 1) are

x̂ =















12 



2 




0 
1

 , x̂ + v = 

0 




30 

0




13 
11 21 




1 

 2 
0 
4 









0
1 
3
2



 , x̂ + v = 
 , x̂ + v = 



1
0 






1 

 26 
40 
4 


0
0
35
13
1 31
0
1
40
1














Polyhedra and Polytopes
polyhedron: the intersection of a finite set of linear
equalities and inequalities.
polytope: bounded polyhedron, that is, for which the
component values are bounded.
Example: The feasible region of any LP is a polyhedron.
extreme point of polyhedron P : point x ∈ P such
that every line segment uv in P containing x will
have either x = u or x = v.
Theorem 6.2 The set of basic feasible solutions for
any linear program is exactly the set of extreme points
for the associated polyhedron.
Proof: Exercise.
36
Polytopes and Convex Hulls
Theorem 6.3 (“Minkowski”) A polytope is the convex hull of its extreme points.
Theorem 6.4 (“Weyl”): The convex hull of any
set of points in ℜn is a polytope in ℜn.
(The proofs of these results are technical, and will not
be presented here.)
Theorem 6.3 can be generalized to unbounded polyhedron, but it involves describing the “linear space”
and “unbounded cone” for P , and so will also not
be presented here. Murty Section 3.7 has a complete treatment of this.
Theorem 6.4 is a key result in the study and solution
of combinatorial problems that can be represented
by linear programs.
37
Faces of Polyhedra
Let P be a polyhedron.
supporting hyperplane for P : Hyperplane H such
that every point P is contained in the half space
H +.
face of P : subset of points of P that lie on some supporting hyperplane for P .
Some special faces:
P itself and the empty set: P is a face with
supporting hyperplane H0,0, and ∅ is a face
with any Hα,β not intersecting P . They are
sometimes called improper faces to distinguish them from the other faces.
vertices: 0-dimensional (singleton) faces.
edges: 1-dimensional faces.
facets: faces of dimension dim(P )-1 (maximum
dimensional proper faces).
38
Example
Consider the feasible region P for Woody’s 3-variable
LP, defined by the inequalities
8x1 + 12x2 + 16x3 ≤ 120
15x2 + 20x3 ≤ 60
3x1 + 6x2 + 9x3 ≤ 48
x1
≥ 0
x2
≥ 0
x3 ≥ 0
The following equalities define supporting hyperplanes for P (P ⊂ Hi−):
H0 : 35x1 + 60x2 + 75x3 = 540
H1 : 11x1 + 18x2 + 25x3 = 168
H2 : 2x1 + 3x2 + 4x3 = 30
and the associated faces of (P ) are
F0 = the vertex (12,2,0)
F1 = the edge between (12,2,0) and (13,0,1)
F2 = the facet which is the convex hull of the
points (12,2,0), (13,0,1), and (15,0,0)
Fact: Any face of polyhedron P will itself be a polyhedron.
39
4
(0,0,3)
3
x2
5
2
4
x3
3
1
(0,4,0)
2
1
(0,0,0)0
0
(7,0,3)
5
(8,4,0)
x1
10
(12,2,0)
(13,0,1)
15
40
(15,0,0)
Theorem 6.5 The set of optimal solutions for a linear program (L) is a face of the feasible region for
(L).
Proof: Let z∗ be the maximum objective function value
for (L). Then every feasible point satisfies cx ≤ z∗,
and so Hc,z∗ will be a supporting hyperplane for this set
−
(P ⊆ Hc,z
).
∗
Theorem 6.6 Any BFS for a linear program (L) is
a vertex of P .
Proof: Let x̂ = [x̂B , x̂N ] = [b̄, 0̄] be a BFS for (L).
Consider the linear inequality
∑
xj ∈xN
xj ≥ 0.
(∗)
Since xj must be nonnegative in any feasible solution,
then every point of P satisfies (∗), so that (∗) is a supporting inequality. Further, the only solutions that satisfy (∗) at equality must have every element of xN equal
to 0, and it follows from the equivalent linear representation xB = b̄ − N̄ xN that x̂ is the only such solution.
Theorem 6.7 For adjacent BFS u and v, the line
segment uv is an edge of P .
Proof: Exercise.
41
Solving Combinatorial Problems Using the
Convex Hull
Combinatorial optimization problems often involve maximizing a linear function over a set S:
max f (x) = cx
(CO)
x∈S
where S is a finite set of points in ℜn representing the
set of solutions that are allowed. In particular, since S
if finite, the assumption of continuity for S does not hold.
Question: Can we use linear programming to solve
this problem?
Answer: Yes, if we can describe S by a system (LP )
of linear equalities and inequalities in such a way
that the optimal solution for the associated LP is
the same as that for (CO).
42
Key Tool: Weyl’s Theorem (6.4), which states that
conv(S) can be represented as a polytope, that is, as
the feasible region for a system of linear equalities and
inequalities.
Lemma 6.11 The extreme points of conv(S) are all
contained in S.
Proof Let p = ∑ri=1 λipi be a convex combination of
points pi ∈ S that is an extreme point of conv(S), and
let r be the smallest number of points for which p can
be expressed as a convex combination. Then 0 < λi < 1
for all λi. If r = 1 then p ∈ S. Otherwise let
λi i
p =
p
i=1 1 − λr
′
r−1
∑
Then p′ ∈conv(S), since it is a combination of points in
S whose multipliers are positive and sum to 1. But now
p = λr pr + (1 − λr )p′.
This means p is a convex combination of pr and p′, so
that by the definition of extreme point p must be equal
to pr or p′. But this means that p can be represented
by a smaller number of points of S, contradicting the
choice of points given above. Therefore p is in S.
43
Theorem 6.8 The basic feasible optimal solutions
to the linear program
(CP )
max cx
x ∈ conv(S)
are optimal to (CO).
Proof By Theorem 6.2 the basic optimal solutions to
(CP ) are extreme points of convS, and by Lemma 6.11
these are in turn in S. But since S ⊂ conv(S), they
then must be optimal to (CO)
Solving combinatorial optimization problem by linear
programming, therefore, requires that you obtain the
description of conv(S) by means of linear equalities and
inequalities. For hard combinatorial problems this is
generally also a difficult problem. For special classes
of combinatorial problems, such as the Traveling Salesperson Problem, however, collections of inequalities have
been found that describe conv(S) fairly accurately. This
allows these problems to be solved much more quickly
than with other methods.
44