Download - Free Documents

A Collection of Important Theorems Theorem The Fritz John Theorem S and B Theorem .
Suppose that f, g , . . .
, gk are C functions of n variables. Suppose that x is a local maximizer of f on the constraint
set dened by the k inequalities g x , . . . , xn b , . . . , gk x , . . . , gk bk Form the Lagrangian
Lx , . . . , xn , , , . . . , k f x g x b . . . k gk x bk with a multiplier for the objective function. Then,
there exist multipliers , , . . . , such that k .
L x x , L , . . . , xn x , ,
. g x b , . . . , gk x bk , k . , . . . , , k . g x b , . . . , gk x bk , . or , . , , . . . , , , . . . , . k
ym . . . . . . xn dened on a ball B about x such that F f x. . ym . xn cm as possibly dening y . If
the determinant of the m m matrix F y . . . . . x is a solution of the above system of
equations.. . . x . . . x . xn . . then there exist C functions y f x . . x n . . . ym . x . . . . . x . . Let
F . . . . . . . . . . . . . . . . . . . x is nonzero. xn c . F ym F . ym fm x . . . . . . ym as implicit
functions of x . . . Fm y . Fm ym evaluated at y . . . . .Theorem The Implicit Function Theorem
S and B Theorem . xn c .. . . . . . . . x n . . fm x. . xn cm for all x x . . . yh . . . . . . . . Fm f x.
Consider the system of equations F y . . Fn Rmn R be C functions. xn . . . . Suppose that y . .
.. . . Fm y . . . . . . . Fm y . Fh . . . . xn in B and y f x . . .. . . . . . . . . . . . . . . . fm x.. . . . ym fm x .
...
.Theorem Cramers Rule S and B Theorem . . the j. Then. . . We will make the following
denition for any n n matrix A. Cramers Rule the unique solution x x . . for i . that is ij times the
determinant of the submatrix obtained by deleting row i and column j from A. A detA adjA . n
detA where Bi is the matrix A with the righthand side b replacing the ith column of A. . . jth
entry is Cij . ith cofactor of A note the switch in indices. for a nonsingular matrix A. . is called
the adjoint of A and is written adj A. The n n matrix whose i. . xn of the n n system Ax b is xi
detBi . jth cofactor of A. . let Cij denote the i.
negative semidenite if x Ax for all x Rn . then A is .Denition Matrix Deniteness Let A be an n
n symmetric matrix. negative denite if x Ax lt for all x Rn . positive denite if x Ax gt for all x Rn
. positive semidenite if x Ax for all x Rn . indenite if x Ax gt for some x Rn and x Ax lt for some
other x Rn .
and x is a strict global max of Q on this constraint set. then Q is indenite on the constraint
set Bx and x is neither a max or a min of Q on this constraint set. and b. . construct the n m n
m symmetric matrix H by bordering the matrix A above and to the left by the coecients B of
the linear constraints H B B Check the signs of the last n m leading principal minors of H.
then Q is positive denite on the constraint set Bx and x is a strict global min of Q on this
constraint set. . . If det H has the same sign as n and if these last n m leading principal
minors alternate in sign. If det H and these last n m leading principal minors all have the
same sign as m . starting with the determinant of H itself. are violated by nonzero leading
principal minors.Theorem Deniteness of Quadratic Forms S and B . To determine the
deniteness of a quadratic form Qx x Ax when restricted to a constraint set given by m linear
equations Bx . then Q is negative denite on the constraint set Bx . . If both of these conditions
a.
gt . .Part III S and B Theorem . Theorem SecondOrder Conditions on Critical Points .
Suppose that F x for i . it is neither a local max nor a local min.Theorem SecondOrder
Conditions on Critical Points . . . . . lt . . Then x is a strict local max of F. Fx x Fx x Fx x Fx x
gt . n xi and that the n leading principal minors of D F x are all positive Fx x Fx x Fx x Fx x Fx
x Fx x Fx x Fx x Fx x Fx x gt . .. .Part I S and B Theorem . . . n xi and that the n leading
principal minors of D F x alternate in sign Fx x Fx x Fx x Fx x Fx x Fx x Fx x Fx x Fx x Fx x lt .
. Theorem SecondOrder Conditions on Critical Points Part II S and B Theorem . Let F U R be
a C function whose domain is an open set U in R . Let F U R be a C function whose domain
is an open set U in Rn . at x . . Then x is a strict local min of F. Let F U R be a C function
whose domain is an open set U in Rn . at x . . . Suppose that F x for i . . . . and . . Suppose
that F x for i . Then x is a saddle point of F . Fx x Fx x Fx x Fx x gt . . n xi and that some
nonzero leading principal minors of D F x violate the sign patters of the hypotheses of the
previous two theorems SampB .
is a critical point of the Lagrangian function Lx . Let f and h be C functions. x . x . f x . and L/x
and .Theorem Firstorder Conditions for Constrained Optimization with Equality Constraints S
and B Theorem . L/x L/ . x c Suppose further that x . each of two variables. x hx . x is not a
critical point of h. x c In other words. x is a solution of the problem maximize f x . at x . x
subject to hx . Suppose that x x . x . Then there is a real number such that x . .
y . y . . y gx. y b. y b . y b Then there is a multiplier such that . suppose that either g g x . x . .
L y x . . y x y In any case. y maximizes f on the constraint set gx. f x. . . Suppose that f and g
are C functions on R and that x . y b. . y or x . . L x x . y. . y .Theorem Maximization in Two
Variables with Inequality Constraints S and B Theorem . If gx . gx . form the Lagrangian
function Lx.
. . . . . g x b . . . . . . . . k gk x bk Then there exist multipliers . . The rank at x of the Jacobian
matrix of the binding constraints g g . . . . xn b . . . . . . . . . xn x x x . . .. . L . gk x bk . g . xn . .
xn x .as large as it can be. k f x g x b . . . . . . gk x . L x x . . gk gk x . . . . Suppose that f. .
assume that the rst k constraints are binding at x and that the last k k constraints are not
binding. Suppose that x Rn is a local maximizer of f on the constraint set dened by the k
inequalities g x . xn x x is k . gk are C functions of n variables. .Theorem Optimization with
Many Inequality Constraints S and B . . k . . such that k . gk x bk . k . . . . . xn bk For ease of
notation. . Suppose that the following nondegenerate constraint qualication is satised at x . . .
. . . . . . g x b . . . . Form the Lagrangian Lx . . . . . .
xn b . .. . . . . . . . . . . . hn x x x is k m. as large as it can be.. .k gk xbk h xc . Form the
Lagrangian Lx . xn . . . xn bk h x . . . x x n . . . L . gk x bk . . . . . .m hm xcm . . . . . . . . . . . . .
xn x x x . such that m k . . Then there exist multiplies . xn c . . . . . . . Suppose that the
following nondegenerate constraints qualication is satised at x . . . . . . Suppose that f. hm m
x . . hm x . . . xn x h h x x . . . . . xn x . . hm C functions of n variables. . gk gk x x . . . . . . . . . .
g x b . . . k . . . gk x bk . . . . . . . . L x x . . . Suppose that x Rn is a local maximizer of f on the
constraint set dened by the k inequalities and m equalities g x . . . . hm x cm . .Theorem
Maximization with Mixed Equality and Inequality Constraints S and B Theorem . gk . . . . k . . .
g . . . h x c . . . . . . . gk x . m f x g xb . . xn cm Without loss of generality. . . g x b . . .the rank
at x of the Jacobian matrix of the equality constraints and the binding inequality constraints g
g . . . . . we may assume that the rst k inequality constraints are binding at x and the other k k
inequality constraints are not binding. . . h . k .
. . . Suppose that the following nondegenerate constraints qualication is satised at x . . . .
Suppose that x Rn is a local minimizer of f on the constraint set dened by the k inequalities
and m equalities g x . . . . . .Theorem Minimization with Mixed Equality and Inequality
Constraints S and B Theorem . . . L x x . . . . h . . . xn x x x . xn c . . Suppose that f. . . g x b . .
. hm m x .m hm xcm . we may assume that the rst k inequality constraints are binding at x
and the other k k inequality constraints are not binding. k . hm x cm . . . . k . . g x b . . . xn cm
Without loss of generality. . . . . . . xn bk h x . . . . . . . . . . . . . . . . . gk gk x x . . . xn x h h x x .
xn b . . . . . . . .. gk x . .. . . . . hn x x x is k m. . . . . xn . . gk x bk . . . . . . . xn x . . . . . such that
m k . . . . hm C functions of n variables. . g . Then there exist multiplies . as large as it can be.
. . L . h x c . . hm x . . gk x bk . . gk . .the rank at x of the Jacobian matrix of the equality
constraints and the binding inequality constraints g g . . . . . . . . m f x g xb . . . . Form the
Lagrangian Lx . k . . . . . . .k gk xbk h xc . x x n . . .
For each choice of the parameter a. . hk x. k a are C functions of a and that nondegenerate
constraint qualication holds. . d L f x a. a . da a Theorem The Envelope Theorem in
Constrained Problems S and B Theorem . . . a. hk Rn R R be C functions. . . . Let x a be a
solution of this problem. x a denote the solution of the problem of maximizing x n f x. . d f x a.
a be a C function of x Rn and a scalar a. consider the unconstrained maximization problem
maximize f x. Suppose that x a is a C function of a. a for any xed choice of the parameter a.
da a where L is the natural Lagrangian for this problem. . Let f. Let x a x a. Then. Suppose
that x a and the Lagrange multipliers a. a x a. . . . h . a with respect to x. a on the constraint
set h x. . . Then. . a. . . a. Let f x. a f x a.Theorem The Envelope Theorem in Unconstrained
Problems S and B Theorem .
x xn k at x . such that k L L L L . . . . There exist . . hk x ck . . v lt . . . .. . that is. . . . Form the
Lagrangian and suppose that . . . . . . Then x is a strict local constrained max of f on Ch . . Dx
Lx . . x lies on the constraint set Ch . Let f. h . . . . is negative denite on the linear constraint
set v Dhx v .Theorem SecondOrder Conditions in Constrained Optimization S and B
Theorem . The Hessian of L with respect to x at x . . . . x . n k . . . Consider the problem of
maximizing f on the constraint set Ch x h x c . . gk be C functions on Rn . . . . . . . . v and Dhx
v Dx Lx .
y . Let f and h be C functions on R . y hx. Form the Lagrangian Lx. y c Suppose that x . h gt
at x . satises . f x. y . y c. y h x h L . y . det x x . y. Consider the problem of maximizing f on
the constraint set Ch x. L . y is a local maximum of f on Ch . .Theorem Secondorder
Conditions for a Simple Lagrangian S and B Theorem . y L xy L y Then x . y hx. L x h y L yx
L at x .
Let f U R be a C concave convex function on U . If the determinant above is negative for all
x. Suppose that f is monotone in that fx gt and fy gt on W. y W . or if x is an interior point of U
.Quasiconcavity/Quasiconvexity S and B Theorem . y W . Let f be a C function on an interval
I in R.Concavity/Convexity S and B Theorem . y I Denition . y W . Then x is a global max of f
on U if and only if Df x x x for all x U . then x is a global max min of f on U if and only if Df x .
if U is open. Then f is concave on I if and only if f y f x f xy x for all x. if f is quasiconcave on
W . y I The function f is convex if and only if f y f x f xy x for all x. Let U be a convex subset of
Rn . Theorem Unconstrained Concave Programming S and B Theorem . In particular. If the
determinant fx fy fx fxx fyx fy fxy fyy is gt for all x. then the determinant above is . Conversely.
.Denition . Let f be a C function on a convex set W in R . then f is quasiconcave on W. if f is
quasiconvex on W the the determinant is for all x. then f is quasiconvex on W .
. . . Consider the programming problem maximize f x subject to x Cb U gi x bi . . k then x is a
global max of f on the constraint set Cb . . k Suppose that one of the constraint qualications
from theorem . . Let g . gi x bi . k f x i i gi x bi If there exist x and such that L x . . f
quasiconcave with nonvanishing gradient. . . . gk U R be C quasiconvex functions. i . for
example. . . . . Let U be a convex open subset of Rn . . Let f U R be a C pseudoconcave
function on U . . . holds. . . xj and i . for j . i gi x bi . for i . . n. . Form the Lagrangian k Lx. .
.Theorem Concave Programming with Constraints S and B Theorem .
. . . gk be C functions of n variables and suppose that x Rn is a local maximizer of f on the
constraint set dened by g x b . k and all t . . . i . Let f. . . . . n has maximal rank j h. NDCQ . .
gh yield binding constraints at x and that gh . . . xyn xn x xn Similarly f is convex on U if and
only if f y f x Df xy x for all x. Rn such that a x . h. gh are concave functions. y U f y f x Df xx y
or in other words f y f x f f xy x . Then f is concave on U if and only if for all x. . g . gk are not
binding at x . . . .Theorem Other Forms of Constraint Qualication S and B Theorem . h there
exists gt and a C curve . Then we can take in the conclusion of the Fritz John Theorem. . . .
KarushKuhnTucker For any vector v Rn with the property that Dgi x v for i . g . . gk x bk For
ease of notation. suppose that g . Suppose that the binding constraint functions satisfy one of
the following properties . b v. .NonDegenerate Constraint Qualication the h n Jag cobian
matrix xi x . . and c gi t bi for all i . . j . . . . . . . gh are convex functions on U and there exists z
U so that each gi z lt bi . . . . . g . Let f be a C function on a convex subset U of Rn . . . . . . . . .
. Slater CQ There is a ball U about x Rn such that g . . . . . . . . . gh are linear functions.
Denition Convexity on Subsets of Rn S and B Theorem . . . y U .
Theorem SecondOrder Conditions for Concavity and Convexity S and B Theorem . Let f be a
C function on an open convex subset U of Rn . The function f is a convex function on U if and
only if D f x is positive semidenite for all x U . . Then f is a concave function on U if and only if
the Hessian D f x is negative semidenite for all x U .