Download NOTES WEEK 04 DAY 2 DEFINITION 0.1. Let V be a vector space

NOTES WEEK 04 DAY 2
SCOT ADAMS
DEFINITION 0.1. Let V be a vector space and let S be a set and
let f : V 99K S and let a P V . Then f pa ` ‚q : pdomrf sq ´ a Ñ S is
defined by pf pa ` ‚qqpvq “ f pa ` vq.
Recall: for any set S, for any t, the constant function CSt : S Ñ ttu
is defined by CSt psq “ t.
DEFINITION 0.2. Let S and T be sets. Then CpS, T q :“ tCSt | t P T u
denotes the set of all constant functions S Ñ T .
Let m, n P N. We set up a dictionary
LpRm , Rn q
Rnˆm ,
Ø
with vector space isomorphisms L ÞÑ rLs and LA Ð[ A.
Let k P N and let j1 , . . . , jk P N. Let S be a set. Let K :“ t1, . . . , ku.
For all ` P K, let J` :“ t1, . . . , jk u. Then we define
S j1 ˆj2 ˆ¨¨¨ˆjk
:“
S J1 ˆJ2 ˆ¨¨¨ˆJk ;
that is, S j1 ˆj2 ˆ¨¨¨ˆjk is the set of functions
J1 ˆ J2 ˆ ¨ ¨ ¨ ˆ Jk
Ñ
S.
Using this notation, for example, C4ˆ5 is the set of 4 ˆ 5 matrices
with complex entries. For any d P N, a “d-tensor” is a d-dimensional
array of objects. (Also, by “0-tensor”, one typically means “scalar”.
So, in this course, a 0-tensor is a real number.) With this terminology,
C4ˆ5ˆ6ˆ7 is the set of 4 ˆ 5 ˆ 6 ˆ 7 tensors with complex entries. Since
this is a four-dimensional array, to picture it on a blackboard, one
would have to “flatten” it, and there are various procedures for doing
this. For example, there’s a natural bijection C4ˆ5ˆ6ˆ7 Ø pC4ˆ5 q6ˆ7 ,
and this allows us to picture an element of C4ˆ5ˆ6ˆ7 as a 6 ˆ 7 matrix,
each entry of which is a 4 ˆ 5 complex matrix. With great patience,
such a matrix of matrices could be written out on a blackboard.
Date: February 9, 2017
Printout date: March 14, 2017.
1
2
SCOT ADAMS
Traditionally, 0-tensors are called “scalars”, 1-tensors are called “vectors”, and 2-tensors are called “matrices”. Beyond two dimensions, one
simply says “tensor”. Some matrices are symmetric, and the idea of
symmetry generalizes in various ways:
Let k P N and let j1 , . . . , jk P N. Let S be a set. Let K :“ t1, . . . , ku.
For all ` P K, let J` :“ t1, . . . , jk u. Let I Ď K. Assume, for all `, m P I,
that j` “ jm . Let Σ denote the set of all bijections σ : K ĂÑą K such
that, @` P KzI, σp`q “ `. For all σ P Σ, for all r P J1 ˆ¨ ¨ ¨ˆJk , we define
σ 1 : J1 ˆ ¨ ¨ ¨ ˆ Jk Ñ J1 ˆ ¨ ¨ ¨ ˆ Jk by σ 1 pn1 , . . . , nk q “ pnσp1q , . . . , nσpkq q.
j1 ˆj2 ˆ¨¨¨ˆjk
denotes the set of all A P S j1 ˆj2 ˆ¨¨¨ˆjk such that, for
Then SsympIq
all r P J1 ˆ ¨ ¨ ¨ ˆ Jk , for all σ P Σ, we have Ar “ Aσ1 prq .
4ˆ4
is the set of symmetric
So, using this notation, for example, Csympt1,2uq
4ˆ4 matrices with complex entries. We’ll usually dispense with the “t”
4ˆ4
4ˆ5ˆ4ˆ6
and “u”, and simply write Csymp1,2q
. For another example, Csymp1,3q
will
4ˆ5ˆ4ˆ6
be the set of all A P C
such that, @i P t1, . . . , 4u, @j P t1, . . . , 5u,
@k P t1, . . . , 4u, @` P t1, . . . , 6u, we have Aijk` “ Akji` .
Let m, n P N. We set up a dictionary
BpRm , Rm , Rn q
Ø
Rmˆmˆn ,
with vector space isomorphisms B ÞÑ rBs and BA Ð[ A. This restricts
mˆmˆn
. Even more, we get
to SBFpRm , Rn q Ø Rsymp1,2q
QpRm , Rn q
Ø
SBFpRm , Rn q
Ø
mˆmˆn
Rsymp1,2q
,
with vector space isomorphisms Q ÞÑ BQ and B ÞÑ rBs and QB Ð[ B
mˆmˆn
and BA Ð[ A. The composite is QpRm , Rn q Ø Rsymp1,2q
, with vector
space isomorphisms Q ÞÑ rQs and QA Ð[ A.
In the last paragraph, if n “ 1, then we can get a simplification
because there’s a vector space isomorphism Rmˆmˆ1 Ø Rmˆm , and anmˆm
other Rmˆmˆ1
symp1,2q Ø Rsym . So, if n “ 1, then, at the quadratic level, we
can work with matrices and symmetric matrices, and avoid higher dimensional tensors. Remember that symmetric matrices can be studied
through the Finite Dimensional Spectral Theorem.
Let m, n P N. We set up a dictionary
T pRm , Rm , Rn q
Ø
Rmˆmˆmˆn ,
NOTES WEEK 04 DAY 2
3
with vector space isomorphisms T ÞÑ rT s and TA Ð[ A. This restricts
mˆmˆmˆn
to STFpRm , Rn q Ø Rsymp1,2,3q
. Let CupRm , Rn q denote the set of homogeneous cubic functions Rm Ñ Rn . Then we get even more:
CupRm , Rn q
Ø
STFpRm , Rn q
Ø
mˆmˆmˆn
Rsymp1,2,3q
,
with vector space isomorphisms K ÞÑ TK and T ÞÑ rT s and KT Ð[ T
mˆmˆmˆn
and TA Ð[ A. The composite is CupRm , Rn q Ø Rsymp1,2,3q
, with vector
space isomorphisms K ÞÑ rKs and KA Ð[ A.
NOTE TO SELF: Think about using K instead of Cu for “cubic”.
Also, think about replacing Rmˆmˆmˆn , above, with Rnˆmˆmˆm . Also,
think about replacing Rmˆmˆn , above, with Rnˆmˆm .
We leave it to the reader to formulate the appropriate definition
of “k-multilinear” in the next definition.
DEFINITION 0.3. Let k P N and let V1 , . . . , Vk be vector spaces.
For any vector space W , the notation Mk pV1 , . . . , Vk , W q will denote
the set of all k-multilinear maps V1 ˆ ¨ ¨ ¨ ˆ Vk Ñ W . Also, we define
Mk pV1 , . . . , Vk q :“ Mk pV1 , . . . , Vk , Rq.
Then @vector spaces V, W , we have LpV, W q “ M1 pV, W q. Also,
@vector spaces V, W, X, we have BpV, W, Xq “ M2 pV, W, Xq. Also,
@vector spaces V, W, X, Y , we have T pV, W, X, Y q “ M3 pV, W, X, Y q.
NOTE TO SELF: Think about SMFk instead of SMk in next defn.
We leave it to the reader to formulate the appropriate definition
of “symmetric” in the next definition.
DEFINITION 0.4. Let k P N and let V be a vector space. For
any vector space W , the notation SMk pV, W q will denote the set of
all F P Mk pV, . . . , V, W q such that F : V ˆ ¨ ¨ ¨ ˆ V Ñ W is symmetric.
Also, we define SMk pV q :“ SMk pV, Rq.
Then @vector spaces V, W , we have LpV, W q “ SM1 pV, W q and
SBFpV, W q “ SM2 pV, W q and STFpV, W q “ SM3 pV, W q.
DEFINITION 0.5. Let k P N. Let V and W be vector spaces. Let
F P SMk pV, W q. Define PF : V Ñ W by PF pvq “ F pv, v, . . . , vq.
Let V, W be vector spaces. Then @L P LpV, W q, PL “ L. Also,
@B P SBFpV, W q, PB “ QB . Also, @T P STFpV, W q, PT “ KT .
The map PF is sometimes called the “diagonal restriction” of F ,
even though, strictly speaking, PF isn’t a restriction of F , because
4
SCOT ADAMS
domrPF s “ V is not a subset of domrF s “ V ˆ V ˆ ¨ ¨ ¨ ˆ V “ V k . Still,
each element v P V can be easily confused with a diagonal element
pv, . . . , vq P V k . So we can picture V as if it were sitting inside V k ,
along the set tpv, . . . , vq | v P V u, which we call the “diagonal” in V k .
DEFINITION 0.6. Let k P N and let V be a vector space. For any
vector space W , we define Pk pV, W q :“ tPF | F P SMk pV, W qu. Also,
we define Pk pV q :“ Pk pV, Rq.
Then @vector spaces V, W , we have LpV, W q “ P1 pV, W q and we
have QpV, W q “ P2 pV, W q and we have CupV, W q “ P3 pV, W q.
Let V be a vector space and let k P N. For any vector space W , elements of Pk pV, W q are called W -valued homogeneous polynomials
on V of degree “ k. Elements of Pk pV q are called homogeneous
polynomials on V of degree “ k, or, sometimes, real-valued homogeneous polynomials on V of degree “ k.
DEFINITION 0.7. Let V be a vector space. For any vector space W ,
we define P0 pV, W q :“ CpV, W q. Also, we define P0 pV q :“ P0 pV, Rq.
DEFINITION 0.8. Let k P N and let V be a vector space. For any
vector space W , we define Pďk pV, W q :“ rP0 pV, W qs`¨ ¨ ¨`rPk pV, W qs.
Also, we define Pďk pV q :“ Pďk pV, Rq.
We now describe THE BIG ALGEBRAIC IDEA: Let k, m, n P N.
In this paragraph, in the notations “Rm , . . . , Rm ” and “m ˆ ¨ ¨ ¨ ˆ m”,
there will always be k copies of “m”. We set up a dictionary
Mk pRm , . . . , Rm , Rn q
Ø
Rmˆ¨¨¨ˆmˆn ,
with vector space isomorphisms F ÞÑ rF s and FA Ð[ A. This restricts
mˆ¨¨¨ˆmˆn
to SMk pRm , Rn q Ø Rsymp1,...,kq
. Even more, we get
Pk pRm , Rn q
Ø
SMk pRm , Rn q
Ø
Rmˆ¨¨¨ˆmˆn
symp1,...,kq ,
with vector space isomorphisms P ÞÑ FP and F ÞÑ rF s and PF Ð[ F
and FA Ð[ A. The composite is Pk pRm , Rn q Ø Rmˆ¨¨¨ˆmˆn
symp1,...,kq , with vector
space isomorphisms P ÞÑ rP s and PA Ð[ A. Thus ends THE BIG
ALGEBRAIC IDEA.
We now aim for THE BIG CALCULUS IDEA, but we first need
some preliminary definitions.
Recall, for all k P N, that 0k :“ p0, . . . , 0q P Rk .
NOTES WEEK 04 DAY 2
5
DEFINITION 0.9. @n P N, @x P Rn , let |x|2 :“
DEFINITION 0.10. @m, n P N, let
such that
be the set of ε : Rm 99K Rn
Omn
‚ domrεs is a nbd of 0m in Rm ,
‚ ε is continuous at 0m
and
DEFINITION 0.11. @m, n P N, let
such that
εp0m q “ 0n .
be the set of ε : Rm 99K Rn
ˆ
Omn
‚ domrεs is a pnbd of 0m in Rm
‚ ε Ñ 0n near 0m .
DEFINITION 0.12. @m, n P N, let
that 0n R imrεs.
a
x21 ` ¨ ¨ ¨ ` x2n .
and
ˆˆ
Omn
be the set of ε P
ˆ
Omn
such
DEFINITION 0.13. @m, n P N, let Omn be the set of α : Rm 99K Rn
such that
‚ domrαs is a nbd of 0m in Rm
and
m
‚ Dnbd U of 0m in R such that α˚ pU q is bounded in Rn .
ˆ
DEFINITION 0.14. @m, n P N, let Omn
be the set of α : Rm 99K Rn
such that
‚ domrαs is a pnbd of 0m in Rm
and
m
‚ Dpnbd P of 0m in R s.t. α˚ pP q is bounded in Rn .
ˆˆ
ˆ
DEFINITION 0.15. @m, n P N, let Omn
be the set of α P Omn
s.t. 0 R imrαs.
For all p ą 0, let | ‚ |p2 : Rn Ñ R be defined by p| ‚ |p2 qpxq “ |x|p2 .
DEFINITION 0.16. @m, n P N, @p ą 0, define
Omn ppq
:“ pOmn q ¨ p| ‚ |p2 q,
ˆ
Omn ppq
p
:“ pOˆ
mn q ¨ p| ‚ |2 q
ˆˆ
Omn ppq
:“ pOˆˆ
mn q ¨ p| ‚
and
|p2 q.
DEFINITION 0.17. @m, n P N, @p ą 0, define
Omn ppq :“ pOmn q ¨ p| ‚ |p2 q,
ˆ
ˆ
Omn
ppq :“ pOmn
q ¨ p| ‚ |p2 q
ˆˆ
ˆˆ
Omn
ppq :“ pOmn
q ¨ p| ‚ |p2 q.
and
6
SCOT ADAMS
In all of the above definitions, if m “ 1 or n “ 1, then we replace R1
by R and 01 “ p0q by 0. So, for example O11 “ O is the set of partial
functions ε : R 99K R such that
‚ domrεs is a nbd of 0,
‚ ε is continuous at 0
and
εp0q “ 0.
Let m, n P N. Then, for all p ą 0, we have Omn ppq Ď Omn and
ˆ
ˆˆ
ˆˆ
Omn
ppq Ď Oˆ
mn and Omn ppq Ď Omn . Thus, for all p, q ą 0, we have
Omn pp ` qq Ď
Omn pqq,
ˆ
pp
Omn
ˆˆ
pp
Omn
` qq Ď
ˆ
Omn pqq,
` qq Ď
ˆˆ
Omn pqq.
and
ˆ
ˆˆ
ˆˆ
Let m, n P N. Then Omn Ď Omn and Oˆ
mn Ď Omn and Omn Ď Omn .
ˆ
Thus, for all p ą 0, we have Omn ppq Ď Omn ppq and Oˆ
mn ppq Ď Omn ppq
ˆˆ
and Oˆˆ
mn ppq Ď Omn ppq.
Let m, n P N. A partial function Rm 99K Rn is said to be “little O” if
it is an element of O Y Oˆ . A partial function Rm 99K Rn is said to be
“big O” if it is an element of O Y Oˆ . A partial function Rm 99K Rn
is said to be “sublinear” if it is an element of rOp1qs Y rOˆ p1qs. A
partial function Rm 99K Rn is said to be “linear order” if it is an
element of rOp1qs Y rOˆ p1qs. A partial function Rm 99K Rn is said
to be “subquadratic” if it is an element of rOp2qs Y rOˆ p2qs. A partial
function Rm 99K Rn is said to be “quadratic order” if it is an element
of rOp2qs Y rOˆ p2qs. We can continue with “subcubic”, “cubic order”,
“subquartic”, “quartic order”, etc.
We now describe THE BIG CALCULUS IDEA: Let m, n P N, let
f : Rm 99K Rn and let a P domrf s. Under some tame assumptions
on f , to be described later, there exist
C P CpRm , Rn q ,
K P CupRm , Rn q ,
L P LpRm , Rn q ,
P4 P P4 pRm , Rn q ,
Q P QpRm , Rn q
P5 P P5 pRm , Rn q, . . .
such that
rf pa ` ‚qs ´ C P Om,n ,
rf pa ` ‚qs ´ pC ` Lq P Om,n p1q,
rf pa ` ‚qs ´ pC ` L ` Qq P Om,n p2q,
rf pa ` ‚qs ´ pC ` L ` Q ` Kq P Om,n p3q,
rf pa ` ‚qs ´ pC ` L ` Q ` K ` P4 q P Om,n p4q,
rf pa ` ‚qs ´ pC ` L ` Q ` K ` P4 ` P5 q P Om,n p5q,
NOTES WEEK 04 DAY 2
7
and so on. So choose C, L, Q, K, P4 , P5 , . . . as above. Another way of
saying the conditions above is:
rf pa ` ‚qs ´ C is little O,
rf pa ` ‚qs ´ pC ` Lq is sublinear,
rf pa ` ‚qs ´ pC ` L ` Qq is subquadratic,
rf pa ` ‚qs ´ pC ` L ` Q ` Kq is subcubic,
rf pa ` ‚qs ´ pC ` L ` Q ` K ` P4 q is subquartic,
rf pa ` ‚qs ´ pC ` L ` Q ` K ` P4 ` P5 q is subquintic,
and so on. Now, C is just constant function, pretty easy. We study
the linear form L through linear algebra. We study the quadratic
form Q through polarization and tensor algebra. (If n “ 1, then the
polarization of Q is described by a symmetric matrix, and we can study
it through linear algebra, especially the Finite Dimensional Spectral
Theorem.) We study the cubic form K through polarization and tensor
algebra. We study the quartic form P4 through polarization and tensor
algebra. We study the quintic form P5 through polarization and tensor
algebra. And so on. Thus ends THE BIG CALCULUS IDEA.
We now aim to apply THE BIG CALCULUS IDEA to get a multivariable version of the Second Derivative Test, but we first need some
preliminary definitions.
DEFINITION 0.18. Let V be a vector space, and let Q P QpV q.
Then Q is positive definite, written Q ą 0, means: @x P V zt0V u,
Qpxq ą 0. Also, Q is positive semidefinite, written Q ě 0, means:
@x P V zt0V u, Qpxq ě 0. Also, Q is negative definite, written Q ă 0,
means: @x P V zt0V u, Qpxq ă 0. Also, Q is negative semidefinite,
written Q ď 0, means: @x P V zt0V u, Qpxq ď 0.
DEFINITION 0.19. Let S be a set, and let W be a vector space.
Then 0SW : S Ñ W denotes the zero function, defined by 0SW psq “ 0W .
We leave it to the reader to formulate the appropriate definitions
of “global strict minimum” and “local strict minimum” below.
Let m P N, f : Rm 99K R. Let C P CpRm q, L P LpRm q, Q P QpRm q.
Assume that rf pa ` ‚qs ´ pC ` L ` Qq P Om1 p2q. Assume that L “ 0V W ,
which is sometimes expressed by saying “f has a critical point at a”.
Finally, assume that Q ą 0. Let R :“ rf pa ` ‚qs ´ pC ` Qq, be the
“second order remainder”. Then f pa ` ‚q “ C ` Q ` R. Also, we have
R “ rf pa ` ‚qs ´ pC ` L ` Qq, and so R P Om1 p2q. We will eventually
8
SCOT ADAMS
show that for any positive definite quadratic form in QpRm q, and any
function in Om1 p2q, when sufficiently close to 0m , the form is enormous
compared to the function. To be more specific, let ε :“ 0.000001. Then
we will eventually show: Dpnbd P of 0m s.t., @x P P , |Rpxq| ă ε¨|Qpxq|.
So choose a pnbd P of 0m such that |R| ă ε ¨ |Q| on P . We conclude
that ´εQ ă R ă εQ on P , and so Q ´ εQ ă Q ` R on P . That is,
p1 ´ εqQ ă Q ` R on P . We define U :“ P Y t0m u. Because p1 ´ εqQ is
positive definite, it follows that p1´εqQ has a global strict minimum at
0m . So, as p1 ´ εqQ ă Q ` R on P , we see that pQ ` Rq|U has a global
strict minimum at 0m . So, since C is constant, pC ` Q ` Rq|U also has
a global strict minimum at 0m . That is, rf pa ` ‚qs|U has a global strict
minimum at 0m . Therefore, f |pU `aq has a global strict minimum at a.
It follows that f has a local strict minimum at a.
This proves the Second Derivative Test: If, at a point, the linear
part of the Taylor series vanishes, and if the quadratic part is positive
definite, then the function has a local strict minimum at that point.
Let V be a vector space. Since we have a 1-1 correspondence
B
ÞÑ
QB
:
SBFpV q
Ñ
QpV q,
the various notions of “definiteness” from Definition 0.18 can be extended from QpV q to SBFpV q, as follows:
DEFINITION 0.20. Let V be a vector space, and let B P SBFpV q.
Then B is positive definite, written B ą 0, means: QB ą 0. Also,
B is positive semidefinite, written B ě 0, means: QB ě 0. Also,
B is negative definite, written B ă 0, means: QB ă 0. Also, B is
negative semidefinite, written B ď 0, means: QB ď 0.
Let V be a vector space, and let B P SBFpV q. Recall that QB is the
diagonal restrction of B. That is, for all x P V , we have
QB pxq
“
Bpx, xq.
Then: ( B ą 0 ) iff ( @x P V zt0V u, Bpx, xq ą 0 ). Similar “iff”s exist
for B ě 0 and B ă 0 and B ď 0.
Let m P N. Since we have a 1-1 correspondence
A
ÞÑ
QA
:
mˆm
Rsym
Ñ
QpRm q,
the various notions of “definiteness” from Definition 0.18 can be extended from QpRm q to Rmˆm
sym , as follows:
NOTES WEEK 04 DAY 2
9
mˆm
DEFINITION 0.21. Let m P N , and let A P Rsym
. Then A is positive definite, written A ą 0, means: QA ą 0. Also, A is positive
semidefinite, written A ě 0, means: QA ě 0. Also, A is negative definite, written A ă 0, means: QA ă 0. Also, A is negative
semidefinite, written A ď 0, means: QA ď 0.
m
Let m P N, and let A P Rmˆm
sym . Then, for all x P R , we have
QA pxq
“
pxH ¨ A ¨ xV q11 .
Then: ( A ą 0 ) iff ( @x P Rm zt0m u, pxH ¨ A ¨ xV q11 ą 0 ). Similar
“iff”s exist for A ě 0 and A ă 0 and A ď 0.