Download Properties of lengths and dis- tances Orthogonal complements

159 Linear Algebra B • Lecture 7
Properties of Euclidean spaces1
Theodore Th. Voronov
MSS/P5 • X63682
[email protected]
www.ma.umist.ac.uk/tv/159.html
Orthogonal complements
Let W be a subspace of an inner product space V . A
vector u in V is said to be orthogonal to W if it is
orthogonal to every vector in W , and the set W ⊥ of
all vectors in V that are orthogonal to W is called the
orthogonal complement of W .
Theorem
Lectures: Monday 14:00 MSS/C19
Tutorials: Tuesday 11:00 MSS/C19
Properties of lengths and distances
Theorem 6.2.2.
Properties of Length.
If u and v are vectors in a Euclidean space V , and if k
is any scalar, then:
(a) kuk ≥ 0
(b) kuk = 0 if and only if u = 0
(c) kkuk = |k| kuk
(d) ku + vk 6 kuk + kvk (Triangle inequality)
Theorem 6.2.3.
Properties of Distance.
If u, v and w are vectors in an inner product space V ,
and if k is any scalar, then:
(a) d(u, v) ≥ 0
(b) d(u, v) = 0 if and only if u = v
(c) d(u, v) = d(v, u)
(d) d(u, v) 6 d(u, w) + d(w, v) (Triangle inequality)
We shall prove part (d) of Theorem 6.2.2 and leave
the remaining parts of Theorems 6.2.2 and 6.2.3 as exercises.
Proof of Theorem 6.2.2d. By definition
ku + vk2
=
hu + v, u + vi
=
hu, ui + 2hu, vi + hv, vi
6
hu, ui + 2|hu, vi| + hv, vi
6
hu, ui + 2kuk kvk + hv, vi
[Property of absolute value]
[By Cauchy–Bunyakovsky]
=
kuk2 + 2kuk kvk + kvk2
=
(kuk + kvk)2 .
Taking square roots gives
6.2.5. Properties of Orthogonal
Complements.
If W is a subspace of a finite-dimensional Euclidean
vector space V , then:
(a) The set W ⊥ is also a subspace of V .
(b) The only vector common to W and W ⊥ is 0.
(c) The orthogonal complement of W ⊥ is W ; that is,
(W ⊥ )⊥ = W .
We shall prove parts (a) and (b).
Proof (a). Note first that h0, wi = 0 for every vector
w in W , so W ⊥ contains at least the zero vector. We
want to show that W ⊥ is closed under addition and
multiplication by scalars; that is, we want to show that
the sum of two vectors in W ⊥ is orthogonal to every
vector in W and that every scalar multiple of a vector
in W ⊥ is orthogonal to every vector in W . Let u and v
be any vectors in W ⊥ , let k be any scalar , and let w be
any vector in W . Then from the definition of W ⊥ we
have hu, wi = 0 and hv, wi = 0. Using basic properties
of the scalar product we have
hu + v, wi = hu, wi + hv, wi = 0 + 0 = 0
hku, wi = khu, wi = k(0) = 0
which proves that u + v and ku are in W ⊥ .
Proof (b). If v is common to W and W ⊥ , then
hv, vi = 0, which implies that v = 0 by the Positivity Axiom (Axiom 4) for scalar products.
A proof for (c) will be omitted. Clearly W ⊂ (W ⊥ )⊥ ,
since hu, wi = 0 for all w ∈ W and u ∈ W ⊥ . Note
that the proof that W = (W ⊥ )⊥ relies on the finitedimensionality of V , which was not used in parts (a)
and (b).
Remark. If dim V = n, dim W = k, then dim W ⊥ =
n − k.
Example. Consider in R3 the subspace spanned by the
the last standard basis vector: W = span(e3 ). Then
W ⊥ = {u ∈ R3 | hu, e3 i = 0} =
{u ∈ R3 | u1 0 + u2 0 + u3 1 = 0} =
{u ∈ R3 | u3 = 0} = {(u1 , u2 , 0) ∈ R3 }
is the coordinate plane of the first two coordinates.
ku + vk 6 kuk + kvk.
Theorem 6.2.6. (Application to linear equations.) If A is an m × n matrix, then:
1 Definitions and theorems numbered as ‘6.2.2’ are taken
from the book: H. Anton, Elementary Linear Algebra, John
Wiley, 2000.
(a) The nullspace of A and the row space of A are orthogonal complements in Rn with respect to the standard Euclidean scalar product.
2
Th. Th. Voronov • 159 Linear Algebra B • Lecture 7 • Properties of Euclidean spaces
(b) The nullspace of AT and the column space of A
are orthogonal complements in Rm with respect to the
standard Euclidean scalar product.
Proof (a). We want to show that the orthogonal complement of the row space of A is the nullspace of A. To
do this we must show that if a vector v is orthogonal to
every vector in the row space, then Av = 0, and conversely, if Av = 0, then v is orthogonal to every vector
in the row space.
Assume first that v is orthogonal to every vector in
the row space of A. Then in particular v is orthogonal
to the row vectors r1 , r2 , . . . , rn of A; that is
r1 ·v = r2 ·v = · · · = rn ·v = 0
(1)
This is precisely Av = 0, since these dot products are
just the components of the column vector Av. Hence if
v is in the orthogonal complement of the row space, it
belongs to the nullspace of A. Assume now that, conversely, v is in the nullspace. Then the dot products
r1 ·v, r2 ·v, . . . , rn ·v vanish. It implies that v is orthogonal to all vectors in the row space, since the vectors
r1 , r2 , . . . , rn span the row space by the definition.
Proof (b). Since the column space of A is the row
space of AT (except for a difference in notation), the
proof follows by applying the result in part (a) to the
matrix AT .
Isomorphism of V and V ∗
For a finite-dimensional Euclidean space V there is a
natural isomorphism between V and its dual space V ∗ .
There is a simple but important statement:
Proposition. In a Euclidean vector space, if a vector is orthogonal to all vectors, it is a zero vector.
Proof. Indeed, let u ∈ V be orthogonal to all vectors
in V . Then, in particular, hu, ui = 0, which implies
u = 0.
the first argument). Consider the kernel of T . By definition, Ker(T ) = {u | hu, vi = 0 for all v ∈ V }. Hence
Ker(T ) = {0}, by the proposition above. Therefore, the
linear transformation T : V → V ∗ is injective. Hence
(here we use the condition that dim V < ∞) it is an
isomorphism.
Notice that both spaces V and V ∗ , both being of dimension n, are of course isomorphic, as are all vector
spaces of the same dimension. Such an isomorphism
can be constructed by a random choice of bases in V
and V ∗ and identifying both spaces with Rn . It is does
not require any Euclidean structure in V . There are
many such isomorphisms. However, the isomorphism
constructed above using a scalar product in V is ‘natural’ in the sense that it does not depend on any random
choices.
Orthonormal bases
A set of vectors in an inner product space is called an
orthogonal set if all pairs of distinct vectors in the set
are orthogonal. An orthogonal set in which each vector
has norm 1 is called orthonormal. We shall be particularly interested in orthogonal and orthonormal
bases.
First of all, orthogonality automatically implies linear independence:
Theorem 6.3.3 If S = {v1 , v2 , . . . , vn } is an orthogonal set of nonzero vectors in a Euclidean vector
space, then S is linearly independent.
Proof. Assume that
k1 v1 + k2 v2 + . . . + kn vn = 0
(2)
To demonstrate that S = {v1 , v2 , . . . , vn } is linearly
independent, we must prove that
k1 = k2 = · · · = kn = 0.
For each vi in S, it follows from (2) that
Theorem. Let V be a finite-dimensional Euclidean
space. Then the map T : V → V ∗ ,
T : u 7→ T (u) = ξu ,
where the linear function ξu is defined by
ξu (v) = hu, vi
for all v ∈ V , is an isomorphism of vector spaces.
Proof. Notice first that ξu for any u ∈ V is indeed a linear function V → R. This follows from the
properties of a scalar product (additivity and homogeneity w.r.t. the second argument). Next, the map
T : u 7→ T (u) = ξu is linear for the same reason (additivity and homogeneity of the scalar product w.r.t.
hk1 v1 + k2 v2 + . . . + kn vn , vi i = h0, vi i = 0
or equivalently,
k1 hv1 , vi i + k2 hv2 , vi i + · · · + kn hvn , vi i = 0
From the orthogonality of S it follows that hvj , vi i = 0
when j 6= i, so that this equation reduces to
ki hvi , vi i = 0
Since the vectors in S are assumed to be nonzero,
hvi , vi i 6= 0 by the positivity axiom for inner products.
Therefore, ki = 0. Since the subscript i is arbitrary, we
have
k1 = k2 = · · · = kn = 0;
3
Th. Th. Voronov • 159 Linear Algebra B • Lecture 7 • Properties of Euclidean spaces
thus, S is linearly independent.
Hence for an orthogonal (moreover, orthonormal) set
of nonzero vectors S to be a basis in V , is equivalent to
the condition that S spans V .
It is very easy to find the coordinates of vectors in
an orthonormal basis:
Theorem 6.3.1 If S = {e1 , e2 , . . . , en } is an orthonormal basis for a Euclidean vector space V , and u
is any vector in V , then
u = hu, e1 ie1 + hu, e2 ie2 + · · · + hu, en ien
The matrix G is known as the Gram matrix of the
basis B. It is defined by the properties of a basis: the
lengths of the basis vectors and angles between them.
Now it is clear that a basis B is orthonormal if its Gram
matrix is the identity matrix:
(
1 if i = j
gij = δij =
0 if i 6= j
(the symbol δij is known as the Kronecker symbol
or Kronecker delta).
It follows that in an orthonormal basis B we
have the following simple formulae. For scalar
product:
X
hu, vi =
u i vi = u 1 v1 + . . . + u n vn .
Proof. Since
S = {e1 , e2 , . . . , en }
i
is a basis, a vector u can be expressed in the form
For norm:
u = k1 e1 + k2 e2 + · · · + kn en
kuk2 =
We shall complete the proof by showing that ki =
hu, ei i for i = 1, 2, . . . , n. For each vector ei in S we
have
= k1 he1 , ei i + k2 he2 , ei i + · · · + kn hen , ei i
Since S = {e1 , e2 , . . . , en } is an orthonormal set, we
have hei , ei i = ||ei ||2 = 1 and hej , ei i = 0 if j 6= i.
Therefore, the above expression for hu, ei i simplifies to
u2i = u21 + . . . + u2n .
i
For distance:
d(u, v) =
hu, ei i = hk1 e1 + k2 e2 + . . . + kn en , ei i
X
p
(u1 − v1 )2 + . . . + (un − vn )2 .
Remark. The formulae above, in particular, mean
that in an orthonormal basis B the scalar product of
two vectors in V equals to the standard scalar product
in Rn of their coordinate column (or row) vectors:
hu, vi = [u]B · [v]B .
hu, ei i = ki .
We shall use this in the sequel.
The main point of considering orthonormal bases is
that all calculations are greatly simplified. To see this,
let B be an arbitrary basis. How one can calculate the
scalar product hu, vi? We have
u = u 1 e1 + . . . + u n en
and
v = v1 e1 + . . . + vn en .
Hence, by the properties of scalar products, we obtain
hu, vi = hu1 e1 + . . . + un en , v1 e1 + . . . + vn en i =
DX
E X
X
ui ei ,
vj ej =
ui vj hei , ej i ,
i
j
i,j
or, introducing a matrix G = (gij ) where
gij = hei , ej i ,
we finally can write
hu, vi =
X
i,j
ui vj gij .
In an orthonormal basis, the natural isomorphism
V → V ∗ of a Euclidean vector space with its dual, looks
especially simple. If T : V → V ∗ , u 7→ T (u) = ξu ,
where ξu (v) = hu, vi as described above, then
ξu = u1 e∗1 + . . . + un e∗n .
Here e∗i ∈ V ∗ denote the covectors of the dual basis
for B, if B = {e1 , . . . , en }. That means that if a basis is orthonormal, then a vector and the corresponding covector have the same components. In particular,
the basis vectors e1 , . . . , en are mapped precisely to the
covectors of the dual basis e∗1 , . . . , e∗n , respectively.
Remark. In an arbitrary basis in a Euclidean space
V , the natural isomorphism V → V ∗ will have the appearance
X
u 7→ ξu =
ui gij e∗j
ij
where (gij ) is the Gram matrix.