Download Vector Spaces

Vector Spaces
Dr. Doreen De Leon
Math 152, Fall 2015
1
Vector Spaces
Vector Spaces
Definition. A vector space is a nonempty set V of objects called vectors on which are defined
two operations: vector addition (which combines two elements of V and is denoted by “+”) and
scalar multiplication (which combines a complex number with an element of V ), that satisfy
the below properties for all u, v, w ∈ V and all scalars α and β.
(1) u + v ∈ V (closure under addition)
(2) αu ∈ V (closure under scalar multiplication)
(3) u + v = v + u (commutativity)
(4) (u + v) + w = u + (v + w) (associativity)
(5) There is a zero vector 0 ∈ V such that u + 0 = u.
(6) For each u ∈ V , there is a vector −u ∈ V such that u + (−u) = 0.
(7) α(u + v) = αu + αv
(8) (α + β)u = αu + βu
(9) α(βu) = (αβ)u
(10) 1u = u
Recall: We discussed previously that Cn is a vector space.
Notes:
(1) The scalars do not need to be complex numbers. Scalars are defined depending on the field
over which our set is defined. Examples of other fields are R, the set of rational numbers,
etc. In that case, we say V is a vector space over the field F .
(2) The objects in V can be anything. They do not necessarily need to be column vectors.
1
Examples of Vector Spaces
There are many other examples of vector spaces. Here are a few.
(1) Pn = the set of all polynomials of degree at most n (i.e., all polynomials of the form
p(t) = a0 + a1 t + a2 t2 + · · · + an tn , where ai ∈ C and t ∈ R) is a vector space.
• Addition: Let p(t) = a0 + a1 t + a2 t2 + · · · + an tn and q(t) = b0 + b1 t + b2 t2 + · · · + bn tn .
Then,
(p + q)(t) = p(t) + q(t)
= (a0 + b0 ) + (a1 + b1 )t + (a2 + b2 )t2 + · · · (an + bn )tn .
• Scalar multiplication: Let p(t) ∈ Pn and let c be a scalar.
Then,
(cp)(t) = cp(t)
= (ca0 ) + (ca1 )t + (ca2 )t2 + · · · + (can )tn .
• The zero vector is the zero polynomial, 0 = 0 + 0t + 0t2 + · · · + 0tn .
The ten axioms can be “easily” verified.
(2) Let F be the set of all real-valued functions defined on R.
• Addition: Let f ∈ F and g ∈ F. Then,
(f + g)(t) = f (t) + g(t).
• Scalar multiplication: Let f ∈ F and let c be a scalar. Then,
(cf )(t) = cf (t).
• The zero vector for F is the zero function, f (t) = 0 for all t.
Properties (1) - (10) can be verified to show that F is a vector space.
Example: Property (2). Let f, g ∈ F. Then,
(f + g)(t) = f (t) + g(t)
= g(t) + f (t)
= (g + f )(t).
(3) Mmn = the set of all m × n matrices is a vector space.
• Addition: matrix addition
• Scalar multiplication: matrix scalar multiplication
• The zero vector is the zero matrix.
2
Example: Let V be the set of positive real numbers with addition defined by x + y = xy and
scalar multiplication defined by cx = xc . Is V a vector space?
Solution: Properties (1) - (4) are straightforward to verify, so we will try to verify properties
(5)-(10).
(5) The zero vector is 1, since for all x ∈ V , x + 1 = x(1) = x.
(6) The additive inverse is 0, since for all x ∈ V , x + 0 = x(0) = 0. Problem: 0 ∈
/ V.
(7)
c(x + y) = (x + y)c
= (xy)c
= xc y c
= (cx)(cy)
= cx + cy.
(8)
(c + d)x = xc+d
= xc xd
= cx + dx.
(9)
c(dx) = cxd
= (xd )c
= xdc
= xcd
= (cd)x.
(10) 1x = x1 = x.
Since V does not satisfy Property (6), V is not a vector space.
Theorem 1. If V is a vector space, then the following properties hold for all vectors v ∈ V
and all scalars α.
• The zero vector 0 is unique.
• The additive inverse −v is unique.
• −v = (−1)v.
• 0u = 0.
• α0 = 0.
3
2
Subspaces
If a vector space H is a subset of vectors from a larger vector space V , then H is a subset of V
(i.e., H ⊆ V ).
Definition. A nonempty subset H of a vector space V is a subspace of V provided that H is
istelf a vector space with the operations of vector addition and scalar multiplication as defined
in V .
Testing Subspaces
Theorem 2. A subset H of a vector space V is a subspace of V if H satisfies the following.
(a) The zero vector of V is in H.
(b) For each u, v ∈ H, u + v ∈ H (closure under addition).
(c) For each u ∈ H and each scalar α, αu ∈ H (closure under scalar multiplication).
NOTES: Assume that property (a) above is satisfied.
* To prove that a subset H of a vector space V is a subspace of V , you need to show that
properties (b) and (c) hold for arbitrary vectors in H.
* If you believe that a subset H of a vector space V is not a subspace of V , it suffices to
find one example showing that either (b) or (c) fails.
Examples:
(1) Let H = {(x, y) ∈ C2 | x ≥ 0, y ≥ 0}. Is H a subspace of C2 ? Why or why not?
(a) 0 = (0, 0) ∈ H.
Let u ∈ H, so u = (u1 , u2 ), where u1 ≥ 0 and u2 ≥ 0; let v ∈ H, so v = (v1 , v2 ), where
v1 ≥ 0 and v2 ≥ 0; and let α be a scalar.
(b) Then, u + v = (u1 + v1 , u2 + v2 ) ∈ H since u1 + v1 ≥ 0 and u2 + v2 ≥ 0.
(c) Let u = (1, 1). Then u ∈ H. Let α = −1. Then, αu = (−1, −1) ∈
/ H, so H is not
closed under scalar multiplication.
Therefore, H is not a subspace of C2 .
 
x
3

(2) Let H be the set of all vectors in C of the form y . Is H a subspace of C3 ? Why or why
0
not?
4
 
0

(a) 0 = 0 ∈ H.
0
 
 
u1
v1



Let u ∈ H, so u = u2 ; let v ∈ H, so v = v2 ; and let α be a scalar.
0
0
(b) Then,


u1 + v1
u + v = u2 + v2  ∈ H,
0
so H is closed under addition.
(c) We have


αu1
αu = αu2  ∈ H,
0
so H is closed under scalar multiplication.
Therefore, H is a subspace of C3 .
(3) Let H = {p ∈ P2 | p(t) = at2 }. Is H a subspace of P2 ? Why or why not?
(a) The zero polynomial satisfies 0 = 0t2 , so 0 ∈ H.
(b) Let p ∈ H =⇒ p(t) = at2 and let q ∈ H =⇒ q(t) = bt2 . Then,
(p + q)(t) = p(t) + q(t)
= (a + b)t2 ∈ H,
so H is closed under addition.
(c) Let p ∈ H =⇒ p(t) = at2 and let α be a scalar. Then,
(αp)(t) = α · p(t)
= (αa)t2 ∈ H,
so H is closed under scalar multiplication.
Therefore, H is a subspace of P2 .
(4) Let H = {p ∈ P3 | p(0) = 8}. is H a subspace of P3 ? Why or why not?
(a) The zero polynomial in P3 is 0 = 0t3 + 0t2 + 0t + 0, which is 0 for all t. So, 0 ∈
/ H.
Therefore H is not a subspace of P3 .
(5) Let H be the set of all upper triangular 2 × 2 matrices. Is H a subspace of M22 ? Why or
why not?
(a) 0 ∈ H since the zero matrix is upper triangular.
5
a b
d e
(b) Let A ∈ H =⇒ A =
and let B ∈ H =⇒ B =
. Then
0 c
0 f
a+d b+e
A+B =
∈ H,
0
c+f
so H is closed under addition.
a b
(c) Let A ∈ H =⇒ A =
and let α be a scalar. Then
0 c
αa αb
αA =
∈ H,
0 αc
so H is closed under scalar addition.
Therefore, H is a subspace of M22 .
(6) C 1 (R) is the set of all real-valued continuous functions with one continuous derivative
defined on R. Is C 1 (R) a subspace of F ? Why or why not?
(a) The zero function, f (t) = 0 for all t ∈ R, is in C 1 (R), since f (t) is continuous and
f 0 (t) = 0, which is also continuous.
Let g, h ∈ C 1 (R), and let c be a scalar.
(b) We have that
(g + h)(t) = g(t) + h(t) is continous on R, and
(g + h)0 (t) = g 0 (t) + h0 (t) is continous on R,
so g + h ∈ C 1 (R), and C 1 (R) is closed under addition.
(c) We have that
(cg)(t) = cg(t) is continous on R, and
(cg)0 (t) = cg 0 (t) is continous on R,
so cg ∈ C 1 (R), and C 1 (R) is closed under scalar mutliplication.
Therefore, C 1 (R) is a subspace of F.
Notes:
• C n (R) is the set of all real-valued continuous functions with n continuous derivatives
defined on R.
• C n [a, b] is the set of all functions that are continuous on [a, b] and have n continuous
derivatives on [a, b].
(7) H is the set of all symmetric 12 × 12 matrices. Is H a subspace of M12,12 ? WHy or why
not?
6
(a) O ∈ H since Ot = O.
Let A, B ∈ H =⇒ At = A and B t = B, and let α be a scalar.
(b) We have that
(A + B)t = At + B t = A + B,
so A + B ∈ H and H is closed under addition.
(c) We have that
(αA)t = αAt = αA,
so αA ∈ H and H is closed under scalar multiplication.
Therefore, H is a subspace of M12,12 .
(8) H is the set of all functions on [0, 1] such that f (0)f (1) ≥ 0. Is H a subspace of F? Why
or why not?
(a) The zero function, f (t) = 0 for all t ∈ R, satisfies f (0)f (1) = 0, so the zero function
is in H.
(b) Let g ∈ H =⇒ g(0)g(1) ≥ 0 and let h ∈ H =⇒ h(0)h(1) ≥ 0. We must determine
if g + h ∈ H. This does not seem likely. We need only come up with an example
where g, h ∈ H, but g + h ∈
/ H.
2
Let g(t) = 1 − t . Then g(0) = 1 and g(1) = 0, so g(0)g(1) = 0 and g ∈ H.
Let h(t) = t2 − 2t. Then h(0) = 0 and h(1) = −1, so h(0)h(1) = 0 and h ∈ H.
But, (g + h)(t) = g(t) + h(t) = 1 − 2t. Since (g + h)(0) = 1 and (g + h)(1) = −1,
[(g + h)(0)][(g + h)(1)] = −1 < 0 and g + h ∈
/ H. So, H is not closed under addition.
Therefore, H is not a subspace of F.
Definition. Given the vector space V , the subspaces V and {0} are each called a trivial
subspace.
Recall:
Definition. The null space of an m × n matrix, denoted N (A) (or Nul(A)), is the set of all
solutions to the homogeneous system of equations, Ax = 0.
We can write
N (A) = {x ∈ Cn | Ax = 0}.
Theorem 3. Let A be an m × n matrix. Then the null space of A, N (A), is a subspace of Cn .
Proof. (a) 0 ∈ N (A) since A0 = 0.
(b) Let u, v ∈ N (A). Then Au = 0 and Av = 0. So,
A(u + v) = Au + Av
= 0 + 0 = 0.
Therefore, u + v ∈ N (A) and N (A) is closed under addition.
7
(c) Let u ∈ N (A) and let α be a scalar. Then
A(αu) = α(Au)
= α0 = 0,
so, αu ∈ N (A) and N (A) is closed under scalar multiplication.
Therefore, N (A) ⊆ Cn is a subspace of Cn .
The Span of a Set
Definition. Let v1 , v2 , . . . , vp be vectors in a vector space V and let α1 , α2 , . . . , αp be scalars.
Then the linear combination of v1 , v2 , . . . , vp with weights α1 , α2 , . . . , αp is given by
α1 v1 + α2 v2 + · · · + αp vp .
Definition. Let V be a vector space and S = {v1 , v2 , . . . , vp } a set of vectors in V . Then, the
span of S, denoted Span(S), is the set of all possible linear combinations of v1 , v2 , . . . , vp .
Theorem 4. If v1 , v2 , . . . , vp are in a vector space V , then Span{v1 , v2 , . . . , vp } is a subspace
of V .
Proof.
(a) The zero vector is in Span(S), because 0 = 0v1 + 0v2 + · · · + 0vp .
(b) Let u, w ∈ Span(S). Then there are constants α1 , α2 , . . . , αp and β1 , β2 , . . . , βp such that
u = α1 v1 + α2 v2 + · · · αp vp
w = β1 v1 + β2 v2 + · · · βp vp
Then,
u + w = (α1 v1 + α2 v2 + · · · αp vp ) + (β1 v1 + β2 v2 + · · · βp vp )
= (α1 + β1 )v1 + (α2 + β2 )v2 + · · · (αp + βp )vp ∈ Span(S).
Therefore, Span(S) is closed under vector addition.
(c) Let u ∈ Span(S) and let α be a scalar. Since u ∈ S, we have
u = α1 v1 + α2 v2 + · · · αp vp .
Then
αu = α(α1 v1 + α2 v2 + · · · αp vp )
= (αα1 )v1 + (αα2 )v2 + · · · + (ααp )vp ∈ Span(S).
Therefore, Span(S) is closed under scalar multiplication.
Since Span(S) ⊆ V contains the zero vector and is closed under vector addition and scalar
multiplication, Span(S) is a subspace of V .
8
So, Span{v1 , v2 , . . . , vp } is the subspace spanned by v1 , v2 , . . . , vp .
Given any subspace H of V , a spanning set for H is a set of vectors v1 , v2 , . . . , vp in H such
that H = Span{v1 , v2 , . . . , vp }.
Example: Let W be the set of all vectors of the form


2b + 3c
 −5b  ,
2c
where b, c are arbitrary complex numbers. Show that W is a subspace of C3 .
Solution: Let x ∈ W . Then


 
 
2b + 3c
2
3





−5b
x=
= b −5 + c 0 .
2c
0
2
    
3 
 2



−5 , 0 =⇒ W is a subspace of C3 .
So, W = Span


0
2
Subspace Constructions
Recall:
Definition. The column space
of an m×n matrix
A, C(A), is the set of all linear combinations
of the columns of A. If A = a1 a2 · · · an , then C(A) = Span{a1 , a2 , . . . , an }.
Theorem 5. The column space of an m × n matrix is a subspace of Cm .
Proof. Let A be an m×n matrix. Then C(A) is the set of all linear combinations of the columns
of A, or C(A) = Span{a1 , a2 , . . . , an }, where ai ∈ Cm for i = 1, 2, . . . , n. Therefore, C(A) is a
subspace of Cm .
Note: If v ∈ C(A) for some m × n matrix A, then v = Ax for some x ∈ Cn (because Ax is a
linear combination of the column vectors of A). So,
C(A) = {b ∈ Cm | b = Ax for some x ∈ Cn }.
Examples:
1 2 3
(1) Consider the matrix A =
.
4 5 6
(a) If C(A) is a subspace of Ck , what is k? Why?
9
(b) If N (A) is a subspace of Ck , what is k? Why?
Solution:
(a) k = 2 since the columns of A are in C2 .
(b) k = 3, because for Ax to be defined, x must be in R3 .
1 2
−2
(2) Let A =
. Is u =
∈ C(A)? In N (A)?
4 8
1
To determine if u ∈ N (A), we must solve Ax = 0.
1 2 | 0 r2 →r2 −4r2 1 2 | 0
−−−−−−→
.
4 8 | 0
0 0 | 0
We see that x2 is free and x1 + 2x2 = 0, so x1 = −2x2 and
−2x2
−2
= x2
x=
.
x2
1
So, u ∈ N (A).
1
We see that C(A) = Span
and so u ∈
/ C(A).
4
(3) Find a matrix A so that W = C(A), where



 2a − b

W =  3b  | a, b ∈ C .


a+b
We have that


2a − b
x ∈ W =⇒ x =  3b 
a+b
 
 
2
−1



3
=a 0 +b
1
1
    
−1 
 2
=⇒ W = Span 0 ,  3 .


1
1


2 −1
3 is one such matrix.
Therefore, A = 0
1
1
10
3
Linear Independence and Spanning Sets
Linear Independence
Definition. A set of vectors S = {v1 , v2 , . . . , vp } in a vector space V is linearly independent
(LI) if the only linear combination of vectors in S that form the zero vector, i.e.,
α1 v1 + α2 v2 + · · · αp v = 0,
is the combination with α1 = α2 = · · · = αp = 0. Otherwise, the set is linearly dependent
(LD). (In other words, S is a linearly dependent set if there exist constants α1 , α2 , . . . , αp not
all zero so that
α1 v1 + α2 v2 + · · · + αp vp = 0,
which is known as a linear dependence relation among v1 , v2 , . . . , vp .)
Theorem 6. Let x1 , x2 , . . . , xn ∈ Cn , and let X = x1 x2 · · · xn . Then, {x1 , x2 , . . . , xp }
is linearly dependent if and only if X is singular.
Proof. The equation c1 x1 + c2 x2 + · · · + cn vn = 0 can be written as
Xc = 0.
This equation has nontrivial solution if and only if X is singular. Thus, {x1 , x2 , . . . , xp } is
linearly dependent if and only if X is singular.
Example: Are the vectors
     
4
2
2
2 , 3 , −5
3
1
2
linearly independent? Why or why not?
Solution: To see if the vectors are linearly independent, we first form the matrix whose columns
are the vectors and then determine if that matrix is singular. So, we need to check if


4 2
2
2 3 −5
3 1
3
is singular. Since
4 2
2
2 3 −5 = 0,
3 1
3
the matrix is singular and so the vectors are not linearly independent.
     
1
−2
1
−1  3 0
    
Example: (Review) Are 
 2 ,  1 , 7 linearly independent? Why or why not?
3
−2
7
11
Solution: We need to determine if the vector equation
 
 
 
1
−2
1
−1
 3
0

 
 
α1 
 2 + α2  1 + α3 7 = 0
3
−2
7
has only the trivial solution. We form the augmented matrix and perform Gaussian elimination.






1 −2 1 | 0
1 −2 1 | 0
1 −2 1 | 0
−1


3 0 | 0
1 1 | 0
1 1 | 0

 −→ 0
 −→ 0
.
 2
0
0
1 7 | 0
5 5 | 0
0 0 | 0
3 −2 7 | 0
0
4 4 | 0
0
0 0 | 0
We see that α3 is a free variable, so there are infinitely many nontrivial solutions. Therefore,
the vectors are linearly dependent.
What about Functions? Matrices?
1 0
3 4
0 0
Example: Are
,
,
linearly independent?
1 1
2 5
−1 6
Solution: We need to determine if the equation
1 0
3 4
0 0
0 0
c1
+ c2
+ c3
=
2 5
−1 6
1 1
0 0
has only the trivial solution. We have
c1 + 3c2
4c2
0 0
=
.
2c1 − c2 + c3 5c1 + 6c2 + c3
0 0
Applying the property of matrix equality gives the system
c1 + 3c2
4c2
2c1 − c2 + c3
5c1 + 6c2 + c3
=0
= 0 =⇒ c2 = 0 =⇒ c1 = 0
= 0 =⇒ c3 = 0
=0
So, c1 = c2 = c3 = 0 is the only solution. Therefore, the matrices are linearly independent.
Example: Are p1 (t) = t2 − 2t + 3, p2 (t) = 2t2 + t + 8, p3 (t) = t2 + 8t + 7 linearly independent?
Solution: We need to determine if the equation c1 p1 (t) + c2 p2 (t) + c3 p3 (t) = 0 has only the
trivial solution. So, we will solve
c1 (t2 − 2t + 3) + c2 (2t2 + t + 8) + c3 (t2 + 8t + 7) = 0
for c1 , c2 , and c3 . Collect terms to obtain
(c1 + 2c2 + c3 )t2 + (−2c1 + c2 + 8c3 )t + 3c1 + 8c2 + 7c3 = 0.
12
Equate coefficients of like terms to obtain the system of equations
c1 + 2c2 + c3 = 0
−2c1 + c2 + 8c3 = 0
3c1 + 8c2 + 7c3 = 0.
This is equivalent to the equation


1 2 1
−2 1 8 c = 0,
3 8 7
which has only the trivial solution if the coefficient matrix is nonsingular. Since
1 2 1
−2 1 8 = 0,
3 8 7
p1 , p2 , and p3 are not linearly independent.
Functions in C n−1 [a, b]
Let f1 , f2 , . . . , fn ∈ C n−1 [a, b]. We start by searching for a direct way to determine if the
functions are linearly dependent. If the functions are linearly dependent, then there exist
weights c1 , c2 , . . . , cn , not all zero such that
c1 f1 (t) + c2 f2 (t) + · · · cn fn (t) = 0
(1)
for each t ∈ [a, b].
Differentiating both sides of (1) with respect to t gives
c1 f10 (t) + c2 f20 (t) + · · · cn fn0 (t) = 0.
We may continue differentiating to obtain a system of n equations in n unknowns (the ci , i =
1, 2, . . . , n).
c1 f1 (t)
c1 f10 (t)
(n−1)
c1 f 1
+ ···
+ ···
..
.
+ c2 f2 (t)
+ c2 f20 (t)
(n−1)
(t) + c2 f2
(t) + · · ·
+ cn fn (t)
+ cn fn0 (t)
(n−1)
+ cn f n
= 0
= 0
..
.
(2)
(t) = 0.
Writing (2) as a matrix equation which for each t ∈ [a, b] has the same solution set as (2):


f1 (t)
f2 (t)
···
fn (t)
 f10 (t)
f20 (t)
···
fn0 (t) 


(3)

 c = 0.
..
..
..


.
.
.
(n−1)
f1
(n−1)
(t) f2
(t) · · ·
13
(n−1)
fn
(t)
So, if the functions are linearly dependent in C n−1 [a, b], for each t ∈ [a, b], the coefficient matrix
in (3) is singular. In other words, the determinant of the coefficient matix in (3) is 0 for each
t ∈ [a, b]. This leads to the following:
Definition. Let f1 , f2 , . . . , fn ∈ C n−1 [a, b]. The Wronskian of f1 , f2 , . . . , fn on [a, b], denoted
W [f1 , f2 , . . . , fn ](t), is the determinant of the coefficient matrix in (3).
Theorem 7. Let f1 , f2 , . . . , fn ∈ C n−1 [a, b]. If there exists a point t0 ∈ [a, b] such that
W [f1 , f2 , . . . , fn ](t0 ) 6= 0 then f1 , f2 , . . . , fn are linearly independent.
Examples:
(1) Determine if the functions f (x) = cos(πx) and g(x) = sin(πx) are linearly independent
in C 1 [0, 1].
f (x) g(x) cos(πx)
sin(πx) W [f, g](x) = 0
=
f (x) g 0 (x) −π sin(πx) π cos(πx)
= π cos2 (πx) + π sin2 (πx) = π 6= 0.
Therefore, f and g are linearly independent in C 1 [0, 1].
(2) Determine if f (t) = 1, g(t) = et + e−t , h(t) = et − e−t are linearly independent.
f (t) g(t) h(t) 1 et + e−t et − e−t W [f, g, h](t) = f 0 (t) g 0 (t) h0 (t) = 0 et − e−t et + e−t f 00 (t) g 00 (t) h00 (t) 0 et + e−t et − e−t t
−t
et + e−t 1+1 e − e
= 1(−1) t
e + e−t et − e−t = (et − e−t )2 − (et + e−t )2
= e2t − 2 + e−2t − (e2t + 2 + e−2t )
= −4 6= 0.
Therefore, the functions are linearly independent.
Spanning Sets
Definition. A subset S of a vector space V is a spanning set of V if V = Span(S). In this
case, we typically say that S spans V .
Examples:
(1) Does the set {x, 1 − x2 , 1 + x2 } span P2 ? Why or why not?
14
1 0
0 1
(2) Does the set
,
span M22 ? Why or why not?
0 1
1 0
Solution:
(1) We need to determine if we can write an arbitrary element in P2 as a linear combination
of x, 1 − x2 , and 1 + x2 . So, let p(x) = a + bx + cx2 . we need to determine if there are
constants c1 , c2 , and c3 such that
c1 x + c2 (1 − x2 ) + c3 (1 + x2 ) = a + bx + cx2 .
We have
(c2 + c3 ) + c1 x + (c2 − c3 )x2 = a + bx + cx2 .
Equating coefficients of like terms gives the system
c1
c2 + c3 = a
=b
c2 − c3 = c.
1
1
We can solve this system directly to obtain c1 = b, c2 = (a + c), c3 = (a − c). Therefore,
2
2
the set spans P2 .
(2) We need to determine if we can write any matrix in M22 as a linear combination of the
matrices in the set. We suspect not, so we will try to find a matrix that won’t work. Let
1 0
A=
.
2 2
Then we have
c1
1 0
0 1
1 0
+ c2
=
,
0 1
1 0
2 2
which gives
c1 c2
1 0
=
.
c2 c1
2 2
By the definition of matrix equality, we obtain
c1
c2
c2
c1
=1
=0
=2
= 2,
a system with no solution. Therefore, A is not in the span of the set, so the set does not
span P2 .
15
Vector Representation
We will just introduce this topic now, although we will come back to it later this semester.
Theorem 8 (Vector Representation Relative to a Basis). Suppose that V is a vector space
and B = {v1 , v2 , . . . , vn } is a linearly independent set of vectors that spans V . Let w be any
vector in V . Then there exist unique scalars a1 , a2 , . . . , an such that
a1 v1 + a2 v2 + · · · + an vn = w.
We will use these constants to define a column vector in Cn with which we can represent w.
4
Bases
Definition. Let V be a vector space. A set S = {b1 , b2 , . . . , bp } in V is a basis of V if
(i) S is linearly independent, and
(ii) S spans V (i.e., V = Span{b1 , b2 , . . . , bp }).
Theorem 9 (The Spanning Set Theorem). Let S = {v1 , v2 , . . . , vp } in vector space V and let
H = Span{v1 , v2 , . . . , vp }.
(a) If vk is a linear combination of the other vectors in S for some k, then the set formed by
removing vk from S spans H.
(b) If H 6= {0}, then some subset of S is a basis of H.
Proof. (a) Without loss of generality, assume that vp is a linear combination of v1 , v2 , . . . , vp−1 .
So,
vp = a1 v1 + a2 v2 + · · · ap−1 vp−1
for some constants a1 , a2 , . . . , ap−1 . Let x ∈ H. Then,
x = c1 v1 + c2 v2 + · · · cp−1 vp−1 + cp vp
for some scalars c1 , c2 , . . . , cp−1 , cp . Then,
x = c1 v1 + c2 v2 + · · · cp−1 vp−1 + cp (a1 v1 + a2 v2 + · · · ap−1 vp−1 )
= (c1 + a1 )v1 + (c2 + a2 )v2 + · · · + (cp−1 + ap−1 )vp−1 .
Therefore, {v1 , v2 , . . . , vp−1 } spans H.
(b) If S is linearly independent, we are done. If not, one vector can be written as a linear
combination of the others and can be deleted. Repeat this procedure until the set is
linearly independent. If at least one vector v 6= 0 remains, then the set is linearly
independent and is, thus, a basis of H.
16
Theorem 10. The set of standard vectors for Cn , B = {e1 , e2 , . . . , en }, is a basis for the vector
space Cn .
We know that B is linearly independent and it is straightforward to show that it is a spanning
set for Cn .
1 0
0 1
0 0
0 0
Example: Show that the set B =
,
,
,
is a basis for M22 .
0 0
0 0
1 0
0 1
Solution: We need to show that B is linearly independent and spans M22 . To verify that B is
linearly independent, we need to show that the vector equation
1 0
0 1
0 0
0 0
0 0
c1
+ c2
+ c3
+ c4
=
0 1
0 0
1 0
0 1
0 0
has only the trivial solution. But this is equivalent to
c1 c2
0 0
=
,
c3 c4
0 0
which has as its unique solution c1 = c2 = c3 = c4 = 0. So, B is linearly independent.
a b
We next show that B spans M22 . Let A =
∈ M22 . Then,
c d
a b
1 0
0 1
0 0
0 0
=a
+b
+c
+d
.
c d
0 0
0 0
1 0
0 1
=⇒ B spans M22 .
Therefore, B is a basis for M22 .
The set B is known as the standard basis for M22 .
Example: Find a basis for the subspace S of P2 , where S = {p ∈ P2 | p(0) = 0}.
Solution: An arbitrary element in S is p(x) = a1 x+a2 x2 . This is a linear combination of x and
x2 . Therefore, the set {x, x2 } spans S. It is also linearly independent (c1 x+c2 x2 = 0 = 0x+0x2
only if c1 = c2 = 0). Therefore {x, x2 } is a basis for S.
Bases and Nonsingular Matrices
We have the following awesome theorem.
Theorem 11. Suppose that A is an n × n matrix. Then the columns of A form a basis for Cn
if and only if A is nonsingular.
Proof. First, we assume that the columns of A form a basis for Cn . Then by definition the set
of columns is linearly independent. Therefore, A is nonsingular.
17
Next, we assume that A is nonsingular. Then, the set of the columns of A is linearly independent. In addition, we know that C(A) = Cn . Therefore, the columns of A span Cn . Since the
columns of A are linearly independent and span Cn , they form a basis for Cn .
We can then add this to the theorem of Nonsingular Matrix Equivalences, obtaining.
Theorem 12 (Nonsingular Matrix Equivalences, Round 6). Suppose that A is a square matrix.
Then the following are equivalent.
(1) A is nonsingular.
(2) A row reduces to the identity matrix.
(3) The null space of A contains only the zero vector (i.e., N (A) = {0}).
(4) The linear system LS(A, b) has a unique solution for every b.
(5) The columns of A are linearly independent.
(6) A is invertible.
(7) The column space of A is Cn (C(A) = Cn ).
(8) The determinant of A is nonzero, i.e., det(A) 6= 0.
(9) The columns of A form a basis for Cn .
Orthonormal Bases and Coordinates
We have seen that orthogonal sets are automatically linearly independent. We get even nicer
properties if the set is orthonormal. Again, we will ignore how we obtain an orthonormal basis
for now.
One good source of an orthonormal basis is a unitary matrix, since the columns of a unitary
matrix are orthonormal. Why? Suppose Q is an n×n orthonormal matrix. Then the n columns
of Q are linearly independent. Since Q is invertible, it is nonsingular. Therefore, the columns
of Q form a basis for Cn . Check out the following theorem.
Theorem 13 (Coordinates and Orthonormal Bases). Suppose that B = {v1 , v2 , . . . , vp } is an
orthonormal basis of the subspace W of Cn . For any w ∈ W , we have
w = hv1 , wiw + hv2 , wiw + · · · + hvp , wiw.
Proof. (Idea) Since B is a basis for W , we can write any vector w ∈ W as a linear combination
of v1 , v2 , . . . , vp . So,
w = α1 v1 + α2 v2 + · · · + αp vp .
We simply compute hvi , wi for i = 1, 2, . . . , p. to see that ai = hv1 , wi for i = 1, 2, . . . , p.
18
Another interesting thing is the following.
Theorem 14. Let A be an n × n matrix and B = {v1 , v2 , . . . , vp } be an orthonormal basis of
Cn . Define
C = {Av1 , Av2 , . . . , Avp }.
Then A is a unitary matrix if and only if C is an orthonormal basis of Cn .
5
Dimension
Almost every vector space we have discussed has contained infinitely many vectors. We seek
some way to measure the “size” of a vector space, which we can use to study the properties of
vector spaces.
Dimension
Before we can make the definition of dimension (at least one that makes sense), we will discuss
the following two theorems.
Theorem 15 (Spanning Sets and Linear Dependence). Suppose that S = {v1 , v2 , . . . , vp } is
a finite set of vectors which spans the vector space V . Then any set of more than p vectors is
linearly dependent.
The idea of the proof is that we take a set R of m > p vectors and show that R is linearly
dependent by writing each vector in the set as a linear combination of the vectors in S, thereby
allowing us to find a nontrivial linear combination of the vectors in R that equal the zero vector.
Theorem 16. Any two bases of vector space V contain the same number of vectors.
Proof. Suppose that we have two bases of V : S, which contains n vectors, and T , which contains
m vectors. We know that m ≤ n, since if m > n, then T is linearly dependent. Similarly, we
have n ≤ m; otherwise, S is linearly dependent. Since m ≥ n and n ≥ m, we have m = n.
Definition. Suppose that V is a vector space and {v1 , v2 , . . . , vn } is a basis of V . Then the
dimension of V is defined by dim(V ) = n, and V is said to be finite-dimensional. If V
has no finite basis, we say that V has infinite dimension or that V is infinite-dimensional. The
dimension of the zero vector space, {0}, is defined to be zero.
Examples:
1. dim(R2 ) = 2, dim(R3 ) = 3, . . . , dim(Rn ) = n
2. dim(Cn ) = n
19
3. dim(P1 ) = 2, dim(P2 ) = 3, . . . , dim(Pn ) = n + 1
4. The set C n (R) is infinite-dimensional.
5. The set Mmn has dimension m · n.
Examples:
(1) Find the dimension of the subspace



 a + 2b 


b − c a, b, c ∈ R .
H=


5c
Solution:


 
 
 
a + 2b
1
2
0
v ∈ H =⇒ v =  b − c  = a 0 + b 1 + c −1 = av1 + bv2 + cv3 .
5c
0
0
5
Therefore, a basis of H is {v1 , v2 , v3 } =⇒ dim(H) = 3.
1
−2
−3
(2) Find dim(H), where H = Span
,
,
.
−5
10
15
1 −2 −3
Solution: First, we need to find a basis of C(A), where A =
.
−5 10 15
1 −2 −3 r2 →r2 +5r1 1 −2 −3
−−−−−−→
.
−5 10 15
0
0
0
1
The first column is the only pivot column, so
is a basis of C(A), so H =
−5
1
Span
. Therefore, dim(H) = 1.
−5
(3) Find dim(S), where S is the set of 2 × 2 symmetric matrices.
Solution:
a b
1 0
0 1
0 0
A ∈ S =⇒ A =
=a
+b
+c
.
b c
0 0
1 0
0 1
Therefore, a basis of S is
1 0
0 1
0 0
,
,
,
0 0
1 0
0 1
and dim(S) = 3.
(4) Find dim(H), where H = {p ∈ P2 | a1 = a2 }.
Solution: p ∈ H =⇒ p(t) = a0 + a1 t + a1 t2 = a0 + a1 (t + t2 ). Therefore, a basis of H is
{1, t + t2 }, and dim(H) = 2.
20
Rank and Nullity of a Matrix
Definition. Suppose that A is an m × n matrix.
• The rank of A, rank(A) , is the dimension of the column space of A (so rank(A) =
dim(C(A))).
• The nullity of A, nullity(A), is the dimension of the null space of A (so, nullity(A) =
dim(N (A))).
Theorem 17 (Rank-Nullity Theorem). The dimensions of the column space and the row space
of an m × n matrix A are equal. This common dimension, rank(A), also equals the number of
pivot positions in A and satisfies
rank(A) + nullity(A) = n.
(4)
Proof.
rank(A) = dim(C(A))
= number of pivot columns in A
= number of pivot columns in an echelon form B of A.
Since B has a nonzero row for each pivot, and these rows form a basis of R(A), rank(A) =
dim(R(A)). Now,
nullity(A) = dim(N (A))
= number of free variables in LS(A, 0)
= number of columns of A that are not pivot columns.
But, (number of pivot columns) + (number of non-pivot columns) = (number of columns). So,
(4) is true.
Example: Suppose a 4 × 7 matrix A has rank 2. Find nullity(A), dim(R(A)), and rank(At ).
Is C(A) = C2 ?
Solution:
dim(C(A)) = 2 =⇒ dim(R(A)) = 2.
rank(A) + nullity(A) = 7 =⇒ nullity(A) = 5.
rank(At ) = dim(C(At )) = dim(R(A)) = 2.
C(A) 6= C2 because each column of A is in C4 .
21
Example: Suppose the solutions of a homogeneous system of five equations in six unknowns
are all multiples of one nonzero solution. Will the system necessarily have a solution for every
possible right-hand side? Why or why not?
Solution: Let A be the 5×6 coefficient matrix of the system. Then nullity(A) = dim(N (A)) =
1. So, dim(C(A)) = 6 − 1 = 5. Since C5 is the only subspace of C5 whose dimension is 5,
C(A) = C5 . Therefore, every nonhomogeneous system of equations has a solution.
What if A were 6 × 5? Then dim(C(A)) = 5 − 1 = 4, so the columns of A do not span C6 .
Rank and Nullity of a Nonsingular Matrix
Theorem 18. Suppose A is an n × n matrix. Then the following are equivalent.
(1) Matrix A is nonsingular.
(2) The rank of A is n (i.e., rank(A) = n).
(3) The nullity of A is zero (i.e., nullity(A) = 0).
Proof.
(1) implies (2): If A is nonsingular, then C(A) = Cn . Therefore, dim(C(A)) = n, or
rank(A) = n.
(2) implies (3): Suppose rank(A) = n. By the Rank-Nullity Theorem, rank(A) +
nullity(A) = n. Therefore, nullity(A) = n − rank(A) = 0.
(3) implies (1): Suppose nullity(A) = 0. Therefore, a basis for nullity(A) is the empty
set. Therefore, N (A) = {0} and A is nonsingular.
This gives us the new Nonsingular Matrix Equivalences theorem.
Theorem 19 (Nonsingular Matrix Equivalences, Round 7). Suppose that A is a square matrix.
Then the following are equivalent.
(1) A is nonsingular.
(2) A row reduces to the identity matrix.
(3) The null space of A contains only the zero vector (i.e., N (A) = {0}).
(4) The linear system LS(A, b) has a unique solution for every b.
(5) The columns of A are linearly independent.
22
(6) A is invertible.
(7) The column space of A is Cn (C(A) = Cn ).
(8) The determinant of A is nonzero, i.e., det(A) 6= 0.
(9) The columns of A form a basis for Cn .
(10) The rank of A is n, rank(A) = n.
(11) The nullity of A is 0, nullity(A) = 0.
6
Properties of Dimension
Goldilock’s Theorem
Theorem 20. Suppose that V is a vector space and S is a linearly
independent set of vectors
S
0
in V . If w is a vector with w ∈
/ Span(S), then the set S = S {w} is linearly independent.
Proof. Let S = {v1 , v2 , . . . , vp }. Consider the equation
c1 v1 + c2 v2 + · · · + cp vp + cp+1 w = 0.
There are two possibilities. If cp+1 = 0, then we are left with
c1 v1 + c2 v2 + · · · + cp vp = 0,
and since S is linearly independent, we have ci = 0 for i = 1, 2, . . . , p. If cp+1 6= 0, then we may
write
cp+1 w = −c1 v1 − c2 v2 − · · · cp vp
c2
cp
c1
v1 + −
v2 + · · · + −
vp ,
=⇒ w = −
cp+1
cp+1
cp+1
a linear combination of the vectors in S.S This contradicts our assumption that w ∈
/ Span(S).
0
Therefore, cp+1 = 0, and the set S = S {w} is linearly independent.
We need this a bit later. Another useful theorem is the following.
Theorem 21. Suppose V is a vector space with dim(V ) = n. Let S = {v1 , v2 , . . . , vm } be a
set of m vectors from V . Then
1. If m > n, then S is linearly dependent.
2. If m < n, then S does not span V .
3. If m = n and S is linearly independent, then S spans V .
23
4. If m = n and S spans V , then S is linearly independent.
Proof of 1 and 2. Let B be a basis for V . Since dim(V ) = n, B is a linearly independent set
of n vectors that spans V .
1. If S were linearly independent, then B is a smaller set of vectors that spans V , which
contradicts the theorem on Spanning Sets and Linear Dependence. Therefore, S must be
linearly dependent.
2. If S spanned V , then B is a larger set of vectors that is linearly independent. This also
contradicts the theorem on Spanning Sets and Linear Dependence. Therefore, S does not
span V .
Theorem 22 (The Basis Theorem). Let V be a p-dimensional vector space with p ≥ 1. Any
linearly independent set of p vectors in V is a basis of V . Any set of p vectors in V that spans
V is a basis of V .
The Basis Theorem follows directly by applying Theorem 21.
Example: Show that {1 + 2t, t2 − t, t − 1} is a basis of P2 .
Solution: Since dim(P2 ) = 3 and the set contains three vectors, it is a basis of P2 if it is
linearly independent.
c1 (1 + 2t) + c2 (t2 − t) + c3 (t − 1) = 0
c2 t2 + (2c1 − c2 + c3 )t + c1 − c3 = 0.
Equate coefficients of like terms.
c2
=0
2c1 − c2 + c3 = 0
c1
− c3 = 0.
Solve using Gaussian elimination.



0
1
0 | 0
2 −1
1 |
r1 ↔r2
2 −1
1 | 0 −−
1
0 |
−→ 0
1
0 −1 | 0
1
0 −1 |

1 −1
2
r3 →r3 −r1

1
0
−−−−−−→ 0
0
1 −3


0
1 −1
2 |
r1 →r1 −r3
0 −−
1
0 |
−−−−→ 0
0
1
0 −1 |


| 0
1 −1
2
r3 →r3 −2r2


| 0 −−−−−−→ 0
1
0
| 0
0
0 −3
Back substitution gives us
−3c3 = 0 =⇒ c3 = 0
c2 = 0
c2 − c2 + 2c3 = 0 =⇒ c1 = 0.
24

0
0
0

| 0
| 0 .
| 0
Since c1 = c2 = c3 = 0 is the only solution, the set is linearly independent. Therefore, it is a
basis of P2 .
Note: We could also have used the Wronskian to determine if the set is linearly independent.
Theorem 23. Let H be a subspace of a finite-dimensional vector space V . Then H is finitedimensional and
dim(H) ≤ dim(V ).
Proof. Let S be a linearly independent set of vectors in H such that H = Span(S). Then
the number of vectors in S cannot exceed dim(V ) or the vectors will be linearly dependent.
Therefore, dim(H) ≤ dim(V ).
So, you may ask, how do we know when dim(H) < dim(V ) and not dim(H) = dim(V )?
Theorem 24. If U and V are subspaces of vector space W such that U ⊆ V (but U 6= V ),
then dim(U ) < dim(V ).
Proof. Suppose dim(U ) = m and dim(V ) = n. Then U has a basis B of size m. If m > n,
then since U ⊆ V , B is linearly dependent, which contradicts the fact that it is a basis for U .
If m = n, then U = Span(B) = V , which contradicts the fact that U 6= V . Therefore, m < n
is the only possibility.
Theorem 25. If U and V are subspaces of vector space W such that U ⊆ V and dim(U ) =
dim(V ), then U = V .
Ranks and Transposes
Theorem 26. If A is an m × n matrix, then rank(A) = rank(At ).
Proof. Suppose A is row-equivalent to matrix B in reduced row echelon form, and B has r
nonzero rows. Then B has r pivot columns, and so A has r pivot columns. This means that
a basis for C(A) consists of the r pivot columns of A and a basis for R(A) consists of the r
nonzero rows of B. Using this, we have
rank(A) = dim(C(A))
=r
= dim(R(A))
= dim(C(At ))
= rank(C(At )).
25