Download Chapter 2 General Vector Spaces

Chapter 2
General Vector Spaces
Outline :
• Real vector spaces
• Subspaces
• Linear independence
• Basis and dimension
• Row Space, Column Space, and Nullspace
2.1 Real Vector Spaces
2.1.1 Example
(1) Let u and v be vectors in R n and k a scalar ( a real number), then we can define
Addition:
u+v,
Scalar multiplication:
ku, kv.
(2) Let P 2 denote the set of all polynomials of the form p=ax 2 +bx+c. For example,
p=0, p=x, p=x+5 are in P 2 . Given any two polynomials p= ax 2 +bx+c and q= a'x 2 +b'x+c',
and a scalar k we can also define
Addition:
p+q= (a+a')x 2 +(b+b')x+(c+c'),
Scalar multiplication: kp= kax 2 +kbx+kc.
(3) Let M22 be the set of all 2 × 2 matrices. Then we can also define the addition and
scalar multiplication in the ordinary way.
About the turn of last century, mathematicians noticed that , in the development of both
mathematics and its applications, many sets of objects were turning up quite naturally with
properties of addition and scalar multiplication in accordance with (A1) through (A7) listed
bellow. Moreover, these properties proved so useful that it was worth while giving a generic
name to all collections of objects that possess such two operations rules. Nowadays they are
called vector spaces( or linear spaces ).
2.1.2 Definition Let V be an arbitrary nonempty set of objects on which two
operations are defined, addition and multiplication by scalars (numbers). By
addition, we mean a rule for associating with each pair of objects u and v in V, an
object called the sum of u and v; by scalar multiplication, we mean a rule for
associating with each scalar k and each object u in V an object k u, called the scalar
multiple of u by k. If the following axioms are satisfied by all objects u, v, w in V
and all scalars k, l, then we call V a vector space and we call the objects in V
vectors:
(A1) If u and v are objects in V, then u + v ∈V
(A2) u + v = v + u
(A3) u + (v + w ) = (u + v ) + w
(A4) There is an object 0 in V, called the zero vector for V such that 0 + u = u + 0 = u for
all u in V.
(A5) For each u in V, there is an object −u in V, called the negative of u, such that
u+(-u)=(-u)+u=0.
(A6) If k ∈ℜ, u ∈V , then ku ∈V
(A7) k( u + v) = ku + kv
(k + l) u = ku + lu, k( lu) = ( kl) u
(A8) 1u = u .
Vector space
R 2 , R 3 , …, R n ,…
P 2 , P 3 ,…,P n ,…,
M 2 2 , M 2 3 ,…,
Exercise:
Verify the conditions (A3) and (A7) for u=(1,-1), v=(0,2), w=(3,3), k=2, l=5.
2.1.3 Proposition
Let V be a vector space and let u ∈ V , k ∈ℜ . Then
(a) 0u = 0
(b) k0 = 0
(c) (−1)u = −u
(d) If ku = 0 , then k = 0 or u = 0
2.1.4 Example
1. The set V = ℜ n with the standard operations of addition and scalar multiplication is a
vector space.
2. The set V of all 2 × 2 matrices with real entries with the standard operations of matrix
addition and matrix scalar multiplication. It is denoted by M22 .
3. The set V of all m × n matrices with real entries under the standard operations of matrix
addition and matrix scalar multiplication. Denote by Mmn .
4. Let V be the set of real-valued functions defined on the entire real line (−∞ , ∞) . Let
f = f ( x ), g = g ( x) be two such func tions in V and let k ∈ℜ. Then
(f + g )( x ) = f ( x ) + g( x ) , (kf )( x ) = kf ( x ) . Then V is a vector space under the defined
operations of addition and scalar multiplication. Denote by F(−∞ , ∞) .
5. The set P n of all polynomials of degree ≤ n is a vector space under the ordinary addition
and scalar multiplication of polynomials.
6. The set V={ (x,y,z): 2x+4y-z=0}, under the standard operations of addition and scalar
multiplication for vectors in ℜ3 is a vector space.
7. Let V = {0} and define addition and scalar multiplication as follows: 0 + 0 = 0 ; k 0 = 0 .
This called the trivial space.
2.1.5 Example The set V = ℜ 2 where addition and scalar multiplication are defined as
follows:
For u = ( u1 , u2 ), v = (v1 , v2 ) , define u + v = (u1 + v1 , u2 + v2 ) ; ku = ( ku1 ,0 ) .
(1) Is condition (A4) satisfied?
(2) Is condition (A7) satisfied?
(3) Is condition (A8) satisfied?
Exercise 2.1
1. Let V=R* be the set of all positive real numbers. For x,y in V and any scalar k, define
x+y=xy, kx=x k .
Determine if V is a vector space.
2. Prove: in a vector space V if k is a scalar and u is a vector s uch that ku=0 then k=0 or
u=0.
3. Show that in any vector space the zero vector is unique..
4.* Let V=R 2 . Suppose the addition of vectors is the usual addition and the scalar
multiplication is defined by k(x,y)=(0,0) if k=0, k(x,y)=(kx, y/k). Determine whether the
axioms (A7) is satisfied.
5*. Find a set V and define the addition + and scalar multiplication such that (V,+,.)
satisfies all axioms except (A7).
2.2 Subspaces
From a given vector space we can construct other new vector spaces. One important type of
spaces obtained from V are the subspaces of V.
2.2.1 Definition A nonempty subset W of a vector space V is called a subspace of V if W is
itself a vector space under the addition and scalar multiplication defined on V.
2.2.2 Remarks
(1) It can be shown that a nonempty subset W of a vector space V is a subspace if and only
if it is closed under addition and scalar multiplication, that is
u, v ∈ W implies u+v ∈ W
ku ∈ W,
and
kv ∈ W for any scalar k.
V
u
v
u+v
ku
W
kv
(2) If B is a subspace and v 1 , v 2 ,K, v r are from B then k1 v 1 +k 2 v 2 +…+k r v r is also
in B for any scalars k1 , k2 ,…,k r .
2.2.3 Example
(1) Let D nn be the set of all the n × n diagonal matrices. Then D nn is a subspace of M nn .
(2) W={(x,y): x+2y=0} is a subspace of R 2 . Is U={(x,y): x+y-1=0} a subspace of
R2 ?
(3) Let B={ (1,2,0,0), (0,1,0,-1)}. Consider W={ k(1,2,0,0)+l(0,1,0,-1): k,l ∈R}.
Then W is a subspace, which is called the subspace spanned by B.
(4) P 2 is a subspace of P 3 . Generally, P m is a subspace of P n if m<n.
(5) Every V has at least two trivial subspaces, V itself and the zero space {0}.
(6) If Ax=0 is a homogeneous linear system of m equations in n unknowns, then the set of
solution vectors is a subspace in ℜn .
Let e1 =(1,0), e2 =(0,1). Then (2,3)=(2,0)+(0,3)=2(1,0)+3(0,1)=2 e1+ 3 e 2. We say the vector
(2,3) is a linear combination of e1 and e2. Every vector in R2 is a linear combination of
e1 and e2.
2.2.4 Definition A vector w is called a linear combination of the vectors v 1 , v 2 ,K, v r , if
w = k1v 1 + k2 v 2 +K+ kr v r ,
for some scalars k1 , k2 ,K, kr ∈ℜ .
2.2.5 Definition Let S = {v 1 , v 2 ,K , v r } be a set of vectors in a vector space V. Let W be the
subspace of V consisting of all linear combinations of the vectors in S. Then W is called the
space spanned by v 1 , v 2 ,K, v r , and we say that the vectors v 1 , v 2 ,K, v r span W. We write
W = span ( S) or W = span {v 1 , v 2 ,K, v r }.
Exercise Show that span{ (1,2),(2,-1)}=R 2.
2.2.6 Remarks
(1) W = span {v 1 , v 2 ,K, v r } is the smallest subspace of V that contains v 1 , v 2 ,K, v r in the
sense that every subspace of V that contains v 1 , v 2 ,K, v r must contain W.
(2) If S = {v 1 , v 2 ,K , v r } and S ′ = {w1 , w2 ,K , wk } are two sets of vectors in a vector space V,
then Span(S)=Span(S') if and only if each vector in S is a linear combination of those in S ′ ,
and conversely, each vector in S ′ is a linear combination of those in S.
Exercise 2.2
1. Determine which of the following are subspaces of the vector space Mnn .
(a) The set of n × n symmetric matrices( A is symmetric if AT =A).
(b) The set of n × n lower triangular matrices
(c) The set of all non-invertible n × n matrices.
2. Determine which of the following are subspaces of F(−∞ , ∞) :
(a) The set of continuous functions.
(b) The set of differentiable functions.
(c) The set of all increasing functions ( f is increasing if x< y implies f(x)<f(y)).
3. Consider the vectors u = (1,2 − 1) and v = (6,4 ,2) in R2. Show that w=(5,2,3) is a linear
combination of u and v, and that w ′ = (4 ,−1,8) is not a linear combination of u and v.
2
4. Determine whether v 1 = (11
, ,2), v 2 = (10
, ,1), v 3 = ( 2,13
, ) span the vector space R .
5. Prove: if a vector u is a linear combination of v 1 , v2 , and each v i is a linear combination
of w1 and w 2 , then u is a linear combination of w 1 and w 2 . Extend this result.
2.3 Linear Independence
For a given space V, generally there are more than one subset that span V. However, among
all such spanning subsets there exist minimal one that contains the smallest number of
vectors. Such a minimal spanning set is called a basis of the space V.
Consider the two groups of vectors B= {(1,2), (1,1),(1,0)} and C={(1,2),(2,3)}.
We can find scalars k,l,m, not all zero, such that
k(1,2)+l(1,1,)+m(1,0)=(0,0).
But
if k(1,2)+l(2,3)=(0,0)
then the only possibility is k=0,l=0.
2.3.1 Definition
Let S = {v 1 , v 2 ,K , v r } be a nonempty set of vectors. Then the vector equation
k1 v1 + k2 v 2 +K kr v r = 0
has at least one solution, namely k1 = 0, k2 = 0 ,K, kr = 0 . If this is the only solution,
then S is called a linearly independent set. If there are other solutions, then S is called a
linearly dependent set.
2.3.2 Examples
(1) { (1,0), (0,1)} is linearly independent in R 2 . In general,
{(1,0,…,0), (0,1,0,…,0), (0,0,…,1)} is linearly independent in R n .
(2) { 1,x,x 2 ,…, x n } is linearly independent in P n .
1 0 0 1  0 0  1 0
(3) { 
, 
 ,
 ,
 } is linearly independent in M 22 .
0 1  0 0  1 0  0 1 
2.3.3 Proposition A set S with two or more vectors is
(a) Linearly dependent if and only if at least one of the vectors in S is expressible as a
linear combination of the other vectors in S.
(b) Linearly independent if and only if no vector of S is expressible as a linear
combination of the other vectors in S.
2.3.4 Examples
(1) If v 1 = (2 ,−1,0,3), v 2 = (1,2,5,− 1), v 3 = (7,−1,5,8 ) , show that
S = {v 1 , v 2 , v 3}
is linearly dependent.
(2) Determine whether v 1 = (1,−2 ,3), v 2 = ( 5,6,− 1), v 3 = (3,2,1) form a linearly dependent set
in R2.
(3) Show that {1+x, x+x2 } is linearly independent in P2.
(4) Show that the functions f1 = x, f 2 = sin x form a linearly independent set of vectors in the
space F(−∞ , ∞) of functions on the real line.
Exercise 2.3
1. Show that the polynomials
p1 = 1 − x, p2 = 5 + 3x − 2 x 2 , p3 = 1 + 3x − x 2 form a linearly dependent
set in P2 .
2. Prove: (a) A finite set of vectors that contains the zero vector is
linearly dependent.
(b) A set with exactly two vectors is linearly independent if and only
if neither vector is a
scalar multiple of the other.
3. Let S = {v 1 , v 2 ,K , v r } be a set of vectors in R2. If r > n , then S is
linearly dependent.
4. Prove if B is a subset of a linearly independent set S, then B is also
linearly independent. Is a subset of linearly dependent set always
linearly dependent?
5. Let S and B be two linearly independent sets of vectors in V. Show
that Span{S} ∩ Span{B} ={0} iff B ∪ S is linearly independent. Find two linearly
independent sets such that their union is linearly dependent.
6. Prove: in the space C(a,b) of all continuous functions defined on [a,b] , the set { 1, x, cos
x} is linearly independent. Is {1, sin 2 x, cos 2 x} linearly independent?
2.4 Basis and Dimension
2.4.1 Definition Let V be a vector space and let S = {v 1 , v 2 ,K , v r } be a set of vectors in V.
Then S is called a basis for V if the following two conditions hold:
•
S is linearly independent;
•
S spans V.
2.4.2 Theorem Let S = {v 1 , v 2 ,K , v n } be a basis for a vector space V . Then every vector v
can be expressed as v = c1v 1 + c2 v 2 +K+cn v n in a unique way.
2.4.3 Definition The scalars c1 , c2 ,K, cn in Theorem 1.4.2 are called the coordinates of v
relative to the basis S. The vector (c1 , c2 ,K, cn ) , denoted by [v] S , in ℜn is called the
coordinate vector of v relative to S.
2.4.4 Remarks For a given basis, the function [.] S : V → ℜn is a bijection.
2.4.5 Examples
(1) Let e 1 = (1, 0, 0, K , 0), e 2 = ( 0,1, 0, K , 0 ),K , e n = (0 , 0, 0, K ,1) . Then S = {e 1 , e 2 , K , e n} is
called the standard basis for ℜn .
(2) { 1,x,x 2 ,…, x n } is called the standard basis of P n .
(3) There is no a finite basis for F(−∞ , ∞) .
2.4.6 Theorem All bases for a vector spaces have the same number of vectors.
2.4.7 Definition
(1) A nonzero vector space is called finite -dimensional if it contains a set of vectors
{v1 , v 2 ,K, v n} that forms a basis. If no such set exists, V is called infinite dimensional. In
addition, the zero vector space is considered to be finite -dimensional,
(2) The dimension of a finite -dimensional vector space V, denoted by dim( V) is defined to
be the number of vectors in a basis for V. In addition, we define the zero vector space to have
dimension zero.
2.4.7 Theorem* If V is an n-dimensional vector space, then:
(a) Every set with more than n vectors is linearly dependent.
(b) If S has n vectors and spans V, then S is a basis.
(c) If S is linearly independent and has n vectors, then S is a basis.
2.4.8 Theorem * Let S be a finite set of vectors in a finite-dimensional vector space V.
(a) If S spans V, but is not a basis for V, then S can be reduced to a basis for V by
removing appropriate vectors from S.
(b) If S is a linearly independent set that is not a basis for V, then S can be enlarged to a
basis for V by inserting appropriate vectors into S.
2.4.9 Examples
(1) dim( ℜn )=n. S = {e 1 , e 2 , K , e n} is a basis for ℜn .
(2) Let v 1 = (1,2,1), v 2 = ( 2,9,0), v 3 = (3,3,4 ) . Show that the set
S = {v 1 , v 2 , v 3} is a basis for ℜ3 .
(3) Let S = {v 1 , v 2 , v 3} be the basis for ℜ3 in (2).
(a) Find the co-ordinate vector of v = (5,−1,9) with respect to S.
(b) Find the vector v in ℜ3 whose co-ordinate vector with respect to S is (v ) S = (−13
, ,2) .
(4) S={ 1,x,…, xn } is a basis for the vector space Pn , so dim( Pn )=n.
The coordinate vector of the polynomial p = a0 + a1 x + a2 x 2 relative to the basis
S = {1, x, x 2 } for P2 is [p] S =(a 0 ,a 1 ,a 2 ).
be the vector space of all polynomials in x . Then B={ 1,x,x 2 ,…, x n , …} is a
(5) Let P[x]
basis of P[x]. So the dimension of P[x] is infinite.
(5) Determine a basis for and the dimension of the solution space of the homogeneous
system
2 x1
+ 2 x2
−
− x1
x1
−
+
+ 2 x3
− 2 x3
x2
x2
+
x5
= 0
− 3 x4
+
−
x5
x5
= 0
= 0
+
+
x5
= 0
x3
x3
x4
Exercise 2.4
1. (1) Prove that S={ (1,2,0),(0,1,1),(1,1,1)} is a basis of ℜ3 . Find the coordinate vector of
v=(1,2,5) with respect to S.
(2) Suppose the coordinate vector of a vector u with respect to S is (2,-1,4). Determine
the vector u.
2. Let S = {v 1 , v 2 ,K , v r } be a linearly independent set in the vector space V. Then S is a basis
for the subspace span{S}.
3. Prove S={ 1+x, x+2x 2 , 1+ x 2 } is a basis of P 2 . Find [a x 2 +bx+c] S .
4. Find the dimension of the solution space of the following linear system
2x-y+z -w=0
x +2z =0
2y +w=0.
5. Prove if S = {v 1 , v 2 ,K , v r } is a basis for V, then for any vector v in V, { v,v1 ,v 2 ,…,v r } is
linearly dependent.
6. Prove that for any vectors u,v and w in a vector space V, { u-v,v-w, w-u}
dependent.
is
linearly
7. Let { v 1 ,v 2 , v3 } be a basis of a space V. Prove:
(i) { v2 , v1 , v3 } is a basis.
(ii) { kv1 ,v 2 , v3 } is a basis for any nonzero scalar k.
(iii) { v1 ,kv 1 +v 2 , v3 } is a basis.
8. Show that if { v1 ,v 2 ,…,v r } is a basis of V, then { v1 , v1 +v 2 ,…,
v 1 +v 2 +…+v r } is also a basis of V.
2.5 Row Space, Column Space, and Nullspace
Given an m n matrix A. The row vectors of A are vectors in R n and the column vectors of A are
vectors in Rm , and the solution vectors of the linear system Ax=0 are vectors in R n . From these
vectors we can construct three vector spaces .
A
Row space
column space
nullspace
I
I
I
n
m
Rn
R
R
2.5.1 Definitions
Let A be an m × n matrix. Then the subspace of ℜn spanned by the row vectors of A is called
the row space of A, and the subspace of ℜm spanned by the column vectors of A is called
the column space of A.
The solution space of the homogeneous system of equations Ax = 0 , which is a subspace of
ℜn , is called the nullspace of A.
2.5.2 Proposition
(1) Elementary row operations do not change the nullspace of a matrix.
(2) Elementary row operations do not change the row space of a matrix.
(3) If a matrix R is in row-echelon form, then the nonzero row vectors
form a basis for the row space of R.
Recall that two matrices are said to be row equivalent if one of them can be obtained from
another by using elementary row operations.
A
B
2.5.3 Proposition Let A and B be row equivalent matrices. Then
(a) A given set of column vectors of A is linearly independent if and only if the
corresponding column vectors of B are linearly independent.
(b) A given set of column vectors of A forms a basis for the column space of A if and
only if the corresponding column vectors of B form a basis for the column space of B.
(c) If a matrix A is in row-echelon form, then the column vectors that contains a leading
1 form a basis for the column space of A.
2.5.4 Remarks
(1) By 1.5.2, to find a basis for the row space of A, we first reduce A to its row echelon
form. Then the rows containing leading 1 form a basis of the row space.
(2) To find a basis of the column space of A, just choose those columns in A
corresponding to the columns in the row echelon form of A that contains a leading 1.
(3) From (1) and (2) it follows that the row space and the column space of a matrix A
have the same dimension.(Why?)
2.5.5 Example Let S={ (1,1,0), (1,2,3), (0,1,3)}. Find a basis of the space Span{S}.
1 1 0
Solution: Let A= 1 2 1 . Then Span{S} is the column space of A.


0 3 3
Reduce A to row echelon form
1 1 0 
A'= 0 1 1  .
0 0 0 
The first and second columns of A' contain leading 1, so the corresponding columns in A
form a basis of A, and therefore of Span{S}, that is { (1,1,0),(1,2,3)}.
2.5.6 Algorithm: Given a set of vectors S = {v 1 , v 2 ,K , v k } in ℜn . To find a subset of these
vectors that form a basis for span(S):
Step 1: Form the matrix A having v 1 , v 2 ,K, v k as its column vectors.
Step 2: Reduce the matrix A to its reduced row-echelon form R, and let w1 , w 2 ,K, w k be
the column vectors of R.
Step 3: Identify the column vectors that contain a leading 1. These vectors form a basis
for span(S). The corresponding column vectors of A are the basis vectors for span( S).
2.5.7 Definition Let A be a matrix. The dimension of the row space ( or the column space) of
A is called the rank of A, denoted by rank( A).
The dimension of the nullspace of A is called the nullity of A, denoted by nullity( A).
2.5.8 Theorem( D imension theorem for matrices) Let A have n columns. Then
rank( A) + nullity( A) = n.
2.5.9 Theorem*
Given An ×n with corresponding linear transformation T :ℜ n → ℜ n , where T(u)=Au. Then the
following statements are equivalent.
(a) A is invertible
(b) Ax = 0 has only the trivial solution.
(c) The reduced row-echelon form of A is In .
(d) A is a product of elementary matrices.
(e) Ax = b is consistent for every n × 1 matrix b .
(f) Ax = b has exactly one solution.
(g) det(A) ≠ 0.
(h) The range of T is ℜn
(i) T is injective.
(j) The column vectors of A are linearly independent
(k) The row vectors of A are linearly independent.
(l) The column vectors of A span ℜn .
(m) The row vectors of A span ℜn .
(n) The column vectors of A form a basis for ℜn .
(o) The row vectors of A form a basis for ℜn .
(p) Rank(A) = n
(q) Nullity(A) = 0
Exercise 1.5
1. Find a basis for the space spanned by the vectors
v 1 = (1,− 2,0,0,3), v 2 = (2 ,−5,−3,−2,6 ), v 3 = (0,51510
, , ,0 ), v 4 = (2 ,6,18,8,6 ) .
What is the dimension of the spanned subspace?
2. Find the rank and the nullity of the matrices below:
2
A=  0
− 1
0
1
1
0
2
1
− 1
1 1 0

0  , B=  2 1 1  .
− 1 0 − 1
1 
Verify the result that rank(A) + nullity(A) = n .
3. Let A be a 7 × 4 matrix. Show that the row vectors of A must be linearly dependent.
4.Give an example to show that elementary row operations may change the column space of a
matrix.
5. Prove that a system of linear equation Ax = b is consistent if and only if b is in the column
space of A.
6. Explain why the row space and the column space of a matrix have the same dimension.
7*. Let A and B be two n × n matrices. Determine which of the following is true for all A
and B:
(a) rank(AB)=rank(A)rank(B),
(b) rank(AB)=max{ rank(A),rank(B)},
(c) rank(AB)=min{rank(A),rank(B)}.