Download Row Space, Column Space, and Null Space

3-8-2013
Row Space, Column Space, and Null Space
Definition. Let A be an m × n matrix.
(a) The row vectors of A are the vectors in F n corresponding to the rows of A. The row space of A
is the subspace of F n spanned by the row vectors of A.
(b) The column vectors of A are the vectors in F n corresponding to the columns of A. The column
space of A is the subspace of F n spanned by the column vectors of A.
Example. Consider the real matrix

1 0
A = 0 1.
0 0

The row vectors are (1, 0), (0, 1), and (0, 0). The row space is the subspace of R2 spanned by these
vectors. Since the first two vectors are the standard basis vectors for R2 , the row space is R2 .
The column vectors are (1, 0, 0) and (0, 1, 0). The column space is the subspace of R3 spanned by these
vectors. Thus, the column space consists of all vectors of the form
a · (1, 0, 0) + b · (0, 1, 0) = (a, b, 0).
Lemma. If E is an elementary row operation and A is a matrix, then E(A) has the same row space as A.
Proof. If E is an operation of the form ri ↔ rj , then E(A) and A have the same rows (except for order), so
it’s clear that their row vectors have the same span.
If E is an operation of the form ri → ari , then A and E(A) agree except in the i-th row. Since
1
a 1 r1 + · · · + a i ri + · · · + a m rm = a 1 r1 + · · · + a i
(ari ) + · · · + am rm ,
a
any element of the row space of A is in the row space of E(A), and vice versa.
Finally, suppose E is a row operation of the form ri → ri + arj . Then
a1 r1 + · · · + ai ri + · · · + am rm = a1 r1 + · · · + ai (ri + arj ) + · · · + (aj − ai a)rj + · · · + am rm ,
which shows that the row space of A is contained in the row space of E(A).
Conversely,
a1 r1 + · · · + ai (ri + arj ) + · · · + aj rj + · · · + am rm = a1 r1 + · · · + ai ri + · · · + (ai a + aj )rj + · · · + am rm ,
so the row space of E(A) is contained in the row space of A.
Definition. Two matrices are row equivalent if one can be obtained from the other via elementary row
operations.
Since row operations preserve row space, row equivalent matrices have the same row space. In particular,
a matrix and its row reduced echelon form have the same row space.
Lemma. Let R = {rij } be a row reduced echelon matrix with nonzero rows r1 , . . . , rp . Suppose the leading
coefficients of R occur at
(1, j1 ), (2, j2 ), . . . , where j1 < j2 < · · · .
1
If v = (v1 , . . . , vn ) and
v = a 1 r1 + · · · + a p rp ,
then vjk = ak .
In other words, this lemma describes the components of a vector in the row space of R.
Proof.
vjk =
p
X
ai rijk .
i=1
But the only nonzero element in column jk is rkjk = 1. Therefore, the only nonzero term in the sum is
ak rkjk = ak .
Example. Here is a row reduced echelon matrix over R:
0
0

0
0

1
0
0
0
2
0
0
0

0 −1 0
1 2 0
.
0 0 1
0 0 0
Note that j1 = 2, j2 = 4, and j3 = 6. Consider the following element of the row space:
5 · (0, 1, 2, 0, −1, 0) + (−3) · (0, 0, 0, 1, 2, 0) + 1 · (0, 0, 0, 0, 0, 1) = (0, 5, 10, −3, −11, 1).
Then
v2 = a1 = 5,
v4 = a2 = −3,
v6 = a3 = 1.
Corollary. The nonzero rows of a row reduced echelon matrix are independent.
Proof. Suppose R is a row reduced echelon matrix with nonzero rows r1 , . . . , rp . Suppose the leading
coefficients of R occur at (1, j1 ), (2, j2 ), . . ., where j1 < j2 < · · ·. If
0 = a 1 r1 + · · · + a p rp ,
then the lemma implies that ak = vjk = 0 for all k. Therefore, the {ri } are independent.
Corollary. The nonzero rows of a row reduced echelon matrix form a basis for the row space of the matrix.
Proof. The nonzero rows span the row space, and are independent, by the preceding corollary.
Algorithm. Let V be a finite-dimensional vector space, and let v1 , . . . , vm be vectors in V . The object is to
find a basis for W = hv1 , . . . , vm i, the subspace spanned by the vi .
Let M be the matrix whose i-th row is vi . The row space of M is W . Let R be a row-reduced echelon
matrix which is row equivalent to M . Then R and M have the same row space W , and the nonzero rows of
R form a basis for W .
Example. Consider the vectors v1 = (1, 0, 1, 1), v2 = (−2, 1, 1, 0), and v3 = (7, −2, 1, 3) in R4 . I’ll find a
basis for the subspace (v1 , v2 , v3 ) spanned by the vectors. Construct a matrix with the vi as its rows and
row reduce:




1 0 1 1
1
0 1 1
 −2 1 1 0  →  0 1 3 2 
0 0 0 0
7 −2 1 3
2
The vectors (1, 0, 1, 1) and (0, 1, 3, 2) form a basis for (v1 , v2 , v3 ).
Example. Determine the dimension of the subspace of R3 spanned by (1, 2, −1), (1, 1, 1), and (2, −2, 1).



1 0 0
1 2 −1
1 1
1  → 0 1 0
2 −2 1
0 0 1

The subspace has dimension 3, since the row reduced echelon matrix has 3 nonzero rows.
Definition. The rank of a matrix is the dimension of its row space.
Example. Consider the following matrix over Z5 :

1 4 2 1
3 3 1 2.
0 1 0 4

Row reduce it:



1 0 2 0
1 4 2 1
3 3 1 2 → 0 1 0 4
0 0 0 0
0 1 0 4

The row reduced echelon matrix has 2 nonzero rows. Therefore, the original matrix has rank 2.
I’ll need the following fact about matrix multiplication for the proof of the next lemma. Consider the
following multiplication:


← r1 →
 ← r2 → 
.
[ a1 a2 · · · an ] 
..


.
← rn →
In doing the multiplication, each a multiplies the corresponding row r. Here’s the picture:
[ a1 a 2 ... an ] a1
a2
r1
r2
an
rn
Therefore,
[ a1
a2
←
←
· · · an ] 


r1
r2
..
.
← rn

→
→
 = [ a 1 r1 + a 2 r2 + · · · + a n rn ] .

→
3
If instead of a single row vector on the left I have an entire matrix, here’s what I get:
a11
 a21
 .
 ..

am1
a12
a22
..
.
···
···

a1n
←
a2n   ←

.. 
. 
am2
· · · amn
r1
r2
..
.
← rn
 
→
←
→ ←
=
 
→
a11 r1 + a12 r2 + · · · + a1n rn
a21 r1 + a22 r2 + · · · + a2n rn
..
.
← am1 r1 + am2 r2 + · · · + amn rn

→
→
.

→
Here’s the point: The rows of the product are linear combinations of the rows r1 , r2 , . . . rn .
Lemma. Let M and N be matrices over a field F which are compatible for multiplication. Then
rank(M N ) ≤ rank N.
Proof. The preceding discussion shows that the rows of M N are linear combinations of the rows of N .
Therefore, the rows of M N are all contained in the row space of N .
The row space of N is a subspace, so it’s closed under taking linear combinations of vectors. Hence,
any linear combination of the rows of M N is in the row space of N . Therefore, the row space of M N is
contained in the row space of N .
From this, it follows that the dimension of the row space of M N is less than or equal to the dimension
of the row space of N — that is, rank(M N ) ≤ rank N .
I already have one algorithm for testing whether a set of vectors in F n is independent. That algorithm
involves constructing a matrix with the vectors as the columns, then row reducing. The algorithm will also
produce a linear combination of the vectors which adds up to the zero vector if the set is dependent.
If all you care about is whether or not a set of vectors in F n is independent — i.e. you don’t care about
a possible dependence relation — the results on rank can be used to give an alternative algorithm. In this
approach, you construct a matrix with the given vectors as the rows.
Algorithm. Let V be a finite-dimensional vector space, and let v1 , . . . , vm be vectors in V . The object is to
determine whether the set {v1 , . . . , vm } is independent.
Let M be the matrix whose i-th row is vi . Let R be a row reduced echelon matrix which is row equivalent
to M . If R has m nonzero rows, then {v1 , . . . , vm } is independent. Otherwise, the set is dependent.
If R has p nonzero rows, then R and M have rank p. (They have the same rank, because they have the
same row space.) Suppose p = m. Since {vi } spans, some subset of {vi } is a basis. However, a basis must
contain p = m elements. Therefore, {vi } must be independent.
Any independent subset of the row space must contain ≤ p elements. Hence, if m > p, {vi } must be
dependent.
Example. The vectors v1 = (1, 0, 1, 1), v2 = (−2, 1, 1, 0), and v3 = (7, −2, 1, 3) in R4 are dependent:

1
0
 −2 1
7 −2



1 0 1 1
1 1
1 0 → 0 1 3 2
0 0 0 0
1 3
The row reduced echelon matrix has only two nonzero rows.
I already know that every matrix can be row reduced to a row reduced echelon matrix. The next result
completes the discussion by showing that the row reduced echelon form is unique
Proposition. Every matrix can be row reduced to a unique row reduced echelon matrix.
Proof. Suppose M row reduces to R, a row reduced echelon matrix with nonzero rows r1 , . . . , rp . Suppose
the leading coefficients of R occur at (1, j1 ), (2, j2 ), . . ., where j1 < j2 < · · ·.
4
Let W be the row space of R and let
v = a 1 r1 + · · · + a p rp
be an element of W . In component form, v = (v1 , . . . , vn ).
Claim: The first nonzero component of v must occur in column jk , for k = 1, 2, . . ..
Suppose ak is the first ai which is nonzero. The sum looks like
a k rk + · · · + a p rp .
The first nonzero element of rk is a 1 at (k, jk ). The first nonzero element in rk+1 , . . . , rp lies to the
right of column jk . Thus, vj = 0 for j < jk , and vjk = ak . Evidently, this is the first nonzero component of
v. This proves the claim.
This establishes that if a row reduced echelon matrix R′ is row equivalent to M , its leading coefficients
must lie in the same columns as those of R. For the rows of R′ are elements of W , and the claim applies.
Next, I’ll show that the nonzero rows of R′ are the same as the nonzero row of R.
Consider, for instance, the first nonzero rows of R and R′ . Their first nonzero components are 1’s lying
in column j1 . Moreover, both r1 and r1′ have zeros in columns j2 , j3 , . . . .
Suppose r1 6= r1′ . Then r1 − r1′ is a nonzero vector in W whose first nonzero component is not in column
j1 , j2 , . . . , which is a contradiction.
The same argument applies to show that rk = rk′ for all k. Therefore, R = R′ .
I showed earlier that you can add vectors to an independent set to make a basis. Here’s how it works in
a particular case.
Example. Add vectors to the following set to make a basis for R5 :
{(2, 4, 3, 0, 7), (1, 2, 0, 3, −1), (1, 2, 2, 0, 9), (−2, −4, −2, −2, −3)} .
If I make a matrix with these vectors as rows and row reduce, the row reduced echelon form will have
the same row space (i.e. the same span) as the original set of vectors:
2
 1

1
−2



1
4
3
0
7
2
0
3 −1 
0
→
0
2
2
0
9
0
−4 −2 −2 −3
2
0
0
0
0
1
0
0
0
0
1
0

0
0

0
1
Since there are four nonzero rows, and since these rows are clearly independent vectors, the original set
of vectors is independent.
By examining the row reduced echelon form, I see that the vector (0, 1, 0, 0, 0) will not be a linear
combination of the others. (I’m choosing a standard basis vector with a “1” in a position not occupied by a
leading coefficient.) Therefore, I can add it to the set and get a new independent set:
{(2, 4, 3, 0, 7), (1, 2, 0, 3, −1), (1, 2, 2, 0, 9), (−2, −4, −2, −2, −3), (0, 1, 0, 0, 0)} .
There are 5 vectors in this set, so it is a basis for R5 .
I also showed earlier that you can remove vectors from a spanning set to get a basis. You can do this
using the same algorithm that gives a basis for the column space of a matrix.
First, here’s a reminder about matrix multiplication. If A is an m × n matrix and V ∈ F n , then the
product Av is a linear combination of the columns of A.
5
In fact, if ci is the i-th column of A and v = (v1 , . . . , vn ),

↑
 c1
↓
↑
c2
↓
 
 v1
↑  v2 

· · · cn  
 ...  = v1 c1 + v2 c2 + · · · + vn cn .
↓
vn
Lemma. Let A be a matrix, and let R be the row reduced echelon matrix which is row equivalent to A.
Suppose the leading coefficients of R occur in columns j1 , . . . , jp , where j1 < · · · < jp , and let ci denote the
i-th column of A. Then {cj1 , . . . , cjp } is independent.
Proof. Suppose that
aj1 cj1 + · · · + ajp cjp = 0,
for
ai ∈ F.
Form the vector v = (vi ), where
vi =
0
ai
if i ∈
/ {j1 , . . . , jp }
if i ∈ {j1 , . . . , jp }
The equation above implies that Av = 0.
It follows that v is in the solution space of the system Ax = 0. Since Rx = 0 has the same solution
space, Rv = 0. Let c′i denote the i-th column of R. Then
0 = Rv = aj1 c′j1 + · · · + ajp cjp .
However, since R is in row reduced echelon form, c′jk is a vector with 1 in the k-th row and 0’s elsewhere.
Hence, {cj1 , . . . , cjp } is independent, and aj1 = · · · = ajp = 0.
Example. Consider the real matrix

1 1
A = 2 0
1 1
It row reduces to

−2 3
−1 1  .
1 0

1 0 0 0
R = 0 1 0 0 .
0 0 1 −1

The leading coefficients occur in the first three columns. Hence, the first three columns of A are
independent:
    

2 
1
 1
 2  ,  0  ,  −1 


1
1
1
In fact, they form a basis for the column space of A.
Example. Find a subset of the following set of vectors which forms a basis for R3 .
  

   
4 
1
−1
 1
 2  ,  1  ,  1  ,  −1 


2
1
−1
1
6
Make a matrix with the vectors as columns and row reduce:

2


1 0
1 −1 1 4

3
 2 1 1 −1  → 
1

0 1 −
1 −1 1 2
3
0 0
0
0




0
1
The leading coefficients occur in columns 1, 2, and 4. Therefore, the corresponding columns of the
original matrix are independent, and form a basis for R3 :
  

 
4 
−1
 1
 2  ,  1  ,  −1  .


2
−1
1
Proposition. Let A be a matrix. Then
row rank(A) = column rank(A).
Proof. Let R be the row reduced echelon matrix which is row equivalent to A. Suppose the leading
coefficients of R occur in columns j1 , . . . , jp , where j1 < · · · < jp , and let ci denote the i-th column of A. By
the preceding lemma, {cj1 , . . . , cjp } is independent. There is one vector in this set for each leading coefficient,
and the number of leading coefficients equals the row rank. Therefore,
row rank(A) ≤ column rank(A).
Now consider AT . This is A with the rows and columns swapped, so
row rank(AT ) = column rank(A),
column rank(AT ) = row rank(A).
Applying the first part of the proof to AT ,
column rank(A) = row rank(AT ) ≤ column rank(AT ) = row rank(A).
Therefore,
column rank(A) = row rank(A).
Remark. The proof provides an algorithm for finding a basis for the column space of a matrix. Specifically,
row reduce the matrix A to a row reduced echelon matrix R. If the leading coefficients of R occur in columns
j1 , . . . , jp , then consider the columns cj1 , . . . , cjp of A. These columns form a basis for the column space of
A.
Example. Consider the real matrix
1 −2
2 1

0 −5
7 1

It row reduces to
1
0

0
0


3 1 1
0 3 1
.
6 −1 1
3 10 4

0 0.6 1.4 0.6
1 −1.2 0.2 −0.2 

0
0
0
0
0
0
0
0
7
The leading coefficients occur in columns 1 and 2. Therefore, (1, 2, 0, 7) and (−2, 1, −5, 1) form a basis
for the column space of A.
Example. If A and B are row equivalent, they don’t necessarily have the same column space. For example,
1 2 1
→
1 2 1
.
1 2 1 r2 → r2 − r1 0 0 0
However, all the elements of the column space of the second matrix have their second component equal
to 0; this is obviously not true of elements of the column space of the first matrix.
Proposition. Let A, B, P and Q be matrices, where P and Q are invertible. Suppose A = P BQ. Then
rank A = rank B.
Proof. I showed earlier that rank M N ≤ rank N . This was row rank; a similar proof shows that
column rank(M N ) ≤ column rank(M ).
Since row rank and column rank are the same, rank M N ≤ rank M .
Now
rank A = rank P BQ ≤ rank BQ = column rank(BQ) ≤ column rank(B) = rank B.
But B = P −1 AQ−1 , so repeating the computation gives rank B ≤ rank A. Therefore, rank A = rank B.
Definition. The null space (or kernel) of a matrix A is the set of vectors ~x such that A~x = ~0. The
dimension of the null space of A is called the nullity of A, and is denoted nullity(A).
Remark. The null space is the same as the solution space of the system of equations A~x = ~0. I showed
earlier that if A is an m × n matrix, then the solution space is a subspace of F n . Thus, the null space of a
matrix is a subspace of F n .
Example. Consider the real matrix
A=
−1 1
.
1 1
3
−1
The vector (1, 2, −1) is in the null space of A, since

1
3 −1 1 
0

2 =
.
−1 1 1
0
−1

The vector (1, 1, 1) is not in the null space of A, since
3
−1
 
1
−1 1  
3
0
1 =
6=
.
1 1
1
0
1
8
Algorithm. Let A be an m × n matrix. The object is to find a basis for the null space of A.
Let ~x = (x1 , x2 , . . . , xn ). Solve the following system by row reducing A to row-reduced echelon form:
A~x = ~0.
In the row reduced echelon form, suppose that {xi1 , xi2 , . . . , xip } are the variables corresponding to the
leading coefficients, and suppose that {xj1 , xj2 , . . . , xjq } are the free variables. Note for future reference that
p + q = n.
As usual, put the solution in parametric form, writing {xi1 , xi2 , . . . , xip } in terms of {xj1 , xj2 , . . . , xjq }:
xi1 = fi1 (xj1 , xj2 , . . . , xjq )
xi2 = fi2 (xj1 , xj2 , . . . , xjq )
..
.
xip = fip (xj1 , xj2 , . . . , xjq )
Plug the expressions for {xi1 , xi2 , . . . , xip } into the general solution vector ~x = (x1 , x2 , . . . , xn ), expressing it in terms of{xj1 , xj2 , . . . , xjq }. Schematically, the result looks like this:
 (f
′
 
 
 
s) 
∗
∗
∗
0
0
1
 x j1 
 
 

 

∗
∗
∗
   (fik ′ s) 
x1
 
 

 

0
1
0
 x j2 

 




 x2  
 .  =  (fik ′ s)  = xj  ∗  + xj  ∗  + · · · + xj  ∗  .
q 
2 
1 




 ..  
..
 ... 
 ... 

 ... 

.
 
 

 

xn
∗
∗

∗

 
 
 
 (fik ′ s) 
1
0

0

x jq
′
∗
∗
∗
(fik s)
ik
In the last expression, the vectors which are being multiplied by xj1 , xj2 , . . . , xjq form a basis for the
null space.
The vectors span the null space, because the equation above has expressed an arbitrary vector in the
null space as a linear combination of the vectors.
The vectors are independent, because (ignoring the ∗’s) the 1’s and 0’s in the j1 , j2 , . . . , jq components
form a q-dimensional standard basis.
This description is probably hard to understand with all the subscripts flying around, but you’ll see in
the examples below that it’s very easy to implement. Before giving an example, here’s an important result
that comes out of this discussion. Notice that p, the number of leading coefficient variables, is the rank of
A. Notice also that q, the number of free variables, is the same as the number of vectors in the basis for the
null space. That is, q = nullity(A). Finally, I observed earlier that p + q = n.
Theorem. Let A be an m × n matrix. Then
n = rank A + nullity A.
Example. Find the nullity and a basis for the null space of the real matrix

1 2
1 2
2 4

0 3
1 −2  .
1 1
9
Row reduce the matrix to row-reduced echelon form:




1 2 0 3
1 2 0 3
 1 2 1 −2  →  0 0 1 −5 
0 0 0 0
2 4 1 1
I’ll use w, x, y, and z as my solution variables. Thinking of the last matrix as representing equations
for a homogeneous system, I have
w + 2x + 3z = 0,
y − 5z = 0,
or
or
w = −2x − 3z,
y = 5z.
Thus,
 





w
−2x − 3z
−2
−3
x

x 
 1 
 0 
 = x
+z
 =
.
y
5z
0
5
z
z
0
1

{(−2, 1, 0, 0), (−3, 0, 5, 1)} is a basis for the null space. The nullity is 2.
Notice that the rank is 2, the number of columns is 4, and 4 = 2 + 2.
Example. Consider the following matrix over Z3 :

1 1 0 2
A = 2 2 1 2.
1 1 1 0

Find bases for the row space, column space, and null space.
Row reduce the matrix:



1 1 0 2
1 1 0 2
2 2 1 2 → 0 0 1 1
0 0 0 0
1 1 1 0

{(1, 1, 0, 2), (0, 0, 1, 1)} is a basis for the row space.
The leading coefficients occur in columns 1 and 3. Taking the first and third columns of the original
matrix, I find that {(1, 2, 1), (0, 1, 1)} is a basis for the column space.
Using a, b, c, and d as variables, I find that the row reduced matrix says
a + b + 2d = 0,
or
a = 2b + d,
c + d = 0,
or
c = 2d.
Thus,

 
  
 
2
2b + d
a
1
1
b  b 
0
 = b  + d .
 =
2d
c
0
2
d
d
0
1
Therefore, {(2, 1, 0, 0), (1, 0, 2, 1)} is a basis for the null space.
c 2013 by Bruce Ikenaga
10