Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Linear Algebra Course Notes 1. Matrix and Determinants 2 1.1
Document related concepts
Quadratic form wikipedia , lookup
Tensor operator wikipedia , lookup
Covariance and contravariance of vectors wikipedia , lookup
Matrix (mathematics) wikipedia , lookup
Cartesian tensor wikipedia , lookup
Determinant wikipedia , lookup
Non-negative matrix factorization wikipedia , lookup
Jordan normal form wikipedia , lookup
Singular-value decomposition wikipedia , lookup
Orthogonal matrix wikipedia , lookup
Perron–Frobenius theorem wikipedia , lookup
Bra–ket notation wikipedia , lookup
System of linear equations wikipedia , lookup
Eigenvalues and eigenvectors wikipedia , lookup
Four-vector wikipedia , lookup
Basis (linear algebra) wikipedia , lookup
Matrix calculus wikipedia , lookup
Cayley–Hamilton theorem wikipedia , lookup
Transcript
Linear Algebra Course Notes
QIRUI LI
C ONTENTS
1. Matrix and Determinants
2
1.1. What is the matrix do for linear algebra
2
1.2. Field and Number Matrix
8
1.3. Matrix Multiplication
11
1.4. Block Matrix Multiplicaiton
19
1.5. Elementary Matrices, Row and Column Transformations
22
1.6. Determinant
42
1.7. Laplacian Expansion, Cofactor, Adjugate, Inverse Matrix Formula
49
2. Linear Equation
55
2.1. Non-Homogeneous Linear Equation, The existence of the solution
60
2.2. Homogeneous Linear Equation, The uniqueness of the solution
63
3. Vector Spaces, Linear Transformation
68
3.1. Actions, Matrix of Actions
68
3.2. Linear Spaces, dimension
71
3.3. Linear Maps and Linear Transformation
102
3.4. Attempting to go back - Isomorphism, Kernel and Image
112
3.5. Invariant subspace for linear transformation. Power-Rank.
118
4. Eigenvalues and Eigenvectors
130
4.1. Finding Eigenvalues and Eigenvectors
130
4.2. Linear Independence of eigenvector, Algebraic and Geometrical multiplicity, Eigen
spaces.Diagonalization.
135
4.3. Polynomial of linear transformation, Spectral mapping theorem, minimal polynomial of linear
transformation.
153
4.4. (Coming soon, Not in Final)Jordan Canonical Form
159
4.5. (Coming soon, Not in Final)Root subspace and classification of invariant subspaces
159
4.6. (Coming soon, Not in Final)Classification of linear transformation commuting with T
159
Date: Last Update:16:22 Wednesday 1st July, 2015;
1
1. M ATRIX AND D ETERMINANTS
Introduction: Before you explore the interesting world of Linear Algebra. You should consolidate your
computation skills and tools. The Matrix and Determinants are the core for all kinds of Linear Algerba
Problems. Like whater and air, we can’t live without them. We will concentrate on calculation in this
chapter
1.1. What is the matrix do for linear algebra. -
1.1.1. Geometric intuition for linear transformations. Introduction: We will introduce conceptually why matrix is important and what should we always think
about when we first study matrices.
Think about how people present a cubic box in a piece of paper:
It is a common sense that angle A of the above picture should be a right angle. But when you use
protractor to measure A, you will see it is an obtuse angle. How does our brain work such that we can
correctly recognize that right angle although it is obtuse as apeared on paper?
Our brain is doing ”linear transformation” everytime so that we can understand space from our eyes.
Now let’s think about how our brain work. If you are good at drawing, how do you draw a cubic box on
paper so that it looks really like a box? (Suppose the distance of you and the box is far enough compared to
the size of the box)
As we trying to make it real to us, we are keeping parallel lines when we draw. This is exactly how
objects appear in our eyes. parallel kept to be parallel. In order to understand the world in the photo, or do
better in arts, we study linear algebra.
Look at following pictures.
2
3
In the first two pictures we map parallel lines to parallel lines, so it’s linear. In the third picure, parallel
lines does not mapped to parallel lines, it’s not linear. In the last graph, they even mapped lines to curves,
this like the world you see in distorting mirrow.
1.1.2. How to represent a linear transformation. We conceptually understand what is linear map, a map where parallel lines are kept. But how to explicitly
represent them?
It is a common sense that rotation is a linear map, because parallel lines rotated to parallel lines. Now
let’s specifically study the rotation counterclockwise by 90 degree.
Problem 1. What is the coordinates of point (1,0) after rotating 90 degree counterclockwise? How about
point (0,1)?
We know that (1,0) is the vector that pointing rightwards, after a rotation of 90 degree counterclockwise,
it would becom a vector that pointing upwards with length invariant, that is, (0,1). The vector (1,0) is facing
upwards, after rotation, it will facing leftwards, that is, (-1,0). Suppose we denote the map as T, then we
know
T((1,0)) = (0,1)
T((0,1)) = (-1,0)
In order to fully understand a linear map, we need to know where every point goes.
Problem 2. Looking at the folowing picture. Do you know where will (3,5) map to after rotating 90 degree
counterclockwise? or what is T(3,5)?
We draw the auxillary nets, we claim these squares in the nets maped to squares in the nets, because rotate
a square counterclockwise by 90 degree is still a square.
4
Before the rotation, we reach the point (3,5) by going right 3 units and going up 5 units.
After rotation conterclockwise, right became up, and up became left. After the rotation, (3,5) maps to
the point that should be got by going up 3 units and going left 5 units, which is exactly (-5,3)
The following pictures will help you to organize what is happening.
Remember the T:
T((1,0)) = (0,1)
T((0,1)) = (-1,0)
Note that (1,0) means right. and (0,1) means up. (0,1) means up and (-1,0) means left. Then we can use
coordinate to explain what is hapenning:
5
All the fact explains T((3,5))=(-5,3)
Not only for (3,5), but also for any arbitrary point, we can know where it will map to by this method. for
example, T((2,1))=(-1,2), T((9,9))=(-9,9). Thus, to figure out a formula for general points. We assume
T((x,y)) = (u,v)
If we can express (u,v) in term of (x,y), we then dare to say we fully understand what is rotation. We know
that (x,y) will rotate to (-y,x), that is u=-y, and v=x. Then we write the system of equation in the following
standard form.
(1)
u = 0x + (-1)y
v = 1x + 0y
With previous understanding, we know that in order to know how a linear transformation on a plane acts
on every point is enough to know how that act on (1,0) and (0,1). And the coefficient in the system of
equation determines the linear map completely. People then refine all the coefficient into a matrix to denote
the specific linear transformation.
0 −1
With previous calculation. Rotation by 90 degree conterclockwise is
1 0
We already studied how to get a matrix from linear trasformation. Now we end this section by showing
an example how to get the linear transformation from matrix.
1 0
Problem 3. How do you understand
? Someone said it is reflection by x-axis, will you believe
0 −1
that?
We know that this means a linear map
T((x,y)) = (u,v)
, where the details of the map shoule be given by the following equation:
(2)
u = 1x + 0y
v = 0x + -1y
6
Let’s see where will (3,4) maps to. Plug (x,y)=(3,4) in that equation, we get (u,v)=(3,-4), which means
T((3,4)) = (3,-4)
It is exactly get by reflection by x axis. With checking more and more points, we found that it make sense
that it is reflection by x-axis.
All the sentences in this section is conceptual. None of them are strict proofs or definitions. With this
interpretations of matrix in mind, you will feel better to study matrix and better in understanding matrix
multiplication. Note that this interpretation only make sense when it is a matrix over R and assuming
basis is given and fixed. We will explain every terminology in the future. Next, we will develop a strict
theory for matrices. This section have nothing to do with the strict theory but makes you feel better on
understanding further concepts. Although matrix have far more meanings than only linear transformation,
keeping geometry in mind will help you understand fast and easy.
7
1.2. Field and Number Matrix. Goal: Conceptually speaking, matrix is describing a linear transformation when elemets are in R,
but matrix itself is more than that. In this section, you are going to learn the most basic, strict, or even
abstract concepts of number matrix, to start with, we should define every term: Number and Matrix. To
define what is numbers, we define the terminology: Fields.
1.2.1. Fields.
Definition 1. We call a set F that equipped with two two-term-operations + and × a Field if this set satisfies
the following axioms
(1) Closed under addition and multiplication if a ∈ F and b ∈ F , then a + b ∈ F and a × b ∈ F
(2) Commutativity of + For any two elements a, b ∈ F , we have a + b = b + a
(2) Associativity of + For any three elements a, b, c ∈ F we have (a + b) + c = a + (b + c)
(3) Existence of 0 There exists an element 0, such that for any elemnt a ∈ F ,a + 0 = a.
(4) Existence of opposite For any element a ∈ F , There exists an element called −a, such that a +
(−a) = 0
(5) Commutativity of × For any two elements a, b ∈ F , we have a × b = b × a
(6) Associativity of × For any three elements a, b, c ∈ F we have (a × b) × c = a × (b × c)
(7) Existence of 1 There exists an element 1, such that for any elemnt a ∈ F , a × 1 = a.
(8) Existence of reciprocal For any non-zero lement a 6= 0 ∈ F , There exists an element called a−1 ,
such that a × a−1 = 1
(9) Distribute property for any a, b, c ∈ F , we have a × (b + c) = a × b + a × c
As you see, there is nothing strage for us. The importance of this axioms is that not only real number or
complex numbers could be numbers. There are more other type of useful numbers
Example 1. A simple example of a field is the field of rational numbers, consisting of numbers which can
be written as fractions ab , where a and b are integers, and b 6= 0. The opposite of such a fraction is simply
− ab , and the reciprocal (provided that a 0) is ab .
Example 2. Suppose F only has two elements, namely 0 and 1, they are zero element and unit element
respectively. And we define 1 + 1 = 0 ,0 + 1 = 1,0 + 0 = 0,0 ∗ 0 = 0,0 ∗ 1 = 0, 1 ∗ 1 = 1. Then this is a
field. This is the minium size field in the universe — field of two element.
Example 3. It is clear that real numbers(all rational and irrational numbers) formed a field. Later on
we will use complex numbers, which can be expressed as a + bi, where i is the mysterious element that
saitisfying i2 = −1. We can verify that complex numbers also satisfies the axioms of field. In the book ,we
denote real number field as R, and complex number field as C
In this book, the element in field are always called number or scalar.
In the study of linear algebra, you will not only encounter calculation, but also proofs. proofs are logic
analysis of why some statement is right based on axioms. To show an example, we proof the following
proposition.
Proposition 1. For any element a ∈ F , a × 0 = 0
Proof. We proof as following:
Assume c = a × 0
We Claim that c = c + c, because:
By definition of 0, we have 0 + 0 = 0.
Thus c = a × 0 = a × (0 + 0) = a × 0 + a × 0 = c + c
So we verified c = c + c
By the existance of −c, we add −c on both side of equation
8
Thus, (−c) + c = (−c) + c + c
So 0 = c
Thus we proved c = 0
This means a × 0 = 0
1.2.2. Matrices.
Definition 2. A Matrix M over a field F or, simply, a matrix M(when the field is clear) is a rectangular
array of numbers aij in F , which are usually presented in the following form:
a11 a12 · · · a1n
a21 a22 · · · a2n
..
..
..
..
.
.
.
.
am1 am2 · · ·
amn
Example 4. The following are matrices over Q (Remember Q is the field of rational numbers, i.e. numbers
of the form ab , where each a and b are integers. Like 12 , 27 ):
1
3
1
2 4 8
0.212121 · · ·
0.5
7
6
1 2 3 , 5 53 1 ,
,
(5),
8
0.24
0.333 · · ·
4
23 0.03
7
6
3
where in the last matrix the dots represents repeating digits, not the ignored elements.
Example 5. The following are matrices over R (Remember R is the field of all rational and irrational
numbers, i.e. √numbers that can be expressed as finitely many integer digit and infinitely many decimal
digits. Like 13 , 2, π)
√
4
π
4
2
5
π , 31 2 , 10− 13 cos 3π 1
5
5 7
e
2
6
10
Example 6. The following are matrices over C (Remember C is the field of all complex numbers, i.e.
numbers that can be expressed as a + bi where a and b are real numbers and i is the element satisfies
i2 = −1)
1 3 2
3 + i 1 53
1+i
2i
, 6 2 8 , π5 + 16 i 5 1
3 + πi 5 + 2i
1 2 4
9 + 9i 1 5i
Example 7. Because we know rational numbers are included in real numbers, real numbers are included
in complex numbers, so a matrix over Q is a matrix over R, and a matrix over R is a matrix over C
a11 a12 · · · a1n
a21 a22 · · · a2n
For a matrix A = ..
..
.. over F,
..
.
.
.
.
am1 am2 · · · amn
the rows of Matrix are called row vectors of Matrix, always denoted as rk , where the subindex are
arranged by order:
1st row vector r1 = a11 a12 · · · a1n 2nd row vector r2 = a21 a22 · · · a2n
···
m’th row vector rm = am1 am2 · · · amn
9
The columns of Matrix are called column vectors of Matrix, always denoted as ck , where the subindex
are arranged by order:
a11
a12
a21
a22
1st column vector c1 = .. 2nd column vector c2 = ..
.
.
am1
· · · n’th column vector cn =
am2
a1n
a2n
..
.
amn
3 1
Example 8. For the matrix over R: A = 1 5
1
3 5
√ r1 = 3 1
2 , r2 =
The column vectors are
√
2
8 . The row vectors are
2
1 5 8 , r3 = 13 5 2
√
3
1
2
c1 = 1 , c2 = 5 , c3 = 8
1
5
2
3
The element aij , which located at i’th row and j’th column are called ij-entry of the Matrix. We frequently
denote a matrix by simply writting A = (aij )1≤i≤m if the formula or rule of aij is explicitly given or clear.
1≤j≤n
1 3 5
Example 9. Suppose (aij ) 1≤i≤3 = 2 1 9 , What is a12 ?
1≤j≤3
7 0 8
Answer: a12 is the element getting by the first row and second column, so a12 = 3
Example 10. What is the matrix (i + j) 1≤i≤3 ?
1≤j≤3
2 3 4
Answer: 3 4 5
4 5 6
Example 11. What is the matrix (i2 + j 2 ) 1≤i≤1 ?
1≤j≤2
Answer: 2 5
Besids the notation (aij )1≤i≤m where aij is bunch of numbers. People can also write a matrix as a row
1≤j≤n
vector of column vectors, or column
vector of row vectors. Explicitely, row vector of column vectors is
like A = c1 c2 · · · cn , this only make sense when
each
the size of each column vector ci are
r1
r2
equal. The column vector of row vectors is like A = .. where each ri are row vectors, and only
.
rm
make sense when size of each of them are equal.
r1
Example 12. Suppose r1 = 6 2 3 , r2 = 4 9 , what is the matrix
?
r2
Answer:This does not make sense because the size of r1 are not equal to the size of r2
10
r1
Example 13. Suppose r1 = 2 3 , r2 = 1 5 , what is the matrix
?
r2
2 3
Answer:
1 5
1
2
Example 14. Suppose c1 =
, c2 =
, what is the matrix c1 c2 ?
0
9
1 2
Answer:
0 9
A matrix with m rows and n columns is called an m by n matrix, written as m × n. The pair of the
number m × n is called the size of the matrix.
Example 15. What is the size of 1 3 ?
Answer: 1 × 2
Two matrices A and B over F are called equal, written A=B, if they have the same size and each of the
corresponing elements are equal. Thus the equality of two m × n matrices is equivalent to a system of mn
equalities, each of the equality corresponds to pair of elements.
x y
1 4
Example 16. Solving the equation
=
z 5
2 5
Answer: as the 22-entry matches each other, this is possible for the equation make sense. By definition of
equal of matrix, we have x = 1, y = 4, z = 2
A matrix of size 1 × n over F, namely, only consists of 1 row, are called row matrix or directly abuse of
notation called row vector.
A matrix of size m × 1 over F, namely, only consists of 1 column, are called column matrix or directly
called column vector
Example 17. Which of the following are row vectors? Which of them are column vectors?
4
1 3 6
2 4 , 2 , (5),
9 1 7
6
Answer: The 1st is a row vector, the 2nd is a column vector, the third is both row vector and column
vector,the last one is neither.
For convenient, people always ignore 0 in the entry of matrices, thus when you see some number matrix
has missing entry, that entry represents 0
0 0 0
1 means 0 0 1
Example 18.
0 0 0
2
0 2 0
6
3 means 6 0 3
2
0 2 0
1.3. Matrix Multiplication. Goal:Our Matrix is defined over a field. We have + and × in field. So it should be inherit by the matrix.
With these operation, we can do basic algebra with matrices. You will see in the future how matrix algebra
simplify all the linear algebra problem.
11
Definition 3. Suppose A = (aij )1≤i≤m and B = (bij )1≤i≤m are two matrices over F and have the same
1≤j≤n
1≤j≤n
size m × n. The sum of the two matrices are defined to be A + B = (aij + bij )1≤i≤m . Explicitly,
1≤j≤n
a11
a21
..
.
a12
a22
..
.
···
···
..
.
a1n
a2n
..
.
+
b11
b21
..
.
b12
b22
..
.
···
···
..
.
b1n
b2n
..
.
=
a11 + b11
a21 + b11
..
.
a12 + b12
a22 + b12
..
.
···
···
..
.
a1n + b1n
a2n + b2n
..
.
bm1 bm2 · · · bmn
am1 + bm1 am2 + bm2 · · · amn + bmn
1 7
1 3 5
Example 19. Does
+ 2 2 make sense? if it is, please calculate.
7 1 0
1
4 9
Answer: This expression doesn’t make sense because the former is of size 2 × 3, and later 3 × 2, so their
size are not equal.
1 7 2
1 3 5
make sense? if it is, please calculate.
Example 20. Does
+
7 1 0
2 41 9
1 7 2
1+1 3+7 5+2
2 10 7
1 3 5
+
Answer:
=
=
7 1 0
2 14 9
7 + 2 1 + 14 0 + 9
9 54 9
am1 am2 · · ·
amn
The definition of matrix multiplication is somewhat complicated. To see how complecated it is, we give
the strict definition:
Definition 4. Suppose A = (aij )1≤i≤m and B = (bjk )1≤j≤n are two matrices over F. Then we define the
1≤j≤n P
1≤k≤p
product of this two matrix to be AB = ( nj=1 aij bjk )1≤i≤m . Explicitly,
1≤k≤p
···
···
..
.
a1n
a2n
..
.
am1 am2 · · ·
amn
a11
a21
..
.
a12
a22
..
.
×
···
···
..
.
b1p
b2p
..
.
bn1 bn2 · · ·
bnp
b11
b21
..
.
b12
b22
..
.
Pn
Pn
a b
a b
···
Pnj=1 1j j1 Pnj=1 1j j2
a
b
a
b
·
··
2j
j1
2j
j2
j=1
j=1
=
..
..
.
..
Pn .
Pn .
j=1 amj bj1
j=1 amj bj2 · · ·
Pn
a b
Pnj=1 1j jp
a
j=1 2j bjp
..
Pn .
j=1 amj bjp
It might cost you whole life to understand this definition by above words. let’s analysis what is going on.
Firstly, to define multiplication. the last number of the size of first matrix should match the first number
of the size of last matrix. In other words, the number of the columns of A should be equal to the number of
rows of B.
Example 21. Suppose A is a 3 × 2 matrix, B is a 5 × 9 matrix, does AB make sense?
Answer: No, because 2 6= 5
1 5
1 8 0
Example 22. Suppose A = 8 1 , B =
, Does AB make sense?
3 7 2
3 9
Answer:
Yes, becausethe size of A is 3 × 2, and B is of size 2 × 3, the final result AB would be a 3 × 3
∗ ∗ ∗
matrix.like ∗ ∗ ∗
∗ ∗ ∗
1 5
1
Example 23. Suppose A = 8 1 1, B =
, Does AB make sense?
8
3 9
12
Answer: Yes,
the size of A is 3 × 2, and B is of size 2 × 1, the final result AB would be a 3 × 1
because
∗
matrix. like ∗
∗
If the matrix is of right size, suppose we have
a11 a12 · · · a1j · · · a1n
b11 b12
a21 a22 · · · a2j · · · a2n b21 b22
..
..
..
..
..
..
..
..
.
.
.
.
.
.
.
.
ai1 ai2 · · · aij · · · ain
b
b
j1
j2
..
..
..
..
..
..
.
.
.
.
.
.
.
.
.
.
.
.
bn1 bn2
am1 am2 · · · amj · · · amn
···
···
..
.
b1k
b2k
..
.
···
···
..
.
···
..
.
···
bjk
..
.
···
..
.
···
bnk
b1p
b2p
..
.
c11
c21
..
.
c12
c22
..
.
···
···
..
.
=
bjp ci1 ci2 · · ·
.
..
..
..
.
. ..
.
bnp
cm1 cm2 · · ·
c1k
c2k
..
.
···
···
..
.
c1p
c2p
..
.
cik
..
.
···
..
.
···
cip
..
.
cmk
Then the element cik is given by multiplying ith row of A and kth column of B:
cik = ai1 b1k + ai2 b2k + ai3 b3k + · · · + ain bnk
5 7
1 5 7
1
×
5
+
5
×
1
+
7
×
3
1
×
7
+
5
×
0
+
7
×
3
66
28
1 0 =
=
Example 24.
2×5+8×0+1×3 2×7+8×0+1×3
26 17
2 8 1
8 3
Definition 5. Suppose A = (aij )1≤i≤m is a matrix over F , and λ ∈ F is a scalar, then we define the scalar
1≤j≤n
multiplication of A to be λA = (λaij )1≤i≤m , to be explicitely,
1≤j≤n
λ
Example 25. 3
a11
a21
..
.
a12
a22
..
.
···
···
..
.
a1n
a2n
..
.
=
am1 am2 · · · amn
3 9 18
1 3 6
=
2 9 2
6 27 6
···
···
..
.
λa1n
λa2n
..
.
λam1 λam2 · · ·
λamn
λa11
λa21
..
.
λa12
λa22
..
.
Definition 6. Suppose A = (aij )1≤i≤m and B = (bjk )1≤j≤n are two matrices over F. Then we define
1≤j≤n
the difference of this two matrix to be A
corresponding entries of A and B
1 5 2
1 2
Example 26. 8 2 1 − 2 4
0 2 3
8 1
1≤k≤p
− B = A + (−1)B. Explicitly, just do substraction for each
0
0
3
2
6 = 6 −2 −5
2
−8 1
1
Definition 7. The zero matrix of size m × n is a m × n matrix with all entries equal to 0, denote as 0m×n
or simply 0 if we konw the size from content.
0 0 0
Example 27.
is a zero matrix.
0 0 0
We call a matrix square matrix if it is of size n × n for some integer n. The multiplication of two square
matrix make sense if and only if they are of the same size, and the result is still a square matrix of the same
size. Now let’s concentrate on square matrix.
diagnal of square matrix: We call the left-right, up-down diagnal as diagnal of matrix.
Pay attention, we don’t call the other diagnal line the diagnal of matrix. When we say diagnal, we always
assume we start from left up and end with right down.
13
cmp
Example 28. In
∗
∗
∗
, all the starts lies in the diagnal of the square matrix, but in
∗
the stars are not lies in the diagnal of the matrix
∗
∗
,
∗
Definition 8. Unit Matrix: Unit Matrix is a square matrix who has entry 1 in diagnals and 0 elsewhere. we
always denote it as In , where n × n is the size of it. Unit Matrix looks like
1
1
1
1
..
.
1
Proposition 2. For any square matrix A of size n × n, we have In A = AIn = A
this means the unit matrix acts like multiplicative identity.
Definition 9. Suppose A is a square matrix of size n × n, if there exists a matrix B of the same size, such
that BA = In , then we call B the inverse of A, and we call the matrix A invertible matrix we always denote
such B as A−1
Proposition 3. If a matrix is invertible, then the inverse of a matrix unique, and is commute with the original
matrix, that is AA−1 = A−1 A = In
The proof needs the fact that if A have left inverse, then A should also have right inverse. We will prove
this proposition after we learned the rank of the matrix.
Proposition 4. The product of two invertible matrix is invertible, and the inverse is given by: (AB)−1 =
B −1 A−1
Proof. We calculate B −1 A−1 AB = B −1 (A−1 A)B = B −1 IB = B −1 B = I, so indeed, we can write(B −1 A−1 )(AB) =
I and by the uniqueness of inverse, this implies (AB)−1 = B −1 A−1
Proposition 5. Suppose A is an invertible matrix
(1) (AT )−1 = (A−1 )T
(2) (A−1 )−1 = A
Proposition 6. If the following notation make sense, then
(1) (A + B) + C = A + (B + C)
(2) A + 0 = 0 + A (Here 0 means 0 matrix)
(3) A + (−A) = (−A) + A = 0
(4) A + B = B + A
(5) k(A + B) = kA + kB (Here k ∈ F is a scalar in feild)
(6) (k + k 0 )A = kA + k 0 A (Here k and k’ are two scalars in feild)
(7) (kk 0 )A = k(k 0 A)
(8) 1A = A
Proposition 7. If the following notation of matrix make sense, then
(1) A(B + C) = AB + AC
(2) (B + C)A = BA + CA
(3) (AB)C = A(BC)
(4) k(AB) = (kA)B = A(kB)
14
With distributive law, we can apply our algebra skills of numbers to matrices, this is what we called
Matrix Algebra. Keep in mind that A and B are not necessarily commute.
Example 29. Suppose A, B are all 2 × 2 matrices, expand (A + B)(A − B), Is this equal to A2 − B 2 ?
When are they equal?
Answer: (A+B)(A−B) = A(A−B)+B(A−B) = AA−AB +BA−BB = A2 −B 2 +(BA−AB).
But A2 −B 2 +(BA−AB) = A2 −B 2 if and only if BA−AB = 0, that is AB = BA. So (A+B)(A−B) =
A2 − B 2 if and only if A and B commute.
This example shows when two matrices commute each other, most of the laws for numbers applies to A
and B. But tipically are not.
Commutativety is a very important problem in linear algebra. It is not exaggeration to say that the
uncertainty of the future is just because of the existance of non-commute matrices.
Definition 10. Suppose A = (aij )1≤i≤m , then we define the transpose of A to be AT = (aji ) 1≤j≤n , to be
1≤j≤n
1≤i≤m
precise,
···
···
..
.
a1n
a2n
..
.
am1 am2 · · ·
amn
a11
a21
..
.
a12
a22
..
.
T
=
···
···
..
.
am1
am2
..
.
a1n a2n · · ·
amn
a11
a12
..
.
a21
a22
..
.
Example 30.
2 0
= 3 6
1 5
T 1 5
1 6
=
6 7
5 7
T
1
3 = 1 3 5
5
2 3 1
0 6 5
T
Proposition 8. Suppose the following notation of matrix make sense, then
(1) (AB)T = B T AT
(2) (A + B)T = AT + B T
(3) (λA)T = λAT
(4) (AT )T = A
We will prove this proposition after we study the block matrix multiplication.
1.3.1. Some special square matrix. Diagnal Matrix: Diagnal Matrix is a square matrix who only has entry in diagnals and 0 elsewhere. we
always denote it as diag(a1 , a2 , · · · , an ), where n × n is the size of it, and each ai is the entry of it. Diagnal
Matrix looks like
a1
a2
a3
diag(a1 , a2 , · · · , an ) =
a4
.
.
.
an
15
Scalar Matrix: Scalar Matrix is a diagnal matrix with the diagnal entry all the same. It could be viewed
as a scalar multiply the unit matrix
Unit matrix is scalar matrix, every scalar matrix is diagnal matrix. Zero square matrix is scalar matrix, is
diagnal matrix.
−1 0 0
Example 31. Diagnal Matrix over Q: 0 21 0
0 0 5
1
−2 0
0
Scalar Matrix over Q: 0 − 12 0
0
0 − 12
Now we discuss what will happen when we multiplying the diagnal matrix.
When left multiplying diagnal matrix on a given matrix, the corresponding entries of diagnal matrix
multiplied on each row of the given matrix. When right multiplying diagnal matrix, the corresponding
entries multiplied on each column of the given matrix. To be precise, look at the following example:
2
2
1
3
4
2
6
10 20 15 = 50 100 75
5
Example 32.
6
100 200 120
600 1200 720
This answer is got by applying the matrix multiplication rule. the final result the first row multiplied by
2, 2nd row by 5, 3rd row by 6.
2 10 300
2
4 50 1800
= 12 350 4800
5
Example 33. 6 70 800
1 20 1200
6
2 100 7200
This answer is got by applying the matrix multiplication rule. the final result the first column multiplied
by 2, 2nd column by 5, 3rd column by 6
Proposition 9. The product of diagnal matrices is diagnal matrix. If the diagnal elements are all non-zero,
then the diagnal matrix is invertible, and the inverse is still a diagnal matrix.
Upper triangular matrix We call a matrix A to be upper triangular, if all the non-zero entries lies on or
above the diagnal. In other words, all entris below the diagnal is 0. The upper triangular matrices looks like:
a11 a12 · · · a1n
a22 · · · a2n
..
.
.
.
.
ann
Proposition 10. The product of two upper triangular matrices is still a upper triangular matrix, and the
entries of the diagnals of the product is the product of their corresponding entries in diagnals. if all the
entries on the diagnal are non-zero, then upper triangular matrix is invertible, and the inverse is still a
upper triangular matrix.
1 3 5
2 5 1
2 14 42
6 7
3 2 =
18 61
Example 34. We compute
9
7
63
This example reflects the fact that the product of two upper triangular matrices is upper triangular, and
the diagnal entries correspond to product of each entries in diagnal.2 = 1 × 2, 18 = 6 × 3, 63 = 9 × 7
We will prove this result after studying the blockwise computation of matrices.
Lower triangular matrix We call a square matrix A to be lower triangular, if all the non-zero entries
lies on or below the diagnal. In other words, all entris above the diagnal is 0. The upper triangular matrices
looks like:
16
a11
a21
..
.
a22
..
.
..
.
an1 an2 · · ·
ann
Proposition 11. The product of two lower triangular matrices is still a lower triangular matrix, and the
entries of the diagnals of the product is the product of their corresponding entries in diagnals. If all the
entries in the diagnal are non-zero, then the lower triangular matrix is invertible, and the inverse is a lower
triangular matrix.
Symmetric matrix :we call a square matrix A to be symmetric, if it saitisfies AT = A.
The entry of symmetric matrix is symmetric with respect to diagnal. See the following examples
Example 35. The
followingare symmetric matrices
1 4 6
1 3
, 4 7 5
3 1
6 5 0
1.3.2. The geometric meaning of matrix multiplication. In this subsection, we fix our field R, this means every entry in the matrix are real numbers. As we said,
the matrix represents a linear map. How about the matrix multiplication?
As we said earlier, matrix multiplication corresponds to linear map composition.
To be concrete, the composition of two linear map just means do the first linear transformation first, and
then do the second one. We will see an example in this subsection, the detail discussion would left to the
future chapters.
We illustrate some of the geometric meaning of matrices we defined:
1 0
Unit matrix defines identity map for example, when n=2, the matrix In =
corresponding to
0 1
the equation u = 1x + 0y; v = 0x + 1y, which means u = x; v = y, so the linear map maps (x,y) to (x,y).
So it seems like doing nothing.
Now we will see a example that is helpful for us to understand the geometric meaning of linear multiplication.
Example 36. Rotation and Rotation again.
If you rotate a graph counterclockwisely by 90 degree, and then do it again, what is the final result? The
final result would be rotate a graph by 180 degree.
17
and
exactly reflects on the matrix, we know that rotationcounterclockwise by 90 degree is given
that is 0 −1
−1 0
by
and rotate by 180 degree is given by
. and the matrix multiplication exactly
1 0
0 −1
reflect
this fact:
0 −1
0 −1
−1 0
=
1 0
1 0
0 −1
√ !
3
1
−
−
2
√2
Example 37. If we know rotate a graph counterclockwise by 120 degree is given by a matrix
,
3
− 21
2
√ !3
3
1
−
−
1 0
2
2
√
show that
=
3
0 1
− 12
2
√ !3
1
−
− 23
√2
Answer:
means rotate three times, each time 120 degree, so finally it will seems like
3
− 21
2
1 0
doing nothing, and matrix for doing nothing is the unit matrix, that is
0 1
18
1.4. Block Matrix Multiplicaiton. Matrix multiplication is not only valid if we do it numberwise, it’s also
make sense blockwise.
Definition 11. A partition of n is an ordered numbers P = (n1 , n2 , · · · , nk ), such that n1 + n2 + · · · +
nk = n . We call each summand nk as k’th part of partition Two partition P1 = (a1 , a2 , · · · , ak ) and
P2 = (b1 , b2 , · · · , bk ) are said to be the same if each correspond part are the same, that is ai = bi
Example 38. Let (2, 3, 1) be a partition of 6, this partition separate object like this: oo|ooo|o
Definition 12. A block matrix is a matrix with partition Pr on its rows and Pc on its columns. These
partition separate matrix into blocks. We denote the ij-block the block located in the i’th part row and j’th
part column
Example 39. Suppose we have a partition Pr = (2, 1) and Pc = (1, 2), and a 3 × 3 matrix A. then we
can separate A by row partition Pr and column partition Pc into a block matrix:
1
2 3
1 2 3
7 9
A= 1 7 9 → 1
0 2 5
2 5
0
A11 A12
and then by this partition we can denote A as a block matrix A =
A
A22
21
1
2 3
where The 11-block is A11 =
, 12-block is A12 =
, 21-block is A21 = 0, 22-block is
1
7 9
A22 = 2 5
Proposition 12. Block Matrix Multiplication: Suppose A is a block matrix, with row partition Pr and
column partition Pc . and B is a block matrix, with row partition Qr and column partition Qc . If Pc = Qr ,
then the product AB is precisely the same with the same rule applied to blocks, to be precise,
Pn
Pn
Pn
A B
A B
A B
···
A11 A12 · · · A1n
B11 B12 · · · B1p
Pnj=1 1j jp
Pnj=1 1j j1 Pnj=1 1j j2
A21 A22 · · · A2n B21 B22 · · · B2p
A
A
B
A
B
·
·
·
j=1 2j Bjp
j=1 2j j1
j=1 2j j2
×
=
..
..
.
.
.
.
.
.
..
.
.
.
..
.. ..
..
..
..
.
.
..
.
.
Pn .
Pn .
Pn .
Am1 Am2 · · · Amn
Bn1 Bn2 · · · Bnp
j=1 Amj Bjp
j=1 Amj Bj1
j=1 Amj Bj2 · · ·
and the row partition of the product is given by Pr , the column partition is given by Qc
We illustrate this property by an example.
1 5 2
1 2 0
Example 40. By the normal method of matrix multiplication, we can compute: 8 1 2 2 0 1 =
1 3 4
0 3 5
11 8 15
10 22 11 . Now we use another computation method:
7 14 23
We separate matrices as follows:
1
2 0
1 5
2
8 1
2 2
0 1
1 3
3 5
4
0
And apply our general method of multiply matrices, first we compute each entries:
1 5
1
2
11
+
0=
8 1
2
2
10
19
1 5
8 1
1 3
2 0
0 1
+
1 3
2 0
0 1
2
2
1
2
3 5
=
8 15
22 11
+4×0=7
+4×
3 5
=
14 23
11
8 15
22 11
Thus, the final result is 10
14 23
7
It doesn’t matter which partition we use to calculate the product.
A very important view of matrix multiplication is by viewing matrix as row vectors of column vectors,
or column vectors of row vectors. We can use block matrix to strict our word. The trivial partition of n is
(n), which means no partition at all. The singleton partition of n is (1, 1, 1, · · · , 1), which means separate
an n-element set in to singletons. Thus, row vector of column vector is a block matrix with trivial partition
of rows and singleton partition on columns, and column vector of row vector is a block matrix with trivial
partition of columns and singleton partition on rows. Now we use this idea to discuss more about matrix
multiplication.
Definition 13. Suppose c1 , c2 , · · · , cn are column vectors over F. a linear combination of column vectors
c1 , c2 , · · · , cn with coefficient a1 , a2 , · · · , an means the column vector formed by c1 a1 + c2 a2 + · · · + cn an .
We also define linear combination of row vectors in the same manner. The lienar combination of row vectors
r1 , r2 , · · · , rn means a1 r1 + a2 r2 + · · · + an rn .
1
1
2
3 , c2 =
1 . Then
4 is a linear combination of
Example 41. Over field Q. Suppose c1 =
5
1
6
2
1
1
4
3
1
c1 and c2 , the coeffitient has only one choice: a1 = 1, a2 = 1, which means
=
+
6
5
1
1
1
2
Example 42. Over field Q. Suppose c1 = 3 , c2 = 1 . Then 7 is not a linear combination
5
1
6
of c1 and c2 , because we can’t find any coefficient that can combine such a vector. (To prove, show that the
entry of any linear combination of c1 and c2 must be a arithmetic sequence)
1
0
1
3
Example 43. Over field Q. Suppose c1 =
, c2 =
, c3 =
. Then
is a linear
0
1
1
4
combination of c1 ,c2 ,c3 , could with coefficient a1 = 3, a2 = 4, a3 = 0, also could with coefficient a1 = 0,
a2 = 1, a3 = 3. There are infinitely many choice of coefficient. In this case, we always say that to represent
3
, these three vectors are redundant, which means we can only choose two of them, such as c1 and c2
4
3
and represents our
uniquely as 3c1 + 4c2 .
4
Proposition 13. Suppose Am×n , and Bn×p are two matrices. Let Cm×p = AB. Then each column of C is
linear combination of columns of A,with coefficient given by the corresponding rows of B. each row of C
is linear combination of rows of B, the coefficeint of i’th row of C comes from i’th row A. The coefficient of
j’th column of C comes from j’th column of B.
20
Suppose A =
Proof.
a11
a21
..
.
a12
a22
..
.
···
···
..
.
a1n
a2n
..
.
, B =
b11
b21
..
.
b12
b22
..
.
···
···
..
.
b1p
b2p
..
.
am1 am2 · · · amn
bn1 bn2 · · · bnp
To prove, write
A
in
to
a
block
matrix,
···
a12
a1n
a11
a21
···
a22
b2n
that is A =
..
..
..
..
.
.
.
.
aa2
···
bmn
am1
To simplify notation, denote as A = (c1 , c2 , · · · , cn ) which ci are column vectors.
Now using blockmatrix multiplication,
b11 b12 · · · b1p
b21 b22 · · · b2p
(c1 , c2 , · · · , cn ) ..
..
.. = (d1 , d2 , · · · , dn ); where d1 = c1 b11 +c2 b21 +· · ·+
..
.
.
.
.
bn1 bn2 · · · bnp
cn bn1 , d2 = c1 b12 + c2 b22 + · · · + cn bn2 ,...
This means the column vectors of product is linear combinations of column vectors of A.
For the rest of the proof, we remain to reader.
1 3 6
1 2 0
Example 44. What is the second column of the matrix product 2 7 8 0 0 2 ?
1 9 0
3 4 3
1 3 6
Answer: The second column of product is linear combination of columns of 2 7 8 , and the
1 9 0
1 2 0
2
coefficient comes from the second column of 0 0 2 , that is 0 . Thus we compute:
3 4 3
4
1
6
26
2 × 2 + 8 × 4 = 36
1
0
2
A11 A12 · · · A1n
A21 A22 · · · A2n
Proposition 14. Transpose of Block Matrix Suppose A = ..
..
.. is a block matrix,
..
.
.
.
.
Am1 Am2 · · ·
T
then A =
AT11
AT12
..
.
AT21
AT22
..
.
AT1m AT2m
···
···
..
.
···
ATn1
ATn2
..
.
ATnm
Amn
This proposition means before we transpose the position of each block, we should transpose each block
itself first.
T
3 2
3
1 2
7
5 6 = 1 5
Example 45. 2
1
1 6
7
2 6
6
21
To illustrate an application of block matrix multiplication, we give the proof of (AB)T = B T AT
Proposition 15. (AB)T = B T AT
Proof.
Separate A and
B into
Block Matrix,
r1
r2
we assume A = .. , B = c1 c2 · · · cp
.
r
m
We see that AB =
r1 c1
r2 c1
..
.
r1 c2
r2 c2
..
.
···
···
..
.
r1 cp
r2 cp
..
.
rm c1 rm c2 · · · rm cp
For column vector c row vector r, we should have rc = cT rT
a1
a2
Assume c = .. , r = b1 b2 · · · bn
.
an
We compute rc = b1 a1 + · · · + bn an , cT rT = a1 b1 + · · · + an bn .
because number multiplication is commute,
Then we have b1 a1 + · · · + bn an = a1 b1 + · · · + an bn
Then we have rc = cT rT
Because a transpose of a number is this number itself, so (rc)T = rc = cT rT
T T T T
T
c1 r1 c1 r2 · · · cT1 rm
cT rT cT rT · · · cT rT
2 2
2 m
2 1
T
Thus, (AB) = ..
..
..
.
.
.
.
.
.
T
cTp r1T cTp r2T · · · cTp rm
On the
other hand,
cT1
cT
2
T
B T = .. , AT = r1T r2T · · · rm
.
cTp
T T T T
T
c1 r1 c1 r2 · · · cT1 rm
cT rT cT rT · · · cT rT
2 2
2 m
2 1
we see that B T AT = ..
..
..
..
.
.
.
.
T
cTp r1T cTp r2T · · · cTp rm
So we conclude (AB)T = B T AT
1.5. Elementary Matrices, Row and Column Transformations.
We have already studied in the last section that we can view the matrix multiplication as linear combination of column vectors of the first matrix, or row vectors of the second matrix. And the coefficient of
matrix multiplication is exactly given by other matrix. This shows that to understand matrix multiplication,
22
we have to study the linear combination of row and column vectors. In this section, we will study the most
basic linear combination of rows and columns, row and column transformation.
1.5.1. Elementary Row transformation. We have three types of row transformation.
row switching This transformation swiches two row of matrix.
1 4 8
3 3 5
r1 ↔r3
Example 46. Switch the 1st and 3rd row of matrix 2 0 9 −−
−−→ 2 0 9
3 3 5
1 4 8
row multiplying This transformation multiplies some row with scalar λ
1 4 8
1 4 8
2×r2
4 0 18
Example 47. Multiply the 2nd row of matrix by 2 : 2 0 9 −−−→
3 3 5
3 3 5
row adding In this transformation, we multiply some row by a scalar, but add that into another row.
1 4 8
1 4 8
r3 +2×r2
Example 48. Add twice of the 2nd row to the 3rd row : 2 0 9 −−
−−−→ 2 0 9
3 3 5
7 3 23
Caution: Write 2 × r instead of r × 2, the reason for that is simple, because scalar is 1 × 1 matrix. In this
view, scalar can only appear in fromt of row vectors.
Simillarly, we can define the column transformation in the same way.
1.5.2. column transformation. column switcing This transformation swiches two column of matrix.
1 4 8
8 4 1
c1 ↔c3
Example 49. Switch the 1st and 3rd column of matrix 2 0 9 −−
−−→ 9 0 2
3 3 5
5 3 3
column multiplying This transformation multiplies some column with scalar λ
1 4 8
1 8 8
c2 ×2
Example 50. Multiply the 2nd column of matrix by 2 : 2 0 9 −−
−→ 2 0 9
3 3 5
3 6 5
column adding In this transformation, we multiply some column
column.
1
2
Example 51. Add twice of the 2nd column to the 3rd column :
3
by a scalar, but add that into another
4 8
1 4 16
c3 +c2 ×2
0 9 −−
−−−→ 2 0 9
3 5
3 3 11
1.5.3. Realization of elementary transformation by matrix multiplication. In the view of last section, row
transformation is equivalent to left multiplication, column transformation is equivalent to right multiplication. In order to make it precise, we define the following Elementary Matrices
23
Definition 14. The Switching matrix is the matrix obtained by swapping ith and jth rows of unit matrix.
Denote by Sij :
i
Sij =
j
1
..
.
0
1
..
1
.
0
..
.
1
Proposition 16. Left multiplyting switching matrix Sij will switch ith and jth rows of the matrix.
1 0 0
Example 52. 0 0 1 is obtained by switching 2nd and 3rd row of unit matrix, left multiplying
0 1 0
1 0 0
0 0 1 will do the same thing to rows of other matrix. As we compute using definition of matrix
0 1 0
multiplication:
1 0 0
1 2 3
1 2 3
0 0 1 4 5 6 = 7 8 9
0 1 0
7 8 9
4 5 6
1 2 3
The result is exactly switch 2nd and 3rd row of 4 5 6
7 8 9
Proposition 17. Right multiplying switching matrix Sij will switch jth and ith columns of the matrix.
1 0
Example 53. 0 0
0 1
1 0 0
ing 0 0 1 will
0 1 0
matrix
multiplication:
1 2 3
1
4 5 6 0
0
7 8 9
0
1 is obtained by switching 2nd and 3rd column of unit matrix, right multiply0
do the same thing to columns of other matrix. As we compute using definition of
0 0
1 3 2
0 1 = 4 6 5
1 0
7 9 8
1 2 3
The result is exactly switch 2nd and 3rd column of 4 5 6
7 8 9
24
Definition 15. The Multiplying matrix is the matrix obtained by multiplying ith row by non-zero scalar λ
of unit matrix. Denote by Mi (λ):
i
Mi (λ) = i
1
..
.
λ
..
.
1
Proposition 18. Left multiplyting multiplying matrix Mi (λ) will multiplies i’th row of the matrix by λ.
1 0 0
Example 54. 0 3 0 is obtained by multiplying the 2nd row of unit matrix by 3, left multiplying
0 0 1
1 0 0
0 3 0 will do the same thing to rows of other matrix. As we compute using definition of matrix
0 0 1
multiplication:
1 0 0
1 2 3
1 2 3
0 3 0 4 5 6 = 12 15 18
0 0 1
7 8 9
7 8 9
1 2 3
The result is exactly multiplying the 2nd row of 4 5 6 by 3
7 8 9
Proposition 19. Right multiplyting multiplying matrix Mi (λ) will multiplies i’th column of the matrix by λ.
1 0 0
Example 55. 0 3 0 is obtained by multiplying the 2nd column of unit matrix by 3, right multi0 0 1
1 0 0
plying 0 3 0 will do the same thing to columns of other matrix. As we compute using definition of
0 0 1
matrix
multiplication:
1 2 3
1 0 0
1 6 3
4 5 6 0 3 0 = 4 15 6
7 8 9
0 0 1
7 24 9
1 2 3
The result is exactly multiplying the 2nd column of 4 5 6 by 3
7 8 9
25
Definition 16. The Addition matrix is the matrix obtained by add jth row by scalar λ to the ith row of unit
matrix. Denote by Aij (λ):
j
i
Aij (λ) =
1
..
.
1
..
λ
.
1
..
.
1
Proposition 20. Left multiplying addition matrix Mi (λ) will add λ times j’th row to the i’th row.
1 2 0
Example 56. 0 1 0 is obtained by adding twice of the 2nd row of unit matrix to 1st row, left
0 0 1
1 2 0
multiplying 0 1 0 will do the same thing to rows of other matrix. As we compute using definition
0 0 1
of matrix
multiplication:
1 2 0
1 2 3
9 12 15
0 1 0 4 5 6 = 4 5 6
0 0 1
7 8 9
7 8 9
1 2 3
The result is exactly adding twice of the 2nd row to the 1st row of 4 5 6
7 8 9
Proposition 21. right multiplying addition matrix Mi (λ) will add λ times i’th column to the j’th column.
1 2 0
Example 57. 0 1 0 is obtained by adding twice of the 1st column of unit matrix to 2nd column,
0 0 1
1 2 0
right multiplying 0 1 0 will do the same thing to columns of other matrix. As we compute using
0 0 1
definition
of
matrix
multiplication:
1 2 3
1 2 0
1 4 3
4 5 6 0 1 0 = 4 13 6
7 8 9
0 0 1
7 22 9
1 2 3
The result is exactly adding twice of the 1st column to the 2nd column of 4 5 6
7 8 9
Caution: when Aij (m) serving at left, it is add m times j’th row to i’th row. But when it serving at right,
it is add m times i’th column to j’th column. Keep in mind that the order is different.
As previous example, we have seen there is an easy way to remember the operation. When left multiplying, like AB, you think how can we get A by row transformation from unit matrix, and then to get the
26
product, do the same row transformation to B. When right multiplying, like BA, you think how to get A by
column transformation from unit matrix, and then to get the product, do the same column transformation to
B.
All the matrices we defined above Sij , Mi (λ), Aij (λ), are called Elementary matrices.
−1
Proposition 22. Elementary matrices are invertible, Indeed, Sij
= Sij , Mi (λ)−1 = Mi (λ−1 ), Aij (λ)−1 =
Aij (−λ)
1.5.4. Row echelon form and column echelon form. Now we want to know by row transformation, how far
could we go. We want to make our matrices looks cleaner and better. Now we begin with an example to see
how neat we can do.
We start by an example:
1
2
1
2
7
2
4
2
6
18
−1
−2
0
−1
−3
Firstly, we use the first row to kill the first entry of other two rows,
r2 − 2 × r1
1
2
1
2
7
1
2
1
2
7
r3 + 1 × r1
2
4
2
6
18 −−−−−−−−−−→
2
4
−1
−2
0
−1
−3
1
1
4
Now, the first column is clear(with clear I mean that column only contains 1 element), funtunately so is
the second column. Now in the third column, we have two 1. But we can’t use the first row to clean the
second row, because if we do that. It would hit the 3-1 entry and 3-2 entry, we want to keep them to be 0.
Because finally we want it to look like stairs. So we arrange each rows from longer to shorter. Thus, we
might swap the last two rows
1
2
1
2
7
1
2
1
2
7
r2 ↔r1
2
4 −−
1
1
4
−−→
1
1
4
2
4
Now we can use the 2nd row to clean up the first row
1
2
1
2
7
1
2
1
3
r1 −1×r2
1
1
4 −−
1
1
4
−−−→
2
4
2
4
Now in the last row, the leading number is 2, we can make it to be 1 by multiplying 21
1
2
1
3
1
2
1
3
1
×r
3
1
1
4 −2−−→
1
1
4
2
4
1
2
Now use the last row to clean up first two rows.
r1 − 1 × r3
1
3
1
2
1
r2 − 1 × r3
1
1
4 −−−−−−−−−−→
1
2
1
2
1
2
This method works for every matrices. The simplest matrix we can reach is the matrix looks like above.
We call it Row Echlen Form
1
2
Definition 17. A m × n matrix is called a Row Echelon Form if it saitiesfying
(1) The first non-zero entry for every row vector is 1, we call this leading 1(or pivot)
27
(2) If a row contains a leading 1, then each row below it contains a leading 1 further to the right
(3) If a column contains a leading 1, then all other entries in that column are 0
Example 58. Which of the following is Row Echelon Form?
(1)
1
1
3
2
1
1
(2)
1
6
1
1
6
6
1
(3)
1
3
2
1
2
1
0
1
1
Answer: The first one is not a row echelon form because there is a row that leading non-zero entry is 2;
The second one is not a echelon form, because if we circle the pivid:
1
6
1
1
6
6
1
In the second column, it contains an element of pivod, but there is other non-zero entry in the second
column.
The third one is a echelon form, and the pivod is
1
1
3
2
1
2
1
0
1
Now let’s summerize the steps we did in the first example and using the matrix multiplication to explicitly
represent that.
28
1
2
−1
2
4
−2
1
2
0
2
6
−1
r2 − 2 × r1
7
1
r3 + 1 × r1
18 −−−−−−−−−−→
−3
r ↔r
2
1
−−
−−→
r −1×r
1
×r3
2
−−−→
2
2
1
7
4
4
1
2
1
1
2
1
2
7
4
4
1
2
0
1
1
1
2
3
4
4
1
2
1
1
1
3
4
2
1
1
2
2
1
1
1
2
−−
−−−→
2
1
r1 − 1 × r3
1
r2 − 1 × r3
−−−−−−−−−−→
2
1
Combined with what we learned by the meaning of left multiplying the elementary matrices, we can
interprete every row transformation langrage by left multiplying a matrix. So It is the same as the following
29
picture
1
2
−1
2
4
−2
1
2
0
2
6
−1
7
18
−3
1
−2 1
1
×
1
×
1
1
1
−−−−−−−−−−−−−−−−−−−−−−−−→
1
×
1
×
2
1
2
7
4
4
1
2
0
1
1
1
2
3
4
4
1
2
1
1
1
3
4
2
1
1
2
2
1
−1
1
1
1
1
2
×
−−−−−−−−−−−→
1
2
1 −1
1
1
×
1
Every step is the same as the following equalities:
1
1
1
−2 1
2
1
1
1
1 −1
1
2
1
2
7
2
4
=
1
1
4
1
−2 1
1
=
1
1
2
1
1
2
1
2
1
2
4
−2
1
2
1
1 1
1
−1
7
4
4
30
1
1 −1
×
1
1
1
−−−−−−−−−−−−−−−−−−−−−−−−−→
7
4
4
1
1
−−−−−−−−−−−−→
2
2
1
1
1
−−−−−−−−−−−→
1
1
1
2
1
2
0
2
4
−2
2
6
−1
1
2
0
2
1
7
18
−3
2
6
−1
7
18
−3
1 −1
1
1
1
1
1
2
0
1
=
1
1
1
2
1
1 −1
1
2
=
−1
1
1
1
1
1
1
=
1
1
2
1
1
2
1
1
1
3
4
2
1
1
1
2
1
2
0
1
1
1 −1
1
1
1
2
6
−1
7
1
18 =
−3
1
2
−1
2
4
−2
1
2
0
1
1
1
1
1
1
2
2
Remark 1. Whenever you see a statement saying that there exists an invertible matrix P such that P M =
N , it is equavalent to say N can be obtained by row transformation from M, sometimes prove the later
sentense is more clear.
Whenever you see a statement saying that there exists an invertible matrix P such that M P = N , it is
equavalent to say N can be obtained by column transformation from M.
1
1
Proposition 23. For every matrix Am×n , there exists a invertible matrix Pm×m , such that P A is a row
echelon form. In this case, P A is called reduced row echelon form of A, denoted as rref(A)
31
1
1
7
18
−3
1
1
1
2
2
6
−1
1
1
−2 1
1
7
18
−3
2
6
−1
1
−2 1
1
1 −1
1
1
2
0
1
1
1
2
1
1
2
4
−2
1
1
Now we compute
:
1
1
−1
1
1
1
1 −1
1
1
1 − 21 −1
2 −1 1
2
−1 12
0
2
4
−2
1
−2 1
1
1 −1
1
1
2
1
2
So our conclusion is
:
1 − 12 −1
1
2 −1 1 2
2
−1
−1 12
0
1
1
2
1
1
1
1
−1
1
3
4
4
1
1
−2 1
=
1
Definition 18. For matrix Am×n , the rank of the matrix A is defined to be the number of non-zero rows of
rref(A)
Proposition 24. The rank of A is at most the number of non-zero rows of A
2 1 6
Example 59. Find the rank of A = 2 9 1
2 2 0
2 1 6
2 9 1
2 2 0
r3 −1×r1
−−−−−→
2 1 6
2 9 1
1 −6
r2 −1×r1
−−−−−→
1
×r1
2
−−−→
2 1 6
8 −5
1 −6
r ↔r
2
3
−−
−−→
r1 − 12 ×r2
3
8 −5
1 −6
1
1
2
3
1 −6
8 −5
1
6
1 −6
8 −5
1
6
1 −6
43
1
6
1 −6
1
1
−−−−−−→
r −8×r
3
2
−−
−−−→
1
×r3
43
−−−−→
r −6×r
1
3
−−
−−−→
r +6×r
2
3
−−
−−−→
1 −6
1
1
2
1
1
1
1
Thus, the rank of A is 3, ref f (A) = I3
32
Now we write it into the language of matrix multiplication. That is
1
2
1
×
1
1
1
43
1
−8
1 − 12
×
1
1
×
1
1
1
× −1 1
1
−6
1
1 6 ×
1
1
1
1
×
1 ×
1
1
2 1 6
1
×
× 2 9 1 =
1
1
1
−1
1
2 2 0
1
1
Now we would like to compute
1
2
1
×
1
1
43
1 − 12
×
1
1
1
−8
1
1
×
1 ×
1
1
1
×
1
−1
1
1
×
1
1
1
× −1 1
1
−6
1
1 6 ×
1
1
1
It is the same as
1 6 ×
1
1
1
2
−6
1
1
1
×
1
1
43
1
×
1
−1
1
1 − 12
×
1
1
1
−8
1
×
×
1
1
1
× −1 1
1
1
1
1
So what we need is do the same row transformation to the identity matrix:
33
1
×
1
1
1 ×
1
1
1
1
r −1×r
3
1
−−
−−−→
1
1
−1
1
r −1×r
2
1
−−
−−−→
1
−1 1
−1
1
1
×r1
2
−−−→
r ↔r
1
2
−1 1
−1
1
2
3
−−
−−→
1
2
−1
1
−1 1
1
− 21
−−−−−−→ −1
1
−1 1
r1 − 12 ×r2
1
− 21
−1
1
7 1 −8
r −8×r
3
2
−−
−−−→
1
×r3
43
−−−−→
1
−1
r1 −6×r3
−−−−−→
r +6×r
−8
43
7
43
1
43
1
43
6
− 43
53
86
7
43
1
43
−8
43
−1
2
3
−−
−−−→
− 21
1
1
43
1
− 43
7
43
6
− 43
6
43
1
43
1
53
86
5
− 43
−8
43
Thus we know
1
43
1
− 43
7
43
6
− 43
53
86
5
− 43
−8
43
2 1 6
1
6
2 9 1 =
1
43
1
2 2 0
1
43
1
6
− 43
43
6
1
−1
Note that by definition of inverse matrix, this exactly means A = − 43
43
7
43
34
1
43
53
86
5
− 43
−8
43
We call a square matrix A of full rank, if the rank of A is equal to the size of A.
Think about Echelon form, now suppose A is a full rank n × n square matrix, A is full rank means
rank(A) = n, this means ref f (A) has n leading 1, but the space in the matrix is square, that forces 1
to distribute in diagnal, and because leading 1 can clear all other entries in its column, so the Reduced
Row Echelon Form of the full rank square matrix should be unit matrix. And thus, we record our row
transformation in matrix P. so we can find a invertible matrix P,such that P A = I, so A is invertible. This
means full rank matrix is invertible. On the other hand, if A is invertible, this means there exists some P,
such that P A = I, but this just means doing some row transformation of A, and the final result is unit
matrix, a Row Echelon form, so we know reff(A)=I, and in this way, rank(A)=n.
Proposition 25. For square matrix A, A is invertible if and only if A is full rank.
This process gives a method to find the inverse of A, which is apply the same row transformation process
to unit matrix. Sometimes people will use this method to do this two steps simutaneously, like following:
35
2 1 6
2 9 1
2 2 0
1
r −1×r
2 1 6
2 9 1
1 −6
r −1×r
2
1
−−
−−−→
1
×r1
2
−−−→
2 1 6
8 −5
1 −6
r ↔r
2
3
−−
−−→
r1 − 12 ×r2
1
r −8×r
3
2
−−
−−−→
1
×r3
43
−−−−→
r −6×r
1
3
−−
−−−→
−−−−−→
1
−1 1
−1
1
1
2
1
2
−1 1
−1
1
1
6
1 −6
43
1
− 12
−1
1
7 1 −8
1
6
1 −6
1
1
−1
1
1 −6
1
r2 +6×r3
1
1
− 21
−1
1
−1 1
1
2
−1
6
1 −6
8 −5
1
3
8 −5
1 −6
1
−1
1
−1 1
−−−−−−→
1
2
1
3
1 −6
8 −5
1
1
3
1
−−
−−−→
1
1
1
1
− 12
1
−8
43
7
43
1
43
1
43
6
− 43
53
86
1
7
43
1
43
−8
43
1
43
1
− 43
7
43
6
− 43
−1
6
43
1
43
53
86
5
− 43
−8
43
When you put unit matrix on the right and doing row transformation to simplify the left matrix to unit, the
inverse of the matrix appears on the right.
The row echelon form is the simplest matrix we can reach by row transformation. But remember we can
still do column transformation, if we continue to do column transformation, things would be more clean.
36
Let’s continue the following example, now let’s see what happen if we do column transformation.
1
2
1
1
2
1
2
0
0
0
0
1
0
0
0
1
1
2
2
0
0
0
0
1
0
0
0
1
0
2
2
0
0
0
0
1
0
0
0
1
0
0
2
0
0
0
0
1
0
0
0
1
0
0
0
1
0
0
0
1
0
0
0
0
0
0
1
0
0
0
0
1
0
0
0
1
0
0
0
0
0
0
1
c2 −c1 ×2
−−
−−−→ 0
0
1
c5 −c1 ×1
−−
−−−→ 0
0
1
c5 −c3 ×2
0
−−−−−→
0
1
c5 −c4 ×2
0
−−−−−→
0
c ↔c
2
3
−−
−−→
c ↔c
3
4
−−
−−→
1
0
0
And in each step, is the same as right multiplying a 5 × 5 matrix, If you want to figure out what is the
matrix,
just do the same transformation
to unit matrix. This matrix is given by
1
0
0
−2
1
0
0
0
1
0
0
1
0
0
−2
0
0
1
0
−2
0
0
0
0
1
Combine with all we discussed, our conclusion is
1 − 21 −1
1
2
1
2
2 −1 1 2
4
2
6
2
−1
−2
0
−1
−1 12
0
1
0
0
0
0
1
0
0
0
= 0
0
0
1
0
0
Now we write it into a block matrix
37
7
18
−3
1
0
0
0
0
0
0
1
0
0
0
0
0
1
0
−2
1
0
0
0
1
0
−2
−2
1
1 0 0
0
0
0 1 0
0
0
0 0 1
0
0
This is the most simplest form we could reach by doing both row and column transformations. The
number of 1 at the end should be the number of rows in reduced row echlon form. So the number of 1 here
reflects the rank. weget this result because the
matrix we are using is of rank 3. If we start with a matrix of
1 0
0 0 0
0 0 0 . We conclude that in the proposition
rank 2, we will get 0 1
0 0
0 0 0
Proposition 26. Suppose Am×n is of rank r, then there exist two invertible square matrices Pm×m , and
Qn×n , such that
Ir
0r×(n−r)
P AQ =
0(m−r)×r 0(m−r)×(n−r)
Here 0 means a matrix with all the element 0.
1.5.5. Some properties of rank. Now we discuss some useful properties of rank. remember rank is defined
to be the number of rows of reduced echelon form of A, and is also the number of 1 when both row and
column transform is allowed.
Proposition 27. rank(A) = rank(AT )
Proof.
Suppose rank(A)=r;
Then we can find some invertible matrix P,Q, such that P AQ =
Ir
0r×(n−r)
.
0(m−r)×r 0(m−r)×(n−r)
Now we transpose both side, we have
T Ir
0r×(m−r)
Ir
0r×(n−r)
=
,
QT AT P T =
0(m−r)×r 0(m−r)×(n−r)
0(n−r)×r 0(n−r)×(m−r)
because QT and P T are all invertible,
then rank(AT ) = r
Proposition 28. If B is an invertible matrix, then rank(BA) = rank(A), rank(AB) = rank(A)
Proof. We prove first, suppose rank(A)=r, then there exists square matrices P Q, such that P AQ =
Ir
0r×(n−r)
0(m−r)×r 0(m−r)×(n−
Now because P AQ = P B −1 BAQ. So [P B −1 ] would serve as ”P” for BA, and Q would serve as ”Q” for
BA. The same method for AB
Proposition 29. The rank of A is at most the number of non-zero rows or the number of non-zero columns
Proof.
We just do our algorithm of reducing to echelon form on non-zero rows, and thus the number
of non-zero row in the reduced row echelon form would less than the number the original non-zero rows.
Because the rank of A and A transpose are the same, the proposition also hold for columns Proposition 30. Consider the block matrix, rank A B ≥ max{rank(A), rank(B)}
Proof. We only have to prove rank(A) ≤ A B , if that is ture, the same method would also works for
B
P
A
P
B
Now we do row transformation such that it is reduced echelon form, that is P A B =
where P A is a reduced echelon form, it has r row. now below r’th row of P A P B every non-zero
entry comes from PB part, so it is further right to the r’th
leading 1. Continue the algorithm, finally we will
have no less than r leading 1. so the rank of A B is at least rank of A.
We list the following facts and leave it as excercise to reader
38
Proposition 31. If Am×n , Bn×q be two matrices. Then rank(AB) ≤ min{rank(A), rank(B)}
Proposition 32. If A, B is two matrices of the same size. Then rank(A + B) ≤ rank(A) + rank(B)
1.5.6. Block Row and Column Transformations. If you separate Matrix into Blocks, can you do Row and
Column transformations? The answer is yes. We define the following elementary block matrices.
We have three types of row transformation.
block row switching This transformation swiches two row of matrix.
8 9
1
2 3
7
br ↔br2
1
5 6 −−1−−−→
Example 60. Switch the 1st and 2nd block row of matrix 4
2 3
8 9
7
4
5 6
block row multiplying This transformation left multiplies an invertible matrix to some block row λ
!
Example 61. Left Multiply the 1st block row of matrix by
13
9
7
1 3
1 2
1
4
(Note this is invertible):
7
1 3
×
2 3
1 2
5 6
−−−−−−−−−
8 9
17 21
12 15
8 9
block row adding In this transformation, we left multiply some row by a matrix, but add that into another
block row.
!
Example 62. Add
8
18
7
1
2
1
4
times the 2nd block row to the 1st block row(note: left multiply) :
7
1
br1 +
×b
2 3
2
5 6
−−−−−−−−−−
8 9
10 12
21 24
8 9
Remark 2. Note that when we do row transformation, we multiply everything on the left, when we do
column transformation, we should multiply on the right
Simillarly, we can define the block column transformation in the same way.
1.5.7. block column transformation. column switching This transformation switches two
trix.
1
2 3
bc ↔bc2
5 6 −−1−−−→
Example 63. Switch the 1st and 2nd block column of matrix 4
8 9
7
column of ma2 3
5 6
8 9
1
4
7
column multiplying This transformation right multiplies some block column with invertible matrix λ
!
Example 64. Right Multiply the 2nd block column by
1
4
7
5 12
11 27
17 42
39
1 3
1 2
1
: 4
7
1 3
2 3
1 2
5 6 −−−−−−−−−−→
8 9
bc2 ×
column adding In this transformation, we right multiplies some column by a matrix, but then add that
into another column. The examples leave to the reader.
Similarly, we also have the following elementary block matrices
Suppose we fix a partition P, then the elementary block matrix is a square matrix with same partition
P = (n1 , n2 , · · · , nk ) on its columns and rows. So the diagonal block of block matrices are square
Definition 19. The Block Switching matrix is the matrix obtained by swapping ith and jth block of unit
matrix. Denote by Sij :
i
Sij =
j
I
..
.
0
..
I
.
I
0
..
.
I
where each I is of the right size nk
Proposition 33. If the row partition of the block matrix A is the same as P, then left multiplying switching
matrix Sij will switch i’th and j’th block rows of A.
Proposition 34. If the column partition of the block matrix A is the same as P, then right multiplying
switching matrix Sij will switch i’th and j’th block column of A.
Definition 20. The Block Multiplying matrix is the matrix obtained by multiplying ith block row by invertible matrix Pni ×ni of unit matrix. Denote by Mi (λ):
i
Mi (P ) = i
I
..
.
P
..
.
I
Proposition 35. If the row partition of the block matrix A is the same as P, then left multiplying multiplication matrix Mi (P ) will left multiplies i’th block row of the matrix by P .
Proposition 36. If the column partition of the block matrix A is the same as P, then right multiplying
multiplication matrix Mi (P ) will right multiplies i’th block column of the matrix by P .
40
Definition 21. The Block Addition matrix is the matrix obtained by add jth block row left multiplied by
matrix Mni ×nj to the ith block row of unit matrix. Denote by Aij (M ):
j
i
Aij (M ) =
I
..
.
I
..
M
.
I
..
.
I
Proposition 37. If the row partition of the block matrix A is the same as P, then left multiplying addition
matrix Aij (M ) will add M times j’th block row to the i’th block row.
Proposition 38. If the column partition of the block matrix A is the same as P, then right multiplying
addition matrix Aij (M ) will add i’th block column times M to the j’th block column.
Proposition 39. Elementary Block Matrices are invertible.
Proposition 40. Elementary Block row or column Transformation does not change the rank.
Example 65. To show an example of the application of the block matrix, Suppose A and B are n × n square
matrices, we shows an identity rank(I − AB) = rank(I − BA)
Proof. We consider the 2n × 2n Matrix blocked by partition (n, n) , that is I A BI.
I A
B I
I
bc −bc ×A
−−2−−−1−−→
B I − BA I
br2 −B×br1
−−−−−−−→
I − BA
I
I A
From here we know rank
= rank
= n + rank(I − BA)
B I
I − BA
P
(Remember in the homework we proved rank
= rank(P ) + rank(Q)) We can also do
Q
another transformation:
I A
B I
I − AB A
bc1 −bc2 ×B
−−−−−−−→
I I
−
AB
br −A×br2
−−1−−−−−→
I
I A
I − AB
From here we know rank
= rank
= rank(I − AB) + n
B I
I
So, n+rank(I-BA)=n+rank(I-AB), thus, rank(I − AB) = rank(I − BA).
41
1.6. Determinant.
Introduction:If you think row vector of a matrix as coordinate of vectors in space, then the geometric
meaning of the rank of the matrix is the dimension of the parallelpepid spanned by them. But we are not
only care about the dimension, sometime we want the volumn of that. Then we will find that determinant is
the right value to work out. Historically speaking, determinant determines when does a system of equation
with n unknowns and n equalities have unique solution. In our matrix language, determinant determines
when a matrix is invertible. You will find out that determinant gives you a fast way to determine when a
matrix is invertible and what is the inverse. In 2 × 2 and 3 × 3 it is fast, but in 4 × 4 or more dimension
cases, the way we learned before might be the most fast way.
We denote the set of all m × n matrices over F by Mm×n
Theorem 4. There exists function f defined on Mn×n . That is assign every n × n matrix over F a value in
F. which saitisfying:
(1) If we switch ith and jth rows of B, then the value becomes to opposite. namely
f (Sij B) = −f (B)
(2) If we multiplied a row of B by λ, then the value is also multiplied by λ(here lambda could be 0).
namely
f (Mi (λ)B) = λf (B)
(3) If we add any scalar times jth row to ith row of B, then the value doesn’t change. namely
f (Aij (λ)B) = f (B)
Then this kind of function exists, and any two such function is differ by a scalar multiple.
As any such two functions differ by a scalar multiple, this means whenever we defines the value of unit
matrix, then the function is uniquely determined. This let us able to make following definition.
Definition 22. The determinant is a unique function on n × n matrices, which assign every n × n matrix
M over F a value in F. denoted by |M | or det(M ) which saitisfying:
(1) If we switch ith and jth rows of a matrix, then the determinant becomes opposite. explicitly:
a11 a12 a13 · · · a1n a11 a12 a13 · · · a1n ··· ··· ··· ··· ··· ··· ··· ··· ··· ··· aj1 aj2 aj3 · · · ajn ai1 ai2 ai3 · · · ain ··· ··· ··· ··· ··· = − ··· ··· ··· ··· ··· aj1 aj2 aj3 · · · ajn ai1 ai2 ai3 · · · ain ··· ··· ··· ··· ··· ··· ··· ··· ··· ··· an1 an2 an3 · · · ann an1 an2 an3 · · · ann (2) If we multiplied a any row by λ, then the determinant will also multiplied by λ(here lambda could
be 0). explicitly:
a11 a12 · · · a1n a11 a12 · · · a1n ··· ··· ··· ··· ···
· · · · · · · · · λai1 λai2 · · · λain = λ ai1 ai2 · · · ain ··· ··· ··· ··· ···
· · · · · · · · · an1 an2 · · · ann an1 an2 · · · ann (3) If we add any scalar times jth row to ith row of B, then the value doesn’t change. explicitly:
42
a11
a12
a13
·
·
·
·
·
·
···
ai1 + λaj1 ai2 + λaj2 ai3 + λaj3
···
···
···
a
a
aj3
j1
j2
·
·
·
·
·
·
·
··
an1
an2
an3
···
···
···
···
···
···
···
a1n
···
ain + λajn
···
ajn
···
ann
=
a11 a12 a13
··· ··· ···
ai1 ai2 ai3
··· ··· ···
aj1 aj2 aj3
··· ··· ···
an1 an2 an3
···
···
···
···
···
···
···
a1n
···
ain
···
ajn
···
ann
(4) The determinant for unit matrix is 1
In other words, determinant is the unique function that saitisfying det(Sij B) = − det(B); det(Mi (λ)B) =
λ det(B); det(Aij (λ)B) = det(B); det(In ) = 1
This definition is basically what is determinant, and all other properties of determinant can all be deduced
from its uniqueness
2 3 ;
Example 66. Compute the determinant of following matrices 4 5 We proceed as following:
2 3
4 5
2 3
= −1
· · · · · · r2 + (−2) × r1
2
= −1
· · · · · · r1 + 3 × r2
1
= 2 × −1
1
= 2 × (−1) × 1
= 2 × (−1) × 1
= −2
Proposition 41.
If one row is a linear combination of other rows, then the determinant is 0
This is a skill when you can see that, but don’t worry if you can’t. you can calculate all determinant just by
row transformation.(also column transformation is valid, we will talk about it in the next few propositions)
2 3 Example 67. Calculate 4 6 Some one can see that second line is twice of the first line, so the determinant should be 0. But one can
also calculate:
43
2 3
4 6
2 3 = 2 3
= 0 × ∗ ∗
= 0
· · · · · · r2 + (−2) × r1
* can be anything
anyway, if one row of determinant is 0, the determinant is just 0
2 3 4 Example 68. Calculate 1 1 1 7 8 9 Some one can see that 3rd row is the first line plus five times the second row, so the determinant should
be 0. But one can also calculate:
2 3 4
1 1 1
7 8 9
2 3 4
= 1 1 1
5 5 5
· · · · · · r3 + (−1) × r1
2 3 4 = 1 1 1 · · · · · · r3 + (−5) × r2
= 0
because the last row of determinant is 0 row
Proof. The first property is clearly from the axiom.
Proposition 42. The determinant for diagonal matrix is the product of all diagonal elements
44
Proof.
a11
a22
a33
..
1
a22
a33
= a11 1
1
a33
= a11 a12 .
ann
..
.
ann
..
.
ann
= ···
1
1
1
= a11 a22 a33 · · · ann ..
.
1
of the upper tirangular matrix is the product of diagonal elements, that is
Proposition 43. The determinant
a11 a12 a13 · · · a1n a22 a23 · · · a2n ai3 · · · ain = a11 a22 a33 · · · ann
· · · · · · ann Proof. We can transfer upper triangular matrix to diagonal matrix by only row addition, with their diagonal
elements kept. But row addition does not change the determinant, so it is the same with the determinant of
diagonal matrix
1 4 5 2 6 Example 69. Calculate 9 Answer: 1 × 2 × 9 = 18
Proposition 44. The matrix M is invertible if and only if det(M ) 6= 0
Proof.
If det(M ) = 0,
then for any elementary row transformation of M, the determinant is 0.
45
in other words, for any invertible matrix P, det(P M ) = 0,
we choose a P such that PM=reff(M).
so by this particular choice, we know det(ref f (M )) = 0,
because reff(M) is upper triangular,
so determinant should be the product of diagonal.
that means there are 0 in diagonal. so M is not of full rank.
If det(M ) 6= 0
then for any elementary row transformation of M, the determinant does not equal to 0
because each time only differ by a non-zero scalar multiple.
That is, for any invertible matrix P, det(P M ) 6= 0.
so choose P such that PM=reff(M),
so det(ref f (M )) 6= 0,
so all element of reff(M) are non zero,
that means reff(M) is an unit matrix.
so M is of full rank,
so M is invertible.
Proposition 45. det(BA) = det(B) det(A)
Proof. If one of A or B is not invertible, then both side of the equation is 0, so true. If A is invertible, then we
det((Sij B)A)
det(Sij (BA))
=
= − det(BA))
consider the function f (B) = det(BA)
det(A) Let’s check f (Sij B) =
det(A)
det(A)
det(A) =
−f (B). And we can also use the same method to check f (Mi (λ)B) = λf (B) and f (Aij (λ)B) = f (B)
and f (In ) = 1. So f(B)=det(B). Then det(BA)
det(A) = det(B), that is det(BA) = det(B) det(A)
Proposition 46. The determinant also saitisfies column transformation property, that is det(BSij ) =
− det(B); det(BMi (λ)) = λ det(B); det(BAij (λ)) = det(B);
Proof. By previous proposition, determinant is multiplicative, so
det(BSij ) = det(B) det(Sij ) = − det(B);
det(BMi (λ)) = det(B) det(Mi (λ)) = λ det(B);
det(BAij (λ)) = det(B) det(Aij (λ)) = det(B);
This means when we calculate determinant we can both use colomn and row transformation.
Proposition 47. det(A) = det(AT )
T) =
Proof. Consider the function f (B) = det(B T ), we have f (Sij B) = det((Sij B)T ) = det(B T Sij
T
T
T
det(B ) det(Sij ) = − det(B ) = −f (B) and we can check f (Mi (λ)B) = λf (B); f (Aij (λ)B) = f (B);
f (In ) = 1 by the same process, that is f (B) = det(B). this means det(B) = det(B T )
Corollary 1. The determinant of lower triangular matrix is the product of all entries in diagonal.
1 2 5
3
)
7 8 1 8
Example 70. Without multiplying two matrix, compute det(
9
9 2 5
46
Answer:
1 2 5
3
)
7 8 1 8
det(
9
9 2 5
1 2 5
3
)
7 8 ) det( 1 8
= det(
9
9 2 5
= 63 × 120
= 7560
Next, we give some calculation method for second order determinant and third order determinant.
Before we proceed, we illustrate that there is one more important property of determinant that worth
using:
a11
a12
···
···
Proposition 48. bi1 + ai1 bi1 + ai2
···
···
an1
an2
···
···
···
···
···
a1n
···
bi1 + ain
···
ann
=
a11 a12
··· ···
ai1 ai1
··· ···
an1 an2
···
···
···
···
···
a1n
···
ai1
···
ann
+
a11 a12
··· ···
bi1 bi1
··· ···
an1 an2
···
···
···
···
···
a1n
···
bi1
···
ann
Proof. This property could also proved by uniqueness of determinant. For A = (aij ) 1≤i≤n consider the
1≤j≤n
function
a11
a12
·
·
·
···
f (A) = bi1 + ai1 bi1 + ai2
···
···
an1
an2
···
···
···
···
···
a1n
···
bi1 + ain
···
ann
−
a11 a12
··· ···
bi1 bi1
··· ···
an1 an2
···
···
···
···
···
a1n
···
bi1
···
ann
As we checked the three properties of f, we should proceed slightly take care of b. We omit proof here,
The reader could do that, a good exercise.
a b
Proposition 49. c d
a b
Proof. c d
= ac − bd
a
=
c d
b a
+
c d = c d
c d
−
b
= ad − bc
a b c Proposition 50. d e f = aei + bf g + cdh − ceg − bdi − af h
g h i Proof. We use the same method as we did before, until we can get an upper triangular, lower triangular or
block upper or lower triangular matrix.
47
a b c d e f g h i a
= d e f
g h i
b
+ d e f
g h i
c
+ d e f g h i a
e
= g h i
a
+ d
f
g h i
b
+
f
g h i
a
e
= g h i
a
− d f
g i
b
+
f
h h i
b
+ d e
g h i
c
c
+
e
f + d
g h i g h i b
− e d
g h g i
c
−
e
i h g
c
+ f d
i g h
= aei − af h + bf g − bdi − ceg + cdh
= aei + bf g + cdh − ceg − bdi − af h
2 5 Example 71. Calculate the determinant of 2 7 Answer: 2 × 7 − 5 × 2 = 4
2 5 6 Example 72. Calculate the determinant of 1 0 2 3 2 0 Answer: 2 ×0×0 + 5×2×3 + 6 ×1 ×2−6×0×3−5 ×1 ×0−2×2×2 = 0 +30 + 12 −0−0−8 = 34
2
Example 73. Calculate the determinant of 1 3 2 9 Answer: I want you to practice property of row adding
2
1 3 2
9
2
3 2
= 9
2
+ 1 0
9
= 2×3×9+0
= 54
48
1.7. Laplacian Expansion, Cofactor, Adjugate, Inverse Matrix Formula.
In this section, We generalize the method that we used to compute the formula of 2 by 2 and 3 by 3
determinants to n by n matirx, the goal is not to write down the explicite formula, but to make use of it to
analyses a method to find the inverse.
Definition 23. The block diagonal matrix is the block matrix with the same partition on rows and columns,
such that all non-zero blocks lies on the diagonal cells. Like:
A1
..
.
An
Example 74. The following could be viewed as block diagonal matrix with approperate partition
1
1
2 5 =
6 7
2 5
6 7
The above matrix could be seen as a block diagonal matrix with partition (1, 2)
2 3
4 6
2 3
4 6
=
2 4 7
1 2 0
3 5 6
6
2 4 7
1 2 0
3 5 6
6
The
seen as block diagonal matrix with partition (1, 3, 2)
above matrix could be
1 3
2 4
5 6
5
7
The above matrix is not a block diagonal matrix, as the partition on its columns and rows are not the
same
Proposition 51. The determinant of block diagonal matrix is the product of the determinant of all diagonal
blocks.
Example 75. Compute the following determinant:
1 2
3 2
2 3
1 5
Answer:
The determinant is just the product of the determinant of diagonal matrices. That is
49
1 2
3 2
2 3
1 5
1 2
= 3 2
2 3
1 5
= (−4) × 7
= −28
Definition 24. The block upper triangular matrix is the block matrix with the same partition on rows and
columns, such that all non-zero blocks lies on or above the diagonal cells. Like:
A11 · · · A1n
..
..
.
.
Ann
Definition 25. The block lower triangular matrix is the block matrix with the same partition on rows and
columns, such that all non-zero blocks lies on or below the diagonal cells. Like:
A11
..
..
.
.
An1 · · · An
Proposition 52. The determinant of block upper triangular matrix or block lower triangular matrix is the
product of determinant of blocks of diagonals.
Definition 26. Let M be a n × n square matrix over F , and 1 ≤ i ≤ n, 1 ≤ j ≤ n, then the cofactor of M
at i,j is defined to be the determinant of (n − 1) × (n − 1) matrix which is obtained by deleting the ith row
and jth column. If we multiply this determinant by (−1)i+j , then the value is called the algebraic cofactor,
denoted by Mij , where the M is the notation for original matrix.
1 2 2
Example 76. Let M= 8 2 8 , What is M12 ?
−1 2 0
Answer: M1 2 is got by computing the determinant of cancelling the 1st row and 2nd column, and times
(−1)i+j i.e.
8 8 −1 0 = −8
1+2 M12 = (−1)
Now we trying to use our method to find the determinant for n × n matirx inductively, and look what we
could find.
50
Example 77. Trying to simplify the determinant of 5 × 5 matrix into the computation of determinant 4 × 4
matrix
1
2
4
5
6
2
4
6
7
1
2
5
7
1
0
We use our standard method:
51
1
4
8
1
0
1
4
6
2
7
1
2
4
5
6
= 1
2
4
5
6
+ = 2
4
6
7
1
4
6
7
1
2
4
5
6
= 1 × −1 × 2
4
5
6
4
6
2
7
+
5
7
1
0
4
6
2
7
2
4
5
6
2
4
6
7
1
5
7
1
0
4
6
7
1
2
4
5
6
4
8
1
0
5
7
1
0
5
7
1
0
+
4
6
2
7
1
4
6
2
7
4
8
1
0
4
8
1
0
+
4
6
2
7
1
4
6
2
7
2
4
5
6
4
6
7
1
5
7
1
0
4
8
1
0
−2×
2
4
5
6
5
7
1
0
4
8
1
0
4
6
2
7
+2×
+1×
2
4
5
6
4
6
7
1
5
7
1
0
4
8
1
0
2
4
5
6
2
5
7
1
0
4
6
7
1
4
8
1
0
4
6
2
7
4
6
2
7
+
4
6
2
7
4
6
2
7
2
4
6
7
1
2
4
5
6
+
−
4
6
2
7
5
7
1
0
4
8
1
0
4
6
7
1
4
6
2
7
4
8
1
0
4
6
7
1
5
7
1
0
1
4
6
2
7
1
4
8
1
0
5
7
1
0
2
4
5
6
4
6
7
1
4
8
1
0
5
7
1
0
4
6
7
1
1
4
8
1
0
1
4
8
1
0
5
7
1
0
4
6
7
1
1
2
4
5
6
− 2
5
7
1
0
2
5
7
1
0
2
4
5
6
4
6
7
1
4
8
1
0
2
4
5
6
4
6
7
1
4
8
1
0
4
6
2
7
As you can see, in the expansion, we expanded out the all the entry of the first row, and left with the
determinant of numbers that in the different row and column with the entry, with the sign alternating, that
exactly the first row times the cofactor corresponds to first row. To conclude, we now have the Laplacian
Expansion:
Definition 27. Suppose A is a square matrix, then we can expand A by any row or any column. namely
Row expansion:
det(A) = ai1 Ai1 + ai2 Ai2 + · · · + ain Ain
52
Column expansion:
det(A) = a1i A1i + a2i A2i + · · · + ani Ani
(Please keep in mind to change the sign, as the term appeared on the formula means algebraic cofactors)
Example 78. Expand the following matrix by the second row
6 8 9 2 4 1 1 2 5 8 9
= −2 × 2 5
+4× 6 9
1 5
−1× 6 8
1 2
−2× 2 6
1 9
Expand the following matrix by the second column
6 8 9 2 4 1 1 2 5 6 9
= −4 × 1 5
+8× 2 1
1 5
As you can see, the method we used above can inductively give the formula, but what is more than that is
the following observation:
What if I change the coefficient of each corfactor by the entry of other row? Then in the previous example
it would be the laplacian expansion of the determinant:
Example 79. In the expansion of
6 8 9 2 4 1 1 2 5 8 9
= −2 × 2 5
+4× 6 9
1 5
−1× 6 8
1 2
Now if we change the coefficient by the corresponding entries of other lines, that is
6 9 6 8 8 9 −5×
−1 × +2×
1 2 2 5 1 5 What the value would be? as you can see, the line that we expanded only determined the coefficient in
front of each algebraic cofacter, not appear in any of the cofacter . if we change the coefficient, the only
change is the line that we expand by.
8
9 8 9 6 9 6 8 6
+2×
−1 × 1 5 − 5 × 1 2 = −1 −2 −5 = 0
2 5 1
2
5 we get the last equation because the second row is scalar multiple of the third row
53
Proposition 53. The othogonal
of cofactor: let A = mataijnn be a matirx, rk =
property
Al1
Al2
be any row of A, and cl = .. be the cofactor of some row. then
.
ak1 ak2 · · · akn
Aln
rk cl = 0 if k 6= l
rk cl = detA if k = l
At this point, we can make the propersition:
Now we construct a matrix, namely, Adjugate Matrix of A, it is constructed by putting Aij in the j’th
row and i’th column(remember the order). Then with the adjugate matrix, we have the following property:
Proposition 54. let A = (aij ) 1≤i≤n be a matirx, we define A∗ = (Aji )1≤j≤n to be the adjunct matrix, then
1≤j≤n
A∗ A = det(A)In
1≤i≤n
Then if the determinant is not 0, simply divide the adjugate matrix by determinant, we can get the inverse
Proposition 55. let A = (aij ) 1≤i≤n be a matirx, we define A∗ = (Aji )1≤j≤n to be the adjunct matrix, then
1≤j≤n
1≤i≤n
A
−1
= (Aji )1≤j≤n
1≤i≤n
2 4 6
Example 80. Find the inverse of 1 3 4
6 7 2
Answer:
2 4 6
1 3 4
6 7 2
=
1
−
2 4 6
1 3 4 6 7 2 3
7
1
6
1
6
4
2
4
2
3
7
−22 34 −2
1
22 −32 −2
= − 64
−11 10
2
54
− − 4
7
2
6
2
6
6
2
6
2
4
7
4 6
3 4
2 6
− 1 4
2 4
1 3
2. L INEAR E QUATION
Introduction: We already get a solid background in computation of matrix. Before we start, we should
learn to solve linear equations, which will be used extensively in the future study. The matrix multiplication
tell us what is output if we input some number. Now, the Linear Equation is the other way, is to know what
is input into the system such that it creates such an output.
Definition 28. A linear equation over field F is an algebraic equation in which each terms is either constant(in F ) or a product of the constant and (the first order of) single variable.
Example 81. The following are linear equations
3x + 2y + z + w = 3
x + 2y = 5
3=7
Definition 29. A system of linear equation is a bunch of equation with the same variables together
Example 82. The following are system of linear equations
3x + 2y + z = 6
4x + y
= 5
x+z
= 2
Definition 30. A solution of a linear equation is an assignment of each variable by a scalar in F , such that
the equation is true when we plug in the scalars for corresponding varibles. A solution of a system of linear
equation is an assignment of each variable by a scalar in F , such that each of the equation is simutaneously
true when we plug in that scalar to corresponding variebles.
Example 83. If we have a system of linear equation like
3x + 2y + z = 6
4x + y
= 5
x+z
= 2
Then the assignment:
x = 1
y = 1
z = 1
is a solution for linear equation, because when we plug in every assignment, the equation is true:
3(1) + 2(1) + (1) = 6
4(1) + (1)
= 5
(1) + (1)
= 2
Another important point of view is that we can write the system of linear equation into matrix by matrix
multiplication, in this way, The system of linear equation is the same as a single matrix equation Ax = b. A
is called the coefficient matrix of this system.
Example 84. We illustrate some examples here. For the linear equation:
3x + 2y + z = 6
4x + y
= 5
x+z
= 2
55
recall the matrix multiplication, we can write each equation as
x
3 2 1
y = 6
z
x
4 1 0 y = 5
z
x
1 0 1 y = 2
z
Now we put them into blocks.
3 2 1
4 1 0
1 0 1
x
y
z
x
y
z
x
y
z
6
= 5
2
Remember the right common factor(resp. left) of a column vector(resp. row vector) could factor out to
the right(resp. to the left), that is
3 2 1
4 1 0
1 0 1
x
y
z
x
y
z
x
y
z
=
3 2 1 x
3 2 1
x
4 1 0 y = 4 1 0 y
z
1 0 1
z
1 0 1
Thus the linear equation is equavalent to say
3 2 1
x
6
4 1 0 y = 5
1 0 1
z
2
Now you can try to expand that by using matrix multiplication and you will find that the single equation
for matrix is exactly the same for the system of linear equations.
Now we say two geometric meanings of linear equation, one is from the view from row vectors of A,
another one is from the view of column vectors of A.
By viewing each row of A, namely, looking at each single equations in the system of equation. each
single equation gives a codimension 1 space in the whole space, and the solution is the intersections of the
spaces. Let’s show some example to make this idea clear.
56
Example 85. The equation
0 1 2
0 0 1
1 0 0
is equavalent to the system of linear equations:
y + 2z
z
x
x
1
y = 1
z
2
= 1
= 1
= 2
And each of the equation could be viewed as a equation of a plane in 3-dimensional space, namely,
H1 , H2 , H3 for each equation. The solution is some assignment of x, y, z, if we view this assignment as
coordinates of some point. Then the coordinates saitisfies each of the equation. Remember the coordinate
of a point saitisfies a equation of a surface is the same as saying the point is on the surface. So the solution
represents a point such that this point is in each of the plane. And this is the same as asking for intersections.
By solving the equation, we find the intersection is (2, −1, 1)
Example 86. Previously we gave a example such that the intersection is a unique point. While it is still
possible to choose other position of planes such that it could have different relative position. Consider the
following example
0 1 0
x
1
0 1 1 y = 3
0 1 −2
z
3
That is,
= 1
y
y+z = 3
y − 2z = 3
Then the geometric picture is like
57
showing that there is no point share on all three plane. And indeed, if we trying to solve this linear
equation, it would have no solution.
Example 87. It could also be the case that a system of linear equation have multiple solutions. Consider
the follwing example
0 1 0
x
1
0 1 1 y = 3
0 1 −2
z
−3
That is,
= 1
y
y+z = 3
y − 2z = −3
And the graph is like
There are multiple points that on each of the plane, clearly, every point with y coordinate 1 and z coordinate 2 would on the intersection. For example (1, 1, 2), (2, 1, 2), (π, 1, 2) .... are all on the intersections.
Indeed, if we solve the equation, we will see there is no restriction for x, so x could be any value, and then
this system of equation have multiple solutions.
Now we look at the equation column by column, as we previously studied, right multiplying a matrix
means create the column combination of the columns of the left matrix, and the coefficient comes from the
corresponding column.
Example 88. Look at the equation
2 5
1 2
x
y
=
9
4
Of course, by looking at rows, we could view the solution as the intersection of two lines.
Now we change another point of view. We look at the columns, this question could viewed this way
x
9
2 5
x
2
5
2
5
=
=
=
x+
y
4
1 2
y
1
2
1
2
y
The last equation is by multiplying by block.
58
So the original equation is the same as asking for the right amount x, y that
2
1
x+
5
2
y=
9
4
9
4
If you treat column vector as coordinates on the plane. Then
is a point on the plane, and
5
2
5
and
are two vectors, which start at original point and ends at
and
respectively.
2
1
2
2
1
Now think, you can only go by the direction of the blue arrow, in how far of each direction you want to go
to get there? of course, wecan’tgo arbitrary amount,because
that might miss our goal. Finally, we figure
2
5
out we should go 2 unit of
, and 1 unit more of
1
2
This is the column picture of system of linear equation – to get the right amount of combination to create
the thing that we disired.
Now let’s focus on how to solve for linear equations.
Definition 31. In the system of linear equation Ax = b, if b is a zero column vector, we call this system
of linear equation, that is Ax = 0, as homogeneous system of linear equation, if b 6= 0, then we call this
system of linear equation non-homogeneous
Example 89. The following is a non-homogeneous linear equation
x + 2y + 3z = 5
The following is a homogeneous linear equation
x + 2y + 3z = 0
The following is a non-homogeneous system of linear equation
x + y + 2z = 1
x+z
= 0
x+y
= 0
59
The following is a homogeneous system of linear equation
x + y + 2z = 0
x+z
= 0
x+y
= 0
2.1. Non-Homogeneous Linear Equation, The existence of the solution. We start by the method to find
the solutions of non-homogeneous linear equation. This method is called Gaussian Elimination
Proposition 56. Given a system of linear equation, we can do the following transformation and that will
not change the solution of the equation.
(1) Switch any of the two equations
(2) Multiply any of the equation by a non-zero scalar λ
(3) Add λ multiples of a equation to another equation.
Example 90. Suppose we have a system of linear equation:
x + y + 2z = 1
x+z
= 2
x+y
= 0
If we switch the first and second equation, of course they are of the same meaning:
= 2
x + y + 2z = 1
x+z
r1 ↔r2
x+z
= 2 −−
x + y + 2z = 1
−−→
x+y
= 0
x+y
= 0
If we add any multiple of the second equation to the first. of course they should give the same solution
x + y + 2z = 1
4x + y + 5z = 7
r1 +3×r2
x+z
= 2 −−
x+z
= 2
−−−→
x+y
= 0
x+y
= 0
If we times any equation by a non-zero number, they also should give the same solution:
x + y + 2z = 1
4x + y + 5z = 7
5×r2
x+z
= 2 −−−→
5x + 5z
= 10
x+y
= 0
x+y
= 0
As you can see the above statement is obvious. In the mean time, it also looks pretty like the row
transformation, is there something to do with the row transformation of the matrix? The answer is definitely
yes. To see the fact, remember that we previously illustrate that the system of equations is equavalant to a
single matrix equation.
Ax = b
We made a statement that if we do row transformation of A and b simutaneously, then it would not change
the solution. This statement is nothing more than claiming that for any invertible matrix P, P Ax = P b and
Ax = b gives the same solution.
To see why, we have to check every solution of Ax = b is a solution of P Ax = P b, and every solution
of P Ax = P b is a solutio of Ax = b. To do the first, suppose the column vector x0 is a solution of Ax = b,
then of course Ax0 = b, now we multiply equation of both side by P, so we get P Ax0 = P b, so we checked
that x0 is a solution of P Ax = b.
Now assume the column vector x1 is a solution of P Ax = P b. then of course P Ax1 = P b, then we
multiply P −1 on both side of the equation, then P canceled, so Ax1 = b, that exactly means x1 is a solution
for Ax = b
60
Then we can use the same method that we use in the row transformation in matrix to reduce the coefficient
matrix to reduced echelon form without changing the solution of that. To show an example, we show both
ways of doing it.
Example 91. Let’s solve the non-homogeneous equation:
3x + 2y + z = 6
x+y
= 2
2x + 2y
= 4
To write it into a matrix form, it is the same as
3 2 1
x
6
1 1 0 y = 2
2 2 0
z
4
In order to do the row transformation simutaneously, we put them in a row of block matrix.
3 2 1
6
1 1 0
2
2 2 0
4
Now we are ready to start doing row transformation on A and b simutaneously.
3 2 1
6
1 1 0
2
2 2 0
4
0 −1 1
r1 −3×r2
−−
−−−→ 1 1 0
2 2 0
0
2
0
0 −1 1
r3 −2×r2
−−
−−−→ 1 1 0
0 0 0
0
2
0
1 1 0
0 −1 1
0 0 0
2
0
0
1 0 1
−−−−−→ 0 −1 1
0 0 0
2
0
0
2
0
0
r ↔r
1
2
−−
−−→
r1 +1×r2
(−1)×r2
−−−−−→
1 0 1
0 1 −1
0 0 0
Now is the same solution as
1 0 1
x
2
0 1 −1 y = 0
0 0 0
z
0
which means x + z = 2 and y − z = 0
61
Remember in the reduced row echelon form, it is possible for leading 1 to appear in the last column. To
show this, now we see another example
Example 92. Let’s solve the non-homogeneous equation:
3x + 2y + z = 6
x+y
= 2
2x + 2y
= 5
3 2 1
1 1 0
2 2 0
6
2
5
0 −1 1
r1 −3×r2
−−
−−−→ 1 1 0
2 2 0
0
2
1
0 −1 1
r3 −2×r2
−−
−−−→ 1 1 0
0 0 0
0
2
1
1 1 0
0 −1 1
0 0 0
2
0
1
1 0 1
r1 +1×r2
−−
−−−→ 0 −1 1
0 0 0
2
0
1
2
0
1
r ↔r
1
2
−−
−−→
(−1)×r2
−−−−−→
1 0 1
0 1 −1
0 0 0
Now is the same solution as
1 0 1
x
2
0 1 −1 y = 0
1
0 0 0
z
which means x + z = 2 and y − z = 0 and 0 = 1
With previous examples, we have seen it might be case that after transformation, we might have equality
that totally doesn’t make sense like 0 = 1, if this equation appears, none of the choice of the varables would
able to make this equality make sense. Then we call this system of linear equation inconsistent. As we
defined solutions to be the numbers saitisfing equations simutaneously, but with 0 = 1, we are not able to
do that, this means this equation does not have solution.
Proposition
57. The system of equation Ax = b exists solution if and only if the reduced echelon form of
A b does not have leading 1 in the column of b.
Now we learn how to write down the solution. The key is to express the varibles of leading 1 in terms of
free varibles. The free varibles are those who does not correspoding to leading 1.
62
Example 93. Suppose we reduced the system of linear equation to
1 0 1
x
2
0 1 −1 y = 0
0 0 0
z
0
which means x + z = 2 and y − z = 0
Now the leading 1 correspond to the variable of x and y, then we can express x and y in term of other
varibles. In our case, it is
x = 2−z
y = z
Then the varible which is not correspond to leading one is called free-varibles, we can assign it to be any
value. For example, we can let z = 1, then we get a solution
x = 1
y = 1
z = 1
2.2. Homogeneous Linear Equation, The uniqueness of the solution. Previously we talked about how
to know the existence of solutions of system of linear equation, is by looking at whether the leading one
appears at the last column. And we defined consistency. Now we start talking about the uniqueness of linear
equation.
Proposition 58. Suppose u and v are two solutions of the equation Ax = b, then u − v is the solution of
the equation Ax = 0
Proof.
Au = b and Av = b
then Au = Av
then Au − Av = 0
that is A(u − v) = 0
Example 94. The system of linear equation
3x + 2y + z = 6
x+y
= 2
2x + 2y
= 4
We can check there exists
of course,
solution
for
such
a equation,
and
there are multiple solutions. Let’s
x
1
x
0
take any two of them, say y = 1 and y = 2 . As you can see, they both saitiesfies
z
1
z
2
1
the equation, and thir difference, namely, −1 should saitisfies the equation:
−1
3x + 2y + z = 0
x+y
= 0
2x + 2y
= 0
and it really is. because take the first equation of the system for example, we have
63
3(1) + 2(−1) + 1(−1)
1
1
0
because −1 = 1 − 2
−1
1
2
= 3(1 − 0) + 2(1 − 2) + 1(1 − 2)
= [3(1) + 2(1) + 1(1)] − [3(0) + 2(−2) + 1(−1)]
1
0
because 1 2 are solutions of 3x + 2y + z = 6
1
2
= 6−6
= 0
With the previous discussion,if we know one particular solution of Ax = b, and all homogeneous
solutions of Ax = 0, then we can know all solutions of Ax = b by sum up particular solution with
homogeneous solutions.
Proposition 59. The solution set of Ax = 0 saitisfies the following
0
0
(1) .. is a solution.
.
0
(2) If u, v are two solutions of equation Ax = 0, then for any scalar λ, µ in field F , we have u×λ+v×µ
also a solution. More generally, linear combination of solutions is also a solution.
Proof. Suppose v1 , v2 , · · · , vn are solutions of Ax = 0, so suppose a1 , a2 , · · · , an are any scalars in field.
then A(v1 a1 + v2 a2 + · · · + vn an ) = (Av1 )a1 + (Av2 )a2 + · · · + (An vn )an = 0a1 + 0a2 + · · · + 0an = 0.
Thus, v1 a1 + v2 a2 + · · · + vn an ) = (Av1 )a1 + (Av2 )a2 + · · · + (An vn )an = 0a1 + 0a2 + · · · + 0an = 0
is a solution
Definition 32. All solutions of the equation Ax = 0 is called the (right) null space of A.
We call it ”space” here because with previous proposition, we will see that they are actually forms a linear
space, we will learn linear space in the future.
Because for any invertible matrix P Ax = 0 is equavalent to say Ax = P −1 0 , this is equavalent to
P Ax = 0. so the solution set of Ax = 0 and P Ax = 0 are the same. Thus, we could choose P such that
P A is the reduced echelon form. and let’s expand to see what does this tells us.
Example 95. Solve the system of linear equation of the reduced echelon form of following:
1
2
1
3
5
1
This is equavalent to the following equation
x1 + 2x2 + 3x4 = 0
x3 + 5x4
= 0
x5
= 0
and thus we can write the solutions like
64
x1
x2
x3
x4
x5
=0
x1 = −2x2 − 3x4
x3 = −5x4
x5 = 0
Thus, the choice of x2 and x4 is free, we have a solution whenever we assign a value for them. and we can
exaust all solutions by assigning different value for them. Look at the oringinal equation, they are varibles
not corresponding to leading 1
The variables that not corresponding to leading 1 is called free varibles. as we can see, we can choose
the value of free varibles whatever we like. and we can express other varibles by the value of free varibles.
That means, all the solutions is one to one correspond to the choice of value of free varibles. And each
choice of free varibles determines a linear combination of solutions. The choose of free varibles to be 1 and
other 0 is called a elementary solution. the number of free varibles is called the dimension of (right) null
space of A.
Example 96. What are elementary solutions for the system of homogeneous linear equation?
x1 + 2x2 + 3x4 = 0
x3 + 5x4
= 0
x5
= 0
write this into matrix form
1
1
1
3
5
1
x1
x2
x3
x4
x5
=
0
0
0
0
0
We discovered that the leading 1 corresponds to x1 , x3 and x5 . And thus the free varibles are x2 and
x4 ,So we express the varible corresponds to leading 1 in term of other varibles:
x1 = −2x2 − 3x4
x3 = −5x4
x5 = 0
The first elementary solution is by setting x2 = 1 and x4 = 0.
x1 = −2
x2 = 1
x3 = 0
x4 = 0
x5 = 0
−2
x1
x2 1
Thus, the first elementary solution is:
x3 = 0
x4 0
x5
0
The second elementary solution is by setting x2 = 0 and x4 = 1 So
65
x1
x2
So the Second elementary solution is:
x3
x4
x5
x1 = −3
x2 = 0
x3 = −5
x4 = 1
x5 = 0
−3
0
= −5
1
0
We denote the number of elementary solutions of homogeneous linear system Ax = 0 as null(A)
Proposition 60. null(A) = n - rank (A), where n is the number of columns of A
Proof. The number of varibles is the number of columns of A, and the rank is the number of leading 1. The
free varible are those varible that not corresponding to leading 1. Thus null(A)=# of free varible = n - # of
leading 1 = n - rank(A)
Proposition 61. Suppose the solution of equation Ax = b exists, then the solution is unique if and only if
any of the following equavalent condition holds:
(1) Ax = 0 only have 0 solution
(2) null(A) = 0
(3) rank(A) = number of columns of A
Corollary 2. Suppose A is a square matrix, then the solution of Ax = b exists and unique if and only if A
any of the equavalent condition holds:
(1) A is invertible
(2) det(A) 6= 0
(3) A is of full rank
In this case, the solution is uniquely given by x = A−1 b.
A∗
, then the solution is given by Crammer’s rule.
If we combine the method of expressing A−1 = det(A)
With previous discussion about existance and uniqueness, we wish to completely solve our linear equation
by representing all its solutions. There are several ways to present them.
Example 97. Represent the solution set of
x1 + 2x2 + 3x4 = 6
x3 + 5x4
= 6
x5
= 1
We know a particular solution for them is
x1
x2
x3
x4
x5
=
66
1
1
1
1
1
So all the solution should be in the form
x1
x2
x3
x4
x5
=
=
=
=
=
1+x
f1
1+x
f2
1+x
f3
1+x
f4
1+x
f5
Where each xei represents the diffrence between any 2 solutions.
Now these xei should saitisfies the homogeneous equation
f1 + 2f
x2 + 3f
x4 = 0
x
x
f3 + 5f
x4
= 0
x
f5
= 0
And their solutions could be represent as
f1 = −2f
x2 − 3f
x4
x
x
f3 = −5f
x4
x
f5 = 0
we can assume x
f2 = t and x
f4 = w, thus we have
x
f1 = −2t − 3w
f2 = t
x
x
f3 = −5w
x
f4 = w
x
f5 = 0
Conbinede with particular solution, we have
x1 = 1 − 2t − 3w
x2 = 1 + t
x3 = 1 − 5w
x4 = 1 + w
x5 = 1
So the solution set is equal to
{
x1
x2
x3
x4
x5
|x1 = 1 − 2t − 3w; x2 = 1 + t; x3 = 1 − 5w; x4 = 1 + w; x5 = 1, where t,w ∈ R}
Now we introduce sevral ways to represent solution sets.
Parametric Form We assign arbitrary values, called parameters, to the free varibles and all the varibles
is determined by the parameters, the solution representing by this form is called parametric form. Like
{
x1
x2
x3
x4
x5
|x1 = 1 − 2t − 3w; x2 = 1 + t; x3 = 1 − 5w; x4 = 1 + w; x5 = 1, where t,w ∈ R}
or directly we represent as
67
x1 = 1 − 2t − 3w
x2 = 1 + t
x3 = 1 − 5w
x4 = 1 + w
x5 = 1
Linear Combination of Column Vector Form We can write the solution as linear combination of column vector form. like
1
−2
−3
1 1
0
{ 1 + 0 t +
−5 w|where t,w ∈ R}
1 0
1
1
0
0
or directly as
x1
1
−2
−3
x2 1 1
0
x3 = 1 + 0 t + −5 w
x4 1 0
1
x5
1
0
0
Matrix Form This presentation is by using the matrix multiplication, and present all the solutions as
1
−2
−3
1 1
0
t
−5
{ 1 + 0
|where t,w ∈ R}
w
1 0
1
1
0
0
or directly as
x1
1
−2
x2 1 1
x3 = 1 + 0
x4 1 0
x5
1
0
Of course, all of the set above are equavalent each other.
−3
0
−5
1
0
t
w
3. V ECTOR S PACES , L INEAR T RANSFORMATION
3.1. Actions, Matrix of Actions. Introduction:We have seen that matrices of numbers, and matrix of
matrix, an natrual think about it is that is there matrix of anything? Now let’s define a border concept,
Actions.
3.1.1. Actions.
Definition 33. Suppose X,Y,Z is three sets, an action is simply a map X × Y −→ Z. In this case, we say
that the element of X left act on the element of Y, result in the element of Z.
Example 98. Suppose you have a rock and an egg, when you want to eat it, the idea is to use the rock act
on it: we can write this expression as multiplication:
Rock hit egg = good rock + damaged egg
68
Then the words”rock hit” could be viewd as an action, it can left act on any object and then gives the
result. Let’s see different result if you throw your rocks on.
Rock hit sea = disappeared rock + sea
Also the action could viewed in language just by simply combine all phrases:
Example 99.
Operator Object
Action
Result
I eat
an apple ”I eat” lef t act on ”an apple” ”I eat an apple”
I peel
the apple ”I peel” lef t act on ”an apple” I peel the apple
Example 100.
Operator Object
Action
Result
I eat
an apple ”I eat” lef t act on ”an apple” ”I eat an apple”
I peel
the apple ”I peel” lef t act on ”an apple” I peel the apple
The action from left and from right should be distinguished, Look at the following example
Example 101. Now consider the composition of two operator ”wear shoes × wear socks”. If your feet
accept action from left. Then this operator left act on your feet, the phinomenon is:
wear shoes × wear socks × yourf eet
= wear shoes × the f eet wearing socks
= the f eet wearing shoes in which the socks is wearing inside the shoes
Now let’s see if your feet is defined to accept action of this kind of operator from right. Then this operator
right act on you feet, the phinomenon is
yourf eet × wear shoes × wear socks
= the f eet waring shoes × wear socks
= the f eet wearing shoes in which the socks is wearing outside of the shoes
Because we choose to write certain composition of operator in one way, so we should have already defined
which direction that the operator should act on, and one can not change the direction.
Definition 34. Suppose we already have an action X × Y → Z, then we define the transpose of X, Y, Z, to
be X T , Y T , Z T respectively, in which the elements are not changed. but the direction of receive action is
reversed. More precisely, is calculated by the rule (ab)T = bT aT
If originally Y accept left action from X, after transpose it. Y T is defined to accept right action from X T .
In other words, everything didn’t change, but looks like the world seeing in the mirror.
Now we give an example
Example 102. Suppose I eat × an apple = I eat an apple, Calculate (an apple)T × (I eat)T
(an apple)T × (I eat)T
ll
= (I eat × an apple)T
= (I eat an apple)T
The action above is the most general actions. In here, we only consider a specific action. We want our
sets to have some addition structure so that we can use Matrix Theory to represent them.
Definition 35. A set G equipped with an operator ”+” is called an abelian semi-group, if any three element
a, b, c of G satisfies the following
(1) a + b is an element in the set G
69
(2) a + b = b + a
(3) a + (b + c) = (a + b) + c
Note that we didn’t say anything about inverse and zero, so generally we can only do addition, no subtracting.
Example 103. Suppose X is a set, G is the set of all subset of X, and define ”+” to be taking union of subset.
Then it clearly satisfies the 3 axioms. So is the taking intersection.
Example 104. Suppose G is all positive integers, we define ”+” to be taking the greatest common divisor,
then it clearly satisfies the 3 axioms.
Now we would like to consider the case that an action of O(operators) act on a semi-group G with result
also an element of G itself. In other words, the action is O × G → G
To illustrate example to play with. let G simply be the collection of all subsets in R2 , and the addition is
taking union of sets. We define O to be set of operations including any rotation and reflections. And we let
O to left act on G. We use the following symbols to represent rotation or reflection.
Example 105. We denote the following
: rotate clockwise by 90 degree
: rotate counterclockwise by 90 degree
: vertically reflection
: horizontally reflection
Now think these as operators, and define the ”+” as union of subsets, then we can calculate the following:
×
×
+
=
×
+
=
+
=
=
+
=
Now if we put all the element into matrix, and regard multiplication as action. addition as the addition
of the semi-groups. Then we can notationally simplify a lot of expressions. To give you an idea how to
Calculate them, see the following expression.
Example 106.
=
=
×
+
+
=
70
×
=
×
×
×
×
=
3.2. Linear Spaces, dimension. Introduction: Now we start studying linear algebra. With the basis and
understandings about matrix before, we can simplify a lot of expressions we need and gives us easier look
to understand.
Definition 36. The (right) linear space over field F is a set V, endowed with an abelian group structure,
which admits action of scalar of F from right, In other words. V is a set equipped with an operator ”+”
satisfying
(1) (V,+) is an abelian group, which means:
1) For any v ∈ V , w ∈ V , u = v + w is an element in V
2) For any v ∈ V , w ∈ V , v + w = w + v
3) For any v ∈ V , u ∈ V , w ∈ V ,(u + v) + w = u + (v + w)
4) There exists 0 ∈ V , such that for any v ∈ V , 0 + v = v
5) For any v ∈ V , There exists −v ∈ V , such that v + (−v) = 0
(2) (V,+) admits an action of field from right, which satisfying
1) For any v ∈ V , v × 1 = v
2) For any v ∈ V , λ ∈ F , v × λ defines an element in V
3) For any v ∈ V , λ, µ ∈ F ,v × (λµ) = (v × λ) × µ
4) For any v ∈ V , λ, µ ∈ F ,v × (λ + µ) = v × λ + v × µ
We often call the element of linear space as vectors
Definition 37. A subspace W of linear space V , is a subset of V , such that for any u,v in W , u×λ+v ×µ ∈
W for any scalars λ and µ
Remark 3. (λv means v × λ)It is an historical mistake to put the scalar action from left. In the future we
will introduce linear map, which people wish to write on the left of the vector. And the fact is that vector
admits scalar action and linear map action, but this two action has to be appear in different side. If linear
map with to appear on the left, then scalar should act on the right, and vise versa. People remember this
mistake though, and some textbook in ascent time trying to keep scalar in the left and write the linear map on
the right. However, that suggestion didn’t become popular because people already get used to write abstract
action on the left. like f (x), cosx. So, because of the historical mistake. The λv will means v × λ. It doesn’t
matter because the scalars are commutative. For example (µλ)v is v × µ × λ, which means µ should act
first and then λ, but the historical mistake symbol will looks like λ acts first and then µ. But it gives the
same result as λ and µ are commutative. In the future study, when you treat modules over non-commutative
rings. please let the scalar act from right, don’t let the historical mistake really misleading you.
71
Example 107. The plane with selected point and structure of vector addition is a linear space over R.
Indeed, the selected point is called origin. and for any point in the plane, it corresponds a vector, this
vector is starting from origin and ending at this point. The sum of two vector composed by applying the
parallelogram rule. The scalar multiplication of the vector is obtained by enlarging the vector by the
corresponding ratio.
Scalar multiplication:
Parallelogram law:
What is the subspace of this space? certainly by definition the space itself could be a subspace. This is
the only 2 dimensional subspace in this space. And 1 dimensional subspaces are lines that passes through
origins. The 0 dimensional subspace is a singleton set :{0}
2-dimensional subspaces: (the cube outside is auxiliary)
The geometric intuition for subspaces is not only a flat space, but also the space should pass though the
origin, or else it would not become a subspace. The most simple reason is that a subspace should contain
the vector 0. Just like the following example.
Example 108. The subset as shown in the following picture does not form a subspace:
The reason is simple, even if we just pick out two vectors, which means starting at origin and ending at
the plane. then by parallelogram law, then ending point of the sub would exceeded the plane. So this subset
is not closed even under vector addition. So a linear subspace should pass though the origin.
Example 109. Denote the set of all the n×1 matrices over F as Mn×1 (F), endowed with additive structure:
The sum of two matrices is just the normal sum of two matrices. the scalar multiplication is naturally
obtained by right multiplying a scalar. Thus, this is indeed a vector space.
72
x
Example 110. Now let V = M3×1 (F), then V is a linear space with elements looks like y . Does the
z
x
set W = { y |x + y + z = 0} a linear subspace?
z
x1
Answer: Yes, it is a linear subspace, we verify it. For any λ, µ ∈ F , we compose x2 λ +
x3
y1
λx1 + µy1
y2 µ = λx2 + µy2 , because we know that each of the vector comes from W, so we have the
y3
λx3 + µy3
condition that
x1 + x2 + x3 = 0
, and
y1 + y2 + y3 = 0
.
Combine this two equation we get
(λx1 + µy1 ) + (λx2 + µy2 ) + (λx3 + µy3 ) = 0
That means
λx1 + µy1
λx2 + µy2 ∈ W
λx3 + µy3
x
Example 111. The same notion as before, is the subset W = { y |x + y + z = 3} a subspace? The
z
1
3
answer is no. because simply take 1 and 0 as two vectors in W, then the sum of the two vectors
1
0
4
4
1 is no longer satisfies the condition x + y + z = 3, so 1 does not lies in the subset. This subset
1
1
is not closed under addition, so is not a subspace.
pass the oringin, and now at least
Another
point of view is gemetrically speaking the subspace should
0
0
0 should in the subset, but this subset W does not contain 0 . This is exactly the same case as
0
0
we illustrate before.
Example 112. Now let V = {ax2 + bx + c|a, b, c ∈ R} to be the polynomials of degree less or equal
than 2 with real number coefficient. The addition structure is simply sum of two polinomials, the scalar
multiplication is defined also in the usual way. This is a linear space.
Example 113. Now consider the subset W = f ∈ V |f (0) = 0, is W a subspace? Yes, it is, indeed, for any
two polynomials f, g ∈ W , we have f (0) = g(0) = 0, then of course any linear combination would give
h(0) = λf (0) + µg(0) = λ0 + µ0 = 0, then h is an element in W, so W is a subspace.
73
Definition 38. Suppose V is a linear space over field F , The linear combination of v1 , v2 , · · · , vm by scalars
a1 , a2 , · · · am means the vector that is obtained by
a1 v1 + a2 v2 + · · · + an vn
Definition 39. Suppose w ∈ V ,v1 , v2 , · · · , vn ∈ V we say w is able to be linearly expressed by{v1 , v2 , · · · , vn }
if w can be written as a linear combination of v1 , v2 , · · · vn with some coefficient.
Example 114. If we draw arbitrary three vectors in a plane, then someone would be the linear combination
of others others.
To see that, imagine you are driving a car from the origin towards the direction of the first vector. At the
same time, your eyes are keep looking on the direction of the second vector. At some point, you will saw
the endpoint of w, and then you made a decision immediately to go the direction towards the second vector.
Finally you got w. In the whole process, we only go by 2 directions. so w is a linear combination of this two
vector with some appropriate amount.
We can also use the same philosophy to imagine every 4 vector in the space, someone should be the
linear combination of other three. (In the general position, you driving along 1 vector, with eyes keep
looking around along the plane that spaned by the other 2.)
Remark 4. The previous one is just a example. When you are driving, please keep your eyes looking on the
road! never look around. No linear algebra when you drive!
2
1
0
Example 115. Let V = M3×1 (R), 4 is a linear combination of 1 and 1 , with the
2
1
0
coefficient 2 and 2, that is
2
1
1
4 = 1 ×2+ 2 ×2
2
1
1
Example 116. Let V = {ax2 + bx + c|a, b, c ∈ R}, the polynomial x2 + 1 is a linear combination of x2 and
1, with coefficient 1, 1. x2 + 1 is also an linear combination of x2 − x and x + 1, also with the coefficient
1,1.
Definition 40. The subspace spanned by {v1 , v2 , · · · , vn } means the subset of all possible linear combinations of {v1 , v2 , · · · , vn }. This set forms a linear subspace, is called
the subspace spanned by {v1 , v2 , · · · , vn }
denoted as span{v1 , v2 , · · · , vn } or span v1 v2 · · · vn
74
Example 117. Geometrically, the subspace spanned by a bouch of vector is the minimal linear subspace
that contains these vector. or it could also be viewed as the maximal subspace that is generated by these
vector, here generate means put all possible linear combination of them in.
1
0
Example 118. Suppose V = M3×1 (R), and v1 = 0 , and v2 = 1 , then the subspace spaned by
0
0
v1 and v2 are all possible linear combinations of v1 and v2 . let’s see
1
0
x
0 ×x+ 1 ×y = y
0
0
0
Then the linear space consists of all column vector with the last entry zero.
Definition 41. Suppose V is a linear space over F , what we call a linear relation of S = {v1 , v2 , · · · , vn }
is a equation of linear combination of these vectors to produce 0, i.e.linear relation is a equation like
a1 v1 + a2 v2 + · · · + an vn = 0
clearly we can choose all the coefficient to be zero, if it does, then we call this as trivial linear relation.
Definition 42. Suppose S = {v1 , v2 , · · · , vn } are subset of linear space V, we call the set S linearly dependent if they have a non-trivial linear relation. That is, there exists some scalar a1 , a2 , · · · , an , such that not
all of them are 0, and
a1 v1 + a2 v2 + · · · + an vn = 0
Definition 43. Suppose S = {v1 , v2 , · · · , vn } are subset of linear space V, we call the set S linearly independent if there exists some scalar a1 , a2 , · · · , an , such that
a1 v1 + a2 v2 + · · · + an vn = 0
forces a1 = a2 = · · · = an = 0. That is, this set of vectors only have trivial linear relation.
Geometrically speaking. Linearly dependence is the case when two vectors are collinear, three vectors
are coplanar, or even more extremely, three vectors collinear. Normally, n vectors should able to span an
n-dimensional space. Whenever they failed to do so, they are linearly dependent, and in this case, we can
always get rid of the redundant vector and minimize the set with keeping its ability to span the space we
want. Look at following picture:
75
These three vectors are linearly dependent because they can only span a plane, now in this case, get rid of
any vector and left with the other two would also able to span the plane. Then the left two would be linearly
independent.
In this case, this three vectors are also not able to span the three dimensional space, but we can only get
rid of the collinear one to make sure they would able to span the plane.
Example 119. The zero vector itself would be a linearly dependent vector, to see why is that, look at the
expression
a0 = 0
we can choose a to be any non-zero scalar to cook up this zero vector. so 0 vector itself is linearly
dependent
Example 120. In the plane, two collinear vectors are linearly dependence, precisely, if w = λv,then the
following is true:
w − λv = 0
the first coefficient is 1 and the second is λ
Example 121. If w can be linearly expressed by v1 , v2 , · · · , vn , then if we put them together, namely
w, v1 , v2 , · · · , vn the set of these vector would be linearly dependent.
w = a1 v1 + a2 v2 + · · · + an vn
then we can write it as
0 = (−1)w + a1 v1 + · · · + an vn
then clear not all of the coefficeint are 0(at least the first -1 is not 0), so they are linearly dependent.
This is exactly where the word dependent come from. linearly dependent means some of the vector could
be expressed as a linear combination of other vectors. So this vector depend on others.
76
1
0
1
Example 122. If V = M3×1 (R), the vectors 0 , 1 and 1 would be linearly dependent,
0
0
0
because we can find some not-all-zero scalar that
0
1
1
0
1 × 1 + 0 × 1 + 1 × (−1) = 0
0
0
0
0
1
When we get rid of 1 , the left two vectors become linearly independent, to see this fact, suppose we
0
have
1
0
0
0 ×x+ 1 ×y = 0
0
0
0
That will forces
x
0
y = 0
0
0
, that is
x = 0
y = 0
Example 123. Suppose we have bunch of vectors S = {v1 , v2 , · · · , vn }, no matter they are linearly independent or dependent, now, we put the zero vector in. T = S ∪ {0} = {0, v1 , v2 , · · · , vn } Then this set of
vectors will become linearly dependent. To see this fact, we only consider to compose them to be 0. Now we
choose the coeffficent of 0 to be 1, and coefficient of others to be 0, now we get
0 × 1 + v1 × 0 + v2 × 0 + · · · + vn × 0 = 0
Now we successfully chosen some not-all-zero scalar to combine them to be 0, so they are linearly dependent.
Keep in mind that why we want to choose a bunch of vector in the space and talked a lot about the span
or linearly Independence? Because there are infinitely many vectors in the space. And they are invisible.
We want to choose finitely many of them so that we can represent them. The method of represent is by
linear combination. We want to choose as much vector as we can to be able to represent every vectors
in the space(So these vectors are able to span the whole space).Thus we would guarantee the existence of
the scalars to represents the vector. In the mean time, we also want to choose as few vector as we can
for our convenience by getting rid of redundant one(linearly dependent one). Thus we could guarantee the
uniqueness of the scalars to represents the vector.
Proposition 62. Let V be a linear space over F , S = {v1 , v2 , · · · , vn } be the set of vectors, then the
following are equivalent
(1) v1 , v2 , · · · , vn spans V
(2) For any w ∈ V , there exists a1 , a2 , · · · , an , such that w = a1 v1 + a2 v2 + · · · + an vn
Proof.
(1) → (2)
Because v1 , v2 , · · · , vn spans V
Then by definition every element of V is a linear combination of v1 , v2 , · · · , vn
That exactly means there exists a1 , a2 , · · · , an such that w = a1 v1 +a2 v2 +· · ·+an vn
77
(2) → (1)
Choose any vector w ∈ V
Suppose there exists a1 , a2 , · · · , an such that w = a1 v1 +a2 v2 +· · ·+an vn
Then w is a linear combination of v1 , v2 , · · · , vn .
now w ∈ span{v1 , v2 , · · · , vn }
So because every vector is in span{v1 , v2 , · · · , vn }, then
V = span{v1 , v2 , · · · , vn }
Proposition 63. Let V be a linear space over F , S = {v1 , v2 , · · · , vn } be the set of vectors, then the
following are equivalent
(1) v1 , v2 , · · · , vn are linearly independent
(2) For w ∈ V that can be linearly expressed by v1 , v2 , · · · , vn , in equation w = a1 v1 + a2 v2 + · · · +
an vn , the choice of scalars is unique
Proof.
(1) → (2)
Because v1 , v2 , · · · , vn are linearly independent
Then a1 v1 + a2 v2 + · · · + an vn = 0 will forces a1 = a2 = · · · = an = 0
Now suppose we have two kinds of expression
w = b1 v1 + b2 v2 + · · · + bn vn = c1 v1 + c2 v2 + · · · cn vn
look at the last equation and then move terms leftwards, we get
(b1 − c1 )v1 + (b2 − c2 )v2 + · · · + (bn − cn )vn = 0
because v1 , v2 , · · · vn are linearly independent
then it will forces the coefficient to be 0.
that is (b1 − c1 ) = · · · = (bn − cn ) = 0
that is b1 = c1 , b2 = c2 ,...bn = cn
So the expression is unique.
(2) → (1)
Suppose for any vector that linearly expressed by v1 , v2 , · · · , vn ,expression is unique.
obviously, 0 can be linearly expressed by v1 , v2 , · · · , vn coefficient is 0.
Because the uniqueness, the coefficent can only be 0
thus if a1 v1 + · · · + an vn = 0, then a1 = · · · = an = 0
This means that v1 , v2 , · · · , vn are linearly independent.
Now we both want uniqueness and existence. That would be great, now we define the best bunch of
vector that have both of them.
Definition 44. If V is a linear space over F , then the set {e1 , e2 , · · · , en } is called a basis of V if
(1) {e1 , e2 , · · · , en } is linearly independent.
(2) {e1 , e2 , · · · , en } span the whole space V
Proposition 64. Suppose {e1 , e2 , · · · , en } is a basis for V, then every vector w ∈ V can be uniquely
expressed as a linear combination of e1 , e2 , · · · , en
Proof. They span the space guarantees the existence of the linear combination. The linearly independence
guarantees the uniqueness.
78
Example 124. If V = {ax2 + bx + c|a, b, c ∈ R}. We said it is a linear space, then the vectors x2 ,x,1 could
be a basis for this linear space, firstly, any polynomial of degree less or equal than 2 is a linear combination
of these three vector over R, besides, if a linear combination of them gives the zero polynomial, then all of
the coefficient would be 0. Besides this standard basis, x2 − 2x + 1, x − 1, 1 could also be a basis. to see
why, we just check the property. it’s clear that for any polynomial, we can substitude x by (x − 1) + 1, and
then expand it in term of (x − 1), will give us the approprate linear combination of these vectors, besides, if
the combination of theses vector gives zero polynomial, then all the coeficcient would be 0. So it is indeed a
basis
1
0
0
Example 125. If V = M3×1 (R), then the 0 , 1 , 0 would be a basis, to see why, firstly,
0
0
1
x
any vector y could be written as a linear combination of these three
z
x
1
0
0
y = 0 ×x+ 1 ×y+ 0 ×z
z
0
0
1
besides, if there is a linear combination of them gives the zero vector, then all the coefficient should be 0,
because
x
1
0
0
0
y = 0 ×x+ 1 ×y+ 0 ×z = 0
z
0
0
1
0
1
0
1
Besides, we can also have a different basis. for example 1 , −1 , 0 could also be a
0
1
0
basis, how to show this is a basis? We left it to reader.
Now we have the concept of basis, let’s define the most important concept of this course – Dimension.
Proposition 65. V is a linear space, suppose S1 = {e1 , e2 , · · · , en } is a basis for V, and S2 = {1 , 2 , · · · , m }
is also a basis for V. Then the number of elements in this two sets are the same. This number is called the
dimention of V
We left this exercise to reader. But, don’t do that right now. Keep reading the next section.
3.2.1. Using Matrix to simplify notation. .
In this section, we emphasis the historical mistake: λv should really means v × λ.
As you can see, we didn’t use any matrix notation until now. and what the method we use to prove is
simply plus, times and equal. we always have to move a lot of terms from the right to the left, and factor
out the common factor. So far so good, because we only treat one equation. In the future, if we stick on this
kind of notation, then we will skew up. That’s why the notion of matrix can simplify all the work. we can
represent a summation without using ”+”. Look at the following:
Example 126.
a1 a2 · · · an
a1 b1 + a2 b2 + · · · + an bn =
b1
b2
..
.
bn
79
Example 127. Previously, we have the equation like
v1 × a1 + v2 × a2 + · · · + vn × an = v1 × b1 + v2 × b2 + · · · + vn × bn
and we conclude by move the right hand side to the left and then minus and then factor out vi to get
v1 × (a1 − b1 ) + v2 × (a2 − b2 ) + · · · + vn × (an − bn ) = 0
But this kind of equation can easily use matrix to represent.
Originally we have
v1 v2 · · · vn
a1
a2
..
.
=
v1 v2
an
b1
b2
· · · vn
···
bn
Simply move the right hand side to the left, and factor out the common row matrix, we get
v1 v2 · · · vn
(
b1
b2
) =
−
···
bn
an
a1
a2
..
.
0
0
..
.
0
And simply use matrix substraction, that is
v1 v2 · · · vn
a1 − b1
a2 − b2
..
.
an − bn
=
0
0
..
.
0
This is not even the worst case, if you stick on using the classical notion, then sometime you have to do
substitution. You substitute n variables by another n variables, and each variable all depend on the new n
variables. If you do that one by one, you would cry and never want to touch math again, now matrix comes
and tell you that it can helps, then the beauty of math comes again to intoxicate you. Matrix multiplication
simplifies substitution of variables And the associativity of matrix multiplication enable you to do the
work around.
Example 128. If
p = 3x + 2y + z
q = 2x + y
r = x + y + 2z
u = p + 2q + 4r
v = 5p + r
w = p+q+r
Please write the u, v, w in terms of x, y, z
If you try substitute and substitute again, it would be a good exercise for calculation toleration. Now lets
do it by matrix.
In the first equation ,we seen that the scalar act on the left of the varibles, we should keep the order.
80
p = 3x + 2y + z
q = 2x + y
r = x + y + 2z
p =
=⇒
q =
r =
3 2 1
2 1 0
1 1 2
x
y
z
x
y
z
x
y
z
p
=⇒ q =
r
3 2 1
2 1 0
1 1 2
x
y
z
x
y
z
x
y
z
The important skill is:
In the column matrix(matrix that only have 1 column), the right common factor (no matter what stuff)
could factor out to the right.
In the row matrix(matrix that only have 1 row), the left common factor(no matter what: same numbers,
same matrices, same hamburgers)could factor out to the left
p
q =
r
3 2 1
2 1 0
1 1 2
x
y
z
x
y
z
x
y
z
p
⇒ q =
r
Now let’s return back, the equation:
p = 3x + 2y + z
q = 2x + y
r = x + y + 2z
3 2 1 3 2 1
x
x
2 1 0
2 1 0 y
y
=
z
z
1 1 2
1 1 2
u = p + 2q + 4r
v = 5p + r
w = p+q+r
tells us nothing more than
p
3 2 1
x
u
1 2 4
p
q = 2 1 0 y v = 5 0 1 q
1 1 2
z
w
1 1 1
r
r
Then to get p, q, r in term of x, y, z, we only need to use the first one to substitute the second one, thus
u
1 2 4
p
1 2 4
3 2 1
x
10 8 9
x
v = 5 0 1 q = 5 0 1 2 1 0 y = 16 11 7 y
w
1 1 1
r
1 1 1
1 1 2
z
6 4 3
z
Thus we know
u = 10x + 8y + 9z
v = 16x + 11y + 7z
w = 6x + 4y + 3z
Now let’s use the language of matrix to redefine everything we did.
81
3.2.2. Matrix interpretation. Previously we put bunch of vectors in sets, while in the future study, you will
find the order of how arrange them is important. So now instead of putting them into a set, it is more
resonable to put them in a row matrix. So previously we always talk about
the linearly independence of
{v1 , v2 , · · · , vn }, now it is more resonable to become v1 v2 · · · vn
Definition 45. Suppose V is a linear space, The linear combination of v1 , v2 , · · · , vm by scalars a1 , a2 , · · · am
means
a1
a2
v1 v2 · · · vn .
..
an
The left is a 1 × n matrix over V, the right is a n × 1 matrix over F
Definition 46. Suppose w
∈ V ,v1
, v2 , · · · , vn ∈ V we say w is able to be linearly expressed by {v1 , v2 , · · · , vn }
x1
x2
if the equation for varible ..
.
xn
w=
v1 v2
x1
x2
..
.
· · · vn
xn
has solution.(Warning, this is not linear equation, because the left matrix is not a number matrix)
Definition 47. The subspace spaned by (v1 , v2 , · · · , vn ) is the collection of all the possible values of the
expression
t1
t2
v1 v2 · · · vn .
..
tn
when each ti ranges from the base field.
Definition
48. The (v1 , v2 , · · · , vn ) is said to be linearly dependent if the following equation for variable
x1
x2
.. has a non-zero solution:
.
xn
0=
v1 v2 · · · vn
x1
x2
..
.
xn
82
Definition
49. The (v1 , v2 , · · · , vn ) is said to be linearly independent if the following equation for variable
x1
x2
.. only has zero solution:
.
xn
0=
v1 v2 · · · vn
x1
x2
..
.
xn
Example 129. Suppose
w=
v1 v2 · · · vn
v1 v2
is linearly independent, and
a1
b1
a2
b2
· · · vn . = v1 v2 · · · vn .
..
..
an
bn
, show the cancellation rule:
a1
a2
..
.
=
an
b1
b2
..
.
bn
Proof.
v1 v2 · · · vn
a1
a2
..
.
=
v1 v2 · · · vn
an
b1
b2
..
.
bn
then
v1 v2 · · · vn
a1
a2
..
.
−
v1 v2 · · · vn
an
b1
b2
..
.
bn
That is
v1 v2 · · · vn
(
a1
a2
..
.
−
an
b1
b2
..
.
) = 0
bn
That is
v1 v2 · · · vn
(
a1 − b1
a2 − b2
..
.
an − bn
83
) = 0
=0
Because v1 v2 · · · vn
that is true is
is linearly independent, view the above as an equation. then the only case
a1 − b1
a2 − b2
..
.
=0
an − bn
, that is
a1
a2
..
.
=
an
b1
b2
..
.
bn
So arithmetically speaking, independent vectors of (right) linear space could cancel from the left. But
lineally depend vectors can’t
Definition 50. Let V be a linear space, then e1 e2 · · · en is called a basis if
(1) e1 e2 · · · en spans the whole space. that is, for any v ∈ V , there exists some scalar
a1 , · · · , an , such that
a1
a2
v = e1 e2 · · · en ..
.
an
(2)
e1 e2 · · · en
is linearly independent, that is, if
e1 e2 · · · en
a1
a2
..
.
= 0, then
an
a1
a2
..
.
=
an
Proposition 66. If e1 e2 · · · en
expressed in unique way as:
0
0
..
.
0
is a basis for vector space V, then every vector v ∈ V could be
v=
e1 e2 · · · en
a1
a2
..
.
an
Proof. The existence of this expression is guaranteed
by the first property of the definition of basis, To show
the uniqueness, remember e1 e2 · · · en is linearly independent, so by previous example we know
the scalars are unique.
84
Definition 51. Suppose V is a linear space,
v=
The column vector over the base field
e1
a1
a2
..
.
e1 e2 · · · en is a basis, then in the expression
a1
a2
e2 · · · en .
..
an
is called the coordinate of v with respect to basis
e1 e2 · · · en
an
Example 130. Let V = {x2 × a + x × b + c|a, b, c ∈ R} be the right vector space over R, (we multiply
scalar on the right because we want it to be a right vector space). Choose x2 x 1 to
a basis.
be
1
then any polynomial could expressed in this way, for example x2 + x + 1 = x2 x 1 1 Then
1
1
the coordinatte of x2 + x + 1 with respect to this basis is 1 . Now let’s change a nother basis, maybe
1
2
x + 1 x + 1 2 . Then we observe that
1
x2 + x + 1 = (x2 + 1) × 1 + (x + 1) × 1 + 2 × (− )
2
1
. Then x2 + x + 1 = x2 + 1 x + 1 2 1 . So the coordinate of this vector with respect to
− 12
1
another basis is 1
− 12
1
0
0
Example 131. Let V = M3×1 (R), now choose the basis 0 1 0 For keeping the
0
0
1
1
0
0
2
1
0 . Now the vector 4 is a linear combination
beauty of mathematics, we write it as 0
0
0
1
2
of basis, namely,
2
1
0
0
4 = 0 ×2+ 1 ×4+ 0 ×2
2
0
0
1
, then it means
1
0
0
2
2
4 = 0
1
0 4
2
0
0
1
2
2
1
0
0
2
1
0 is 4 . (Looks like matrix
, so the coordinates of 4 with respect to the basis 0
0
0
1
2
2
multiplication huh?)
85
1
0
0
1
0 we observe that
Now we choose another basis 1
0
1
1
2
1
0
0
4 = 1 ×2+ 1 ×2+ 0 ×0
1
1
2
0
This means
2
1
0
0
4 = 1
1
0
2
0
1
1
.
2
This means the coordinate of 4 with respect to the basis
2
2
2
0
1
1
0
0
1
1
0
2
0 is 2
1
0
We schetch a very important point of view. when basis is chosen, we can treat vectors as column
matrices.
Proposition 67. Suppose V is a linear space over F , v, w ∈ V , e1 e2 · · · en is a basis of V. let
a1
b1
a2
b2
v = e1 e2 · · · en .. ; w = e1 e2 · · · en ..
.
.
an
bn
Then:
v+w =
e1 e2 · · · en
a1 + b1
a2 + b2
..
.
an + bn
v−w =
e1 e2 · · · en
v×λ=
e1 e2 · · · en
a1 − b1
a2 − b2
..
.
an − bn
a1
a2
.. × λ
.
an
This means, although linear space is abstract object, when the base is chosen, we can treat them as if
a column vector by looking at its coordinates. Then the sum of vector is just sum of the coordinate, the
difference of two vector is difference of coordinate. The scalar multiple of a vector is a scalar multiple of
coordinate. So the most important vector space we should handel well is the vector space of column matrices
M n1F . If we can understand this vector space well, other vector space is all the same with this one if we
choose a base.
Previously with the basis chosen, every single vector have unique coordinate, and we are able to handle
it by just think it as a column matrix.
86
But how to know bunch of vectors are linearly independent? How to know whether they span the whole
space or not? How do we know two bunch of vectors actually would span the same space?
With basis chosen, every vector transferred to a coordinate, coordinate is like a name card of the vector, a
realization of abstract object by explicitly computable and understandable column matrix. When we want to
talk about the property of the vector, then it is enough to look at its name card - the column matrix. To study
the properties of bunch of vector, we should put them together, so in fact we should put their coordinate
together to form a matrix, and link the property of the bunch of vectors to the property of matrix. And use
this matrix to give the algorithm of finding these vectors are weather linearly independent, span the whole
space, or became a new basis.
Let’s begin with the strict definition of the coordinate matrix we want.
e
e
·
·
·
e
, then for every bunch of vector
Proposition 68. Suppose
V
is
a
linear
space,
with
basis
1
2
n
v1 v2 · · · vm , There exists a unique n × m matrix P, such that we can put them into the equation:
v1 v2 · · · vm
=
e1 e2 · · · en
P
and each column of P is the coordinate of each vi with respect to the basis e1 e2 · · · en
Proof. Because e1 e2 · · · en is a basis, then for each vi , where i ranges from 1 to m, there exists a
unique coordinate column matrix such that
p1i
p2i
vi = e1 e2 · · · en ..
.
pni
, then put all the column vectors together,Let
P =
···
···
..
.
p1m
p2m
..
.
pn1 pn2 · · ·
pnm
p11
p21
..
.
p12
p22
..
.
Then we have exactly
v1 v2 · · · vm
=
e1 e2 · · · en
P
The uniqueness of this matrix comes from the uniqueness of the coordinate with respect to basis. Each
column of this matrix is exactly the coordinate of corresponding vi .
Theorem 5. Suppose V is a linear space over F , if there exists a basis e1 e2 · · · en consists of
finitely many vectors, say n, Then Any basis of V will consists of n vectors.
Proof.
Suppose we have two bases of linear space,
namely, e1 e2 · · · en , 1 2 · · · m
because e1 e2 · · · en is a basis, there exists a unique matrix Pn×m such that
1 2 · · · m = e1 e2 · · · en P
because 1 2 · · · m is a baies, there exists a unique matrix Qm×n , such that
e1 e2 · · · en = 1 2 · · · m Q
substitute this equation to previous one we have
1 2 · · · m = 1 2 · · · m QP
87
because we also have the obvious identity
1 2 · · · m = 1 2 · · · m Im
Then because of basis 1 2 · · · m , the coordinate is unique, that is
QP = Im
Similarly, we will have
P Q = In
.
Using the identity
det(Im + QP ) = det(In + P Q)
we now know
2m = 2n
So m = n
Thus the element of e1 e2 · · · en , 1 2 · · · m are the same.
By the arbitrary choice of basis, we showed every basis consists of same number of vector.
Corollary 3. Suppose
V is a linear space over F , with basis e1 e2 · · · en , then the set of vectors
v1 v2 · · · vn is a basis if and only if in the expression
v1 v2 · · · vn = e1 e2 · · · en P
, P is invertible.
Proof.
Suppose
v1 v2 · · · vn is a basis
Then there would exists a unique square matrix Q, such that
e1 e2 · · · en = v1 v2 · · · vn Q
Then doing substitution,
e1 e2 · · · en
e1 e2 · · · en
=
We know P Q = In ,
so P is invertible.
On the other hand, Suppose P is invertible
If
v1 v2 · · · vn
a1
a2
..
.
=0
an
That means
e1 e2 · · · en
P
a1
a2
..
.
an
88
=0
PQ
That means
P
a1
a2
..
.
=
an
0
0
..
.
0
so
a1
a2
..
.
= P −1
an
0
0
..
.
=
0
0
0
..
.
0
So v1 v2 · · · vn is linearly independent.
For any vector w ∈ V there exists expression
c1
c2
w = e1 e2 · · · en ..
.
cn
doing substitution, we have
w=
v1 v2 · · · vn
P
c1
c2
..
.
cn
Thus
So any vector is a linear
combination of vi
v1 v2 · · · vn spans the whole space
v1 v2 · · · vn is a basis.
Now the previous work gives us the first link, basis linked to invertible matrices.
Example 132. Suppose V is a linear space over F with a basis e1 e2 e3 , and suppose we know
v1 = 3e1 + 2e2
v2 = 5e1 + e2 + e3
v3 = 2e1 + 6e3
Does v1 v2 v3 a basis?
Answer: Write this into matrix form:
3
5
2
v1 v2 v3 = e1 e2 e3 2 1 0
0 1 6
3 5 2
, then because e1 e2 e3 is a basis. Then it is the same with asking whether 2 1 0 is invert0 1 6
ible. By calculating its determinant
3 5 2 2 1 0 = −38 6= 0
0 1 6 89
We know that it is invertible, thus
v1 v2 v3
form a basis.
Definition 52. Let V be a linear space over F , then the number of vectors in a basis of V does not depend
on the choice of basis. this number is then called the dimension of V.
Example 133. In our daily life. Suppose origin is chosen. a point is 0-dimensional. an infinite line is 1dimensional over R, an infinite flat plane is 2-dimensional over R, with time terminate and regardless of the
curvature of our space, then the world we live is 3-dimensional over R, if you would like to add the freedom
on time. Then the world we live is 4-dimensional over R, but what is the relation between time and length?
roughly speaking, you could think as if
√
1s = 299792458 −1m.
Example 134. Let V = M2×2 (R), with usual additionand scalar
multiplication,
then V
is 4-dimensional
1 0
0 1
0 0
0 0
vector space over R, because it has a basis consisting of
,
,
, and
.
0 0
0 0
1 0
0 1
a b
a b
1
0
Now let’s consider it’s subset {
|
=
}, This is actually a subspace, why?
c d
c d
2
0
because if we have 2 such matrix in the subset, say suppose we know
a1 b1
1
0
a2 b2
1
0
=
=
c1 d1
2
0
c2 d2
2
0
Then any linear combination of this two matrix would give the same property:
a1 b1
a2 b2
1
(λ
+µ
)
c1 d1
c2 d2
2
= λ
=
0
0
a1 b1
c1 d1
0
0
+
1
2
+µ
a2 b2
c2 d2
1
2
0
=
0
Now how many dimension is this linear subspace? As we can see the equation
a b
1
0
=
c d
2
0
gave us 2 linear relation. With 2 restrictions, we expect the dimension is 4-2=2. Let’s check. By matrix
multiplication, we have the following relation:
a + 2b = 0
c + 2d = 0
So we have a = −2b, c = −2d, so we can represent the element in this set as
−2b b
−2d d
There are no restriction onb and d now.
are free now.
We −2 1
Then, clear it has basis
and
, so the subspace is 2-dimensional.
−2 1
90
Proposition 69. Suppose V is an n-dimensional space over F , and suppose v1 v2 · · · vn consists
of n vectors, where n equal to the dimension and linearly independent. Then v1 v2 · · · vn is a basis
of V
Proof.
Taking a basis of V, say e1 e2 · · · en
then there is a matrix P, such that v1 v2 · · · vn = e1 e2 · · · en P
independent
then rank(P ) = n
Now we claim because v1 v2 · · · vn are linearly
x1
x2
This is because e1 e2 · · · en P .. = 0 only has 0 solution.
.
xn
This means P
x1
x2
..
.
=
0
0
..
.
only has 0 sulution.
xn
0
This means there is no free variables,
because the number of free variables is equal to n − rank(P )
Then n − rank(P ) = 0
So rank(P ) = n
Thus P is invertible,
Then v1 v2 · · · vn become another basis.
Proposition 70. Suppose V is an n-dimensional space over F , then any n+1 vectors in V will be linearly
dependent.
Proof.
Suppose those n+1 vectors together is linearly independent.
Then any n of them is linearly independent.
Then any n of them forms a basis.
Then the left one can be linearly represent by those n vectors.
They are actually linearly dependent.
Contradiction.
Corollary 4. Suppose V is a linear space over F , W is a subspace of V , then
dim(W ) ≤ dim(V )
Proof. Choose a basis of W, then the vectors in that is also linearly independent in V, so the statement is
clear.
Example 135. Show that if A is an 2 × 2 matrix over R, then show that there exists a polynomial f (x) =
ax2 + bx + c with degree at most 2, such that f (A) = A2 + bA + cI is not invertible.
2
Proof. We consider the space V = M2×1 (R), and take a non-zero vector, say
, and consider the
1
following three vectors:
2
2
2
2
I
;A
;A
;
1
1
1
91
Then they are three vectors in the 2 dimensional space V. So they must be linearly dependent. So there exists
a non-trivial linear relation
2
2
2
0
2
aA
+ bA
+ cI
=
1
1
1
0
Then it is the same thing as
2
(aA + bA + cI)
That is
f (A)
2
So f (A) is not invertible, because if it is, then
1
2
1
2
1
=
=
=
0
0
0
0
[f (A)]−1
0
0
=
0
0
, this is an contradiction.
Example 136. Let Px2 = {ax2 + bx + c|a, b, c ∈ R} the space of all polynomials with degree at most 2.
Show that the following polynomials forms a basis.
u(x) = x(x − 1); v(x) = x(x − 2); w(x) = (x − 1)(x − 2)
Proof. As we know Px2 is 3-dimensional because we can choose the basis x2 x 1 , so in order to
show 3 vectors form a basis, it is enough to show they are linearly independent. To show this, we suppose
there is any linear relation, that is
ax(x − 1) + bx(x − 2) + c(x − 1)(x − 2) = 0
Then this means this equation is ture for any value of x. Now plug x=1 in this equation, we get
−b = 0
Now plug x=2 in this equation, we get
2a = 0
Now plug x=0 in this equation, we get
2c = 0
That means a = b = c = 0, so they are linearly independent, with the fact that the number of them is
euqal to the dimension, they form a basis.
Change of Basis
Suppose we have two bases in the linear space, what is the relationship of coordinate of the same vector?
We have studied in a vector
of the same number of vectors. Then suppose
space, any two bases consists
we have e1 e2 · · · en , and 1 2 · · · n as two basis, then there exists an unique invertible
matrix P, such that
1 2 · · · n = e1 e2 · · · en P
, this P is called the change of basis matrix
There are several ways to change basis, look at this example. We show three method to do it. Each of the
method is actually totally same, but with different habit of notation.
Example 137. (Classical)Suppose V is a linear space with basis e1 e2 e3 , and suppose we have
three vectors
e1
1 =
2 = e1 + e2
3 = e2 + e3
If v = e1 + 2e2 + 4e3 , what is the coordinate of v in the basis 1 2 3 ?
92
Answer: We are trying to compose a linear combination. suppose the coefficient is x, y, z, then we list the
equation:
xe1 + y(e1 + e2 ) + z(e2 + e3 ) = e1 + 2e2 + 4e3
distribute that and combine terms with same ei , we get
(x + y)e1 + (y + z)e2 + ze3 = e1 + 2e2 + 4e3
By compare the coefficient in front of e1 , e2 , e3 respectively, we have
x+y = 1
y+z = 2
z
= 4
Solve this equation, we get x = 3, y = −2, z = 4
Now we turn to the second method, because our basis is chosen and fixed, let’s treat them exactly like
column vectors.
1
1
1
0
2
Example 138. (Simplified)What is the coordinate of
in the basis { 0 , 1 , 1 }?
4
0
0
1
Answer: let these three column vector to combine by scalar x, y, z, now we list the equation
1
1
0
1
0 ×x+ 1 ×y+ 1 ×z = 2
0
0
1
4
This is the same as asking
x+y
1
y+z = 2
z
4
1
1
2
2
, then we solve x = 3, y = −2, z = 4. So the coordinate of
in the new basis is
4
4
Now let’s see the third method.
Example 139. (Strict)Suppose
V is a linear space with basis
1 1 0
e1 e2 e3 0 1 1 . If v ∈ V has coordinate
0 0 1
coordinate of v with basis 1 2 3 ?
Answer: Suppose
x
1
3 x2
x3
1 2
v=
e1 e2 e3 , another basis 1 2 3 =
1
2 with basis e1 e2 e3 , what is the
4
, then because
1 1 0
0 1 1
0 0 1
1 2 3
=
e1 e2 e3
So we have
v=
e1 e2 e3
1 1 0
x1
0 1 1 x2
0 0 1
x3
93
On the other hand
v=
e1 e2 e3
1
2
4
So
1
1 1 0
x1
0 1 1 x2 = 2
4
0 0 1
x3
Then
−1
x1
1 1 0
1
x2 = 0 1 1 2
x3
0 0 1
4
1 −1 1
1
= 0 1 −1 2
0
1
4
0
3
= −2
4
Thus we know the coordinate of the vector v in the basis
1 2 3
3
is −2
4
The Algorithm for linearly independent of vectors. .
Now we figure out an algorithm to determine a bunch of vector v1 v2 · · · vm is linearly independent or not. Now with our basis chosen, it is the same as asking how to determine
a bunch of column matrix
is linearly independent or not. Keep in mind in the notion v1 v2 · · · vm = e1 e2 · · · en P ,
each column of P is exactly the coordinate of each vi , so our intuition tells us that the linear independence
of vi is exactly linear independence of each column of P.
Proposition 71. Now let V be a n-dimensional linear space over F , e1 e2 · · · en is a fixed
basis.
and v1 v2 · · · vm is bunch of vectors, and v1 v2 · · · vm = e1 e2 · · · en P then
the following are equivalent
(1) v1 v2 · · · vm is linearly independent
(2) Each column of P is linearly independent
Proof. The proof is super clear, because a linear relation of v1 v2 · · · vm is the same as linear
relation of the column of P. what I mean is
0
a1
a1
a1
a2 0
a2
a
2
v1 v2 · · · vm . = 0 ⇔ e1 e2 · · · en P . = 0 ⇔ P . = .
.. ..
..
..
am
am
am
0
So their linear independence is equivalent.
Example 140. Suppose V is a linear space with basis e1 e2 e3 . And suppose
v1 = 2e1 + 3e2
v2 = e 1 + e 2
v3 = e 2
94
Then clear
v1 v2 v3
has a linear relation:
v1 − 2v2 − v3 = 0
.
Now with the basis chosen, every vector has coordinate,
2
3
e1 e2 e3
v1 =
0
1
e1 e2 e3 1
v2 =
0
0
1
e
e
e
v
=
1
2
3
3
0
As you see the same linear relation also holds for its coordinates:
2
1
0
0
3 − 1 ×2− 1 = 0
0
0
0
0
So when base is chosen, the coordinate represents the vector. Now the question comes to how to determine
whether a bunch of column matrix is linearly independent or not.
Proposition 72. The following are equivalent.
(1) The columns of matrix
P is linearly
independent.
x1
0
x2 0
(2) The linear equation P .. = .. only have 0 solution.
. .
xn
0
(3) The rank of P is the number of its columns
Ir
(4) The reduced row echelon form for P is
0
Proof. The proof is very easy, if you don’t know how to prove, please send me an email.
We use this proposition, together with what we learned before to show several examples.
Example 141. Suppose V is a linear space with e1 e2 e3 as basis, and
v1 = 2e1 + 3e2
v2 = e 1 + e 2
Show that v1 v2 is linearly independent.
Proof: Writting the coordinate of v1 v2 , we get
2
1
v1 v2 = e 1 e 2 e 3 3 1
0 0
First Method:
2 1
The rank of 3 1 is 2, equal to its columns, so
0 0
Second Method:
95
v1 v2
is linearly independent.
2 1 x
0
1
We find that The linear equation 3 1
=
only has zero solution. that means
x2
0
0 0
x1
2 1
x
1
e1 e2 e3 3 1
only has
= 0 only has zero solution. Because v1 v2
x2
x2
0 0
zero solution, then by definition, v1 v2 is linearly independent.
Third Method:
2 1
1 0
We reduced 3 1 to the reduced row echelon form to get 0 1 , then the upper block is exactly
0 0
0 0
an identity matrix, thus each columns are linearly independent.
Get rid of redundant vectors
Suppose we have bunch of vector v1 v2 · · · vm , they span a subspace W , and they are not
necessarily linearly independent. Can you choose the minimal set of vi such that they still span W but
linearly independent?
In order to do it, we need to transfer our linear space to which we familiar with. Look at following:
a1
a1
a2
a2
v1 v2 · · · vm . = 0 ⇐⇒
e1 e2 · · · en P . = 0
..
..
am
am
⇐⇒ P
a1
a2
..
.
am
=
0
0
..
.
0
The first equivalence is get by substitution, the second equavalence is get by linearly independence of
basis. This formal computation showed us that a linear relation of v1 v2 · · · vm is also a linear
relation of columns of P, which is coordinates of v1 v2 · · · vm
Thus, the study of linear independence of vectors is equiavalent to study the linear independence of their
coordinates.
But even for coordinates, it’s also hard for us now to compute their linear independence in mind, look at
the following example.
Example 142. Let linear space V = M5×1 (R). Consider the following column matrices:
10
0
5
1
0 1 0 0
2 2 0 0
3 1 1 2
4
2
5
1
Are they linearly independent? If not, can you get rid of some of them such that they become linearly
independent?
As you can see, this takes a little bit time to discuss. But at least we know we shouldn’t get rid of the
second one. Because the second coordinate of second vector is 1 while others is 0. So getting rid of second
one would result in a subspace that the second coordinate is 0. But that subspace is definitely smaller than
96
the span of these four vectors, so the second vector is not redundant, taking it away will make the subspace
collapse.
With the above discussion, you may find it is easier to know which vector is VIP if they have a non-zero
coordinate entry while other vectors are 0 in the corresponding position. This might tell us to find a way to
simplify general vectors to the vectors we want with their linear relation keeping.
Proposition 73. row transformation does not change the linear relation of column matrices
We have sevral ways to understand this fact.
Matrix interpretation(Strict Language):
Put these column matrices in a matrix, a linear relation of these column matrices is simply a
a1
a2
..
.
,
an
such that
P
a1
a2
..
.
=
an
0
0
..
.
0
Doing row transformation is nothing more than left multiply an invertible matrix Q, if we do that we have
a1
0
0
a2
0 0
QP .. = Q .. = ..
.
. .
an
0
0
So a linear relation of columns of P , is a linear relation of columns of QP , a linear relation of QP , is a
linear relation of Q−1 QP = P .
Directly understanding(More on sense)
If we have a linear relation for each columns, that means
c11
cm1
0
c12
cm2
0
.. × a1 + · · · + .. × am = ..
.
.
.
cn
cm n
0
And these is simply n equations for numbers at each corresponding entry.
c1i a1 + · · · + cmi am = 0
, clear, row transformation is simply the equation adding, and each step is invertible. Then it does not change
the linear relation.
No matter which way you choose to understand this fact, now remember we can reduce a matrix to the
reduced row echelon form, and now the question is simply asking which column of reduced echelon form
should take away.
We copy the definition of reduced row echelon form here to review:
Definition 53. A m × n matrix is called a Row Echelon Form if it saitiesfying
(1) The first non-zero entry for every row vector is 1, we call this leading 1(or pivot)
(2) If a row contains a leading 1, then each row below it contains a leading 1 further to the right
(3) If a column contains a leading 1, then all other entries in that column are 0
97
Remember the concept of reduced row echelon form
Example 143. The following are reduced row echelon form
(1)
1
1
1
1
1
3
(2)
1
6
1
1
6
1
(3)
1
2
3
1
1
2
1
1
Can you figure out which vector in the column of reduced row echelon form is redundant? Definitely
easy. For each column of reduced echelon form, if this column does not contain leading 1, then this column
is a linear combination of those leading 1. Every columns without leading 1 is redundant vector.
Now we left with columns with leading 1, are they linearly independent now? Definitely is, because each
leading 1 of the column vector does not appear in the same location. So each column have a nonzero entry
while other column has 0 in the corresponding position. In this way getting rid of any vector would result
the collapse of space. And therefore they must be linearly independent.
Now we summarize the method of getting rid of redundant vectors.
The method of getting rid of redundant vectors
Suppose we have a bunch of vectors v1 v2 · · · vm in the space V
Step 1: Choose a basis of the space V: e1 e2 · · · en
Step 2: Write down the coordinates of each vector v1 v2 · · · vm = e1 e2 · · · en P
Step 3: Transform the coordinate matrix P to reduced row echelon form.
Step 4: Then the columns of leading corresponding the useful vector, others are redundant.
Example 144. Suppose there are 3 banks: A bank, B bank, C bank respectively. And there are 3 product.
If you buy the first product, 10% of your money would go to bank A, 90% of your money would go to bank
B. If you buy the second product, 50% of your money would go to bank B, 50% of your money would go to
bank C. If you buy the third product, 5% of your money would go to bank A, 70% of your money would go to
bank B, and 25% of your money would go to bank C. As a manager, you want to invest your money to these
3 product. But actually only consider 2 of them is enough, why?
The most simple reason should be the product C seems a linear combination of product B and product
A, suppose buying A and B together is the same as buying 2C, why do we waste time consider C? The less
factor we consider, the easier life would be.
Now let’s do that by rigorous process.
Step 1: Choose a basis: Let
e1 = Dollar that willing to go to Bank A
e2 = Dollar that willing to go to Bank B
e3 = Dollar that willing to go to Bank C
This is the basis of the vector space of distribution of dollars. Note that each vector here is dollar itself, not
the variable represents amount of dollar. The difference of the dollar e1 , e2 and e3 is their dream bank that
98
they are willing to go. (Don’t keep this kind of dollar with you, they might jump out of your pocket and run
to Bank A)
Step 2: Write down the coordinate of bunch of vectors.
Now suppose v1 = Product 1, v2 = Product 2, v3 = Product 3 So we have
v1 = 0.1e1 + 0.9e2
v2 =
0.5e2 + 0.5e3
v3 = 0.05e1 + 0.7e2 + 0.25e3
We put coordinates into matrix.
v1 v2 v3
=
e1 e2 e3
Step 3: Now doing row transformation of the matrix.
99
0.1
0.05
0.9 0.5 0.7
0.5 0.25
0.1
0.05
0.9 0.5 0.7
0.5 0.25
r3 −0.5×r1
−−
−−−−→
0.1
0.9
0.5 0.5
0.25 0.25
0.1 0.9
r3 −0.5×r2
0.5 0.5
−−
−−−−→
r1 −1.8×r2
−−
−−−−→
10×r
1
−−−−→
1
2×r
2
−−−→
0.1
1
−0.9
0.5 0.5
−9
0.5 0.5
−9
1
1
Because the leading 1 is at 1st and 2nd column, then the third column is redundant, correspondingly, v3
is redundant.
Step 4: Conclusion.
So v3 is redundant, after getting rid of v3 , then v1 , v2 would be linearly independent.
That’s why we only have to consider the product A and product B.
Example 145. Suppose V = M5×1 (R), what is the dimension of the subspace W ⊂ V spanned by
1
1
3
3 1 7
2 , 6 , 10
0 1 1
2
9
13
3
1
1
3 1 7
? Simplify span{
2 , 6 , 10 }
0 1 1
13
9
2
In this example, we already have natural basis
1
0
0 1
0 , 0 ,
0 0
0
0
0
0
1
0
0
100
,
0
0
0
1
0
,
0
0
0
0
1
, so do this directly, put them into matrix.
1
1
3
3
1
7
2
6
10 =
0
1
1
2
9
13
1
0
0
0
0
0
1
0
0
0
The reduced row echelon form of this matrix is
1
0
0
1
0
0
1
0
0
0
1
0
0
0
0
0
1
1
3
2
0
2
1
1
6
1
9
3
7
10
1
13
2
1
Remark : Now it not only tell us the first and second one can already span the subspace, it also tells us
that the third one is twice of the first combine with the second.
Now we know that the first two vector are linearly independent, and they span the whole subspace. So it
is an basis, so the subspace is 2-dimensional.
And,
1
1
3
1
1
3 1 7
3 1
span{
2 , 6 , 10 } = span{ 2 , 6 }
0 1 1
0 1
2
9
13
2
9
101
3.3. Linear Maps and Linear Transformation. .
Introduction:Now we are heading to the most important part of linear algebra. Previously we’ve talked
the notational meaning of the matrix- to strict the language we use. But I should say the geometrical
meaning of the matrix is more important and more intuitive and should be the soul of linear algebra. You
can use a matrix to represent a linear transformation, and the matrix product is just the composition of
the two transformation. Sometimes linear transformation are losing data, then we use kernel to measure
in what extent did we lose the information. We also should know which value is possible to get by linear
transformation, so we talk about the image to be all possible values that can get by the linear transformation.
Now we start.
Geometrically speaking, linear transformation is the transformation that keeping the parallel lines.
Now that the transformation is keeping the parallel lines, can I completely understand every point maps
to where?
Of course, because parallelogram is keeping, so the parallelogram law is still true for the vector that maps
to., if we know where two vector goes, we will predict out where their sum goes. And so is for the scalar
multiples.
Practically speaking, linear transformation is a process of encoding data that preserves linear combination
(What do I mean by that? I’ll show an example.)
Example 146. In Game room, we have two kinds of game coin, namely, gold game coin and silver game
coin. Every gold one worth 2 dollars, and every silver one worth 1 dollar.
Then the price of the game coins is a linear transformation. How to understand that?
Think the word ”The price of” as an action on the coins that you are going to buy, like
102
The price of (2 gold + 1 silver) = 5 dollars
The price of (5 gold + 2 silver) = 12 dollars
Now, the game coins are putted into a bag. Suppose you don’t know how many coins and what types the
coins are in the bag. But your friend know and only tells you the price. He said the coins in red bag worth 6
dollars, and the coins in dark bag worth 5 dollars.
Problem 6. What is the price of the coins if you put the coins in two bags together?
In our general sense, we don’t have to pull all coins out and calculate price one by one again, what we
only have to do is sum up 5 and 6, and the total would be 11 dollars.
Problem 7. What is the price of the coins of 3 black bag and 2 red bag?
You also don’t need to pull them out and add one by one, simply do 5 dollars × 3 + 6 dollars × 2 =
27 dollars
Without knowing coins, simply do the combination we can know the information which is the same as we
do if we pulling out all the conins and sum up. This kind of encoded data is the typicall example of linear
map, and this example is exactly what I mean by preserving linear combination.
Now let’s see an example of non-linear map
Example 147. The diamonds are embed on the ring, and the price of the ring is propotional to the squate
of the number of diamond on the ring, for example, we have the following price
Ring with 1 diamond $100
Ring with 2 diamonds $400
Ring with 3 diamonds $900
Now suppose I have 2 rings, the left one has 2 diamond, worth $400 and the right one also has 2 diamond,
worth $400. What is the price of the ring if I dig out the all the diamond on this two ring and embed on a
new ring?
Obviously, the price is not just only 400 + 400 = 800, instead, the price goes up to 1600, this map, the
price of the ring, is not linear to the diamond.
With this understanding of linear maps, Let’s define what is linear map.
Definition 54. Suppose V1 , V2 is two linear space over F , A map T : V1 −→ V2 is called a linear map if
for any vector v, w ∈ V1 , and any scalar λ, we have
(1) T (v + w) = T v + T w
(2) T (λv) = λT v
We call the space V1 Domain Space, and space V2 the Target Space. In the case that Domain space and
Target Space are the same, we often call it Linear Transformation
Now return back to the case
Example 148. As we previously discussed bags of coins, that is not actually a linear space. Since the
number of bags are integers, instead of decimals. Also the number of bags does not allow negative numbers,
but in the concept of linear space we allow negative number. But still, if we can think decimal bags and
negative bags, then taking its price would be a linear map. because the price of the sum of two bags is the
sum of the price, and the price of scalar multiple of bag is scalar multiple of price.
This is the example where domain space and target space are not the same. So it is a linear map.
Now look at the most popular linear transformation: Rotation.
103
Example 149. Suppose we already choose an orthogonal axies(But we didn’t define orthogonal yet). And
we select our basis to be 1 unit
arrow on x-axis and y-axis(Standard Orthogonal Basis, which we also didn’t
define), denote as e1 e2 respectively, and consider rotation by the angle θ
This is the example where domain and target are the same, so
it isan linear transformation.
x
, What will this linear transformation
Every element in this plane can be expressed as e1 e2
y
do for this vector?
To understand the behavior of the arbitrary vector, we test this linear transformation on the basis first,
using the trigonometric function we have:
Then we can represents our work into following equations:
T e1 = cos θe1 + sin θe2
T e2 = − sin θe1 + cos θe2
It is clear that rotation preserve the parralel lines, so rotation is actually a linear transformation. With
the property of transformation ,we are able to calculate where any point goes.
For example, let’s calculate where the vector e1 + e2 goes.
T (e1 + e2 )
= T e1 + T e2
= (cos θe1 + sin θe2 ) + (− sin θe1 + cos θe2 )
= (cos θ − sin θ)e1 + (sin θ + cos θ)e2
This is not the only method, remember we can represent things into matrix. To put our work into matrix,
we have:
104
T
e1 e2
T e1 T e2
=
=
1
1
e1 e2
cos θ − sin θ
sin θ cos θ
So we calculate:
T (e1 + e2 )
= T
e1 e2
=
e1 e2
=
e1 e2
cos θ − sin θ
sin θ cos θ
cos θ − sin θ
cos θ + sin θ
1
1
As you can see again, matrix put everything together with neat work. And it seems linear transformation
is actually like matrix multiplication. If the basis is chosen, we regard vectors as column matrices, then the
linear transformation in this world is like a matrix. Let’s define now the matrix representation of T.
Definition 55. Suppose T : V −→ W is a linear map, and e1 e2 · · · en is basis for V, 1 2 · · · m
is basis for W. Then there exists a unique matrix P, with size n × m, such that
T
e1 e2 · · · en
=
1 2 · · · m
P
Then this matrix P is called the matrix representation
of T with respect to basis
Domain Space and basis 1 2 · · · m of Target Space
e1 e2 · · · en
of
Now let’s find matrices with respect to the bases. We introduce several examples including some practical
example.
Example 150. There are 3 brothers. Namely, A,B,C. They have some money originally. Suppose there is
neither income and costs. And they like to share their money. At the end of every month, each of them will
split his money into two equal part and give it to other two. For example. Suppose A has 300 dollars at first
month. At second month, A with give 150 to B, and give 150 to C.
(1) Suppose in a month, A has $1000, B has $ 500, C has $ 200, what is the money of them next month.
Answer: Of course we can do it directly, As A will receive half of B and half of C, so A would
have 350. B will receive half of A and half of C, so B would have 600, C would receive half of A and
half of B, so C would have 750.
But, we want to use higher understanding to this process. If we view all possible status of the
money each one has as a linear space. OK, then it is 3 dimensional. And at the end of every month,
the status changes, and this change, is linear. So it is the same as saying every month we do an
linear transformation of current status, and the out put is the status next month.
Let V be the linear space of the current account status of A,B,C. We choose a basis for V
e1 = a dollar that belongs to A
e2 = a dollar that belongs to B
e3 = a dollar that belongs to C
And let T be the operator that changes the status to the next month. What is T e1 , T e2 , T e3
respectively?
105
Because A will split every dollar of him and give it to B and C. So a dollar belongs to A now
become two quarter belongs to B and two quarter belongs to C. That is
1
1
T e1 = e2 + e3
2
2
And we write everything down, that is
T e1 =
T e2 =
T e3 =
1
2 e1
1
2 e1
1
2 e2
+ 12 e3
+ 12 e3
1
+ 2 e2
Now put our work into matrix, that is
1
2
e1 e2 e3
T
=
e1 e2 e3
1
2
1
2
1
2
1
2
1
2
Because A has 1000, B has 500, C has 200. This status corresponding to vectors
1000
e1 e2 e3 500
200
And plug in the matrix representation form of T. the linear transformation act on this would result
matrix multiplication on the coordinates:
T
e1 e2 e3
1000
500 =
200
e1 e2 e3
1
2
1
2
1
2
1
2
1
2
1
2
1000
500
200
e1 e2
=
350
e3 600
750
= 350e1 + 600e2 + 750e3
That means the status of next month. 350 dollars belongs to A, 600 dollars belongs to B, and 750
dollars belongs to C.
1
1
In this example, the matrix
basis.
1
2
1
2
2
1
2
2
1
2
is the matrix representation of T with respect to the
Example 151. Suppose V = M2×2 (R) is the linear space of all 2 × 2 matrices, then we define a map by
1
1
T (A) =
A−A
2
2
(1) Is this map a linear transformation?
106
Answer: Yes, it is, let A, B be two matrices, and λ, µ be two scalars, we found that
T (λA + µB) =
1
2
=
1
= λ
1
1
= λ(
2
− µB
B − µB
2
1
2
B−B
2
1
2
1
1
) + µ(
1
1
2
+µ
2
A−A
2
µB − λA
1
A − λA
2
2
1
2
1
λA +
2
(λA + µB) − (λA + µB)
1
2
)
= λT (A) + µT (B)
T
So, indeed, this is a linear map.
If you are proficient with matrix notation, you can also check by matrices. But remember in matrix
notation, it is better to represent scalar multiplication by λI2 where the subindex is the number of
columns if it multiplied on the right. and rows if it is multiplied on the left.
Now let’s show the neater work:
µI2
µI2
µI2
1
1
A B
A B
=
− A B
λI2
2
λI2
λI2
2
=
1
A
2
1
B
2
µI2
λI2
µI2
λI2
µI2
λI2
−
µI2
2
1
λI2
2
A B
=
1
A
2
1
B
2
=
1
A
2
1
B
2
µI
2 2
A B
1
λI2
2
−
1
1
µI2
λI2
µI2
λI2
2
A B
−
1
1
2
=
=
A
2
=
1
2
A−A
T (A) T (B)
1
2
µI2
λI2
107
B
2
1
1
µI2
λI2
1
2
−
A
2
B−B
1
B
1
2
1
2
µI2
λI2
Whenever the operator T can pass though, it is an linear map. Because it defined in the same
space, it is an linear transformation.
(2) Find the matrix representation of this linear map with respect to the basis
1
1
1
1
−1
1
We proceed by apply T on each of the vectors in the basis, and then represent the result vector by
this basis again.
1
T(
1
) =
1
T(
1
−1
= 0
1
1
) =
1
2
=
2
1
−
1
+0
+0
1
1
2
1
−
1
1
+ (−1)
−1
1
+0
1
1
2
1
= 0
1
1
−1
1
+0
1
−
+0
1
1
−1
2
1
= 0
+0
= (−1)
T(
1
+0
−1
1
1
2
−1
1
1
) =
1
1
1
+0
0 0
0 0
=
1
2
T(
1
) =
1
1
−
1
1
= 0
1
2
0 0
0 0
=
1
1
+0
1
−1
108
+0
1
+1
1
So we know that
T
1
1
1
−1
1
1
=
1
1
1
−1
1
1
0
0
0
0
0
0
0
0
0
0
−1
0
0
0
0
1
The right matrix is the matrix representation for linear transformation T.
As you can see, the expression on the right hand side make sense. Because scalar right act on
matrix as multiplication.
So indeed, this matrix is the matrix representation of T.
This might falls you into the worry. Did we create a work that can put 2 × 8 matrix and 4 × 4
matrix multiply together ?!
No, we didn’t, As you can see, because now our object that the scalars act on is matrix. So the
left matrix is actually 1 × 4 instead of 2 × 8. So 1 × 4 matrix times 4 × 4 matrix make sense.
Now we stop by here.if you still, maybe, would like to put into block matrix work, and you still
think the left hand side as a 2 × 8 matrix. Now, remember the scalar multiplication is actually scalar
matrix multiplication, which I mean is:
a b
c d
is actually
λ −−−−−→
a b
c d
λ
λ
So if you want your notation work as block matrix multiplication, you should really goes like:
1
1
1
1
−1
1
0
0
0
0
0
0
0
0
0
0 0
0 0
0 0
0
0
0
0
0
0
0
0
0
0
0 0
0 0
0 0
0
0
0
0
0
−1
0
0
0
0
0 0
0 0
−1
0
0
0
0
0
0
0
0
1
0
0
0
0
0
0
0
0
1
Now it is 2 × 8 multiply 8 × 8, it make sense now.
But, in this kind of problems, we view matrices as object, so we still think the left matrix as 1 × 4,
and right matrix as 4 × 4. And the matrix representation of the linear transformation is of course
4 × 4 We haven’t create any work that can put 2 × 8 and 4 × 4 matrix multiply together.
Example 152. Suppose V = M2×1 (R), W = M3×1 (R), And suppose we have a linear transformation
T : V −→ W defined by
1
1
T
= 1
0
0
0
0
T
= 1
1
1
109
1
0
Find the matrix representation of T with respect to the basis
in domain space and
0
1
1
0
0
basis 0 1 0 in target space
0
0
1
Answer: We proceed like following: we test our linear transformation on each of the vectors of basis:
1
1
0
0
1
1
0
1
0 ×0
T
=
=
×1+
×1+
0
0
0
0
1
0
1
0
0
0
T
= 1 = 0 ×0+ 1 ×1+ 0 ×1
1
1
0
0
1
.
Then we put our work into matrix, we get
1 0
1
0
0
1
0
1
0 1 1
= 0
T
0
1
0 1
0
0
1
1 0
So the matrix representation of T with respect to that basis is 1 1 .
0 1
a
Now let’s look what is T
? Because in the natural basis, the coordinate of column matrix is just
b
this matrix itself. In this way, we have
a
T
b
= T
1
0
=
0
1
0
0
1
0
1
0
a
b
0
1 0 a
0 1 1
b
1
0 1
1 0 a
= 1 1
b
0 1
So we see in this example, the linear map that took place from column matrices to column matrices is just
realized by matrix multiplication. And this matrix is obtained by represent the linear map with respect to
those basis.
We have already told that with a basis, we can identify a vector with a column vector. With this identification, the linear relation, linear independence, and linear dependence are all the same.
So what is the linear transformation like if we using this identification. As you see in the previous
problem. linear map for column vectors is just multiplying the matrix. So we wish our abstract linear map
could also be realized as matrix multiplication on the coordinates.
110
Proposition 74. Suppose
V, W is two linear spaces over F , with bases e1 e2 · · · en of V and
1 2 · · · m of W. Suppose T : V −→ W is a linear map. And let P be the matrix representing
this linear transformation with this two bases.
T e1 e2 · · · en = 1 2 · · · m P
Then for any vector with coordinate expression
e1 e2 · · · en
v=
a1
a2
..
.
an
Then the coordinate form of T v is
1 2 · · · m
Tv =
P
a1
a2
..
.
an
Proof. Apply T to the second equation and use first equation to substitute we get the third equation.
Although this is an trivial statement, but this tells us if we only look at coordinate, then what the linear
map did is just multiply it by a matrix P.
Example 153. Suppose V = Px2 = {ax2 + bx + c|a, b, c ∈ R} is the linear space of polynomial with real
coefficient with degree at most 2. T : V −→ R is a linear transformation defined by evaluating by plug in
x=2
(1) with the basis x2 x 1 in V, and (1) in R, write down the matrix that representing the linear
transformation.
Answer: We test it on each of the basis:
T (x2 ) = 1
T (x) = 1
T (1) = 1
So we have
T
x2 x 1
= (1)
1 1 1
1 1 1
1
(2) Suppose v ∈ V has coordinate 3 with respect to basis
5
of T v with respect to basis (1)?
Answer: Just multiply the matrix
1
1 1 1 3 =9
5
So the matrix representation
is
So the coordinate of T v with respect to basis (1) is 9.
111
x2 x 1 . what is the coordinate
Example 154. Previously we have seen with respect to the basis e1 e2 , our rotation has representation
cos θ − sin θ
sin θ cos θ
3
, what is the coordinate of T v?
. If v = e1 e2
4
Answer: Just multiply the matrix on it, the coordinate of T v with respect to the basis e1 e2 is going
to be
cos θ − sin θ
3
sin θ cos θ
4
So it is
3 cos θ − 4 sin θ
3 sin θ + 4 cos θ
In other words,
3 cos θ − 4 sin θ
T v = e1 e2
3 sin θ + 4 cos θ
To conclude, there is an identification of two world with bases, the left hand side is abstract linear space,
and linear transformation among them. The right hand side is the coordinate and matrix multiplication. We
conclude our result into the following form.
For Spaces, with chosen bases we have following:
vectors
↔ Coordinates
bunch of vectors
↔ Coordinate Matrix P
bunch of vectors form a basis
↔ Coordinate Matrix P is invertible
bunch of vectors are linearly independent ↔ rank(P) = number of columns of P
bunch of vectors spans the whole space
↔ rank(P) = number of rows of P
Linear Transformation
↔ Matrix multiplication
I’ll say one more word of the last row of the form. The matrix is given by represent linear transformation
into matrix by those basis.
3.4. Attempting to go back - Isomorphism, Kernel and Image. Now we know the concept of linear
transformation, but typically the most interesting question is, what is the inverse?
Example 155. Let’s keep previous problem of 3 brothers. Remember at the end of each month they split
their money and give it to other two.
Question: Suppose A,B,C have $350, $600, $750 respectively this month, how much do they have previous
month?
We can also figure out this by elementary method, it is the money of A should originally given by B and C,
it represent half of the money of B and C. so B,C together has $ 700 in total last month. And in the process
of distributing, the total amount of money does not change, and the total amount is $1700, So that means A
has $1000 last month. and we can figure out by the same process that B has $ 600, C has $ 750.
Now let’s do linear algebra. previously we know that
1
1
2
2
1
T e1 e2 e3 = e1 e2 e3 1
2
1
2
112
1
2
2
Now suppose T −1 is the inverse change, then T −1 T , is just go back and go forward, so would change
nothing. We apply(left multiply) T −1 on both side, we have
1
1
2
2
1
e1 e2 e3 = T −1 e1 e2 e3 1
2
1
2
and we’d like to move matrix to the other side, right multiply by
e1 e2 e3
1
2
1
2
1
2
1
2
1
2
1
2
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
−1
, we have
−1
= T −1
e1 e2 e3
That means the matrix representation of linear transformation T −1 is the inverse of the matrix. Let’s
calculate this matrix out.
1
2
1
2
1
2
1
2
1
2
1
2
−1
−1 1
1
= 1 −1 1
1
1 −1
That means
T −1
e1 e2 e3
=
e1 e2 e3
Now that the status now is
−1 1
1
1 −1 1
1
1 −1
e1 e2 e3
350
600
750
, so apply T −1 on that will result matrix multiplication on the coordinates. so previous month, the status
would be
−1
1
1
350
1000
e1 e2 e3 1 −1 1 600 = e1 e2 e3 500
1
1 −1
750
200
So in the previous month, A has 1000 dollars, B has 500 dollars, C has 200 dollars.
Now the question comes, can any linear transformation goes back? Think about the question that we
talked about previously, the coins of game room. a gold coin worth $ 2, and silver worth $ 1. And now they
are packed in bags and you know the price of conins in the bag.
Now see the linear map case first. Suppose we map X to Y, now we want to go back from Y to X.
Problem 8. Suppose you know the total price of coins in red bag is $ 7, do you know how many gold how
many silver are there?
This is a go back question. We want to know the original status of gold and silver, solely on its total price.
You will see the answer is not unique. You could have the following combinations:
3 gold coin + 1 silver coin
2 gold coin + 3 silver coin
7dollars =
1 gold coin + 5 silver coin
0 gold coin + 7 silver coin
113
If you know some of the answer, how do you know each of the asnwer? As You can see, each time you
subtract 1 gold coin, you supplement 2 silver coin. So we can decribe this process by you add 2 silver- 1
gold coin. So this ammount
2 silver - 1 gold
does not change the total price, but change the vector in your domain. Thus, we see the combination such
that the price is 0 is very important. It measures the difference betweeen your prediction and the real fact.
This is the first factor prevent you from predicting the original value, the possible combination that result to
0. Now let’s define the kernel of a linear map
Definition 56. Suppose V W are linear spaces over F , and T : V −→ W is a linear map, Then the kernel
of T is defined to be the set
ker(T ) = {v ∈ V |T v = 0}
Then it is a linear subspace in the domain.
As we can see one reason for us to failed to going back is the kernel. When the kernel is not zero space,
we can not tell exactly which vector is the departure.
Sometimes the case is even worse, if no vector maps to here, it is even impossible for going back. In the
previous question, the question is like
Problem 9. For coins in the bag. What kind of price can be result?
We know every price could be result if we treat the amount of these coins could be decimal or negative
numbers.
Now let’s define the image of a linear map. Only if you are cosidering the vector in image could you goes
back.
Definition 57. Suppose V W are linear spaces over F , and T : V −→ W is a linear map, Then the image
of T is defined to be the set
Im(T ) = {w ∈ W |w = T v for some v}
Then it is a linear subspace in the domain.
The kernel is the subspace of domain definede by the restriction T v = 0, Then the method to find the
kernel is by getting rid of the restriction by parametrization, and then write every vector in the kernel as the
linear combination by these parameters and then we get a basis of the subspace.
The image is the subspace of target defined by any possible vectors that can mapped from domain, thus,
because every vector is a linear combination of the basis in the domain, so every vector should mapped to
the linear combination of the image of the basis. This tells us the image is the subspace spanned by the
image of basis. To find the image of linear map, we simply get rid of redundant vectors.
Example 156. Suppose V1 , V2are linear spaces over R, and T : V1 −→ V2 is a linear map. e1 e2 e3
is a basis for V, and 1 2 is basis for V2 . Suppose we know
T e1 = 1 + 32
T e2 = 21 + 2
T e3 = 1 − 52
(1) Find the matrix representation of T with respect to the basis e1 e2 e3 and 1 2 .
Answer:
1 2 1
T e1 e2 e3 = 1 2
3 1 −5
114
(2) Find the kernel of T
Answer:
e1
The kernel of T is asking for v =
T
e1 e2
x
e2 e3 y such that
z
x
e3 y = 0
z
Plug in the matrix representation, that is, asking for
x
1 2 1
y =
1 2
3 1 −5
z
1 2
0
0
Now we are ready to solve x,y,z. Note that we can cancel linear independent vectors on both side.
So we have
x
1 2 1
y = 0
3 1 −5
0
z
Reduce to reduced row echelon form we have
x
x
1
2
1
0
1
− 11
5
y
y
=P
=
8
3 1 −5
0
1
5
z
z
we get the solution:
x − 11
5 z = 0
y + 85 z = 0
Now assign free variable z by parameter 5t, we have
x
11t
11
y = −8t = −8 t
z
5t
5
Thus any v ∈ ker(T ) have the form
v =
e1 e2
x
e3 y
z
=
e1 e2 e3
11
−8 t
5
= t(11e1 − 8e2 + 5e3 )
So and element in the kernel is linear combination of 11e1 − 8e2 + 5e3 . Thus,
ker(T ) = span{11e1 − 8e2 + 5e3 }
115
(3) Find the image of T
1 2
The image is asking for all the possible values for w =
a
b
That can get by the
form
w = Tv = T
e1 e2
x
e3 y
z
= xT e1 + yT e2 + zT e3
Thus, Im(T ) = span{T e1 , T e2 , T e3 }.
But now we have to get rid of redundantvectors.
1 2 1
As T e1 T e2 T e3 = 1 2
3 1 −5
Then we look at the coordinate matrix and do row transformation P to reduce to reduced row
echelon form.
1
− 11
5
T e1 T e2 T e3 = 1 2 P
8
1
5
As you can see, in the reduced row echelon form, The piviot columns are the first and second
column, then We get rid of T e3 , thus
span{T e1 , T e2 , T e3 } = span{T e1 , T e2 }
= span{1 + 32 , 21 + 2 }
Final answer:
Im(T ) = span{1 + 32 , 21 + 2 }
We illustrate an important formula in linear algebra.
Theorem 10. Suppose T : V −→ W is a linear map, then the following formula is true:
dim(ker(T )) + dim(im(T )) = dim(V )
Please note that in the right hand side of the equation it is dimension of domain
Proof. Suppose V has basis e1 e2 · · · en and W has basis 1 2 · · · m , then by finding
its matrix representation form
T e1 e2 · · · en = 1 2 · · · m A
, where A is an m × n matrix.
As we know because the linear relation of T e1 T e2 · · · T en is the same
as linear relations of
the columns of A. By getting rid of redundant vectors of T e1 T e2 · · · T en , We found a basis in the
image. That means after getting rid of redundant vectors of A, the number of linearly independent columns
of A is dim(imT ), they are corresponding to number of leading 1.
116
Now when we are trying to find the kernel, the last step is always find solution of Ax = 0, while in
there, the number of elementary solution, is the number of free variables, and elementary solution is linearly
independent, left times our basis will gives the basis in the kernel. So the number of free variables is
dim(kerT )
Remember that the number of free variables correspond to those columns that does not have leading 1.
So, sum them together, we would get dim(ImT )+dim(KerT ) equal to the number of columns of A, which
is equal to n, dimenstion of the domain.
The above way shows us the method of finding image. As you know our motivation for kernel and image
is trying to go back. To conclude, we define the two type of good map
Suppose T : V −→ W is a linear map.
Definition 58.
(1) T is called an injective if for any v1 6= v2 ∈ V , we have T v1 6= T v2 .
(2) T is called an surjective if for any w ∈ W , we can find some v ∈ V , such that w = T v
To see in another way, for injective, the preimage of every vector is always unique, but not necessarily
exists. For surjective, the preimage of every vector is always exists, but not necessarily unique. And for
bijective, the preimage exists and unique. Thus, only bijective is invertible. To use our language of kernel
and image, we conclude in the following preoposition
Proposition 75. Suppose T : V −→ W is a linear map, then
(1) T is injective if and only if Ker(T ) = {0}
(2) T is surjective if and only if Im(T ) = W
(3) T is bijective if and only if Ker(T ) = {0} and Im(T ) = W .
Some more explaination about injective preserves linearly independence, and surjective preserves
the span.
Proposition 76. Suppose T : V −→ W is a linear map, then
(1) If T is injective, then dim(V ) ≤ dim(W )
(2) T is surjective then dim(V ) ≥ dim(W )
(3) T is bijective then dim(V ) = dim(W )
Definition 59. Two linear space V,W is called isomorphic if there exists a linear map between V,W such that
this linear map is isomorphic.
Proposition 77. Two linear space is isomorphic if and only if their dimension are equal.
more explaination goes here
3.4.1. Composition, Inverse and Restriction. .
Composition
If we think linear map T : V −→ W , and R : W −→ U as two machine, for T, it accepts input from
vectors of V, and its output is the vector in W, for R, the input is vectors in W, and the output is an vector in
U. now, If we combine this two machine to be one machine, simply concatenate the output pipe of T with
the input pipe of R together, and it will become a macine that receive input from V, and gives output in U.
To see the whole process, we give some vector v as input for the first machine, it would give some result w
in the space W, but now w is immediately becomes the input of R, and then R act on it and the final out put
will become an vector of U.
We see some set theoretic explaination of composition
Nice picture goes here
Proposition 78. The matrix appeared in the matrix representation of the composition of two linear map is
the multiplication of these two matrices.
117
More explaination goes here
The inverse of linear map
The invertible linear map, that can only happens between the space with same dimension. And thus, the
matrix appeared in the matrix representation is a square matrix.
Proposition 79. Suppose T : V −→ W is an isomorphism, and V has basis e1 e2 · · · en , and W
has basis 1 2 · · · n , Suppose
T
e1 e2 · · · en
=
1 2 · · · n
P
T −1
1 2 · · · n
=
e1 e2 · · · en
P −1
Then
example of plug in x=t+1 and t=x-1 as example of inverse map. and find their matrices and then
say that this two matrix is invertible each other.
The restriction of linear map
Here goes the picture example of restrictino on sets
Definition 60. Suppose T : V −→ W is a linear map, and U is a subspace of V , then T could be defined
on U by the same way. Denoted as
T |U : U −→ W
, we call this construction of linear map by restrct T on subspace U in the domain
For a linear map, restriction on the subspace in the domain is for free. You can restrict linear map on
any subspace you like. Simply ignore others and don’t care where others go, you only keep track of your
element.
A example of calculation for the restriction goes here
Whenever we have a linear map, we always consider two things. What is the kerenel? what is the image.
The kernel of T |U is the element in U that killed by T |U , because T |U maps by the same law as T , so
killed by T |U is the same as saying killed by T . Thus we know all element in U killed by T is simply
ker(T ) ∩ U .
Proposition 80.
ker(T |U ) = ker(T ) ∩ U
So how about image of T |U ? we can’t give the precise relation. because we only know the image of T |U
should be a subset of T , because we didn’t care about others, our members might not fill all the possible
positions of Im(T ), so we know
Proposition 81.
Im(T |U ) ⊂ Im(T )
3.5. Invariant subspace for linear transformation. Power-Rank. .
The difference between map and transformation When we talk about linear map, we treat two spaces
separately, so that is why we could discuss restriction in domain or target. And you see the restriction in the
domain comes for free, and the restriction in the target needs some discussion, and if you want to restrict
yourself simultaneously in the subset of domain and subset of target, you should first restrict domain, and
then check whether your target works.
118
Now we discuss linear transformation. Because we assume that we are using the same space, so whenever
we choose a subspace in the domain, this subspace will also be a subspace in the target. And if we only
restrict in the domain or target, then it is not linear transformation again, instead, becomes a linear map. So,
in order to make restriction of linear transformation again a linear transformation. We need to restrict us
simutaneously in the same subspace in the domain and target. And by the above discussion, we have the
following definition of invariant subspace.
Before define what is invariant subspace, let’s just look at set for example ,for which kind of subset can
we restrict our self on? Clear, we need that transformation not going to map things out from our subset.
Look at the following picture.
We call the subset that we could restrict our transformation as invariant subset. Does each of those would
be the set that we could restrict our transformation on?
119
In the first picture, yes it is. Every element in the blue circle does not go out, although they all collapse
to the center, but whenever it doesn’t blow out, we call it invariant. So invariance only means not going out,
collapse is also call invariance.
In the second picture, yes, it is. Everyone in the blue circle stays, some might gradually collapse to some
point, there are also some guy playing in a circle, they don’t collapse, just rotating and rotating all the way.
This is an invariant subspace.
In the third picture, it is not invariant subset, because some guy that originally in the blue circle. After
one time transformation, it goes out.
To sum up, invariant subset means the subset you chose is able to restrict all the element in there no matter
how many times you will apply transformations. In this case, we can restrict our transform on that subset,
otherwise, we can’t restrict.
For our linear transformation, the idea are the same.
Definition 61. Let T : V −→ V to be a linear transformation on V. Then the subspace W ⊂ V is called an
invariant subspace of T, if
T (W ) ⊂ W
Definition 62. Let T : V −→ V be an linear transformation, W is an invariant subspace, Then as a
linear map, we can restrict T on W both on domain, and target. We can do that because W ⊂ T (W ).
After this restriction, it becomes a linear transformation on subset. This is called the restriction of linear
transformation on the invariant subspace W
Remark 5. Please distinguish the restriction of linear map and linear transformation, As a linear map, we
discuss domain and target separately, we can choose a set in domain, and another set in target. Restrict
linear map is still a linear map. But in linear transformation case, the target and domain are identified
together, thus, choosing a subset in the domain is the same as choosing a subset in the target. So linear
tranformation can only restrict on invariant subspace to make sure it is still a linear transformation. In the
case that you just want to restrict on domain, you should say, forget about the linear transformation and
treat it as a linear map, and if you do that, the result is just a linear map.
Proposition 82. Suppose T : V −→ V is a linear transformation, then ker T and ImT are invariant
subspaces.
Proof. Easy
Proposition 83. Suppose T : V −→ V , R : V −→ V are two linear maps on the same space, saitisfiying
T R = RT , then ker R and ImR are invariant subspace for T
Proof. Easy, Just check the definition of kernel and image, show that any element in the kernel of R mapped
to the kernel of R. And every element in the image of R mapped to the image of R
examples of invariant subspaces and restriction of linear transformation on it
Example 157. Suppose V is 3-dimensional space over Q, with basis e1 e2 e3 , T : V −→ V is a
linear transformation, such that
0
−1
−2
3
2
T e1 e2 e3 = e1 e2 e3 4
−1 0
2
Determine whether the subspace W = span e3 − 2e2 e3 − e1 is an invariant subspace, if it is, find
the matrix representation of T |W with respect to this basis.
Answer: Denote 1 2 = e3 − 2e2 e3 − e1 , To show W is an invariant subspace, we only
need T (1 ), and T (2 ) in W.
120
T (1 ) = T (e3 − 2e2 )
= (−2e1 + 2e2 + 2e3 ) − 2(3e2 − e1 )
= −4e2 + 2e3
T (2 ) = T (e3 − e1 )
−1
= T e1 e2 e3 0
1
0
−1 −2
−1
e1 e2 e3 4
3
2 0
=
−1 0
2
1
e1 e2 e3
=
−2
−2
3
= −2e1 − 2e2 + 3e3
e3 − 2e2 e3 − e1 ,
Now in order to show −4e2 +2e3 and −2e1 −2e2 +3e3 are in subspace spanned by
we make use of reduced row echelon form.
1 2 T 1 T 2
=
e1 e2 e3
0
−2
1
−1
0
1
0
−4
2
−2
−2
3
By applying the reduced process, we found
0
−2
1
−1
0
1
0
−4
2
−2
1
−2 = P
3
2
1
1
2
for some invertible P.
And this exactly means T 1 = 21 , and T 2 = 1 + 22 , So W is indeed an invariantsubspace.
Thus, for T |W , the domain and target for T |W are both W, and with basis 1 2 , the matrix representation form for linear transformation T is
T |W
1 2
=
1 2
2 1
2
Image of high power
Before discussing this part, let’s see some example in set theory.
Example in set theory
Now we consider what will happen for the high power of the linear transformation.
Let’s still use set theory picture to consider, see the following picture.
121
If we keep track of all element and act by T onece. The whole set would collapse down to a smaller
subset, the image. And if we act twice, this subset may become more smaller. If we act several times, how
small would it collapse? Is there any way to describe this process?
Now let’s think why the subset collapse under the transformation of set. It is because someone in the
center is absorbing points. Look at the following picture:
As you can see, points in the ker T goes to the black hole after 1 step. points in the ker T 2 goes to ker T,
and then goes to black hole, it took them two steps. Points in ker T 3 takes 3 steps to collapse. And those
absorbed by the kernel is exactly the reason of collapse of the image of subset.
Now we want to measure its collapsion. We put these two graph together for you to compare:
122
The total number of element in whole set is 19. After 1 time of transformation, there are 12 element, after
2 time of transformation, there are only 6 element. Look at 19 −→ 12 −→ 6. First time we lost 7 element,
second time we lost 6 element, it seems that the element we lose is decreasing. How to measure the element
we lose?
As you can see, the element we lose is reflect in ker T. because he is absorbing points. From the whole
set to the Im T, the number of points we lose is exactly the points that absorbed by kernel. and from Im T
to Im T 2 , the number of the point we lose is the number of points in ker T ∩ Im T . So ker T ∩ Im T k
measures the point we lose from Im T k to Im T k+1
Actually this is just an example for you to understand better. In set theory, we can’t say what is kernel,
because there is nothing like zero-vector in set. With this picture in mind, let’s now discuss the case in linear
transformation.
Rank-power in linear transformation
Now suppose T : V −→ V is a linear transformation, how can we say something about the dimension of
their image and kernel of its high power?
Proposition 84. Suppose T : V −→ V is a linear transformation, then we have the chain of following
invariant subspaces.
{0} ⊂ ker T ⊂ ker T 2 ⊂ ker T 3 ⊂ · · ·
V ⊃ Im T ⊃ Im T 2 ⊃ Im T 3 ⊃ · · ·
Proof. If x ∈ ker T n , then T n (x) = 0, then of course T n+1 (x) = T 0 = 0, so x ∈ ker T n+1 , thus
ker T n ⊂ ker T n+1 .
For y ∈ Im T n+1 , there exists x such that y = T n+1 x = T n (T x), so y ∈ Im T n
In order to keep track of the map, we consider the following sequence in the space:
T
T |Im
T|
2
T|
3
Im T
Im T
V −−−−−−−−→ Im T −−−−→ Im T 2 −−−
−−→ Im T 3 −−−
−−→ Im T 4 →
− ···
In each step, we might lose some dimension, and the dimension we lost measured by kernel. The dimension formula tells us
T
Dimension lost = dim(Domain) − dim(Image)
= dim(Ker)
= Dimension of element in domain killed by transformation
123
Example 158. Which subspace measures the dimension lost from Im T 3 to Im T 5
Answer: From Im T 3 to Im T 5 , the linear transformation is T 2 , the dimension lost is measured by the
space inside of Im T 3 that killed by T 2 , so this space is
Im(T 3 ) ∩ ker(T 2 )
So
dim(Im T 3 ) − dim(Im T 5 ) = dim(Im T 3 ∩ ker T 2 )
To say this strictly, consider the linear transformation:
T 2 |Im T 3 : Im T 3 −→ Im T 3
The kernel of this transformation is ker T 2 ∩ Im T 3 , the image of this transformation is Im T 5 , and the
domain is Im T 3 , so by applying the dimension formula we have
dim(ker T 2 |Im T 3 ) + dim(Im T 2 |Im T 3 ) = dim(Im T 3 )
This is equivalent to
dim(ker T 2 ∩ Im T 3 ) + dim(Im T 5 ) = dim(Im T 3 )
Thus
dim(ker T 2 ∩ Im T 3 ) = dim(Im T 3 ) − dim(Im T 5 )
Now let’s go back to the sequence:
T
T |Im
T|
2
T|
3
Im T
Im T
V −−−−−−−−→ Im T −−−−→ Im T 2 −−−
−−→ Im T 3 −−−
−−→ Im T 4 →
− ···
We understand for each step, the dimension lost is the dimension of
T
ker T ∩ ImT k
But we have a subspace chain
ker T ⊃ ker T ∩ Im T ⊃ ker T ∩ Im T 2 ⊃ ker T ∩ Im T 3 ⊃ · · ·
And thus
dim(ker T ) ≥ dim(ker T ∩ Im T ) ≥ dim(ker T ∩ Im T 2 ) ≥ dim(ker T ∩ Im T 3 ) ≥ · · ·
With the understanding that it measures the lost of dimension, we have
dim(ImT 0 ) − dim(ImT 1 ) ≥ dim(ImT 1 ) − dim(ImT 2 ) ≥ dim(ImT 2 ) − dim(ImT 3 ) ≥ · · ·
Proposition 85. The function f (n) = dim(ImT n ) satiesfies the following property:
(1) f never increase, that is
f (n + 1) ≤ f (n)
(2) The decreasement
∆f (n) = f (n) − f (n + 1)
never increase. that is
f (n + 1) − f (n + 2) ≤ f (n) − f (n + 1)
Suppose we have a basis e1 e2 · · · en , then we can
correspond a linear transformation to a matrix, namely T e1 e2 · · · en = e1 e2 · · · en A. And the dimension of image is simply the
rank. So we transfer dim(ImT n ) to the rank(An ). And conversely, when we have an n × n matrix, we can
define a linear transformation of Mn×1 (F), simply left multiply it on elements of Mn×1 (F). Thus, we also
have the same property for rank of the matrix
124
Corollary 5. For arbitrary square matrix A rank(An ) satiesfies the following property:
(1) rank(An ) never increase, that is
rank(An+1 ) ≤ rank(An )
(2) The decreasement of rank(An ) never increase. that is
rank(An+1 ) − rank(An+2 ) ≤ rank(An ) − rank(An+1 )
In the whole process, as the incresement become smaller and smaller, finally it will goes to 0. and after
that, because it can not grow or decrease, the rank will become a stable number as the power goes very large.
Thus we define the stable number of the power rank.
Definition 63. For a linear transformation T (or a matrix A), the rank stable number of T is defined to be
the minimal number n, such that T n = T n+1 (resp. rank(An ) = rank(An+1 )).
Definition 64. if N is the power-rank stable number of T, then we call dim(Im(T N )) (resp, rank(AN ))as
the stable rank. and we call dim(ker(T N )) as the stable nullity.
Proposition 86. The rank stable number is less than the stable nullity.
Proof. Because the stable nullity measures the total amount of decrease needed to come to the stable point,
while each step before stable point one decrease at least 1. So less than the stable nullity times, it would
already decrease to the stable point.
Example 159. Suppose V is 10-dimensional space, T : V −→ V is a linear transformation, Suppose Im T
is 8- dimensional, Im T 3 is 5-dimensional. And T 10 = 0
(1) Find the dimension of subspace Im T k for all integer k.
(2) Find the dimension of ker T 3 ∩ Im T 5
(3) What is the rank stable number?
Answer: First let’s see the dimension of Im T 2 , it could be 8,7,6,5. Of couse it could not be 8 or
5 because if it is, then it will become stable all the way, forces impossible for T 10 = 0.
If it is 7, it also impossible, because in that case, Im T to Im T 2 dimension drops 1. That forces
Im T 2 to Im T 3 the dimension drops at most 1. but Im T 3 is 5-dimensional. 7−1 > 5. So the only
case is Im T 2 is 6-dimensional. And then, because Im T 3 is 5-dimensional, from here each step
the dimension losing is at most 1. As we know T 1 0 = 0, that means it will keep losing dimension
until it become 0. so Im T 4 is 4-dimensional, Im T 5 is 3-dimensional, Im T 6 is 2-dimensional,
Im T 7 is 1-dimensional, Im T 8 = 0 is 0-dimensional.
For (2) we know dim(ker T 3 ∩ Im T 5 ) = dim(Im T 5 ) − dim(Im T 8 ) = 3 − 0 = 3. So this
space is 3-dimensional.
For (3) The rank stable number is 8 because 8 is the first time for the rank to be stable.
power-rank curve
Now let’s using graphics to help us understand better, because we know rank(An ) as a function it is nonincrease and non-accelertes, so it is concave down, The power-rank curve ssaitisfies the following properties:
Proposition 87.
(1) The slope of the curve is never possitive
(2) All the curve lies above any link of two adjacent points.
(3) This curve only pass though integer points.
The third condition is most strong one. It is not like the curve that we general studied, because in this
world, we only have integer, so this curve is actually like segment. To give you an idea, lets look at the
previous example.
125
Example 160. Suppose V is 10-dimensional space, T : V −→ V is a linear transformation, Suppose Im T
is 8- dimensional, Im T 3 is 5-dimensional. And T 10 = 0, Try to draw the power-rank curve
We set up the points that we already have here:
Now you see, Now you know the curve, from 0 to 3, drops 5 unit. So dropping 5 unit in 3 steps. So what
is the first drop.
Is that possible to drop by 1 at first?
It’s impossible, because it is non-accelerate, so dropping by 1 would result either 1 every step or 0. So
We will still
even we could try our best:
miss the point at (3,5)
So the first step can only drop at least 2.
Is that possible to drop by 3 at first?
If drop 3 at first, we would reach 6 at 1:
126
But now your position is embarrasing. How to get that 5 at 3? you have two step far and you want to
drop totally by 1? remember the curve can only pass though integer points, so it has no choice: The only
way for it to get 5 at 3 is to:
And if he do that, he will stable all the way. So he will definitely miss that 0 at 10.
127
Now drop by 3 even hard, no wonder it is impossible to drop by 4 at first place.
So the only case is at first it drops by 2, and it drops by 2, every follow steps could choose 2 or 1 or 0.
Now it reach 7 at 1. then it wants to touch 5 at 3, the only step for it to be non-stable is keep dropping
and each step is 1. so
128
Now we saw that it could only drop by 1 or 0 afterwards, and dropping by 0 means it stable, at clear,
stable at any rank other than 0 would costs contradiction because it would miss that 0 at 10. So in can only
stable at rank 0. This exactly means it will keep dropping by 1 each step until it touches the bottom:
So this is the only case that it could be, and it tells us everything. The stable index is 8. and the stable-rank
is 0 and stable nullity is 9.
dim(V ) = 9
dim(Im(T )) = 7
dim(Im(T 2 )) = 6
dim(Im(T 3 )) = 5
dim(Im(T 4 )) = 4
dim(Im(T 5 )) = 3
dim(Im(T 6 )) = 2
dim(Im(T 7 )) = 1
dim(Im(T 8 )) = 0
129
4. E IGENVALUES AND E IGENVECTORS
4.1. Finding Eigenvalues and Eigenvectors. .
Remember what we talked about last time.
Problem 11. There are 3 brothers. Namely, A,B,C. They have some money originally. Suppose there is
neither income and costs. And they like to share their money. At the end of every month, each of them will
split his money into two equal part and give it to other two.
Suppose this month, A has $8192, B has $4096, C has $8192. What amount will they have after 1 year?
This is the most classical problem, A process of the system. We can keep track of it several times, but if
we asked to keep track a lot of time, it would be a pain.
Let’s start with the most simple basis:
e1 = A dollar that belongs to A
e2 = A dollar that belongs to B
e3 = A dollar that belongs to C
Then the linear transformation would have a very ugly look:
T e1 e2 e3 = e1 e2 e3 12
1
2
1
2
1
2
1
2
1
2
However, if we choose another basis, like
1 = e1 + e2 + e3
2 =
e1 − e2
3 =
e1 − e3
Then 1 , 2 , 3 together form a basis. And the behavior of linear transformation is also nice:
1 −→ 1 −→ 1 −→ 1 −→ 1 −→ · · · −→ 1
1
1
1
1
1
2
2 −→ − 2 −→ 2 −→ − 2 −→ 2 −→ · · · −→
2
4
8
16
4096
1
1
1
1
1
3 −→ − 3 −→ 3 −→ − 3 −→ 3 −→ · · · −→
3
2
4
8
16
4096
This means in the basis 1 2 3 , the matrix representation of T 1 2 has a very simple form, namely
1
1
T 12 1 2 3 = 1 2 3
4096
1
4096
By replacing the change of basis relation:
1 2 3
=
e1 e2
1
1
1
e3 1 −1 0
1 0 −1
We would have
T 12
e1 e2 e3
1 1
1
1 −1 0 =
1 0 −1
1
1 1
1
1 −1 0
1 0 −1
e1 e2 e3
130
1
4096
1
4096
1 1
1
Simply move the matrix 1 −1 0 to the right, we have
1 0 −1
1
1 1
1
12
e1 e2 e3
1 −1 0
e1 e2 e3
=
T
1 0 −1
−1
1 1
1
1 −1 0
1 0 −1
1
4096
1
4096
e1 e2
=
1
1
1
1
1
e3 3
1 −1 0
1 0 −1
1
4096
1
4096
1 1
1
1 −2 1
1 1 −2
You can finally calculate it out and find the answer.
In the above exercise, We saw that the key step to simplify the linear transformation is to find some vector
v such that
Tv = v × λ
. And we expect, that these vector would able to give us a basis.
Definition 65. Suppose T : V −→ V is a linear transformation. A non-zero vector v is called an eigenvector
if and only if there is some λ, such that
Tv = v × λ
And in this case, λ is called the eigenvalue of T.
We directly do that, suppose our space has a basis e1 e2 · · · en , Then we want to find the vector
v such that T v = v × λ. With the basis e1 e2 · · · en , we assume
a1
a2
v = e1 e2 · · · en ..
.
an
, and suppose
T
e1 e2 · · · en
=
e1 e2 · · · en
A
, where A is a matrix. we proceed like follows:
Tv = v × λ
means
T
e1 e2
· · · en
a1
a2
..
.
=
e1 e2
· · · en
an
a1
a2
..
.
λ
an
Then
e1 e2 · · · en
A
a1
a2
..
.
=
e1 e2 · · · en
a1 λ
a2 λ
..
.
an λ
an
That is
131
e1 e2 · · · en
a1
a2
..
.
A
=
e1 e2 · · · en
an
λa1
λa2
..
.
λan
We preceed on,
e1 e2 · · · en
A
a1
a2
..
.
=
e1 e2
an
λ
λ
· · · en
λ
λ
..
.
a1
a2
..
.
an
λ
Cancel the basis on both side, we get
A
a1
a2
..
.
λ
λ
=
an
λ
λ
..
.
a1
a2
..
.
an
λ
We use neater symbol
A
a1
a2
..
.
= λI
a1
a2
..
.
an
an
Move the left hand side to right hand side and finally we get.
a1
0
a2 0
(λI − A) .. = ..
. .
an
0
Now we know that
v is non-zero vector ⇔
a1
a2
..
.
an
6=
0
0
..
.
⇔ λI − A is not invetible ⇔ det(λI − A) = 0
0
Proposition 88. λ is an eigenvalue of the linear transformation T if and only if det(λI − A) = 0, where A
is the matrix appeared in the matrix representation of A by some basis. We call the polynomial det(λI − A)
as the characteristic polynomial of T.
Now as you can see, we define some polynomial of T by its matrix representation. But for the same T it
might have different matrix to represent it. Does this definition don’t contradict?
132
Proposition 89. If A, B two matrices that represents the same lienar transformation T on some bases. Then
there exists some P, such that
B = P −1 AP
Proof. Suppose T e1 e2 · · · en = e1 e2 · · · en A, and T 1 2 · · · n = 1 2 · · · n B,
and 1 2 · · · n = e1 e2 · · · en P , then substitude and use uniqueness of coordinates, we
will get
B = P −1 AP
Definition 66. The matrix A and B are called similar, if there exists an invertible matrix P, such that
B = P −1 AP
Now let’s discuss does det(λI − A) and det(λI − B) are the same? We proceed as follows.
det(λI − B) = det(λI − P −1 AP )
= det(P −1 λIP − P −1 AP )
= det(P −1 (λI − A)P )
= det(P −1 ) det(λI − A) det P
= (det P )−1 det P det(λI − A)
= det(λI − A)
Proposition 90. The eigen polynomial for similar matrices are the same. Thus, similarity does not change
eigenvalues and its multiplicity as a root.
Proof. This is what we already showed prevously, we showed that det(λI − A) = det(λI − B), if A and
B are similar matrices.
So no matter which basis you chose, the characteristic polynomial does not change. Now let’s do some
examples on how to find and calculate eigenvalues and eigenvectors.
Example 161. Let V be two dimensional vector space over C, with basis e1 e2 , and suppose we have
a linear transformation T switching two vectors in the basis, that is
T e1 = e2
T e2 = e1
, Find the eigenvalue and eigenvector for T.
Our intuition feels like if they are switching things, then we combine this two thing by the way e1 + e2 ,
and e1 − e2 , then they will just keep it or just multiply it by -1. Let’s find them by eigenvalue process.
Write down the matrix form, we get
0 1
T e1 e2 = e1 e2
1 0
0 1
And here we set A =
1 0
So it’s characteristic polynomial is
133
λ −1 = λ2 − 1
det(λI − A) = −1 λ Now we find two roots of this characteristic polynomial, they are λ = 1 and λ = −1
Then for these two value, each value would have least one eigen vector, because if the coefficient matrix
is not invertible then the homogeneous linear equation will have non-zero solution.
If λ = 1, then we solve the equation
1 −1
x
0
=
−1 1
y
0
.
1
We found the elementary solution for this equation is
1
If λ = −1, then we solve the equation
−1 −1
x
0
=
−1 −1
y
0
1
We found the elementary solution for this equation is
−1
So This linear
transformation
has
two
eigen
values
and
and an eigenvector corresponds to 1 is 1 =
1
e1 e2
, and the eigenvector corresponds to -1 is 2 = e1 e2 . And fortunately, we can
1
check that they form a basis for the linear space, and thus this linear transformation would able to have a
very simple representation.
1 0
T 1 2 = 1 2
0 −1
Example 162. Now let’s come back to our oringinal problem, the three boys distributing their money problem. As you can see, by the basis we chose, we simplify the problem to a linear transformation
1
1
2
2
1
T e1 e2 e3 = e1 e2 e3 12
2
1
2
1
2
Now let’s finding eigenvalues and eigen vectors for this linear transformation, we have the eigen polynomial.
λ −1 −1 2
2 − 1 λ − 1 = λ3 − 3 λ − 1 = (λ − 1)(λ − 1 )2
2
2 4
4
2
−1 −1 λ 2
2
Thus, this linear transformation has two different eigenvalues, namely, 1, and 21 . Now let’s find the
eigenvectors one by one.
For 1, we solving the equation
1 − 12 − 12
0
x
−1 1 −1 y = 0
2
2
z
0
− 21 − 12 1
By reduced row echelon form, we see this has only one elementary solution, which is
x
1
y = 1
z
1
134
Now let’s continue to solve for eigenvalue 21
By solving the equation
1
− 2 − 12 − 12
x
0
−1 −1 −1 y = 0
2
2
2
z
0
− 21 − 12 − 12
We get two elementary solutions. which is
x
1
y = −1
z
0
x
1
y = 0
z
−1
Then we know the eigenvector for 1 is
1
1 = e1 e2 e3 1 = e1 + e2 + e3
1
The eigenvector for 2 are
1
2 = e1 e2 e3 −1 = e1 − e2
0
1
3 = e1 e2 e3 0 = e1 − e3
−1
Fortunately they form a basis.
In the above situation, we see two intresting facts, the eigenvector seems automatically forms a basis,
they are automatically linearly independet? Another fact is that the number of elementary solutions for the
eigen value seems equal to the algebraic multiplicity of it in the equation, is that true? We will solve this
question in our study next section
4.2. Linear Independence of eigenvector, Algebraic and Geometrical multiplicity, Eigen spaces.Diagonalization.
The first step to the basis–Linearly independence
We have following useful theorem.
Theorem 12. Suppose T : V −→ V is an linear transformation with distinct eigenvalues λ1 , λ2 , · · · ,
λs . (Here if an eigenvalue appears several times we just conted one here[In the future we will call this as
”algebraic multiplicity not considered”]).
Then for each one of them, we choose an eigenvector vi belongs
to λi . Then v1 v2 · · · vs are linearly independet.
Proof. Suppose there is a linear relation
v1 v2 · · · vs
a1
a2
..
.
as
. Then act the linear transformation on, we have
135
=0
0 = T0
= T
v1 v2 · · · vs
a1
a2
..
.
as
=
v1 × λ1 v2 × λ2 · · · vn × λn
a1
a2
..
.
as
=
v1 v2
λ1
λ2
· · · vs
λ3
λ4
..
.
λs
=
v1 v2 · · · vs
λ1 a1
λ2 a2
..
.
λs as
The above calculation tells us if
v1 v2 · · · vs
a1
a2
..
.
=0
as
then by applying T,
v1 v2 · · · vs
λ1 a1
λ2 a2
..
.
=0
λs as
, By applying T and T again, we get
v1 v2 · · · vs
v1 v2 · · · vs
λ21 a1
λ22 a2
..
.
λ2s as
λ31 a1
λ32 a2
..
.
λ3s as
=0
=0
···
Then apply T until s − 1 times and put the vectors together, we get
136
a1
a2
..
.
as
a1
a2
a3
..
.
λ 1 a1
λ 2 a2
λ 3 a3
..
.
λ21 a1
λ22 a2
λ23 a3
..
.
···
···
···
..
.
λs−1
1 a1
λs−1
2 a2
λs−1
3 a3
..
.
as
λs as
λ2s as
···
λs−1
s as
OK we know numbers can commute, so
a1 λ 1
a1
a2
a2 λ 2
a3
a3 λ 3
v1 v2 · · · vs
..
..
.
.
as λ s
as
a1 λ21
a2 λ22
a3 λ23
..
.
···
···
···
..
.
a1 λs−1
1
a2 λs−1
2
a3 λs−1
3
..
.
as λ2s
···
as λs−1
s
v1 v2
· · · vs
=
0 0 0 ··· 0
0 0 0 ··· 0
=
And you can see each row has a common left factor ai , let’s factor that out row by row.
v1 v2
a1
a2
· · · vs
a3
..
.
..
.
an
1
1
1
..
.
1
λ1
λ2
λ3
..
.
λ21
λ22
λ23
..
.
···
···
···
..
.
λs−1
1
λs−1
2
λs−1
3
..
.
λs
λ2s
···
λs−1
s
=
0 0 0 ··· 0
Now the matrix appeared on the right is Vandemonde matrix. Because we didn’t consider algebraic multiplicity, which means the eigenvalue I listed here are all distinct.By Quiz 2 bonus question, we know the
Vandemonde Matrix is invertible(because polynomial have two kind of basis and Vandemonde play the role
of change of basis matrix.). Thus, we invert it to the right, we have:
v1 v2
· · · vs
a1
a2
a3
..
.
..
=
.
0 0 0 ··· 0
an
1
1
1
..
.
λ1
λ2
λ3
..
.
λ21
λ22
λ23
..
.
···
···
···
..
.
λs−1
1
λs−1
2
λs−1
3
..
.
1
λs
λ2s
···
λs−1
s
That is
v1 v2
· · · vs
a1
a2
a3
..
.
..
.
=
0 0 0 ··· 0
an
We know the coordinate of zero vector is all the way zero, so we have
a1 = a2 = · · · = 0
137
−1
Then we proved the linearly independence of the eigenvector that belongs to different eigenvalues.
Thus, if you have bunch of eigenvectors, to check that they are linearly indepedent, you don’t have to
check them by put together, instead, you only have to check each small family of eigenvectos belongs to the
same eigenvalue are linearly independent. To see this, we give an example
Example 163. Suppose T is a linear transformation T : V −→ V , e1 e2 e3 e4 e5 is a basis for
V. And the matrix representation of T is:
T
e1 e2 e3 e4 e5
e1 e2 e3 e4
=
2 1
2 3
e5
1
3
2
2
1
−1
5
−3
−6
−2
2
2
1
3
1
Are you scared by this 5 × 5 matrix? Oh don’t worry, We found a perfect fact that this is actually block
upper triangular. Now calculate the characteristic polynomial. Let’s call the matrix appeared on the right
as A.
det(λI − A) = λ − 2 −1
−2 λ − 3
λ − 2 −1
= −2 λ − 3
−1
−2
1
−3
−2
−5
λ − 2 −1
3
−2 λ − 3
6
−1
−1 λ + 2
λ − 2 −1
3
−2 λ − 3
6
−1
−1 λ + 2
Now as you can see, this is just the product of characteristic polynomial of the diagonal blocks.
Now using tricks to calculate each polynomial (If you calculate characteristic polynomial by just expanding the determinant, please stop. That would be a waste of life. Please keep on reading to find the tricks.
The quick method to determine coefficient of your characteristic polynomial.)
λ − 2 −1
−2 λ − 2
= λ2 − 5λ + 4 = (λ − 1)(λ − 4)
And use tricks to calculate 3 × 3 we have
λ − 2 −1
3
−2 λ − 3
6
−1
−1 λ + 2
= λ3 − 3λ2 + 3λ − 1 = (λ − 1)3
Now the characteristic polynomial for T is
fT (λ) = (λ − 1)4 (λ − 4)
So the algebraic multiplicity for eigenvalue 1 is 4, and the algebraic multiplicity for eigenvalue 4 is 1.
Because the geometric multiplicity can not exceed the the algebraic multiplicity and it would be at least one.
Then the geometric multiplicity for 4 is also 1.
Now, find the eigenvector for 1. we look at
138
2 1
2 3
A−I =
1
3
2
2
1
2
2
1
3
1
−1
5
−3
−6
−2
1
−
−3 −1
1 −3
=
1
1
1
1
Now we are going to find out the null space of the matrix
1 1
1 2 −1
2 2
3 2 5
1 1 −3
2 2 −6
1 1 −3
1
3
1
2
1
2
2
1
2
1
−1
5
−3
−6
−3
Doing row reduce a little bit (Never do it completely to reduced row echelon form, it would cost you a lot
of time. You only reduce to the one when you could find solution. The knowledge is going to serve us, not to
kill us.)
1 1
1 2 −1
2 2
3 2 5
1
1
−3
2 2 −6
1 1 −3
1 1
2 2
r4 −2×r3 ,r5 −r3
−−−−−−−−−→
r2 −2×r1
−−−−−→
r2 −r3
−−−−→
1 1
1 1
1 2 −1
3 2 5
1 1 −3
1 2 −1
1 −2 7
1 1 −3
2 −1
−3 10
1 1 −3
1
OK, now the equation looks better, without completely reduced that to reduced row echelon form, we
could start by finding
1 1
1 2 −1
a
0
−3 10 b 0
1 1 −3
c = 0
d 0
e
0
139
The second equation tells you 10e = 3d. So we assume d = 10t and e = 3t. And plug in the third
equation ,we get
c + 10t − 9t = 0
This means c = −t. Now plug the material you had in first equation, you have
a + b − t + 20t − 3t = 0
Which means
a + b = −16t
Now b comes as another free varible, associate b = s then
a = −16t − s
Put all your work together, you have
a =
b =
c =
d
=
e =
−16t − s
s
−t
10t
3t
And by setting t=1,s=0; and t=0,s=1, we get elementary solution.
−16
0
−1
10
3
,
−1
1
0
0
0
Remember, this two vector is only for the space of matrix, we should go back to the original abstract
linear space. So the actual vector in the linear space we should left multiply the basis on. So the actual
eigenvector should be
1 =
e1 e2 e3 e4
e5
−16
0
−1
10
3
, 2 =
e1 e2 e3 e4
e5
−1
1
0
0
0
=
e1 e2
−1
1
Because the elementary solution automatically gives us the basis of solution space, so we don’t have to
check linearly independece, they become linearly independent automatically in the same eigenspace. And
because in the diffrent eigenspace, the linear independece is guaranteed by our theorem. So we also don’t
need to check linear independency.
Now we found the geometric multiplicity for 1 is 2. It is far less than its algebraic multiplicity, which is
4. Only this point is sufficient to say it is not diagonalizable.
Now let’s work for eigenvalue 4.
140
2 1
2 3
A−4I =
1
3
2
2
1
2
2
1
3
1
−1
5
−3
−6
−2
4
−
−2 1
2 −1
=
4
4
4
4
1 2 −1
3 2 5
−2 1 −3
2 −1 −6
1
1 −6
Now we solve the equation
−2 1
2 −1
1 2 −1
3 2 5
−2 1 −3
2 −1 −6
1
1 −6
a
b
c
d
e
=
0
0
0
0
0
Before you start reducing to reduced row echelon form, wait!!! you found something.
Pick out the last three equation and rewrite it into matrix form:
−2 1 −3
c
0
2 −1 −6 d = 0
1
1 −6
e
0
But simply noticed that the matrix
−2 1 −3
2 −1 −6
1
1 −6
is invertible!!! This means the homogeneous euqation would only have 0 solution , so we simply get
c = 0, d = 0, e = 0
OK, now with that, let’s going back, you are finding
−2 1
2 −1
1 2 −1
3 2 5
−2 1 −3
2 −1 −6
1
1 −6
a
b
0
0
0
0
0
=
This is the same as finding
−2 1
2 −1
a
1
That’s too easy, just let
=
b
2
So we have
a
b
c
d
e
a
b
1
2
0
0
0
=
=
So the eigenvector for eigenvalue 4 for this linear transformation is
141
0
0
0
0
0
3 =
e1 e2 e3 e4
e5
1
2
0
0
0
=
e1 e2
1
2
Now because we only get one solution in the process of finding elementary solution . So this eigen space
would be 1 dimensional. And the theorem tells us that eigenvector come from different eigenspace are
automatically linearly independet. So we know the vectors:
1 = −16e1 − e3 + 10e4 + 3e5 , 2 = e2 − e1 , 3 = e1 + 2e2
are linearly independent.
All the eigenvector of eigenvalue 1 lives in the subspace span(1 , 2 ) and all the eigenvector of eigenvalue
4 lives in the subspace span(3 )
Let’s use the viewpoint of space to see eigen vectors.
Proposition 91. Suppose T : V −→ V is a linear transformation, and λ is an eigenvalue of T, then all the
eigenvector of eigenvalue λ forms a subspace of T. We call this subspace as eigensubspace of the eigenvalue
λ
Proof. Because eigen vector of eigenvalue λ means the kernel of the linear tranformation T − λE, where E
is the identity map. and of course kernel is a linear subspace.
We also have another proof
Proof. Consider the set of all eigenvectors of eigenvalue λ, namely, W = {v|T v = λv}. We can check for
any two vector in W, any linear combination of them is still in W. And thus it become a linear subspace. Definition 67. Suppose T : V −→ V is a linear map, with characteristice polynomial fT = (λ−λ1 )n1 (λ−
λ2 )n2 (λ − λ3 )n3 · · · (λ − λm )nm . Then for the eigenvalue λ1 , the multiplicity for it to appear as a root of
characteristic polynomial is called the algebraic multiplicity of λ1 . And the dimension of the eigenspace of
λ1 is called the geometric multiplicity of λ1
Theorem 13. Suppose T : V −→ V is a linear transformation. λ0 is an eigenvalue, then the geometric
multiplicity is less than the algebraic multiplicity.
Proof.
T
v1
Fix λ0 , and suppose the geometric multiplicity of λ0 is g.
Then we could able to find g many linear independent eigen vectors v1 , · · · , vg
Extend it to a basis of whole space, v1 · · · vg g+1 · · · n
With such basis, the matrix representation form of linear transformation looks like
λ0
..
.
λ0
· · · vg g+1 · · · n = v1 · · · vg g+1 · · · n
Call the right matrix by A, Now by calculating its characteristic polynomial, we have
142
∗
∗
det(λI − A) = λ − λ0
..
∗
.
λ − λ0
λ−∗
∗
∗
|
{z
g
}
λ−∗ ∗
∗
.
g
..
= (λ − λ0 ) ∗
∗
∗
∗ λ−∗
∗
∗
..
.
∗
∗ λ−∗
= (λ − λ0 )g h(λ)
So the characteristic polynomial have at least g multiple root of λ0 .
Because the arbitrary choose of λ0 ,
We proved that for any eigenvalue, the geometric multiplicity is less than its algebraic multiplicity
Please remember why we want to study eigenvalue and eigenvectors. It is because we want to simplify
our problem, we wish there would be nice basis in our space such that every vector in the basis is eigenvector.
Definition 68. Let V be a linear space over F , then a linear transformation T : V −→ V is called
diagonalizable if there exists an eigenbasis for T in V. Where eigenbasis means a basis with all vector as
eigenvector.
Let’s translate this definition to matrix.
Definition 69. Let A be an n × n matrix, then A is called diagonalizable if it is similar to a diagonal matrix.
Of course, a linear transformation is diagonalizable if and only if its matrix in some matrix representation
is diagonalizable.
Corollary 6. A linear transformation is diagonalizable if and only if the geometric multiplicity is equal to
the algebraic multiplicity for each of its eigenvalues.
Proof. We only need to find an eigenbasis. Suppose geometric multiplicity is equal to algebraic. Then for
each distinct eigenvalue, we could find linearly independet eigenvectors as many as the algebraic multiplicity. And by going over all the eigenvalue, we found eigenvectors as many as the dimension of all space. And
because eigenvectors in each eigenspace is linearly independent, and eigenvectors comes from eigenspace
for different eigenvalue is automatically linearly independet, so when we put them together, they are linearly
independet. Thus, they form a basis.
On the other hand, if a linear transformation is diagonalizable, then we could find eigen basis. Because
geometric multiplicity could not exceeded algebraic. So the total number of those basis is at most the sum
of geometric, so is at most the sum of the algebraic multiplicity. But it reach the sum. The only case allow
it happen is geometric multiplicity equal to algebraic multiplicity.
Example 164. Previously we calculate the eigenvector and eigenvalue for the linear transformation
143
T
e1 e2 e3 e4 e5
=
e1 e2 e3 e4
2 1
2 3
e5
1
3
2
2
1
2
2
1
3
1
−1
5
−3
−6
−2
And we do found it has eigenvalue 1 and 4 and characteristic polynomial is
(λ − 1)4 (λ − 4)
For 1, it has eigen vector(basis of eigenspace of eigenvalue 1):
1 = −16e1 − e3 + 10e4 + 3e5 , 2 = e2 − e1
For 4, it has eigen vector(basis of eigenspace of eigenvalue 4):
3 = e1 + 2e2
From the polynomial, we know the multiplicity of the factor (λ − 1) is 4, and the multiplicity of the factor
(λ − 4) is 1. So we know, algebraic multiplicity for 1 is 4, and algebraic multiplicity for 4 is 1. And because
for 1, we only get at most 2 linearly independent eigenvectors, so the geometric multiplicity for 1 is 2. and
the algebraic multiplicity for 4 is 1. To sum up in the chart see
Eigenvalues
1 4
Algebraic multiplicity 4 1
Geometric multiplicity 2 1
So for eigenvalue 1, the geometric can not touch it . This linear transformation is not diagonalizable
because for eigenvalue 1 geometric multiplicity fails to fill all the algebraic multiplicity
Example 165. Although so many times we are not lucky enough to make sure our transformation is diagonalizable. But some cases, it is. Remember the original problem that 3 brothers are splitting money? The
transformation is like: V 3-dimensional, and
T e1 =
T e2 =
T e3 =
And we found the characteristic polynomial is
1
2 e2
1
2 e1
1
2 e1
+ 12 e3
+ 12 e3
+ 12 e2
1
(λ − 1)(λ + )2
2
So at this point, it has eigenvalue 1, and −1
,
the
algebraic
multiplicity for 1 is 1, and for −1
2
2 is 2.
OK and then we found eigenvectors. For 1 we found the eigenvector e1 + e2 + e3 , so the geometric
multiplicity for 1 is 1.And for −1
2 we found two linearly independent eigenvectors e1 − e2 and e2 − e3 , so the
geometric multiplicity for −1
is
2. So to sum up, we have:
2
Eigenvalues
1 − 12
Algebraic multiplicity 1 2
Geometric multiplicity 1 2
Because all the numbers match up, and it is diagonalizable. If it is, then pick up all the eigenvector you
found, in this case is exactly
1 = e1 + e2 + e3
2 = e1 − e2
3 = e2 − e3
144
They are automatically linearly independet because our theorem garantee that, and because geometric
is equal to algebraic. Then the total number of them would equal the dimension of whole space. and if this
happen ,they form a bsis. So let your transformation T act on this basis, and you would have
T
1 2 3
=
1 2 3
1
− 12
− 12
Because you have a diagonal matrix, so we call this process as diagonalization.
More Information on Characteristic polynomial
Now we are ready to explain the tricks to quickly detemine the characteristic polynomial. Like the
adjugate method. This method is only quick for 2 by 2 and 3 by 3 matrix, for 4 by 4 or even more, it is better
to change your method. But this section is important because it set up the relation between the eigenvalue
and the appearance of matrix.
As we see we could use determinant to find the characteristic polynomial, is there any meaning of the
coefficient of it? Let’s now discuss about this part.
Definition 70. Suppose A is a square matrix, then the trace of A is defined to be the sum of its diagonals
Definition 71. For a square matrix A. The determinant of a submatrix in which the diagonal of the submatrix
lies on the diagonal of A is called a principal minor.
Example 166. Suppose we have a 4 × 4 matrix
1
3
A=
5
2
4
1
5
1
1
8
3
0
1
4
2
6
. Find all principal minors of A of size 2.
Answer, to find the principal minors of size 2, firstly, you pick arbitrarily 2 elements on the diagonal.
1
3
5
2
4
1
5
1
1
8
3
0
1
4
2
6
And then you draw verticle and horizontal line from the element you picked out, and take out the element
in the intersection of your lines.(I can’t draw picture due to the limited technology here. I would like to find
someone to help me out.)
1
3
5
2
4
1
5
1
1
8
3
0
1
4
2
6
And then put those 4 element together into determinant. it is a principal minor
1 4 1 6 Because we have 4 element in diagonal, and each choice of two element gives us a principal minor. And
there are 6 choice to choose 2 from 4. So we have 6 principal minor. Please practice that method to find
other five. And finally, you would found that they are:
145
1 4
3 1
1 1
,
5 3
1 1
,
2 6
1 4
,
1 6
1 8
,
5 3
3 2
,
0 6
Theorem 14. If A is a square matrix and fA (λ) is its characteristic polynomial. Suppose
fA (λ) = λn − a1 λn−1 + a2 λn−2 − a3 λn−3 + · · · + (−1)n an
. Then
ai = The sum of all principal minor of size i
In particular, a1 is the trace of the matrix, and an is the determinant of the matrix.
Proof.
For a square matrix A, suppose
det(λI − A) = λn − a1 λn−1 + a2 λn−2 − a3 λn−3 + · · · + (−1)n an
Then factor out λ we get
1
A) = λn − a1 λn−1 + a2 λn−2 − a3 λn−3 + · · · + (−1)n an
λ
Devide λn to right hand side, we have
λn det(I −
det(I −
1
λn
λn−1
λn−2
λn−3
1
A) = n − a1 n + a2 n − a3 n + · · · + (−1)n an n
λ
λ
λ
λ
λ
λ
This is
1
A) = 1 − a1 λ−1 + a2 λ−2 − a3 λ−3 + · · · + (−1)n an λ−n
λ
Now substitute λ by −λ
det(I −
1
A) = 1 + a1 λ−1 + a2 λ−2 + a3 λ−3 + · · · + an λ−n
λ
Now substitute λ by λ1 . We have
det(I +
det(I + λA) = an λn + an−1 λn−1 + · · · + a1 λ + 1
By doing derivative, you found the k’th coefficient could be found by
dk
det(I + λA)|λ=0
dλk
Remember the derivative of determinant is the sum of the determinant with derivative row by
k!ak =
row.
Then the k’th derivative of the matrix I + λA is the sum of detreminant of those matrix with
selected k many row of I + λA being diffrentiated, and if plug in λ = 0 That is exactly a determinant of
matrix A only with k many row selected and for position outside of the row there are only 1 on diagonal and
other position 0. The determinant of that is exactly a principal minor. And it has been counted k! times due
to the order. so
k!ak = k!(sum of the principal minor of size k)
Then ak =sum of the principal minor of size k.
146
Example 167. Yes, this is exactly what I mensioned as trick. I think in the future you will do a lot of
characteristic polynomial calculation in other courses, learn this and do this faster than others. Hahahaha!
Let’s Just take previous one forexample. I mensioned
that use the trick to calculate characteristic poly
2 1 −3
2 1
nomial of matrix
and 2 3 −6
2 3
1 1 −2
Now lets calculate the first 2 by 2 matrix.
2 1
2 3
You know it is a 2 × 2 matrix, so you write on your paper.
λ2
And the sum of principal minor of size 1 is just the trace, so 2+3=5, so you write
λ2 − 5λ
(Please take care, -,+,-,+,-,+ is change alternating. be careful!)
and calculate the determinant: 2 × 3 − 1 × 2 = 4. So you write
f (λ) = λ2 − 5λ + 4
Yeah, we won!
OK and for the3 × 3 matrix.
2 1 −3
2 3 −6
1 1 −2
You know it is 3 × 3, so you write down
λ3
And then the next one would be the trace, so 2 + 3 + (−2) = 3, you write down (take care to change sign
alternatively!)
λ3 − 3λ2
OK, next coefficient, is the sum of principal minor of size 2. All principal minor is
2 1 = 4, 3 −6 = 0, 2 −3 = −1
2 3 1 −2 1 −2 And the sum of them are 4 + 0 − 1 = 3. So it is the coefficient with λ, you write down
λ3 − 3λ2 + 3λ
OK the last coefficient is the determinat. We found that determinant is 1. But now, take care! the last step
we should change the sign. So we wrote:
f (λ) = λ3 − 3λ2 + 3λ − 1
OK, we won!
Remember that the coefficient of polynomial is automatically have its own interpretation, in term of roots.
This is called Vieta’s formulas(You could Wikipedia it to know more!)
Lemma 1. Vieta’s formula Suppose P (x) is a polynomial of x of degree n over C with roots x1 , x2 · · · , xn
(considered in C, so all degree n polynomial would have n roots) Suppose
P (x) = xn − a1 xn−1 + a2 xn−2 − · · · + (−1)n a0
Then
147
a1
a2
..
.
a
n
= x1 + x2 + · · · + xn
= (x1 x2 + x1 x3 + · · · + x1 xn ) + (x2 x3 + · · · + x2 xn ) + · · · + xn−1 xn
= x1 x2 x3 · · · xn
Remark 6. We putting polynomial as alternating sum because only in alternating sum and leading coefficient one can those formula have the most simple look, or otherwise it would come with power of -1 and
divede something. And as you can see, in the characteristic polynomial case, if we put alternating sum.
then each coefficient is just the simple sum of principal minors. So putting alternating sum and has leading
coefficient 1 is the most natrual way to write polynomial. We are following natural, not trying to make you
in trouble : )
Now we have two different interpretation of characteristic polynomial, and keep in mind that the roots
are eigenvalues, so we have the following property.
Corollary 7. Suppose T : V −→ V is a linear transformation in n-dimensional space and A is its representation on some basis of V. Then
tr(A)
= a1 = λ1 + · · · + λn
sum of principal minor of size 2 = a2 = (λ1 λ2 + λ1 λ3 + · · · + λ1 λn ) + (λ2 λ3 + · · · + λ2 λn ) + · · · + λn−1 λn
..
.
det(A)
= an = λ1 λ2 λ3 · · · xn
With those fun property, We can do some interesting things.
Example 168. Suppose A is a 2 × 2 matrix, and suppose tr(A) = 3 and tr(A2 ) = 5, find det(A).
What? Are you sure? Yes! We could able to know determinant if we know it is 2 by 2 and its trace of A
and A square! It’s fun!
Answer:
Because A is a 2 × 2 matrix, then it would have 2 eigenvalue(multiplicity considered), suppose it is
λ1 , λ2
Then the eigenvalue of
A2
would be
λ21 , λ22
(This is by spectral mapping theorem, you would know in next few pages.)
So because trace is the sum of two eigenvalues. So
3 = tr(A) = λ1 + λ2
5 = tr(A2 ) = λ21 + λ22
Then sqare the first equation we get
9 = λ21 + 2λ1 λ2 + λ22
Because we already know from the second equation that
5 = λ21 + λ22
So take the difference of them, we get
4 = 2λ1 λ2
So that is
λ1 λ2 = 2
148
And we know the product of two eigenvalue is equal to the determinant, so
det(A) = 2
Example 169. We call the n × n matrix A that has some power equal to n as nilpotent matrix. Show that if
Am = 0 for some m, then all the eigenvalue for A is equal to 0
Answer: We don’t know the eigenvalue for A actually, but we can assume the eigenvalue for A is
λ1 , λ2 , · · · , λn
So with this assumption, the eigenvalue for Am is
m
m
λm
1 , λ2 , · · · , λn
But Am = 0, we already know the eigenvalue for 0 matrix. because the characteristic polynomial for 0
matrix is det(λI − 0) = λn , so all the eigenvalue for 0 matrix is 0. In another hand, this means
λm
= 0
1
λm = 0
2
..
.
λm = 0
n
So we should have
λ1
λ2
..
.
λ
n
= 0
= 0
= 0
Now let’s look another theorem powered by the det(I − AB) = det(I − BA) Actually by replacing
A by λA, we could have general formula like det(I − λAB) = det(I − λBA) and this is also works for
nullity. We have null(I − λAB) = null(I − λBA)
Theorem 15. Suppose A is an m × n matrix and B is an n × m matrix. Then the characteristic polynomial
of AB and BA are only different by the factor of a power of λ
Proof.
det(λI − AB) = λm det(I − λ1 AB)
= λm det(I − λ1 BA)
= λm−n det(λI − BA)
So the only difference of them is just by the difference of their size.
Corollary 8.
tr(AB) = tr(BA)
Proof. because we know the characteristic polynomial of AB and BA are just different by a factor of λm−n .
And the trace of AB is just the opposite second coefficient of characteristic polynomial of AB. and trace
of BA is just the opposite second coefficient of characteristic polynomial of BA. But now because this two
polynomial has the same second coefficient(because this two polynomial only different by a factor of λm−n ).
so they are the same.
149
Upper triangulize a matrix by similarity
Last course we said that we are going to diagonalize a matrix, but as you see diagonalizable needs strong
condition that the algebraic multiplicity should be equal to the geometric multiplicity. As a result, the case
always happen is that we can’t diagnalize a matrix. In this case, some property would be hard to look into.
Instead, we could still find matrix P and make our matrix similar to an upper triangular matrix.
Firstly, let’s look into upper triangular matrix.
Proposition 92. If M is an upper triangular or lower triangular matrix, then the eigenvalue of M is exactly
appear on the diagonal, the multiplicity of appearance is exactly the same as algebraic multiplicity.
Proof.
Suppose
a11 a12 · · ·
a22 · · ·
A=
..
.
a1n
a2n
..
.
ann
Then the characteristic polynomial of A is
λ − a11 −a12 · · ·
λ − a22 · · ·
fA (λ) = ..
.
−a1n
−a2n
..
.
λ − ann
Then because the determinant of upper triangular matrix is the product of diagonal.
Then
fA (λ) = (λ − a11 )(λ − a22 ) · · · (λ − ann )
Thus, the diagonal are exactly the root of this polynomial
And the numerical multiplicity are exactly the same as the multiplicity it appears on diagonal. Example 170. Find the eigenvalue and eigenvector for the matrix
1 2 3
6 7
A=
9
Answer: For this upper triangular matrix. we observe that 1,2,9 are all on diagonal, then 1,2,9 are its
eigenvalue. For 1, we find
−2 −3
I − A = −5 −7
−8
By setting up the equation
−2 −3
x
0
−5 −7 y = 0
0
−8
z
We found 1 elementary solutions.
1
0
c1 =
0
Now, let’s continue
150
x
5 −2 −3
−7
(6I − A) y =
z
−3
We found another eigenvector
2
5
c2 =
0
Now let’s also continue
x
8 −2 −3
3 −7
(9I − A) y =
z
x
0
y = 0
z
0
x
0
y = 0
z
0
We found our last eigenvector:
23
c3 = 56
24
Now let’s see the method to similar a matrix to upper triangular matrix.
Lemma 2. For any matrix A over C(It is C! we need every polynomial to have roots), we are able to find
P, such that P −1 AP is a block upper trangular matrix like
∗
∗
···
∗
λ
∗
∗
···
∗
∗
∗
···
∗
P −1 AP =
..
..
..
.
.
.
.
.
.
∗
∗
···
∗
Proof.
We view the n × n matrix A as a linear transformation by left multiplication on Mn×1 (C).
Then the characteristic polynomial of A is det(λI − A)
This polynomial would at least has 1 root, say λ0
c11
c12
Then we found a eigen vector for λ0 , say c1 = ..
.
c1n
Now because c1 is a non-zero column matrix, then some of the entry in c1 is not zero.
let’s say c1i 6= 0
Then we construct other column vector ck to be different column vectors that having
1 entry at place different than i and other entries 0.
Then c1 , c2 · · · , cn form a basis in Mn×1 (C)and we know Ac1 = λc1 .
So
∗
∗
···
∗
λ
∗
∗
···
∗
∗
∗
···
∗
A c1 c2 · · · cn = c1 c2 · · · cn
..
..
..
.
.
.
.
.
.
∗
∗
···
∗
151
Let P =
c1 c2 · · · cn
λ
AP = P
So
λ
P −1 AP =
, we have
∗
∗
∗
..
.
∗
∗
∗
..
.
···
···
···
..
.
∗
∗
∗
..
.
∗
∗
···
∗
∗
∗
∗
..
.
∗
∗
∗
..
.
···
···
···
..
.
∗
∗
∗
..
.
∗
∗
···
∗
Proposition 93. For any matrix A over C, we are able to find some invertible matrix P, such that P −1 AP is
an upper triangular matrix.
Proof. Firstly, let’s find P1 , such that
P1−1 AP1
λ1 r1
B
λ2 r2
C
=
And for B, we find P2 , such that
P2−1 BP
And we have
1
−1
P2
P1−1 AP1
=
1
P2
=
−1 1
P2
λ1
λ1
=
=
r1 P2
−1
P2 BP2
λ1 r1
B
1
P2
r1 P2
λ2 r2
C
And we continue doing this for C and continue and continue, finally we will get an upper triangular
matrix.
So we have use the similarity to relate every matrix to an upper triangular matrix. Which property is
preserved under similarity? Let’s look
Summary 1. The following property are preserved under similarity.
(1) det(A) = det(P −1 AP )
(2) rank(A) = rank(P −1 AP )
(3) Characteristic polynomial and eigenvalues.
(4) Algebraical multiplicity of an eigenvalue λ
(5) Geometrical multiplicity of an eigenvalue λ
152
Proof.
For (1), det(P −1 AP ) = det(P )−1 det(A) det(P ) = det(P )−1 det(P ) det(A) = det(A).
For (2), because rank does not change under right or left multiplication of invertible matrix. then
rank(P −1 AP ) = rank(AP ) = rank(A)
For (3) We already proved that, because characteristic polynomial are preserved, then the eigenvalue is the root of characteristic polynomial, so of course it is preserved.
For (4) Because characteristic polynomial are preserved, so of course, the algebraical multiplicity
of an eigenvalue is preserved.
For (5) Because Geometrical multiplicity is rank(λI −A), and λI −A is similar to λI −P −1 AP ,
so by (2), it is preserved.
With the discussion above, We may change a matrix to an upper triangular matrix without changing
essential property of it and then discover the relation of algebraical multiplicity and geometrical multiplicity.
Because any matrix over C could similar to an upper triangular matrix, thus it is an useful tool to explain
everything.
4.3. Polynomial of linear transformation, Spectral mapping theorem, minimal polynomial of linear
transformation. In this section, we are going to discuss more detail about algebraic multiplicity and geometrical multiplicity, and related it to the power-rank curve. And then discuss all the properties of a
polynomial of linear transformation has.
We will see the upepr-triangular matrices would be helpful.
Lemma 3. Suppose f (x) = xn +a1 xn−1 +· · ·+an is an polynomial over F, then f (P −1 AP ) = P −1 f (A)P
We call this property that similarity doesn’t change the polynomial of linear transformation.
Proof. We discovered
−1
−1
−1
−1
(P −1 AP )n = P
{z AP × · · · × P AP}
| AP × P AP × P
n many
= P −1 AP × P −1 AP × · · · × P −1 AP
= P −1 |A × A ×
{z· · · × A} P
nmany
= P −1 An P
Now let f (x) = xn + a1 xn−1 + · · · + an , then
f (P −1 AP ) =
=
=
=
(P −1 AP )n + a1 (P −1 AP )n−1 + · · · + an I
P −1 An P + a1 P −1 An−1 P + · · · + an
P −1 (An + a1 An−1 + · · · + an I)P
P −1 f (A)P
Theorem 16. Spectral mapping theoremSuppose A is a n × n matrix over C, then A has n eigenvalues(multiplicity considered), namely, λ1 , λ2 , · · · , λn . f is a polynomial, then the eigenvalues of f (A) is
exactly f (λ1 ), f (λ2 ), · · · , f (λn ).
Proof. Choose P such that P −1 AP is an upper-triangular matrix. Then the element on the diagonal is
exactly the eigenvalues with multiplicity considered. Now we act polynomial on, then the diagonal of
f (P −1 AP ) is exactly the polynomial on each entries on diagonal, so P −1 f (A)P has eigenvalues f (λ1 ), f (λ2 ), · · · , f (λn ).
So is f (A)
153
Example 171. Suppose A is an n × n matrix with characteristic polynomial
(λ − 1)3 (λ − 2)2
Find the characteristic polynomial of A2 + I
Answer: Characteristic polynomial is the product of the factor of eigenvalue where algebraic multiplicity
counted. So this fact tells us that:
Eigenvalues for A is1, 1, 1, 2, 2
Now A2 + I is getting just by applying the polynomial x2 + 1, so the eigenvalue of A2 + I is getting by
apply the polynomial x2 + 1 to the eigenvalue of A. then A2 + I has eigenvalue
Eigenvalues for A2 + Iis2, 2, 2, 5, 5
So A2 + I has characteristic polynomial (λ − 2)3 (λ − 5)2
Thus, the spectral mapping theorem helps us to determine the eigenvalues and algebraic multiplicity of
an eigenvalue, but we still don’t understand the geometric multiplicity part.
Let’s recall what is geometric multiplicity. For an n × n matrix A, and λ be one of its eigenvalue. Then
the geometric multiplicity of A is
null(A − λI) = n − rank(A)
. Thus, to know the geometric multiplicity of a polynomial, say, f (A), we has to know null(f (A) − f (λ)I),
and this is enough to study the nullity of a polynomial expression of linear transformation.
Firstly, if we come a polynomial, say λ2 − 3λ + 2, then we could factor it to be like (λ − 1)(λ − 2) So
in general we could write a polynomial as (λ − λ1 )n1 (λ − λ2 )n2 · · · (λ − λk )nk
The following lemma would be useful, that means the nullity of coprime factor could split up.
Lemma 4. Suppose α and β be different number, and A is an n × n matrix, then
null((A − αI)r (A − βI)s ) = null((A − αI)r ) + null((A − βI)s )
Proof. Let us denote B = A − βI, and let γ = β − α, so B − γI = A − αI
We only need to show
null((B − γI)r B s ) = null((B − γI)r ) + null(B s )
and because null(B − γI) = null(γI − B), so lets show
null((γI − B)r B s ) = null((γI − B)r ) + null(B s )
For the right hand side of the equation, we have
r
s
null((B − γI) ) + null(B ) = null
(B − γI)r
Bs
Now let’s do row transformation. We can use matrices. please remember an important formula we learned
in middle school saying
ak − bk = (a − b)(ak−1 + ak−2 b + ak−3 b2 + · · · + bk−1 )
k
k
−b
So aa−b
= ak−1 + ak−2 b + ak−3 b2 + · · · + bk−1 . This looks like fractional, but not fractional. We will
use this trick.
Starting to do row transformation
154
(γI − B)r
Bs
s
s
I−B r
bc2 +bc1 ×( γγI−B
)
(γI − B)r (γ s I − B s )r
Bs
(γI − B)r γ sr I + B s g(B s )
Bs
(γI − B)r γ sr I
Bs
(γI − B)r
γ sr I
1
r
s
− γ sr (γI − B) B
γ sr I
−−−−−−−−−−−−→
=
br1 −B s g(B s )×br2
−−−−−−−−−−−→
s
br2 − γBsr ×br1
−−−−−−−−→
bc1 −bc2 ×
(γI−B)r
γ sr
−−−−−−−−−−−→
bc1 ↔bc2
−−−−−−−−−−−−−→
− γ1sr (γI − B)r B s
−−−−−→
bc1 × γ 1sr ;bc2 ×(−γ sr )
γ sr I
− γ1sr (γI − B)r B s
I
(γI − B)r B s
here the g is some polynomial.
From above, we know
null(γI − B)r + null(B s ) = null(I) + null((γI − B)r B s )
and with null(I) = 0, we get the formula
null(γI − B)r + null(B s ) = null((γI − B)r B s )
Example 172. Suppose A is an n × n matrix with eigenvalues 1,2,3. And the geometric multiplicity for 1 is
2, for 2 is 2, for 3 is 2. Find the geometric multiplicity for the matrix A2 − 3A of eigenvalue -2
Yes, it has eigenvalue -2 by spectrual mapping theorem. To find the geometric multiplicity of A2 − 3A of
eigenvalue -2. We need to find
null(A2 − 3A + 2I)
We found A2 − 3A + 2I = (A − I)(A − 2I), so
null(A2 − 3A + 2I) = null((A − I)(A − 2I)) = null(A − I) + null(A − 2I) = 2 + 2
Because remember the nullity of A − I is the geometric multiplicity of eigenvalue 1, and the nullity of
A − 2I is the geometric multiplicity of 2. but we all know they are 2 from the assumption.
Previously, we know geometric multiplicity of eigenvalue λ0 of A is equal to null(A − λ0 I), but we still
didn’t give an explanation of what algebraic multiplicity is. With our powerful power-rank curve, we could
have a look in it.
155
Theorem 17. Suppose A is an n × n matrix, and λ0 be one of its eigenvalue. Then the stable nullity of
A − λ0 I is equal to the algebraic multiplicity of λ0 .
Proof. Firstly, by considering I, A, A2 , · · · , An , they are n+1 vectors in n-dimensional space. So they are
linearly dependent. Thus there exists a polynomial f (λ) such that f (A) = 0. Now suppose
f (λ) = (λ − λ1 )k1 (λ − λ2 )k2 · · · (λ − λm )km
Because the right hand side is 0, we can raise the power km to be large enough such that the nullity of
each factor stable.
So we know
(A − λ1 )k1 (A − λ2 )k2 · · · (A − λm )km = 0
take nullity and use theorem on both side we have
null((A − λ1 )k1 ) + null((A − λ2 )k2 ) + · · · + null((A − λm )km ) = n
Now change all the ki by si , where si is the rank-stable number, which is the minimal number that rank
stop dropping.
If some si = 0, then null((A − λi )si ) = null(I) = 0 so we could miss that because it has 0 contribution
to the equality. Then we only write down those term with stable number larger than zero
null((A − λ1 )s1 ) + null((A − λ2 )s2 ) + · · · + null((A − λm )sm ) = n
Because stable number larger than zero means not invertible. So each of the λi is eigenvalue. And we
know null((A − λ1 )s1 ) is the geometric multiplicity of eigenvalue 0 of (A − λ1 )s1 , which can not exceeded
algebraic multiplicity of eigenvalue 0 of (A−λ1 )s1 , which is equal to the algebraic multiplicity of eigenvalue
λ1 of A due to spectrual mapping theorem. So we know
null((A − λi )si ) ≤ ai
where ai is the algebraic multiplicity of λi .
So
n = null((A − λ1 )s1 ) + null((A − λ2 )s2 ) + · · · + null((A − λm )sm )
≤ a1 + a2 + · · · + am
= n
So because restrict by the n on both side of inequality. The only case that could be happen is equal. So
the stable nullity is equal to the algebraic multiplicity.
Now we give an explaination of the algebraic multiplicity, it is the stable nullity. In the mean time,
geometric multiplicity is just the nullity that you go by the first step, no wonder why it is smaller.
Example 173. Suppose A is an 8 × 8 matrix with characteristic polynomial
(λ − 1)5 (λ − 2)3
Suppose we know the geometric multiplicity of eigenvalue 2 is 1, then what is rank(A − 2I), rank((A −
2I)2 ), and rank((A − 2I)3 ), rank((A − 2I)4 )?
Answer:
Because we know the geometric multiplicity of 2 is just null(A − 2I) = 1, and we know the stable nullity
is algebraic multiplicity of 2, which is 5. So the rank curve, would start from 8. drop by 1 at first step, and
finally reach 8-5=3 as its stable rank. Because at first it drops by 1, and non-acceleration garantees that
each step it could only drop by 1 or 0. but 0 means stable. so it drop by 1 each time until it touches 3. So
rank(A − 2I) = 7, rank((A − 2I)2 ) = 6, rank((A − 2I)3 = 5, rank((A − 2I)4 = 4, rank((A − 2I)5 ) =
3, rank((A − 2I)6 ) = 3, rank((A − 2I)7 ) = 3, · · ·
156
Definition 72. Suppose A is a n × n matrix, We call a polynomial p(λ) as annhilating polynomial for A if
p(A) = 0
2
Example 174. For example, if A =
is 2 × 2 matrix, then p(λ) = λ2 − 3λ + 2 could be an
1
annhilating polynomial because if we plug our matrix in it, it would become
4
2
1
0 0
2
A − 3A + 2I =
−3
+2
=
1
1
1
0 0
Corollary 9. For n × n matrix A, Its characteristic polynomial is annhilating polynomial.
Proof. Suppose we have
null((A − λ1 )s1 ) + null((A − λ2 )s2 ) + · · · + null((A − λm )sm ) = n
in which each of the λi are eigenvalue, and si is the rank stable number. Because the algebraic multiplicity
for λi , which we denote as ai , serves as stable nullity. And the previous theorem says stable nullity is always
larger than the stable number, so ai ≥ si . So at ai , it already stable, so
null((A − λ1 )a1 ) + null((A − λ2 )a2 ) + · · · + null((A − λm )am ) = n
That is
(A − λ1 )a1 (A − λ2 )a2 · · · (A − λm )am = 0
But left hand side is exactly the characteristic polynomial apply on A. So this tells us characteristic
polynomial kills A.
This is a very useful corollary, let’s see some example
Example 175. Suppose A is an 2 × 2 matrix with tr(A) = −1 and det(A) = 1, show that A3 = I
We don’t know what happened actually, but we could try. because we have trace and determinant, from
here we know the characteristic polynomial of A is
fA (λ) = λ2 + λ + 1
Now given that characteristic polynomial is annhilating, we know
A2 + A + I = 0
, that is
A2 = −A − I
Let’s repetidly using this and test out what is A3 , and replace A2 by −A − I whenever it appears.
Multiply A we know
A3 =
=
=
=
−A2 − A
−(−A − I) − A
A+I −A
I
But sometimes the characteristic polynomial is not the most efficient one. Just think identity matrix for
example. The eigenvalue for identity matrix is al l the one. So the characteristic polynomial for identity is
(λ − 1)n , where n is the size of it. But actually λ − 1 is enough to kill it. The later one, we would call
minimal polynomial.
157
Definition 73. Suppose A is an n × n matrix, the minimal polynomial of A is defined to be the lowest degree
with leading coefficient one annihilating polynomial
1 1
Example 176. Consider the matrix A =
1
Then A is an upper triangular matrix, with eigenvalue 1 and 1. So we know the eigenpolynomial kills A,
so at least
(A − I)2 = 0
But is it minimal? Yes, it is minimal, because we couldn’t lower any degree of each factor. if we plug in
A into A − I, the only matrix we will get is
1
0 0
A−I =
6=
0 0
And we still need to square it to get zero zero.
Previously, with the method showing in the nullity. We see only the factor of eigenvalue contributes
nullity. The minimal polynomial is looking for those smallest power such that it could provide enough
nullity, so of course it should be the rank-stable number. Because rank-stable number is the smallest number
that the nullity does not increase more.
Theorem 18. Suppose A is an n × n matrix, and λ1 , λ2 , · · · , λk are all its distinct eigenvalues, then the
minimal polynomial of A is equal to
(λ − λ1 )s1 (λ − λ2 )s2 · · · (λ − λk )sk
Where each sk is the rank stable number for A − λk I
Example 177. Suppose T : V −→ V is a linear transformation of 15 dimensional space, and its characteristic polynomial is (λ − 1)(λ − 2)2 (λ − 3)6 (λ − 4)6 and suppose we know
dim(Im(T − 2I)) = 14
dim(Im(T − 3I)3 ) = 11
dim(Im(T − 4I)2 ) = 12
Find the minimal polynomial of T.
Answer: Finding the minimal polynomial is the same as finding the rank-stable number. Combine with
the fact that stable nullity of T − λ0 I is the algebraic multiplicity for λ0 , we know that
dim(Im(T − 2I)∞ ) = 15 − (algebraic multiplicity of 2) = 15 − 2 = 13
Similarly,
dim(Im(T − I)∞ ) = 14
dim(Im(T − 3I)∞ ) = 9
dim(Im(T − 4I)∞ ) = 9
After plotting out the power-rank curve, you would find that the rank-stable number for T − I is 1, for
T − 2I is 2, for T − 3I is 5, for T − 4I is also 5. So the minimal polynomial is just obtained by putting all
the rank stable number to each corresponding factor, so minimal polynomial of T is
(λ − 1)(λ − 2)2 (λ − 3)5 (λ − 4)5
158
Example 178. Suppose A is a square matrix with minimal polynomial λ2 − 3λ + 1 = 0, expressing A−1 in
term of a polynomial of A.
Answer: In this case,
A2 − 3A + I = 0
We could write it as
(A − 3I)A + I = 0
Thus
(A − 3I)A = −I
So
(3I − A)A = I
then
A−1 = 3I − A
4.4. (Coming soon, Not in Final)Jordan Canonical Form.
4.5. (Coming soon, Not in Final)Root subspace and classification of invariant subspaces.
4.6. (Coming soon, Not in Final)Classification of linear transformation commuting with T. coursenote
ends here, I’ll continue writting
159
