Download Systems of Linear Equations and Matrices

Linear Algebra
 Lecturer:
Heru Suhartanto, PhD, [email protected]
 Schedule (at C3113):
 Monday, 10.00 – 11.40 AM
 Wednesday, 08.00 – 08.50 AM
 Marking scheme:
 2 exams (mid 25%, final 35 %): reqs = attendance 75%
 2 quizes, 20%
 4 assignments, 20%
 Reference:
 Horward Anton, Elementary Linear Algebra, 8-th Ed, John Wiley &
Sons, Inc, 2000
 More information (will be updated soon) at
http://telaga.cs.ui.ac.id/WebKuliah/LinearAlgebra06/

Linear Algebra - Chapter 1 [YR2005]
1
Why studying Linear Algebra?
 It is mostly used in Computer Graphics,
animation, cars and aero plane designs, etc.
 It is frequently used in economics, and other
subjects,
 It is frequently used in Scientific Computation
and industrial application algorithm and
application?
 It will be used in any area in the future that
you will notice later 
Linear Algebra - Chapter 1 [YR2005]
2
Systems of Linear
Equation and Matrices
CHAPTER 1
FASILKOM UI 05
Introduction ~ Matrices
 Information in science and mathematics is
often organized into rows and columns to
form rectangular arrays.
 Tables of numerical data that arise from
physical observations
 Example: (to solve linear equations)
5x  y  3
2x  y  4

5
2

1
1
3
4 
Solution is obtained by performing appropriate
operations on this matrix
Linear Algebra - Chapter 1 [YR2005]
4
1.1 Introduction to
Systems of Linear Equations
Linear Equations

In x y variables (straight line in the xy-plane)
a1 x  a2 y  b

where a1, a2, & b are real constants,
In n variables
a1 x1  a2 x2  ...  an xn  b
where a1, …, an & b are real constants
x1, …, xn = unknowns.
 Example 1 Linear Equations
1
x1  2x2  3x3  x4  7
x  3 y  7 y  2 x  3z  1

The equations are linear (does not involve any
products or roots of variables).
Linear Algebra - Chapter 1 [YR2005]
6
Linear Equations
x3 y 5


3 x  2 y  z  xz  4
y  sin x
The equations are not linear.
A solution of a1x1  a2 x2  ...  anxn  b is a sequence of
n numbers s1, s2, ..., sn Э they satisfy the equation
when x1=s1, x2=s2, ..., xn=sn (solution set).
 Example 2 Finding a Solution Set
4x  2 y  1

1 equation and 2 unknown, set one var as the
parameter (assign any value)
x  12 t  14 , y  t
x  t , y  2t  12
or

1 equation and 3 unknown, set 2 vars as parameter

x1  4 x2  7 x3  5
x1  5  4s  7t , x2  s, x3  t
Linear Algebra - Chapter 1 [YR2005]
7
Linear Systems / System of
Linear Equations
 Is A finite set of linear equations in the vars x1, ..., xn
 s1, ..., sn is called a solution if x1=s1, ..., xn=sn is a solution
of every equation in the system.
4 x1  x 2  3x3  1
 Ex.
3x1  x 2  9 x3  4





x1=1, x2=2, x3=-1 the solution
x1=1, x2=8, x3=1 is not, satisfy only the first eq.
System that has no solution : inconsistent
System that has at least one solution: consistent
Consider:
l1 : a1 x  b1 y  c1, a1  0  b1  0
l 2 : a 2 x  b 2 y  c 2, a1  0  b1  0
Linear Algebra - Chapter 1 [YR2005]
8
Linear Systems
(x,y) lies on a line if and only if the numbers x and y satisfy
the equation of the line. Solution: points of intersection l1 & l2
y
 l1 and l2 may be parallel:
l1 l 2
x
no intersection, no solution
x

y l1 l 2
 l1 and l2 may intersect
at only one point: one solution
x
y
 l1 and l2 may coincide:
infinite many points of intersection,
infinitely many solutions
Linear Algebra - Chapter 1 [YR2005]
l1 &l 2
x
9
Linear Systems
 In general: Every system of linear equations has
either no solutions, exactly one solution, or infinitely
many solutions.

 An arbitrary system of m linear equations in n
unknowns:
a11x1 + a12x2 + ... + a1nxn = b1
a21x1 + a22x2 + ... + a2nxn = b2



+ ... + a x
am1x1 + am2x2
mn n = bm
 x1, ..., xn = unknowns, a’s and b’s are constants
 aij, i indicates the equation in which the coefficient
occurs and j indicates which unknown it multiplies
Linear Algebra - Chapter 1 [YR2005]
10
Augmented Matrices
 Example:
 a11 a12
 a 21 a 22

 


 am1 am 2



a1n
a2n

amn
b1 
b 2 
 

bm 
x1  x 2  2 x3  9
2 x1  4 x 2  3x3  1
1
2
9
1
2

4

3
1


 3
6  5 0 
3x1  6 x 2  5 x3  0
 Remark: when constructing, the unknowns must be
written in the same order in each equation and the
constants must be on the right.
Linear Algebra - Chapter 1 [YR2005]
11
Augmented Matrices
 Basic method of solving system linear equations
Step 1: multiply an equation through by a nonzero
constant.
 Step 2: interchange two equations.
 Step 3: add a multiple of one equation to another.
 On the augmented matrix (elementary row
operations):
 Step 1: multiply a row through by a nonzero
constant.
 Step 2: interchange two rows.
 Step 3: add a multiple of one equation to another.

Linear Algebra - Chapter 1 [YR2005]
12
Elementary Row Operations
(Example)
r1 : x  y  2 z  9
r 2 : 2 x  4 y  3z  1
r 3 : 3x  6 y  5 z  0
 r2= -2r1 + r2
r1 : x  y  2 z  9
r 2 : 2 y  7 z  17
r 3 : 3x  6 y  5 z  0
 r3 = -3r1 + r3
r1 : x  y  2 z  9
r 2 : 2 y  7 z  17
r 3 : 3 y  11z  27
1
2

 3
1
4
6
2
3
5
 1
 0

 3
1
2
6
2
7
5
 1
 0

 0
Linear Algebra - Chapter 1 [YR2005]
1
2
3
9
1
0




9 
 17
0 
2
9 
 7  17 
 11  27
13
Elementary Row Operations
(Example)
 r2 = ½ r2
r1 : x  y  2 z  9
r 2 : y  72 z   172
r 3 : 3 y  11z  27
 r3 = -3r2 + r3
r1 : x  y  2 z  9
r 2 : y  72 z   172
r 3 :  12 z   32
 r3 = -2r3 r1 : x  y  2 z  9
r 2 : y  72 z   172
r3 : z  3
 1
 0

 0
1
1
3
2
9 
 72  172 
 11  27
 1
 0

 0
1
1
0
2
 72
 12
9 
 172 
 32 
 1
 0

 0
1
1
0
2
 72
1
9 
 172 
3 
Linear Algebra - Chapter 1 [YR2005]
14
Elementary Row Operations
(Example)
 r 1 = r1 – r 2
r1 : x  112 z 
35
2
r 2 : y  72 z   172
r3 : z  3
 1
 0

 0
 r1 = -11/2 r3 + r1 r1 : x  1
0
1
0
11
2
7
2

1
0
1
7
17 
r
2: y 2 z   2
1
 r2 = 7/2 r3 + r2
0
 0
0
r3 : z  3
 Solution:
r1 : x  1
1 0 0 1 
0 1 0 2
r2 : y  2


0 0 1 3
r3 : z  3
Linear Algebra - Chapter 1 [YR2005]

 
3 
35
2
17
2
0
1 
 72  172 
1
3 
15
1.2 Gaussian Elimination
Echelon Forms

Reduced row-echelon form, a matrix must have
the following properties:
1.
2.
3.
4.
If a row does not consist entirely of zeros the the
first nonzero number in the row is a 1 = leading 1
If there are any rows that consist entirely of
zeros, then they are grouped together at the
bottom of the matrix.
In any two successive rows that do not consist
entirely of zeros, the leading 1 in the lower row
occurs farther to the right than the leading 1 in
the higher row.
Each column that contains a leading 1 has zeros
everywhere else.
Linear Algebra - Chapter 1 [YR2005]
17
Echelon Forms
 A matrix that has the first three properties is
said to be in row-echelon form.
 Example:

Reduced row-echelon form:
1
0

 0

0
1
0
0
0
1

4  1 0 0 
7 , 0 1 0, 




 1 0 0 1 

0
0
0
0
2
0
0
0
1
0
0
0
0
1
0
0
1
3
0
0

 0 0

, 
 0 0


Row-echelon form:
1
0

 0
4
1
0
3
6
1
7  1 1 0   0
2  , 0 1 0  ,  0
5  0 0 0  0
1
0
0
Linear Algebra - Chapter 1 [YR2005]
2
1
0
6
1
0
0
0 
1 
18
Elimination Methods
 Step 1: Locate the
leftmost non zero
column
 Step 2: Interchange
r 2 ↔ r 1.
 Step 3: r1 = ½ r1.
 Step 4: r3 = r3 – 2r1.
 0
 2

 2
0
4
4
2
 10
5
0
6
6
7
12
5
12 
28 
 1 
 2
 0

 2
4
0
4
 10
2
5
6
0
6
12
7
5
28 
12 
 1 
 1
 0

 2
2
0
4
5
2
5
3
0
6
6
7
5
14 
12 
 1
 1
 0

 0
2
0
0
5
2
5
3
0
0
6
14 
7
12 
 17  29
Linear Algebra - Chapter 1 [YR2005]
19
Elimination Methods
 Step 5 : continue do all steps above until the entire
matrix is in row-echelon form.



r2 = -½ r2
 1
 0

 0
2
0
0
5
1
5
3
0
0
6 14 
 72  6 
 17  29
r3 = r3 – 5r2
1
0

 0
2
0
0
5
1
0
3
0
0
6
 72
1
0

 0
2
0
0
5
1
0
3
0
0
6
 72
1
r3 = 2r3
Linear Algebra - Chapter 1 [YR2005]
1
2
14 
 6
1 
14 
 6
2 
20
Elimination Methods
 Step 6 : add suitable multiplies of each row to the
rows above to introduce zeros above the leading 1’s.
 r2
 r1
 r1
= 7/2 r3 + r2
1
0

 0
2
0
0
5
1
0
3
0
0
6
0
1
14 
1 
2 
= -6r3 + r1
1
0

 0
2
0
0
5
1
0
3
0
0
0
0
1
2
1
2
= 5r2 + r1




1 2 0 3 0 7
0 0 1 0 0 1 


0 0 0 0 1 2
Linear Algebra - Chapter 1 [YR2005]
21
Elimination Methods
 1-5 steps produce a row-echelon form
(Gaussian Elimination). Step 6 is producing
a reduced row-echelon (Gauss-Jordan
Elimination).
 Remark: Every matrix has a unique reduced
row-echelon form, no matter how the row
operations are varied. Row-echelon form of
matrix is not unique: different sequences of
row operations can produce different rowechelon forms.
Linear Algebra - Chapter 1 [YR2005]
22
Back-substitution
 Bring the augmented matrix into row-echelon form
only and then solve the corresponding system of
equations by back-substitution.
 Example: [Solved by back substitution]
1
0

0

0
3 2
0 1
0 0
0 0
0
2
0
0
2
0
0
0
0 0
3 1 
1 13 

0 0
x1  3x2  2 x3  2 x5  0
x3  2 x4  3x6  1
x6  13
Step 1. x1  3x2  2 x3  2 x5
x3  1  2 x4  3x6
x6 
1
3
Linear Algebra - Chapter 1 [YR2005]
23
Back-Substitution
Step 3. Assign arbitrary values
to the free variables
[parameters], if any
Step 2.
Substituti ng x6 
1
3
x1  3x2  2 x3  2 x5
x1  3r  4 s  2t
x3  2 x4
x6 
1
x2  r
3
Substituti ng x3  2 x4
x1  3x2  4 x4  2 x5
x3  2 x4
x6 
1
x3  2 s
x4  s
x5  t
x6 
1
3
3
Linear Algebra - Chapter 1 [YR2005]
24
Homogeneous Linear Systems
 A system of linear equations is said to be
homogeneous if the constant terms are all zero.
a11x1  a12 x2    a1n xn  0
a21x1  a22 x2    a2 n xn  0




am1 x1  am 2 x2    amn xn  0
 Every homogeneous sytem of linear equations is
consistent, since all such systems have
x1=0,x2=0,...,xn=0 as a solution [trivial solution]. Other
solutions are called nontrivial solutions.
Linear Algebra - Chapter 1 [YR2005]
25
Homogeneous Linear Systems
 Example: [Gauss-Jordan Elimination]
2 x1  2 x2  x3  x5  0
 x1  x2  2 x3  3x4  x5  0
x1  x2  2 x3  x5  0
x3  x4  x5  0
2
 1

1

0
2
1
1
0
1
2
2
1
0
3
0
1
1
1
1
1
0 1
0 0


0 0
 
0  0
1
0
0
0
0
1
0
0
0
0
1
0
1
1
0
0
0
0
0

0
Linear Algebra - Chapter 1 [YR2005]
26
Homogeneous Linear Systems
The corresponding system of equations is
x1  x2  x5  0
x3  x5  0
x4  0
Solving for the leading variables yields
x1   x2  x5
x3   x5
x4  0
The general solution is
x1  s  t , x2  s, x3  t , x4  0, x5  t
The trivial solution is obtained when s=t=0
Linear Algebra - Chapter 1 [YR2005]
27
Homogeneous Linear Systems
 Theorem:
A homogeneous system of linear equations
with more unknowns than equations has
infinitely many solutions.
Linear Algebra - Chapter 1 [YR2005]
28
1.3 Matrices and Matrix Operations
Matrix Notation and Terminology
 A matrix is a rectangular array of numbers with rows
and columns.
 The numbers in the array are called the entries in the
matrix.
e   2 
1 2
 Examples: 

 1

 3 0, 2 1 0  3, 0
0
 1 4




1
2
0
1 ,  , 4
3
0   
The size of a matrix is described in terms of the
number of rows and columns its contains.
A matrix with only one column is called a column
matrix or a column vector.
A matrix with only one row is called a row matrix or
a row vector.
Linear Algebra - Chapter 1 [YR2005]
30
Matrix Notation and Terminology



aij = (A)ij = the entry in row i and column j of a
matrix A.
1 x n row matrix a = [a1 a2 ... an]
b 
m x 1 column matrix
1
b 
b 2 
 
 
bm 
 a11 a12  a1n 
a

 21 a22  a2 n 
 

 


an1 an 2  ann 

n x n matrix

A matrix A with n rows and n columns is called
a square matrix of order n. Main diagonal of A
= {a11, a22, ..., ann}
Linear Algebra - Chapter 1 [YR2005]
31
Operations on Matrices
 Definition:
Two matrices are defined to be equal if they have the
same size and their corresponding entries are equal.
 If A = [aij] and B = [bij] have the same size, then A=B if
and only if (A)ij=(B)ij, or equivalently aij=bij for all i and j.
 Definition:
 If A and B are matrices of the same size, then the sum
A+B is the matrix obtained by adding the entries of B to
the corresponding entries of A, and the difference A–B
is the matrix obtained by subtracting the entries of B
from the corresponding entries of A. Matrices of
different sizes cannot be added or subtracted.

Linear Algebra - Chapter 1 [YR2005]
32
Operations on Matrices

If A = [aij] and B = [bij] have the same size, then
(A+B)ij = (A)ij + (B)ij = aij + bij and
(A-B)ij = (A)ij – (B)ij = aij - bij
 Definition:
 If A is any matrix and c is any scalar, then the product
cA is the matrix obtained by multiplying each entry of
the matrix A by c. The matrix cA is said to be a scalar
multiple of A.
 If A = [aij], then (cA)ij = c(A)ij = caij.
Linear Algebra - Chapter 1 [YR2005]
33
Operations on Matrices
 Definition
If A is an mxr matrix and B is an rxn matrix,
then the product AB is the mxn matrix whose
entries are determined as follows. To find the
entry in row i and column j of AB, single out
row i from the matrix A and column j from the
matrix B. Multiply the corresponding entries
from the row and column together and then
add up the resulting products.
Linear Algebra - Chapter 1 [YR2005]
34
Partitioned Matrices
 A matrix can be subdivided or partitioned into smaller
matrices by inserting horizontal and vertical rules
between selected rows and columns.
 For example, nexts are three possible partitions of a
general 3 x 4 matrix A –



the first is a partition of A into four submatrices
A11,A12,A21, and A22.
The second is a partitioned of A into its row matrices
r1,r2, and r3.
The third is a partitioned of A into its column matrices
c1,c2,c3 and c4.
Linear Algebra - Chapter 1 [YR2005]
35
A

Partitioned Matrices
Linear Algebra - Chapter 1 [YR2005]
36
Matrix Multiplication
by Columns and by Rows
 Let A and B be matrices whose product is
noted as AB,
 J-th column of matrix AB = A [ j-th column
matrix of B],
 I-th row of AB = [ I-th row of matrix A] B
Linear Algebra - Chapter 1 [YR2005]
37
Matrix Products as Linear Combinations
 a11
a
A   21


 am1
then
a1n 
 x1 
x 
a2 n 
 and x   2 






amn 
 xn 
a12
a22
am 2
 a11 x1  a12 x2 
a x a x 
2
22
Ax   21 1


 am1 x1  am 2 x2 
 a12 
 a11 
a 
a 
21
  x  22  
x1 
2








a
a
2
m
1
m




 a1n xn 
 a2 n xn 
 


 amn xn 
 a1n 
a 
 xn  2 n 




a
mn


Linear Algebra - Chapter 1 [YR2005]
38
Matrix Form of a Linear System
a11 x1  a12 x2 
 a1n xn  b1
a21 x1  a22 x2 
 a2 n xn  b2
am1 x1  am 2 x2 
 amn xn  bm
this can be written as a product of
a1n   x1   b1 
 a11 a12
a
  x  b 
a
a
22
2n   2 
 21
 2

   

   
a
a
a
m2
mn   xm 
 m1
bm 
if we designated these matrices respectively as A,x and b then
Ax  b
A is called the coefficient matrix.
Linear Algebra - Chapter 1 [YR2005]
39
Transpose of a Matrix
 If A is any m x n matrix then the transpose of
A, denoted by AT, is defined to be the n x m
matrix that results from interchanging the
rows and the columns of A.
 For example
2 3
2 1 5


T
B  1 4  , B  
3 4 6 

 5 6 
or in general the element of the matrix can
be written as
BijT  B ji
Linear Algebra - Chapter 1 [YR2005]
40
1.4 Inverses; Rules of Matrix Arithmetic
Properties of Matrix Operations
 ab = ba for real numbers a & b, but AB ≠ BA
even if both AB & BA are defined and have
the same size.
 Example:
  1 0
1 2
A
,B  


 2 3
3 0
  1  2
 3 6
AB  
, BA  


11
4

3
0




Linear Algebra - Chapter 1 [YR2005]
42
Properties of Matrix Operations

Theorem: Properties of
h)
A+B = B+A
A+(B+C) = (A+B)+C
A(BC) = (AB)C
A(B+C) = AB+AC
(B+C)A = BA+CA
A(B-C) = AB-AC
(B-C)A = BA-CA
a(B+C) = aB+aC
i)
a(B-C) = aB-aC
a)
b)
c)
d)
e)
f)
g)
Math Arithmetic
(Commutative law for addition)
(Associative law for addition)
(Associative for multiplication)
(Left distributive law)
(Right distributive law)
j) (a+b)C = aC+bC
k) (a-b)C = aC-bC
l) a(bC) = (ab)C
m) a(BC) = (aB)C
Linear Algebra - Chapter 1 [YR2005]
43
Properties of Matrix Operations
 Proof (d):
 Proof for both have the same size:
 Let size A be r x m matrix, B & C be m x n (same
size).
 This makes A(B+C) an r x n matrix, follows that
AB+AC is also an r x n matrix.
 Proof that corresponding entries are equal:
 Let A=[aij], B=[bij], C=[cij]
 Need to show that [A(B+C)]ij = [AB+AC]ij for all values
of i and j.
 Use the definitions of matrix addition and matrix
multiplication.
Linear Algebra - Chapter 1 [YR2005]
44
Properties of Matrix Operations
[ A( B  C )]ij  ai1(b1 j  c1 j )  ai 2(b 2 j  c 2 j )    aim(bmj  cmj)
 (ai1b1 j  ai 2b 2 j    aimbmj)  (ai1c1 j  ai 2c 2 j    aimcmj)
 [ AB]ij  [ac]ij
 [ AB  AC ]ij
 Remark: In general, given any sum or any product of
matrices, pairs of parentheses can be inserted or
deleted anywhere within the expression without
affecting the end result.
Linear Algebra - Chapter 1 [YR2005]
45
Zero Matrices
 A matrix, all of whose entries are zero, such as
0 
0 0 0 
0 
0
0
0
0
0 0  


,
 , 0
,
0
0
0
,
0 0  




  0 0 0   0 0 0 0  0 
 


0 
 A zero matrix will be denoted by 0 or 0mxn for the mxn zero
matrix. 0 for zero matrix with one column.
 Properties of zero matrices:
 A+0=0+A=A
 A–A=0
 0 – A = -A
 A0 = 0; 0A = 0
Linear Algebra - Chapter 1 [YR2005]
46
Identity Matrices
 Square matrices with 1’s on the main diagonal and
0’s off the main diagonal, such as
1
0

1
0 
, 0

1
0
0
1
0
1
0 
0
0, 
0

1 
0
0
1
0
0
0
0
1
0
0
0
0

1
 Notation: In = n x n identity matrix.
 If A = m x n matrix, then:
 AIn = A and InA = A
Linear Algebra - Chapter 1 [YR2005]
47
Identity Matrices
 Example:
 a11 a12
A
 a 21 a 22
a13 
a 23
1 0  a11 a12 a13  a11 a12 a13
I 2A  

A




0 1  a 21 a 22 a 23  a 21 a 22 a 23
 a11 a12
AI 3  
 a 21 a 22
1 0 0
a13  
 a11 a12 a13 

0 1 0  
A



a 23
a 21 a 22 a 23

0 0 1
 Theorem: If R is the reduced row-echelon form of an
n x n matrix A, then either R has a row of zeros or R
is the identity matrix In.
Linear Algebra - Chapter 1 [YR2005]
48
Identity Matrices
 Definition: If A & B is a square matrix and same size
Э AB = BA = I, then A is said to be invertible and B is
called an inverse of A. If no such matrix B can be
found, then A is said to be singular.
 Example:
3 5
 2  5
B
,A


1
2

1
3




 2  5 3 5 1
AB  




 1 3  1 2 0
3 5  2  5 1
BA  




1 2  1 3  0
Linear Algebra - Chapter 1 [YR2005]
0
I

1
0
I

1
49
Properties of Inverses
 Theorem:
 If B and C are both inverses of the matrix A, then B = C.
 If A is invertible, then its inverse will be denoted by the
symbol A-1.
 The matrix
a b 
A

c
d


is invertible if ad-bc ≠ 0, in which case the inverse is
given by the formula
b 
 d

d

b




1
1
ad

bc
ad

bc
A 



a 
ad  bc  c a   c

 ad  bc ad  bc  50
Linear Algebra - Chapter 1 [YR2005]
Properties of Inverses
 Theorem: If A and B are invertible matrices of the
same size, then AB is invertible and (AB)-1 = B-1A-1.
 A product of any number of invertible matrices is
invertible, and the inverse of the product is the
product of the inverses in the reverse order.

Example:
1 2
 3 2
7 6
A
,
B

,
AB


 2 2
9 8
1
3






 4  3
1

1
3

2




1
1
A1  
,
B

,
(
AB
)
 9 7 



3
 2 2 
 1 1 
 1 2 
 1  1  3  2  4  3
1 1
B A 
 9 7 

3 

 1 2   1 1   2 2 
Linear Algebra - Chapter 1 [YR2005]
51
Powers of a Matrix
 If A is a square matrix, then we define the nonnegative integer
powers of A to be
A0=I
An = AA...A
(n>0)
n factors
 Moreover, if A is invertible, then we define the negative integer
prowers to be
A-n = (A-1)n = A-1A-1...A-1
n factors
 Theorem: Laws of Exponents
 If A is a square matrix, and r and s are integers, then ArAs =
Ar+s, (Ar)s=Ars
 If A is an invertible matrix, then



A-1 is invertible and (A-1)-1 = A
An is invertible and (An)-1 = (A-1)n for n = 0, 1, 2, ...
For any nonzero scalar k, the matrix kA is invertible and
(kA)-1 = 1/k A-1.
Linear Algebra - Chapter 1 [YR2005]
52
Powers of a Matrix
 Example:
1 2
A

1
3


 3  2
A 

 1 1 
1
1 2 1 2 1 2 11 30
A 







1 3 1 3 1 3 15 41
3
 3  2  3  2  3  2  41  30
A  (A )  








1
1

1
1

1
1

15
11



 

3
1 3
Linear Algebra - Chapter 1 [YR2005]
53
Polynomial Expressions Involving
Matrices
 If A is a square matrix, m x m, and if
p( x)  a0  a1 x    an x n
is any polynomial, then we define
p( A)  a0 I  a1 A    an An
 Example:
p ( x)  2 x 2  3 x  4
2

1
2


 1 2 1 0
2
p( A)  2 A  3 A  4 I  2
 3
 4



0
3
0
3
0
1



 

2 8   3 6 4 0 9 2 








0
18
0
9
0
4
0
13

 
 
 

Linear Algebra - Chapter 1 [YR2005]
54
Properties of the Transpose
 Theorem: If the sizes of the matrices are
such that the stated operations can be
performed, then
a)
b)
c)
d)
((A)T)T = A
(A+B)T = AT + BT and (A-B)T = AT – BT
(kA)T = kAT, where k is any scalar
(AB)T = BTAT
 The transpose of a product of any number of
matrices is equal to the product of their
transpose in the reverse order.
Linear Algebra - Chapter 1 [YR2005]
55
Invertibility of a Transpose
 Theorem: If A is an
invertible matrix, then AT
is also invertible and
(AT)-1 = (A-1)T
 Example:
 5  3 T  5 2
A
,A 


2
1

3
1




3  1 T 1  2 T 1 1  2
1
1
A 
,(A )  
,(A )  




2

5
3

5
3

5






Linear Algebra - Chapter 1 [YR2005]
56
Exercise
 Show that if a square matrix A satisfies A2-3A+I=0,
then A-1=3I-A
 Let A be the matrix
1
1

0
0
1
1
1
0
1
Determine whether A is invertible, and if so, find its
inverse. [Hint. Solve AX = I by equating
corresponding entries on the two sides.]
Linear Algebra - Chapter 1 [YR2005]
57
1.5 Elementary Matrices and
a Method for Finding A-1
Elementary Matrices

Definition:
An n x n matrix is called an elementary matrix if
it can be obtained from the n x n identity matrix In
by performing a single elementary row
operation.
1 0 0 0
1 0 3


Example:
0 0 0 1
1 0 




,3 : 0 1 0
1: 
,2 : 



0  3 0 0 1 0 0 0 1

 

0
1
0
0


2.
Multiply the second row of I2 by -3.
Interchange the second and fourth rows of I4.
3.
Add 3 times the third row of I3 to the first row.
1.
Linear Algebra - Chapter 1 [YR2005]
59
Elementary Matrices
 Theorem: (Row Operations by Matrix Multiplication)
If the elementary matrix E results from performing a certain
row operation on Im and if A is an m x n matrix, then the
product of EA is the matrix that results when this same row
operation is performed on A.
 Example:

1 0 2 3
1 0 0 
A   2 1 3 6  , E  0 1 0 
1 4 4 0 
 3 0 1 
1 0 2 3
EA   2 1 3 6 
 4 4 10 9 

EA is precisely the same matrix that results when we add 3
times the first row of A to the third row.
Linear Algebra - Chapter 1 [YR2005]
60
Elementary Matrices
 If an elementary row operation is applied to an
identity matrix I to produce an elementary matrix E,
then there is a second row operation that, when
applied to E, produces I back again.
 Inverse operation
Row operation on I that
produces E
Row operation on E that
reproduces I
Multiply row i by c ≠ 0
Multiply row i by 1/c
Interchange rows i and j
Interchange rows i and j
Add c times row i to row j
Add –c times row i to row j
Linear Algebra - Chapter 1 [YR2005]
61
Elementary Matrices
 Theorem: Every elementary matrix is invertible,
and the inverse is also an elementary matrix.
 Theorem: (Equivalent Statements)

If A is an n x n matrix, then the following
statements are equivalent, that is, all true or all
false.
a)
b)
c)
d)
A is invertible
Ax = 0 has only the trivial solution.
The reduced row-echelon form of A is In.
A is expressible as a product of elementary
matrices.
Linear Algebra - Chapter 1 [YR2005]
62
Elementary Matrices

Proof:
(a )  (b)  (c)  (d )  (a)
(a)  (b)
Assume A is invertible and let x0 be any solution of Ax=0.
A1 ( Ax0 )  A1 0, ( A1 A) x0  0, Ix0  0, x0  0
(b)  (c)
Let Ax=0 be the matrix form of the system
a11 x1    a1n xn  0

an1 x1    ann xn  0
 a11 a12  a1n
a
 21 a22  a2 n
 



an1 an 2  ann
1
0 
0


0
0
 


0 
 0
0
0

0
1
0

0
0

1


0

0
0
0
1
Linear Algebra - Chapter 1 [YR2005]
0
0 
0


0 
63
Elementary Matrices
(c )  ( d )
Assumed that the reduced row-echelon form of A is In by a finite
sequence of elementary row operations, such that:
Ek  E2 E1 A  I n
By theorem, E1,…,En are invertible. Multiplying both sides of equation
on the left we obtain:
A  E11 E21  Ek1 I n  E11 E21  Ek1
This equation expresses A as a product of elementary matrices.
(d )  (a)
If A is a product of elementary matrices, then the matrix A is a product
of invertible matrices, and hence is invertible.
 Matrices that can be obtained from one another by a finite
sequence of elementary row operations are said to be row
equivalent.
 An n x n matrix A is invertible if and only if it is row equivalent to
the n x n identity matrix.
Linear Algebra - Chapter 1 [YR2005]
64
A Method for Inverting Matrices
 To find the inverse of an invertible matrix, we must
find a sequence of elementary row operations that
reduces A to the identity and then perform this same
sequence of operations on In to obtain A-1.
1 2 3
 Example:


A  2 5 3
1 0 8
 Adjoin the identity matrix to the right side of A,
thereby producing a matrix of the form [A|I]
 Apply row operations to this matrix until the left side
is reduced to I, so the final matrix will have the form
[I|A-1].
Linear Algebra - Chapter 1 [YR2005]
65
A Method for Inverting Matrices
1
2


1
2
5
0
1
0


0
2
1
2
1
0


0
2
1
0
1
0


0
2
1
0
3 2
1
5
1
0


0
2
1
0
0  14
0 13
1 5
6
5
2
3 
 3

 1

1
0


0
0
1
0
0  40
0 13
1
5
16
5
2
9 
 3

 1

31
30
80
0
0

1

0
1
0
3
1
3 2
5 1
3
1
3 2
1  5
3
1
0
1
0
0
1
2
0
1
2
0
0

1

0
0

1

0 
0 

 1

Added –2 times the first row to the
second and –1 times the first row to the
third.
Added 2 times the second row to the
third.
Multiplied the third row by –1.
Added 3 times the third row to the
second and –3 times the third row to the
first.
We added –2 times the second row to
the first.
Linear Algebra - Chapter 1 [YR2005]
66
A Method for Inverting Matrices
 Often it will not be known in advance whether a given matrix is
invertible.
 If elementary row operations are attempted on a matrix that is
not invertible, then at some point in the computations a row of
zeros will occur on the left side.
1 6 4
 Example:
A   2 4 1 
1

2
 1
1

0
0
1

0
0
6
4
2
4 1
1 0
5 0
0

0
1
0
1
0
6
8
8
4 1
9 2
9 1
0
1
0
0

0
1
6
8
0
4 1
9 2
0 1
1
1

0
1
 1 2  5
Added -2 times the first row to the second and
added the first row to the third.
0 0 Added the second row to the third.
Linear Algebra - Chapter 1 [YR2005]
67
Exercises

Consider the matrices
1
5
1
3 4
8 1
3 4
A  2  7  1, B  2  7  1, C  2  7  1
8 1
3 4
2  7 3 
5 
1 
Find elementary matrices, E1, E2, E3, and E4, such
that
a.
b.
c.
d.
E1A=B
E2B=A
E3A=C
E4C=A
Linear Algebra - Chapter 1 [YR2005]
68
Exercises
 Express the matrix:
1 7 8
0
1

3
3
8


 2  5 1  8
in the form A = E F G R, where E, F, G are
elementary matrices, and R is in row-echelon form.
Linear Algebra - Chapter 1 [YR2005]
69
1.6 Further Results on
Systems of Equations and Invertibility
Linear Systems
 Theorem:
Solving Linear Systems by Matrix Inversion:
If A is an invertible n x n matrix, then for each n x 1
matrix b, the system of equations Ax = b has exactly
one solution, namely, x = A-1b.
 Linear systems with a common coefficient matrix.
Ax=b1, Ax=b2, Ax=b3, ..., Ax=bk
If A is invertible, then the solutions
x1=A-1b1, x2=A-1b2, x3=A-1b3, ..., xk=A-1bk
This can be efficiently done using Gauss-Jordan
Elimination on [A|b1|b2|...|bk]
Linear Algebra - Chapter 1 [YR2005]
71
Linear Systems
 Example: (a) x1  2 x2  3x3  4 (b) x1  2 x2  3 x3  1
2 x1  5 x2  3x3  5
2 x1  5 x2  3 x3  6
x1  8 x3  9
1

2
1

2
5
1

0
0

0
0
1
0
x1  8 x3  6
1 

6 
8 9  6

3 4
3 5
2 

0 0 1 
1 1  1

0 1
The solution:
(a) x1=1, x2=0, x3=1
(b) x1=2, x2=1, x3=-1
Linear Algebra - Chapter 1 [YR2005]
72
Properties of Invertible Matrices

Theorem: Let A be a square matrix.
a)
b)

If B is a square matrix satisfying BA=I, then B=A-1.
If B is a square matrix satisfying AB=I, then B=A-1.
Theorem: Equivalent Statements
a)
b)
c)
d)
e)
f)
A is invertible
Ax=0 has only the trivial solutions
The reduced row-echelon form of A is In
A is expresssible as a product of elementary matrices
Ax=b is consistent for every n x 1 matrix b
Ax=b has exactly one solution for every n x 1 matrix b
Linear Algebra - Chapter 1 [YR2005]
73
Properties of Invertible Matrices
 Theorem: Let A and B be square matrices of
the same size. If AB is invertible, then A and B
must also be invertible.
 A fundamental problem.
Let A be a fixed m x n matrix. Find all m x 1
matrices b such that the system of equations
Ax=b is consistent.
Linear Algebra - Chapter 1 [YR2005]
74
Exercises
 Solve the system by inverting the coefficient matrix.
 x  2 y  3z  0
w  x  4 y  4z  7
w  3x  7 y  9 z  4
 w  2x  4 y  6z  6
 Find condition that b’s must satisfy for the system to
be consistent.
6 x1  4 x2  b1
3x1  2 x2  b2
Linear Algebra - Chapter 1 [YR2005]
75
1.7 Diagonal, Triangular,
and Symmetric Matrices
Diagonal Matrices
 A square matrix in which all the entries off the main
6 0
1 0 0 
2 0  
 , 0  4
,
0
1
0
0  5 


 0 0 1  0 0

 0 0

diagonal are zero. Example:
0
0
0
0
0
0
0

8
 A diagonal matrix is invertible if and only if all of its
diagonal entries are nonzero.
d1 0  0 
0 d  0 
2

D
 



0
0

d
n




D 1  



1
d1
0

0
0
1
d2

0



d1k
0 


0  k  0
 D 
 
 

1 
 0
d n 
Linear Algebra - Chapter 1 [YR2005]
0
k
d2

0
0 

0 
 
k
 d n 


77
Diagonal Matrices
 Example:
1 0 0
A  0  3 0
0 0 2
d1 0
0 d
2

 0 0
 a11
a
 21
a31
A 1

1

 0

0


0
1

3
0   a11 a12
0  a21 a22
d 3  a31 a32
a12 a13  d1
a22 a23   0
a32 a33   0
0

0

0

1
2

1
A5  0
0
0
3
0
a14   d1a11
a23 a24   d 2 a21
a33 a34   d 3a31
0 0   d1a11
d 2 0   d1a21
0 d 3  d1a31
a13
Linear Algebra - Chapter 1 [YR2005]
0
0
1
0 A5  0  1 243

2
0
0
d1a12
d1a13
d 2 a22 d 2 a23
d 3 a32
d 3 a33
d 2 a12
d 3 a13 
d 3a23 
d 3a33 
d 2 a22
d 2 a32
0
0 
1 
32
d1a14 
d 2 a24 
d 3 a34 
78
Triangular Matrices
 Lower triangular = a square matrix in which all the
entries above the main diagonal are zero.
 Upper triangular = a square matrix in which all the
entries under the main diagonal are zero.
 Triangular = a matrix that is either upper triangular or
lower triangular.
a11 a12
0 a
22

0
0

0
0
a13
a23
a33
0
a14 
a24 
a34 

a44 
 a11 0
a
 21 a22
a31 a32

a41 a42
0
0
a33
a43
Linear Algebra - Chapter 1 [YR2005]
0
0 
0

a44 
79
Triangular Matrices
 Theorem: (basic properties of triangular matrices)
 The transpose of a lower triangular matrix is upper
triangular, and the transpose of an upper triangular
matrix is lower triangular.
 The product of lower triangular matrices is lower
triangular, and the product of upper triangular matrices
is upper triangular.
 A triangular matrix is invertible if and only its diagonal
entries are all nonzero.
 The inverse of an invertible lower triangular matrix is
lower triangular, and the inverse of an invertible upper
triangular matrix is upper triangular.
Linear Algebra - Chapter 1 [YR2005]
80
Triangular Matrices
1 3  1
3  2 2 
A  0 2 4 , B  0 0  1
0 0 5 
0 0 1 
The matrix A is invertible, since its diagonal entries are
nonzero, but the matrix B is not.
 Example:

1  3 2 7 5 
 3  2  2
A1  0 1 2  2 5 , AB  0 0 2 
1 
0 0
0 0 5 
5 


This inverse is upper triangular.
This product is upper triangular.
Linear Algebra - Chapter 1 [YR2005]
81
Symmetric Matrices
 A square matrix A is called symmetric if A = AT.
d1 0
1
4
5

 
0 d2
7

3

 


 3 5 , 4  3 0,  0 0

 5 0 7 

 0 0

0
0
d3
0
0
0 
0

d4 
 A matrix A = [aij] is symmetric if and only if aij=aji for all
values of i and j.
1 4 5 
4  3 0


5 0 7 
Linear Algebra - Chapter 1 [YR2005]
82
Symmetric Matrices
 Theorem: If A and B are symmetric matrices
with the same size, and if k is any scalar, then



AT is symmetric
A+B and A-B are symmetric
kA is symmetric
 Theorem:
 If A is an invertible matrix, then A-1 is
symmetric.
 If A is an invertible matrix, then AAT and ATA
are also invertible.
Linear Algebra - Chapter 1 [YR2005]
83
Exercise
 Find all values of a, b, and c for which A is symmetric.
2 a  2b  2c 2a  b  c 
A  3
5
a  c 
0
2
7 
 Find all values of a and b for which A and B are both
not invertible.
0
 a  b  1 0
5

A
,B  


0
3
0
2
a

3
b

7




Linear Algebra - Chapter 1 [YR2005]
84