Download Objective: introduce basic concepts and skills in matrix algebra

Webpage:
http://mail.thu.edu.tw/~wenwei/course-notes.htm
or
http://140.128.104.155/wenwei/course-notes.htm
Then, click on “Matrix Algebra”
Objective: introduce
basic concepts and skills in matrix
algebra. In addition, some applications of
matrix algebra in statistics are described.
Section 1. Introduction and Matrix Operations
Definition of r  c matrix:
An r  c matrix A is a rectangular array of rc real numbers arranged in r
rizontal rows and c ertical columns:
 a11
a
A   21
 

 ar1
a12
a22

ar 2
The i’th row of A is




a1c 
a2 c 
 .

arc 
rowi ( A)  ai1 ai 2  aic , i  1,2,, r, ,
and the j’th column of A is
1
 a1 j 
a 
2j
col j ( A)   , j  1,2,, c.
  
 
 arj 
We often write A as
 
A  aij  Ar c .
Matrix operations:
Let
A  Arc
 a11 a12
a
a
 [aij ]   21 22
 


 ar 1 ar 2
 a1c 
 a2c 
  ,

 arc 
B  Bcs
b11 b12
b b
 [bij ]   21 22



bc1 bc 2
 b1s 
 b2 s 
  ,

 bcs 
D  Drc
 d11 d12
d
d
 [d ij ]   21 22
 


 d r1 d r 2
 d1c 
 d 2c 
  .

 d rc 
Then,
1.
Matrix addition, scalar multiplication and transpose of a matrix:
2
 (a11  d11 ) (a12  d12 )
( a  d ) ( a  d )
21
22
22
A  D  [aij  d ij ]   21




 (ar1  d r1 ) ( ar 2  d r 2 )
 pa11
 pa
pA  [ paij ]   21
 

 par1
pa12
pa22

par 2
 (a1c  d1c ) 
 (a2c  d 2c )
,



 (arc  d rc ) 
 pa1c 
 pa2c 
, p  R.

 

 parc 
and the transpose of A is denoted as
At  Actr
 a11 a21
a
a
 [a ji ]   12 22



a1c a2c
 ar1 
 ar 2 
  

 arc 
Example 1:
Let
 1
A
 4
3
5
3
1
B

and
8
0

7
1
0
.
1
Then,
 1 3
A B  
 4  8
 1 2
2A  
 4  2
37
5 1
3 2
5 2
1  0  4

0  1
 4
1 2   2

0  2
  8
and
3
4
6
6
10
1
,
1

2
0

1  4

At  
 3 5 .

1 0 

2.
Matrix multiplication:
We first define the dot product or inner product of n-vectors.
Definition of dot product:
The dot product or inner product of the n-vectors
a  a1 a2  ac 
and
 b1 
b 
b   2
  ,
 
bc 
are
c
a  b  a1b1  a2b2    ac bc   ai bi
i 1
.
Example 1:
4
Let a  1 2 3 and b  5 . Then, a  b  1 4  2  5  3  6  32 .
 
6
Definition of matrix multiplication:
E  E r s
 e11 e12
e
e
 eij   21 22



er1 er 2
 
 e1s 
 e2 s 
  

 ers 
4
 row1 ( A)  col1 ( B) row1 ( A)  col2 ( B)
row ( A)  col ( B) row ( A)  col ( B)
1
2
2
 2




rowr ( A)  col1 ( B) rowr ( A)  col2 ( B)
 row1 ( A)  col s ( B) 
 row2 ( A)  col s ( B)




 rowr ( A)  col s ( B)
 row1 ( A) 
row ( A)
2
col ( B ) col ( B)  col ( B )

2
s
 
 1


row
(
A
)
r


 a11 a12
a
a22
  21
 


 ar1 ar 2
 a1c  b11 b12
 a2 c  b21 b22
   


 arc  bc1 bc 2
 b1s 
 b2 s 
 Arc Bcs
  

 bcs 
That is,
eij  rowi ( A)  col j ( B)  ai1b1 j  ai 2 b2 j    aicbcj , i  1,, r , j  1,, s.
Example 2:
1 2 
0 1 3 
A22  
,
B

 23  1 0  2 .
3  1


Then,
 row ( A)  col1 ( B) row1 ( A)  col 2 ( B) row1 ( A)  col3 ( B)   2 1  1
E23   1


row2 ( A)  col1 ( B) row2 ( A)  col 2 ( B) row2 ( A)  col3 ( B)  1 3 11 
since
0
0
row1 ( A)  col1 ( B)  1 2   2 , row2 ( A)  col1 ( B)  3  1   1
 1
 1
1
row1 ( A)  col2 ( B )  1 2   1 , row2 ( A)  col 2 ( B)  3  11  3
0 
0 
5
 3 
row1 ( A)  col3 ( B)  1 2   1 , row2 ( A)  col3 ( B)  3
  2
 3 
 1   11 .
  2
Example 3
a31
1
1
 4 5 




 2, b12  4 5  a 31b12  24 5   8 10 
3
3
12 15
Another expression of matrix multiplication:
Arc Bcs
 row1 ( B) 
row ( B)
 col1 ( A) col 2 ( A)  col c ( A) 2 
  


rowc ( B) 
c
 col1 ( A)row1 ( B)  col 2 ( A)row2 ( B)    col c ( A)rowc ( B)   coli ( A)rowi ( B)
i 1
where coli ( A)rowi ( B) are r  s matrices.
Example 2 (continue):
 row ( B) 
A22 B23  col1 ( A) col 2 ( A) 1   col1 ( A)row1 ( B)  col 2 ( A)row2 ( B)
row2 ( B)
1
2
 0 1 3   2 0  4  2 1  1
  0 1 3    1 0  2  
   1 0 2    1 3 11 
3

1
0
3
9
 
 

 
 

Note:
 row1 ( A) 
row ( A)
2
 and
Heuristically, the matrices A and B, 





 rowr ( A) 
6
col1 ( B)
col 2 ( B)  col s ( B) , can be thought as r  1 and 1  s vectors.
Thus,
Arc Bcs
 row1 ( A) 
row ( A)
2
col ( B) col ( B)  col ( B)

2
s
 
 1


row
(
A
)
r


can be thought as the multiplication of r  1 and 1  s vectors. Similarly,
Arc Bcs
 row1 ( B) 
row ( B)
2

 col1 ( A) col2 ( A)  colc ( A)
 



row
(
B
)
c


can be thought as the multiplication of 1  c and c  1 vectors.
Note:
I.
AB
is not necessarily equal to
1
A
2
3
2
and
B

0
 1


II.
AC  BC
BA . For instance,
 1
2 
 2 5  0 7 
AB  
  4  2  BA .
4

4

 

 A
might be not equal to
1 3
2
A  , B  
0 1
2
 2
 AC  
1
B . For instance,
4
 1  2
and
C

 1 2 
3


4
 BC but A  B
2
7
III.
AB  0 , it is not necessary that A  0

IV.
1
A
1
0
AB  
0
or
B  0 . For instance,
1
 1  1
and
B

 1 1 
1


0
 BA but A  0, B  0.
0
A p  A  A A , A p  Aq  A pq , ( A p ) q  A pq
p factors
Also, ( AB) p is not necessarily equal to A p B p .
V.
 AB
3.
The trace of a matrix:
t
 B t At .
Definition of the trace of a matrix:
The sum of the diagonal elements of a r  r square matrix is called the trace of
the matrix, written
tr ( A) , i.e., for
 a11
a
A   21
 

 a r1
a12
a 22




ar 2

a1r 
a 2 r 
 ,

a rr 
r
tr ( A)  a11  a22    arr   aii
i 1
8
.
Example 4:
1 5 6 
Let A  4 2 7 . Then, tr ( A)  1  2  3  6 .


8 9 3
Homework 1
1. Prove
tr ( AB)  tr ( BA ) , where A and B are
r  c and c r
matrices, respectively.
2.
(a) When does
 A  B  A  B   A2  B 2 ?
tr( AB)  tr( AB t )
(b) When
A t  A.
(c) When
X t XGX t X  X t X
Prove
, prove
9
X t XG t X t X  X t X