Download An Overview of Matrix Algebra

3_3 An Useful Overview of
Matrix Algebra
Definitions
Operations
SAS/IML matrix
commands
What is it?
Matrix algebra is a means of making
calculations upon arrays of numbers (or
data).
Most data sets are matrix-type
Why use it?
Matrix algebra makes mathematical
expression and computation easier.
It allows you to get rid of cumbersome
notation, concentrate on the concepts
involved and understand where your results
come from.
Definitions - scalar
 a scalar is a number
– (denoted with regular type: 1 or 22)
Definitions - vector
Vector: a single row or column of numbers
– denoted with bold small letters
– row vector
a = 1 2 3 4 5
– column vector
b=
1 
2
3
4
 5 
Definitions - Matrix
A matrix is an array of numbers
A=
 a11 a12 a13
a 21 a 22 a 23
Denoted with a bold Capital letter
All matrices have an order (or
dimension):
that is, the number of rows  the number
of columns. So, A is 2 by 3 or (2  3).
Definitions
A square matrix is a matrix that has the
same number of rows and columns (n  n)
Matrix Equality
 Two matrices are equal if and only if
– they both have the same number of rows and
the same number of columns
– their corresponding elements are equal
Matrix Operations
Transposition
Addition and Subtraction
Multiplication
Inversion
The Transpose of a Matrix: A'
The transpose of a matrix is a new matrix
that is formed by interchanging the rows
and columns.
The transpose of A is denoted by A' or (AT)
Example of a transpose
Thus,
 a11 a12 
A  a21 a22 
a a 
 31 32 
 a11 a21 a31 
A'  
a12 a22 a32 
If A = A', then A is symmetric
Addition and Subtraction
Two matrices may be added (or subtracted)
iff they are the same order.
Simply add (or subtract) the corresponding
elements. So, A + B = C yields
Addition and Subtraction (cont.)
a11 a12  b11 b12  c11 c12 
a
  b
  c

a
b
c
 21 22   21 22   21 22 
a31 a32  b31 b32  c31 c32 
Where
a11  b11  c11
a12  b12  c12
a21  b21  c 21
a22  b22  c 22
a31  b31  c31
a32  b32  c32
Matrix Multiplication
To multiply a scalar times a matrix, simply
multiply each element of the matrix by the
scalar quantity
 a11 a12   ka11 ka12 
k


a21 a22  ka21 ka22 
Matrix Multiplication (cont.)
To multiply a matrix times a matrix, we
write
• AB (A times B)
This is pre-multiplying B by A, or post-
multiplying A by B.
Matrix Multiplication (cont.)
In order to multiply matrices, they must be
CONFORMABLE
that is, the number of columns in A must
equal the number of rows in B
So,
A  B = C
(m  n)  (n  p) = (m  p)
Matrix Multiplication (cont.)
(m  n)  (p  n) = cannot be done
(1  n)  (n  1) = a scalar (1x1)
Matrix Multiplication (cont.)
Thus
where
a11 a12 a13  b11 b12  c11 c12 
a a
 x b b   c

a
c
 21 22 23   21 22   21 22 
a31 a32 a33  b31 b32  c31 c32 
c11  a11b11  a12b21  a13b31
c12  a11b12  a12b22  a13b32
c 21  a21b11  a22b21  a23b31
c 22  a21b12  a22b22  a23b32
c31  a31b11  a32b21  a33b31
c32  a31b12  a32b22  a33b32
Matrix Multiplication- an
example
Thus
1 4 7 1 4 c11 c12  30 66
2 5 8 x 2 5  c
  36 81
c

 
  21 22  

3 6 9 3 6 c31 c32  42 96
where
c11  1 * 1  4 * 2  7 * 3  30
c12  1 * 4  4 * 5  7 * 6  66
c 21  2 * 1  5 * 2  8 * 3  36
c 22  2 * 4  5 * 5  8 * 6  81
c31  3 * 1  6 * 2  9 * 3  42
c32  3 * 4  6 * 5  9 * 6  96
Properties
AB does not necessarily equal BA
(BA may even be an impossible operation)
For example,
A
(2  3)
B
(3  2)

B
 (3  2)

A
 (2  3)
= C
= (2  2)
= D
= (3  3)
Properties
Matrix multiplication is Associative
A(BC) = (AB)C
Multiplication and transposition
(AB)' = B'A'
A popular matrix: X'X
1 x11 
1 x12 
X  
  
1 x 
1n 

X' X
1 1
 
 x11 x12
1 x11 


1 x12 
 1

  
  
 x1n 


1 x 
 1n 


 n
n
 x
1i

i 1

x1i 

i 1

n
2
x1i


i 1
n
Another popular matrix: e'e
e

e' e
 e1 
e 
 2

 
en 

e1
e2
 en 
 e1 
e 
  2

 
en 

n
2
e
i
i 1
Special matrices
There are a number of special matrices
– Diagonal
– Null
– Identity
Diagonal Matrices
– A diagonal matrix is a square matrix that has
values on the diagonal with all off-diagonal
entities being zero.
0
0
a11 0
0 a

0
0
22


0
0 a33 0 


0
0 a44 
0
Identity Matrix
An identity matrix is a diagonal matrix
where the diagonal elements all equal one.
I=
1
0

0

0
0
1
0
0
0
0
1
0
0
0
0

1
A I =A
Null Matrix
 A square matrix where all elements equal zero.
0
0

0

0
0
0
0
0
0
0
0
0
0
0
0

0
The Determinant of a Matrix
The determinant of a matrix A is denoted by
|A| (or det(A)).
Determinants exist only for square matrices.
They are a matrix characteristic, and they
are also difficult to compute
The Determinant for a 2x2 matrix
If A =
a11 a12 
a

 21 a22 
Then
A  a11a22  a12a21
Properties of Determinates
 Determinants have several mathematical
properties which are useful in matrix
manipulations.
– 1 |A|=|A'|.
– 2. If a row or column of A = 0, then |A|= 0.
– 3. If every value in a row or column is multiplied by
k, then |A| = k|A|.
– 4. If two rows (or columns) are interchanged the
sign, but not value, of |A| changes.
– 5. If two rows or columns are identical, |A| = 0.
– 6. If two rows or columns are linear combination of
each other, |A| = 0
Properties of Determinants
– 7. |A| remains unchanged if each element of a
row or each element multiplied by a constant, is
added to any other row.
– 8. |AB| = |A| |B|
– 9. Det of a diagonal matrix = product of the
diagonal elements
Rank
 The rank of a matrix is defined as
 rank(A) = number of linearly independent rows
= the number of linearly independent columns.
 A set of vectors is said to be linearly dependent if
scalars c1, c2, …, cn (not all zero) can be found
such that
c1a1 + c2a2 + … + cnan = 0
For example,
a = [1 21 12] and b = [1/3 7 4] are
linearly dependent
A matrix A of dimension n  p (p < n) is of
rank p. Then A has maximum possible rank
and is said to be of full rank.
In general, the maximum possible rank of
an n  p matrix A is min(n,p).
The Inverse of a Matrix
-1
(A )
 For an n  n matrix A, there may be a B such that
AB = I = BA.
 The inverse is analogous to a reciprocal
 A matrix which has an inverse is nonsingular.
 A matrix which does not have an inverse is
singular.
 An inverse exists only if
A 0
Properties of inverse matrices

 AB 
 B A

 A' 

A 


1
1
-1 1
-1
-1
A 
-1
A
'
How to find inverse matrixes?
determinants? and more?
If A
A
-1
a b 
 

c d 

and |A|  0
1
det( A)
 d  b
 c a 


Otherwise, use SAS/IML an easier way