Download B632_06lect8a

Matrix Algebra, or:
Is this torture really necessary?!
• What for?
– Permits compact, intuitive depiction of regression
analysis
– Flexible, in that it can handle any number of
independent variables
– Generally used in statistical presentation, for OLS and
other techniques
• You need to be able to interpret it.
March 7, 2006
Lecture 8a
Slide #1
The Basics
• Matrix form:
5 8 
10
A  
,
where
a

0
23
12 1 0

Elements of a matrix can be indentified as
:
a11 a12 a13 
A  
a21 a22 a23 

March 7, 2006
Lecture 8a
Slide #2
Transpose (“prime”) of a Matrix
5 8 
10
A  
12 1 0

10
A  5

8
March 7, 2006
12 
1 
0 

Lecture 8a
Slide #3
Vectors
• Vectors are essentially single rows or
columns:
6 
1
Y   
8

11

Y  6 1 8 11
March 7, 2006
Lecture 8a
Slide #4
Adding Matrices
Addition works only if matrices have the same dimension:
X1  X 2
4 3 8 1 
 

 
2 0  4 5
4  8 3  1  12 2 
 

 
2  4 0  (5) 6 5 
March 7, 2006
Lecture 8a
Slide #5
Multiplication of Matrices
Dimensions: A(r*q) * B(q*c) = C(r*c),
So the number of columns in the first matrix must match
the number of rows in the second matrix
2 5 

 4 2 1

1 0  

 5 7 2
6 2 
(2  4)  (5  5)
(2  2)  (5  7) (2  1)  (5  2) 


  (1  4)  (0  5)
(1 2)  (0  7)
(1  1)  (0  2) 


(6  4)  (2  5) (6  2)  (2  7) (6  1)  (2  2)
33 39 12


 4 2 1 


14
2
2


March 7, 2006
Lecture 8a
Slide #6
Rules for Matrix Multiplication
• Are matrices conformable?
A x B
=
(r x q) (q x c)
C
(r x c)
• Vector times a matrix:
A x B
=
C
(r x c) (c x 1)
(r x 1)
• Row and column vectors:
March 7, 2006
A x B
=
(r x 1) (1 x p)
C
(r x p)
A x B
(1 x r) (r x 1)
C
(1 x 1) a scalar
=
Lecture 8a
Slide #7
Identity Matrices
Square matrices with 1’s on diagonal and 0’s elsewhere:
1
0
I4  
0

0
0 0
1 0
0
0
0 1 0
0 0 1 

4 x 4 identity matrix
Identity matrices act like 1’s in familiar algebra:
I x B
=
(r x r) (r x c)
March 7, 2006
B
(r x c)
Lecture 8a
Slide #8
Matrix Inversion
Acts a bit like division in algebra: any matrix multiplied
by its inverse is equal to the identity matrix:
1
1
CC  C C  I
The text uses the following example:
10 4 
0.112245 0.04082
1
C  
so C  

3 11
0.03061 0.102041

Inversion works only for square matrices
March 7, 2006
Lecture 8a
Slide #9
Finding the Identity Matrix:
An Example
3 1  1 a
C  

C


2 4 

b
c  1 0 

I
d 
 
0 1 

2a + 4b = 0 so 2a = -4b and a = -2b
3a + b = 1 so 3(-2b) + b = 1, and -5b=1 so b = -1/5
Therefore: a = -2(-1/5) so a = 2/5
3c + d = 0 so d = -3c
2c + 4d = 1 so 2c + 4(-3c) = 1 and -10c = 1 so c = -1/10
Therefore d = -3(-1/10) so d = 3/10
March 7, 2006
Lecture 8a
Slide #10
Example Continued
Now we can check and see the result of C x C-1:
 2  1 
3 1
1
5
10


C 
C
1
3


2
4




 5 10 
( 2  3)  ( 1  2) ( 2  1)  ( 1  4) 
10
5
10

  5
( 1  3)  ( 3  2) ( 1  1)  ( 3  4)
 5

10
5
10
1 0
 
0 1

March 7, 2006
Lecture 8a
Slide #11
Regression in Matrix Form
• Assume a model using n observations, with K-1
Xi (independent) variables
Y (n  1) is a column vector of the observed dependent variable
Yˆ (n  1) is a column vector of predicted Y values
X (n  K) each column is of observations on an X,
first column 1' s
B (K  1) a row vector of regression coefficients (first is b
0
)
U (n  1) is a column vector of n residual values
March 7, 2006
Lecture 8a
Slide #12
Regression in Matrix Form
Y  XB  U
Yˆ  XB
B  ( X X) 1 X Y
Note: we can’t uniquely define (X’X) if any
column in the X matrix is a linear function of
any other column(s) in X. Why is that?
March 7, 2006
Lecture 8a
Slide #13
The X’X Matrix
 n
 X1
( X X)  
X2


 X3
X X
 X 
 X  X X  X X 

X
X
X
X
X


 
 X X  X X  X 
1
2
1
2
3
2
1
2
2
2
1
1
3
3
2
1
3
2
3
2
3
Note that you can obtain the basis for all the
necessary means, variances and covariances
from the (X’X) matrix
March 7, 2006
Lecture 8a
Slide #14
An Example of Matrix Regression
Using a sample of 7 observations, where X has
Elements {X0, X1, X2, X3}
6
1 4 5 4
 6.48 
 0.48
11
1 7 2 3
10.02
 0.98 
 






4
1 2 6 4
 4.41 
  0.41
 






Y   3  X  1 1 9 6 Ŷ =  2.51  U   0.49  B = 3.96 1.06 0.04  0.49
5
1 3 4 5
 4.89 
 0.11 
 






9
1 7 3 4
 9.58 
 0.58
10
1 8 2 5
10.11
  0.11
 






March 7, 2006
Lecture 8a
Slide #15
New Dataset: Scientists
• Sample of AAAS members
– US and EU
– Collected in 2002
– Focus on science, security, GCC…
• Available on the class data page
– See the codebook and do file
March 7, 2006
Lecture 8a
Slide #16
Application of Multivariate
Regression Analysis
• Predict expected temperature change (c4_34_tc),
using the following independent variables:
–
–
–
–
–
Age (c5_3_age)
Gender (c5_4_gen)
Ideology (c4_1_ide))
Fragile nature (c4_2_nat)
US or EU (recoded: usa_c)
• Run the model
• Evaluate the Output
• Draw Initial Conclusions
March 7, 2006
Lecture 8a
Slide #17
Break for analysis...
• Feel free to work in
groups
• Discuss Analyses
• Take 20 minutes
March 7, 2006
Lecture 8a
Slide #18