Download Matrix Algebra (and why it`s important!)

Matrix Algebra (and why it’s
important!)
Methods for Dummies
FIL
October 2007
Steve Fleming & Verity Leeson
Sources and further information
 Jon Machtynger & Jen Marchant’s slides!
 Human Brain Function textbook (for GLM)
 SPM course
http://www.fil.ion.ucl.ac.uk/spm/course/
 Web Guides
– http://mathworld.wolfram.com/LinearAlgebra.html
– http://www.maths.surrey.ac.uk/explore/emmaspages/opt
ion1.html
– http://www.inf.ed.ac.uk/teaching/courses/fmcs1/
(Formal Modelling in Cognitive Science course)
– http://www.wikipedia.org
Scalars, vectors and matrices
 Scalar: Variable described by a single
number – e.g. Image intensity (pixel value)
 Vector: Variable described by magnitude and direction – e.g. Image
intensity at a particular time
 Matrix: Rectangular array of vectors defined by number of rows and
columns
2
(Roman Catholic)
3
Square (3 x 3)
Rectangular (3 x 2)
d r c : rth row, cth column
Matrices in Matlab
 Vector formation:
 Matrix formation:
X = [1 2 3; 4 5 6; 7 8 9]
=
Subscripting – each element of a matrix
can be addressed with a pair of numbers;
row first, column second (Roman Catholic)
e.g.
X(2,3) = 6
X(3, :) =
7
8
‘;’ is used to signal
end of a row
[1 2 3]
9
5
X( [2 3], 2) =  
8
 
1

4
7

2
5
8
3

6
9 
‘:’ is used to
signify all rows or
columns
“Special” matrix commands:
• zeros(3,1) =
• ones(2) =
• magic(3)
more to come…
0
 
0
0
 
1

1
1

1
Matrix addition
Addition (matrix of same size)
– Commutative: A+B=B+A
– Associative: (A+B)+C=A+(B+C)
Subtraction (consider as the addition of a negative matrix)
Matrix multiplication
• Scalar multiplication:
n
• Rule for multiplication of
vectors/matrices:
m
Matrix multiplication rule:
“When A is a mxn matrix & B is a
kxl matrix, AB is only viable if n=k.
The result will be an mxl matrix”
l
a11
a12
a13
a21
a22
a23
a31
a32
a33
a41
a42
a43
b11
b12
b21
b22
b31
b32
x
b11
b12
b21
b22
b31
b32
a11
a12
a13
a21
a22
a23
a31
a32
a33
a41
a42
a43
X
k
Multiplication method l
l
• Sum over
product of
respective rows
and columns
• For larger
matrices, following
method might be
helpful:
m
m
0
2
 X 
3
3
1

2
r
1

1
 c11

 c 21
=
c
=
• Matlab does all this for you!
• Simply type: C = A * B
• N.B. If you want to do element-wise
multiplication, use: A .* B
=
2  0

49
2

13
c12 

c 22 
Define
output
matrix
1  0  Sum
 over
2  3  crc
1

5
Transposition
column
→
row
row
→
column
Mrc = Mcr
• In Matlab: AT = A’
Outer and inner products of
vectors
Two vectors:
Inner product = scalar
Outer product = matrix
(1xn)(nx1)  (1X1)
(nx1)(1xn)  (nXn)
Identity matrices
 Is there a matrix which plays a similar role as
the number 1 in number multiplication?
Consider the nxn matrix:
1

0
0

0
1
0
Worked example
A In = A
for a 3x3 matrix:
A square nxn matrix A has one
A I n = In A = A
0

0
1 
An nxm matrix A has two!!
In A = A & A I m = A
1
2
3
4
5
6
7
8
9
X
1
0
0
0
1
0
0
0
1
=
1+0+0
0+2+0
0+0+3
4+0+0
0+5+0
0+0+6
7+0+0
0+8+0
0+0+9
• In Matlab: eye(r, c) produces
an r x c identity matrix
Inverse matrices
 Definition. A matrix A is nonsingular or invertible if there exists a
matrix B such that:
worked example:
1
1
-1
2
X
2
3
-1
3
1
3
1
3
=
2+1
3 3
-1 + 1
3 3
-2+ 2
3 3
1+2
3 3
=
1
0
0
1
 Common notation for the inverse of a matrix A is A-1
 The inverse matrix A-1 is unique when it exists.
 If A is invertible, A-1 is also invertible  A is the inverse matrix of A-1.
• Matrix division:
 If A is an invertible matrix, then (AT)-1 = (A-1)T
• In Matlab: A-1 = inv(A)
A/B = AB-1
Determinants
 Determinant is a
function:
– Input is nxn matrix
– Output is a single
number (real or
complex) called the
determinant
 A matrix A has an inverse matrix A-1 if
and only if det(A)≠0 (see next slide)
• In Matlab: det(A) = det(A)
Calculation of inversion using determinants
Or you can just type
inv(A)!
thus
Note: det(A)≠0
More complex matrices can be inverted using methods such as the Gauss-Jordan
elimination, Gaussian elimination or LU decomposition
Applications
 SEM
http://www.maths.soton.ac.uk/~jav/soton/MATH1007/workbook_8/8_2_inv
_mtrx_sim_lin_eqnpdf.pdf
 Neural Networks
http://csn.beckman.uiuc.edu/k12/nn_matrix.pdf
 SPM
http://imaging.mrc-cbu.cam.ac.uk/imaging/PrinciplesStatistics
Solving simultaneous equations
 For one linear equation ax=b where the unknown
is x and a and b are constants
 3 possibilities
b
If a  0 then x   a 1b thus there is single solution
a
If a  0 , b  0 then the equation ax  b becomes 0  0
and any value of x will do
If a  0 , b  0 then ax  b becomes 0  b which is a
contradiction
With >1 equation and >1 unknown
1
x

a
b from the single
 Can use solution
equation to solve
 For example
2 x  3x  5
1
2
x1  2 x2  1
3   x1   5 
1  2  x    1

 2   
 In matrix form 2
AX = B
 Need to find determinant of matrix A (because X
=A-1B)
 From earlier

2 3 
1  2 


a b
det( A) 
 ad  bc
c d
(2 x -2) – (3 x 1) = -4 – 3 = -7
 So determinant is -7
 To find A-1
1  2  3 1 2 3 
A 
 



( 7 )   1 2  7  1  2 
1
if B is
1 
 4
 
1 2 3  1  1  14  2
X 
  



7 1  2   4  7   7   1
So
x  a 1b
Neural Networks
 Neural networks create a mathematical
model of the connections in a neural system
 Connections are the excitatory and inhibitory
synapses between neurons
Excitatory Connection
Input Neuron
Output Neuron
Inhibitory Connection
Input Neuron
Output Neuron
Scenario 1
Input Neuron
If
If
Output Neuron
then
then
Scenario 2
•The combination of
both an active
excitatory and active
inhibitory input will
cancel out
•No net activity
+
=
Matrix Representations of Neural
Connections –Scenario 2 again
Excitatory = positive
influence on post synaptic
cell
#2
-1
+1
#1
1
+1
#3
Inhibitory = negative
influence
=
With the synapses labelled (1-3) and activity level
specified we can translate this information into a set of
vectors (1 row matrices)
#2
-1
+1
#1
1
+1
#3
 Input vector = (1 1) relates
to activity (#1 #2)
 Weight vector = (1 -1)
relates to connection
weight (#1 #2)
=
Activity of Neuron 3
Input x weight
1
1 1.   (11)  (1 1)  (1)  (1)  0
 1
With varying input (activity) and weight, neuron 3 can
take on a wide range of values
How are matrices relevant to fMRI
data?
 Consider that data measured includes
– Response variable
e.g BOLD signal at a particular voxel
Many scalars for this one voxel
– Explanatory variables
These are assumed to be measured without error
May be continuous
May be dummy indicating levels of an
experimental factor
With a single explanatory variable
Y
=
X
.
β
+
ε
Observed = Predictors * Parameters + Error
BOLD
= Design Matrix * Betas + Error
Y=
X.β +ε
Y
Time
 Y is a matrix of BOLD
signals
 Each column
represents a single
voxel sampled at
successive time points.
 Each voxel is
considered as
independent
observation
 Analysis of individual
voxels over time, not
groups over space
Preprocessing ...
Intensity
Design Matrix
Y
Scan no
Voxel 1
•
57.84
1
X1
Task
difficulty
5
X2
Y=
X1
X.β +ε
X2
Constant
variable
1
2
57.58
4
1
3
57.14
4
1
4
55.15
2
1
5
55.90
3
1
6
55.67
1
1
7
58.14
6
1
8
55.82
3
1
9
55.10
1
1
10
58.65
6
1
11
56.89
5
1
12
55.69
2
1
Most –ve nearest
black, most +ve
nearest white
Matrix
Rows :
values of X for a single predictor
Columns : different predictors
Solve equation for
β – tells us how
much of the BOLD
signal is explained
by X
A complex version
a
m
b3
b4
b5
=
b6
+
b7
b8
b9
Y
=
X
×
b
+
e
The End…
Any (easy)
questions?!