Download Matrix Calculus

Fin500J Mathematical Foundations in Finance
Topic 2: Matrix Calculus
Philip H. Dybvig
Reference: Matrix Calculus, appendix from Introduction to Finite Element
Methods book
Slides designed by Yajun Wang
Fin500J Topic 2
Fall 2010 Olin Business School
1
Outline
 The Derivatives of Vector Functions
 The Chain Rule for Vector Functions
Fin500J Topic 2
Fall 2010 Olin Business School
2
1 The Derivatives of Vector Functions
Fin500J Topic 2
Fall 2010 Olin Business School
3
1.1 Derivative of Vector with Respect to Vector
Fin500J Topic 2
Fall 2010 Olin Business School
4
1.2 Derivative of a Scalar with Respect to Vector
If y is a scalar
It is also called the gradient of y with respect to a vector
variable x, denoted by y .
1.3 Derivative of Vector with Respect to Scalar
Fin500J Topic 2
Fall 2010 Olin Business School
5
Example 1
Given
and
Fin500J Topic 2
Fall 2010 Olin Business School
6
In Matlab
>> syms x1 x2 x3 real;
>> y1=x1^2-x2;
>> y2=x3^2+3*x2;
>> J = jacobian([y1;y2], [x1 x2 x3])
J =
[ 2*x1,
[
0,
-1,
0]
3, 2*x3]
Note: Matlab defines the derivatives as the transposes of those given in this lecture.
>> J'
ans =
[ 2*x1,
0]
[
3]
[
-1,
0, 2*x3]
Fin500J Topic 2
Fall 2010 Olin Business School
7
Some useful vector derivative formulas
 Cx
 CT
x
 xT C
C
x
 xT x
 2x
x
Homework
Fin500J Topic 2
C11 C12
C
 21 C22
 


Cn1 Cn 2




n


C1n   x1   t 1 xt C1t 
n
C2 n   x2    xt C2t 


 t 1
     
   n

Cnn   xn   
xt Cnt 
t 1

 c11

 Cx  c12


x

c
 1n
Fall 2010 Olin Business School
c21  cn1 

c22  cn 2 
T

C
   

c2 n  cnn 

8
Important Property of Quadratic Form xTCx
 (xTCx)
  C  CT  x
x
C11 C12
C
 21 C22
 


Cn1 Cn 2
Proof:


x T Cx    xi   x j Cij  
i 1 
j 1

n
n


 n  n

   xi   x j Cij   
 (x T Cx)
 i 1  j 1



 

xk
xk
n
n
j 1
i 1




n


x
C
t
1
t
C1n   x1   t 1

n
C2 n   x2    xt C2t 


 t 1
     
   n

Cnn   xn   
xt Cnt 
t 1

n


 n


  xk   x j Ckj      xi xk Cik 

 j 1

i 1





xk
xk
  x j Ckj   xi Cik
 (x T Cx)

 Cx  CT x   C  CT  x
x
If C is symmetric,
Fin500J Topic 2
 (xTCx)
 2C x
x
Fall 2010 Olin Business School
9
2 The Chain Rule for Vector Functions
Let
where z is a function of y, which is in turn a function of x, we can write
Each entry of this matrix may be expanded as
Fin500J Topic 2
Fall 2010 Olin Business School
10
The Chain Rule for Vector Functions (Cont.)
Then
On transposing both sides, we finally obtain
This is the chain rule for vectors (different from the conventional chain
rule of calculus, the chain of matrices builds toward the left)
Fin500J Topic 2
Fall 2010 Olin Business School
11
Example 2
x, y are as in Example 1 and z is a function of y defined as
 z1 
 
z2
z    , and
 z3 
 
 z4 
 z1

z  y1

y  z1

 y2
 z1  y12  2 y2

2
 z2  y2  y1
, we have

2
2
 z3  y1  y2
z  2 y  y
 4
1
2
z2
y1
z3
y1
z2
y2
z3
y2
z4 
y1   2 y1 1 2 y1 2 

.
z4   2 2 y2 2 y2 1

y2 
Therefore,
2 x1
 4 x1 y1
 2 x1 0 
2
y

1
2
y
2


z y z 


1

  1 3   1
   2 y1  6 1  6 y2
x x y 
  2 2 y2 2 y2 1  4 x
0
2
x
4 x3 y2
3

3

Fin500J Topic 2
Fall 2010 Olin Business School
4 x1 

 2 y2  6 y2 1 
4 x3 y2
2 x3 
4 x1 y1
12
In Matlab
>> z1=y1^2-2*y2;
>> z2=y2^2-y1;
>> z3=y1^2+y2^2;
>> z4=2*y1+y2;
>> Jzx=jacobian([z1; z2; z3; z4],[x1 x2 x3])
Jzx =
[
4*(x1^2-x2)*x1,
-2*x1^2+2*x2-6,
-4*x3]
[
-2*x1,
6*x3^2+18*x2+1,
4*(x3^2+3*x2)*x3]
4*(x1^2-x2)*x1, -2*x1^2+20*x2+6*x3^2,
4*(x3^2+3*x2)*x3]
[
[
4*x1,
1,
2*x3]
>> Jzx’
ans =
[ 4*(x1^2-x2)*x1,
-2*x1,
4*(x1^2-x2)*x1,
4*x1]
[ -2*x1^2+2*x2-6,
6*x3^2+18*x2+1, -2*x1^2+20*x2+6*x3^2,
[
4*(x3^2+3*x2)*x3,
-4*x3,
Fin500J Topic 2
Fall 2010 Olin Business School
4*(x3^2+3*x2)*x3,
1]
2*x3]
13