Download Matrix Algebra Linear Models A system of m linear equations in n

Matrix Algebra

Linear Models
A system of m linear equations in n variables:
a11 x1  a12 x 2  ...  a1n x n  d1
 a11
a
a 21 x1  a 22 x 2  ...  a 2 n x n  d 2
21

 ...
....

a m1 x1  a m 2 x 2  ...  a mn x n  d m am1
a12
a22
...
am 2
... a1n   x1   d1 
... a2 n   x2   d 2 

 Ax  d
... ...   ...   ... 
   
... amn   xn  d m 
where
 a11
a
A   21
 ...

a m1
a12
a 22
...
am2
... a1n 
... a 2 n 
... ... 

... a mn 
 x1 
x 
x   2
 ... 
 
 xn 
Each of A, x, d constitutes a matrix.

Dimension of the Matrix
m × n matrix ( m rows and n columns)
 a11
a
 21
 ...

am1
a12
a22
...
am 2
... a1n 
... a2 n 
... ... 

... amn 
or
aij 
mn
Special matrices:
1.
2.
m = n  n × n square matrix
n × 1 column vector
 x1 
x 
x   2
 ... 
 
 xn  n1
3. 1 × n row vector
x  x1 x2 ... xn 1n
1
 d1 
d 
d  2
 ... 
 
d m 

Matrix Operations
1.
Equality
aij   bij   aij  bij
mn
mn
2.
Addition
aij   bij   cij 
where aij  bij  cij
mn
mn
mn
3.
Subtraction
aij   bij   dij 
where aij  bij  d ij
mn
mn
mn
4.
Scalar Multiplication
k aij 
5.
mn
 kaij 
mn
Multiplication of Matrices
 aik mn bkj  n p  cij  mn


3.
4.
In general, AB  BA
( AB)C  A( BC )  ABC (Associative law of multiplication)
5.
A( B  C)  AB  AC; ( B  C) A  BA  CA (Distributive Law)
Identity Matrices I
0 ... 0 
1 ... 0 
(Square matrix)
... ... ...

0 ... 1  nn
IA  AI  A
Null Matrices 
0
0

...

0

k 1
Laws
1. A  B  B  A (Commutative law of addition)
2. ( A  B)  C  A  ( B  C )  A  B  C (Associative law of addition)
1
0
In  
...

0

n
where cij   aik bkj
0 ... 0 
0 ... 0 
A      A ; A   ; A  
... ... ...

0 ... 0  mn
AB   does NOT imply A   or B  
2

Transpose A or AT
A  aij 
1.
2.
3.

mn

A  a ji 
nm
( A)  A
( A  B)  A  B
( AB)  BA
Inverses A1
AA1  A1 A  I
1. A1 is defined only if A is a square matrix
2. Not every square matrix has an inverse. If A has an inverse, A is said to be
nonsingular. If A does not have an inverse, A is said to be singular.
3.
4.
5.

A and A1 are inverses of each other.
If A is n × n, A1 must be n × n
If an inverse exists, it is unique.
Evaluating a Determinant
1. First-order determinant
a11  a11
2.
3.
4.
Second-order determinant
a11
a12
a 21
a 22
 a11a 22  a12 a 21
Third-order determinant
a11 a12 a13
a21 a22
a23  a11a22 a33  a12 a23a31  a21a32 a13  a13a22 a31  a12 a21a33  a23a32 a11
a31
a33
a32
nth-order determinant
Minor M ij is obtained by deleting i row and j column.
Cofactor Cij  (-1)i  j M ij
n
A   aij Cij Expansion by the ith row
j 1
n
A   aij Cij Expansion by the jth column
i 1
3

Finding the Inverse Matrix
A1 

 C11

C12
adjA  C   
 

 C1n
1
adjA
A
C 21
C 22

C2n
C n1 

... C n 2 
 

... C nn 
...
Cramer’s Rule
Ax  d  x j 
Aj
Aj 
A
a11
a12
...
a21
a22
... d 2


an1

d1

...
a1n
... a2 n

an 2  d n  ann
Exercise
4.6
 0 4
3  8
1 0 9
1. Given A  
,B  
and C  


 , find A , B  and C  .
  1 3
0 1 
6 1 1 
2. Use the matrices given in Prob. 1 to verify that
(a)
5.2
 A  B   A  B 
(a) 4 0 1
6 0 3
a b
c
(e) b c a
c a b
4. Test whether the following matrices are nonsingular:
4 0 1 
(a) 19 1  3
 7 1 0 
5.4
 AC   C A
1. Evaluate the following determinants:
8 1 3
5.3
(b)
 4 9 5
(d)  3 0 1 
 10 8 6
4. Find the inverse of each of the following matrices:
4  2 1
(a) E  7 3 0
2 0 1
1 0 0
(c) G  0 0 1


0 1 0
4
5.5
3. Use Cramer’s rule to solve the following equation system:
8 x1  x 2
 16
(a)
2 x 2  5 x3  5
2 x1
 3 x3

7
x y z  a
(d)
x
y z  b
x
y z  c
5
Differentiation

The Definition of Derivative
f x  

dy
f  x  x   f x 
 lim
dx x0
x
Rules of Differentiation
1.
2.
3.
4.
dc
0
dx
(c: constant; Constant-function Rule)
dx n
 nx n 1 (Power-function Rule)
dx
d
cf ( x)  c d f ( x)  cf ( x)
dx
dx
d
 f ( x)  g ( x)  d f ( x)  d g ( x)  f ( x)  g ( x) (Sum-difference
dx
dx
dx
Rule)
5.
6.
7.
8.
d
 f ( x) g ( x)  f ( x) g ( x)  f ( x) g ( x)
(Product Rule)
dx
d f ( x) f ( x) g ( x)  f ( x) g ( x)

(Quotient Rule)
dx g ( x)
g 2 ( x)
dx
1

dy  dy 
 
 dx 
(Inverse-function Rule)
z  f ( y ); y  g ( x) 
dz dz dy

dx dy dx
(Chain Rule)

The Definition of Partial Derivative
f x1  x1 , x2 ,..., xn   f x1 , x2 ,..., xn 
y
f1 
 lim
x1 x0
x1

Total Differentials
U  U ( x1 , x 2 ,..., x n )  dU 
1.
dc  0
2.
3.
du n  nu n 1 du
d (u  v)  du  dv
4.
U
U
U
dx1 
dx 2  ... 
dx n
x1
x 2
x n
d uv   vdu  udv
6
5.
6.
7.

u 1
d    2 vdu  udv 
v v
d (u  v  w)  du  dv  dw
d uvw  vwdu  uwdv  uvdw
Total Derivatives
Q  QK t , Lt , t 
Q
Q
Q
dK 
dL 
dt
K
L
t
dQ Q dK Q dL Q




dt K dt L dt t
 dQ 

Derivatives of Implicit Functions
Explicit function: y  f ( x1 , x2 ,..., xn )
Implicit function: F ( x1 , x2 ,..., xn )  0
F ( x, y )  0  dF 
F
F
F
dy
dx 
dy  Fx dx  Fy dy  0 
 x
x
y
dx
Fy
7
Exponential Functions

Simple Exponential Function y  f (t )  b t

Graphical Form
y
(b>0; y>0)
y = 2t
2
1
t
0

A Preferred Base
e  lim (1 
m 
1
y = et
1 m
)  2.71828...
m
Logarithmic Functions

The Meaning of Logarithm
Log functions are inverse functions of certain exponential functions.
y  b t  t  log b y

(b>0; y>0)
Common Log and Natural Log
10 is the base  Common Log (e.g. log10100 = log100 = 2)
e is the base  Natural Log (e.g. loge100 = ln100 = 4.60517…)

Rules of Logarithm
Rule 1: logc(ab) = logc(a) + logc(b)
Rule 2: logc(a/b) = logc(a) – logc(b)
Rule 3: logc (ba) =a logc(b)
Rule 4: logc (a) = logc (b) logb (a)
Rule 5: logc (a) = 1/ logc (a)

The Graphical Form
8
y
y
y = et
t = ln y
1
45˚
t
0

45˚
0
1
Log-Function Rule
d
1
ln t 
dt
t

Exponential-Function Rule
d t
e  et
dt

The Rules Generalized
d f (t )
e
 f ' (t )e f (t )
dt
d
f ' (t )
ln f (t ) 
dt
f (t )

t
The Case of Base b
d t
b  b t ln b
dt
d
1
log b t 
dt
t ln b
More general formulas:
d f (t )
b
 f ' (t )b f (t ) ln b
dt
d
f ' (t ) 1
log b f (t ) 
dt
f (t ) ln b
9
Exercise
7.1 2. Find the following:
(a)

d
 x 4
dx

d
 au b
du
(f)
7.2 3.
Differentiate the following by using the product rule:
(b) 3x  10 6 x 2  7 x
(e) 2  3x1  xx  2
8.3 2.
Use the rules of differentials to find dy from the following functions
x1
2 x1 x 2
(a) y 
(b) y 
x1  x 2
x1  x 2
Given y  3x1 2 x2  1x3  5
3.


(a) Find dy by rule VII
(b) Find the differential of y, if dx2  dx3  0 .
8.4 1.
Find the total derivative dz/dy, given
(a) z  f x, y   5x  xy  y 2 , where x  g  y   3y 2
10.5 1.
Find the derivatives of:
bx c
(e)
y  e ax
3.
(g)
 2x 
y  ln 

1 x 
4.
(b)
y  log 2 t  1
2
(f)
y  xe x
(c)
y  132t 3
10
(f)
y  x 2 log 3 x
Integration

Indefinite Integrals
d
F  x   f  x    f  x dx  F  x   c
dx


Rules of Integration
1.
x n 1
 x dx  n  1  c  n  1
2.
 e dx  e
3.
1
 xdx  ln x  c  x  0  the logarithmic rule 
4.
  f  x   g  x dx   f  x dx   g  x dx  the integral of a sum 
5.
 k f  x  dx  k  f  x dx  the integral of a multiple 
6.
du
 f  u  x   dx dx   f  u du  F u   c  the substitution rule 
7.
 vdu  uv   udv  integration by parts 
x
x
 c  the exponential rule 
Definite Integrals

b
a

 the power rule 
n
f  x dx  F  x  a  F  b   F  a 
b
Properties of Definite Integrals
a
f  x dx   f  x dx
b
1.

2.
 f  x dx  F  a   F  a   0
3.
4.
5.
b
a
a
a

d

b

b
a
a
a
f  x dx   f  x dx   f  x dx   f  x dx  a  b  c  d 
b
c
d
a
b
c
 f  x dx   f  x dx
b
a
kf  x dx  k  f  x dx
b
a
11

Improper Integrals
-
Infinite Limits of Integration



b


f  x dx  lim  f  x dx;
b
b  a
a

 f  x dx
f  x dx  lim
 f  x dx
b
a  a

-
f  x dx  lim
b
a  a
b 
Infinite Integrand
Assume that f  x    as x  p , where p is a point in the interval (a, b); then
 f  x dx   f  x dx   f  x dx
b
p
a
b
a
p
The given integral on the left can be considered as convergent if and only if each
subintegral has a limit.
Exercise
14.2 1. Find the following:
(d)
2.
1.
(e)
dx
x
4x
dx
1
(e)
2
  2ax  b   ax
 3e
 2 x  7 
(e)
dx
 4 xe
x2 3
dx
Find:
(a)
14.3
2 x
Find:
(d)
4.
 2e
  x  3 x  1
12
(b)
dx
 x ln xdx
Evaluate the following:
(c)
3

1
3 xdx
(e)
  ax
1
1
2

 bx  c dx
2. Evaluate the following:
14.4
3.
1 
1


dx
e
 x 1 x 
Evaluate the following:
(d)

(c)

6
1
0
x2 3dx
(d)

0

ert dt
(e)
12

5
1
dx
x2
2

7
 bx dx
Optimization Problems
One variable

First derivative versus Second derivative
f ( x)  f ( x)  f ( x)
f x0   0  f x  
at x0

f x0   0  f x  
f x0   0  f x  
at x0

f x0   0  f x  

Relative (local) versus Absolute (global) Extremum
1.
Absolute (global) extremum  Relative (local) Extremum
First-derivative test for relative extremum
If f x0   0
1.
2.
3.

A relative minimum if f x  change its sign -  + from the immediate
left of x 0 to its immediate right.
Neither a relative maximum nor a relative minimum if f x  has the same
sign on both the immediate left and immediate right of x 0
Second-derivative test for relative extremum
If f x0   0
1.
A relative maximum if f x0   0 .
2.
A relative minimum if f x0   0
3.

A relative maximum if f x  change its sign +  - from the immediate
left of x 0 to its immediate right.
Either a relative maximum, or a relative minimum, or an inflection point
if f x0   0 .
Conditions for a relative extremum y  f (x)
Condition
First-order condition
Second-order condition
Maximum
f x   0
f x  0
Minimum
f x   0
f x  0
13

Nth-derivative test for relative extremum
If f x0   0 and if the first nonzero derivative is that of Nth derivative,
f
(N)
x0   0 .
2.
A relative maximum if f ( N ) x   0 and N is an even number.
A relative minimum if f ( N ) x   0 and N is an even number.
3.
An inflection point if N is odd.
1.
More than one variable

Conditions for a relative extremum z  f ( x, y )
Condition
Maximum
fx  fy  0
First-order condition
f xx  0;
Second-order condition

Minimum
f
fx  fy  0
yy
 0 ;
f xx  0;
f xx f yy  f xy2
Hessian (determinant)
f
yy
 0 ;
f xx f yy  f xy2
H
A Hessian is a determinant with the second-order partial derivatives as its
elements.
1.
In the two-variable case,
H 
f xx
f xy
f yx
f yy
 f xx f yy  f xy f yx  f xx f yy  f xy
2
Note that f xy  f yx (Young’s theorem)
2.
In the n-variable case,
H 
f 11
f 12
...
f 1n
f 21
f 22
...
f 2n
...
...
...
...
f n1
f n2
...
f nn
H1  f11 ; H 2 
;
f11
f12
f 21
f 22
f 11
f 12
f 13
; H 3  f 21
f 31
f 22
f 23 ; … … H n  H
f 33
14
f 32

Determinantal Test for a relative extremum z  f ( x1 , x2 ,..., xn )
Condition
First-order condition
Second-order condition
Maximum
f1  f 2  ...  f n  0
Minimum
f1  f 2  ...  f n  0
H 2  0; H 3  0...
H 2 , H 3 ,..., H n  0
 1n H n
0
Optimization with Equity Constraints


Free optimum versus constrained optimum
Lagrange-Multiplier Method
z  f ( x1 , x2 ,..., xn ) subject to the constrain g ( x1 , x2 ,..., xn )  c

Lagrangian function Z  f ( x1, x 2 ,..., x n )   c  g ( x1 , x 2 ,..., x n )

First-order condition
Z   c  g ( x1 , x 2 ,..., x n )  0
Z 1  f 1  g 1  0
Z 2  f 2  g 2  0
........................
Z n  f n  g n  0

Bordered Hessian
H
0
g1
g2
...
g1
Z 11
Z 12
... Z 1n
H  g2
Z 21
Z 22
... Z 2 n
...
...
...
gn
Z n1
Z n2
0
g1
g2
H 2  g1
Z 11
g2
Z 21
...
gn
...
... Z nn
0
g1
g2
g3
g
Z 12 ; H 3  1
g2
Z 22
g3
Z 11
Z 12
Z 13
Z 21
Z 22
Z 23
Z 31
Z 32
Z 33
15
;….. H n  H

Determinantal Test for a relative extremum z  f ( x1 , x2 ,..., xn ) subject to
g ( x1 , x2 ,..., xn )  c ; with Z  f ( x1, x 2 ,..., x n )   c  g ( x1 , x 2 ,..., x n )
Condition
First-order condition
Second-order
condition
Maximum
Z   Z1  Z 2  ...  Z n  0
H 2  0; H 3  0; H 4  0...
 1n H n
Minimum
Z   Z1  Z 2  ...  Z n  0
H 2 , H 3 ,..., H n  0
0
Exercise
11.4 Find the extreme values, if any, of the following four functions. Check whether
they are maxima or minima by the determinantal test.
1. z  x12  3x 22  3x1 x 2  4 x 2 x3  6 x32
2. z  29  x12  x 22  x32 
3. z  x1 x3  x12  x 2  x 2 x3  x 22  3x32
12.2 1. Use the Lagrange-multiplier method to find the stationary value of z:
(a) z  xy , subject to x  2 y  2
(b) z  x y  4 , subject to x  y  8
(c) z  x  3 y  xy , subject to x  y  6
(d) z  7  y  x 2 , subject to x  y  0
16