Download Chapter 4 The Multivariate Normal Distribution

1
Chapter 4 The Multivariate Normal Distribution
4.1 Definition
Intuition:
Let

X ~ N  , 2
 . Then, the density function is
1
2
   x   2 
 1 
f x   
exp 

2 
2
 2 
 2

1
2
1
2

1
 1   1
x    1 x   

exp 
 

Var  X 
 2  Var  X  
 2

Definition (Multivariate Normal Random Variable):
A random vector
 X1 
X 
2 
X  
~ N  ,  
  


X p 


with E X   , Cov X    has the density function
p
1
 1 2  1 2
1
x   t  1 x   
f x   f x1 , x2 ,  , x p   
exp
 


 2   det  
2

Moment Generating Function of Multivariate Normal Random
Variable:
Let
 X1 
 t1 
X 
t 
2
2

X 
~ N  ,  , t   
  
 .


 
 X p 
t p 
2
Then, the moment generating function for X is
  
M X t   M X t1 , t 2 ,, t p   E exp t t X


 E exp t1 X 1  t 2 X 2    t p X p 
1


 exp  t t   t t t 
2


Result:
If X ~ N  ,  and C is a
pn
matrix of rank p, then

CX ~ N C , CC t

.
[proof:]
Let
Y  CX . Then,
     
 s  C t 

 E exp s X 

M Y t   E exp t t Y  E exp t t CX
t
t
t
t
 s  t C
1


 exp  s t   s t s 
2


1


 exp  t t C  t t CC t t 
2


Since
M Y t 
is the moment generating function of



N C , CC t ,
CX ~ N C , CC t

.
◆
3
Result:

2
If X ~ N  , I

then

TX ~ N T ,  2 I

,
where T is an orthogonal matrix.
Result:
If X ~ N  ,  , then the marginal distribution of subset of the
elements of Y is also multivariate normal.

 X1 
X 
2
X 
~ N  ,  
, then
  


X

 p

 X i1 
X 
i

X   2  ~ N   , 
  


X

i
 m



,
where
 i1i1  i1i2
  i1 

 
 i i  i2i2
i


2
m  p, i1 , i2 ,  , im  1,2,  , p ,    ,    2 1
 
  


 
  im 
 imi1  imi2
  i1im 

  i2im 
  

  imim 
Useful Splus Commands:
>ir=rbind(iris[,,1],iris[,,2],iris[,,3])
>irmean=c(mean(ir[,1]),mean(ir[,2]),mean(ir[,3]),mean(ir[,4])) # mean vector
>irvar=var(ir)
# variance-covariance matrix
>ircor=cor(ir)
# correlation matrix
>plot(ir[,1],ir[,2])
# scatter plot
>boxplot(ir[,1],ir[,2],ir[,3],ir[,4])
# box plot
>pairs(ir)
# all pairwise scatter plots
4
>brush(ir)
# brush a matrix of scatter plots
Result:
(i) If
X1 and
X 2 are independent random vecotors, then
Cov X 1 , X 2   0 .
  1   11
 X1 
  , 
~
N
(ii) If 

 
X
 2
  2   21
12  

X1 and X 2
 22   , then
 12  0 .
are independent
(iii) If X1 and X 2 are independent and are distributed as
N 1 ,11 
and
N 2 ,22  , respectively, then
  1  11
 X1 
  , 
~
N
X 
 
 2
  1  0
0 

 22  
Example 1:
 X1 
X 
Let X   2  be
X3 
 
X 4 
N , 
with
4
1

0

0
1
3
0
0
X 
X 
Then,  1  and  3  are independent.
X 2 
X 4 
Result:
0
0
1
2
0
0
2 .

5
5
  1   11
 X1 
  , 
~
N
Let 

 
X
 2
  2   21
12  

 22   and
 22  0 . Then the
conditional distribution of X 1 , given that X 2  x2 , is normal and has
mean  1  12 221 x2   2 
and
Covariance  11  12 221 21 .
[proof:]
We only need to prove

X 1  1  12 221 x2   2  ~ N 0, 11  12 221 21
then,

X1 ~ N 1  12221 x2  2 , 11  12 22121
,
.
By taking
I
A
0
 12  221 

I

so
 X 1  1   I  12 221   X 1  1   X 1  1  12 221  X 2  2 
A
.



X


X


X 2  2
0
I
2
2
 2

 2


By the result (page 2 or Result 4.3 in textbook),
 X 1  1   X 1  1  12 221  X 2  2 
t
A

 ~ N 0, AA ,

X 2  2
 X 2  2  

where

 I  12 221  11 12  
I
AA  



I   21  22    12  221
0
t
Since

X1  1  12221  X 2  2 

t

0t  11  12 221 21 0 t 

.
I 
0
 22 
X 2  2
and
have
zero
covariance, they are independent. Therefore, the conditional
distribution
X 1  1  12221 x2  2 
is
the
same
as
the
6
unconditional distribution of
X1  1  12221  X 2  2  . That is,

X 1  1  12 221 x2   2  ~ N 0, 11  12 221 21
.
Theorem: Let X ~ N p  ,   with   0 . Then
Q   X     1  X    ~  p2
t
[proof:]
Since

orthogonal
1
0



0

  AAt ,
is positive definite,
matrix
0

2




0
where
A
AAt  At A  I
(
0
0
 . Then,
 

 p 

 1  A1 At  A1  At
 . Thus,
Q   X     1  X   
t
  X    A1 At  X   
t
 Z t 1Z
where
Z  At  X    . Further,
is a real
)
and
7
Q  Z t 1Z

 Z1
Z2

1

 1
0
Zp 


0

2
i
1

 Z 
Z

  i 
  
i 1 i
i 1 
i 
p
0
p
Therefore, if we can prove
2

0 
  Z1 
 
0  Z2 

  
  
1   Z p 

 p 




0

2
Z i ~ N 0, i 
and
Zi
are
mutually independent, then
 Zi
~ N 0,1, Q   
 
i
i 1 
i
p
Zi
The proof is as follows. Since

2

 ~  p2


.
Z  At  X    , then

Z ~ N 0, At A  N 0,   , where
At A  At AAt A   .
That is,
Z i ~ N 0, i  .
Useful Splus Commands:
comatrix=matrix(0,2,2)
comatrix[1,]=c(2,1)
comatrix[2,]=c(1,1)
mvector=c(0,2)
### Density, cumulative probability, and random generation for the multivariate
 0 2 1 
### normal N   , 
 
 2 1 1 
8
msample=rmvnorm(10,mvector,comatrix)
pmvnorm(c(0,2),mvector,comatrix)
## 10 random observations
## P X 1  0, X 2  2
dmvnorm(c(0,2),mvector,comatrix)
##
f 0,2  1
2
### Density, cumulative probability, and random generation for the multivariate
 0 1 0 
### normal N   , 
 
 0 0 1 
msample=rmvnorm(20)
## 20 random observations
pmvnorm(c(0,0))
## P X 1  0, X 2  0
dmvnorm(c(0,0))
##
f 0,0  1
2
Theorem:
Let
X 1 , X 2 ,, X n
be mutually independent with
X j ~ N  j ,  . Then,
 n
 n 2 
V1  c1 X 1  c2 X 2    cn X n ~ N   c j  j ,   c j   .
 j 1

 j 1  

Moreover, V1 and V2  b1 X 1  b2 X 2    bn X n are jointly
multivariate normal with covariance matrix
 n 2 
  c j 
 j 1 

 bt c 

 

bc 

n

 .
  b 2j  


 j 1  
 
t
Consequently, V1 and V2 are independent if b c 
t
n
b c
j 1
j
j
0.