Download Document

Jointly distributed Random
variables
Multivariate distributions
Discrete Random Variables
The joint probability function;
p(x,y) = P[X = x, Y = y]
1.
0  p  x, y   1
2.
 p  x, y   1
x
3.
y
P  X , Y   A   p  x, y 
 x, y   A
Continuous Random Variables
Definition: Two random variable are said to have
joint probability density function f(x,y) if
1.
0  f  x, y 
 
2.
  f  x, y  dxdy  1
 
3.
P  X , Y   A    f  x, y  dxdy
A
If 0  f  x, y  then z  f  x, y 
defines a surface over the x – y plane
  f  x, y  dxdy
A
Multiple Integration
  f  x, y  dxdy
A
f(x,y)
If the region A = {(x,y)| a ≤ x ≤ b, c ≤ y ≤ d} is a
rectangular region with sides parallel to the
coordinate axes:
y
d
c
a
Then
x
b
  f  x, y  dxdy
A




    f  x, y  dx  dy     f  x, y  dy  dx




d
b
b
d
c
a
a
c
To evaluate
d b
  f  x, y  dxdy    f  x, y  dxdy
c a
d b
A


    f  x, y  dx  dy
c a

First evaluate the inner integral
b
G  y    f  x, y  dx
a
Then evaluate the outer integral
d b
d
c a
c
  f  x, y  dxdy   G  y  dy
y
d
dy
y
c
b
a
b
x
G  y    f  x, y  dx = area under surface above the
line where y is constant
a
d b
d
c a
c
  f  x, y  dxdy   G  y  dy
Infinitesimal volume under
surface above the line where
y is constant
The same quantity can be calculated by integrating
first with respect to y, than x.
b d
  f  x, y  dxdy    f  x, y  dydx
a c
b d
A


    f  x, y  dy  dx
a c

First evaluate the inner integral
d
H  x    f  x, y  dy
c
Then evaluate the outer integral
b d
b
a c
a
  f  x, y  dydx   H  x  dx
y
dx
d
c
d
a
x
b
x
H  x    f  x, y  dy = area under surface above the
line where x is constant
c
b d
b
a c
a
  f  x, y  dydx   H  x  dx
Infinitesimal volume under
surface above the line where
x is constant
Example: Compute
1 1
 
1 1

0 0
Now
1 1
 
0 0


x 2 y  xy 3 dxdy    x 2 y  xy 3 dydx
0 0
1


2
3
2
3
x y  xy dxdy     x y  xy dx  dy
0 0

x

1
1 3
2
x
x 3 
 dy
  y y
3
2
0
x 0 


1
2
4 y 1
1 3
1y 1y
1
   y  y  dy 

3
2 
3 2 2 4 y 0
0
1 1 7
  
6 8 24

1


The same quantity can be computed by reversing
the order of integration
1 1
 
0 0
1


2
3
2
3
x y  xy dydx     x y  xy dy  dx
0 0


1


y 1
4
 y2

y
2
   x  x  dx
2
4 y 0 
0


1
2 x 1
1x 1x
1 2 1 
   x  x  dx 

2
4 
2 3 4 2
0
1 1 7
  
6 8 24
1
3
x 0
Integration over non rectangular
regions
Suppose the region A is defined as follows
A = {(x,y)| a(y) ≤ x ≤ b(y), c ≤ y ≤ d}
y
d
c
Then

A
b(y)
a(y)
x
 b y 

f  x, y  dxdy     f  x, y  dx  dy

c 
 a y 
d
If the region A is defined as follows
A = {(x,y)| a ≤ x ≤ b, c(x) ≤ y ≤ d(x) }
y
d(x)
c(x)
Then

A
b
a
x
d  x

f  x, y  dxdy     f  x, y  dy  dx

a
 c x 
b
In general the region A can be partitioned into
regions of either type
y
A2
A1
A3
A
A4
x
Example:
Compute the volume under f(x,y) = x2y + xy3 over the
region A = {(x,y)| x + y ≤ 1, 0 ≤ x, 0 ≤ y}
y
(0, 1)
x+y=1
(1, 0)
x
Integrating first with respect to x than y
y
(0, 1)
x+y=1
(1 - y, y)
(0, y)
(1, 0)
x
 x
A
2

1 1 y
y  xy dxdy 
3
 x
0 0
1 1 y
2

y  xy dxdy
3


2
3
    x y  xy dx  dy

0 
0


and
x 1 y
3
2




x
x
2
3
3
x
y

xy
dx
dy

    3 y  2 y  dy
0  0 


0
x 0 


3
2
1
1 y
1 y 3 


 
y
y  dy
3
2

0

1
 y  3 y 2  3 y3  y 4 y3  2 y 4  y5 
 

 dy
3
2

0
1
1 y
1
 1  43  15 14  52  16


3
2
 16  13  14  151  81  15  121
1
2

20 4030815 2410
120
3
 120

1
40
Now integrating first with respect to y than x
y
(0, 1)
x+y=1
(x, 1 – x )
(1, 0)
(x, 0)
 
A

x
1 1 x
x 2 y  xy 3 dydx 

0 0
1 1 x

x 2 y  xy 3 dydx


2
3
    x y  xy dy  dx
0  0



Hence

0  0
1
1 x
y 1 x
2
4




y
y

2
2
3


x
 x y  xy  dy  dx 0  2  x 4   dx
y 0 


1
1
x
0
2
1  x 
2
2
1 x

x
4
4
dx
x 2  2 x3  x 4 x  4 x 2  6 x3  4 x 4  x5


dx
2
4
0
1
x  2 x 2  2 x3  2 x 4  x5

dx
4
0
1
1512 6
4
 18  16  18  101  201  1520120
 120
Continuous Random Variables
Definition: Two random variable are said to have
joint probability density function f(x,y) if
1.
0  f  x, y 
 
2.
  f  x, y  dxdy  1
 
3.
P  X , Y   A    f  x, y  dxdy
A
Definition: Let X and Y denote two random
variables with joint probability density function
f(x,y) then
the marginal density of X is
fX  x 

 f  x, y  dy

the marginal density of Y is
fY  y  

 f  x, y  dx

Definition: Let X and Y denote two random
variables with joint probability density function
f(x,y) and marginal densities fX(x), fY(y) then
the conditional density of Y given X = x
fY X  y x  
f  x, y 
fX  x
conditional density of X given Y = y
fX Y  x y 
f  x, y 
fY  y 
The bivariate Normal distribution
Let
f  x1 , x2  
1
 2   1 2
1 
2
e
1
 Q  x1 , x2 
2
where
2
 x   2






x


x


x


1
1
1
1
2
2
2
2

2






 
 
  1   2    2  
  1 
Q  x1 , x2  
1  2
This distribution is called the bivariate
Normal distribution.
The parameters are 1, 2 , 1, 2 and .
Surface Plots of the bivariate
Normal distribution
Note:
f  x1 , x2  
1
 2   1 2
1 
2
e
1
 Q  x1 , x2 
2
is constant when
2
 x   2






x


x


x


1
1
1
1
2
2
2
2

2






 
 
  1   2    2  
  1 
Q  x1 , x2  
1  2
is constant.
This is true when x1, x2 lie on an ellipse
centered at 1, 2 .
Marginal and Conditional
distributions
Marginal distributions for the Bivariate Normal
distribution
Recall the definition of marginal distributions
for continuous random variables:
f1  x1  



f  x1 , x2  dx2
and
f 2  x2  

 f  x , x  dx
1
2

It can be shown that in the case of the bivariate
normal distribution the marginal distribution of xi
is Normal with mean i and standard deviation i.
1
Proof:
The marginal distributions of x2 is
f 2  x2  

 f  x , x  dx
1
2
1



1
 2 1 2
1  2
e
1
 Q x1 , x2 
2
dx1

where
2

 x1  1  x2  2
 x1  1 

  2 

 1 
  1   2


Q  x1 , x2  
1  2
  x2  2  


 
  2  

2
Now:
2
 x   2






x


x


x


1
1
1
1
2
2
2
2

2






 
 
  1   2    2  
  1 
Q  x1 , x2  
1  2
2
2
x

a
x
a
a
 1

1


c

 2 2 x1  2  c

2
b
b
b
 b 
2

x12
1
x2  2
 
2 

2
2
 1  
 2 1 1   2
   1  



2
1
 1  
2
1
2


 2


 x2  2 
 2 1 1  
2

1

x2  2 



 x1

2
 22 1   2 
Hence
Also

b2   1   2

or b  1 1   2
1
x2  2
a
 

2
2
b
 1  
 2 1 1   2






1
1
 
    x2   
2  1
2
 1    

and
1
a  1  
 x2   
2
Finally
x2  2 
x2  2 



a
c  2
 2
1  2
2
2
2
b
1 1  
 2 1 1  
 2 1  2
2

c


2
1
 1  
2
1
2

2
1
 1  
2
1
2
2
1


2

 2
 2

 x2  2 
 2 1 1  
2
 x2  2 
 2 1 1  
2





1
 1     x2    
2


  1   2 
2
 x2  2 
2
x2  2 


2
1 
1

 22 1   2 
 22 1   2 

a2
 2
b
and
 2
1
12
1
2
c 2
  2 1  x2  2   2  x2  2 
2  1
2
2
1 1    
2

 
1
  1    x2  2  
2

 
 12
1
2
2
 2
1     x2  2  
2  2 
1 1      2

 x2  2 


 2 
2
Summarizing
2
 x   2






x


x


x


1
1
1
1
2
2
2
2

  2 


 
  1   2    2  
  1 
Q  x1 , x2  
1  2
 x1  a 

 c
 b 
2
where
and
b  1 1   2
1
a  1  
 x2   
2
2
 x2   2 
c

 2 
Thus

f 2  x2  
 f  x , x  dx
1
2
1



1
 2 1 2



1 
e
2
1
2 be
 2 1 2
1
2 2
e
1  2
e
1  2
2
2

1  x1  a 
 
c
2  b 

dx1


c 2
1  x  
  2 2
2 2 
dx1


 2 1 2
1
 Q x1 , x2 
2


1
e
2 b
1  x a 
  1 
2 b 
2
dx1
Thus the marginal distribution of x2 is Normal
with mean 2 and standard deviation 2.
Similarly the marginal distribution of x1 is Normal
with mean 1 and standard deviation 1.
Conditional distributions for the Bivariate Normal
distribution
Recall the definition of conditional distributions
for continuous random variables:
f1 2  x1 x2  
f  x1 , x2 
f 2  x2 
and f 21  x2 x1  
f  x1 , x2 
f1  x1 
It can be shown that in the case of the bivariate
normal distribution the conditional distribution of
xi given xj is Normal with:
i
mean i j  i    x j   j  and
j
standard deviation
i j  i 1  2
Proof
f 21  x2 x1  
f  x1 , x2 
f1  x1 
e
1
 Q  x1 , x2 
2
2   1 2


1  2
1
2 2

e
1
1  x  
 Q  x1 , x2    2 2 
2
2 2 
2 1 1  
2
2

e
e
1  x  
  2 2
2 2 
2
2
 1  x   2
1  x1  a 
 
c   2 2 

2  b 
 2   2 
2 1 1   2
where
and
Hence
b  1 1   2
1
a  1  
 x2   
2
2
 x2   2 
c

 2 
1
f1 2  x1 x2  
e
2 b
1  x a 
  1 
2 b 
2
Thus the conditional distribution of x2 given x1 is Normal
with:
1
 x2  2  and
mean a  1 2  1  
2
standard deviation
b  1 2  1 1   2
Bivariate Normal Distribution with marginal
distributions
Bivariate Normal Distribution with
conditional distribution
x2
( 1, 2)
Major axis of
ellipses
Regression
Regression to the
mean
2
21  2    x1  2 
1
x1