Download Chapter 2 Multivariate Distributions

Chapter 2
Multivariate Distributions
Math 6203
Fall 2009
Instructor: Ayona Chatterjee
Random Vector
• Given a random experiment with a sample
space C. Consider two random variables X1
and X2 which assign to each element c of C
one and only one ordered pair of numbers
X1(c)=x1 and X2(c)=x2. Then we say that (X1, X2)
is a random vector.
• The space of (X1, X2) is the set of ordered pairs
D={(x1, x2) : X1(c)=x1 and X2(c)=x2 }
Cumulative Distribution Function
• The joint cumulative distribution function of
(X1, X2) is denoted by FX1,X2 (x1, x2) and is given
as FX1,X2 (x1, x2) =P[X1≤x1, X2 ≤x2)].
• A random vector (X1, X2 )is a discrete random
variable is its space D is finite or countable.
• A random vector (X1, X2 ) with space D is
continuous if its cdf FX1,X2 (x1, x2) is continuous.
Probability Mass Function
• For discrete random variables X1 and X2, the
joint pmf is defined as
p X1 X 2 ( x1 , x2 )  P[ X 1  x1 , X 2  x2 ]
Note
* 0  p X1 X 2 ( x1 , x2 )  1
*  p X1 X 2 ( x1 , x2 )  1
D
Probability Density Function
• For a continuous random vector
FX1 X 2 ( x1 , x2 ) 
x1 x2
f
X1 X 2
( w1 , w2 )dw1dw2
  
 2 FX 1 X 2 ( x1 , x2 )
x1x2
 f X 1 X 2 ( x1 , x2 )
Note
* f X 1 X 2 ( x1 , x2 )  0
*  f X1 X 2 ( x1 , x2 )dx1dx2  1
D
Marginals
• The marginal distributions can be obtained
from the joint probability density function.
• For a discrete and continuous random vector
the marginals can be obtained as below:
p X1 ( x1 ) 
p
x2  
X1 X 2
( x1 , x2 )

f X1 ( x1 ) 
f

X1 X 2
( x1 , x2 )dx2
Expectation
• Suppose (X1, X2) is of the continuous type.
Then E(Y) exists if
 
  g(x , x ) f
1
2
X1 X 2
( x1 , x2 )dx1dx2  
  
then
 
E (Y ) 
  g(x , x ) f
1
  
2
X1 X 2
( x1 , x2 )dx1dx2
Theorem
• Let (X1, X2) be a random vector. Let Y1 = g1(X1,
X2) and Y2 = g2 (X1, X2) be a random variable
whose expectations exits. Then for any real
numbers k1 and k2.
E(k1 Y1 + k2 Y2 )= k1E(Y1 ) + k2 E(Y2 )
Note
 
E ( g ( X 2 )) 
  g(x ) f
2
  

X1 X 2
( x1 , x2 )dx1dx2 
 g(x ) f
2

X2
( x2 )dx2
Moment Generating Function
• Let X = (X1. X2 )’ be a random vector. If
E(et1x1+t2x2 ) exists for |t1 |<h1 and |t2 |<h2
where h1 and h2 are positive, the mgf is given
as
M X1 X 2 (t )  E[e ]
t'X
where
t  (t1 , t 2 ) '
M X1 X 2 (t1 ,0) is the mgf of X1 and M X1 X 2 (0 , t 2 ) is the mgf of X 2 .
2.3 CONDITIONAL DISTRIBUTIONS AND
EXPECTATIONS
• So far we know
– How to find marginals given the joint distribution.
• Now
– Look at conditional distribution, distribution of
one of the random variable when the other has a
specific value.
Conditional pmf
• We define
P( X 1  x1 , X 2  x2 ) p X1, X 2 ( x1 , x2 )
p X 2 | X1 ( x2 | x1 ) 

P( X 1  x1 )
p X1 ( x1 )
x2  S X 2
• SX2 is the support of X2.
• Here we assume pX1 (x1) > 0.
• Thus conditional probability is the joint
divvied by the marginal.
Conditional pdf
• Let fX1x2 (x1, x2 ) be the joint pdf and fx1 (x1) and
fx2 (x2) be the marginals for X1 and X2
respectively then the conditional pdf of X2,
given X1 is
f X 2 | X 1 ( x2 | x1 ) 
f X 1 ( x1 )  0
f X 1, X 2 ( x1 , x2 )
f X 1 ( x1 )
 f 2|1 ( x2 | x1 )
Note
* f 2|1 ( x2 | x1 )  0

*  f 2|1 ( x2 | x1 )dx2  1

Conditional Expectation and Variance
If u(X 2 ) is a function of X 2 , the conditiona l expectatio n of
u(X 2 ), given that X1  x1if it exists, is given by

E[u ( X 2 ) | x1 ]   u ( x2 ) f 2|1 ( x2 | x1 )dx2

var( X 2 | x1 )  E ( X | x1 )  [ E ( X 2 | x1 )]
2
2
2
Theorem
• Let (X1, X2) be a random vector such that the
variance of X2 is finite. Then
– E[E(X2 |X1)]=E(X2)
– Var[E(X2 |X1 )]≤ var(X2 )
2.4 The Correlation Coefficient
E[( X  1 )(Y   2 )]  E ( XY )  E ( X ) E (Y )
Cov( X , Y )  E[( X  1 )(Y   2 )]

Cov( X , Y )
 1 2
Here ρ is called the correlation coefficient
of X and Y.
Cov(X,Y) is the covariance between X and Y.
The Correlation Coefficient
• Note that -1 ≤ ρ≤ 1.
• For the bivariate case
– If ρ = 1, the graph of the line y = a + bx (b > 0)
contains all the probability of the distribution of X
and Y.
– For ρ = -1, the above is true for the line y = a + bx
with b < 0.
– For the non-extreme case, ρ can be looked as a
measure of the intensity of the concentration of
the probability of X and Y about a line y = a + bx.
Theorem
• Suppose (X,Y) have a joint distribution with
the variance of X and Y finite and positive.
Denote the means and variances of X and Y by
µ1 , µ2 and σ12 , σ22 respectively, and let ρ be
the correlation coefficient between X and Y. If
E(Y|X) is linear in X then
2
E (Y | X )   2  
( X  1 )
1
E (var( Y | X ))   22 (1   2 )
2.5 Independent random Variables
• If the conditional pdf f2|1 (x2|x1) does not depend
upon x1 then the marginal pdf of X2 equals the
conditional pdf f2|1 (x2|x1) .
• Let the random variables X and Y have joint pdf
f(x,y) and the marginals fx (x) and fy (y)
respectively. The random variables X and Y are
said to be independent if and only if
– f(x,y)= fx (x) fy (y)
– Similar defintion can be wriiten for discrete random
variables.
– Random variables that are not independent are said
to be dependent.
Theorem
• Let the random variables X and Y have support
S1 and S2, respectively and have the joint pdf
f(x,y). Then X and Y are independent if and
only if f(x,y) can be written as a product of a
nonnegative function of x and a nonnegative
function of y. That is f(x,y)=g(x)h(y) where
g(x)>0 and h(y)>0.
Note
• In general X and Y must be dependent of the
space of positive probability density of X and Y
is bounded by a curve that is neither a
horizontal or vertical line.
• Example; f(x,y)=8xy, 0< x< y < 1
– S={(x,y): 0< x< y < 1} This is not a product space.
Theorems
• Let (X, Y) have the joint cfd F(x,y) and let A and
Y have the marginal cdfs Fx (x) and Fy (y)
respectively. Then X and Y are independent if
and only if
– F(x,y)= Fx (x)Fy (y)
• The random variable X and Y are independent
if and only if the following condition holds.
– P(a < X≤ b, c < Y ≤ d)= P(a < X≤ b)P( c < Y ≤ d)
– For ever a < b, c < d and a,b,c and are constants.
Theorems
• Suppose X and Y are independent and that
E(u(X)) and E(v(Y)) exist, then
– E[u(x), v(Y)]=E[u(X)]E[v(Y)]
• Suppose the joint mgf M(t1,t2) exists for the
random variables X and Y. Then X and Y are
independent if and only if
– M(t1,t2) = M(t1,0)M(0,t2)
• That is the joint mfg if the product of the marginal
mgfs.
Note
• If X and Y are independent then the
correlation coefficient is zero.
• However a zero correlation coefficient does
not imply independence.