Download CHAPTER 4 Multiple Random Variable

CHAPTER 4
Multiple Random Variable








4.1 Vector Random Variables
4.2 Pairs of Random Variables
4.3 Independence of Two Random Variables
4.4 Conditional Probability and Conditional
Expectation
4.5 Multiple Random Variables
4.6 Functions of Several Random Variables
4.7 Expected Value of Functions of Random
Variables
4.8 Jointly Gaussian Random Variables
4.1 Vector Random Variables
A vector random variable X is a function that assigns a vector of real
numbers to each outcome ζ in S, the sample space of the random
experiment .
EXAMPLE 4.1
Let a random experiment consist of selecting a student’s name form
an urn. Let ζdenote the outcome of this experiment, and define the
following three functions :
H    height of sudent  in inches,
W    weight of student ζ in pounds, and
A   age of student ζ in years.
Events and Probabilities
EXAMPLE 4.4
Consider the tow-dimensional random variable X = (X, Y). Find the
region of the plane corresponding to the events
A  X  Y  10,
B  min( X , Y )  5, and


C  X 2  Y 2  100 .
The regions corresponding to events A and C are
straightforward to find and are shown in Fig. 4.1.
For the n-dimensional random variable X = (X1,…,Xn), we are
particularly interested in events that have the product form
A  X 1 in A1 X 2 in A2   X n in An ,
(4.1)
where Ak is a one-dimensional event (ie., subset of the real line) that
involves Xk only.
A fundamental problem in modeling a system with a vector
random variable X = (X1,…, Xn) involves specifying the probability of
product-form events :
PA  PX 1 in A1 X 2 in A2    X n in An 
 PX 1 in A1 ,, X n in An .
(4.2)
In principle, the probability in Eq. (4.2) is obtained by finding the
probability of the equivalent event in the underlying sample space,
PA  P in S such that X  in A.
EXAMPLE 4.5
None of the events in Example 4.4 are of product form. Event B is
the union of two product-form events :
B  X  5 and Y   X  5 and Y  5.
The probability of a non-product-form event B is found as
follow : First, B is approximated by the union of disjoint product-form
events, say, B1, B2,…, Bn ; the probability of B is then approximated by


PB  P  Bk    PBk .
k
 k
The approximation becomes exact in the limit as the Bk’s become
arbitrarily fine.
Independence
If the one-dimensional random variable X and Y are “independent,” if
A1 is any event that involves X only and A2 is any event that involves
Y only, then
PX in A1, Y in A2   PX in A1 PY in A2 .
In the general case of n random variables, we say that the random
variables X1, X2,…, Xn are independent if
PX 1 in A1 ,, X n in An   PX 1 in A1  PX n in An  ,
where the Ak is an event that involves Xk only.
(4.3)
4.2 PAIRS OF RANDOM VARIABLES
Pairs of Discrete Random Variable
Let the vector random variable X = (X,Y) assume values from some
countable set S  ( x j , yk ), j  1,2,, k  1,2, . The joint
probability mass function of X specifies the probabilities of the
product-form event X  x j  Y  yk :

p X ,Y ( x j ,



y )  PX  x  Y  y 
k

j
 P X  x j , Y  yk

k
j  1,2,
k  1,2,
(4.4)
The probability of any event A is the sum of the pmf over the
outcomes in A :
PX in A 
 p
( x j , yk ) in A
X ,Y
( x j , yk ) .
(4.5)
The fact that the probability of the sample space S is 1 gives


 p
j 1 k 1
X ,Y
( x j , yk )  1 .
(4.6)
The marginal probability mass functions :


 PX  x , Y  anything 
 PX  x and Y  y  X  x
pX (x j )  P X  x j
j
j
1
j

and Y  y2  

  p X,Y (x j ,yk ) ,
(4.7a)
k 1
and similarly,
pY ( yk )  PY  yk 

  p X,Y ( x j ,yk ) .
j 1
(4.7b)
EXAMPLE 4.7
The number of bytes N in a message has a geometric distribution
with parameter 1-p and range SN={0, 1, 2, …}. Suppose that
messages are broken into packets of maximum length M bytes . Let
Q be the number of full packets in a message and let R be the
number of bytes left over. Find the joint pmf and the marginal pmf’s
of Q and R.
SQ={0, 1, 2,….} and SR={0, 1, 2, ….M – 1} .
The probability of the elementary event {(q, r)} is given by
PQ  q, R  r   PN  qM  r   1  p  p qM  r .
The marginal pmf of Q is
PQ  q   PN in qM , qM  1,, qM  ( M  1)
M 1
  1  p  p qM  r
k 0
 1  p  p

 1 p
M
qM
1 pM
1 p
q  0 ,1,2 ,
 p  .
M q
The marginal pmf of R is
PR  r   PN in r , M  r ,2 M  r , )

  1  p  p qM  r
q 0

1  p  p r
1 p
M
r  0,1, ,M-1.
The Joint cdf of X and Y
The joint cumulative distribution function of X and Y is defined as the
probability of the product-form event X  x1 Y  y1":
FX ,Y ( x1 , y1 )  PX  x1 , Y  y1 .
(4.8)
The joint cdf is nondecreasing in the “northeast” direction,
(i)
FX,Y (x1,y1 )  FX,Y (x2 ,y2 ) if x1  x2 and y1  y2 ,
It is impossible for either X or Y to assume a value less than  ,
therefore
(ii)
FX,Y (  ,y1 )  FX,Y (x2 ,  )  0
It is certain that X and Y will assume values less than infinity,
therefore
(iii)
FX,Y (,)  1.
If we let one of the variables approach infinity while keeping the
other fixed, we obtain the marginal cumulative distribution functions
(iv)
and
FX ( x)  FX ,Y ( x, )  PX  x, Y    PX  x
FY ( y )  FX ,Y (, y )  PY  y .
Recall that the cdf for a single random variable is continuous
form the right. It can be shown that the joint cdf is continuous from
the “north” and from the “east”
(v)
lim FX ,Y ( x, y )  FX ,Y (a, y )
x a 
and
lim FX ,Y ( x, y)  FX ,Y ( x, b)
y b 
EXAMPLE 4.8
The joint cdf for the vector of random variable X = (X,Y) is given by


 1  e x 1  e  x
FX ,Y ( x, y)  
0

x  0, y  0
elsewhere.
Find the marginal cdf’s.
The marginal cdf’s are obtained by letting one of the variables
approach infinity :
FX ( x)  lim FX ,Y ( x, y)  1  e x
x0
FY ( y)  lim FX ,Y ( x, y)  1  e x
y0
y 
x
The cdf can be used to find the probability of events that can be
expressed as the union and intersection of semi-infinite rectangles.
Consider the strip defined by x1  X  x2 and Y  y1, denoted by
the region B in Fig. 4.6(a) .
By the third axiom of probability we have that
FX ,Y ( x2 , y1 )  FX ,Y ( x1 , y1 )  Px1  X  x2 , Y  y1 
The probability of the semi-infinite strip is therefore
Px1  X  x2 , Y  y1   FX ,Y ( x2 , y1 )  FX ,Y ( x1 , y1 ) .
Consider next the rectangle x1  X  x2 , y1  Y  y2  denoted
by the region A in Fig 4.6 (b).
FX ,Y ( x2 , y2 )  Px1  X  x2 , y1  Y  y2 
 FX ,Y ( x2 , y1 )  FX ,Y ( x1 , y1 )  FX ,Y ( x1 , y2 ) .
The probability of the rectangle is thus
(vi)
Px  X  x2 , y1  Y  y2 
 FX,Y (x2 ,y2 )  FX,Y (x2 ,y1 )  FX,Y (x1,y2 )  FX,Y (x1,y1 )
EXAMPLE 4.9
Find the probability of the events A  X  1, Y  1, B  X  x, Y  y,
where x > 0 and y > 0, and D  1  X  2,2  Y  5 in Example 4.8
The probability of A is given directly by the cdf :



PA  PX  1, Y  1  FX ,Y (1,1)  1  e 1  e  .
The probability of B requires more work. Consider Bc
B c  (X  x Y  y)c  X  x Y  y ,
The probability of the union of two events :
 
P B c  PX  x   PY  y   PX  x, Y  y 

 
 

 1  e x  1  e  y  1  e x 1  e  y

 1  e x e  y .
The probability of B :
 
PB  1  P B c  ex e y .
The probability of event D is found by applying property vi of the
joint cdf :
P1  X  2,2  Y  5
 FX ,Y (2,5)  FX ,Y (2,2)  FX ,Y (1,5)  FX ,Y (1,2)


 


 1  e 1  e   1  e 1  e 
 1  e  2 1  e 5   1  e  2 1  e  2 

5 

2 
The Joint pdf of Two Jointly Continuous Random
Variables
We say that the random variables X and Y are jointly continuous
if the probabilities of events involving (X, Y) can be expressed as an
integral of a pdf. There is a nonnegative function fX,Y(x,y), called the
joint probability density function, that is defined on the real plane
such that for every event A, a subset of the plane,
PX in A  
A
f
X ,Y
( x' , y' )dx' dy' ,
(4.9)
as shown in Fig. 4.7. When a is the entire plane, the integral must
equal one :
1 



  
f X ,Y ( x' , y' )dx' dy' .
(4.10)
The joint cdf can be obtained in terms of the joint pdf of jointly
continuous random variables by integrating over the semi-infinite
rectangle defined by (x, y) :
FX ,Y ( x, y)  
x

y
 
(4.11)
f X ,Y ( x' , y' )dx' dy'
It then follows that if X and Y are jointly continuous random variables,
then the pdf can be obtained from the cdf by differentiation :
f X ,Y ( x, y ) 
FX ,Y ( x, y )
.
(4.12)
xy
The probability of a rectangle region is obtained by letting
A  x, y  : a1  x  b1 and a2  y  b2  in Eq. (4.9) :
Pa1  X  b1, a2  Y  b2   
b1 b2

a1 a2
f X ,Y ( x' , y ' )dx' dy ' .
Px  X  x  dx, y  Y  y  dy   
x  dx
x

y  dy
y
(4.13)
f X ,Y ( x' , y ' )dx' dy '
 f X,Y (x,y)dxdy
(4.14)
The marginal pdf’s fX(x) and fY(y) are obtained by taking the
derivative of the corresponding marginal cdf’s , FX ( x)  FX ,Y ( x, )
and FY ( y )  FX ,Y (, y ) .
d x
FX ( x) 
dx 






f X ,Y ( x' , y ' )dy ' dx'
f X ,Y ( x, y ' )dy ' .
(4.15a)
FY ( y)   f X ,Y ( x' , y)dx' .
(4.15b)

Similarly,


EXAMPLE 4.10 Jointly Uniform Random Variables
A randomly selected point (X, Y) in the unit square has the uniform
joint pdf given by
1 0  x  1 and 0  y  1
f X ,Y ( x, y )  
0 elsewhere.
Find the joint cdf.
There are five cases in this problem, corresponding to the five
regions shown in Fig. 4.9.
1. If x < 0 or y < 0, the pdf is zero and Eq. (4.12) implies
FX ,Y ( x, y )  0
2. If (x,y) is inside the unit interval,
FX ,Y ( x, y)  
x
0
y
 1dx' dy'  xy .
0
3. If 0  x  1 and y  1 ,
FX ,Y ( x, y)  
x 1
0
 1dx' dy'  x .
0
4. Similarly, if x  1 and 0  y  1 ,
FX ,Y ( x, y )  y .
5. Finally, if x  1 and y  1 ,
1 1
FX ,Y ( x, y)  
 1dx' dy'  1.
0 0
EXAMPLE 4.11
Find the normalization constant c and the marginal pdf’s for the
following joint pdf :
ce  x e  y 0  y  x  
f X ,Y ( x, y)  
elsewhere .
0
The constant c is found from the normalization condition
specified by Eq. (4.10) :
 x

c
x  y
1    ce e dydx   ce  x 1  e  x dx  .
0 0
0
2
Therefore c= 2. The marginal pdf’s are found by evaluating Eq.
(4.15a) and (4.15b) :



0
0


0
0


f X ( x)   f X ,Y ( x, y)dy   2e x e y dy  2e x 1  e x

0 x
and
fY ( y)   f X ,Y ( x, y)dx   2e x e y dx  2e2 y
0 y
EXAMPLE 4.13 Jointly Gaussian Random Variables
The joint pdf of X and Y, shown in Fig. 4.11 is
1
f X ,Y ( x, y ) 
exp  ( x 2  2 xy  y 2 ) / 2(1   2 )    x,y  
2 1   2
We say that X and Y are jointly Gaussian. Find the marginal pdf’s.


The marginal pdf of X is found by integrating fX,Y(x,y) over y :
f X ( x) 
e
 x 2 / 2 (1  2 )







exp  y 2  2 xy 2(1   2 ) dy.
2 1  
We complete the square of the argument of the exponent by adding
2
2
2 2
2 2
2 2
and subtracting ρ2x2 , that is y  2 xy   x   x   y  x    x
f X ( x) 

2
e
 x 2 / 2 (1  2 )
2 1  
e
 x2 / 2
2



2
e
-
-



exp   y  x    2 x 2 2(1   2 ) dy
2
  y  x 2 2 (1  2 )

2 1  
2

dy 
e
 x2 / 2
2
,
Random Variables That Differ in Type
EXAMPLE 4.14 A Communication Channel with Discrete Input and
continuous Output
Let X be the input , Y be output and N be noise.
PX  1  PX  1  1/ 2 , and Y  X  N where N : U (2,2)
Find PX  1, Y  0.
PX  k , Y  y  PY  y | X  k PX  k ; therefore
PX  1, Y  y  PY  y | X  1PX  1 ,
where P[X = +1] = 1 / 2. When the input X = 1, the output Y is
uniformly distributed in the interval [-1, 3]; therefore
y 1
PY  y | X  1 
for  1  x  3 .
4
1 1 1
PX  1, Y  0  PY  0 | X  1PX  1   
4 2 8
4.3 INDEPENDENCE OF TWO RANDOM
VARIABLES
X and Y are independent random variables if any event A1 defined in
terms of X is independent of any event A2 defined in terms of Y ;
PX in A1 , Y in A2   PX in A1 PY in A2 .
(4,17)
Suppose that X and Y are a pair of discrete random variables. If we
let A1  X  x j and A2  Y  yk , then the independence of X and
Y implies that
p X ,Y ( x j , yk )  P X  x j , Y  yk




 PX  x PY  y 
j
 p X ( x j ) pY ( yk )
k
for all x j and yk .
(4.18)
Therefore, if X and Y are independent discrete random variables, then the
joint pmf is equal to the product of the marginal pmf’s
Let A  A1  A2 be a product-form event as above, then
PA 
 p
X ,Y
( x j , yk )
x j in A1 y k in A2

 p
X
( x j ) pY ( yk )
x j in A1 y k in A2
 PA1 PA2  ,
(4.19)
We say, The “discrete random variables X and Y are independent if and
only if the joint pmf is equal to the product of the marginal pmf’s for all xj,
yk ”
EXAMPLE 4.16
Are Q and R in Example 4.7 independent? From Example 4.7 we
have

 11pp  p
PQ  q PR  r   1  p M p M
r
M
 1  p  p Mq r
 PQ  q, R  r 
for all q  0,1,
r  0, , M  1 .
Therefore Q and R are independent.
It can shown that the random variables X and Y are independent if and
only if their joint cdf is equal to the product of its marginal cdf’s :
FX ,Y ( x, y )  FX ( x) FY ( y )
for all x and y .
(4.20)
Similarly, if X and Y are jointly continuous, then X and Y are
independent if and only if their joint pdf is equal to the product of the
marginal pdf’s :
f X ,Y ( x, y )  f X ( x) fY ( y )
for all x and y .
(4.21)
EXAMPLE 4.18
Are the random variables X and Y in Example 4.13 independent?
The product of the marginal pdf’s of X and Y in Example 4.13 is
1 x 2  y 2 / 2
f X ( x) fY ( y ) 
e
   x, y  
2
The jointly Gaussian r.v’s X and Y are indepdent if and only if ρ=0.
EXAMPLE 4.19
Are the random variables X and Y independent in Example 4.8? If we
multiple the marginal cdf’s found in Example 4.8 we find



FX ( x) FY ( y)  1  ex 1  e y  FX ,Y ( x, y)
for all x and y
so X and Y are independent.
If X and Y are independent random variables, then the random
variables defined by any air of functions g(X) and h(Y) are also
independent.
1. Consider the one-dimensional events A and B.
2. Let A’ be the set of all values of x such that if x is in A’ then
g(x) is in A,
3. Similarly, let B’ be the set of all values of y. then
Pg ( x) in A, h( y ) in B  PX in A' , Y in B'
 PX in A'PY in B'
 Pg ( x) in APh( y ) in B.
4.4 CONDITIONAL PROBABILITY AND
CONDITIONAL EXPECTATION
Conditional Probability
In Section 2.4, we know
PY in A | X  x 
PY in A, X  x
.
PX  x
(4.22)
If X is discrete, then Eq. (4.22) can be used to obtain the
conditional cdf of Y given X = xk :
FY ( y | xk ) 
PY  y, X  xk 
,
P X  x k 
for PX  xk   0 .
(4.23)
The conditional pdf of Y given X = xk , if the derivative exists, is given
by
d
fY ( y | xk ) 
FY ( y | xk ) .
(4.24)
dy
Integrating the conditional pdf :
PY in A | X  x   
y in A
(4.25)
fY ( y | xk )dy .
Note that if X and Y are independent, PY  y, X  xk   PY  y *
PX  xk  so FY ( y | x)  FY ( y) and fY ( y | x)  fY ( y) .
If X and Y are discrete, then

pY ( y j | xk )  P Y  y j


p (x , y )
| X  x 

.
PX  x 
p (x )
P X  xk , Y  y j
x, y
k
j
k
k
X
k
(4.26)
for xk such that PX  xk   0 . We defined pY ( y j | xk )  0 for xk such
that PX  xk   0 . The probability of any event A given X = xk is
found by
PY in A | X  xk  
p
Y
y j in A
( y j | xk ) .
(4.27)
Note that if X and Y are independent, then
pY ( y j | xk ) 

PX  xk P Y  y j
PX  xk 
  PY  y   p ( y ) .
j
Y
j
EXAMPLE 4.20
Let X be the input and Y the output of the communication channel
discussed in Example 4.14. Find the probability that Y is negative
given that X is +1.
If X =+1, then Y is uniformly distributed in the interval [-1, 3], that
is ,
1

fY ( y | 1)   4
1
1  y  3
elsewhere
Thus
dy 1
PY  0 | X  1  
 .
1 4
4
0
If X is a continuous random variable, then P[X = x] = 0 so Eq.
(4.22) is undefined. We define the conditional cdf of Y given X = x by
the following limiting procedure:
FY ( y | x)  lim FY ( y | x  X  x  h) .
h 0
The conditional cdf on the right side of Eq. (4.28) is :
PY  y, x  X  x  h
FY ( y | x  X  x  h) 
Px  X  x  h
y
 

xh
 x

xh
x
f X ,Y ( x' , y ' )dx' dy '
f X ( x' )dx'
(4.28)


y

f X ,Y ( x, y ' )dy ' h
f X ( x)h
.
(4.29)
As we let h approach zero,

F ( y | x) 
y

Y
f X ,Y ( x, y ' )dy '
f X ( x)
.
(4.30)
The conditional pdf of Y given X = x is obtained by
fY ( y | xk ) 
f ( x, y)
d
FY ( y | xk )  X ,Y
.
dy
f X ( x)
(4.31)
Note that if X and Y are independent, then f X ,Y ( x, y )  f X ( x) *
fY ( y) and fY ( y | x)  fY ( y) and FY ( y | x)  FY ( y) .
EXAMPLE 4.21
Let X and Y be the random variables introduced in Example 4.11.
Find f X ( x | y) and fY ( y | x) .
Using the marginal pdf’s
2e  x e  y
 x  y 
f X ( x | y) 

e
2e 2 y
and
for x  y
2e  x e  y
e y
f Y ( y | x)   x

x
2e 1  e
1  e x


for 0  y  x .

ce  x e  y
f X ,Y ( x, y)  
0
x
0 y x
elsewhere .

f X ( x)   f X ,Y ( x, y )dy   2e  x e  y dy  2e  x 1  e  x
0
0


0
0
fY ( y)   f X ,Y ( x, y)dx   2e xe y dx  2e2 y

0x
0 y
If we multiply Eq. (4.26) by P[ X = xk ], then

 

P X  xk , Y  y j  P Y  y j | X  xk PX  xk 
(4.32)
p x , y ( xk , y j )  pY ( y j | xk ) p X ( xk ) .
Suppose we are interested in the probability that Y is in A :
PY in A 
 p
X ,Y
( xk , y j )
all xk y j in A

 p
Y
( y j | xk ) p X ( x k )
all xk y j in A

PY in A 
p
all xk
X
 PY
all xk
( xk )
p
Y
( y j | xk )
y j in A
in A | X  xk  p X ( xk ) .
(4.33)
If X and Y are continuous, we multiply Eq. (4.31) by fX(x)
f X ,Y ( x, y )  fY ( y | xk ) f X ( x) .
(4.34)
To replace summations with integrals and pmf’s with pdf’s ,
PY in A   PY in A | X  x f X ( x)dx .


(4.35)
EXAMPLE 4.22 Number of Defects in a Region; Random Splitting
of Poisson Counts
The total number of defects X on a chip is a Poisson random variable
with mean α. Suppose that each defect has a probability p of falling
in a specific region R and that the location of each defect is
independent of the locations of all other defects. Find the pmf of the
number of defects Y that fall in the region R.
Form Eq. (4.33)

PY  j    PY  j | X  k PX  k .
k 0
The total number of defect : X = k,
the number of defects that fall in the region R is a binomial r.v with k,
p
jk
0

PY  j | X  k    k  j
k j




p
1

p
0 jk.
 j 
 
Noting that k  j ,

PY  j   
k j
k
k!

k

j
p j 1  p 
e 
j!(k  j )!
k!

p  j e 

j!


k j
1  p  k  j
1-p
(k  j )!

p  j e  1 p  p  j p

e

e .
j!
j!
Thus Y is a Poisson r.v with mean αp.
p
EXAMPLE 4.23 Number of Arrivals During a Customer’s Service
Time
The number of customers that arrive at a service station during a
time t is a Poisson random variable with parameter βt. The time
required to service each customer is an exponential random variable
with parameter α. Find the pmf for the number of customers N that
arrive during the service time T of a specific customer. Assume that
the customer arrivals are independent of the customer service time.
PN  k    PN  k | T  t  fT (t )dt

0


0

t k e te t dt
 k
k!
k!


0
k    t
t e
dt .


 nt n1e t
(n  1)!
0


0
k  t
t e
dt  1
dt 
k!
 k 1
Let r = (α+β)t, then

 k
k r
PN  k  
r
e dr
k 1 0
k!   

    





k 1
   
        
k
k
where we have used the fact that the last integral is a gamma
function and is equal to k!.
Conditional Expectation
The conditional expectation of Y given X = x is defined by
EY | x   yfY ( y | x)dy .


(4.36a)
If X and Y are both discrete random variables, we have
EY | x   ypY ( y j | x)
yj
(4.36b)
We now show that
EY   EEY | X  ,
(4.37)
where the right-hand side is
EEY | X    EY | x f X ( x)dx
X continuous
EEY | X    EY | xk p X ( xk )
X discrete


and
xk
We prove Eq. (4.37) for the case where X and Y are jointly
continuous random variables, then
EEY | X    EY | x f X ( x)dx






 
yfY ( y | x)dyf X ( x)dx




  y  f X ,Y ( x, y )dxdy
  yfY ( y)dy  EY .


The above result also holds for the expected value of a function of Y :
Eh( y)  EEh(Y ) | X .
The kth moment of Y is given by
  

E Yk  E E Yk | X .
EXAMPLE 4.25 Average Number of Defects in a Region
Find the mean of Y in Example 4.22 using conditional expectation.


k 0
k 0
E Y    EY | X  k PX  k    kpPX  k   pEX   p .
EXAMPLE 4.26 Average Number of Arrivals in a Service Time
Find the mean and variance of the number of customer arrivals N
during the service time T of a specific customer in Example 4.23 .
We will need the first two conditional moments of N given T = t:
EN | T  t   t


E N 2 | T  t  t  t  ,
2
The first two moment of N are
EN    EN | T  t  fT (t )dt   tfT (t )dt  βET 
 


0
0






E N   E N | T  t fT (t )dt   t   2t 2 fT (t )dt
2
0
2
 
 βE T   β 2 E T 2 .
0
The variance of N is then
 
  E T  ET    ET 
VARN   E N 2  EN 
2
2
2
2
2
  2 VART   ET .
If T is exponential with parameter α, then E[T] = 1/α and VAR[T] =
1/α2 , so
β
EN  
α
and
2 
VAR N   2  .
 
4.5 MULTIPLE RANDOM VARIABLES
Joint Distributions
The joint cumulative distribution function of X1, X2,…., Xn is defined
as the probability of an n-dimensional semi-infinite rectangle
associate with the point (x1,…, xn):
FX1 , X 2 ,X n ( x1 , x2 , xn )  PX 1  x1 , X 2  x2 ,, X n  xn . (4.38)
The joint cdf is defined for discrete, continuous, and random
variables of mixed type.
EXAMPLE 4.27
Let the event A be defined as follows :
A  max  X 1 , X 2 , X 3   5.
Find the probability of A .
The maximum of three numbers is less than 5 if and only if
each of the three numbers is less than 5 ; therefore
PA  PX 1  5 X 2  5 X 3  5
 FX1 , X 2 , X 3 (5,5,5) .
The joint probability mass function of n discrete random
variables is defined by
p X1 , X 2 ,X n ( x1 , x2 , xn )  PX 1  x1 , X 2  x2 , X n  xn .
(4.39)
The probability of any n-dimensional event A is found by summing
the pmf over the points in the event
P X1 , X 2 , X n  in A   
x in A
p
X1 , X 2 ,X n
( x1 , x2 , xn ) . (4.40)
One-dimensional pmf of Xj is found by adding the joint pmf over all
variables other than xj:


p X j ( x j )  P X j  x j    p X1 , X 2 ,, X n ( x1 , x2 , xn ) . (4.41)
x1
x j 1 x j 1
xn
The marginal pmf for X1,…,Xn-1 is given by
p X1 , X 2 ,, X n1 ( x1 , x2 , xn1 )   p X1 , X 2 ,, X n ( x1 , x2 , xn ) .
xn
(4.42)
A family of conditional pmf’s is obtained from the joint pmf by
conditioning on different subsets of the random variables.
p X n xn | x1 ,, xn1  
p X1 ,, X n x1 ,, xn1 
p X1 ,, X n 1 xn | x1 ,, xn1 
,
(4.43a)
if p X1 ,, X n x1 ,, xn 1   0 . Repeated applications of Eq. (4.43a) yield
p X1 ,, X n x1 ,, xn 1   p X n xn | x1 ,, xn 1 
 p X n1 xn 1 | x1 ,, xn  2  p X 2 x2 | x1  p X1 x1 
(4.43b)
EXAMPLE 4.28
A computer system receives messages over three communications
lines. Let Xj be the number of messages received on line j in one
hour. Suppose that the joint pmf of X1, X2, and X3 is given by
p X1 , X 2 , X 3 x1 , x2 , x3   1  a1 1  a2 1  a3 a1x1 a2x2 a3x3
x1  0,x2  0,x3  0
Find p(x1, x2) and p(x1) given that 0< ai < 1.

p X1 , X 2 x1 , x2   1  a1 1  a2 1  a3  a1x1 a2x2 a3x3
x3 0
 1  a1 1  a2 a1x1 a2x2 .

p X1 x1   1  a1 1  a2  a1x1 a2x2
 1  a1 a1x1 .
x2  0
If r.v’s X1, X2,…,Xn are jointly continuous random variables, then
the probability of any n-dimensional event A is
P X 1 ,, X n  in A      f X1 ,, X n ( x1' ,, xn' )dx1'  dxn' ,
x in A
(4.44)
'
'
where f X1 ,, X n ( x1 ,, xn ) is the joint probability density function
The joint cdf of X is obtained from the joint pdf by integration :
x1
xn


FX1 , X 2 ,, X n ( x1 , x2 ,, xn )    f X1 ,, X n ( x1' ,, xn' )dx1' dxn' , (4.45)
The joint pdf (if the derivative exists) is given by
n
'
'
f X1 , X 2 , X n ( x1 ,, xn ) 
FX1 ,, X n ( x1 ,, xn ) .
(4.46)
x1 xn
The marginal pdf for a subset of the random variables is
obtained b integrating the other variables out. The marginal of X1 is




f X1 ( x1 )     f X1 ,, X n ( x1 , x2' ,, xn' )dx2' dxn' .
(4.47)
The marginal pdf for X1,…,Xn-1 is given by

f X1 ,, X n1 ( x1 ,, xn1 )   f X1 ,, X n ( x1 ,, xn1 , xn' )dxn' .

(4.48)
The pdf of Xn given the values of X1,…,Xn-1 is given by
f X n ( xn | x1 ,, xn1 ) 
f X1 ,, X n ( x1 ,, xn )
f X1 ,, X n1 ( x1 ,, xn1 )
(4.49a)
if f X 1 ,, X n1 ( x1 , , xn 1 )  0
Repeated applications of Eq. (4.49a) yield
f X1 ,, X n ( x1 ,, xn )  f X n ( xn | x1 ,, xn 1 )
 f X n1 ( xn 1 | x1 ,, xn  2 )  f X 2 ( x2 | x1 ) f X1 ( x1 )
(4.49b)
EXAMPLE 4.29
The r.v’s X1, X2, and X3 have the joint Gaussian pdf
f X1 , X 2 , X 3 x1 , x2 , x3  
e
1 

 x12  x22  2 x1 x2  x32 
2 

2 
.
Find the marginal pdf of X1 and X3 .
f X1 , X 3  x1 , x3  
e

e 
dx2 .


2
2 / 2
 x32 / 2
 x12  x22  2 x1 x2

The above integral was carried out in Example 4.13 with   1 / 2
e  x3 / 2 e  x1 / 2
f X1 , X 3  x1 , x3  
.
2 2
2
2
Independence
X1,…,Xn-1 are independent if
PX 1 in A1 ,, X n in An   PX 1 in A1  PX n in An 
for any one-dimensional events A1,…,An. It can be shown that
X1,…,Xn are independent if and only if
FX1 ,, X n ( x1 ,, xn )  FX1 ( x1 )  FX n ( xn )
(4.50)
for all x1,…,xn. If the random variables are discrete,
p X1 ,, X n ( x1 ,, xn )  p X1 ( x1 )  p X n ( xn )
for all x1, ,xn .
If the random variables are jointly continuous,
f X1 ,, X n ( x1 ,, xn )  f X1 ( x1 )  f X n ( xn )
for all x1, ,xn .
4.6 FUNCTIONS OF SEVERAL RANDOM
VARIABLES
One Function of Several Random Variables
Let the random variable Z be defined as a function of several random
variables:
Z  g  X 1 , X 2 ,, X n  .
(4.51)
The cdf of Z is found by first finding the equivalent event of Z  z,
that is, the set RZ  x  x1 ,, xn  such that g x   z, then
FZ ( z )  PX in Rz 
 
x in Rz



f X1 ,, X n x1' ,, xn' dx1'  dxn' .
(4.52)
EXAMPLE 4.31 Sum of Two Random Variables
Let Z = X + Y. Find FZ(z) and fZ(z) in terms of the joint pdf of X and Y.
The cdf of Z is
FZ ( z)  


z  x'
  
f X ,Y ( x' , y' )dy' dx' .
The pdf of Z is

d
f Z ( z) 
FZ ( z )   f X ,Y ( x' , z  x' )dx' .
(4.53)


dz
Thus the pdf for the sum of two random variables is given by a
superposition integral.
If X and Y are independent random variables, then by Eq. (4.21)
the pdf is given by the convolution integral of the margial pdf’s of X
and Y :

f Z ( z )   f X ( x' ) fY ( z  x' )dx' .

(4.54)
EXAMPLE 4.32 Sum of Nonindependent Gaussian Random
Variables
Find the pdf of the sum Z = X + Y of two zero-mean, unit-variance
Gaussian random variables with correlation coefficient ρ= -1 / 2.

f Z ( z )   f X ,Y ( x' , z  x' )dx'



 x'2 2 x' z  x'   z  x'2 

exp 
dx'
2
2 1 2  
2 1 
2 1  



 x '2  x ' z  z 2 
1

exp 
dx' .
1 2  
23 / 4 
2 3 / 4

After completing the square of the argument in the exponent we
obtain
 z2 / 2
e
f Z ( z) 
.
2

1






Let Z = g(X, Y), and suppose we are given that Y = y, then Z =
g(X, y) is a function of one random variable. And the pdf of Z given Y
= y: fZ(z | Y = y). The pdf of Z is found from

f Z ( z)   f Z ( z | y' ) fY ( y' )dy' .

EXAMPLE 4.34
Let Z = X / Y. Find the pdf of Z if X and Y are independent and both
exponentially distributed with mean one.
Assume Y = y, then
f Z ( z | y)  y f X ( yz | y) .
The pdf of Z is
f Z ( z)  





y ' f X ( y ' z | y ' ) fY ( y ' )dy '
y ' f X ,Y ( y ' z , y ' )dy ' .

f Z ( z )   y ' f X ( y ' z ) fY ( y ' )dy '
0

  y ' e  y ' z e  y ' dy '
0

1
1  z 2
z  0.
Transformations of Random Vectors
Let X1,…, Xn be random variables associate with some experiment,
and let the random variables Z1,…, Zn be defined by n functions of X
= (X1,…, Xn) :
Z1  g1 ( X)
Z 2  g 2 ( X)  Z n  g n ( X) .
The joint cdf of Z1,…, Zn at the point z = (z1,…, zn) is equal to
the probability of the region of x where g k (x)  zk for k  1,..., n :
FZ1 ,, Z n ( z1 ,, zn )  Pg1 ( X)  z1 ,, g n ( X)  zn .
(4.55a)
If X1,…, Xn have a joint pdf, then
FZ1 ,,Z n ( z1 ,, zn ) 
EXAMPLE 4.35
  f
'
'
'
'
(
x
,...,
x
)
dx

dx
(4.55b)
X 1 ,..., X n
1
n
1
n .
x ':g k ( x ') z k
Let the random variables W and Z be defined by
W  min( X , Y )
and
Z  max( X , Y ) .
Find the joint cdf of W and Z in terms of the joint cdf of X and Y.
FW , Z ( w, z )  Pmin( X , Y )  w max  X , Y   z.
If z > w, the above probability is the probability of the semi-infinite
rectangle defined by the point (z, z) minus the square region denote
by A.
FW , Z ( w, z )  FX ,Y ( z , z )  PA
 FX ,Y ( z , z )
 FX ,Y ( z , z )  FX ,Y ( w, z )  FX ,Y ( z , w)  FX ,Y ( w, w)
 FX ,Y ( w, z )  FX ,Y ( z , w)  FX ,Y ( w, w)
If z < w then
FW , Z ( w, z )  FX ,Y ( z, z ) .
pdf of Linear Transformations
We consider first the linear transformation of two random variables :
V  a b  X 
V  aX  bY
or
W    c e  Y  .
W  cX  eY
  
 
Denote the above matrix by A. We will assume A has an inverse, so
each point (v, w) has a unique corresponding point (x, y) obtained
from
x 
1 v 

A
 y
 w .
 
 
(4.56)
In Fig. 4.15, the infinitesimal rectangle and the parallelogram are
equivalent events, so their probabilities must be equal. Thus
f X ,Y ( x, y )dxdy  fV ,W (v, w)dP
where dP is the area of the parallelogram. The joint pdf of V and W is
thus given by
fV ,W (v, w) 
f X ,Y ( x, y )
,
(4.57)
dP
dxdy
where x an y are related to (v, w) by Eq. (4.56)
It can be shown that dP   ae  bc dxdy ,so the “stretch factor” is
ae  bc dxdy
dP

 ae  bc  A ,
dxdy
dxdy
where |A| is the determinant of A.
Let the n-dimensional vector Z be
Z  AX,
where A is an n n invertible matrix. The joint of Z is then
f Z (z)  f Z1 ,,Z n z1 ,, zn  
f X1 ,, X n x1 ,, xn 
A
x  A1 z
f x A1z 

(4.58)
A
EXAMPLE 4.36 Linear Transformation of Jointly Gaussian Random
Variables
Let X and Y be the jointly Gaussian random variables introduced in
Example 4.13. Let V and W be obtained from (X, Y) by
V  1  1 1  X 
W  
 1 1 Y  
2
 
 
X 
A  .
Y 
Find the joint pdf of V and W.
|A| = 1,
 X  1 1  1 V 
Y  
1 1  W  ,
2
 
 
so X  V  W 
2 and Y  V  W 
2.
vw v w
fV ,W (v, w)  f X ,Y 
,
,
2 
 2
where
f X ,Y x, y  
1
e
 
 x 2  2 xy  y 2 / 2 1  2
2 1   2
.
By substituting for x and y, the argument of the exponent becomes
v  w2 / 2  2 v  wv  w / 2  v  w2 / 2 
2 1   2 
Thus
fV ,W (v, w) 
1

e
1/ 2
2 1   2 

v2
w2

.
21    21   
 v 2 / 2 1    w2 / 2 1  
 .
pdf of General Transformations
Let the r.v’s V and W be defined by two nonlinear functions of X and
Y:
V  g1 ( X , Y ) and W  g2 ( X , Y ) .
(4.59)
Assume that the functions v(x, y) and w(x, y) are invertible, then
x  h1 (v, w) and
y  h2 (v, w) .
In Fig. 4.17(b) , make the approximation

g k ( x  dx, y )  g k ( x, y )  g k ( x, y )dx k  1,2
x
and similarly for the y variable. The probabilities of the infinitesimal
rectangle and the parallelogram are approximately equal. therefore
f X ,Y ( x, y )dxdy  fV ,W (v, w)dP
and
fV ,W (v, w) 
f X ,Y (h1 (v, w), h2 (v, w))
dP
dxdy
,
(4.60)
where dP is the area of the parallelogram. The “stretch factor” at the
point (v, w) is given by the determinant of a matrix of partial
derivatives :
 v
 x
J ( x, y )  det 
 w
 x
v 
y 
.
w 
y 
The determinant J(x, y) is called the Jacobian of the transformation.
The Jacobian of the inverse transformation is given by
 x
 v
J (v, w)  det 
y

 v
x 
w  .
y 

w 
It can be shown that
1
J (v, w) 
.
J ( x, y)
We therefore conclude that the joint pdf of V and W can be found
using either of the following expressions :
fV ,W (v, w) 
f X ,Y (h1 (v, w), h2 (v, w))
J  x, y 
 f X ,Y (h1 (v, w), h2 (v, w)) J v, w
(4.61a)
(4.61b)
EXAMPLE 4.37 Radius an Angle of Independent Gaussian Random
Variables
Let X and Y be zero-mean, unit-variance independent Gaussian
random Variables. Find the joint pdf of V and W defined by

V  X Y
2

2 1/ 2
W   X , Y  ,
where  denotes the angle in the range (0.2π) that is defined by
the point (x, y).
The inverse transformation is given by
x  v cos w
and
y  v sin w .
The Jacobian is given by
cos w  v sin w
J (v, w) 
 v.
sin w v cos w
Thus
v v 2 cos2 ( w)  v 2 sin2 ( w) / 2
fV ,W (v, w) 
e
2
1 v 2 / 2

ve
v  0, 0  w  2π .
2
The pdf of a Rayleigh random variable is given by
fV (v)  ve v
2
/2
v  0.
We therefore conclude that the radius V and the angle W are
independent random variables.
1
(1) fW ( w) 
, 0  w  2
2
(2) W : uniform random variable
EXAMPLE 4.38 Student’s t-distribution
Let X be a zero-mean, unit-variable Gaussian random variable and
let Y be a chi-square random variable with n degrees of freedom.
Assume that X and Y are independent. Find the pdf of V  X / Y / n
Define the auxiliary function W = Y. Then
X  V W/n and Y  W .
The Jacobian of the inverse transformation is
J (v, w) 
fV ,W (v, w) 
e
 x2 / 2
2
w/ n
(v / 2) wn
0
1
 y / 2
 w/ n .
e
2n / 2
(1) f X ,Y ( x, y) 
e
 x2 / 2
( w / 2)
e 

2 n n / 2
 n 1 / 2
n / 2 1  y / 2
J (v, w)
x v w / n
yw
( y / 2) n / 21 e y / 2
, y0
2(n / 2)
2
  w / 2  1 v 2 / n

.
The pdf of V is

1
 n 1 / 2  w / 2 1 v 2 / n 
fV ( v ) 
( w / 2)
e
dw .

0
2 n n / 2
  n 1 / 2
2

1

v
/
n
2
n 1/ 2  w'
let w'  w / 2(v / n  1) ,
fV (v) 
(
w
'
)
e dw' .

0
n n / 2
We finally obtain the Student’s t-distribution

1  v / n
(v) 
  n 1 / 2
n  1 / 2
n n / 2
2
fV

(2)
 (w / 2)
( n 1) / 2
e
 ( w / 2)(1 v 2 / n )
dw 
0

1
2
( n 1) / 2


( n21 ) !
1 v / n
2
2

n1 1
2

1
2

( n 1) / 2
w
e
( n 1) / 2 
w
(1 v 2 / n )
2
dw
0
2( n21 )!
1  v / n 
2
n 1
2


0
k  t
t e
dt 
k!
 k 1
k  (n  1) / 2 : integer
Problem:
1
 1 2

f X ,Y ( x, y ) 
exp    x  y 2  
2
 2

 v  3 x  5 y  3 5   x 
 w   x  2 y   1 2  y 
  
 
 
1
 x  3 5   v   2 1  v 
 y   1 2  w   1 3   w
  
   
 


1
2
2
1 
(1) fV ,W (v, w) 
exp  2  2v  5w    v  3w  


2
1
a b 
1  d b 
(2) 


 c a 
c
d
ad

bc




Consider the problem of finding the joint pdf for n functions of n
random variables X = (X1,…, Xn):
Z1  g1 (X) ,
Z 2  g 2 (X) ,, Z n  g n (X) ,
We assume as before that the set of equations
z1  g1 (x) ,
z2  g 2 (x) ,, zn  g n (x) ,
( 4.62)
has a unique solution given by
x1  h1 (x) ,
x2  h2 (x) ,, xn  hn (x) ,
The joint pdf of Z is then given by
f Z1 ,, Z n ( z1 ,, z n ) 
f X1 ,, X n h1 ( x ), h2 ( x), , hn ( x) 
J x1 , x2 ,, xn 
(4.63a)
 f X1 ,, X n h1 ( x), h2 ( x), , hn ( x)  J z1 , z 2 ,, z n  , (4.63b)
where J x1 ,, xn  and J z1 ,, zn  are the determinants of the
transformation and the inverse transformation, respectively,
 g1
 x
 1
J  x1 ,  , xn   det  
 g n
 x
 1
g1 
xn 

 
g n 

xn 

and
 h1
 z
 1
J  z1 ,  , z n   det  
 hn
 z
 1
h1 
z n 

 
hn 

z n 

4.7 EXPECTED VALUE OF FUNCTIONS OF
RANDOM VARIABLES
The expected value of Z = g(X, Y) can be found using the following
expressions :
   g x, y  f ( x, y )
X ,Y
 
EZ   
 g ( xi , yn ) p X ,Y ( xi , yn )
 i n
X, Y jointly continuous
X,Y discrete.
(4.64)
EXAMPLE 4.39 Sum of Random Variables
Let Z = X + Y . Find E[Z].
EZ   EX  Y 


 x' y' f ( x' , y' )dx' dy'
   x' f ( x' , y ' )dy ' dx'   y ' f ( x' , y ' )dx' dy '
  x' f ( x' )dx'  y ' f ( y ' )dy '  EX   EY .

X ,Y
 



X ,Y
 



 
X ,Y

X

Y
(4.65)
Thus, the result shows that the expected value of a sum of n random
variables is equal to the sum of the expected values :
E X 1  X 2    X n   E X 1     E X n .
(4.66)
In general if X1,…, Xn are independent random variables, then
E g1  X 1 g 2  X 2 x  g n  X n   E g1  X 1   E g 2  X 2   E g n  X n .
(4.67)
The Correlation and covariance of Two Random Variables
The jkth joint moment of X and Y is defined by

E X jY k

   x j y k f ( x, y)
X ,Y
 

j k
x
 i yn p X ,Y ( xi , yn )
 i n
X, Y jointly continuous
X,Y discrete.
If j = 0, to obtain the moments of Y,
If k = 0, to obtain the moments of X ,
If j = 1, k = 1, to call E[XY] as the correlation of X and Y.
If E[XY]=0, we say that X and Y are orthogonal.
(4.68)
The jkth central moment of X and Y is defined as the joint
moment of the centered random variables, X – E[X] and Y – E[Y] :


E  X  E X  Y  E Y  .
j
k
Note: j = 2 k = 0 gives VAR(X)
j = 0 k = 2 gives VAR(Y),
j = k =1, that is defined as the covariance of X and Y
COV( X , Y )  E X  E( X )Y  E(Y ).
(4.69)
COV ( X , Y )  EXY  XEY   YE X   EX EY 
 EXY   2 EX EY   EX EY 
 EXY   EX EY .
(4.70)
EXAMPLE 4.41 Covariance of Independent Random Variables
Let X and Y are independent random variables. Find their covariance.
COV ( X , Y )  E X  E ( X ) Y  E (Y ) 
 EX  E ( X )EY  E (Y )
 0,
Therefore pairs of independent random variables have covariance zero.
The correlation coefficient of X and Y is defined by
 X ,Y 
COV  X , Y 
 XY

E XY   E X E Y 
 XY
,
(4.71)
where  X  VAR X  and  Y  VARY  are the standard
deviations of X and Y, respectively
The correlation coefficient is a number that is at most 1 in
magnitude :
 1   X ,Y  1
(4.72)
proof :
 X  EX  Y  EY   2 
 
0  E 

Y  
  X

 1  2  X ,Y  1
 21   X ,Y .
The extreme values of ρX,Y are achieved when X an Y are
related linearly, Y = aX + b; ρX,Y =1 if a > 0 and ρX,Y = -1 if a < 0.
X and Y are said to be uncorrelated if ρX,Y = 0. If X and Y are
independent（獨立）, then X and Y are uncorrelated. In Example 4.18,
we saw that if X and Y are jointly Gaussian and ρX,Y = 0 , then X and Y are
independent Gaussian random variables.
EXAMPLE 4.42 Uncorrelated but Dependent Random Variables
Let  be uniformly distributed in the interval (0,2π). Let
X  cos  and Y  sin 
The point (X, Y) then corresponds to the point on the unit circle
specified by the angle , as shown in Fig. 4.18. This is not the case
in Example 3.28, so X and Y are dependent (相依).
We now show that X and Y are uncorrelated (不相關）:
1 2
E XY   E sin  cos  
sin  cos d

0
2
1 2

sin 2d  0.

0
4
*Joint Characteristic Function
The joint characteristic function of n random variables is defined as


 X1 , X 2 ,X n (w1 , w2 ,wn )  E e j w1 X1  w2 X 2  wn X n  .
Consider


 X ,Y (w1 , w2 )  E e j w1 X  w2Y  .
(4.73a)
(4.73b)
If X and Y are jointly continuous random variables, then
 X ,Y (w1 , w2 )  



  
f X ,Y ( x, y)e j w1x w2 y dxdy .
(4.73c)
The inversion formula for the Fourier transform implies that the joint
pdf is given by
f X ,Y ( x, y ) 

1
4
2
 

 
 X ,Y ( w1 , w2 )e j  w1x  w2 y dw1dw2 .
(4.74)
The marginal characteristic functions can be obtained form the joint
characteristic function :
 X ( w)   X ,Y ( w,0)
Y ( w)   X ,Y (0, w) .
(4.75)
If X and Y are independent random variables, then

 
 E e E e   
 X ,Y ( w1 , w2 )  E e j  w1 X  w2Y   E e jw1 X e jw2Y
jw1 X
jw2Y
X

( w1 ) Y ( w2 )
(4.76)
The characteristic function of the sum Z = aX + bY can be obtained
from the joint characteristic function of X and Y as follows:

 

 Z (w)  E e jwaX bY   E e j waX  wbY    X ,Y (aw, bw) .
(4.77a)
If X and Y are independent random variables, the characteristic
function of Z = aX + bY is then
 Z ( w)   X ,Y (aw, bw)   X (aw)Y (bw) .
(4.77b)
The joint moments of X and Y can be obtained by taking
derivatives of the joint characteristic funciton.

 X ,Y ( w1 , w2 )  E e j  w1 X  w2Y 

   jw1 X i
 E 
i!
 i 0


 
i 0 k 0


 jw2Y k 


k!
k 0
i
k




jw
jw
2
E X iY k  1
.
i!
k!
derivatives :

i
EXY
k

1
j
ik
i k
 X ,Y ( w1 , w2 )
.
i
k
w1w2
w 0, w 0
1
2
(4.78)
EXAMPLE 4.44
m  0,  2  1
Suppose U and V are independent zero-mean, unit-variance
Gaussian random variables, and let
X  U V
Y  2U  V .
Find the joint characteristic function of X and Y, and find E[XY].
The joint characteristic function of X and Y is

 E e 
 
 X ,Y ( w1 , w2 )  E e j  w1 X  w2Y   E e j  w1 (U V )  w2 ( 2U V ) 
j w1  2 w2 U   w1  w2 V 
.

Since U and V are independent random variables, then



1
 w1  w2 2
2

 X ,Y ( w1 , w2 )  E e j  w1  2 w2 U E e j  w1  w2 V  U w1  2w2 V w1  w2 
e

1
 w1  2 w2 2
2
e

e

1
2 w12  6 w1w2  5 w22
2

,
The correlation E[XY] is found from Eq. (4.78) with i = 1and k =1:
1 2
EXY   2
 X ,Y ( w1 , w2 )
j w1w2
w 0, w 0
1
 e


1
2 w12  6 w1w2  5 w22
2

2
6w1  10w2  1 4w1  6w2 
4
1  2 2 w12  6 w1w2 5 w22 
6 w1 0,w2 0
 e
2
 3. f
1
4.8 JOINTLY GAUSSIAN RANDOM VARIABLES
The random variables X and Y are said to be jointly Gaussian if
their joint pdf has the form
f X ,Y ( x, y )

1
exp 
2
2
1


X ,Y




2
 x  m  2
 x  m1  y  m2   y  m2   
1
  2  X ,Y 

  
  

  1 
  1   2    2   
21 2 1   X2 ,Y
(4.79)
for    x   and    y  
The pdf is constant for values x and y for which the argument
of the exponent is constant :
2
 x  m  2
 x  m1  y  m2   y  m2  
1
  2  X ,Y 

  
   constant

  1 
  1   2    2  
When ρX,Y = 0, X and Y are independent ; when ρX,Y ≠ 0, the major
axis of the ellipse is oriented along the angle
 2   
1
  arctan  X2 ,Y 1 2 2  .
2
 1   2 
(4.80)
Note that the angle is 45º when the variance are equal.
The marginal pdf of X is found by integrating fX,Y(x, y) over all y.
f X ( x) 
e
  x  m1 2 / 2 12
2  1
,
that is, X is a Gaussian random variable with mean m1 and
2
variance  1 .
(4.81)
The conditional pdf of X given Y = y is
f ( x, y )
f X ( x | y )  X ,Y
fY ( y )
2


 
1
2
 y  m2   m1  
exp 
x   X ,Y
2
2 
1
 2 1   X ,Y  1 
 

.
2
2
21 1   X ,Y




(4.82)
We now show that the ρX,Y in Eq. (4.79) is indeed the
correlation coefficient between X and Y. The covariance between X
and Y is defined by
COV  X , Y   E X  m1 Y  m2 
 EE X  m1 Y  m2  | Y .
Now the conditional expectation of (X – m1)(Y – m2) given Y = y is
E X  m1 Y  m2  | Y  y    y  m2 EX  m1 | Y  y 
  y  m2 EX | Y  y   m1 


1

 y  m2  ,
  y  m2   X ,Y
2


where we have used the fact that the conditional mean of X given Y =
y is m1   X ,Y  1 /  2  y  m2  . Therefore
E  X  m1 Y  m2  | Y    X ,Y
1
Y  m2 2 ,
2
and
COV  X , Y   EE X  m1 Y  m2  | Y    X ,Y
  X ,Y  1 2 .

1
2
E Y  m2 
2

EXAMPLE 4.45
The amount of yearly rainfall in city 1 and in city 2 is modeled by a
pair of jointly Gaussian random variables, X and Y, with pdf given by
Eq. (4,79). Find the most likely value of X given that we know Y = y.
The conditional pdf of X given Y = y is given by Eq. (4.82),
which is maximum at the conditional mean
E X | y   m1   X ,Y
1
 y  m2  .
2
n Jointly Gaussian Random Variables
The random variables X1, X2,…, Xn are said to be jointly Gaussian if
their joint pdf is given by
 1

T
exp  x  m K 1 x  m
 2
,
f X (x )  f X1 , X 2 ,, X n ( x1 , x2 , xn ) 
2 n / 2 k 1/ 2
where x and m are column vectors defined by
 x1 
 m1   EX 1 
x 
m   EX 
2 
x   2 , m   2  

    EX 3 
 
  

x
m


E
X
4 
 n
 n 
and K is the covariance matrix that is defined by
(4.83)
COV  X 2 , X 1   COV  X 1 , X n 
 VAR X 1 
COV  X , X 





VAR
X

COV
X
,
X
2
1
2
2
n 
K 











COV
X
,
X

VAR
X
n
1
n


(4.84)
Equation (4.83) shows that the pdf of jointly Gaussian random
variables is completely specified by the individual means and variances and
the pairwise covariances.
EXAMPLE 4.46
Verify that the tow-dimensional Gaussian pdf given in Eq. (4.79) has
the form of Eq. (4.83).
The covariance matrix for the two-dimensional case is given by
  12
K 
  X ,Y  1 2
 X ,Y  1 2 
,
2
2 
The inverse of the covariance matrix is
2


1
2
K 1  2 2

 1  2 1   X2 ,Y   X ,Y  1 2


The term in the exponent is therefore
  X ,Y  1 2 
.
2
1


 22
1
x  m1 , y  m2 
2 2
2
 1  2 1   X ,Y
  X ,Y  1 2


  X ,Y  1 2   x  m1 


y

m
 12
2

  22  x  m1    X ,Y  1 2  y  m2  
1
x  m1 , y  m2 
 2 2

2
2








x

m


y

m
 1  2 1   X ,Y
1
1
2 
 X ,Y 1 2



x  m1  /  1 2  2  X ,Y x  m1  /  1  y  m2  /  2    y  m2  /  2 2
1   
2
X ,Y
.
EXAMPLE 4.48 Independence of Uncorrelated Jointly Gaussian
Random Variables
Suppose X1, X2,…, Xn are jointly Gaussian random variables with
COV X i , X j  0 for i  j . Show that X1, X2,…, Xn are independent
random variables.


 
K  diag VAR X i   diag  i2
Therefore
1 
K  diag  2 
 i 
1
and


 i

x  mT K 1 x  m    xi  mi 
n
i 1
2
.
Thus form Eq. (4.83)
 1 n
 1
2
2




exp  i 1 xi  mi  / 2 
exp

x

m
/
2


i
i
n
 2

 2

f X (x ) 

2 n / 2 K 1/ 2
i 1
2i2
n
  f X i ( xi ) .
i 1
2-dimensional Gaussian pdf, n=2
 1
exp   x  m1
 2
1. f ( x, y ) 
 1 2   x  m1  

2  
y

m
2  
2 
1
 
y  m2  
  1 2
2
1

2 1 2 1   2
  r 2  2  rs  s 2  
exp 

2
2 1    


,
2
2 1 2 1  

x  m1
y  m2 
,s 
r 




1
2

2.  =0,
 1
f ( x, y ) 
exp   x  m1
2 1 2
 2
1
 12 0   x  m1  
y  m2  

2  
y

m

2 
 0 2 
2
2


1
 1  x  m1   y  m2   

exp  
 
 
2 1 2
 2   1    2   
2
12 0 


0 
1
1
3. K  
 K 
2
2 
 0 2 
 0 2 
4. If  12  2,  22  1, m1  m2  0,   0, then
 x2 y 2 
f ( x, y ) 
exp   
2
2 2
 4
1