Download Two random variables

Random Variables and Stochastic
Processes – 0903720
Lecture#11
Dr. Ghazi Al Sukkar
Email: [email protected]
Office Hours: Refer to the website
Course Website:
http://www2.ju.edu.jo/sites/academic/ghazi.alsukkar
1
Chapter 6
Two Random Variables
(Bivariate Distributions)





Joint Sample Space
Joint Distribution and Its properties
Joint density Functions
Independence or Random Variables
Line masses
2
Two Random Variables
• Often encountered when dealing with combined experiments or
repeated trials of a single experiment.
• Two random variables are basically two-dimensional functions
defined on a sample space of a combined experiment.
• Examples:
– Consider the random experiment of launching a dart on a
circular dartboard. The random variables 𝑋 and 𝑌 can be used to
map a (physical) point (𝜁𝑖 ) on the dartboard to a point in the
plane, which is within the unit circle: {𝑋(𝜁𝑖 ), 𝑌(𝜁𝑖 )}.
– Consider choosing a student at random from a population of our
university students. Two random variables can be defined to map
from the sample space of university students to the
measurements of height and weight: 𝐻(𝜁𝑖 ) and 𝑊 𝜁𝑖 .
3
Joint Sample Space
S
𝜁2
ℝ
Function
Y
𝑌(𝜁4 )
𝜁1
𝑌(𝜁2 )
Function
X
𝑆𝐽
𝑋(𝜁2 )
𝑋(𝜁1 ),𝑌(𝜁2 )
𝑋(𝜁1 )
ℝ
• Let 𝑋 and 𝑌 denote two random variables defined on the sample
space 𝑆, where specific values of 𝑋 and 𝑌 are denoted by 𝑥 and 𝑦
respectively.
• Any ordered pair of numbers (𝑥, 𝑦) may be considered as a random
point (plane) in the 𝑥𝑦 plane.
• The plane of all points (𝑥, 𝑦) in the ranges of 𝑋 and 𝑌 may be
considered as a new sample space, called: range sample space,
two-dimensional product space, and Joint sample space 𝑆𝐽 .
4
Joint Statistics
S
𝐴
𝑦
𝑆𝐽
𝐴= 𝑋≤𝑥
𝐴∩𝐵
𝐵
𝐴 ∩ 𝐵 = 𝑋 ≤ 𝑥, 𝑌 ≤ 𝑦
𝐵= 𝑌≤𝑦
𝑥
• Let 𝐴 be an event for 𝑋 define as 𝐴 = 𝑋 ≤ 𝑥 .
• Let 𝐵 be an event for 𝑌 defined as 𝐵 = 𝑌 ≤ 𝑦 .
• The event 𝐴 ∩ 𝐵 defined on 𝑆 corresponds to the joint
event 𝑋 ≤ 𝑥 ∩ 𝑌 ≤ 𝑦 = 𝑋 ≤ 𝑥, 𝑌 ≤ 𝑦 defined on 𝑆𝐽 .
5
• 𝑃 𝐴 = 𝑃 𝑋 ≤ 𝑥 = 𝐹𝑋 (𝑥) which is the
distribution of the R.V. 𝑋, it determines its
separate (marginal) statistics.
• 𝑃 𝐵 = 𝑃 𝑌 ≤ 𝑦 = 𝐹𝑌 (𝑦) which is the
distribution of the R.V. 𝑌, it determines its
separate (marginal) statistics.
• But probability of the joint event 𝑋 ≤ 𝑥, 𝑌 ≤ 𝑦
can not be expressed in terms of 𝐹𝑋 (𝑥) and
𝐹𝑌 (𝑦).
⟹ The joint statistics of the R.V.s 𝑋 and 𝑌 are
completely determined if P 𝑋 ≤ 𝑥, 𝑌 ≤ 𝑦 is known
for every 𝑥 and 𝑦.
6
Joint (Bivariate) Distribution
• What about the probability that the pair of R.V.s
(𝑋, 𝑌) belongs to an arbitrary region D in the 𝑥𝑦 plane?
• Towards this, we define the joint probability
distribution function of 𝑋 and 𝑌 to be:
𝐹𝑋𝑌 𝑥, 𝑦 = 𝑃 𝑋 ≤ 𝑥, 𝑌 ≤ 𝑦 = 𝑃 (𝑋, 𝑌) ∈ 𝐷1
where 𝑥 and 𝑦 are arbitrary real numbers.
𝑌
𝑦
𝐷1
𝑥
𝑋
7
Properties of 𝐹𝑋𝑌 𝑥, 𝑦
• FXY (, y )  FXY ( x,)  0, FXY (,)  1 .
Since  X ( )  , Y ( )  y    X ( )    , we get
FXY (, y )  P X ( )     0.
Similarly  X ( )  , Y ( )     S , we get FXY (, )  P( S )  1.
• Px1  X ( )  x 2 , Y ( )  y   FXY ( x 2 , y )  FXY ( x1 , y ).
To prove this , we note that for 𝑥2 > 𝑥1
 X ( )  x2 , Y ( )  y    X ( )  x1 , Y ( )  y   x1  X ( )  x2 , Y ( )  y 
and the mutually exclusive property of the events on the
right side gives
P X ( )  x 2 , Y ( )  y   P X ( )  x1 , Y ( )  y   P x1  X ( )  x 2 , Y ( )  y 
𝑦
𝑌
𝑥1
𝑥2
𝑋
8
•
P  X ( )  x, y1  Y ( )  y 2   FXY ( x, y 2 )  FXY ( x, y1 ).
𝑦2
𝑌
𝑦1
𝑥
𝑋
• Px1  X ( )  x2 , y1  Y ( )  y 2   FXY ( x2 , y 2 )  FXY ( x2 , y1 ) .
 FXY ( x1 , y 2 )  FXY ( x1 , y1 ).
This is the probability that (X,Y) belongs to the rectangle 𝑅𝑜 .
To prove this, use:
x1  X ( )  x2 , Y ( )  y 2   x1  X ( )  x2 , Y ( )  y1   x1  X ( )  x2 , y1  Y ( )  y 2 .
Y
y2
R0
y1
X
x1
x2
9
Joint probability density function
(Joint PDF)
• The joint PDF of 𝑋 and 𝑌 is given by
• Hence
 2 FXY ( x, y )
f XY ( x, y ) 
.
x y
FXY ( x, y )  
x


y

f XY (u, v) dudv.
• And




 
f XY ( x, y ) dxdy  1.
• 𝑓𝑋𝑌 (𝑥, 𝑦) ≥ 0.
10
Joint Probability
• The probability that the point (𝑋, 𝑌)belongs to an arbitrary
region 𝐷 in the 𝑥𝑦 plane is the integral of 𝑓𝑋𝑌 (𝑥, 𝑦) in 𝐷.
P ( X , Y )  D  
• Proof:

( x , y )D
f XY ( x, y )dxdy.
P x  X ( )  x  x, y  Y ( )  y  y   FXY ( x  x, y  y )
 FXY ( x, y  y )  FXY ( x  x, y )  FXY ( x, y )

x  x
x

y  y
y
f XY (u , v)dudv  f XY ( x, y )xy.
• Thus the probability that (𝑋, 𝑌) belongs to a differential rectangle
𝑥 𝑦 equals 𝑓𝑋𝑌 (𝑥, 𝑦)∆𝑥∆𝑦, then repeating this procedure over
the union of no overlapping differential rectangles in D.
Y
D
y
x
X
11
Marginal Statistics
• In the context of several R.V.s, the statistics of each
individual ones are called marginal statistics.
• Thus 𝐹𝑋 (𝑥) is the marginal probability distribution
function of 𝑋, and 𝑓𝑋 (𝑥) is the marginal PDF of 𝑋. It is
interesting to note that all marginals can be obtained
from the joint PDF.
• In fact
FX ( x)  FXY ( x,),
• And
f X ( x)  


FY ( y)  FXY (, y).
f XY ( x, y )dy, fY ( y )  


f XY ( x, y )dx.
12
• Proof:
( X  x )  ( X  x )  (Y  )
FX ( x)  P X  x   P X  x,Y    FXY ( x,).
So:
x

• Also: FX ( x)  FXY ( x,)    f XY (u, y ) dudy
Take the derivative with respect
to 𝑥:

f X ( x)  

f XY ( x, y )dy.
• At this point, it is useful to know the Liebnitz formula for
differentiation under integrals:
b( x )
Let
H ( x) 
h( x, y )dy.

a( x)
Then its derivative with respect to x is given by
b ( x ) h ( x, y )
dH ( x ) db( x )
da ( x )

h ( x , b) 
h( x, a )  
dy.
a
(
x
)
dx
dx
dx
x
13
𝑋 > 𝑥, 𝑌 > 𝑦
𝑦
𝐴= 𝑋≤𝑥
𝐴 ∩ 𝐵 = 𝑋 ≤ 𝑥, 𝑌 ≤ 𝑦
𝐵= 𝑌≤𝑦
𝑥
• 𝑃 𝑋 > 𝑥, 𝑌 > 𝑦 = 1 − 𝑃 𝑋 ≤ 𝑥 − 𝑃 𝑌 ≤ 𝑦 +
𝑃 𝑋 ≤ 𝑥, 𝑌 ≤ 𝑦 = 1 − 𝐹𝑋 𝑥 − 𝐹𝑌 𝑦 + 𝐹𝑋𝑌 (𝑥, 𝑦)
14
• Example: given 𝑓𝑋𝑌 𝑥, 𝑦 =
1
− 𝑥 2 +𝑦 2 2𝜎 2
𝑒
find the probability that
2
2𝜋𝜎
(𝑥, 𝑦) falls in the region 𝐷 which is a circle
𝑥 2 + 𝑦 2 ≤ 𝑎2 with radius 𝑎.
m  P ( X , Y )  D    
f ( x, y ) dxdy.
• Use polar transformation:
Let 𝑥 = 𝑟𝑐𝑜𝑠 𝜃 and 𝑦 = 𝑟𝑠𝑖𝑛 𝜃
𝑎 𝜋
1
− 𝑟 2 2𝜎 2
𝑚=
𝑒
𝑟𝑑𝜃𝑑𝑟
2
2𝜋𝜎 0 −𝜋
−𝑎2 2𝜎 2
=1−𝑒
( x , y )D
XY
15
• The joint PDF represents complete information about the
R.V.s, and their marginal PDFs can be evaluated from the
joint PDF. However, given marginals, (most often) it will
not be possible to compute the joint PDF.
Y
• Example: Given
c,
f XY ( x, y )  
0,
0  x  y  1,
otherwise .
1
y
𝑐 is constant,
0
1
Obtain the marginal PDFs 𝑓𝑋 (𝑥) and 𝑓𝑌 (𝑦)
Sol.: It is given that the joint PDF 𝑓𝑋𝑌 (𝑥, 𝑦) is a constant in the
shaded region
2 1

 



y
1
cy


f XY ( x, y )dxdy     c  dx dy   cydy 
y 0  x 0
y 0

2
Thus c = 2.
1

0
X
c
 1.
2
16
• Moreover
f ( x )   f ( x, y )dy  

X

XY
1
yx
• And similarly
f ( y )   f ( x, y )dx  

Y

XY
2dy  2(1  x ),
y
x 0
0  x  1,
2dx  2 y, 0  y  1.
• Clearly, in this case given 𝑓𝑋 (𝑥) and 𝑓𝑌 (𝑦), it
will not be possible to obtain the original joint
PDF.
17
Joint Normality
• X and Y are said to be jointly normal (Gaussian) distributed, if
their joint PDF has the following form:
f XY ( x, y ) 
1
2 X  Y 1  
2
e
1  ( x   X ) 2 2  ( x   X )( y  Y ) ( y  Y ) 2



 XY
2 (1  2 )   X2
 Y2




,
   x  ,    y  , |  | 1.
• 𝜌 is indeed the correlation coefficient.
• By direct integration and completing the square, it can be

1
e ( x   ) / 2
shown that f X ( x)    f XY ( x, y )dy 
2
2
X
2
X
2 X
⟹ 𝑋~𝑁(𝜇𝑋 , 𝜎𝑋2 )
And
f Y ( y) 



f XY ( x, y ) dx 
⟹
1
2 Y2
𝑌~𝑁(𝜇𝑌 , 𝜎𝑌2 )
e  ( y  Y )
2
/ 2 Y2
18
• Following the above notation, we will denote the jointly
normal R.V.s as 𝑁(𝜇𝑋 , 𝜎𝑋2 , 𝜇𝑌 , 𝜎𝑌2 , 𝜌)
• If two R.V.s are jointly Gaussian, they are also marginally
Gaussian, the converse is not always true.
• Once again, knowing the marginals alone doesn’t tell us
everything about the joint PDF.
• The only situation where the marginal PDFs can be used
to recover the joint PDF is when the random variables
are statistically independent.
19
Joint Gaussian PDF and contour, 𝜎𝑋 = 𝜎𝑌 = 1, 𝜌 = 0
20
Joint Gaussian PDF and contour, 𝜎𝑋 = 𝜎𝑌 = 1, 𝜌 = 0.3
21
Joint Gaussian PDF and contour, 𝜎𝑋 = 𝜎𝑌 = 1, 𝜌 = 0.7
22
Joint Gaussian PDF and contour, 𝜎𝑋 = 𝜎𝑌 = 1, 𝜌 = 0.95
23
Independence of Random Variables
• Definition: The random variables X and Y are said to be
statistically independent if the events 𝑋 ∈ 𝐴 and
𝑌 ∈ 𝐵 are independent events for any two arbitrary
sets 𝐴 and 𝐵 in 𝑥 and 𝑦 axes respectively.
𝑃 𝑋 ∈ 𝐴, 𝑌 ∈ 𝐵 = 𝑃 𝑋 ∈ 𝐴 𝑃 𝑌 ∈ 𝐵
• Applying the above definition to the events 𝑋 ≤ 𝑥 and
𝑌 ≤ 𝑦 we conclude that, if the R.V.s 𝑋 and 𝑌 are
independent, then: 𝑃 𝑋 ≤ 𝑥, 𝑌 ≤ 𝑦 = 𝑃 𝑋 ≤ 𝑥 𝑃{𝑌 ≤
24
• The procedure to test for independence. Given 𝑓𝑋𝑌 𝑥, 𝑦
obtain the marginal PDFs 𝑓𝑋 (𝑥) and 𝑓𝑌 (𝑦) Then examine
whether
𝑓𝑋𝑌 𝑥, 𝑦 = 𝑓𝑋 (𝑥)𝑓𝑌 (𝑦)
is valid.
• If so, the R.V.s are independent, otherwise they are
dependent.
• For the Joint Gaussian, 𝑋 and 𝑌 are independent if 𝜌 = 0.
 xy2e y , 0  y  , 0  x  1,
• Example: Given f XY ( x, y )  

0,
otherwise.
Determine whether X and Y are independent.
25
• Sol.:


f X ( x )   f XY ( x, y )dy  x  y 2e  y dy
0
• Similarly.
0

y 

 x   2 ye
 2  ye  y dy   2 x,
0
0


fY ( y )  
1
0
y2 y
f XY ( x, y )dx 
e ,
2
0  x  1.
0  y  .
• ⟹ f XY ( x, y)  f X ( x) fY ( y),
• Hence X and Y are independent random
variables.
26
Circular Symmetry
•
•
The Joint density of two R.V.s 𝑋 and 𝑌 is circularly symmetrical if it depends only
on the distance from the origin, i.e.:
𝑓𝑋𝑌 𝑥, 𝑦 = 𝑔 𝑟 , 𝑟 = 𝑥 2 + 𝑦 2
Theorem: if the R.V.s 𝑋 and 𝑌 are circularly symmetrical and independent, then
𝑋~𝑁 0, 𝜎 2 𝑎𝑛𝑑 𝑌~𝑁 0, 𝜎 2 .
Proof: 𝑓𝑋𝑌 𝑥, 𝑦 = 𝑔
𝑥 2 + 𝑦 2 = 𝑓𝑋 (𝑥)𝑓𝑌 (𝑦)
𝜕𝑔(𝑟)
𝑑𝑔(𝑟) 𝜕𝑟
𝜕𝑟
𝑥
=
and
=
𝜕𝑥
𝑑𝑥 𝜕𝑥
𝜕𝑥
𝑟
𝜕𝑔 𝑟
𝑥 ′
= 𝑔 𝑟 = 𝑓𝑋′ (𝑥)𝑓𝑌 (𝑦)
𝜕𝑥
𝑟
But:
⟹
Dividing both sides by 𝑥𝑔 𝑟 = 𝑥𝑓𝑋 (𝑥)𝑓𝑌 (𝑦)
1 𝑔′ 𝑟
1 𝑓𝑋′ (𝑥)
⟹
=
𝑟 𝑔(𝑟) 𝑥 𝑓𝑋 (𝑥)
But since the right hand side is a function of only 𝑥 and the left hand side is a function
of both 𝑥 and 𝑦, the only way these two term will equal is when they are constants.
1 𝑔′ 𝑟
⟹
=𝛼
𝑟𝑔 𝑟
2
2
2
⟹ 𝑔 𝑟 = 𝐴𝑒 𝛼𝑟 2 = 𝐴𝑒 𝛼 𝑥 +𝑦 2
27
Discrete-Type R.V.s
• If X and Y are discrete R.Vs taking the values 𝑥𝑖 and 𝑦𝑗 ,
then 𝑝𝑖𝑗 = 𝑃 𝑋 = 𝑥𝑖 , 𝑌 = 𝑦𝑗 represents their joint
probability mass, and their respective marginal
probability masses are given by:
𝑝𝑖 = 𝑃 𝑋 = 𝑥𝑖 =
𝑗𝑃
𝑋 = 𝑥𝑖 , 𝑌 = 𝑦𝑗 =
𝑗 𝑝𝑖𝑗
𝑝𝑗 = 𝑃 𝑌 = 𝑦𝑗 =
𝑖𝑃
𝑋 = 𝑥𝑖 , 𝑌 = 𝑦𝑗 =
𝑖 𝑝𝑖𝑗
p
• If they are independent then:
ij
i
𝑝𝑖𝑗 = 𝑃 𝑋 = 𝑥𝑖 , 𝑌 = 𝑦𝑗 = 𝑝𝑖 𝑝𝑗
p
ij
j
p11
p21

pi1

pm1
p12
p22

pi 2

pm 2
 p1 j
 p2 j


 pij


 pmj
 p1n
 p2 n


 pin


 pmn
28
• Example: let 𝑋 ∈ −1,0,1 with probabilities
⟹ 𝑌 = 𝑋 3 ∈ −1,0,1 with probabilities
⟹ 𝑍 = 𝑋 2 ∈ 0,1 with probabilities
1 1
,
2 2
1 1 1
, ,
4 2 4
1 1 1
, ,
4 2 4
.
.
.
The joint PDF of 𝑌 and 𝑍:
𝑃 𝑌 = −1, 𝑍 = 0 = 0
𝑃 𝑌 = −1, 𝑍 = 1 = 𝑃 𝑋 = −1 = 1/4
𝑃 𝑌 = 0, 𝑍 = 0 = 𝑃 𝑋 = 0 = 1/2
𝑃 𝑌 = 0, 𝑍 = 1 = 0
𝑃 𝑌 = 1, 𝑍 = 0 = 0
𝑃 𝑌 = 1, 𝑍 = 1 = 𝑃 𝑋 = 1 = 1/4
29
Line Masses
• If 𝑋 is discrete-type R.V. taking the values 𝑥𝑖 and 𝑌 is of
continuous-type, then all the probability masses are on
the vertical lines 𝑥 = 𝑥𝑖 .
𝑌
𝑦2
𝑦1
𝑦
𝑥𝑖
𝑋
• The probability mass between 𝑦1 and 𝑦2 on the line 𝑥𝑖 is:
𝑃 𝑋 = 𝑥𝑖 , 𝑦1 ≤ 𝑦 ≤ 𝑦2
30
• If Y = 𝑔(𝑋) where 𝑋 is a continuous-type R.V. then all
the masses are on the curve 𝑦 = 𝑔(𝑥).
In this case 𝐹𝑋𝑌 (𝑥, 𝑦) can be expressed in terms of 𝐹𝑋 (𝑥).
e.g. for 𝑥 and 𝑦 given on the figure:
𝑦 = 𝑔 𝑥1 = 𝑔 𝑥2 = 𝑔(𝑥3 )
𝑔(𝑥)
𝐹𝑋𝑌 𝑥, 𝑦 = 𝐹𝑋 𝑥1 + 𝐹𝑋 𝑥3 − 𝐹𝑋 (𝑥2 )
𝑦
𝑥1
𝑥2
𝑥3
𝑥
𝑥
31
• If 𝑋 = 𝑔(𝑍) and 𝑌 = ℎ(𝑍) where 𝑍 is a continuous-type
R.V., then the probability masses are on the curve
specified parametrically by 𝑥 = 𝑔 𝑧 , 𝑦 = ℎ(𝑧). The joint
statistics of 𝑋 and 𝑌 can be expressed in terms of 𝐹𝑍 (𝑧).
e.g., if 𝑔 𝑧 = cos 𝑧 , ℎ 𝑧 = sin 𝑧, then the curve is a circle.
𝑌 = sin 𝑍
1
𝑋 = cos 𝑍
32