Download MTHE/STAT 351: 8 – Joint Distributions Discrete Random Variables

Discrete Random Variables
MTHE/STAT 351: 8 – Joint Distributions
We are often interested in the probabilistic description of how two or
more random variables behave together. We will consider the case of two
random variables.
T. Linder
Definition Suppose X and Y are two discrete r.v.’s defined on the same
sample space, having possible values in the sets X and Y, respectively.
The joint probability mass function (pmf) of X and Y is defined by
Queen’s University
p(x, y) = P (X = x, Y = y)
Fall 2016
Remark: P (X = x, Y = y) is a simplified notation for the probability
P (X = x and Y = y) = P {X = x} \ {Y = y}
MTHE/STAT 351: 8 – Joint Distributions
1 / 80
Properties of the joint pmf
1. p(x, y)
MTHE/STAT 351: 8 – Joint Distributions
2 / 80
Given the joint pmf, we can recover the individual pmf’s of X and Y :
0 for all x and y.
pX (x) = P (X = x)
=
2. p(x, y) = 0 if x 2
/ X or y 2
/ Y.
=
3.
=
XX
P (X = x and Y 2 Y)
X
P (X = x, Y = y)
y2Y
p(x, y) = 1
x2X y2Y
X
p(x, y)
y2Y
Proof: Follows the same lines as the proof of
P
Thus
P (X = x) = 1
x2X
4) For any A ⇢ R2 ,
P (X, Y ) 2 A =
X
pX (x) =
X
p(x, y)
X
p(x, y)
y2Y
Similarly,
p(x, y)
pY (y) =
x2X ,y2Y: (x,y)2A
x2X
Proof: Follows the same lines as the proof of
P
P (X 2 B) =
P (X = x)
The pmf’s pX (x) and pY (y) are called the marginal pmf’s of X and Y .
x2X : x2B
MTHE/STAT 351: 8 – Joint Distributions
3 / 80
MTHE/STAT 351: 8 – Joint Distributions
4 / 80
Example: The joint pmf p(x, y) of X and Y are given in the table below.
Find the marginal pmf’s and P (X + Y  2).
Example: Five cards are drawn from a standard deck of 52 cards without
replacement. Let X be the number of spades drawn and Y the number
of clubs drawn. Find the joint pmf of X and Y .
x
Solution: We have X = Y = {0, 1, 2, 3, 4, 5}. If x + y > 5, then
P (X = x, Y = y) = 0. Otherwise
P (X = x, Y = y) =
13
x
13
y
0
0.1
0.3
0.1
y
0
1
2
26
5 x y
52
5
1
0.05
0.2
0.1
2
0.05
0.1
0
Solution:
Thus
p(x, y) =
8
>
<
>
:
13
x
13
y
26
5 x y
52
5
if 0  x, y  5, x + y  5
otherwise
0
pX (0)
=
0.1 + 0.3 + 0.1 = 0.5
pX (1)
=
0.05 + 0.2 + 0.1 = 0.35
pX (2)
=
0.05 + 0.1 + 0 = 0.15
Similarly,
pY (0) = 0.2,
MTHE/STAT 351: 8 – Joint Distributions
5 / 80
pY (1) = 0.6,
pY (2) = 0.2
MTHE/STAT 351: 8 – Joint Distributions
6 / 80
We obtain
Remark: The joint pmf p(x, y) uniquely determines the marginal pmf’s
through the relationship
X
X
pX (x) =
p(x, y),
pY (y) =
p(x, y)
x
y
0
1
2
pX (x)
0
0.1
0.3
0.1
0.5
1
0.05
0.2
0.1
0.35
2
0.05
0.1
0
0.15
pY (y)
0.2
0.6
0.2
y2Y
x2X
However, the reverse is not true. For given pX (x) and pY (y) there are
(infinitely) many joint pmf’s whose marginals are pX (x) and pY (y).
For example, compare this table and the previous example:
(This is why pX (x) and pY (y) are called the marginal pmf’s.)
x
P (X + Y  2)
=
X
y
0
1
2
pX (x)
p(x, y)
x+y2
=
p(0, 0) + p(0, 1) + p(1, 0) + p(1, 1) + p(0, 2) + p(2, 0)
=
0.1 + 0.3 + 0.05 + 0.2 + 0.1 + 0.05
=
0.8
MTHE/STAT 351: 8 – Joint Distributions
7 / 80
MTHE/STAT 351: 8 – Joint Distributions
0
0.1
0.3
0.1
0.5
1
0.07
0.21
0.07
0.35
2
0.03
0.09
0.03
0.15
pY (y)
0.2
0.6
0.2
8 / 80
Expected value
Corollary 2
If X and Y are discrete r.v.’s, then
Note that from the joint pmf we can compute the marginal pmf’s and so
the expected values of X and Y :
X
X
E(X) =
xpX (x) and E(Y ) =
ypY (y)
x2X
E(X + Y ) = E(X) + E(Y )
Proof: With h(x, y) = x + y we have
XX
E(X + Y ) =
(x + y)p(x, y)
y2Y
If h : R2 ! R is a real function of two variables, h(X, Y ) is a discrete
random variable.
x2X y2Y
XX
=
Theorem 1
x2X y2Y
If X and Y are discrete r.v.’s with joint pmf p(x, y) and h(x, y) is a real
function of two variables, then
E[h(X, Y )] =
XX
X
=
x
x2X
X
=
P
=
g(x)P (X = x).
X
p(x, y) +
y2Y
y2Y
|
x2X
Proof: Follows the lines of the proof of E[g(X)] =
yp(x, y)
x2X y2Y
y
X
p(x, y)
x2X
| {z }
{z }
pY (y)
pX (x)
X
X
xpX (x) +
ypY (y)
h(x, y)p(x, y)
x2X y2Y
XX
xp(x, y) +
y2Y
E(X) + E(Y )
⇤
x2X
MTHE/STAT 351: 8 – Joint Distributions
9 / 80
MTHE/STAT 351: 8 – Joint Distributions
10 / 80
Jointly Continuous Random Variables
Properties of jointly continuous r.v.’s
1. f (x, y)
Definition The random variables X and Y are said to be jointly
continuous if there exists a nonnegative function f (x, y) such that for
any “reasonable” planar set C ⇢ R2 ,
P (X, Y ) 2 C =
ZZ
2.
Z
1
1
Z
1
f (x, y) dxdy = 1
1
Proof:
f (x, y) dxdy
Z
C
The function f (x, y) is called the joint probability density function (joint
pdf) of X and Y .
MTHE/STAT 351: 8 – Joint Distributions
0 (by definition).
11 / 80
1
1
Z
1
f (x, y) dxdy
=
ZZ
=
P (X, Y ) 2 R2
=
1
1
MTHE/STAT 351: 8 – Joint Distributions
f (x, y) dxdy
R2
⇤
12 / 80
Similarly to the discrete case, the marginal pdf’s can be obtained from
the joint pdf:
3. For any a  b and c  d,
P (a  X  b, c  Y  d) =
Z
d
c
Z
b
fX (x) =
f (x, y) dxdy
a
Proof: Let C = {(x, y) : a  x  b, c  y  d}. Then
P (a  X  b, c  Y  d)
=
=
=
Z
d
c
Z
b
f (x, y) dxdy
a
1
and
f (x, y) dy
fY (y) =
1
Z
1
f (x, y) dx
1
Proof sketch:
P (X, Y ) 2 C
ZZ
f (x, y) dxdy
C
Z
P (a  X  b)
P (a  X  b, 1 < Y < 1)
Z 1Z b
f (x, y) dxdy
=
=
1
⇤
Z b ✓Z
=
a
1
a
Note: Letting a = b = u and c = d = v, this shows that
Thus the function g(x) =
P (X = u, Y = v) = 0
R1
1
f (x, y) dy dx
1
f (x, y) dy satisfies
Z
P (a  X  b) =
for all u and v.
◆
b
g(x) dx
a
for all a < b, and so it must equal to the pdf fX of X.
MTHE/STAT 351: 8 – Joint Distributions
13 / 80
Example: Let the joint pdf of X and Y be given by
8
<k xy 0  x, y  1
f (x, y) =
:0
otherwise
MTHE/STAT 351: 8 – Joint Distributions
(b) Find the marginal pdf’s.
Solution: For 0  x  1,
fX (x)
(a) Determine k.
1
=
=
Z
1
0
Z
=
1
f (x, y) dxdy
✓Z
kxy dxdy
◆✓Z 1
◆
x dx
y dy
=
k
=
✓ ◆2
1
1
k
=k
2
4
0
0
By symmetry
fY (y) =
Thus k = 4.
MTHE/STAT 351: 8 – Joint Distributions
Z
Z
1
f (x, y) dy
1
1
4xy dy
0
2x
Note that f (x, y) = 0 if x 2
/ [0, 1], so fX (x) = 0 if x 2
/ [0, 1]. Thus
8
<2x 0  x  1
fX (x) =
:0
otherwise
1
1
1Z 1
0
1
=
=
Solution:
Z
14 / 80
15 / 80
MTHE/STAT 351: 8 – Joint Distributions
8
<2y
:0
0y1
otherwise
16 / 80
Example: Suppose the joint pdf of X and Y is given by
8
<10xy 2 0  x  y  1
f (x, y) =
:0
otherwise
(c) Find P (X + Y  1).
Solution: Note that P (X + Y  1) = P (X, Y ) 2 C , where
C = {(x, y) : x + y  1}
(a) Find P (Y  2X).
Thus
P (X + Y  1)
ZZ
=
f (x, y) dxdy =
C
Z
=
Z
=
1
0
1
Z
ZZ
x+y1
1 y
4xy dxdy
0
C
2y(1
2
y) dxdy
=
0
 2
y
2
(y 2y + y ) dy = 2
2
0
✓
◆
1 2 1
1
2
+
=
2 3 4
6
=
=
Solution: Note that f (x, y) = 0 outside the region
A = {(x, y) : 0  x  y  1}. Letting C = {(x, y) : y  2x}, we have
ZZ
ZZ
P (Y  2X) =
f (x, y) dxdy =
10xy 2 dxdy
f (x, y) dxdy
Z
1
2
3
2y 3
y4
+
3
4
1
=
0
=
MTHE/STAT 351: 8 – Joint Distributions
17 / 80
Z
5
1
0
Z
Z
1
0
15
4
Z
C\A
y
10xy 2 dxdy
y/2
✓
y2 y2
y2
4
1
y 4 dy =
0
3
4
◆
dy
MTHE/STAT 351: 8 – Joint Distributions
18 / 80
(b) Find the marginal pdf’s fX (x) and fY (y).
Geometric Probability
Solution: For x 2 [0, 1],
Suppose a point is drawn randomly from a bounded planar region
B ⇢ R2 . The mathematical model is a pair of random variables (X, Y )
with joint pdf given by
8
<c if (x, y) 2 B
f (x, y) =
:0 otherwise
fX (x) =
Z
1
f (x, y) dy =
1
Thus
fX (x) =
Similarly, for y 2 [0, 1],
Z
fY (y) =
1
Z
8
< 10 x(1
3
10xy 2 dy =
x
x3 )
f (x, y) dx =
hence
fY (y) =
8
<5y 4
:0
10
x(1
3
x3 )
0x1
:0
1
MTHE/STAT 351: 8 – Joint Distributions
1
otherwise
where c > 0 is a constant.
Z
The constant is determined by
ZZ
ZZ
1 =
f (x, y) dxdy =
c dxdy
y
2
10xy dx = 5y
4
0
R2
B
c · area(B)
0y1
=
otherwise
so that c =
19 / 80
(since f (x, y) = 0 outside B)
1
.
area(B)
MTHE/STAT 351: 8 – Joint Distributions
20 / 80
Let’s compute P (X, Y ) 2 A for some A ⇢ R2 :
ZZ
P (X, Y ) 2 A =
f (x, y) dxdy
Example: Let (X, Y ) be a point randomly drawn from the unit square
B = {(x, y) : 0  x, y  1}. Calculate P |X Y | 1/2 .
Solution: Note that area(B) = 1. The probability can be calculated by
integrating the pdf
8
<1 if 0  x, y  1
f (x, y) =
:0 otherwise
A
ZZ
1
=
dxdy
area(B)
A\B
ZZ
1
=
dxdy
area(B)
A\B
area(A \ B)
area(B)
=
over the region A = {(x, y) : |x
1/2}.
However, since A \ B is a union of two congruent isosceles right triangles
of side length 1/2, we have
Thus if (X, Y ) is a point randomly drawn from B, then for all
“reasonable” A ⇢ R2 ,
P |X
area(A \ B)
P (X, Y ) 2 A =
area(B)
MTHE/STAT 351: 8 – Joint Distributions
y|
21 / 80
Y|
1/2 =
2·
area(A \ B)
=
area(B)
MTHE/STAT 351: 8 – Joint Distributions
1
2
·
1
1 2
2
=
1
4
22 / 80
Corollary 4
Expected value
If X and Y are jointly continuous r.v.’s, then
E(X + Y ) = E(X) + E(Y )
Just as in the discrete case, using the joint pdf we can compute the
marginal pdf’s and thus the expected values of X and Y :
Z 1
Z 1
E(X) =
xfX (x) dx and E(Y ) =
yfY (y) dy
1
Proof Similar to the discrete case, but the sums are replaced with
integrals. Letting h(x, y) = x + y we have
Z 1Z 1
E(X + Y ) =
(x + y)f (x, y) dxdy
1
1
Z 1Z 1
Z 1Z 1
=
xf (x, y) dxdy +
yf (x, y) dxdy
1
1
1
1
◆
◆
Z 1 ✓Z 1
Z 1 ✓Z 1
=
x
f (x, y) dy dx +
y
f (x, y) dx dy
1
1
1
1
|
{z
}
|
{z
}
fX (x)
fY (y)
Z 1
Z 1
=
xfX (x) dx +
yfY (y) dy
1
The following theorem shows how to calculated the expected value of the
r.v. h(X, Y ).
Theorem 3
If X and Y are continuous r.v.’s with joint pdf f (x, y) and h(x, y) is a
real function of two variables, then
E[h(X, Y )] =
Z
1
1
Z
1
h(x, y)f (x, y) dxdy
1
=
MTHE/STAT 351: 8 – Joint Distributions
23 / 80
1
E(X) + E(Y )
MTHE/STAT 351: 8 – Joint Distributions
⇤
1
24 / 80
Example: Let X and Y have joint pdf
8
<10xy 2 0  x  y  1
f (x, y) =
:0
otherwise
Remark: The main theorem about the expected value of h(X, Y ) can be
used for both discrete and jointly continuous r.v.’s to show the the
following “linearity of expectation” property:
(a) Find E(X 2 + Y 2 ).
Solution: Using the linearity of expectation,
⇥
⇤
⇥
⇤
⇥
⇤
E ↵g1 (X, Y ) + g2 (X, Y ) = ↵E g1 (X, Y ) + E g2 (X, Y )
E(X 2 + Y 2 )
for any two functions g1 , g2 : R2 ! R.
=
=
=
=
MTHE/STAT 351: 8 – Joint Distributions
25 / 80
E(X 2 ) + E(Y 2 )
Z 1Z 1
Z
2
x f (x, y) dxdy +
Z
1
1
1Z y
2
2
x 10xy dxdy +
0
0
5
5
15
+ =
14 7
14
Z
1
0
Z
1
1
y
Z
1
y 2 f (x, y) dxdy
1
y 2 10xy 2 dxdy
0
MTHE/STAT 351: 8 – Joint Distributions
26 / 80
Independence of Random Variables
(b) Find E(X 2 Y 2 ).
Definition Two random variables X and Y defined on the same
probability space are independent if for all “reasonable” A, B ⇢ R, the
events {X 2 A} and {Y 2 B} are independent, i.e.,
Solution:
E(X 2 Y 2 )
Z
=
Z
=
=
5
14
1
x2 y 2 f (x, y) dxdy
P (X 2 A, Y 2 B) = P (X 2 A)P (Y 2 B)
x y 10xy 2 dxdy
0
5
18
·
Z
1
1
1Z y
2 2
10
=
Note that E(X 2 )E(Y 2 ) =
1
5
7
Z
0
1
0
=
Z
y
Let F (s, t) denote the joint distribution function of X and Y :
x3 y 4 dxdy
0
25
98 ,
F (s, t) = P (X  s, Y  t)
It can be shown using the axioms of probability that X and Y are
independent if and only if
so that
E(X 2 Y 2 ) 6= E(X 2 )E(Y 2 )
F (s, t) = FX (s)FY (t)
i.e., the joint distribution function is the product of the marginal
distribution functions.
MTHE/STAT 351: 8 – Joint Distributions
27 / 80
MTHE/STAT 351: 8 – Joint Distributions
28 / 80
Example: The joint pmf of X and Y is given by
The following theorem gives an easy to check characterization of
independence for discrete random variables.
x
0
1
2
Theorem 5
0
0.1
0.05
0.05
Let X and Y be discrete r.v.’s with joint pmf p(x, y). Then X and Y are
independent if and only if
1
0.3
0.2
0.1
2
0.1
0.1
0
y
Are X and Y independent?
p(x, y) = pX (x)pY (y)
Solution: The marginal pmf’s are
i.e., the joint pmf is the product of the marginal pmf’s.
pX (0) = 0.5,
Proof If X and Y are independent, then {X = x} and {Y = y} are
independent events, so p(x, y) = pX (x)pY (y) follows.
pX (1) = 0.35,
pX (2) = 0.15
and
pY (0) = 0.2,
The proof that p(x, y) = pX (x)pY (y) for all x, y implies that X and Y
are independent is left as an exercise.
⇤
pY (1) = 0.6,
pY (2) = 0.2
Clearly, X and Y are not independent since e.g.,
pX (2)pY (2) = 0.15 · 0.2 = 0.03 6= 0 = p(2, 2)
MTHE/STAT 351: 8 – Joint Distributions
29 / 80
MTHE/STAT 351: 8 – Joint Distributions
30 / 80
Example: If X and Y have pmf given below, are they independent?
x
Independence of continuous r.v.’s
0
1
2
0
0.1
0.07
0.03
1
0.3
0.21
0.09
2
0.1
0.07
0.03
y
The following characterizes the independence of jointly continuous
random variables in term of their pdf’s:
Theorem 6
Let X and Y be jointly continuous r.v.’s with joint pdf f (x, y). Then X
and Y are independent if and only if
Solution: They are independent. Calculate the marginal pmf’s and check
that p(x, y) = pX (x)pY (y) for all x and y:
f (x, y) = fX (x)fY (y)
x
0
1
2
pY (y)
0
0.1
0.07
0.03
0.2
1
0.3
0.21
0.09
0.6
2
0.1
0.07
0.03
0.2
pX (x)
0.5
0.35
0.15
y
MTHE/STAT 351: 8 – Joint Distributions
i.e., the joint pdf is the product of the marginal pdf’s.
31 / 80
MTHE/STAT 351: 8 – Joint Distributions
32 / 80
fX (x) =
Example: Let ⌦ be the planar region defined by
⌦ = {(x, y) : 0  x + y  1, 0  x  1, 0  y  1}
Z
1
f (x, y) dy =
8R
< 1
f (x, y) dx =
8R
< 1
1
and
and the joint pdf of X and Y be given by
8
<6x (x, y) 2 ⌦
f (x, y) =
:0
otherwise
fY (y) =
Z
1
1
x
0
6x dy = 6x(1
0x1
x)
:0
otherwise
y
0
6x dx = 3(1
y)2
0y1
:0
otherwise
Clearly, f (x, y) 6= fX (x)fY (y). For example,
Are X and Y independent?
fX (x)fY (y) > 0
Solution: Let’s calculate fX (x) and fY (y):
for all (x, y) 2 (0, 1)2
⌦
but
f (x, y) = 0 if (x, y) 2 (0, 1)2
⌦
Thus X and Y are not independent.
MTHE/STAT 351: 8 – Joint Distributions
33 / 80
MTHE/STAT 351: 8 – Joint Distributions
34 / 80
Example: A point (X, Y ) is selected at random from the rectangle
R = {(x, y) : 0  x  a, 0  y  b}
Similarly
Are X are Y independent ?
Solution: Recall that the joint pdf is given by
8
1
1
<
=
if (x, y) 2 R
area(R)
ab
f (x, y) =
:
0
otherwise
fY (y) =
fX (x) =
1
f (x, y) dy =
1
MTHE/STAT 351: 8 – Joint Distributions
8R
< b
1
0 ab
:0
dy =
1
a
1
f (x, y) dx =
1
8R
< a
1
0 ab
:0
dx =
1
b
0yb
otherwise
Clearly, f (x, y) = fX (x)fY (y) for all x and y, so X and Y are
independent.
Thus
Z
Z
0xa
otherwise
35 / 80
MTHE/STAT 351: 8 – Joint Distributions
36 / 80
Example: Let X and Y be independent exponential r.v.’s with pdf’s
8
8
<↵e ↵x x 0
< e y y 0
fX (x) =
fY (y) =
:0
:0
otherwise,
otherwise
P (X < Y )
=
Find P (X < Y ).
=
Solution: Since X and Y are independent,
8
<↵ e
f (x, y) = fX (x)fY (y) =
:0
=
↵x
y
x, y
0
=
P (X < Y ) =
1
Z
y
↵ e ↵x y dxdy
✓Z y
◆
Z 1
y
↵x
↵ e
e
dx dy
0
Z0 1
1 e ↵y e y dy
Z0 1
Z 1
e y dy
e (↵+ )y dy
0
0
0
otherwise
We will calculate
ZZ
Z
f (x, y) dxdy
=
1
=
↵
↵+
0
↵+
(x,y): x<y
MTHE/STAT 351: 8 – Joint Distributions
37 / 80
MTHE/STAT 351: 8 – Joint Distributions
38 / 80
Theorem 8
Let X and Y be discrete or jointly continuous independent random
variables and h(x) and g(y) real functions. Then
Some consequences of independence
E[h(X)g(Y )] = E[h(X)]E[g(Y )]
Theorem 7
Let X and Y be discrete or jointly continuous independent random
variables and h(x) and g(y) real functions. Then h(X) and g(Y ) are
independent r.v.’s.
Proof Assume X and Y are jointly continuous; the proof for the
discrete case is similar:
Z 1Z 1
E[h(X)g(Y )] =
h(x)g(y)f (x, y) dxdy
1
1
Z 1Z 1
=
h(x)g(y)fX (x)fY (y) dxdy (by independence)
1
1
✓Z 1
◆
Z 1
=
g(y)fY (y)
h(x)fX (x) dx dy
1
1
Z 1
= E[h(X)]
g(y)fY (y) dy
For example, if X and Y are independent, then sin X and cos Y are
independent, cos X and sin Y are independent, X 2 and e Y are
independent, etc.
=
MTHE/STAT 351: 8 – Joint Distributions
39 / 80
1
E[h(X)]E[g(Y )]
MTHE/STAT 351: 8 – Joint Distributions
⇤
40 / 80
Example: Let R1 , R2 , R3 , and R4 be the square regions defined by
R1 = 0  x  12 ,
R3 =
Corollary 9
1
2
 x  0,
1
2
 y  1 , R2 =
1y
1
2
, R4 =
1x
1
2
1
2,
 x  1,
0y
1
2
1
2
y0
and assume (X, Y ) is randomly drawn from A = R1 [ R2 [ R3 [ R4 .
Then the joint pdf is
8 1
<
= 1 (x, y) 2 A
f (x, y) = area(A)
:
0
otherwise
If X and Y are independent random variables, then
E(XY ) = E(X)E(Y )
Remark: The converse of the corollary is not true, i.e., the fact that
E(XY ) = E(X)E(Y ) does not imply that X and Y are independent.
It is easy to see that the marginal pdf’s are
8
8
<1/2
<1/2
1x1
fX (x) =
fY (y) =
:0
:0
otherwise,
1y1
otherwise,
Thus both X and Y are uniform r.v.’s on the interval [ 1, 1].
MTHE/STAT 351: 8 – Joint Distributions
41 / 80
Clearly, f (x, y) 6= fX (x)fY (y), so X and Y are not independent.
MTHE/STAT 351: 8 – Joint Distributions
42 / 80
We have
E(X) = E(Y ) = 0
and
E(XY )
=
=
=
=
Z
1
1
ZZ
xyf (x, y) dxdy
Solution: The joint pdf f (x, y) = fX (x)fY (y) is given by
8
<1 0 < x, y < 1
f (x, y) =
:0 otherwise
1
xy dxdy
4 ZZ
X
i=1
Z 1
+
1
R1 [R2 [R3 [R4
1/2
=
Z
Example: Let X and Y be independent uniform r.v.’s on the interval
(0, 1). Find the pdf of Z = XY and use it to calculate E(Z).
Z
Z
xy dxdy
Ri
1/2
xy dxdy +
0
1
1/2
Z
0
Z
1/2
0
xy dxdy +
1/2
Z
Z
Since Z = XY and X, Y 2 (0, 1), we have Z = XY 2 (0, 1), and so
1/2
xy dxdy
1
0
1/2
Z
1
FZ (t) = P (Z  t) =
xy dxdy
1/2
✓ ◆✓ ◆ ✓ ◆✓
◆ ✓
◆✓
◆ ✓ ◆✓
◆
1
3
1
3
1
3
3
1
+
+
+
=0
8
8
8
8
8
8
8
8
8
>
>
<0
>
>
:
t0
P (XY  t)
0<t<1
1
t
1
Thus E(X)E(Y ) = E(XY ), but X and Y are not independent.
MTHE/STAT 351: 8 – Joint Distributions
43 / 80
MTHE/STAT 351: 8 – Joint Distributions
44 / 80
For 0 < t < 1 we have from the geometry of the problem
P (XY  t)
Using integration by parts with u =
✓
◆
Z 1 Z t/x
t
P Y 
=t·1+
dydx
X
t
0
Z 1
t
t+
dx = t t ln t
t x
=
=
E(Z) =
Z
( t ln t) dt
E(Z) =
tfZ (t) dt =
1
Z
t2
ln t
2
Z
0 0+
=
0
1
+
0
1
0
Z
1 2
t 1
· dt
2 t
0
t
dt
2
1
4
=
Compare the above with the the following simple calculation which uses
the fact that E(XY ) = E(X)E(Y ) since X and Y are independent:
E(XY ) = E(X)E(Y )
We can calculate E(Z) as
1

1
=
Thus the pdf of Z = XY is given by
8
< d (t t ln t) 0 < t < 1
dt
fZ (t) = FZ0 (t) =
:0
otherwise
8
< ln t 0 < t < 1
=
:0
otherwise
Z
ln t and dv = t, we obtain
=
=
1
( t ln t) dt
✓Z
1
x dx
0
◆✓Z
✓ ◆2
1
1
=
2
4
1
y dy
0
◆
0
MTHE/STAT 351: 8 – Joint Distributions
45 / 80
MTHE/STAT 351: 8 – Joint Distributions
46 / 80
Conditional Distributions
Discrete distributions
Let X and Y be discrete random variables with joint pmf p(x, y). If we
don’t know anything about the the value of Y , the probabilities
concerning X are calculated from the marginal pmf
X
pX (x) =
p(x, y)
Recall the definition of conditional probability of an event A given
another event B (such that P (B) > 0):
P (A|B) =
P (AB)
P (B)
y2Y
Now assume that we know that Y = y. Then we have extra knowledge
about the probabilities concerning X in the form of the conditional
probabilities
We want to extend this notion to a pair of random variables X and Y .
Specifically, we are interested in how the knowledge of the value of one of
them “a↵ects” the probability distribution of the other.
P (X = x|Y = y) =
MTHE/STAT 351: 8 – Joint Distributions
47 / 80
MTHE/STAT 351: 8 – Joint Distributions
P (X = x, Y = y)
p(x, y)
=
P (Y = y)
pY (y)
48 / 80
Remarks:
Definition Let X and Y be discrete r.v.’s with joint pmf p(x, y). The
conditional pmf of X given Y = y is defined by
1. The conditional pmf pY |X (y|x) is similarly defined.
p(x, y)
pX|Y (x|y) =
pY (y)
2. The conditional pmf can be used to calculate the conditional
probability of events in the form {X 2 C} given {Y = y}:
whenever pY (y) > 0.
P (X 2 C|Y = y) =
Note: For fixed y 2 Y, the function pX|Y (x|y) is a pmf in x. Indeed,
pX|Y (x|y) 0 and
X
pX|Y (x|y)
=
x2X
=
=
3. If X and Y are independent, then p(x, y) = pX (x)pY (y), so
pX|Y (x|y) =
Example: Let X be the number of spades and Y the number of clubs in
a randomly drawn poker hand. We have seen that
8 13 13
26
>
< x y 5 x y
if 0  x, y  5, x + y  5
52
p(x, y) =
5
>
:
0
otherwise
MTHE/STAT 351: 8 – Joint Distributions
,
P (X  1|Y = 3)
p(x, y)
pY (y)
=
=
MTHE/STAT 351: 8 – Joint Distributions
X
pX|Y (x|3)
x1
=
0y5
13
26
52
y
5 x y / 5
13
39
52
y
5 y / 5
13
26
x
5 x y
0x
39
5 y
=
=
⇡
Thus
pX|Y (x|y) =
50 / 80
Let’s calculate P (X  1|Y = 3):
Let’s calculate the conditional pmf of X given Y . We have
pY (y) =
p(x, y)
pX (x)pY (y)
=
= pX (x)
pY (y)
pY (y)
Thus in this case pX|Y (x|y) = pX (x). The converse is also true,
and in fact X and Y are independent if and only if
pX|Y (x|y) = pX (x) for all x and y such that pY (y) > 0.
49 / 80
39
5 y
52
5
pX|Y (x|y)
x2C
Proof This is essentially the law of total probability. The formal
proof is left as an exercise.
X p(x, y)
pY (y)
x2X
1 X
p(x, y)
pY (y)
x2X
| {z }
pY (y)
1
pY (y) = 1
pY (y)
MTHE/STAT 351: 8 – Joint Distributions
13
y
X
pX|Y (0|3) + pX|Y (1|3)
13
0
26
2
39
2
+
13
1
26
1
39
2
0.895
13
x
5
y
51 / 80
MTHE/STAT 351: 8 – Joint Distributions
52 / 80
Remarks:
Continuous distributions
1. Just as in the discrete case, it is easy to show that fX|Y (x|y) is a
valid pdf for fixed y:
Z 1
Z 1
f (x, y)
fX|Y (x|y) dx =
dx
1
1 fY (y)
Z 1
1
=
f (x, y) dx
fY (y) 1
|
{z
}
fY (y)
1
=
fY (y) = 1
fY (y)
It is not immediately clear how to condition on the value of a continuous
random variable Y since P (Y = y) = 0 for all y.
Definition Let X and Y be jointly continuous r.v.’s with joint pdf
f (x, y). The conditional pdf of X given Y = y is defined by
fX|Y (x|y) =
f (x, y)
fY (y)
whenever fY (y) > 0. Similarly, the conditional pdf of Y given X = x is
fY |X (y|x) =
2. Similarly to the discrete case, it can be shown that X and Y are
independent if and only if
f (x, y)
fX (x)
fX|Y (x|y) = fX (x)
whenever fX (x) > 0.
for all x and y such that fY (y) > 0.
MTHE/STAT 351: 8 – Joint Distributions
53 / 80
P (Y 2 B|X = x) =
B
(a) Find fX|Y (x|y) and fY |X (y|x).
fY |X (y|x) dy
Solution: We have seen that
fY (y) =
Note: The meaning of P (Y 2 B|X = x) is not as obvious as in the
discrete case, since we are conditioning on the event {X = x} of
probability zero. We will see later how useful P (Y 2 B|X = x) can be.
MTHE/STAT 351: 8 – Joint Distributions
54 / 80
Example: Let the joint pdf of X and Y be
8
<10xy 2 0  x  y  1
f (x, y) =
:0
otherwise
Definition For a reasonable set B ⇢ R, the conditional probability of
the event {Y 2 B} given X = x is defined by
Z
MTHE/STAT 351: 8 – Joint Distributions
8
<5y 4
:0
0y1
otherwise
We have fY (y) > 0 if y 2 (0, 1]. Thus for all y 2 (0, 1],
8
2x
> 10xy 2
= 2
0x<y
f (x, y) <
5y 4
y
fX|Y (x|y) =
=
>
fY (y)
:0
otherwise
55 / 80
MTHE/STAT 351: 8 – Joint Distributions
56 / 80
(b) Find P (X  12 |Y = 34 ).
Previously we have also calculated fX (x):
8
< 10 x(1 x3 )
fX (x) = 3
:0
Solution: We have
8
2x
32
>
< 3 =
x
2
9
(
)
fX|Y (x|3/4) =
4
>
:0
0x1
otherwise
Thus for x 2 (0, 1) (so that fX (x) > 0),
8
3y 2
> 10xy 2
=
f (x, y) < 10
1 x3
x3 )
fY |X (y|x) =
=
3 x(1
>
fX (x)
:0
Thus
P (X  12 |Y = 34 )
xy<1
otherwise
=
=
=
MTHE/STAT 351: 8 – Joint Distributions
57 / 80
The conditional probability P (Y 2 B|X = x) can be very useful in
calculating probabilities via the following version of the law of total
probability:
x2X
If X and Y are jointly continuous random variables, then
MTHE/STAT 351: 8 – Joint Distributions
1
1
otherwise
1/2
fX|Y (x|y) dx
1
Z
32 1/2
x dx
9 0
32 1
4
· =
9 8
9
MTHE/STAT 351: 8 – Joint Distributions
Z
Let X and Y be discrete random variables. Then
X
P (Y 2 B) =
P (Y 2 B|X = x)pX (x)
Z
3
4
58 / 80
Proof We only do the continuous case:
Theorem 10 (Law of total probability)
P (Y 2 B) =
Z
0x
1
1
P (Y 2 B|X = x)fX (x) dx
◆
Z 1 ✓Z
=
fY |X (y|x) dy fX (x) dx
1
B
◆
Z ✓Z 1
=
fY |X (y|x)fX (x) dx dy
1
ZB
=
fY (y) dy
B
=
P (Y 2 B|X = x)fX (x) dx
59 / 80
MTHE/STAT 351: 8 – Joint Distributions
P (Y 2 B)
⇤
60 / 80
Example: Suppose X is uniformly distributed on [0, 1]. Given X = x, let
Y be a random point in the interval [0, x]. Calculate the probability
P (Y
1/2).
Thus from the law of total probability we have
Z 1
P (Y
1/2) =
P (Y
1/2|X = x)fX (x) dx
Solution: We could calculate fX,Y (x, y) and obtain P (Y
1/2) from a
double integral. Instead, we will use the law of total probability.
We have
8
<1
fY |X (y|x) = x
:0
and so
P (Y
1/2|X = x) =
=
=
0yx1
otherwise
8
<x
:
1/2
x
=
1/2  x  1
=
otherwise
0
MTHE/STAT 351: 8 – Joint Distributions
61 / 80
Z
Z

1
1
x
1/2
dx
x
1/2
1
1
dx
2x
1
1/2
x
1
1
2
1
ln x
2
1
1/2
ln 2
MTHE/STAT 351: 8 – Joint Distributions
62 / 80
Conditional Expectation
Note: For fixed y, the conditional expectation of X given Y = y is
simply an expectation calculated according to the conditional distribution
of X given Y = y.
We can generalize the expected value to conditional distributions in the
following way.
Definition Let X and Y be random variables defined on the same
sample space. If X and Y are discrete, then the conditional expectation
of X given Y = y is defined by
E(X|Y = y) =
X
Thus all the basic properties we derived for (unconditional) expectations
still hold. For example,
E(h(X)|Y = y) =
xpX|Y (x|y)
X
h(x)pX|Y (x|y)
x2X
x2X
for discrete r.v.’s, and
whenever pY (y) > 0. If X and Y are jointly continuous, the
corresponding definition is
E(X|Y = y) =
Z
1
E(h(X)|Y = y) =
xfX|Y (x|y) dx
Z
1
h(x)fX|Y (x|y) dx
1
for continuous r.v.’s.
1
whenever fY (y) > 0.
MTHE/STAT 351: 8 – Joint Distributions
63 / 80
MTHE/STAT 351: 8 – Joint Distributions
64 / 80
Example: Let the joint pdf of X and Y be
8
<10xy 2 0  x  y  1
f (x, y) =
:0
otherwise
We obtain for all y 2 [0, 1],
E(X|Y = y)
(a) Find E(X|Y = y).
=
Solution: We previously derived
8 2x
<
fX|Y (x|y) = y 2
:
0
0x<y1
=
otherwise
Thus for all y 2 [0, 1] we need to calculate
Z 1
Z
E(X|Y = y) =
xfX|Y (x|y) dx =
1
=
y
x
0
65 / 80
(b) Find E(Y |X = x).
=
=
=
MTHE/STAT 351: 8 – Joint Distributions
Z
1
1
66 / 80
Theorem 11 (Law of total expectation)
0x<y1
Let X and Y be random variables defined on the same sample space. If
X and Y are discrete, then
otherwise
E(Y ) =
yfY |X (y|x) dy =
3
x3
1
3
Z
·
2x2
dx
y2
0
Z y
2
x2 dx
y2 0
2 y3
·
y2 3
2
y
3
MTHE/STAT 351: 8 – Joint Distributions
X
x2X
=
y
Conditional expectation can greatly simplify the calculation of expected
value.
Solution: From previous calculations,
8
2
< 3y
fY |X (y|x) = 1 x3
:
0
E(Y |X = x)
Z
2x
dx
y2
MTHE/STAT 351: 8 – Joint Distributions
Hence for x 2 [0, 1)
=
1
Z
1
x
3y 3
1 x3
If X and Y are jointly continuous, then
y 3 dy
E(Y ) =
x
1
E(Y |X = x)pX (x)
x4
1 x3
4
4
3 1 x
·
4 1 x3
Z
1
1
E(Y |X = x)fX (x) dx
Note: The proof is very similar to the proof of the law of total
probability and is left as an exercise. The theorem also holds if the roles
of X and Y are exchanged.
67 / 80
MTHE/STAT 351: 8 – Joint Distributions
68 / 80
Transformations of Two Random Variables
Example: Suppose X is uniformly distributed on [0, 1]. Given X = x, let
Y be a random point in the interval [0, x]. Calculate E(Y ).
Solution: Since Y is uniformly distributed on [0, x] given X = x, we have
Here we consider the two-dimensional generalization of the problem of
finding the pdf of h(X) for an invertible h : R ! R and continuous
r.v. X.
x
E(Y |X = x) =
2
Thus by the law of total expectation
Z 1
Z
E(Y ) =
E(Y |X = x)fX (x) dx =
1
1
0
Suppose X and Y are jointly continuous and h1 , h2 : R2 ! R are real
functions. We want to determine the joint pdf of the pair of r.v.’s
x
1
dx =
2
4
U = h1 (X, Y ),
We will be able to do this if h1 and h2 satisfy certain regularity
conditions.
Exercise: Calculate E(Y ) by finding fY (y) first. Which solution is
simpler?
MTHE/STAT 351: 8 – Joint Distributions
69 / 80
1. The mapping (h1 , h2 ) : R2 ! R2 is invertible, i.e., for any u and v
the pair of equations
h1 (x, y) = u,
h2 (x, y) = v
y = g2 (u, v)
70 / 80
Theorem 12 (Joint pdf of transformed random variables)
2. The functions h1 and h2 have continuous partial derivatives
@h1 @h1 @h2 @h2
,
,
,
.
@x @y @x @y
3. The Jacobian of (h1 , h1 ) is nonzero everywhere, i.e., the following
determinant is nonzero for all x and y:
2
3
@h1 @h1
6 @x
@h1 @h2
@h1 @h2
@y 7
7
J(x, y) = det 6
6= 0
4 @h2 @h2 5 = @x @y
@y @x
@x
@y
MTHE/STAT 351: 8 – Joint Distributions
MTHE/STAT 351: 8 – Joint Distributions
Let Jb denote the Jacobian of (g1 , g2 ), i.e.,
2
3
@g1 @g1
1
@v 7
b v) = det 6
J(u,
4 @u
5=
@g2 @g2
J(g1 (u, v), g2 (u, v))
@u
@v
has at most one solution
x = g1 (u, v),
V = h2 (X, Y )
Assume X and Y have joint pdf fX,Y (x, y). If h1 and h2 satisfy the
above conditions, then the joint pdf of U = h1 (X, Y ) and V = h2 (X, Y )
is given by
b v)|
fU,V (u, v) = fX,Y g1 (u, v), g2 (u, v) |J(u,
Remark: The proof of the theorem relies on the change of variable
formula for double integrals.
71 / 80
MTHE/STAT 351: 8 – Joint Distributions
72 / 80
Example: Let X and Y be independent standard normal r.v.’s. Let R
and ⇥ be the polar coordinates of the point (X, Y ), i.e.,
p
R = X 2 + Y 2,
⇥ = arctan Y /X
The partial derivatives of g1 and g2 are
@g1
@
=
(r cos ✓) = cos ✓,
@r
@r
Find the joint pdf of R and ⇥.
Solution: We have h1 (x, y) =
We have to solve
r=
p
p
and
x2
x2 + y 2 ,
+
y2
and h2 (x, y) = arctan y/x .
✓ = arctan y/x
Note that 1 < x < 1 and
0  ✓ < 2⇡.
y = r sin ✓ = g2 (r, ✓)
73 / 80
=
=
1
fX (x)fY (y) = p e
2⇡
1
(x2 +y 2 )/2
e
2⇡
x2 /2
1
p e
2⇡
y 2 /2
The marginal pdf’s are
8
<re r2 /2
fR (r) =
:0
Therefore
fR,⇥ (r, ✓)
=
=
=
ˆ ✓)|
fX,Y g1 (r, ✓), g2 (r, ✓) |J(r,
ˆ ✓)|
fX,Y (r cos ✓, r sin ✓)|J(r,
1
e
2⇡
r 2 /2
r
74 / 80
r 2 /2
r>0
otherwise
r > 0, 0  ✓ < 2⇡
otherwise
f⇥ (✓) =
8
<
1
2⇡
:0
0  ✓ < 2⇡
otherwise
Since fR,⇥ (r, ✓) = fR (r)f⇥ (✓), the random variables R and ⇥ are
independent. Also note that ⇥ is uniformly distributed on [0, 2⇡).
for all r > 0 and 0  ✓ < 2⇡.
MTHE/STAT 351: 8 – Joint Distributions
MTHE/STAT 351: 8 – Joint Distributions
In conclusion, we obtained that
8
< 1 re
fR,⇥ (r, ✓) = 2⇡
:0
Since X and Y are independent,
fX,Y (x, y)
@g2
= r cos ✓
@✓
since cos2 ✓ + sin2 ✓ = 1.
1 < y < 1, while 0  r < 1 and
MTHE/STAT 351: 8 – Joint Distributions
r sin ✓
Thus the Jacobian of (g1 , g2 ) is
"
#
cos ✓
r sin ✓
ˆ
J(r, ✓) = det
= r cos2 ✓ + r sin2 ✓ = r
sin ✓ r cos ✓
for x and y. The solution (of course) is the well-known expression
x = r cos ✓ = g1 (r, ✓),
@g2
@
=
(r sin ✓) = sin ✓,
@r
@r
@g1
=
@✓
75 / 80
MTHE/STAT 351: 8 – Joint Distributions
76 / 80
Thus
Sum of two independent random variables
fU (u) =
Assume X and Y are independent and let U = X + Y . We want to find
the pdf of U in terms of the pdf’s of X and Y .
FU0 (u)
=
=
One way to do this is to calculate the joint pdf fU,V (u, v) of the pair
U = X + Y , V = X Y , and then find fU (u) as the marginal of
fU,V (u, v).
=
Instead, we follow a more direct approach:
FU (u) = P (U  u)
=
=
x+yu
=
We obtained
P (X + Y  u)
ZZ
fX,Y (x, y) dxdy
Z
1
1
Z
fU (u) =
fX (x)fY (y) dxdy
1
fU (u) =
77 / 80
Remark: The integral
1
fX (u
Z
y)fY (y) dy
Solution: We have fU = fX ⇤ fY , where
8
<1
fX (t) = fY (t) =
:0
For 1  u  2,
fX (u
1
MTHE/STAT 351: 8 – Joint Distributions
1
fX (x)fY (y) dxdy
✓Z
1
u y
◆
fX (x) dx fY (y) dy
{z
}
d
F
(u
y)
du X
1
fX (u
y)fY (y) dy
fX (u
y)fY (y) dy
1
1
1
Z
1
fX (x)fY (u
x) dx
1
78 / 80
0t1
otherwise
y)fY (y) dy =
Z
Z
fX (u
y) dy =
0
1
fX (u
y) dy =
0
fU (u) =
1
fX (u
y) dy
Z
1
Putting these together, we obtain
8
>
>
<u
Thus
fU (u) =
d
du
1
|
u y
MTHE/STAT 351: 8 – Joint Distributions
1
Example: Let X and Y be independent uniform r.v.’s on [0, 1]. Find the
pdf of U = X + Y .
1
Z
1
Z
If u < 0 or u > 2, then fU (u) = 0. For 0  u < 1 we have
is called the convolution of fX (x) and fY (y) and is denoted by fX ⇤ fY ,
Z
Z
1
By symmetry, the same proof also implies that
u y
MTHE/STAT 351: 8 – Joint Distributions
Z
Z
d
du
Z 1
>
>
:
2
Z
u
dy = u
0
1
dy = 2
u
u 1
0u<1
u
0
1u2
otherwise
This function is called the triangular pdf.
0
79 / 80
MTHE/STAT 351: 8 – Joint Distributions
80 / 80