Download Lecture 15: Independent Random Variables

Lecture 15:
Independent Random Variables
Dan Sloughter
Furman University
Mathematics 37
February 5, 2004
15.1
Independence: discrete variables
Definition 15.1. We say random variables X1 , X2 , . . . , Xn are independent
if for any sets A1 , A2 , . . . , An of real numbers we have
P (X1 ∈ A1 , X2 ∈ A2 , . . . , Xn ∈ An ) = P (X1 ∈ An )P (X2 ∈ An ) · · · P (Xn ∈ An ).
Theorem 15.1. Suppose X and Y are discrete random variables with joint
probability function p and marginal probability functions pX and pY . Then
X and Y are independent if and only if
p(x, y) = pX (x)pY (y)
for all (x, y) ∈ R2 .
Proof. First suppose X and Y are independent. Then for any (x, y) ∈ R,
p(x, y) = P (X = x, Y = y) = P (X = x)P (Y = y) = pX (x)pY (y).
Now suppose p(x, y) = pX (x)pY (y) for all (x, y) ∈ R. Then for any A ⊂ R
and B ⊂ R we have
XX
P (X ∈ A, Y ∈ B) =
P (X = x, Y = y)
x∈A y∈B
=
XX
x∈A y∈B
1
pX (x)pY (y)
=
X
pX (x)
x∈A
X
pY (y)
y∈B
!
=
X
pX (x)
x∈A
!
X
pY (y)
y∈B
= P (X ∈ A)P (Y ∈ B).
Hence X and Y are independent.
More generally, discrete random variables X1 , X2 , . . . Xn with joint probability function p are independent if and only if
p(x1 , x2 , . . . , xn ) = pX1 (x1 )pX2 (x2 ) · · · pXn (xn )
for all (x1 , x2 , . . . , xn ) ∈ Rn .
Example 15.1. Suppose X and Y are independent Poisson random variables, both with mean λ. Then the joint probability function of X and Y
is

λx e−λ λy e−λ



·
, if x = 0, 1, 2, . . . , y = 0, 1, 2, . . . ,

x!
y!
p(x, y) =



0,
otherwise,

λx+y e−2λ



 x!y! , if x = 0, 1, 2, . . . , y = 0, 1, 2, . . . ,
=



0,
otherwise.
15.2
Independence: jointly continuous variables
Theorem 15.2. If X and Y are random variables with joint distribution
function F and marginal distribution functions FX and FY , then X and Y
are independent if and only if
F (x, y) = FX (x)FY (y)
for all (x, y) ∈ R2 .
2
Proof. If X and Y are independent, then
F (x, y) = P (X ≤ x, Y ≤ y) = P (X ≤ x)P (Y ≤ y) = FX (x)FY (y).
Now suppose F (x, y) = FX (x)FY (y) for all (x, y) ∈ R2 and let A = (a, b] and
B = (c, d]. Then
P (X ∈ A, Y ∈ B) = P (a < X ≤ b, c < Y ≤ d)
= F (b, d) − F (a, d) − F (b, c) + F (a, c)
= FX (b)FY (d) − FX (a)FY (d) − FX (b)FY (c) + FX (a)FY (c)
= (FX (b) − FX (a))(FY (d) − FY (c))
= P (a < X ≤ b)P (c < Y ≤ d)
= P (X ∈ A)P (Y ∈ B).
It may now be shown that P (X ∈ A, Y ∈ B) = P (X ∈ A)P (Y ∈ B) for any
sets A ⊂ R and B ⊂ R. Thus X and Y are independent.
The following result for jointly continuous random variables now follows.
Theorem 15.3. Suppose X and Y are jointly continuous random variables
with joint density function f and marginal density functions fX and fY .
Then X and Y are independent if and only if
f (x, y) = fX (x)fY (y)
for all (x, y) ∈ R2 .
More generally, jointly continuous random variables X1 , X2 , . . . Xn with
joint density function f and marginal densities fXk , k = 1, 2, . . . , n, are
independent if and only if
f (x1 , x2 , . . . , xn ) = fX1 (x1 )fX2 (x2 ) · · · fXn (xn )
for all (x1 , x2 , . . . , xn ) ∈ Rn .
Example 15.2. We have seen that if X and Y have joint probability density
(
e−y , if 0 < x < y < ∞,
f (x, y) =
0,
otherwise,
3
then the marginals of X and Y are given by
(
e−x , if x > 0,
fX (x) =
0,
otherwise,
and
Hence
(
ye−y , if y > 0,
fY (y) =
0,
otherwise.
(
ye−x−y , if x > 0, y > 0,
fX (x)fY (y) =
0,
otherwise,
which is clearly not equal to f (x, y). Hence X and Y are not independent.
Note that if the set C = {(x, y) : f (x, y) > 0} is not of the form A × B,
where A = {x : fX (x) > 0} and B = {y : fY (y) > 0}, then X and Y cannot
be independent.
Example 15.3. Suppose X and Y are independent, exponential random
variables with parameters λ and β, respectively. Then the marginals of X
and Y are
(
λe−λx , if x > 0,
fX (x) =
0,
otherwise,
and
(
βe−βy , if y > 0,
fY (y) =
0,
otherwise,
so the joint probability density function of X and Y is
(
λβe−λx−βy , if x > 0, y > 0,
f (x, y) =
0,
otherwise.
Then, for example,
Z
∞
Z
P (X < Y ) =
Z0 ∞
=
0
y
λβe−λx−βy dxdy
0
y βe−βy −e−λx 0 dy
4
Z
∞
βe−βy − βe−(λ+β)y dy
0
Z b
= lim
βe−βy − βe−(λ+β)y dy
b→∞ 0
β −(λ+β)y b
−βy b
= lim −e
+
e
0
0
b→∞
λ+β
β
βb
−(λ+β)b
= lim 1 − e +
(e
− 1)
b→∞
λ+β
β
=1−
λ+β
λ
=
.
λ+β
=
Theorem 15.4. If X1 , X2 , . . . , Xn are independent random variables and
rk : R → R, k = 1, 2, . . . , n, then Y1 = r1 (X1 ), Y2 = r2 (X2 ), . . . , Yn = rn (Xn )
are independent.
Proof. For any B ⊂ R, let rk−1 (B) = {x : r(x) ∈ B}. Then for any A1 ⊂
R, A2 ⊂ R, . . . , An ⊂ R, we have
P (r1 (X1 ) ∈ A1 , . . . , rn (Xn ) ∈ Xn ) = P (X1 ∈ r1−1 (A1 ), . . . , Xn ∈ rn−1 (An ))
= P (X1 ∈ r1−1 (A1 )) · · · P (Xn ∈ rn−1 (An ))
= P (r1 (X1 ) ∈ A1 ) · · · P (rn (Xn ) ∈ An ).
Hence Y1 = r1 (X1 ), Y2 = r2 (X2 ), . . . , Yn = rn (Xn ) are independent.
Example 15.4. If X and Y are independent, then so are X 2 and sin(Y ).
5