Download Continuous random variables

Continuous random variables
So far we have been concentrating on discrete random variables, whose distributions are not
continuous. Now we deal with the so-called continuous random variables.
A random variable X is called a continuous random variable if its distribution function F is a
continuous function on R, or equivalently, if
P(X = x) = 0,
for every x ∈ R.
Of course there are random variables which are neither discrete nor continuous. But in this
course we will concentrate on discrete random variables and continuous random variables. Among
continuous random variables, we will concentrate on a subclass.
Definition 1 A random variable X is called an absolutely continuous random variable if there is
a nonnegative function f on R such that
Z x
P(X ≤ x) =
f (t)dt,
for every x ∈ R.
−∞
The function f in the definition above is called the probability density of the absolutely continuous random variable X and it must satisfy
Z ∞
f (t)dt = 1.
−∞
Definition 2 A nonnegative function f on R satisfying
Z ∞
f (t)dt = 1.
−∞
is called a probability density function.
It can be shown that if f is a probability density function, then then there exists an absolutely
continuous random variable X with f as its probability density function.
R∞
Given a nonnegative function g on R such that −∞ g(t)dt = c is a finite positive number, then
the function f (x) = 1c g(x) is a probability density function.
From Definition 1, we can easily see that an absolutely continuous random variable is a continuous random variable. However, there are continuous random variables which are not absolutely
continuous and we call these random variables singularly continuous random variables. In this
course, we will not deal with singularly continuous random variables.
If X is an absolutely continuous random variable with density f , then its distribution function
is given by
Z
x
F (x) =
f (t)dt,
−∞
1
for every x ∈ R.
Conversely, if X is an absolutely continuous random variable with distribution function F , then its
density function is given by
f (x) = F 0 (x)
at those points x where F is differentiable and f (x) = 0 at those points x where F is not differentiable.
If X is an absolutely continuous random variable with density f , then for any real numbers
a < b we have
Z
b
P(X ∈ (a, b)) = P(X ∈ [a, b]) =
f (t)dt.
a
Example 3 A point is chosen randomly from the unit disk so that each point in the disk is equally
likely. Let X the distance of the point from the center of the disk. Then the distribution function
of X is given by


x≤0

0,
F (x) =
x2 ,


1,
0<x≤1
x > 1.
This random variable is an absolutely continuous random variable with density
(
2x, x ∈ (0, 1)
f (x) =
0,
otherwise.
Example 4 Suppose that X is an absolutely continuous random variable whose density function
is given by
(
C(4x − 2x2 ), 0 < x < 2
f (x) =
0,
otherwise.
(a) What is the value of C? (b) Find P(X > 1).
Solution. (a) Since f is a probability distribution we must have
Z ∞
Z 2
f (x)dx = C
(4x − 2x2 )dx = 1.
R2
−∞
0
Since 0 (4x − 2x2 )dx = 83 , we have C = 83 .
R∞
R2
(b) P(X > 1) = 1 f (x)dx = 83 1 (4x − 2x2 )dx = 12 .
Definition 5 If X is an absolutely continuous random variable with density f , then the expectation
of X is defined as
Z
∞
xf (x)dx.
E[X] =
−∞
Example 6 Suppose that X is an absolutely continuous random variable with density given by
(
2x, x ∈ (0, 1)
f (x) =
0,
otherwise.
Then the expectation of X is
Z
Z
∞
xf (x)dx =
E[X] =
0
−∞
2
1
2
2x2 dx = .
3
The following theorem tells us that if X is an absolutely continuous random variable with
density f , then we can use the density f of X to find the expectation of any function of X.
Theorem 7 If X is an absolutely continuous random variable with density f , then for any realvalued function φ on R,
Z ∞
E[φ(X)] =
φ(x)f (x)dx.
−∞
Definition 8 If X is an absolutely continuous random variable, then its variance is defined as
Var(X) = E[(X − E[X])2 ].
Just as in the discrete case, we have
Var(X) = E[X 2 ] − (E[X])2 .
As a consequence of Theorem 7, we immediately we get the following result which we have seen
in the discrete case.
Proposition 9 If X is an absolutely continuous random variable, and a, b are real numbers, then
Var(aX + b) = a2 Var(X).
E[aX + b] = aE[X] + b,
Example 10 Find the variance of the random variable in Example 6.
R1
1
.
Solution. E[X 2 ] = 0 2x3 dx = 21 . Thus Var(X) = 21 − ( 23 )2 = 18
Example 11 Suppose that X is an absolutely continuous random variable with density given by
(
1, x ∈ (0, 1)
f (x) =
0, otherwise.
Find the expectation of eX .
Solution.
Z
E[eX ] =
1
ex dx = e − 1.
0
Now we will introduce some important classes of absolutely continuous random variables. The
first class is the uniform random variables.
A random variable X is said to be uniformly distributed over the interval (a, b) if it is absolutely
continuous with density given by
(
1
, x ∈ (a, b)
f (x) = b−a
0,
otherwise.
If X is uniformly distributed over the interval (a, b), then
E[X] =
a+b
,
2
var(X) =
3
(b − a)2
.
12
Example 12 Buses arrive at a specified bus stop at 15 minutes intervals starting at 7 am. That is,
they arrive at 7, 7:15, 7:30, and so on. If a passenger arrives at a stop at a time that is uniformly
distributed between 7 and 7:30, find the probability that he waits (a) less than 5 minutes for a bus;
(b) more than 10 minutes for a bus.
Example 13 A stick of length 1 is split at a point X that is uniformly distributed over (0, 1).
Suppose that p is a point in (0, 1). Determine the expected length of the piece that contains the
point p.
Now we introduce the normal random variables. Consider the function
g(x) = e−x
It is easy to see that the integral
Z
2 /2
,
x ∈ R.
∞
c=
g(x)dx
−∞
converges and thus c is a finite positive number. There is no simple formula for the antiderivative
of g and so we can not evaluate the value of c by the fundamental theorem of calculus. The easiest
way to evaluate c is by a very special trick in which we we write c as a two-dimensional integral
and introduce polar coordinates.
Z ∞
Z ∞
Z ∞Z ∞
2
2
2
−x2 /2
−y 2 /2
c =
e
dx
e
dy =
e−(x +y )/2 dxdy
−∞
−∞
−∞ −∞
¶
Z ∞ µZ π
Z ∞
2
2
=
e−r /2 rdθ dr = 2π
re−r /2 dr
0
=
−π
−r2 /2 ∞
−2πe
|0
0
= 2π.
Therefore the function
1
2
f (x) = √ e−x /2 , x ∈ R
2π
is a probability density and it is called the standard normal density. An absolutely continuous
random variable whose density is a standard normal density is called a standard normal random
variable.
There is no simple formula for the distribution function of a standard normal random variable.
It is traditional to denote the distribution function of a standard normal random variable by
Z x
1
2
e−t /2 dt, x ∈ R.
Φ(x) = √
2π −∞
One can find the values of Φ(x) by tables given in probability textbooks or by computer. It is easy
to check that Φ satsfies the following relation
Φ(−x) = 1 − Φ(x),
x ∈ R.
Suppose that X is a standard normal random variable, µ is a real number and σ > 0. Let
consider the random variable Y = µ + σX. For any y ∈ R,
P(Y ≤ y) = P(µ + σX ≤ y) = P(X ≤ (y − µ)/σ)
4
=
1
√
2π
Z
(y−µ)/σ
2 /2
e−t
dt.
−∞
By the chain rule and the fundamental of calculus we know that Y is an absolutely continuous
random variable whose density is given by
1
2
2
f (y) = √ e(y−µ) /2σ ,
σ 2π
y ∈ R.
The probability density above is called a normal density with parameters µ and σ 2 , and will be
denoted by n(µ, σ 2 ) or n(y; µ, σ 2 ). An absolutely continuous random variable whose density is a
normal density n(µ, σ 2 ) is called a normal random variable with parameters µ and σ 2 . The standard
normal density is simply a normal density with parameters 0 and 1 n(0, 1). Thus we have shown
that if X is a standard normal random variable, µ is a real number and σ > 0, then the random
variable Y = µ + σX is a normal random variable with parameters µ and σ 2 .
Similar to the paragraph above, we can show that, if X is a normal random variable with
parameters µ and σ 2 , then Z = (X − µ)/σ is a standard normal random variable. This fact will
be very useful in finding the probabilities involving general normal random variables.
Theorem 14 If X is a normal random variable with parameters µ and σ 2 , then
E[X] = µ,
Var(X) = σ 2 .
Proof Since X can be written as µ + σZ, where Z is a standard normal random variable, so
we only need to prove the theorem when X is a standard normal random variable. So we assume
that X is a standard normal random variable.
2
Since the function x √12π e−x /2 is an odd function, we get that
Z
E[X] =
∞
1
2
x √ e−x /2 dx = 0.
2π
−∞
2
Using integration by parts (with u = x, dv = xe−x /2 ) we get
Z ∞
1
2
2
√
Var(X) = E[X ] =
x2 e−x /2 dx
2π −∞
µ
¶
Z ∞
1
−x2 /2 ∞
−x2 /2
= √
−xe
|−∞ +
e
dx
2π
−∞
Z ∞
1
2
e−x /2 dx = 1.
= √
2π −∞
Example 15 If X is a normal random variable with parameters µ = 3 and σ 2 = 9, find (a)
P(2 < X < 5); (b) P(X > 0); (c) P(|X − 3| > 6).
Solution (a).
¶
µ
¶
µ
1
2−3
X −3
5−3
X −3
2
=P − <
P(2 < X < 5) = P
<
<
<
3
3
3
3
3
3
2
1
= Φ( ) − Φ(− ) ≈ .3799
3
3
5
(b).
µ
¶
µ
¶
X −3
X −3
0−3
P(X > 0) = P
>
=P
> −1
3
3
3
= 1 − Φ(−1) = Φ(1) ≈ .8413.
(c).
P(|X − 3| > 6) = P(X − 3 > 6) + P(X − 3 < −6)
¶
µ
¶
µ
X −3
X −3
>2 +P
< −2
= P
3
3
= 1 − Φ(2) + Φ(−2) = 2(1 − Φ(2)) =≈ .0456.
Example 16 When we say that grades for a test is curved, we mean that, after the test, the
instructor finds the mean and variance of the scores, and then assigns the letter grade A to those
whose test score is greater than µ + σ, B to those whose score is between µ and µ + σ, C to those
whose score is between µ − σ and µ, and D to those whose score is between µ − 2σ and µ − σ, and
F to those whose score is below µ − σ. Assuming that the test score follows a normal distribution,
find the probability of each letter grade.
Solution
P(X > µ + σ) = 1 − Φ(1) ≈ .1587
P(µ < X ≤ µ + σ) = Φ(1) − Φ(0) ≈ .3413
P(µ − σ < X ≤ µ) = Φ(0) − Φ(−1) ≈ .3413
P(µ − 2σ < X ≤ µ − σ) = Φ(−1) − Φ(−2) ≈ .1359
P(X < µ − 2σ) = Φ(−2) ≈ .0228.
Example 17 Suppose that X is a normal random variable with parameters µ = 0 and σ 2 = 4.
Find the value of c so that P(|X| > c) = .1.
Solution
X
c
X
c
c
P(|X| > c) = P(X > c) + P(X < −c) = P( > ) + P( < − ) = 2(1 − Φ( )).
2
2
2
2
2
In order that P(|X| > c) = .1, we need 2(1 − Φ( 2c )) = .1, that is, Φ( 2c ) = .995. From the table we
get that 2c = 2.58, and so c = 5.16.
Now we introduce the exponential random variables. For any λ > 0, the following function
(
λe−λx x > 0
f (x) =
0
x≤0
is a probability density function and it is called the exponential density with parameter λ.
For any λ > 0, an absolutely continuous random variable X is called an exponential random
variable with parameter λ if its density function is an exponential density with parameter λ.
Suppose that X is an exponential random variable with parameter λ, then for any x > 0, we
have
Z x
λe−λt dt = 1 − e−λx .
P(X ≤ x) =
0
6
So the distribution function of X is given by
(
1 − e−λx
F (x) =
0
x≥0
x<0
Theorem 18 Suppose that X is an exponential random variable with parameter λ, then E[X] =
and Var(X) = λ12 .
1
λ
Proof For any positive integer n, using integration by parts, we have
Z ∞
n
E[X ] =
xn λe−λx dx
0
Z ∞
n −λx ∞
= −x e
|−∞ +
nxn−1 e−λx dx
0
Z
n ∞ n−1 −λx
n
=
x
λe
dx = E[X n−1 ].
λ 0
λ
Letting n = 1 and then n = 2, we get E[X] =
1
λ
and E[X 2 ] = λ2 E[X] =
2
.
λ2
Thus Var(X) =
1
.
λ2
Example 19 Suppose that the length of a phone call in minutes is an exponential random variable
1
with parameter λ = 10
. If someone arrives immediately ahead of you at a public telephone booth,
find the probability that you have to wait (a) more than 10 minutes; (b( between 10 and 20 minutes.
We say that a positive random variable X is memoryless if
P(X > s + t|X > t) = P(X > s),
for all s, t > 0.
It is very easy to check that any exponential random variable is memoryless. In fact, we can prove
that if a positive random variable X is memoryless in the sense above, it must be an exponential
random variable. This suggests that exponential random variables are very useful in modeling
lifetimes of certain equipment when the memoryless property is approximately satisfied.
To introduce the next type of random variable, we recall the definition of Gamma function. For
any α > 0, the Gamma function is defined by
Z ∞
Γ(α) =
xα−1 e−x dx.
0
There is no explicit formula for this function. One can easily see that Γ(1) = 1. In fact, we can
easily find a formula for Γ(n) for any positive integer n. Before this, we observe the following
Proposition 20 For any α > 0, we have Γ(α + 1) = αΓ(α).
proof
Z
Γ(α + 1) =
∞
xα e−x dx
0
Z
= −xα e−x |∞
0 +
= αΓ(α).
7
0
∞
αxα−1 e−x dx
Using this proposition, we immediately get Γ(n) = (n − 1)!.
One can easily check that, for any α > 0 and λ > 0, the function
( α
λ
xα−1 e−λx x > 0
f (x) = Γ(α)
0
x≤0
is a probability density function and it is called a Gamma density with parameters α and λ.
For any α > 0 and λ > 0, an absolutely continuous random variable is called a Gamma random
variable with parameters α and λ if its density is a Gamma density with parameters α and λ.
A Gamma random variable with parameters α = 1 and λ is simply an exponential random
variable with parameter λ.
Theorem 21 Suppose that X is a Gamma random variable with parameters α and λ, then E[X] =
α
α
λ and Var(X) = λ2 .
Proof
E[X] =
=
=
Z ∞
1
λxe−λx (λx)α−1 dx
Γ(α) 0
Z ∞
1
λe−λx (λx)α dx
λΓ(α) 0
Γ(α + 1)
α
= .
λΓ(α)
λ
Similarly, we can find E[X 2 ] and then find that Var(X) =
α
.
λ2
Example 22 Suppose that X is a normal random variable with parameters µ = 0 and σ 2 . Find
the density of Y = X 2 .
Solution Let F be the distribution function of X and f the density function of X. Y is a
nonnegative random variable. For any y > 0, we have
√
√
P(Y ≤ y) = P(X 2 ≤ y) = P(− y ≤ X ≤ y)
√
√
= F ( y) − F (− y).
Thus the density of Y is


g(y) =
√
1
√
2 y (f ( y)
√
+ f (− y)) y > 0
0
y ≤ 0.
Plugging in the expression for f , we get that the density of Y is

 √1 e−y/(2σ2 ) y > 0
g(y) = σ 2πy
0
y ≤ 0.
(0.1)
This example shows that, if X is a normal random variable with parameters µ = 0 and σ 2 , then
is a Gamma random variable with parameters α = 12 and λ = 2σ1 2 . By comparing (0.1) with
definition of Gamma density we see that
X2
√
1
Γ( ) = π.
2
8
From this and Proposition 20 we immediately get that for any positive odd integer n
√
π(n − 1)!
n
Γ( ) = n−1 ¡ n−1 ¢ .
2
!
2
2
Now we deal with the following problem: Suppose that X is an absolutely continuous random
variable with density f and φ is a function on R, how do we find the density of the random variable
Y = φ(X). Our strategy is to first find the distribution of Y in terms of f and then find the density
of Y . We illustrate this by a few examples.
Example 23 Let X be uniformly distributed on (0, 1) and λ > 0 a constant. Find the density of
Y = −λ−1 ln(1 − X).
Solution. Obviously Y is a nonnegative random variable. For any y > 0,
P(Y ≤ y) = P(−λ−1 ln(1 − X) ≤ y) = P(ln(1 − X) ≥ −λy)
= P(1 − X ≥ e−λy ) = P(X ≤ 1 − e−λy )
= 1 − e−λy .
Hence the density of Y is given by
g(y) =
(
λe−λy
0
y>0
y≤0
.
Example 24 Let X be an exponential random variable with parameter λ and β 6= 0 a constant.
Find the density of Y = X 1/β .
Solution. We deal with this example in two separate cases: β > 0 and β < 0. The two cases
are similar and we only go over the case β < 0. Y is obviously a nonnegative random variable. For
any y > 0,
P(Y ≤ y) = P(X 1/β ≤ y) = P(X ≥ y β )
β
= e−λy .
Hence the density of Y is given by
g(y) =
(
β
|β|λy β−1 e−λy
0
y>0
y≤0
Remark on Notations
9
.