Download Continuous Random Variables

Continuous Random Variables
Chia-Ping Chen
Professor
Department of Computer Science and Engineering
National Sun Yat-sen University
Probability
Introduction
We have studied discrete random variables, whose
values are discrete.
Now we introduce continuous random variables, whose
values are continuous.
Examples
delay times of Romeo and Juliet
wheel of fortune
noise
landing point
Prof. C. Chen
Continuous Random Variables
Alternative Definition of Random Variable
Let (Ω, F, P(·)) be a probability model. A random variable
X defined on Ω is such that
{X ≤ x} = {ω | X(ω) ≤ x}
is an event in F for any x ∈ R.
Prof. C. Chen
Continuous Random Variables
Cumulative Distribution Function
Since
{X ≤ x} = {ω | X(ω) ≤ x}
is an event in F for any x ∈ R, we can define
FX (x) = P(X ≤ x)
as the cumulative distribution function (CDF) of random
variable X.
Prof. C. Chen
Continuous Random Variables
From Discrete to Continuous Random Variables
Let’s look at the CDFs of a few discrete random variables.
Bernoulli
binomial
geometric
Prof. C. Chen
Continuous Random Variables
Continuous Random Variable
A random variable X is continuous if there exists a function
fX (x) such that the probability that X takes a value in any
interval I can be written as an integral
P(X ∈ I) =
Z
fX (x)dx
I
For a small interval
P(x ≤ X ≤ x + dx) =
Z x+dx
fX (x)dx ≈ fX (x)dx
x
P(x ≤ X ≤ x + dx)
dx
Thus, fX (x) is the probability per unit length, i.e. density, at x.
fX (x) is called probability density function (PDF).
⇒ fX (x) ≈
Prof. C. Chen
Continuous Random Variables
PDF and CDF
PDF is the derivative of CDF.
Let X be a continuous random variable.
FX (x) = P(X ≤ x) =
⇒ fX (x) =
Prof. C. Chen
Z x
−∞
fX (x0 )dx0
dFX (x)
dx
Continuous Random Variables
Example 3.1 Wheel of Fortune
A gambler spins a wheel of fortune, which is continuously
calibrated between 0 and 1, and observes the resulting number.
Assuming that any two subintervals of [0, 1] of the same length
have the same probability, this experiment can be modelled in
terms of a random variable X with PDF
(
fX (x) =
0≤x≤1
otherwise
c,
0,
The constant c can be decided through normalization
Z
fX (x)dx = 1 ⇒
Z 1
c dx = 1 ⇒ c = 1
0
Prof. C. Chen
Continuous Random Variables
Specifying a Continuous Random Variable
A continuous random variable is completely specified by a PDF.
Prof. C. Chen
Continuous Random Variables
Continuous Uniform Random Variable
For a continuous uniform random variable X
(
fX (x) =
1
b−a ,
0,
a≤x≤b
otherwise
where a < b.
Prof. C. Chen
Continuous Random Variables
Example 3.2 Driving Time
Alvin’s driving time to work is between 15 and 20 minutes in a
sunny day, and between 20 and 25 minutes in a rainy day, with
all times being equally likely in each case. Assume that a day is
sunny with probability 2/3, and rainy with probability 1/3.
What is the PDF of the driving time, viewed as a random
variable X?
Prof. C. Chen
Continuous Random Variables
Example 3.3
Consider a random variable X with PDF

1
 √
,
fX (x) = 2 x
0,
0<x≤1
otherwise
Although fX (x) becomes arbitrarily large as x approaches 0, it
is a valid PDF nonetheless, since it is non-negative and
Z 1
Z
fX (x)dx =
0
Prof. C. Chen
1
√ dx = 1
2 x
Continuous Random Variables
PMF and CDF
For a discrete random variable, PMF is the difference of CDF.
The elements in the image of a discrete random variable X can
be sorted
−∞ < x1 < x2 < · · · < ∞
⇒ FX (xi ) = P(X ≤ xi )
= pX (x1 ) + · · · + pX (xi−1 ) + pX (xi )
= FX (xi−1 ) + pX (xi )
⇒ pX (xi ) = FX (xi ) − FX (xi−1 )
Prof. C. Chen
Continuous Random Variables
Example 3.6 Tests
You are allowed to take a certain test 3 times, and your final
score will be the maximum of the test scores. Assume that your
score in each test takes one of the values from 1 to 10 with
equal probability 1/10, independent of the scores in other tests.
What is the PMF of the final score X?
Prof. C. Chen
Continuous Random Variables
Using CDF simplifies matters. Let Xi be the score of the ith
test and X be the final score. Then
X = max{X1 , X2 , X3 }
⇒ {X ≤ x} = {X1 ≤ x} ∩ {X2 ≤ x} ∩ {X3 ≤ x}
⇒ FX (x) = FX1 (x)FX2 (x)FX3 (x) =
x
10
3
⇒ pX (k) = FX (k) − FX (k − 1) =
Prof. C. Chen
k
10
−
3
k−1
10
Continuous Random Variables
3
Expectation, Variance, and Moments
expectation
Z ∞
E[X] =
−∞
variance
var(X) = E[(X − E[X])2 ]
Z ∞
=
−∞
moments
xfX (x)dx
(x − E[X])2 fX (x)dx
mn (X) = E[X n ]
Z ∞
=
−∞
Prof. C. Chen
xn fX (x)dx
Continuous Random Variables
Example 3.4 Uniform Random Variable
Consider a continuous random variable X uniformly distributed
over the interval [a, b]. What is the expectation and variance of
X?
Prof. C. Chen
Continuous Random Variables
Exponential Random Variables
For an exponential random variable T , the PDF is an
exponential function
(
fT (t) =
λe−λt , t ≥ 0
0,
otherwise
where λ > 0 is a parameter. This is denoted by
T ∼ exponential(λ)
CDF
Z t
FT (t) =
−∞
(
0
0
fT (t )dt =
Prof. C. Chen
1 − e−λt , t ≥ 0
0,
t<0
Continuous Random Variables
mean
Z ∞
E[T ] =
t fT (t)dt =
−∞
Z ∞ t λe−λt dt
0
= (−te
−λt
∞ Z ∞
) +
e−λt dt
0
0
1
=
λ
variance
E[T 2 ] =
=
Z ∞
t2 fT (t)dt =
−∞
∞
−t2 e−λt λ
0
Z ∞
t2 λe−λt dt
0
+
1
λ
Z ∞
2te−λt dt
0
2
= 2
λ
⇒ var(T ) = E[T 2 ] − E 2 [T ] =
Prof. C. Chen
1
λ2
Continuous Random Variables
Example 3.5 Meteorite
The time until a small meteorite first lands anywhere in the
Sahara desert is modeled as an exponential random variable
with a mean of 10 days. The time is currently midnight. What
is the probability that a meteorite first lands sometime between
6 a.m. and 6 p.m. of the first day?
E[T ] = 10 =
1
3
⇒ P
≤T ≤
4
4
1
1
⇒ λ=
λ
10
3
4
Z
=
1
4
3
4
Z
fT (t)dt =
1
4
λe−λt dt
3
1
−λt 4
= −e
4
1
− 40
=e
Prof. C. Chen
3
− e− 40
Continuous Random Variables
Gaussian Random Variables
For a Gaussian random variable X, the PDF is
fX (x) = √
(x−µ)2
1
e− 2σ2
2πσ
where parameters µ and σ 2 are parameters. This is denoted by
by
X ∼ N (µ, σ 2 )
Prof. C. Chen
Continuous Random Variables
Properties
normalization
Z ∞
−∞
1
2
2
√
e−(x−µ) /2σ dx = 1
2πσ
expectation
E[X] = µ
variance
var(X) = σ 2
Prof. C. Chen
Continuous Random Variables
Normalization
Z ∞
α=
−∞
1
=√
2π
√
1
2
2
e−(x−µ) /2σ dx
2πσ
Z ∞
e−y
2 /2
dy
−∞
∞
1
1
2
√
⇒ α = √
e−x /2 dx
2π −∞
2π
Z
Z
1 ∞ ∞ −(x2 +y2 )/2
=
e
dxdy
2π −∞ −∞
Z
Z
1 2π ∞ −r2 /2
=
e
rdrdθ
2π 0
0
=1
2
Z
Prof. C. Chen
Z ∞
e
−y 2 /2
−∞
Continuous Random Variables
dy
Variance
var(X) = E[(X − µ)2 ]
Z ∞
1
2
2
√
=
(x − µ)2 e−(x−µ) /2σ dx
2πσ
−∞
Z ∞
1
2
2
=√
z 2 e−z /2σ dz
2πσ −∞
Z ∞
σ2
2
y 2 e−y /2 dy
=√
2π −∞
Z ∞
∞
σ2
−y 2 /2 −y 2 /2
=√
−ye
+
e
dy
−∞
2π
−∞
σ2 √
=√
2π
2π
= σ2
Prof. C. Chen
Continuous Random Variables
Standard Gaussian
A standard Gaussian is a Gaussian with zero mean and unit
variance. That is
Y ∼ N (0, 1)
From a Gaussian
X ∼ N (µ, σ 2 )
we can get a standard Gaussian through a linear function
Y =
X −µ
∼ N (0, 1)
σ
Prof. C. Chen
Continuous Random Variables
Standard Normal Table
A standard normal table tabulates the values of the CDF of
a standard Gaussian
1
Φ(y) = √
2π
Z y
2 /2
e−t
dt
−∞
for y > 0.
FY (y) + FY (−y) = 1
⇒ FY (y) = 1 − FY (−y)
⇒ Φ(y) = 1 − Φ(−y)
which can be used to get Φ(y) for y < 0.
Prof. C. Chen
Continuous Random Variables
Example 3.7 Snowfall at Mountain Rainier
The yearly snowfall at Mountain Rainier is modeled as a
normal random variable with a mean of µ = 60 and a standard
deviation of σ = 20. What is the probability that this year’s
snowfall will be at least 80 inches?
X − 60
80 − 60
P(X ≥ 80) = P
≥
20
20
= P(Y ≥ 1)
= 1 − P(Y ≤ 1)
= 1 − Φ(1)
= 1 − 0.8413
= 0.1587
Prof. C. Chen
Continuous Random Variables
Example 3.8 Message Transmission
A binary message is transmitted as a signal S, which is either
−1 or +1. The communication channel corrupts the
transmission with additive normal noise with mean µ = 0 and
variance σ 2 . The receiver concludes that the signal −1 (or +1)
was transmitted if the value received is < 0 (or ≥ 0). What is
the probability of error?
Prof. C. Chen
Continuous Random Variables
P(E) = P(E, S = 1) + P(E, S = −1)
= P(E|S = 1)P(S = 1) + P(E|S = −1)P(S = −1)
= P(S + Z < 0|S = 1)P(S = 1)
+ P(S + Z > 0|S = −1)P(S = −1)
= P(Z ≤ −1)P(S = 1) + P(Z ≥ 1)P(S = −1)
= P(Z ≥ 1)
= 1 − P(Z < 1)
1−0
Z −0
<
=1−P
σ
σ
1
=1−Φ
σ
Prof. C. Chen
Continuous Random Variables
CDF of Continuous Random Variables
Let’s look at the CDFs of some continuous random variables.
uniform
exponential
Gaussian
Prof. C. Chen
Continuous Random Variables
Multiple Continuous Random Variables
Prof. C. Chen
Continuous Random Variables
Joint Cumulative Distribution Function
The joint cumulative distribution function (joint CDF) of
random variables X and Y is
FXY (x, y) = P(X ≤ x, Y ≤ y)
It is the probability of the region left-and-below (x, y).
Prof. C. Chen
Continuous Random Variables
Joint Probability Density Function
For a small rectangle
P(x ≤ X ≤ x + dx, y ≤ Y ≤ y + dy)
= P(X ≤ x + dx, Y ≤ y + dy) − P(X ≤ x, Y ≤ y + dy)
− P(X ≤ x + dx, Y ≤ y) + P(X ≤ x, Y ≤ y)
= FXY (x + dx, y + dy) − FXY (x, y + dy)
− FXY (x + dx, y) + FXY (x, y)
=
∂ 2 FXY (x, y)
dxdy
∂x∂y
So we can see
fXY (x, y) =
∂ 2 FXY (x, y)
∂x∂y
and it is the probability per unit area. It is called joint
probability density function.
Prof. C. Chen
Continuous Random Variables
Marginal PDF and Joint PDF
Z ∞
fX (x) =
−∞
fXY (x, y)dy
FX (t) = P(X ≤ t) = P(X ≤ t, Y < ∞)
Z ∞ Z t
=
−∞ −∞
Z t Z ∞
=
−∞ −∞
Z t
=
−∞
fXY (x, y)dxdy
fXY (x, y)dy dx
fX (x)dx
⇒ fX (x) =
Z ∞
Prof. C. Chen
−∞
fXY (x, y)dy
Continuous Random Variables
Example 3.9 Romeo and Juliet
Consider the dating of Romeo and Juliet. What is the joint
PDF for the random delays of Romeo and Juliet?
Prof. C. Chen
Continuous Random Variables
Example 3.10 Uniform Random Variables
Suppose that the joint PDF of the random variables X and Y is
a constant c on the set S shown in Fig. 3.12 and is zero outside.
Determine the value of c and the PDF of X.
Prof. C. Chen
Continuous Random Variables
Example 3.11 Buffon’s Needle
A surface is ruled with parallel lines, which are at distance d
from each other. Suppose that we throw a needle of length l on
the surface at random. We assume that l < d. What is the
probability that the needle will intersect one of the lines?
Prof. C. Chen
Continuous Random Variables
Let X be the distance from the center of needle to nearest line,
and Θ be the angle between needle and line. Assume X and Θ
are uniform and independent
fXΘ (x, θ) = fX (x)fΘ (θ) =
4
,
πd
d
π
, 0≤θ≤
2
2
0≤x≤
l
⇒ P(intersecting) = P X ≤ sin Θ
2
Z Z
=
x≤ 2l sin θ
Z
=
π
2
0
2l
=
πd
2l
=
πd
Prof. C. Chen
l
2
Z
0
Z
π
2
sin θ
fXΘ (x, θ)dxdθ
4
dxdθ
πd
sin θdθ
0
Continuous Random Variables
Example 3.12 Unit Square
Let X and Y have a joint uniform PDF on the unit square.
⇒ fXY (x, y) = 1
⇒ FXY (x, y) = xy
Prof. C. Chen
Continuous Random Variables
Conditional Probability of Continuous Random
Variables
Prof. C. Chen
Continuous Random Variables
Conditional CDF and Conditional PDF
Let X be a continuous random variable. Conditioned on a
non-null event A, the conditional CDF of X is the
conditional probability
FX|A (x) = P(X ≤ x|A)
as a function of x.
The derivative of a conditional CDF is a conditional PDF
fX|A (x) =
Prof. C. Chen
d
F
(x)
dx X|A
Continuous Random Variables
Example 3.13 Exponential Random Variable
The time T until a new light bulb burns out is an exponential
random variable with parameter λ. Alice turns the light on,
leaves the room, and when she returns, t time units later, finds
that the light bulb is still on. Let X be the additional time until
the light bulb burns out. What is the conditional PDF of X?
FX|{T >t} (x) = P(X ≤ x|T > t)
= P(T − t ≤ x|T > t)
P(t < T ≤ t + x)
=
P(T > t)
e−λt − e−λ(t+x)
e−λt
= 1 − e−λx
=
⇒ fX|{T >t} (x) = λe−λx = fT (x)
Prof. C. Chen
Continuous Random Variables
Total Probability Theorem for PDF
Let A1 , . . . , An be a partition of the sample space Ω with
P(Ai ) > 0, and X be a continuous random variable. Then
fX (x) =
n
X
P(Ai )fX|Ai (x)
i=1
Unconditional probability density can be synthesized by
conditional probability densities.
Prof. C. Chen
Continuous Random Variables
Proof
FX (x) = P(X ≤ x) =
n
X
P(X ≤ x, Ai )
i=1
=
=
n
X
i=1
n
X
P(Ai )P(X ≤ x|Ai )
Z x
P(Ai )
i=1
=
Z x X
n
−∞ i=1
Z x
=
−∞
−∞
fX|Ai (x0 )dx0
P(Ai )fX|Ai (x0 )dx0
fX (x0 )dx0
⇒ fX (x) =
n
X
P(Ai )fX|Ai (x)
i=1
Prof. C. Chen
Continuous Random Variables
Example 3.14 Waiting Time
A train arrives at a station every quarter hour starting at 6:00
a.m. You walk into the station every morning between 7:10
a.m. and 7:30 a.m., and your arrival time is a uniform random
variable over this interval. What is the PDF of the time you
have to wait for the first train to arrive?
Let X be the waiting time.
A = {catch the 7:15 train}
⇒ fX (x) = P(A)fX|A (x) + P(Ac )fX|Ac (x)
(
=
1
4
1
4
1
· 15 + 34 · 15
,
3
1
· 0 + 4 · 15 ,
Prof. C. Chen
0≤x≤5
5 ≤ x ≤ 15
Continuous Random Variables
Conditional PDF Conditioned on a Random Variable
Let X and Y be continuous random variables. The conditional
PDF of X given Y = y with fY (y) > 0 is defined by
fX|Y (x|y) =
fXY (x, y)
fY (y)
P(x ≤ X ≤ x + dx | y ≤ Y ≤ y + dy)
P(x ≤ X ≤ x + dx, y ≤ Y ≤ y + dy)
=
P(y ≤ Y ≤ y + dy)
fXY (x, y)dxdy
≈
fY (y)dy
= fX|Y (x|y)dx
So fX|Y (x|y) is the probability per unit length (density) of x
near X = x and Y = y.
Prof. C. Chen
Continuous Random Variables
Example 3.15 Darting
Ben throws a dart at a circular target of radius r. Let the point
of impact be (X, Y ). We assume that he always hits the target,
and that all points of impact are equally likely. What is the
conditional PDF fX|Y (x|y)?
fX|Y (x|y) =
Z √r2 −y2
Z
fY (y) =
fXY (x, y)
fY (y)
fXY (x, y)dx =
⇒ fX|Y (x|y) =
√
−
r2 −y 2
1
2 q 2
dx
=
r − y2
πr2
πr2
fXY (x, y)
1
= p 2
,
fY (y)
2 r − y2
Prof. C. Chen
|x| ≤
q
r2 − y2
Continuous Random Variables
Example 3.16 Police Radar
The speed of a vehicle that drives past a police radar is modeled
as an exponential random variable X with mean 50 miles per
hour. The police radar’s measurement Y of the vehicle’s speed
has an error which is modeled as a normal random variable
with zero mean and standard deviation equal to one tenth of
the vehicle’s speed. What is the joint PDF of X and Y ?
Prof. C. Chen
Continuous Random Variables
The distribution of reading depends on the speed of vehicle. In
this case, the joint PDF of X and Y is naturally factorized into
fXY (x, y) = fX (x)fY |X (y|x)
−
1
1
e
= e−λx √
50
2πσY |X
x
1
1
= e− 50 √
50
2π
Prof. C. Chen
−
x
10
e
(y−µY |X )2
2σ 2
Y |X
(y−x)2
x 2
2 10
( )
Continuous Random Variables
Comments
From joint PDF to conditional PDF and marginal PDF
fY |X (y|x) =
fXY (x, y)
fXY (x, y)
=R
fX (x)
fXY (x, y 0 )dy 0
From conditional PDF and marginal PDF to joint PDF
fXY (x, y) = fX (x)fY |X (y|x)
Prof. C. Chen
Continuous Random Variables
Conditional Expectation
The conditional expectation of a continuous random
variable X conditioned on an event A is defined by
Z ∞
E[X|A] =
−∞
x fX|A (x)dx
The conditional expectation of a continuous random
variable X conditioned on Y = y (Y is another continuous
random variable) is defined by
Z ∞
E[X|Y = y] =
−∞
Prof. C. Chen
x fX|Y (x|y)dx
Continuous Random Variables
Total Expectation Theorem I
Let X and Y be continuous random variables. Then
Z ∞
E[X] =
−∞
E[X|Y = y] fY (y)dy
Z ∞
E[X] =
−∞
Z ∞
xfX (x)dx
Z ∞
x
=
−∞
Z ∞
fXY (x, y)dydx
−∞
Z ∞
x
=
fX|Y (x|y)fY (y)dydx
−∞
−∞
Z ∞ Z ∞
=
−∞
Z ∞
=
−∞
−∞
xfX|Y (x|y)dx fY (y)dy
E[X|Y = y] fY (y)dy
Prof. C. Chen
Continuous Random Variables
Total Expectation Theorem II
Let X be a continuous random variable, and A1 , . . . , An be a
partition of the sample space with P(Ai ) > 0. Then
E[X] =
n
X
P(Ai )E[X|Ai ]
i=1
Z ∞
E[X] =
−∞
Z ∞
x
=
−∞
=
=
xfX (x)dx
n
X
i=1
n
X
n
X
P(Ai )fX|Ai (x)dx
i=1
Z ∞
P(Ai )
−∞
xfX|Ai (x)dx
P(Ai )E[X|Ai ]
i=1
Prof. C. Chen
Continuous Random Variables
Example 3.17
Let X be a continuous random variable with piecewise constant
PDF



1/3, 0 ≤ x ≤ 1
fX (x) = 2/3, 1 < x ≤ 2


0,
otherwise
Define events
A1 = {X lies in the interval [0, 1]}
A2 = {X lies in the interval [1, 2]}
Find E[X] and var(X) via the total expectation theorem.
Prof. C. Chen
Continuous Random Variables
Independent Random Variables
Let X and Y be continuous random variables. They are said to
be independent if
fXY (x, y) = fX (x)fY (y)
Equivalent to
fX|Y (x|y) = fX (x)
and
fY |X (y|x) = fY (y)
Prof. C. Chen
Continuous Random Variables
Example 3.18 Independent Gaussian Random Variables
Let X and Y be independent normal random variables with
means µx , µy and variances σx2 , σy2 , respectively. What is the
joint PDF of X and Y ?
Prof. C. Chen
Continuous Random Variables
Bayes’ Rule
Prof. C. Chen
Continuous Random Variables
Continuous Random Variables
Let X and Y be continuous random variables. Let the PDF of
X be fX (x) and the conditional PDF of Y given X = x be
fY |X (y|x). The conditional PDF of X given Y = y is
fX|Y (x|y) = R
fX (x)fY |X (y|x)
fX (x0 )fY |X (y|x0 )dx0
fXY (x, y) = fX (x)fY |X (y|x)
⇒ fY (y) =
Z
Z
fXY (x, y)dx =
⇒ fX|Y (x|y) =
fX (x0 )fY |X (y|x0 )dx0
fX (x)fY |X (y|x)
fXY (x, y)
=R
fY (y)
fX (x0 )fY |X (y|x0 )dx0
Prof. C. Chen
Continuous Random Variables
Example 3.19 Light Bulb
A light bulb is known to have an exponentially distributed
lifetime Y . However, the manufacturing company is
experiencing quality control problems, so the parameter Λ of
the PDF of Y is random, uniformly distributed in the interval
[1, 3/2]. We test a light bulb and record its lifetime y. What
can we say about the parameter Λ?
fΛY (λ, y)
fY (y)
fΛ (λ)fY |Λ (y|λ)
=R
fΛ (λ0 )fY |Λ (y|λ0 )dλ0
fΛ|Y (λ|y) =
3
= 1≤λ≤
2

? R 3
2
1
Prof. C. Chen
2λe−λy
2λ0 e−λ0 y dλ0

 : 0
Continuous Random Variables
Mixed Random Variables
Let N be a discrete random variable with PMF pN (n), and Y
be a continuous random variable with conditional PDF
fY |N (y|n). Then
pN (n)fY |N (y|n)
pN |Y (n|y) = P
pN (n0 )fY |N (y|n0 )
n0
fN Y (n, y) = pN (n)fY |N (y|n)
⇒ fY (y) =
X
fN Y (n, y) =
n
⇒ pN |Y (n|y) =
X
pN (n)fY |N (y|n)
n
pN (n)fY |N (y|n)
fN Y (n, y)
=P
fY (y)
pN (n0 )fY |N (y|n0 )
n0
Prof. C. Chen
Continuous Random Variables
Example 3.20 Binary Signal Transmission
A binary signal S is transmitted, and we are given that
P(S = 1) = p and P(S = −1) = 1 − p. The received signal is
Y = S + N , where N is a normal noise with zero mean and unit
variance. What is the probability that S = 1, as a function of
the observed value y of Y ?
PS|Y (1|y) =
=
=
pS (1)fY |S (y|1)
pS (1)fY |S (y|1) + pS (−1)fY |S (y| − 1)
p √12π e−(y−1)
2 /2
p √12π e−(y−1)2 /2 + (1 − p) √12π e−(y+1)2 /2
pey
pey + (1 − p)e−y
Prof. C. Chen
Continuous Random Variables