Download Document

Chapter 3
General Random Variables
Peng-Hua Wang
Graduate Institute of Communication Engineering
National Taipei University
Chapter Contents
3.1 Continuous Random Variables and PDFs
3.2 Cumulative Distribution Functions
3.3 Normal Random Variables
3.4 Joint PDFs of Multiple Random Variables
3.5 Conditioning
3.6 The Continuous Bayes’ Rule
3.7 Summary and Discussion
Peng-Hua Wang, April 24, 2012
Probability, Chap 2 - p. 2/37
3.1 Continuous Random Variables
and PDFs
Peng-Hua Wang, April 24, 2012
Probability, Chap 2 - p. 3/37
Concepts
■
Let X be the arrival time of a bus. X is a continuous
random variable taking value between p and q.
■
The sample space Ω = [ p, q ]. For any point c ∈ Ω,
P( X = a ) = 0.
■
We can find P( x ≤ a ), P( a < X ≤ b ), or
P( a < X ≤ a + δ ). In general, we can find P( X ∈ B) for
B ⊂ Ω.
■
Let FX ( x ) = P( X ≤ x ). We know that
FX (−∞) = P( X ≤ −∞) = 0,
Peng-Hua Wang, April 24, 2012
FX ( ∞) = P( X ≤ ∞) = 1
Probability, Chap 2 - p. 4/37
Concepts
■
Let
FX ( x + δ ) − FX ( x )
dFX ( x )
f X ( x ) , lim
=
δ
dx
δ →0
■
We have
P( X ≤ x ) = FX ( x ) =
■
Z x
−∞
f X ( t) dt
Therefore, P( X ∈ B) can be evaluated in terms of
integral of f X ( x ). For example,
P( a < X ≤ b ) =
Peng-Hua Wang, April 24, 2012
Z b
a
f X ( x ) dx
Probability, Chap 2 - p. 5/37
PDF
■
f X ( x ) is called the probability density function (PDF) of
continuous random variable X.
■
FX ( x ) = P( X ≤ x ) is called the cumulative distribution
function (CDF).
◆ f X ( x) ≥ 0
R∞
◆
−∞ f X ( x ) dx = P(− ∞ < X ≤ ∞ ) = 1
◆ P( x < X < x + δ ) ≈ f X ( x ) · δ if δ is small.
Peng-Hua Wang, April 24, 2012
Probability, Chap 2 - p. 6/37
Example 3.1. Uniform distribution.
f X ( x) =
(
c,
a≤x≤b
0, otherwise.
Find c.
Peng-Hua Wang, April 24, 2012
Probability, Chap 2 - p. 7/37
Example 3.3. Uniform distribution.
c
f X ( x) = √ ,
x
0<x≤1
Find c.
■ A PDF can take arbitrarily large values.
Peng-Hua Wang, April 24, 2012
Probability, Chap 2 - p. 8/37
Expectation
■
■
■
The mean or expectation of a continuous random
R∞
variable X is defined by E[ X ] = −∞ x f X ( x ) dx.
R∞ k
k
The kth moment is E[ X ] = −∞ x f X ( x ) dx.
The variance of X is
Var( X ) = E[( X − E[ X ])2 ] = E[ X 2 ] − ( E[ X ])2
■
The mean of new RV Y = g( X ) is
E[ g( X )] =
Peng-Hua Wang, April 24, 2012
Z ∞
−∞
g( x ) f X ( x ) dx.
Probability, Chap 2 - p. 9/37
Expectation
■
The expectation is well-defined if
E[| X |] =
■
Z ∞
−∞
| x | f X ( x ) dx < ∞.
A not-well-defined random variable: Cauchy RV. Its PDF
is
c
f X ( x) =
, −∞ < x < ∞.
2
1+x
(Please find c). E[ X ] is not well-defined.
Peng-Hua Wang, April 24, 2012
Probability, Chap 2 - p. 10/37
Example 3.4. Uniformly RV
1
f X ( x) =
,
b−a
Find E[ X ] and Var( X ).
Peng-Hua Wang, April 24, 2012
a ≤ x ≤ b.
Probability, Chap 2 - p. 11/37
Exponential RV
f X ( x ) = ke−λx ,
Find k, E[ X ] and Var( X ).
Peng-Hua Wang, April 24, 2012
0 ≤ x < ∞.
Probability, Chap 2 - p. 12/37
3.2 Cumulative Distribution
Functions
Peng-Hua Wang, April 24, 2012
Probability, Chap 2 - p. 13/37
Cumulative Distribution Functions
■
The cumulative distribution function (CDF) of a rv X,
denoted by FX ( x ), is defined by

∑
if X is discrete,
k ≤ x p X ( k ),
FX ( x ) , P( X ≤ x ) = R x

− ∞ f X ( x ) dx, if X is continuous.
■
CDF is exactly probability. PDF is NOT probability.
■
Note the “ ≤ ” in the definition.
■
“Any random variable associated with a given
probability model has a CDF, regardless of whether it is
discrete or continuous.”
Peng-Hua Wang, April 24, 2012
Probability, Chap 2 - p. 14/37
Fig 3.6
CDFs of some discrete random variables
Peng-Hua Wang, April 24, 2012
Probability, Chap 2 - p. 15/37
Fig 3.7
CDFs of some continuous random variables
Peng-Hua Wang, April 24, 2012
Probability, Chap 2 - p. 16/37
Properties of a CDF
■
■
Definition: FX ( x ) , P( X ≤ x )
Monotonically nondecreasing: If x ≤ y, then
FX ( x ) ≤ FX ( y).
■
FX (−∞) = 0, FX (+∞) = 1
■
If X is discrete, FX ( x) is piecewise constant. If X is
continuous, FX ( x) is continuous.
■
If X is discrete,
p X ( k ) = FX ( k ) − FX ( k − 1)
■
If X is continuous,
d
f X ( x) =
FX ( x )
dx
Peng-Hua Wang, April 24, 2012
Probability, Chap 2 - p. 17/37
Example 3.6
■
Let X1 , X2 and X3 be 3 independent discrete random
variables with identical PMFs.
X = max { X1 , X2 , X3 }
Find PMF of X.
■
Let X1 , X2 and X3 be 3 independent continuous random
variables with identical PDFs.
X = max { X1 , X2 , X3 }
Find PDF of X.
Peng-Hua Wang, April 24, 2012
Probability, Chap 2 - p. 18/37
3.3 Normal Random Variables
Peng-Hua Wang, April 24, 2012
Probability, Chap 2 - p. 19/37
Definition
A continuous random variable X is said to be normal or
Gaussian if it has a PDF of the form
f X ( x ) = ce
−( x −µ)2 /(2σ2 )
, −∞ < x < ∞.
√1
2πσ
■
c=
■
E[ X ] = µ
■
Var( X ) = σ2
Peng-Hua Wang, April 24, 2012
Probability, Chap 2 - p. 20/37
Standard Normal
■
A standard normal rv Y is the normal rv with µ = 0 and
σ = 1. Its CDF is denoted by Φ( y)
Φ(y) =
■
■
Z y
−∞
1 −t2 /2
√ e
dt
2π
Φ(−y) = 1 − Φ( y) because The PDF of standard normal
is even.
For normal rv X with mean µ and variance σ2 , we know
that X = σY + µ. Thus,
x−µ
P( X ≤ x ) = P(σY + µ ≤ x ) = P Y ≤
σ
Peng-Hua Wang, April 24, 2012
=Φ
x−µ
σ
Probability, Chap 2 - p. 21/37
Example 3.7
The annual snowfall at a particular geographic location is
modeled as a normal random variable with a mean of
µ = 60 inches and a standard deviation of σ = 20. What is
the probability that this year’s snowfall will be at least 80
inches?
Y = 20X + 60
P(Y ≥ 80) =?
Peng-Hua Wang, April 24, 2012
Probability, Chap 2 - p. 22/37
Example 3.8
A binary message is transmitted as a signal s, which is
either −l or +1. The communication channel corrupts the
transmission with additive normal noise N with mean
µ = 0 and variance σ2 . The receiver concludes that the
signal −1 (or +1) was transmitted if the value received is
< 0 (or ≥ 0, respectively). What is the probability of
error?
Peng-Hua Wang, April 24, 2012
Probability, Chap 2 - p. 23/37
3.4 Joint PDFs Of Multiple Random
Variables
Peng-Hua Wang, April 24, 2012
Probability, Chap 2 - p. 24/37
Definitions
■
X and Y are two continuous random variables. Their
joint CDF is
FX,Y ( x, y) , P( X ≤ x, Y ≤ Y )
■
Joint PDF is
∂2
FX,Y ( x, y)
f X,Y ( x, y) ,
∂x∂y
Peng-Hua Wang, April 24, 2012
Probability, Chap 2 - p. 25/37
Properties
1. P(( X, Y ) ∈ B) =
ZZ
( x,y)∈ B
f X,Y ( x, y) dxdy
2. P( a < X ≤ b, c < Y ≤ d) =
3. FX,Y ( x, y) =
4.
Z ∞
Z ∞
Z x
x =−∞ y=−∞
Z y
s=−∞ t=−∞
Z b Z d
x = a y=c
f X,Y ( x, y) dydx
f X,Y ( s, t) dtds
f X,Y ( x, y) dydx = 1
5. P( x < X ≤ x + δ, y < Y ≤ y + delta ) ≈ f X,Y ( x, y) δ2
Peng-Hua Wang, April 24, 2012
Probability, Chap 2 - p. 26/37
Marginal PDF
FX ( x ) = P( X ≤ x ) = P( X ≤ x, −∞ < Y < ∞)
=
Z x
Z ∞
s=−∞ y=−∞
f X,Y ( s, y) dyds
dFX ( x )
⇒ f X ( x) =
=
dx
Z ∞
y=−∞
f X,Y ( x, y) dy
Similarly,
fY (y) =
Peng-Hua Wang, April 24, 2012
Z ∞
−∞
f X,Y ( x, y) dx
Probability, Chap 2 - p. 27/37
Example 3.9 Jointly Uniform PDF
f X,Y ( x, y) = c,
■
If a < x < b and c < y < d, find c.
■
If | x | + | y| ≤ r, find c.
p
If x2 + y2 ≤ r, find c.
■
Peng-Hua Wang, April 24, 2012
Probability, Chap 2 - p. 28/37
Expectation
E[ g( X, Y )] =
Z ∞
Z ∞
x =−∞ y=−∞
g( x, y) f X,Y ( x, y) dydx
■
E[ aX + bY + c] = aE[ X ] + bE[Y ] + c
■
You can easily extend the results of two random
variables to more joint random variables.
Peng-Hua Wang, April 24, 2012
Probability, Chap 2 - p. 29/37
3.5 Conditioning
Peng-Hua Wang, April 24, 2012
Probability, Chap 2 - p. 30/37
Conditioning an RV on an Event
■
Conditioning CDF and PDF
P({ X ≤ x } ∩ A)
FX | A ( x ) = P( X ≤ x | A) =
P( A )
d
FX | A ( x )
f X| A ( x) =
dx
■
Special case: A = { X ∈ B}
FX | X ∈ B ( x ) = P( X ≤ x | X ∈ B) =
R
t ≤ x,t ∈ B f X ( t ) dt
=
P( X ∈ B )
f X|X∈B( x) =
Peng-Hua Wang, April 24, 2012
P({ X ≤ x } ∩ { X ∈ B})
P( X ∈ B )
d
f X ( x)
FX | X ∈ B ( x ) =
,
dx
P( X ∈ B )
x ∈ B.
Probability, Chap 2 - p. 31/37
Example 3.13. The Exponential RV
The time Tuntil a new light bulb burns out is an
exponential rv with parameter λ. You turn the light on,
leaves the room, and when you returns, t time units later,
finds that the light bulb is still on, which corresponds to
the event A = { T > t}. Let X be the additional time until
the light bulb burns out. What is the conditional CDF of X,
given the event A?
Hint. P( X > x | A) = P( T > T + x | T > t) =?
Peng-Hua Wang, April 24, 2012
Probability, Chap 2 - p. 32/37
Example 3.14.
The metro train arrives at the station near your home every
quarter hour starting at 6:00 a.m. You walk into the station
every morning between 7:10 and 7:30 a.m., and your
arrival time is a uniform random variable over this
interval. What is the PDF of the time you have to wait for
the first train to arrive?
Hint. Let X be the time of your arrival. X is a uniform
random variable over the interval from 7:10 to 7:30. Let Y
be the waiting time. Let A and B be the events
A = {7 : 10 ≤ X ≤ 7 : 15} = {you board the 7:15 train},
B = {7 : 15 < X ≤ 7 : 30} = {you board the 7:30 train}.
f Y ( y ) = P( A ) f Y | A ( y ) + P ( B ) f Y | B ( y )
Peng-Hua Wang, April 24, 2012
Probability, Chap 2 - p. 33/37
Conditioning one RV on Another
■
Let X and Y be continuous random variables with joint
PDF f X,Y ( x, y). The conditional PDF of X given that
Y = y, is defined by
f X,Y ( x, y)
f X |Y ( x | y ) =
fY (y)
Peng-Hua Wang, April 24, 2012
Probability, Chap 2 - p. 34/37
Example 3.15.
f X,Y ( x, y) = c,
Find c, f Y ( y) and f X |Y ( x | y).
Peng-Hua Wang, April 24, 2012
x 2 + y2 ≤ r 2
Probability, Chap 2 - p. 35/37
Conditional Expectations
■
Definition.
Z ∞
E[ X | A] =
E [ X |Y = y ] =
■
x f X | A ( x ) dx
−∞
Z ∞
−∞
x f X |Y ( x | y) dx
Total expectation. Let A1 , A2 , . . . , An form a partition of
the sample space.
E[ X ] =
∑ P( A i ) E [ X | A i ]
i
E[ X ] =
Z ∞
Peng-Hua Wang, April 24, 2012
−∞
f Y ( y) E[ X |Y = y] dy = EY [ E[ X |Y = y]]
Probability, Chap 2 - p. 36/37
Independence
■
Two continuous random variables X and Y are
independent if
f X,Y ( x, y) = f X ( x ) f Y ( y)
■
Three continuous random variables X, Y and Y are
independent if
f X,Y,Z ( x, y, z ) = f X ( x ) f Y ( y) f Z ( z )
■
If X and Y are independent, we have
E[ XY ] = E[ X ] E[Y ]
Peng-Hua Wang, April 24, 2012
Probability, Chap 2 - p. 37/37