Download Continuous Random Variables, Moments and Moment Generating

Continuous Random Variables,
Moments and Moment
Generating Function
Continuous random variable
I
Definition: A random variable X is continuous if the CDF
FX (x) = P(X ≤ x) is a continuous function of x.
I
Example 1: Weight of a new born baby;
I
Example 2: Waiting time in a bus stop.
Density function
I
Definition: The probability density function fX (x) of a
continuous random variable X is the function that satisfies
Rx
FX (x) = −∞ fX (t)dt.
I
Remark: There exist continuous random variables which
do not have densities. We will only discuss the case where
the density exists.
CDF and density function
I
If X has a density fX (x), then FX (x) =
Rx
−∞ fX (u)du.
If
FX (x) is differentiable, then
d
FX (x)|x=x0 = FX0 (x0 )
dx
but FX0 (x0 ) is not necessary to be the same as fX (x0 ).
I
If fX (x) is continuous at point x0 , then
d
FX (x)|x=x0 = FX0 (x0 ) = fX (x0 ).
dx
Example

 2x
Define FX (x) =
 1x +
2
0≤x <
1
2



2 0 ≤ x < 13


1
1
fX (x) =
2
3 ≤x ≤1



 0 otherwise
(a) FX (x) =
Rx
−∞ fX (t)dt
=
1
3
1
3
≤x ≤1


2 0 ≤ x < 31 , x 6=





 1 x=1
6
∗
fX (x) =

1
1


2
3 ≤x ≤1



 0 otherwise
Rx
∗
−∞ fX (t)dt.
(b)
dFX (x)
dx |x= 13
does not exist but fX ( 13 ) = 12 .
(c)
dFX (x)
dx |x= 16
= fX ( 61 ) 6= fX∗ ( 16 ).
1
6
Density is not probability
I
Note that fX (t) is not probability. Actually, P(X = t) = 0 for
any t if X is a continuous random variable. Because
{X = t} ⊂ {t − < X ≤ t} for any P(X = t) ≤ P(t − < X ≤ t) = FX (t) − FX (t − ).
Hence 0 ≤ P(X = t) ≤ lim→0 [FX (t) − FX (t − )] = 0 by
continuity of FX .
I
Any meaningful statement about probability must consider
X lying in some interval. Probability is interpreted as the
area under the density function.
Example: Logistic distribution
A random variable X with logistic distribution if FX (x) =
1
.
1+e−x
Then
fX (x) =
dFX (x)
e−x
=
dx
(1 + e−x )2
P(a < X < b) = FX (b) − FX (a)
Z b
Z
=
fX (x)dx −
−∞
a
Z
fX (x)dx =
−∞
If ∆x is small, P(a ≤ x ≤ a + ∆x) ≈ fX (a)∆x.
b
fX (x)dx.
a
Quantile and median
I
Definition: Let X be a random variable with CDF FX (x).
For any 0 < α < 1, an quantile of X is any number xα
satisfying
FX (xα ) ≥ α
and FX (xα− ) ≤ α.
The median is x0.5 .
I
Quantiles defined above may not be unique. To make it
unique, we usually define the quantile as
xα = inf{x : FX (x) ≥ α}.
Example
Let X have CDF
F (u) =


0







u 2 /2


3
4




3


4 +



 1
u−2
4
u<0
0≤u<1
1≤x <2
2≤u<3
u ≥ 3.
Where are 0.6, 0.65, 0,75 quantiles and where is the median?
Expected value
I
Definition: Let X be a continuous random variable with
density f (x). Then the expected value of a random variable
g(X ) is
Z
E(g(X )) =
provided that
I
R
g(x)f (x)dx
|g(x)|f (x)dx exists.
Linearity:
E(ag1 (X ) + bg2 (X ) + c) = aE(g1 (X )) + bE(g2 (X )) + c.
Examples
I
Example 1: If X has exponential(λ), i.e.,
fX (x) =
x
1
exp(− ) x ≥ 0 and λ > 0.
λ
λ
What is the expectation of X ?
I
Example 2: If X has Cauchy distribution, the density of X is
fX (x) =
1 1
π 1 + x2
What is the expectation of X ?
− ∞ < x < ∞.
Mixture of continuous and discrete random variables
I
A random variable X could take continuous and discrete
values.
P(X ∈ A) = α
X
Z
fd (x) + (1 − α)
fc (x)dx
A
x∈A
for any Borel set A and 0 < α < 1, where fd (x) is a pmf
and fc (x) is a density.
I
The expectation of X is
E(X ) = α
X
x
Z
xfd (x) + (1 − α)
xfc (x)dx.
Example: Jelly Donut Problem
Suppose that the Jelly Donut Company want to decide how
many jelly donuts to bake every day. Package sells for s dollar
and cost c dollars to make. The demand D is a continuous
random variable with density f and CDF F . To maximize the
profit, how many packages the company should make?
Variance
I
For any random variable X , the variance is
Var(X ) = E[(X − E(X ))2 ].
I
Example: If X has exponential(λ), what is the variance of
X?
Moments
Definition: For each n, the nth moment of X is µn = E(X n ) and
the nth central moment of X is κn = E(X − µ)n where
µ = E(X ).
The moments can be used to measure some aspects of a
distribution. For example, we can measure degree of
asymmetric of a distribution by coefficient of skewness:
γ1 =
κ3
σ3
and for symmetric densities (pmfs), we can measure
peakedness by coefficient of kurtosis γ2 =
σ 2 = Var(X ).
κ4
σ4
− 3, where
Symmetric
I
A random variable is said to symmetric about 0 if X and
−X have the same distribution.
I
If X and −X have the same distribution,
F (u) + F (−u) = 1 + P(X = −u).
I
If X is continuous and fX (x) = fX (−x) for all x except
countable many x, then X is symmetric about 0. If X is
discrete, we require that fX (x) = fX (−x) for all x.
Left skew and right skew
Examples: Skewness
What are the coefficients of skewness for exponential(λ) and
Binomial(n, p)?
0.4
0.3
p=0.1
p=0.8
0.0
0.1
0.1
0.2
0.3
probability
0.4
λ=2
λ=5
0.0
density
Binomial Distribution(n=10)
0.2
0.5
Exponential Distribution
0
2
4
6
x
8
10
0
2
4
6
x
8
10
Kurtosis
Kurtosis γ2 =
κ4
σ4
− 3 is compared with the kurtosis of the
standard normal distribution, which has kurtosis 0.
Moments might not fully determine a distribution
I
Moments reflects some aspects of a distribution, but it can
not determine a distribution. Two totally different
distributions could have all the moments to be the same.
I
Example: Consider two random variables X and Y having
densities
fX (x) = √
1
2πx
exp(−(log x)2 /2)
0<x <∞
fY (y ) = fX (y )(1 + sin(2π log(y))) 0 < y < ∞.
Example continuation
0.6
0.4
0.2
0.0
Densities
0.8
1.0
1.2
fy(y)
fx(x)
0
2
4
6
x
8
10
Moment generating function
I
Let X be a random variable with CDF FX . The moment
generating function (MGF) of X , denoted by MX (t) is
MX (t) = E(etX ) provided that the expectation exists for t in
some neighborhood of 0. That is for all −h < t < h, E(etX )
exists.
I
For continuous random variables with density function
R
fX (x), MX (t) = etx fX (x)dx.
I
For discrete random variables with probability mass
P
function pX (x), MX (t) = x etx pX (x).
Properties
I
For any constants a and b, the MGF of the random variable
aX + b is given by MaX +b (t) = ebt MX (at).
I
For independent and identically distributed random
variables X , X1 , · · · , Xn . Let Sn = X1 + X2 + X3 + · · · + Xn .
Then MSn (t) = MXn (t), where MX (t) is the MGF of X .
Calculate moments using MGF
If X has MGF MX (t), then
E(X n ) =
dn
MX (t)|t=0 .
dt n
That is, the n-th moment is equal to the n-th derivatives of
MX (t) evaluated at t = 0.
Examples
I
(Discrete case) If X has Binomial(n, p), what is the
moment generating function of X ?
I
(Continuous case) If X has Exponential(λ), what is the
moment generating function of X ?
MGF and CDF
I
Let FX (x) and FY (y ) be two CDFs. If the moment
generating functions exist and MX (t) = MY (t) for all t in
some neighborhood of 0, then FX (u) = FY (u) for all u.
I
Suppose {Xn : n = 1, 2, · · · } is a sequence of random
variables, each with MGF MXn . Further,
lim MXn (t) = MX (t)
n→∞
for all t in a neighborhood of 0
and MX (t) is MGF of X . Then there exist a unique CDF
FX (x) whose moments are determined by MX (t) and for all
x where FX (x) is continuous, we have
lim FXn (x) = FX (x).
n→∞