Download 4.4 Moments

STAT 421 Lecture Notes
4.4
105
Moments
Let X be a random variable and k denote a positive integer. The kth moment of X is defined
to be E(X k ). The kth moment need not exist, though the kth and lower moments exist if
E(|X|k ) exists.
Theorem 4.4.1. If E(|X k |) < ∞ for some positive k, then E(|X j |) < ∞ for all positive
integers j < k.
A sketch of the proof for the case of X with a continuous distribution follows. Suppose
that E(|X k |) < ∞ for some positive k and j is a positive integer less than k. Then,
∫ ∞
j
E(|X |) =
|xj |f (x) dx
−∞
∫ 1
∫
j
|x |f (x) dx +
|xj |f (x) dx
=
−1
1
∫
≤
{x||x|>1}
∫
f (x) dx +
{x||x|>1}
k
−1
|xk |f (x) dx
≤ Pr(|X| ≤ 1) + E(|X |)
< ∞.
Central Moments Suppose that X is a random variable with E(X) = µ. For every positive
integer k, the kth central moment is defined to be
E[(X − µ)k ].
For example, the second central moment is the variance E[(X − µ)2 ].
Suppose that the distribution of X is symmetric about µ and E[(X − µ)k ] exists. Then, if
k is odd, E[(X − µ)k ] = 0 because
∫ 0
∫ ∞
k
k
E[(X − µ) ] =
(x − µ) f (x) dx +
(x − µ)k f (x) dx
−∞
0
∫ ∞
∫ ∞
k
= −
(x − µ) f (x) dx +
(x − µ)k f (x) dx = 0.
0
0
Example 4.4.1. Suppose that X is continuous with p.d.f.
f (x) = ce−(x−3)
2 /2
, x ∈ R.
The p.d.f. is symmetric about 3, and so the median and mean of X is µ = 3. It can be
shown that for every positive k,
∫ ∞
2
k
E(|X |) =
|xk |ce−(x−3) /2 dx < ∞.
−∞
STAT 421 Lecture Notes
106
By symmetry of f , E[(X − µ)k ] = 0 when k is an odd positive integer. For the even central
moments, a recursive formula can be developed. Let 2n = k and y = x − µ. Then,
∫ ∞
2
k
E[(X − µ) ] =
y 2n ce−y /2 dx.
−∞
Using integration by parts, set
u = y 2n−1 ⇒ dy = (2n − 1)y 2n−2
dv = cye−y
2 /2
dy ⇒ v = −ce−y
2 /2
.
Then,
E[(X − µ) ] = −y
k
2n−1
∞
−y 2 /2 ce
−∞
∫
∞
+
(2n − 1)y 2n−2 ce−y
−∞
2n−2
= 0 + (2n − 1)E[(X − µ)
2 /2
dy
]
= (2n − 1)E[(X − µ)2n−2 ].
Now, E[(X − µ)0 ] = E(1) = 1, E[(X − µ)2 ] = (2 − 1) × 1 = 1, E[(X − µ)4 ] = (4 − 1) × 1 = 3,
E[(X − µ)6 ] = (6 − 1) × 3 = 15, and so on.
Skewness is a measure of lack of symmetric based on the third central moment. Since the
third central moment is 0 if the distribution is symmetric, the difference between E[(X −µ)3 ]
and 0 reflects lack of symmetry. For interpretability, E[(X − µ)3 ] is scaled by the second
central moment, and the measure of skewness is
E[(X − µ)3 ]
E[(X − µ)3 ]
=
.
E[(X − µ)2 ]
σ2
The figure below and left shows the probability function for the binomial distribution as a
function of p. Values of p are drawn from the set {.1, .15, .2, . . . , .85, .9}. Skewness is plotted
against p in the figure below and to the right.
107
−0.5
0.0
skewness
0.2
0.1
0.0
Pr(X=x)
0.3
0.5
0.4
STAT 421 Lecture Notes
0
5
10
15
0.2
20
0.4
0.6
0.8
p
x
Moment Generating Functions
Definition 4.4.2. Let X be a random variable. The moment generating function (m.g.f.)
of X is defined to be
ψ(t) = E(etX ), t ∈ R.
Any two random variables with the same distribution will have the same moment generating
function. The m.g.f. may or may not exist (though if X is bounded, then ψ(t) will exist).
Note that ψ(0) = E(1) = 1 always exists, but is uninteresting.
Theorem 4.4.2. Let X be a random variable whose m.g.f. ψ(t) is finite for all values of
t in an open interval containing t = 0. Then, for each positive integer n, E(X n ), the nth
moment of X, is finite and equals the nth derivative of ψ(t) evaluated at t = 0. The nth
derivative evaluated at 0 is denoted by ψ (n) (0).
The proof of the theorem depends strongly on the following result:
( n tX )
dn E(etX )
d e
(n)
ψ =
= E
.
n
dt
dtn
Accepting the truth of this statement leads to
)
detX ψ (0) = E
dt ( 0·X )t=0
= E Xe
(
(1)
= E(X).
STAT 421 Lecture Notes
108
Example 4.4.3. Suppose that X has the following p.d.f.
f (x) = e−x I{r|r>0} (x).
The m.g.f. of X is
( )
ψ(t) = E etX
∫ ∞
=
etx e−x dx
∫0 ∞
=
e(t−1)x dx.
0
This integral is finite provided that t ∈ (−∞, 1), and for t ∈ (−∞, 1),
ψ(t) = (1 − t)−1 .
To determine the first and second moments, and the variance of X, we compute
ψ (1) (t) = (1 − t)−2 ⇒ ψ (1) (0) = E(X) = 1
ψ (2) (t) = 2(1 − t)−3 ⇒ ψ (2) (0) = E(X 2 ) = 2
Var(X) = ψ (2) (0) − [ψ (1) (0)]2 = 1.
Question 1 : Compute E(X) where X has the following p.f.:
f (x) =
λx e−λ
I{0,1,2,...} (x).
x!
Properties of moment generating functions Let X be a random variable with mg.f. ψ1 (t)
and Y = aX + b where a, b are constants. Let ψ2 (t) denote the m.g.f. of Y . Then, for every
value of t for which ψ1 (at) is finite,
ψ2 (t) = ebt ψ1 (at).
The proof proceeds as follows:
]
[
ψ2 (t) = E e(aX+b)t
[
]
= E ebt eaXt
[
]
= ebt E eatX
= ebt ψ1 (at).
Example 4.4.4. Suppose that X has the following p.d.f.
f (x) = e−x I{r|r>0} (x),
STAT 421 Lecture Notes
109
and so the m.g.f. of X is ψ1 (t) = (1 − t)−1 . If Y = 3 − 2X, then the m.g.f. of Y will be
finite for t > −1/2 and will have the value
ψ2 (t) = e3t ψ1 (−2t) =
e3t
.
1 + 2t
Sums of independent random variables (but not necessarily identically distributed random
variables) are important statistics, and computing the m.g.f. of the sum is a convenient
method for determining the distribution, or at least the moments, of the sum. The following
theorem is key:
Theorem 4.4.4. Suppose that X1 , . . . , Xn are independent random variables with m.g.f.s
∑
ψ1 , . . . , ψn . Let Y = ni=1 Xi and let ψ denote the m.g.f. of Y . Then, for every value of t
for which ψ1 (t), . . . , ψn (t) are finite,
ψ(t) =
n
∏
ψi (t).
i=1
The proof proceeds as follows:
)
( ∑n
ψ(t) = E et i=1 Xi
( n
)
∏
= E
etXi
i=1
=
n
∏
(
)
E etXi (by independence of X1 , . . . , Xn )
i=1
=
n
∏
ψi (t).
i=1
The moment generating function for the binomial random variable
Suppose that X ∼ Binom(n, p) and that Xi ∼ Binom(1, p), i = 1, . . . , n are independent.
∑
Then, X =
Xi . Furthermore, the m.g.f. of Xi is
∑
ψi (t) =
etx px (1 − p)1−x
x∈{0,1}
= 1 − p + pet .
By Theorem 4.4.4., the m.g.f. of X is
ψ(t) =
n
∏
ψi (t)
i=1
= (1 − p + pet )n .
Theorem 4.4.5. If the m.g.f.s of random variables X and Y are finite and identical in an
open interval containing t = 0, then the probability distributions of X and Y are identical.
STAT 421 Lecture Notes
110
The proof is beyond the scope of the book.
Theorem 4.4.6. Suppose that X ∼ Binom(n1 , p) and Y ∼ Binom(n2 , p) are independent
(note that p is the same for both random variables). Then, X + Y ∼ Binom(n1 + n2 , p). To
prove the claim, let ψ1 (t) and ψ2 (t) denote the m.g.f.s of X and Y , and let ψ(t) denote the
m.g.f. of X + Y . By Theorem 4.4.4.,
ψ(t) = ψ1 (t)ψ2 (t)
= (1 − p + pet )n1 (1 − p + pet )n2
= (1 − p + pet )n1 +n2 .
The m.g.f. of Y , ψ(t), is the m.g.f. of a Binom(n1 + n2 , p) random variable. By Theorem
4.4.5., the distribution of Y is that of a Binom(n1 + n2 , p) random variable.