Download The Moment Generating Function

The Moment Generating Function
Gary Schurman, MBE, CFA
October, 2011
The moment generating function (Mx (t)) of the random variable x as a function of the variable t is...
Mx (t) = E etx
(1)
Note that etx can be approximated around zero using a Taylor Series Expansion. When x changes from zero to a
non-zero value the approximate value of Equation (1) at the new value of x via a Taylor Series Expansion is...
1 δ 2 M0 (t) 2 1 δ 3 M0 (t) 3
1 δ n M0 (t) n
δM0 (t)
δx +
δx
+
δx
+
...
+
δx
Mx (t) = E M0 (t) +
δx
2 δx2
6 δx3
n! δxn
1
= E et0 + tet0 (x − 0) + t2 et0 (x − 0)2 + t3 et0 (x − 0)3 + ... + tn et0 (x − 0)n
2
1
1
1
= E 1 + tx + t2 x2 + t3 x3 + ... + tn xn
2
6
n!
1
1
1
= 1 + t E x + t2 E x2 + t3 E x3 + ... + tn E xn
(2)
2
6
n!
To calculate the first moment of the distribution (i.e. M1, which is the expected value of x) we take the first
derivative of Equation (2) with respect to t and evaluate it at t = 0 because...
M 1 = lim
t→0
δMx (t)
δt
1 2
1
2
3
n−1
= lim E x + t E x + t E x + ... +
t
E xn
t→0
2
(n − 1)!
=E x
(3)
To calculate the second moment of the distribution (i.e. M2, which is the expected value of the square of x) we
take the second derivative of Equation (2) with respect to t and evaluate it at t = 0 because...
δ 2 Mx (t)
2
t→0
δt
M 2 = lim
1
tn−2 E xn
= lim E x2 + t E x3 + ... +
t→0
(n − 2)!
= E x2
(4)
And so on...
The Normal Distribution
The normal distribution is a distribution of continuous random variables. If the random variable x is continuous
the moment generating function of the random variable x where f (x) is the probability density function can be
defined as...
Z∞
Mx (t) =
etx f (x) δx
(5)
−∞
1
The equation for the probability density function of the normal distribution with mean m and variance v is...
f (x) = √
2
1
1
e− 2v (x−m)
2πv
(6)
If we combine Equations (5) and (6) from above we can write the equation for the moment generating function of
the normal distribution as...
Z∞
2
1
1
etx √
Mx (t) =
e− 2v (x−m) δx
(7)
2πv
−∞
To solve Equation (7) we will first define a new random variable z to be the normalized random variable x.
equation for the random variable z is...
x−m
z= √
v
The random variable x as a function of the new random variable z, which was defined in Equation (8) above,
√
x=m+z v
The
(8)
is...
(9)
The derivative of Equation (9) with respect to the random variable z is...
δx √
= v
(10)
δz
To solve Equation (7) we will next replace the variable x with the variable z as defined in Equations (8), (9) and
(10) above. Equation (7) rewritten to be a function of the variable z is...
Z∞
√
1
− 12 z 2 δx t(m+z v)
√
Mx (t) =
e
e
δz δz
2πv
−∞
Z∞
=
−∞
Z∞
=
−∞
√
√ 1
− 21 z 2 +z v t mt √
e
e v δz
2πv
√
1 2
1
√ e− 2 z +z v t emt δz
2π
Noting that after completing the square...
2
√
√
√
1
1 2
1
1
2
− z − v t = − z − 2z v t + vt = − z 2 + z v t − vt2
2
2
2
2
(11)
(12)
We can use Equation (12) to rewrite Equation (11), which becomes...
Z∞
Mx (t) =
−∞
√
2
2
1
1
1
√ e− 2 (z− v t) emt e 2 vt δz
2π
=e
mt+ 21 vt2
=e
mt+ 12 vt2
Z∞
−∞
√
2
1
1
√ e− 2 (z− v t) δz
2π
(13)
The first derivative of Equation (13) with respect to the variable t is...
2
1
δMx (t)
= (m + vt) emt+ 2 vt
δt
The first moment of the distribution will be the limit of Equation (14) as t goes to zero
δMx (t)
E x = lim
t→0
δt
1
= (m + v(0)) em(0)+ 2 v(0)
=m
(14)
2
(15)
2
The second derivative of Equation (13) with respect to the variable t is...
2
2
1
1
δ 2 Mx (t)
= v emt+ 2 vt + (m + vt)2 emt+ 2 vt
2
δt
(16)
The second moment of the distribution will be the limit of Equation (16) as t goes to zero
δ 2 Mx (t)
E x2 = lim
t→0
δt2
1
2
1
= v em(0)+ 2 v(0) + (m + v(0))2 em(0)+ 2 v(0)
2
= v + m2
(17)
The Binomial Distribution
The binomial distribution is a distribution of discrete random variables. If the random variable k, which is the
number of successes out of n trials, is discrete then the moment generating function of the random variable k where
P (k) is the probability mass function can be defined as...
Mk (t) =
n
X
etk P (k)
(18)
k=0
The equation for the probability mass function of the binomial distribution with the probability of success equal to
p and the probability of failure equal to 1 − p is...
P (k) =
n!
pk (1 − p)n−k
k!(n − k)!
(19)
If we combine Equations (18) and (19) from above we can write the equation for the moment generating function
of the binomial distribution as...
Mk (t) =
=
n
X
k=0
n
X
k=0
n!
pk (1 − p)n−k etk
k!(n − k)!
n!
(pet )k (1 − p)n−k
k!(n − k)!
(20)
Pascal’s rule says that...
n
X
k=0
n!
ak bn−k = (a + b)n
k!(n − k)!
(21)
If we define a = pet and b = 1 − p then Equation (20) becomes...
Mk (t) = (pet + 1 − p)n
(22)
The first derivative of Equation (22) with respect to the variable t is...
δMk (t)
= n(pet + 1 − p)n−1 pet
δt
(23)
The first moment of the distribution will be the limit of Equation (23) as t goes to zero
δMk (t)
E k = lim
t→0
δt
= n(pe0 + 1 − p)n−1 pe0
= np
(24)
The second derivative of Equation (23) with respect to the variable t is...
δ 2 Mk (t)
= n(n − 1)(pet + 1 − p)n−2 pet pet + n(pet + 1 − p)n−1 pet
δt2
3
(25)
The second moment of the distribution will be the limit of Equation (23) as t goes to zero
δ 2 Mk (t)
E k 2 = lim
t→0
δt2
= n(n − 1)(pe0 + 1 − p)n−2 pe0 pe0 + n(pe0 + 1 − p)n−1 pe0
= n(n − 1)p2 + np
= np − np2 + n2 p2
= np(1 − p) + n2 p2
(26)
4