Download Use of moment generating functions

Use of moment generating
functions
Definition
Let X denote a random variable with probability
density function f(x) if continuous (probability mass
function p(x) if discrete)
Then
mX(t) = the moment generating function of X
 
 E etX
  tx
  e f  x  dx if X is continuous
  
  etx p  x 
if X is discrete
 x
The distribution of a random variable X is
described by either
1. The density function f(x) if X continuous (probability
mass function p(x) if X discrete), or
2. The cumulative distribution function F(x), or
3. The moment generating function mX(t)
Properties
1. mX(0) = 1
2. mXk   0   k th derivative of mX  t  at t  0.
 
 k  E X
k  E  X
3.
k

k
k

 x f  x  dx

k

  x p  x
mX  t   1  1t 
2
2!
t 
2
X continuous
X discrete
3
3!
t 
3

k
k!
t 
k
.
4. Let X be a random variable with moment
generating function mX(t). Let Y = bX + a
Then mY(t) = mbX + a(t)
= E(e [bX + a]t) = eatmX (bt)
5. Let X and Y be two independent random
variables with moment generating function
mX(t) and mY(t) .
Then mX+Y(t) = mX (t) mY (t)
6. Let X and Y be two random variables with
moment generating function mX(t) and mY(t)
and two distribution functions FX(x) and
FY(y) respectively.
Let mX (t) = mY (t) then FX(x) = FY(x).
This ensures that the distribution of a random
variable can be identified by its moment
generating function
M. G. F.’s - Continuous distributions
Name
Continuous
Uniform
Exponential
Gamma
2
d.f.
Normal
Moment generating
function MX(t)
ebt-eat
[b-a]t
  
for t < 


  t 

  
for t < 


  t 
 1 
1-2t


/2
for t < 1/2
t+(1/2)t22
e
M. G. F.’s - Discrete distributions
Name
Discrete
Uniform
Bernoulli
Binomial
Geometric
Negative
Binomial
Poisson
Moment
generating
function MX(t)
et etN-1
N et-1
q + pet
(q + pet)N
pet
1-qet
 pet  k


t
1-qe


(et-1)
e
Moment generating function of the
gamma distribution
 
mX  t   E etX 

tx
e
 f  x  dx

where

 
 1   x
x
e

f  x      

0

x0
x0

    e f  x  dx
mX  t   E e
tX
tx




 1   x
 e
x e dx
  
0
 

 1   t  x

x e
dx

   0
tx

using
or
ba a 1 bx
0   a  x e dx  1

 a
a 1  bx
0 x e dx  ba
then



 1   t  x
mX  t  
x e
dx

   0
    


      t 

  


  t 
t
Moment generating function of the
Standard Normal distribution

 
mX  t   E etX 
tx
e
 f  x  dx

where
1
f  x 
e
2
x2

2
thus
mX  t  

e

tx
1
e
2
x2

2

dx 


1
e
2
x2
  tx
2
dx


We will use
0
mX  t  









e
1
e
2
x2
  tx
2
x2  2tx

2


1
e
2 b
t2
2
2
x a 


2b2
dx  1
dx
1
e
dx
2
2
x t 

x 2  2 tx  t 2 t 2
t2 
1  2
1  2
2
2
e
e dx  e 
e
dx
2
2

Note:
2
3
4
x
x
x
ex  1  x    
2! 3! 4!
2
3
t  t 
   
t2
2
2 2
t

2
mX  t   e  1  


2
2!
3!
2
2
4
6
t
t
t
 1  2  3 
2 2 2! 2 3!
Also
mX  t   1  1t 
2
2!
t 
2
2
2m
t
 m 
2 m!
3
3!
t 
3
Note:
2
3
4
x
x
x
ex  1  x    
2! 3! 4!
2
3
t  t 
   
t2
2
2 2
t

2
mX  t   e  1  


2
2!
3!
2
2
Also
4
2
6
2m
t
t
t
t
 1  2  3   m 
2 2 2! 2 3!
2 m!
 2 2 3 3
mX  t   1  1t  t  t 
2!  3!
k  k th moment 


x k f  x  dx
Equating coefficients of tk, we get
k  0 if k is odd and
2 m
1
for k  2m then m 
2 m !  2m  !
hence 1  0, 2  1, 3  0, 4  3
Using of moment generating
functions to find the distribution of
functions of Random Variables
Example
Suppose that X has a normal distribution with
mean  and standard deviation .
Find the distribution of Y = aX + b
Solution:
 2t 2
t 
mX  t   e 2
2
 2 at
  at  
bt
maX b  t   e mX  at   e e
bt
 a   b t 
e
 
2
 2 a 2t 2
2
= the moment generating function of the normal
distribution with mean a + b and variance a22.
Thus Y = aX + b has a normal distribution with
mean a + b and variance a22.
Special Case: the z transformation
Z
X 

1
  
   X 
  aX  b
 
  
1
  
Z  a  b      
0
 
  
2
1 2
2
2 2
Z  a      1
 
Thus Z has a standard normal distribution .
Example
Suppose that X and Y are independent each having a
normal distribution with means X and Y , standard
deviations X and Y
Find the distribution of S = X + Y
Solution:
mX  t   e
mY  t   e
X t 
Y t 
 X2 t 2
2
 Y2 t 2
2
Now
mX Y  t   mX  t  mY  t   e
X t 
 X2 t 2
2
e
Y t 
 Y2 t 2
2
or
m X Y  t   e
 X X


t 
2
X

 Y2 t 2
2
= the moment generating function of the
normal distribution with mean X + Y and
2
2
variance  X   Y
Thus Y = X + Y has a normal distribution
2
2
with mean X + Y and variance  X   Y
Example
Suppose that X and Y are independent each having a
normal distribution with means X and Y , standard
deviations X and Y
Find the distribution of L = aX + bY
Solution:
mX  t   e
X t 
 X2 t 2
mY  t   e
2
Y t 
 Y2 t 2
2
Now
maX bY t   maX t  mbY t   mX  at  mY bt 
e
 X  at  
 X2  at 
2
2
e
Y  bt  
 Y2  bt 
2
2
or
a

t 
2
maX bY  t   e
 a  X  b X
2
X

 b2 Y2 t 2
2
= the moment generating function of the
normal distribution with mean aX + bY
2 2
2 2
and variance a  X  b  Y
Thus Y = aX + bY has a normal
distribution with mean aX + BY and
2 2
2 2
variance a  X  b  Y
Special Case:
a = +1 and b = -1.
Thus Y = X - Y has a normal distribution
with mean X - Y and variance
 1
2
 X   1  Y   X   Y
2
2
2
2
2
Example (Extension to n independent RV’s)
Suppose that X1, X2, …, Xn are independent each having a
normal distribution with means i, standard deviations i
(for i = 1, 2, … , n)
Find the distribution of L = a1X1 + a1X2 + …+ anXn
Solution:
mX i  t   e
Now
ma1 X1 
i t 
 i2t 2
(for i = 1, 2, … , n)
2
 an X n
t   ma X t 
1 1
 mX1  a1t 
e
1  a1t  
man X n t 
mX n  ant 
12  a1t 
2
2
e
n  an t  
 n2  an t 
2
2
or
ma1 X1 
 an X n
t   e
 a11 ... an n
a

t 
2 2
2 2

...

a
1 1
n n
t
2
2
= the moment generating function of the
normal distribution with mean a11  ...  an n
2 2
2 2
and variance a1  1  ...  an  n
Thus Y = a1X1 + … + anXn has a normal
distribution with mean a11 + …+ ann
2 2
2 2
and variance a1  1  ...  an  n
Special case:
a1  a2 
1  2 
 12   12 
1
 an 
n
 n  
  12   2
In this case X1, X2, …, Xn is a sample from a
normal distribution with mean , and standard
deviations ,and
1
L   X1  X 2   X n 
n
 X  the sample mean
Thus
Y  x  a1 x1  ...  an xn
 n
 1
 n x
x1  ...  1
n
has a normal distribution with mean
 x  a11  ...  an n
 1   ...  1   
n
n
and variance
 
 
 x2  a12 12  ...  an2 n2
2
1
1
1

  2
  2
  2
     ...      n    
n
n
n
n
2
2
2
Summary
If x1, x2, …, xn is a sample from a normal
distribution with mean , and standard
deviations ,then x  the sample mean
has a normal distribution with mean
x  
and variance
 
2
x

2
n
 

 standard deviation  x 

n

Sampling distribution
of x
0.4
0.3
Population
0.2
0.1
0
20
30
40
50
60
The Central Limit theorem
If x1, x2, …, xn is a sample from a distribution
with mean , and standard deviations ,then
if n is large x  the sample mean
has a normal distribution with mean
x  
and variance
 
2
x
 
2
 
standard deviation  x 


n 
n
Proof: (use moment generating functions)
We will use the following fact:
Let
m1(t), m2(t), …
denote a sequence of moment generating functions
corresponding to the sequence of distribution
functions:
F1(x) , F2(x), …
Let m(t) be a moment generating function
corresponding to the distribution function F(x) then
if
lim mi  t   m  t  for all t in an interval about 0.
i 
then lim Fi  x   F  x  for all x.
i 
Let x1, x2, … denote a sequence of independent
random variables coming from a distribution with
moment generating function m(t) and distribution
function F(x).
Let Sn = x1 + x2 + … + xn then
mSn t   mx1  x2 
=  m  t  
 xn
t  =mx t  mx t 
1
2
mxn t 
n
x1  x2 
now x 
n
 xn
Sn

n
n
 t    t 
or mx  t   m 1   t   mSn    m   
  Sn
 n    n 
n
Let z 
x 


n
then mz  t   e

n

n

t
x
n

 nt  
mx 
  e
  
n

t
  nt  
m 
 
   n  
  t 
and ln mz  t   
t  n ln m 


   n 
n
n
t
t2
Let u 
or n 
and n  2 2
u
 u
 n
t
  t 
Then ln mz  t   
t  n ln m 


   n 
t 2
t2
  2  2 2 ln  m  u 
 u  u
n
t 2 ln m  u   u
 2

u2



Now lim ln  mz  t    lim ln  mz  t  
n 
u 0
t2



ln  m  u   u
2 u 0
t2

lim
lim
2 u 0

u2
m  u 

m u 
2u
using L'Hopital's rule
m  u  m  u    m  u 

t
2

lim
2 u 0
 m  u 
2
2
2
using L'Hopital's ru
m  u  m  u    m  u 

t
 m  u 
2
2

lim
2 u 0
t m  0   m  0 
 2

2
2
 
2
2
using L'Hopital's rule again
2
t E x   E  xi 
t2
 2


2
2
2
2
i

2

2
t
thus lim ln  mz  t  
and lim  mz  t    e
n
n
2
t2
2
Now m  t   e
t2
2
Is the moment generating function of the standard
normal distribution
Thus the limiting distribution of z is the standard
normal distribution
i.e. lim Fz  x  
n
x


1
e
2
u2

2
du
Q.E.D.