Download Document

Use of moment generating
functions
1. Using the moment generating functions of
X, Y, Z, …determine the moment
generating function of W = h(X, Y, Z, …).
2. Identify the distribution of W from its
moment generating function
This procedure works well for sums, linear
combinations etc.
Therorem
Let X and Y denote a independent random variables
each having a gamma distribution with parameters
(l,a1) and (l,a2). Then W = X + Y has a gamma
distribution with parameters (l, a1 + a2).
Proof:
a1
 l 
mX  t   

 l t 
a2
 l 
and mY  t   

 l t 
Therefore mX Y  t   mX  t  mY t 
a1
a2
a1 a 2
 l   l 
 l 

 
 

 l t   l t 
 l t 
Recognizing that this is the moment generating
function of the gamma distribution with parameters
(l, a1 + a2) we conclude that W = X + Y has a
gamma distribution with parameters (l, a1 + a2).
Therorem (extension to n RV’s)
Let x1, x2, … , xn denote n independent random
variables each having a gamma distribution with
parameters (l,ai), i = 1, 2, …, n.
Then W = x1 + x2 + … + xn has a gamma distribution
with parameters (l, a1 + a2 +… + an).
Proof:
ai
 l 
mxi  t   

 l t 
i  1, 2..., n
Therefore
mx1  x2 ... xn  t   mx1 t  mx2 t  ...mxn t 
a1
a2
an
 l   l 
 l 

...
 



 l t   l t 
 l t 
a1 a 2 ...a n
 l 


 l t 
Recognizing that this is the moment generating
function of the gamma distribution with parameters
(l, a1 + a2 +…+ an) we conclude that
W = x1 + x2 + … + xn has a gamma distribution with
parameters (l, a1 + a2 +…+ an).
Therorem
Suppose that x is a random variable having a
gamma distribution with parameters (l,a).
Then W = ax has a gamma distribution with
parameters (l/a, a).
a
Proof:
 l 
mx  t   

 l t 
a
a
l 

 l  
a 
then max  t   mx  at   


 l  at   l  t 
 a 
Special Cases
1. Let X and Y be independent random variables
having a c2 distribution with n1 and n2 degrees of
freedom respectively then X + Y has a c2
distribution with degrees of freedom n1 + n2.
2. Let x1, x2,…, xn, be independent random variables
having a c2 distribution with n1 , n2 ,…, nn degrees
of freedom respectively then x1+ x2 +…+ xn has a
c2 distribution with degrees of freedom n1 +…+ nn.
Both of these properties follow from the fact that a
c2 random variable with n degrees of freedom is a
G random variable with l = ½ and a = n/2.
Recall
If z has a Standard Normal distribution then z2 has a
c2 distribution with 1 degree of freedom.
Thus if z1, z2,…, zn are independent random variables
each having Standard Normal distribution then
U  z12  z22  ...  zn2
has a c2 distribution with n degrees of freedom.
Therorem
Suppose that U1 and U2 are independent random
variables and that U = U1 + U2 Suppose that U1
and U have a c2 distribution with degrees of
freedom n1andn respectively. (n1 < n)
Then U2 has a c2 distribution with degrees of
freedom n2 =n -n1
Proof:
 12 
Now mU1  t    1 
 2 t 
v1
2
 12 
and mU  t    1 
 2 t 
v
2
Also mU t   mU1 t  mU2 t 
Hence mU 2  t  
mU  t 
mU1  t 
 12 
1 
2 t 


v
2
v  v1
2 2
 12 
v1  

1
2
 12 
 2 t 
1 
 2 t 
Q.E.D.
Distribution of the sample
variance
n
s2 
 (x  x )
i 1
i
n 1
2
Properties of the sample variance
n
n
i 1
i 1
n
n
i 1
i 1
2
2
2
(
x

x
)

(
x

a
)

n
(
x

a
)
 i
 i
Proof:
2
2
(
x

x
)

(
x

a

a

x
)
 i
 i
n
  ( xi  a) 2  2( xi  a)( x  a)  ( x  a) 2 
i 1
n
n
  ( xi  a)  2( x  a) ( xi  a)  n( x  a)
2
i 1
i 1
n
  ( xi  a) 2  2n( x  a ) 2  n( x  a ) 2
i 1
n
  ( xi  a )  n( x  a )
2
i 1
2
2
n
n
i 1
i 1
2
2
2
(
x

x
)

(
x

a
)

n
(
x

a
)
 i
 i
Special Cases


  xi 
n
n
n
i 1


2
2
2
2
(
x

x
)

x

nx

x




i
i
i
n
i 1
i 1
i 1
1. Setting a = 0.
Computing formula
n
2
Setting a = .
2.
n
n
i 1
i 1
2
2
2
(
x

x
)

(
x


)

n
(
x


)
 i
 i
or
n
n
i 1
i 1
2
2
2
(
x


)

(
x

x
)

n
(
x


)
 i
 i
Distribution of the sample variance
Let x1, x2, …, xn denote a sample from the
normal distribution with mean  and variance s2.
Let
z1 
x1  
s
,
, zn 
xn  
s
n
Then
z 
2
1
z 
2
n
 x   
i 1
2
i
s2
has a c2 distribution with n degrees of freedom.
Note:
n
2
(
x


)
 i
i 1
s
2
n

2
(
x

x
)
 i
i 1
s
2

n( x   )
2
s2
or U = U2 + U1
n
U
 (x  )
i 1
2
i
s
2
has a c2 distribution with n degrees of freedom.
We also know that x
has normal distribution with mean  and
variance s2/n
Thus
z
x 
s
n
has a Standard Normal distribution and
U1  z 
2
nx  
2
s2
has a c2 distribution with 1 degree of freedom.
If we can show that U1 and U2 are independent
then
n
U2 
2
(
x

x
)
 i
i 1
s
2

(n  1) s
2
s2
has a c2 distribution with n - 1 degrees of freedom.
The final task would be to show that
n
2
(
x

x
)
and x
 i
i 1
are independent
Summary
Let x1, x2, …, xn denote a sample from the
normal distribution with mean  and variance s2.
1. than x has normal distribution with
mean  and variance s2/n
n
2.
U
 n  1 s
s
2
2

 (x  x )
i 1
2
i
s
2
has a c2 distribution with n = n - 1 degrees of
freedom.
The Transformation Method
Theorem
Let X denote a random variable with
probability density function f(x) and U = h(X).
Assume that h(x) is either strictly increasing
(or decreasing) then the probability density of
U is:


1
dh (u )
dx
g  u   f h (u )
 f  x
du
du
1
Proof
Use the distribution function method.
Step 1 Find the distribution function, G(u)
Step 2 Differentiate G (u ) to find the
probability density function g(u)
G  u   P U  u   P  h  X   u 
 P  X  h 1 (u )  h strictly increasing
 


1
 X  h (u )  h strictly decreasing
P




 F h 1 (u )


1
1

F
h
(
u
)



h strictly increasing

h strictly decreasing
hence
g u   G u 

dh  u 
1
 F   h u 

du

1
 F  h 1 u dh  u 
 


du

1
h strictly increasing
h strictly decreasing
or
1
dh
(u )
dx
1
g  u   f h (u )
 f  x
du
du


Example
Suppose that X has a Normal distribution
with mean  and variance s2.
Find the distribution of U = h(x) = eX.
Solution:
2
x 


1
2s 2
f  x 
e
2s
1
dh  u  d ln  u  1
1
h  u   ln  u  and


du
du
u
hence
1
dh
(u )
dx
1
g  u   f h (u )
 f  x
du
du



1 1 
e
2s u
 ln  u    
2s 2
2
for u  0
This distribution is called the log-normal
distribution
log-normal distribution
0.1
0.08
0.06
0.04
0.02
0
0
10
20
30
40
The Transfomation Method
Theorem
(many variables)
Let x1, x2,…, xn denote random variables
with joint probability density function
f(x1, x2,…, xn )
Let u1 = h1(x1, x2,…, xn).
u2 = h2(x1, x2,…, xn).
un = hn(x1, x2,…, xn).
define an invertible transformation from the x’s to the u’s
Then the joint probability density function of
u1, u2,…, un is given by:
g  u1 ,
, un   f  x1 ,
 f  x1 ,
, xn 
d  x1 ,
d  u1 ,
, xn  J
, xn 
, un 
 dx1
 du
 1
d  x1 , , xn 
where J 
 det 
d  u1 , , un 

 dxn
 du1
Jacobian of the transformation
dx1 
dun 



dxn 
dun 
Example
Suppose that x1, x2 are independent with density
functions f1 (x1) and f2(x2)
Find the distribution of
u1 = x1+ x2
u2 = x1 - x2
Solving for x1 and x2 we get the inverse transformation
u1  u2
x1 
2
u1  u2
x2 
2
The Jacobian of the transformation
J
d  x1 , x2 
d  u1 , u2 
1
2
 det 
1
 2
 dx1
 du
1

 det
 dx2
 du
 1
dx1 
du2 

dx2 

du2 
1 
2    1   1    1  1    1
  




1   2  2   2  2 
2

2 
The joint density of x1, x2 is
f(x1, x2) = f1 (x1) f2(x2)
Hence the joint density of u1 and u2 is:
g  u1 , u2   f  x1 , x2  J
 u1  u2   u1  u2  1
 f1 
 f2 

 2   2 2
From
 u1  u2   u1  u2  1
g  u1 , u2   f1 
 f2 

 2   2 2
We can determine the distribution of u1= x1 + x2
g1  u1  

 g  u , u  du
1





2
2
 u1  u2   u1  u2  1
f1 
 f2 
 du2
 2   2 2
u1  u2
u1  u2
dv 1
put v 
then
 u1  v,

2
2
du2 2
Hence
g1  u1  



 u1  u2   u1  u2  1
f1 
 f2 
 du2
 2   2 2


 f  v  f u
1
2
1
 v  dv

This is called the convolution of the two
densities f1 and f2.
Example: The ex-Gaussian distribution
Let X and Y be two independent random
variables such that:
1. X has an exponential distribution with
parameter l.
2. Y has a normal (Gaussian) distribution with
mean  and standard deviation s.
Find the distribution of U = X + Y.
This distribution is used in psychology as a model
for response time to perform a task.
Now
lel x
f1  x   
 0
x0
x0
2
x 


1
2s 2
f2  y  
e
2s
The density of U = X + Y is :.
g u  

 f  v  f  u  v  dv
1
2


  le
0
 lv
1
e
2s
2
u v   


2s 2
dv
or

l
g u  
e

2s 0

2
u v   


l v
l

e

2s 0

2s 2
2
u v     2s 2l v


2s 2

l

e

2s 0
l

e
2s
dv
dv
v2  2 u   v   u     2s 2l v
2
2s 2
2
u  


2s
2

e
0

dv
v 2  2  u    s 2 l  v
2s 2
dv
l

e
2s
or

u  
2s
2
2

e

v 2  2  u    s 2 l  v
2s 2
0
2
l

e
2s
 u   2   u   s 2l  

2s 2
e

v2  2  u    s 2 l  v   u    s 2l 
 u   2   u   s 2l  

2s 2

0
 le
2
2s 2
dv
0
2
 le
dv

1
e
2s
 u   2   u   s 2l 

2s 2
v 2  2  u    s 2l  v   u   s 2l 
2
P V  0
2s 2
2
dv
Where V has a Normal distribution with mean
V  u     s l 
2
and variance s2.
Hence
g u   le

s 2l 
 l  u    

2



    s 2l   u  

1    


2


s




Where (z) is the cdf of the standard Normal
distribution
The ex-Gaussian distribution
0.09
g(u)
0.06
0.03
0
0
10
20
30