Download Example: Student`s t distribution

Functions of Random Variables
Methods for determining the distribution of
functions of Random Variables
1. Distribution function method
2. Moment generating function method
3. Transformation method
Distribution function method
Let X, Y, Z …. have joint density f(x,y,z, …)
Let W = h( X, Y, Z, …)
First step
Find the distribution function of W
G(w) = P[W ≤ w] = P[h( X, Y, Z, …) ≤ w]
Second step
Find the density function of W
g(w) = G'(w).
Example: Student’s t distribution
Let Z and U be two independent random
variables with:
1. Z having a Standard Normal distribution
and
2. U having a c2 distribution with n degrees
of freedom
Find the distribution of
Z
t
U
n
The density of Z is:
1
f  z 
e
2
The density of U is:
n
z2

2
 1 2
  n 1  u
2 2

h u  
u e 2
n 
 
2
Therefore the joint density of Z and U is:
n
 1 2
n
z 2 u
 
1 
2

2
f  z, u   f  z  h  u  
u e 2
n 
2   
2
The distribution function of T is:


Z
t



G  t   P T  t   P
 t  P Z 
 U

n



n

U

Then
n 1 
n 1
n 1



  t2  2
2
2


t
2

g (t )  G  t   
 1
 K   1

n
n   n



n   
2
where
n 1 


2 

K
n 
n   
2
Student’s t distribution
 t2 
g (t )  K   1
n

where
n 1

n 1 


2 

K
n 
n   
2
2
Student – W.W. Gosset
Worked for a distillery
Not allowed to publish
Published under the
pseudonym “Student
t distribution
standard normal distribution
Distribution of the Max and Min
Statistics
Let x1, x2, … , xn denote a sample of size n from
the density f(x).
Let M = max(xi) then determine the distribution
of M.
Repeat this computation for m = min(xi)
Assume that the density is the uniform density
from 0 to q.
Hence
1

f ( x )  q
 
0  x q
elsewhere
and the distribution function
0
x

F ( x)  P  X  x   
q
 1
x0
0  x q
x q
Finding the distribution function of M.
G (t )  P  M  t   P  max  xi   t 
 P  x1  t ,
, xn  t 
 P  x1  t 
P  xn  t 
 0

n
 t 
  
 q 
 1

t0
0  t q
t q
Differentiating we find the density function of M.
 nt n 1
 n
g  t   G  t    q
 0

0.12
0  t q
otherwise
0.6
f(x)
0.1
0.5
0.08
0.4
0.06
0.3
0.04
0.2
0.02
0.1
0
0
0
2
4
6
8
10
g(t)
0
2
4
6
8
10
Finding the distribution function of m.
G (t )  P  m  t   P  min  xi   t 
 1  P  x1  t ,
, xn  t 
 1  P  x1  t 
P  xn  t 
0


n
  t
 1  1  
  q

1

t0
0  t q
t q
Differentiating we find the density function of m.
n  t 
 1  
g  t   G   t   q  q 

0

n 1
0  t q
otherwise
f(x)
g(t)
0.12
0.6
0.1
0.5
0.08
0.4
0.06
0.3
0.04
0.2
0.02
0.1
0
0
0
2
4
6
8
10
0
2
4
6
8
10
The probability integral transformation
This transformation allows one to convert
observations that come from a uniform
distribution from 0 to 1 to observations that
come from an arbitrary distribution.
Let U denote an observation having a uniform
distribution from 0 to 1.
1 0  u  1
g (u )  
 elsewhere
Let f(x) denote an arbitrary density function and
F(x) its corresponding cumulative distribution
function.
1
X

F
(U )
Let
Find the distribution of X.
1

G  x   P  X  x   P  F (U )  x 
 P U  F  x  
 F  x
Hence.
g  x   G  x   F   x   f  x 
Thus if U has a uniform distribution from 0 to 1.
Then
1
X  F (U )
has density f(x).
X  F 1 (U )
U
Use of moment generating
functions
Definition
Let X denote a random variable with probability
density function f(x) if continuous (probability mass
function p(x) if discrete)
Then
mX(t) = the moment generating function of X
 
 E etX
  tx
  e f  x  dx if X is continuous
  
  etx p  x 
if X is discrete
 x
The distribution of a random variable X is
described by either
1. The density function f(x) if X continuous (probability
mass function p(x) if X discrete), or
2. The cumulative distribution function F(x), or
3. The moment generating function mX(t)
Properties
1. mX(0) = 1
2. mXk   0   k th derivative of mX  t  at t  0.
 
 k  E X
k  E  X
3.
k

k
k

 x f  x  dx

k

  x p  x
mX  t   1  1t 
2
2!
t 
2
X continuous
X discrete
3
3!
t 
3

k
k!
t 
k
.
4. Let X be a random variable with moment
generating function mX(t). Let Y = bX + a
Then mY(t) = mbX + a(t)
= E(e [bX + a]t) = eatmX (bt)
5. Let X and Y be two independent random
variables with moment generating function
mX(t) and mY(t) .
Then mX+Y(t) = mX (t) mY (t)
6. Let X and Y be two random variables with
moment generating function mX(t) and mY(t)
and two distribution functions FX(x) and
FY(y) respectively.
Let mX (t) = mY (t) then FX(x) = FY(x).
This ensures that the distribution of a random
variable can be identified by its moment
generating function
M. G. F.’s - Continuous distributions
Name
Continuous
Uniform
Exponential
Gamma
c2
nd.f.
Normal
Moment generating
function MX(t)
ebt-eat
[b-a]t
  
for t < 


  t 

  
for t < 


  t 
 1 
1-2t


n/2
for t < 1/2
t+(1/2)t22
e
M. G. F.’s - Discrete distributions
Name
Discrete
Uniform
Bernoulli
Binomial
Geometric
Negative
Binomial
Poisson
Moment
generating
function MX(t)
et etN-1
N et-1
q + pet
(q + pet)N
pet
1-qet
 pet  k


t
1-qe


(et-1)
e
Moment generating function of the
gamma distribution
 
mX  t   E etX 

tx
e
 f  x  dx

where

 
 1   x
x
e

f  x      

0

x0
x0

    e f  x  dx
mX  t   E e
tX
tx




 1   x
 e
x e dx
  
0
 

 1   t  x

x e
dx

   0
tx

using
or
ba a 1 bx
0   a  x e dx  1

 a
a 1  bx
0 x e dx  ba
then



 1   t  x
mX  t  
x e
dx

   0
    


      t 

  


  t 
t
Moment generating function of the
Standard Normal distribution

 
mX  t   E etX 
tx
e
 f  x  dx

where
1
f  x 
e
2
x2

2
thus
mX  t  

e

tx
1
e
2
x2

2

dx 


1
e
2
x2
  tx
2
dx


We will use
0
mX  t  









e
1
e
2
x2
  tx
2
x2  2tx

2


1
e
2 b
t2
2
2
x a 


2b2
dx  1
dx
1
e
dx
2
2
x t 

x 2  2 tx  t 2 t 2
t2 
1  2
1  2
2
2
e
e dx  e 
e
dx
2
2

Note:
2
3
4
x
x
x
ex  1  x    
2! 3! 4!
2
3
t  t 
   
t2
2
2 2
t

2
mX  t   e  1  


2
2!
3!
2
2
4
6
t
t
t
 1  2  3 
2 2 2! 2 3!
Also
mX  t   1  1t 
2
2!
t 
2
2
2m
t
 m 
2 m!
3
3!
t 
3
Note:
2
3
4
x
x
x
ex  1  x    
2! 3! 4!
2
3
t  t 
   
t2
2
2 2
t

2
mX  t   e  1  


2
2!
3!
2
2
Also
4
2
6
2m
t
t
t
t
 1  2  3   m 
2 2 2! 2 3!
2 m!
 2 2 3 3
mX  t   1  1t  t  t 
2!  3!
k  k th moment 


x k f  x  dx
Equating coefficients of tk, we get
k  0 if k is odd and
2 m
1
for k  2m then m 
2 m !  2m  !
hence 1  0, 2  1, 3  0, 4  3
Using of moment generating
functions to find the distribution of
functions of Random Variables
Example
Suppose that X has a normal distribution with
mean  and standard deviation .
Find the distribution of Y = aX + b
Solution:
 2t 2
t 
mX  t   e 2
2
 2 at
  at  
bt
maX b  t   e mX  at   e e
bt
 a   b t 
e
 
2
 2 a 2t 2
2
= the moment generating function of the normal
distribution with mean a + b and variance a22.
Thus Y = aX + b has a normal distribution with
mean a + b and variance a22.
Special Case: the z transformation
Z
X 

1
  
   X 
  aX  b
 
  
1
  
Z  a  b      
0
 
  
2
1 2
2
2 2
Z  a      1
 
Thus Z has a standard normal distribution .
Example
Suppose that X and Y are independent each having a
normal distribution with means X and Y , standard
deviations X and Y
Find the distribution of S = X + Y
Solution:
mX  t   e
mY  t   e
X t 
Y t 
 X2 t 2
2
 Y2 t 2
2
Now
mX Y  t   mX  t  mY  t   e
X t 
 X2 t 2
2
e
Y t 
 Y2 t 2
2
or
mX Y  t   e
  X  Y


t 
2
X

 Y2 t 2
2
= the moment generating function of the
normal distribution with mean X + Y and
2
2
variance  X   Y
Thus Y = X + Y has a normal distribution
2
2
with mean X + Y and variance  X   Y
Example
Suppose that X and Y are independent each having a
normal distribution with means X and Y , standard
deviations X and Y
Find the distribution of L = aX + bY
Solution:
mX  t   e
X t 
 X2 t 2
mY  t   e
2
Y t 
 Y2 t 2
2
Now
maX bY t   maX t  mbY t   mX  at  mY bt 
e
 X  at  
 X2  at 
2
2
e
Y  bt  
 Y2  bt 
2
2
or
a

t 
2
maX bY  t   e
 a X bY
2
X

 b2 Y2 t 2
2
= the moment generating function of the
normal distribution with mean aX + bY
2 2
2 2
and variance a  X  b  Y
Thus Y = aX + bY has a normal
distribution with mean aX + bY and
2 2
2 2
variance a  X  b  Y
Special Case:
a = +1 and b = -1.
Thus Y = X - Y has a normal distribution
with mean X - Y and variance
 1
2
 X   1  Y   X   Y
2
2
2
2
2
Example (Extension to n independent RV’s)
Suppose that X1, X2, …, Xn are independent each having a
normal distribution with means i, standard deviations i
(for i = 1, 2, … , n)
Find the distribution of L = a1X1 + a1X2 + …+ anXn
Solution:
mX i  t   e
Now
ma1 X1 
i t 
 i2t 2
(for i = 1, 2, … , n)
2
 an X n
t   ma X t 
1 1
 mX1  a1t 
e
1  a1t  
man X n t 
mX n  ant 
12  a1t 
2
2
e
n  an t  
 n2  an t 
2
2
or
ma1 X1 
 an X n
t   e
 a11 ... an n
a

t 
2 2
2 2

...

a
1 1
n n
t
2
2
= the moment generating function of the
normal distribution with mean a11  ...  an n
2 2
2 2
and variance a1  1  ...  an  n
Thus Y = a1X1 + … + anXn has a normal
distribution with mean a11 + …+ ann
2 2
2 2
and variance a1  1  ...  an  n
Special case:
a1  a2 
1  2 
 12   12 
1
 an 
n
 n  
  12   2
In this case X1, X2, …, Xn is a sample from a
normal distribution with mean , and standard
deviations ,and
1
L   X1  X 2   X n 
n
 X  the sample mean
Thus
Y  x  a1 x1  ...  an xn
 n
 1
 n x
x1  ...  1
n
has a normal distribution with mean
 x  a11  ...  an n
 1   ...  1   
n
n
and variance
 
 
 x2  a12 12  ...  an2 n2
2
1
1
1

  2
  2
  2
     ...      n    
n
n
n
n
2
2
2
Summary
If x1, x2, …, xn is a sample from a normal
distribution with mean , and standard
deviations ,then x  the sample mean
has a normal distribution with mean
x  
and variance
 
2
x

2
n
 

 standard deviation  x 

n

Sampling distribution
of x
0.4
0.3
Population
0.2
0.1
0
20
30
40
50
60
The Central Limit theorem
If x1, x2, …, xn is a sample from a distribution
with mean , and standard deviations ,then
if n is large x  the sample mean
has a normal distribution with mean
x  
and variance
 
2
x
 
2
 
standard deviation  x 


n 
n
Proof: (use moment generating functions)
We will use the following fact:
Let
m1(t), m2(t), …
denote a sequence of moment generating functions
corresponding to the sequence of distribution
functions:
F1(x) , F2(x), …
Let m(t) be a moment generating function
corresponding to the distribution function F(x) then
if
lim mi  t   m  t  for all t in an interval about 0.
i 
then lim Fi  x   F  x  for all x.
i 
Let x1, x2, … denote a sequence of independent
random variables coming from a distribution with
moment generating function m(t) and distribution
function F(x).
Let Sn = x1 + x2 + … + xn then
mSn t   mx1  x2 
=  m  t  
 xn
t  =mx t  mx t 
1
2
mxn t 
n
x1  x2 
now x 
n
 xn
Sn

n
n
 t    t 
or mx  t   m 1   t   mSn    m   
  Sn
 n    n 
n
Let z 
x 


n
then mz  t   e

n

n

t
x
n

 nt  
mx 
  e
  
n

t
  nt  
m 
 
   n  
  t 
and ln mz  t   
t  n ln m 


   n 
n
n
t
t2
Let u 
or n 
and n  2 2
u
 u
 n
t
  t 
Then ln mz  t   
t  n ln m 


   n 
t 2
t2
  2  2 2 ln  m  u 
 u  u
n
t 2 ln m  u   u
 2

u2



Now lim ln  mz  t    lim ln  mz  t  
n 
u 0


t2

t2

lim
ln  m  u   u
2 u 0
lim
2 u 0
u2
m  u 

m u 
using L'Hopital's rule
2u
m  u  m  u    m  u 

t
2

lim
2 u 0

 m  u 
2
2
2
using L'Hopital's rule again
m  u  m  u    m  u 

t
 m  u 
2
2

lim
2 u 0
t m  0   m  0 
 2

2
2
 
2
2
using L'Hopital's rule again
2
t E x   E  xi 
t2
 2


2
2
2
2
i

2

2
t
thus lim ln  mz  t  
and lim  mz  t    e
n
n
2
t2
2
Now m  t   e
t2
2
Is the moment generating function of the standard
normal distribution
Thus the limiting distribution of z is the standard
normal distribution
i.e. lim Fz  x  
n
x


1
e
2
u2

2
du
Q.E.D.