Download Week11

Convergence in Distribution
• Recall: X n   in probability if lim P X n       0.
n
• Definition
Let X1, X2,…be a sequence of random variables with cumulative
distribution functions F1, F2,… and let X be a random variable with cdf
FX(x). We say that the sequence {Xn} converges in distribution to X if
lim FX n  x   FX  x 
n 
at every point x in which F is continuous.
• This can also be stated as: {Xn} converges in distribution to X if for all x  R
such that P(X = x) = 0
lim P X n  x   P X  x 
n 
• Convergence in distribution is also called “weak convergence”. It is weaker
then convergence in probability. We can show that convergence in
probability implies convergence in distribution.
week 11
1
Simple Example
• Assume n is a positive integer. Further, suppose that the probability mass
function of Xn is:
P X n  0 
1 1

2 n
,
P X n  1 
1 1

2 n
Note that this is a valid p.m.f for n ≥ 2.
• For n ≥ 2, {Xn} convergence in distribution to X which has p.m.f
P(X = 0) = P(X = 1) = ½ i.e. X ~ Bernoulli(1/2)
week 11
2
Example
• X1, X2,…is a sequence of i.i.d random variables with E(Xi) = μ < ∞.
•
1 n
Let X n   X i. Then, by the WLLN for any a > 0
n i 1


P X n    a  0 as n  ∞.
• So…
week 11
3
Continuity Theorem for MGFs
• Let X be a random variable such that for some t0 > 0 we have mX(t) < ∞ for
t   t0 , t0  . Further, if X1, X2,…is a sequence of random variables with
m X n t    and lim m X n t   m X t  for all t   t0 , t0 
n 
then {Xn} converges in distribution to X.
• This theorem can also be stated as follows:
Let Fn be a sequence of cdfs with corresponding mgf mn. Let F be a cdf with
mgf m. If mn(t)  m(t) for all t in an open interval containing zero, then
Fn(x)  F(x) at all continuity points of F.
• Example:
Poisson distribution can be approximated by a Normal distribution for large λ.
week 11
4
Example to illustrate the Continuity Theorem
• Let λ1, λ2,…be an increasing sequence with λn ∞ as n  ∞ and let {Xi} be
a sequence of Poisson random variables with the corresponding parameters.
We know that E(Xn) = λn = V(Xn).
X  E  X n  X n  n
• Let Z n  n
then we have that E(Zn) = 0, V(Zn) = 1.

V X n 
n
• We can show that the mgf of Zn is the mgf of a Standard Normal random
variable.
• We say that Zn convergence in distribution to Z ~ N(0,1).
week 11
5
Example
• Suppose X is Poisson(900) random variable. Find P(X > 950).
week 11
6
Central Limit Theorem
• The central limit theorem is concerned with the limiting property of sums
of random variables.
• If X1, X2,…is a sequencen of i.i.d random variables with mean μ and
variance σ2 and , S   X
n
i 1
i
then by the WLLN we have that
Sn
  in probability.
n
• The CLT concerned not just with the fact of convergence but how Sn /n
fluctuates around μ.
• Note that E(Sn) = nμ and V(Sn) = nσ2. The standardized version of Sn is
S n  n
Zn 
and we have that E(Zn) = 0, V(Zn) = 1.
 n
week 11
7
The Central Limit Theorem
• Let X1, X2,…be a sequence of i.i.d random variables with E(Xi) = μ < ∞
and Var(Xi) = σ2 < ∞. Suppose the common distribution function FX(x) and
the common moment generating function mX(t) are defined in a
neighborhood of 0. Let
n
Sn   X i
i 1
Then, lim P S n  n  x   x  for - ∞ < x < ∞
n
  n

where Ф(x) is the cdf for the standard normal distribution.
• This is equivalent to saying that Z n  S n  n converges in distribution to
 n
Z ~ N(0,1).
•
 Xn  


P
 x     x 
Also, lim
n   
n


i.e. Z n  X n   converges in distribution to Z ~ N(0,1).
 n
week 11
8
Example
• Suppose X1, X2,…are i.i.d random variables and each has the Poisson(3)
distribution. So E(Xi) = V(Xi) = 3.


• The CLT says that P X 1    X n  3n  x 3n  x as n  ∞.
week 11
9
Examples
• A very common application of the CLT is the Normal approximation to the
Binomial distribution.
• Suppose X1, X2,…are i.i.d random variables and each has the Bernoulli(p)
distribution. So E(Xi) = p and V(Xi) = p(1- p).


• The CLT says that P X 1    X n  np  x np1  p  x as n  ∞.
• Let Yn = X1 + … + Xn then Yn has a Binomial(n, p) distribution.

So for large n, PYn  y   P Yn  np

 np1  p 

 y  np 
y  np 

 



np1  p  
 np1  p  
• Suppose we flip a biased coin 1000 times and the probability of heads on
any one toss is 0.6. Find the probability of getting at least 550 heads.
• Suppose we toss a coin 100 times and observed 60 heads. Is the coin fair?
week 11
10