Download Handout 9 - UIUC Math

Limit Theorems
Inequalities
Proposition 1 (Markov Inequality) If X is a nonnegative random variable with mean µ, then
for any a > 0,
µ
P(X ≥ a) ≤ .
a
Proposition 2 (Chebyshev Inequality) If X is a random variable with mean µ and variance
σ 2 , then for any a > 0,
σ2
P(|X − µ| ≥ a) ≤ 2 .
a
Example 3 Suppose that it is known that the number of items produced in a factory during a week
is a random variable with mean 50.
(a) What can be said about the probability that this week’s production will exceed 75?
(b) If the variance of a weeks production is known to be 25, what can be said about the probability
that this week’s production will between 40 and 60?
Limit Theorems
Theorem 4 (Weak Law of Large Numbers) Let X1 , X2 , . . . be a sequence of independent and
identically distributed random variables with common mean µ. Then for any ² > 0,
¯
µ¯
¶
¯ X1 + · · · + Xn
¯
¯
¯
lim P ¯
− µ¯ ≥ ² = 0.
n→∞
n
Proof We will prove this theorem under the additional assumption that random variables has
a finite variance σ 2 .
Theorem 5 (Strong Law of Large Numbers) Let X1 , X2 , . . . be a sequence of independent
and identically distributed random variables with common mean µ. Then,
µ
¶
X1 + · · · + Xn
P lim
= µ = 1.
n→∞
n
Theorem 6 (Central Limit Theorem) Let X1 , X2 , . . . be a sequence of independent and identically distributed random variables with common mean µ and common variance σ 2 . Then the
distribution of
X1 + · · · + Xn − nµ
√
σ n
1
converges to the standard normal distribution. That is, for any a ∈ R,
µ
¶
Z a
X1 + · · · + Xn − nµ
1
2
√
e−x /2 dx.
lim P
≤a = √
n→∞
σ n
2π −∞
The key to the proof of the central limit theorem is the following result, which we state without
proof.
Proposition 7 Let Z1 , Z2 , . . . be a sequence of random variables with distribution functions FZn
and moment generating functions MZn , n ≥ 1; let Z be a random variable with distribution function
FZ and moment generating function MZ . If MZn (t) → MZ (t) for all t, then FZn (t) → FZ (t) for
all t which is a continuous point for FZ .
Proof of the Central Limit Theorem We will prove the theorem under the assumption that
the moment generating function M (t) of Xi is finite for all t.
√
First we assume that µ = 0 and σ 2 = 1. The moment generating function of Xi / n is
t
MXi /√n (t) = M ( √ ),
n
and thus
µ
MPni=1 Xi /√n (t) =
¶n
t
M(√ ) .
n
Put L(t) = ln M (t). Then we can easily get that L(0) = 0, L0 (0) = 0 and L0 (0) = 1. To prove the
theorem, we must show that
µ
¶n
t2
t
M(√ )
→e2,
n
or equivalently
√
t2
nL(t/ n) → .
2
By using L’Hopital’s rule twice, we get
√
√
√
L(t/ n)
−L0 (t/ n)n−3/2 t
L0 (t/ n)t
= lim
= lim
lim nL(t/ n) = lim
n→∞
n→∞
n→∞
n→∞ 2n−1/2
n−1
−2n−2
√
−3/2
2
2
2
00
√ t
−L (t/ n)n
t
t
= lim
= lim L00 (t/ n) = .
−3/2
n→∞
n→∞
2
2
−2n
√
Thus the central limit theorem is true when µ = 0 and σ 2 = 1. The general case follows by
fi = (Xi − µ)/σ, since E[X
fi ] = 0 and Var(X
fi ) = 1.
considering the random variables X
Applications of the Central Limit Theorem
Example 8 A certain basketball player makes 80 percent of his free throws on average. What is the
probability that in 100 attempts he will be successful more than 85 times? Assume independence.
Solution In this case n = 100 and each Xi is a Bernoulli random variable with parameter .8.
Thus µ = .8 and σ 2 = .16. Consequently by the central limit theorem
¶
µ Pn
n
n
X
X
85.5 − 80
i=1 Xi − 80
≈ 1 − Φ(1.375) ≈ .085.
P(
≥
Xi > 85) = P(
Xi ≥ 85.5) = P
4
4
i=1
i=1
2
Example 9 The number of students enrolled in a calculus class is a Poisson random variable with
parameter 100. If the number enrolling is 120 or more, the class will be taught in two sections,
otherwise it will be taught in one section. Find the probability that the class has to be taught in two
sections.
Solution The exact solution
∞
X
(100)i
e−100
i!
i=120
does not readily yields a numerical answer. However, recalling that a Poisson random variable with
parameter 100 is the sum of 100 independent Poisson random variables with parameter 1, we can
use the central limit theorem to find an approximate answer. If X denote the number of students
enrolled in the course, then
µ
¶
X − 100
119.5 − 100
P(X ≥ 120) = P(X ≥ 119.5) = P
≥
≈ 1 − Φ(1.95) ≈ .0265.
10
10
Example 10 Let Xi , i = 1, 2, . . . , 48 be independent random variables, each uniformly distributed
P
in (0, 1). Find an approximation to P( 48
i=1 Xi > 27).
Solution Since E[Xi ] = 1/2 and Var(Xi ) = 1/12, we have by the central limit theorem
!
ÃP
48
48
X
X
−
24
27
−
24
i
i=1
P(
>
Xi > 27) = P
≈ 1 − Φ(1.5) ≈ .0668.
2
2
i=1
Applications to Polling
The central limit theorem can be regarded as a refinement of the weak law of large numbers. In
fact, let X1 , X2 , . . . be a sequence of independent and identically distributed random variables with
P
mean µ and variance σ 2 . Put Sn = ni=1 Xi . Then by the central limit theorem, for any c > 0,
¯
µ¯
¶
¯ Sn
¯
¯
¯
P ¯
− µ¯ ≥ c
= P(Sn ≤ nµ − nc) + P(Sn ≥ nµ + nc)
n
µ
¶
µ
¶
nc
nc
√
≈ Φ − √
+1−Φ
σ n
σ n
µ
µ √ ¶¶
c n
.
= 2 1−Φ
σ
In other words
¯
µ¯
¶
¯ Sn
¯
¯
¯
P ¯
− µ¯ ≥ c ≈ 2(1 − Φ(δ)),
n
where
δ=
√
c n
.
σ
(0.1)
(0.2)
Example 11 A sample of size n is to be taken to determine the percentage of the population
planning to vote a certain candidate A in an upcoming election. Let Xi = 1 if the i-th person
sampled plans to vote for A and Xi = 0 otherwise. We assume that X1 , X2 , . . . , Xn are independent
and identically distributed with P(Xi = 1) = p and P(Xi = 0) = 1−p. Then µ = p and σ 2 = p(1−p).
p
WE also assume that p is close to .5 so that σ = p(1 − p) can be approximated satisfactorily by
σ ≈ 1/2 (note that σ has a maximum of 1/2 at p = .5, and that, as p ranges over .3 ≤ p ≤ .7,
3
σ stays above .458 which is close to 1/2). The random variable Sn /n denotes the fraction of the
people sampled that pan to vote for A and can be used to estimate the true unknown probability p.
We will use the central limit theorem to solve the following three problems:
(i) Suppose n = 900. Find the probability that
¯
¯
¯ Sn
¯
¯
¯ ≥ .025.
−
p
¯n
¯
(ii) Suppose n = 900. Find c such that
¯
¶
µ¯
¯ Sn
¯
¯
¯
P ¯
− p¯ ≥ c = .01.
n
(iii) Find n such that
¯
¶
µ¯
¯
¯ Sn
¯
¯
− p¯ ≥ .025 = .01.
P ¯
n
Solution (i) By (0.2)
so by (0.1)
√
(.025) 900
= 1.5,
δ=
.5
¯
µ¯
¶
¯ Sn
¯
P ¯¯
− p¯¯ ≥ .025 ≈ 2(1 − Φ(1.5)) = .134.
n
(ii) We first choose δ so that 2(1 − Φ(δ)) = .01 or φ(δ) = .995. Using the table we know that
δ = 2.58. Solving (0.2) for c we get
δσ
(2.58)(.5)
c= √ = √
= .043.
n
900
(iii) As in (ii) we have δ = 2.58. Solving (0.2) for n we find
n=
δ2σ2
(2.58)2 (.25)
=
= 2663.
2
c
(.025)2
Remark on Notations
4