Download Lecture 5: Weak Laws of Large Numbers 1.) L2 Weak Laws We

Lecture 5: Weak Laws of Large Numbers
1.) L2 Weak Laws
We begin by defining some modes of convergence for random variables. Suppose that Xn , n ≥ 1
and X are random variables defined on the same probability space.
We say that Xn converges to X in Lp if E|X − Xn |p → 0 as n → ∞. In contrast, we say
p
that Xn converges to X in probability and write Xn → X if for every > 0, we have
P{|Xn − X| > } → 0.
Lemma (1.5.3): If r > 0 and E|Xn |r → 0, then Xn → 0 in probability.
Proof: The result follows from Chebyshev’s inequality which shows that
P{|Xn | > } ≤ −r E|Xn |r → 0.
We say that a family of random variables, Xi , i ∈ I, is uncorrelated if EXi2 < ∞ for every
i ∈ I and EXi Xj = 0 whenever i 6= j.
Lemma (1.5.1): Let X1 , · · · , Xn be uncorrelated. Then
Var (X1 + · · · + Xn ) =
n
X
Var(Xi ).
i=1
L2 weak law. Let X1 , X2 , · · · be uncorrelated random variables with EXi = µ and Var(Xi ) ≤
C < ∞. If Sn = X1 + · · · + Xn , then as n → ∞, Sn /n → µ in L2 and in probability.
Proof: To prove L2 convergence, observe that E[Sn /n] = µ, so
E Sn /n − µ
2
= Var(Sn /n) =
n
1 X
Cn
Var(Xi ) ≤ 2 → 0.
2
n
n
i=1
Convergence in probability then follows from Lemma (1.5.3).
2.) Triangular Arrays
Many limit theorems in probability address the asymptotic behavior of the row sums of arrays
Xn,k , 1 ≤ k ≤ n of random variables.
Theorem (1.5.4): Let µn = ESn and σn2 = Var(Sn ). If σn2 /b2n → 0, then (Sn − µn )/bn → 0 in
probability.
1
Proof: The result follows from Lemma (1.5.3) since
Sn − µn 2
= b−2
E
n Var(Sn ) → 0.
bn
Example: Coupon Collector’s Problem. Let Y1 , Y2 , · · · be i.i.d. uniform on {1, 2, · · · , n}
and define
τkn = inf m : |{Y1 , · · · , Ym }| = k
to be the first time that we have k different items in the sample. Here we will study the
asymptotics of Tn = τnn , the time to collect a complete set. Set τ0n = 0 and note that τ1n = 1.
n
For 1 ≤ k ≤ n, Xn,k = τkn − τk−1
is the time elapsed until we sample an item distinct from
the first k − 1. Notice that Xn,k is a geometric random variable with parameter 1 − (k − 1)/n
and is independent of the earlier waiting times Xn,j , 1 ≤ j < k. Recall that if X is a geometric
random variable with parameter p, then EX = 1/p and Var(X) = (1 − p)/p2 . Writing Tn =
Xn,1 + · · · + Xn,n , we have
n n
X
X
k − 1 −1
1−
ETn =
= n
k −1 ∼ n log(n)
n
k=1
k=1
−2
n n
X
X
k−1
π2
2
Var(Tn ) ≤
1−
= n
k −2 ≤ n2 .
n
6
k=1
k=1
Taking bn = n log(n), Theorem (1.5.4) implies that
P
Tn − n nk=1 k −1
→ 0 in probability
n log(n)
and so Tn /(n log(n)) → 1 in probability.
3.) Truncation
We can use truncation to extend the weak law to random variables without a second moment.
Theorem (1.5.5): Weak law for triangular arrays. For each n ≥ 1, let Xn,k , 1 ≤ k ≤ n be
a collection of independent random variables. Let bn , n > 1 be a collection of real numbers with
bn → ∞ and let X̄n,k = Xn,k 1(|Xn,k |≤bn ) . Suppose that as n → ∞
Pn
1.
k=1 P(|Xn,k | > bn ) → 0, and
Pn
2
2. b−2
n
k=1 EX̄n,k → 0.
P
If Sn = Xn,1 + · · · + Xn,n and an = nk=1 EX̄n,k , then (Sn − an )/bn → 0 in probability.
Proof: Let S̄n = X̄n,1 + · · · + X̄n,n . Then
Sn − an S̄n − an > ≤ P(Sn 6= S̄n ) + P P bn > .
bn 2
To estimate the first term on the RHS, note that
n
X
P(Sn 6= S̄n ) ≤ P ∪nk=1 {X̄n,k 6= Xn,k } ≤
P(|Xn,k | > bn ) → 0
k=1
by (i). For the second term, we can use (ii) along with Chebyshev’s inequality and the fact that
Var(Xn ) ≤ EXn2 to show that
2
S̄n − an −2 S̄n − an −2 −2
>
≤
E
P bn = bn Var(S̄n )
bn n
n
X
X
−2
−2
= (bn )
var(X̄n,k ) ≤ (bn )
E(X̄n,k )2 → 0.
k=1
k=1
Lemma (1.5.7): If Y ≥ 0 and p > 0, then
Z ∞
p
EY =
py p−1 P(Y > y)dy.
0
Proof: Using Fubini’s Theorem, we can calculate
Z ∞
Z ∞Z
p−1
py P(Y > y)dy =
py p−1 1(Y >y) dPdy
0
Ω
Z0 Z ∞
py p−1 1(Y >y) dydP
=
Ω 0
Z Z Y
Z
p−1
=
py dydP =
Y p dP = EY p .
Ω
0
Ω
Theorem (1.5.6): Weak Law of Large Numbers. Let X1 , X2 , · · · be i.i.d. with
xP(|X1 | > x) → 0 as x → ∞.
Let Sn = X1 + · · · + Xn and µn = E X1 1(|X1 |≤n) . Then Sn /n − µn → 0 in probability.
(?)
Proof: We will apply Theorem (1.5.5) with Xn,k = Xk and bn = n. To check condition (i) in
that theorem, observe that (?) implies that
n
X
P(|Xn,k | > n) = nP(|X1 | > n) → 0.
k=1
2 → 0. First observe that
For condition (ii), we need to show that n−2 · nEX̄n,1
1
1
2
EX̄n,1
=
n
n
Z
0
∞
1
2yP(|X̄n,1 | > y)dy ≤
n
3
Z
n
2yP(|X1 | > y)dy
0
since P(|X̄n,1 | > y) = 0 for y ≥ n. To show that the last term tends to 0 as n → ∞, let
g(y) = 2yP(|X1 | > y) and notice that 0 ≤ g(y) ≤ 2y and g(y) → 0 as y → ∞ imply that
M = sup g(y) < ∞. If we define K = sup{g(y) : y > K}, then for n > K
Z n
2yP(|X1 | > y)dy ≤ KM + (n − K)K .
0
Dividing by n and letting n → ∞ gives
Z
1 n
lim sup
2yP(|X1 | > y)dy ≤ K .
n→∞ n 0
The result then follows upon noting that K → 0 as K → ∞.
The familiar form of the weak law of large numbers is:
Corollary (1.5.8): Let X1 , X2 , · · · be i.i.d. with E|Xi | < ∞. Let Sn = X1 + · · · + Xn and
µ = EX1 . Then Sn /n → µ in probability.
Proof: Chebyshev’s inequality and the dominated convergence theorem imply that
xP(|X1 | > x) ≤ E |X1 |1(|X1 |>x) → 0 as x → ∞
µn = E X1 1(|X1 |≤n) → EX1 = µ as n → ∞.
Using Theorem (1.5.6), we see that if > 0, then P(|Sn /n − µn | > /2) → 0. Since µn → µ, it
follows that P(|Sn /n − µ| > ) → 0.
4