Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Random variable wikipedia , lookup
Birthday problem wikipedia , lookup
Inductive probability wikipedia , lookup
Ars Conjectandi wikipedia , lookup
Infinite monkey theorem wikipedia , lookup
Probability interpretations wikipedia , lookup
Conditioning (probability) wikipedia , lookup
Lecture 5: Weak Laws of Large Numbers 1.) L2 Weak Laws We begin by defining some modes of convergence for random variables. Suppose that Xn , n ≥ 1 and X are random variables defined on the same probability space. We say that Xn converges to X in Lp if E|X − Xn |p → 0 as n → ∞. In contrast, we say p that Xn converges to X in probability and write Xn → X if for every > 0, we have P{|Xn − X| > } → 0. Lemma (1.5.3): If r > 0 and E|Xn |r → 0, then Xn → 0 in probability. Proof: The result follows from Chebyshev’s inequality which shows that P{|Xn | > } ≤ −r E|Xn |r → 0. We say that a family of random variables, Xi , i ∈ I, is uncorrelated if EXi2 < ∞ for every i ∈ I and EXi Xj = 0 whenever i 6= j. Lemma (1.5.1): Let X1 , · · · , Xn be uncorrelated. Then Var (X1 + · · · + Xn ) = n X Var(Xi ). i=1 L2 weak law. Let X1 , X2 , · · · be uncorrelated random variables with EXi = µ and Var(Xi ) ≤ C < ∞. If Sn = X1 + · · · + Xn , then as n → ∞, Sn /n → µ in L2 and in probability. Proof: To prove L2 convergence, observe that E[Sn /n] = µ, so E Sn /n − µ 2 = Var(Sn /n) = n 1 X Cn Var(Xi ) ≤ 2 → 0. 2 n n i=1 Convergence in probability then follows from Lemma (1.5.3). 2.) Triangular Arrays Many limit theorems in probability address the asymptotic behavior of the row sums of arrays Xn,k , 1 ≤ k ≤ n of random variables. Theorem (1.5.4): Let µn = ESn and σn2 = Var(Sn ). If σn2 /b2n → 0, then (Sn − µn )/bn → 0 in probability. 1 Proof: The result follows from Lemma (1.5.3) since Sn − µn 2 = b−2 E n Var(Sn ) → 0. bn Example: Coupon Collector’s Problem. Let Y1 , Y2 , · · · be i.i.d. uniform on {1, 2, · · · , n} and define τkn = inf m : |{Y1 , · · · , Ym }| = k to be the first time that we have k different items in the sample. Here we will study the asymptotics of Tn = τnn , the time to collect a complete set. Set τ0n = 0 and note that τ1n = 1. n For 1 ≤ k ≤ n, Xn,k = τkn − τk−1 is the time elapsed until we sample an item distinct from the first k − 1. Notice that Xn,k is a geometric random variable with parameter 1 − (k − 1)/n and is independent of the earlier waiting times Xn,j , 1 ≤ j < k. Recall that if X is a geometric random variable with parameter p, then EX = 1/p and Var(X) = (1 − p)/p2 . Writing Tn = Xn,1 + · · · + Xn,n , we have n n X X k − 1 −1 1− ETn = = n k −1 ∼ n log(n) n k=1 k=1 −2 n n X X k−1 π2 2 Var(Tn ) ≤ 1− = n k −2 ≤ n2 . n 6 k=1 k=1 Taking bn = n log(n), Theorem (1.5.4) implies that P Tn − n nk=1 k −1 → 0 in probability n log(n) and so Tn /(n log(n)) → 1 in probability. 3.) Truncation We can use truncation to extend the weak law to random variables without a second moment. Theorem (1.5.5): Weak law for triangular arrays. For each n ≥ 1, let Xn,k , 1 ≤ k ≤ n be a collection of independent random variables. Let bn , n > 1 be a collection of real numbers with bn → ∞ and let X̄n,k = Xn,k 1(|Xn,k |≤bn ) . Suppose that as n → ∞ Pn 1. k=1 P(|Xn,k | > bn ) → 0, and Pn 2 2. b−2 n k=1 EX̄n,k → 0. P If Sn = Xn,1 + · · · + Xn,n and an = nk=1 EX̄n,k , then (Sn − an )/bn → 0 in probability. Proof: Let S̄n = X̄n,1 + · · · + X̄n,n . Then Sn − an S̄n − an > ≤ P(Sn 6= S̄n ) + P P bn > . bn 2 To estimate the first term on the RHS, note that n X P(Sn 6= S̄n ) ≤ P ∪nk=1 {X̄n,k 6= Xn,k } ≤ P(|Xn,k | > bn ) → 0 k=1 by (i). For the second term, we can use (ii) along with Chebyshev’s inequality and the fact that Var(Xn ) ≤ EXn2 to show that 2 S̄n − an −2 S̄n − an −2 −2 > ≤ E P bn = bn Var(S̄n ) bn n n X X −2 −2 = (bn ) var(X̄n,k ) ≤ (bn ) E(X̄n,k )2 → 0. k=1 k=1 Lemma (1.5.7): If Y ≥ 0 and p > 0, then Z ∞ p EY = py p−1 P(Y > y)dy. 0 Proof: Using Fubini’s Theorem, we can calculate Z ∞ Z ∞Z p−1 py P(Y > y)dy = py p−1 1(Y >y) dPdy 0 Ω Z0 Z ∞ py p−1 1(Y >y) dydP = Ω 0 Z Z Y Z p−1 = py dydP = Y p dP = EY p . Ω 0 Ω Theorem (1.5.6): Weak Law of Large Numbers. Let X1 , X2 , · · · be i.i.d. with xP(|X1 | > x) → 0 as x → ∞. Let Sn = X1 + · · · + Xn and µn = E X1 1(|X1 |≤n) . Then Sn /n − µn → 0 in probability. (?) Proof: We will apply Theorem (1.5.5) with Xn,k = Xk and bn = n. To check condition (i) in that theorem, observe that (?) implies that n X P(|Xn,k | > n) = nP(|X1 | > n) → 0. k=1 2 → 0. First observe that For condition (ii), we need to show that n−2 · nEX̄n,1 1 1 2 EX̄n,1 = n n Z 0 ∞ 1 2yP(|X̄n,1 | > y)dy ≤ n 3 Z n 2yP(|X1 | > y)dy 0 since P(|X̄n,1 | > y) = 0 for y ≥ n. To show that the last term tends to 0 as n → ∞, let g(y) = 2yP(|X1 | > y) and notice that 0 ≤ g(y) ≤ 2y and g(y) → 0 as y → ∞ imply that M = sup g(y) < ∞. If we define K = sup{g(y) : y > K}, then for n > K Z n 2yP(|X1 | > y)dy ≤ KM + (n − K)K . 0 Dividing by n and letting n → ∞ gives Z 1 n lim sup 2yP(|X1 | > y)dy ≤ K . n→∞ n 0 The result then follows upon noting that K → 0 as K → ∞. The familiar form of the weak law of large numbers is: Corollary (1.5.8): Let X1 , X2 , · · · be i.i.d. with E|Xi | < ∞. Let Sn = X1 + · · · + Xn and µ = EX1 . Then Sn /n → µ in probability. Proof: Chebyshev’s inequality and the dominated convergence theorem imply that xP(|X1 | > x) ≤ E |X1 |1(|X1 |>x) → 0 as x → ∞ µn = E X1 1(|X1 |≤n) → EX1 = µ as n → ∞. Using Theorem (1.5.6), we see that if > 0, then P(|Sn /n − µn | > /2) → 0. Since µn → µ, it follows that P(|Sn /n − µ| > ) → 0. 4