Download Weak law of large numbers

Weak law of large numbers
Dean P Foster
October 17, 2011
Administrivia
• In class exam next Monday
• Last section of Helms to read: 4.4
Definition: Variance
• V ar(X) = E(X − µ)2 where µ = E(X)
• Mathemtically better than SD or E(|X − µ|)
• Draw some pictures
Covariance
• Cov(X, Y ) = E(X − µX )(Y − µY )
• If X ⊥ Y , then Cov(X, Y ) = 0. Not the other way around!
(Need all polynomials, not just easy one.)
1
• V ar(X + Y ) = V ar(X) + V ar(Y ) + 2 ∗ Cov(X, Y )
• V ar(X + Y ) = Cov(X + Y, X + Y ) = Cov(X, X) + Cov(X, Y ) +
Cov(X, Y ) + Cov(Y, X)
P
P
• V ar( Xi ) = i,j Cov(Xi , Xj )
IID
• independent
• identically distributed
• IID for short
• {Xi }ni=1
• Interested in
P
Xi
• interseted in average X
Computing means and variances of sums
Consider sums of IID random variables:
P
• Compute E( X) (linearity)
P
• Compute E( X)2 (algebra)
P
• Compute V ar( X)2 (easy by covariance formula)
2
Summaries
Let Xi be a sequence of IID random variables. Let S =
Pn
i=1 Xi .
Then:
E(S) = nE(X)
V ar(S) = nV ar(X)
E(S s ) = φS (s) = φX (s)n
If the RHS’s exist.
Application
Plug in the blanks:
P (S > nk) ≤ E(X)/k if X ≥ 0
P (|S − nµ| > k) ≤ nV ar(X)/k 2
P (S > k) ≤ MX (1)−k/n
Patience
Keynes: “In the long run, we’re all dead.”
Singularity: “Maybe.”
If ΩM = 6, the universe will crush itself in 15 billion years.
If ΩM > 1, sometime we will get crushed.
Proton’s decay maybe at 1033 years.
(Measured value is about .05 to .3, so no crush iminent.)
Heat death: 10100 years.
Not even close to the limit. Let’s think of a really big number
compared to 10100 , how about 100 ∗ 10100 = 10102 . Lots more where
that came from.
3
Limit theorem: Law of large numbers
We want to talk about a limit theorems.
Theorem 1 (SLLN) lim X n = µ, where X n =
IID and E(X) = µ.
Pn
Xi /n and Xi are
Problem: Why does it matter? Won’t we be dead before it comes
into play?
Sample paths
What does rn → r∞ mean?
• Statement about where rn isn’t found.
• Excludes two rectangles
• Plot on “arc-tan scale”
• Draw / δ statement of limit
Pictures of average
• Sketch picture using whole black board
• Compress using arc-tan say
• Draw arbitary boxes.
• Where do these boxes lie? Hard question.
• Picture via top half of “log / log” plot
4
• Ah–nice straight line, almost
Theorem 2 (Iterated logorithm) IID, µ = 0, σ finite:
p
lim sup X n /(σ 2n log(log n)) = 1
Key insight
• Useful to give concrete bounds at fixed times
• Can’t say “always”
• So give probability at fixed time
• Most often stated as “weak law”
Two versions of weak law
Theorem 3 (WLLN) P (|X n − µ| ≥ ) → 0 (if IID, σ < ∞, µ =
E(X))
Proof:
• V (X n ) = σ 2 /n
• E(X n ) = µ
• P (|X n − µ| ≥ ) < σ 2 /(n2 ) → 0
qed
Theorem 4 (Concrete) P (|X n −µ| ≥ ) ≤ σ 2 /(n2 ) (if IID, σ < ∞,
µ = E(X))
5
Cauchy
f (x) = 1/π(1 + x2 )
Conclusions: Good limits / bad limits
Theorem: All population models which have a cap on the total size of
the population, will go extenct at some point.
Proof: Model has a chance of each individual dieing. If they all die
at the exact same time extention occurs. The probabily of that one
dies is > 0. The probability that all die is n . So, the probabilty of
making it past time T is less than (1 − n )T → 0.
• Not interesting since there is a lot of interesting stuff that can
happen before limit sets in
• other effects will make an interesting story first
• Model will break down long before limit sets in
Good limit:
• Ideally concrete bounds
• Model holds up to expected time of limit being approximately
true
• Other effects don’t take over first
Conclusions:
• Law of large number as typically stated is ambigious about whether
it is a good limit or a bad limit
• Concrete version shows it to be a good limit
• Weak law is very good limit theorem
6
Un used material
Recall:
Let Xi be a sequecne of IID random variables. Let S =
Pn
i=1 Xi .
Then:
E(S) = nE(X)
V ar(S) = nV ar(X)
MS (1) = MX (1)n
Gambling
• The setup:
– Suppose round i you bet fraction of your wealth on gamble
Yi .
– If it is a fair bet, then your gain is W E(Y ) (which is a
ranodm variable!) or just zero.
– Suppose your initial wealth is 1.
– What is your expected wealth at time T ? Also 1.
• So, by markov P (W > k) ≤ 1/k.
• Converting to sums:
– But if your bet either pays out or doesn’t pay out.
– Let S be the number of times it pays out.
– Then W = (1 + a)S (1 − b)n−S .
7
• Plugging in:
P (W > k) = P ((1 + a)S (1 − b)n−S > k)
S
1 + a
= P(
(1 − b)n > k)
1−
S
= P (α > k(1 − b)−n )
= P (S log(α) > log(k(1 − b)−n ))
= P (S > log(k(1 − b)−n )/ log(α))
≤ 1/k
Writing this differently, let
v
log(α)v
αv
(1 − b)n αv
=
=
=
=
log(k(1 − b)−n )/ log(α)
log(k(1 − b)−n )
k(1 − b)−n
k
So,
P (S > v) ≤ α−v /(1 − b)n
8