Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Law of Large Numbers • Toss a coin n times. • Suppose 1 Xi 0 if i th toss came up H if i th toss came up T • Xi’s are Bernoulli random variables with p = ½ and E(Xi) = ½. 1 n • The proportion of heads is X n X i . n i 1 • Intuitively X n approaches ½ as n ∞ . week 12 1 Markov’s Inequality • If X is a non-negative random variable with E(X) < ∞ and a >0 then, P X a EX a week 12 2 Chebyshev’s Inequality • For a random variable X with E(X) < ∞ and V(X) < ∞, for any a >0 P X E X a V X a2 • Proof: week 12 3 Back to the Law of Large Numbers • Interested in sequence of random variables X1, X2, X3,… such that the random variables are independent and identically distributed (i.i.d). Let 1 n Xn Xi n i 1 Suppose E(Xi) = μ , V(Xi) = σ2, then 1 n 1 n E X n E X i E X i n i 1 n i 1 and 1 n 1 V X n V X i 2 n i 1 n n V X i 1 i 2 n • Intuitively, as n ∞, V X n 0 so X n E X n week 12 4 • Formally, the Weak Law of Large Numbers (WLLN) states the following: • Suppose X1, X2, X3,…are i.i.d with E(Xi) = μ < ∞ , V(Xi) = σ2 < ∞, then for any positive number a P Xn a 0 as n ∞ . This is called Convergence in Probability. Proof: week 12 5 Example • Flip a coin 10,000 times. Let 1 Xi 0 if i th toss came up H if i th toss came up T • E(Xi) = ½ and V(Xi) = ¼ . • Take a = 0.01, then by Chebyshev’s Inequality 1 1 1 1 P X n 0.01 2 2 4 410,000 0.01 • Chebyshev Inequality gives a very weak upper bound. • Chebyshev Inequality works regardless of the distribution of the Xi’s. week 12 6 Strong Law of Large Number • Suppose X1, X2, X3,…are i.i.d with E(Xi) = μ < ∞ , then X n converges to μ as n ∞ with probability 1. That is 1 P lim X 1 X 2 X n 1 n n • This is called convergence almost surely. week 12 7 Continuity Theorem for MGFs • Let X be a random variable such that for some t0 > 0 we have mX(t) < ∞ for t t0 , t0 . Further, if X1, X2,…is a sequence of random variables with m X n t and lim m X n t m X t for all t t0 , t0 n then {Xn} converges in distribution to X. • This theorem can also be stated as follows: Let Fn be a sequence of cdfs with corresponding mgf mn. Let F be a cdf with mgf m. If mn(t) m(t) for all t in an open interval containing zero, then Fn(x) F(x) at all continuity points of F. • Example: Poisson distribution can be approximated by a Normal distribution for large λ. week 12 8 Example to illustrate the Continuity Theorem • Let λ1, λ2,…be an increasing sequence with λn ∞ as n ∞ and let {Xi} be a sequence of Poisson random variables with the corresponding parameters. We know that E(Xn) = λn = V(Xn). X E X n X n n • Let Z n n then we have that E(Zn) = 0, V(Zn) = 1. V X n n • We can show that the mgf of Zn is the mgf of a Standard Normal random variable. • We say that Zn convergence in distribution to Z ~ N(0,1). week 12 9 Example • Suppose X is Poisson(900) random variable. Find P(X > 950). week 12 10 Central Limit Theorem • The central limit theorem is concerned with the limiting property of sums of random variables. • If X1, X2,…is a sequencen of i.i.d random variables with mean μ and variance σ2 and , S X n i 1 i then by the WLLN we have that Sn in probability. n • The CLT concerned not just with the fact of convergence but how Sn /n fluctuates around μ. • Note that E(Sn) = nμ and V(Sn) = nσ2. The standardized version of Sn is S n n Zn and we have that E(Zn) = 0, V(Zn) = 1. n week 12 11 The Central Limit Theorem • Let X1, X2,…be a sequence of i.i.d random variables with E(Xi) = μ < ∞ and Var(Xi) = σ2 < ∞. Suppose the common distribution function FX(x) and the common moment generating function mX(t) are defined in a neighborhood of 0. Let n Sn X i i 1 Then, lim P S n n x x for - ∞ < x < ∞ n n where Ф(x) is the cdf for the standard normal distribution. • This is equivalent to saying that Z n S n n converges in distribution to n Z ~ N(0,1). • Xn P x x Also, lim n n i.e. Z n X n converges in distribution to Z ~ N(0,1). n week 12 12 Example • Suppose X1, X2,…are i.i.d random variables and each has the Poisson(3) distribution. So E(Xi) = V(Xi) = 3. • The CLT says that P X 1 X n 3n x 3n x as n ∞. week 12 13 Examples • A very common application of the CLT is the Normal approximation to the Binomial distribution. • Suppose X1, X2,…are i.i.d random variables and each has the Bernoulli(p) distribution. So E(Xi) = p and V(Xi) = p(1- p). • The CLT says that P X 1 X n np x np1 p x as n ∞. • Let Yn = X1 + … + Xn then Yn has a Binomial(n, p) distribution. So for large n, PYn y P Yn np np1 p y np y np np1 p np1 p • Suppose we flip a biased coin 1000 times and the probability of heads on any one toss is 0.6. Find the probability of getting at least 550 heads. • Suppose we toss a coin 100 times and observed 60 heads. Is the coin fair? week 12 14