* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Handout 9 - UIUC Math
Survey
Document related concepts
Georg Cantor's first set theory article wikipedia , lookup
List of important publications in mathematics wikipedia , lookup
Fermat's Last Theorem wikipedia , lookup
Wiles's proof of Fermat's Last Theorem wikipedia , lookup
Four color theorem wikipedia , lookup
Nyquist–Shannon sampling theorem wikipedia , lookup
Brouwer fixed-point theorem wikipedia , lookup
Infinite monkey theorem wikipedia , lookup
Fundamental theorem of algebra wikipedia , lookup
Tweedie distribution wikipedia , lookup
Transcript
Limit Theorems Inequalities Proposition 1 (Markov Inequality) If X is a nonnegative random variable with mean µ, then for any a > 0, µ P(X ≥ a) ≤ . a Proposition 2 (Chebyshev Inequality) If X is a random variable with mean µ and variance σ 2 , then for any a > 0, σ2 P(|X − µ| ≥ a) ≤ 2 . a Example 3 Suppose that it is known that the number of items produced in a factory during a week is a random variable with mean 50. (a) What can be said about the probability that this week’s production will exceed 75? (b) If the variance of a weeks production is known to be 25, what can be said about the probability that this week’s production will between 40 and 60? Limit Theorems Theorem 4 (Weak Law of Large Numbers) Let X1 , X2 , . . . be a sequence of independent and identically distributed random variables with common mean µ. Then for any ² > 0, ¯ µ¯ ¶ ¯ X1 + · · · + Xn ¯ ¯ ¯ lim P ¯ − µ¯ ≥ ² = 0. n→∞ n Proof We will prove this theorem under the additional assumption that random variables has a finite variance σ 2 . Theorem 5 (Strong Law of Large Numbers) Let X1 , X2 , . . . be a sequence of independent and identically distributed random variables with common mean µ. Then, µ ¶ X1 + · · · + Xn P lim = µ = 1. n→∞ n Theorem 6 (Central Limit Theorem) Let X1 , X2 , . . . be a sequence of independent and identically distributed random variables with common mean µ and common variance σ 2 . Then the distribution of X1 + · · · + Xn − nµ √ σ n 1 converges to the standard normal distribution. That is, for any a ∈ R, µ ¶ Z a X1 + · · · + Xn − nµ 1 2 √ e−x /2 dx. lim P ≤a = √ n→∞ σ n 2π −∞ The key to the proof of the central limit theorem is the following result, which we state without proof. Proposition 7 Let Z1 , Z2 , . . . be a sequence of random variables with distribution functions FZn and moment generating functions MZn , n ≥ 1; let Z be a random variable with distribution function FZ and moment generating function MZ . If MZn (t) → MZ (t) for all t, then FZn (t) → FZ (t) for all t which is a continuous point for FZ . Proof of the Central Limit Theorem We will prove the theorem under the assumption that the moment generating function M (t) of Xi is finite for all t. √ First we assume that µ = 0 and σ 2 = 1. The moment generating function of Xi / n is t MXi /√n (t) = M ( √ ), n and thus µ MPni=1 Xi /√n (t) = ¶n t M(√ ) . n Put L(t) = ln M (t). Then we can easily get that L(0) = 0, L0 (0) = 0 and L0 (0) = 1. To prove the theorem, we must show that µ ¶n t2 t M(√ ) →e2, n or equivalently √ t2 nL(t/ n) → . 2 By using L’Hopital’s rule twice, we get √ √ √ L(t/ n) −L0 (t/ n)n−3/2 t L0 (t/ n)t = lim = lim lim nL(t/ n) = lim n→∞ n→∞ n→∞ n→∞ 2n−1/2 n−1 −2n−2 √ −3/2 2 2 2 00 √ t −L (t/ n)n t t = lim = lim L00 (t/ n) = . −3/2 n→∞ n→∞ 2 2 −2n √ Thus the central limit theorem is true when µ = 0 and σ 2 = 1. The general case follows by fi = (Xi − µ)/σ, since E[X fi ] = 0 and Var(X fi ) = 1. considering the random variables X Applications of the Central Limit Theorem Example 8 A certain basketball player makes 80 percent of his free throws on average. What is the probability that in 100 attempts he will be successful more than 85 times? Assume independence. Solution In this case n = 100 and each Xi is a Bernoulli random variable with parameter .8. Thus µ = .8 and σ 2 = .16. Consequently by the central limit theorem ¶ µ Pn n n X X 85.5 − 80 i=1 Xi − 80 ≈ 1 − Φ(1.375) ≈ .085. P( ≥ Xi > 85) = P( Xi ≥ 85.5) = P 4 4 i=1 i=1 2 Example 9 The number of students enrolled in a calculus class is a Poisson random variable with parameter 100. If the number enrolling is 120 or more, the class will be taught in two sections, otherwise it will be taught in one section. Find the probability that the class has to be taught in two sections. Solution The exact solution ∞ X (100)i e−100 i! i=120 does not readily yields a numerical answer. However, recalling that a Poisson random variable with parameter 100 is the sum of 100 independent Poisson random variables with parameter 1, we can use the central limit theorem to find an approximate answer. If X denote the number of students enrolled in the course, then µ ¶ X − 100 119.5 − 100 P(X ≥ 120) = P(X ≥ 119.5) = P ≥ ≈ 1 − Φ(1.95) ≈ .0265. 10 10 Example 10 Let Xi , i = 1, 2, . . . , 48 be independent random variables, each uniformly distributed P in (0, 1). Find an approximation to P( 48 i=1 Xi > 27). Solution Since E[Xi ] = 1/2 and Var(Xi ) = 1/12, we have by the central limit theorem ! ÃP 48 48 X X − 24 27 − 24 i i=1 P( > Xi > 27) = P ≈ 1 − Φ(1.5) ≈ .0668. 2 2 i=1 Applications to Polling The central limit theorem can be regarded as a refinement of the weak law of large numbers. In fact, let X1 , X2 , . . . be a sequence of independent and identically distributed random variables with P mean µ and variance σ 2 . Put Sn = ni=1 Xi . Then by the central limit theorem, for any c > 0, ¯ µ¯ ¶ ¯ Sn ¯ ¯ ¯ P ¯ − µ¯ ≥ c = P(Sn ≤ nµ − nc) + P(Sn ≥ nµ + nc) n µ ¶ µ ¶ nc nc √ ≈ Φ − √ +1−Φ σ n σ n µ µ √ ¶¶ c n . = 2 1−Φ σ In other words ¯ µ¯ ¶ ¯ Sn ¯ ¯ ¯ P ¯ − µ¯ ≥ c ≈ 2(1 − Φ(δ)), n where δ= √ c n . σ (0.1) (0.2) Example 11 A sample of size n is to be taken to determine the percentage of the population planning to vote a certain candidate A in an upcoming election. Let Xi = 1 if the i-th person sampled plans to vote for A and Xi = 0 otherwise. We assume that X1 , X2 , . . . , Xn are independent and identically distributed with P(Xi = 1) = p and P(Xi = 0) = 1−p. Then µ = p and σ 2 = p(1−p). p WE also assume that p is close to .5 so that σ = p(1 − p) can be approximated satisfactorily by σ ≈ 1/2 (note that σ has a maximum of 1/2 at p = .5, and that, as p ranges over .3 ≤ p ≤ .7, 3 σ stays above .458 which is close to 1/2). The random variable Sn /n denotes the fraction of the people sampled that pan to vote for A and can be used to estimate the true unknown probability p. We will use the central limit theorem to solve the following three problems: (i) Suppose n = 900. Find the probability that ¯ ¯ ¯ Sn ¯ ¯ ¯ ≥ .025. − p ¯n ¯ (ii) Suppose n = 900. Find c such that ¯ ¶ µ¯ ¯ Sn ¯ ¯ ¯ P ¯ − p¯ ≥ c = .01. n (iii) Find n such that ¯ ¶ µ¯ ¯ ¯ Sn ¯ ¯ − p¯ ≥ .025 = .01. P ¯ n Solution (i) By (0.2) so by (0.1) √ (.025) 900 = 1.5, δ= .5 ¯ µ¯ ¶ ¯ Sn ¯ P ¯¯ − p¯¯ ≥ .025 ≈ 2(1 − Φ(1.5)) = .134. n (ii) We first choose δ so that 2(1 − Φ(δ)) = .01 or φ(δ) = .995. Using the table we know that δ = 2.58. Solving (0.2) for c we get δσ (2.58)(.5) c= √ = √ = .043. n 900 (iii) As in (ii) we have δ = 2.58. Solving (0.2) for n we find n= δ2σ2 (2.58)2 (.25) = = 2663. 2 c (.025)2 Remark on Notations 4