Survey

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Document related concepts

Transcript

THE UNIVERSITY OF HONG KONG DEPARTMENT OF STATISTICS AND ACTUARIAL SCIENCE STAT1301 PROBABILITY AND STATISTICS I, FALL 2010 EXAMPLE CLASS 10 Convergence Modes and Limit Theorems Elements of Theory 1. Infinite Sequence of random variables and Its Partial Sum Sequence +. An infinite sequence of random variables is an infinite sequence Infinite sequence of random variables : * + where of functions. The partial sum sequence associated with the infinite sequence : * ∑ is called the -th partial sum of the sequence * functions. +. The partial sum sequence is also an infinite sequence of 2. The Three Modes of Probabilistic Convergence ( ) ( ) ( ) Special Notations: → where should be replaced by one of the following symbols or abbreviations: mode notation meaning (sure) convergence → almost surely, with probability 1 in probability in law, in distribution, or weakly → mathematical definition .{ }/ ,| | ( ) ( ) Convergence in Probability to a constant and Convergence in Distribution to a constant (singleton) for some constant . 3. Independent and Identically Distributed Sequence and Basic Statistical Notions If the ’s are independent and identically distributed as a same distribution (known or unknown), then * is called an independent and identically distributed (i.i.d.) sequence of random variables. + An iid collection of random variables is the prime device of statistics. In the population-sample language of statistics, the collection * + is called a sampling frame of size ; +, is called a sample; the realized values of the collection, denoted as * the common distribution that all ’s follow is called the population. The fundamental problem of statistics is to infer properties of the unknown population + obtained under the sampling frame * properties of the many samples * The sample mean ̅ is a random variable based on the sampling frame * by investigating the +. +, defined as ̅ The sample mean depends on the size . As the size increases, we obtain a sequence of sample means vividly depicted as the “running sample mean”. The Laws of large numbers concern the convergence of the running sample mean. The Central limit theorem concerns the limit distribution of the (standardized) sample mean. All under the iid condition. 4. Weak and Strong Laws of Large Numbers (for iid r.v. sequence) Both versions of LLN state that the limit of the rolling sample mean is the population expectation. Weak Law of Large Numbers: Running Sample Mean Converges to Population Expectation with high probability. Strong Law of Large Numbers: Running Sample Mean Converges to Population Expectation with probability 1 (almost surely). The Strong Law implies the Weak Law because a.s. convergence implies in-probability convergence. The Weak law nevertheless occupies important position in probability and statistics and can be called “the fundamental theorem of statistics” for the reason next:The difference between the two laws could be heuristically understood as how long one should wait to see the effect of the respective law to manifest: After fixing an , you need to wait much longer to see the strong law effect than to see the weak law effect. Formal representation of the Laws of Large Numbers + be an i.i.d. sequence of random variables. The requirement ( ) ( Let * Now the condition is enough for us to summon both versions of the law of large numbers: ) is necessary. Weak version (*| ̅ ̅ +) | Remark: The Weak Law is about convergence in probability to the constant , thus it is equivalent to convergence in distribution to that constant (singleton). Strong version .{ ̅ }/ ̅ → 5. Central Limit Theorem (for iid r.v. sequence) CLT states that the standardized rolling sample mean’s limiting distribution is the standard normal distribution. Formal representation of CLT Let * ( + be an i.i.d. sequence of random variables following any distribution. The conditions of ( ) ) and ( ) are enough to summon the CLT: ̅ ( ) √ Problems . 1. (December 2009 Exam Q4) Let (a) Compute ( ) and (b) What distribution does (c) Compute ( ) and (d) For ( /. Then converges in distribution to ; i.e., . )? [5 marks] have? (For full credit, be sure to provide the values of all parameters.) [5 marks] ( ) [5 marks] relatively large, give an approximation for ( ) that does not depend on . [5 marks] 2. Markov’s and Chebyshev’s Inequalities and WLLN 1) An archer is aiming at a circular target of radius 20 inches. Her arrows hit on average 5 inches away from the center, each shot being independent. Show that the next arrow will miss the target with probability at most . 2) Let be a sequence of iid random variables and assume the population expectation and variance are finite and equals to and , respectively. Prove WLLN using the Chebyshev’s Inequality, that is, show the following: 3) A fair coin is tossed independently times. Let be the number of heads obtained. Use the Chebyshev inequality to find a lower bound of the probability that , and (iii) differs from by less than when (i) , (ii) . 3. True or False a) Convergence in distribution concerns the convergence of a function sequence while almost sure convergence concerns only the convergence of a point sequence. b) Almost sure Convergence is also known as almost everywhere convergence. c) Convergence in distribution implies almost sure convergence. d) Convergence in probability implies convergence in distribution. e) Convergence in distribution to a constant implies convergence in probability to that same constant. f) If a sequence of functions does not converge, the sequence must not converge almost surely. g) If a sequence of functions does not converge in distribution, the sequence must not converge. h) Both the Strong Law of Large Numbers and the Central Limit Theorem imply the Weak Law of Large Number. i) In most practical repetitive random phenomena, the effect of the Weak Law of Large Numbers manifests much earlier than does the effect of the Strong Law of Large Numbers. j) The Central Limit Theorem is a consequence of the de Moivre-Laplace theorem; this means the central limit theorem describes a nature of a sequence of Bernoulli trials. k) The Central Limit Theorem is a theorem concerning the limiting distribution of iid sum. l) When the population variance is finite, we can use Markov’s inequality to prove the Weak Law of Large Numbers and in that case the independence requirement for the sequence cannot be relaxed to the requirement of pairwisely uncorrelated sequence. m) The Poisson distribution cannot be the distribution of any iid sum and therefore we cannot use the Normal distribution to approximate a Poisson distribution. n) We can use the Normal distribution to approximate a Gamma distribution; this is directly because the Gamma distribution is independently additive in its first parameter. o) The Gamma distribution is the inter-arrival time distribution of a Poisson process. The exponential distribution is the waiting time distribution of a Poisson process. p) We can use the Normal distribution to approximate a Negative Binomial distribution because the Negative Binomial distribution can be expressed as the iid sum of Geometric distribution. q) The 4. CLT distribution equals in distribution the sum of independent ’s WLLN Show that CLT implies WLLN, under iid and finite population expectation and variance conditions. 5. CLT formal exercise Demonstrate that , .{ }/ ( √ ) ∫ 6. Elementary Statistics Estimate the probability that the number of heads in 10,000 independent tosses of a fair coin differs by less than 1% from 5000. What is the probability that the number of heads will be greater than 5100? How many independent tosses of a fair coin are required for the probability that the average number of heads differs from 0.5 by less than 1% to be at least 0.99?