Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Mathematics for Computer Science MIT 6.042J/18.062J Deviation of Repeated Trials Copyright © Albert R. Meyer, 2005. All rights reserved. December 12, 2005 lec15M.1 Jacob D. Bernoulli (1659 – 1705) Copyright © Albert R. Meyer, 2005. All rights reserved. December 12, 2005 lec15M.2 Jacob D. Bernoulli (1659 – 1705) Even the stupidest man---by some instinct of nature per se and by no previous instruction (this is truly amazing) -- knows for sure that the more observations ...that are taken, the less the danger will be of straying from the mark. ---Ars Conjectandi (The Art of Guessing), 1713* *taken from Grinstead \& Snell, http://www.dartmouth.edu/~chance/teaching_aids/books_articles/probability_book/book.html Introduction to Probability, American Mathematical Society, p. 310. Copyright © Albert R. Meyer, 2005. All rights reserved. December 12, 2005 lec15M.3 Jacob D. Bernoulli (1659 – 1705) It certainly remains to be inquired whether after the number of observations has been increased, the probability…of obtaining the true ratio…finally exceeds any given degree of certainty; or whether the problem has, so to speak, its own asymptote---that is, whether some degree of certainty is given which one can never exceed. Copyright © Albert R. Meyer, 2005. All rights reserved. December 12, 2005 lec15M.4 Deviation from the Mean Pr{observed value far from expected value} is SMALL How small? Copyright © Albert R. Meyer, 2005. All rights reserved. December 12, 2005 lec15M.5 Deviation from the Mean Observed value means random variable, R. far from may mean: • distance or • amount above (or below) Copyright © Albert R. Meyer, 2005. All rights reserved. December 12, 2005 lec15M.6 Markov Bound far Copyright © Albert R. Meyer, 2005. All rights reserved. December 12, 2005 { { μ Pr{R above x} ≤ x small lec15M.7 Chebychev Bound Copyright © Albert R. Meyer, 2005. All rights reserved. far December 12, 2005 x 2 { distance { { Pr{|R–µ| > x} ≤ 2 small lec15M.8 Binomial Bound x 2 2 Pr{|Bn,p–µ| > x} e Copyright © Albert R. Meyer, 2005. All rights reserved. December 12, 2005 lec15M.9 Weak Law of Large Numbers An ::= Avg. of n independent trials ::= E[single trial] ? Copyright © Albert R. Meyer, 2005. All rights reserved. December 12, 2005 { { { lim P r{ An μ ε} 0 far distance n small lec15M.11 Jacob D. Bernoulli (1659 – 1705) Therefore, this is the problem which I now set forth and make known after I have pondered over it for twenty years. Both its novelty and its very great usefulness, coupled with its just as great difficulty, can exceed in weight and value all the remaining chapters of this thesis. Copyright © Albert R. Meyer, 2005. All rights reserved. December 12, 2005 lec15M.12 The Principle Behind: • Estimation (polling) • Algorithm analysis • Design against failure • Communication thru noise • Gambling Copyright © Albert R. Meyer, 2005. All rights reserved. December 12, 2005 lec15M.13 Not Usable as Stated Need to know the rate of convergence to 0 for any application. Copyright © Albert R. Meyer, 2005. All rights reserved. December 12, 2005 lec15M.14 Repeated Trials X1,, Xn independent 2 with mean, , and variance An ::= (X1 + + Xn)/n E[An] = n/n = Copyright © Albert R. Meyer, 2005. All rights reserved. December 12, 2005 lec15M.15 Repeated Trials Var[X1 + + Xn] = 2 n (by independence) Var[An] = 2 2 n /n σ n 2 decreases with # trials Copyright © Albert R. Meyer, 2005. All rights reserved. December 12, 2005 lec15M.16 Repeated Trials So by Chebychev 1 Pr{| An μ | ε} ( ε) n 2 as n Copyright © Albert R. Meyer, 2005. All rights reserved. 0 December 12, 2005 lec15M.17 Weak Law of Large Numbers Therefore lim P r{ An μ ε} 0 n QED Copyright © Albert R. Meyer, 2005. All rights reserved. December 12, 2005 lec15M.18