Download slides15m

Mathematics for Computer Science
MIT 6.042J/18.062J
Deviation of
Repeated Trials
Copyright © Albert R. Meyer, 2005. All rights reserved.
December 12, 2005
lec15M.1
Jacob D. Bernoulli (1659 – 1705)
Copyright © Albert R. Meyer, 2005. All rights reserved.
December 12, 2005
lec15M.2
Jacob D. Bernoulli (1659 – 1705)
Even the stupidest man---by some instinct of
nature per se and by no previous instruction
(this is truly amazing) -- knows for sure that
the more observations ...that are taken, the
less the danger will be of straying from the
mark.
---Ars Conjectandi (The Art of Guessing), 1713*
*taken from Grinstead \& Snell,
http://www.dartmouth.edu/~chance/teaching_aids/books_articles/probability_book/book.html
Introduction to Probability, American Mathematical Society, p. 310.
Copyright © Albert R. Meyer, 2005. All rights reserved.
December 12, 2005
lec15M.3
Jacob D. Bernoulli (1659 – 1705)
It certainly remains to be inquired whether
after the number of observations has been
increased, the probability…of obtaining the
true ratio…finally exceeds any given
degree of certainty; or whether the problem
has, so to speak, its own asymptote---that
is, whether some degree of certainty is
given which one can never exceed.
Copyright © Albert R. Meyer, 2005. All rights reserved.
December 12, 2005
lec15M.4
Deviation from the Mean
Pr{observed value far from
expected value}
is SMALL
How small?
Copyright © Albert R. Meyer, 2005. All rights reserved.
December 12, 2005
lec15M.5
Deviation from the Mean
Observed value means
random variable, R.
far from may mean:
• distance or
• amount above (or below)
Copyright © Albert R. Meyer, 2005. All rights reserved.
December 12, 2005
lec15M.6
Markov Bound
far
Copyright © Albert R. Meyer, 2005. All rights reserved.
December 12, 2005
{
{
μ
Pr{R above x} ≤
x
small
lec15M.7
Chebychev Bound
Copyright © Albert R. Meyer, 2005. All rights reserved.
far
December 12, 2005
x
2
{
distance
{
{
Pr{|R–µ| > x} ≤

2
small
lec15M.8
Binomial Bound
x 
2
2
Pr{|Bn,p–µ| > x}  e
Copyright © Albert R. Meyer, 2005. All rights reserved.
December 12, 2005
lec15M.9
Weak Law of Large Numbers
An ::= Avg. of n independent trials
::= E[single trial]
?
Copyright © Albert R. Meyer, 2005. All rights reserved.
December 12, 2005
{
{
{
lim  P r{ An  μ  ε}  0
far
distance
n
small
lec15M.11
Jacob D. Bernoulli (1659 – 1705)
Therefore, this is the problem which I
now set forth and make known after I
have pondered over it for twenty years.
Both its novelty and its very great
usefulness, coupled with its just as
great difficulty, can exceed in
weight and value all the remaining
chapters of this thesis.
Copyright © Albert R. Meyer, 2005. All rights reserved.
December 12, 2005
lec15M.12
The Principle Behind:
• Estimation (polling)
• Algorithm analysis
• Design against failure
• Communication thru noise
• Gambling
Copyright © Albert R. Meyer, 2005. All rights reserved.
December 12, 2005
lec15M.13
Not Usable as Stated
Need to know the
rate of convergence to 0
for any application.
Copyright © Albert R. Meyer, 2005. All rights reserved.
December 12, 2005
lec15M.14
Repeated Trials
X1,, Xn independent
2
with mean, , and variance 
An ::= (X1 +  + Xn)/n
E[An] = n/n = 
Copyright © Albert R. Meyer, 2005. All rights reserved.
December 12, 2005
lec15M.15
Repeated Trials
Var[X1 +  + Xn] =
2
n
(by independence)
Var[An] =
2
2
n /n
σ

n
2
decreases with # trials
Copyright © Albert R. Meyer, 2005. All rights reserved.
December 12, 2005
lec15M.16
Repeated Trials
So by Chebychev
1
Pr{| An  μ |  ε}  ( ε) 
n
2
as n  
Copyright © Albert R. Meyer, 2005. All rights reserved.
0
December 12, 2005
lec15M.17
Weak Law of Large Numbers
Therefore
lim  P r{ An  μ  ε}  0
n
QED
Copyright © Albert R. Meyer, 2005. All rights reserved.
December 12, 2005
lec15M.18