Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
What if there was no Law? What if There Were No Law of Large Numbers? We said that the Law of Large Numbers applies whenever we make independent observations on a random variable X that has an expected value. In those cases the Law guarantees that the sequence of sample means will eventually converge to the expected value of the random variable, E(X), or the "mean of the distribution." But not all distributions have an expected value. Here is an example of one that does not: 0.0 0.1 f(x) 0.2 0.3 Probability Density Function: Cauchy Distribution (2.5) -5 0 5 10 x Authors: Blume, Greevy Bios 311 Lecture Notes Page 1 of 9 What if there was no Law? This is the density function of a random variable that has a "Cauchy" probability distribution. Here it is again, this time with the normal density with mean 2.5 and standard deviation 1.5 shown for comparison. 0.0 0.1 f(x) 0.2 0.3 Probability Density Function: Cauchy Distribution (2.5,1.5) with Normal(2.5, SD=1.5) -5 0 5 10 x The Cauchy probability density function is f (x) = 1 (1 + (x )2 ) < x< The density is centered at θ ( θ = 2.5 in the above graphs), and θ is the median of the Cauchy distribution. Authors: Blume, Greevy Bios 311 Lecture Notes Page 2 of 9 What if there was no Law? If this distribution did have an expected value, that expected value would also equal θ, the center of the distribution. But it does not. In this case (Cauchy distribution) the integral that we use to define the expected value of X does not exist. E( X ) = x f (x) dx = ??? Big deal? So what? Let's see what happens if we repeat the experiment that we used to demonstrate the Law of Large Numbers in action: Generate a sequence of independent random variables, all with this Cauchy distribution (centered at 2.5), and look at the sequence of sample means. Authors: Blume, Greevy Bios 311 Lecture Notes Page 3 of 9 What if there was no Law? Here are the first 10 means: 2 4 6 8 10 Sample Means for a Sequence of IID Random Variables Cauchy(2.5,1) Probability Distribution 0 last mean= 3.25 2 4 6 8 10 sample size 8 10 Sample Means for a Sequence of IID Random Variables Cauchy(2.5,1) Probability Distribution 0 2 4 6 last mean= 5.35 0 20 40 60 80 100 sample size Authors: Blume, Greevy Bios 311 Lecture Notes Page 4 of 9 What if there was no Law? Here is 1000 observations: 2 4 6 8 10 Sample Means for a Sequence of IID Random Variables Cauchy(2.5,1) Probability Distribution 0 last mean= 3.05 0 200 400 600 800 1000 sample size Close enough you say? Here is another 1000: 0 2 4 Sample Means for a Sequence of IID Random Variables Cauchy(2.5,1) Probability Distribution -10 -8 -6 -4 -2 last mean= 1.16 0 200 400 600 800 1000 sample size Authors: Blume, Greevy Bios 311 Lecture Notes Page 5 of 9 What if there was no Law? Here is another 1000: -6 -4 -2 0 2 4 Sample Means for a Sequence of IID Random Variables Cauchy (2.5,1) Probability Distribution -8 last mean= -7.57 0 200 400 600 800 1000 sample size And here is 10,000: 2 4 6 Sample Means for a Sequence of IID Random Variables Cauchy (2.5,1) Probability Distribution -2 0 last mean= -0.0794 0 2000 4000 6000 8000 10000 sample size Authors: Blume, Greevy Bios 311 Lecture Notes Page 6 of 9 What if there was no Law? The sequence of sample means never settles down. It does not converge to the value 2.5 (or to any other value). In this case, increasing the sample size does nothing because we cannot learn about E(X) by watching the sequence of sample means. Does the fact that the sequence of means will never settle down and eventually leads us to θ mean that we cannot learn about the value of this parameter (θ) by making more and more independent observations on the Cauchy distribution? No. We must look for a different way to estimate θ because the sample mean doesn't work. Here's an idea -- the parameter we want to estimate is the median of the distribution. How about estimating it by the sample median? If instead of the sample means we look at the sequence of sample medians, everything is OK. Authors: Blume, Greevy Bios 311 Lecture Notes Page 7 of 9 What if there was no Law? Here is the first 100 sample medians for the previous sequence of 10,000 Cauchy(2.5,1) random variables. 3 4 Sample Means for a Sequence of IID Random Variables Cauchy (2.5,1) Probability Distribution 0 1 2 last median= 2.67 0 20 40 60 80 100 sample size We only need 1,000 observations to see that the Law is again working: 3 4 Sample Means for a Sequence of IID Random Variables Cauchy (2.5,1) Probability Distribution 0 1 2 last median= 2.42 0 200 400 600 800 1000 sample size Authors: Blume, Greevy Bios 311 Lecture Notes Page 8 of 9 What if there was no Law? The explanation for why the sequence of sample medians goes to the median of the distribution is also found in the Law of Large Numbers. 3 4 Sample Means for a Sequence of IID Random Variables Cauchy (2.5,1) Probability Distribution 0 1 2 last median= 2.45 0 200 400 600 800 1000 sample size 3 4 Sample Means for a Sequence of IID Random Variables Cauchy (2.5,1) Probability Distribution 0 1 2 last median= 2.48 0 200 400 600 800 1000 sample size Authors: Blume, Greevy Bios 311 Lecture Notes Page 9 of 9