Download Chapter 10 – Statistical Estimation

Chapter 10 – Statistical Estimation
Keep in mind the distinction between a SAMPLE and a POPULATION.
Remember that this shows up in notation as the difference between Roman and Greek letters.
Sample mean = X-bar; Sample standard deviation = s
Population mean = mu; Population standard deviation = sigma
Statistical inference: drawing conclusions about the population on the basis of a sample.
A statistic is the number (mean, variance, slope of regression line) calculated on the basis of a sample; statistical
inference will enable us to conclude how accurately this statistic is likely to reflect the unknown truth about the
population.
The variance of the statistic is the key ingredient in statistical inference. Note that this is NOT the same
as the variance of the population or the variance of the sample. (not the same... NOT ... NOT !!!
Not making this distinction is the primary reason for mistakes in this material.)
Statistics are themselves random variables. Their exact value depends on the sample we have
randomly chosen from the population. There is not “a sample mean” in the same sense as there is “a population
mean” -- we will get a different sample mean every time we take a different sample.
To interpret statistics correctly, you need to know:
a. Whether they are unbiased -- if their expected value is the same as that of the population parameter.
The next best thing to being unbiased is being consistent, that is, converging to the population parameter
given enough observations.
b. What the variance of the statistic or the standard error of the statistic is. [Note that the phrase is
“standard error” rather than “standard deviation”, though both mean the square root of the variance].
It is nice if you have a statistic which is both unbiased and minimum variance, but it is not always possible to
meet both criteria, and there is no automatic rule about which is more desirable.
The Central Limit Theorem (183-188) assures us that whatever the distribution of the population, the
distribution of the mean (and other statistics) tends towards the normal distribution as the sample size
becomes larger.
For a demonstration of the Central Limit Theorem, you can run the EcLS programs
(cltdemo)
which will allow you to choose a sample size (choose the default value of 50 to begin)
and a distribution to generate the data.
Unsurprisingly, normal and Student's t distributions for the data result in normal distributions of the
mean; more surprisingly, uniform and chi-squared distributions, which are not close to normal in appearance,
have a normal distribution of means even for fairly small samples.
Even very highly skewed distributions, such as that resulting from squaring the data generated by a chisquared distribution, will have a normal distribution if you take a large sample (but not in small or medium sized
samples; the Central Limit Theorem, as the name suggests, holds in the limit, and you should be cautious in
relying on it in very highly skewed distributions).
(cltdemo2) provides another, somewhat more automated, demonstration of the Central Limit Theorem,
which will allow you to choose a distribution, then automatically draw a graph of means for different sample
sizes. Close the graphs of means in turn to see the gradual convergence of the means to a normal distribution and
to a smaller and smaller variance as the sample size increases.
Chapter 10 -- Homework Problems
Problem 1. The random variable X is distributed N (7, 3).
Take a random sample of 16 observations and caculate the sample mean Xbar.
What is the expected value and variance of the sample mean?
The expected value of the sample mean is the expected value of the population.
The variance of the sample mean is the variance of the population divided by the number of observations
Here Var(xbar) = 9 / 16, so SE mean = 3 /4.
Given these values, and the fact that the Central Limit Theorem indicates that we can use the normal distribution
when calculating the probability of getting a sample mean in any interval, we can calculate the probability that
Xbar will be in the interval:
INTERVAL
Z-Scores
(normal-cdf upper) - (normal-cdf lower)
Probability
7 to 7.5
0 and (7.5 - 7) / 0.75
Pr (Z < 2/3) - Pr (Z < 0) = .7486 - .5000
= .2486
7.5 to 8.0
2/3 and 4/3
Pr(Z < 4/3) - Pr (Z < 2/3) = .9082 - .7486
= .1596
8.0 to 8.5
4/3 and 2.25
Pr (Z < 2.25) - Pr (Z < 4/3) = .9878 - .9082 = .0796
8.5 to 9.0
2.25 and 3.0
Pr (Z < 3.0) - Pr (Z < 2.25) = .9987 - .9878 = .0009
9.0 to 9.5
3.0 and 3.75
Pr (Z < 3.75) - Pr (Z < 3.0) = .9999 - .9987 =.0012
[This last required the computer to compute (normal-cdf 3.75).
Problem 2. Given that X1, X2 and X3 are three independent, normal random variables which share a common
mean (mu) and standard deviation (sigma) -- as they would if they were three independent draws from a
distribution with mean mu and standard deviation sigma.
Use the weighted average V = 0.750 X1 + 0.125 X2 + 0.125 X3 as an estimator of the population mean.
Is V unbiased?
YES. Proof:
E(V) = 0.750 E(X1) + 0.125 E(X2) + 0.125 E(X3) = 0.750 mu + 0.125 mu + 0.125 mu = mu
Is V a minimum variance estimator? Compare it to the standard average Xbar = (X1 + X2 + X3) / 3.
Variance of V =
(0.750 squared) * Var (X1) + (0.125 squared) * Var (X2) + (0.125 squared) Var (X3)
= (0.5625) Var X + (0.015625) Var X + (0.015625) Var X = 0.59375 Var X.
Alternative estimator Xbar has a smaller variance:
(1/9) Var X1 + (1/9) Var X2 + (1/9) Var X3 = Var X / 3 = 0.33333 Var X
Problem 3. Again, take X1, X2 and X3 as independent, normal random variables which share a common mean
mu and standard deviation (sigma).
This time, take the estimator W = 1/2 X1 + 1/2 X2 + 1/2 X3.
Is W unbiased? No; its expected value is E(W) = 0.5 E(X1) + 0.5 E(X2) + 0.5 E(X3) = 1.5 mu
Variance of W = 1/4 Var X + 1/4 Var X + 1/4 Var X = 0.75 Var X.
Problem 4. Let X be a binomial random variable for N trials, each with probability of success p.
Is p-hat = S / N, where S is the number of successes in N trials, an unbiased estimate of the true probability p ?
Yes, since E (p-hat) = E (Xi / n) = E(Xi) / n = np / n (since E(Xi) = np, see p. 147) and of course np/n = p.
Hence E(p-hat) = p
Problem 5. What is the variance of p-hat in the last problem?
For each of the N trials, we have Var(p-hat) = Var (X/n) = Var(X/n-squared)
= np (1-p) / n-squared = p (1-p) / n
Problem 6. Is taking two observations on the binomial random variable X sufficient to have an unbiased
estimator?
Yes: [ E(X1) + E(X2)] /2 = 2p / 2 = p
But the variance of this estimator is p (1-p) / 2; with three observations the variance would be p (1-p) / 3
and so forth.