Download Chapter 7 Sampling and Point Estimation

Chapter 7
Sampling and Point Estimation
Sample This
Chapter 7A
This Week in Prob & Stat
today
fine print warning: while today’s presentation is mostly conceptual,
Thursday’s presentation will be much more mathematical.
7-1 Introduction
• The field of statistical inference consists of those methods used to
make decisions or to draw conclusions about a population.
• These methods utilize the information contained in a sample from the
population in drawing conclusions.
• Statistical inference may be divided into two major areas:
• Parameter estimation
• Hypothesis testing
Fundamental Problem: Given that X1, X2, …, Xn is a random sample
from some unknown population, what can be said about the population?
For example, what is the distribution, mean, variance, median, range, etc.
The Big Picture Again
deduction
sample
population
descriptive
statistics
parameter
(e.g. mean)
probability
theory
descriptive
statistics
statistic
(e.g. sample
mean)
induction (inferential statistics)
Statistics and Sampling

Statistical Inference:




Draw conclusions about a population based on sample.
Hypothesis tests and parameter estimation.
Population:
 Generally impossible or impractical to observe
an entire population.
 Be aware that population may change over time.
Sample:
 A subset of observations from a population.
 Must be representative of the population.
 Must be chosen randomly to avoid bias.
Parameter Estimation
a population
Estimators
Sampling – A Pictorial Presentation
f(x)
Xi ~ Population(,2)
Population
2
X

Std dev( X )   X
Sample

Random Sample
X1, X2, …, Xn

n
n

X 
X
i 1
n
i
Sampling Distributions
The probability distribution of a statistic is
called a sampling distribution.
- Definition makes sense. Statistic is a
property of a sample from a population.
- Depends on the population distribution,
sample size, and method of sample selection.
- Key statistics are things like the sample
mean, variance, proportion, and difference of
two means.
Definition of a Statistic
Statistic – any function of the observations in a
random sample. Examples of point estimates:
1 n
ˆ  x   xi
n i 1
n
1
2
ˆ 2  s 2 
(
x

x
)
 i
n  1 i 1
pˆ  x / n
x1  x2
pˆ 1  pˆ 2
A Sampling Distribution is the probability distribution of a
statistic.
Sampling Distributions cont’d
X 1  X 2  ...  X n
X
n
    ...  
so that  X 

n
2
2
2
2




...



and  X2 

2
n
n
If the Xi have a normal distribution, then so does the
sample mean. The Xi are I.I.D.R.V.
7.2 Sampling Distributions and the
Central Limit Theorem
Statistical inference is concerned with making decisions about a
population based on the information contained in a random
sample from that population.
Definitions:
The Central Limit Theorem
Figure 7-1
Distributions of
average scores from
throwing dice. [Adapted
with permission from Box,
Hunter, and Hunter (1978).]
What does it take
to become normal?
Who are
you calling
normal?
More Normalcy
The CLT in Action
Ten people from a population having a mean weight of 190 lb.
with a variance of 400 lb2 get on an elevator having a weight
capacity of 2000 lb. What is the probability that their average
weight exceeds 200 lb. and they all fall to their death?
X approx n   x  190,  x2  400 /10 
 X   200  190 
Pr  X  200  Pr 

  Pr  z  1.581  .0569
 / n 20 / 10 
Some Normal Thoughts on the
Central Limit Theorem (CLT)


CLT tells us that a distribution of means will always
be nearly normal if the sample is large enough.
Heights of adults are a result of both genetic and
environmental factors.




they are polygenic – influenced by many different genes
also many environmental factors; e.g. nutrition and
childhood diseases
eventual height is then an average sample from the large
population of “height factors”
Height therefore has a normal distribution
The tall and the short of it.
However All is not Normal

Weights of individuals are not normally distribution



In a similar manner, income is not normally
distributed


It is not just the cumulative result of many small factors
In addition, there may be one or two dominant causes of
obesity; e.g. a glandular disturbance
as a result of a dominant factor – e.g. inherited wealth
On the other hand, mental test scores tend to be
normally distributed

due to many determinants: e.g. genetics and long-term
environmental conditions
CLT Revisited
The Main Result (of all time!):
If X 1 , X 2 ,..., X n is a random sample of size n taken
from a population (finite or infinite) with mean  and
n
finite variance  , and if Y=  X i , the
2
i 1
limiting form of the distribution of
Y  y
Z
y
as n  , is the standard normal distribution.
The sum of all Random Variables



A doctor spends an average (mean) of 20 minutes with each
patient with a standard deviation of 8 minutes. Today’s
appointment book shows 10 patients scheduled this morning (8 –
12).
The good doctor has a luncheon appointment at noon before her
afternoon golf outing.
What is the probability she will make the luncheon on time?
10
Y   Xi
i 1
n(  y  200,  y  640)
 Y   y 240  200 
Pr{Y  240}  Pr 

  Pr{Z  1.581}  .9431
25.3 
  y
Difference in Sample Means
Approximate Sampling Distribution
More Mean Differences
X1
n( 1 ,  12 ); X 2
n( 2 ,  22 )
Y  X1  X 2
E Y    y  E  X 1  X 2   E  X 1   E  X 2   1  2
V Y    y2  V  X 1  X 2   V  X 1   V  X 2  
Z
Y  y
y

X 1  X 2  ( 1  2 )
 12 / n1   22 / n2
 12
n

 22
n
A Normal Difference Example


The section 1 class in ENM 661 consisting of 24 students had
an average score of 82.7 on their midterm while section 2
consisting of 16 students scored an average of 81.4.
What is the probability that their average scores would differ
by at least 1.3 if 1 - 2 = 0. Assume the population standard
deviations are known where 1 = 10 and 2 = 12.
z
X 1  X 2  ( 1  2 )
 12 / n1   22 / n2
Pr Z  .358  .36
82.7  81.4   0


 .358
100 144

24 16
Now begins the discussion on
point estimation
The discussion on the central limit
theorem has now ended.
Definition of Point Estimate
Point Estimate
 qˆ is a point estimate of some population parameter q
of a statistic Q̂ .
 Q̂ is a point estimator of q . After a sample has been
selected Q̂ takes on a particular valueqˆ .
 Q̂ is a random variable, qˆ is not, e.g.
X vs. x , or S 2 vs. s 2
s2 is a population parameter, S2 is a point estimator
of s2. The estimate of S2 is s2. S2 has a sampling
distribution. But s2 does not – it is just a number.
Properties of Estimators
What makes a good estimator? What is the best
estimator for a population parameter?
•
Bias - does it hit the target?
•
Variance – estimate is based on a sample
•
Standard Error and Estimated Standard Error
•
Mean Squared Error and Efficiency
•
Consistency – how does the estimator behave as
the sample size increases?
•
Sufficiency – does the estimator use all the
information that is available?
Bias of the Estimator
Def: The point estimator Q̂ is an unbiased estimator for
ˆ  q . If the estimator is not
the parameter q if E Q
 
 
ˆ  q is called the bias
unbiased, then the difference E Q
of the estimator Q̂ .
Is the sample mean unbiased?
 n

X
 i  1  n
n
 1 n
i 1


E[ X ]  E
 E  X i    E[ X i ] 

n
 n  n  i 1  n i 1


7-3 General Concepts of Point
Estimation
7-3.1 Unbiased Estimators
Definition
Example 7-1
Example 7-1 (continued)
A Biased Estimator
Define an estimator for the population variance to be:
1 n
1 n 2
2
S   ( X i  X )   X i  nX 2
n i 1
n i 1
2
  n 2

 n 2
2
E  nS   E   X i  nX    E   X i   E  nX 2  
 i 1
   i 1


2


 2
2
2 
 nE  X 1   n     

 n
 
 2

 2
2
2 
 n       
  

 n
 
  n  1  2 
2
2
 n

n




n


using:  2  E[ X 2 ]   2
2
2
2
2  n 1 


E S    
 

n
n


n
2
E  S  
E  S 2    2
n 1
7-3.2 Variance of a Point Estimator
Definition
Figure 7-5 The sampling
distributions of two
unbiased estimators
Qˆ 1 and Qˆ 2 .
Variance of Estimator
Sample mean is the MVUE for the population mean for a
population with normal distribution.
Generally, the stat package you use is making the
reasonable choices for you.
Example of bad choice: sample size n=2
Method 1: estimate mean as (X1 + X2)/2
Method 2: estimate mean as (X1 + 2X2)/3
Variance of method 1 is 2/2
Variance of method 2 is 52/9
7-3.2 Variance of a Point Estimator
I just knew it was
going to be the
sample mean.
BLUE Estimator

Best Linear Unbiased Estimator (BLUE)


Best is defined as the minimum variance
estimator from among all unbiased linear
estimators
Is the sample mean a BLUE estimator
for the population
mean?
n
X
X
i 1
n
i
An Engineering Management
Bonus Round!!!!
A real world example of sampling,
parameter estimation, and fishing
for the correct answer.
How many Fish are in the Lake?





Let N = the number of fish in the lake
k = the number of fish caught and tagged, and released
back into the lake
allow for the tagged fish to be uniformly dispersed within
the lake
X = a RV, the number of tagged fish caught in the followon sample of size n
Then an estimate for the number of fish in the lake is
found by assuming
k X

N n
kn
ˆ
Then N 
X
Is N-hat unbiased, BLUE, or
MVUE?
kn
ˆ
N
;
X
k n
1
ˆ


E  N   E    kn E  
X 
X 
k n
1
2
ˆ


Var  N   Var     kn  Var  
X 
X 
X
hypergeometric  N , n, k 
counting the fish
in the lake
Point Estimation
- to be continued
next time
- making standard errors
- those magic moments
- maximizing likelihoods
ENM 500 students engaged
in random sampling.
ENM 500 students caught
discussing the central limit
theorem.