Download Chapter 7 Estimation - Bioinformatics at SDSU

Chapter 7
Estimation
Instructor: Xijin Ge
SDSU Dept. of Math/Stat
1
Sampling situation: how many hours do SDSU
students spend in social networking sites?
Population:11,400 students in SDSU
Random Sample #1:
100 students
X
1 n
S 
( X i  X )2

n  1 i 1
2

2
2
Evaluating estimators by repeated sampling and
estimation
Population:11,400 students in SDSU
Random Sample #1
X1
Random Sample #2
X2
Random Sample #3
X3
Sample mean
3
An Experiment
• Suppose r.v. X is a number we got from rolling an
honest die
– p.d.f.
f ( x)  1 ,
x  1,2,3,4,5,6.
6
The average value for x is :
  E( x)   x f ( x)  3.5
all x
– This is the exact value of the mean that we derived
theoretically.
4
Estimation of the mean
• Pretend that we don’t know the exact value of
the mean and want to evaluate it through
experiments.
• Rolling die 30 times and calculate average X
• 56 students did the experiments
3.43
3.33
3.6
2.97
3.5
4.2
3.07
3.57
3.47
3.8
3.53
3.2
3.13
3.63
4
3.83
3.43
3.9
3.07
3.47
3.23
3.33
3.47
3.73
3.9
3.32
4.21
3.63
Point estimate
Point
estimator
3.33
3.33
3.2
3.67
3.86
3.8
3
3.3
3.4
3.67
3.33
3.56
3.47
4.33
3.53
3.57
3.76
3.5
3.7
4.13
3.97
3.42
3.53
3.43
3.4
3.53
3.63
3.42
5
Histogram of all point estimates
x = scan(“data.txt”)
hist(x,xlim=c(2.5,4.5))
10
0
5
Frequency
15
Histogram of x
2.5
3.0
3.5
4.0
4.5
x
Unbiased: centered at the right spot.
6
Defining “Unbiased” estimator
An estimator ˆ is an unbiased estimator for a
parameter  if and only if E (ˆ)   .
X
To test if the statistic is an unbiased estimator we need to repeat the sampling
and estimation many, many times.
If the average of the estimated values approaches the true/theoretical value,
then it is unbiased.
7
Theorem 7.1.1
Let X 1 , X 2 , X 3 , X n be a random sample of size n
from a distributi on with mean .
The sample mean X is an unbiased estimation for .
Proof:
E ( X )  E[ 1 n ( X 1  X 2  X 3    X n )]

1
n
( E[ X 1 ]  E[ X 2 ]  E[ X 3 ]    E[ X n ])
 1n ( 


 

  )
8
Are unbiased estimations accurate?
We sampled 100 students and
X  1.5 hr
Can we guarantee that the true mean is close to
1.5hr?
9
Desirable properties of point estimator
• Unbiased, and
• Small variance for large sample sizes.
10
Let X be the sample mean based on a random sample
of size n drawn from a distributi on with mean  and variance  2 . Then
Var X  ?
Proof.
P58, rules of
variance
Var X  Var[ 1n ( X 1  X 2  X 3    X n )]
10 minutes
group
quiz
2
  1n  Var[ X 1  X 2  X 3    X n ]
(total
5 points)
2
  1n  [Var X 1  Var X 2  Var X 3    Var X n ]
1.Give 2answer (1 point)
2
2
2
2
1


]









[

n
2.Prove
it (2 points)
2
1 2



n

n
3.Discuss
it (2 points)

2
n
11
Theorem 7.1.2. Let X be the sample mean based on a random sample
of size n from a distributi on with mean  and variance  2 . Then
Var X 
Proof.
2
n
Var X  Var[ 1n ( X 1  X 2  X 3    X n )]
  1n  Var[ X 1  X 2  X 3    X n ]
2
  1n  [Var X 1  Var X 2  Var X 3    Var X n ]
2
  1n  [ 2
2
P58, rules of
variance
 2
 2
   2 ]
  1n  n 2
2

2
n
12
For larger sample size, the difference between
repeated estimation is smaller.
Population:11,400 students in SDSU
Random Sample #1
X1
Random Sample #2
X2
Random Sample #3
X3
Sample mean
13
Discussions on Theorem 7.1.2
• Sample means based on small sample may
differ significantly from actual population
mean.
• Sample mean based on a large sample can be
expected to lie reasonable close to actual
population mean.



.
• Standard deviation of X is given by This
is
n
n
called standard error of the mean.
2
14
Th. 7.1.3. Estimation of variance
n
1
2
Let S 2 
(
X

X
)
be the sample variance

i
n  1 i 1
based on a random sample of size n from a distributi on
with mean  and variance  2 .
S 2 is an unbiased estimator for  2 .
Proof in Appendix C.
Note S is not an unbiased estimator of σ.
15
Distribution of X
Properties of Moment Generating Functions:
If mX (t )  mY (t ), for all t in some open interval about 0,
then X and Y have the same distributi on.
If X 1 and X 2 are independen t, let Y  X 1  X 2
then mY (t )  mX1 (t )mX 2 (t ).
t
Let Y    X , then mY (t )  e mX (t )
16
X is normally distributed !
Let X 1 , X 2 , X 3 , X n be a random sample of size n
from a distributi on with mean .
2

Then X is normally distribute d with mean  and variance
.
n
17
Expected value of your learning outcome
• What is an unbiased estimator?
• Sample mean is an unbiased estimator of
population mean.
• Variance of sample mean decrease linearly
with the increase of sample size.
Var X 
2
n
• Unbiased estimation of variance is S^2
• X is normally distributed!
18