Download LECTURE 8

SYSTEMS 302
LECTURE 8
• BIASED ESTIMATORS
• Jensen’s Inequality
• EFFICIENT ESTIMATION
• Efficiency of the Sample Mean
• REGRESSION ANALYSIS
• Simple Linear Regression Model
• Estimation of Parameters
• For next time:
• Devore, Section 12.1-12.2
ESTIMATION OF
STANDARD DEVIATION
Given the unbiased estimate of variance:
s 
2
n
1
n 1
2
(
X

X
)
 i1 i n
n
it is natural to estimate standard deviation by:
sn 
1
n 1

n
2
(
X

X
)
i
n
i 1
By continuity, this is clearly a consistent estimator of
  var( X ) . But unfortunately it is biased.
EXAMPLE:
Suppose X ~ Bernuolli (.3)
so that
var( X )  .3(1  .3)  .21
  ( X )  .21  .458
ESTIMATION OF CORRELATION
The same argument for variance shows that for a
given sample ( xi , yi : i  1,.., n ) of jointly distributed
random variables X and Y , an unbiased estimate of
their covariance, cov( X ,Y ) , is given by (try it!):
(1)
s xy 
1
n 1

n
i 1
( xi  xn )( yi  yn )
This suggests that a natural estimate of correlation,
(2)
 ( X ,Y ) 
cov( X ,Y )
 ( X ) (Y )
is given by
(3)
s xy
r ( x, y ) 

sx s y
1
n 1

1
n 1
 r ( x, y ) 

s xy
s x2 s 2y
n
i 1
( xi  xn )( yi  yn )
 i1 ( xi  xn )2
n

n
i 1
1
n 1

n
2
(
y

y
)
i
n
i 1
( xi  xn )( yi  yn )
 i1 ( xi  xn )2
n
 called sample correlation

n
2
y
y
(

)
i
n
i 1
MINIMIZATION OF VARIANCE
Given the constrained minimization problem:
minimize:
2
a
 i1 i
subject to:

n
n
a 1
i 1 i
First solve for a1 in terms of a2 ,.., an as

n
a  1  a1  1   j 1 a j
i 1 i
and let S ( a2 ,.., an ) be defined by

S ( a2 ,.., an )  1   j 1 a j

2
  j 1 a 2j
Then minimize this (uncontrained) function by
taking first-order conditions:
0

ai


S ( a2 ,.., an )  2 1   j 1 a j ( 1)  2ai
  2( a1 )  2ai
 ai  a1 , i  2,.., n
 a1    an 
1
n
BEST LINEAR UNBIASED
ESTIMATORS
For any random sample ( X 1 ,.., X n ) from a distribution with
parameter θ , each unbiased estimator of θ which is of the
form
n
(1)
θ n = å ai X i
i =1
is called a linear unbiased estimator of θ . With this
definition we have the following fundamental property
of the sample mean:
THEOREM. The sample mean X n is the unique
linear unbiased estimator of the mean µ with
smallest variance.
→ Hence X n is called a Best Linear Unbiased (BLU)
estimator of µ .
SIMPLE LINEAR
REGRESSION MODEL
There exist parameters β 0 , β1 and σ 2 such that for any
fixed value of the independent (explanatory) variable, x ,
the dependent variable, Y , is related to x by the equation
(1)
Y = β 0 + β1 x + ε
The quantity, ε , is a random variable (called the random
error or random deviation), and is assumed to be normally
distributed as
(2)
ε : N (0,σ 2 )
Note that since E (ε ) = 0 , the expected value of Y for each
value of x is given by
(3)
E (Y | x) = β 0 + β1 x
This linear relation is called the regression line with slope
parameter, β1 , and intercept parameter, β 0
SIMPLE (BIVARIATE) REGESSION
samples
pop
Bivariate
Conditional
(Y , X )
Pr(Y | X  x )
Linearity
E (Y | x )   0   1 x
Normality
  Y  E (Y | x ) ~ N (0, 2 )
MODEL
Random
Sample
Parameters
Y   0   1 x   ,  ~ N ( 0, 2 )
(Yi , xi ) , i  1,.., n
(  0 ,  1 , )
2
( yi , xi ) , i  1,.., n
ESTIMATES ??
ROD WEIGHT MODEL
Suppose that the statistical population of finished rod
weights, Y , for each rough weight value, x , is (locally)
representable by the linear model
(1)
Y = .30 + .60 x + ε
with random error
(2)
ε : N (0,.04)
Q1. If the rough casting weight of a given rod is 2.72 oz,
then what is the expected finished weight of the rod?
Q2. If rough casting weight increases by one ounce, then
what is the expected increase in finished weight?
Q3. Suppose that the acceptable range of finished weights
for rods is between 1.8 and 2.3 ounces. If the rough
casting weight of a given rod is 2.72 oz (as above)
then what is the chance that the finished rod will
be accepted?
Q3 SOLUTION:
Find: P (1.8 ≤ Y ≤ 2.3 =
= P (1.8 ≤ 1.93 + ε ≤ 2.3)
| x 2.72)
= P ( −.13 ≤ ε ≤ .37)
But: =
σ
= .2 ⇒
.04
ε
.2
~ N (0,1)
 .13 ε .37 
⇒ P ( −.13 ≤ ε ≤ .37) = P  −
≤ ≤

 .2 .2 .2 
= P ( −.65 ≤ Z ≤ 1.85)
= P ( Z ≤ 1.85) − P ( Z ≤ −.65)
= Φ(1.85) − Φ( −.65)
= .9678 − .2578 [Table A3]
= .71
⇒ 71% acceptance rate.