Download Review of Probability and Statistics

Multiple Regression Analysis
y = b0 + b1x1 + b2x2 + . . . bkxk + u
2. Hypothesis Testing
1
Variance of the OLS Estimators
Now we know that the sampling
distribution of our estimated coefficients are
centered around the true parameters
Want to know how accurate/reliable our
estimators are
This is called hypothesis testing
2
So far, we know that given the GaussMarkov assumptions, OLS is BLUE
In order to do hypothesis testing, we need
to add another assumption (beyond the
Gauss-Markov assumptions)
Assume that u is independent of x1, x2,…, xk
and u is normally distributed with zero
mean and variance s2: u ~ Normal(0,s2)
3
Classical Linear Model
Under these assumptions, OLS is not only
BLUE (best linear unbiased estimator), but
is the minimum variance unbiased
estimator (meaning most accurate among all
possible models that give unbiased
estimators)
4
Under these assumptions,
 
bˆ j ~ Normal  b j ,Var bˆ j  , so that

bˆ j  b j


 
sd bˆ j
~ Normal  0,1
5
Population vs Sample
Mean
Variance
Standard Deviation
True Parameter
Sample mean
Sample Variance
Standard Error
Coefficient Estimate
6
The t Test
Under these assumptions
 bˆ  b 
j
j
 
se bˆ j
~ tn  k 1
Note this is a t distribution with degrees of
freedom n  k  1(vs normal)
2
ˆ
because we have to estimate s by s
2
7
The t Test (cont)
Start with a null hypothesis
For example, H0: bj=0
This null says that xj has no incremental
effect on y, beyond the effects from other
x’s
8
The t Test (cont) (Important)
To perform our test w e first need to form
ˆ
b
" the" t statistic for bˆ j : t bˆ  j
j
se bˆ
 
j
We will then use our t statistic along with
a rejection rule to determine whether t o
accept the null hypothesis , H 0
9
t Test
Besides our null, H0, we need an alternative
hypothesis, H1, and a significance level
H1: bj  0
If we want to have only a 5% probability of
rejecting H0 if it is really true, then we say
our significance level is 5%
10
t Test
If the sample is not too small (>30 observations),
Reject the null if the magnitude(=absolute
value) of our t statistic is greater than 2.
If the magnitude of our t statistic is less than 2,
then we fail to reject the null
11
If we reject the null, we say “xj is statistically
significant at the 5% significance level”, or
simply “xj is statistically significant”
If we fail to reject the null, we say “xj is
statistically insignificant at the 5% level”, or
simply “xj is statistically insignificant”.
12
p-values
An alternative to look up what percentile
the t statistic is in the appropriate t
distribution – this is the p-value.
Roughly speaking, p-value is the
probability that we would observe this t
statistic (or more extreme values) if the null
were true (no significant coefficient)
13
For example, if p-value=0.04,
This means that, if the null were true (no
significant coefficient), your chance of seeing
the results that you have seen is just 4%.
So the coefficient most likely is significant.
14
p-values
Most computer packages will compute the
p-value for you.
If p-value is <0.05, the coefficient is
significant, reject the null.
15