Download Regression Analysis

Regression Analysis
Regression Analysis
1
CHAPTER OUTLINE
• INTRODUCTION TO EMPIRICAL MODELS
• LEAST SQUARES ESTIMATION OF THE PARAMETERS
• PROPERTIES OF THE LEAST SQUARES ESTIMATORS
AND ESTIMATION OF s
2
• HYPOTHESIS TESTING IN LINEAR REGRESSION
• CONFIDENCE INTERVALS IN LINEAR REGRESSION
• PREDICTION OF NEW OBSERVATIONS
• ASSESSING THE ADEQUACY OF THE REGRESSION MODEL
Regression Analysis
2
Definitions

Regress


The act of reasoning backward
Regression

A functional relationship between two or more correlated
variables that is often empirically determined from data and
is used esp. to predict values of one variable when given
values of the others.
Regression Analysis
3
Models




Abstraction/simplification of the system used as a proxy for the
system itself
Can try wide-ranging ideas in the model
 Make your mistakes on the computer where they don’t
count, rather for real where they do count
Issue of model validity
Two types of models
 Physical (iconic)
 Logical/Mathematical -- quantitative and logical assumptions,
approximations
Regression Analysis
4
What Do You Do with a Logical Model?


If model is simple enough, use traditional mathematics
(queueing theory, differential equations, linear programming) to
get “answers”
 Nice in the sense that you get “exact” answers to the model
 But might involve many simplifying assumptions to make the
model analytically tractable -- validity??
Many complex systems require complex models for validity —
simulation needed
Regression Analysis
5
INTRODUCTION TO EMPIRICAL MODELS
• models
• theoretical (mechanical) model
• empirical model
• scatter diagram
Regression Analysis
6
Regression Analysis
7
Regression Analysis
8
• linear model
(equation)
• probabilistic linear
model
• simple linear
regression model
• regression
coefficients
Regression Analysis
9
• multiple regression model
• multiple linear regression
model
• intercept
• partial regression coefficients
• contour plot
Regression Analysis
10
• dependent variable or response y may be related to k
independent or regressor variables
• interaction
• any regression model that is linear in parameters (the
b’s) is a linear regression model, regardless of the
shape of the surface that it generates.
Regression Analysis
11
Regression Analysis
12
Regression Analysis
13
LEAST SQUARES ESTIMATION OF THE PARAMETERS
Simple Linear Regression
Y  0  1 x  
Regression Analysis
14
• method of least squares
• least squares normal equations
L      yi  0  1 xi 
2
2
i
• fitted or estimated regression line
^
^
^
y   0 1 x
• residual
^
ei  yi  y i
Regression Analysis
15
Regression Analysis
16

S xx   xi  x


2
S xy   yi xi  x

then,
^
1 
S xy
S xx
Example 10-1, pp. 436
Regression Analysis
17
Regression Analysis
18
Regression Analysis
19
Multiple Linear Regression
Regression Analysis
20
Regression Analysis
21
Regression Analysis
22
Regression Analysis
23
PROPERTIES OF THE LEAST SQUARES
ESTIMATORS AND ESTIMATION OF s2
• unbiased estimators
• covariance matrix
• estimated standard error
• residual mean square (or error mean square)
Regression Analysis
24
Hypothesis Testing on 0and 1, pp. 447
Regression Analysis
25
HYPOTHESIS TESTING IN LINEAR
REGRESSION
Regression Analysis
26
Regression Analysis
27
*k=p-1
Regression Analysis
28
Regression Analysis
29
Tests on Individual Regression Coefficients
Regression Analysis
30
Confidence Intervals on
Individual Regression Coefficients
Regression Analysis
31
Regression Analysis
32
Confidence Interval on the Mean Response
Regression Analysis
33
Regression Analysis
34
Regression Analysis
35
PREDICTION OF NEW OBSERVATIONS
Regression Analysis
36
Regression Analysis
37
• simple linear regression
Regression Analysis
38
Regression Analysis
39
ASSESSING THE ADEQUACY OF THE
REGRESSION MODEL
• normal probability plot of residuals
• standardize
• outlier
Regression Analysis
40
Regression Analysis
41
Regression Analysis
42
Regression Analysis
43
Regression Analysis
44
Regression Analysis
45
Regression Analysis
46
Regression Analysis
47
Regression Analysis
48
Coefficient of Multiple Determination
Regression Analysis
49
Regression Analysis
50
Influential Observations
Regression Analysis
51
Regression Analysis
52
Regression Analysis
53
Regression Analysis
54