Download Ch 12

Statistics for
Business and Economics
8th Edition
Chapter 12
Multiple Regression
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-1
Chapter Goals
After completing this chapter, you should be able to:






Apply multiple regression analysis to business decisionmaking situations
Analyze and interpret the computer output for a multiple
regression model
Perform a hypothesis test for all regression coefficients
or for a subset of coefficients
Fit and interpret nonlinear regression models
Incorporate qualitative variables into the regression
model by using dummy variables
Discuss model specification and analyze residuals
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-2
12.1
The Multiple Regression Model
Idea: Examine the linear relationship between
1 dependent (Y) & 2 or more independent variables (Xi)
Multiple Regression Model with K Independent Variables:
Y-intercept
Population slopes
Random Error
Y  β0  β1X1  β2 X2    βK XK  ε
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-3
Multiple Regression Equation
The coefficients of the multiple regression model are
estimated using sample data
Multiple regression equation with K independent variables:
Estimated
(or predicted)
value of y
Estimated
intercept
Estimated slope coefficients
yˆ i  b0  b1x1i  b2 x 2i    bK xKi
In this chapter we will always use a computer to obtain the
regression slope coefficients and other regression
summary measures.
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-4
Three Dimensional Graphing
Two variable model
y
yˆ  b0  b1x1  b2 x 2
x2
x1
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-5
Three Dimensional Graphing
(continued)
Two variable model
y
yi
Sample
observation
yˆ  b0  b1x1  b2 x 2
<
<
yi
ei = (yi – yi)
x2i
x1
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
<
x1i
x2
The best fit equation, y ,
is found by minimizing the
sum of squared errors, e2
Ch. 12-6
12.2
Estimation of Coefficients
Standard Multiple Regression Assumptions



1. The xji terms are fixed numbers, or they are realizations
of random variables Xj that are independent of the error
terms, εi
2. The expected value of the random variable Y is a linear
function of the independent Xj variables.
3. The error terms are normally distributed random
variables with mean 0 and a constant variance, 2.
E[ε i ]  0 and E[ε i2 ]  σ 2
for (i  1, ,n)
(The constant variance property is called homoscedasticity)
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-7
Standard Multiple Regression
Assumptions
(continued)

4. The random error terms, εi , are not correlated with
one another, so that
E[ε iε j ]  0

for all i  j
5. It is not possible to find a set of numbers,
c0, c1, . . . , ck, such that
c 0  c1x1i  c 2 x 2i    cK xKi  0
(This is the property of no linear relation for the Xjs)
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-8
Example:
2 Independent Variables

A distributor of frozen desert pies wants to
evaluate factors thought to influence demand



Dependent variable:
Pie sales (units per week)
Independent variables: Price (in $)
Advertising ($100’s)
Data are collected for 15 weeks
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-9
Pie Sales Example
Week
Pie
Sales
Price
($)
Advertising
($100s)
1
350
5.50
3.3
2
460
7.50
3.3
3
350
8.00
3.0
4
430
8.00
4.5
5
350
6.80
3.0
6
380
7.50
4.0
7
430
4.50
3.0
8
470
6.40
3.7
9
450
7.00
3.5
10
490
5.00
4.0
11
340
7.20
3.5
12
300
7.90
3.2
13
440
5.90
4.0
14
450
5.00
3.5
15
300
7.00
2.7
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Multiple regression equation:
Sales = b0 + b1 (Price)
+ b2 (Advertising)
Ch. 12-10
Estimating a Multiple Linear
Regression Equation

Excel can be used to generate the coefficients and
measures of goodness of fit for multiple regression

Data / Data Analysis / Regression
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-11
Multiple Regression Output
Regression Statistics
Multiple R
0.72213
R Square
0.52148
Adjusted R Square
0.44172
Standard Error
47.46341
Observations
ANOVA
Regression
Sales  306.526 - 24.975(Pri ce)  74.131(Advertising)
15
df
SS
MS
F
2
29460.027
14730.013
Residual
12
27033.306
2252.776
Total
14
56493.333
Coefficients
Standard Error
Intercept
306.52619
114.25389
2.68285
0.01993
57.58835
555.46404
Price
-24.97509
10.83213
-2.30565
0.03979
-48.57626
-1.37392
74.13096
25.96732
2.85478
0.01449
17.55303
130.70888
Advertising
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
t Stat
6.53861
Significance F
P-value
0.01201
Lower 95%
Upper 95%
Ch. 12-12
The Multiple Regression Equation
Sales  306.526 - 24.975(Pri ce)  74.131(Advertising)
where
Sales is in number of pies per week
Price is in $
Advertising is in $100’s.
b1 = -24.975: sales
will decrease, on
average, by 24.975
pies per week for
each $1 increase in
selling price, net of
the effects of changes
due to advertising
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
b2 = 74.131: sales will
increase, on average,
by 74.131 pies per
week for each $100
increase in
advertising, net of the
effects of changes
due to price
Ch. 12-13
Example 12.3








Y profit margines
X1: net revenue for deposit dollar
X2 nunber of offices
Y = 1.56 + 0.237X1 -0.000249X2
Corrleations
Y
X1
X1
-0.704
X2
-.0.868
0.941


Y = 1.33 -0.169X1
Y = 1.55 – 0.000120X2
12.3
Explanatory Power of a
Multiple Regression Equation
Coefficient of Determination, R2

Reports the proportion of total variation in y explained
by all x variables taken together
SSR regression sum of squares
R 

SST
total sum of squares
2

This is the ratio of the explained variability to total
sample variability
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-16
Coefficient of Determination, R2
Regression Statistics
Multiple R
0.72213
R Square
0.52148
Adjusted R Square
0.44172
Standard Error
Regression
52.1% of the variation in pie sales
is explained by the variation in
price and advertising
47.46341
Observations
ANOVA
SSR 29460.0
R 

 .52148
SST 56493.3
2
15
df
SS
MS
F
2
29460.027
14730.013
Residual
12
27033.306
2252.776
Total
14
56493.333
Coefficients
Standard Error
Intercept
306.52619
114.25389
2.68285
0.01993
57.58835
555.46404
Price
-24.97509
10.83213
-2.30565
0.03979
-48.57626
-1.37392
74.13096
25.96732
2.85478
0.01449
17.55303
130.70888
Advertising
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
t Stat
6.53861
Significance F
P-value
0.01201
Lower 95%
Upper 95%
Ch. 12-17
Estimation of Error Variance

Consider the population regression model
yi  β0  β1x1i  β2 x 2i    βK xKi  ε i

The unbiased estimate of the variance of the errors is
n
s2e 
2
e
 i
SSE
n  K 1 n  K 1
i1

where ei  y i  yˆ i

The square root of the variance, se , is called the
standard error of the estimate
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-18
Standard Error, se
Regression Statistics
Multiple R
0.72213
R Square
0.52148
Adjusted R Square
0.44172
Standard Error
Regression
The magnitude of this
value can be compared to
the average y value
47.46341
Observations
ANOVA
se  47.463
15
df
SS
MS
F
2
29460.027
14730.013
Residual
12
27033.306
2252.776
Total
14
56493.333
Coefficients
Standard Error
Intercept
306.52619
114.25389
2.68285
0.01993
57.58835
555.46404
Price
-24.97509
10.83213
-2.30565
0.03979
-48.57626
-1.37392
74.13096
25.96732
2.85478
0.01449
17.55303
130.70888
Advertising
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
t Stat
6.53861
Significance F
P-value
0.01201
Lower 95%
Upper 95%
Ch. 12-19
Adjusted Coefficient of
2
Determination, R


R2 never decreases when a new X variable is
added to the model, even if the new variable is
not an important predictor variable
 This can be a disadvantage when comparing
models
What is the net effect of adding a new variable?
 We lose a degree of freedom when a new X
variable is added
 Did the new X variable add enough
explanatory power to offset the loss of one
degree of freedom?
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-20
Adjusted Coefficient of
2
Determination, R
(continued)

Used to correct for the fact that adding non-relevant
independent variables will still reduce the error sum of
squares
SSE / (n  K  1)
2
R  1
SST / (n  1)
(where n = sample size, K = number of independent variables)



Adjusted R2 provides a better comparison between
multiple regression models with different numbers of
independent variables
Penalize excessive use of unimportant independent
variables
Value is less than R2
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-21
R
Regression Statistics
Multiple R
0.72213
R Square
0.52148
Adjusted R Square
0.44172
Standard Error
47.46341
Observations
ANOVA
Regression
15
df
2
R  .44172
2
44.2% of the variation in pie sales is
explained by the variation in price and
advertising, taking into account the sample
size and number of independent variables
SS
MS
F
2
29460.027
14730.013
Residual
12
27033.306
2252.776
Total
14
56493.333
Coefficients
Standard Error
Intercept
306.52619
114.25389
2.68285
0.01993
57.58835
555.46404
Price
-24.97509
10.83213
-2.30565
0.03979
-48.57626
-1.37392
74.13096
25.96732
2.85478
0.01449
17.55303
130.70888
Advertising
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
t Stat
6.53861
Significance F
P-value
0.01201
Lower 95%
Upper 95%
Ch. 12-22
Coefficient of Multiple
Correlation

The coefficient of multiple correlation is the correlation
between the predicted value and the observed value of
the dependent variable
R  r(yˆ , y)  R2



Is the square root of the multiple coefficient of
determination
Used as another measure of the strength of the linear
relationship between the dependent variable and the
independent variables
Comparable to the correlation between Y and X in
simple regression
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-23
Conf.
Intervals
and
Hypothesis
12.4
Tests for Regression Coefficients
The variance of a coefficient estimate is
affected by:




the sample size
the spread of the X variables
the correlations between the independent variables, and
the model error term
We are typically more interested in the regression
coefficients bj than in the constant or intercept b0
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-24
Confidence Intervals
Confidence interval limits for the population slope βj
b j  tnK 1,α/2Sb j
Coefficients
Standard Error
Intercept
306.52619
114.25389
Price
-24.97509
10.83213
74.13096
25.96732
Advertising
where t has
(n – K – 1) d.f.
Here, t has
(15 – 2 – 1) = 12 d.f.
Example: Form a 95% confidence interval for the effect of
changes in price (x1) on pie sales:
-24.975 ± (2.1788)(10.832)
So the interval is
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
-48.576 < β1 < -1.374
Ch. 12-25
Confidence Intervals
(continued)
Confidence interval for the population slope βi
Coefficients
Standard Error
…
Intercept
306.52619
114.25389
…
57.58835
555.46404
Price
-24.97509
10.83213
…
-48.57626
-1.37392
74.13096
25.96732
…
17.55303
130.70888
Advertising
Lower 95%
Upper 95%
Example: Excel output also reports these interval endpoints:
Weekly sales are estimated to be reduced by between 1.37 to
48.58 pies for each increase of $1 in the selling price
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-26
Hypothesis Tests

Use t-tests for individual coefficients

Shows if a specific independent variable is
conditionally important

Hypotheses:


H0: βj = 0 (no linear relationship)
H1: βj ≠ 0 (linear relationship does exist
between xj and y)
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-27
Evaluating Individual
Regression Coefficients
(continued)
H0: βj = 0 (no linear relationship)
H1: βj ≠ 0 (linear relationship does exist
between xi and y)
Test Statistic:
t
bj  0
(df = n – k – 1)
Sb j
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-28
Evaluating Individual
Regression Coefficients
(continued)
Regression Statistics
Multiple R
0.72213
R Square
0.52148
Adjusted R Square
0.44172
Standard Error
47.46341
Observations
ANOVA
Regression
15
df
t-value for Price is t = -2.306, with
p-value .0398
t-value for Advertising is t = 2.855,
with p-value .0145
SS
MS
F
2
29460.027
14730.013
Residual
12
27033.306
2252.776
Total
14
56493.333
Coefficients
Standard Error
Intercept
306.52619
114.25389
2.68285
0.01993
57.58835
555.46404
Price
-24.97509
10.83213
-2.30565
0.03979
-48.57626
-1.37392
74.13096
25.96732
2.85478
0.01449
17.55303
130.70888
Advertising
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
t Stat
6.53861
Significance F
P-value
0.01201
Lower 95%
Upper 95%
Ch. 12-29
Example: Evaluating Individual
Regression Coefficients
From Excel output:
H0: βj = 0
H1: βj  0
Coefficients
Standard Error
-24.97509
74.13096
Price
Advertising
d.f. = 15-2-1 = 12
 = .05
t Stat
P-value
10.83213
-2.30565
0.03979
25.96732
2.85478
0.01449
The test statistic for each variable falls
in the rejection region (p-values < .05)
t12, .025 = 2.1788
Decision:
/2=.025
/2=.025
Reject H0 for each variable
Conclusion:
Reject H0
Do not reject H0
-tα/2
-2.1788
0
Reject H0
tα/2
2.1788
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
There is evidence that both
Price and Advertising affect
pie sales at  = .05
Ch. 12-30
12.5
Tests on Regression Coefficients
Tests on All Coefficients

F-Test for Overall Significance of the Model

Shows if there is a linear relationship between all
of the X variables considered together and Y

Use F test statistic

Hypotheses:
H0: β1 = β2 = … = βK = 0 (no linear relationship)
H1: at least one βi ≠ 0 (at least one independent
variable affects Y)
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-31
F-Test for Overall Significance

Test statistic:
MSR
SSR/K
F 2 
se
SSE/(n  K  1)
where F has K (numerator) and
(n – K – 1) (denominator)
degrees of freedom

The decision rule is
MSR
Reject H0 if F  2  FK,nK 1,α
se
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-32
F-Test for Overall Significance
(continued)
Regression Statistics
Multiple R
0.72213
R Square
0.52148
Adjusted R Square
0.44172
Standard Error
47.46341
Observations
ANOVA
Regression
15
df
MSR 14730.0
F

 6.5386
MSE
2252.8
With 2 and 12 degrees
of freedom
SS
MS
P-value for
the F-Test
F
2
29460.027
14730.013
Residual
12
27033.306
2252.776
Total
14
56493.333
Coefficients
Standard Error
Intercept
306.52619
114.25389
2.68285
0.01993
57.58835
555.46404
Price
-24.97509
10.83213
-2.30565
0.03979
-48.57626
-1.37392
74.13096
25.96732
2.85478
0.01449
17.55303
130.70888
Advertising
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
t Stat
6.53861
Significance F
P-value
0.01201
Lower 95%
Upper 95%
Ch. 12-33
F-Test for Overall Significance
(continued)
H0: β1 = β2 = 0
H1: β1 and β2 not both zero
 = .05
df1= 2
df2 = 12
Critical
Value:
 = .05
Do not
reject H0
Reject H0
MSR
F
 6.5386
MSE
Decision:
Since F test statistic is in
the rejection region (pvalue < .05), reject H0
F = 3.885
0
Test Statistic:
Conclusion:
F
F.05 = 3.885
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
There is evidence that at least one
independent variable affects Y
Ch. 12-34
Test on a Subset of
Regression Coefficients

Consider a multiple regression model involving
variables Xj and Zj , and the null hypothesis that the Z
variable coefficients are all zero:
y  β0  β1x1    βK xK  α1z1  αR zR  ε
H0 : α1  α2    αR  0
H1 : at least one of α j  0 (j  1,..., R)
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-35
Tests on a Subset of
Regression Coefficients
(continued)

Goal: compare the error sum of squares for the
complete model with the error sum of squares for the
restricted model

First run a regression for the complete model and obtain SSE

Next run a restricted regression that excludes the Z variables
(the number of variables excluded is R) and obtain the
restricted error sum of squares SSE(R)

Compute the F statistic and apply the decision rule for a
significance level 
( SSE(R)  SSE ) / R
Reject H0 if F 
 FR,nK R 1,α
2
se
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-36
Newbold 12.41










n=30 families
Explain household milk consumption
Y: milk consumption (quarts per week)
X1: family income (dolars per week)
X2: family size
LS estimators :
b0:-0.025, b1:0.052, b2:1.14
SST: 162.1, SSE: 88.2
: X3: number of preschool childeren ?
SSE 83.7


Test the null hypothesis that
Number of preschool childeren in a family dose
not significantly ieffects weekly family milk
consumption
12.41 Let  3 be the coefficient on the number of preschool children in the household
( SSE ( R)  SSE ) / R (88.2  83.7) /1

 1.398 , F 1,26,.05 = 4.23
H 0 : 3  0, H1 : 3  0 , F 
2
se
83.7 / 26
Therefore, do not reject H 0 at the 5% level
12.6

Prediction
Given a population regression model
y i  β0  β1x1i  β 2 x 2i    βK x Ki  ε i (i  1,2,, n)

then given a new observation of a data point
(x1,n+1, x 2,n+1, . . . , x K,n+1)
the best linear unbiased forecast of y^n+1 is
yˆ n1  b0  b1x1,n1  b 2 x 2,n1    bK x K,n1

It is risky to forecast for new X values outside the range of the data used
to estimate the model coefficients, because we do not have data to
support that the linear model extends beyond the observed range.
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-40
Predictions from a
Multiple Regression Model
Predict sales for a week in which the selling
price is $5.50 and advertising is $350:
Sales  306.526 - 24.975(Pri ce)  74.131(Adv ertising)
 306.526 - 24.975 (5.50)  74.131 (3.5)
 428.62
Predicted sales
is 428.62 pies
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Note that Advertising is
in $100’s, so $350
means that X2 = 3.5
Ch. 12-41
Transformations for
Nonlinear Regression Models
12.7



The relationship between the dependent
variable and an independent variable may
not be linear
Can review the scatter diagram to check for
non-linear relationships
Example: Quadratic model
Y  β0  β1X1  β2 X12  ε

The second independent variable is the square
of the first variable
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-42
Quadratic Model Transformations
Quadratic model form:
Let
z1  x1 and z2  x12
And specify the model as
y i  β0  β1z1i  β2 z 2i  ε i

where:
β0 = Y intercept
β1 = regression coefficient for linear effect of X on Y
β2 = regression coefficient for quadratic effect on Y
εi = random error in Y for observation i
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-43
Linear vs. Nonlinear Fit
Y
Y
X
X
Linear fit does not give
random residuals
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
residuals
residuals
X
X

Nonlinear fit gives
random residuals
Ch. 12-44
Quadratic Regression Model
Yi  β0  β1X1i  β 2 X  ε i
2
1i
Quadratic models may be considered when the scatter
diagram takes on one of the following shapes:
Y
Y
β1 < 0
β2 > 0
X1
Y
β1 > 0
β2 > 0
X1
Y
β1 < 0
β2 < 0
X1
β1 > 0
β2 < 0
X1
β1 = the coefficient of the linear term
β2 = the coefficient of the squared term
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-45
Testing for Significance:
Quadratic Effect

Testing the Quadratic Effect

Compare the linear regression estimate
yˆ  b0  b1x1

with quadratic regression estimate
yˆ  b0  b1x1  b2 x12

Hypotheses

H0: β2 = 0 (The quadratic term does not improve the model)

H1: β2  0 (The quadratic term improves the model)
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-46
Testing for Significance:
Quadratic Effect
(continued)

Testing the Quadratic Effect
Hypotheses


H0: β2 = 0
(The quadratic term does not improve the model)

H1: β2  0
(The quadratic term improves the model)
The test statistic is
b2  β2
t
sb 2
d.f.  n  3
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
where:
b2 = squared term slope
coefficient
β2 = hypothesized slope (zero)
Sb = standard error of the slope
2
Ch. 12-47
Testing for Significance:
Quadratic Effect
(continued)

Testing the Quadratic Effect
Compare R2 from simple regression to
R2 from the quadratic model

If R2 from the quadratic model is larger than
R2 from the simple model, then the
quadratic model is a better model
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-48
Exmple



Performance by age
First increases and then falls again for older
ages
Quadartic relation
Y ( Performance)
X ( age) X2 (age square)
38
18
52
27
61
58
11
42
51
36
20
12
56
75
44
38
83
64
59
17
400
144
3136
5625
1936
1444
6889
4096
3481
289
70
60
50
40
30
20
10
0
Series1
0
20
40
60
80
100
Regression Statistics
Multiple R
0,98763
R Square
0,975412
Adjusted R Square
0,968387
Standard Error2,989524
Observations
10
ANOVA
df
Regression
Residual
Total
SS
MS
2 2481,839 1240,92
7 62,56076 8,937252
9
2544,4
Coefficients
Standard Error t Stat
Intercept
-9,125595 3,782392 -2,412652
X Variable 1
3,01394 0,190829 15,79391
X Variable 2 -0,03372 0,002036 -16,55886
F Significance F
138,848 2,33E-06
P-value
0,046592
9,88E-07
7,15E-07
Example: Quadratic Model
3
1
7
2
8
3
15
5
22
7
33
8
40
10
54
12
67
13
70
14
78
15
85
15
87
16
99
17

Purity increases as filter time
increases:
Purity vs. Time
100
80
Purity
Purity
Filter
Time
60
40
20
0
0
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
5
10
15
20
Time
Ch. 12-52
Example: Quadratic Model
(continued)

Simple regression results:
y^ = -11.283 + 5.985 Time
Coefficients
Standard
Error
-11.28267
3.46805
-3.25332
0.00691
5.98520
0.30966
19.32819
2.078E-10
Intercept
Time
t Stat
P-value
Regression Statistics
0.96888
Adjusted R Square
0.96628
Standard Error
6.15997
Time Residual Plot
Significance F
373.57904
t statistic, F statistic, and
R2 are all high, but the
residuals are not random:
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
10
2.0778E-10
Residuals
R Square
F
5
0
-5 0
5
10
15
20
-10
Time
Ch. 12-53
Example: Quadratic Model
(continued)
 Quadratic regression results:
y^ = 1.539 + 1.565 Time + 0.245 (Time)2
Coefficients
Standard
Error
Intercept
1.53870
2.24465
0.68550
0.50722
Time
1.56496
0.60179
2.60052
0.02467
Time-squared
0.24516
0.03258
7.52406
1.165E-05
Time Residual Plot
P-value
10
Residuals
t Stat
5
0
-5
R Square
0.99494
Adjusted R Square
0.99402
Standard Error
2.59513
F
5
1080.7330
2.368E-13
The quadratic term is significant and
improves the model: R2 is higher and se is
lower, residuals are now random
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
10
15
20
Time
Significance F
Time-squared Residual Plot
10
Residuals
Regression Statistics
0
5
0
-5
0
100
200
300
400
Time-squared
Ch. 12-54
Logarithmic Transformations
The Exponential Model:

Original exponential model
Y  β0 X X ε
β1
1

β2
2
Transformed logarithmic model
log(Y)  log(β0 )  β1log(X1 )  β 2log(X 2 )  log(ε)
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-55
Interpretation of coefficients
For the logarithmic model:
log Yi  log β0  β1 log X1i  log ε i

When both dependent and independent
variables are logged:

The estimated coefficient bk of the independent
variable Xk can be interpreted as
a 1 percent change in Xk leads to an estimated bk
percentage change in the average value of Y

bk is the elasticity of Y with respect to a change in Xk
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-56
Example Cobb-Douglas Production
Function

Production function
output as a function of
labor and capital
Labor Capital
20
25
18
40
55
28
26
51
61
49
115
220
318
196
185
217
256
175
193
201
Output
410,15
568,54
499,15
911,15
958,15
749,95
682,85
904,48
1148,55
928,56
Original and new Variables

Original data and new
variables
Labor
Capital Output Ln(Y/x2) ln(X1/X2)
20
25
18
40
55
28
26
51
61
49
115
220
318
196
185
217
256
175
193
201
410,15
568,54
499,15
911,15
958,15
749,95
682,85
904,48
1148,55
928,56
1,271591
0,949444
0,450855
1,536593
1,644649
1,240109
0,981098
1,642574
1,783565
1,53033
-1,7492
-2,174752
-2,87168
-1,589235
-1,213023
-2,047693
-2,287081
-1,23296
-1,151816
-1,411485
Regression Output
SUMMARY OUTPUT
Regression Statistics
Multiple R
0,98371
R Square 0,967686
Adjusted R Square
0,963646
Standard Error
0,078586
Observations
10
ANOVA
df
Regression
Residual
Total
SS
MS
1 1,479531 1,479531
8 0,049407 0,006176
9 1,528937
F Significance F
239,568 3,02E-07
Coefficients
Standard Error t Stat
P-value Lower 95%Upper 95%Lower 95,0%
Upper 95,0%
Intercept
2,577262 0,085991 29,97116 1,67E-09 2,378965 2,775558 2,378965 2,775558
X Variable 1 0,718702 0,046434 15,47799 3,02E-07 0,611625 0,825778 0,611625 0,825778
Parameter valeus





b1 =0.71
B2= 1- 0.71 = 0.29
Ln(b0) = 2.577 so b0 = exp(2.577) = 13.15
The estimated cobb-douglas Prodcution
Function is
Output = 13.15*L0.71K0.29,
Dummy Variables for
Regression Models
12.8

A dummy variable is a categorical independent
variable with two levels:





yes or no, on or off, male or female
recorded as 0 or 1
Regression intercepts are different if the
variable is significant
Assumes equal slopes for other variables
If more than two levels, the number of dummy
variables needed is (number of levels - 1)
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-61
Dummy Variable Example
yˆ  b0  b1x1  b2 x 2
Let:
y = Pie Sales
x1 = Price
x2 = Holiday (X2 = 1 if a holiday occurred during the week)
(X2 = 0 if there was no holiday that week)
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-62
Dummy Variable Example
(continued)
yˆ  b0  b1x1  b 2 (1)  (b 0  b 2 )  b1x1
Holiday
yˆ  b0  b1x1  b 2 (0) 
No Holiday
y (sales)
b0 + b2
b0
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
 b1 x 1
b0
Different
intercept
Same
slope
If H0: β2 = 0 is
rejected, then
“Holiday” has a
significant effect
on pie sales
x1 (Price)
Ch. 12-63
Interpreting the
Dummy Variable Coefficient
Example:
Sales  300 - 30(Price)  15(Holiday)
Sales: number of pies sold per week
Price: pie price in $
1 If a holiday occurred during the week
Holiday:
0 If no holiday occurred
b2 = 15: on average, sales were 15 pies greater in
weeks with a holiday than in weeks without a
holiday, given the same price
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-64
Differences in Slope

Hypothesizes interaction between pairs of x
variables


Response to one x variable may vary at different
levels of another x variable
Contains two-way cross product terms

yˆ  b0  b1x1  b 2 x 2  b3 x 3
 b0  b1x1  b 2 x 2  b3 (x1x 2 )
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-65
Effect of Interaction

Given:
Y  β0  β 2 X 2  (β1  β3 X 2 )X1
 β0  β1X1  β2 X 2  β3 X1X 2



Without interaction term, effect of X1 on Y is
measured by β1
With interaction term, effect of X1 on Y is
measured by β1 + β3 X2
Effect changes as X2 changes
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-66
Interaction Example
Suppose x2 is a dummy variable and the estimated
regression equation is yˆ  1 2x1  3x2  4x1x2
y
12
x2 = 1:
^y = 1 + 2x + 3(1) + 4x (1) = 4 + 6x
1
1
1
8
4
x2 = 0:
^y = 1 + 2x + 3(0) + 4x (0) = 1 + 2x
1
1
1
0
0
0.5
1
1.5
x1
Slopes are different if the effect of x1 on y depends on x2 value
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-67
Significance of Interaction Term


The coefficient b3 is an estimate of the difference
in the coefficient of x1 when x2 = 1 compared to
when x2 = 0
The t statistic for b3 can be used to test the
hypothesis
H0 : β3  0 | β1  0, β 2  0
H1 : β3  0 | β1  0, β 2  0

If we reject the null hypothesis we conclude that there is
a difference in the slope coefficient for the two
subgroups
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-68
12.9
Multiple Regression Analysis
Application Procedure
Errors (residuals) from the regression model:
<
ei = (yi – yi)
Assumptions:
 The errors are normally distributed
 Errors have a constant variance
 The model errors are independent
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-69
Analysis of Residuals

These residual plots are used in multiple
regression:
<

Residuals vs. yi

Residuals vs. x1i

Residuals vs. x2i

Residuals vs. time (if time series data)
Use the residual plots to check for
violations of regression assumptions
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-70
Chapter Summary









Developed the multiple regression model
Tested the significance of the multiple regression model
Discussed adjusted R2 ( R2 )
Tested individual regression coefficients
Tested portions of the regression model
Used quadratic terms and log transformations in
regression models
Explained dummy variables
Evaluated interaction effects
Discussed using residual plots to check model
assumptions
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-71
Rcrtv,dr 12.78


n=93
rate scale of 1(low)-10(high)


levels of satisfactions



opinion of residence hall life
room mates, floor, hall, resident advisor
Y. overall opinion
X1, X4










Source
model
error
total
param
inter.
x1
x2
x3
x4
df SS
MSS F
4 37.016 9.2540 9.958
88 81.780 0.9293
92 118.79
estimate t std error
3.950
5.84
0.676
0.106
1.69
0.063
0.122
1.70
0.072
0.092 1.75
0.053
0.169
2.64 0.064
R-sq
0.312
31.2% of the variation in the overall opinion of residence hall can be explained by the
variation in the satisfaction with roommates, with floor, with hall, and with resident
advisor. The F-test of the significance of the overall model shows that we reject H 0
that all of the slope coefficients are jointly equal to zero in favor of H1 that at least
one slope coefficient is not equal to zero. The F-test yielded a p-value of .000.
All of the variables are significant at the .1 level of significance but not at the .05
level of significance except the variable – satisfaction with resident advisor. The
variable – satisfaction with resident advisor is significant at all common levels of
alpha.
Exercise 12.82





developing countiry - 48 countries
Y:per manifacturing growth
x1 per agricultural growth
x2 per exorts growth
x3 per rate of inflation
Regression Analysis: y_manufgrowt versus x1_aggrowth, x2_exportgro, ...
The regression equation is
y_manufgrowth = 2.15 + 0.493 x1_aggrowth + 0.270 x2_exportgrowth
- 0.117 x3_inflation
Predictor
Coef
SE Coef
T
P
VIF
Constant
2.1505
0.9695
2.22
0.032
x1_aggro
0.4934
0.2020
2.44
0.019
1.0
x2_expor
0.26991
0.06494
4.16
0.000
1.0
x3_infla
-0.11709
0.05204
-2.25
0.030
1.0
S = 3.624
R-Sq = 39.3%
R-Sq(adj) = 35.1%
Analysis of Variance
Source
DF
SS
MS
F
Regression
3
373.98
124.66
9.49
Residual Error
44
577.97
13.14
Total
47
951.95
P
0.000
39.3% of the variation in the manufacturing growth can be explained by the variation
in agricultural growth, exports growth, and rate of inflation.
The F-test of the significance of the overall model shows that we reject H 0 that all of
the slope coefficients are jointly equal to zero in favor of H1 that at least one slope
coefficient is not equal to zero. The F-test yielded a p-value of .000.
All the independent variables are significant at the .1 and .05 levels of significance.
Exports growth is significant in explaining the manufacturing growth at all common
levels of alpha.
All else being equal, the agricultural growth and the exports growth variables have the
expected sign. This is because, as the percentage of agricultural growth and the
exports growth increases, the percentage of manufacturing growth also increases.
The rate of inflation variable is negative which indicates that higher the inflation rate,
lower is the manufacturing growth.
All rights reserved. No part of this publication may be reproduced, stored in a retrieval
system, or transmitted, in any form or by any means, electronic, mechanical, photocopying,
recording, or otherwise, without the prior written permission of the publisher.
Printed in the United States of America.
Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 12-78