Download Research Methods I

Research Methods I
Lecture 10: Regression Analysis
on SPSS
Introduction
• Lecture 8: looked at descriptive
statistics; relationships between
variables: correlation; cross-tabulation
• This lecture: regression
• Focus on how regression can be done
on SPSS
• Focus on OLS, although SPSS is
capable of WLS and logistic regression
Regression: fundamentals
• Relationship between two variables lead
to prediction of values of a dependent
variable
• Prediction of Yi based on Y(bar), model
(effect of X on Y) and residual
• Regression equation:
Y^ = a^ + b^X + e^
Regression: fundamentals
• Actual values of X will predict values of
Y^ based upon intercept term (a^),
slope coefficient (b^) and residual (e^)
• Ordinary Least Squares finds values of
a^ and b^ which minimise sum of
squared residuals from the regression
• Under certain assumptions, OLS is
BLUE
Evaluating the regression
• SPSS allows various ways to evaluate
the generated regression equation:
• Goodness of fit
• Individual significance of regressors
• Tests of classical assumptions
(necessary for inference)
Goodness of fit
• R2=ESS/TSS
• F-test: F=ESS/TSS
• Check outliers via standardised
residuals or studentised residuals
• Can check for significant outliers, i.e.,
ones which would affect the value of the
regressors
Significant outliers
•
•
•
•
Cook’s distance (critical>1)
Leverage values (critical: > 2(k+1)/n
Mahalanobis distance
Covariance ratio: if CVRi>/<1 +
[3(k+1)/n], deleting case
damages/improves parameter precision
• Casewise diagnostics
• CI of regression coefficients
Methods of regression
• Hierarchical: when there is a good
statistical or theoretical reason for
including one variable first
• Forward (specific to general) or
Backward (general to specific): based
on statistical criteria
• Stepwise: forward + removal test
• Use Enter command to do usual
regression
Regression equation
• Individual coefficients: unstandardised b
and standardised b
• SPSS will give t-statistics, s.e.(b), and
p-value for significance of b
• If p<0.05, coefficient significant at 5%
• This is all OK for description but for
inference, require specific assumptions:
need to test these
“Diagnostic” tests
• No direct specification test (no
equivalent of RESET in Microfit)
• Variation in ind. variables (2≠0): test by
prior observation
• SPSS provides many tests for
multicollinearity: e.g. VIF (critical
value=1), tolerance (=1/VIF), covariance
matrix, partial correlations between
regressors
“Diagnostic” tests
• Autocorrelation: DW test (DW(2) not
directly available)
• Linearity and homoscedasticity can be
tested via scatter plots of standardised
residuals versus standardised predicted
residuals (from the assump)
• Normality: ask SPSS for histogram and
P-P plot of the regression residuals (can
also do K-S and S-W tests)
Conclusions
• SPSS is capable of doing regression
analysis of various types
• SPSS offers a range of tests of
specification, fit and underlying
assumptions of regression
• Tests are more extensive but less userfriendly than a package like Microfit