Generalized Linear Models (GLMs) & Categorical Data Analysis (CDA) in R Hong Tran, April 21, 2015 Laboratory for Interdisciplinary Statistical Analysis LISA helps VT researchers benefit from the use of Statistics Collaboration: Visit our website to request personalized statistical advice and assistance with: Designing Experiments • Analyzing Data • Interpreting Results Grant Proposals • Software (R, SAS, JMP, Minitab...) LISA statistical collaborators aim to explain concepts in ways useful for your research. Great advice right now: Meet with LISA before collecting your data. LISA also offers: Educational Short Courses: Designed to help graduate students apply statistics in their research Walk-In Consulting: Available Monday-Friday from 1-3 PM in the Old Security Building (OSB) for questions <30 mins. See our website for additional times and locations. All services are FREE for VT researchers. We assist with research—not class projects or homework. www.lisa.stat.vt.edu Outline 1.What is CDA? 2.Contingency Table 3.Measures of Association 4.Test of Independence 5.What is GLM? When should we use it? 6.How to evaluate the GLM models? 7.Logistic Regression 8.Poisson Regression What is CDA? Dependent Variable (Y) Independent Variables (X) Model Continuous (Normal) Continuous Linear Regression Continuous Categorical ANOVA Continuous Mixed ANCOVA Categorical Categorical CDA Contingency Table I rows for categories in X J rows for categories in Y Values in cell=possible outcomes Example 1 of Contingency Table the relationship between smoking and epidermoid/undifferentiated pulmonary carcinoma (cancer) Cohort study conducted 2x2 contingency table Does smoking increase the risk of having epidermoid/undifferentiated pulmonary carcinoma? Generating Contingency Table in R Input the 2×2 table in R as a 2×2 matrix Change the matrix to table using the function as.table(), because some functions are happier with tables than matrices Measure of Association Continuous Variables-Pearson Correlation Coefficient Ordinal Variables-Pearson Correlation Coefficient Nominal Variables-Phi Coefficient and Cramer’s V Pearson Correlation Pearson Correlation Example 2 mtcars in R 1974 Motor Trend US magazine mpg: miles per gallon wt: weight drat: rare axle ratio Phi Coefficient measures the association between two binary variables. Its value ranges from -1 to +1, where +1/-1 indicates perfect positive association/negative association, 0 indicates no association. The square of the phi coefficient is related to the chi-squared statistic for a 2×2 contingency table. Cramer’s V Cramer’s V measures the association between two nominal variables. It varies from 0 (no association) to 1 (complete association) and can reach 1 only when the two variables are equal to each other. Measures of Association Comments: 1, When the two variables are binary, Cramer’s V is the same as Phi Coefficient 2, In R, under library(psych), use function phi() for Phi Coefficient 3, In R, under library(vcd), use function assocstats() for Cramer’s V Test of Independence Large Sample Size Chi-square Test Small Sample Size Fisher’s Exact Test Test of Independence (Chi-square Test) Back to Example 1 Cases Control Total Smoke 18/313 13/313 31/313 Non-smoker 46/313 236/313 282/313 Total 64/313 249/313 1 Test of Independence (Chi-square Test) Test of Independence (Fisher’s Exact Test) When any of the expected counts fall below 5, Chi-square test is not appropriate. Instead, we use Fisher’s Exact Test. Example 3: The following data are from a Stanford University study of the effectiveness of the antidepressant Celexain the treatment of compulsive shopping. Worse Same Better Celexain 2 3 7 Placebo 2 8 2 Test of Independence Chi-Square Test Use R function chisq.test() Fisher’s Exact Test Use R function fisher.test() Generalized Linear Models When the response variables are not continuous, not normally distributed Count numbers: 1, 2, 3,… Binary: 0 and 1 Comparison General Linear Model Generalized Linear Model Special cases ANOVA, ANCOVA, MANOVA, MANCOVA, linear regression, mixed model Linear regression, logistic regression, Poisson regression Function in R lm glm Typical method estimation Least Square Maximum Likelihood Ordinary Linear Regression Ordinary Linear Regression (OLR) investigates and models the linear relationship between independent variables and dependent variables that are continuous. The simplest regression is Simple Linear regression, which models the linear relationship between a single independent variable and a single dependent variable. Simple Linear Regression Model: Assumptions in OLR The assumptions are: The true relationship between x and y is linear. The errors are normally distributed with mean zero and unknown common variance 2 . The errors are uncorrelated. The possible approaches when the assumptions of a normally distributed dependent variable with constant variance are violated: Data transformations Weighted least squares Generalized linear model (GLM) GLM Model function is called the link function because it connects the mean and the linear predictor Dependent variable’s distribution must come from the Exponential Family of Distributions Includes Normal, Bernoulli, Binomial, Poisson, Gamma, etc. 3 Components Random: Identifies dependent Y and its probability distribution Systematic: Independent variables in a linear predictor function Link function: Invertible function .that links the mean of the dependent variable to the systematic component. Response Distribution Types of GLMs GLM and OLR Ordinary linear regression is a special case of GLM In OLR, the 3 components for GLM are: Random: the dependent variable is normally distributed with mean and variance 2 Systematic: Independent variables in a linear predictor function Link function: Identity link ()= Therefore, the GLM model for Ordinary linear regression is Model Evaluation: Deviance Deviance: measures how close the predicted values from the fitted model match the actual values from the raw data. Definition: Deviance = -2[log-likelihood(proposed model)-loglikelihood(saturated model)] A saturated model is a model that fits the data perfectly, so its loglikelihood is the maximum. It has as many parameters as observations and hence it provides no simplification at all. The deviance has a chi-squared asymptotic null distribution. The degree of freedom is n-p, where n is the number of observations and p is the number of model parameters. Smaller deviance, the better the model Inference in GLM Goodness of Fit test ─ The null hypothesis is that the model is a good alternative to the saturated model. ─ Deviance is the Likelihood Ratio Statistic Likelihood Ratio test - Allows for the comparison of one model to another model by looking at the difference in deviance of the two models. -Null Hypothesis: the predictor variables in Model 1 that are not found in Model 2 are not significant to the model fit. -Alternative Hypothesis: the predictor variables in Model 1 that are not found in Model 2 are significant to the model fit. ─ LRS is distributed as Chi-square distribution. ─ Simpler models have larger deviance. Model Comparison in GLM Two additional measures for model comparison are: ─ AkaikeInformation Criterion (AIC) •Penalizes model for having many parameters •AIC=-2logLikelihood+2*p where p is the number of parameters in the model •The smaller AIC, the better the model ─ Bayesian Information Criterion (BIC) •BIC=-2logLikelihood+ln(n)*p where p is the number of parameters in the model and n is the number of observations •Usually stronger penalization for additional parameter than AIC •The smaller BIC, the better the model Summary Setup of GLM Inference in GLM Deviance and Likelihood Ratio Test ─ Test goodness of fit for the proposed GLM model ─ Test the significance of a predictor variable or set of predictor variables in the model Model Comparison in GLM ─ AIC ─ BIC Logistic Regression Logistic regression is a regression technique for predicting the outcome of a binary dependent variable. Example: y=1-Success, 0-Failure Random Component: the dependent variable follows a Bernoulli distribution ─ Probability of Success: ─ Probability of Failure: 1- ─ The probability of obtaining y=1 or y=0 is given by Bernoulli Distribution: ─ Mean(Y): μ= Logistic Regression Logistic Regression Steps for Logistic Regression in R 1.Create a single vector of 0’s and 1’s for the response variable. 2.Use the function glm() family=binomial to fit the model. 3.Test for goodness of fit and significance of predictors. 4.Interpretation Poisson Regressions Poisson regression is a regression technique for predicting the outcome of a count dependent variable. Dependent variable measures the number of occurrences in a given time frame. Outcomes equal to 0,1,2,… Examples: Number of penalties during a football game. Number of customers shop at a grocery store on a given day. Number of car accidents at an intersection during a period of time. Poisson Regression Poisson Regression Steps for Poisson Regression in R 1.Input data where y is a column of counts. 2.Use the function glm() family=poisson to fit the model. 3.Test for goodness of fit and significance of predictors.