Survey

* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project

Survey

Document related concepts

Data assimilation wikipedia , lookup

Instrumental variables estimation wikipedia , lookup

Forecasting wikipedia , lookup

Regression toward the mean wikipedia , lookup

Choice modelling wikipedia , lookup

Time series wikipedia , lookup

Regression analysis wikipedia , lookup

Transcript

STAC67H3: Regression Analysis Assignment # 02 Solution November 13, 2015 Problem 1: Exercise 3.10 Per capita earnings Page 149 [20 Marks] A sociologist employed linear regression model (2.1) to relate per capita earnings (Y ) to average number of years of schooling (X) for 12 cities. The fitted values Ŷi and the semistudentized e∗i follow. i: Ŷi : e∗i : 1 9.9 -1.12 2 9.3 0.81 ··· ··· ··· 3 10.2 -0.76 10 15.6 -3.78 11 11.2 0.74 12 13.1 0.32 2 a. [10 Marks] Plot the semistudentized residuals against the fitted values. What does the plot suggest? ● 1 ● ● ● ● ● 0 ● −1 ● −2 ● −3 Semistudentized Residuals ● ● −4 ● 9 10 11 12 13 14 15 Fitted Values Figure 1: Plot of the semistudentized residuals against the fitted values. Most of the semistudentized residuals are around zero line, suggesting a linear fit might be appropriate. One of the residuals lies outside of the interval (−3, 3) suggesting the presence of a potential outlying observation. ## R Codes ## p1 yhat <- p1[,1] studres <- p1[,2] ## Data: Per capita earnings ## Fitted Values ## Semistudentized Residuals 1 plot(yhat, studres, xlab = "Fitted Values", ylab = "Semistudentized Residuals") abline(h = 0, lty = 2) b. [10 Marks] How many semistudentized residuals are outside ±1 standard deviation? Approximately how many would you expect to see if the normal error model is appropriate? Four of the 12 (.33%) residuals are outside of ±1 standard deviation. We expect 68% of the residuals fall within one standard deviation from the mean, here the mean of the residuals is zero. In this example 67% of the residuals are within one standard deviation from the mean. Hence, the normal error model is reasonable. ## R Codes ## sum((studres < - 1) | (studres > 1)) mean((studres < - 1) | (studres > 1)) 1 - mean((studres < - 1) | (studres > 1)) Problem 2: Exercise 3.11 Drug concentration – Page 149 [25 Marks] A pharmacologist employed linear regression model (2.1) to study the relation between the concentration of a drug in plasma (Y ) and the log-dose of the drug (X). The residuals and log-doge levels follow. i: Xi : ei : 1 -1 0.5 2 0 2.1 3 1 -3.4 4 -1 0.3 5 0 -1.7 6 1 4.2 7 -1 -0.6 8 0 2.6 9 1 -4.0 a. [10 Marks] Plot the residuals ei against Xi . What conclusions do you draw from the plot? 4 ● ● ● ● 0 Residuals 2 ● ● −2 ● −4 ● ● −1.0 −0.5 0.0 0.5 1.0 Predictor Variable Figure 2: Plot of the residuals against predictor variable. The residuals are around zero line, suggesting a linear fit might be appropriate. However, the variability of residuals is larger for larger values of the predictor variable suggesting an apparent violation of the constant variance assumption. 2 ## R Codes ## x <- c(-1, 0, 1, -1, 0, 1, -1, 0, 1) e <- c(0.5, 2.1, -3.4, 0.3, -1.7, 4.2, -0.6, 2.6, -4.0) ## predictor variable ## residuals plot(x, e, xlab = "Predictor Variable", ylab = "Residuals") abline(h = 0, lty = 2) b. [15 Marks] Assume that (3.10) is appropriate and conduct the Breusch-Pagan test to determine whether or not the error variance varies with log-dose of the drug (X). Use α = 0.05. State the alternatives, decision rule, and conclusion. Does your conclusion support your preliminary findings in part (a)? The error sum of squares (SSE) of the model Yi = β0 + β1 Xi + i is SSE = 59.96. Consider the following model log σi2 = γ0 + γ1 Xi and the hypotheses H0 : γ1 = 0 and HA : γ1 6= 0. The alternative hypothesis states that the constant variance assumption does not hold for the first model. The second provides regression sum of squares as SSR∗ = 330.04. Hence, the BP test statistic 2 SSE SSR∗ 2 / = 3.717906. χBP = 2 n Decision Rule: At α = 0.05 level of significance, the null hypothesis is rejected if χ2BP ≥ χ2 (0.95, 1) = 3.84. Conclusion: The null hypothesis can’t be rejected at 0.05 level of significance. The parts (a) and (b) give differing results: Perhaps for the small sample size. ## R Codes ## n <- length(x) SSE <- sum(e^2) reg <- lm(e^2 ~ x) anova(reg) SSR <- 330.04 chiBP <- (SSR/2)/(SSE/n)^2 chiBP Problem 3: Exercise 3.17 Sales growth – Page 150 [45 Marks] A marketing researcher studied annual sales of a product that had been introduced 10 years ago. The data are as follows, where X is the year (coded) and Y is sales in thousands of units: i: Xi : Yi : 1 0 98 2 1 135 3 2 162 4 3 178 5 4 221 6 5 232 7 6 283 8 7 300 9 8 374 10 9 395 a. [5 Marks] Prepare a scatter plot of the data. Does a linear relation appear adequate here? 3 400 ● 300 ● 250 ● ● ● 200 Sales in thousands of units 350 ● ● 150 ● 100 ● ● 0 2 4 6 8 Year Figure 3: Plot of the response against predictor variable. More or less, a linear relation appears to be reasonable for the two variables. ## R Codes ## x <- c(0:9) y <- c(98, 135, 162, 178, 221, 232, 283, 300, 374, 395) plot(x, y, xlab = "Year", ylab = "Sales in thousands of units") reg <- lm(y ~ x) abline(reg) b. [15 Marks] Use the Box-Cox procedure and standardization (3.36) to find an appropriate power transformation of Y . Evaluate SSE for λ = 0.3, 0.4, 0.5, 0.6, 0.7. What transformation of Y is suggested? The Box-Cox procedure is performed using R (the codes are appended below). The value of λ = 0.5 minimized the residual sum of squares (SSE). Hence, a square root transformation of the original response Y seems reasonable. ## R Codes ## resSS <- function(x, y, lambda){ n <- length(y) k2 <- (prod(y))^(1/n) k1 <- 1/(lambda * (k2^(lambda - 1))) w <- rep(NA, n) for(i in 1:n){ w[i] <- ifelse(lambda == 0, (k2 * log(y[i])), (k1 * (y[i]^lambda - 1))) } reg_fit <- lm(w ~ x) SSE <- deviance(reg_fit) return(SSE) } lambda = c(0.3, 0.4, 0.5, 0.6, 0.7) 4 1100 1050 SSE 1000 950 0.3 0.4 0.5 0.6 0.7 λ Figure 4: Plot of the SSE against λ. SSE <- rep(NA, length(lambda)) for(i in 1:length(lambda)){ SSE[i] <- resSS(x, y, lambda[i]) } plot(lambda, SSE, type = "l", lty = 1, lwd = 4, xlab = expression(lambda), ylab = "SSE") √ c. [10 Marks] Use the transformation Y 0 = Y and obtain the estimated linear regression function for the transformed data. The square root transformation is obtained using R (please see the codes below). The estimated regression function for the transformed data is Ŷi0 = 10.26 + 1.08Xi . ## R Codes ## yp <- sqrt(y) regt <- lm(yp ~ x) summary(regt) d. [5 Marks] Plot the estimated regression line and the transformed data. Does the regression line appear to be a good fit to the transformed data? The estimated regression line for the transformed data shows a better fit than that is obtained in part (a). ## R Codes ## plot(x, yp, xlab = "Year", ylab = "Squared transformation of sales") abline(regt) e. [5 Marks] Obtain the residuals and plot them against the fitted values. Also prepare a normal probability plot. What do your plots show? 5 20 ● 18 ● 16 ● ● 14 Squared transformation of sales ● ● ● 12 ● 10 ● ● 0 2 4 6 8 Year 1 2 3 Figure 5: Estimated regression line for the transformed data. ● 0 ● ● ● ● ● ● −3 −2 −1 Rediduals ● ● ● 10 12 14 16 18 20 Fitted Values Figure 6: Plot of the residuals against fitted values for the transformed data. From the plot of residuals against fitted values for the transformed data, we see that the values are nicely scattered around the zero line suggesting a linear fit for the transformed data is reasonable. The residuals are not in a straight line against the expected residuals. The normal probability plot is suggesting weak to moderate departure of the residuals from normality. ## R Codes ## ## plot of residuals for against fitted values ## e <- residuals(regt) yhat <- fitted(regt) 6 0.6 0.4 ● ● ● 0.2 ● 0.0 Residuals ● ● −0.2 ● −0.4 ● ● −0.6 ● −0.6 −0.4 −0.2 0.0 0.2 0.4 0.6 Expected Score Figure 7: Normal probability plot of the residuals for the transformed data. plot(yhat, e, xlab = "Fitted Values", ylab = "Rediduals", ylim = c(-3, 3)) abline(h = 0, lty = 2) ## normal-probability plot ## n <- length(e) MSE <- sum(e^2)/(n - 2) RankofRes <- rank(e) Zscore <- qnorm((RankofRes - 0.375)/(n + 0.25)) ExpRes <- Zscore * sqrt(MSE) plot(ExpRes, e, xlab = "Expected Score", ylab = "Residuals", xlim = c(-0.6, 0.6), ylim = c(-0.6, 0.6)) abline(a = 0, b = 1) f. [5 Marks] Express the estimated regression function in the original units. Figure 8 shows the plot of the estimated regression function in original units. The estimated regression function is clearly a good fit to the original data. ## R Codes ## yhat.ori <- (yhat)^2 x <- c(0:9) y <- c(98, 135, 162, 178, 221, 232, 283, 300, 374, 395) plot(x, y, xlab = "Year", ylab = "Sales in thousands of units") points(x, yhat.ori, type = "l") Problem 4: Exercises 4.6 – Page 172 [25 Marks] Refer to Muscle mass Problem 1.27. a. [10 Marks] Obtain Bonferroni joint confidence intervals for β0 and β1 , using a 99 percent family confidence coefficient. Interpret your confidence intervals. 7 400 ● 300 ● 250 ● ● ● 200 Sales in thousands of units 350 ● ● 150 ● 100 ● ● 0 2 4 6 8 Year Figure 8: Plot of the estimated regression function in original units. The 100(1 − α) Bonferroni joint confidence intervals for β0 and β1 is b0 ± Bs(b0 ) b1 ± Bs(b1 ) where B = t(1 − α/4; n − 2). Hence, the 99% Bonferroni joint confidence intervals for β0 and β1 is (−1.45, −0.93) (140.26, 172.43) where b0 = 156.346564, b1 = −1.189996, s(b0 ) = 5.51226249, s(b1 ) = 0.09019725 and B = t(1 − α/4; n − 2) = 2.918394. In repeated sampling of the data of same size, the 99% of the estimates of β0 and β1 will be in between (140.26, 172.43) and (−1.45, −0.93), respectively. ## R Codes ## x <- MusMass[,2] y <- MusMass[,1] n <- length(x) reg <- lm(y ~ x) b <- coef(reg) se <- sqrt(diag(vcov(reg))) alpha <- 0.01 tval <- qt(1 - alpha/4, n - 2) lower.lim <- b - tval * se lower.lim upper.lim <- b + tval * se upper.lim b. [10 Marks] Will b0 and b1 tend to err in the same direction or in opposite direction here? Explain. 8 The covariance between b0 and b1 can be expressed as σ(b0 , b1 ) = −X̄σ 2 (b1 ). If X̄ is positive, b0 and b1 are negatively correlated, implying that if the estimate b1 is too high, the estimate b0 is likely to be too low, and vice versa. In this example X̄ = 59.98; hence the covariance between b0 and b1 is negative. This implies that the estimators b0 and b1 here tend to err in opposite directions. c. [5 Marks] A researcher has suggested that β0 should equal to approximately 160 and that β1 should be between −1.9 and −1.5. Do the joint confidence intervals in part (a) support this expectation? The suggested values for β1 are outside of the confidence limits obtained in part (a). So the joint confidence intervals do not support this expectation. Problem 5: Exercises 5.5 Consumer finance (page 210). The data below show, for a consumer finance company operating in six cities, the number of competing loan companies operating in the city (X) and the number per thousands of the company’s loans made in that city that are currently delinquent (Y ). i: Xi : Yi : 1 4 16 2 1 5 3 2 10 4 3 15 5 3 13 6 4 22 Assume that first-order regression model (2.1) is applicable. Using matrix methods, find (1) Y0 Y, (2) X X, (3) X0 Y. [3 × 5 = 15 Marks] Here, Y0 Y = 1259. 0 6 17 XX= . 17 55 81 0 XY= . 261 0 ## R Codes ## x <- c(4, 1, 2, 3, 3, 4) y <- c(16, 5, 10, 15, 13, 22) Y <- matrix(y, ncol = 1) X <- cbind(1, x) YY <- t(Y) %*% Y YY XX <- t(X) %*% X XX XY <- t(X) %*% Y XY Exercises 5.13 (page 211) Refer to Consumer finance Problem 5.5 . Find (X0 X) Here, 1 55 −17 −1 . (X0 X) = −17 6 41 −1 . [10 Marks] Exercises 5.24 (page 212) Refer to Consumer finance Problems 5.5 and 5.13. [60 Marks] 9 a. [7 × 5 = 35 Marks] Using matrix methods, obtain the following: (1) vector of estimated regression coefficients, (2) vector of residuals, (3) SSR, (4) SSE, (5) estimated variance-covariance matrix of b, (6) point estimate of E(Yh ) when Xh = 4, (7) s2 {pred} when Xh = 4. Here, (1) the vector of estimated regression coefficients is 0.4390244 −1 b = (X0 X) X0 Y = . 4.6097561 (2) the vector of residuals is −2.87804878 −0.04878049 0.34146341 0.73170732 −1.26829268 3.12195122 e = Y − Xb = . (3) the regression sum of squares SSR = b0 X0 Y = 1238.707. (4) the error sum of squares 0 SSE = e0 e = (Y − Xb) (Y − Xb) = 20.29268. (5) the estimated variance-covariance matrix of b 0 Var-Cov(b) = M SE X X −1 = 6.805473 −2.103510 −2.1035098 0.7424152 . (6) point estimate of E(Yh ) when Xh = 4 dh ) = Xh b = 18.87805. E(Y (7) s2 {pred} when Xh = 4. −1 s2 {pred} = 1 + X0h (X0 X) Xh M SE = 6.929209. b. [3 × 5 = 15 Marks] From your estimated variance-covariance matrix in part (a5), obtain the following (1) s{b0 , b1 }; (2) s2 {b0 }; (3) s{b1 }. √ Here, (1) s{b0 , b1 } = −2.1035098; (2) s2 {b0 } = 6.805473; (3) s{b1 } = 0.7424152. c. [5 Marks] Find the hat matrix H. Here, H = X (X0 X) −1 0 X = 0.36585366 −0.1463415 0.02439024 −0.14634146 0.6585366 0.39024390 0.02439024 0.3902439 0.26829268 0.19512195 0.1219512 0.14634146 0.19512195 0.1219512 0.14634146 0.36585366 −0.1463415 0.02439024 d. [5 Marks] Find s2 {e}. 10 0.1951220 0.1219512 0.1463415 0.1707317 0.1707317 0.1951220 0.1951220 0.1219512 0.1463415 0.1707317 0.1707317 0.1951220 0.36585366 −0.14634146 0.02439024 0.19512195 0.19512195 0.36585366 Here, 2 s {e} = (I − H) M SE = 3.2171327 0.7424152 0.7424152 1.7323022 −0.1237359 −1.9797739 −0.9898870 −0.6186794 −0.9898870 −0.6186794 −1.8560381 0.7424152 −0.1237359 −1.9797739 3.7120761 −0.7424152 −0.7424152 −0.1237359 −0.9898870 −0.6186794 −0.7424152 4.2070196 −0.8661511 −0.9898870 −0.9898870 −0.6186794 −0.7424152 −0.8661511 4.2070196 −0.9898870 −1.8560381 0.7424152 −0.1237359 −0.9898870 −0.9898870 3.2171327 ## R Codes ## b <- solve(XX) %*% XY b e <- Y - X %*% b e SSR <- t(b) %*% t(X) %*% Y SSR SSE <- t(e) %*% e SSE MSE <- SSE/4 VarCovB <- solve(XX) * c(MSE) VarCovB Xh <- c(1, 4) YhHat <- Xh %*% b YhHat s2YhPred <- (1 + t(Xh) %*% solve(XX) %*% Xh) * c(MSE) s2YhPred H <- X %*% solve(XX) %*% t(X) H I <- matrix(0, nrow = dim(H)[[1]], ncol = dim(H)[[1]]) diag(I) <- 1 s2e <- (I - H) * c(MSE) s2e Problem 6: Exercises 6.15 Patient satisfaction (page 250) [65 Marks] A hospital administrator wished to study the relation between patient satisfaction (Y ) and patient’s age (X1 , in years), severity of illness (X2 , an index), and anxiety level (X3 , an index). The administrator randomly selected 46 patients and collected the data presented below, where larger values of Y , X2 , and X3 are, respectively, associated with more satisfaction, increased of severity of illness, and more anxiety. i X1i X2i X3i Yi : : : : : 1 50 51 2.3 48 2 36 46 2.3 57 3 40 48 2.2 66 ··· ··· ··· ··· ··· 44 45 51 2.2 68 45 37 53 2.1 59 46 28 46 1.8 92 a. [5 Marks] Prepare a stem-and-leaf plot for each of the predictor variables. Are any noteworthy features revealed by these plots? The stem-and-leaf plots for X1 , X2 and X3 are produced below. None of the plots revealed any noteworthy features: No outlying observations, no special distributional shapes etc. However, one can get similar information from histograms too. 11 2 | 23 2 | 58899999 3 | 012233344 3 | 66678 4 | 0012233344 4 | 557779 5 | 0233 5 | 55 Stem-and-leaf plot for X1, patient’s age. 40 | 0 42 | 00 44 | 0 46 | 00000 48 | 0000000000 50 | 000000000000 52 | 000000 54 | 0000 56 | 00 58 | 0 60 | 0 62 | 0 Stem-and-leaf plot for severity of illness (X2, an index). 18 | 000000 20 | 00000000 22 | 0000000000000 24 | 00000000000 26 | 0000 28 | 0000 Stem-and-leaf plot for anxiety level (X3, an index). ## R Codes ## y <- PS[, 1] x1 <- PS[, 2] x2 <- PS[, 3] x3 <- PS[, 4] stem(x1) stem(x2) stem(x3) b. [10 Marks] Obtain the scatter plot matrix and the correlation matrix. Interpret these and state your principal findings. Table 1: Correlation matrix of Y , X1 , X2 and X3 . Y X1 X2 X3 Y 1.0000 -0.7868 -0.6029 -0.6446 X1 -0.7868 1.0000 0.5680 0.5697 12 X2 -0.6029 0.5680 1.0000 0.6705 X3 -0.6446 0.5697 0.6705 1.0000 ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● x1 ● ●● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ●● ●● ● ● x2 ● ● ● ● ● ● ●● ● ●● ●● ● ● ●● ● ● ● ● ● ● 1.8 ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ●● ● ● ●●●● ●●●● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ●● 2.0 ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 2.6 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●●● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● 2.8 ● ● ● ● ● 2.4 ● ● ● ● ● ● ● ● ● ● ●● ● ●● 2.2 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● 60 25 30 35 40 45 50 55 ● ● ● ● ● ● ● ● ● ● ● ● ● ● 2.8 ● ● ● ● 2.6 ● ● ● ● ● ● ●● ● ● ●● 2.4 ● ● ● ● ● 2.2 ● 55 ● ● 2.0 ● 50 ● ● ● ● ● ● ● ● ● ● ● ● ● 45 ● ● ● ● ● ● ● ●● ● ●● ● ● ● y 1.8 ● ● ● ● ● 30 40 50 60 70 80 90 25 30 35 40 45 50 55 ● ● ● x3 ●● ● ● ● ● ● ● 30 40 50 60 70 80 90 ●● ● 45 ● 50 55 60 Figure 9: Scatter plot matrix of Y , X1 , X2 and X3 . Figure 9 and Table 1 show the scatter plot and correlation matrix among Y , X1 , X2 and X3 . The response variable Y is negatively correlated with each of the predictor variables X1 , X2 and X3 . The predictor variables X1 , X2 and X3 are also moderately positively correlated which might introduce multicollinearity in the problem. ## R Codes ## pairs(PS) cor(PS) c. [10 Marks] Fit regression model (6.5) for three predictor variables to the data and state the estimated regression function. How is b2 interpreted here? The estimated regression function is d ) = 158.49 − 1.14X1 − 0.44X2 − 13.47X3 . E(Y Interpretation of b2 = −0.44: For one unit increase in severity of illness X2 , an average decrease in patient satisfaction (Y) is −0.44 unit given patient’s age X1 and anxiety level X3 are held fixed. ## R Codes ## attach(PS) reg <- lm(y ~ x1 + x2 + x3, data = PS) summary(reg) d. [10 Marks] Obtain the residuals and prepare a box plot of the residuals. Do there appear to be any outliers? The box-plot of the residuals shows a symmetric distribution. No evidence of outliers is found. ## R Codes ## e <- residuals(reg) e boxplot(e, ylab = "Residuals") 13 Table 2: Residuals 1 0.1129334 2 -9.0796538 3 4.0237858 ··· ··· 45 -5.5380448 46 10.0523698 0 −15 −10 −5 Residuals 5 10 15 i: ei : Figure 10: Box-plot of the residuals. e. [15 Marks] Plot the residuals against Ŷ , each of the predictor variables, and each two-factor interaction term on separate graphs. Also prepare a normal probability plot. Interpret your plots and summarize your findings. The plots of the residuals against predictor variables do not show any striking pattern. The plot of the residuals against fitted values shows that the constant variance assumption might not hold for this data set. The normal probability plot shows that the residuals have longer tails in both directions. ## R Codes ## yhat <- fitted(reg) plot(yhat, e, xlab = "Fitted Values", ylab = "Residuals", ylim = c(-20, 20)) abline(h = 0, lty = 2) plot(x1, e, xlab = "X1", ylab = "Residuals", ylim = c(-20, 20)) abline(h = 0, lty = 2) plot(x2, e, xlab = "X2", ylab = "Residuals", ylim = c(-20, 20)) abline(h = 0, lty = 2) plot(x3, e, xlab = "X3", ylab = "Residuals", ylim = c(-20, 20)) abline(h = 0, lty = 2) install.packages("scatterplot3d") library(scatterplot3d) scatterplot3d(x1, x2, e, xlab = "X1", ylab = "X2", zlab = "Residuals") 14 ● ● ● ● ● 10 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0 ● ● ● ● ● ● 0 Residuals ● 0 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● Residuals ● ● ● ● ● ● 10 10 ●● ● ● ● ● ● ● ● ● 20 ● ● ● ● ● Residuals (c) Residuals against X2 . 20 (b) Residuals against X1 . 20 (a) Residuals against Ŷ . ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 70 80 −20 ● −20 60 90 25 30 35 40 Fitted Values 45 50 55 45 50 X1 55 60 X2 (e) Residuals against X1 and X3 . (f) Residuals against X1 and X3 . 20 (d) Residuals against X3 . ● ● ● ● −20 ● 50 ● ● ● ● ● ● ● ● ● 40 ● −10 ● ● −10 −10 ● ● ● ● ● ● ● ● ● ● 10 ● ● ● ● ● 65 ● 60 ● ● ● ● −10 ● ● ● ● ● ● ● ● 2.8 ● 2.6 ● 50 ● ● ● ● ● ● ● 2.4 2.2 ● ● 45 2.0 1.8 2.0 2.2 2.4 2.6 −20 −20 −20 ● 40 20 25 30 35 2.8 40 45 50 55 1.8 20 25 30 35 X1 40 45 X1 X3 (g) Residuals against X2 and X3 . (h) Normal probability plot of the residuals. ● 15 ● ● 10 ● ● ● ●● ● 5 3.0 ● ● ● ● ● ● ● ● ● ● ● ● −10 ● ● ● 0 ● ●● −10 ● ●● ●● ● ● ●● ● ● 2.6 ● ● ● 2.8 ● 0 2.4 ● ●● ● ● 2.2 ● ● −20 2.0 45 50 55 60 ● ● ● ● 1.8 40 ● −15 Residuals ● ● ● ● Residuals ● ● ●● ●● ● −5 10 ● ● ● ● ● ● ●● ● X3 20 ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 65 −20 X2 −10 0 10 20 Expected Score Figure 11: Plots of the residuals. scatterplot3d(x1, x3, e, xlab = "X1", ylab = "X3", zlab = "Residuals") scatterplot3d(x2, x3, e, xlab = "X2", ylab = "X3", zlab = "Residuals") n <- length(e) MSE <- sum(e^2)/(n - 4) RankofRes <- rank(e) 15 ● 3.0 ● ● ● 55 ● ● ● ● ● ● −10 ● ● ● ● ● ● ● ● ● ● ● 0 ● ● ● ● ● ● Residuals 10 ● ● −10 ● ● 0 Residuals ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● 20 ● ● ● ● ● ● ● ● ● ● 50 55 X3 ● ● ● ● ● ● X2 ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● 0 ● ● ● ● ● Residuals ● ● ● 20 10 ● Zscore <- qnorm((RankofRes - 0.375)/(n + 0.25)) ExpRes <- Zscore * sqrt(MSE) plot(ExpRes, e, xlab = "Expected Score", ylab = "Residuals") abline(a = 0, b = 1) f. [5 Marks] Can you conduct a formal test for lack of fit here? In this data set the observations were not repeated. So a formal test for lack of fit is not recommended here. g. [10 Marks] Conduct the Breusch-Pagan test for constancy of the error variance, assuming log σi2 = γ0 + γ1 Xi1 + γ2 Xi2 + γ3 Xi3 ; use α − 0.01. State the alternatives, decision rule, and conclusion. Here, SSE and SSR∗ are 4248.841 and 21355.53, respectively. For n = 46, the test statistic is χ2BP = (SSR∗ /2)/(SSE/n)2 = 1.25157, which follows χ-squared distribution with 3 degrees of freedoms. At α = 0.01 level of significance, the tabulated value is χ2T AB = χ2 (1 − α; 3) = 11.34487. The assumption of constant variance can’t be rejected at 0.01 level of significance as χ2BP < χ2T AB . ## R Codes ## SSE <- sum(e^2) SSE n <- length(e) reg2 <- lm(e^2 ~ x1 + x2 + x3) y2hat <- fitted(reg2) SSR2 <- sum((y2hat - mean(y2hat))^2) SSR2 chiBP <- (SSR2/2) / (SSE/n)^2 chiBP chiTAB <- qchisq(0.99, 3) chiTAB Exercises 6.16 (page 251) Refer to Patient satisfaction Problem 6.15 [25 Marks] Assume that regression model (6.5) for three predictor variables with independent normal error terms is appropriate. a. [10 Marks] Test whether there is a regression relation; use α = 0.10. State the alternatives, decision rule, and conclusion. What does your test imply about β1 , β2 , and β3 ? What is the P -value of the test? Consider the multiple linear regression model with three predictors Yi = β0 + β1 Xi1 + β2 Xi2 + β3 Xi3 + i . In order to test if there is any regression relation, we want to test the null hypothesis H0 : β1 = β2 = β3 = 0 against alternative HA : at least one β is not equal to zero. 16 The test statistic is FCAL = (SSR/3)/(SSE/n − 4) = 30.05, which follows 3 and 42 degrees of freedoms. At α = 0.10 level of significance, the tabulated value is FT AB = F (1 − α; 3, 42) = 2.219059 Decision rule: As FCAL > FT AB the null hypothesis is rejected at α = 0.10 level of significance. That means at least one of the β’s are not equal to zero. The P -value of the test is 1.542−10 . ## R Codes ## reg <- lm(y ~ x1 + x2 + x3) summary(reg) qf(0.90, 3, 42) b. [10 Marks] Obtain joint interval estimates of β1 , β2 , and β3 , using a 90 percent family confidence coefficient. Interpret your results. The 100(1 − α)% Bonferroni joint confidence intervals for β1 , β2 and β3 are b1 ± Bs{b1 } b2 ± Bs{b2 } b3 ± Bs{b3 } where B = t(1 − α/6; n − 4). Using R, the joint 90% confidence intervals are (−1.614248, −0.6689755) (−1.524510, 0.6405013) (−29.092028, 2.1517012). In repeated samples of size 46, the 90% estimates of β1 , β2 and β3 will be in (−1.614248, −0.6689755), (−1.524510, 0.6405013) and (−29.092028, 2.1517012), respectively. ## R Codes ## b <- summary(reg)$coef[, 1] b seb <- summary(reg)$coef[, 2] seb alpha <- 0.10 tval <- qt(1 - alpha/6, 42) lower.lim <- b - tval * seb lower.lim upper.lim <- b + tval * seb upper.lim c. [5 Marks] Calculate the coefficient of multiple determination. What does it indicate here? The coefficient of multiple determination is computed as R2 = SSR/SST = 0.6822. The 68.22% variability in the response variable, patient satisfaction, is explained by the fitted model. ## R Codes ## R2 <- summary(reg)$r.squared R2 #$ 17 Exercises 6.17 (page 251) Refer to Patient satisfaction Problem 6.15 [20 Marks] Assume that regression model (6.5) for three predictor variables with independent normal error terms is appropriate. a. [10 Marks] Obtain an interval estimate of the mean satisfaction when Xh1 = 35, Xh2 = 45, and Xh3 = 2.2. Use a 90 percent confidence coefficient. Interpret your confidence interval. Here, the estimated mean regression is E(Yd |Xh ) = Xh b = 69.01029. The estimated variance of the estimated mean regression is −1 d Var(Ŷ |Xh ) = M SE × XTh XT X Xh = 7.100156. Hence, the 100(1 − α) confidence interval of E(Y |Xh ) is q d E(Yd |Xh ) ± t(1 − α/2; n − 4) × Var(Ŷ |Xh ) = (64.52854, 73.49204). In repeated sampling of size n = 46, 90 percent of the times the estimates of E(Y |Xh ) will be in (64.52854, 73.49204). ## R Codes ## X <- cbind(1, x1, x2, x3) Y <- matrix(y, ncol = 1) XX <- t(X) %*% X XY <- t(X) %*% Y b <- solve(XX) %*% XY Xh <- matrix(c(1, 35, 45, 2.2), nrow = 1) EYh <- Xh %*% b EYh VarEYh <- (Xh %*% solve(XX) %*% t(Xh)) * MSE VarEYh alpha <- 0.10 tval <- qt(1 - alpha/2, 42) c(EYh - tval * sqrt(VarEYh), EYh + tval * sqrt(VarEYh)) b. [10 Marks] Obtain a prediction interval for a new patient’s satisfaction when Xh1 = 35, Xh2 = 45, and Xh3 = 2.2. Use a 90 percent confidence coefficient. Interpret your confidence interval. Here, the estimated prediction is Ŷ |Xh = Xh b = 69.01029. The estimated variance of the prediction error is −1 d Var(pred) = M SE × 1 + XTh XT X Xh = 108.263. Hence, the 100(1 − α) confidence interval of Yh |Xh ) is q d = (51.50965, 86.51092). Ŷ |Xh ± t(1 − α/2; n − 4) × Var(pred) In repeated sampling of size n = 46, 90 percent of the times the predictions of Yh |Xh will be in (51.50965, 86.51092). 18 ## R Codes ## pred <- Xh %*% b pred VarPred <- (1 + (Xh %*% solve(XX) %*% t(Xh))) * MSE VarPred c(pred - tval * sqrt(VarPred), pred + tval * sqrt(VarPred)) 19