Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Theory of Regression - Jeremy Miles`s Page
Document related concepts
Types of artificial neural networks wikipedia , lookup
Foundations of statistics wikipedia , lookup
Bootstrapping (statistics) wikipedia , lookup
Degrees of freedom (statistics) wikipedia , lookup
History of statistics wikipedia , lookup
Taylor's law wikipedia , lookup
Transcript
Applying Regression 1 The Course β’ 14 (or so) lessons β Some flexibility β’ Depends how we feel β’ What we get through 2 Part I: Theory of Regression 1. Models in statistics 2. Models with more than one parameter: regression 3. Samples to populations 4. Introducing multiple regression 5. More on multiple regression 3 Part 2: Application of regression 6. 7. 8. 9. 10. 11. 12. Categorical predictor variables Assumptions in regression analysis Issues in regression analysis Non-linear regression Categorical and count variables Moderators (interactions) in regression Mediation and path analysis Part 3:Taking Regression Further (Kind of brief) 13. Introducing longitudinal multilevel models 4 Bonuses Bonus lesson1: Why is it called regression? Bonus lesson 2: Other types of regression. 5 House Rules β’ Jeremy must remember β Not to talk too fast β’ If you donβt understand β Ask β Any time β’ If you think Iβm wrong β Ask. (Iβm not always right) 6 The Assistants β’ Carla Xena - [email protected] β’ Eugenia Suarez Moran [email protected] β’ Arian [email protected] Learning New Techniques β’ Best kind of data to learn a new technique β Data that you know well, and understand β’ Your own data β In computer labs (esp later on) β Use your own data if you like β’ My data β Iβll provide you with β Simple examples, small sample sizes β’ Conceptually simple (even silly) 8 Computer Programs β’ Stata β Mostly β’ Iβll explain SPSS options β’ Youβll like Stata more β’ Excel β For calculations β Semi-optional β’ GPower 9 Lesson 1: Models in statistics Models, parsimony, error, mean, OLS estimators 10 What is a Model? 11 What is a model? β’ Representation β Of reality β Not reality β’ Model aeroplane represents a real aeroplane β If model aeroplane = real aeroplane, it isnβt a model 12 β’ Statistics is about modelling β Representing and simplifying β’ Sifting β What is important from what is not important β’ Parsimony β In statistical models we seek parsimony β Parsimony ο simplicity 13 Parsimony in Science β’ A model should be: β 1: able to explain a lot β 2: use as few concepts as possible β’ More it explains β The more you get β’ Fewer concepts β The lower the price β’ Is it worth paying a higher price for a better model? 14 The Mean as a Model 15 The (Arithmetic) Mean β’ We all know the mean β The βaverageβ β Learned about it at school β Forget (didnβt know) about how clever the mean is β’ The mean is: β An Ordinary Least Squares (OLS) estimator β Best Linear Unbiased Estimator (BLUE) 16 Mean as OLS Estimator β’ Going back a step or two β’ MODEL was a representation of DATA β We said we want a model that explains a lot β How much does a model explain? DATA = MODEL + ERROR ERROR = DATA - MODEL β We want a model with as little ERROR as possible 17 β’ What is error? Data (Y) Model (b0) mean Error (e) 1.40 -0.20 1.55 -0.05 1.80 1.60 0.20 1.62 0.02 1.63 0.03 18 β’ How can we calculate the βamountβ of error? β’ Sum of errors? β’ Sum of absolute errors? ERROR ο½ οei ο½ ο(Yi ο YΛ) ο½ ο(Yi ο b0 ) ο½ ο¨ο 0.20 ο© ο« ο¨ο 0.05 ο© ο« 0.20 ο« 0.02 ο« 0.03 ο½0 19 β’ Are small and large errors equivalent? β One error of 4 β Four errors of 1 β The same? β What happens with different data? β’ Y = (2, 2, 5) β b0 = 2 β Not very representative β’ Y = (2, 2, 4, 4) β b0 = any value from 2 - 4 β Indeterminate β’ There are an infinite number of solutions which would satisfy our criteria for minimum error 20 β’ Sum of squared errors (SSE) 2 i ERROR ο½ οe 2 Λ ο½ ο(Yi ο Y ) ο½ ο(Yi ο b0 ) 2 ο½ ο¨ο 0.20 ο© ο« ο¨ο 0.05 ο© ο« 0.202 ο« 0.022 ο« 0.032 2 2 ο½ 0.08 21 β’ Determinate β Always gives one answer β’ If we minimise SSE β Get the mean β’ Shown in graph β SSE plotted against b0 β Min value of SSE occurs when β b0 = mean 22 2 1.8 1.6 1.4 SSE 1.2 1 0.8 0.6 0.4 0.2 0 1 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2 b0 23 The Mean as an OLS Estimate 24 Mean as OLS Estimate β’ The mean is an Ordinary Least Squares (OLS) estimate β As are lots of other things β’ This is exciting because β OLS estimators are BLUE β Best Linear Unbiased Estimators β Proven with Gauss-Markov Theorem β’ Which we wonβt worry about 25 BLUE Estimators β’ Best β Minimum variance (of all possible unbiased estimators) β Narrower distribution than other estimators β’ e.g. median, mode Y ο½Y 26 SSE and the Standard Deviation β’ Tying up a loose end 2 Λ SSE ο½ ο(Yi ο Y ) sο½ 2 Λ ο(Yi ο Y ) n ο³ο½ 2 Λ ο(Yi ο Y ) n ο1 27 β’ SSE closely related to SD β’ Sample standard deviation β s β Biased estimator of population SD β’ Population standard deviation - ο³ β Need to know the mean to calculate SD β’ Reduces N by 1 β’ Hence divide by N-1, not N β Like losing one df 28 Proof β’ That the mean minimises SSE β Not that difficult β As statistical proofs go β’ Available in β Maxwell and Delaney β Designing experiments and analysing data β Judd and McClelland β Data Analysis: a model comparison approach β’ (out of print?) 29 Whatβs a df? β’ The number of parameters free to vary β When one is fixed β’ Term comes from engineering β Movement available to structures 30 Back to the Data β’ Mean has 5 (N) df β 1st moment β’ ο³ has N β1 df β Mean has been fixed β 2nd moment β Can think of it as amount of cases vary away from the mean 31 While we are at it β¦ β’ Skewness has N β 2 df β 3rd moment β’ Kurtosis has N β 3 df β 4rd moment β Amount cases vary from ο³ 32 Parsimony and df β’ Number of df remaining β Measure of parsimony β’ Model which contained all the data β Has 0 df β Not a parsimonious model β’ Normal distribution β Can be described in terms of mean and ο³ β’ 2 parameters β (z with 0 parameters) 33 Summary of Lesson 1 β’ Statistics is about modelling DATA β Models have parameters β Fewer parameters, more parsimony, better β’ Models need to minimise ERROR β Best model, least ERROR β Depends on how we define ERROR β If we define error as sum of squared deviations from predicted value β Mean is best MODEL 34 Lesson 1a β’ A really brief introduction to Stata 35 Command review Commands Output Variable list Commands 36 Stata Commands β’ Can use menus β But commands are easy β’ All have similar format: β’ command variables , options β’ Stata is case sensitive β BEDS, beds, Beds β’ Stata lets you shorten β summarize sqft β su sq 37 More Stata Commands β’ Open exercise 1.4.dta β Run β’ summarize sqm β’ table beds β’ mean price β’ histogram price β Or β’ su be β’ tab be β’ mean pr β’ hist pr 38 Lesson 2: Models with one more parameter - regression 39 In Lesson 1 we said β¦ β’ Use a model to predict and describe data β Mean is a simple, one parameter model 40 More Models Slopes and Intercepts 41 More Models β’ The mean is OK β As far as it goes β It just doesnβt go very far β Very simple prediction, uses very little information β’ We often have more information than that β We want to use more information than that 42 House Prices β’ Look at house prices in one area of Los Angeles β’ Predictors of house prices β’ Using: β Sale price, size, number of bedrooms, size of lot, year built β¦ 43 House Prices address listprice beds baths sqft 3628 OLYMPIAD Dr 649500 4 3 2575 3673 OLYMPIAD Dr 450000 2 3 1910 3838 CHANSON Dr 489900 3 2 2856 3838 West 58TH Pl 330000 4 2 1651 3919 West 58TH Pl 349000 3 2 1466 3954 FAIRWAY Blvd 514900 3 2.25 2018 4044 OLYMPIAD Dr 649000 4 2.5 3019 4336 DON LUIS Dr 474000 2 2.5 2188 4421 West 59TH St 460000 3 2 1519 4518 WHELAN Pl 388000 2 1.5 1403 4670 West 63RD St 259500 3 2 1491 5000 ANGELES VISTA Blvd 678800 5 4 3808 46 One Parameter Model β’ The mean Y ο½ 415.69 YΛ ο½ b0 ο½ Y SSE ο½ 341683 βHow much is that house worth?β $415,689 Use 1 df to say that 47 Adding More Parameters β’ We have more information than this β We might as well use it β Add a linear function of number of size (square feet) (x1) YΛ ο½ b0 ο« b1 x1 48 Alternative Expression β’ Estimate of Y (expected value of Y) YΛ ο½ b0 ο« b1 x1 β’ Value of Y Yi ο½ b0 ο« b1 xi1 ο« ei 49 Estimating the Model β’ We can estimate this model in four different, equivalent ways β Provides more than one way of thinking about it 1. 2. 3. 4. Estimating the slope which minimises SSE Examining the proportional reduction in SSE Calculating the covariance Looking at the efficiency of the predictions 50 Estimate the Slope to Minimise SSE 51 Estimate the Slope β’ Stage 1 β Draw a scatterplot β x-axis at mean β’ Not at zero β’ Mark errors on it β Called βresidualsβ β Sum and square these to find SSE 52 700 500 600 160 140 120 300 400 100 80 1.5 2 2.5 3 3.5 4 4.5 5 5.5 60 200 40 20 1000 0 2000 3000 4000 SQFT LAST SALE PRICE Fitted values 53 160 140 120 100 80 60 40 20 0 1.5 2 2.5 3 3.5 4 4.5 5 5.5 β’ Add another slope to the chart β Redraw residuals β Recalculate SSE β Move the line around to find slope which minimises SSE β’ Find the slope 55 β’ First attempt: 56 β’ Any straight line can be defined with two parameters β The location (height) of the slope β’ b0 β Sometimes called a β The gradient of the slope β’ b1 57 β’ Gradient b1 units 1 unit 58 β’ Height b0 units 59 β’ Height β’ If we fix slope to zero β Height becomes mean β Hence mean is b0 β’ Height is defined as the point that the slope hits the y-axis β The constant β The y-intercept 60 β’ Why the constant? β’ beds (x1) x0 £ (000s) β b0x0 1 1 77 β Where x0 is 1.00 for every case 2 1 74 β’ i.e. x0 is constant 1 1 88 3 1 62 Implicit in Stata 5 1 90 β (And SPSS, SAS, R) 5 1 136 β Some packages force you to2 make1it 35 explicit 5 1 134 β (Later on weβll need to make 4 it explicit) 1 138 1 1 55 61 β’ Why the intercept? β Where the regression line intercepts the yaxis β Sometimes called y-intercept 62 Finding the Slope β’ How do we find the values of b0 and b1? β Start with we jiggle the values, to find the best estimates which minimise SSE β Iterative approach β’ Computer intensive β used to matter, doesnβt really any more β’ (With fast computers and sensible search algorithms β more on that later) 63 β’ Start with β b0=416 (mean) β b1=0.5 (nice round number) β’ SSE = 365,774 β b0=300, b1=0.5, SSE=341,683 β b0=300, b1=0.6, SSE=310,240 β b0=300, b1=0.8, SSE=264,573 β b0=300, b1=1, SSE=301, 797 β b0=250, b1=1, SSE=255,366 β β¦.. 64 β’ Quite a long time later β b0 = 216.357 β b1 = 1.084 β SSE = 145,636.78 β’ Gives the position of the β Regression line (or) β Line of best fit β’ Better than guessing β’ Not necessarily the only method β But it is OLS, so it is the best (it is BLUE) 65 700 160 600 140 100 Price 400 500 120 80 300 60 Actual Price Predicted Price 40 200 20 0 1000 0.5 1 1.5 2000 2 2.5 3 SQFT 3.5 30004 4.5 5 4000 5.5 Number of Bedrooms Fitted values LAST SALE PRICE 66 β’ We now know β A zero square metre house is worth ο» $216,000 β Adding a square meter adds $1,080 β’ Told us two things β Donβt extrapolate to meaningless values of x-axis β Constant is not necessarily useful β’ It is necessary to estimate the equation 67 Exercise 2a, 2b 68 Standardised Regression Line β’ One big but: β Scale dependent β’ Values change β £ to β¬, inflation β’ Scales change β £, £000, £00? β’ Need to deal with this 69 β’ Donβt express in βrawβ units β Express in SD units β ο³x1=183.82 β ο³y=114.637 β’ b1 = 1.103 β’ We increase x1 by 1, and ΕΆ increases by 1.084 1.084 ο½ (1.084 / 114.637) SDs ο½ 0.00945SDs β’ So we increase x1 by 1 and ΕΆ increases by 0.0094 SDs 70 β’ Similarly, 1 unit of x1 = 1/69.017 SDs β Increase x1 by 1 SD β ΕΆ increases by 1.103 ο΄ (69.017/1) = 76.126 β’ Put them both together b1 ο΄ ο³ x1 ο³y 71 1.080 ο΄ 69.071 ο½ 0.653 114.637 β’ The standardised regression line β Change (in SDs) in ΕΆ associated with a change of 1 SD in x1 β’ A different route to the same answer β Standardise both variables (divide by SD) β Find line of best fit 72 β’ The standardised regression line has a special name The Correlation Coefficient (r) (r stands for βregressionβ, but more on that later) β’ Correlation coefficient is a standardised regression slope β Relative change, in terms of SDs 73 Exercise 2c 74 Proportional Reduction in Error 75 Proportional Reduction in Error β’ We might be interested in the level of improvement of the model β How much less error (as proportion) do we have β Proportional Reduction in Error (PRE) β’ Mean only β Error(model 0) = 341,683 β’ Mean + slope β Error(model 1) = 196,046 76 ERROR(0) ο ERROR(1) PRE ο½ ERROR(0) ERROR(1) PRE ο½ 1 ο ERROR(0) 196046 PRE ο½ 1 ο 341683 PRE ο½ 0.426 77 β’ But we squared all the errors in the first place β So we could take the square root 0.426 ο½ 0.653 β’ This is the correlation coefficient β’ Correlation coefficient is the square root of the proportion of variance explained 78 Standardised Covariance 79 Standardised Covariance β’ We are still iterating β Need a βclosed-formβ equation β Equation to solve to get the parameter estimates β’ Answer is a standardised covariance β A variable has variance β Amount of βdifferentnessβ β’ We have used SSE so far 80 β’ SSE varies with N β Higher N, higher SSE β’ Divide by N β Gives SSE per person (or house) β (Actually N β 1, we have lost a df to the mean) β’ Gives us the variance β’ Same as SD2 β We thought of SSE as a scattergram β’ Y plotted against X β (repeated image follows) 81 160 140 120 100 80 1.5 2 2.5 3 3.5 4 4.5 5 5.5 60 40 20 0 82 β’ Or we could plot Y against Y β Axes meet at the mean (415) β Draw a square for each point β Calculate an area for each square β Sum the areas β’ Sum of areas β SSE β’ Sum of areas divided by N β Variance 83 Plot of Y against Y 180 160 140 120 100 0 20 40 60 80 80 100 120 140 160 180 60 40 20 0 84 Draw Squares 180 Area = 40.1 x 40.1 = 1608.1 160 138 β 88.9 = 40.1 140 138 β 88.9 = 40.1 120 100 0 20 35 β 88.9 = -53.9 40 60 80 80 100 120 140 160 180 60 40 35 β 88.9 = -53.9 20 Area = -53.9 x -53.9 = 2905.21 0 85 β’ What if we do the same procedure β Instead of Y against Y β Y against X β’ β’ β’ β’ Draw rectangles (not squares) Sum the area Divide by N - 1 This gives us the variance of x with y β The Covariance β Shortened to Cov(x, y) 86 87 Area = (-33.9) x (-2) = 67.8 55 β 88.9 = -33.9 138-88.9 = 49.1 4-3=1 1 - 3 = -2 Area = 49.1 x 1 = 49.1 88 β’ More formally (and easily) β’ We can state what we are doing as an equation β Where Cov(x, y) is the covariance ο( x ο x )( y ο y ) Cov( x , y ) ο½ N ο1 β’ Cov(x,y)=5165 β’ What do points in different sectors do to the covariance? 89 β’ Problem with the covariance β Tells us about two things β The variance of X and Y β The covariance β’ Need to standardise it β Like the slope β’ Two ways to standardise the covariance β Standardise the variables first β’ Subtract from mean and divide by SD β Standardise the covariance afterwards 90 β’ First approach β Much more computationally expensive β’ Too much like hard work to do by hand β Need to standardise every value β’ Second approach β Much easier β Standardise the final value only β’ Need the combined variance β Multiply two variances β Find square root (were multiplied in first place) 91 β’ Standardised covariance ο½ Cov( x, y ) Var ( x ) ο΄ Var ( y ) 5165 ο½ 69.02 ο΄ 114.64 ο½ 0.653 92 β’ The correlation coefficient β A standardised covariance is a correlation coefficient rο½ Covariance ο¨ variance ο΄ variance ο© 93 β’ Expanded β¦ rο½ ο¦ ο( x ο x )( y ο y ) οΆ ο§ ο· N ο1 ο¨ οΈ 2 2 ο¦ ο( x ο x ) ο( y ο y ) οΆ ο§ο§ ο·ο· ο΄ N ο1 οΈ ο¨ N ο1 94 β’ This means β¦ β We now have a closed form equation to calculate the correlation β Which is the standardised slope β Which we can use to calculate the unstandardised slope 95 We know that: rο½ b1 ο΄ ο³ x1 ο³y We know that: b1 ο½ r ο΄ο³ y ο³x 1 96 b1 ο½ r ο΄ο³ y ο³x 1 0.659 ο΄ 114.64 b1 ο½ 69.017 b1 ο½ 1.080 β’ So value of b1 is the same as the iterative approach 97 β’ The intercept β Just while we are at it β’ The variables are centred at zero β We subtracted the mean from both variables β Intercept is zero, because the axes cross at the mean 98 β’ Add mean of y to the constant β Adjusts for centring y β’ Subtract mean of x β But not the whole mean of x β Need to correct it for the slope c ο½ y ο b1 x1 c ο½ 415.7 ο 1.08 ο΄ 183.81 c ο½ 216.35 99 Accuracy of Prediction 100 One More (Last One) β’ We have one more way to calculate the correlation β Looking at the accuracy of the prediction β’ Use the parameters β b0 and b1 β To calculate a predicted value for each case 101 Beds 239.2 177.4 265.3 153.4 136.2 187.5 280.5 203.3 141.1 130.3 Actual Predicted Price Price 605.0 475.8 400.0 408.8 529.5 504.1 315.0 382.7 341.0 364.0 525.0 419.7 585.0 520.5 430.0 436.8 436.0 369.4 390.0 357.7 β’ Plot actual price against predicted price β From the model 102 600 140 500 Predicted Value 120 100 80 400 60 40 300 20 20 40 200 60 300 80 100 400Actual Value 500 LAST SALE PRICE 120 600 140 160 700 103 β’ r = 0.653 β’ The correlation between actual and predicted value β’ Seems a futile thing to do β And at this stage, it is β But later on, we will see why 104 Some More Formulae β’ For hand calculation rο½ οxy οx 2 οy 2 β’ Point biserial ο¨ M rο½ y1 ο M y 0 ο©ο΄ PQ sd y 105 β’ Phi (f) β Used for 2 dichotomous variables Vote P Vote Q Homeowner A: 19 B: 54 Not homeowner C: 60 D:53 BC ο AD rο½ ( A ο« B)(C ο« D)( A ο« C )( B ο« D) 106 β’ Problem with the phi correlation β Unless Px= Py (or Px = 1 β Py) β’ Maximum (absolute) value is < 1.00 β’ Tetrachoric correlation can be used to correct this β’ Rank (Spearman) correlation β Used where data are ranked 6οd rο½ 2 n(n ο 1) 2 107 Summary β’ Mean is an OLS estimate β OLS estimates are BLUE β’ Regression line β Best prediction of outcome from predictor β OLS estimate (like mean) β’ Standardised regression line β A correlation 108 β’ Four ways to think about a correlation β 1. β 2. β 3. β 4. Standardised regression line Proportional Reduction in Error (PRE) Standardised covariance Accuracy of prediction 109 Regression and Correlation in Stata β’ Correlation: β’ correlate x y β’ correlate x y , cov β’ regress y x β’ Or β’ regress price sqm 110 Post-Estimation β’ Stata commands βleave behindβ something β’ You can run post-estimation commands β They mean βfrom the last regressionβ β’ Get predicted values: β predict my_preds This comes after the comma, so itβs an option β’ Get residuals: β predict my_res, residuals 111 Graphs β’ Scatterplot β’ scatter price beds β’ Regression line β lfit price beds β’ Both graphs β twoway (scatter price beds) (lfit price beds) 112 β’ What happens if you run reg without a predictor? β regress price 113 Exercises 114 Lesson 3: Samples to Populations β Standard Errors and Statistical Significance 115 The Problem β’ In Social Sciences β We investigate samples β’ Theoretically β Randomly taken from a specified population β Every member has an equal chance of being sampled β Sampling one member does not alter the chances of sampling another β’ Not the case in (say) physics, biology, etc. 116 Population β’ But itβs the population that we are interested in β Not the sample β Population statistic represented with Greek letter β Hat means βestimateβ Λ bο½ο’ x ο½ο Λx 117 β’ Sample statistics (e.g. mean) estimate population parameters β’ Want to know β Likely size of the parameter β If it is > 0 118 Sampling Distribution β’ We need to know the sampling distribution of a parameter estimate β How much does it vary from sample to sample β’ If we make some assumptions β We can know the sampling distribution of many statistics β Start with the mean 119 Sampling Distribution of the Mean β’ Given β Normal distribution β Random sample β Continuous data β’ Mean has a known sampling distribution β Repeatedly sampling will give a known distribution of means β Centred around the true (population) mean (ο) 120 Analysis Example: Memory β’ Difference in memory for different words β 10 participants given a list of 30 words to learn, and then tested β Two types of word β’ Abstract: e.g. love, justice β’ Concrete: e.g. carrot, table 121 Concrete Abstract 12 4 11 7 4 6 9 12 8 6 12 10 9 8 8 5 12 10 8 4 Diff (x) 8 4 -2 -3 2 2 1 3 2 4 x ο½ 2.1 ο³ x ο½ 3.11 N ο½ 10 122 Confidence Intervals β’ This means β If we know the mean in our sample β We can estimate where the mean in the population (ο) is likely to be β’ Using β The standard error (se) of the mean β Represents the standard deviation of the sampling distribution of the mean 123 1 SD contains 68% Almost 2 SDs contain 95% 124 β’ We know the sampling distribution of the mean β t distributed if N < 30 β Normal with large N (>30) β’ Asymptotically normal β’ Know the range within means from other samples will fall β Therefore the likely range of ο ο³x se( x ) ο½ n 125 β’ Two implications of equation β Increasing N decreases SE β’ But only a bit (SE halfs if N is 400 times bigger) β Decreasing SD decreases SE β’ Calculate Confidence Intervals β From standard errors β’ 95% is a standard level of CI β 95% of samples the true mean will lie within the 95% CIs β In large samples: 95% CI = 1.96 ο΄ SE β In smaller samples: depends on t distribution (df=N-1=9) 126 x ο½ 2.1, ο³ x ο½ 3.11, N ο½ 10 ο³x 3.11 se( x ) ο½ ο½ ο½ 0.98 n 10 127 95% CI ο½ 2.26 ο΄ 0.98 ο½ 2.22 x ο CI ο£ ο ο£ x ο« CI -0.12 ο£ ο ο£ 4.32 128 What is a CI? β’ (For 95% CI): β’ 95% chance that the true (population) value lies within the confidence interval? No; β’ 95% of samples, true mean will land within the confidence interval? 129 Significance Test β’ Probability that ο is a certain value β Almost always 0 β’ Doesnβt have to be though β’ We want to test the hypothesis that the difference is equal to 0 β i.e. find the probability of this difference occurring in our sample IF ο=0 β (Not the same as the probability that ο=0) 130 β’ Calculate SE, and then t β t has a known sampling distribution β Can test probability that a certain value is included x tο½ se(x ) 2.1 tο½ ο½ 2.14 0.98 p ο½ 0.061 131 Other Parameter Estimates β’ Same approach β Prediction, slope, intercept, predicted values β At this point, prediction and slope are the same β’ Wonβt be later on β’ One predictor only β More complicated with > 1 132 Testing the Degree of Prediction β’ Prediction is correlation of Y with ΕΆ β The correlation β when we have one IV β’ Use F, rather than t β’ Started with SSE for the mean only β This is SStotal β Divide this into SSresidual β SSregression β’ SStot = SSreg + SSres 133 Fο½ SSreg df1 SS res df 2 df1 ο½ k df 2 ο½ N ο k ο 1, 134 β’ Back to the house prices β Original SSE (SStotal) = 341683 β SSresidual = 196046 β’ What is left after our model β SSregression = 341683β 196046= 145636 β’ What our model explains 135 SSreg df1 Fο½ SS res df 2 145636 1 Fο½ ο½ 18.57 196046 ( 25 ο 1 ο 1) df1 ο½ k ο½ 1 df 2 ο½ N ο k ο 1 ο½ 8 136 β’ F = 18.6, df = 1, 25, p = 0.0002 β Can reject H0 β’ H0: Prediction is not better than chance β A significant effect 137 Statistical Significance: What does a p-value (really) mean? 138 A Quiz β’ Six questions, each true or false β’ Write down your answers (if you like) β’ An experiment has been done. Carried out perfectly. All assumptions perfectly satisfied. Absolutely no problems. β’ P = 0.01 β Which of the following can we say? 139 1. You have absolutely disproved the null hypothesis (that is, there is no difference between the population means). 140 2. You have found the probability of the null hypothesis being true. 141 3. You have absolutely proved your experimental hypothesis (that there is a difference between the population means). 142 4. You can deduce the probability of the experimental hypothesis being true. 143 5. You know, if you decide to reject the null hypothesis, the probability that you are making the wrong decision. 144 6. You have a reliable experimental finding in the sense that if, hypothetically, the experiment were repeated a great number of times, you would obtain a significant result on 99% of occasions. 145 OK, What is a p-value β’ Cohen (1994) β[a p-value] does not tell us what we want to know, and we so much want to know what we want to know that, out of desperation, we nevertheless believe it doesβ (p 997). 146 OK, What is a p-value β’ Sorry, didnβt answer the question β’ Itβs βThe probability of obtaining a result as or more extreme than the result we have in the study, given that the null hypothesis is trueβ β’ Not probability the null hypothesis is true 147 A Bit of Notation β’ Not because we like notation β But we have to say a lot less β’ β’ β’ β’ Probability β P Null hypothesis is true β H Result (data) β D Given - | 148 Whatβs a P Value β’ P(D|H) β Probability of the data occurring if the null hypothesis is true β’ Not β’ P(H|D) (what we want to know) β Probability that the null hypothesis is true, given that we have the data = p(H) β’ P(H|D) β P(D|H) 149 β’ What is probability you are prime minister β Given that you are British β P(M|B) β Very low β’ What is probability you are British β Given you are prime minister β P(B|M) β Very high β’ P(M|B) β P(B|M) 150 β’ Thereβs been a murder β Someone murdered an instructor (perhaps they talked too much) β’ The police have DNA β’ The police have your DNA β They match(!) β’ DNA matches 1 in 1,000,000 people β’ Whatβs the probability you didnβt do the murder, given the DNA match (H|D) 151 β’ Police say: β P(D|H) = 1/1,000,000 β’ Luckily, you have Jeremy on your defence team β’ We say: β P(D|H) β P(H|D) β’ Probability that someone matches the DNA, who didnβt do the murder β Incredibly high 152 Back to the Questions β’ Haller and Kraus (2002) β Asked those questions of groups in Germany β Psychology Students β Psychology lecturers and professors (who didnβt teach stats) β Psychology lecturers and professors (who did teach stats) 153 1. You have absolutely disproved the null hypothesis (that is, there is no difference between the population means). β’ True β’ β’ β’ β’ β’ 34% of students 15% of professors/lecturers, 10% of professors/lecturers teaching statistics False We have found evidence against the null hypothesis 154 2. You have found the probability of the null hypothesis being true. β 32% of students β 26% of professors/lecturers β 17% of professors/lecturers teaching statistics β’ β’ False We donβt know 155 3. You have absolutely proved your experimental hypothesis (that there is a difference between the population means). β β β β’ 20% of students 13% of professors/lecturers 10% of professors/lecturers teaching statistics False 156 4. You can deduce the probability of the experimental hypothesis being true. β 59% of students β 33% of professors/lecturers β 33% of professors/lecturers teaching statistics β’ False 157 5. You know, if you decide to reject the null hypothesis, the probability that you are making the wrong decision. β’ β’ β’ β’ β’ 68% of students 67% of professors/lecturers 73% of professors professors/lecturers teaching statistics False Can be worked out β P(replication) 158 6. You have a reliable experimental finding in the sense that if, hypothetically, the experiment were repeated a great number of times, you would obtain a significant result on 99% of occasions. β 41% of students β 49% of professors/lecturers β 37% of professors professors/lecturers teaching statistics β’ β’ False Another tricky one β It can be worked out 159 One Last Quiz β’ I carry out a study β All assumptions perfectly satisfied β Random sample from population β I find p = 0.05 β’ You replicate the study exactly β What is probability you find p < 0.05? 160 β’ I carry out a study β All assumptions perfectly satisfied β Random sample from population β I find p = 0.01 β’ You replicate the study exactly β What is probability you find p < 0.05? 161 β’ Significance testing creates boundaries and gaps where none exist. β’ Significance testing means that we find it hard to build upon knowledge β we donβt get an accumulation of knowledge 162 β’ Yates (1951) "the emphasis given to formal tests of significance ... has resulted in ... an undue concentration of effort by mathematical statisticians on investigations of tests of significance applicable to problems which are of little or no practical importance ... and ... it has caused scientific research workers to pay undue attention to the results of the tests of significance ... and too little to the estimates of the magnitude of the effects they are investigating 163 Testing the Slope β’ Same idea as with the mean β Estimate 95% CI of slope β Estimate significance of difference from a value (usually 0) β’ Need to know the SD of the slope β Similar to SD of the mean 164 s y. x ο½ 2 Λ ο(Y ο Y ) N ο k ο1 s y.x ο½ SSres N ο k ο1 s y.x ο½ 5921 ο½ 27.2 8 165 β’ Similar to equation for SD of mean β’ Then we need standard error - Similar (ish) β’ When we have standard error β Can go on to 95% CI β Significance of difference 166 se(by. x ) ο½ s y.x ο( x ο x ) 2 27.2 se(by. x ) ο½ ο½ 5.24 26.9 167 β’ Confidence Limits β’ 95% CI β t dist with N - k - 1 df is 2.31 β CI = 5.24 ο΄ 2.31 = 12.06 β’ 95% confidence limits 14.8 ο 12.1 ο£ ο’ ο£ 14.8 ο« 12.1 2.7 ο£ ο’ ο£ 26.9 168 β’ Significance of difference from zero β i.e. probability of getting result if ο’=0 β’ Not probability that ο’ = 0 b 14.7 tο½ ο½ ο½ 2.81 se(b) 5.2 df ο½ N ο k ο 1 ο½ 8 p ο½ 0.02 β’ This probability is (of course) the same as the value for the prediction 169 Testing the Standardised Slope (Correlation) β’ Correlation is bounded between β1 and +1 β Does not have symmetrical distribution, except around 0 β’ Need to transform it β Fisher zβ transformation β approximately normal zο’ ο½ 0.5[ln( 1 ο« r ) ο ln( 1 ο r )] 1 SE zο’ ο½ nο3 170 zο’ ο½ 0.5[ln( 1 ο« 0.706) ο ln( 1 ο 0.706)] zο’ ο½ 0.879 1 1 SEzο’ ο½ ο½ ο½ 0.38 nο3 10 ο 3 β’ 95% CIs β 0.879 β 1.96 * 0.38 = 0.13 β 0.879 + 1.96 * 0.38 = 1.62 171 β’ Transform back to correlation e ο1 r ο½ 2y e ο«1 2y β’ 95% CIs = 0.13 to 0.92 β’ Very wide β Because of small sample size β Maybe thatβs why CIs are not reported? 172 Using Excel β’ Functions in excel β Fisher() β to carry out Fisher transformation β Fisherinv() β to transform back to correlation 173 The Others β’ Same ideas for calculation of CIs and SEs for β Predicted score β Gives expected range of values given X β’ Same for intercept β But we have probably had enough 174 One more tricky thing β’ (Donβt worry if you donβt understand) β’ For means, regression estimates, etc β Estimate β’ 1.0000 β 95% confidence intervals β’ 0.0000, 2.0000 β P = 0.05000 β’ They match 175 β’ For correlations, odds ratios, etc β No longer match β’ 95% CIs β 0.0000, 0.50000 β’ P-value β 0.052000 β’ Because of the sampling distribution of the mean β Does not depend on the value β’ The sampling distribution of a proportion β Does depend on the value β More certainty around 0.9 than around 0.00. 176 Lesson 4: Introducing Multiple Regression 177 Residuals β’ We said Y = b0 + b1x1 β’ We could have said Yi = b0 + b1xi1 + ei β’ We ignored the i on the Y β’ And we ignored the ei β Itβs called error, after all β’ But it isnβt just error β Trying to tell us something 178 What Error Tells Us β’ Error tells us that a case has a different score for Y than we predict β There is something about that case β’ Called the residual β What is left over, after the model β’ Contains information β Something is making the residual οΉ 0 β But what? 179 700 160 600 140 100 Price 400 500 120 80 300 60 Actual Price Predicted Price 40 200 20 0 1000 0.5 1 1.5 2000 2 2.5 3 SQFT 3.5 30004 Number of Bedrooms LAST SALE PRICE 4.5 Fitted values 5 4000 5.5 181 β’ The residual (+ the mean) is the expected value of Y If all cases were equal on X β’ It is the value of Y, controlling for X β’ Other words: β Holding constant β Partialling β Residualising (residualised scores) β Conditioned on 182 β’ Sometimes adjustment is enough on its own β Measure performance against criteria β’ Teenage pregnancy rate β Measure pregnancy and abortion rate in areas β Control for socio-economic deprivation, religion, rural/urban and anything else important β See which areas have lower teenage pregnancy and abortion rate, given same level of deprivation β’ Value added education tables β Measure school performance β Control for initial intake 183 Adj Value (mean + resid) Sqm Price Predicted Residual 239.2 605.0 475.77 129.23 544.8 177.4 400.0 408.78 -8.78 406.8 265.3 529.5 504.08 25.42 441.0 153.4 315.0 382.69 -67.69 347.9 136.2 341.0 364.05 -23.05 392.6 187.5 525.0 419.66 105.34 520.9 280.5 585.0 520.51 64.49 480.1 203.3 430.0 436.79 -6.79 408.8 141.1 436.0 369.39 66.61 482.2 130.3 390.0 357.70 32.30 447.9 184 Control? β’ In experimental research β Use experimental control β e.g. same conditions, materials, time of day, accurate measures, random assignment to conditions β’ In non-experimental research β Canβt use experimental control β Use statistical control instead 185 Analysis of Residuals β’ What predicts differences in crime rate β After controlling for socio-economic deprivation β Number of police? β Crime prevention schemes? β Rural/Urban proportions? β Something else β’ This is (mostly) what multiple regression is about 186 β’ Exam performance β Consider number of books a student read (books) β Number of lectures (max 20) a student attended (attend) β’ Books and attend as IV, grade as outcome 187 Book s Attend 0 1 0 2 4 4 1 4 3 0 9 15 10 16 10 20 11 20 15 15 Grade 45 57 45 51 65 88 44 87 89 59 First 10 cases 188 β’ Use books as IV β R=0.492, F=12.1, df=1, 28, p=0.001 β b0=52.1, b1=5.7 β (Intercept makes sense) β’ Use attend as IV β R=0.482, F=11.5, df=1, 38, p=0.002 β b0=37.0, b1=1.9 β (Intercept makes less sense) 189 100 90 80 70 Grade (100) 60 50 40 30 -1 0 1 2 3 4 5 Books 190 100 90 80 70 60 Grade 50 40 30 5 7 9 11 13 15 17 19 21 Attend 191 Problem β’ Use R2 to give proportion of shared variance β Books = 24% β Attend = 23% β’ So we have explained 24% + 23% = 47% of the variance β NO!!!!! 192 β’ Look at the correlation matrix BOOKS 1 ATTEND 0.44 1 GRADE 0.49 0.48 1 BOOKS ATTEND GRADE β’ Correlation of books and attend is (unsurprisingly) not zero β Some of the variance that books shares with grade, is also shared by attend 193 β’ I have access to 2 cars β’ My wife has access to 2 cars β We have access to four cars? β No. We need to know how many of my 2 cars are shared β’ Similarly with regression β But we can do this with the residuals β Residuals are what is left after (say) books β See if residual variance is explained by attend β Can use this new residual variance to calculate SSres, SStotal and SSreg 194 β’ Well. Almost. β This would give us correct values for SS β Would not be correct for slopes, etc β’ Because assumes that the variables have a causal priority β Why should attend have to take what is left from books? β Why should books have to take what is left by attend? β’ Use OLS again; take variance they share 195 β’ Simultaneously estimate 2 parameters β b1 and b2 β Y = b0 + b1x1 + b2x2 β x1 and x2 are IVs β’ Shared variance β’ Not trying to fit a line any more β Trying to fit a plane β’ Can solve iteratively β Closed form equations better β But they are unwieldy 196 3D scatterplot (2points only) y x2 x1 197 b2 y b1 b0 x2 x1 198 199 Increasing Power β’ What if the predictors donβt correlate? β’ Regression is still good β It increases the power to detect effects β (More on power later) β’ Less variance left over β’ When do we know the two predictors donβt correlate? 200 (Really) Ridiculous Equations 2 ο¨ οο¨ y ο y ο©ο¨x1 ο x1 ο©ο©ο¨οο¨x2 ο x2 ο© ο© ο ο¨οο¨ y ο y ο©ο¨x2 ο x2 ο©ο©ο¨οο¨x1 ο x1 ο©ο¨x2 ο x2 ο©ο© b1 ο½ 2 2 2 οο¨x1 ο x1 ο© οο¨x2 ο x2 ο© ο ο¨οο¨x1 ο x1 ο©ο¨x2 ο x2 ο©ο© 2 ο¨ οο¨ y ο y ο©ο¨x2 ο x2 ο©ο©ο¨οο¨x1 ο x1 ο© ο© ο ο¨οο¨ y ο y ο©ο¨x1 ο x1 ο©ο©ο¨οο¨x2 ο x2 ο©ο¨x1 ο x1 ο©ο© b2 ο½ 2 2 2 οο¨x2 ο x2 ο© οο¨x1 ο x1 ο© ο ο¨οο¨x2 ο x2 ο©ο¨x1 ο x1 ο©ο© b0 ο½ y ο b1 x1 ο b2 x2 201 β’ The good news β There is an easier way β’ The bad news β It involves matrix algebra β’ The good news β We donβt really need to know how to do it 202 β’ Weβre not programming computers β So we usually donβt care β’ Very, very occasionally it helps to know what the computer is doing 203 Back to the Good News β’ We can calculate the standardised parameters as B=Rxx-1 x Rxy β’ Where β B is the vector of regression weights β Rxx-1 is the inverse of the correlation matrix of the independent (x) variables β Rxy is the vector of correlations of the correlations of the x and y variables 204 Exercise 4.2 205 Exercises β’ Exercise 4.1 β Grades data in Excel β’ Exercise 4.2 β Repeat in Stata β’ Exercise 4.3 β Zero correlation β’ Exercise 4.4 β Repeat therapy data β’ Exercise 4.5 β PTSD in families. 206 Lesson 5: More on Multiple Regression 207 Contents β’ More on parameter estimates β Standard errors of coefficients β’ R, R2, adjusted R2 β’ Extra bits β Suppressors β Decisions about control variables β Standardized estimates > 1 β Variable entry techniques 208 More on Parameter Estimates 209 Parameter Estimates β’ Parameter estimates (b1, b2 β¦ bk) were standardised β Because we analysed a correlation matrix β’ Represent the correlation of each IV with the outcome β When all other IVs are held constant 210 β’ Can also be unstandardised β’ Unstandardised represent the unit (rather than SDβs) change in the outcome associated with a 1 unit change in the IV β When all the other variables are held constant β’ Parameters have standard errors associated with them β As with one IV β Hence t-test, and associated probability can be calculated β’ Trickier than with one IV 211 Standard Error of Regression Coefficient β’ Standardised is easier 1ο R 1 SEο’i ο½ 2 n ο k ο1 1 ο R i 2 Y β R2i is the value of R2 when all other predictors are used as predictors of that variable β’ Note that if R2i = 0, the equation is the same as for previous 212 Multiple R 213 Multiple R β’ The degree of prediction β R (or Multiple R) β No longer equal to b β’ R2 Might be equal to the sum of squares of B β Only if all xβs are uncorrelated 214 In Terms of Variance β’ Can also think of R2 in terms of variance explained. β Each IV explains some variance in the outcome β The IVs share some of their variance β’ Canβt share the same variance twice 215 Variance in Y accounted for by x1 rx1y2 = 0.36 The total variance of Y =1 Variance in Y accounted for by x2 rx2y2 = 0.36 216 β’ In this model β R2 = ryx12 + ryx22 β R2 = 0.36 + 0.36 = 0.72 β R = ο0.72 = 0.85 β’ But β If x1 and x2 are correlated β No longer the case 217 Variance in Y accounted for by x1 rx1y2 = 0.36 The total variance of Y =1 Variance shared between x1 and x2 (not equal to rx1x2) Variance in Y accounted for by x2 rx2y2 = 0.36 218 β’ So β We can no longer sum the r2 β Need to sum them, and subtract the shared variance β i.e. the correlation β’ But β Itβs not the correlation between them β Itβs the correlation between them as a proportion of the variance of Y β’ Two different ways 219 β’ Based on estimates 2 R ο½ b1ryx1 ο« b2 ryx2 β’ If rx1x2 = 0 β rxy = bx1 β Equivalent to ryx12 + ryx22 220 β’ Based on correlations 2 R ο½ 2 yx1 r 2 yx2 ο«r ο 2ryx1 ryx2 rx1 x2 2 x1 x2 1οr β’ rx1x2 = 0 β Equivalent to ryx12 + ryx22 221 β’ Can also be calculated using methods we have seen β Based on PRE (predicted value) β Based on correlation with prediction β’ Same procedure with >2 IVs 222 Adjusted R2 β’ R2 is on average an overestimate of population value of R2 β Any x will not correlate 0 with Y β Any variation away from 0 increases R β Variation from 0 more pronounced with lower N β’ Need to correct R2 β Adjusted R2 223 β’ Calculation of Adj. R2 N ο1 Adj. R ο½ 1 ο (1 ο R ) N ο k ο1 2 2 β’ 1 β R2 β Proportion of unexplained variance β We multiple this by an adjustment β’ More variables β greater adjustment β’ More people β less adjustment 224 N ο1 N ο k ο1 N ο½ 20, k ο½ 3 20 ο 1 19 ο½ ο½ 1.1875 20 ο 3 ο 1 16 N ο½ 10, k ο½ 8 N ο½ 10, k ο½ 3 10 ο 1 9 ο½ ο½9 10 ο 8 ο 1 1 10 ο 1 9 ο½ ο½ 1.5 10 ο 3 ο 1 6 225 Extra Bits β’ Some stranger things that can happen β Counter-intuitive 226 Suppressor variables β’ Can be hard to understand β Very counter-intuitive β’ Definition β A predictor which increases the size of the parameters associated with other predictors above the size of their correlations 227 β’ An example (based on Horst, 1941) β Success of trainee pilots β Mechanical ability (x1), verbal ability (x2), success (y) β’ Correlation matrix Mech Mech Verb Success 1 0.5 0.3 Verb 0.5 1 0 Success 0.3 0 1 228 β Mechanical ability correlates 0.3 with success β Verbal ability correlates 0.0 with success β What will the parameter estimates be? β (Donβt look ahead until you have had a guess) 229 β’ Mechanical ability β b = 0.4 β Larger than r! β’ Verbal ability β b = -0.2 β Smaller than r!! β’ So what is happening? β You need verbal ability to do the mechanical ability test β Not actually related to mechanical ability β’ Measure of mechanical ability is contaminated by verbal ability 230 β’ High mech, low verbal β High mech β’ This is positive (.4) β Low verbal β’ Negative, because we are talking about standardised scores (-(-.2) ο (.2) β’ Your mech is really high β you did well on the mechanical test, without being good at the words β’ High mech, high verbal β Well, you had a head start on mech, because of verbal, and need to be brought down a bit 231 Another suppressor? x1 x2 y x1 1 0.5 0.3 x2 0.5 1 0.2 y 0.3 0.2 1 b1 = b2 = 232 Another suppressor? x1 x2 y x1 1 0.5 0.3 x2 0.5 1 0.2 y 0.3 0.2 1 b1 =0.26 b2 = -0.06 233 And another? x1 x2 y x1 1 0.5 0.3 x2 0.5 1 -0.2 y 0.3 -0.2 1 b1 = b2 = 234 And another? x1 x2 y x1 1 0.5 0.3 x2 0.5 1 -0.2 y 0.3 -0.2 1 b1 = 0.53 b2 = -0.47 235 One more? x1 x2 y x1 1 -0.5 0.3 x2 -0.5 1 0.2 y 0.3 0.2 1 b1 = b2 = 236 One more? x1 x2 y x1 1 -0.5 0.3 x2 -0.5 1 0.2 y 0.3 0.2 1 b1 = 0.53 b2 = 0.47 237 β’ Suppression happens when two opposing forces are happening together β And have opposite effects β’ Donβt throw away your IVs, β Just because they are uncorrelated with the outcome β’ Be careful in interpretation of regression estimates β Really need the correlations too, to interpret what is going on β Cannot compare between studies with different predictors β Think about what you want to know β’ Before throwing variables into the analysis 238 What to Control For? β’ What is the added value of a βbetterβ college β In terms of salary β More academic people go to βbetterβ colleges β Control for: β’ Ability? Social class? Motherβs education? Parentβs income? Course? Ethnic group? β¦ 239 β’ Decisions about control variables β Guided from theory β’ Effect of gender β Controlling for hair length and skirt wearing? 240 241 β’ Do dogs make kids healthier? β What to control for? Parentβs weight? β’ Yes: Obese parents are more likely to have obese kids, kids who are thinner, relative to the parents are thinner. β’ No: Dog might make parent thinner. By controlling for parental weight, youβre controlling for the effect of dog 242 Bad control vars Bad control vars Kidβs health Dog Good control vars Parent Weight Child Asthma Kidβs health Dog Rural/Urban? House/apartment? Income Standardised Estimates > 1 β’ Correlations are bounded -1.00 β€ r β€ +1.00 β We think of standardised regression estimates as being similarly bounded β’ But they are not β Can go >1.00, <-1.00 β R cannot, because that is a proportion of variance 246 β’ Three measures of ability β Mechanical ability, verbal ability 1, verbal ability 2 β Score on science exam Mech Mech Verbal1 Verbal2 Scores 1 0.1 0.1 0.6 Verbal1 0.1 1 0.9 0.6 Verbal2 0.1 0.9 1 0.3 Scores 0.6 0.6 0.3 1 βBefore reading on, what are the parameter estimates? 247 Mech Verbal1 Verbal2 0.56 1.71 -1.29 β’ Mechanical β About where we expect β’ Verbal 1 β Very high β’ Verbal 2 β Very low 248 β’ What is going on β Itβs a suppressor again β a predictor which increases the size of the parameters associated with other predictors above the size of their correlations β’ Verbal 1 and verbal 2 are correlated so highly β They need to cancel each other out 249 Variable Selection β’ What are the appropriate predictors to use in a model? β Depends what you are trying to do β’ Multiple regression has two separate uses β Prediction β Explanation 250 β’ Prediction β What will happen in the future? β Emphasis on practical application β Variables selected (more) empirically β Value free β’ Explanation β Why did something happen? β Emphasis on understanding phenomena β Variables selected theoretically β Not value free 251 β’ Visiting the doctor β Precedes suicide attempts β Predicts suicide β’ Does not explain suicide β’ More on causality later on β¦ β’ Which are appropriate variables β To collect data on? β To include in analysis? β Decision needs to be based on theoretical knowledge of the behaviour of those variables β Statistical analysis of those variables (later) β’ Unless you didnβt collect the data β Common sense (not a useful thing to say) 252 Variable Entry Techniques β’ Entry-wise β All variables entered simultaneously β’ Hierarchical β Variables entered in a predetermined order β’ Stepwise β Variables entered according to change in R2 β Actually a family of techniques 253 β’ Entrywise regression β All variables entered simultaneously β All treated equally β’ Hierarchical regression β Entered in a theoretically determined order β Change in R2 is assessed, and tested for significance β e.g. sex and age β’ Should not be treated equally with other variables β’ Sex and age MUST be first (unchangeable) β Confused with hierarchical linear modelling (MLM) 254 R-Squared Change ο¨ SSE1 ο SSE0 ο© /( df1 ο df 0 ) Fο½ SSE0 / df 0 β’ SSE0, df0 β’ SSE and df for first (smaller) model β’ SSE1, df1 β’ SSE and df for second (larger) model 255 β’ Stepwise β Variables entered empirically β Variable which increases R2 the most goes first β’ Then the next β¦ β Variables which have no effect can be removed from the equation β’ Example β House prices β whatβs important? β Size, lot size, list price, 256 β’ Stepwise Analysis β Data determines the order β Model 1: listing price, R2 = 0.87 β Model 2: listing price + lot size, R2 = 0.89 List Lot size b 0.81 0.02 p <0.001 0.02 257 β’ Hierarchical analysis β Theory determines the order β Model 1: Lot size+ House size, R2 = 0.499 β Model 2: + List price, R2 = 0.905 β Change in R2 = 0.41, p < 0.001 2 House size Lot size List price 0.18 0.15 0.75 0.20 0.03 <0.001 258 β’ Which is the best model? β Entrywise β OK β Stepwise β excluded age β’ Excluded size β MOST IMPORTANT PREDICTOR β Hierarchical β’ Listing price accounted for additional variance β Whoever decides the price has information that we donβt β’ Other problems with stepwise β F and df are wrong (cheats with df) β Unstable results β’ Small changes (sampling variance) β large differences in models 259 β Uses a lot of paper β Donβt use a stepwise procedure to pack your suitcase 260 Is Stepwise Always Evil? β’ Yes β’ All right, no β’ Research goal is entirely predictive (technological) β Not explanatory (scientific) β What happens, not why β’ N is large β 40 people per predictor, Cohen, Cohen, Aiken, West (2003) β’ Cross validation takes place 261 β’ Alternatives to stepwise regression β More recently developed β Used for genetic studies β’ 1000s of predictors, one outcome, small samples β Least Angle Regression β’ LARS (least angle regression) β’ Lasso (Least absolute shrinkage and selection operator) 262 Entry Methods in Stata β’ Entrywise β What regress does β’ Hierarchical β Two ways β Use hireg β Add on module β’ net search hireg β’ Then install 263 Hierarchical Regression β’ Use (on one line) β hireg outcome (block1var1 block1var2) (block2var1 block2var2) β’ Hireg reports β Parameter estimates for the two regressions β R2 for each model, change in R2 264 Model R2 1: 0.022 2: 0.513 p 0.136 0.000 F(df) 2.256(1,98) 50.987(2,97) R2 change 0.490 P value for the R2 F(df)change 97.497(1,97) p 0.000 P value for the change in R2 265 Hierarchical Regression (Contβ¦) β’ I donβt like hireg, for two reasons β Itβs different to regression β It only works for OLS regression, not logistic, multinomial, Poisson, etc β’ Alternative 2: β Use test β The p-value associated with the change in R2 for a variable β’ Equal to the p-value for that variable. 266 Hierarchical Regression (Contβ¦) β’ Example (using cars) β Parameters from final model: β hireg price () (extro) car | extro | Coef. .463 Std. Err. .1296 t 3.57 P>|t| 0.001 [95% Conf. Interval] .2004 .72626 β R2 change statistics R2 change 0.128 F(df) change 12.773(1,36) p 0.001 β (What is relationship between t and F?) β’ We know the p-value of the R2 change β When there is one predictor in the block β What about when thereβs more than one? 267 Hierarchical Regression (Cont) β’ test isnβt exactly what we want β But it is the same as what we want β’ Advantage of test β You can always use it β’ (I can always remember how it works) 268 (For SPSS) β’ SPSS calls them βblocksβ β’ Enter some variables, click βnext blockβ β Enter more variables β’ Click on βStatisticsβ β Click on R-squared change 269 Stepwise Regression β’ Add stepwise: prefix β’ With β Pr() β probability value to be removed from equation β Pe() β probability value to be entered into equation β’ stepwise, pe(0.05) pr(0.2): reg price sqm lotsize originallis 270 A quick note on R2 R2 is sometimes regarded as the βfitβ of a regression model β Bad idea β’ If good fit is required β maximise R2 β Leads to entering variables which do not make theoretical sense 271 Propensity Scores β’ Another method of controlling for variables β’ Ensure that predictors are uncorrelated with one predictor β Donβt need to control for them 272 xβs Uncorrelated? β’ Two cases when xβs are uncorrelated β’ Experimental design β Predictors are uncorrelated β We randomly assigned people to conditions to ensure that was the case β’ Sample weights β We can deliberately sample β’ Ensure that they are uncorrelated 273 β’ 20 β’ 20 β’ 20 β’ 20 women with college degree women without college degree men with college degree men without college degree β Or use post hoc sample weights β’ Propensity weighting β Weight to ensure that variables are uncorrelated β Usually done to avoid having to control β E.g. ethnic differences in PTSD symptoms β Can incorporate many more control variables β’ 100+ 274 Propensity Scores β’ Race profiling of police stops β Same time, place, area, etc β www.youtube.com/watch?v=Oot0BOaQTZI 275 Critique of Multiple Regression β’ Goertzel (2002) β βMyths of murder and multiple regressionβ β Skeptical Inquirer (Paper B1) β’ Econometrics and regression are βjunk scienceβ β Multiple regression models (in US) β Used to guide social policy 276 More Guns, Less Crime β (controlling for other factors) β’ Lott and Mustard: A 1% increase in gun ownership β 3.3% decrease in murder rates β’ But: β More guns in rural Southern US β More crime in urban North (crack cocaine epidemic at time of data) 277 Executions Cut Crime β’ No difference between crimes in states in US with or without death penalty β’ Ehrlich (1975) controlled all variables that affect crime rates β Death penalty had effect in reducing crime rate β’ No statistical way to decide whoβs right 278 Legalised Abortion β’ Donohue and Levitt (1999) β Legalised abortion in 1970βs cut crime in 1990βs β’ Lott and Whitley (2001) β βLegalising abortion decreased murder rates by β¦ 0.5 to 7 per cent.β β’ Itβs impossible to model these data β Controlling for other historical events β Crack cocaine (again) 279 β’ Crime is still dropping in the US β Despite the recession β’ Levitt says itβs mysterious, because the abortion effect should be over β’ Some suggest Xboxes, Playstations, etc β’ Netflix, DVRs β (Violent movies reduce crime). 280 Another Critique β’ Berk (2003) β Regression analysis: a constructive critique (Sage) β’ Three cheers for regression β As a descriptive technique β’ Two cheers for regression β As an inferential technique β’ One cheer for regression β As a causal analysis 281 Is Regression Useless? β’ Do regression carefully β Donβt go beyond data which you have a strong theoretical understanding of β’ Validate models β Where possible, validate predictive power of models in other areas, times, groups β’ Particularly important with stepwise 282 Lesson 6: Categorical Predictors 283 Introduction 284 Introduction β’ So far, just looked at continuous predictors β’ Also possible to use categorical (nominal, qualitative) predictors β e.g. Sex; Job; Religion; Region; Type (of anything) β’ Usually analysed with t-test/ANOVA 285 Historical Note β’ But these (t-test/ANOVA) are special cases of regression analysis β Aspects of General Linear Models (GLMs) β’ So why treat them differently? β Fisherβs fault β Computersβ fault β’ Regression, as we have seen, is computationally difficult β Matrix inversion and multiplication β Canβt do it, without a computer 286 β’ In the special cases where: β’ You have one categorical predictor β’ Your IVs are uncorrelated β It is much easier to do it by partitioning of sums of squares β’ These cases β Very rare in βappliedβ research β Very common in βexperimentalβ research β’ Fisher worked at Rothamsted agricultural research station β’ Never have problems manipulating wheat, pigs, cabbages, etc 287 β’ In psychology β Led to a split between βexperimentalβ psychologists and βcorrelationalβ psychologists β Experimental psychologists (until recently) would not think in terms of continuous variables β’ Still (too) common to dichotomise a variable β Too difficult to analyse it properly β Equivalent to discarding 1/3 of your data 288 The Approach 289 The Approach β’ Recode the nominal variable β Into one, or more, variables to represent that variable β’ Names are slightly confusing β Some texts talk of βdummy codingβ to refer to all of these techniques β Some (most) refer to βdummy codingβ to refer to one of them β Most have more than one name 290 β’ If a variable has g possible categories it is represented by g-1 variables β’ Simplest case: β Smokes: Yes or No β Variable 1 represents βYesβ β Variable 2 is redundant β’ If it isnβt yes, itβs no 291 The Techniques 292 β’ We will examine two coding schemes β Dummy coding β’ For two groups β’ For >2 groups β Effect coding β’ For >2 groups β’ Look at analysis of change β Equivalent to ANCOVA β Pretest-posttest designs 293 Dummy Coding β 2 Groups β’ Sometimes called βsimple codingβ β’ A categorical variable with two groups β’ One group chosen as a reference group β The other group is represented in a variable β’ e.g. 2 groups: Experimental (Group 1) and Control (Group 0) β Control is the reference group β Dummy variable represents experimental group β’ Call this variable βgroup1β 294 β’ For variable βgroup1β β 1 = βYesβ, 0=βNoβ Original Category Exp Con New Variable 1 0 295 β’ Some data β’ Group is x, score is y Control Group Experiment 1 Experiment 2 Experiment 3 Experimental Group 10 10 10 20 10 30 296 β’ Control Group = 0 β Intercept = Score on Y when x = 0 β Intercept = mean of control group β’ Experimental Group = 1 β b = change in Y when x increases 1 unit β b = difference between experimental group and control group 297 35 30 Gradient of slope 25 represents difference between means 20 15 10 5 0 Control Group Experiment 1 Experimental Group Experiment 2 Experiment 3 298 Dummy Coding β 3+ Groups β’ With three groups the approach is the similar β’ g = 3, therefore g-1 = 2 variables needed β’ 3 Groups β Control β Experimental Group 1 β Experimental Group 2 299 Original Category Con Gp1 Gp2 Gp1 Gp2 0 1 0 0 0 1 β’ Recoded into two variables β Note β do not need a 3rd variable β’ If we are not in group 1 or group 2 MUST be in control group β’ 3rd variable would add no information β’ (What would happen to determinant?) 300 β’ F and associated p β Tests H0 that g1 ο½ g 2 ο½ g3 β’ b1 and b2 and associated p-values β Test difference between each experimental group and the reference group β’ To test difference between experimental groups β Need to rerun analysis (or just do ANOVA with post-hoc tests) 301 β’ One more complication β Have now run multiple comparisons β Increases a β i.e. probability of type I error β’ Need to correct for this β Bonferroni correction β Multiply given p-values by two/three (depending how many comparisons were made) 302 Effect Coding β’ Usually used for 3+ groups β’ Compares each group (except the reference group) to the mean of all groups β Dummy coding compares each group to the reference group. β’ Example with 5 groups β 1 group selected as reference group β’ Group 5 303 β’ Each group (except reference) has a variable β 1 if the individual is in that group β 0 if not β -1 if in reference group group 1 2 3 4 5 group_1 group_2 group_3 group_4 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 -1 -1 -1 -1 304 Examples β’ Dummy coding and Effect Coding β’ Group 1 chosen as reference group each time β’ Data Group Mean SD 1 52.40 4.60 2 56.30 5.70 3 60.10 5.00 Total 56.27 5.88 305 β’ Dummy Group dummy2 dummy3 1 2 3 0 1 0 0 0 1 Group Effect2 effect3 1 2 3 -1 1 0 -1 0 1 β’ Effect 306 Dummy R=0.543, F=5.7, df=2, 27, p=0.009 b0 = 52.4, b1 = 3.9, p=0.100 b2 = 7.7, p=0.002 Effect R=0.543, F=5.7, df=2, 27, p=0.009 b0 = 56.27, b1 = 0.03, p=0.980 b2 = 3.8, p=0.007 b0 ο½ g1 b0 ο½ G b1 ο½ g2 ο g1 b1 ο½ g2 ο G b2 ο½ g3 ο g1 b2 ο½ g3 ο G 307 In Stata β’ Use xi: prefix for dummy coding β’ Use xi3: module for more codings β’ But β I donβt like it, I do it by hand β I donβt understand what itβs doing β It makes very long variables β’ And then I canβt use test β BUT: If doing stepwise, you need to keep the variables together β’ Example: xi: reg outcome contpred i.catpred This has changed in Stata 11. xi: no longer needed Put i. in front of categorical 308 predictors xi: reg salary i.job_description -----------------------------------------------------salary | Coef. Std. Err. t P>|t| -------------+---------------------------------------_Ijob_desc~2 | 3100.34 2023.76 1.53 0.126 _Ijob_desc~3 | 36139.2 1228.352 29.42 0.000 _cons | 27838.5 532.4865 52.28 0.000 ------------------------------------------------------ 309 Exercise 6.1 β’ 5 golf balls β Which is best? 310 In SPSS β’ SPSS provides two equivalent procedures for regression β Regression β GLM β GLM will: β Automatically code categorical variables β Automatically calculate interaction terms β Allow you to not understand β’ GLM wonβt: β Give standardised effects β Give hierarchical R2 p-values 311 ANCOVA and Regression 312 β’ Test β (Which is a trick; but itβs designed to make you think about it) β’ Use bank data (Ex 5.3) β Compare the pay rise (difference between salbegin and salary) β For ethnic minority and non-minority staff β’ What do you find? 313 ANCOVA and Regression β’ Dummy coding approach has one special use β In ANCOVA, for the analysis of change β’ Pre-test post-test experimental design β Control group and (one or more) experimental groups β Tempting to use difference score + t-test / mixed design ANOVA β Inappropriate 314 β’ Salivary cortisol levels β Used as a measure of stress β Not absolute level, but change in level over day may be interesting β’ Test at: 9.00am, 9.00pm β’ Two groups β High stress group (cancer biopsy) β’ Group 1 β Low stress group (no biopsy) β’ Group 0 315 High Stress Low Stress AM 20.1 22.3 PM 6.8 11.8 Diff 13.3 10.5 β’ Correlation of AM and PM = 0.493 (p=0.008) β’ Has there been a significant difference in the rate of change of salivary cortisol? β 3 different approaches 316 β’ Approach 1 β find the differences, do a t-test β t = 1.31, df=26, p=0.203 β’ Approach 2 β mixed ANOVA, look for interaction effect β F = 1.71, df = 1, 26, p = 0.203 β F = t2 β’ Approach 3 β regression (ANCOVA) based approach 317 β IVs: AM and group β outcome: PM β b1 (group) = 3.59, standardised b1=0.432, p = 0.01 β’ Why is the regression approach better? β The other two approaches took the difference β Assumes that r = 1.00 β Any difference from r = 1.00 and you add error variance β’ Subtracting error is the same as adding error 318 β’ Using regression β Ensures that all the variance that is subtracted is true β Reduces the error variance β’ Two effects β Adjusts the means β’ Compensates for differences between groups β Removes error variance β’ Data is am-pm cortisol 319 More on Change β’ If difference score is correlated with either pre-test or post-test β Subtraction fails to remove the difference between the scores β If two scores are uncorrelated β’ Difference will be correlated with both β’ Failure to control β Equal SDs, r = 0 β’ Correlation of change and pre-score =0.707 320 Even More on Change β’ A topic of surprising complexity β What I said about difference scores isnβt always true β’ Lordβs paradox β it depends on the precise question you want to answer β Collins and Horn (1993). Best methods for the analysis of change β Collins and Sayer (2001). New methods for the analysis of change β More later 321 Lesson 7: Assumptions in Regression Analysis 322 The Assumptions 1. The distribution of residuals is normal (at each value of the outcome). 2. The variance of the residuals for every set of values for the predictor is equal. β’ violation is called heteroscedasticity. 3. The error term is additive β’ no interactions. 4. At every value of the outcome the expected (mean) value of the residuals is zero β’ No non-linear relationships 323 5. The expected correlation between residuals, for any two cases, is 0. β’ The independence assumption (lack of autocorrelation) 6. All predictors are uncorrelated with the error term. 7. No predictors are a perfect linear function of other predictors (no perfect multicollinearity) 8. The mean of the error term is zero. 324 What are we going to do β¦ β’ Deal with some of these assumptions in some detail β’ Deal with others in passing only β look at them again later on 325 Assumption 1: The Distribution of Residuals is Normal at Every Value of the outcome 326 Look at Normal Distributions β’ A normal distribution β symmetrical, bell-shaped (so they say) 327 What can go wrong? β’ Skew β non-symmetricality β one tail longer than the other β’ Kurtosis β too flat or too peaked β kurtosed β’ Outliers β Individual cases which are far from the distribution 328 Effects on the Mean β’ Skew β biases the mean, in direction of skew β’ Kurtosis β mean not biased β standard deviation is β and hence standard errors, and significance tests 329 Examining Univariate Distributions β’ Graphs β Histograms β Boxplots β P-P plots β’ Calculation based methods 330 Histograms 30 A and B 30 20 20 10 10 0 0 331 β’ C and D 40 14 12 30 10 8 20 6 4 10 2 0 0 332 β’E&F 20 10 0 333 Histograms can be tricky β¦. 7 6 6 5 5 4 4 3 3 2 2 1 1 0 0 0 7 7 6 6 6 5 5 6 5 4 3 2 1 5 4 4 4 3 3 2 2 1 1 0 0 3 2 1 0 334 Boxplots 335 P-P Plots β’A&B 1.00 1.00 .75 .75 .50 .50 .25 .25 0.00 0.00 .25 .50 .75 1.00 0.00 0.00 .25 .50 .75 1.00 336 β’C&D 1.00 1.00 .75 .75 .50 .50 .25 .25 0.00 0.00 .25 .50 .75 1.00 0.00 0.00 .25 .50 .75 1.00 337 β’E&F 1.00 1.00 .75 .75 .50 .50 .25 .25 0.00 0.00 .25 .50 .75 1.00 0.00 0.00 .25 .50 .75 1.00 338 Calculation Based β’ Skew and Kurtosis statistics β’ Outlier detection statistics 339 Skew and Kurtosis Statistics β’ Normal distribution β skew = 0 β kurtosis = 0 β’ Two methods for calculation β Fisherβs and Pearsonβs β Very similar answers β’ Associated standard error β can be used for significance (t-test) of departure from normality β not actually very useful β’ Never normal above N = 400 340 Skewness Kurtosis A B C D E F -0.12 0.271 0.454 0.117 2.106 0.171 -0.084 0.265 1.885 -1.081 5.75 -0.21 341 Outlier Detection β’ Calculate distance from mean β z-score (number of standard deviations) β deleted z-score β’ that case biased the mean, so remove it β Look up expected distance from mean β’ 1% 3+ SDs 342 Non-Normality in Regression 343 Effects on OLS Estimates β’ The mean is an OLS estimate β’ The regression line is an OLS estimate β’ Lack of normality β biases the position of the regression slope β makes the standard errors wrong β’ probability values attached to statistical significance wrong 344 Checks on Normality β’ Check residuals are normally distributed β Draw histogram residuals β’ Use regression diagnostics β Lots of them β Most arenβt very interesting 345 Regression Diagnostics β’ Residuals β Standardised, studentised-deleted β look for cases > |3| (?) β’ Influence statistics β Look for the effect a case has β If we remove that case, do we get a different answer? β DFBeta, Standardised DFBeta β’ changes in b 346 β DfFit, Standardised DfFit β’ change in predicted value β’ Distances β measures of βdistanceβ from the centroid β some include IV, some donβt 347 More on Residuals β’ Residuals are trickier than you might have imagined β’ Raw residuals β OK β’ Standardised residuals β Residuals divided by SD se ο½ οe n ο k ο1 2 348 Standardised / Studentised β’ Now we can calculate the standardised residuals β SPSS calls them studentised residuals β Also called internally studentised residuals ei eiο’ ο½ se 1 ο hi 349 Deleted Studentised Residuals β’ Studentised residuals do not have a known distribution β Cannot use them for inference β’ Deleted studentised residuals β Externally studentised residuals β Studentized (jackknifed) residuals β’ Distributed as t β’ With df = N β k β 1 350 Testing Significance β’ We can calculate the probability of a residual β Is it sampled from the same population β’ BUT β Massive type I error rate β Bonferroni correct it β’ Multiply p value by N 351 Bivariate Normality β’ We didnβt just say βresiduals normally distributedβ β’ We said βat every value of the outcomesβ β’ Two variables can be normally distributed β univariate, β but not bivariate 352 β’ Coupleβs IQs β male and female FEMALE MALE 8 6 5 6 4 4 3 2 Frequency 2 0 60.0 70.0 80.0 90.0 100.0 110.0 120.0 130.0 1 0 140.0 60.0 70.0 80.0 90.0 100.0 110.0 120.0 130.0 140.0 βSeem reasonably normal 353 β’ But wait!! 160 140 120 100 80 MALE 60 40 40 60 80 100 120 140 160 FEMALE 354 β’ When we look at bivariate normality β not normal β there is an outlier β’ So plot X against Y β’ OK for bivariate β but β may be a multivariate outlier β Need to draw graph in 3+ dimensions β canβt draw a graph in 3 dimensions β’ But we can look at the residuals instead β¦ 355 β’ IQ histogram of residuals 12 10 8 6 4 2 0 356 Multivariate Outliers β¦ β’ Will be explored later in the exercises β’ So we move on β¦ 357 What to do about NonNormality β’ Skew and Kurtosis β Skew β much easier to deal with β Kurtosis β less serious anyway β’ Transform data β removes skew β positive skew β log transform β negative skew - square 358 Transformation β’ May need to transform IV and/or outcome β More often outcome β’ time, income, symptoms (e.g. depression) all positively skewed β can cause non-linear effects (more later) if only one is transformed β alters interpretation of unstandardised parameter β May alter meaning of variable β’ Some people say that this is such a big problem β Never transform β May add / remove non-linear and moderator effects 359 β’ Change measures β increase sensitivity at ranges β’ avoiding floor and ceiling effects β’ Outliers β Can be tricky β Why did the outlier occur? β’ Error? Delete them. β’ Weird person? Probably delete them β’ Normal person? Tricky. 360 β You are trying to model a process β’ is the data point βoutsideβ the process β’ e.g. lottery winners, when looking at salary β’ yawn, when looking at reaction time β Which is better? β’ A good model, which explains 99% of your data? (because we threw outliers out) β’ A poor model, which explains all of it (because we keep outliers in) β’ I prefer a good model 361 More on House Prices β’ Zillow.com tracks and predicts house prices β In the USA β’ Sometimes detects outliers β We donβt trust this selling price β We havenβt used it 362 Example in Stata β’ reg salary educ β’ predict res, res β’ hist res β’ gen logsalary= log(salary) β’ reg logsalary educ β’ predict logres, res β’ hist logres 363 4.0e-05 3.0e-05 0 Density 2.0e-05 1.0e-05 -20000 0 20000 40000 Residuals 60000 80000 2 1 0 Density 1.5 .5 -1 -.5 0 Residuals .5 1 But β¦ β’ Parameter estimates change β’ Interpretation of parameter estimate is different β’ Exercise 7.0, 7.1 366 Bootstrapping β’ Bootstrapping is very, very cool β’ And very, very clever β’ But very, very simple 367 Bootstrapping β’ When we estimate a test statistic (F or r or t or c2) β’ We rely on knowing the sampling distribution β’ Which we know β If the distributional assumptions are satisfied 368 Estimate the Distribution β’ Bootstrapping lets you: β Skip the bit about distribution β Estimate the sampling distribution from the data β’ This shouldnβt be allowed β Hence bootstrapping β But it is 369 How to Bootstrap β’ We resample, with replacement β’ Take our sample β Sample 1 individual β’ Put that individual back, so that they can be sampled again β Sample another individual β’ Keep going until weβve sampled as many people as were in the sample β’ Analyze the data β’ Repeat the process B times β Where B is a big number 370 Example Original 1 2 3 4 5 6 7 8 9 10 B1 1 1 3 3 3 3 7 7 9 9 B2 1 2 3 4 4 4 8 8 9 10 B3 2 2 3 2 4 4 6 7 9 9 371 β’ Analyze each dataset β Sampling distribution of statistic β’ Gives sampling distribution β’ 2 approaches to CI or P β’ Semi-parametric β Calculate standard error of statistic β Call that the standard deviation β Does not make assumption about distribution of data β’ Makes assumption about sampling distribution 372 β’ Non-parametric β Stata calls this percentile β’ Count. β If you have 1000 samples β 25th is lower CI β 975th is upper CI β P-value is proportion that cross zero β’ Non-parametric needs more samples 373 Bootstrapping in Stata β’ Very easy: β Use bootstrap: (or bs: or bstrap: ) prefix or β (Better) use vce(bootstrap) option β’ By default does 50 samples β Not enough β Use reps() β At least 1000 374 Example reg salary salbegin educ, vce(bootstrap, reps(50)) | Observed Bootstrap | Coef. Std. Err. z -----------+--------------------------------salbegin | 1.672631 .0863302 19.37 β’ Again salbegin | 1.672631 .0737315 22.69 375 More Reps β’ 1,000 reps β Z = 17.31 β’ Again β Z = 17.59 β’ 10,000 reps β 17.23 β 17.02 376 β’ Exercise 7.2, 7.3 377 Assumption 2: The variance of the residuals for every set of values for the predictor is equal. 378 Heteroscedasticity β’ This assumption is a about heteroscedasticity of the residuals β Hetero=different β Scedastic = scattered β’ We donβt want heteroscedasticity β we want our data to be homoscedastic β’ Draw a scatterplot to investigate 379 160 140 120 100 80 MALE 60 40 40 60 80 100 120 140 160 380 FEMALE β’ Only works with one IV β need every combination of IVs β’ Easy to get β use predicted values β use residuals there β’ Plot predicted values against residuals β’ A bit like turning the scatterplot to make the line of best fit flat 381 Good β no heteroscedasticity Predicted Value 382 Bad β heteroscedasticity Predicted Value 383 Testing Heteroscedasticity β’ Whiteβs test 1. 2. 3. 4. Do regression, save residuals. Square residuals Square IVs Calculate interactions of IVs β e.g. x1β’x2, x1β’x3, x2 β’ x3 384 5. Run regression using β squared residuals as outcome β IVs, squared IVs, and interactions as IVs 6. Test statistic = N x R2 β Distributed as c2 β Df = k (for second regression) β’ Use education and salbegin to predict salary (employee data.sav) β R2 = 0.113, N=474, c2 = 53.5, df=5, p < 0.0001 β’ Automatic in Stata β estat imtest, white 385 60000 40000 0 20000 -20000 -40000 Residuals Plot of Predicted and Residual 0 50000 100000 150000 Linear prediction 386 Whiteβs Test as Test of Interest β’ Possible to have a theory that predicts heteroscedasticity β’ Lupien, et al, 2006 β Heteroscedasticity in relationship of hippocampal volume and age 387 Magnitude of Heteroscedasticity β’ Chop data into 5 βslicesβ β Calculate variance of each slice β Check ratio of smallest to largest β Less than 5 β’ OK 388 gen slice = 1 replace slice replace slice replace slice replace slice = = = = 2 3 4 5 if if if if pred pred pred pred > > > > 30000 60000 90000 120000 bysort slice: su pred 1: 3954 5: 17116 (Doesnβt look too bad, thanks to skew in predictors) 389 Dealing with Heteroscedasticity β’ Use Huber-White (robust) estimates β β β’ Also called sandwich estimates Also called empirical estimates Use survey techniques β β Relatively straightforward in SAS and Stata, fiddly in SPSS Google: SPSS Huber-White 390 Whyβs it a Sandwich? β’ SE can be calculated with: ο1 n (X ' X ) ο1 β’ Sandwich estimator: ο1 ο1 ο1 ο1 ( n X ' X ) ( n X ' οX )( n X ' X ) ο1 391 Example β’ reg salary educ β Standard errors: β 204, 2821 β’ reg salary educ , robust β Standard errors: β 267 β 3347 β’ SEs usually go up, can go down 392 Heteroscedasticity β Implications and Meanings Implications β’ What happens as a result of heteroscedasticity? β Parameter estimates are correct β’ not biased β Standard errors (hence p-values) are incorrect 393 However β¦ β’ If there is no skew in predicted scores β P-values a tiny bit wrong β’ If skewed, β P-values can be very wrong β’ Exercise 7.4 394 Robust SE Haiku T-stat looks too good. Use robust standard errors significance gone 395 Meaning β’ What is heteroscedasticity trying to tell us? β Our model is wrong β it is misspecified β Something important is happening that we have not accounted for β’ e.g. amount of money given to charity (given) β depends on: β’ earnings β’ degree of importance person assigns to the charity (import) 396 β’ Do the regression analysis β R2 = 0.60,, p < 0.001 β’ seems quite good β b0 = 0.24, p=0.97 β b1 = 0.71, p < 0.001 β b2 = 0.23, p = 0.031 β’ Whiteβs test β c2 = 18.6, df=5, p=0.002 β’ The plot of predicted values against residuals β¦ 397 20 10 0 -10 -20 30 40 50 Linear prediction 60 70 β’ Plot shows heteroscedastic relationship 398 β’ Which means β¦ β the effects of the variables are not additive β If you think that what a charity does is important β’ you might give more money β’ how much more depends on how much money you have 399 70 60 50 40 30 20 5 10 import given Fitted values 15 given Fitted values 400 β’ One more thing about heteroscedasticity β it is the equivalent of homogeneity of variance in ANOVA/t-tests 401 β’ Exercise 7.4, 7.5, 7.6 402 Assumption 3: The Error Term is Additive 403 Additivity β’ What heteroscedasticity shows you β effects of variables need to be additive (assume no interaction between the variables) β’ Heteroscedasticity doesnβt always show it to you β can test for it, but hard work β (same as homogeneity of covariance assumption in ANCOVA) β’ Have to know it from your theory β’ A specification error 404 Additivity and Theory β’ Two IVs β Alcohol has sedative effect β’ A bit makes you a bit tired β’ A lot makes you very tired β Some painkillers have sedative effect β’ A bit makes you a bit tired β’ A lot makes you very tired β A bit of alcohol and a bit of painkiller doesnβt make you very tired β Effects multiply together, donβt add together 405 β’ If you donβt test for it β Itβs very hard to know that it will happen β’ So many possible non-additive effects β Cannot test for all of them β Can test for obvious β’ In medicine β Choose to test for salient non-additive effects β e.g. sex, race β’ More on this, when we look at moderators 406 β’ Exercise 7.6 β’ Exercise 7.7 407 Assumption 4: At every value of the outcome the expected (mean) value of the residuals is zero 408 Linearity β’ Relationships between variables should be linear β best represented by a straight line β’ Not a very common problem in social sciences β measures are not sufficiently accurate (much measurement error) to make a difference β’ R2 too low β’ unlike, say, physics 409 Fuel β’ Relationship between speed of travel and fuel used Speed 410 β’ R2 = 0.938 β looks pretty good β know speed, make a good prediction of fuel β’ BUT β look at the chart β if we know speed we can make a perfect prediction of fuel used β R2 should be 1.00 411 Detecting Non-Linearity β’ Residual plot β just like heteroscedasticity β’ Using this example β very, very obvious β usually pretty obvious 412 Residual plot 413 Linearity: A Case of Additivity β’ Linearity = additivity along the range of the IV β’ Jeremy rides his bicycle harder β Increase in speed depends on current speed β Not additive, multiplicative β MacCallum and Mar (1995). Distinguishing between moderator and quadratic effects in multiple regression. Psychological Bulletin. 414 Assumption 5: The expected correlation between residuals, for any two cases, is 0. The independence assumption (lack of autocorrelation) 415 Independence Assumption β’ Also: lack of autocorrelation β’ Tricky one β often ignored β exists for almost all tests β’ All cases should be independent of one another β knowing the value of one case should not tell you anything about the value of other cases 416 How is it Detected? β’ Can be difficult β need some clever statistics (multilevel models) β’ Better off avoiding situations where it arises β Or handling it when it does arise β’ Residual Plots 417 Residual Plots β’ Were data collected in time order? β If so plot ID number against the residuals β Look for any pattern β’ Test for linear relationship β’ Non-linear relationship β’ Heteroscedasticity 418 2 Residual 1 0 -1 -2 0 10 20 30 40 Participant Number 419 How does it arise? Two main ways β’ time-series analyses β When cases are time periods β’ weather on Tuesday and weather on Wednesday correlated β’ inflation 1972, inflation 1973 are correlated β’ clusters of cases β patients treated by three doctors β children from different classes β people assessed in groups 420 Why does it matter? β’ Standard errors can be wrong β therefore significance tests can be wrong β’ Parameter estimates can be wrong β really, really wrong β from positive to negative β’ An example β students do an exam (on statistics) β choose one of three questions β’ IV: time β’ outcome: grade 421 β’Result, with line of best fit 90 80 70 60 50 40 Grade 30 20 10 10 Time 20 30 40 50 60 70 422 β’ Result shows that β people who spent longer in the exam, achieve better grades β’ BUT β¦ β we havenβt considered which question people answered β we might have violated the independence assumption β’ outcome will be autocorrelated β’ Look again β with questions marked 423 β’ Now somewhat different 90 80 70 60 50 40 Question 30 Grade 3 20 2 10 10 1 20 30 40 50 60 70 Time 424 β’ Now, people that spent longer got lower grades β questions differed in difficulty β do a hard one, get better grade β if you can do it, you can do it quickly 425 Dealing with NonIndependence β’ For time series data β Time series analysis (another course) β Multilevel models (hard, some another course) β’ For clustered data β Robust standard errors β Generalized estimating equations β Multilevel models 426 Cluster Robust Standard Errors β’ Predictor: School size β’ Outcome: Grades β’ Sample: β 20 schools β 20 children per school β’ What is the N? 427 Robust Standard Errors β’ Sample is: β 400 children β is it 400? β Not really β’ Each child adds information β’ First child in a school adds lots of information about that school β 100th child in a school adds less informationβ β How much less depends on how similar the children in the school are β 20 schools β’ Itβs more than 20 428 Robust SE in Stata β’ Very easy β’ reg predictor outcome , robust cluster(clusterid) β’ BUT β Only to be used where clustering is a nuisance only β’ Only adjusts standard errors, not parameter estimates β’ Only to be used where parameter estimates shouldnβt be affected by clustering 429 Example of Robust SE β’ Effects of incentives for attendance at adult literacy class β Some students rewarded for attendance β Others not rewarded β’ 152 classes randomly assigned to each condition β Scores measured at mid term and final 430 Example of Robust SE β’ Naïve β reg postscore tx midscore β Est: -.6798066 SE: .7218797 β’ Clustered β reg postscore tx midscore, robust cluster(classid) β Est: -.6798066 SE .9329929 431 Problem with Robust Estimates β’ Only corrects standard error β Does not correct estimate β’ Other predictors must be uncorrelated with predictors of group membership β Or estimates wrong β’ Two alternatives: β Generalized estimating equations (gee) β Multilevel models 432 Independence + Heteroscedasticity β’ Assumption is that residuals are: β Independently and identically distributed β’ i.i.d. β’ Same procedure used for both problems β Really, same problem 433 β’ Exercise 7.9, exercise 7.10 434 Assumption 6: All predictor variables are uncorrelated with the error term. 435 Uncorrelated with the Error Term β’ A curious assumption β by definition, the residuals are uncorrelated with the predictors (try it and see, if you like) β’ There are no other predictors that are important β That correlate with the error β i.e. Have an effect 436 β’ Problem in economics β Demand increases supply β Supply increases wages β Higher wages increase demand β’ OLS estimates will be (badly) biased in this case β need a different estimation procedure β two-stage least squares β’ simultaneous equation modelling β Instrumental variables 437 Another Haiku Supply and demand: without a good instrument, not identified. 438 Assumption 7: No predictors are a perfect linear function of other predictors no perfect multicollinearity 439 No Perfect Multicollinearity β’ IVs must not be linear functions of one another β matrix of correlations of IVs is not positive definite β cannot be inverted β analysis cannot proceed β’ Have seen this with β age, age start, time working (canβt have all three in the model) β also occurs with subscale and total in model at the same time 440 β’ Large amounts of collinearity β a problem (as we shall see) sometimes β not an assumption β’ Exercise 7.11 441 Assumption 8: The mean of the error term is zero. You will like this one. 442 Mean of the Error Term = 0 β’ Mean of the residuals = 0 β’ That is what the constant is for β if the mean of the error term deviates from zero, the constant soaks it up Y ο½ ο’ 0 ο« ο’1 x1 ο« ο₯ Y ο½ ( ο’ 0 ο« 3) ο« ο’1 x1 ο« (ο₯ ο 3) - note, Greek letters because we are talking about population values 443 β’ Can do regression without the constant β Usually a bad idea β E.g R2 = 0.995, p < 0.001 β’ Looks good 444 13 12 y 11 10 9 8 7 6 7 8 9 10 11 12 13 x1 445 Lesson 8: Issues in Regression Analysis Things that alter the interpretation of the regression equation 446 The Four Issues β’ β’ β’ β’ Causality Sample sizes Collinearity Measurement error 447 Causality 448 What is a Cause? β’ Debate about definition of cause β some statistics (and philosophy) books try to avoid it completely β We are not going into depth β’ just going to show why it is hard β’ Two dimensions of cause β Ultimate versus proximal cause β Determinate versus probabilistic 449 Proximal versus Ultimate β’ Why am I here? β I walked here because β This is the location of the class because β Eric Tanenbaum asked me because β (I donβt know) β because I was in my office when he rang because β I was a lecturer at Derby University because β I saw an advert in the paper because 450 β I exist because β My parents met because β My father had a job β¦ β’ Proximal cause β the direct and immediate cause of something β’ Ultimate cause β the thing that started the process off β I fell off my bicycle because of the bump β I fell off because I was going too fast 451 Determinate versus Probabilistic Cause β’ Why did I fall off my bicycle? β I was going too fast β But every time I ride too fast, I donβt fall off β Probabilistic cause β’ Why did my tyre go flat? β A nail was stuck in my tyre β Every time a nail sticks in my tyre, the tyre goes flat β Deterministic cause 452 β’ Can get into trouble by mixing them together β Eating deep fried Mars Bars and doing no exercise are causes of heart disease β βMy Grandad ate three deep fried Mars Bars every day, and the most exercise he ever got was when he walked to the shop next door to buy oneβ β (Deliberately?) confusing deterministic and probabilistic causes 453 Criteria for Causation β’ Association (correlation) β’ Direction of Influence (a ο b) β’ Isolation (not c ο a and c ο b) 454 Association β’ Correlation does not mean causation β we all know β’ But β Causation does mean correlation β’ Need to show that two things are related β may be correlation β may be regression when controlling for third (or more) factor 455 β’ Relationship between price and sales β suppliers may be cunning β when people want it more β’ stick the price up Price Price Demand Sales 1 0.6 0 Demand 0.6 1 0.6 Sales 0 0.6 1 β So β no relationship between price and sales 456 β Until (or course) we control for demand β b1 (Price) = -0.56 β b2 (Demand) = 0.94 β’ But which variables do we enter? 457 Direction of Influence β’ Relationship between A and B β three possible processes A B A B B causes A A B C causes A & B C A causes B 458 β’ How do we establish the direction of influence? β Longitudinally? Barometer Drops Storm β Now if we could just get that barometer needle to stay where it is β¦ β’ Where the role of theory comes in (more on this later) 459 Isolation β’ Isolate the outcome from all other influences β as experimenters try to do β’ Cannot do this β can statistically isolate the effect β using multiple regression 460 Role of Theory β’ Strong theory is crucial to making causal statements β’ Fisher said: to make causal statements βmake your theories elaborate.β β donβt rely purely on statistical analysis β’ Need strong theory to guide analyses β what critics of non-experimental research donβt understand 461 β’ S.J. Gould β a critic β says correlate price of petrol and his age, for the last 10 years β find a correlation β Ha! (He says) that doesnβt mean there is a causal link β Of course not! (We say). β’ No social scientist would do that analysis without first thinking (very hard) about the possible causal relations between the variables of interest β’ Would control for time, prices, etc β¦ 462 β’ Atkinson, et al. (1996) β relationship between college grades and number of hours worked β negative correlation β Need to control for other variables β ability, intelligence β’ Gould says βMost correlations are noncausalβ (1982, p243) β Of course!!!! 463 I drink a lot of beer 16 causal relations 120 non-causal correlations laugh bathroom jokes (about statistics) children wake early karaoke curtains closed sleeping headache equations (beermat) thirsty fried breakfast no beer curry chips falling over lose keys 464 β’ Abelson (1995) elaborates on this β βmethod of signaturesβ β’ A collection of correlations relating to the process β the βsignatureβ of the process β’ e.g. tobacco smoking and lung cancer β can we account for all of these findings with any other theory? 465 1. 2. 3. 4. 5. 6. 7. 8. The longer a person has smoked cigarettes, the greater the risk of cancer. The more cigarettes a person smokes over a given time period, the greater the risk of cancer. People who stop smoking have lower cancer rates than do those who keep smoking. Smokerβs cancers tend to occur in the lungs, and be of a particular type. Smokers have elevated rates of other diseases. People who smoke cigars or pipes, and do not usually inhale, have abnormally high rates of lip cancer. Smokers of filter-tipped cigarettes have lower cancer rates than other cigarette smokers. Non-smokers who live with smokers have elevated cancer rates. (Abelson, 1995: 183-184) 466 β In addition, should be no anomalous correlations β’ If smokers had more fallen arches than nonsmokers, not consistent with theory β’ Failure to use theory to select appropriate variables β specification error β e.g. in previous example β Predict wealth from price and sales β’ increase price, price increases β’ Increase sales, price increases 467 β’ Sometimes these are indicators of the process, not the process itself β e.g. barometer β stopping the needle wonβt help β e.g. inflation? Indicator or cause of economic health? 468 No Causation without Experimentation β’ Blatantly untrue β I donβt doubt that the sun shining makes us warm β’ Why the aversion? β Pearl (2000) says problem is that there is no mathematical operator (e.g. β=β) β No one realised that you needed one β Until you build a robot 469 AI and Causality β’ A robot needs to make judgements about causality β’ Needs to have a mathematical representation of causality β Suddenly, a problem! β Doesnβt exist β’ Most operators are non-directional β’ Causality is directional 470 Sample Sizes βHow many subjects does it take to run a regression analysis?β 471 Introduction β’ Social scientists donβt worry enough about the sample size required β βWhy didnβt you get a significant result?β β βI didnβt have a large enough sampleβ β’ Not a common answer, but very common reason β’ More recently awareness of sample size is increasing β use too few β no point doing the research β use too many β waste their time 472 β’ Research funding bodies β’ Ethical review panels β both become more interested in sample size calculations β’ We will look at two approaches β Rules of thumb (quite quickly) β Power Analysis (more slowly) 473 Rules of Thumb β’ Lots of simple rules of thumb exist β 10 cases per IV β and at least 100 cases β Green (1991) more sophisticated β’ To test significance of R2 β N = 50 + 8k β’ To test significance of slopes, N = 104 + k β’ Rules of thumb donβt take into account all the information that we have β Power analysis does 474 Power Analysis Introducing Power Analysis β’ Hypothesis test β tells us the probability of a result of that magnitude occurring, if the null hypothesis is correct (i.e. there is no effect in the population) β’ Doesnβt tell us β the probability of that result, if the null hypothesis is false (i.e., there actually is an effect in the population) 475 β’ According to Cohen (1982) all null hypotheses are false β everything that might have an effect, does have an effect β’ it is just that the effect is often very tiny 476 Type I Errors β’ Type I error is false rejection of H0 β’ Probability of making a type I error β a β the significance value cut-off β’ usually 0.05 (by convention) β’ Always this value β’ Not affected by β sample size β type of test 477 Type II errors β’ Type II error is false acceptance of the null hypothesis β Much, much trickier β’ We think we have some idea β we almost certainly donβt β’ Example β I do an experiment (random sampling, all assumptions perfectly satisfied) β I find p = 0.05 478 β You repeat the experiment exactly β’ different random sample from same population β What is probability you will find p < 0.05? β Answer: 0.5 β Another experiment, I find p = 0.01 β Probability you find p < 0.05? β Answer: 0.79 β’ Very hard to work out β not intuitive β need to understand non-central sampling distributions (more in a minute) 479 β’ Probability of type II error = beta (ο’) β same as population regression parameter (to be confusing) β’ Power = 1 β Beta β Probability of getting a significant result (given that there is a significant result to be found) 480 State of the World Research Findings H0 True (no effect to be found) H0 false (effect to be found) H0 true (we find no effect β p > 0.05) οΌ Type II error p=ο’ power = 1 - ο’ H0 false (we find an effect β p < 0.05) Type I error p=a οΌ 481 β’ Four parameters in power analysis β a β prob. of Type I error β ο’ β prob. of Type II error (power = 1 β ο’) β Effect size β size of effect in population βN β’ Know any three, can calculate the fourth β Look at them one at a time 482 β’ a Probability of Type I error β Usually set to 0.05 β Somewhat arbitrary β’ sometimes adjusted because of circumstances β rarely because of power analysis β May want to adjust it, based on power analysis 483 β’ ο’ β Probability of type II error β Power (probability of finding a result) =1βο’ β Standard is 80% β’ Some argue for 90% β Implication that Type I error is 4 times more serious than type II error β’ adjust ratio with compromise power analysis 484 β’ Effect size in the population β Most problematic to determine β Three ways 1. What effect size would be useful to find? β’ R2 = 0.01 - no use (probably) 2. Base it on previous research β what have other people found? 3. Use Cohenβs conventions β small R2 = 0.02 β medium R2 = 0.13 β large R2 = 0.26 485 β Effect size usually measured as f2 β For R2 2 R f ο½ 2 1ο R 2 486 β For (standardised) slopes 2 sri f ο½ 2 1ο R 2 β Where sr2 is the contribution to the variance accounted for by the variable of interest β i.e. sr2 = R2 (with variable) β R2 (without) β’ change in R2 in hierarchical regression 487 β’ N β the sample size β usually use other three parameters to determine this β sometimes adjust other parameters (a) based on this β e.g. You can have 50 participants. No more. 488 Doing power analysis β’ With power analysis program β SamplePower, Gpower (free), Nquery β With Stata command sampsi β’ Which I find very confusing β’ But weβll use it anyway 489 sampsi β’ Limited in usefulness β A categorical, two group predictor β’ sampsi 0 0.5, pre(1) r01(0.5) n1(50) sd(1) β Find power for detecting an effect of 0.5 β’ When thereβs one other variable at baseline β’ Which correlates 0.5 β’ 50 people in each group β’ When sd is 1.0 490 sampsi β¦ Method: ANCOVA relative efficiency = adjustment to sd = adjusted sd1 = 1.143 0.935 0.935 Estimated power: power = 0.762 491 GPower β’ Better for regression designs 492 Underpowered Studies β’ Research in the social sciences is often underpowered β Why? β See Paper B11 β βthe persistence of underpowered studiesβ 495 Extra Reading β’ Power traditionally focuses on p values β What about CIs? β Paper B8 β βObtaining regression coefficients that are accurate, not simply significantβ 496 β’ Exercise 8.1 497 Collinearity 498 Collinearity as Issue and Assumption β’ Collinearity (multicollinearity) β the extent to which the predictors are (multiply) correlated β’ If R2 for any IV, using other IVs = 1.00 β perfect collinearity β variable is linear sum of other variables β regression will not proceed β (SPSS will arbitrarily throw out a variable) 499 β’ R2 < 1.00, but high β other problems may arise β’ Four things to look at in collinearity β meaning β implications β detection β actions 500 Meaning of Collinearity β’ Literally βco-linearityβ β lying along the same line β’ Perfect collinearity β when some IVs predict another β Total = S1 + S2 + S3 + S4 β S1 = Total β (S2 + S3 + S4) β rare 501 β’ Less than perfect β when some IVs are close to predicting other IVs β correlations between IVs are high (usually, but not always) ο high multiple correlations 502 Implications β’ Effects the stability of the parameter estimates β and so the standard errors of the parameter estimates β and so the significance and CIs β’ Because β shared variance, which the regression procedure doesnβt know where to put 503 β’ Sex differences β due to genetics? β due to upbringing? β (almost) perfect collinearity β’ statistically impossible to tell 504 β’ When collinearity is less than perfect β increases variability of estimates between samples β estimates are unstable β reflected in the variances, and hence standard errors 505 Detecting Collinearity β’ Look at the parameter estimates β large standardised parameter estimates (>0.3?), which are not significant β’ be suspicious β’ Run a series of regressions β each IV as outcome β all other IVs as IVs β’ for each IV 506 β’ Sounds like hard work? β SPSS does it for us! β’ Ask for collinearity diagnostics β Tolerance β calculated for every IV Tolerance ο½ 1-R 2 β Variance Inflation Factor β’ sq. root of amount s.e. has been increased 1 VIF ο½ Tolerance 507 Actions What you can do about collinearity βno quick fixβ (Fox, 1991) 1. Get new data β’ β’ β’ avoids the problem address the question in a different way e.g. find people who have been raised as the βwrongβ gender β’ β’ exist, but rare Not a very useful suggestion 508 2. Collect more data β’ β’ β’ not different data, more data collinearity increases standard error (se) se decreases as N increases β’ get a bigger N 3. Remove / Combine variables β’ β’ β’ If an IV correlates highly with other IVs Not telling us much new If you have two (or more) IVs which are very similar β’ e.g. 2 measures of depression, socioeconomic status, achievement, etc 509 β’ β’ sum them, average them, remove one Many measures β’ use principal components analysis to reduce them 3. Use stepwise regression (or some flavour of) β’ β’ See previous comments Can be useful in theoretical vacuum 4. Ridge regression β’ β’ not very useful behaves weirdly 510 β’ Exercise 8.2, 8.3, 8.4 511 Measurement Error 512 What is Measurement Error β’ In social science, it is unlikely that we measure any variable perfectly β measurement error represents this imperfection β’ We assume that we have a true score β T β’ A measure of that score βx 513 xο½T ο«e β’ just like a regression equation β standardise the parameters β T is the reliability β’ the amount of variance in x which comes from T β’ but, like a regression equation β assume that e is random and has mean of zero β more on that later 514 Simple Effects of Measurement Error β’ Lowers the measured correlation β between two variables β’ Real correlation β true scores (x* and y*) β’ Measured correlation β measured scores (x and y) 515 True correlation of x and y rx*y* x* e y* Reliability of x rxx Reliability of y ryy x y Measured correlation of x and y rxy e 516 β’ Attenuation of correlation rxy ο½ rx * y * ο΄ rxx ryy β’ Attenuation corrected correlation rx * y * ο½ rxy rxx ryy 517 β’ Example rxx ο½ 0.7 ryy ο½ 0.8 rxy ο½ 0.3 rx* y* ο½ rx* y* rxy rxx ryy 0.3 ο½ ο½ 0.40 0.7 ο΄ 0.8 518 Complex Effects of Measurement Error β’ Really horribly complex β’ Measurement error reduces correlations β reduces estimate of ο’ β reducing one estimate β’ increases others β because of effects of control β combined with effects of suppressor variables β exercise to examine this 519 Dealing with Measurement Error β’ Attenuation correction β very dangerous β not recommended β’ Avoid in the first place β use reliable measures β donβt discard information β’ donβt categorise β’ Age: 10-20, 21-30, 31-40 β¦ 520 Complications β’ Assume measurement error is β additive β linear β’ Additive β e.g. weight β people may under-report / overreport at the extremes β’ Linear β particularly the case when using proxy variables 521 β’ e.g. proxy measures β Want to know effort on childcare, count number of children β’ 1st child is more effort than 19th child β Want to know financial status, count income β’ 1st £1 much greater effect on financial status than the 1,000,000th. 522 β’ Exercise 8.5 523 Lesson 9: Non-Linear Analysis in Regression 524 Introduction β’ Non-linear effect occurs β when the effect of one predictor β is not consistent across the range of the IV β’ Assumption is violated β expected value of residuals = 0 β no longer the case 525 Some Examples 526 Skill A Learning Curve Experience 527 Performance Yerkes-Dodson Law of Arousal Arousal 528 Suicidal Enthusiastic Enthusiasm Levels over a Lesson on Regression 0 Time 3.5 529 β’ Learning β line changed direction once β’ Yerkes-Dodson β line changed direction once β’ Enthusiasm β line changed direction twice 530 Everything is Non-Linear β’ Every relationship we look at is nonlinear, for two reasons β Exam results cannot keep increasing with reading more books β’ Linear in the range we examine β For small departures from linearity β’ Cannot detect the difference β’ Non-parsimonious solution 531 Non-Linear Transformations 532 Bending the Line β’ Non-linear regression is hard β We cheat, and linearise the data β’ Do linear regression Transformations β’ We need to transform the data β rather than estimating a curved line β’ which would be very difficult β’ may not work with OLS β we can take a straight line, and bend it β or take a curved line, and straighten it β’ back to linear (OLS) regression 533 β’ We still do linear regression β Linear in the parameters β Y = b1x + b2x2 + β¦ β’ Can do non-linear regression β Non-linear in the parameters β Y = b1x + b2x2 + β¦ β’ Much trickier β Statistical theory either breaks down OR becomes harder 534 β’ Linear transformations β multiply by a constant β add a constant β change the slope and the intercept 535 y=2x y y=x + 3 y=x x 536 β’ Linear transformations are no use β alter the slope and intercept β donβt alter the standardised parameter estimate β’ Non-linear transformation β will bend the slope β quadratic transformation y = x2 β one change of direction 537 β Cubic transformation y = x2 + x3 β two changes of direction 538 β’ To estimate a non-linear regression β we donβt actually estimate anything nonlinear β we transform the x-variable to a non-linear version β can estimate that straight line β represents the curve β we donβt bend the line, we stretch the space around the line, and make it flat 539 Detecting Non-linearity 540 Draw a Scatterplot β’ Draw a scatterplot of y plotted against x β see if it looks a bit non-linear β e.g. Education and beginning salary β’ from bank data β’ with line of best fit 541 A Real Example β’ Starting salary and years of education β From employee data.sav 542 80000 0 20000 40000 60000 Expected value of error (residual) is > 0 5 10 Expected 20 value of error beginning salary (residual) is < 0 15 educational level (years) Fitted values 543 Use Residual Plot β’ Scatterplot is only good for one variable β use the residual plot (that we used for heteroscedasticity) β’ Good for many variables 544 β’ We want β points to lie in a nice straight sausage 545 β’ We donβt want β a nasty bent sausage 546 -20000 0 20000 40000 60000 β’ Educational level and starting salary 5000 10000 15000 20000 Linear prediction Fitted values 25000 30000 Residuals 547 Carrying Out Non-Linear Regression 548 Linear Transformation β’ Linear transformation doesnβt change β interpretation of slope β standardised slope β se, t, or p of slope β R2 β’ Can change β effect of a transformation 549 β’ Actually more complex β with some transformations can add a constant with no effect (e.g. quadratic) β’ With others does have an effect β inverse, log β’ Sometimes it is necessary to add a constant β negative numbers have no square root β 0 has no log 550 Education and Salary Linear Regression β’ Saw previously that the assumption of expected errors = 0 was violated β’ Anyway β¦ β R2 = 0.401, p < 0.001 β salbegin = -6290 + 1727 ο΄ educ β Standardised β’ b1 (educ) = 0.633 β Both parameters make sense 551 Non-linear Effect β’ Compute new variable β quadratic β educ2 = educ2 β’ Add this variable to the equation β R2 = 0.585, p < 0.001 β salbegin = 46263 + -6542 ο΄ educ + 310 ο΄ educ2 β’ slightly curious β Standardised β’ b1 (educ) = -2.4 β’ b2 (educ2) = 3.1 β What is going on? 552 β’ Collinearity β is what is going on β Correlation of educ and educ2 β’ r = 0.990 β Regression equation becomes difficult (impossible?) to interpret β’ Need hierarchical regression β what is the change in R2 β is that change significant? β R2 (change) = 0.184, p < 0.001 553 Cubic Effect β’ While we are at it, letβs look at the cubic effect β R2 (change) = 0.004, p = 0.045 β 19138 + 103 ο΄ e + -206 ο΄ e2 + 12 ο΄ e3 β Standardised: b1(e) = 0.04 b2(e2) = -2.04 b3(e3) = 2.71 554 Fourth Power β’ Keep going while we are ahead? β When do we stop? 555 Interpretation β’ Tricky, given that parameter estimates are a bit nonsensical β’ Two methods β’ 1: Use R2 change β Save predicted values β’ or calculate predicted values to plot line of best fit β Save them from equation β Plot against IV 556 80000 60000 40000 0 20000 5 10 15 educational level (years) Linear prediction Linear prediction Linear prediction 20 Linear prediction Linear prediction beginning salary 557 β’ Differentiate with respect to e β’ We said: s = 19138 + 103 ο΄ e + -206 ο΄ e2 + 12 ο΄ e3 β but first we will simplify it to quadratic s = 46263 + -6542 ο΄ e + 310 ο΄ e2 β’ dy/dx = -6542 + 310 x 2 x e 558 Education Slope 9 -962 10 -342 11 278 12 898 13 1518 14 2138 15 2758 16 3378 17 3998 18 4618 19 5238 20 5858 1 year of education at the higher end of the scale, better than 1 year at the lower end of the scale. MBA versus GCSE 559 β’ Differentiate Cubic 19138 + 103 ο΄ e + -206 ο΄ e2 + 12 ο΄ e3 dy/dx = 103 β 206 ο΄ 2 ο΄ e + 12 ο΄ 3 ο΄ e2 β’ Can calculate slopes for quadratic and cubic at different values 560 Education Slope (Quad) Slope (Cub) 9 -962 -689 10 -342 -417 11 278 -73 12 898 343 13 1518 831 14 2138 1391 15 2758 2023 16 3378 2727 17 3998 3503 18 4618 4351 19 5238 5271 20 5858 6263 561 A Quick Note on Differentiation β’ For y = xp β dx/dy = pxp-1 β’ For equations such as y =b1x + b2xP dy/dx = b1 + b2pxp-1 β’ y = 3x + 4x2 β dy/dx = 3 + 4 β’ 2x 562 β’ y = b1x + b2x2 + b3x3 β dy/dx = b1 + b2 β’ 2x + b3 β’ 3 β’ x2 β’ y = 4x + 5x2 + 6x3 β’ dx/dy = 4 + 5 β’ 2 β’ x + 6 β’ 3 β’ x2 β’ Many functions are simple to differentiate β Not all though 563 Splines and Knots β’ Estimate a different slope following an event β Lines are splines β Events are knots β’ Event might be known β Marriage β’ Might be unknown β How many years after brain injury does recovery start 564 Lesson 10: Regression for Counts and Categories Dichotomous/Nominal outcomes 565 Contents β’ General and Generalized Linear Models β’ Dichotomous β logistic / probit β’ Counts β Poisson and negative binomial 566 GLMs and GLMs β’ General linear models β Ordinary least squares regression based models β Identity link function β Regression, ANOVA, correlation, etc β’ Generalized linear models β More links β More error structures β General linear models are a subset of generalized linear models 567 Dichotomous β’ Often in social sciences, we have a dichotomous/nominal outcome β we will look at dichotomous first, then a quick look at multinomial β’ Dichotomous outcome β’ e.g. β β β β guilty/not guilty pass/fail won/lost Alive/dead (used in medicine) 568 Why Wonβt OLS Do? 569 Example: PTSD in Veterans β’ How does length of deployment affect probability of PTSD? β Have PTSD, or donβt. β We might be interested in severity β’ Army are not β’ If you have PTSD, you need help β Not going back β’ Develop a selection procedure β Two predictor variables β Rank β 1 =Staff Sgt, 5 = Private, β Deployment length (months) 570 β’ 1st ten cases Rank 5 1 1 4 1 1 4 1 3 4 Months 6 15 12 6 15 6 16 10 12 26 PTSD 0 0 0 0 1 0 1 1 0 1 571 β’ outcome β PTSD (1 = Yes, 0 = No) β’ Just consider score first β Carry out regression β Rank as predictor, PTSD as outcome β R2 = 0.097, F = 4.1, df = 1, 48, p = 0.028. β b0 = 0.190 β b1 = 0.110, p=0.028 β’ Seems OK 572 β’ Residual plot 573 β’ Problems 1 and 2 β strange distributions of residuals β parameter estimates may be wrong β standard errors will certainly be wrong 574 β’ 2nd problem β interpretation β I have rank 2 β Pass = 0.190 + 0.110 ο΄ 2 = 0.41 β I have rank 8 β Pass = 0.190 + 0.110 ο΄ 8 = 1.07 β’ Seems OK, but β What does it mean? β Cannot score 0.41 or 1.07 β’ can only score 0 or 1 β’ Cannot be interpreted β need a different approach 575 A Different Approach Logistic Regression 576 Logit Transformation β’ In lesson 9, transformed IVs β now transform the outcome β’ Need a transformation which gives us β graduated scores (between 0 and 1) β No upper limit β’ we canβt predict someone will pass twice β No lower limit β’ you canβt do worse than fail 577 Step 1: Convert to Probability β’ First, stop talking about values β talk about probability β for each value of score, calculate probability of pass β’ Solves the problem of graduated scales 578 probability of PTSD given a rank of 1 is 0.7 Score 1 2 3 4 5 No N 7 5 6 4 2 PTSD P 0.7 0.5 0.6 0.4 0.2 N 3 5 4 6 8 PTSD P 0.3 0.5 0.4 0.6 0.8 probability of PTSD given a rank of 5 is 0.2 579 This is better β’ Now a score of 0.41 has a meaning β a 0.41 probability of pass β’ But a score of 1.07 has no meaning β cannot have a probability > 1 (or < 0) β Need another transformation 580 Step 2: Convert to Odds-Ratio Need to remove upper limit β’ Convert to odds β’ Odds, as used by betting shops β 5:1, 1:2 β’ Slightly different from odds in speech β a 1 in 2 chance β odds are 1:1 (evens) β 50% 581 β’ Odds ratio = (number of times it happened) / (number of times it didnβt happen) p(event) p(event ) odds ratio ο½ ο½ p(not event ) 1 ο p(event ) 582 β’ 0.8 = 0.8/0.2 = 4 β equivalent to 4:1 (odds on) β 4 times out of five β’ 0.2 = 0.2/0.8 = 0.25 β equivalent to 1:4 (4:1 against) β 1 time out of five 583 β’ Now we have solved the upper bound problem β we can interpret 1.07, 2.07, 1000000.07 β’ But we still have the zero problem β we cannot interpret predicted scores less than zero 584 Step 3: The Log β’ Log10 of a number(x) log( x ) 10 ο½x β’ log(10) = 1 β’ log(100) = 2 β’ log(1000) = 3 585 β’ log(1) = 0 β’ log(0.1) = -1 β’ log(0.00001) = -5 586 Natural Logs and e β’ Donβt use log10 β Use loge β’ Natural log, ln β’ Has some desirable properties, that log10 doesnβt β β β β For us If y = ln(x) + c dy/dx = 1/x Not true for any other logarithm 587 β’ Be careful β calculators and stats packages are not consistent when they use log β Sometimes log10, sometimes loge 588 Take the natural log of the odds ratio β’ Goes from -ο₯ ο +ο₯ β can interpret any predicted value 589 Putting them all together β’ Logit transformation β log-odds ratio β not bounded at zero or one 590 Score 1 No PTSD PTSD N P N P Odds (No PTSD) log(odds)No PTSD 2 3 4 7 5 6 4 0.7 0.5 0.6 0.4 3 5 4 6 0.3 0.5 0.4 0.6 5 2 0.2 8 0.8 2.33 1.00 1.50 0.67 0.25 0.85 0.00 0.41 -0.41 -1.39 591 probability 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 Probability gets closer to zero, but never reaches it as logit goes down. -3.5 -3 -2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5 3 Logit 592 3.5 β’ Hooray! Problem solved, lesson over β errrmmmβ¦ almost β’ Because we are now using log-odds ratio, we canβt use OLS β we need a new technique, called Maximum Likelihood (ML) to estimate the parameters 593 Parameter Estimation using ML ML tries to find estimates of model parameters that are most likely to give rise to the pattern of observations in the sample data β’ All gets a bit complicated β OLS is a special case of ML β the mean is an ML estimator 594 β’ Donβt have closed form equations β must be solved iteratively β estimates parameters that are most likely to give rise to the patterns observed in the data β by maximising the likelihood function (LF) β’ We arenβt going to worry about this β except to note that sometimes, the estimates do not converge β’ ML cannot find a solution 595 R2 in Logistic Regression β’ A dichotomous variable doesnβt have variance β If you know the mean (proportion) you know the variance β You canβt have R2. β’ There are several pseudo-R2 β’ None are perfect β Thereβs something better 596 Logistic Regression in Stata β’ Exercise 10.1 β’ Two (almost) equivalent commands β logistic ptsd rank deployment β logit ptsd rank deployment 597 Logit β’ Gives output in log-odds β’ logit ptsd rank deployment -----------------------------------------------------------------------------pass | Odds Ratio Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------deployment | 1.158213 .0841987 2.02 0.043 1.004404 1.335575 rank | 1.333192 .4011279 0.96 0.339 .7392395 2.404365 ------------------------------------------------------------------------------ 598 Logistic β’ Gives output in odds ratios β’ β No intercept logit ptsd rank deployment -----------------------------------------------------------------------------pass | Odds Ratio Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------deployment | 1.158213 .0841987 2.02 0.043 1.004404 1.335575 rank | 1.333192 .4011279 0.96 0.339 .7392395 2.404365 ------------------------------------------------------------------------------ 599 β’ SPSS produces a classification table β And Stata produces it if you ask β predictions of model β based on cut-off of 0.5 (by default) β predicted values x actual values β’ DO NOT USE IT! β’ Will this person go to prison? β No. β You will be right 99.9% of the time β Doesnβt mean you have a good model β (Gottman and Murray β Blink) 600 Classification Tablea Predicted PASS Observed Step 1 PASS 0 Percentage Correct 1 0 18 8 69.2 1 12 12 50.0 Overall Percentage 60.0 a. The cut value is .500 601 Model parameters β’B β Change in the logged odds associated with a change of 1 unit in IV β just like OLS regression β difficult to interpret β’ SE (B) β Standard error β Multiply by 1.96 to get 95% CIs 602 β’ Constant β i.e. score = 0 β B = 1.314 β Exp(B) = eB = e1.314 = 3.720 β OR = 3.720, p = 1 β (1 / (OR + 1)) = 1 β (1 / (3.720 + 1)) β p = 0.788 603 β’ Score 1 β Constant b = 1.314 β Score B = -0.467 β Exp(1.314 β 0.467) = Exp(0.847) = 2.332 β OR = 2.332 β p = 1 β (1 / (2.332 + 1)) = 0.699 604 Standard Errors and CIs β’ Symmetrical in B β Non-symmetrical (sometimes very) in exp(B) 605 β’ The odds of failing the test are multiplied by 0.63 (CIs = 0.408, 0.962 p = 0.033), for every additional point on the aptitude test. 606 Hierarchical Logistic Regression β’ In OLS regression β Use R2 change β’ In logistic regression β Use chi-square change β’ Difference in chi-square = chi-square β’ Difference in df = df 607 Hierarchical Logistic Regression β’ Model 1: Experience β’ Model 2: Experience + Score β’ Model 1: β Chi-square =4.83, df = 1 β’ Model 2: β Chi-square =5.77, df = 2 608 β’ Difference: β Chi-square = 5.77-4.83= 1.94, β Df = 2 β 1 = 1 β’ gen p = 1 - chi2(1, 1.94) β’ tab p β’ p = 0.332 β’ P-value from SE = 0.339 β’ Why? 609 More on Standard Errors β’ Because of Wald standard errors β Wald SEs are overestimated β Make p-value in estimates is wrong β too high β (CIs still correct) 610 β’ Two estimates use slightly different information β P-value says βwhat if no effectβ β CI says βwhat if there is this effectβ β’ Variance depends on the hypothesised ratio of the number of people in the two groups β’ Can calculate likelihood ratio based pvalues β If you can be bothered β Some packages provide them automatically 611 Probit Regression β’ Very similar to logistic β much more complex initial transformation (to normal distribution) β Very similar results to logistic (multiplied by 1.7) β’ Swap logistic for probit in Stata command β Harder to interpret β’ Parameter doesnβt mean something β like log odds 612 Differentiating Between Probit and Logistic β’ Depends on shape of the error term β Normal or logistic β Graphs are very similar to each other β’ Could distinguish quality of fit β Given enormous sample size β’ Logistic = probit x 1.7 β Actually 1.6998 β’ Probit advantage β Understand the distribution β’ Logistic advantage β Much simpler to get back to the probability 613 3 2.8 2.6 2.4 2.2 2 1.8 1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0 -0.2 -0.4 -0.6 -0.8 1 -1 -1.2 -1.4 -1.6 -1.8 -2 -2.2 -2.4 -2.6 -2.8 -3 1.2 Normal (Probit) Logistic 0.8 0.6 0.4 0.2 0 614 Infinite Parameters β’ Non-convergence can happen because of infinite parameters β Insoluble model β’ Three kinds: β’ Complete separation β The groups are completely distinct β’ Pass group all score more than 10 β’ Fail group all score less than 10 615 β’ Quasi-complete separation β Separation with some overlap β’ Pass group all score 10 or more β’ Fail group all score 10 or less β’ Both cases: β No convergence β’ Close to this β Curious estimates β Curious standard errors 616 β’ Categorical Predictors β Can cause separation β Especially if correlated β’ Need people in every cell Male White Non-White Female White Non-White Below Poverty Line Above Poverty Line 617 Logistic Regression and Diagnosis β’ Logistic regression can be used for diagnostic tests β For every score β’ Calculate probability that result is positive β’ Calculate proportion of people with that score (or lower) who have a positive result β’ Calculate c statistic β Measure of discriminative power β % of all possible cases, where the model gives a higher probability to a correct case than to an incorrect case 618 β Perfect c-statistic = 1.0 β Random c-statistic = 0.5 619 Sensitivity and Specificity β’ Sensitivity: β Probability of saying someone has a positive result β β’ If they do: p(pos)|pos β’ Specificity β Probability of saying someone has a negative result β’ If they do: p(neg)|neg 620 C-Statistic, Sensitivity and Specificity β’ After logistic β lroc β’ Gives c-statistic β Better than R-squared 621 1.00 0.75 0.50 0.25 0.00 0.00 0.25 Area under ROC curve = 0.7469 0.50 1 - Specificity 0.75 1.00 More Advanced Techniques β’ Multinomial Logistic Regression more than two categories in outcome β same procedure β one category chosen as reference group β’ odds of being in category other than reference β’ Ordinal multinomial logistic regression β For ordinal outcome variables 623 More on Odds Ratios β’ Odds ratios are horrid β’ We use them because they have nice distributional properties β’ Example: β 40% in group 1 get PTSD β 60% in group 2 get PTSD β Whatβs the odds ratio? β How is this confusing? 624 Alternatives to Odds Ratios β’ Risk difference β 20 percentage points higher β’ Relative risk β Probability is 1.5 times higher β This is what you would think an odds ratio meant β’ Can we use these in regression? β RD β maybe. Sometimes. β RR β yes. But we need to do something else first 625 Final Thoughts β’ Logistic Regression can be extended β dummy variables β non-linear effects β interactions β’ Same issues as OLS β collinearity β outliers 626 β’ Same additional options as regress β xi: β cluster β robust 627 Poisson Regression 628 Counts and the Poisson Distribution β’ Von Bortkiewicz (1898) β Numbers of Prussian soldiers kicked to death by horses 120 100 80 60 0 1 2 3 4 5 109 65 22 3 1 0 40 20 0 0 1 2 3 4 5 629 β’ The data fitted a Poisson probability distribution β When counts of events occur, poisson distribution is common β E.g. papers published by researchers, police arrests, number of murders, ship accidents β’ Common approach β Log transform and treat as normal β’ Problems β Censored at 0 β Integers only allowed β Heteroscedasticity 630 The Poisson Distribution 0.7 0.6 Probability 0.5 0.5 1 4 8 0.4 0.3 0.2 0.1 0 0 1 2 3 4 5 6 7 8 9 Count 10 11 12 13 14 15 16 17 631 exp( ο ο ) ο p ( y | x) ο½ y! y 632 exp( οο ) ο p ( y | x) ο½ y! y Excel has a Poisson function you can use. β’ Where: β y is the count β ο is the mean of the Poisson distribution β’ In a Poisson distribution β The mean = the variance (hence heteroscedasticity issue)) β ο ο½ ο³2 633 Poisson Probabilities Mean Score 0 1 2 3 4 5 6 7 8 9 10 1 0.37 0.37 0.18 0.06 0.02 0.00 0.00 0.00 0.00 0.00 0.00 2 0.14 0.27 0.27 0.18 0.09 0.04 0.01 0.00 0.00 0.00 0.00 3 0.05 0.15 0.22 0.22 0.17 0.10 0.05 0.02 0.01 0.00 0.00 10 0.00 0.00 0.00 0.01 0.02 0.04 0.06 0.09 0.11 0.13 0.13 634 Issues with Estimation β’ Just as with logistic β We canβt predict a mean below zero β’ Donβt predict the mean β Predict the log of the mean 635 Poisson Regression in Stata β’ Adult literacy study β Number of sessions attended β Count variable β’ Poisson regression 636 poisson sessions tx ---------------------------------------------------------------------------sessions | Coef. Std. Err. z P>|z| [95% Conf. Interval] -----------+---------------------------------------------------------------tx | -.2359546 .06668 -3.54 0.000 -.366645 -.1052642 _cons | 1.899973 .046225 41.10 0.000 1.809374 1.990572 ---------------------------------------------------------------------------- poisson sessions tx, irr -----------------------------------------------------------------------------sessions | IRR Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------tx | .7898166 .052665 -3.54 0.000 .6930557 .9000867 ------------------------------------------------------------------------------ 637 But was it Poisson? β’ Look at predicted probabilities β Compare with actual probabilities β’ Predicted means β Control: exp(1.899) = 6.86 β Intervention: exp(1.899-0.236) = 5.28 β’ Get means and SDs 638 bysort tx: sum sessions -> tx = 0 VObs Mean Std. Dev. -------------+-------------------------------------sessions | 70 6.685714 3.495516 ----------------------------------------------------> tx = 1ariable | Variable | Obs Mean Std. Dev. -------------+-------------------------------------sessions | 82 5.280488 2.709263 639 β’ Do OK on the means β Donβt do OK on the variances β Variances are too high β’ Compare predicted probabilities with actual probabilities β’ tab session tx,col nofreq β’ Draw graphs β Not horrible β Except the zeroes 640 0.25 0.20 0.15 Predicted - control Predicted - intervention Actual - control Actual - intervention 0.10 0.05 0.00 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Test for Goodness of Fit to Poisson Distribution β’ After running Poisson β estat gof Goodness-of-fit chi2 Prob > chi2(150) = = 314.139 0.0000 β’ Highly significant β Poisson distribution doesnβt fit 642 Overdispersion β’ Problem in Poisson regression β Too many zeroes β’ Causes β c2 inflation β Standard error deflation β’ Hence p-values too low β Higher type I error rate β’ Two solutions β Negative binomial regression β Robust standard errors 643 Robust Standard Errors poisson sessions tx, robust --------------------------------------------------------| Robust sessions | Coef. Std. Err. z P>|z| -------------+------------------------------------------tx | -.2359546 .0840648 -2.81 0.005 _cons | 1.899973 .0622477 30.52 0.000 --------------------------------------------------------- β’ Robust SEs are larger 644 Negative Binomial Regression β’ Adds a βhurdleβ to account for the zeroes β Called alpha β’ nbreg sessions tx β’ OR β’ nbreg sessions tx, robust 645 Back to Categorical Outcomes β’ We said: β Odds ratios are not good β’ We like relative risk instead β What is the ratio of the risks? β’ What analysis technique do we know that gives ratios of means 646 β’ Poisson regression! β’ Wait. It wonβt work. The distribution is wrong. β’ Robust estimates! 647 Poisson Regression in SPSS β’ SPSS 15 (and above), has added it β’ Under generalized linear models 648 Lesson 11: Mediation and Path Analysis 649 Introduction β’ Moderator β Level of one variable influences effect of another variable β’ Mediator β One variable influences another via a third variable β’ All relationships are really mediated β are we interested in the mediators? β can we make the process more explicit 650 β’ In examples with bank education beginning salary β’ Why? β What is the process? β Are we making assumptions about the process? β Should we test those assumptions? 651 job skills expectations beginning salary education negotiating skills kudos for bank 652 Direct and Indirect Influences X may affect Y in two ways β’ Directly β X has a direct (causal) influence on Y β (or maybe mediated by other variables) β’ Indirectly β X affects Y via a mediating variable - M 653 β’ e.g. how does going to the pub effect comprehension on a Summer school course β on, say, regression not reading books on regression Having fun in pub in evening less knowledge Anything here? 654 not reading books on regression Having fun in pub in evening less knowledge fatigue Still needed? 655 β’ Mediators needed β to cope with more sophisticated theory in social sciences β make explicit assumptions made about processes β examine direct and indirect influences 656 Detecting Mediation 657 βClassic Approachβ 4 Steps From Baron and Kenny (1986) β’ To establish that the effect of X on Y is mediated by M 1. Show that X predicts Y 2. Show that X predicts M 3. Show that M predicts Y, controlling for X 4. If effect of X controlling for M is zero, M is complete mediator of the relationship β’ (3 and 4 in same analysis) 658 Example: Book habits Enjoy Books ο Buy books ο Read Books 659 Three Variables β’ Enjoy β How much an individual enjoys books β’ Buy β How many books an individual buys (in a year) β’ Read β How many books an individual reads (in a year) 660 ENJOY BUY READ ENJOY BUY READ 1.00 0.64 0.73 0.64 1.00 0.75 0.73 0.75 1.00 661 β’ The Theory enjoy buy read 662 β’ Step 1 1. Show that X (enjoy) predicts Y (read) β b1 = 0.487, p < 0.001 β standardised b1 = 0.732 β OK 663 2. Show that X (enjoy) predicts M (buy) β b1 = 0.974, p < 0.001 β standardised b1 = 0.643 β OK 664 3. Show that M (buy) predicts Y (read), controlling for X (enjoy) β b1 = 0.469, p < 0.001 β standardised b1 = 0.206 β OK 665 4. If effect of X controlling for M is zero, M is complete mediator of the relationship β (Same as analysis for step 3.) β b2 = 0.287, p = 0.001 β standardised b2 = 0.431 β Hmmmmβ¦ β’ Significant, therefore not a complete mediator 666 0.287 (step 4) enjoy read buy 0.974 (from step 2) 0.206 (from step 3) 667 The Mediation Coefficient β’ Amount of mediation = Step 1 β Step 4 =0.487 β 0.287 = 0.200 β’ OR Step 2 x Step 3 =0.974 x 0.206 = 0.200 668 SE of Mediator enjoy buy a (from step 2) read b (from step 2) β’ sa = se(a) β’ sb = se(b) 669 β’ Sobel test β Standard error of mediation coefficient can be calculated se ο½ b s + a s - s s 2 2 a a = 0.974 sa = 0.189 2 2 b 2 2 a b b = 0.206 sb = 0.054 670 β’ Indirect effect = 0.200 β se = 0.056 β t =3.52, p = 0.001 β’ Online Sobel test: http://quantpsy.org 671 Problems with the Sobel test β’ Recently β Move in methodological literature away from this conventional approach β’ Problems of power: β Several tests, all of which must be significant β’ Type I error rate = 0.05 * 0.05 = 0.0025 β’ Must affect power 672 β’ Distributional Assumption β We assume that the sampling distribution of the coefficient is normally distributed β’ Standard error is standard deviation β’ If: β a (x ο m) is normal and not zero β b (m ο y) is normal and not zero β’ Then: β a×b β Is not normally distributed β’ Assumption is violated β Test is incorrect 673 β’ Solution: β Bootstrap β’ Computer intensive semi-parametric procedure β’ Removes distributional assumption β Bootstrapping suggested as alternative β’ For Stata: β’ www.ats.ucla.edu/stat/stata/faq/mediat ion_cativ.htm β’ For SAS, SPSS: β www.quantpsy.org 674 Cross Sectional Bias β’ If everything is measured at one time β Likely to be bias β’ Ideally: β Three variables, measured on three occasions 675 x x x m m m y y y 676 β’ Kind of hard work β Collecting data on three occasions β’ BUT: Stationarity assumption can save us x x x m m m y y y 677 β’ We assume that the effect from M to Y is stable over time β Only need two time points β’ Cole and Maxwell (2003) 678 Power in Mediation β’ Really hard to work out β’ Need to run simulations β’ Power depends on β Size of a β Size of b β’ Fritz and Mackinnon (2007) β Table of power for different effects 679 More Information on Mediation β’ Mackinnon, Fritz and Fairchild β Annual Review of Psychology β’ Mackinnon β Introduction to statistical mediation β’ Iacobucci β Mediation analysis (little green book) β’ Mackinnonβs website (Google: mackinnon mediation) β’ Facebook group β (No, really) 680 Lesson 12: Moderators in Regression βdifferent slopes for different folksβ 681 Introduction β’ Moderator relationships have many different names β interactions (from ANOVA) β multiplicative β non-linear (just confusing) β non-additive β’ All talking about the same thing 682 A moderated relationship occurs β’ when the effect of one variable depends upon the level of another variable 683 β’ Hang on β¦ β That seems very like a nonlinear relationship β Moderator β’ Effect of one variable depends on level of another β Non-linear β’ Effect of one variable depends on level of itself β’ Where there is collinearity β Can be hard to distinguish between them β Paper B5 β Should (usually) compare effect sizes 684 β’ e.g. How much it hurts when I drop a computer on my foot depends on β x1: how much alcohol I have drunk β x2: how high the computer was dropped from β but if x1 is high enough β x2 will have no effect 685 β’ e.g. Likelihood of injury in a car accident β depends on β x1: speed of car β x2: if I was wearing a seatbelt β but if x1 is low enough β x2 will have no effect 686 30 25 Injury 20 15 10 5 0 5 15 25 35 45 Speed (mph) Seatbelt No Seatbelt 687 β’ e.g. number of words (from a list) I can remember β depends on β x1: type of words (abstract, e.g. βjusticeβ, or concrete, e.g. βcarrotβ) β x2: Method of testing (recognition β i.e. multiple choice, or free recall) β but if using recognition β x1: will not make a difference 688 β’ We looked at three kinds of moderator β’ alcohol x height = pain β continuous x continuous β’ speed x seatbelt = injury β continuous x categorical β’ word type x test type β categorical x categorical β’ We will look at them in reverse order 689 How do we know to look for moderators? Theoretical rationale β’ Often the most powerful β’ Many theories predict additive/linear effects β Fewer predict moderator effects Presence of heteroscedasticity β’ Clue there may be a moderated relationship missing 690 Two Categorical Predictors 691 β’ 2 IVs Data β word type (concrete [e.g. Carrot, table], abstract [e.g. Love, justice) β test method (multiple choice, recall ) β’ 20 Participants in one of four groups β β β β Concrete, MC Concrete, recall Abstract, MC Abstract, recall β’ 5 per group β’ lesson12.1-words.dta 692 MC Mean Concrete SD Mean Abstract SD Mean Total SD 15.4 2.5 15.6 1.5 15.5 2.0 Recall Total 15.2 15.3 3.2 2.7 7.0 11.3 1.6 4.8 11.1 13.3 4.9 4.3 693 β’ Graph of means 20 15 10 Concrete Abstract 5 0 MC Recall 694 Procedure for Testing 1: Convert to dummy coding β Already done 2: Calculate interaction term β Multiply dummy codes together β (Can also use xi: for this) β Call interaction mxc 695 β’ Interaction term (wxt) β multiply effect coded variables together Concrete 0 0 1 1 MC 0 1 0 1 mxc 0 0 0 1 696 3: Carry out regression β Hierarchical β linear effects first β interaction effect in next block 697 β’ b0(intercept)= 7.0 β Mean score when MC = 0 and concrete = 0 β’ b1 (mc) = 8.6 β When concrete is zero, effect of MC β’ b2 (concrete) = 8.2 β When mc is zero, effect of concrete MC Recall Total Concrete Abstract Total 15.4 15.2 15.30 15.6 7 11.30 15.50 11.10 13.30 698 β’ b3 (mc x con) = -8.4 β grand mean β’ Given other estimates, whatβs the predicted mean of concrete, MC? β 7.0 + 8.6 + 8.2 = 23.8 β’ What is it? β 15.4 Recog Recall Total Concrete Abstract Total 15.4 15.2 15.3 15.6 7.0 11.3 15.5 11.1 13.3 699 β’ Have: β’ Expect: β’ Difference: 15.4 23.8 -8.4 700 Back to the Graph Slope for concrete words 15.2-15.4=-0.2 20 15 10 Concrete Abstract 5 Difference in slopes 0 -8.6-(-0.2) = -8.4 MC Recall Slope for abstract words 7.0-15.6=-8.6 701 b associated with interaction β’ The difference in the slopes OR β’ The change in slope, away from the average, associated with a 1 unit change in the moderating variable 702 β’ Another way to look at it Y = 7 + 8.6 ο΄ m + 8.2 ο΄ c + -8.4 ο΄ m ο΄ c β’ Examine concrete words group (c = 1) β substitute values into the equation Y(conc) = Y = 7 + 8.6 ο΄ m + 8.2 ο΄ 1 + -8.4 ο΄ m ο΄ 1 Y(conc) = Y = 7 + 8.6 ο΄ m + 8.2 + -8.4 ο΄ m Y(conc) = Y = 7 + 8.2 + 8.6 ο΄ m -8.4 ο΄ m Y(conc) = Y = 15.2 + 0.2 ο΄ m 703 Categorical x Continuous 704 Note on Dichotomisation β’ Very common to see people dichotomise a variable β Makes the analysis easier β Very bad idea β’ Paper B6 705 Data A chain of 60 supermarkets β’ examining the relationship between profitability, shop size, and local competition β’ 2 IVs β shop size β comp (local competition, 0=no, 1=yes) β’ outcome β profit 706 β’ Data, βlesson 12.2.dtaβ Shopsize 4 10 7 10 10 29 12 6 14 62 Comp 1 1 0 0 1 1 0 1 0 0 Profit 23 25 19 9 18 33 17 20 21 8 707 1st Analysis Two IVs β’ R2=0.367, df=2, 57, p < 0.001 β’ Unstandardised estimates β b1 (shopsize) = 0.083 (p=0.001) β b2 (comp) =- 5.883 (p<0.001) β’ Standardised estimates β b1 (shopsize) = 0.356 β b2 (comp) = 0.448 708 β’ Suspicions β Presence of competition is likely to have an effect β Residual plot shows a little heteroscedasticity 709 10 5 -5 0 Residuals -10 15 20 25 Linear prediction 30 Procedure for Testing β’ Very similar to last time β convert βcompβ to dummy coding β’ (if itβs not already) β Compute interaction term β’ comp (effect coded) x size β Hierarchical regression 711 Result β’ Estimates β b1 (shopsize) = 0.12, SE = 0.03 β b2 (comp) = -1.67, SE 2.50 β b3 (sxc) = -0.10, SE 0.05 712 β’ comp now non-significant β shows importance of hierarchical β it obviously is important 713 Interpretation β’ Draw graph with lines of best fit β graph twoway (scatter profit shopsize if comp==1) (lfit profit shopsize if comp==1) (scatter profit shopsize if comp==0) (lfit profit shopsize if comp==0), legend(off) 714 40 30 20 10 0 0 20 40 60 80 100 shopsize 715 β’ Substitute into equation β’ Effects of size β (can ignore the constant) β’ Y=sizeο΄0.12 + compο΄(-1.67) + sizeο΄compο΄(-0.09) β Competition present (comp = 1) β’ Y=sizeο΄0.12 + 1ο΄(-1.67) + sizeο΄1ο΄(-0.09) β’ Y=sizeο΄0.12 + sizeο΄ (-0.09) β’ Y=sizeο΄0.03 716 β’ Y=sizeο΄0.12 + compο΄(-1.67) + sizeο΄compο΄(-0.09) β Competition present (x2 = 0) β’ Y=sizeο΄0.12 + 0 β’ Y=sizeο΄0.12 ο΄(-1.67) + sizeο΄ 0 ο΄(-0.09) 717 Two Continuous Variables 718 Data β’ Bank Employees β only using clerical staff β 363 cases β predicting starting salary β previous experience β age β age x experience β (exercise 6.3) 719 β’ Correlation matrix β only one significant LOGSB AGESTARTPREVEXP LOGSB 1.00 -0.09 0.08 AGESTART -0.09 1.00 0.77 PREVEXP 0.08 0.77 1.00 720 Initial Estimates (no moderator) β’ (standardised) β R2 = 0.063, p<0.001 β Age at start = -0.37, p<0.001 β Previous experience = 0.36, p<0.001 β’ Suppressing each other β Age and experience compensate for one another β Older, with no experience, bad β Younger, with experience, good 721 The Procedure β’ Very similar to previous β create multiplicative interaction term β BUT β’ Center variables (subtract mean) β Not always necessary β Can make life easier 722 β’ Hierarchical regression β two linear effects first β moderator effect in second 723 β’ Change in R2 β 0.085, p<0.001 β’ Estimates (standardised) β b1 (agestart) = -0.52 β b2 (prevexp) = 0.93 β b3 (age x exp) = -0.56 724 Interpretation 1: Pick-a-Point β’ Graph is tricky β canβt have two continuous variables β Choose specific points (pick-a-point) β’ Graph the line of best fit of one variable at others β Two ways to pick a point β’ 1: Choose high (z = +1), medium (z = 0) and low (z = -1) β’ Choose βsensibleβ values β age 20, 50, 80? 725 β’ We know: β Y = e ο΄ 0.94 + a ο΄ -0.53 + a ο΄ e ο΄ -0.58 β Where a = agestart, and e = experience β’ We can rewrite this as: β Y = (e ο΄ 0.94) + (a ο΄ -0.53) + (a ο΄ e ο΄ -0.58) β Take a out of the brackets β Y = (e ο΄ 0.94) + (-0.53 + e ο΄ -0.58)a β’ Bracketed terms are simple intercept and simple slope β intercept and slope for agestart β ο·0= (e ο΄ 0.10) β ο·1= (-0.53 + e ο΄ -0.58)a β Y = ο·0 + ο·1a 726 β’ Pick any value of e, and we know the slope for a β Standardised, so itβs easy β’ e = -1 β ο·0= (-1 ο΄ 0.94) = -0.94 β ο·1= (-0.53 + -1 ο΄ -0.58)a = -0.05a β’ e=0 β ο·0= (0 ο΄ 0.10) = 0 β ο·1= (-0.53+ 0 ο΄ -0.58)a = -0.53a β’ e=1 β ο·0= (1 ο΄ 0.10) = 0.10 β ο·1= (-0.53 + 1 ο΄ -0.58)a = -1.11a 727 Graph the Three Lines 1.5 1 e = -1 e=0 e=1 Log(salary) 0.5 0 -0.5 -1 -1.5 -1 -0.9 -0.8 -0.7 -0.6 -0.5 -0.4 -0.3 -0.2 -0.1 0 Age 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 728 Do This in Stata β’ The easy way β’ Create some pseudo cases β Some fake people β With sensible scores for the variables β Regression equation βstays behindβ β’ Calculate predicted scores with predict β (Can be in a new dataset) 729 β’ Then draw graph β’ drop if _n < 364 β’ graph twoway (lfit pred agestart if prevexp==0, lcolor(red)) (lfit pred agestart if prevexp==85, lcolor(black)) (lfit pred agestart if prevexp==170, lcolor(green)) , legend(off) 731 10 9.8 9.6 9.4 20 30 40 agestart 50 β’ (Also works in SPSS; in SAS, use Proc Score) 733 Interpretation 2: P-Values and CIs β’ Second way β Newer, rarely done β’ Calculate CIs of the slope β At any point β’ Calculate p-value β At any point β’ Give ranges of significance 734 What do you need? β’ The variance and covariance of the estimates β SPSS doesnβt provide estimates for intercept β Need to do it manually β’ In options, exclude intercept β Create intercept β c = 1 β Use it in the regression 735 β’ Enter information into web page: β’ www.people.ku.edu/~preacher/interact /mlr2.htm β’ Get results β’ Calculations in Bauer and Curran (in press: Multivariate Behavioral Research) β Paper B13 736 4.1 4.2 Y 4.3 4.4 4.5 MLR 2-Way Interaction Plot 4.0 CVz1(1) CVz1(2) CVz1(3) -1.0 -0.5 0.0 X 0.5 1.0 737 Areas of Significance 0.0 -0.2 -0.4 -0.6 Simple Slope 0.2 0.4 Confidence Bands -4 -2 0 2 4 Experience 738 β’ 2 complications β 1: Constant differed β 2: outcome was logged, hence non-linear β’ effect of 1 unit depends on where the unit is β See paper A2 739 Finally β¦ 740 Unlimited Moderators β’ Moderator effects are not limited to β 2 variables β linear effects 741 Three Interacting Variables β’ Age, Sex, Exp β’ Block 1 β Age, Sex, Exp β’ Block 2 β Age x Sex, Age x Exp, Sex x Exp β’ Block 3 β Age x Sex x Exp 742 β’ Results β All two way interactions significant β Three way not significant β Effect of Age depends on sex β Effect of experience depends on sex β Size of the age x experience interaction does not depend on sex (phew!) 743 Moderated Non-Linear Relationships β’ Enter non-linear effect β’ Enter non-linear effect x moderator β if significant indicates degree of nonlinearity differs by moderator 744 745 Lesson 13: Longitudinal Models 746 Advantages of Longitudinal Data β’ You get more data from the same number of people β’ You can test causal relationships β Although you canβt rule them out β’ You can examine change β’ You can control for individual differences 747 Disadvantage of Longitudinal Data β’ Itβs much harder to analyze 748 Longitudinal Research β’ For comparing repeated measures β Clusters are people β’ Data are usually short and fat ID V1 V2 V3 V4 1 2 3 4 7 2 3 6 8 4 3 2 5 7 5 749 Converting Data β’ Change data to tall and thin β’ Use reshape in stata β’ Use Data, Restructure in SPSS β’ Clusters are ID ID V X 1 1 2 1 2 3 1 3 4 1 4 7 2 1 3 2 2 6 2 3 8 2 4 4 3 1 2 3 2 5 3 3 7 3 4 5 750 Predict Salary Change β’ Use exercise5.3-bank salary.dta β Compare beginning salary and salary β Would normally use paired samples t-test β’ Difference = $17,403, 95% CIs $16,427.407, $18,379.555 751 Predict Salary Change β’ Donβt take the difference in salary and salbegin β Why not? β’ reg salary agestart salbegin β’ Est: -207.8, 95% CIs -267.4, -148.2 752 Restructure the Data β’ gen id = _n β’ rename salbegin sal1 β’ rename salary sal2 β’ reshape long sal, i(id) j(t) β’ replace t = t-1 753 Restructure the Data β’ Do it again β With data tall and thin β’ Do a regression β What do we find? ID Time Cash 1 0 $18,750 1 1 $21,450 2 0 $12,000 2 1 $21,900 3 0 $13,200 3 1 $45,000 754 Results β’ We have violated the independence assumption β’ We have the wrong answer β’ Simplest way to solve it: β’ regress sal t, cluster(id) β’ Assumes that ID is just an irritant β Rather inflexible 755 However β¦ β’ That has one advantage β Missing data doesnβt mean that we exclude the case β If data are missing at random (or missing completely at random) estimates will be unbiased 756 β’ If everyone has β Score at time 1 β Score at time 2 β’ Analysis is easy β’ If half the people have β Score at time 1 β Score at time 2 β’ Analysis is easy β’ But what if some have β Time 1 β Time 2 β Time 1 and 2 757 β’ If β Missing at random T1 β’ MAR T2 β Missing completely at random 10 15 7 6 8 2 5 8 β’ MCAR β (Crappy names) β’ No problem Interesting β¦ β’ That wasnβt very interesting β What is more interesting is when: β’ We have missing data β Which we wonβt talk about more (much) β’ We have multiple measurements of the same people β Which we will talk about 759 Modelling Change β’ Can plot and assess trajectories over time β’ How do people change? β’ What predicts the rate of change? 760 Plotting Individuals Salary Person 1 T1 T2 761 Plotting Individuals Person 3 Salary Person 1 Person 2 T1 T2 762 0 0 100000 150000 50000 0 100000 150000 50000 0 100000 150000 50000 0 100000 150000 50000 0 100000 150000 50000 0 100000 150000 50000 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 .5 Graphs by id 1 0 .5 1 0 .5 1 t 0 .5 1 0 .5 1 0 .5 1 0 50000 0 .2 .4 .6 t .8 1 Estimation β’ Each individual has an intercept β Sampled from the population of intercepts β’ Each individual has a slope β Sampled from the population of slopes β’ Can we estimate the average of each β And a measure of their variance? β’ Yes! With multilevel models 765 Multilevel Models β’ Can do all kinds of clever things β We wonβt worry about most of them β’ Used when β Level 1 units (measures) β Are nested within β Level 2 units (people) β’ Same person measured twice β Violated indepence 766 Levels β’ In regression β Everything is at one level β’ In multilevel models β We have multiple levels β’ Hierarchical levels (hence hierarchical linear models) β’ Random effects (random effects models) β’ Mixed effects (mixed models) 767 Levels β’ Level 1 units β First level of measurement β Are clustered within β’ Level 2 units β Second level of measurement 768 Some Equations β’ (This is very hard. Itβs not important). β’ In regression yi ο½ b0 ο« b1 xi1 ο« ei β’ If x is time β And we have one person β Reference time with I β And call it T 769 β’ Single person equation yi ο½ b0 ο« b1Ti ο« ei β’ But what if we have lots of people? β Weβve used i for time β Weβll use j for people yij ο½ b0 ο« b1Tij ο« eij β’ But everything is fixed β We want to have some random effects 770 β’ Letβs make intercepts random β Everyone has their own intercept yij ο½ b0 j ο« b1Tij ο« eij β’ Look! We added a little j β Now itβs a multilevel model β’ And we need an equation for the intercept 771 β’ Equation for each personβs intercept b0 j ο½ g 00 ο« ο0 j β’ Your intercept (b0j) is equal to: β Mean intercept β’ g00 (Gamma) β Plus residual (for that individual) β’ ο0j (mu) β This is level 2 model β’ Level 2 residuals β’ i.i.d, etc 772 β’ So now we have yij ο½ b0 j ο«b1 jTij ο« eij b0 j ο½ g 00 ο« ο0 j β’ Or yij ο½ (g 00 ο« ο0 j ) ο« b1Tij ο« eij 773 Make Time Random β’ Value of the time parameter can vary β Amongst people β Everyone can have a different effect of time yij ο½ b0 j ο« b1 jTij ο« eij 774 β’ Time is random β Everyone has a slope parameter btj ο½ g t 0 ο« ο tj β’ So: yij ο½ b0 j ο« b1Tij ο« eij b0 j ο½ g 00 ο« ο 0 j b1 j ο½ g 10 ο« ο1 j 775 β’ Or yij ο½ b0 j ο« b1 jTij ο« eij b0 j ο½ g 00 ο« ο 0 j btj ο½ g t 0 ο« οtj yij ο½ ο¨g t 0 ο« ο1 j ο© ο« ο¨g 10 ο« ο1 j ο© ο« eij 776 Time Invariant Covariates β’ Can be added at level 2 β We can predict personβs intercept β’ Starting point β And rate of change 777 Employee Data β’ Level 1: β Pay measures β’ Two of them β Clustered within β’ Level 2: β People β Level 2 measures: age, sex, job, etc 778 Regression with Time β’ Do a regression analysis β On one person β’ Time is the predictor β’ Get a regression line for that person 779 Fixed vs Random Effects β’ Fixed effects β Effect is the same across all clusters (people) β Variation is only measurement error β’ Random effects β Effect varies across people β Additional parameter in the model β’ Less parsimonious 780 Fixed vs Random Effects β’ If an effect has variance β It might have covariance β With any other effects which also have variance β’ More parameters β Less parsimony 781 Covariates β’ Two kinds of covariates β Time invariant β’ Fixed for a person β Age when study started β Sex β Time variant β’ Can change over time β Time β Marital Status 782 Time Invariant β’ Look at effect of age β Add age to the fixed effects β Is that significant? β Are random effects (still?) significant 783 Multilevel Models in Stata β’ Use xtmixed β Stata 10 added xtmelogit, xtmepoisson β’ Continuous variables are hard enough β’ In SPSS, continuous variables only 784 Multilevel Models in Stata β’ xtmixed sal t β’ Does regression β’ Need to tell it about the people: β’ xtmixed sal t ||id: 785 -----------------------------------------------------------------------------sal | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------t | 17403.48 496.7321 35.04 0.000 16429.9 18377.06 _cons | 17016.09 610.6691 27.86 0.000 15819.2 18212.98 ----------------------------------------------------------------------------------------------------------------------------------------------------------Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval] -----------------------------+-----------------------------------------------id: Identity | sd(_cons) | 10875.87 449.592 10029.44 11793.73 -----------------------------+-----------------------------------------------sd(Residual) | 7647.093 248.6285 7174.992 8150.258 -----------------------------------------------------------------------------LR test vs. linear regression: chibar2(01) = 280.88 Prob >= chibar2 = 0.0000 786 Average -----------------------------------------------------------------------------slope sal | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------t | 17403.48 496.7321 35.04 0.000 16429.9 18377.06 _cons | 17016.09 610.6691 27.86 0.000 15819.2 18212.98 ------------------------------------------------------------------------------ Average -----------------------------------------------------------------------------intercept Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval] SD of the -----------------------------+-----------------------------------------------id: Identity | slopes sd(_cons) | 10875.87 449.592 10029.44 11793.73 -----------------------------+-----------------------------------------------sd(Residual) | 7647.093 248.6285 7174.992 8150.258 -----------------------------------------------------------------------------LR test vs. linear regression: chibar2(01) = 280.88 Prob >= chibar2 = 0.0000 SD of the slopes 787 β’ Gives random intercepts only β Letβs look at them β’ predict rand_int β’ xtline rand_int , overlay t(t) i(id) legend(off) 788 0 .2 .4 .6 t .8 1 0 20000 40000 60000 80000 Random Slopes β’ Everyone has the same slope β Maybe thatβs not true β Make slopes random β’ xtmixed sal t ||id: t 790 -----------------------------------------------------------------------------sal | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------t | 17403.48 496.7273 35.04 0.000 16429.91 18377.05 _cons | 17016.09 361.5138 47.07 0.000 16307.53 17724.64 ----------------------------------------------------------------------------------------------------------------------------------------------------------Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval] -----------------------------+-----------------------------------------------id: Independent | sd(t) | 10814.52 351.6071 10146.88 11526.09 sd(_cons) | 7870.712 255.9014 7384.801 8388.595 -----------------------------+-----------------------------------------------sd(Residual) | .709687 .3436148 .2747476 1.833158 ------------------------------------------------------------------------------ β’ predict rand_int_slope, fitte β’ xtline rand_int_slope , overlay t(t) i(id) legend(off) 792 0 50000 0 .2 .4 .6 t .8 1 Structure of the Covariances β’ We have been forcing slope and intercepts to be uncorrelated β’ Letβs correlate them β’ xtmixed sal t ||id: t , cov(un) 794 -----------------------------------------------------------------------------sal | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------t | 17403.48 496.7316 35.04 0.000 16429.91 18377.06 _cons | 17016.09 361.5085 47.07 0.000 16307.54 17724.63 ----------------------------------------------------------------------------------------------------------------------------------------------------------Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval] -----------------------------+-----------------------------------------------id: Unstructured | sd(t) | 10103.18 11670.32 1050.081 97205.97 sd(_cons) | 7382.783 7985.867 886.1058 61511.25 corr(t,_cons) | .8550486 3.491476 -1 1 -----------------------------+-----------------------------------------------sd(Residual) | 2727.788 21601.34 .0004956 1.50e+10 ------------------------------------------------------------------------------ Predicting Change β’ Does another variable moderate the effect of time β This means that the effect of time varies β As a function of the slope β’ xi: xtmixed sal i.t*agestart ||id: t 796 -----------------------------------------------------------------------------sal | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------_It_1 | 25483.16 1640.544 15.53 0.000 22267.75 28698.57 agestart | -5.157022 30.85168 -0.17 0.867 -65.6252 55.31115 _ItXagest_1 | -212.5138 41.25203 -5.15 0.000 -293.3663 -131.6613 _cons | 17205.18 1226.935 14.02 0.000 14800.43 19609.93 ----------------------------------------------------------------------------------------------------------------------------------------------------------Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval] -----------------------------+-----------------------------------------------id: Unstructured | sd(t) | 9871.434 9785.792 1414.372 68896.47 sd(_cons) | 7437.632 6495.359 1342.987 41190.55 corr(t,_cons) | .8622535 2.921366 -1 1 -----------------------------+-----------------------------------------------sd(Residual) | 2619.971 18422.9 .0027095 2.53e+09 ------------------------------------------------------------------------------ Exercises β’ 13.1, 13.2 798 Fixed Effects Models β’ A second way of looking at longitudinal data β’ Multilevel (mixed) models β Assume that intercepts are random β’ Fixed effects models β Assume they are fixed β If they are fixed they can correlate β’ With all other predictors 799 Fixed Effects Models β’ Allowing intercepts to correlate β Has the effect of controlling for ALL time invariant predictors β Even those you didnβt measure β Each person is their own control 800 Fixed Effects Models β’ Regression asks: β Are people who are higher on x also higher on y? β’ Fixed effects asks: β When a person is higher on x are they also higher on y β Effects are within people, not between people 801 Fixed Effects in Stata β’ Make data long, then β xtreg sal t agestart, i(id) 802 -----------------------------------------------------------------------------sal | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------t | 17403.48 496.7319 35.04 0.000 16427.41 18379.56 _cons | 17016.09 351.2425 48.45 0.000 16325.9 17706.28 -------------+---------------------------------------------------------------sigma_u | 12145.928 sigma_e | 7647.0911 rho | .71612838 (fraction of variance due to u_i) -----------------------------------------------------------------------------F test that all u_i=0: F(473, 473) = 5.05 Prob > F = 0.0000 Fixed Effects Regression β’ Can we look at the effect of time invariant predictors? β’ xtreg sal t agestart, i(id) fe β’ Why not? 804 Interactions β’ But we can look at interactions of time invariant predictors β’ xi: xtreg sal i.t*agestart, i(id) fe 805 -----------------------------------------------------------------------------sal | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------_It_1 | 25483.16 1640.538 15.53 0.000 22259.48 28706.84 agestart | (dropped) _ItXagest_1 | -212.5138 41.25188 -5.15 0.000 -293.5743 -131.4533 _cons | 17009.25 342.8103 49.62 0.000 16335.62 17682.88 -------------+---------------------------------------------------------------sigma_u | 12087.77 sigma_e | 7455.6318 rho | .72441115 (fraction of variance due to u_i) ------------------------------------------------------------------------------ Exercises β’ 13.3, 13.4 807 Bonus Lesson 1: Why Regression? A little aside, where we look at why regression has such a curious name. 808 Regression The or an act of regression; reversion; return towards the mean; return to an earlier stage of development, as in an adultβs or an adolescentβs behaving like a child (From Latin gradi, to go) β’ So why name a statistical technique which is about prediction and explanation? 809 β’ Francis Galton β Charles Darwinβs cousin β Studying heritability β’ Tall fathers have shorter sons β’ Short fathers have taller sons β βFilial regression toward mediocrityβ β Regression to the mean 810 β’ Galton thought this was biological fact β Evolutionary basis? β’ Then did the analysis backward β Tall sons have shorter fathers β Short sons have taller fathers β’ Regression to the mean β Not biological fact, statistical artefact 811 Other Examples β’ Secrist (1933): The Triumph of Mediocrity in Business β’ Second albums often tend to not be as good as first β’ Sequel to a film is not as good as the first one β’ Sports Illustrated Cover Jinx β’ Parents think that punishing bad behaviour works, but rewarding good behaviour doesnβt 812 β’ Accident reduction schemes β Always reduce accidents β’ Poor radiologists improve after training β’ Any treatment for a cold will work β Or for most illnesses β’ Deaths due to methadone in Utah β High last year β Must take action! 813 Pair Link Diagram β’ An alternative to a scatterplot x y 814 r=1.00 x x x x x x x 815 r=0.00 x x x x x 816 From Regression to Correlation β’ Where do we predict an individualβs score on y will be, based on their score on x? β Depends on the correlation β’ r = 1.00 β we know exactly where they will be β’ r = 0.00 β we have no idea β’ r = 0.50 β we have some idea 817 r=1.00 Starts here Will end up here x y 818 r=0.00 Starts here Could end anywhere here x y 819 r=0.50 Probably end somewhere here Starts here x y 820 Galton Squeeze Diagram β’ Donβt show individuals β Show groups of individuals, from the same (or similar) starting point β Shows regression to the mean 821 r=0.00 Ends here Group starts here x Group starts here y 822 r=0.50 x y 823 r=1.00 x y 824 1 unit r units x y β’ Correlation is amount of regression that doesnβt occur 825 β’ No regression β’ r=1.00 x y 826 β’ Some regression β’ r=0.50 x y 827 r=0.00 β’ Lots (maximum) regression β’ r=0.00 x y 828 Formula zΛ y ο½ rxy z x 829 Conclusion β’ Regression towards mean is statistical necessity regression = perfection β correlation β’ Very non-intuitive β’ Interest in regression and correlation β From examining the extent of regression towards mean β By Pearson β worked with Galton β Stuck with curious name β’ See also Paper B3 830 β’ Correcting for regression to the mean β Possible β Makes lots of tricky assumptions β’ To appear to do well in your job / life β Do something after someone has failed β You probably canβt do worse β Hospital / school / department / class / study / experiment β’ If it fails, volunteer to do it 831 Bonus Lesson 2: Other Kinds of Regression 832 Introduction β’ Weβve covered a few kinds of regression β There are many more, for specific types of outcomes 833 Beta Regression β’ Used when the outcome variable is beta distributed β’ Rates and proportions β Bounded by zero and 1 β Uniform, or strange shaped distributions 834 Cox Proportional Hazards Regression β’ Type of survival model β’ Used for time to an event β When the event might not occur β’ Developed for medical research 835 Cox Proportional Hazards Regression β’ E.g. How long does it take for a car to break down β The car crashes, and is scrapped β Weβll never know β but we want to know the information β Discarding the data point would lead to bias 836 Competing Risks Survival β’ Time to multiple events β Several things are trying to kill you β Which one succeeds 837 Data Mining Techniques β’ Avoid problems with stepwise regression β Used as alternatives to logistic β’ Boosted regression (Stata command: boost) β Semi-parametric alternative to logistic regression β’ Least Angle Regression (LARS) 838 β’ Classification trees 839 Seemingly Unrelated Regression β’ Used for multiple outcomes β With correlated error terms β’ Some say it should be seemingly related β’ Stata command: sureg 840 Instrumental Variables Regression β’ Used for mediator models β Although economists donβt call them that. β’ Use ivregress in Stata 841 Quantile Regression β’ Why do we always try to predict the mean? β’ What about predicting the median? The 25th percentile? β’ Thatβs quantile regression 842 Non-Parametric Regression β’ (Might also include LARS/Lasso/Boost) β’ Donβt force any functional form on the relationship β’ LO(W)ESS β locally weighted scatterplot smoothing β Will find any relationship 843 Robust Regression β’ (Careful, not sandwich estimators) β’ Trimmed of outliers, estimated with bootstrap β’ Lots of publications by Wilcox 844 Censored Regression β’ Tobit regression β When a measure is censored β E.g. unemployed people work 0 hours. 845
