Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Psy B07 CATEGORICAL DATA & χ2 Chapter 6 Slide 1 Psy B07 A Quick Look Back Reminder about hypothesis testing: 1) Assume what you believe (H1) is wrong. Construct H0 and accept it as a default. 2) Show that some event is of sufficiently low probability given H0***. 3) Reject H0. *** In order to do this, we need to know the distribution associated with H0, because we use that distribution as the basis for our probability calculation. Chapter 6 Slide 2 Psy B07 z-score Use when we have acquired some data set, then want to ask questions concerning the probability of certain specific data values (e.g., do certain values seem extreme?). In this case, the distribution associated with H0 is described by X and S2 because the data points reflect a continuous variable that is normally distributed. Chapter 6 Slide 3 Psy B07 Chi-square (χ2) test The Chi-square test is a general purpose test for use with discrete variables. It has a number of uses, including the detection of bizarre outcomes given some a priori probability for binomial situation, and for multinomial situations. Chapter 6 Slide 4 Psy B07 Chi-square (χ2) test In addition, it allows us to go beyond questions of bizarreness, and move into the question of whether pairs of variables are related. For example: Female Male Legalize Do not Legalize 9 9 23 7 It does so by mapping the discreet variables unto a continuous distribution assuming H0, the chi-square distribution. Chapter 6 Slide 5 Psy B07 The chi-square distribution Let’s reconsider a simple binomial problem. Say, we have a batter who hits .300 [i.e., P(Hit)=0.30], and we want to know whether it is abnormal for him to go 6 for 10 (i.e., 6 hits in 10 at bats). We could do this using the binomial stuff that I did not cover in Chapter 5 (and for which you are not responsible) But we can also do it with a chi-square test Chapter 6 Slide 6 Psy B07 The way of the chi2 We can put our values into a contingency table as follows: Observed Expected? Hits Outs 6 4 Then consider the distribution of the following formula given H0: (Observed Expected) 2 Expected Chapter 6 Slide 7 Psy B07 The way of the chi2 Attempt Expected (E) 1 2 3 4 5 6 7 8 9 10 Chapter 6 Observed (O) 3/7 3/7 3/7 3/7 3/7 3/7 3/7 3/7 3/7 3/7 Slide 8 Psy B07 The way of the chi2 Attempt Expected (E) 1 2 3 4 5 6 7 8 9 10 Chapter 6 3/7 3/7 3/7 3/7 3/7 3/7 3/7 3/7 3/7 3/7 Observed (O) hit out out hit hit hit out hit out hit Slide 9 Psy B07 The way of the chi2 In-Class Example: Note that while the observed values are discreet, the derived score is continuous. If we calculated enough of these derived scores, we could plot a frequency distribution which would be a chi-square distribution with 1 degree of freedom or 2(1). Given this distribution and appropriate tables, we can then find the probability associated with any particular 2 value. Chapter 6 Slide 10 Psy B07 The way of the chi2 Continuing the Baseball Example: Observed Expected 2 ( O E ) 2 E (6 3) 2 (4 7) 2 3 7 9 9 4.29 3 7 Chapter 6 Hits Outs 6 3 4 7 So if the probability of obtaining a 2 of 4.29 or greater is less than , then the observed outcome can be considered bizarre (i.e., the result of something other than a .300 hitter getting lucky). Slide 11 Psy B07 The way of the chi2 Just like the t-test, chi2 distribution is based on degrees of freedom Thus, since our obtained 2 value of 4.29 is greater than 3.84, we can reject H0 and assume that hitting 6 of 10 reflects more than just chance performance. Chapter 6 Slide 12 Psy B07 The way of the chi2 Going a Step Further: Suppose we complicate the previous example by taking walks and hit by pitches into account. That is, suppose the average batter gets a hit with a probability of 0.28, gets walked with a probability of .08, gets hit by a pitch (HBP) with a probability of .02, and gets out the rest of the time. Chapter 6 Slide 13 Psy B07 The way of the chi2 Now we ask, can you reject H0 (that this batter is typical of the average batter) given the following outcomes from 50 at bats? Observed Expected Hit 12 Walk HBP Out 3 8 27 1) Calculate expected values (Np). 2) Calculate 2 obtained. 3) Figure out the appropriate df (C-1). 4) Find 2critical and compare 2 obtained to it. Chapter 6 Slide 14 Psy B07 The way of the chi2 Observed Expected Hit 12 14 Walk HBP Out 3 8 27 4 1 31 2 ( O E ) 2 E (12 14) 2 (3 4) 2 (8 1) 2 (27 31) 2 14 4 1 31 50.51 Chapter 6 Slide 15 Psy B07 Two types of chi2 tests So far, all the tests have been to assess whether some observation or set of observations seems out-of-line with some expected distribution. This is also known as the goodness-of-fit chi-square test However, the logic of the chi-square test can be extended to examine the issue of whether two variables are independent (i.e., not systematically related) or dependent (i.e., systematically related). Chapter 6 Slide 16 Psy B07 χ2 test for independence Consider the following data set again: Female Male Legalize Do not Legalize 9 9 23 7 Are the variables of gender and opinion concerning the legalization of marijuana independent? Chapter 6 Slide 17 Psy B07 χ2 test for independence Legalize Do Not Legalize Total Female 9 23 32 Male 9 7 16 Total 18 30 48 From the marginal totals we can calculate: P(Female) = 32/48 = 0.667 P(Male) = 16/48 = 0.333 P(Legalize) = 18/48 = 0.375 P(Do Not Legalize) = 30/48 = 0.625 Chapter 6 Slide 18 Psy B07 χ2 test for independence If these two variables are independent, then by the multiplicative law, we expect that: P(Female,Legalize) = P(Female) x P(Legalize) = .667 x .375 = .250125 EV(Female, Legalize) = Np = 48 x .250125 = 12 Chapter 6 Slide 19 Psy B07 χ2 test for independence If we do this for all four cells, we get: Legalize Female Male Total Chapter 6 Do Not Legalize Total 9 23 Expect: 12 Expect: 20 9 7 Expect: 6 Expect: 10 18 30 32 16 48 Slide 20 Psy B07 χ2 test for independence Are the observed values different enough from the expected values to reject the notion that the differences are due to chance variation? (O E ) E 2 2 2 2 (9 12) (23 20) (9 6) (7 10) 12 20 6 10 3.6 2 2 Chapter 6 Slide 21 Psy B07 χ2 test for independence The df associated with 2 variable contingency tables can be calculated using the formula: df = (C-1)(R-1) where C is the number of columns and R is the number of rows. Chapter 6 Slide 22 Psy B07 χ2 test for independence Thus, to finish our previous example, the 2 critical with alpha equal .05 and 1 df equals 3.84. Since our 2 is not bigger than that (i.e., 3.6) we cannot reject H0. Chapter 6 Slide 23 Psy B07 Assumptions of χ2 Independence of observations: Chi-square analyses are only valid when the actual observations within the cells are independent. This independence of observations is different from the issue of whether the variables are independent, that is what the chi-square is testing. Chapter 6 Slide 24 Psy B07 Assumptions of χ2 Independence of observations: You know your observations are not independent when the grand total is larger than the number of subjects. Example: The activity level of 5 rats was tested over 4 days, producing these values: Chapter 6 Low Activity Medium High 3 7 10 Slide 25 Psy B07 Assumptions of χ2 Normality: Use of the chi-square distribution for finding critical values assumes that the expected values (i.e., Np) are normally distributed. This assumption breaks down when the expected values are small (specifically, the distribution of Np becomes more and more positively skewed as Np gets small). Chapter 6 Slide 26 Psy B07 Assumptions of χ2 Normality: Thus, one should be cautious using the chi-square test when the expected values are small. How small? This is debatable but if expected values are as low as 5, you should be worried. Chapter 6 Slide 27 Psy B07 Assumptions of χ2 Inclusion of Non-Occurrences: The chi-square test assumes that all outcomes (occurrences and nonoccurrences) are considered in the contingency table. As an example of a failure to include a non-occurrence, see page 160 of the text. Chapter 6 Slide 28 Psy B07 A tale of tails We only reject H0 when values of 2 are larger than 2 obtained. This suggests that the 2 test is always one-tailed and, in terms of the rejection region, it is. In a different sense, however, the test is actually multiple tailed. Chapter 6 Slide 29 Psy B07 A tale of tails Reconsider the following “marking scheme” example: Option 1 Option 2 Option 3 38 57 5 If we do not specify how we expect the results to fall out then any outcome with a high enough 2 obtained can be used to reject H0. However, if we specify our outcome, we are allowed to increase our alpha - in the example we can increase alpha to 0.30 if we specified the exact ordering (in advance) that was observed. Chapter 6 Slide 30 Psy B07 Measures of Association The chi-square test only tells us whether two variables are independent, it does not say anything about the magnitude of the dependency if one is found to exist. Stealing from the book, consider the following two cases, both of which produce a significant 2 obtained, but which imply different strengths of relation: Chapter 6 Slide 31 Psy B07 Measures of Association Smoking Behaviour Male Female Nonsmoker 400 350 Smoker 100 150 13.333 2 Primary Food Shopper Male Female Chapter 6 Yes 400 100 No 100 400 2 12.737 Slide 32 Psy B07 Measures of Association There are a number of ways to quantify the strength of a relation (see sections in the text on the contingency coefficient, Phi, & Odds Ratios), but the two most relevant to psychologists are Cramer’s Phi and Cohen’s Kappa. Chapter 6 Slide 33 Psy B07 Measures of Association Cramer’s Phi (φc) can be used with any contingency table and is calculated as: 2 c N(k 1) Values of range from 0 to 1. The values the tables on the previous page are 0.12 and 0.60 respectively, indicating a much stronger relation in the second example. Chapter 6 Slide 34 Psy B07 Measures of Association Often, in psychology, we will ask some “judge” to categorize things into specific categories. For example, imagine a beer brewing competition where we asked a judge to categorize beers as Yucky, OK, or Yummy. Obviously, we are eventually interested in knowing something about the beers after they are categorized. Chapter 6 Slide 35 Psy B07 Measures of Association However, one issue that arises is the judges abilities to tell the difference between the beers. One way around this is to get two judges and show that a given beer is reliably rated across the judges (i.e., that both judges tend to categorize things in a similar way). Chapter 6 Slide 36 Psy B07 Measures of Association Such a finding would suggest that the judges are sensitive to some underlying quality of the beers as opposed to just guessing. Judge 2 Yuck! OK Yummy Chapter 6 Yuck! 3 1 0 Judge 1 OK 2 15 1 Yummy 3 2 3 Slide 37 Psy B07 Measures of Association Note that if you just looked at the proportion of decisions that me and Judge 2 agreed on, it looks like we are doing OK: P(Agree)=21/30 = 0.70 or 70% Chapter 6 Slide 38 Psy B07 Measures of Association There is a problem here, however, because both judges are biased to judge a beer as OK such that even if they were guessing, the agreement would seem high because both would guess OK on a lot of trials and would therefore agree a lot. Solution: fO – fE k= =? N – fE Chapter 6 Slide 39 Psy B07 Measures of Association Such a finding would suggest that the judges are sensitive to some underlying quality of the beers as opposed to just guessing. Judge 2 Yuck! OK Yummy Chapter 6 Yuck! 3 (1.07) 1 0 Judge 1 OK 2 15(10.8) 1 Yummy 3 2 3(1.07) Slide 40 Psy B07 O E N E 21 12.94 30 12.94 .47 Chapter 6 Slide 41