Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Statistical Power - Illinois State University Department of Psychology
Document related concepts
Transcript
Statistics for the Social Sciences Psychology 340 Fall 2006 Effect sizes & Statistical Power Outline Statistics for the Social Sciences • Error types revisited • Effect size: Cohen’s d • Statistical Power Analysis Performing your statistical test Statistics for the Social Sciences Real world (‘truth’) There really isn’t an effect Experimenter’s conclusions H0 is correct Reject H0 Fail to Reject H0 H0 is wrong There really is an effect Performing your statistical test Statistics for the Social Sciences Real world (‘truth’) H0 is correct H0 is wrong Performing your statistical test Statistics for the Social Sciences Real world (‘truth’) Real world (‘truth’) H0 is correct H0 is correct H0 is wrong Type I error Type II error Real world (‘truth’) H0 is wrong Type I error Type II error H0 is correct So there is only one distribution The original (null) distribution H0 is wrong So there are two distributions The new (treatment) The original (null) distribution distribution Performing your statistical test Real world (‘truth’) Statistics for the Social Sciences H0 is correct Real world (‘truth’) H0 is wrong Type I error Type II error H0 is correct So there is only one distribution The original (null) distribution H0 is wrong So there are two distributions The new (treatment) The original (null) distribution distribution Effect Size Real world (‘truth’) Statistics for the Social Sciences • Hypothesis test tells us whether the observed difference is probably due to chance or not • It does not tell us how big the difference is – Effect size tells us how much the two populations don’t overlap H0 is correct H0 is wrong Type I error Type II error H0 is wrong So there are two distributions The new (treatment) The original (null) distribution distribution Effect Size Statistics for the Social Sciences • Figuring effect size 1 2 But this is tied to the particular units of measurement The new (treatment) The original (null) distribution distribution – Effect size tells us how much the two populations don’t overlap 2 1 Effect Size Statistics for the Social Sciences • Standardized effect size Cohen’s d 1 2 d – Puts into neutral units for comparison (same logic as zscores) The new (treatment) The original (null) distribution distribution – Effect size tells us how much the two populations don’t overlap 2 1 Effect Size Statistics for the Social Sciences • Effect size conventions – small – medium – large d = .2 d = .5 d = .8 1 2 d The new (treatment) The original (null) distribution distribution – Effect size tells us how much the two populations don’t overlap 2 1 Error types Statistics for the Social Sciences There really isn’t an effect I conclude that there is an effect Real world (‘truth’) H0 is correct Reject H0 Experimenter’s conclusions Fail to Reject H0 I can’t detect an effect H0 is wrong There really is an effect Error types Statistics for the Social Sciences Type I error (): concluding that there is a difference between groups (“an effect”) when there really isn’t. Reject H0 Experimenter’s conclusions Fail to Reject H0 Real world (‘truth’) H0 is correct H0 is wrong Type I error Type II error (): concluding that there isn’t an effect, when there really is. Type II error Statistical Power Statistics for the Social Sciences • The probability of making a Type II error is related to Statistical Power – Statistical Power: The probability that the study will produce a statistically significant results if the research hypothesis is true (there is an effect) Power 1 • So how do we compute this? Statistical Power Statistics for the Social Sciences Real world (‘truth’) H0 is correct H0 is wrong Real world (‘truth’) H0: is true (is no treatment effect) Type I error The original (null) distribution Type II error = 0.05 Reject H0 Fail to reject H0 Statistical Power Statistics for the Social Sciences Real world (‘truth’) H0 is correct H0 is wrong Real world (‘truth’) H0: is false (is a treatment effect) Type I error Type II error The new (treatment) distribution The original (null) distribution = 0.05 Reject H0 Fail to reject H0 Statistical Power Statistics for the Social Sciences Real world (‘truth’) H0 is correct H0 is wrong Real world (‘truth’) H0: is false (is a treatment effect) Type I error Type II error The new (treatment) distribution = 0.05 Reject H0 The original (null) distribution = probability of a Type II error Fail to reject H0 Failing to Reject H0, even though there is a treatment effect Statistical Power Statistics for the Social Sciences Real world (‘truth’) H0 is correct H0 is wrong Real world (‘truth’) H0: is false (is a treatment effect) Type I error Type II error The new (treatment) distribution = 0.05 Power = 1 - Probability of (correctly) Rejecting H0 Reject H0 The original (null) distribution = probability of a Type II error Fail to reject H0 Failing to Reject H0, even though there is a treatment effect Statistical Power Statistics for the Social Sciences • Steps for figuring power 1) Gather the needed information: mean and standard deviation of the Null Population and the predicted mean of Treatment Population 1 60; 2.5 2 55; 2.5 2 1 Statistical Power Statistics for the Social Sciences • Steps for figuring power 2) Figure the raw-score cutoff point on the comparison distribution to reject the null hypothesis 1 60; 2.5 From the unit normal = 0.05 table: Z = -1.645 Transform this z-score to a raw score 1 raw score 1 (Z ) 60 (2.5)(1.645) 55.89 Statistical Power Statistics for the Social Sciences • Steps for figuring power 3) Figure the Z score for this same point, but on the distribution of means for treatment Population 2 55; 2.5 0.355 Z X Remember to use the properties of the treatment population! 55.88 55 2.5 Transform this raw score to a z-score 55.89 Statistical Power Statistics for the Social Sciences • Steps for figuring power 4) Use the normal curve table to figure the probability of getting a score more extreme than that Z score = probability of a Type II error From the unit normal table: Z(0.355) = 0.3594 0.355 Power = 1 - Power 1 0.3594 0.64 55.89 The probability of detecting this an effect of this size from these populations is 64% Statistical Power Statistics for the Social Sciences Factors that affect Power: -level – Sample size – Population standard deviation – Effect size – 1-tail vs. 2-tailed Statistical Power Statistics for the Social Sciences Factors that affect Power: -level Change from = 0.05 to 0.01 = 0.05 Power = 1 - Reject H0 Fail to reject H0 Statistical Power Statistics for the Social Sciences Factors that affect Power: -level Change from = 0.05 to 0.01 = 0.01 = 0.05 Power = 1 - Reject H0 Fail to reject H0 Statistical Power Statistics for the Social Sciences Factors that affect Power: -level Change from = 0.05 to 0.01 = 0.01 = 0.05 Power = 1 - Reject H0 Fail to reject H0 Statistical Power Statistics for the Social Sciences Factors that affect Power: -level Change from = 0.05 to 0.01 = 0.01 = 0.05 Power = 1 - Reject H0 Fail to reject H0 Statistical Power Statistics for the Social Sciences Factors that affect Power: -level Change from = 0.05 to 0.01 = 0.01 = 0.05 Power = 1 - Reject H0 Fail to reject H0 Statistical Power Statistics for the Social Sciences Factors that affect Power: -level So as the level gets smaller, so does the Power of the test Change from = 0.05 to 0.01 = 0.01 = 0.05 Power = 1 - Reject H0 Fail to reject H0 Statistical Power Statistics for the Social Sciences Factors that affect Power: Sample size Recall that sample size is related to the spread of the distribution Change from n = 25 to 100 = 0.05 Power = 1 - Reject H0 Fail to reject H0 X n Statistical Power Statistics for the Social Sciences Factors that affect Power: Sample size Change from n = 25 to 100 = 0.05 Power = 1 - Reject H0 Fail to reject H0 Statistical Power Statistics for the Social Sciences Factors that affect Power: Sample size Change from n = 25 to 100 = 0.05 Power = 1 - Reject H0 Fail to reject H0 Statistical Power Statistics for the Social Sciences Factors that affect Power: Sample size Change from n = 25 to 100 = 0.05 Power = 1 - Reject H0 Fail to reject H0 Statistical Power Statistics for the Social Sciences Factors that affect Power: Sample size Change from n = 25 to 100 = 0.05 Power = 1 - Reject H0 Fail to reject H0 As the sample gets bigger, the standard error gets smaller and the Power gets larger Statistical Power Statistics for the Social Sciences Factors that affect Power: Population standard deviation Change from = 25 to 20 Recall that standard error is related to the spread of the distribution = 0.05 X Power = 1 - Reject H0 Fail to reject H0 n Statistical Power Statistics for the Social Sciences Factors that affect Power: Population standard deviation Change from = 25 to 20 = 0.05 Power = 1 - Reject H0 Fail to reject H0 Statistical Power Statistics for the Social Sciences Factors that affect Power: Population standard deviation Change from = 25 to 20 = 0.05 Power = 1 - Reject H0 Fail to reject H0 Statistical Power Statistics for the Social Sciences Factors that affect Power: Population standard deviation Change from = 25 to 20 = 0.05 Power = 1 - Reject H0 Fail to reject H0 Statistical Power Statistics for the Social Sciences Factors that affect Power: Population standard deviation Change from = 25 to 20 = 0.05 Power = 1 - Reject H0 Fail to reject H0 As the gets smaller, the standard error gets smaller and the Power gets larger Statistical Power Statistics for the Social Sciences Factors that affect Power: Effect size Compare a small effect (difference) to a big effect = 0.05 Power = 1 - Reject H0 treatment Fail to reject H0 no treatment Statistical Power Statistics for the Social Sciences Factors that affect Power: Effect size Compare a small effect (difference) to a big effect = 0.05 Power = 1 - Reject H0 treatment Fail to reject H0 no treatment Statistical Power Statistics for the Social Sciences Factors that affect Power: Effect size Compare a small effect (difference) to a big effect = 0.05 Power = 1 - Reject H0 Fail to reject H0 Statistical Power Statistics for the Social Sciences Factors that affect Power: Effect size Compare a small effect (difference) to a big effect = 0.05 Power = 1 - Reject H0 Fail to reject H0 Statistical Power Statistics for the Social Sciences Factors that affect Power: Effect size Compare a small effect (difference) to a big effect = 0.05 Power = 1 - Reject H0 Fail to reject H0 Statistical Power Statistics for the Social Sciences Factors that affect Power: Effect size Compare a small effect (difference) to a big effect = 0.05 Power = 1 - Reject H0 Fail to reject H0 Statistical Power Statistics for the Social Sciences Factors that affect Power: Effect size Compare a small effect (difference) to a big effect = 0.05 Power = 1 - Reject H0 Fail to reject H0 Statistical Power Statistics for the Social Sciences Factors that affect Power: Effect size Compare a small effect (difference) to a big effect = 0.05 Power = 1 - Reject H0 Fail to reject H0 As the effect gets bigger, the Power gets larger Statistical Power Statistics for the Social Sciences Factors that affect Power: 1-tail vs. 2-tailed Change from = 0.05 two-tailed to = 0.05 two-tailed = 0.05 Power = 1 - Reject H0 Fail to reject H0 Statistical Power Statistics for the Social Sciences Factors that affect Power: 1-tail vs. 2-tailed Change from = 0.05 two-tailed to = 0.05 two-tailed = 0.05 p = 0.025 Power = 1 - Reject H0 p = 0.025 Fail to reject H0 Statistical Power Statistics for the Social Sciences Factors that affect Power: 1-tail vs. 2-tailed Change from = 0.05 two-tailed to = 0.05 two-tailed = 0.05 p = 0.025 Power = 1 - Reject H0 p = 0.025 Fail to reject H0 Statistical Power Statistics for the Social Sciences Factors that affect Power: 1-tail vs. 2-tailed Change from = 0.05 two-tailed to = 0.05 two-tailed = 0.05 p = 0.025 Power = 1 - Reject H0 p = 0.025 Fail to reject H0 Statistical Power Statistics for the Social Sciences Factors that affect Power: 1-tail vs. 2-tailed Change from = 0.05 two-tailed to = 0.05 two-tailed = 0.05 p = 0.025 Power = 1 - Reject H0 p = 0.025 Fail to reject H0 Statistical Power Statistics for the Social Sciences Factors that affect Power: 1-tail vs. 2-tailed Change from = 0.05 two-tailed to = 0.05 two-tailed = 0.05 p = 0.025 Power = 1 - Reject H0 Two tailed functionally cuts the -level in half, which decreases the power. p = 0.025 Fail to reject H0 Statistical Power Statistics for the Social Sciences Factors that affect Power: -level: So as the level gets smaller, so does the Power of the test – Sample size: As the sample gets bigger, the standard error gets smaller and the Power gets larger – Population standard deviation: As the population standard deviation gets smaller, the standard error gets smaller and the Power gets larger – Effect size: As the effect gets bigger, the Power gets larger – 1-tail vs. 2-tailed: Two tailed functionally cuts the -level in half, which decreases the power Why care about Power? Statistics for the Social Sciences • Determining your sample size – Using an estimate of effect size, and population standard deviation, you can determine how many participants need to achieve a particular level of power • When a result if not statistically significant – Is is because there is no effect, or not enough power • When a result is significant – Statistical significance versus practical significance Ways of Increasing Power Statistics for the Social Sciences
