Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Part of the Web Site of Dr. Jeffrey Oescher EDF 802 Index Page Risks and Errors, Power and Sample Size, and Effect Size Revised – 30 January 2014 Formative Exercises - Topic 2 I. Errors and risks A. Statistical decisions to reject or accept the null hypothesis 1. Correct decisions a. Accept H0 when it is true b. Reject H0 when it is false 2. Errors a. Type I – reject H0 when it is true b. Type II – accept H0 when it is false Table 1 Type I and Type II Errors Reject H0 Statistical Conclusion B. Accept H0 Reality H0 is True H0 is False Type I Error Correct (α) Decision (1-β) Correct Type II Error Decision (1-α) (β) Consequences of Type I and Type II errors are a function of the researcher’s values 1. Educational example a. Consider the use of an expensive computerized writing program. If H0 is rejected and it is true (i.e., Type I error), the program is selected even though it is very expensive and will not result in increased achievement. If H0 is accepted but a difference really does exist (i.e., Type II error), the program will not be used and the students will not have the advantage of using an effective program. b. Which error would you like to protect against if the program director wants to showcase your schools and is willing to provide them with all of II. the hardware, software, and training necessary to use this program effectively? c. Which error would you like to protect against if you want the students to benefit from an exemplary program? 2. Medical example a. Consider the use of a new, very costly drug. If H0 is rejected and it is true (i.e., Type I error), the new drug is used even though it is costly and not helpful. If H0 is accepted but a difference really does exist (i.e., Type II error), the drug will not be used and the patients will not receive the benefits from it. b. Which error would you like to protect against if several effective drugs are available already? c. Which error would you like to protect against if there are no known effective drugs and the health problems prompting the use of this drug are increasing at an alarming rate? 3. Urban study example a. Consider the consequences of an impact study related to the revitalization of a major urban recreation area. The predicative models suggest the changes to the park will increase usage and revenues to a level that the facilities would become self-sustaining. If H0 is rejected and it is true (i.e., Type I error), the renovations are completed even though they are very expensive and will not result in increased usage or revenue. If H0 is accepted but the changes really would create a self-sustaining recreational area (i.e., Type II error), the renovation will not be undertaken and the citizens will not have the advantage of using an improved, financially viable recreation park. b. Which error would you like to protect against if the city is in desperate need of a recreation area that would support its youth and older citizens and, perhaps more importantly, is willing to pay whatever costs are involved in developing it? c. Which error would you like to protect against if the city has indicated they cannot provide any financial support for this project, although the need exists. Controlling errors A. The goal of any researcher is to minimize Type I and Type II B. C. III. errors 1. As Type I errors are minimized, Type II errors are maximized 2. As Type II errors are minimized, Type I errors are maximized An error can be judged only relative to reality 1. Reality is never known 2. This is the reason statisticians work in probabilities No error exists until a decision is made 1. The only time a Type I error is made is when H0 is rejected a. The probability of this error is alpha (α) b. Protecting against a Type I error i. Decrease alpha (α) – that is, α=.05 rather than .10 or α=.01 rather than .10) ii. As a consequence beta (β) increases c. See Figures 1A, 1B, and 1C 2. The only time a Type II error is made is when H0 is accepted a. The probability of this error is beta (β) b. Beta (β) is not as straightforward as alpha (α) c. Protecting against a Type II error i. Increase alpha (α) – that is α=.05 rather than .01 or α=.10 rather than .05) ii. As a consequence β decreases iii. Increase sample size d. See Figures 2A and 2B Power A. Correctly rejecting H0 when it is false (i.e., making a correct decision) B. Algebraically noted as (1-β) C. Consider an appropriate level of power to be a 4:1 ratio of α to β D. Factors affecting power 1. Directional nature of H0 2. Level of significance a. If you increase alpha (α), you increase power (1β) b. If you decrease alpha (α), you decrease power (1β) 3. Effect size a. The difference between the value specified in H0 and the value specified in H1 b. Increasing effect size separates the two sampling distributions E. Sample size and power 1. Larger samples mean smaller standard errors which IV. effectively reduces the overlap between distributions and thus power increases 2. This is the only factor affecting power over which the researcher has a reasonable level of control a. The directional nature of H0 is determined by the nature of the research question b. Alpha level is set a-priori based on the researcher’s concern for Type I and Type II errors c. Effect size is directly related to the success of the treatment 3. Two perspectives a. A-priori – Tables for sample sizes based on assumption about effect size, power, and alpha (See Hinkle, Wiersma, and Jurs or Cohen) b. Post-hoc – See specific SPSS Windows output from procedures like ANOVA, GLM, etc. Practical significance A. Distinguishing statistical and practical significance 1. The effect of large samples on statistical significance 2. Concern for the “meaning“ of results B. Estimates of practical significance 1. Strength of association – the proportion of the variance in the dependent variable that can be explained by the independent variable a. r2 b. Eta squared c. Omega squared 2. Standardized effect size – a method of expressing mean differences in standard deviation units 3. See Table 2 Table 4 Interpreting Selected Measures of Effect Size Analysis Measure of Effect Size Correlations Means r Eta squared Partial Eta Squared Omega squared Partial Omega Squared Cohen’s f Mean Difference (d) Small .10 .01 .01 .01 .01 .10 .20 Effect Size Moderate .30 .06 .06 .06 .06 .25 .50 Large .50 .14 .14 .14 .14 .40 .80