Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Lecture 4 Sample size determination 4.1 Criteria for sample size determination 4.2 Finding the sample size 4.3 Some simple variations 4.4 Further considerations RDP Statistical Methods in Scientific Research - Lecture 4 1 4.1 Criteria for sample size determination Suppose that we are to conduct an investigation comparing populations, PA and PB Sample A comprises nA units of observation from PA Sample B comprises nB units of observation from PB Suppose that nA = nB and that n = nA + nB The responses will be quantitative, and the analysis will use a t-test How should we choose n? RDP Statistical Methods in Scientific Research - Lecture 4 2 Let mA = mean response for PA mB = mean response for PB Null hypothesis is H0: mA = mB From the data, we will obtain the sample means x A and x B and sample standard deviations SA and SB for groups A and B Once we have the data, we can: - Reject H0 and say that mA > mB - Reject H0 and say that mA < mB - Not reject H0 RDP Statistical Methods in Scientific Research - Lecture 4 3 When nA = nB = n/2, the t-statistic is t xA - xB 1 1 S n n B A xA - xB n 2S where S n A - 1 SA2 n B - 1 SB2 nA nB - 2 S2A S2B 2 t will tend to be positive if mA > mB, negative if mA < mB and close to zero if mA = mB RDP Statistical Methods in Scientific Research - Lecture 4 4 We will: - Reject H0 and say that mA > mB if t k - Reject H0 and say that mA < mB if t -k - Not reject H0 if -k < t < k Say mA < mB Say mA > mB Do not reject H0 -k 0 k t Now we need to find both n and k RDP Statistical Methods in Scientific Research - Lecture 4 5 Suppose that, in truth, mA = mB This does not mean that we will observe x A x B nor t = 0 In fact, we may observe t k or t -k, just by chance This means that we might reject H0 when H0 is true This is called type I error RDP Statistical Methods in Scientific Research - Lecture 4 6 Suppose that, in truth, mA = mB + d where d > 0, and is of a magnitude that would be scientifically worth detecting We may still observe t k by chance This means that we might fail to reject H0 when H0 is false This is called type II error RDP Statistical Methods in Scientific Research - Lecture 4 7 The probability that t k or t -k, when mA = mB, is called the risk of type I error, and is denoted by a (This is for a two-sided alternative: the probability that t k, when mA = mB, is the risk of type I error for a one-sided alternative and is equal to a/2) The probability that t k, when mA = mB + d is called the risk of type II error, and is denoted by b The probability that t k, when mA = mB + d is called the power, and is equal to 1 - b RDP Statistical Methods in Scientific Research - Lecture 4 8 Reducing type I error Increase k – make it difficult to reject H0 Increasing power Decrease k – make it easy to reject H0 Reducing type I error and increasing power simultaneously Increase n – this will make the study more informative, but it will cost more RDP Statistical Methods in Scientific Research - Lecture 4 9 4.2 Finding the sample size Suppose that the true standard deviation within each of the populations PA and PB is s Then t Z where xA - xB Z n 2s Z follows the normal distribution, with standard deviation 1 When mA = mB, Z has mean 0 When mA = mB + d, Z has mean dn/(2s) RDP Statistical Methods in Scientific Research - Lecture 4 10 Specify that the type I risk of error (two-sided) should be a: P( Z k or Z -k : mA = mB) = a (1) Under H0, Z is normally distributed with mean 0 and st dev 1 k is the value exceeded by a normal (0, 1) random variable with prob a/2 RDP Statistical Methods in Scientific Research - Lecture 4 11 Specify that the type II risk of error should be b: P( Z k : mA = mB + d) = b (2) Under H0, Z is normally distributed with mean dn/(2s) and st dev 1 k - dn/(2s) is the value exceeded by a normal (0, 1) random variable with prob 1 - b RDP Statistical Methods in Scientific Research - Lecture 4 12 For a = 0.05 and 1 – b = 0.90, we have k = 1.960 and k - dn/(2s) = -1.282 Thus 2 1.960 1.282 n 4s 2 d2 s n d Type I error: a RDP s 42.030 d 2 Power: 1 - b 2 0.8 0.9 0.95 0.1 24.730 34.255 43.289 0.05 31.396 42.030 51.979 0.01 46.716 59.518 71.257 Statistical Methods in Scientific Research - Lecture 4 13 Sample size increases: - as s increases - as d decreases - as a decreases - as 1 - b increases RDP Statistical Methods in Scientific Research - Lecture 4 14 4.3 Some simple variations Unequal randomisation nEnC The power of a study depends on n which, for equal sample sizes is equal to n / 2 n / 2 n n 4 For nE = RnC, n = RnC + nC and so n E n C Rn / R 1n / R 1 Rn 2 n n R 1 RDP Statistical Methods in Scientific Research - Lecture 4 15 Unequal randomisation So, the overall sample size is multiplied by the factor 2 R 1 4Rn Fn 2 4R R 1 and nE by FE and nC by FC, where R 1 R 1 FE and FC 2 2R RDP R 1 2 3 5 10 F FE 1 1 1.125 1.500 1.333 2.000 1.800 3.000 3.025 5.500 FC 1 0.750 0.667 0.600 0.550 Statistical Methods in Scientific Research - Lecture 4 16 Unknown standard deviation The sample size formula depends on guessing s If this guess is smaller than the truth, the sample size will be too small and the study underpowered If this guess is larger than the truth, the sample size will be too large and the sample size unnecessarily large A more accurate calculation can be based on the t-distribution rather than the normal, but this makes little difference and does not overcome the dependence on s RDP Statistical Methods in Scientific Research - Lecture 4 17 Unknown standard deviation Often, the final analysis will be based on a linear model, not just a t-test The formulae given can still be used, but s is now the residual standard deviation (the SD about the fitted model) Fitting the right factors will reduce the residual standard deviation, and so the sample size will also be reduced - but you have to guess what s will be in advance! RDP Statistical Methods in Scientific Research - Lecture 4 18 Sample size for estimation The sample size can be determined to give a confidence interval of specified width W The 95% confidence interval for d = mA - mB is of the form 1 1 1 1 x A - x C - 1.96 s d x A - x C 1.96 s nA nB nA nB when sample sizes are large (Lecture 1, Slide 24) When nA = nB = n/2, this has length s 4 2 1.96 s 7.84 n n RDP Statistical Methods in Scientific Research - Lecture 4 19 Sample size for estimation We need to set s 7.84 W n which means that s n 61.47 W RDP 2 Statistical Methods in Scientific Research - Lecture 4 20 Binary data For R = 1, a = 0.05 and 1 – b = 0.90, we have n 4 1.960 1.282 2 p 1 - p 2 42.030 2 p 1 - p where pE log e 1 - pE pC Rp E pC and p - log e R 1 1 - pC pC is the anticipated success rate in PC, and pE the improved rate in PE to be detected with power 1 - b RDP Statistical Methods in Scientific Research - Lecture 4 21 Examples for binary data: R = 1, a = 0.05 and 1 – b = 0.90 RDP pC pE p 1 - p 0.1 0.2 0.811 0.1275 502 0.1 0.3 1.350 0.1600 144 0.1 0.5 2.197 0.2100 42 0.3 0.4 0.442 0.2275 946 0.3 0.5 0.847 0.2400 244 0.3 0.7 1.695 0.2500 60 0.4 0.5 0.405 0.2475 1034 0.4 0.6 0.811 0.2500 256 0.4 0.8 1.792 0.2400 56 0.5 0.6 0.405 0.2475 1034 0.5 0.7 0.847 0.2400 244 0.5 0.9 2.197 0.2100 42 Statistical Methods in Scientific Research - Lecture 4 n 22 Binary data This approach is based on the log-odds ratio pE log e 1 - pE pC - log e 1 p C Many other approximate formulae exist All give similar answers when sample sizes are large: exact calculations can be made for small sample sizes RDP Statistical Methods in Scientific Research - Lecture 4 23 4.4 Further considerations Setting the values for a and b The standard scientific convention is to ensure that a will be small, and allow any risks to be taken with b For example, if an SD or a control success rate is underestimated at the design stage, the study will be underpowered – the analysis maintains the type I error a at the cost of losing power a is the community’s risk of being given a false conclusion b is the scientist’s risk of not proving his/her point RDP Statistical Methods in Scientific Research - Lecture 4 24 Exceptions If the scientist wishes to prove the null hypothesis (equivalence testing) - then b should be kept small, while a can be inflated if necessary In a pilot study, preliminary to a larger confirmatory study - type I errors can be rectified in the next study, but type II errors will mean that the next study is not conducted at all RDP Statistical Methods in Scientific Research - Lecture 4 25 Finally: Many more sample size formulae exist – see Machin et al. (1997) Software also exists: nQuery advisor, PASS Ensure that the sample size formula used matches the intended final analysis In complicated situations, the whole study can be simulated on the computer in advance to determine its power RDP Statistical Methods in Scientific Research - Lecture 4 26