Download Significance of getting the right sample size

Forum Research
research-collins/OF/NH2
01/10/2007
17:19
Page 1
Significance of getting
the right sample size
Claire Collins gives an overview of the principles and variables
that underpin sample size estimation
WHAT SAMPLE SIZE DO I NEED? is probably the question I am
most often asked; however, reference to the sample size estimation is often excluded from research proposals. The
sample size estimation approach depends ,among other
things, on the design of the study, descriptive or comparative, independent or matched, and the type of variable being
analysed.
While some researchers apply the appropriate formulae to
calculate the sample size required, most seem to utilise the
services of a statistician or one of the available sample size
calculators. Whichever you use, you will need to provide certain information in order to determine the sample size
required.
In a single group study, a precision-based sample size is
based on the confidence level and the required width of the
confidence interval. In a two-group comparison, the sample
size is based on significance level, the minimum difference
to be detected at that significance level, and the power to
detect that difference.
If you are testing for an effect, you need to specify the size
of the effect you wish to be able to detect and the probability of detecting it, if it exits (the power of the test). You also
need some estimate of the variability between subjects.
The four principles underlying sample size therefore are:
• Variance and sampling variation: standard deviation (SD)
– variability of individual observations of continuous measurements around a mean in a sample; standard error (SE)
– variability of a mean or percentage from one sample to
another sample. Precision increases as the sample size
increases
• Effect size: This refers to the estimated difference you
expect to see between the two groups under study and is
determined from previous experience/studies or on an estimate of what would be of scientific/clinical interest. The
bigger the effect, the smaller the sample size required to
detect it
• Significance: How likely is it that the observed difference
is due to chance when the true difference is zero? Type I/α
error is where you reject the null hypotheses incorrectly. A
5% level is usual chosen
• Power: How likely you are to detect a difference for a given
sample size, effect size and level of significance. Note if
you increase your sample size by X, your power increases
by X. Type II/β error is where you accept the null hypotheses incorrectly. A 20% or lower level is recommended (ie.
an 80% or higher power).
Often in a study there is more than one variable of inter-
√
est. Ideally, one should calculate the sample size required
for each and select the largest sample suggested. The alternative is to base the calculation on one key variable but this
requires a decision regarding which of the variables will
underpin the sample size estimation.
Note that many of formulae and examples used in demonstration articles and most sample size tables assume that
the sample is being selected using a simple random sample
technique and are not applicable to samples selected using
alternative techniques.
The sample size estimations do not allow for non-contacts
and non-response. You should consider increasing your
target sample size to allow for these. In addition, your
sample may need to be increased to permit internal comparisons (eg. between males and females, etc.).
The sample must be sufficiently big enough that an effect
of scientific significance will also be statistically significant
but not so big that an effect of little scientific importance is
statistically detectable.
In reality, the size of the sample selected is a balance
between precision and feasibility, with consideration given
to economic (cost and time resources) and ethical (risk of
involvement) concerns.
In most circumstances, you are likely to estimate the
sample with varying effect size and significance and power
levels in order to determine what is best suited to your project, given all considerations. The references below, used to
compile this summary piece, contain many examples of
sample size calculation.
Claire Collins is ICGP director of research
References
1. Daly, LE, Bourke, GJ. Interpretation and Uses of Medical Statistics 5th
edition. UK, Blackwell Science Ltd., 2000
2. du V Florey, C. Sample size for beginners. BMJ 1993; 306: 1181-1184
3. Israel, GD. Determining sample size. http://edis.ifas.ufl.edu/PD006.
University of Florida. Original publication Nov 1992. Reviewed June 2003
4. Lenth, RV. Some practical guidelines for effective sample size
determination. www.stat.uiowa.edu/techrep/tr303.pdf. University of Iowa,
2001
FORUM October 2007 25