Download Permutation tests - People Server at UNCW

• For a permutation test, we have H0:
F1(x) = F2(x) vs
Ha: F1(x) <= F2(x)
• Note this alternative means that the
density of the first population is
larger than that of the second...
sketch to see this!
• Special case is the shift alternative:
Ha: F1(x) = F2(x-D), where D > 0.
Sketch this!
• We may also have the alternative Ha:
F1(x) >= F2(x) and the two-sided
alternative
Ha: F1(x) <= F2(x) or F1(x) >= F2(x),
for all x, with strict inequality
occurring for at least one x. For the
shift alternative, this is D ne 0.
Think of D as the difference between
medians of the two populations...
• Permutation tests may also be
performed on other statistics besides
the mean – of course, if population(s)
are normal then mean are probably
best – the textbook mentions ones
based on the median and the trimmed
mean. This gives permutation tests
much flexibility ...
• Note in Table 2.2.1 that changing the
max. value has no effect on the
medians but could impact the mean
• p-values obtained from permutation
distributions of test statistics are
exact in the sense that they are not
dependent upon unverified
assumptions about the underlying
population distribution ...
• Approximate p-values may be
obtained from random sampling of
permutations and for large number of
random samples, the error can be
quite small – see bottom of page 32
for margin of error...
Permutation tests
• We may also get approximate p-values by randomly
sampling the permutations, instead of trying to write
them all down. This is useful when m+n is large…
• Do as before:
– assign experimental units to the two groups at
random and compute the difference between the
means of the two groups, Dobs . There are m units
assigned to group1 and n units to group2 (m+n total
units).
– randomly "sample" all the m+n observations so there
are m in group1 and n in group2.
– compute the difference between the means of the two
groups of the "sampled" vector, D.
– repeat this procedure a large number of times (1000
or larger). For an upper-tailed test, calculate the
empirical p-value: # of times D>= Dobs / 1000
– make your decision about rejecting the null
hypothesis based on this empirical p.
– this empirical p-value is approximately normal with
mean = true p and standard deviation =
sqrt(p(1-p)/R)
where R=# of randomly sampled permutations (1000
above)
• Do example 2.3.1 on page 33 - use various test
statistics with R.