Download QMIN: Introduction

Chapter 1
Introduction
Why study statistics in neuroscience? There is one and only one reason—lab
efficiency.
1.1
Neuroscience Does Not Need Statistics
60
20
40
Calls per Hour
80
100
Huh? Yes, you read that correctly.
There really is no ontological need for
statistics in neuroscience or, for that Figure 1.1: Number of calls to directory
matter, in many other sciences. Fig- assistance before and after implementaure ?? is an adaptation from Camp- tion of a fee.
bell & Stanley (1963) that plots the
frequency of telephone calls to directory assistance before and after a
charge was instituted by a telephone
company for the call. Can you guess
the exact time when the charge was
first levied? Did the charge influence
the number of calls?
The answer to both questions is
perfectly obvious. The charge was instituted shortly after the 600th time
period (627, to be exact), and the
number of calls dropped almost to the
level when the number of calls was
first recorded. The figure also illustrates that the cost initially inhibited
folks from calling directory assistance
but afterwards the rate of increase in calls was pretty much the same as the rate
of increase in the period before the charge.
There are statistical methods used by econometricians that could analyze
0
200
400
600
Time
1
800
1000
CHAPTER 1. INTRODUCTION
2
these data, but applying them (interrupted time series) is not easy for most
people and the process is overkill, akin to duck hunting with a canon. There is
no need to perform any statistics to test whether these observations are “due to
chance.” Were an obstreperous reviewer to demand such statistics, one would
naturally calculate them in order to get the paper published (or grant funded).
But one is reminded of Ralph Waldo Emerson’s observation that “A foolish
consistency is the hobgoblin of small minds.”
If a neuroscientist were interested in whether a treatment alters a response,
then several methods are available to assess the hypothesis without using statistics. If the response could be measured over many time points, then a plot like
the one in Figure ?? be constructed for every animal tested. The skeptic might
(quite rightly) respond that this approach would work if the effect of the treatment were very large. What would one do if the treatment effect was small?
One could still answer the question without statistics by testing a very large
number of animals and plotting the mean response at each time point. The
larger the number of animals, the more the plot of the means before and after
the response will resemble smooth curves and the greater the ability to visually
see the effect of the response. How many aminals might it take? That depends
upon the magnitude of the effect and the variability of the response from one
animal to another. For the moderate effect sizes in behavioral neuroscience, it
may take several thousand animals.
Many responses in neuroscience cannot be measured over time. In such cases,
one assigns some animals randomly to a control group and others to a treatment
group. Once again, there is no need to use statistics to test if the treatment
group differs from the control group. Just test a sufficiently large number of
animals so that visual inspection of their histograms clearly demonstrates that
the distributions differ. Again, “sufficiently large” could mean several hundreds
to several thousands per group.
Naturally, no sensible neuroscientist would take this approach. Why? No
matter how you phrase the answer, it always reduces to lab efficiency. Comments such as “take too much time,” “not worth all the trouble,” and “would
be prohibitively expensive” all say in some way that it would stretch lab resources to take such an approach to science. Instead, the customary approach
is to make a reasonable guess about the effect size of the hypothetical response
and use statistics to arrive at a reasonable sample size that will give a good
likelihood of observing a real effect. Thus, the use of statistics reduces the number of animals tested at the cost of slightly increased uncertainty in the results
(all that “reasonable” and “good likelihood” stuff in the previous sentence. The
more certainty, the more animals are needed. Statistics provide an objective
compromise between absolute certainty and the cost and time needed to arrive
at reasonable inferences.
At the same time, when one views results that do not need statistics, then
the lab that produced them either had a great idea or was very inefficient in
testing their hypothesis. In some research, sample sizes are constrained—e.g.,
you have already performed a study, but want to perform a new assay on tissue
saved from the study. Here, a “visually obvious” result is serendipitous.
CHAPTER 1. INTRODUCTION
1.2
3
A Little Statistics Can Go a Long Way
Panels A and B of Figure 1.2 present the same data. Do you notice antyhing
awkward about panel B? If you were to submit a paper or a grant proposal with
panel B, what response would you expect from the reviewers?
(A)
(B)
Figure 1.2: Mean response for four doses of a drug.
Panel B is not “wrong.” After all, it contains the same data as in panel A.
It is just that panel A presents the data in such a way as to make it easier to
assess the dose-response relationship. Hence, although panel B is not wrong, it
is stupid.
Without consciously being aware of it, many scientists treat their data in
a “panel B” fashion. Again, there is nothing “wrong” with this. It is just that
it is awkward, leads to lab inefficiency, and sometimes obscures science. Let
us spend a few minutes explaining why this is so.The study for Figure 1.2 uses
four groups, but they are not groups in the sense of dividing people in religious
groups such as Jewish, Christian, Muslim, Hindu, etc. The order of the religious
groups is arbitrary as is the order of male or female. Religious affiliation and sex
are strictly categorical variables. The groups in Figure 1.2 have an underlying
metric associated with them.
When faced with “groups,” many researchers automatically think of a statistical procedure called the analysis of variance or ANOVA. But using ANOVA
with these data is treating them as if they religion or sex. In short, ANOVA
is a “panel B” approach to the statistical analysis. The ANOVA is not wrong.
CHAPTER 1. INTRODUCTION
4
It will inform us whether there are indeed differences somewhere among the
groups and using statistical gymnastics such as multiple comparisons, one may
even get some insight into where those differences occur.
The “panel A” approach would capitalize on the quantitative information
that orders the group. Here, the subjects would be given actual numerical
values for the amount of drug administered and the data would be analyzed by
regression, a close cousin to ANOVA. The regression is more powerful than the
ANOVA. That is, it is more likely to detect real differences among the groups
than the ANOVA. Hence, planning a study using a regression will sometimes
result in the use of fewer animals. Also, the regression might also be more
informative. Visual inspection of panel A suggests that the response asymptotes
between 15 and 20 mgs. Is this real or, by dumb luck, did the 15 mg group scored
higher than its population value? The regression can answer this better than
the ANOVA.
How much extra time would it take to perform the regression, even the more
complicated one that asked whether the response does in fact asymptote? Perhaps a few seconds. In today’s world of point-and-click statistics, both ANOVA
and regression are contained in a single technique called the General Linear
Model or GLM. Hence, the only difference in is clicking on a few different boxes
in the GLM. So ask yourself, “Is it worth a second or two?”
1.3
. . . but Seldom All the Way
Some people view statistics with mysticism. For most of these folks, the mysticism takes the form of suspicion. Others, however, have the mistaken impression
that statistics will magically pull a rabbit from a hat and reveal vast undiscovered relationships in their data. These people spend days and days performing
ever more sophisticated analyses on a single problem. While extensive data
analysis is justified for large epidemiological data sets and for some genomic
and imaging studies, it is not appropriate for the small data sets in most experimental neuroscience.
1.4
References:
Campbell, D. T. & Stanley, J.C. (1963). Experimental And Quasi-experimental
Designs For Research. Chicago: Rand McNally.