Download â•œObjective Testsâ•š and Human Decisions

"Objective Tests" and Human Decisions
By A. Peter Ruderman
Professor of Health Administration,
The purpose of sampling is to be able to draw sensible
conclusions about a population without having to examine all
of it. The justification for random sampling is that it is
subject to objective tests based on a well-developed theory of
probability. Indeed, statisticians often use the word "subjective" or "judgment" as an antonym to "random." It is
easy to forget in the welter of 5-place tables and meticulously
rechecked worksheets that the objectivity of statistical tests
stops just short of the fmal decision as to significance, and
that decision making is a subjective human process.
Consider the following dialogue between an entomologist
and a statistician, to whom we shall refer as ENTO and
STAT!
ENTO: I have a problem. Here are 2 samples of female
mosquitoes, representing the Chagres and Gatun strains of
Anopheles albimanus which I caught wild in Panama. I want
to know whether DDT makes one strain more irritable than
the other.
STAT: Then set up your experiment. Make sure that all
extraneous influences are eliminated. Make sure that your
samples are completely random.
ENTO: I assure you this has been taken care of. Now I
plan to introduce the mosquitoes, one at a time, into a clear
plastic container with DDT-impregnated paper at the bottom.
I give each mosquito 3 minutes to settle down, and I count
the number of times she takes off from the paper in the next
15 minutes. La donna e mobile.
STAT: Excellent. Then you will calculate the mean
number of takeoffs for each of the 2 strains and test the
significance of the difference between the means. How many
mosquitoes do you have?
ENTO: Ten of each kind. They languish in captivity.
STAT: This is known as a Small Sample. 1 should say that
t would be an appropriate test.
(Interlude while the entomologist runs the experiment and
records his results.)
STAT: I see that the Chagres mosquitoes had a mean of 6
takeoffs in 15 minutes, and the Gatun strain a mean of 12.
Just let me compute the standard enor of the difference
between means. It turns out to be 2.6, so we divide 6 by 2.6
and get t = 2.3. Then we consult the t table ...
ENTO: Why do you put your finger on n = 18? I had 20
mosquitoes.
STAT (a bit snobbishly): These are Degrees of Freedom.
One of our more arcane mysteries.
ENTO: I see it says t = 2.101 when P = 0.05. I don't see
any 2.3 in the table. Does this mean that my 2 strains of
mosquitoes are significantly different in number of takeoffs?
STAT: Yes, at the 5% level.
ENTO: Please give me a straightforward answer with no
if's, at's, or percent's. Is the difference significant?
STAT: No, not significant. That is, at the 1% level.
ENTO: I see we are getting no closer to a decision. At least
tell me this: Why did you pick the 5% level in the first place?
STAT: I am more candid than most members of the
Profession so I shall put it bluntly. The truth is that 2 is close
to 1.96. Well, fairly close. Now if you had caught an infinite
1This dialogue developed at a time when one of the
author's duties was to teach statistics at the former WHO
Malaria Eradication Training Centre in Kingston, ] amaica,
and many perspiring malariologists are thanked for the
stimulus they provided.
University of Toronto
number of mosquitoes instead of just 20, the t distribution
would have been the same as the normal distribution. And if
you go plus or minus 1.96 standard errors from the mean of a
normal sampling distribution you cut off 95% of the area
under the curve. That leaves 5%, see?
ENTO: I see, but can't say I'm very much enlightened.
STAT: If you really want to know, 2 is a convenient
number to talk about. Easy to remember. Dramatic. And if
you don't like the 5% level I suggest the 1% level, because
2.576 is fairly close to 3. Of course, some lecturers like to say
"close to two-and-a-half" but I always used to tell my classes
3.
ENTO: Look, I have to make some decisions. The whole
purpose of my experiment was to learn if that difference was
really significant. What can I do now?
STAT: Well, there arc lots of other points you can use.
Would you like 10% or 2.5% or 0.5%? Each one has a column
in the t table.2
ENTO: I want some logical confidence percentage. You
tell me which one to choose.
STAT: It's like this - with 5% you can usually get some
kind of usable result with a small sample. At 1% your
confidence is higher but you will probably need a larger
sample. Now if you want to be really confident, let me
suggest that you try for 0.1% with a sample of 2000
mosquitoes.
ENTO: Do you know how long it takes to watch 2000
mosquitoes for 18 minutes each? And the money it costs?
There must be something else I can do!
STAT: Well, if you wish you can forget about the
confidence limits in the table. Just take the t of 2.3 which we
computed and see what is the precise probability associated
with it. Here it is! Your answer is 3.362'70! I must say that is
not a very round figure, though the number of decimal places
is impressive.
ENTO: I must formulate my conclusion very carefully, for
the results of the experiment may be vital. This means that
there are 3.362 chances in 100 that 2 independent random
samples of 10 mosquitoes each, with a difference of 6 or
more between the respective mean takeoffs, could have been
drawn from a population in which the Gatun and Chagres
strains had the same mean number of takeoffs.
STAT: Exactly so. Now do you wish to accept or rejllet
the hypothesis that the difference was significant?
ENTO: Do I have any alternative?
STAT: Of course. You can look at the problem in another
way altogether. Don't worry about accepting or rejecting
hypotheses at all, but look at the confidence limits of the
difference.
ENTO: How do I do that?
STAT: You remember that the standard error of the
difference between means was 2.6? Well, the 95% confidence
limits give you a range from 0.5 to 11.5 (with n = 18, twas
2.101), which means that there are 95 chances in 100 that
your samples which had a difference of 6 in mean takeoffs
came from a population where the mean difference was
somewhere between 0.5 and 11.5.
ENTO: I don't like this. I see that your 95% point is just
like the 5% point, and has no more logical justification.
Besides, let's ignore a half takeoff, and just say that the
2There was a copy
Statisticians on the table.
22
of the Biometrika
Tables for
difference is somewhere between 1 and 11 or 12. That lower
figure doesn't imply much difference in irritability but the
upper one docs, and I think next you're going to tell me that
the 99% limits arc anything from -1.5 to 13.5.
STAT: That's right.
ENTO: I am not even surprised by a negative number, for I
take it to be a population where the Chagres mosquitoes were
more irritable than the Gatun instead of the other way
around. I think I am farther away from a sensible conclusion
than ever. What do you think I should do?
STAT: I'm sorry. As a statistician I can set out the values
which form the background for decision, but the actual
decision is up to you.
ENTO: You mean there is no rule for accepting or
rejecting the hypothesis?
STAT: Not at all. Strict application of the rules for
random sampling has given you a whole array of values of t,
each associated with a given probability of happening at
random in samples of given size, with which you can compare
the t from your experiment. Now your personal knowledge
of mosquitoes must take over.
ENTO: Well, in my samples it looked as if the Gatun
mosquitos were twice as irritated by DDT as the Chagres
mosquitoes, but the samples were small and your t test said
"yes" at the 5% point and "no" at the 1% point, which the
exact probability of 3.362 bears out. I must say I fmd this
business of 5 or 1% very arbitrary, but on the other hand
what does this figure of 3.362 really tell me?
STAT: That's for you, not me, to decide.
ENTO: Well, all I can decide at this point is that I am
twice as irritated as those mosquitoes!
If this imaginary dialogue represented an isolated case,
there would not be a problem. However, in the author's
experience with statistics in a variety of fields, the point has
arisen frequently. The user of sample data often gets a
spurious impression of precision because of the attention
paid to computational accuracy, the scientific look of the
formulas, and the number of decimal places in the testing
tables. It would be useful to remember that the objectivity of
tests based on random sampling stops once the probabilities
have been stated.
The final act of decision - though often based on
convenient confidence percentages such as 5 and 1 - involves
a nonobjective human choice by the subject-matter specialist.
FUTURE MEETING
2nd AIBS INTERDISCIPLINARY MEETING
$ 5.00 -
University of Wyoming, Laramie
27 June - I July 1970
$ 8.00 -
The central theme of the 2nd AIBS Interdisciplinary
Meeting will be, "Co-action among Natural Resource Biologists," and will guide the 3-day session for the eight
participating societies: American Fisheries Society, American
Society of Agronomy, American Society of Range Management, Ecological Society of America, Entomological Society
of America, Society of American Foresters, Soil Conservation
Society of America, and The Wildlife Society.
The purpose of the 2nd AIBS Interdisciplinary Meetingwhich does not supplant the regular AIBS Annual Meetingsis to provide the forum to establish closer communication,
better understanding, and greater cooperation among natural
resource biologists, and, consequently, to form the basis for
exerting a more proportionate influence on natural resource
management.
Registration begins on Saturday, 27 June and one or more
field trips will be held on Sunday, 28 June, under the
direction of Dr. Herbert G. Fisser. Meeting sessions will be
held 29-30 June and 1 July.
Selected discussion panels, composed of representatives of
the several disciplines involved, will each day present ad lib
discussion, comments, or questions to the speakers concerning the topic of the day. During the afternoon, all registrants
at the meeting will assemble in workshop groups, thus
providing them the opportunity to participate individually in
a free exchange of ideas.
AIBS will accept advance registration and housing applications until 29 May. Mter 29 May there will be a late
registration fee of $3.00. Register early and avoid the
inconvenience of long lines at the meeting.
$ 3.00 -
Graduate or undergraduate students who
register in advance by mail on or before
29 May.
Graduate or undergraduate students who
register after 29 May by mail or at the
meeting.
Spouse accompanying a registrant, not planning to attend scientific sessions.
Registration Acknowledgments
AIBS will acknowledge all advance registrations received
up to 29 May. These acknowledgments must be presented
upon arrival at the Registration Center, located in Washakie
Center on campus.
Refunds
AIBS will refund "prepaid" advance registration fees on all
cancellations received in writing or telegram before noon on
Saturday, 27 June. NO REFUNDS WILL BE MADE AFTER
THIS DATE.
For further information and registration forms write to:
AIBS REGISTRA nON
3900 Wisconsin Avenue, N. W.
Washington, D. C. 20016
INTERNATIONAL BIOLOGICAL PROGRAM
The Environmental Physiology Subcommittee of the U.S.
National Committee for the International Biological Program
sponsored a working conference centered on chemical interactions between organisms. It emphasized control of insects
by plants, and control of plants by other plants. This
conference was held at the University of California, Santa
Barbara, March 18-22, 1968.
The proceedings of the plant-insect portion "Insect-Plant
Interactions" are available. This publication consists of
highlights of the conference, recommendations, summaries of
the 36 papers presented, and bibliographies. Copies may be
obtained without charge from: International Biological Program, National Academy of Sciences, 2101 Constitution
Ave., Washington, D. C. 20418.
Registration Fees
$15.00
Advance registrations received by mail on or
before 29 May.
$18.00
Late registrations received after 29 May by
mail or at the meeting.
23