Download Classical Conditioning Since Pavlov

Review of General Psychology
2006, Vol. 10, No. 4, 365–376
Copyright 2006 by the American Psychological Association
1089-2680/06/$12.00 DOI: 10.1037/1089-2680.10.4.365
Classical Conditioning Since Pavlov
M. E. Bitterman
University of Hawaii
In the three quarters of a century following Pavlov’s work, the accumulation of
factual information about classical conditioning has continued, but there has been
little conceptual progress. The only thing we have now that approximates a
workable general theory of conditioning was introduced more than 30 years ago and
continues to receive a good deal of respectful consideration despite a variety of
generally recognized shortcomings that little has been done to repair; nor does a
systematic review of recent papers in leading journals give any good reason to think
that a more satisfactory theory is in the making. A remedial strategy, recommended
long ago by C. L. Hull and by E. R. Hilgard, is proposed and exemplified by some
research with honeybees.
Keywords: classical conditioning, Pavlov, Hilgard, fishes, honeybees
Today, we remember Ernest R. Hilgard, a
scholar of enormous accomplishment, whose
first experimental work was on classical conditioning (Hilgard, 1931). Early in his career, he
published with Donald Marquis a pivotal analysis of classical and instrumental conditioning
that continues to be required reading (Hilgard &
Marquis, 1940). I first came under his influence
soon after my arrival as a graduate student in
Howard Liddell’s laboratory at Cornell, when I
was referred to that important book.
Liddell’s laboratory was an old farm, called
the Behavior Farm, and the subjects were farm
animals (Liddell, James, & Anderson, 1934). In
my first semester there, I began to earn my
Pavlovian credentials in an experiment on aversive conditioning in a sheep, with brief, unavoidable shock to a limb as the unconditioned
stimulus. Various responses of the animal, skeletal and autonomic, were captured by what was
then a state-of-the-art recorder that put as much
ink on me as on the recording paper but even so
was preferable to the smoked drums we still
were using in the physiology laboratory. At the
same time, I read Pavlov, starting with his Nobel lecture of almost exactly a century ago (Pavlov, 1904/1967), in which he described his initial observations of conditioning in the course of
the work on the digestive glands for which the
prize was given. Then I settled down with Conditioned Reflexes (Pavlov, 1927), a review of
the quarter century or so of intensive work in
Pavlov’s laboratory that followed his initial observations. Liddell, who had close ties to the
psychiatric community, was interested primarily in the pathogenic possibilities of what was
called defensive conditioning, and so it remained for Hilgard and Marquis to provide me
with a broader psychological perspective.
Where We Are Now
The work with honeybees that is described was supported
by a series of grants from the National Science Foundation,
currently Grant IBN03-46546.
This article is based on an address by the author, as
recipient of the Ernest R. Hilgard Award for Distinguished
Contributions to General Psychology, at the 113th Annual
Convention of the American Psychological Association in
Washington, DC, August 2005.
Correspondence concerning this article should be addressed to M. E. Bitterman, Békésy Laboratory of Neurobiology, University of Hawaii, 1993 East-West Road, Honolulu, HI 96822. E-mail: [email protected]
Pavlov learned a great deal about conditioning, and it is interesting to ask how much more
we have learned about it since then. In my
opinion, we have made little conceptual
progress. We are still today, as Hilgard (1948)
regretted in the first edition of his influential
Theories of Learning, and, as Gray (1979) put it
so well several decades later, in the uncomfortable position of having many facts that seem to
provide significant clues to the underlying processes but no adequate theory to account for
them. Other people working in the field are
365
366
BITTERMAN
more complacent—Rescorla (1988), for example, has claimed “dramatic” conceptual
progress— but the complacency is not well
founded. Some ideas that are supposed to be
new are not at all new, as a reading of Pavlov
would show. Some really are new, but untenable.
A common claim is that Pavlov was wrong
about the importance of conditioned stimulus–
unconditioned stimulus (CS-US) contiguity.
The new idea is that conditioning is based not
on contiguity but on contingency—that conditioning occurs only when the probability of the
US in the presence of the CS is different than in
its absence. In the primordial contingency experiment advertised by Rescorla (1988), one
group of rats was shocked only in the presence
of a CS, whereas a second group was shocked
with the same probability both in the presence
of the CS and in its absence. Although the
number of CS-US parings was the same for both
groups, the animals of the first group responded
differentially to the CS but the poor animals of
the second group (shocked five times as often as
the first) did not, which Pavlov might have
found perfectly understandable since, for those
animals, both the CS and the context were conditioned.
The general acceptance of the contingency
idea is discouraging, not only in view of the
limitations of the work that first suggested it,
but also because it is directly contradicted by
the results of many subsequent experiments
(Papini & Bitterman, 1990). Consider only one
of them, an autoshaping experiment by Brandon
(1981): A single group of pigeons is trained
with three colors that follow each other in haphazard sequence. In the presence of one color,
free food is given twice per minute on average;
in the presence of the second color, once per
minute; and in the presence of the third color,
never. The pigeons come to respond differentially to the second color (less than to the first,
more than to the third), although the probability
of reinforcement is exactly the same in its presence as in its absence. Like results for goldfish
have been obtained in analogous experiments
both with food and with shock as reinforcement
(Brandon, Satake, & Bitterman, 1982).
Almost everybody points proudly to the discovery of blocking (Kamin, 1969), which is said
again to demonstrate the insufficiency of contiguity. The prototypical design of a blocking
experiment, given uncritically in the textbooks,
is this: A blocking group is conditioned first to
stimulus A and then to a compound of two
stimuli, A and B; a control group is conditioned
only to the compound; and then both groups are
tested with B. Although the contiguity of B and
reinforcement is the same for both groups, there
is less responding to B in the first group than in
the second. A variety of other control procedures have been used in efforts to establish that
it is the conditioning of A which is critical,
rather than the prior reinforcement apart from
experience with A, or the prior experience with
A apart from reinforcement. A has been backwardly paired or explicitly unpaired with reinforcement, or reinforced trials with a different
stimulus, C, have been substituted for reinforced trials with A, all with the same reported
outcome: less responding by the blocking animals than by control animals in the test with B.
The traditional explanation is that learning
about B on reinforced trials with the AB compound is somehow impaired by the conditioning
of A, which competes with B for associative
strength or for attention, but similar results have
been obtained in recent work with B entirely
absent in the training (Blaser, Couvillon, &
Bitterman, 2006). The new results suggest that
differential responding to B in conventional between-groups blocking experiments may be
traced, not to different learning about B, but to
generalized effects of the different treatments of
other stimuli in the blocking and control groups.
It now seems fair to say that no such demonstration of blocking is credible unless accompanied by null results for B-absent controls. After
a good many years of work on blocking, we are
pretty much back at the beginning.
In any case, Pavlov knew very well that performance in conditioning situations cannot be
understood in terms of contiguity alone. It was
he, after all, who discovered overshadowing:
strong responding to B after reinforced training
with B alone but very little after reinforced
training with AB. Diminished responding to B
alone after the AB training did not mean, Pavlov
noted, that little had been learned about it; clear
evidence to the contrary was provided by
greater responding to AB than to A alone. We
are still not sure about what causes overshadowing or its opposite, called potentiation, which
sometimes occurs.
CLASSICAL CONDITIONING SINCE PAVLOV
Of course, we can all agree that there has
been at least some conceptual progress in the
years since Pavlov. (a) We now have a somewhat better appreciation than he had of the
generality of Pavlovian conditioning, both over
animals and over systems within animals. With
a drop of sucrose solution as the US and extension of the proboscis as the response, harnessed
honeybees perform like Pavlov’s dogs in a wide
range of comparative experiments (Bitterman,
Menzel, Fietz, & Schäfer, 1983), although the
nearest common ancestor lived hundreds of millions of years ago and probably had only an
extremely simple central nervous system. (b)
There is better methodological understanding.
Pavlovian conditioning now is studied not only
in classical situations (with reinforcement independent of response), but also in Thorndikian or
instrumental situations, as Hilgard and Marquis
(1940) termed them. Awkward early efforts to
treat classical training as instrumental training
in disguise were put to rest after a time by
omission experiments as conceived by Sheffield
(1965). We understand now that quite the opposite is true—that Pavlovian processes are embedded in Thorndikian procedures, which afford new opportunities to study them; when an
animal makes an instrumental response, it is, in
effect, giving itself a classical conditioning trial.
(c) We are clearer about the nature of the associations that are formed. Pavlov (1904/1967)
speculated from the very outset on the possibility of connections between sensory centers in
the brain—what later were called S-S, as distinct from stimulus–response (S-R) connections—and the idea has since been amply supported by several lines of work dating back to
an experiment by Brogden (1939) on sensory
preconditioning. (d) Thinking about inhibition
has advanced. Pavlov believed that a stimulus
becomes inhibitory in the course of continued
action, but the modern idea is that a stimulus
becomes inhibitory only when encountered
without reinforcement in an otherwise excitatory context (Konorski, 1948). (e) Discrimination experiments have long pointed to the need
for some such concept as attention and its modification with training (Sutherland & Mackintosh, 1971). Over the years, there have been
many interesting suggestions as to how the idea
might be formalized, although their potential
still is largely unrealized.
367
So there has been some conceptual progress,
although not very much to show for three quarters of a century of work. It is interesting, too,
that none of these post-Pavlovian developments
is very new. The only thing we have now that
begins to approximate a general theory of conditioning was introduced more than 30 years
ago by Rescorla and Wagner (1972). One interesting (and widely credited) postulate of the
theory is that the components of a compound
stimulus share the changes in associative
strength generated by reinforcement and nonreinforcement of the compound. Another is that
excitation and inhibition lie on a single continuum—that inhibition is negative excitation—
which (with little empirical support) contradicts
Pavlov’s assumption that a stimulus can have
both excitatory and inhibitory properties at the
same time. An especially attractive feature of
the theory is its statement in equational form,
the old linear equation of Bush and Mosteller
(1951) in a different and now much more familiar notation, which opens the door to quantitative prediction. That door, unfortunately, remains unentered. Without values for the several
parameters of the equation, associative strength
cannot be computed, which means that predictions from the theory can be no more than
ordinal, and even those predictions are made on
the naı̈ve assumption of a one-to-one relation
between associative strength and performance.
Despite a variety of shortcomings (Miller, Barnet, & Grahame, 1995), the theory has continued to receive a great deal of respectful consideration, generating forests of competing bar
graphs. Looking for alternatives, we find only a
disjointed array of rather insular conceptions
(Pearce & Bouton, 2001; Wasserman & Miller,
1997).
If we do not have a satisfactory general theory of conditioning, are we at least on the way
to developing one? I am afraid not. We simply
do not begin to have what we need for the
purpose, which is first of all a homogeneous
body of data, exactly replicable and readily extendable, to which people working in different
laboratories contribute. That, of course, would
entail agreement on a well-characterized model
animal and on a set of highly standardized and
efficient training techniques, but there is no
indication that the need is even recognized. A
few months ago, I suggested to Rachel Blaser, a
doctoral candidate at the University of Hawaii,
368
BITTERMAN
that it might be useful to have a systematic look
at relevant papers—their subjects, methods, and
topics—published during the past 10 years in
some of our leading journals. Table 1 summarizes what she found in 236 papers appearing
from 1995 to 2004 in Animal Learning & Behavior (Animal now has been dropped from the
title) and in the Journal of Experimental Psychology: Animal Behavior Processes.
As should come as no surprise, the experiments were conducted mostly with rats and
pigeons, although there are bits and pieces of
data also for an odd assortment of other animals, the reasons for the choice of which are not
often evident. Training techniques varied
widely, some of them terribly crude. In experiments on conditioned food aversion, an animal
ingests some food that waits in its gut for a
poison given later, or sometimes even before
the food, to take effect. In any case, there is no
meaningful control either of the CS, the US, or
the CS-US interval. In “freezing” experiments,
an animal is shocked in a box, removed, and
then returned some time later, when any restriction of its movement is noted. I remember, in a
report of one such experiment, an earnest attempt to describe exactly how the animals were
carried to the box, certainly a possible source of
unspecified conditioned stimuli. More sophisticated techniques also were used, but they differed so much in detail as to make it impossible
to pool results from different laboratories. It is
interesting to note a decline (relative to my own
count of papers in the same journals for the
years 1981–1990) in the use of the old conditioned suppression technique, which not only is
rather inefficient (most of the experimenter’s
time is spent in preparing for a few terminal
measurements) but also has the not-inconsiderable disadvantage, as commonly used, of failing
to track the course of acquisition. The topics
also were rather diverse. There were some notable convergences of interest, but the concerns
so scattered that fewer than two thirds of them
could be meaningfully classified, even with
rather broad categories. Some of the results
reported are trivial, others are intuitively interesting although largely undigested, and there
are numerous unresolved contradictions.
Consider this problem in compound conditioning: One group of animals has reinforced
training with three individual stimuli, A, B, and
C. A second group has reinforced training with
the three two-component compounds composed
of the same three stimuli, AB, AC, and BC. Then
both groups are tested with the three-stimulus
compound, ABC. Which will respond more?
(Note the ordinal nature of the question.) Rescorla–Wagner theory suggests that the first
group will respond more, whereas Pearce’s
(2002) configurational variant of the theory suggests that the second will respond more. What
actually happens depends, presumably, on
whether you do a rabbit eyelid-conditioning experiment with light, tone, and tactile stimuli in
New Haven, where the first group responds
Table 1
Summary of 236 Papers—Animals, Techniques, and Topics—From 1995 to 2004
Animal
Rat
Pigeon
Rabbit
Mouse
Quail
Miscellaneous (snail, ferret, shrew,
drosophila, gerbil, wasp,
goldfish, etc.)
Technique
76
11
4
3
3
3
Foodbox approach or entry
Food aversion
Conditioned suppression
Autoshaping
Freezing
Mazes or runways
Eyelid conditioning
Sign-tracking
Heart rate
Miscellaneous behavioral
observations (activity, startle,
rearing, retching, orienting,
shuttling, etc.)
Topic
21
15
15
11
9
8
5
3
1
12
Compound conditioning
Extinction
Latent inhibition
ITI and ISI effects
Sensory preconditioning
Conditioned inhibition
US properties
Miscellaneous (not readily
classified)
21
11
10
7
5
4
4
38
Note. The classification is by Rachel E. Blaser of 236 papers in Animal Learning & Behavior and the Journal of
Experimental Psychology: Animal Behavior Processes (1995–2004). The data are percentages of papers. ITI ⫽ intertrial
interval; ISI ⫽ interstimulus interval.
CLASSICAL CONDITIONING SINCE PAVLOV
A Way Forward
The time may have come for us to turn again
to a once widely admired strategy pioneered
long ago by Hull (1935), who unfortunately did
not have the computer resources necessary to
implement it properly. The strategy was forcefully recommended by Hilgard (1948, 1956),
who called attention to the importance of “miniature systems” in the early development of the
physical sciences. The idea is to begin with (a)
a carefully chosen model animal; (b) some basic
questions; (c) an efficient, objective, and highly
standardized training technique designed to
yield results that are exactly reproducible; and
(d) the simplest conceivable theoretical account
of the results in the form of a set of equations
with meaningful parameters that are carefully
evaluated. Then there are new experiments designed to test exact predictions from the theory,
which is modified systematically as required by
expansion of the database and (Hilgard’s
phrase) progressive broadening of the boundary
conditions.
Some contemporary work with honeybees,
still in an early stage, may provide a useful
example. Individual subjects are pretrained to
forage at a laboratory window, where on arrival
they find two targets— disposable petri dishes
labeled with different colors or odors—that are
set out on the deep shelf of the window (Couvillon & Bitterman, 1985). One target contains
a drop of 50% sucrose solution and the other
contains a drop of water, which is unacceptable
and can be distinguished from the sucrose only
by taste. If the animal goes first to the water, it
is permitted to correct its choice. After filling its
“social stomach,” the animal leaves for the hive
to deposit the sucrose it has collected, returning
to the window a few minutes later to make
another choice. Of course, the lateral arrangement of the two targets is changed quasirandomly from visit to visit. Figure 1 shows a
1.0
CHOICE OF A
more (Myers, Vogel, Shin, & Wagner, 2001), or
a pigeon key-pecking experiment with differently positioned colored lights in Cardiff, where
the second group responds more (Pearce, Aydin, & Redhead, 1997). A third conception of
compounding (Wagner, 2003)—the “replaced
elements” conception—provides an explanation
of either outcome, although only after the fact.
How are we to move on?
369
0.8
0.6
0.4
Obtained
Simulated
0.2
0
0
5
10
VISITS
15
20
Figure 1. Obtained and simulated proportions of honeybees choosing A, the originally reinforced of two odors, on
each trial of a l0-trial reversal problem (Couvillon & Bitterman, 1985).
typical acquisition curve for a discrimination
between two odors, with the positive and negative alternatives reversed after 10 trials; the
performance of a group of 8 subjects is plotted
there in terms of the probability of correct
choice on each trial.
To explain the results of this and several
other rather basic discrimination experiments
performed at the same time, we began with a
simple contiguity theory consisting of two postulates in equational form, one dealing with
learning and the second with performance. The
first postulate is that the attractiveness of a
target is given by the strength of its association
with sucrose, which increases with reinforcement to some asymptotic value and decreases to
zero with nonreinforcement. The second is that
the probability of choosing a given alternative is
a power function of its relative associative
strength. Each of the equations has several parameters to which values must be assigned if
performance is to be computed; for the learning
equation, they are the salience of the CS, the
rates of growth and decline in associative
strength with reinforcement and nonreinforcement, and the asymptotic level of associative
strength determined by the magnitude of reinforcement; for the performance equation, slope
and curvature parameters. The question then
asked was whether we could find values for the
parameters that would yield a close simulation
of all the data; the answer, as Figure 1 illustrates, is that we could, which is perhaps not
very surprising. (Who was it who once said that,
370
BITTERMAN
given a few free parameters, he could draw a
picture of an elephant?) A more challenging
question, hardly ever asked, was whether, with
the same parameter-values, we could exactly
predict the results of new experiments; the answer again is that we could. Figure 2 shows, for
example, the predicted and obtained results of a
new experiment with the same stimuli in which
four initial training trials were followed by
seven 4-trial reversals (Couvillon & Bitterman,
1986).
Because the question of how learning is
translated into performance is so often ignored,
it may be interesting to look at the choice function (intermediate between what have been described as “maximizing” and “matching”) that
yielded the best fit to the original data. The
function is plotted in Figure 3, along with the
good predictions it provided of the results of a
subsequent experiment on probability learning
that was designed especially to constrain it
(Fischer, Couvillon, & Bitterman, 1993). Seven
of nine groups of honeybees (individually
trained, as always) began with one of two alternatives (odors) consistently reinforced, the
other never reinforced (a 100:0 problem), and
then were either continued at that ratio or
shifted to one of several others (90:10, 80:20,
70:30, 50:50, 20:80, or 0:100). An eighth group
began at 70:30 and was shifted to 30:70. For a
ninth group, the ratio was 50:50 throughout.
The terminal choice-ratios, predicted and obtained, show very good agreement.
To see how well the theory could deal with
performance in somewhat more complex problems, we turned from simple color or odor
learning to learning about color-odor com-
CHOICE OF A
1.0
Obtained
Predicted
0.8
0.6
0.4
0.2
0
0
4
8
12
16
20
VISITS
24
28
pounds (Couvillon & Bitterman, 1988). In experiments with two colors, A and B, and two
odors, X and Y, honeybees were trained to discriminate between a compound and its components, the compound reinforced in some cases
(AX⫹/A⫺/X⫺) and the separate components
reinforced in others (A⫹/X⫹/AX⫺). When the
results made it clear that, even for these animals, a compound is more than the sum of its
components, we considered the simplest alternative that had yet been suggested in the vertebrate literature—the Rescorla–Wagner assumption that the components of a compound stimulus interact to generate a new, compoundunique component, Q, which functions like any
other component. With an intermediate value
for the salience of Q, the results of the entire set
of compounding experiments could be simulated readily. Consider an experiment on conditional discrimination in which the task was, for
example, to choose A in the presence of X, but
B in the presence of Y: As Figure 4 shows, the
performance of the animals in this difficult
problem was captured rather nicely.
Although the theory provides a rigorous
quantitative account of a now quite substantial
set of data, it is still rather simple. For one thing,
the components of a compound, including the
compound-unique component, do not compete
for associative strength, but gain and lose it
independently with reinforcement and nonreinforcement. Furthermore, there is no inhibitory
assumption; the effect of nonreinforcement is
simply to reduce associative strength, which
never falls below zero. Now, however, there are
some data for honeybees that do suggest an
inhibitory process (Couvillon, Bumanglag, &
Bitterman, 2003), as well as some sort of competition for attention (Shapiro & Bitterman,
1998), data that point inescapably to the inadequacy of the theory, which must either be elaborated or abandoned. We would be happy for
colleagues in other laboratories with facilities
for working with honeybees to join in the effort,
and we would be glad to provide any information that would promote the growth of a common pool both of data and ideas.
32
Figure 2. Obtained and predicted proportions of honeybees choosing A, the originally reinforced of two odors, on
each trial of a four-trial reversal problem (Couvillon &
Bitterman, 1986).
A New Model Animal
It is only the strategy represented by this
work, however, that I (along with Hull and
Hilgard) am recommending again to the larger
1.0
0.9
Maximizing
Best Fit
0.8
Matching
0.7
371
1.0
CHOICE RATIO
PROBABILITY OF CHOICE
CLASSICAL CONDITIONING SINCE PAVLOV
0.6
0.8
Obtained
Predicted
0.6
0.4
0.2
0.5
0.0
0.5 0.6 0.7 0.8 0.9 1.0
0.0 0.2 0.4 0.6 0.8 1.0
RELATIVE ASSOCIATIVE STRENGTH REINFORCEMENT RATIO
Figure 3. (Left) Three functions relating probability of choice to relative associative
strength: maximizing, matching, and the (intermediate) function of best fit to the data of all
previous choice experiments with honeybees. (Right) Obtained and predicted relations between asymptotic choice ratio and reinforcement ratio in an array of choice problems; the
straight line of best fit to the obtained data also is shown (Fischer et al., 1993).
world, and not the honeybee as a model animal.
Among its other limitations, the foraging
honeybee is a hive-bound creature that must be
depended on to come to the laboratory of its
own accord and to expose itself to the stimuli.
In proboscis-extension conditioning with harnessed foragers, there is better stimulus control,
but the technique is of limited value, because
extension of the proboscis is evoked only by a
very narrow range of conditioned stimuli, and
because one must be concerned about the wellbeing of subjects restrained for too long a time.
In any case, we certainly will want to continue
to study learning in vertebrates. Given the long
history of work with rats and pigeons, it might
seem reasonable to begin with one of those
animals, but since we would be starting all over
again with newly standardized techniques to
build a common body of data, my own prefer-
CORRECT CHOICE
1.0
0.8
0.6
0.4
Obtained
Simulated
0.2
0
0
4
8
12
16
20
VISITS
24
28
32
Figure 4. Obtained and simulated proportions of honeybees choosing correctly on each trial of a conditional color–
odor problem (Couvillon & Bitterman, 1988).
ence would be to begin with a fish (Powers,
1989). Small freshwater fish are abundant, relatively inexpensive, and live well in the laboratory. We already have for them several excellent conditioning techniques that compare favorably indeed with those commonly used for
rats and pigeons—techniques that are readily
standardized and fully automated, that require
little advance preparation of the subjects, and
that provide reliable, trial-by-trial measures of
responding to a wide range of stimuli.
A good deal of work has already been conducted with a modern version of a simple appetitive technique conceived long ago for the
study of discriminative learning in fish (Washburn & Bentley, 1906). As illustrated in Figure
5, a plastic disk in the training tank has at its
center a nipple through which thickened liquid
food can be pumped. The disk is suspended
from a thin rod running to a strain gauge so that
contacts with it can be recorded, and stimuli
paired with food pumped through the nipple
soon begin to produce anticipatory responses to
the nipple. Consider some sample results for a
group of small goldfish in an experiment on the
generalization of excitation (Tennant & Bitterman, 1975). After reinforced trials with a tone
of either 200 or 800 Hz, nonreinforced tests
with tones of the original and several different
frequencies yielded the generalization gradient
plotted in Figure 6.
Useful also is a modern version of a technique, pioneered by Froloff (1925), that is
sketched in Figure 7. The CS is a light or a tone,
372
BITTERMAN
Figure 5. Apparatus for the study of appetitive conditioning in fish. Liquid food is pumped through the nipple in the
center of a disk mounted on a thin rod. Strikes at the disk in
response to the food, or to stimuli paired with food, are
recorded.
kin, & Bitterman, 1963). For one group, the
pairing was consistent; for a second group, the
US occurred on only half the trials. When reinforcement was terminated altogether, responding in both groups declined, more precipitously
in the consistently reinforced group—the familiar partial reinforcement extinction effect.
The shuttlebox, designed originally for the
study of avoidance conditioning in rats
(Warner, 1932), is suitable also for the study of
classical conditioning in fish, as well as of instrumental conditioning and the relation be-
MEAN RESPONSE MAGNITUDE
RELATIVE RESPONSE FREQUENCY
and the US is a brief shock that activates the
animal, which is confined in a small chamber.
The activity, both conditioned and unconditioned, is reflected by movement in the water of
a paddle linked to a strain gauge, and a simple
integrating circuit provides a measure of response-magnitude. The results for two groups
of small Tilapia, trained in daily 10-trial sessions with light as the CS and a 10-s CS-US
interval, are shown in Figure 8 (Gonzalez, Es-
Figure 7. Apparatus for the study of aversive conditioning
in fish. General activity produced by brief shock, or by
stimuli paired with the shock, generates water currents that
move the paddle.
35
30
25
20
15
0
1
2
3
200-HZ STEPS FROM
TRAINING STIMULUS
Figure 6. Auditory frequency generalization in a group of
goldfish after classical appetitive conditioning with a tone as
the CS and food as the US (Tennant & Bitterman, 1975).
70
Consistent
60
50
40
30
Partial
20
10
0
0
5
10 15 20
SESSIONS
25
Figure 8. Performance in the activity-conditioning situation of two groups of Tilapia extinguished after partial
versus consistent reinforcement with light as the CS and
brief shock as the US (Gonzalez, Eskin, & Bitterman,
1963).
CLASSICAL CONDITIONING SINCE PAVLOV
373
Figure 9. A shuttlebox for fish. Shuttling responses generated by shock, or by stimuli paired
with shock, are detected by the photocells.
PROBABILITY OF RESPONSE
tween the two. A shuttlebox for fish is diagrammed in Figure 9. Plotted in Figure 10 are
the results for a group of small goldfish classically trained with general illumination as the CS
and brief, unavoidable shock as the US (Woodard & Bitterman, 1971). Two colors were
used— one reinforced and the other not (a standard control for sensitization)—and differential
responding to them soon appeared. I should
emphasize that the procedure was purely clas-
1.0
CS+
0.8
0.6
CS-
0.4
0.2
0
0
5
10
15
SESSIONS
20
Figure 10. Differential classical conditioning of a group
of goldfish in a shuttlebox with colored lights as stimuli
(CS⫹ and CS⫺) and brief (unavoidable) shock as the US
(Woodard & Bitterman, 1971).
sical, the structure of the apparatus serving to
channel the conditioned activity generated by
the CS. Some promising first steps in the development of a quantitative two-process theory of
shuttlebox conditioning in goldfish have already
been taken (Zhuikov, Couvillon, & Bitterman,
1994).
I began a long time ago to develop these
conditioning techniques for fish in the belief
that it would be useful to have some systematic comparative information about learning
in older vertebrate lines (Bitterman, Wodinsky, & Candland, 1958), and you might wonder why I have not continued to do much
work with them. The answer is simple: The
funding ran out. At a time when the influence
of the ethologists loomed large and there was
a lot of loose talk about “evolutionary constraints” on learning, the funding agencies
were willing to support an inquiry into the
possibility that learning in fish might be different in some fundamental respects from
learning in the birds (pigeons) and mammals
(rats) with which most people were working,
but their interest waned as little indisputable
evidence of differences in learning, qua learning, turned up (Bitterman, 1975). The fact, for
example, that the choice ratios of several species of fish in experiments on probability
learning tended closely to approximate the
reinforcement ratios, whereas rats either max-
374
BITTERMAN
imized or behaved “hypothetically” in the old
Krechevskian manner, could be taken to reflect a difference only in the use rather than in
the acquisition of information; the fact that
large and sudden decrement in reward magnitude did not produce in goldfish the dramatic disruption of performance found in rats
could be put down to a difference in temperament.
It was then, with even Konrad Lorenz himself joining the ranks of the general-process
learning theorists (Lorenz, 1977), that P. A.
Couvillon and I turned to the study of learning in honeybees (Couvillon & Bitterman,
1980), a still more divergent species. For that
work, paradoxically enough, we have found
continuing support precisely because of many
unexpected and quite detailed similarities in
the learning of honeybees and vertebrates
(Bitterman, 1988, 1996) that have captured
the interest of funding agencies. Now, perhaps, it may be worth thinking again about
work with fish. The conditioning techniques
that I have described can be readily adapted
for the zebra fish, which would be a wonderful model animal. Support for work with it
should not be difficult to find, given that it is
now, and probably for a long time will continue to be, the darling of the biomedical
research community, which can be counted on
to welcome the results; nor is it unreasonable
to suppose that success with the zebra fish
would attract support for equally sophisticated comparative work with judiciously selected vertebrates of other classes. The central
question (Hull, 1945) would be whether the
equations that describe learning in the various
animals are of the same form, differing only
in their constants—as, in Hull’s memorable
example, the gravitational constant at Hammerfest and Madras— or whether the equations themselves are different.
In concluding, I should say that I regret not
being able to bring you a happier message about
progress to date in work on classical conditioning, but only the sort of message that the messenger is likely to be shot for bringing. It may
be of some comfort to those of us involved in
that work to know that we have not been alone
in our floundering. In a review of research on
human memory and verbal learning, Tulving
and Madigan (1970) concluded that, at least as
of then, “nothing very much [had] changed over
the past hundred years,” or even since Aristotle,
“in the understanding of how people learn and
remember things” (pp. 476 – 477), although
Tulving now tells me that there has been some
significant progress at least in the understanding
of episodic memory. In any case, recognition of
shortcomings is the first step on the road to
improvement, and there does seem to be a way
forward for us, as Hull and Hilgard advised, that
promises more rapid advance than in the past. I
would like to think that, together, we will begin
at last to act on that advice.
References
Bitterman, M. E. (1975). The comparative analysis of
learning. Science, 188, 699 –709.
Bitterman, M. E. (1988). Vertebrate-invertebrate
comparisons. In H. J. Jerison & I. Jerison (Eds.),
Intelligence and evolutionary biology (pp. 251–
276). Berlin: Springer.
Bitterman, M. E. (1996). Comparative analysis of
learning in honeybees. Animal Learning & Behavior, 24, 123–141.
Bitterman, M. E., Menzel, R., Fietz, A., & Schäfer, S.
(1983). Classical conditioning of proboscis extension in honeybees (Apis mellifera). Journal of
Comparative Psychology, 97, 107–119.
Bitterman, M. E., Wodinsky, J., & Candland, D. K.
(1958). Some comparative psychology. American
Journal of Psychology, 71, 94 –110.
Blaser, R. E., Couvillon, P. A., & Bitterman, M. E.
(2006). Blocking and pseudoblocking in honeybees. Quarterly Journal of Experimental Psychology, 59, 68 –76.
Brandon, S. E. (1981). Key-light-specific associations and factors determining key pecking in noncontingent schedules. Journal of Experimental
Psychology: Animal Behavior Processes, 7, 348 –
361.
Brandon, S. E., Satake, N., & Bitterman, M. E.
(1982). Performance of goldfish trained on multiple schedules of response-independent reinforcement. Journal of Comparative & Physiological
Psychology, 96, 467– 475.
Brogden, W. J. (1939). Sensory pre-conditioning.
Journal of Experimental Psychology, 25, 323–332.
Bush, R. R., & Mosteller, F. (1951). A mathematical
model for simple learning. Psychological Review, 58, 313–323.
Couvillon, P. A., & Bitterman, M. E. (1980). Some
phenomena of associative learning in honeybees.
Journal of Comparative and Physiological Psychology, 94, 878 – 885.
CLASSICAL CONDITIONING SINCE PAVLOV
Couvillon, P. A., & Bitterman, M. E. (1985). Analysis of choice in honeybees. Animal Learning &
Behavior, 13, 246 –252.
Couvillon, P. A., & Bitterman, M. E. (1986). Performance of honeybees in reversal and ambiguouscue problems. Animal Learning & Behavior, 14,
225–231.
Couvillon, P. A., & Bitterman, M. E. (1988). Compound-component and conditional discrimination
of colors and odors by honeybees: Further tests of
a continuity model. Animal Learning & Behavior, 16, 67–74.
Couvillon, P. A., Bumanglag, A., & Bitterman, M. E.
(2003). Inhibitory conditioning in honeybees.
Quarterly Journal of Experimental Psychology,
56B, 359 –370.
Fischer, M. E., Couvillon. P. A., & Bitterman, M. E.
(1993). Choice in honeybees as a function of the
probability of reward. Animal Learning & Behavior, 21, 187–195.
Froloff, G. P. (1925). Bedingte Refelexe bei Fischen
[Conditioned reflexes in fish]. Pflügers Archiv,
208, 261–271.
Gonzalez, R. C., Eskin, R., & Bitterman, M. E.
(1963). Further experiments on partial reinforcement in the fish. American Journal of Psychology, 76, 366 –375.
Gray, J. A. (1979). Ivan Pavlov. New York: Viking
Press.
Hilgard, E. R. (1931). Conditioned eyelid responses
to a light stimulus based on the reflex wink to
sound. Psychological Monographs, 41, No. 84.
Hilgard, E. R. (1948). Theories of learning. New
York: Appleton-Century-Crofts.
Hilgard, E. R. (1956). Theories of learning (2nd ed.).
New York: Appleton-Century-Crofts.
Hilgard, E. R., & Marquis, D. G. (1940). Conditioning and learning. New York: Appleton-Century.
Hull, C. L. (1935). The conflicting psychologies of
learning—A way out. Psychological Review, 42,
491–516.
Hull, C. L. (1945). The place of innate individual and
species differences in a natural-science theory of
behavior. Psychological Review, 52, 55– 60.
Kamin, L. J. (1969). Predictability, surprise, attention
and conditioning. In B. A. Campbell & R. M.
Church (Eds.), Punishment and aversive behavior
(pp. 279 –296). New York: Appleton-CenturyCrofts.
Konorski, J. (1948). Conditioned reflexes and neuron
organization. Cambridge, England: Cambridge
University Press.
Liddell, H. S., James, W. T., & Anderson, O. D.
(1934). The comparative physiology of the conditioned motor reflex: Based on experiments with
pig, dog, sheep, goat, and rabbit. Comparative
Psychology Monographs, 11, No. 51.
375
Lorenz, K. (1977). Behind the mirror. New York:
Harcourt Brace Jovanovich.
Miller, R. R., Barnet, R. C., & Grahame, N. J. (1995).
Assessment of the Rescorla–Wagner model.
(1995). Psychological Bulletin, 117, 363–386.
Myers, K. M., Vogel, E. H., Shin, J., & Wagner,
A. R. (2001). A comparison of the Rescorla–
Wagner and Pearce models in a negative patterning and a summation problem. Animal Learning &
Behavior, 29, 36 – 45.
Papini, M. R., & Bitterman, M. E. (1990). The role of
contingency in classical conditioning. Psychological Review, 97, 396 – 403.
Pavlov, I. P. (1927). Conditioned reflexes. Oxford,
England: Oxford University Press.
Pavlov, I. P. (1967). Physiology of digestion. In Nobel lectures, physiology or medicine 1901–1921.
Amsterdam, the Netherlands: Elsevier. Retrieved
from http://nobelprize.org/medicine/laureates/1904/
pavlov-lecture.html (Original work published in
1904)
Pearce, J. M. (2002). Evaluation and development of
a connectionist theory of configural learning. Animal Learning and Behavior, 30, 73–95.
Pearce, J. M., Aydin, A., & Redhead, E. S. (1997).
Configural analysis of summation in autoshaping.
Journal of Experimental Psychology: Animal Behavior Processes, 23, 84 –94.
Pearce, J. M., & Bouton, M. E. (2001). Theories of
associative learning in animals. Annual Review of
Psychology, 52, 111–139.
Powers, D. A. (1989). Fish as model systems. Science, 246, 352–358.
Rescorla, R. A. (1988). Pavlovian conditioning: It’s
not what you think it is. American Psychologist, 43, 151–160.
Rescorla, R. A., & Wagner, A. R. (1972). A theory of
classical conditioning: Variation in the effectiveness of reinforcement and nonreinforcement. In
A. H. Black & W. F. Prokasy (Eds.), Classical
conditioning: II. Current research and theory (pp.
64 –99). New York: Appleton-Century-Crofts.
Shapiro, M. S., & Bitterman, M. E. (1998). Intramodal competition for attention in honeybees.
Psychonomic Bulletin & Review, 5, 334 –338.
Sheffield, F. D. (1965). Relation between classical conditioning and instrumental learning. In W. F. Prokasy
(Ed.), Classical conditioning: A symposium (pp.
302–322). New York: Appleton-Century-Crofts.
Sutherland, N. S., & Mackintosh, N. J. (1971). Mechanisms of animal discrimination learning. New
York: Academic Press.
Tennant, W. A., & Bitterman, M. E. (1975). Extradimensional transfer in the discriminative learning of
goldfish. Animal Learning & Behavior, 3, 201–
204.
Tulving, E., & Madigan, S. A. (1970). Memory and verbal
learning. Annual Review of Psychology, 10, 437–477.
376
BITTERMAN
Wagner, A. R. (2003). Context-sensitive elemental
theory. Quarterly Journal of Experimental Psychology B, 56, 7–29.
Warner, L. H. (1932). The association span of the
white rat. Journal of Genetic Psychology, 41, 57–
90.
Washburn, M. F., & Bentley, I. M. (1906). The
establishment of an association involving colordiscrimination in the creek chub. Journal of Comparative Neurology, 16, 113–125.
Wasserman, E. A., & Miller, R. R. (1997). What’s
elementary about associative learning? Annual Review of Psychology, 48, 575– 607.
Woodard, W. T., & Bitterman, M. E. (1971). Classical conditioning of goldfish in the shuttlebox. Behavior Research Methods & Instrumentation, 3,
193–194.
Zhuikov, A. Y., Couvillon, P. A., & Bitterman, M. E.
(1994). A quantitative two-process analysis of
avoidance conditioning in goldfish. Journal of Experimental Psychology: Animal Behavior Processes, 19, 342–352.
Received November 24, 2005
Revision received February 17, 2006
Accepted March 15, 2006 䡲