Download Avoidance

Avoidance
The “Problem of Avoidance”
l
The fact that animals can learn to avoid aversive
stimuli presented a serious challenge to theorists in
the 1940s and 50s.
l
l
l
There is no ongoing shock prior to the response, and no
shock immediately after.
Thus there appeared to be no immediate consequence of
the behavior that could serve to reinforce it. Yet avoidance
behavior was acquired and maintained.
This problem led to a series of attempts to account
theoretically for avoidance behavior, each involving
a new type of avoidance conditioning procedure.
The Shuttlebox
l
l
l
A shuttlebox is a test
chamber divided into two
compartments by a low
hurdle or by a doorway
(pictured).
The grid floor of the
shuttlebox can be electrified
to deliver the aversive
stimulus (shock).
It is also equipped with a
light or speaker for
delivering a warning
stimulus.
1
Two-Way “Shuttlebox”
Avoidance: The Procedure
l
l
l
l
l
In two -way avoidance, the subject (typically a rat) is
placed in one compartment of a shuttlebox.
After a short while, a tone sounds and continues for
20 seconds, at which time the shocker is turned on.
If the rat shuttles to the other compartment during
the shock, this turns off the shock and tone.
If the rat shuttles before the shock, the tone ends
immediately and no shock is delivered.
After a short while, the tone comes on again and the
rat must shuttle back to the first compartment to
terminate the tone or the tone and shock.
Two-Way “Shuttlebox”
Avoidance: Performance
l
l
l
l
Initially the rat waits until the shock begins, then
runs, sooner or later ending up in the other
compartment and ending the shock.
After more experience, we find the rat orienting
toward the hurdle or doorway during the tone, then
running quickly to the other compartment when the
shock commences.
After even more experience, the rat “jumps the gun”
and runs before the shock begins.
Soon the rat is efficiently running to the opposite
compartment each time the tone comes on, thus
avoiding the shock on most occasions.
Accounting For Shuttlebox
Avoidance
l
Mowrer’s two-factor theory attempts to explain
shuttlebox avoidance by appear to two factors:
l
l
l
Classical conditioning: During the early phases, each tone
ends in shock. This classically conditions fear to the tone.
Instrumental conditioning: Now that the tone elicits strong
fear, any response that immediately terminates the tone
will be negatively reinforced (escape).
Thus, the rats are not shuttling during the tone in
order to avoid the shock; they are shuttling in order
to terminate the (now aversive) tone.
2
Sidman Avoidance
l
This procedure, developed by Murray Sidman in the
early 1950s, investigated avoidance in an operant
chamber equipped with a lever:
l
l
l
l
In the absence of lever-pressing, a ½ second shock was
scheduled every 5 seconds (the S-S or shock-shock
interval.
After a lever-press, shock was delivered at the end of a 20second R-S or response-shock interval, if no further
responses occurred, after which the S-S interval was
reinstated.
Each lever-press during the R-S interval reset the R-S
timer to the beginning, postponing the shock.
In this procedure, there is no explicit CS as there is
in two -way shuttlebox avoidance.
Performance on Sidman
Avoidance Schedules
l
l
l
Subjects (typically rats) acquire lever-press
avoidance responding on this schedule and tend to
respond rather steadily at a rate of about one leverpress every second or so.
A few times during each session, the rat may fail to
respond during the R-S interval and then receives a
shock.
Often several minutes elapse after the start of the
session during which even a well -trained animal fails
to respond on the lever. This phenomenon has
been called “warm-up”.
Sidman’s Explanation for Sidman
Avoidance Performance
l
Sidman proposed an explanation the may be
called “differential punishment of other
behavior.” According to this explanation,
l
l
l
Responding on the lever is never followed closely
by shock because of the R-S interval.
Any other ongoing behavior occasionally will be
paired with shock delivery, tending to suppress
those behaviors.
Eventually the only behavior not suppressed is
lever-pressing; as a result it occurs frequently.
3
Anger’s “CATS” Theory of
Avoidance
l
Douglas Anger proposed an alternative to Sidman’s
differential punishment theory:
l
l
l
l
Time-correlated internally -generated stimuli mark the
passage of time during the R-S interval.
Those occurring just before shock delivery (near the end of
the R-S interval) become aversive through classical
conditioning. Anger called these “conditioned aversive
temporal stimuli” or “CATS.”
A lever-press resets the interval and thus replaces highly
aversive end-interval stimuli with less aversive stimuli
marking the beginning of the interval.
Lever-presses are thus negatively reinforced by escape
from CATS. This is essentially Mowrer’stwo-factor theory
applied to Sidman avoidance.
Sidman’s Evidence Against
CATS
l
l
l
Sidman created a variant of his original avoidance
schedule. In this variant, a tone sounded during the
last few seconds of the R-S interval.
The rats now waited for the tone to sound. They
then casually approached the lever, pressing it at
the last moment before the shock would have been
delivered.
This explicit CS seemed to act more as a
discriminative stimulus for responding than as an
elicitor of fear. Although the rats could have kept it
from appearing, they did not.
Herrnstein and Hineline’s
(1966) Avoidance Schedule
l
Herrnstein and Hineline (1966) devised an
avoidance schedule that eliminated internal, timecorrelated stimuli that might function as CATS:
l
l
l
l
Two variable-time (VT) shock programs were arranged to
run simultaneously. In the absence of lever-pressing, a VT
2-minute schedule determined shock delivery.
Each response transferred control over shock delivery to
the second, VT 4-minute schedule.
Each shock-delivery transferred control back to the VT 2minute schedule.
By responding at least once after each shock, the rats
could spend most of the session on the VT 2-minute
schedule, thus avoiding about half of the shocks that they
would have received if they had never responded.
4
Explaining Performance on the
Herrnstein & Hineline Schedule
l
l
l
l
l
Most rats acquired the avoidance response, although only after
long training.
Note that if the VT 2-minute schedule were about to schedule a
shock, a response at that moment would be followed immediately
by shock.
All that the subjects could learn on this schedule is that shocks
tend to occur less often in the presence of responding than in t he
absence of responding.
Herrnstein and Hineline therefore proposed that what reinforces
avoidance responding on this schedule is shock-frequency
reduction .
This explanation refers to the long -term average effect of
responding (a molar explanation) rather than to the immediate
effect of responding (a molecular explanation).
Seligman’s Cognitive Theory
of Avoidance
l
Martin Seligman proposed that, when responding on
traditional avoidance schedules, subjects learn
expectancies:
l
l
l
They learn to expect that a shock will be delivered if they
do not respond.
They learn to expect that a shock will not be delivered if
they do respond.
Subjects prefer not to receive a shock; so they
respond. Essentially, this theory states that the
reinforcer for avoidance responding is the avoidance
of the shock.
Avoidance Responding: The
Problem of Extinction
l
l
l
Early theorists attempted to extinguish
avoidance responding in the shuttlebox by
disconnecting the shocker.
However, subjects continued to respond,
hour after hour. Extinction eventually
occurred, but only after a very long time.
This led some theorists to propose that there
must be something special about avoidance
responding that made it especially resistant
to extinction.
5
The Proper Extinction of
Avoidance
l
l
l
Is disconnecting the shocker the proper extinction
procedure for avoidance responding?
No. In this procedure, responses continue to be
followed by the absence of shock, just as before.
So long as the subject responds reliably, there are
no opportunities to learn that shocks no longer occur
in the absence of responding.
The proper method of extinction would be to deliver
the shocks whether or not the subject responded.
With this procedure, extinction is quite rapid.
The Cognitive Explanation
l
The slow extinction of avoidance responding
following disconnection of the shocker can be
understood readily in terms of Seligman’s cognitive
theory of avoidance:
l
l
Responding continues to be followed by shock absence,
confirming the expectation learned during training.
Failing to respond no longer leads to shock, but because
the subject almost always respond, there are few
opportunities to experience this change and revise the
expectation. Because subjects continue to expect shock if
they do not respond, they continue to respond.
Differences in Ease of
Acquisition
l
Bob Bolles noticed that some forms of
avoidance behavior seemed easier for
subjects to acquire than others:
l
l
l
l
One-way shuttlebox avoidance: Rapidly acquired
Two -way shuttlebox avoidance: Many trials
required
Lever-press avoidance: Many, many trials, if ever;
very difficult to get subjects responding
Bolles developed an explanation for these
differences that appealed to “SSDRs .”
6
“SSDRs”
l
l
Organisms have evolved innate behavior
patterns to deal with various threats. Bolles
termed these behaviors “SSDRs,” or
“species-specific defense reactions .”
In general, SSDRs include:
l
l
l
Fighting (self-defense when attacked)
Fleeing (running, preferable to a place of safety)
Freezing (immobility, when the threat is still
relatively distant or uncertain)
Bolles’ Explanation for
Differences in Acquisition Rate
l
One-Way Avoidance
l
l
Two -Way Avoidance
l
l
Subject has only to move from a place of danger to a place
of safety. This is compatible with the subject’s innate
reaction (fleeing).
Subject has to move from a place of danger to one only
immediately less dangerous. Neither compatible nor
incompatible with fleeing.
Lever-Press Avoidance
l
No place to run or hide, nothing to attack, so subject tends
to freeze. Generally incompatible with lever-pressing
(although subjects sometimes attack the lever).
Is Lever-Press Avoidance
Really Harder to Learn?
l
l
l
My own (limited) experience with lever-press
avoidance (in this case, Sidman avoidance) does
not confirm Bolles’ assertion that this type of
avoidance is difficult for rats to acquire.
In Sidman avoidance, the frequent shocks
programmed under the S-S interval tend to break up
freezing, increasing the chances that the subject will
depress the lever in the course of its activities.
In other situations involving relatively infrequent
shocks (averaging one per two minutes), rats almost
always acquired the required lever-pressing
behavior within a session or two. Freezing tended
to go away after subjects had some experience in
the situation.
7