Download lecture 16

Extinction of Conditioned Behavior
Chapter 9
• Effects of Extinction
• Extinction and Original Learning
• What is learned during Extinction
Effects of Extinction
Extinction involves omitting the US or reinforcer
 In Pavlovian conditioning, this means repeated
presentations of the CS alone
 In operant conditioning, this means not giving
the reinforcer after the response
Two main effects of extinction procedures on behavior
 the rate of responding decreases
 response variability increases
Extinction Increases Variability in Behavior
Neuringer, Kornell, & Olufs (2001)
Three levers present and Var group had to vary sequence
All groups increased
variability during
extinction
Extinction and Original Learning
•
•
•
•
Spontaneous Recovery
Renewal
Reinstatement
US-devaluation
Spontaneous Recovery
Rescorla (1996)
Rats trained to make 2 different responses for food
Lever-press – food
Nose-poke – food
At end of training phase, rats were making about 17 resp/min
After acquisition, the rats were given 2 20-min extinction sessions
with each response
For one of the responses, the extinction phase occurred shortly
before the recovery test (R-No rest)
For the other response, the extinction phase occurred 7 days
before the recovery test (R-Rest)
At end of extinction phase, rats were making about 2 resp/min
Spontaneous Recovery
Shows importance of passage of time
Renewal
Bouton & King (1983)
Rats were first trained to bar-press for food
Then, Context A
T – Shock
Conditioning resulted in T
suppressing BP response
During extinction, the rats were divided into 3 groups
Group A: CS alone in Context A
Group B: CS alone in Context B
Group NE: did not receive extinction
After extinction, all rats received test trials in Context A
Renewal
Reinstatement
Could be due to context conditioning
In initial studies, the excitatory conditioning, extinction and
subsequent US exposures were all given in same context
Recent research indicates that context is important but not
because it permits summation
The role of context appears to be to disambiguate the significance
of a stimulus that has a mixed history of conditioning and excitation
Context has little effect on stimuli that have a history of only
conditioning
Reinstatement
Bouton (1984)
Conditioned suppression with rats
Phase 1
CS – weak shock
CS – weak shock
CS – strong shock
CS – strong shock
Phase 2
No treatment
No treatment
Extinction
Extinction
Reinstatement
Shock-same
Shock-different
Shock-same
Shock-different
Test
CS
CS
CS
CS
Reinstatement was introduced when the level of responding was
similar in all subjects
However, some rats got to that point by receiving only conditioning,
whereas for others the CS was both conditioned and extinguished
Reinstatement
Reinstatement
For rats that did not receive extinction, it did not matter whether
the reinstatement shocks were given in the same context or a
different context
- shows that contextual conditioning did not summate with the
suppression elicited by the CS
For rats that did receive extinction, reinstatement shocks given in
the same context produced more suppression than shocks given in
a different context
- shows that contextual conditioning facilitates the reinstatement
effect
According to Bouton, reinstatement US presentations given in the
test context serve to restore the excitatory properties of the
contextual cues and enable those cues to be more effective in
reactivating the memory of excitatory conditioning of the CS
US-Devaluation
Rescorla (1993)
Used devaluation procedure in operant experiment to show that
the original R-O association was not abolished by extinction
Phase 1
R1 – O1
R2 – O2
R3 – O1
R4 – O2
Phase 2
R1 extinguished
R2 extinguished
US-devaluation
O1 – LiCl
Test
R1 vs R2
and
R3 vs R4
The extinguished responses were re-trained with a third reinforcer
before the devaluation procedure to make sure there would be some
responding during the final test
Effect of Devaluation - Operant
Shows that extinction does not eliminate the R-O association
What is learned in extinction?
The experiments on spontaneous recovery, renewal, reinstatement
and US-devaluation all suggest that extinction does not abolish the
original learning
So why does extinction produce a decrease in responding?
Extinction may produce an inhibitory S-R association
The non-reinforcement of a response in the presence of a specific
stimulus produces an inhibitory S-R association that suppresses
that response whenever S is present
Inhibitory S-R Association
Paradoxical Reward Effects
• Overtraining Extinction Effect
• Magnitude of Reinforcement Extinction Effect
• Partial Reinforcement Extinction Effect
Partial Reinforcement Extinction Effect
In addition to behavioral effects of extinction, there are also
emotional effects (i.e. frustration response)
The most important variable that determines the magnitude
of both the behavioral and emotional effects of an extinction
procedure is the schedule of reinforcement that was in effect
before extinction was introduced
The dominant schedule characteristic that determines extinction
effects is whether the response was reinforced every time it
occurred (CRF) or only some of the time it occurred
(partial or intermittent reinforcement)
Partial Reinforcement Extinction Effect
The general finding is that extinction is slower, and involves
fewer frustration reactions, if a partial reinforcement schedule,
rather than CRF, was in effect before extinction — PREE
The PREE was previously thought to be counterintuitive
- if a reward strengthens a response, then CRF should
produce a stronger response and slower extinction
Mechanisms of the Partial Reinforcement
Extinction Effect
• Discrimination hypothesis
• Frustration hypothesis
• Sequential memory hypothesis
Discrimination Hypothesis
• introduction of extinction is easier to detect
after CRF than partial reinforcement
• with CRF, easier to notice when reinforcement
is absent during extinction
• with PRF, not as easy because reinforcement
was sometimes absent during training
Discrimination Hypothesis
While this idea makes intuitive sense, it was discarded a long
time ago.
Jenkins (1962)
2 groups:
Group 1:
Group 2:
Phase 1
CRF
partial
Phase 2
CRF
CRF
Extinction
Extinction
Extinction procedure should be equally discriminable in the
2 groups
However, group 2 was slower to extinguish
These results suggest that the PREE is not just a discrimination
problem but that subjects actually learned something during partial
reinforcement that affects the rate of extinction
Frustration Theory
Early in training, after nonreward, the organism is frustrated
and expects not to be rewarded on the next trial.
But on some trials following nonreward, the organism is
rewarded
Thus the stimuli associated with nonreward (frustration)
become associated with reward, and the organism learns to
respond in their presence – they become SDs for responding.
SF:
R
SAppetitive
With F = frustration
SF is present during extinction, thus the organism
makes many responses.
Sequential Theory
Early in training, when the organism is rewarded on
a trial after one or more nonreward trials, the
organism remembers not being rewarded on those
previous trials.
That memory then becomes associated with reward
on the present trial.
SN:
R
SAppetitive
With N = memory of
nonreward
Thus the animal learns to make the response when the
memory of nonreward, including sequences of
nonreward, is present.
During extinction, the stimulus conditions are always
those of non-reward.
Animals that were on PR schedules should make
many responses in extinction. They have learned to
respond in the presence of stimuli associated with
nonreward (either frustration or memory of
nonreward).
In contrast, animals that were on CRF, should make
few responses during extinction. They have never
experienced nonreward, and never learned to respond
in its presence. They quickly become frustrated and
give up responding.