Download Eksperimentsituasjonen - Psykologi bachelor, UiT&#248

The partial reinforcement extinction
effect (PREE)
Frode Svartdal
University of Tromsø
Extinction: Basics


Extinction is defined in terms of a
reinforcement process
Extinction contingencies



The stimulus (SR or US) is discontinued
The learning contingency is discontinued
Extinction process


The conditioned response is reduced (strength,
frequency, etc.)
Relearning, … not forgetting
Extinction: Basics
Operant conditioning
Catania, 1984)
Extinction: Basics
Classical conditioning
Factors affecting the extinction rate


In general: Fast acquisition / high rate of
responding  fast extinction
Amount of reward


Variability




High  fast extinction
Stimulus
Response
Reinforcement
= high ext. persistence
Some forms of learning do not extinguish
(easily)

Evaluative conditioning (e.g., Diaz, Ruiz, & Beyens, 2005)
Factors affecting the extinction rate

Partial Reinforcement Extinction Effect


Partial (Intermittent) Reinforcement (PRF)
 increased extinction response
Continuous Reinforcement (CRF)
 reduced extinction persistence
First demonstrations
Operant conditioning;
free operant; rats;
Skinner (1938)
50%
100%
Classical conditioning;
blink response; students;
Humphreys (1939)
Free operant
Ferster & Culbertson, 1975
Free operant
Compared to CRF:
PRF 
• higher asymptotes
• more persistent
responding under
extinction
EXTINCTION
PRF
CRF
Rats, maze running speed under extinction
(Weinstock, 1954)
PRF (30%)
CRF
Classical conditioning (rats): PREE
Extinction
25%
PRF response rate
LOWER than CRF
response rate
50%
100%
15%
Classical conditioning; eyelid; human subjects
(Svartdal & Flaten, in prep.)
Operant conditioning; humans;
Svartdal, 2003, Exp. 4
Conclusions (… preliminary)

PREE is a very robust outcome

Measures & species






Bar pressing, rats
Maze running, rats
Pecking, pigeons
Blink reflex, humans, rabbits
…
Contingency

Operant/instrumental



Discrete trial
Free operant
Classical
But…

How general is the PREE?



Alternative methods of analysis


Reversed PREE observed under some conditions
Generalized PREE observed under some conditions
Nevin (1988): ”PREE is an artefact because of
wrong method of analzing extinction performance”
Response unit issue

PREE or not dependig on how the response is
defined (Mowrer & Jones, 1945!
Reversed PREE
What happens if the subject
is exposed to a mixture
of PRF and CRF contingencies?
Reversed PREE
Pavlik & Carlton, 1965: Rats; bar
pressing, free operant



Gr. 1: Single contingency; CRF
Gr. 2: Single contingency; PRF
Gr. 3: Two signalled schedules
alternated for the same subjects; CRF
+ PRF
Reversed PREE
Conventional
PREE
Reversed PREE
Reversed
PREE
Reversed PREE
Pavlik & Carlton (1965):


Single reinforcement schedules (CRF vs. PRF) in betweengroups experiments  PREE
Two schedules (CRF vs. PRF) for the same subjects 
Reversed PREE
Other research



Reversed PREE observed
Generalized PREE (overall increased persistence, but no
difference between conditions)
Conventional PREE rarely if ever observed in within-subjects
manipulations of CRF - PRF
PREE as a generalization:
Ecological validity
If applied to a situation
with a very specific
schecule for a specific
behavior  PREE
Example:
Single mother – child is
begging for toys
only from mom
If applied to various
situations with mixed
contingencies 
Reversed PREE
Generalized PREE
Example:
Mother and father –
child begs for toys
from both
PREE as a generalization

Relevance to ADHD


Complex schedules  RPREE or GPREE,
not PREE
Factors associated with slower learning 
slower extinction


Attentional problems, difficulties with
concentration, memory, …
Would add to biological factors
Response unit issue
Free operant responding: What is the response unit?
Mowrer & Jones,1945:
What should be counted as the response
unit - single responses or the unit of
responses required for reinforcement?


Free-operant
Intermittent reinforcemet, e.g., FR4
Response unit
FR4
Reinforced responses
PREE
Total responses
Reversed PREE
Total responses /
reinforcement ratio
Nevin: PREE is an artefact
PREE: Alternative analyses
Nevin, 1988: Behavioral momentum
• ”RPREE” is the rule – the response is stronger
following CRF
•
•
•
in free-operant responding (but not in discrete-trial
experiments)
following extended training
Extinction performance
•
•
Traditional measure: Number of responses
Nevin: Slope of the extinction curve
SHORT
LONG
Nevin, 1988
Absolute number
of responses
PREE
Relative to initial
ext response
level
RPREE
PREE vs. RPREE – important variables

Dependent measure


Type of situations


Free operant vs. discrete trial
Complexity of situation


No. of responses vs. relative change
One vs. more schedules (e.g., multiple schedule)
Design

Between groups vs. within subjects
PREE typically observed
Measure
Number of responses
Situation
Discrete trial
Schedule
Single
Design
Between-groups manipulation of
reinforcer rate
Other
CRF schedule must be 100%
PREE: My interests


Interaction PREE & Reversed PREE
Cognition (verbalization) related to
behavioral PREE
The experimental situation
”Computer responses”
presented
Left, right
Subject responses
recorded
Left, right
The experimental situation
Task

Complete a four-response chain of responses started by the
computer


”Obtain as many correct answers as you can.”
Rules (depending on experiment)



Subject: R L
Instructed task: Identify and apply the functional rule(s)


E.g.: Computer: L R
”Repeat computer sequence”
”Reverse computer sequence”
Feedback (visual, autitory) for correct answer; nothing happens
if answer is incorrect
The experimental situation
Manipulations (between
groups and/or within groups)
Rule
Reverse (typically used)
Repeat
Contingency
CRF (100%)
PRF (20-60%)
The experimental situation

Reward rate manipulated



Between groups
Within subjects (multiple schedule)
Discrete trial situation; fixed
number of trials


180 acquisition trials
40 extinction trials
Conventinal PREE; operant responding;
students; Svartdal, 2003, Exp. 4
Reversed & conventional PREE; operant
responding; students; Svartdal, 2000
Reversed PREE



Purpose: Explore the relationship
between PREE and RPREE
PREE vs. RPREE: Contradiction or
compatible effects?
Method


Independent groups: PRF and CRF
Within: CRF and PRF
Svartdal, 2000 ctd.


Multiple schedule, alternating
Group 40/40



PRF
Group 80/80



Half trials (signalled): 40%
Half trials (signalled): 40%
Half trials (signalled): 80%
Half trials (signalled): 80%
”CRF”
Group 80/40


Half trials (signalled): 80%
Half trials (signalled): 40%
”CRF” + PRF
* No. of responses: RPREE
* Relative change: No difference
PREE
80%
40%
Svartdal, 2000 ctd.


Relationship between schedule components
Simplest assumption: Modulation between
component schedules:
 60% + context = 60%  reference
 60% + context = 100%  reduced
persistence
 60% + context = 20%  increaced
persistence
Performance of a 60% schedule depending on
other schedule = 100%, 60%, or 20%
Svartdal, 2000
Svartdal, F. (2000). Persistence during extinction:
Conventional and Reversed PREE under multiple
schedules. Learning and Motivation, 31, 21-40.
Cognition in PREE
•
Currently: Strong cognitive arguments to interpret conditioning
in terms of cognition
•
•
•
•
•
Classical conditioning: Lovibond & Shanks, 2002
Operant conditioning: Shanks & St John, 1994
Implicit learning doubted: Shanks, 2005
Extinction: Lovibond, 2004
Basic argument:
CONTINGENCY  CONSCIOUS APPREHENSION  BEHAVIORAL CHANGE
CONTINGENCY  CONSCIOUS APPREHENSION  NO BEHAVIORAL CHANGE
•
Large number of studies supporting this assumption
Cognition in PREE


So, since the behvioral PREE is very
robust, a ”cognitive PREE” must be easy
to measure
Basic prosedure:
 Behavioral acquisition under 100%
vs. 60% reinforcer rate
 Measurement of verbalized PREE
Cognition in PREE
Prediction of persistence:
”How likely is it that you will continue
responding if reward no longer appears?”
Several experiments have
demonstrated no sensitivity
to learning history in
predictions
3 extinction trials;
immediate behavioral
sensitivity
No difference
in predictions
1,0
0,9
0,8
0,7
0,6
100/100
60/60
0,5
0,4
0,3
Responses (proportion of possible)
0,2
0,1
0,0
Trl1
Trl3
Trl5
Trl7
Trl9
Trl11
Trl13
Trl15
Trl17
TExt
Svartdal & Silvera, in prep.
Cognition in PREE
Retrospective judgments:
”How many responses did you emit after
reward no longer appeared?”
Subjects are very
accurate in descrbing
their own behavior,
including their own
extinction persistence
Cognition in PREE
Svartdal, F. (2003). Extinction after partial
reinforcement: Predicted vs. judged persistence.
Scandinavian Journal of Psychology, 44, 55-64.
Meta-cognitive PREE?


We all have long experience with
various contingencies
Maybe a ”meta-cognition” evolves:


Uncertain outcomes  Persist
Certain outcomes  Quit
Meta-cognitive PREE?

Scenarioes presented to subjects,
manipulation



Reliable outcome vs.
Unreliable outcome
Persistence judgments of behavior
Meta-cognitive PREE?
Naive students: No effect of
outcome manipulation
Meta-cognitive PREE?
Psychology students
Naive students
(have read about PREE)
Meta-cognitive PREE?
Svartdal, F. (2000). Persistence during extinction:
Are judgments of persistence affected by contingency
information? Scandinavian Journal of Psychology,
41, 315-328.
PREE: Theory
Mowrer & Jones: Diskriminasjonshypotesen

PRF:



Læringbetingelsene  ekstinksjonsbetingelsene
Generalisering til ekstinksjon
CRF:


Læringbetingelsene # ekstinksjonsbetingelsene
Liten generalisering til ekstinksjon
PREE: Theory
Amsel: Frustrasjonshypotesen

PRF:





Forventning om belønning  frustrasjon når belønning
uteblir
Frustrasjons-cues assosieres med læringssituasjonen
Under ekstinksjon: Frustrasjon pga uteblitt belønning
Læringssituasjonen  ekstinksjonssituasjonen
CRF:


Frustrasjon oppstår ikke under læring
Læringssituasjonen # ekstinksjonssituasjonen
PREE: Theory
Capaldi: Sequential hypothesis

PRF:




Ikke-belønnede trials blir signal på at belønning snart vil
følge: … N N N R N N N R …
Dvs.: Det opparbeides en forventning om belønning når
belønning uteblir
Under ekstinksjon: Mange responser pga forventning om
belønning
CRF:


Ingen erfaring med uteblitt belønning under læring
Under ekstinksjon: Få responser
PREE: Theory

Status:





Discrete-trial-situasjonen


Diskriminasjonshypotesen står svakt
Amsels hypotese står rimelig sterkt
Capaldis hypotese står ganske sterkt
Nevins modell: Ingen hypotese i vanlig forstand
Capaldi og Amsel dominerende
Fri-operant-situasjonen

Svak teoretisk forståelse
Evaluative conditioning

Neutral stimulus



ζ
Neutral stimulus paired with US


E.g.,
E.g.,
ζ
+
MURDER
Extinction

ζ alone
Test

ζ is evaluated in accordance with MURDER