Download 5 - smw15.org

PSY 445: Learning & Memory
Chapter 5:
Instrumental Conditioning:
Nonreward, Punishment, & Avoidance



Nonreward
◦ Extinction
◦ Omission
Punishment
Avoidance learning
◦ Negative Reinforcement
The non-reinforcement of a previously
reinforced response
 The result is the decrease in the strength of
the initially reinforced response
Extinction
Hi
Strength
of R
Lo
Acquisition
R  No S*

A selected response prevents a positive
reinforcer from occurring
Consists of the removal of a stimulus (usually
one that is considered to be pleasant)
following a response that leads to a decrease
in the future strength of that response

R ------------------------------- S

(tease sister)
(no computer games)
Unpleasant Emotional Effects
 Frustration; Aggression
Extinction Burst
 Temporary increase in the nonreinforced behavior
 Alessandri, Sullivan, & Lewis (1990)
Infant mobile 
Spontaneous Recovery
 After a delay the response recovers
See next slide
Unpleasant Emotional Effects
 Frustration; Aggression
Extinction Burst
 Temporary increase in the nonreinforced behavior
 Alessandri, Sullivan, & Lewis (1990)
Infant mobile 
Spontaneous Recovery
 After a delay the response recovers
See next slide
Spontaneous Recovery
Hi
Time
passes
Strength
of R
Lo
Acquistion
Extinction
S+ occurs
When old behavior doesn’t work, new behaviors may replace it
Neuringer, Kornell, & Olafs (2001)
Procedure
 Rats trained to press three specific levers (of five present) to get
food pellet reward
 Extinction phase food pellets withheld
Results
 Rats tried new sequence of lever pushing to see if they could get
reward
Interpretation
 Real-world adaptation: If food is not obtained with one behavior,
survival depends on us to be flexible enough to try novel and
creative behavior
 This runs counter to Thorndike’s old interpretation where cats
continued with stereotyped behavior

Extinction is slower if a partial reinforcement
schedule (PRS), rather than a continuous
reinforcement schedule (CRS) was in effect
before extinction
◦ Acquisition: CRS rats ran faster to look for reward
◦ Extinction: PRS rats ran faster to look for reward
Explanations for resistance
Discrimination Hypothesis
 For CRS rats, the onset of extinction is readily
discriminated from the previous continuous schedule
of rewards; something has changed
 For PRS rats, the first extinction trial is not considered
permanent
Frustration Hypothesis
 For PRS rats, the frustrating aftereffects of nonreward
become associated with the subsequent occurrence of
reward
 Frustration experiences on one trial becomes SD for
reward
Sequential Hypothesis
 At the start of a new trial, PRS rats remember
the outcome of the previous trial and
associates it with the outcome of the current
trial


Less trials needed here than with frustration idea
(memory quicker to form than is frustration)
Trials can be spaced further apart as memory
persists longer than frustration

An aversive stimulus is presented immediately after a
behavior in order to decrease the future probability of
that behavior (or strength of this response)
For example: when your dog chases a car

R --------------------------------------- S
(run s into traffic)
(scolded)
Punishing stimuli
 Shock used in most animal studies
Response-contingent punishment
 Be sure specific response is being punished
Intensity
 Strong better than weak; don’t use gradual increases
Delay of punishment
 More effective when applied immediately
Solomon, Turner, & Lessac (1968)
 Dogs punished immediately, after 5 seconds,; after 15
seconds
 Immediate worked best; fifteen second delay was least
effective
Schedule of punishment

Continuous schedule is generally better than partial
schedule; however, spontaneous recovery is more likely
Incompatible responses


Be careful to punish in a way that will lead to the
desired outcome
Unfortunately, at times the punishing stimulus can
elicit behaviors that are incompatible with the desired
outcome
Concurrent reinforcement

The effects of punishment can be neutralized if positive
reinforcement of the inappropriate behavior occurs
along with punishment
Providing a verbal rationale
Less severe punishment is enhanced by including an
explanation
Cheyne, Goyeche, & Walters (1969)
Procedure
 Different types of punishment with or without verbal
rationale were given to children attempting to reach
for a toy
Results
 No rationale most severe punishment is most effect
 With rationale lesser punishment worked significantly
better
See next slide 

Results 
Interpretation
 Strong emotional responses that accompany
punishment may interfere with the ability to attend to
and/or learn from the situation (narrowing of
attention)
Cheyne, Goyeche, & Walters (1969)
Individual differences
 Some people or animals may be less reactive to
punishment than others



Conditioned fear avoidance
◦ The place where punishment takes place or the person
administering the punishment may become CS leading to CR
Aggression
◦ Punishment-aggression hypothesis
 Miller (1948): rats exhibited shock-aggression behavior
 Aggression is reinforced
 Generalized aggression
Paradoxical rewarding effects
◦ The pairing of a punishing stimulus with a positive reinforcer
can convert the punisher into a secondary reinforcer
◦ The punishing stimulus can inadvertently enforce behavior
rather than suppress in it
 Masserman (1943)
Masserman (1943)
Procedure
 Trained cats to lever press for unpleasant blasts of air
 Phase 1: Lever press for reward (food)
 Phase 2: Blasts only occasionally and mild
 Phase 3: Gradually increased air blasts and decreased
food rewards
Results
 Cats would bar press for punishment
Interpretation
 Knowing the history of an organism can explain
irrational behavior

Frustration produced by nonreward can elicit
aggression (similar to punishment)

Does punishment work in everyday life?
Are punished children better behaved?

Methodological issues

 Different types of punishment
 Who decides if the child is misbehaving

Conflicting research
◦ Some say more problems; others say neither benefits
nor deficits

Correlational issues
◦ Not necessarily cause and effect relationship



Resistance to extinction is the extent to
which responding continues after an
extinction procedure has begun
Persistent responding = high resistance to
extinction
Responding stops quickly = low resistance to
extinction



Schedule of reinforcement
◦ Partial during acquisition
Magnitude of the reinforcer
◦ Smaller during acquisition
Delay of the reinforcer
◦ Delayed during acquisition
Generalized Persistence
 In certain cases, responding is resilient
 Can carry over from one situation to another
Eisenberger et al. (1979)
Procedure
 Depressed patients asked to perform chores
 Phase 1: Gratitude was either on continuous or partial
reinforcement schedule
 Phase 2: New person requests favor and shows gratitude on
partial reinforcement schedule
Results
 Patients spent more time on a chore (punching computer cards) if
initially on partial reinforcement schedule
Interpretation
 Persistence generalized beyond initial task to another task and
from another person
An instrumental response escapes or prevents
an aversive stimulus
 Often referred to as negative reinforcement
because the instrumental response increases
in frequency

Two types of Negative Reinforcement
◦ Escape Conditioning
 When a behavior has terminated an aversive
stimulus
◦ Avoidance Conditioning
 When a behavior has prevented an aversive
stimulus
Typical procedure


Learning task begins with warning signal which is
followed by shock
On succeeding trials, selected instrumental response
prevents aversive stimulus and terminates both the
warning signal and trial
Shuttle Box 


Active Avoidance
◦ Active performance of a certain response
prevent the aversive stimulus
Passive Avoidance
◦ Withholding a response prevents the
aversive outcome



Conditioning Theory
◦ Watson-Mowrer Two-Process Theory
Cognitive Theory
Functional Approach
Watson-Mowrer Two-Process Theory (Mowrer , 1947)
 Avoidance learning involved two processes - (1) classical
conditioning and (2) instrumental conditioning
Process 1:
 Dangerous, painful, aversive stimuli (US) causes an innate fear
response (UR)
 Other stimuli present at the time get associated with fear through
classical conditioning.
 When these other stimuli (CSs) are encountered again, they evoke
a fear response (CR)
Process 2:
 The presence of fear and all of its visceral effects is aversive
 Any response that removes these fear-evoking stimuli will be
negatively reinforced
Solomon & Wynne (1953)
 Illustrates Two-Process Theory

Shuttle box consists of:
◦ 2 chambers separated by a barrier several inches high
◦ Could freely move between chambers
◦ Separate lights
◦ Metal floor that could be electrified to deliver shock
Solomon & Wynne (1953)
Procedure
 Experimental Session – 10 trials each
 Dog placed in one compartment of a shuttle box
 After a time, light in the compartment that dog was in went off,
while other light stayed on
 10 seconds later, the dog received a shock through the floor until
jumped over barrier
 Measured response latency
Results
 Escape first: First few responses were usually longer than 10
seconds. So dog was getting shocked and then escaping.
 Avoidance response: By roughly the 5th trial, dogs response
latency was less than 10 seconds. So dog never received shock in
these cases.
Interpretation
 Both processes are involved
Limitations
 Apparently endless persistence in avoidance
behavior
◦ Fear to warnings never seem to go away
◦ Examples: people with phobias
 Seems to fit better with classical conditioning
ideas
Eliminating avoidance behavior requires
modifying two expectations:
 Response - expect to avoid aversive outcome
 No response – expect to get shocked
Subject must learn that the warning stimulus no
longer signals danger
Species-Specific Defense Responses (SSDRs)
 Studying the natural behaviors of the
organism’s reactions to threats in the wild
 Innate responses that are primed in a fearful or
threatening situation
 These behaviors are witnessed in lab setting
(for example, mice in fearful situations will
“freeze”)
When a behavior has two opposing outcomes
reward and punishment what is the result?
 Relative intensity
 Proximity to the consequence (physically or
temporally)
Approach Coping
 Confronting the stressor; problem-focused
coping
Avoidance Coping
 Shutting down and trying to deny or suppress
the unpleasant thoughts and feelings;
repressive coping
 Distraction can be an adaptive form of
avoidance coping
Averill & Rosenn (1972)
 Shock-avoidance experiment
Procedure
 Audiotape with two tracks
 Track 1: Music
 Track 2: Tone of the impending shock
Results
 Some students listened to music exclusively
Interpretation
 Sometimes the anticipation of unpleasantness
is worse than the actual aversive stimulus
Which style is best? It depends…
 Is stressor controllable or not?
 Habituation and counterconditioning process
processes are involved
 Are thoughts intrusive?
 Ironic processes may be involved (Wegner,
1987)
Learning that there is an explicit lack of
contingency between responses and an aversive
outcome
Seligman & Maier (1967)
Procedure
 Phase 1: Dogs receive either: escapable shock (press button with
nose), inescapable shock, no shock
 Phase 2: All placed in shuttle box (two components)
 Light warned of shock on one side
 Escape shock by moving to other side
Results
 Dogs that had been exposed to escapable shock and no shock
quickly learned to escape
 Inescapable shock dogs “quietly whined"
Interpretation
 Inescapable shock dogs developed a perceived loss of control
Hiroto (1974)
Procedure
 College students exposed to loud tone pulses that
were “unpleasant but not harmful”
 Phase 1: Escapable group and inescapable group
 Phase 2: Both groups now could escape tone
Results
 Inescapable group was slow to respond in Phase 2
Interpretation
 Inescapable college students developed a perceived
loss of control
Depression


Symptoms of both helpless and depressed mimic
each other – reductions in activity; loss of motivation;
disruptions in eating, sleeping, and sexual behavior
Assertiveness training can be beneficial to these
individuals
Causal Attribution Theory


This is the idea that helplessness is ultimately
determined by the belief that our behavior is
ineffectual in determining what happens to us
Insofar as their causal attributions are concerned, helpless
individuals have an external locus of control
Classroom Education
 Locus of control is involved here as well
Physical Health
 Correlational findings associate negative
attitudes with physical health problems
Amygdala is involved in classical conditioning
of fear
 Strong evidence from studies on both animals
and humans
Pet Containment Systems
 Invisible Fencing technique applies WatsonMowrer theory of conditioning
 Potential to provoke aggression is a concern
Treatment of Obsessive-Compulsive Disorder
 Stimulus extinction and response prevention
have been used to treat these individuals
 Usually combined with other methods
(cognitive therapies; pharmacological
treatments)
Some slides prepared with the help of the following websites:
 www.radford.edu/~pjackson/ExtinctIC.ppt
 www.public.coe.edu/~mbaker/baker/learning/lectures
 http://pjackson.asp.radford.edu/punishment3.pdf
 http://web.mnstate.edu/malonech/Psy342/Terry%20Notes/Instru
mental%20Cond.%20T4T5.htm