Download Extinction

Avoidance:
Is negative REINFORCEMENT!
What is avoidance behavior?
• Avoidance contingencies INCREASE the rate of an
operant response
– NOT punishment, but actually negative reinforcement
– Remember, punishment DECREASES the response
• But with both avoidance and punishment the
organism reduces contact with an aversive
stimulus!
– Increasing periods of safety
– Avoidance procedures = ACTIVE avoidance
– Punishment procedures = passive avoidance
Avoidance Tests:
Discriminated Avoidance
• Discriminated Avoidance:
–
–
–
–
Procedure for studying negative reinforcement and avoidance
A response CANCELS a shock
Organism is responding for food reinforcers
When light comes on, must press another lever to AVOID the shock
• IF the response does not occur during the S+ the stimulus is
followed by a shock
• IF the response does occur during the S+, the shock is cancelled
– thus: signal or sD for shock
– if this were an escape: response could also occur DURING the shock
to shut off shock
Shuttle Avoidance
• 1-way shuttle avoidance:
– Animal on one side of a shuttle
– Cue comes on: Must jump to other (safety) side or receive
a shock.
– Placed back in original side for each trial.
• 2-way shuttle avoidance
– Animal on one side of a shuttle
– Cue comes on: Must jump to other (safety) side or receive
a shock.
– But: previously SAFE side then becomes shock side
– Animal must jump back into “shock” side to get away from
shock.
– VERY Difficult for animal to learn.
Theories of Avoidance:
Two Factor theory
• Two things happen during avoidance
conditioning: CC and then OC
– Animal learns to fear S+ via class. conditioning
• CS (light)---> US (shock): UR (fear)
• animal learns to fear light via pairing with shock
– Animal will then learn a response to AVOID shock
and thus remove/lessen their fear
– Thus: not getting shocked reduces fear that was
signaled by the CS
Theories of Avoidance:
Two Factor theory
• Experimental evidence:
– On initial training trials: light/CS produces physiological
symptoms of fear
• Escape response results in decrease in these physiological
symptoms
– On later trials:
• little or no evidence of physiological fear with CS
presentation
• suggests fear has been reduced/replaced by the escape
response
– In sense: forms a negative feedback loop
Problems with 2-factor theory:
• Signs of fear dissipate w/time:
– as animal gets "better" at avoidance response
– thus: no fear to be avoided
• The CS is not as important in avoidance
learning as 2-factor theory states:
– Animals can learn to avoid in a discriminated
avoidance situation long before there is any sign
that they are responding to/detecting the CS
Two Avoidance Procedures:
• Sidman Avoidance:
– The response POSTPONES or DELAYS the shock
– Thus: only temporary solution
– Must keep responding to keep delaying the shock
– Results in lots of responding
– Again: some signal may be used to signal when
must respond
Two Avoidance Procedures:
• Herrnstein and Hineline Procedure:
– The response reduces the rate of the shock
– Does NOT delays or cancel, just slows down rate of
delivery
– The response switches the schedule of shock to a
lower rate
– Note: cannot entirely AVOID shock in this procedure:
• once animal receives shock on lowered schedule, reverts
back to original schedule
• animal must respond again to switch schedule again
Herrnstein and Hineline:
Test of 2-factor theory
• Test of 2 factor theory:
– Two groups of rats used
– Group 1: Can turn off light, but still get shock
– Group 2: Can turn off shock, light still on
• 2-factor theory would predict that Group 1 should
respond more, because this would be cancelling the CS
that produces fear
• Results: group 2 responds much more accurately, faster
Alternative: One-Factor Theory
• Responses occur whenever they reduce the rate
at which aversive events occur
• When a CS is present: only providing information
about the effectiveness of a response
• Fear may be a by-product of avoidance training,
but not crucial to learning/ maintaining an
avoidance response
Evidence for One-Factor theory
• Almost postulating a "cognitive" theory of avoidance:
• Seligman and Jonston (1973) did postulate cognitive
theory:
– like Rescorla Wagner theory in that deals with predictability
• Basic premise:
– Learning occurs only when there is a discrepancy between
observation and expectation
– Subjects' behavior will change in avoidance task whenever
there is a discrepancy between expectancy and observation
Evidence for One-Factor theory
• Two important expectations in avoidance task:
– Expectation about consequences of a response
– Expectation about consequences of not responding
• Data support One-factor theory
– On trial 1: no expectations
– On trial 2 (and more): expectation about what will happen
• no shock will occur if response is made
– Shock will occur if no response is made
– Animal prefers no shock to shock- so responds
• Contingency is what is important in avoidance, fear is by-product!
Safety-Signal Hypothesis
• In avoidance: Cannot forget that there is positive reinforcement
through conditioned inhibition of fear
– That is, are rewarded for reducing fear through the avoidance
response
– Thus avoidance response is (obviously) reinforced
– But so are signals that occur with that response
• Conditioned safety signals!
– Performance of an avoidance behavior reduces conditioned fear
– Safety signals associated with this avoidance behavior are then signals
for reinforcement
– These safety signals may not be “outside” or external cues, but
feelings and behaviors within the organism
– Feeling of relief is reinforcing!
Characteristics of Avoidance Behavior
• Negatively reinforced behavior is difficult to
extinguish:
– escape behaviors take long time to go away
– e.g.: rat in 1-way shuttle still runs when light
comes on-even after hundreds of EXT trials
• BUT: Will extinguish quickly if animal/human
can detect change from conditioning to EXT
situation
Characteristics of Avoidance Behavior
• Extremely variable:
– from subject to subject
– from session to session with SAME subject
– procedure to procedure
• Choice of response is important
– determines how quickly will learn contingency
– how well learning is maintained
• Example: 1-way vs 2-way shuttle avoidance tests:
– Rat learns to run to the safe side shuttle box when the light comes on to avoid
shock
– 1-way shuttle: run to other (always the same) area when light comes on
– 2-way shuttle: run to opposite (changing) area when light comes on
• Why do animals have a difficult time learning 2-way shuttle avoidance
Biological Boundaries on
Avoidance Behavior
• Different animals “avoid” in different ways,
using different behaviors
• BUT: Highly similar pattern of FEAR-FREEZEFLIGHT-FIGHT sequence
• Suggests that avoidance has highly biological
organization, but that contingency is learned
Species specific Defense reactions
• SSDR’s
– Bob Bolles (1970, 1971)
– Behaviors which animal does naturally in time of
danger
– Includes: freezing, fleeing, fighting
• Why?
– Animal has innate behaviors does when avoiding
noxious stimulus– Can't make it go against its nature
Examples of SSDR’s
• Rat in shuttle box:
– If shock back feet: it easily escapes and jumps over
barrier
– If shock front feet: Avoidance behavior greatly
reduced
• Rats when shocked bite/grab, defensive
burying
• Pigeons when shocked WING flap, then peck
Negative Reinforcement in Humans
• Most often "reinforcement" technique used in real world
– Often used because is cheaper, easier, more natural
– Produces "bad" side effects: avoidance responses to SD = boss,
principal, spouse, etc.
• Data show it is a highly ineffective reinforcement procedure with
many side effects
• People develop interesting behaviors towards signals of the
aversive:
– Avoidance
– Anger/frustration
– Negative thoughts and feelings towards that individual/situation
• Given uneven and unpredictable behavior with avoidance
techniques, avoid them in applied situations.
Avoidance behavior in Humans
• Humans have many ineffective and/or irrational fears
– Often involve avoidance responses due to original fear
– Develop odd avoidance behaviors as a result
• Maintained by decrease in fear
• E.g., banging two sticks to keep the tigers away
• Symptoms of obsessive/compulsive disorders:
– Compulsions = repeated, stereotyped, ritualized actions
• individual feels compelled to engage in them
– Obsessions = compulsive thoughts (no actual actions)
– Many, many examples of this
– Can begin to interfere in life
An Aside: Flooding as an aversive:
• To extinguish an inappropriate response: must make
contact with "changed reinforcement or punishment"
situation
• Sometimes used as alternative to systematic
desensitization
• Flood with presentation of fear-provoking stimulus
– Again, no actual consequence occurs
– Continue presentations until the response is extinguished
• Problem: may "scare the patient to death"
Perceived Control and Avoidance
• Significant side effects may be produced by
avoidance tasks
– Animal psychosis or experimental psychosis
– Animal stops eating, drinking
– Animal may engage in self injurious behavior
• Appears to be due to implementation of an
avoidance contingency under certain conditions
• Most severe: Learned Helplessness
Learned helplessness
Marty Seligman
• Four groups of dogs
Grp
Grp
Grp
Grp
Training I and II
I Escapable/escapeable
II Inescapable/inescapable
III Escapable/inescapable
III Inescapable/escapable
result Lasting effects
run
None
not run
None
not run
None
not run
Severe
Remember, Seligman’s hypothesis was that NONE of the
dogs would be significantly harmed.
Key Factor = inescapability
• Once learned not to escape (learned to be
helpless)= not change
Characteristics of L.H.
• Inescapability that produces phenomenon,
not the shock itself
• Works under variety of procedures,
conditions
• Very generalizeable, transferable
• if take far enough, can make it a
contingency rule for the animal, rather
than specific contingency for specific
situation(s)
Symptoms of L.H.
•
•
•
•
Passivity
Learned laziness
Retardation of learning
Somatic effects
• Reduction of helplessness with time
Clinical expressions of
learned helplessness
• School phobias
and math anxiety
• Abusive
Relationships
• Depression
• Cultural learned
helplessness
“Curing” or eliminating learned
helplessness
• Unlearn the rule
• Reshape or recondition
• Must be done in situation where
organism cannot fail
• Difficult to do- animals can “not”
respond
• UPenn program on relearning thoughts
during test taking
Why?
• Only when shock contingent on behavior do
animals develop LH
– Animals in no control/no control condition do not
develop
• Showed generalization very quickly
– In situations where there WAS a contingency, the
lack of behavior sabotaged results
How is this an example of the
importance of contingency?
• Got themselves into contingency trap
• If they don’t work, no reward, only punishment
• This reinforced contingency rule that THEY were the
cause of the bad consequences
• Self sabotage
• And it was true!
• Thus: treatment must be to learn better
contingencies and eliminate the bad (and in
their head) contingency rule
Why is this important for humans?
• Helps explain the “misbehavior” of humans
with some disorders
• Drug addicts and those with schizophrenia
make “poor” choices
– May be due to physiology of the
addiction or disease
– “bad choices” may be due to effect of
DA
– Real changes may be occurring in the
brain which prevent the addict from
being sensitive to changes in his or her
life rewards
• May also explain some of the perseverative
and off-task behaviors observed in these
individuals
What “causes” LH?
• Newer research: original theory of learned
helplessness NOT account for people's varying
reactions to situations that can cause learned
helplessness
• Learned helplessness sometimes remains specific to
one situation
• At other times generalizes across situations
• At first, difficult to predict which will occur in a given
situation
Factors affecting development of
learned helplessness in humans
• Attributional style
– Optimistic vs. pessimistic attributions
– Global vs. specific attributions
– Stable vs. unstable attributions
– Intrinsic vs. extrinsic attributions
• Data suggest these may be both innate and
learned
Punishment
Definitions?
• Any consequence which decreases the frequency of behavior
• Learning from the consequences that produce pain or
discomfort
• The loss of reinforcers: This has survival value for the
individual and for the species.
• Punishment teaches us not to repeat responses that cause us
harm
• Any stimulus or event, when used contingently, decreases the
probability of a response.
Nature of Punishment
• Punishment is defined neither by
– The actions of the person delivering the
consequences
– Nor by the nature of those consequences.
• A decrease in the future frequency of the
occurrence of the behavior must be observed
before a consequence qualifies as
punishment.
Definitions of Punishment
• Positive or Type I Punishment :
– Presentation of a stimulus (or an increase in the intensity of an
already present stimulus) immediately following a behavior
– Results in a decrease in the frequency of the behavior.
– Spanking, electric shock, etc.
• Negative or Type II Punishment:
– Termination of an already present stimulus (or a decrease in the
intensity of an already present stimulus) immediately following
a behavior
– Results in a decrease in the future frequency of the behavior.
– Response cost, overcorrection
Discriminative Effects
• Stimulus condition in the presence of which a response
has a lower probability of occurrence than it does in its
absence
• Response-contingent punishment:
– Delivery occurs in the presence of a stimulus cue
– Punishment only occurs when emitting contingent response.
• Stimulus control is important:
– If punishment occurs only in some stimulus conditions and not in others:
the suppressive effects of punishment will be most prevalent under those
conditions
– Organism learns the setting conditions as to when punishment will occur
– E.g.: Teacher has to see you misbehave
Punishment aka Aversive Control
• Note that aversive events are associated with both:
– positive punishment
– negative reinforcement
• Term aversive control is often used to describe
intervention involving either or both of these two
principles.
Aversive but not contingent?
• Aversive stimuli can also affect operant
behavior when given noncontingently
– That is, a targeted behavior neither produces nor
prevents the punisher
– when aversive stimuli occur independently of
responding.
• Most famous example is conditioned
emotional response (CER)
Why are CER’s important?
• The “threat” of an upcoming aversive event can decrease
responding
– Even when noncontingent
– Unpredictability produces GREAT CERs
• Because the aversive event is likely, organism “prepares”
or “gets ready” for the event
– Can be cued or uncued
– Really gets in the way of ongoing responses
• E.g., in dog training: if dog is afraid of being punished
– Rate of overall responding goes down
– Animal is afraid of punisher, so doesn’t ‘risk’ behaving
Parameters of Punishment
• Is all we learned about positive reinforcement
true, in mirror-image form, of punishment?
– Yes and no.
– Are some additional characteristics of punishment
• Any operant punishment situation is really a
punishment plus reinforcement situation.
– Just like any reinforcement situation =Sr + P
– If get punished, don’t get the reinforcer.
– If get the reinforcer, don’t get punished.
Punishment + Reinforcement?
• For punishment to suppress operant responding, responses
must already be occurring with some frequency.
• For responses to occur, they must be producing
reinforcement.
• So, effect of punishment reflects interaction of two
contingencies--reinforcement and punishment.
• Jointly operate in most situations.
Punishment Effectiveness
• Punishing only reinforced response is often not an effective
procedure.
– Suppresses responses
– Doesn’t provide a “replacement” response
• If you give organism an alternative, unpunished route to
reinforcement, then effects of punishment are enhanced.
• Always shape an incompatible response!
– A response that is the opposite of the inappropriate response
– E.g., punish getting out of seat but reinforce sitting in seat
Punishment Intensity:
• As intensity of punishing stimulus increases,
degree of suppression increases.
• If very intense shock is used, then suppression
may be virtually complete.
• Partially due to physiological effects; partially
due to contingency effects
Role of past experience!
• Experience with the punisher is important
– If never experienced shock, don’t know it hurts!
– Punishment effects are relative: What has been prior experience
and how is the punisher COMPARED to previous punishers?
• Lots of research showing this:
– Suppressive effect of intermediate shock intensity depends on
animal’s past experience with shock.
– If animal has experienced intensities going from mild to
intermediate, then there will be little suppression.
– If animal has experienced intensities going from severe to
intermediate, then there will be substantial suppression.
Immediate is Best!
• For punishment to be maximally effective, it must
immediately follow operant response.
• As delay interval between response and
punishment increases, amount of suppression
decreases.
• Too many other behaviors can occur between R
and P; contingency not as effective when initially
using P
Probability of a punisher
• Initially: Should be certain and follow each
operant response.
– Probability of punishment should be 1.0
– When responses are punished intermittently,
effectiveness of punishment procedure is reduced.
• Can shape towards partial schedule of
punishment
• Is this different than what observe with
reinforcement? Think about it!
Recovery from Punishment: Extinction
• When punishment is discontinued, suppressive
effects on responding ARE not permanent
• The rate of responding after punishment is
discontinued will
– not only recover
– But briefly exceed level at which it was occurring prior to
punishment
– Opposite of extinction burst
– Is this really surprising?
Unconditioned Punishers
• Unconditioned punisher: stimulus whose presentation
functions as punishment without having been paired with any
other punishers.
– Innate
– Biologically relevant
– Still, all organisms not respond the same!
• Unconditioned punishers will suppress any behavior that
precedes their onset.
• Again, is this similar to reinforcement?
How are Conditioned Punishers different?
• Conditioned punisher is a stimulus that functions as
punishment as a result of a person’s conditioning
history.
– E.g., the word “no” is a conditioned stimulus
– A child’s name can become a punisher!!!!
• Acquires capability to function as punisher through
stimulus-stimulus pairing with one or more
unconditioned or conditioned punishers.
Extinction of Conditioned Punishers
• If responses occur in absence of punisher, the
response will return, potentially to pre-punishment
levels
• If the conditioned punisher is repeatedly presented
without the punisher with which it was initially
paired, effectiveness as punishment will diminish
until it is no longer a punisher.
Generalization and Discrimination
• Stimulus that has been paired with numerous
forms of unconditioned and conditioned
punishers becomes a generalized conditioned
punisher.
• Stimulus that has been paired with only one
specific conditioned or unconditioned
punisher becomes a discriminated punisher.
Other factors influencing effectiveness of
punishment:
• Schedule or frequency of punishment
– Continuous punishment schedules knock down behavior more quickly
– Partial punishment schedules keep behavior suppressed more
effectively
• Availability of reinforcement for the target behavior
– Must eliminate inadvertent sources of reinforcement for your
behavior targeted for punishment
– Teacher may punish, but the other kids may keep reinforcing “class
clown” behavior
• Availability of reinforcement for an alternative behavior.
– Punishment more effective if reinforce the opposite behavior
– Again, must give organism alternative path to the reinforcer that was
maintaining the unwanted behavior
Punishment can lead to Aggression
• Reflexive aggression:
– When punished, act out aggressively
– Is called reflexive because appears to be innate
– When in pain, you bite!
• Operant Aggression
– Learn to be aggressive contingently
– retaliatory
Punishment and Aggression
• Aggression may model aggression
– That is, aggression breeds aggression
– Use of aggression as punishment may provide model of
how to “solve problems”
• Social Disruption
– The person who delivers the punishment/situation in
which punishment occurs become tainted
– Poisoning cues
– Also called social disruption:
• Behavior is disrupted in presence of cues that predict punishment
• May get freezing, reduced effort, etc., in presence of punisher
Is Time Out a Punisher?
• Yes, by definition it is a negative punisher
– Losing the opportunity to get reinforcement from
many other sources
– Again, to be effective, must really isolate so can’t get
reinforced.
• Time out is not isolation
– Time out is brief, focused and contingent
– Isolation is of long duration, often noncontingent, and
often not effective.
Rules for Using Time-Out
• 1 minute per year of age
– Not really effective for children under 6-9 mos
– For really little ones, VERY brief withdrawal of attention
• Must be quiet to get the timer to start
• Cannot use for dangerous, disruptive or selfstimulatory behavior
• Must really be “time out” from other rewards
Negative Punishment
• Response cost: your response costs you
something or some behavior
• OVERCORRECTION: two parts
– Restitution: reinstatement of environment (clean
up)
– Positive practice: practice better response for
situation
– Can also use satiation/habituation
If Punishment is so bad, why would a
behavioral therapist use it? Examples:
• Severe self injurious behavior (SIB)
• Highly aggressive behavior to others
• Behavior which creates in immediate danger
for self or others
– Running into the street
– Pulling a hot pan off the stove
Guidelines for using
positive punishment
• Behavior must be (immediately) dangerous to person or
others
• Rate of responding is so high that there is NO chance to
interrupt and reinforce “good” behavior
• Must have tried other alternatives
• An example:
• https://www.youtube.com/watch?v=qhKVJTe59F4
• http://www.youtube.com/watch?v=13UcT1FVVts