* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Extinction
Survey
Document related concepts
Abnormal psychology wikipedia , lookup
Thin-slicing wikipedia , lookup
Psychophysics wikipedia , lookup
Neuroeconomics wikipedia , lookup
Classical conditioning wikipedia , lookup
Shock collar wikipedia , lookup
Theory of planned behavior wikipedia , lookup
Verbal Behavior wikipedia , lookup
Applied behavior analysis wikipedia , lookup
Theory of reasoned action wikipedia , lookup
Sociobiology wikipedia , lookup
Psychological behaviorism wikipedia , lookup
Attribution (psychology) wikipedia , lookup
Adherence management coaching wikipedia , lookup
Flagellation wikipedia , lookup
Behavior analysis of child development wikipedia , lookup
Behaviorism wikipedia , lookup
Transcript
Avoidance: Is negative REINFORCEMENT! What is avoidance behavior? • Avoidance contingencies INCREASE the rate of an operant response – NOT punishment, but actually negative reinforcement – Remember, punishment DECREASES the response • But with both avoidance and punishment the organism reduces contact with an aversive stimulus! – Increasing periods of safety – Avoidance procedures = ACTIVE avoidance – Punishment procedures = passive avoidance Avoidance Tests: Discriminated Avoidance • Discriminated Avoidance: – – – – Procedure for studying negative reinforcement and avoidance A response CANCELS a shock Organism is responding for food reinforcers When light comes on, must press another lever to AVOID the shock • IF the response does not occur during the S+ the stimulus is followed by a shock • IF the response does occur during the S+, the shock is cancelled – thus: signal or sD for shock – if this were an escape: response could also occur DURING the shock to shut off shock Shuttle Avoidance • 1-way shuttle avoidance: – Animal on one side of a shuttle – Cue comes on: Must jump to other (safety) side or receive a shock. – Placed back in original side for each trial. • 2-way shuttle avoidance – Animal on one side of a shuttle – Cue comes on: Must jump to other (safety) side or receive a shock. – But: previously SAFE side then becomes shock side – Animal must jump back into “shock” side to get away from shock. – VERY Difficult for animal to learn. Theories of Avoidance: Two Factor theory • Two things happen during avoidance conditioning: CC and then OC – Animal learns to fear S+ via class. conditioning • CS (light)---> US (shock): UR (fear) • animal learns to fear light via pairing with shock – Animal will then learn a response to AVOID shock and thus remove/lessen their fear – Thus: not getting shocked reduces fear that was signaled by the CS Theories of Avoidance: Two Factor theory • Experimental evidence: – On initial training trials: light/CS produces physiological symptoms of fear • Escape response results in decrease in these physiological symptoms – On later trials: • little or no evidence of physiological fear with CS presentation • suggests fear has been reduced/replaced by the escape response – In sense: forms a negative feedback loop Problems with 2-factor theory: • Signs of fear dissipate w/time: – as animal gets "better" at avoidance response – thus: no fear to be avoided • The CS is not as important in avoidance learning as 2-factor theory states: – Animals can learn to avoid in a discriminated avoidance situation long before there is any sign that they are responding to/detecting the CS Two Avoidance Procedures: • Sidman Avoidance: – The response POSTPONES or DELAYS the shock – Thus: only temporary solution – Must keep responding to keep delaying the shock – Results in lots of responding – Again: some signal may be used to signal when must respond Two Avoidance Procedures: • Herrnstein and Hineline Procedure: – The response reduces the rate of the shock – Does NOT delays or cancel, just slows down rate of delivery – The response switches the schedule of shock to a lower rate – Note: cannot entirely AVOID shock in this procedure: • once animal receives shock on lowered schedule, reverts back to original schedule • animal must respond again to switch schedule again Herrnstein and Hineline: Test of 2-factor theory • Test of 2 factor theory: – Two groups of rats used – Group 1: Can turn off light, but still get shock – Group 2: Can turn off shock, light still on • 2-factor theory would predict that Group 1 should respond more, because this would be cancelling the CS that produces fear • Results: group 2 responds much more accurately, faster Alternative: One-Factor Theory • Responses occur whenever they reduce the rate at which aversive events occur • When a CS is present: only providing information about the effectiveness of a response • Fear may be a by-product of avoidance training, but not crucial to learning/ maintaining an avoidance response Evidence for One-Factor theory • Almost postulating a "cognitive" theory of avoidance: • Seligman and Jonston (1973) did postulate cognitive theory: – like Rescorla Wagner theory in that deals with predictability • Basic premise: – Learning occurs only when there is a discrepancy between observation and expectation – Subjects' behavior will change in avoidance task whenever there is a discrepancy between expectancy and observation Evidence for One-Factor theory • Two important expectations in avoidance task: – Expectation about consequences of a response – Expectation about consequences of not responding • Data support One-factor theory – On trial 1: no expectations – On trial 2 (and more): expectation about what will happen • no shock will occur if response is made – Shock will occur if no response is made – Animal prefers no shock to shock- so responds • Contingency is what is important in avoidance, fear is by-product! Safety-Signal Hypothesis • In avoidance: Cannot forget that there is positive reinforcement through conditioned inhibition of fear – That is, are rewarded for reducing fear through the avoidance response – Thus avoidance response is (obviously) reinforced – But so are signals that occur with that response • Conditioned safety signals! – Performance of an avoidance behavior reduces conditioned fear – Safety signals associated with this avoidance behavior are then signals for reinforcement – These safety signals may not be “outside” or external cues, but feelings and behaviors within the organism – Feeling of relief is reinforcing! Characteristics of Avoidance Behavior • Negatively reinforced behavior is difficult to extinguish: – escape behaviors take long time to go away – e.g.: rat in 1-way shuttle still runs when light comes on-even after hundreds of EXT trials • BUT: Will extinguish quickly if animal/human can detect change from conditioning to EXT situation Characteristics of Avoidance Behavior • Extremely variable: – from subject to subject – from session to session with SAME subject – procedure to procedure • Choice of response is important – determines how quickly will learn contingency – how well learning is maintained • Example: 1-way vs 2-way shuttle avoidance tests: – Rat learns to run to the safe side shuttle box when the light comes on to avoid shock – 1-way shuttle: run to other (always the same) area when light comes on – 2-way shuttle: run to opposite (changing) area when light comes on • Why do animals have a difficult time learning 2-way shuttle avoidance Biological Boundaries on Avoidance Behavior • Different animals “avoid” in different ways, using different behaviors • BUT: Highly similar pattern of FEAR-FREEZEFLIGHT-FIGHT sequence • Suggests that avoidance has highly biological organization, but that contingency is learned Species specific Defense reactions • SSDR’s – Bob Bolles (1970, 1971) – Behaviors which animal does naturally in time of danger – Includes: freezing, fleeing, fighting • Why? – Animal has innate behaviors does when avoiding noxious stimulus– Can't make it go against its nature Examples of SSDR’s • Rat in shuttle box: – If shock back feet: it easily escapes and jumps over barrier – If shock front feet: Avoidance behavior greatly reduced • Rats when shocked bite/grab, defensive burying • Pigeons when shocked WING flap, then peck Negative Reinforcement in Humans • Most often "reinforcement" technique used in real world – Often used because is cheaper, easier, more natural – Produces "bad" side effects: avoidance responses to SD = boss, principal, spouse, etc. • Data show it is a highly ineffective reinforcement procedure with many side effects • People develop interesting behaviors towards signals of the aversive: – Avoidance – Anger/frustration – Negative thoughts and feelings towards that individual/situation • Given uneven and unpredictable behavior with avoidance techniques, avoid them in applied situations. Avoidance behavior in Humans • Humans have many ineffective and/or irrational fears – Often involve avoidance responses due to original fear – Develop odd avoidance behaviors as a result • Maintained by decrease in fear • E.g., banging two sticks to keep the tigers away • Symptoms of obsessive/compulsive disorders: – Compulsions = repeated, stereotyped, ritualized actions • individual feels compelled to engage in them – Obsessions = compulsive thoughts (no actual actions) – Many, many examples of this – Can begin to interfere in life An Aside: Flooding as an aversive: • To extinguish an inappropriate response: must make contact with "changed reinforcement or punishment" situation • Sometimes used as alternative to systematic desensitization • Flood with presentation of fear-provoking stimulus – Again, no actual consequence occurs – Continue presentations until the response is extinguished • Problem: may "scare the patient to death" Perceived Control and Avoidance • Significant side effects may be produced by avoidance tasks – Animal psychosis or experimental psychosis – Animal stops eating, drinking – Animal may engage in self injurious behavior • Appears to be due to implementation of an avoidance contingency under certain conditions • Most severe: Learned Helplessness Learned helplessness Marty Seligman • Four groups of dogs Grp Grp Grp Grp Training I and II I Escapable/escapeable II Inescapable/inescapable III Escapable/inescapable III Inescapable/escapable result Lasting effects run None not run None not run None not run Severe Remember, Seligman’s hypothesis was that NONE of the dogs would be significantly harmed. Key Factor = inescapability • Once learned not to escape (learned to be helpless)= not change Characteristics of L.H. • Inescapability that produces phenomenon, not the shock itself • Works under variety of procedures, conditions • Very generalizeable, transferable • if take far enough, can make it a contingency rule for the animal, rather than specific contingency for specific situation(s) Symptoms of L.H. • • • • Passivity Learned laziness Retardation of learning Somatic effects • Reduction of helplessness with time Clinical expressions of learned helplessness • School phobias and math anxiety • Abusive Relationships • Depression • Cultural learned helplessness “Curing” or eliminating learned helplessness • Unlearn the rule • Reshape or recondition • Must be done in situation where organism cannot fail • Difficult to do- animals can “not” respond • UPenn program on relearning thoughts during test taking Why? • Only when shock contingent on behavior do animals develop LH – Animals in no control/no control condition do not develop • Showed generalization very quickly – In situations where there WAS a contingency, the lack of behavior sabotaged results How is this an example of the importance of contingency? • Got themselves into contingency trap • If they don’t work, no reward, only punishment • This reinforced contingency rule that THEY were the cause of the bad consequences • Self sabotage • And it was true! • Thus: treatment must be to learn better contingencies and eliminate the bad (and in their head) contingency rule Why is this important for humans? • Helps explain the “misbehavior” of humans with some disorders • Drug addicts and those with schizophrenia make “poor” choices – May be due to physiology of the addiction or disease – “bad choices” may be due to effect of DA – Real changes may be occurring in the brain which prevent the addict from being sensitive to changes in his or her life rewards • May also explain some of the perseverative and off-task behaviors observed in these individuals What “causes” LH? • Newer research: original theory of learned helplessness NOT account for people's varying reactions to situations that can cause learned helplessness • Learned helplessness sometimes remains specific to one situation • At other times generalizes across situations • At first, difficult to predict which will occur in a given situation Factors affecting development of learned helplessness in humans • Attributional style – Optimistic vs. pessimistic attributions – Global vs. specific attributions – Stable vs. unstable attributions – Intrinsic vs. extrinsic attributions • Data suggest these may be both innate and learned Punishment Definitions? • Any consequence which decreases the frequency of behavior • Learning from the consequences that produce pain or discomfort • The loss of reinforcers: This has survival value for the individual and for the species. • Punishment teaches us not to repeat responses that cause us harm • Any stimulus or event, when used contingently, decreases the probability of a response. Nature of Punishment • Punishment is defined neither by – The actions of the person delivering the consequences – Nor by the nature of those consequences. • A decrease in the future frequency of the occurrence of the behavior must be observed before a consequence qualifies as punishment. Definitions of Punishment • Positive or Type I Punishment : – Presentation of a stimulus (or an increase in the intensity of an already present stimulus) immediately following a behavior – Results in a decrease in the frequency of the behavior. – Spanking, electric shock, etc. • Negative or Type II Punishment: – Termination of an already present stimulus (or a decrease in the intensity of an already present stimulus) immediately following a behavior – Results in a decrease in the future frequency of the behavior. – Response cost, overcorrection Discriminative Effects • Stimulus condition in the presence of which a response has a lower probability of occurrence than it does in its absence • Response-contingent punishment: – Delivery occurs in the presence of a stimulus cue – Punishment only occurs when emitting contingent response. • Stimulus control is important: – If punishment occurs only in some stimulus conditions and not in others: the suppressive effects of punishment will be most prevalent under those conditions – Organism learns the setting conditions as to when punishment will occur – E.g.: Teacher has to see you misbehave Punishment aka Aversive Control • Note that aversive events are associated with both: – positive punishment – negative reinforcement • Term aversive control is often used to describe intervention involving either or both of these two principles. Aversive but not contingent? • Aversive stimuli can also affect operant behavior when given noncontingently – That is, a targeted behavior neither produces nor prevents the punisher – when aversive stimuli occur independently of responding. • Most famous example is conditioned emotional response (CER) Why are CER’s important? • The “threat” of an upcoming aversive event can decrease responding – Even when noncontingent – Unpredictability produces GREAT CERs • Because the aversive event is likely, organism “prepares” or “gets ready” for the event – Can be cued or uncued – Really gets in the way of ongoing responses • E.g., in dog training: if dog is afraid of being punished – Rate of overall responding goes down – Animal is afraid of punisher, so doesn’t ‘risk’ behaving Parameters of Punishment • Is all we learned about positive reinforcement true, in mirror-image form, of punishment? – Yes and no. – Are some additional characteristics of punishment • Any operant punishment situation is really a punishment plus reinforcement situation. – Just like any reinforcement situation =Sr + P – If get punished, don’t get the reinforcer. – If get the reinforcer, don’t get punished. Punishment + Reinforcement? • For punishment to suppress operant responding, responses must already be occurring with some frequency. • For responses to occur, they must be producing reinforcement. • So, effect of punishment reflects interaction of two contingencies--reinforcement and punishment. • Jointly operate in most situations. Punishment Effectiveness • Punishing only reinforced response is often not an effective procedure. – Suppresses responses – Doesn’t provide a “replacement” response • If you give organism an alternative, unpunished route to reinforcement, then effects of punishment are enhanced. • Always shape an incompatible response! – A response that is the opposite of the inappropriate response – E.g., punish getting out of seat but reinforce sitting in seat Punishment Intensity: • As intensity of punishing stimulus increases, degree of suppression increases. • If very intense shock is used, then suppression may be virtually complete. • Partially due to physiological effects; partially due to contingency effects Role of past experience! • Experience with the punisher is important – If never experienced shock, don’t know it hurts! – Punishment effects are relative: What has been prior experience and how is the punisher COMPARED to previous punishers? • Lots of research showing this: – Suppressive effect of intermediate shock intensity depends on animal’s past experience with shock. – If animal has experienced intensities going from mild to intermediate, then there will be little suppression. – If animal has experienced intensities going from severe to intermediate, then there will be substantial suppression. Immediate is Best! • For punishment to be maximally effective, it must immediately follow operant response. • As delay interval between response and punishment increases, amount of suppression decreases. • Too many other behaviors can occur between R and P; contingency not as effective when initially using P Probability of a punisher • Initially: Should be certain and follow each operant response. – Probability of punishment should be 1.0 – When responses are punished intermittently, effectiveness of punishment procedure is reduced. • Can shape towards partial schedule of punishment • Is this different than what observe with reinforcement? Think about it! Recovery from Punishment: Extinction • When punishment is discontinued, suppressive effects on responding ARE not permanent • The rate of responding after punishment is discontinued will – not only recover – But briefly exceed level at which it was occurring prior to punishment – Opposite of extinction burst – Is this really surprising? Unconditioned Punishers • Unconditioned punisher: stimulus whose presentation functions as punishment without having been paired with any other punishers. – Innate – Biologically relevant – Still, all organisms not respond the same! • Unconditioned punishers will suppress any behavior that precedes their onset. • Again, is this similar to reinforcement? How are Conditioned Punishers different? • Conditioned punisher is a stimulus that functions as punishment as a result of a person’s conditioning history. – E.g., the word “no” is a conditioned stimulus – A child’s name can become a punisher!!!! • Acquires capability to function as punisher through stimulus-stimulus pairing with one or more unconditioned or conditioned punishers. Extinction of Conditioned Punishers • If responses occur in absence of punisher, the response will return, potentially to pre-punishment levels • If the conditioned punisher is repeatedly presented without the punisher with which it was initially paired, effectiveness as punishment will diminish until it is no longer a punisher. Generalization and Discrimination • Stimulus that has been paired with numerous forms of unconditioned and conditioned punishers becomes a generalized conditioned punisher. • Stimulus that has been paired with only one specific conditioned or unconditioned punisher becomes a discriminated punisher. Other factors influencing effectiveness of punishment: • Schedule or frequency of punishment – Continuous punishment schedules knock down behavior more quickly – Partial punishment schedules keep behavior suppressed more effectively • Availability of reinforcement for the target behavior – Must eliminate inadvertent sources of reinforcement for your behavior targeted for punishment – Teacher may punish, but the other kids may keep reinforcing “class clown” behavior • Availability of reinforcement for an alternative behavior. – Punishment more effective if reinforce the opposite behavior – Again, must give organism alternative path to the reinforcer that was maintaining the unwanted behavior Punishment can lead to Aggression • Reflexive aggression: – When punished, act out aggressively – Is called reflexive because appears to be innate – When in pain, you bite! • Operant Aggression – Learn to be aggressive contingently – retaliatory Punishment and Aggression • Aggression may model aggression – That is, aggression breeds aggression – Use of aggression as punishment may provide model of how to “solve problems” • Social Disruption – The person who delivers the punishment/situation in which punishment occurs become tainted – Poisoning cues – Also called social disruption: • Behavior is disrupted in presence of cues that predict punishment • May get freezing, reduced effort, etc., in presence of punisher Is Time Out a Punisher? • Yes, by definition it is a negative punisher – Losing the opportunity to get reinforcement from many other sources – Again, to be effective, must really isolate so can’t get reinforced. • Time out is not isolation – Time out is brief, focused and contingent – Isolation is of long duration, often noncontingent, and often not effective. Rules for Using Time-Out • 1 minute per year of age – Not really effective for children under 6-9 mos – For really little ones, VERY brief withdrawal of attention • Must be quiet to get the timer to start • Cannot use for dangerous, disruptive or selfstimulatory behavior • Must really be “time out” from other rewards Negative Punishment • Response cost: your response costs you something or some behavior • OVERCORRECTION: two parts – Restitution: reinstatement of environment (clean up) – Positive practice: practice better response for situation – Can also use satiation/habituation If Punishment is so bad, why would a behavioral therapist use it? Examples: • Severe self injurious behavior (SIB) • Highly aggressive behavior to others • Behavior which creates in immediate danger for self or others – Running into the street – Pulling a hot pan off the stove Guidelines for using positive punishment • Behavior must be (immediately) dangerous to person or others • Rate of responding is so high that there is NO chance to interrupt and reinforce “good” behavior • Must have tried other alternatives • An example: • https://www.youtube.com/watch?v=qhKVJTe59F4 • http://www.youtube.com/watch?v=13UcT1FVVts