* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Classical conditioning of instrumental conditioning?
Attitude change wikipedia , lookup
Behavioral modernity wikipedia , lookup
Insufficient justification wikipedia , lookup
Thin-slicing wikipedia , lookup
Applied behavior analysis wikipedia , lookup
Verbal Behavior wikipedia , lookup
Attribution (psychology) wikipedia , lookup
Social perception wikipedia , lookup
Impulsivity wikipedia , lookup
Classical conditioning wikipedia , lookup
Sociobiology wikipedia , lookup
Descriptive psychology wikipedia , lookup
Theory of planned behavior wikipedia , lookup
Adherence management coaching wikipedia , lookup
Transtheoretical model wikipedia , lookup
Theory of reasoned action wikipedia , lookup
Behavior analysis of child development wikipedia , lookup
Psychological behaviorism wikipedia , lookup
Behaviorism wikipedia , lookup
CLASSICAL CONDITIONING OF INSTRUMENTAL CONDITIONING? WHOA…THEY ARE SUPPOSED TO BE SEPARATE! THORNDIKE’S ROLE IN CC-OC CONNECTION • Thorndike wrote that instrumental conditioning occurs in context of specific environmental stimuli • Cues involved that tell you “contingency in effect” • Really a THREE term contingency: S,R and O • • • Stimulus Response Outcome • S: RConsequence • Stimulus can be an S+ • Stimulus can be an S• Can predict a reinforcer OR a punisher S-R ASSOCIATION AND LAW OF EFFECT • Contextual stimuli (S) predict the instrumental response contingency (R or RO). • Is an association between the S and the R • Is an association between the R and the O • According to THORNDIKE: Role of the reinforcer is to “stamp in” or “strengthen” the S-R connections • NOT the RO contingency • Motivation for engaging in RO contingency are the setting contextual stimuli (S) • Data did NOT support this model • • Association IS formed between the R and O However, contextual cues are VERY important in operant conditioning EXPECTANCY OF REWARD • Reward expectancy can be a motivator for the operant response! • Cues for this expectancy are classically conditioned • Expectancy cues can be highly varied and complex • • • • The situation The individual(s) Signal learning Cues paired with RO contingency because they predict the contingency • Clark Hull and Kenneth Spence: • Behavior is result of interaction between organism and its environment. • Environment provides the stimuli; The organism responds (all of which is observable) • Hull’s theory = drive reduction theory • Basic premise of their theory • Instrumental response increases during conditioning because • Presence of S evokes instrumental response through CC • Instrumental response is emitted because of operant contingency TWO PROCESS THEORY OF OPERANT CONDITIONING • Assumes 2 distinct types of learning: • Pavlovian/CC • Instrumental/Operant • Related together in special way: • • • Presence of stimuli (S) come to predict R-O relationship R-O relationship is also strengthened Thus S-O become connected • Rescorla and Solomon assume S-O association activates emotional state • Motivates the operant behavior • This emotional state assumed to be positive or negative depending on the consequating stimulus TEST OF 2-PROCESS THEORY • Pavlovian Instrumental Transfer experiment: • Can counterbalance phase 1 and phase 2 • Critical transfer phase it the transfer test: Will the rat lever press to tone alone? WHAT DO THE DATA SAY? • Large research area • Organisms respond more in presence of S if S is positive (appetitive) • Organisms respond less in presence of S if S is negative (aversive) • BUT: still respond in absence of the S • Response interactions occur in Pavlovian Instrumental Transfer • Evidence suggests do develop emotional responses to the S • Also develop sign tracking behavior (now sign tracking begins to make sense!) KRANK, ET. AL., 2008 • Rat study! • 2 response levers, 1 on either side of water bottle • Trained to press either response lever for drop of sweetened water, then ethanol • CONC VI 20 sec VI 20 sec schedule • 8 Pavlovian sessions: • No response levers • CS (light) of 10 sec on either left or right side (above lever hole) 0.2 ml of ethanol • Unpaired group: CS and ethanol separated by 10 sec. • Paired group: CS just before ethanol: GOT sign tracking • Replaced levers and added Pavlovian Transfer Test • CS light periodically presented while rats responding for ethanol; alternated over both levers KRANK, ET. AL., 2008 • What happened? • Prats pressed each response lever about 2x/min prior • • to CS Unpaired group: did not change much when CS added Paired group: significant increase in lever pressing during CS presentation IF presented to same side as training trials • Why is this important? • Shows that CS facilitated lever pressing • Demonstrates importance of classical conditioning in • operant conditioning contingencies Fairly specific as well- not just any CS, but the correct-side CS CONDITIONAL EMOTIONAL STATES? REWARD SPECIFIC EXPECTANCIES? • 2-process theory assumes classical conditioning mediates instrumental conditioning through conditioning of positive or negative emotions • BUT: also develop specific reward expectancies • These reward specific expectancies can undermine the emotional conditioning • Kruse, et. al., 1983: • • Food pellets vs. sugar solution • Suggested rats formed specific expectancies about what the reward would be. CS+ for food pellets elicited more instrumental responding when pellets were the reward than when sugar and vice versa R-O AND S(R-O) RELATIONSHIPS IMPORTANT • HIERARCHICAL S(R-O) RELATIONS • In addition to the simple associations of 2 elements (i.e., S-R, S-O, R-O), can develop hierarchical • associations • the (S) signals the relationship between a response and its outcome S -> (R -> O) the (S) becomes an occasion setter that signals when a specific response will be followed by a specific reinforcer • (S) can be a context or a specific cue • EXPERIMENTAL APPROACH • One (S) tone signals (R) lever push – (O) food • Another (S) light signals (R) pull string – (O) sucrose • Then switch the (S) – (R-O) combinations • Animals confused by the switch- responding decreases • Suggests that reward expectancies are formed • Considerable support for S(R-O) relationships WANTING VS NEEDING • WANTING IS THE MOTIVATIONAL CONSEQUENCES OF REWARDS • Is an underlying incentive salience • Motivation to get the reward • Wanting and Needs usually go together • Needing the food that you Want • WANTING CAN BE SEPARATED FROM NEEDS • Drug addiction, gambling and overeating: wants or physiological needs • Impulse control disorders where wanting has much more intensity • Self-control study with children • Get one marsh mellow now or two later • Incentive salience is the psychological process underlying temptation • Produces “surges of motivation to obtain and consume the reward” WANTING VS NEEDING • INCENTIVE SALIENCE INTENSITY MODULATED BY • • Needs: really are hungry • If needs are paired with emotional arousal and stress: Now stress cues “need seeking” behavior • Needs can become “wants” when paired with emotional arousal Individual personality differences important • Different people have different levels of emotional arousal and stress • Individual differences in types of pairing history. WANTING VS NEEDING • INCENTIVE SALIENCE OF SPECIFIC REWARDS CHANGES WITH EXPERIENCE • Foods such as chocolate • Drugs such as cocaine • Reward system becomes sensitized • High levels of responding to reward cues • The sight of food, drugs or other incentives • SENSITIZED INCENTIVE SALIENCE PRODUCES IMPULSIVE BEHAVIOR • Difficult to exercise self-control • External conditions such as stress reduce self-control • HIGH LEVELS OF RELAPSE IN DRUG ADDICTS • Not well explained by pleasure “liking” to get high • Not well explained by withdrawal “avoiding” the discomfort • Mostly cues with incentive salience that produce excessive wanting WHAT IS ADDICTION? • Physiologic dependence and withdrawal avoidance do not explain addiction • Neurobiology of addiction attempts to explain the mechanisms by which drug seeking behaviors are consolidated into compulsive use: • • -long persistence of relapse risk -drug-associated cues control behavior ALTHOUGH ADDICTIVE DRUGS ARE PHARMACOLOGICALLY DIVERSE… • Stimulants (act as a serotonin-norepinephrine-dopamine reuptake inhibitors) • Cocaine, amphetamines, MDMA • Opioids (agonist action) • Heroin, morphine, oxycodone, fentanyl • GABAergic agonists/modulators • Alcohol, benzodiazepines, barbiturates • Cannabis (binds cannabinoid receptors) …THEY ALL LEAD TO A COMMON PATHWAY • All addictive drugs pharmacologically release dopamine in the nucleus accumbens • Why? • Addiction is essentially a learned process • Predictive cues along with operant response • On of best examples of associative learning with instrumental conditioning. REINFORCEMENT AND DOPAMINE? • Olds and Milner: • Brain stimulation = lots of behavior • Animals would work until death to gain access to this brain stimulation • Thought had discovered pleasure center: • • Nucleus accumbens Mesolimbic pathway • Dopamine (DA) was neurotransmitter involved in these areas THE DOPAMINE SYSTEM REINFORCEMENT AND DA • EBS = electrical brain stimulation • • • • releasing LOTS of dopamine (DA) Results in lots of locomotion or exploratory behavior Salamone and Schultz’ modern work has shown this release DA release modulates “appetitive behavior” • • • Occurs in modes or modules related to terminal event E.g., food modes, sex modes Which mode depends on context of environment • Search, capture, prepare, consume • DA not affect consummatory behavior DA IS REGULATED IN TWO WAYS: • DA is released in pulses (phasic) and has an overall tone in synapse (tonic) • Result is feedback system • DA release in response to stimuli in environment • Motivated or energize appropriate behavior • Feedback system follows Rescorla Wagner model DA REGULATION • DA also has a constant level or overall tone in the synapse: • • • • • More released when “surprised above what expected Less when “surprised” below what expected When get what expected- behavior is “well learned” and appears to become habit (not sure how this works yet) Think Rescorla Wagner model Thus, fluctuations in DA as learn, and then serve as feedback regarding state of environment HOW RELATE TO CHOICE? • The value of the choice should affect the size of the DA burst • • Experience can affect DA release • • • • • Small rewards = less DA release than larger rewards Use to large rewards; less responsive to smaller rewards Becomes a problem with gambling, other addicitons Used to large bursts of DA via your gambling, drug, etc. More typical rewards such as a paycheck, family, friends, etc., no longer elicit a response or as large a response as the “addictive” responses Question: can we reshape/retrain the brain? THE DOPAMINE REWARD PATHWAY HOW DOPAMINE LEADS TO BEHAVIOR CHANGE • Dopamine required for natural stimuli (food, opportunity for mating, etc) to be rewarding and drive behavior • Natural rewards and addictive drugs both cause dopamine release in the Nucleus Acumbens • Addictive drugs mimic effects of natural rewards and thus shape behavior THE DOPAMINE REWARD PATHWAY HOW DOPAMINE LEADS TO BEHAVIOR CHANGE • Survival demands that organisms find and obtain needed resources (food, shelter) and opportunity for mating despite risks -survival relevant goals • These goals have natural “rewards” (eating, safety, sex) • Behaviors with rewarding goals persist to a conclusion and increase over time as they are positively reinforcing • Dopamine is a “feedback” system”: • If it is a rewarding behavior….then do it again! • If it is not a rewarding behavior, don’t do it. THE DOPAMINE REWARD PATHWAY HOW DOPAMINE LEADS TO BEHAVIOR CHANGE • Internal states (hunger) increase value of goal-related cues and increase pleasure of consumption • As internal states increase, seeking behavior for a resolution to that increases • Thus: likelihood that complex behavioral sequence (hunting) will be brought to successful conclusion THE DOPAMINE REWARD PATHWAY HOW DOPAMINE LEADS TO BEHAVIOR CHANGE • Behavioral sequences involved in obtaining reward (steps required to hunt) become overlearned/automatized • Automatized behavioral repertoires can be activated by cues which are predictive of reward PREDICTION ERROR HYPOTHESIS: SCHULTZ, 1999; 2005 • Exposure to an unexpected reward causes transient firing of dopamine neurons which signals brain to learn a cue. • Once cue is learned, burst of firing occurs at cue, not at reward. • • If the reward does not arrive, dopamine firing will decrease below baseline levels serves as an error signal about reward predictions • If reward comes at unexpected time, dopamine firing will increase positive predictive error signal: “better than expected!” • Remember Rescorla-Wagner! DOPAMINE GATING HYPOTHESIS: BERRINGER • Because drugs cause dopamine release (due to pharmacological actions), dopamine firing upon use does not decay over time brain repeatedly gets positive predictive error signal: “better than expected!” • Drug cues become ubiquitous (drug cues difficult to extinguish) • Cues that predict drug availability take on enormous incentive salience (consolidates drug seeking behavior) • Drug cues will become powerfully over-weighted compared to other choices (contributes to loss of control over drug use) CLINICAL IMPLICATIONS • Addictive behaviors are a important and normal part of human behavior • Addictive drugs pharmacologically modify functioning of reward circuits to overvalue drug rewards and reduce the comparative value of other rewards • Intention to stop use is not enough to stably quit substance use. SO: TREATMENT OF ADDICTION • First: detox…..get drugs out of system • Next: work on breaking • The operant response of seeking • The cues that predict the drug • Easy-peasy, correct? IS RELAPSE HIGHER FOR DRUG ADDICTION? • Yes, it is up there and comparable to other “behavioral” diseases • How are addiction, hypertension and asthma behavioral disorders? WHAT DOES WORK? • Understanding the importance of environmental setting cues and conditions • Drug paraphernalia itself • Environmental cues: location, time of day, activities etc. • People: Family, friends, etc. • Stressors: what stressors are related to drug addiction • Two potential types of alchoholism: • • • Excitement seeking Risk/aversion avoidance Treatment of “triggers” will be very different WHAT DOES WORK? • Must rebuild new cues linked to positive behavior • • • • New rules for living New friends New social skills New skills for dealing with stressors, etc. • Sober Living House programs are very effective • Allow slow transition back into life • Allows for learning to fluency the new skills and cues IS THIS JUST TRUE FOR ADDICTION? • What about other “habits”? • • • Takes 2-3 weeks to change a habit That is about how long it takes to change neural circuits Any long term behavior change must involve change of cues as well as change in behavior • • • Think of dieting, etc. Transitioning of treatment programs: special schools placement back to regular schools Others?