* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Gluck_OutlinePPT_Ch08 short
Caridoid escape reaction wikipedia , lookup
Synaptic gating wikipedia , lookup
Neurophilosophy wikipedia , lookup
Brain Rules wikipedia , lookup
History of neuroimaging wikipedia , lookup
Haemodynamic response wikipedia , lookup
Neurolinguistics wikipedia , lookup
Neuroplasticity wikipedia , lookup
Neuroanatomy wikipedia , lookup
Parent management training wikipedia , lookup
Cognitive neuroscience wikipedia , lookup
Donald O. Hebb wikipedia , lookup
Neuropsychology wikipedia , lookup
Stimulus (physiology) wikipedia , lookup
Neuroethology wikipedia , lookup
Aging brain wikipedia , lookup
Feature detection (nervous system) wikipedia , lookup
Eyeblink conditioning wikipedia , lookup
Time perception wikipedia , lookup
Metastability in the brain wikipedia , lookup
Neuropsychopharmacology wikipedia , lookup
Behavior analysis of child development wikipedia , lookup
Psychological behaviorism wikipedia , lookup
Behaviorism wikipedia , lookup
Neuroeconomics wikipedia , lookup
Clinical neurochemistry wikipedia , lookup
Chapter 8 Instrumental Conditioning: Learning the Consequences of Behavior 8.1 Behavioral Processes 8.1 Behavioral Processes • The “Discovery” of Instrumental Conditioning • Components of the Learned Association • Putting It All Together: Building the S–R–C Association • Learning and Memory in Everyday Life— The Problem with Punishment • Choice Behavior 3 The “Discovery” of Instrumental Conditioning Instrumental conditioning—developing a contingency between response and outcome. • Organism learns to make responses to obtain or avoid important consequences. e.g., trained circus animals, waterskiing squirrels AP/Wide World Photos • 4 Free-Operant Learning • Operant conditioning—a type of instrumental learning. • Skinner’s free-operant paradigm: Replaces Thorndike’s discrete trials. Learner can operate apparatus (e.g. Skinner box) at will. Learner’s trial-independent responses are measured with a cumulative recorder. 5 Operant Conditioning 6 Free-Operant Learning • Reinforcement—consequences increase behavior probability. • Punishment—consequences decrease behavior probability. 7 Components of the Learned Association • Three components to instrumental conditioning: Stimulus (S) Response (R) Consequence (C) 8 Stimulus • A discriminative stimulus is a cue, not a US or CS. A signal for when response will lead to consequence. Examples: Starting whistle for racing swimmers Potty seat for toilet-training toddler • Can increase behavior probability, but does NOT elicit behavior. 9 Response: Shaping • Shaping—successive approximations to the desired response are reinforced. Collect baseline data on current behavior (establish operant level). Identify target behavior. Reinforce successive approximations of the target response. 10 Response: Shaping • Example: Helping autistic children learn language (Ivar Lovaas, 1987). Say target word (e.g., child’s name). Reinforce with food any sound, then closer imitations. Introduce new words. 11 Response: Chaining • Chaining—learning a complicated sequence of responses by adding one discrete “link” (step) at a time. Backward chaining—training steps in reverse order. • Examples: Teaching pets unusual tricks. Teaching workers a sequential manufacturing process. 12 Skinner • http://www.youtube.com/watch?v=mm5FGr QEyBY 13 Consequence: Primary Reinforcers • Reinforcer—behavioral consequence that makes future behavior more likely. • Primary reinforcers: Reinforcing events that occur because of their natural characteristics and inherent ability to reinforce behavior (drive reduction theory). Examples: Food or Water Sleep Sex 14 Consequence: Secondary Reinforcers • Secondary (conditioned) reinforcers: Reinforcing events that function as reinforcers because they are consistently associated with one or more primary reinforcers. Example: Money No biological imperative. Can be exchanged for primary reinforcers (e.g., food or shelter). 15 Consequence: Punishers • Punishers: Behavioral consequence leads to a reduction of future behavior. • Strong and enduring aversive stimuli are the most effective suppressors. Aversive stimuli of low intensity may reinforce behavior we intend to suppress! Apply aversive stimuli immediately after the targeted behavior. Delaying punisher decreases contingency. 16 The Negative Contrast Effect Data from Kobre and Lipsitt, 1972. 17 Learning and Memory in Everyday Life— The Problem with Punishment • Use of corporal punishment is controversial. • Alternatives: Scolding Time-out Grounding Withholding allowance • Avoid attention for punished behavior. • Reinforce appropriate behavior. 18 Putting It All Together: Building the S–R–C Association • Timing affects learning: Immediate consequence = best learning Instrumental conditioning faster if R–C interval is short (temporal contiguity). • Timing can also impact: Punishment Immediate punishment more effective than delayed punishment. Self-control Forego immediate reward for greater future reward. 19 8.1 Interim Summary • Instrumental conditioning = learning a threeway association (S → R → C) between: Discriminative stimulus (S) Response (R) Consequence (C) C may be reinforcer or punisher. • In instrumental conditioning, C occurs only if R is made; whereas, In classical conditioning, the consequence (US) occurs automatically after the stimulus (CS). 20 8.1 Interim Summary • Four classes of instrumental conditioning: Positive reinforcement Negative reinforcement Positive punishment Negative punishment. • “Negative” and “positive” show if consequence is subtracted or added. • “Reinforcement” and “punishment” show response increase or decrease with learning. 21 8.1 Interim Summary • Operant conditioning: subclass of instrumental conditioning Organism responds at its own rate. • Complex responses may be trained by: Shaping Reinforcement of progressive approximations. Chaining Training a sequence of responses, one step at a time. 22 8.2 Brain Substrates 8.2 Brain Substrates • The Basal Ganglia (BG) and Instrumental Conditioning • Mechanisms of Reinforcement in the Brain 24 BG and Instrumental Conditioning • BG help connect information from the sensory and motor cortices to make a behavioral response. • BG may serve as storage for S–R associations (especially those in which R is a movement). • With BG lesions (in dorsolateral striatum): Rats learned to lever-press for food. But showed impaired discriminative S training. 25 Basal Ganglia 26 Reinforcement in the Brain: This figure shows that instrumental learning may involve the interaction of several neural systems. 27 Electrical Brain Stimulation • One of the “pleasure centers” is the ventral tegmental area (VTA) in the brainstem. The VTA is the center for dopamine neuromodulation. • VTA stimulation = powerful reinforcement 28 Electrical Brain Stimulation: Brain stimulation may directly activate the brain's “reinforcement” system, eliminating the need for natural reinforcers (e.g., food). Stimulus S (Sight of lever) Visual System (e.g. visual cortex) Motor System (e.g. basal ganglia) Reinforcement System Taste System (e.g.brainstem gustatory nuclei) Response R (Press lever) Consequence C: Electrical Brain Stimulation Hungry? 29 Dopamine and Reinforcement • Some VTA axons extend to the nucleus accumbens in BG. Nucleus accumbens sends dopamine to motor areas in the striatum. • Dopamine may be the physiological basis for the “wanting” aspect of reinforcement. “Motivation” or “wanting” in chemical form May contribute to addictive behavior. 30 Reward Prediction by Dopamine Neurons • Schultz (2002) trained monkeys to press a lever for food. • Electrophysiological recordings indicate that dopamine neurons in a monkey’s midbrain signal reward (or omission of reward). 31 Reward Prediction by Dopamine Neurons • In study: Dopamine neurons in a monkey’s midbrain respond strongly after unexpected rewards. If light occurs before food, dopamine neurons increase activation after light, but not after food. Dopamine neurons decrease activity after an expected reward does NOT occur (omission). • Illustrates reward prediction hypothesis i.e., dopamine is involved in predicting future reward. 32 (A) Unexpected juice reward: (B) Reward is predicted by light stimulus: (C) Predicted reward is omitted: Adapted from Schultz, 2002. 33 Opioids and Hedonic (Liking) Value • Endogenous opioids (endorphins) may mediate “liking.” • Opiates (heroin, morphine) bind to the brain’s natural opiate receptors. • Opiates may provide information about “liking” that helps stimulate VTA’s “wanting” system. 34 8.2 Interim Summary • In the brain, instrumental S-R-C associations may be stored in corticocortical connections and via basal ganglia. • Brain’s reinforcement system may include release of dopamine from ventral tegmental area to basal ganglia. • Drugs that interfere with the dopamine system disrupt instrumental conditioning. 35 8.2 Interim Summary • Several hypotheses on interaction of dopamine and reinforcement. Anhedonia hypothesis: Dopamine gives reinforcers their “goodness.” Incentive salience hypothesis: Dopamine modulates “wanting” rather than “liking” (how hard an organism is willing to work for reinforcement). Reward prediction hypothesis: Dopamine signals whether reinforcement is expected. 36 8.2 Interim Summary • Whereas dopamine may be involved in “wanting,” endogenous opioids may be involved in “liking.” Drugs that affect brain opiate receptors affect hedonic (“goodness”) value of primary reinforcers and punishers (e.g., food and pain). 37 8.3 Clinical Perspectives 8.3 Clinical Perspectives • Drug Addiction • Behavioral Addiction • Treatments 39 Drug Addiction • Pathological addiction—a strong habit maintained despite harmful consequences. Involves craving a high “euphoria” and avoiding withdrawal. Seeking pleasure involves positive reinforcement. Avoiding pain involves negative reinforcement. • As indicated by the incentive salience hypothesis, dopamine is involved in “wanting” a drug. 40 Effects of Drugs on Dopaminergic Neurons 41 Behavioral Addiction • Behavioral addiction—addiction to certain behaviors, rather than drugs. • Examples: Compulsive gambling, eating, sex, Internet use, shopping, exercise, work Everynight Images/Alamy Produces euphoria. Understanding drug addiction may help understand/treat behavioral addictions. 42 Behavioral Addiction http://www.youtube.com/watch?v=Bz2VT5Ky 7Kw Everynight Images/Alamy • 43 Treatments • Naltrexone (drug) treatment: Indirectly inhibits dopamine production; may help treat heroin addicts and compulsive gamblers. • (Cognitive) behavior therapies: e.g., extinction, distancing, reinforcement of alternative behaviors, delayed reinforcement Based on instrumental conditioning principles. 44 8.3 Interim Summary • Addictive drugs (e.g., heroin, caffeine) may hijack brain’s reinforcement system. May be psychological as well as physiological addiction. • Behavioral addictions may reflect same brain processes as drug addictions. 45 8.3 Interim Summary • Treatment for people with addictions may include: Cognitive therapy Medication Behavioral therapy Including principles learned from instrumental conditioning. 46