Download Ch18 Lecture

18
Actions, Habits, and the
Cortico-Striatal System
Some Behaviors Are Modified by the Outcomes They Produce
To appreciate the basic problems addressed in this chapter,
you might think back to when you initially learned to drive
a car. In order to competently drive you had to learn and
coordinate many complex behaviors, such as:
•
•
•
•
•
•
•
Insert the ignition key
Turn on the ignition key
Release the handbrake
Put the car in gear
Put your foot on the accelerator
Generate just the right amount of gas
Adjust the steering wheel to maintain or change the
trajectory
• Make proper fine-grain adjustments to oncoming traffic
• Apply just the right amount of pressure on the brake to
stop
The Concept of Instrumental Behavior
Most psychologists use the term
instrumental learning or instrumental
behavior when referring to the study of
how behavior is modified by the outcome it
produces.
This term recognizes that our behaviors can
be viewed as instruments that can change
or modify our environments. For example,
when you turn the ignition key, the engine
starts.
The Scientific Study of Instrumental Behavior Began with E.L.
Thorndike
Thorndike’s Law of Effect
The essence of Thorndike’s theory is that outcomes
produced by behavior ultimately adapt the animal to the
situation by strengthening and weakening existing
stimulus–response (S–R) connections. Outcomes that
reward behavior strengthen S–R connections;
nonrewarding outcomes weaken connections.
Tolman’s Cognitive Expectancy Theory
According to E.C. Tolman:
Instrumental behaviors are organized around
goals and mediated by expectancies.
An expectancy is a three-term association
(S1–R–S2) that includes a representation of
the stimulus situation (S1) that preceded the
response, a representation of the response
(R), and a representation of the outcome
(S2) produced by the response.
Comparison of the S–R Habit versus Expectancy Representation
of the Cat’s Solution to the Puzzle Box
Instrumental Behaviors Can Be Classified as Either Actions or
Habits
Actions and habits differ
on four dimensions.
How Can You Tell If an Instrumental Behavior Is an Action or a
Habit? The Reward Devaluation Strategy
The reward
devaluation strategy
centers on changing
the value of the
outcome after the
animal has learned
to solve the
problem.
The Rat’s Lever-Pressing Response Becomes a Habit with
Extensive Training
Instrumental behaviors are
initially controlled by the action
system. This conclusion is
based on the fact that after
limited training the animals
behavior is reduced following
reward devaluation.
However, with extensive
practice the behaviors become
habits. This conclusion is
based on the fact that after
extensive training, reward
devaluation has no effect
compared to the control
condition.
Conceptual Model for Actions and Habits
Actions versus Habits
Action and habit systems compete for control.
Once an instrumental response is well learned,
the two systems can work synergistically to
support the same behavior, with the habit
system dominating. However, when the
contingencies are reversed, the action system
attempts to rapidly adjust to the system but the
output from the habit system can interfere with
adaptations.
• Example: An American crossing the street in
England has to overcome the habit of looking
to the left before crossing.
The Acquisition of Instrumental Behaviors Depends on the
Striatum
Rats with damage
to the striatum
could not learn
that immediately
turning left or
right to enter the
adjacent arm
would be
rewarded.
The Striatum Integrates Information from Many Brain Regions to
Create Instrumental Behaviors
Neural Support for the Action System: Dorsomedial Striatum
Supports Actions
Rats with damage to the dorsomedial striatum are
not sensitive to reward devaluation.
Number of Response/min
Neural Support for the Action System: The Basolateral Amygdala
Attaches Value Outcomes
Con
Deval
Sham
Amygdala
The amygdala is critical for
attaching value to a particular
outcome. Primates and rodents
with damage to this region learn
simple discriminations but are
insensitive to reward devaluation.
Neural Support for Actions: Prelimbic Prefrontal Cortex Supports
Action Learning but Is Not a Storage Site
Number of Response/min
Prelimbic
Con
Deval
Prior
After
The prelimbic region is critical in the
acquisition of the associations that
support an action. We know this
because when this region is
damaged prior to training, reward
devaluation has no effect,
meaning the behavior is a habit.
The prelimbic region is not the site
in the brain where these
associations are stored. We know
this because if this region is
damaged after training, rats are
sensitive to reward devaluation.
Neural Support for Habits: Dorsolateral Striatum
Number of Response/min
DLS damage
Con
Deval
Control
DLS lesion
Damage to the dorsolateral striatum
prevents the development of habits.
Following extensive lever press
training, normal rats are insensitive
to reward devaluation. However, rats
with damage to the dorsolateral
striatum are still sensitive to
devaluation, indicating that the
behavior never became a habit.
Infralimbic Medial Prefrontal Cortex Suppresses the Action System
Number of Response/min
Infralimbic
Damage to this region prior to training prevents
instrumental behaviors from becoming habits. Rats
with such damage remain sensitive to reward
devaluation even after extensive training.
Con
Deval
Moreover, if this region is damaged after rats have
be extensively trained and the instrumental
response has become a habit, the rats again
become sensitive to reward devaluation.
Prior
After
All this suggests that with extensive training the
infralimbic region comes to suppress the output of
the action system.
The Contributions of the Prelimbic and Infralimbic Regions of the
Prefrontal Cortex to Instrumental Behavior
The Contributions of the Prelimbic and Infralimbic Regions of the
Medial Prefrontal Cortex to Instrumental Behavior
Summary
Prelimbic prefrontal cortex contributes to creating action
based behavior. However, its not a storage site for the
action pattern because damage to it after the action
has been learned does not affect animals sensitive to
reward devaluation.
Infralimbic prefrontal cortex suppresses the action
system. If this region is damaged before training the
animal never becomes insensitive to reward
devaluation even after extensive training. Moreover
after extensive training, when the behavior is normally
a habit, if the infralimbic region is damaged it again
becomes action—sensitive to reward devaluation.
The Mesolimbic Dopamine System and the Dopamine Theory
of Reinforcement
(A) The mesolimbic dopamine
system. The dopamine neurons are
located in the ventral tegmental area
(VTA) and their fibers project into the
nucleus accumbens.
(B) The dopamine theory of
reinforcement. An outcome–reward
has two functions: (1) It generates a
representation (O), and (2) it
activates dopamine neurons in the
VTA that release dopamine. This
strengthens synaptic connections
between the representations of the
stimulus (S) complex and the
response (R) and perhaps between
the representations of the response
and the outcome.
The Incentive Salience Hypothesis of How Rewards Influence
Instrumental Behavior
The incentive salience
hypothesis assumes that the
reward turns on dopamine
neurons in the VTA. In the
normal sequence of events that
establish instrumental behavior,
the stimulus not only can
associate with the response, it
also can become associated
with the incentive properties of
the outcome. The stimulus
itself can Then elicit strong
urges or wants that lead the
individual to seek out the
reward.
Neural Support for Action and Habit Systems: Summary
Instrumental Behavior
Habit System
Action System
Dorsomedial
striatum
Basolateral
amygdala
Dorsolateral
striatum
Prelimbic prefrontal
cortex
Mesolimbic
dopamine system
Infralimbic prefrontal
cortex