Download - CUNY Academic Works

City University of New York (CUNY)
CUNY Academic Works
Dissertations, Theses, and Capstone Projects
Graduate Center
9-30-2015
The Role of the VTA NMDA Receptors, VTA DA
Cells and VTA Terminal Regions in RewardRelated Learning
Karen Kest
Graduate Center, City University of New York
How does access to this work benefit you? Let us know!
Follow this and additional works at: http://academicworks.cuny.edu/gc_etds
Part of the Biology Commons, and the Psychology Commons
Recommended Citation
Kest, Karen, "The Role of the VTA NMDA Receptors, VTA DA Cells and VTA Terminal Regions in Reward-Related Learning"
(2015). CUNY Academic Works.
http://academicworks.cuny.edu/gc_etds/1005
This Dissertation is brought to you by CUNY Academic Works. It has been accepted for inclusion in All Graduate Works by Year: Dissertations, Theses,
and Capstone Projects by an authorized administrator of CUNY Academic Works. For more information, please contact [email protected].
THE ROLE OF THE VTA NMDA RECEPTORS, VTA DA CELLS AND THE VTA
TERMINAL REGIONS IN REWARD-RELATED LEARNING
by
KAREN KEST
A dissertation submitted to the Graduate Faculty in Psychology in partial fulfillment
of the requirements for the degree of Doctor of Philosophy, The City University of
New York
2015
ii © 2015
Karen Kest
All Rights Reserved
This manuscript has been read and accepted by the
Graduate Faculty in Psychology in satisfaction of the
dissertation requirement for the degree of Doctor of Philosophy
________________ Date ________________ Date Dr. Robert Ranaldi
_____________________________________________ Chair of Examining Committee
Dr. Joshua Brumberg
_____________________________________________ Executive Officer Dr. Richard Bodnar
Dr. Caroline Pytte
Dr. Kenneth Carr
Dr. Jon Horvitz
Supervisory Committee
THE CITY UNIVERSITY OF NEW YORK
iii iv Abstract
THE ROLE OF THE VTA NMDA RECEPTORS, VTA DA CELLS AND THE VTA
TERMINAL REGIONS IN REWARD RELATED LEARNING
by
Karen Kest
Adviser: Robert Ranaldi, Ph.D.
Reward-related learning occurs when an initially neutral stimulus acquires the capacity to
elicit responses similar to an unconditioned stimulus (US) with which it is associated, in
which case the stimulus now functions as a conditioned stimulus (CS). The mechanisms
whereby stimuli come to function as CSs are not fully understood and comprise the
theme of this dissertation. We have previously proposed that coincident signals from an
unconditioned and the eventual conditioned stimulus (US and CS) signals on dopamine
(DA) cells of the ventral tegmental area (VTA) leads to strengthening of CS synapses,
allowing the CS to acquire the ability to activate VTA DA cells on its own and elicit
conditioned approach, thereby functioning as a CS. Furthermore, we proposed that this
type of learning is VTA NMDA receptor dependent. This dissertation was designed to
test this model by specifically testing the following hypotheses: (1) A food US will
activate VTA DA cells; (2) A food-associated CS will activate VTA DA cells and cause
conditioned approach; (3) Blockade of NMDA receptors in the VTA will prevent a foodassociated stimulus from acquiring the capacity to function as a CS (i.e., cause
v conditioned approach) and to cause conditioned activation of the mesocorticolimbic DA
system.
To test the hypothesis that a US, in this case food, activates VTA DA cells, male rats
were maintained at 85% of their free feeding weights for the duration of this study. Rats
were exposed to an eating protocol in which the rats were able to eat or not eat food.
Rat brains were then removed and immunostained for c-Fos followed by tyrosine
hydroxylase (TH) to examine VTA DA cell activation. As expected, rats that ate the food
demonstrated a significantly greater number of VTA DA (TH-labeled) cells expressing cFos than rats that did not receive food.
To test the hypothesis that a CS, in this case a food-associated light, activates VTA DA
cells, male rats were maintained at 85% of their free feeding weights for the duration of
this study. Rats were trained to retrieve a food pellet after a light presentation (the CS)
and then tested for the expression of the food checking response with only CS
presentations. As expected, a light functioning as a CS caused a significantly greater
number of VTA DA cells to express c-Fos than a light not functioning as a CS.
We also hypothesized that VTA NMDA receptor stimulation is necessary for a foodassociated stimulus (CS; also a food-associated light) to elicit conditioned approach via
conditioned activation of VTA DA cell terminal regions (mesocorticolimbic DA terminal
regions). Rats were prepared with indwelling cannulae positioned so as to allow bilateral
microinjections of AP-5 (a NMDA receptor antagonist) in the VTA and were maintained
vi at 85% of their free feeding weights. Male rats were exposed to an acquisition or
expression of learning protocol. Subsequently, all rat brains were processed for c-Fos
labeling to examine activation of mesocorticolimbic DA terminal regions. As expected,
AP-5 significantly impaired the acquisition of conditioned approach and significantly
reduced the amount of c-Fos expressed in the cells in the mesocorticolimbic DA terminal
regions in response to the CS. Also, AP-5 did not impair the expression of the already
learned conditioned approach response.
All together, the results support the model - that a CS acquires, via the VTA NMDA
receptor, the capacity to cause conditioned activation of VTA DA cells and
mesocorticolimbic DA terminal regions therefore eliciting conditioned approach in a
manner similar to a US.
vii ACKNOWLEDGEMNTS
There are many people I’d like to thank for supporting me over the course of my
work that have enabled me to complete this dissertation. I am especially grateful to my
mentor, Dr. Robert Ranaldi. You had confidence in a young undergraduate student that
approached you in your Behavioral Neuroscience class. From then till now, you have
shown an unwavering bolstering in my abilities to comprehend and execute research. In
addition to your constant confidence and support, you have imparted attributes that will
always remain with me – in addition to training me with the necessary procedural skills I
needed to conduct my research you have also trained me to write and think in clear and
focused manner, as well as that patience, attention detail, and hard work are the key to
success. For all this I thank you!
Thank you Dan Hong Chen, Ivonne Cruz, Daniel Lubelski, Jonathan Muller and
Michelle Saliba for your commitment to my doctoral research as undergraduate students.
Your ability to work together, comprehension of the tasks at hand and your attention to
details were contributing factors to my research results.
To my husband Asher and son Avi, thank you for your effortless and endless
support in me. Asher, thank you for your calm words of encouragement and advice at
times I’ve felt overwhelmed.
Lastly, I want to thank the members of my dissertation committee: Drs. Robert
Ranaldi, Richard Bodnar, Caroline Pytte, Kenneth Carr and Jon Horvitz for their
willingness to participate in this dissertation.
viii Table of Contents
List of Figures …………………………………………………………………………x
Chapter 1. ...................................................................................................................... 1
1. What is reward-related learning? ........................................................................... 1
2. The physiology and function of DA and its role in reward-related learning ...... 2
2.a. Physiological features of DA and its receptors .................................................... 2
2.b. Functions of DA ..................................................................................................... 5
2.c. DA and its role in reward-related learning.......................................................... 8
3. A proposed Hebbian Model of Reward-Related Learning ................................. 11
4. Framework of the VTA and its DA projection sites ............................................ 12
5. VTA and reward-related learning......................................................................... 16
6. A more detailed proposed Hebbian Model of reward-related learning in the
VTA .............................................................................................................................. 18
7. Hypothesis................................................................................................................ 20
Chapter 2. .................................................................................................................... 23
1. Introduction: A food-associated CS activates c-Fos in VTA DA neurons and
elicits conditioned approach ...................................................................................... 23
2. Methods.................................................................................................................... 27
2.a. Subjects ................................................................................................................. 27
2.b. Apparatus ............................................................................................................. 28
2.b.i. Conditioning chambers ..................................................................................... 28
2.b.ii. Foot-shock chambers........................................................................................ 28
2.b.iii. Locomotor activity chambers ......................................................................... 28
2.c. Experiments .......................................................................................................... 29
2.c.i. Experiment 1: The effects of a US (food) on c-Fos in VTA TH-labeled cells 29
2.c.ii. Experiment 2: The effects of a CS on c-Fos in VTA TH-labeled cells ......... 29
2.c.iii. Experiment 3: The effects of reward omission on locomotor activity ........ 30
2.d. Immunohistochemistry ....................................................................................... 31
2.e. Data Analysis ........................................................................................................ 32
2.e.i. Experiment 1: The effects of a US (food) on c-Fos in VTA TH-labeled cells 33
2.e.ii. Experiment 2: The effects of a CS on c-Fos in VTA TH-labeled cells ......... 33
2.e.iii. Experiment 3: The effects of reward omission on locomotor activity ........ 33
3. Results ...................................................................................................................... 34
3.a. Experiment 1: The effects of a US (food) on c-Fos in VTA TH-labeled cells . 34
3.b. Experiment 2: The effects of a CS on c-Fos in VTA TH-labeled cells ............ 34
3.c. Experiment 3: The effects of reward omission on locomotor activity ............. 35
4. Discussion ................................................................................................................ 43
Chapter 3. .................................................................................................................... 47
ix 1. Introduction: NMDA receptor stimulation in the VTA is necessary for the
acquisition of conditioned approach and the capacity of a conditioned stimulus to
activate dopamine terminal regions .......................................................................... 47
2. Methods.................................................................................................................... 50
2.a. Subjects ................................................................................................................. 50
2.b. Surgical procedure............................................................................................... 51
2.c. Apparatus ............................................................................................................. 51
2.d. Microinjection procedure ................................................................................... 51
2.e. Drugs ..................................................................................................................... 52
2.f. Conditioning experiments .................................................................................... 52
2.f.i. Conditioning procedure for acquisition study ................................................. 53
2.f.ii. Conditioning procedure for expression study................................................. 54
2.g. Histology and Immunohistochemistry ............................................................... 54
2.g.i. Acquisition study ............................................................................................... 54
2.g.ii. Expression study ............................................................................................... 55
2.h. Data Analysis........................................................................................................ 56
3. Results ...................................................................................................................... 57
3.a. Histology ............................................................................................................... 57
3.b. Acquisition study ................................................................................................. 57
3.c. Expression study .................................................................................................. 58
3.d. Immunohistochemistry ....................................................................................... 59
4. Discussion ................................................................................................................ 68
Chapter 4………………………………………………………………...…………...73
1. General Discussion .................................................................................................. 72
1.a. Results ................................................................................................................... 72
1.b. The role of the VTA ............................................................................................. 73
1.c. Understanding the role of Glu ............................................................................ 75
1.d. DA activation from unconditioned and conditioned stimuli ........................... 76
1.e. The role of VTA NMDA receptors ..................................................................... 76
1.f. A detailed Hebbian model................................................................................... 77
1.g. Possible shortcomings of the proposed Hebbian model in reward-related
learning ........................................................................................................................ 81
1.h. The importance of understanding the neural mechanisms underlying rewardrelated learning ........................................................................................................... 83
References…………………………………………………………………………….84
x List of Figures
Figure 1. An illustration of the proposed Hebbian Model of reward-­‐related learning in the VTA 22 Figure 2. VTA TH-labeled cells expressing c-Fos in the VTA for rats in the US (food
pellets) and no US (no food pellets) groups
36
Figure 3: Representative VTA slices showing double-labeled TH/c-Fos cells in rats
receiving US or no US. Below is an image showing a representative TH/c-Fos labeled
cell (black arrow) and a representative TH-only labeled cell (white arrow) in the VTA 37
Figure 4: CS/Pre-CS food trough head entry ratios for EP and NEP groups during the 3
conditioning and CS-only test sessions
38
Figure 5: VTA TH-labeled cells expressing c-Fos in the VTA for animals in the EP and
NEP groups
39
Figure 6: Representative VTA slices showing double-labeled TH/c-Fos cells in the EP
and NEP groups
40
Figure 7: Locomotor counts for the groups receiving foot shock or no foot shock just
prior to placement in the locomotor activity chambers
41
Figure 8: A. CS/Pre-CS food trough head entry ratios for EP and NEP groups during the
3 conditioning and CS-only test sessions. B. Locomotor counts for the EP and NEP
groups tested immediately after the CS-only test session
42
Figure 9. Histological reconstruction of injection sites adapted from Paxinos and Watson
(1997)
60
Figure 10. Average number of food trough head entries for each group across three
conditioning and one CS-only test session of the acquisition study
62
Figure 11. Average CS/pre-CS ratios for each group across three conditioning and one
CS-only test session of the acquisition study
63
Figure 12. Average number of food trough head entries for each group across seven
conditioning and one CS-only test session of the expression study
64
Figure 13. Average CS/pre-CS ratios for each group across seven conditioning and one
CS-only test session of the expression study
65
xi Figure 14. A. c-Fos counts in the NAcc core and shell, medial and lateral caudate and
frontal cortex in rats completing the acquisition study. B. Average CS/pre-CS ratios for
the two groups mentioned in A for the CS-only test session of the acquisition study
66
Figure 15. Illustrations representing c-Fos in identical portions of the NAcc core in one
rat that received bilateral microinjections of 0.5 µg AP-5, and one that received vehicle,
immediately preceding the CS-only test session in the acquisition study
67
1 Chapter 1.
1. What is reward-related learning?
Reward-related learning is an essential adaptive function for survival in which
organisms acquire behaviors that place them in contact with stimuli that function as
primary rewards (unconditioned stimuli, USs; e.g., food, water), which consequently
elicit approach behaviors and reinforce these behaviors. Neutral stimuli associated with
primary rewards acquire the ability to act as conditioned stimuli (CSs), which can elicit
conditioned approach behaviors similar to the primary rewards with which they are
associated. Involved in this type of learning is the brain’s reward system, the
mesocorticolimbic dopamine (DA) system, which originates in the ventral tegmental area
(VTA) and projects to forebrain regions such as the nucleus accumbens (NAcc),
amygdala (AMG) and prefrontal cortex (PFC). These forebrain regions will be referred to
as the DA terminal regions. The neural mechanisms underlying this type of rewardrelated learning in this system are not fully understood, particularly how neutral stimuli
become CSs. One possibility is that a conditioned stimulus (CS), by virtue of its
contiguity with an unconditioned stimulus (US), acquires the capacity to activate the
same neural substrates activated by the US and thus elicit conditioned approach behaviors
similar to the US (Ranaldi, 2014; Zellner & Ranaldi, 2010). The present experiments
were designed to test a particular neurobiological model focused on the VTA and the DA
terminal regions for this type of learning.
2 2. The physiology and function of DA and its role in reward-related learning
2.a. Physiological features of DA and its receptors
There is a large body of evidence implicating DA in reward-related learning (see
(Beninger, 1989; Beninger & Miller, 1998; Wise, 2006; Wise & RomprÇ, 1989; Wise &
Rompre, 1989; Zellner & Ranaldi, 2010) for reviews). Hence, a mention of the complex
physiology of DA is merited. DA is a neurotransmitter in the catecholamine family that
cannot cross the blood brain barrier so it is synthesized from cells within the central
nervous system. DA is then transported from cytosol into synaptic vesicles by vesicular
monoamine transporter 2 (VMAT2). DA remains in the vesicle until an action potential
occurs and forces the vesicles to merge with the cell membrane, thus spilling DA into the
synaptic cleft. Once in the synaptic cleft, DA binds to and activates postsynaptic DA
receptors. DA can also bind to presynaptic DA receptors which can either excite or
inhibit the DA neurons depending on their electrical potential; thus presynaptic DA
receptors act as autoreceptors which result in the inhibition of DA neurotransmitter
synthesis and release to help normalize DA levels.
DA neurons are small, unmyelinated neurons that ascend within the
mesocorticolimbic DA system. There are several dopaminergic pathways in the central
nervous system involved in reward. First is the mesocorticolimbic DA system, in which
DA neurons located in the VTA (A10) ascend to the DA terminal regions. DA within this
system has been shown to affect neurons depending on their current neurophysiological
state. For instance, DA can maintain and prolong a cell’s excited states (Lavin et al.,
2005; Lewis & O'Donnell, 2000; O'Donnell & Grace, 1995). To exemplify this, Lavin
and her colleagues (Lavin, et al., 2005) found that post synaptic potentials in the mPFC
from VTA stimulation are not mediated by mPFC DA receptors, but rather initiated by
3 mPFC Glu receptors and maintained by mPFC DA receptors. Second is the nigrostriatal
system that arises in the substantia nigra pars compacta (A9) and ascends to the
neostriatum (caudate and putamen), part of the basal ganglia (BG) – a region involved in
movement.
Beninger and colleagues (Beninger, D'Amico, & Ranaldi, 1993)
demonstrated this system’s involvement in food-reinforced operant responding by
injecting trained rats with a DA antagonist in the striatum which resulted in decreased
food-reinforced lever pressing. For the purpose of staying focused, only the
mesocorticolimbic DA system will be discussed here.
Levels of extracellular DA are modulated by tonic and phasic DA transmission.
Tonic DA transmission occurs when small amounts of DA are released impartially of
neuronal firing. In contrast, phasic DA transmission, or rapid burst firing, results from the
firing of DA neurons (Grace & Bunney, 1984). Concentrated bursts result in greater
amounts of extracellular DA. The amount of DA signal in the mesocorticolimbic DA
terminal regions, time-locked to reward events, has been suggested to be a necessary
input for the synaptic plasticity in this neuronal system that leads to reward-related
learning (Beninger, 1983; Beninger & Ranaldi, 1994; Kelley, 1999). In behaving
primates, DA neurons of the NAcc display short-latency, phasic firing to primary reward
and conditioned cues associated with reward. In rats, transient DA release in terminal
regions that mimics that seen during burst firing has been demonstrated during
presentation of reward-related cues. Taken together, these studies suggest that phasic
dopamine release is a critical mediator of reward-related processes (Heien & Wightman,
2006).
DA binds to and activates postsynaptic DA receptors to mediate its physiological
4 effects on the target. DA can have long-term effects on neurons by playing a role in
synaptic plasticity (Beninger & Miller, 1998; Jay, 2003). DA can act on stimulatory or
inhibitory metabotropic DA receptor families (Niznik & van Tol, 1992) (all DA receptors
are metabotropic). Activation of the stimulatory DA receptor families, receptors D1 and
D5, results in a long-term molecular change in a neuron by triggering a conformational
change in the DA receptor and therefore exposing a binding site for a G-protein. The
newly formed G-protein coupled receptor (GPCR) causes displacement of a guanine
diphosphate (GDP) with guanine triphosphate (GTP) on its alpha subunit, causing this
alpha subunit to dissociate and, in the presence of adenosine triphosphate (ATP), form
adenylyl cyclase (AC). AC converts ATP into cyclic adenosine monophosphate (cAMP;
a second messenger). In the presence of cAMP the alpha subunit, along with GTP,
becomes protein kinase A (PKA) (cAMP binds to the regulatory subunit of PKA
allowing its catalytic subunit to become active), which can translocate to the nucleus and
phosphorylates cAMP response element binding protein (Creb) 1. Creb2 (which when
bound to a gene represses it) is bound to a cAMP response element (CRE; a DNA
promotor region). A phosphorylated Creb1 displaces Creb2, causing activation of CRE
that results in the expression of neighboring genes, in other words gene transcription.
This pathway has been implicated in activity-dependent long-term sensitization in the
Aplysia (Kandel, 2001) as well as activity-dependent plasticity involved in behavioral
classical conditioning (Antonov, Antonova, Kandel, & Hawkins, 2001) supporting the
idea that synaptic plasticity is a mechanism of learning and memory more generally.
Conversely, activation of the inhibitory DA receptor families, receptors D2, D3
and D4, inhibits AC therefore preventing long-term molecular changes. For instance, D3
5 receptor mutant mice exhibit potentiated acquisition of conditioned place preference with
cocaine compared to wild-type mice (Kong, Kuang, Li, & Xu, 2011). Activation of CaM
kinase 2 (CaMKII; a calcium sensitive serine/therine protien kinase that when activated
can phosphorylate and thus regulate Glu receptors (Barria, Muller, Derkach, Griffith, &
Soderling, 1997)) in the NAcc, AMG and PFC is also potentiated in D3 receptor mutant
mice compared to that in wild-type mice following conditioned place preference
expression. These results support the notion that inhibitory DA receptors modulate
reward-related learning induced by cocaine by inhibiting the activation of CaMKII in the
brain reward circuit. DA can also have short-term effects on neurons which are usually
modulatory (Greengard, 2001) and tend to increase post-synaptic excitability via
coupling to the D1 family of receptors, which result in a cascade of processes leading to
increased expression of glutamate (Glu) receptors.
2.b. Functions of DA
The heterogeneity of effects of DA – short v. long-term consequences, the brain
region in question, the relative concentration of receptor subtypes and the current state of
affected neurons – accounts for the fact that a number of competing views on the function
of DA exist. These perspectives include the following: (1) The anhedonia hypothesis,
which emphasizes DA’s role in basic reward. Wise (Wise, Spindler, deWit, & Gerber,
1978) provided support for this hypothesis by blocking DA receptors with pimozide
which resulted in blunting the rewarding effects of food. (2) The learning and
reinforcement hypothesis, which emphasizes the idea that DA plays an important role in
establishing and reinforcing food-seeking behaviors (Beninger, 1983; Wise, 2006). (3)
The effort hypothesis, which emphasizes that the effort required in obtaining a reward is
6 DA dependent. Salamone and his colleagues (Salamone, Correa, Farrar, & Mingote,
2007) provided support for this hypothesis by blocking DA in the NAcc during foodseeking behaviors that were critically dependent upon the work requirements of the task –
lever pressing schedules that have minimal work requirements are largely unaffected by
blocking DA in the NAcc, whereas reinforcement schedules that have high work
requirements are substantially impaired by blocking DA in the NAcc. (4) The behavioral
switching hypothesis, which emphasizes that short latency firing in the mesencephalic
dopaminergic neurons in response to neutral stimuli that are behaviorally significant, is
DA dependent (Nicola, 2007; Redgrave, Prescott, & Gurney, 1999). Behavioral
switching may underlie associative learning in that the initial dopaminergic neuron firing
could represent an essential component in the process of switching behavioral selections
to unexpected, behaviorally important stimuli. (5) The reward prediction error
hypothesis, which emphasizes that reward prediction error is conveyed via DA itself
(Ljungberg, Apicella, & Schultz, 1992; Schultz, 1998). Reward prediction error is the
phasic response of DA neurons observed when an unexpected reward is presented.
Specifically, Montague and colleagues (Montague, Dayan, & Sejnowski, 1996) provided
support for this hypothesis by demonstrating that the release of DA from the VTA cells
into cortical targets conveys information about prediction errors between the expected
amount of reward and the actual reward. Under this postulate, increases in DA release
indicate that the current state is better than expected whereas decreases indicate the
current state is worse than expected. These responses transfer to the onset of a CS after
repeated pairings with the reward. (6) The incentive-salience hypothesis, which
emphasizes the role of DA in reward “wanting” as opposed to “liking” (Berridge, 2007).
7 Incentive salience is a motivational wanting attribute given by the brain to rewardpredicting stimuli. This “wanting” is unlike “liking” in that “liking” is a pleasure
immediately gained from consumption or other contact with rewarding stimuli, while the
“wanting” is a motivational state aimed at a salient incentive stimulus that makes it a
desirable goal and thus transforming it from a sensory experience into something that
commands
attention,
and
therefore
elicits
approach
behavior.
For
instance,
electrophysiological studies conducted in animals suggest that DA neurons are activated
in response to CSs predictive of food reward to a greater extent than when animals
actually eat the food (Naranjo, Tremblay, & Busto, 2001). DA plays a role in salience of
rewarding stimuli in that it assists in attaining the most rewarding stimuli to the one
concerned.
Each of these perspectives encompasses a particular aspect of DA function, and a
comprehensive account of DA’s function may involve one or a combination of these
perspectives. For instance, Schultz and his colleagues (Schultz, Tremblay, & Hollerman,
2000) recorded DA neurons situated in the mesocorticolimbic DA system during rewardrelated tasks and demonstrated that primates respond to specific aspects of reward
behavior such as during the detection of a reward, reward predicting cues (i.e. CSs of
predictive food reward) and reward expectation period. Berridge and Robinson (Berridge
& Robinson, 1998) argued that activation of midbrain DA pathways attributes incentive
salience to and approach toward CSs, and that the CS continues to reinforce incentive
salience and approach as long as the reward is present. Nicola and Deadwyler (Nicola &
Deadwyler, 2000) reported that neurons of the NAcc fired between, but not during, selfadministrated cocaine infusions thus supporting DA’s involvement in motivation for
8 cocaine and therefore ruling out DA’s involvement in the “liking” associated with
rewarding stimuli. Altogether, these findings suggested that DA in the mesocorticolimbic
system plays a role in reward-related learning, and that there may be discrepancies in the
particular aspects of reward it mediates. This dissertation proposes that DA plays several
roles in reward-related learning (see the next section, 2.c.)
encompassing different
aspects on the aforementioned perspectives.
2.c. DA and its role in reward-related learning
Many studies have demonstrated that DA released from within the
mesocorticolimbic DA system is strongly implicated in reward-related behaviors. In an
early animal study Olds and Milner found that if animals are given the choice between
food and direct electrical stimulation of various regions of the mesocorticolimbic DA
system, animals preferred the direct stimulation and consequently starved (Olds &
Milner, 1954). Later animal studies suggest that DA strengthens and/or reinforces
associations between rewarding stimuli and responses (Kelley, 1999; White & Milner,
1992). More specifically, it has been well established that mesocorticolimbic DA plays a
critical role in mediating the behavioral and neural effects of primary rewards (Robinson
& Berridge, 1993; Wise, 2004; Wise, et al., 1978). Consumption of primary rewards is
associated with VTA DA cell activation (Kest, Cruz, Chen, Galaj, & Ranaldi, 2012;
Yoshida et al., 1992), the release of DA in terminal regions of the mesocorticolimbic DA
system (J. J. Day, Roitman, Wightman, & Carelli, 2007; Hernandez & Hoebel, 1988;
Pfaus, Damsma, Wenkstern, & Fibiger, 1995; Yoshida, et al., 1992) and increased
approach behavior (Ikemoto, 2007). Blockade of DA neurotransmission in cortical,
striatal and midbrain regions reduces or eliminates the rewarding effects of USs such as
9 food (Wise, 1978), brain stimulation and drugs of abuse (for a detailed review see (Wise,
2004)). Thus it appears that mesocorticolimbic DA mediates the effects of primary
rewards.
In addition to primary rewards, the mesocorticolimbic DA system also plays an
important role in mediating the behavioral and neural effects of CSs. Definitively, a foodassociated CS caused increased behavioral approach and elevated levels of VTA DA cell
activation (Kest, et al., 2012; A. G. Phillips, Atkinson, Blackburn, & Blaha, 1993) and
elevated levels of DA in the NAcc and caudate nucleus in rats (A. G. Phillips, et al.,
1993). Furthermore, presentation of CSs is associated with neural activity in the VTA
and/or DA release in the DA terminal regions (see, for example, (Blackburn & Phillips,
1989; Carelli, 2000; Carelli & Ijames, 2001; Childress et al., 1999; Gratton & Wise,
1994; Janak, Chang, & Woodward, 1999; Talmi, Seymour, Dayan, & Dolan, 2008).
Neuronal activity in the VTA, the source of mesolimbic DA, is also correlated with the
presentations of CSs, and indeed may be a necessary condition for responding to CSs (Di
Ciano & Everitt, 2004; Murschall & Hauber, 2006). This CS-induced DA activity, in the
VTA and the mesocorticolimbic DA terminal regions, is associated with increased
approach behaviors; increases in mesolimbic DA can reinstate extinguished lever
pressing (Ranaldi, Pocock, Zereik, & Wise, 1999; Stewart, 1984) and are observed just
prior to reinforced lever presses (Gratton & Wise, 1994; Kiyatkin & Gratton, 1994;
Richardson & Gratton, 1996).
Thus, it appears that, in reward-related learning,
mesocorticolimbic DA activity not only mediates the effects of primary rewards but also
comes under the control of CSs.
10 We are particularly interested in the activation of approach behaviors associated
with mesocorticolimbic DA activity. There is evidence that mesocorticolimbic
stimulation results in increased forward locomotion, seeking and appetitive behaviors,
and that approach behavior is linked with mesocorticolimbic cell activity (Beninger,
1983; Berridge, 2007; Berridge & Robinson, 1998; Mogenson, 1987; Panksepp, 1998;
Phillipson, 1979; Wise & Bozarth, 1987). Therefore, it appears that mesocorticolimbic
DA plays a key role in the incentive-motivational effects of primary rewards, increasing
approach and seeking (Ljungberg, et al., 1992; P. E. Phillips, Stuber, Heien, Wightman,
& Carelli, 2003; Romo & Schultz, 1990; Yun, Wakabayashi, Fields, & Nicola, 2004),
and therefore maximizing the animals proximity to rewards.
In light of the heterogeneous effects of DA, we argue that activation of the
mesocorticolimbic DA neurons enhances current motivated behavior to reward-related
stimuli and facilitates associative processes underlying reward-related learning.
Therefore, we believe that DA probably plays a role in eliciting approach behaviors
several times in a sequence during reward-related learning. First, DA release from the
VTA during the encounter with primary rewards initially spurs increased investigatory
and approach behavior. Second, synaptic changes in the VTA allow neutral stimuli
associated with the US to additionally recruit DA cells, augmenting DA release and
approach behavior, putting the animal in greater proximity to rewards and related cues.
Third, DA release at the terminal regions due to USs and CSs facilitate associative
processes that underlie this type of learning. Once CS associations have been acquired,
either primary rewards or CSs can continue to activate VTA cells, providing for DA
release that will maintain approach behaviors and motivational processes. This suggests
11 that activation of DA pathways within the mesocorticolimbic DA system, in response to
USs and CSs, will elicit approach behavior.
3. A proposed Hebbian Model of Reward-Related Learning
Several neural models of reward-related learning have been developed, most of which
are based on the Hebbian postulate that when a synapse is active at the same time the
postsynaptic neuron is active the synapse changes in strength. Furthermore, when two
synapses (for example, one associated with a US and another with a CS) are active at the
same time the post-synaptic neuron is active, the convergence of the two synaptic events
induces neuronal plasticity (Hebb, 1949; Kandel, 2001). We propose here, and others
have elsewhere (Beninger, 1983; Kelley, 1999; Ranaldi, 2014; Testa et al., 2005; Zellner
& Ranaldi, 2010), that reward-related learning involves similar kinds of neural processes.
Furthermore, we propose that the VTA, the source of mesocorticolimbic DA neurons
which project to terminal regions such as the NAcc, AMG and mPFC, may constitute a
candidate node for where this kind of neural plasticity occurs; the VTA is strongly
implicated in reward, receives neuronal afferents that may signal USs and CSs, and has
been shown to undergo long term potentiation (LTP) (Bonci & Malenka, 1999; Overton,
Richards, Berry, & Clark, 1999) - a form of synaptic plasticity and a putative neural
mechanism of learning (Kandel, 2001).
It has been suggested that CSs gain access to the same neural circuits, plausibly the
mesocorticolimbic DA system, as primary rewards and therefore come to activate these
circuits on their own (Bindra, 1974; Ranaldi & Beninger, 1994; Wise, 2004). Others and
we have suggested that one component of this process by 3which CSs gain access to
these circuits is by way of synaptic plasticity in the VTA (Bonci & Malenka, 1999;
12 Harris, Wimmer, Byrne, & Aston-Jones, 2004; Sharf & Ranaldi, 2006a; Stuber et al.,
2008; Zellner, Kest, & Ranaldi, 2009a). One possibility is that a CS, by virtue of its
contiguity with a US, acquires the capacity to activate VTA DA cells and hence the DA
terminal regions—the same neural substrate activated by the US—, therefore causing
conditioned approach and other reward-related behavior similar to the US. We propose a
neurobiological model of reward-related learning where the VTA constitutes a primary
site where such neural plasticity occurs (Kest, et al., 2012; Ranaldi et al., 2011; Sharf &
Ranaldi, 2006a; Zellner & Ranaldi, 2010). In this model the VTA receives signals about
both USs and eventual CSs. These signals converge on VTA DA neurons leading to the
acquisition by the CS of the capacity to activate DA neurons, and hence DA terminal
regions, on its own and elicit conditioned approach.
4. Framework of the VTA and its DA projection sites
The mesocorticolimbic DA system begins with the VTA. The VTA is a group of
neurons located medial to the substantia nigra, ventral to the red nucleus and between the
caudal hypothalamus and brainstem reticular formation. There are different types of
neurons located in the VTA which can be divided into distinct subpopulations which
participate in the various dopaminergic pathways that mediate the pleasurable effects and
the regulation and control of behavior in reward-related learning. The functions of these
subpopulations are determined by the neurotransmitter they receive from VTA afferents
and those released by VTA efferents. The neurons located in the VTA include DA
(Margolis et al., 2006; Swanson, 1982), GABA (Carr & Sesack, 2000), and Glu (Lavin,
et al., 2005). The VTA is partially populated (50%) with dopaminergic neurons
(Margolis, et al., 2006); DA neurons are the “principal” cells of the VTA. The GABA
13 neurons are the “secondary” cells of the VTA (Johnson & North, 1992). Both DA and
GABA neurons project within the VTA itself and outside the VTA (Carr & Sesack,
2000). Recent evidence demonstrates the presence of Glu neurons in the VTA that are
non-dopaminergic and non-GABAergic, and may therefore constitute a third type of
VTA neuron (Yamaguchi, Sheen, & Morales, 2007). The VTA releases DA in the
terminal regions of the mesocorticolimbic DA system.
The VTA receives a large number of inputs from various brain regions. There are
glutamatergic inputs from the mPFC (Christie, Bridge, James, & Beart, 1985) (Sesack &
Pickel, 1992), lateral hypothalamus (LH) (Fields, Hjelmstead, Margolis, & Nicola, 2007),
bed nucleus of the stria terminalis (Georges & Aston-Jones, 2002), and the superior
colliculus (Geisler & Zahm, 2005). It receives additional glutamatergic, and GABAergic
information (Cornwall & Phillipson, 1988; Oakman, Faris, Kerr, Cozzari, & Hartman,
1995) from the pedunculopontine tegmental nucleus (PPTg), AMG, and the posterior
lateral dorsal tegmental nucleus (LDT) (Fibiger, Phillips, & Brown, 1992). GABAergic
inputs to the VTA also arise from the ventral pallidum (Geisler & Zahm, 2005) and the
NAcc (Conrad & Pfaff, 1976). There are also cholinergic afferents from the PPTg and
LDT (Fibiger, et al., 1992), serotonergic projections from the dorsal raphe nucleus as
well as inputs from the pontine, cerebellar and medullary nuclei (Geisler & Zahm, 2005)
and AMG (Wallace, Magnuson, & Gray, 1992).
The diverse afferents to the VTA release a variety of neurotransmitters to specific
targeted subpopulations of VTA neurons. The glutamatergic afferents from the PFC,
raphe nucleus and LH target the subpopulation of VTA DA neurons which project back
to the PFC (Fields, et al., 2007). Additional inputs to the VTA DA neurons include
14 cholinergic afferents (Omelchenko & Sesack, 2006). Furthermore, the LDT has
glutamatergic and GABAergic afferents to the VTA which synapse onto dopaminergic
and GABAergic VTA neurons which also project back to the PFC (Fields, et al., 2007).
These diverse inputs mediate complex synaptic actions within the VTA which in turn
determine the type of outputs leaving the VTA.
Efferents of the VTA project to targeted brain regions, which receive inputs from
a distinct subpopulation of VTA neurons. Dopaminergic fibers, which receive input from
the PFC, LH, PPTg and LTD, target the NAcc (Floresco, West, Ash, Moore, & Grace,
2003), lateral septal area, AMG (Bjorklund & Dunnett, 2007) - specifically to the
basolateral AMG (BLA) (Fallon & Ciofi, 1992; Howland, Taepavarapruk, & Phillips,
2002), mPFC (Fuxe & et al., 1974; Swanson, 1982) via the mediodorsal thalamus, and
the hippocampus. There are glutamatergic (Lavin, et al., 2005) and GABAergic (Carr &
Sesack, 2000; Margolis, et al., 2006) efferents, that receive input from the LTD, which
project to the NAcc and PFC (Wise, 2004) . It is within these multifaceted inputs and
outputs from subsets of VTA neurons that separate circuits are shaped and formed which
help form the underlying mechanisms of reward-related learning.
Each DA terminal region has two things in common: (1) they all receive
information from the VTA and (2) they all send information back to the VTA. Efferents
of the NAcc are GABAergic and project back to the VTA and the ventral pallidum (VP;
(Heimer, Zaborszky, Zahm, & Alheid, 1987)). Mogenson and his colleagues (Mogenson,
Jones, & Yim, 1980) proposed that the NAcc projections to the VP translate mesolimbic
motivation signals into motor output, a neural circuit which may contribute to approach
behaviors. The inputs and outputs of the NAcc could play an important role in behaving
15 animals to adapt to different environmental conditions. The BLA sends Glu fibers to the
mPFC and NAcc (Spooren, Veening, Groenewegen, & Cools, 1991). Activation of the
NAcc, from the BLA, results in activation of the VP, which then initiates motor systems
involved particularly in goal-directed behaviors – an underlying theme in reward-related
learning. Also, Floresco and colleagues (Floresco, Yang, Phillips, & Blaha, 1998)
suggest that the glutamatergic BLA inputs to NAcc dopamine terminals synaptically
facilitate or depress dopamine efflux, and these effects are independent of DA neuronal
firing activity. Moreover, they also imply that changes in NAcc DA levels following
presentation of reward-related stimuli may be mediated, in part, by the BLA. The
glutamatergic fibers of the central nuclues of the AMG connect back to the VTA
providing feedback to its DA projection site, perhaps enhancing the effects of the
connections from the VTA to the BLA. The mPFC sends its glutamatergic efferents
(from the pyramidal cells in the pyramidal layer) back to the VTA, the NAcc (Sesack,
Deutch, Roth, & Bunney, 1989; Wright & Groenewegen, 1995) and the BLA forming a
closed neural loop in the mesocorticolimbic DA system. Furthermore, stimulation of the
mPFC results in Glu and DA release in the NAcc; Glu release may be directly from the
mPFC pyramidal cells to the VTA and then to the NAcc (Rossetti & Wise, 1995), and the
DA release may be indirect through the VTA . The role of the mPFC in this neural loop
may involve the formation of the stimulus-reward associations, specifically in higher
order information processing such as extracting and retaining the “associative
significance” of a stimulus independent of its physical properties (Watanabe, 1990). The
inputs and outputs of the mesocorticolimbic DA system, as well as the subpopulations of
nuclei, are linked together and allow for communication amongst many brain regions.
16 This interconnection allows for the modification, or in other words “synaptic plasticity”
including the strengthening and weakening, of incoming and outgoing signals that
underlie reward-related learning.
5. VTA and reward-related learning
There is evidence that the VTA receives signals about USs. The VTA receives
acetylcholine (ACh) afferents from PPTg and LDT (Bolton, Cornwall, & Phillipson,
1993; Garzon, Vaughan, Uhl, Kuhar, & Pickel, 1999; Henderson & Sherriff, 1991;
Oakman, et al., 1995). In the VTA, extracellular concentrations of ACh increase during
eating, drinking and self-stimulation of the lateral hypothalamus (Rada, Mark, Yeomans,
& Hoebel, 2000). In addition, injections of muscarinic ACh (mACh) receptor antagonists
in the VTA reduce eating both in our laboratory (Sharf, McKelvey, & Ranaldi, 2006;
Sharf & Ranaldi, 2006a, 2006b) and in others (Rada, et al., 2000) and reduces approach
and consummatory responses for food (Ikemoto & Panksepp, 1996). In-vivo (Gronier &
Rasmussen, 1998) and in-vitro (Calabresi, Lacey, & North, 1989) studies show that
stimulation of mACh receptors in the VTA depolarizes presumed DA neurons and
releases DA in terminal regions of the mesocorticolimbic system (A. D. Miller & Blaha,
2005; Westerink, Enrico, Feimann, & De Vries, 1998). These findings support the idea
that VTA mACh receptor stimulation is involved in mediating some of the unconditioned
(i.e., rewarding or incentive motivational) effects of primary rewards (USs), including
food, through activation of VTA DA neurons.
It may be presumed that there is a CS signal to the VTA mediated by
glutamatergic afferents originating in the hindbrain and the forebrain (Geisler, Derst,
Veh, & Zahm, 2007), including from regions known to process environmental stimuli
17 such as PFC, AMG, bed nucleus of the stria terminalis, and superior colliculus (Sesack &
Pickel, 1992; Smith, Charara, & Parent, 1996). It is possible, then, that the VTA receives
a Glu signal that is in some way related to reward-associated environmental stimuli and
which might be involved in the acquisition of reward-related learning. For instance,
convergence of signals from reward-associated stimuli and USs on VTA DA neurons
may lead to LTP-like neural plasticity, resulting in strengthening of CS-related synapses
causing conditioned activation of DA neurons and conditioned reward responding.
Indeed, some evidence of this exists. Microinjections of NMDA receptor antagonists in
the VTA block the development of morphine conditioned place preference (Harris, et al.,
2004), the acquisition of a food-reinforced operant response (Zellner, Kest, & Ranaldi,
2009b) and the acquisition of a conditioned approach response (Ranaldi, et al., 2011;
Stuber, et al., 2008).
This convergent stimulation creates the conditions for LTP which is in many
cases dependent on NMDA receptor stimulation (Citri & Malenka, 2007). NMDA
receptors are found in the VTA (Rodriguez, Doherty, & Pickel, 2008) and LTP has been
demonstrated in the VTA DA neurons where it is NMDA receptor dependent (Bonci &
Malenka, 1999). NMDA receptor stimulation can result in initiation of second messenger
cascades implicated in LTP and other intracellular events that lead to neurochemical and
structural changes associated with learning. Such synaptic changes could result in what
may initially be a weak glutamate signal (i.e., one that is not able to robustly activate
VTA DA neurons) becoming one that is able to robustly activate DA cells and causing
behaviorally adequate DA release. This increased DA release, triggered by strengthened
CS-related synapses in the VTA, might lead to increased approach behaviors elicited by
18 CSs, increasing the likelihood of direct contact with rewards or processes downstream in
DA terminal regions which will facilitate other regions involved in reward-related
behaviors.
6. A more detailed proposed Hebbian Model of reward-related learning in the VTA
Our proposed Hebbian model of synaptic plasticity in the acquisition of rewardrelated learning involves activity dependent changes in the strength of synaptic
connections among US and CS sets of neurons, brought about by the conjoint activity of
these neurons on DA neurons of the VTA. Specifically, we propose that when animals
consume a reward in a given environment, US-stimulated ACh release activates VTA DA
cells, leading to increased DA in mesocorticolimbic terminal regions (A. D. Miller &
Blaha, 2005; Schilstrom, Svensson, Svensson, & Nomikos, 1998; Westerink, et al.,
1998), which stimulates increased approach behavior (Beninger, 1983; Berridge, 2007;
Mogenson, 1987; Wise & Bozarth, 1987). As the VTA DA cells are stimulated by the
ACh signal (i.e., the US), they also receive relatively weak excitation from Glu afferents
conveying information about environmental stimuli. This coincident stimulation of ACh
and Glu receptors on VTA DA neurons induces LTP-like processes in VTA DA neurons,
resulting in the previously weak synapses becoming strong enough to activate the VTA
DA cells in their own right. These strengthened synapses now allow reward-associated
stimuli to function as CSs, increasing the levels of approach toward rewards. Up to date,
much research indicates that VTA DA activity comes under the control of CSs and
suggests that this activity is a necessary component of the associative processes
underlying reward-related learning.
19 On a physiological level, we propose that a US (food) stimulates ACh release
which activates mACh receptors at the level of the VTA resulting in strong
depolarization of VTA DA neurons; depolarization of the VTA DA cells removes a
magnesium ion (Wu & Johnson, 1996) blocking the NMDA receptor channel. Removal
of the magnesium block, allows for calcium to flow through the NMDA receptor in
response to CS-stimulated Glu release at the level of the VTA leading to the initiation of
calcium-dependent second messenger cascades. Activation of the NMDA receptors allow
for an influx of intracellular calcium. High levels of calcium allow itself to bind to
calmodulin (a calcium binding protein) thus forming CaM which can activate CaM
dependent kinases such as CamKII (a calcium sensitive serine/threonine protein kinase) –
an enzyme which causes autophosphorylation and/or phosphorylation and is crucial for
LTP that underlies reward-related learning (Lisman, Schulman, & Cline, 2002). CamKII,
in addition to PKA, regulates (phosphorylates) Creb, which results in gene transcription
(see Figure 1 depicted from (Ranaldi, 2014)). In addition to NMDA receptor activation,
mACh receptor activation also results in a long-term molecular change in VTA DA cells.
Ach can trigger a conformational change in the mACh receptor therefore exposing a
binding site for a G-protein. The newly formed GPCR causes displacement of a GDP
with a GTP on its alpha subunit, activating phosphoinositide (PI) located in the cell
membrane. PI forms phosphoinositide phosphate (PIP) and in the presence of protein
kinase C (PKC), PIP is broken down into inositol triphosphate (IP3) and diglyceride
(DAG); IP3 increases the amount of calcium channels in the endoplasmic reticulum and
DAG is IP3 dependent – it stimulates PKC when there are high levels of Ca therefore
enhancing LTP and the strengthening of future synaptic events. Thus, if US (mACh
20 receptor) and CS (NMDA receptors) stimulation at the level of the VTA occurs
simultaneously, then the originally weak excitations from Glu afferents (presumably
mediating reward-associated stimulus information) may become strengthened.
7. Hypothesis
According to the Hebbian model coincident US (from ACh stimulation) and CS
(from Glu stimulation) signals on VTA DA cells are proposed to lead to CSs to acquire
the ability to activate VTA DA cells and corticolimbic DA terminal regions on their own
and elicit conditioned approach, in a manner similar to the US. Furthermore, this type of
learning has been proposed to be dependent on VTA NMDA receptors. To test this model
of reward-related learning, we hypothesized that (1) a food US will activate VTA DA
cells, (2) a food-associated CS (light) will acquire the capacity to activate VTA DA cells
and cause conditioned approach, and (3) that AP-5 (2-amino-5-phosphonopentanoic acid;
a competitive NMDA receptor antagonist) in the VTA will impair the acquisition, but not
the expression, of conditioned approach on a behavioral level and the expression of c-Fos
in DA terminal regions on a neural level. c-Fos is an amino-acid protein encoded by an
immediate early gene which allows for increased expression of that particular gene. The
expression of c-Fos has been shown to be an indication of neuronal activity (Dragunow &
Faull, 1989), and specifically recent neuronal activity (H. E. Day, Kryskow, Nyhuis,
Herlihy, & Campeau, 2008). Since c-Fos indicates recent neuronal activity, we will
measure cellular activity by labeling and then quantifying c-Fos.
To test the first hypothesis, rats were exposed to a food study in which we
hypothesized that rats consuming the food pellets will display a significantly greater
number of VTA DA (tyrosine hydroxylase (TH)-labeled) cells expressing c-Fos than rats
that are not presented with food pellets. To test the second hypothesis, rats were exposed
21 to a conditioned-approach study in which we hypothesized that a light functioning as a
CS will cause a significantly greater number of VTA DA cells to express c-Fos than the
same light not functioning as a CS. To test the third hypothesis, rats were surgically
prepared with indwelling cannulae positioned so as to allow bilateral microinjections of
AP-5 or vehicle in the VTA. Rats were then exposed to either an acquisition or an
expression study in which we hypothesized that AP-5 will significantly impair the
acquisition of conditioned approach and significantly reduce the amount of c-Fos
expressed in the mesocorticolimbic DA terminal regions in response to the CS relative to
vehicle treatment. It was also expected that AP-5 would not impair the expression of the
already learned conditioned approach response. Altogether our expected results support
our model of reward-related learning, namely that a CS acquires, via the VTA NMDA
receptor, the capacity to cause conditioned activation of VTA DA cells and
mesocorticolimbic DA terminal regions therefore eliciting conditioned approach.
22 Figure 1: An illustration of the proposed Hebbian Model of reward-related learning in
the VTA. Shown are the components of the proposed neural mechanisms necessary for a
neutral stimulus the ability to acquire the capacity to activate a VTA DA neuron on thier
own therefore acting as a conditioned stimulus that can cause a conditioned approach
response. US, mAch receptor stimulation allows for activation of VTA DA neurons and
then activation of the DA terminal regions that cause approach. CS, NMDA receptor
stimulation, allows for the conditioned activation of VTA DA neurons and then
conditioned activation of the DA terminal regions that cause conditioned approach. When
there is coincident mAch and NMDA receptors stimulation in the VTA DA neurons,
calcium flowing through the NMDA receptor initiates intracellular cascades resulting in
long-term changes in neural activity. It is proposed that these changes in neural activity
allow CS the ability to acquire the capacity to activate VTA DA neurons on their own
and elicit conditioned approach. Image was prepared and displayed by Dr. Robert
Ranaldi (Ranaldi, 2014).
23 Chapter 2.
1. Introduction: A food-associated CS activates c-Fos in VTA DA neurons and elicits
conditioned approach
Reward-related learning is an essential adaptive function for survival in which
organisms acquire behaviors that place them in contact with stimuli that function as
primary rewards (unconditioned stimuli, USs; e.g., food, water) – which consequently
elicit approach behaviors and reinforce these behaviors. Neutral stimuli associated with
primary rewards acquire the ability to act as conditioned stimuli (CSs), which can elicit
conditioned approach responses similar to the primary rewards with which they are
associated. The neural mechanisms underlying this type of reward-related learning are
not fully understood.
We propose that conditioned approach learning involves the
acquisition by CSs of the capacity to cause conditioned activation of the same neural
systems activated by primary rewards, thereby eliciting the conditioned response (CR).
The present experiments were designed to test this model.
Several neural models of reward-related learning have been developed, most of
which are based on the Hebbian postulate that when a synapse is active at the same time
the postsynaptic neuron is active the synapse changes in strength. Furthermore, when two
synapses (for example, one activated by a US and another by a stimulus with a
contiguous relation with the US) are active at the same time the postsynaptic neuron is
active, the convergence of the two synaptic events induces neuronal plasticity (Hebb,
1949; Kandel, 2001). We propose here, as others have elsewhere (Beninger & Hahn,
1983; Beninger & Ranaldi, 1994; Kelley, 1999; Zellner & Ranaldi, 2010) that rewardrelated learning involves similar kinds of neural processes. Furthermore, we propose that
the ventral tegmental area (VTA), the source of mesocorticolimbic dopamine (DA)
24 neurons, may constitute a candidate node for where this kind of neural plasticity occurs;
the VTA, as detailed below, is strongly implicated in reward, receives neuronal afferents
that may signal USs and CSs, and has been shown to undergo long term potentiation
(LTP) (Bonci & Malenka, 1999; Overton, et al., 1999) - a form of neuronal plasticity and
a putative neural mechanism of learning.
Mesocorticolimbic DA plays a critical role in mediating the behavioral effects of
primary rewards (Robinson & Berridge, 1993; Wise, 2004). It is well established that
consumption of primary rewards is associated with the release of DA in terminal regions
of the mesocorticolimbic DA system (J. J. Day, et al., 2007; Hernandez & Hoebel, 1988;
Pfaus, et al., 1995; Yoshida, et al., 1992) and increased approach behavior (Ikemoto,
2007). Blockade of DA neurotransmission in cortical, striatal and midbrain regions
reduces or eliminates the rewarding effects of USs such as food, brain stimulation and
drugs of abuse (for a detailed review see (Wise, 2004; Zellner & Ranaldi, 2010). Thus it
appears that mesocorticolimbic DA mediates the effects of primary rewards.
In addition to primary rewards, the mesocorticolimbic DA system also plays an
important role in mediating the behavioral effects of CSs (Wise, 2004; Zellner & Ranaldi,
2010). Phillips and colleagues (A. G. Phillips, et al., 1993) demonstrated that a foodassociated CS caused increased behavioral approach and elevated levels of DA in the
NAcc and caudate nucleus in rats. This CS-induced DA activity is associated with
increased approach behaviors; increases in mesolimbic DA can reinstate extinguished
lever pressing (Ranaldi, et al., 1999; Stewart, 1984) and are observed just prior to
reinforced lever presses (Gratton & Wise, 1994; Kiyatkin & Gratton, 1994; Richardson &
Gratton, 1996). Thus, it appears that, in reward-related learning, mesocorticolimbic DA
25 activity comes under the control of CSs. Precisely how this happens remains to be
determined. One possibility is that a CS, by virtue of its contiguity with a US, acquires
the capacity to activate VTA DA neurons – the same neural substrate activated by the US
–and thus causes conditioned approach and other reward-related behavior similar to the
US. We propose a neurobiological model of reward-related learning where the VTA
constitutes a primary site where such neural plasticity occurs (Sharf & Ranaldi, 2006a;
Zellner & Ranaldi, 2010). In this model the VTA receives signals about both USs and
eventual CSs that converge on DA neurons leading to the acquisition by the CS of the
capacity to activate DA neurons on its own and elicit conditioned approach (the CR).
There is evidence that the VTA receives signals about USs. The VTA receives
acetylcholine (ACh) afferents from the pedunculopontine tegmental nucleus (PPTg) and
the posterior lateral dorsal tegmental nucleus (LDT) (Bolton, et al., 1993; Garzon, et al.,
1999; Henderson & Sherriff, 1991; Oakman, et al., 1995). In the VTA, extracellular
concentrations of ACh increase during eating, drinking and self-stimulation of the lateral
hypothalamus (Rada, et al., 2000). In addition, injections of muscarinic ACh (mACh)
receptor antagonists in the VTA reduce eating both in our laboratory (Sharf, et al., 2006;
Sharf & Ranaldi, 2006a, 2006b) and others (Rada, et al., 2000) and reduces approach and
consummatory responses for food (Ikemoto & Panksepp, 1996). In-vivo (Gronier &
Rasmussen, 1998) and in-vitro (Calabresi, et al., 1989) studies show that stimulation of
mACh receptors in the VTA depolarizes presumed DA neurons and releases DA in
terminal regions of the mesocorticolimbic system (A. D. Miller & Blaha, 2005;
Westerink, et al., 1998). These findings support the hypothesis that VTA mACh receptor
26 stimulation is involved in mediating some of the unconditioned (i.e., rewarding or
incentive motivational) effects of primary rewards (USs), including food.
There is also evidence of a putative CS signal in the VTA mediated by
glutamatergic afferents originating in the hindbrain and the forebrain (Geisler, et al.,
2007), including from regions known to process environmental stimuli such as frontal
cortical areas, amygdala, bed nucleus of the stria terminalis, superior colliculus and
others (Sesack & Pickel, 1992; Smith, et al., 1996). It is possible, then, that the VTA
receives glutamate (Glu) signals that are in some way related to reward-associated
environmental stimuli and which might be involved in the acquisition of reward-related
learning. Indeed, some evidence of this exists.
Microinjections of NMDA receptor
antagonists in the VTA block the development of morphine conditioned place preference
(Harris, et al., 2004), the acquisition of a food-reinforced operant response (Zellner, et al.,
2009b) and the acquisition of a conditioned approach response (Ranaldi, et al., 2011;
Stuber, et al., 2008).
Given the role of mesocorticolimbic DA in reward and the evidence that the VTA
receives US- and CS-related signals, we propose that the VTA is a crucial site for neural
plasticity underlying reward-related learning.
In this model, conditioned approach
learning occurs when a reward-associated stimulus acquires the capacity to activate VTA
DA neurons and elicit conditioned approach (the CR). If this model is correct then a US
(e.g., food) should activate VTA DA neurons and a CS (e.g., food-paired light) should
acquire the capacity to activate VTA DA neurons and elicit conditioned approach.
Recent data from Sombers and colleagues (Sombers, Beyene, Carelli, & Wightman,
2009) is in accordance with this prediction. These researchers found that a cue that
27 signaled availability of brain stimulation reward (BSR) activated VTA DA cells and
initiated self-stimulation.
This demonstrates that a tonic (contextual) cue that is
predictive of artificial reward (i.e., BSR) ‘availability’ causes conditioned activation of
VTA DA neurons. In the present experiments we tested whether or not a phasic, discrete
cue associated with natural reward (food) and which elicits conditioned responses per se,
can activate VTA DA cells. Specifically, we hypothesized that a food-associated CS
(light) would acquire the capacity to elicit conditioned approach and activate VTA DA
neurons. To quantify DA cell activation we used immunohistochemical procedures that
labeled neurons with tyrosine hydroxylase (TH) to identify them as DA neurons and
stained for c-Fos to identify activated neurons; cells labeled with both TH and c-Fos
represented activated DA cells.
2. Methods
The protocols used in the present experiments were in accordance with the
National Institutes of Health Guide for Care and Use of Laboratory Animals and were
approved by the Queens College Institutional Animal Care and Use Committee.
2.a. Subjects
Subjects consisted of 68 male Long Evans rats, facility-bred from males and
females obtained from Charles River Laboratories (Raleigh, NC), with initial free-feeding
weights between 300 and 400 grams. All rats were individually housed and maintained
on a 12 h light:12 h dark cycle (lights off at 0600). All experimental sessions were
conducted during the dark phase in order to test the rats during their active periods. All
rats had unlimited access to food (Purina rat chow) until experimental sessions started, at
28 which time access was restricted to daily rations that maintained their weights at 85% of
their free-feeding values.
2.b. Apparatus
2.b.i. Conditioning chambers
All behavioral testing was conducted in eight conditioning chambers each
measuring 30 x 21 x 18 cm (l x w x h). One wall was equipped with a food trough and
two white stimulus lights, each situated one inch above and two inches to the right or left
of the food trough. Each chamber was housed in a ventilated, sound-attenuating box. A
PC through a MED Associates interface controlled the chambers.
2.b.ii. Foot-shock chambers
All foot-shock stimuli were administered in 8 chambers each measuring 30 x 21 x
18 cm (l x w x h). These chambers were equipped with aluminum rod floors. Each rod
was connected to a constant-current aversive stimulator. Each chamber was housed in a
ventilated, sound-attenuating box. A PC through a MED Associates interface controlled
the foot-shock administration.
2.b.iii. Locomotor activity chambers
Locomotor activity was measured in 6 open field activity monitors, each
measuring 44.5 x 44.5 x 30.5 cm (l x w x h). Each chamber was equipped with 16 photoemitters/photocells along the length and 16 along the width, all 4 cm above the floor.
Locomotor activity counts were registered by the photocells when adjacent beams were
broken consecutively.
29 2.c. Experiments
Starting one day after beginning the food restriction diet each rat was given 20
food pellets (45 mg, Bioserv, Frenchtown, NJ) in its home cage on each of three days.
All rats were then tested in either a feeding (Experiment 1), a conditioned approach
learning (Experiment 2) or a stress experiment (Experiment 3). The feeding experiment
tested whether or not food (the US) can activate c-Fos in VTA DA [tyrosine-hydroxylase
(TH)-labeled] cells. The conditioned approach experiment tested whether or not a foodassociated CS (a 3-s light presentation) can cause conditioned approach and activation of
c-Fos in VTA TH-labeled cells.
The stress experiment was an additional control
experiment testing the possibility that reward omission may cause stress, measured as
suppressed locomotor activity.
2.c.i. Experiment 1: The effects of a US (food) on c-Fos in VTA TH-labeled cells
Rats were exposed to a protocol that consisted of 4 daily 10 min sessions in test
chambers. For sessions 1-3 all rats were habituated to the test chambers with no US.
During session 4, half the rats were given 30 food pellets in a dish to eat in the test
chamber while the other half was given an empty dish (US vs. no US). Rat brains were
then harvested and immunostained for c-Fos followed by TH.
2.c.ii. Experiment 2: The effects of a CS on c-Fos in VTA TH-labeled cells
Rats were exposed to a protocol that consisted of 6 daily sessions. During session
1 all rats were given 20 min of magazine training in which 20 food pellets were delivered
on a random time schedule. For sessions 2-4 rats were randomly assigned to one of two
30 groups, US-CS explicitly paired (EP) or US-CS not explicitly paired (NEP) groups. Each
of these sessions was 60 min long. During these conditioning sessions, 30 food pellets
were delivered on a random time 120-s schedule (range of 15 to 245 s). For rats in the
EP group, each pellet delivery was preceded by a 3-s presentation of a light on the left
side of the trough. For rats in the NEP group the light presentations (on left side of the
trough) and food deliveries were not correlated. After the 3 conditioning sessions all rats
received one 30-min session during which there were no light or food presentations. This
was followed by a CS-only test session in which rats were presented with light
presentations under the same random time schedule as in conditioning but with no food.
Rat brains were then harvested and immunostained for c-Fos followed by TH. For all
rats head entries during each session were counted and analyzed (see Data Analysis
section below for details).
2.c.iii. Experiment 3: The effects of reward omission on locomotor activity
Four groups of rats were used in this experiment. Two groups of rats were
exposed to a protocol that consisted of 4 daily, 1-h sessions in the shock chambers each
followed by 30 min sessions in the locomotor activity chambers. For sessions 1-3 all rats
were habituated to the test chambers with no shock. After the third locomotor activity
session, all rats were assigned to a shock or a no-shock group with group assignment
matched to their locomotor activity levels during session 3. During session 4, the rats in
the shock group received randomly delivered foot-shocks (0.8 mA; 1 s on; mean off
period of 120 s) in the shock chambers while the no-shock group did not. All rats were
then placed in the locomotor activity chambers for 30 min and locomotor activity was
measured.
31 The other two groups were exposed to the same conditioned approach protocol
described in Experiment 2 with the exception of the following (one group was exposed to
the EP and the other to the NEP protocol). First, for 3 days prior to the conditioned
approach procedure, rats were habituated to the locomotor activity chambers for 30 min
each day. Second, immediately after the CS-only test session, rats were placed in the
locomotor chambers for 30 min and locomotor activity was measured.
2.d. Immunohistochemistry
Seventy-five minutes after the end of the last session in Experiments 1 and 2 all
rats were anesthetized with sodium pentobarbital in preparation for perfusion. While
under deep anesthesia the rats were perfused through the heart first with 0.9% saline
followed by a phosphate-buffered (0.1 M) fixative containing 4% paraformaldehyde.
Brains were removed from the skull and fixed with 4% paraformaldehyde overnight at 4°
C and sectioned through the ventral tegmental area in the coronal plane on a vibratome.
Free-floating
sections
(40
µm)
were
collected
into
different
wells
for
immunocytochemistry. Sections were first washed with phosphate-buffered saline (PBS)
and then blocked in 5% NGS and 0.2% Triton X-100 for 1 h. Sections were then
incubated with primary antibodies (rabbit anti-CFOS 1:5000, Calbiochem) in 0.1% Triton
X-100, 2.5% NGS, and PBS at 4° C overnight. Sections were rinsed several times with
2.5% NGS in PBS and then incubated in biotinylated secondary antibodies (biotinylated
goat anti-rabbit; 1:200, Vector Labs, Burlingame, California) for 2 h at room temperature.
Sections were rinsed several times with PBS and incubated for 1 h in an avidinhorseradish peroxidase mixture (Vector Labs, Burlingame, California). Sections were
rinsed in PBS and then reacted with nickel chloride and 0.05% diaminobenzidine (DAB)
32 in the presence of 0.0015% H2O2. This procedure was completed twice, first for c-Fos
and then for TH using a TH primary antibody (TH anti-TH). Sections were collected
onto gelatin-coated slides, dried for several hours, and coverslipped with Cytoseal.
2.e. Data Analysis
Neural activity data (from Experiments 1 and 2) consisted of c-Fos counts in VTA
TH-labeled cells. To quantify VTA TH-labeled cells expressing c-Fos we identified from
Paxinos and Watson (1986) brain slices for the anterior (plate 37), medial (plate 39) and
posterior (plate 41) planes of the VTA that were common to all rats. VTA TH-labeled
cells expressing c-Fos were counted on an Olympus (Tokyo, Japan) BX51W microscope
with a motorized stage at 4x magnification.
Stereo Investigator software
(MicroBrightField, Williston, VT) was used to outline regions of the VTA. The numbers
of VTA TH-labeled cells expressing c-Fos in each VTA region were added to arrive at a
VTA total count for each rat. To prevent bias that may skew the results, all counting was
conducted blind to the treatment groups.
Conditioned approach data (from Experiments 2 and 3) consisted of the number
of food trough head entries made during (1) a 6-s period immediately preceding the onset
of the CS (pre-CS period), (2) the 3-s period during the CS, (3) a 3-s period immediately
following the offset of the CS and (4) at all other times (non-CS period). For analyses the
3-s periods during and immediately following the CS were combined for a 6-s total
period and referred to as the CS period. For each session the total number of head entries
during the CS periods and the total number of head entries during the pre-CS periods
were used to calculate the CS/pre-CS ratio. This ratio indicates the magnitude of the
33 conditioned approach response (i.e., the degree to which food trough head entries were
elicited by the CS).
Locomotor activity data (Experiment 3) consisted of locomotor activity counts
(consecutive beam breaks) during 30 min sessions in the locomotor chambers.
2.e.i. Experiment 1: The effects of a US (food) on c-Fos in VTA TH-labeled cells
A one-way analysis of variance (ANOVA) with group (US and no US) as the
factor was used to analyze the amounts of VTA TH-labeled cells expressing c-Fos in the
groups.
2.e.ii. Experiment 2: The effects of a CS on c-Fos in VTA TH-labeled cells
A mixed-design ANOVA, with group (EP and NEP) as a between groups factor
and session as a repeated measures factor was conducted on the CS/pre-CS ratio data
from the 3 conditioning sessions (sessions 2-4). A one-way, between groups ANOVA
was conducted on the CS/pre-CS ratio data from the CS-only test session.
A one-way ANOVA, with group (EP and NEP) as the factor was conducted to
analyze differences in the amount of VTA TH-labeled cells expressing c-Fos between
groups.
2.e.iii. Experiment 3: The effects of reward omission on locomotor activity
Foot shock groups: An independent samples, one-tailed t-test was used to
compare the locomotor activity counts between the shock and no-shock groups.
Conditioned approach/locomotor activity tested groups: A one-way, between
groups ANOVA was conducted on the CS/pre-CS ratio data from the CS-only test
34 session. An independent samples, one-tailed t-test was used to compare the locomotor
activity counts between the EP and NEP groups.
Criterion for significance was p < 0.05 in all cases.
3. Results
3.a. Experiment 1: The effects of a US (food) on c-Fos in VTA TH-labeled cells
The US group (i.e., rats who ate the food pellets) demonstrated a significantly
greater number of VTA TH-labeled cells expressing c-Fos than the no US group (i.e., rats
that did not eat food) (see Figure 2). A one-way ANOVA revealed a significant group
effect (F(1,14) 12.819, p < 0.003).
Figure 3 contains representative images of VTA slices, in identical regions of the
VTA, showing double-labeled TH/c-Fos cells in rats receiving US or no US as well as a
representative image of a double-labeled TH/c-Fos cell and a TH labeled cell in the VTA.
3.b. Experiment 2: The effects of a CS on c-Fos in VTA TH-labeled cells
The EP group (i.e., rats who had the US and CS explicitly paired) demonstrated
significantly more food trough head entries during the CS presentations than during no
CS presentations, as indicated by significantly greater CS/pre-CS ratios, than the NEP
group (i.e, rats who did not have the US and CS explicitly paired) during the 3
conditioning sessions (see Figure 4). A two way ANOVA with group and session
(repeated measures) as factors showed significant group (F(1,18)=22.574,p<0.0005),
session (F(1,16)=4.984,p<0.012), and session X group (F(2,36)=5.10,p<0.009) effects. Tests
of simple effect of session in each group revealed that the EP group showed a significant
(F(1,36)=10.032,p=.01) increase in the CS/pre-CS ratio across the 3 conditioning sessions
35 whereas the NEP group did not. The EP group also demonstrated a significantly greater
CS/pre-CS ratio than the NEP group during the CS-only test session (see Figure 4). A
one-way ANOVA showed a significant group effect (F(1,18)=4.288,p<0.05).
The EP group demonstrated a greater number of VTA TH-labeled cells
expressing c-Fos than the NEP group during the CS-only test session (see Figure 5). A
one-way ANOVA showed a significant group (F(1,18)=22.460,p<0.001) effect.
Figure 6 depicts representative images of VTA slices, in identical regions of the
VTA, showing double-labeled TH/c-Fos cells in rats from the EP and NEP groups.
3.c. Experiment 3: The effects of reward omission on locomotor activity
The group that received intermittent foot shock prior to the locomotor activity test
demonstrated significantly less locomotor activity than the group that did not receive foot
shock prior to this test (t(14) = -1.756, p < 0.05) (see Figure 7). Of the conditioned
approach groups that were later tested in locomotor activity the EP group demonstrated
significantly more food trough head entries during the CS presentations than when no CS
was present, indicated by significantly greater CS/pre-CS ratios, than the NEP group
during the CS-only test session (see Figure 8A). A one-way ANOVA revealed a
significant group effect (F(1, 14) = 30.10, p < 0.001). Immediately following this CS-only
test was the locomotor activity test, which showed virtually identical amounts of
locomotor activity in the EP and NEP groups (see Figure 8B; an independent samples ttest revealed no significant effect between the groups).
36 Number of TH/c-Fos labeled cells
2500
*
2000
US
no US
1500
1000
500
0
Group
Figure 2: Mean (±SEM) counts of VTA TH-labeled cells expressing c-Fos in the VTA
for rats in the US (food pellets) and no US (no food pellets) groups. * represents a
significant group difference
37 Figure 3: Representative VTA slices showing double-labeled TH/c-Fos cells in rats
receiving US and no US. Below is an image showing a representative TH/c-Fos labeled
cell (black arrow) and a representative TH-only labeled cell (white arrow) in the VTA.
These representative images were taken at 4x magnification that was used to quantify cell
counts.
38 Figure 4: Mean (±SEM) CS/Pre-CS food trough head entry ratios for EP and NEP
groups during the 3 conditioning and CS-only test sessions. * represents a significant
group difference.
Number of TH/c-Fos labeled cells
400
39 *
300
EP
NEP
200
100
0
Group
Figure 5: Mean (±SEM) counts of VTA TH-labeled cells expressing c-Fos in the VTA
for animals in the EP and NEP groups. * represents a significant group difference.
40 Figure 6: Representative VTA slices showing double-labeled TH/c-Fos cells in the EP
and NEP groups. These representative images were taken at 4x magnification that was
used to quantify cell counts.
41 1200
Locomotor counts
1000
*
800
no shock
shock
600
400
200
0
Group
Figure 7: Mean (±SEM) locomotor counts for the groups receiving foot shock or no foot
shock just prior to placement in the locomotor activity chambers. * represents a
significant group difference.
42 A
B
8
EP
NEP
1600
*
1400
Locomotor counts
6
CS/pre-CS ratio
EP
NEP
1800
4
2
1200
1000
800
600
400
200
0
0
1
2
3
Session
Test
Group
Figure 8: A. Mean (±SEM) CS/Pre-CS food trough head entry ratios for EP and NEP
groups during the 3 conditioning and CS-only test sessions. * represents a significant
group difference. B. Mean (±SEM) locomotor counts for the EP and NEP groups tested
immediately after the CS-only test
session.
43 4. Discussion
The data from Experiment 1 demonstrated a greater number of VTA TH-labeled
cells expressing c-Fos in animals that ate food than in animals that did not. These data
strongly suggest that food, a US, activates VTA DA neurons. The data from Experiment
2 demonstrated that a stimulus explicitly paired (EP) with the US (food reward) produced
a greater number of VTA-TH labeled cells expressing c-Fos and greater food trough head
entries (the conditioned approach response) than a stimulus not explicitly paired (NEP)
with the US. In Experiment 3, the fact that the EP and NEP groups showed identical
levels of locomotor activity rules out the possibility that the greater VTA DA cell
activation observed in the EP group of Experiment 2 was due to group differences in
reward omission-induced stress. Altogether, these results strongly suggest that a foodassociated CS acquires the capacity to cause conditioned activation of VTA DA neurons.
Although this experiment did not directly test the causal relation between conditioned
activation of VTA DA neurons and conditioned approach learning, the finding that
significantly more VTA DA cell activation was observed when the light stimulus served
a behavioral CS function than when it didn’t suggests such a causal relation. Overall, the
present results provide support for our neurobiological model of reward-related learning
(Zellner and Ranaldi, 2010; (Ranaldi, 2014).
An alternative explanation for the significant amount of VTA TH-labeled cells
expressing c-Fos in Experiment 2 is that this expression is due to the light itself, apart
from its role as a CS. In our conditioned approach learning experiment, the US-CS EP
and US-CS NEP groups received the same number of light presentations. If c-Fos
expression in these regions was due simply to light presentations then we should have
observed equal amounts of c-Fos in the EP and NEP groups. The fact that we observed
44 significantly more c-Fos activation in the EP group argues that it is the CS attribute of the
light stimulus, not the sensory effects per se, that is responsible for the significant
enhancement of c-Fos activation in the EP group.
Studies have shown that the presentation of a US or CS can translate into synaptic
overflow of DA into the terminal regions of the mesocorticolimbic DA system (Fiorino,
Coury, Fibiger, & Phillips, 1993); (Nicola, Taha, Kim, & Fields, 2005); (Sombers, et al.,
2009), however, the manner in which this DA release is caused is unclear. It may result
from DA cell activation from signals in the VTA, or from presynaptic stimulation of DA
terminals. The current study suggests that conditioned approach occurs through signal
activation of DA cells, at the level of the cell bodies, in the VTA. We proposed this
signal is glutamatergic in nature (Sharf, et al., 2006; Sharf & Ranaldi, 2006a; Zellner &
Ranaldi, 2010). If this is true then one would expect reduced responding maintained by
reward-associated stimuli and reduced dopamine release in the mesolimbic DA terminal
regions after blockade of glutamate signaling in the VTA and in fact this is what Sombers
and colleagues found (Sombers, et al., 2009). These findings suggest a causal link
between a CS’s capacity to cause activation of VTA DA cells to elicit conditioned
approach. Furthermore, these findings suggest that unconditioned and conditioned food
reward use similar neural mechanisms in mediating their effects on learned and motivated
behavior.
In reward-related learning, natural and drug rewards trigger approach behaviors
primarily because they elevate mesolimbic DA levels (Wise, 2004). Under conditions of
DA receptor blockade, rewards such as food (Wise, et al., 1978), brain stimulation
(Fouriezos, Hansson, & Wise, 1978) and psychomotor stimulants like amphetamine
45 (Yokel & Wise, 1975) and cocaine (de Wit & Wise, 1977) fail to maintain responding in
trained animals. We hypothesized that the capacity of CSs to function as such also
requires that they elevate mesolimbic DA levels and we have proposed (Sharf & Ranaldi,
2006a; Zellner & Ranaldi, 2010) that their capacity to stimulate DA neurons depends on
Glu neurotransmission in the VTA. For instance, You et al (You, Wang, Zitzman, Azari,
& Wise, 2007) have demonstrated that exposure to a cocaine self-administration context
causes Glu release in the VTA and that this CS-induced glutamatergic signal is necessary
for cocaine seeking. It is also well-established that reward predictive stimuli cause burst
firing of VTA DA neurons (Schultz, 1997). And although the mechanisms through
which this happens have not been definitively established they likely involve Glu
neurotransmission. Several brain regions, including the prefrontal cortex, amygdala, the
midbrain tegmental nuclei, and the bed nucleus of the stria terminalis project directly to
the VTA (Georges & Aston-Jones, 2002; Harris, et al., 2004). Most of these inputs are
glutamatergic and excite VTA DA neurons (Georges & Aston-Jones, 2002; Takahata &
Moghaddam, 1998) suggesting that one or more of these signals may play a necessary
role in CS-induced mesolimbic DA activation in reward-related learning.
In summary, we tested the hypothesis that a phasic, discrete food-associated CS
can acquire the capacity to cause conditioned activation of VTA DA neurons. We found
that consumption of food (presentation of a US) was associated with significant
elevations in VTA TH-labeled cells expressing c-Fos. We also found that animals show
a significantly greater amount of VTA TH-labeled cells expressing c-Fos in response to a
food-associated light stimulus that served a behavioral CS function than in response to a
46 light stimulus that did not serve such a function. These results support our hypothesis
that conditioned activation of VTA DA cells underlies the capacity of CSs to elicit CRs.
47 Chapter 3.
1. Introduction: NMDA receptor stimulation in the VTA is necessary for the
acquisition of conditioned approach and the capacity of a conditioned stimulus to
activate dopamine terminal regions
The mesocorticolimbic dopamine (DA) system plays an important role in
mediating the behavioral and neural effects of unconditioned stimuli (USs; Berridge and
Robinson, 1998; Wise, 2004). Rats approaching food-related stimuli demonstrate
elevated nucleus accumbens (NAcc) DA levels whereas rats not approaching do not
(Bassereo et al., 2007). Furthermore, DA depletion in the NAcc, caudate and frontal
cortex result in diminished rewarding effects of natural (Beninger and Ranaldi, 1993;
Aberman et al., 1998) and drug rewards (Hiroi and White, 1991; McGregor and Roberts,
1993 and 1995). In addition to primary rewards, the mesocorticolimbic DA system also
plays an important role in mediating the behavioral and neural effects of conditioned
stimuli (CSs; Phillips et al., 1993). AP-5 in the NAcc core blocks the acquisition of
conditioned approach for food reward (Di Ciano et al., 2001). Presentations of CSs are
associated with neural activity or DA release in terminal regions of the mesocorticolimbic
DA system, including the NAcc and prefrontal cortex (PFC; Blackburn and Phillips,
1989; Carelli, 2000, 2002; Gratton and Wise, 1994; Talmi et al., 2008). Depletions of DA
using 6-hydroxydopamine lesions in the NAcc core impair Pavlovian approach to a
conditioned stimulus (Parkinson, et al., 1999). Accordingly, it appears that, in rewardrelated learning mesocorticolimbic DA activity comes under the control of CSs allowing
for the activation of DA terminal regions and the elicitation of a conditioned approach
response. Precisely how this happens remains to be determined. One possibility is that
48 neural plasticity in the ventral tegmental area (VTA), the source of midbrain DA in the
mesocorticolimbic DA system, may be involved.
Several neural models of the associative properties of reward-related learning
have been developed, most of which are based on the Hebbian postulate that when a
synapse is active at the same time the postsynaptic neuron is active the synapse changes
in strength. Furthermore, when two synapses (an unconditioned stimulus [US] and
conditioned stimulus [CS]) are active at the same time a post synaptic neuron is active,
the convergence of the two synaptic events induces neuronal plasticity (Hebb, 1949). We
propose here, as we and others have elsewhere (Beninger, 1983; Beninger and Ranaldi
1993), that reward-related learning involves similar kinds of neural processes. Synaptic
plasticity, such as long term potentiation (LTP), in the VTA may underlie the neural
activity that develops during the formation of associations between rewards and neutral
stimuli (see Zellner et al., 2009 and Zellner and Ranaldi, 2010 for review). Studies have
found that LTP is one of the main mechanisms of learning (Kandel, 2001) and that it is
VTA NMDA receptor dependant (Bonci and Malenka, 1999; Stuber et al., 2008). For the
VTA to be considered a possible site where CS-US associations may be formed it must
be a site where convergence of afferents carrying US signals and afferents carrying CS
signals occurs. It appears that the VTA has these types of afferents in the form of
acetylcholine (possible US signals) and glutamate (possible CS signals) afferents.
The VTA has acetylcholine (ACh) secreting terminals of neurons originating in
the pedunculopontine nucleus (PPN) and laterodorsal tegmental nuclei (LDT; Bolton,
1993; Garzon, 1999). In the VTA, extracellular concentrations of ACh increase during
eating, drinking and self-stimulation of the lateral hypothalamus (Rada, 2000). In vitro
49 studies show that stimulation of metabotropic ACh receptors (mAchR) in the VTA
induces a membrane depolarization in the DA neurons (Kalivas, 1993) and DA release in
terminal regions of the mesolimbic system (Marchi, 1991). In addition, mACh receptor
antagonists in the VTA reduce eating both in our laboratory (Sharf and Ranaldi, 2006)
and others (Rada, 2000) and approach and consummatory responses for food (Ikemoto,
1996). These findings suggest that VTA mACh receptor stimulation is involved in
mediating some of the US signals about rewards, including food.
The VTA also receives glutamate (Glu) projections from structures ranging from
the brainstem to the forebrain (Geisler et al., 2007), including the PPN (Charara 1996)
and medial PFC (Smith 1996). In the VTA, extracellular concentrations of Glu increase
in response to environmental stimuli. For instance, microinjections of NMDA and AMPA
receptor antagonists delivered simultaneously into the VTA blocks the development of
cocaine place preference (Harris and Aston-Jones, 2003) and, when administered
individually, blocks the development of morphine place preference (Harris et al., 2004).
Accordingly, it appears that Glu activation in the VTA is involved in drug-related
learning and may therefore also be involved in reward-related learning. The VTA may
receive a Glu signal that is related to environmental stimuli, possibly mediating the
effects of CSs, and which might be involved in the acquisition of reward-related learning.
Given this confluence of signals, of both ACh and Glu stimulation in the VTA,
and the kinds of information these signals transmit, it is reasonable to suppose that these
signals are integrated in some way when learning about reward-producing responses,
such as conditioned approach responses. According to the Hebbian model of associative
learning the convergence of ACh (the US) and Glu (the CS) at the level of the VTA can
50 lead to the CS acquiring the ability to induce DA release and thus activation of regions
downstream of the VTA, the DA terminal regions, and consequently to stimulate
approach. Therefore, we hypothesized that NMDA receptor stimulation in the VTA is
necessary for the acquisition, but not the expression, of a CS to elicit conditioned
approach and conditioned activation of DA terminal regions. In particular, we predicted
that AP-5 in the VTA would impair the acquisition, but not the expression, of conditioned
approach on a behavioral level and the expression of c-fos in DA terminal regions on a
neural level.
2. Methods
The protocols used in the present experiments were in accordance with the
National Institutes of Health Guide for Care and Use of Laboratory Animals and were
approved by the Queens College Institutional Animal Care and Use Committee.
2.a. Subjects
Subjects consisted of 65 male Long Evans rats, facility-bred from males and
females obtained from Charles River Laboratories (Raleigh, NC), with initial free-feeding
weights between 375 and 450 grams at the time of surgery. All rats were individually
housed and maintained on a 12 h light:12 h dark cycle (lights off at 6 AM). All
experimental sessions were conducted during the dark phase in order to test the rats
during their active periods. All rats had unlimited access to food (Purina rat chow) until
experimental sessions began, at which time access was restricted to daily rations that
maintained their weights at 85% of their free-feeding values.
51 2.b. Surgical procedure
All rats received an intraperitoneal (IP) injection of atropine sulfate (0.1 ml) and
were anesthetized by sodium pentobarbital (65 mg/kg). Stainless steel guide cannulae
(0.635 mm outer diameter, 0.3302 mm inner diameter) were bilaterally implanted to a
depth that allowed for microinjections into the VTA using the following coordinates: -5.6
mm caudal to bregma, ±2.0 mm from the midline at a 10° angle toward the midline and 8.3 mm below the surface of the skull (Paxinos and Watson, 1986). The cannulae were
fixed in dental acrylic anchored to the skull by four stainless steel screws. Obturators
(0.3048 mm diameter), extending 1 mm beyond the tip of the cannulae, were inserted at
all times except during microinjections.
2.c. Apparatus
All behavioral testing was conducted in eight conditioning chambers each
measuring 30 x 21 x 18 cm (l x w x h). One wall was equipped with a food trough and
two white stimulus lights, each situated one inch above and two inches to the right or left
of the food trough. Each chamber was housed in a ventilated, sound-attenuating box. The
chambers were controlled by a PC through a MED Associates interface.
2.d. Microinjection procedure
Immediately prior to the appropriate sessions the obturator was removed from one
of the guide cannulae and a stainless steel injector tube was inserted to extend 1 mm
beyond the end of the guide cannula. The injector was connected by polyethylene tubing
to a 10 µl Hamilton syringe (Reno, NV) preloaded with vehicle or AP-5. The compound
was delivered manually over a 30-s period and the injector was kept in place for an
52 additional 60 seconds before being removed and the obturator replaced. This procedure
was repeated on the contralateral side after which the animal was placed in the
conditioning chamber and the session started.
2.e. Drugs
AP-5 (Sigma-Aldrich, St. Louis, MO) was dissolved in 0.9% saline before the
start of the experiments. Each microinjection was delivered in a volume of 0.5 µl at a
dose of 0.25 or 0.5 µg/ 0.5 µl which was chosen based on our previous experiments
showing that the high dose of AP-5 significantly attenuated the acquisition of foodreinforced operant responding without affecting food motivation or food reward value
(Zeller et al., 2009).
2.f. Conditioning experiments
One week after surgery rats began the food restriction diet to reduce their weights
to 85% of their free-feeding values where they were maintained for the duration of the
experiments. At least one day after food restriction began all rats were given 20 food
pellets (45 mg, Bioserv, Frenchtown, NJ) in their home cages on each of three days. All
rats were then tested in one of two versions of a conditioned approach paradigm. The
first version, the acquisition study, was used to investigate the effects of treatment on the
acquisition of conditioned approach (behavioral level) and activity in the DA terminal
regions (neural level). The second version, the expression study, was used to investigate
the effects of treatment on the expression of a learned conditioned approach.
53 2.f.i. Conditioning procedure for acquisition study
Rats were given one 20-min magazine training session in the conditioning
chambers in which 20 food pellets were delivered on a random time schedule, to allow
rats to become acquainted with magazine delivery of food pellets. Rats were then
randomly assigned to groups receiving bilateral intra-VTA microinjections of AP-5 or
vehicle immediately before each of three daily, consecutively held 60-min conditioning
sessions. During conditioning sessions, 30 food pellets were delivered on a random time
120-s schedule (range of 15 to 245 s). In a explicitly-paired (EP) group each pellet
delivery was preceded by a 3-s presentation of a light on the left side of the food trough.
In a non-explicitly paired (NEP) group the light presentations and food deliveries were
not correlated to each other. After the 3 conditioning sessions with intra-VTA treatment,
all rats were exposed to one 30-min extinction session with no treatment during which no
light or food presentations were programmed. The extinction session represents goaldirected behavior and is indicative of remembered reward. This was followed by a 60min CS-only test session in which rats were presented with light presentations under the
same random time schedule as in conditioning but with no further consequences (no
food). All rats received intra-VTA vehicle injections prior to the CS-only test session.
For all rats the number of head entries during each session was counted and analyzed (see
Data analysis section for details). After the last session the rats were sacrificed and their
brains were extracted and prepared for histological cannula verification and some for
immunohistochemistry.
54 2.f.ii. Conditioning procedure for expression study
This procedure was similar to the one described in the acquisition test but with the
following differences. The number of conditioning sessions was 7 (to ensure a learned
response), no microinjections were made prior to the conditioning sessions and
microinjections of AP-5 or vehicle were made prior to the CS-only test session. No
brains were prepared for immunohistochemistry.
2.g. Histology and Immunohistochemistry
2.g.i. Acquisition study
Immediately after the end of the last session rats analyzed on a behavioral level
were anesthetized with an overdose of sodium pentobarbital, perfused with 0.9% saline
followed by 4% formalin, and decapitated. The brains were removed and stored in 4%
formalin for at least seven days before being sectioned in the coronal plane on a cryostat,
stained with cresyl violet and inspected for cannulae implantations and injection sites. All
rats included in the data analysis had verified cannulae placements in VTA.
Seventy five minutes after the end of the last session rats analyzed on a neural
level were anesthetized with an overdose of sodium pentobarbital, perfused with a
phosphate-buffered saline (PBS; 0.1 M) followed by a fixative containing 4%
paraformaldehyde, and decapitated. The brains were removed and stored in 4%
paraformaldehyde overnight at 4°C and sectioned through the VTA, NAcc core and shell,
caudate, and frontal cortex in the coronal plane on a vibratome. Free-floating sections (40
µm) were collected into different wells for immunocytochemistry. Sections were first
rinsed 3 times in PBS for 10 min each. For use with conventional light microscopy
sections were incubated in 0.5% hydrogen peroxide (H2O2) and methanol in PBS for 20
55 min. Sections were rinsed 3 times in PBS for 10 min each and then blocked in 5% normal
goat serum (NGS) and 0.2% Triton X-100 in PBS for 1 h. Sections were then incubated
with primary antibody (rabbit anti-C-FOS 1:5000, Calbiochem) in 0.1% Triton X-100,
2.5% NGS, and PBS at 4°C overnight. Sections were rinsed 3 times in PBS for 10 min
each and then incubated in biotinylated secondary antibody (biotinylated goat anti-rabbit;
1:200, Vector Labs, Burlingame, California) and 2.5% NGS in PBS for 2 h at room
temperature. Sections were rinsed 3 times with PBS for 10 min each and then incubated
in an avidin-horseradish peroxidase mixture (Vector Labs, Burlingame, California) for 2
h. Sections were rinsed 3 times in PBS for 10 min each and then reacted with 0.05%
diaminobenzidine in the presence of 0.0015% H2O2. Sections were collected onto
gelatin-coated slides, dried for several hours, and coverslipped with Cytoseal for VTA
cannula verification and c-Fos quantification (black labeled c-Fos nuclei indicate
activated neurons are easily identifiable and thus easily counted for statistical analyses).
All rats included in the data analysis had verified cannulae placements in VTA.
2.g.ii. Expression study
Immediately after the end of the last session rats were anesthetized with an
overdose of sodium pentobarbital, perfused with 0.9% saline followed by 4% formalin,
and decapitated. The brains were removed and stored in 4% formalin for at least seven
days before being sectioned in the coronal plane on a cryostat, stained with cresyl violet
and inspected for cannulae implantations and injection sites. All rats included in the data
analysis had verified cannulae placements in VTA.
56 2.h. Data Analysis
For all rats in the conditioning procedures the data consisted of the number of
food trough head entries made during (1) a 6-s period immediately preceding the onset of
the CS (pre-CS period), (2) the 3-s period during the CS, (3) a 3-s period immediately
following the offset of the CS and (4) at all other times (non-CS period). For analyses the
period during the CS and the period immediately following the CS were combined for a
6-s total period and referred to as the CS-period. For each session the total number of
head entries during the CS periods and the total number of head entries during the pre-CS
periods are used to calculate the CS/pre-CS ratio. This ratio indicates the magnitude of
the conditioned approach response (re: the degree to which food trough head entries are
elicited by the CS).
Separate two-way, mixed-design ANOVAs, with group (dose of AP-5) as a
between groups factor and session as a repeated measures factor were conducted on the
CS/pre-CS ratio data from sessions 1 to 3 (acquisition) or 1 to 7 (expression) for the
acquisition and expression tests, respectively. Significant interactions were followed by
tests of simple effect of session in each group (dose). Separate one-way ANOVAs were
conducted on the CS/pre-CS ratio data from the CS-only test session for the conditioning
tests. Separate one-way ANOVAs were conducted on the total head entries from the CSonly test session for the conditioned procedures.
C-Fos labeled nuclei were counted on an Olympus (Tokyo, Japan) BX51W
microscope with a motorized stage at 4x magnification. Stereo Investigator software
(MicroBrightField, Williston, VT) was used to outline the NAcc core and shell, medial
and lateral caudate, and frontal cortex. To prevent bias that may skew the results, all
counting was conducted blind to the treatment groups. Planned independent sample t-
57 tests with bonferroni adjusted p values comparing c-Fos counts in the AP-5 and vehicle
groups were conducted for each brain region.
3. Results
3.a. Histology
Most VTA microinjection sites were localized in the caudal portion of the VTA (5.6 to -6.04 mm posterior to bregma) with some injections occurring in the central
portion (-5.2 to -5.3 mm posterior to bregma) (Figure 9).
3.b. Acquisition study
Rats receiving AP-5 prior to conditioning sessions made the greatest number of
food trough head entries during sessions 1 to 3 while rats receiving vehicle made the
fewest [dose effect: F(3,32) = 7.625, P < .001]. Across sessions 1 to 3, the total number
of head entries did not change for the vehicle group while it declined for the 0.25 µg AP5 and random control groups and increased in the 0.5 µg AP-5 group [session x dose:
F(6,64) = 4.601, P < .001]. In the test session, when all rats were treated with vehicle
injections, the total number of head entries was similar for all groups (Figure 10).
The left panel in Figure 11 shows the ratio of CS/pre-CS food trough head entries
in the three conditioning sessions. The pattern of change across sessions for this ratio
was different among the different treatment groups [a two-way ANOVA revealed a
significant session x dose interaction, F(6,64) = 3.747, p < .005]. The vehicle and 0.25
µg AP-5 groups both showed progressively larger CS/pre-CS ratios across sessions while
for the 0.5 µg AP-5 and random control groups this ratio did not change (test of simple
effect of session at each dose was < .01 for vehicle and < .001 for 0.25 µg AP-5). The
58 right panel in Figure 11 shows the CS/pre-CS ratio in the CS-only test session. The ratios
for the vehicle and 0.25 µg AP-5 groups were much higher than for the 0.5 µg AP-5 and
random control groups (a one-way ANOVA revealed a significant group effect; F(3,32) =
3.15, P < .05). The ratios for the vehicle and 0.25 µg AP-5 were similar and the ratios for
the 0.5 µg AP-5 and random control groups were similar (LSD post hoc tests showed that
each of the vehicle and 0.25 µg AP-5 groups differed significantly from each of the 0.5
µg AP-5 and random control groups, all ps < .05).
3.c. Expression study
Rats receiving 0.5 µg AP-5 or vehicle prior to the CS-only test session made
similar amounts of food trough head entries during sessions 1 to 7 (with no treatment)
that declined across these sessions (Figure 12; analyses revealed a significant session
effect, F(6,78) = 2.219, P < .05, that did not significantly interact with group). In the CSonly test session, the group receiving 0.5 µg AP-5 showed a somewhat greater number of
overall head entries than the group receiving vehicle but this difference was not
significant.
The left panel in Figure 13 shows the ratio of CS/pre-CS food trough head entries
in the seven conditioning sessions.
Although both groups depicted received no
treatments prior to session 1 to 7—that is, they were identical—the data are depicted
separately according to the treatment that they would receive in the CS-only test session.
The pattern of change in the CS/pre-CS ratio was similar for both groups across the
conditioning sessions; both groups showed increasing ratios across the sessions that
appeared to level off in the last three sessions or so (a two-way ANOVA revealed a
significant session effect; F(6,78) = 4.438, P < .005). The right panel in Figure 13 shows
59 the ratio of CS/pre-CS food trough head entries in the CS-only test session after receiving
0.5 µg AP-5 or vehicle. Both groups showed similar ratios.
3.d. Immunohistochemistry
To test the hypothesis that intra-VTA AP-5 reduction in CS/pre-CS ratios is
associated with reduced activity in DA terminal regions we tested two additional groups
(on a neural level) in the acquisition study, one with 0.5 µg AP-5 and the other with
vehicle, both stained for c-Fos after the CS-only test. Similar to the previous groups
tested (on a behavioral level) in the same conditions, the vehicle group showed
significantly greater CS/pre-CS ratios than the AP-5 group during the CS-only test
(Figure 14B). Accordingly, c-Fos expression was lower in all regions tested in the AP-5
group. (see Figure 14A). Our planned comparisons revealed that in the NAcc core the
amount of c-Fos in the AP-5 group was significantly lower than in the vehicle group
[t(11)=2.038, p < .03); t-tests for the NAcc shell, medial caudate and frontal cortex
approached significance, Ps = .1 for each]. Figure 15 shows representative brain sections
of identical locations in the VTA, one from a vehicle (left) and the other from an AP-5
(right) animal, showing c-Fos in the NAcc core at the same level of magnification used
during counting.
60 -­‐5.2 mm -­‐5.6 mm -­‐5.8 mm -­‐6.04 mm -­‐6.3 mm 61 Figure 9. Histological reconstruction of injection sites adapted from Paxinos and Watson
(1997). Black circles represent the injection sites in the VTA. The numbers to the right of
each section indicates the distance posterior to the bregma.
62 1000
Total head entries
800
Vehicle
AP-5 0.25
AP-5 0.5
Random control
600
400
200
0
1
2
3
Test
Session
Figure 10. Average number of food trough head entries for each group across three
conditioning
and one CS-only test session of the acquisition study. Bilateral
microinjections of 0.25 or 0.5 µg of AP-5 or vehicle immediately preceded conditioning
sessions whereas only vehicle immediately preceded the CS-only test session. The
vertical lines represent the standard error of the mean.
Ratio of CS/Pre-CS head entries
8
Vehicle
0.25 µg AP-5
0.5 µg AP-5
Random Control
7
6
63 #
*
5
4
+
3
+
*#
2
1
0
1
2
3
CS-only
Session
Figure 11. Average CS/pre-CS ratios for each group across three conditioning and one
CS-only test session of the acquisition study. Bilateral microinjections of 0.25 or 0.5 µg
of AP-5 or vehicle immediately preceded conditioning sessions whereas only vehicle
immediately preceded the CS-only session. * and # represent significant group
differences in CS-only test; + represents significant increase across the three conditioning
sessions.
64 1000
Total head entries
800
Vehicle
AP-5 0.5
600
400
200
0
1
2
3
4
5
6
7
Test
Session
Figure 12. Average number of food trough head entries for each group across seven
conditioning
and one CS-only test session of the expression study. Bilateral
microinjections of 0.5 µg of AP-5 or vehicle immediately preceded the CS-only test
session. The vertical lines represent the standard error of the mean.
Ratio of CS/Pre-CS head entries
10
65 Vehicle
0.5 µg AP-5
9
8
7
+
6
5
+
4
3
2
1
0
1
2
3
4
5
6
7
CS-only
Session
Figure 13. Average CS/pre-CS ratios for each group across seven conditioning and one
CS- only test session of the expression study. Bilateral microinjections 0.5 µg of AP-5 or
vehicle immediately preceded the CS-only test session. + represents significant increase
across the seven conditioning sessions.
66 A
B
cFOS count
100
Ratio of CS/Pre-CS head entries
120
vehicle
AP-5
80
60
40
20
0
*
NA
co
NA
sh
Cau
med
Cau
lat
6
5
4
*
vehicle
AP-5
3
2
1
0
Front
cortex
Brain region
Figure 14. A. c-Fos counts in the NAcc core and shell, medial and lateral caudate and
frontal cortex in rats completing the acquisition study. Bilateral microinjections of 0.5 µg
AP-5 or vehicle immediately preceded conditioning sessions whereas only vehicle
immediately preceded the CS-only test session. * represents significant group differences
at that site. B. Average CS/pre-CS ratios for the two groups mentioned in A for the CSonly test session of the acquisition study. * represents significant group differences. For
A and B vertical lines represent the standard error of the mean.
Vehicle
67 0.5 µg AP-5
Figure 15. Illustrations representing c-Fos in identical portions of the NAcc core in one
rat that received bilateral microinjections of 0.5 µg AP-5, and one that received vehicle,
immediately preceding the CS-only test session in the acquisition study. Illustrations
were taken with the same microscope set to the same level of 4x magnification. The
white arrow indicates c-Fos.
68 4. Discussion
Based on the results of the present study we can conclude that the NMDA
receptors in the VTA are necessary for the acquisition, but not the expression, of a
conditioned approach response and for a CS to significantly activate the NAcc core. In
the acquisition study, rats that received intra-VTA injections of 0.5 µg AP-5 showed no
evidence of acquiring conditioned approach in response to the 3-s light presentation
preceding each pellet delivery. This was first indicated by a lack of increase in the ratio
of CS/pre-CS head entries across three conditioning sessions by the 0.5 µg AP-5 and
random control groups while the ratio of the 0.25 µg AP-5 and vehicle groups did
increase across those sessions. More importantly, during the CS-only test session, which
directly followed vehicle injection for all groups, the ratio of CS/pre-CS was significantly
higher in the 0.25 µg AP-5 and vehicle groups than the 0.5 µg AP-5 group, which in turn
was not significantly different from the random control group. Because the CS-only test
session was conducted without any acute effects of AP-5 treatment, the lack of
conditioned approach provides extremely strong evidence that NMDA receptor blockade
in the VTA prevented the acquisition of learning in relation to the CS. On a similar note,
in the expression study, intra-VTA injections of 0.5 µg AP-5 and vehicle administered
prior to CS-only test session had no effect on conditioned approach behavior indicating
that the NMDA receptor stimulation in the VTA is not necessary for the expression of
conditioned approach
In addition, our findings demonstrate that NMDA receptor stimulation in the VTA
is necessary for CS-associated activity in DA terminal regions, specifically in the NAcc
core. In the acquisition study, rats treated with intra-VTA injections of 0.5 µg AP-5 prior
to conditioning sessions not only failed to acquire the conditioned approach response, but
69 also showed significantly less c-Fos expression in the NAcc core in response to the CS
during the CS-only test session than rats treated with vehicle. These results suggest two
things. First, that expression of CS responding involves CS induced activation of cells in
the NAcc core. Second, that the neural plasticity that allows a CS to acquire the ability to
elicit a response similar to that of a US and CS induced NAcc core activity both require
NMDA receptor stimulation in the VTA during acquisition. All in all, our results
demonstrate that NMDA receptor antagonism in the VTA leads to impairment in the
acquisition of a conditioned approach response and in the acquisition of CS-induced
neuronal activity.
The behavioral and neural findings of this study are consistent with a previous
study from our lab, as well as from others. On the behavioral level, we have previously
reported that intra-VTA AP-5 blocked the acquisition of food reinforced instrumental
responding but not its expression (Zellner et al., 2009). Furthermore, intra-VTA treatment
with an NMDA receptor antagonist impaired the acquisition of morphine induced place
preference (Harris et al, 2004). These studies support our findings that the NMDA
receptors in the VTA play a role in the acquisition of reward-related learning for both
natural and drug reward. On the neural level, excitotoxic lesions of the NAcc core
(Parkinson et al., 2000) or depletion of DA in the NAcc using 6-hydroxydopamine
(Parkinson et al., 2002) impaired the acquisition of approach to a reward-related stimulus.
Altogether, these studies support our hypotheses that NMDA receptor stimulation in the
VTA is necessary for the acquisition, but not the expression, of a CS to elicit conditioned
approach and conditioned activation of DA terminal regions.
70 The present findings make a significant contribution to the understanding of the
neural plasticity underlying reward-related learning. We propose that when rats consume
a reward in a given environment, ACh transmission in the VTA activates DA cells,
leading to increased DA in forebrain regions which stimulates increased approach
behavior. As the VTA DA cells are stimulated by the ACh signal of reward, they also
receive excitation from Glu afferents. Because of this coincident stimulation of ACh and
Glu receptors, LTP occurs at the Glu synapses. Through the process of LTP, the
environmental signals acquire the ability to activate VTA DA cells in their own right,
increasing the levels of conditioned approach towards stimuli in the environment which
are contiguously associated with the reward. From this model we can predict that NMDA
receptor stimulation in the VTA and activation of DA terminal regions play a role in
these proposed associative processes of reward-related learning, a prediction that is
supported by the present data.
The increase in head entries across the three conditioning sessions during the
acquisition study emitted by the 0.5 µg AP-5 group deserves closer examination. This
change in behavior suggests the possibility that some kind of learning was taking place in
these rats, although we are unable to determine specifically what that might have been. It
is possible these rats were acquiring some association between reward and the
conditioning chamber, or the food trough, but in the absence of the ability to associate
food delivery with the light presentation they were indiscriminately increasing their
investigation of the food trough. It is also possible that the stimulation of food reward, in
the context of an inability to link reward to any specific stimuli, resulted in heightened
activity. The ability of intra-VTA AP-5 to facilitate context learning or other aspects of
71 reward-learning which are not related to discrete cues therefore merits additional
investigation. The scant evidence which exists to date would argue against this being the
case, at least in the case of drug administration with VTA NMDA blockade (Harris, et al.,
2004) where NMDA receptor blockade is associated with impaired context conditioning;
to our knowledge context-related learning with food reward has not been tested with this
manipulation and therefore remains an empirical question.
The results of this experiment also argue against context learning; although the
total head entry data during days 1 to 3 suggest a possibility of learning, pre-CS
responding on the test day was no higher than controls, which ought to have been the
case if AP-5 treated rats had learned a reward-association with the environment. In any
case, given that AP-5 rats on the test day with no drug pre-treatment emitted similarly
low levels of head entries to vehicle-treated rats; it appears that even if some kind of
learning were taking place across the training sessions, AP-5 rats were specifically
impaired in being able to acquire an association with the discrete reward-paired cue.
Another possibility for the increase in head entries seen in the 0.5 µg AP-5 group
is that there occurred sensitization to the stimulant effects of this drug. Intra-VTA
injections of 0.5 µg AP-5 causes an increase in locomotor activity (Zellner et al., 2009)
which might become sensitized with repeated use as in the present study. However, a
number of studies do not support this possibility and demonstrate that repeated
administration of the NMDA receptor antagonists AP-5 (Licata et al., 2004) or MK-801
(Kalivas and Alesdatter, 1993) in the VTA prevents the development of sensitization.
72 Chapter 4.
1. General Discussion
1.a. Results
The neural
mechanisms
underlying
reward-related
learning
in
the
mesocorticolimbic dopamine (DA) system are not fully understood, particularly how
neutral stimuli become conditioned stimuli (CSs). We proposed, based of the Hebbian
model, that for this type of learning the ventral tegmental area (VTA) receives signals
about both unconditional stimuli (USs) and eventual CSs which converge on DA neurons
leading to the acquisition by the conditioned stimulus (CS) of the capacity to activate DA
neurons, and hence DA terminal regions (specifically the nucleus accumbens, amygdala
and prefrontal cortex) on its own and elicit conditioned approach. To test this model we
hypothesized that (1) a food US activates VTA DA cells, (2) a food-associated CS (light)
acquires the capacity to activate VTA DA cells and cause conditioned approach, and (3)
that blockade of NMDA receptors in the VTA would impair the acquisition, but not the
expression, of conditioned approach on a behavioral level and the expression of c-Fos in
DA terminal regions on a neural level.
Our results support our hypotheses. First, that food (the US) activates VTA DA
neurons. Second, that a stimulus explicitly paired with the US, the conditioned stimulus
(CS; light) also activates VTA DA neurons and produces a significantly greater
conditioned approach response than a stimulus not explicitly paired with the US. Third,
that NMDA receptor stimulation in the VTA is necessary for the acquisition, but not
expression, of conditioned approach behavior and for a CS to significantly activate the
nucleus accumbens (NAcc) core. Altogether, it appears that a discrete food-associated
73 CS can acquire the capacity to cause conditioned activation of VTA DA neurons, and that
this is VTA NMDA receptor dependent. The results of the present studies lead us to
conclude that conditioned activation of VTA DA cells underlies the capacity of CSs to
elicit a conditioned approach response similar to that of a US, and that this is VTA
NMDA receptor dependent. Furthermore, that the neural plasticity underlying rewardrelated learning involves CS-induced activation of DA terminal regions, specifically the
NAcc core.
1.b. The role of the VTA
The VTA is an essential node in the mesocorticolimbic DA system because it
constitutes a node for where this kind of neuronal plasticity occurs. We (Bonci &
Malenka, 1999; Harris, et al., 2004; Ranaldi, 2014; Zellner & Ranaldi, 2010) suggest that
one constituent of the process by which CSs gain access to the same neuronal circuits as
USs is via synaptic plasticity in which US and CS relevant cells activate the same VTA
DA neuron concurrently. This convergent stimulation creates the conditions for long term
potentiation (LTP) which has been demonstrated in the VTA DA neurons (Bonci &
Malenka, 1999; Chen et al., 2008; Luu & Malenka, 2008) as well as to be VTA NMDA
receptor dependent (Bonci & Malenka, 1999; Citri & Malenka, 2007; Nugent, Hwong,
Udaka, & Kauer, 2008; Stuber, et al., 2008). We have shown that injections of an NMDA
receptor antagonist directly into the VTA block the acquisition of reward-related learning
during both operant (Zellner, et al., 2009b) and classical (Ranaldi, et al., 2011)
conditioning tasks for natural reward (food). The same has been shown for drug reward.
Harris and Aston-Jones (Harris & Aston-Jones, 2003) found that stimulation of NMDA
receptors in the VTA is essential for the acquisition of cocaine conditioned place
74 preference. Therefore, we proposed here that, if the reward neurons are firing in response
to muscarinic acetylcholine (ACh) receptor activation, then simultaneous relatively weak
stimulation of the NMDA receptor via glutamate (Glu) signals can result in the initiation
of second messenger cascades implicated in LTP (Harris, et al., 2004), as well as other
intracellular events that lead to neurochemical and structural changes associated with
synaptic plasticity underlying reward-related learning. Such synaptic changes could result
in VTA Glu acquiring the capacity to robustly activate VTA DA cells, causing enhanced
DA release. This increase in DA might lead to increased approach, increasing the
likelihood of direct contact with rewards or emission of behaviors that lead to rewards or
to associative processes downstream that facilitate other aspects of conditioned
behaviors. The VTA acts as a node where this type of neuronal plasticity occurs because
it receives substantial afferents able to provide coincident stimulation representing both
USs and CSs, contains NMDA receptors that can set in motion intracellular processes
that lead to neural plasticity following such coincident stimulation such as LTP.
The VTA receives and is modulated by many neurotransmitters including DA, ACh,
Glu, GABA, serotonin, norepinephrine, opioids and peptides including CCK and orexin.
In this dissertation I focus only on the role of ACh and Glu. ACh likely mediates the
primary reward signal of various USs to the VTA whereas Glu likely carries signals
about environmental stimuli, which provides for the stimulation of the VTA NMDA
receptor. VTA NMDA receptor stimulation mediates LTP, and is the signal by which
CSs are likely to continue to activate VTA cells, and the DA terminal regions, after
associative learning has taken place.
75 1.c. Understanding the role of Glu
It is thought that in reward-related learning, natural and drug rewards trigger
approach behaviors primarily because they elevate mesolimbic DA levels (Wise, 2004).
Under conditions of DA receptor blockade, rewards such as food (Wise, et al., 1978),
brain stimulation (Fouriezos, et al., 1978) and psychomotor stimulants like amphetamine
(Yokel & Wise, 1975) and cocaine (de Wit & Wise, 1977) fail to maintain responding in
trained animals. We hypothesized that the capacity of CSs to function as such also
requires that they elevate mesolimbic DA levels and we have proposed (Ranaldi, 2014;
Sharf & Ranaldi, 2006a; Zellner & Ranaldi, 2010) that their capacity to stimulate DA
neurons depends on Glu neurotransmission in the VTA. On a similar note, You et al.
(You, et al., 2007) have demonstrated that exposure to a cocaine self-administration
context causes Glu release in the VTA and that this CS-induced glutamatergic signal is
necessary for cocaine seeking. It is also well-established that reward predictive stimuli
cause burst firing of VTA DA neurons (Schultz, 1997), and although the mechanisms
through which this happens have not been definitively established they likely involve Glu
neurotransmission. Several brain regions, including the prefrontal cortex, amygdala
(AMG), the midbrain tegmental nuclei, and the bed nucleus of the stria terminalis project
directly to the VTA (Georges & Aston-Jones, 2002; Harris, et al., 2004). Most of these
inputs are glutamatergic and excite VTA DA neurons (Georges & Aston-Jones, 2002;
Takahata & Moghaddam, 1998) suggesting that one or more of these signals may play a
necessary role in CS-induced mesolimbic DA activation in reward-related learning.
76 1.d. DA activation from unconditioned and conditioned stimuli
Studies have shown that the presentation of a US or CS can translate into synaptic
overflow of DA into the terminal regions of the mesocorticolimbic DA system (Fiorino,
et al., 1993; Sombers, et al., 2009), however, the manner in which this DA release is
initiated is unclear. It may result from DA cell activation from signals in the VTA, or
from presynaptic stimulation of DA terminals. The current studies suggest that
conditioned approach occurs through signal activation of DA cells, at the level of the cell
bodies, in the VTA. We proposed this signal is glutamatergic in nature (Sharf, et al.,
2006; Sharf & Ranaldi, 2006a; Zellner & Ranaldi, 2010). If this were true then one
would expect reduced responding maintained by reward-associated stimuli and reduced
dopamine release in the mesolimbic DA terminal regions after blockade of glutamate
signaling in the VTA and in fact this is what Sombers and colleagues found (Sombers, et
al., 2009). Moreover, in a previous study we demonstrated that blockade of NMDA
receptors in the VTA prevented the acquisition of conditioned approach and this was
associated with significantly reduced c-Fos activation in the NAcc core in response to the
(non-effective in this case) CS (Ranaldi, et al., 2011). Altogether these findings suggest a
causal link between a CS’s capacity to cause activation of VTA DA cells and mesolimbic
terminal regions and its capacity to elicit conditioned approach. In addition, our findings
demonstrate that NMDA receptor stimulation in the VTA is necessary for CS-associated
activity in DA terminal regions.
1.e. The role of VTA NMDA receptors
There is an accumulation of evidence (See Chapter 4, 1.b.) that NMDA receptor
stimulation in the VTA plays a necessary role in reward-related learning for both natural
77 (e.g., food) and drug reward. And altogether, these studies support our hypotheses that
NMDA receptor stimulation in the VTA is necessary for the acquisition, but not the
expression (Addy, Nunes, & Wickham, 2015), of a CS to elicit conditioned approach and
conditioned activation of DA terminal regions.
1.f. A detailed Hebbian model
We have proposed a model of reward-related learning (Sharf & Ranaldi, 2006a)
(Zellner, et al., 2009a; Zellner & Ranaldi, 2010) predicated on the assumption that this
learning occurs when the CS acquires the ability to activate the same neural system that
produces unconditioned approach (Bindra, 1974). This is a Hebbian model that proposes
that the VTA is a site where signals from reward-associated and US (i.e., primary
rewards) converge onto DA cells and, through NMDA-dependent LTP, CS signals
acquire the capacity to activate VTA DA cells by themselves. In the VTA, muscarinic
ACh receptor stimulation has been shown repeatedly to mediate the rewarding effects of
USs like food (Rada, et al., 2000; Sharf & Ranaldi, 2006a), brain stimulation reward
(Kofman & Yeomans, 1988; Rada, et al., 2000; Yeomans, Mathur, & Tampakeras, 1993)
and cocaine (You, Wang, Zitzman, & Wise, 2008) and therefore could serve as the US
signal in this model.
Also in the VTA, Glu is released from afferents of neurons
originating in frontal cortex, pedunculopontine nuclei and other brain regions that are
involved in processing information about environmental stimuli. Thus, the VTA is a site
where convergence of US and CS signals is possible. Through the process of LTP at the
Glu synapse, the environmental signals acquire the ability to activate VTA DA cells in
their own right, increasing the levels of conditioned approach towards stimuli in the
environment which are contiguously associated with the reward. Support for this model
78 comes from previous studies showing that blockade of muscarinic ACh receptor
stimulation in the VTA blocks the acquisition, but not the expression, of food-reinforced
instrumental responding (Sharf, et al., 2006) and other food-related learning (Sharf &
Ranaldi, 2006a).
The present findings make a significant contribution to the
understanding of the neural plasticity underlying reward-related learning. Support for the
role of NMDA receptor stimulation in this model comes from the present and other
(Stuber, et al., 2008) findings that blockade of NMDA receptors impairs the acquisition
of the conditioned approach or instrumental (Zellner, et al., 2009a) response but not the
expression of this learning.
Additional evidence supporting our model is that the VTA and its DA terminal
regions show progressive changes in activity as neutral stimuli are paired with rewards,
indicating that associative processes are taking place. Midbrain DA neurons respond to
both primary rewards and CSs (see (Horvitz, 2000) for review), but responding to CSs
develops over time. Generally, midbrain DA neurons fire in response to reward receipt
until animals are well trained at which time responding of DA cells comes primarily
under the control of CSs (Ljungberg, et al., 1992; Pan, Schmidt, Wickens, & Hyland,
2005). The present findings that intra-VTA AP-5, in addition to impairing the acquisition
of conditioned approach also resulted in significantly less c-Fos in NAcc core and a trend
toward less c-Fos in other DA terminal regions in response to the CS, is consistent with
the hypothesis that the reward-associated stimulus failed to control behavior because it
failed to acquire the capacity to activate DA cells, resulting in less cellular activation in
DA terminal regions. However, some findings do not support this model. Stuber and his
colleagues (Stuber, et al., 2008) have found that although conditioned approach was
79 associated with synaptic strengthening onto VTA DA cells to the CS, this LTP-like
enhanced response was temporary; it developed during acquisition of conditioned
approach for food but dissipated after the behavioral response stabilized. These findings
suggest that VTA DA cell activation facilitates the transformation of neutral stimuli into
salient reward predictive cues but is not required to maintain the US-CS association. The
maintenance of the US-CS association may rely on synaptic plasticity outside the VTA
and possibly in the DA terminal regions.
It seems probable that NMDA receptor stimulation in the VTA is needed to
produce a DA signal downstream that is itself essential for synaptic plasticity. Phasic, or
burst, firing of VTA DA cells is at least partly dependent on NMDA receptor stimulation
(Chergui et al., 1991), and this NMDA effect appears dependent on intact signaling from
afferents originating in the laterodorsal tegmentum (Lodge & Grace, 2006). It has long
been hypothesized that a DA signal in terminal regions, time-locked to reward events,
serves as a necessary input for the synaptic plasticity in these regions that leads to
reward-related learning (Beninger, 1983; Beninger & Ranaldi, 1994). In this model,
stimuli associated with reward provide a glutamatergic signal to output neurons in DA
terminal regions while primary rewards provide a DA signal to the same.
This
convergent stimulation would produce synaptic plasticity in these output neurons
allowing reward-related stimuli to function as CSs, activating these output neurons to
produce conditioned responding. Kelley and colleagues have conducted several studies
demonstrating the necessity of both DA and NMDA receptor stimulation in NAcc
(Smith-Roe & Kelley, 2000), prefrontal cortex (Baldwin, Sadeghian, Holahan, & Kelley,
2002) and AMG (Andrzejewski, Spencer, & Kelley, 2005) for the acquisition, but not
80 performance, of instrumental responding. But in addition to the stimulus-response type
of learning that might be represented by the Kelley studies it is also possible that DA
signals in terminal regions mediate the synaptic plasticity that underlies stimulus-reward
associations; such plasticity may then result in amplified CS-related glutamate signals
back to the VTA (You, et al., 2007) causing the CS-induced activation of DA cells
(Ljungberg, et al., 1992; Pan, et al., 2005) and CS-induced DA release (Bassareo & Di
Chiara, 1999; Wilson, Nomikos, Collu, & Fibiger, 1995; Wolterink et al., 1993) that is
observed in reward-related learning. Definitely, Glu in the VTA has been shown to have
a role in conveying information about CSs. You and colleagues (You, et al., 2007)
demonstrated that instrumental responding influenced by cocaine CSs is correlated with
VTA Glu release and reduced with VTA Glu blockade. Moreover, inhibition of Glu
release in the VTA during heroin self-administration reduces drug-context induced
reinstatement of already learned operant responding for drug (Bossert, Liu, & Shaham,
2004). These findings suggest that a route by which signals relating to CSs excite the
VTA, and by which those signals come to be able to excite the VTA in and of
themselves, thereby becoming CS signals, is via glutamate stimulation of the NMDA
receptor. Thus in both of these cases, where phasic DA activity is necessary for stimulusresponse or stimulus-reward associative learning, if phasic DA activity is dependent on
NMDA receptor stimulation in the VTA (Chergui, et al., 1991; Lodge & Grace, 2006),
then antagonism of this neurochemical pathway should impair acquisition of conditioned
approach learning.
In the case where phasic DA activity is necessary for synaptic
plasticity leading to enhanced CS-related glutamate signals to VTA, blockade of VTA
NMDA receptors during learning should also result in the diminished capacity of CSs to
81 activate VTA DA cells leading to reduced cellular activity in DA terminal regions in
response to CSs, a finding observed in the present study.
1.g. Possible shortcomings of the proposed Hebbian model in reward-related
learning
In this dissertation we surmised evidence that supports our model of rewardrelated learning. Specifically, we assessed the multitude of data demonstrating that VTA
ACh provides a US signal to the VTA, that VTA Glu provides a CS signal to the VTA
and coincident stimulation of the NDMA receptor is necessary for its synaptic
strengthening allowing environmental stimuli to recruit DA cells on their own. Therefore,
we propose that reward-related stimuli gain access to the same motivational neural
circuits as primary rewards and thereafter come to activate these circuits and elicit
motivational states, specifically approach behaviors, similar to the primary rewards.
In light of this proposition, there are some who suppose that DA may not have a
role in reward-related learning or that it plays a different role than approach. It has been
demonstrated that mesocorticolimbic DA neurons actually respond to salient and novel
stimuli, and furthermore, that DA neurons respond to salient events that extend beyond
that of reward stimuli (Horvitz, 2000); in other words, DA neurons respond to salient and
arousing changes in environmental conditions, regardless of the motivational valence of
that change. Specifically, Horvitz (Horvitz, Stewart, & Jacobs, 1997) demonstrated that
DA neurons in the VTA are activated by non-conditioned auditory and visual stimuli.
Other studies found that cells in the VTA increase their firing to sound, tail pressing, tail
pricks and spontaneous movement not associated with rewards (Kiyatkin & Rebec, 1998;
J. D. Miller, Sanghera, & German, 1981). Though these findings demonstrate that the
82 VTA and its terminal regions respond to novel USs, in the absence of reinforcement,
these responses have been shown to habituate (Ljungberg, et al., 1992; Schultz, 1998). In
both primates (Ljungberg, et al., 1992) and rats (Pan, et al., 2005), midbrain DA neurons
fire in response to reward receipt until animals are well trained and then respond only to
other unexpected rewards. Furthermore, in the awake rat, a transient increase in NAcc
DA to a novel stimulus disappears on subsequent sessions when not paired with reward
(Kiyatkin & Stein, 1996). Though these findings don’t directly demonstrate a role for
DA in approach in reward-related learning, they do show an important role in sensory or
attentional processes that may be necessary in the early stages of reward-related learning,
but not later on. Altogether, it appears that midbrain DA neurons respond to novel and
aversive (Young, Ahier, Upton, Joseph, & Gray, 1998) events, in addition to rewards and
conditioned stimuli.
Given the significant amount of data assessed in this dissertation, it seems
reasonable to state that CSs have to the ability to activate the same neural systems as
primary rewards (USs) and can thus directly influence behavior. Additionally, it appears
that after years of hypotheticals and experimentation we are currently in the process of
understanding the underlying neural mechanisms of how CSs acquire this capacity. We
hope that further investigation in this specific area will contribute to our basic knowledge
of this important adaptive function as well as in addressing pathologies and diseases of
reward-related behaviors that involve abnormal strengthening of reward-related cues.
83 1.h. The importance of understanding the neural mechanisms underlying rewardrelated learning
Elucidating the neural mechanisms underlying reward-related learning is
important because it adds insight into understanding pathologies, such as drug addiction,
that involve this type of learning. For instance, since drug-taking is similar to other types
of reinforced learning (Goldberg, 1973) it is possible that our “Hebbian-based” neural
model of reward-related learning applies to drug-related learning occurring in addiction
as well. Drug-associated stimuli, which cause craving (Childress, Ehrman, McLellan, &
O'Brien, 1988; Ehrman, Robbins, Childress, & O'Brien, 1992), drug-seeking (Goldberg,
1975; Ranaldi & Roberts, 1996) and reinstatement of drug seeking in animals (Fuchs,
Tran-Nguyen, Specio, Groff, & Neisewander, 1998; Grimm, Hope, Wise, & Shaham,
2001; Meil & See, 1996; Semenova & Markou, 2003), do so in part because of their
capacity to elevate DA levels in the mesocorticolimbic DA system (Robinson &
Berridge, 1993; Stewart, de Wit, & Eikelboom, 1984; Wise, 1994). It is important to
note that the mesocorticolimbic DA system did not evolve to respond to drugs of abuse or
drug-associated stimuli but rather to play a role in adaptive behaviors for survival such as
reward-related learning. It has been suggested that cue-induced drug craving and seeking
results from direct activation of ventral tegmental area DA neurons. If this is so, then
strategies that can prevent drug cue signals from activating these neurons may prove
beneficial in treating relapse. Thus, understanding the neurobiology underlying the role
of the mesocorticolimbic DA system in reward-related learning may help address
disorders and diseases (such as schizophrenia, Parkinson’s, and attention deficit
hyperactivity disorder), and potential therapies or improved treatment strategies.
84 References
Addy, N. A., Nunes, E. J., & Wickham, R. J. (2015). Muscarinic, but not nicotinic,
acetylcholine receptor blockade in the ventral tegmental area attenuates cueinduced sucrose-seeking. Behavioural brain research, 291, 372-376. doi:
10.1016/j.bbr.2015.05.036
Andrzejewski, M. E., Spencer, R. C., & Kelley, A. E. (2005). Instrumental learning, but
not performance, requires dopamine D1-receptor activation in the amygdala.
Neuroscience, 135(2), 335-345.
Antonov, I., Antonova, I., Kandel, E. R., & Hawkins, R. D. (2001). The contribution of
activity-dependent synaptic plasticity to classical conditioning in Aplysia. [In
Vitro Research Support, U.S. Gov't, P.H.S.]. The Journal of neuroscience : the
official journal of the Society for Neuroscience, 21(16), 6413-6422.
Baldwin, A. E., Sadeghian, K., Holahan, M. R., & Kelley, A. E. (2002). Appetitive
instrumental learning is impaired by inhibition of cAMP-dependent protein kinase
within the nucleus accumbens. Neurobiology of Learning and Memory, 77(1), 4462.
Barria, A., Muller, D., Derkach, V., Griffith, L. C., & Soderling, T. R. (1997). Regulatory
phosphorylation of AMPA-type glutamate receptors by CaM-KII during longterm potentiation. [Comment In Vitro Research Support, Non-U.S. Gov't
Research Support, U.S. Gov't, P.H.S.]. Science, 276(5321), 2042-2045.
Bassareo, V., & Di Chiara, G. (1999). Modulation of feeding-induced activation of
mesolimbic dopamine transmission by appetitive stimuli and its relation to
motivational state. European Journal of Neuroscience, 11, 4389-4397.
Beninger, R. J. (1983). The role of dopamine in locomotor activity and learning. Brain
Research Reviews, 6, 173-196.
Beninger, R. J. (1989). Dissociating the effects of altered dopaminergic function on
performance and learning. Brain Research Bulletin, 23, 365-371.
Beninger, R. J., D'Amico, C. M., & Ranaldi, R. (1993). Microinjections of flupenthixol
into the caudate putamen of rats produce intrasession declines in food-rewarded
operant responding. Pharmacol Biochem Behav, 45(2), 343-350. doi: 00913057(93)90249-S [pii]
Beninger, R. J., & Hahn, B. L. (1983). Pimozide blocks establishment but not expression
of amphetamine-produced environment-specific conditioning. Science, 220, 13041306.
Beninger, R. J., & Miller, R. (1998). Dopamine D1-like receptors and reward-related
incentive learning. Neuroscience & Biobehavioral Reviews, 22(2), 335-345.
Beninger, R. J., & Ranaldi, R. (1994). Dopaminergic agents with different mechanisms of
action differentially affect responding for conditioned reward. In T. Palomo & T.
Archer (Eds.), Strategies for Studying Brain Disorders. Vol. 1: Depressive,
Anxiety and Drug Abuse Disorders. London: Farrand Press. (Reprinted from: IN
FILE).
Berridge, K. C. (2007). The debate over dopamine's role in reward: the case for incentive
salience. Psychopharmacology (Berl), 191(3), 391-431.
85 Berridge, K. C., & Robinson, T. E. (1998). What is the role of dopamine in
reward:hedonic impact, reward learning, or incentive salience? Brain Research
Reviews, 28, 309-369.
Bindra, D. (1974). A motivational view of learning, performance, and behavior
modification. Psychological Reviews, 81, 199-213.
Bjorklund, A., & Dunnett, S. B. (2007). Fifty years of dopamine research. [Editorial
Review].
Trends
in
neurosciences,
30(5),
185-187.
doi:
10.1016/j.tins.2007.03.004
Blackburn, J. R., & Phillips, A. G. (1989). Dopamine and preparatory behavior: II. a
neurochemical analysis. Behavioral Neuroscience, 103, 15-23.
Bolton, R. F., Cornwall, J., & Phillipson, O. T. (1993). Collateral axons of cholinergic
pontine neurones projecting to midline, mediodorsal and parafascicular thalamic
nuclei in the rat. Journal of Chemical Neuroanatomy, 6 (2 ), 101 - 114
Bonci, A., & Malenka, R. C. (1999). Properties and plasticity of excitatory synapses on
dopaminergic and GABAergic cells in the ventral tegmental area. Journal of
Neuroscience, 19(10), 3723-3730.
Bossert, J. M., Liu, S. Y., & Shaham, Y. (2004). A role of ventral tegmental area
glutamate in contextual cue-induced relapse to heroin seeking. Journal of
Neuroscience, 24(47), 10726-10730.
Calabresi, P., Lacey, M. G., & North, R. A. (1989). Nicotinic excitation of rat ventral
tegmental neurones in vitro studied by intracellular recording. British Journal of
Pharmacology, 98 (1 ), 135 - 140.
Carelli, R. M. (2000). Activation of accumbens cell firing by stimuli associated with
cocaine delivery during self-administration. Synapse, 35, 238-242.
Carelli, R. M., & Ijames, S. G. (2001). Selective activation of accumbens neurons by
cocaine-associated stimuli during a water/cocaine multiple schedule. Brain
Research, 907(1-2), 156-161.
Carr, D. B., & Sesack, S. R. (2000). GABA-containing neurons in the rat ventral
tegmental area project to the prefrontal cortex. Synapse, 38(2), 114-123.
Chen, B. T., Bowers, M. S., Martin, M., Hopf, F. W., Guillory, A. M., Carelli, R. M., . . .
Bonci, A. (2008). Cocaine but not natural reward self-administration nor passive
cocaine infusion produces persistent LTP in the VTA. Neuron, 59(2), 288-297.
Chergui, K., Charlety, P. J., Akaoka, H., Brunet, J. L., Saunir, C. F., Buda, M., . . .
Chouvet, G. (1991). Participation of NMDA receptors in spontaneous burst firing
of dopaminergic mesencephalic neurons. C R Academy des Sciences III, 313(2),
139-144.
Childress, A. R., Ehrman, R. N., McLellan, A. T., & O'Brien, C. P. (1988). Conditioned
craving and arousal in cocaine addiction: a preliminary report.
NIDA.Res.Monogr., 81, 74-80.
Childress, A. R., Mozley, P. D., McElgin, W., Fitzgerald, J., Reivich, M., & O'Brien, C.
P. (1999). Limbic activation during cue-induced cocaine craving. American
Journal of Psychiatry, 156(1), 11-18.
Christie, M. J., Bridge, S., James, L. B., & Beart, P. M. (1985). Excitotoxin lesions
suggest an aspartatergic projection from rat medial prefrontal cortex to ventral
tegmental area. Brain Research, 333, 169-172.
86 Citri, A., & Malenka, R. C. (2007). Synaptic plasticity: multiple forms, functions, and
mechanisms. Neuropsychopharmacology, 33(1), 18-41.
Conrad, L. C., & Pfaff, D. W. (1976). Autoradiographic tracing of nucleus accumbens
efferents in the rat. Brain research, 113(3), 589-596.
Cornwall, J., & Phillipson, O. T. (1988). Afferent projections to the dorsal thalamus of
the rat as shown by retrograde lectin transport-I. the mediodorsal nucleus.
Neuroscience, 24, 1035-1049.
Day, H. E., Kryskow, E. M., Nyhuis, T. J., Herlihy, L., & Campeau, S. (2008).
Conditioned fear inhibits c-fos mRNA expression in the central extended
amygdala. [Research Support, N.I.H., Extramural]. Brain research, 1229, 137146. doi: 10.1016/j.brainres.2008.06.085
Day, J. J., Roitman, M. F., Wightman, R. M., & Carelli, R. M. (2007). Associative
learning mediates dynamic shifts in dopamine signaling in the nucleus
accumbens. [Research Support, N.I.H., Extramural]. Nature neuroscience, 10(8),
1020-1028. doi: 10.1038/nn1923
de Wit, H., & Wise, R. A. (1977). Blockade of cocaine reinforcement in rats with the
dopamine receptor blocker pimozide, but not with noradrenergic blockers
phentolamine and phenoxybenzamine. Canadian Journal of Psychology, 31, 195203.
Di Ciano, P., & Everitt, B. J. (2004). Contribution of the ventral tegmental area to
cocaine-seeking maintained by a drug-paired conditioned stimulus in rats.
European Journal of Neuroscience, 19(6), 1661-1667.
Dragunow, M., & Faull, R. (1989). The use of c-fos as a metabolic marker in neuronal
pathway tracing. [Review]. Journal of neuroscience methods, 29(3), 261-265.
Ehrman, R. N., Robbins, S. J., Childress, A. R., & O'Brien, C. P. (1992). Conditioned
responses to cocaine-related stimuli in cocaine abuse patients.
Psychopharmacology, 107(4), 523-529.
Fallon, J. H., & Ciofi, P. (1992). Distribution of monoamines within the amygdala The
Amgydala: Neurobiological Aspects of Emotion, Memory, and Mental
Dysfunction (pp. 97-114). (Reprinted from: IN FILE).
Fibiger, H. C., Phillips, A. G., & Brown, E. E. (1992). The neurobiology of cocaineinduced reinforcement. Ciba Foundation symposium, 166, 96-111.
Fields, H. L., Hjelmstead, G. O., Margolis, E. B., & Nicola, S. M. (2007). Ventral
tegmental area neurons in learned appetitive behavior and positive reinforcement.
Annual Reviews in Neuroscience, 30, 289-316.
Fiorino, D. F., Coury, A., Fibiger, H. C., & Phillips, A. G. (1993). Electrical stimulation
of reward sites in the ventral tegmental area increases dopamine transmission in
the nucleus accumbens of the rat. Behavioral Brain Research, 55, 131-141.
Floresco, S. B., West, A. R., Ash, B., Moore, H., & Grace, A. A. (2003). Afferent
modulation of dopamine neuron firing differentially regulates tonic and phasic
dopamine transmission. Nat.Neurosci., 6(9), 968-973.
Floresco, S. B., Yang, C. R., Phillips, A. G., & Blaha, C. D. (1998). Basolateral amygdala
stimulation evokes glutamate receptor-dependent dopamine efflux in the nucleus
accumbens of the anaesthetized rat. European Journal of Neuroscience, 10(4),
1241-1251.
87 Fouriezos, G., Hansson, P., & Wise, R. A. (1978). Neuroleptic-induced attenuation of
brain stimulation reward in rats. Journal of Comparative and Physiological
Psychology, 92, 661.
Fuchs, R. A., Tran-Nguyen, L. T., Specio, S. E., Groff, R. S., & Neisewander, J. L.
(1998). Predictive validity of the extinction/reinstatement model of drug craving.
[Comparative Study Research Support, Non-U.S. Gov't Research Support, U.S.
Gov't, P.H.S.]. Psychopharmacology, 135(2), 151-160.
Fuxe, K., & et al. (1974). Central catecholamine neurons, behavior and neuroleptic drugs:
An analysis to understand the involvement of catecholamines in schizophrenia.
Journal of Psychiatric Research, 11, 151-161.
Garzon, M., Vaughan, R. A., Uhl, G. R., Kuhar, M. J., & Pickel, V. M. (1999).
Cholinergic axon terminals in the ventral tegmental area target a subpopulation of
neurons expressing low levels of the dopamine transporter. Journal of
Comparative Neurology, 410 (2 ), 197 - 210.
Geisler, S., Derst, C., Veh, R. W., & Zahm, D. S. (2007). Glutamatergic afferents of the
ventral tegmental area in the rat. Journal of Neuroscience, 27(21), 5730-5743.
Geisler, S., & Zahm, D. S. (2005). Afferents of the ventral tegmental area in the ratanatomical substratum for integrative functions. Journal of Comparative
Neurology, 490(3), 270-294.
Georges, F., & Aston-Jones, G. (2002). Activation of ventral tegmental area cells by the
bed nucleus of the stria terminalis: a novel excitatory amino acid input to
midbrain dopamine neurons. Journal of Neuroscience, 22 (12 ), 5173 - 5187.
Goldberg, S. R. (1973). Comparable behavior maintained under fixed-ratio and secondorder schedules of food presentation, cocaine injection or d-amphetamine
injection in the squirrel monkey. The Journal of pharmacology and experimental
therapeutics, 186(1), 18-30.
Goldberg, S. R. (1975). Stimuli associated with drug injections as events that control
behavior. [Research Support, U.S. Gov't, Non-P.H.S. Research Support, U.S.
Gov't, P.H.S.]. Pharmacological reviews, 27(3), 325-340.
Grace, A. A., & Bunney, B. S. (1984). The control of firing pattern in nigral dopamine
neurons: single spike firing. J.Neurosci, 4(11), 2866-2876.
Gratton, A., & Wise, R. A. (1994). Drug- and behavior-associated changes in dopaminerelated electrochemical signals during intravenous cocaine self-administration in
rats. Journal of Neuroscience, 14, 4130-4146.
Greengard, P. (2001). The neurobiology of slow synaptic transmission. Science, 294,
1024-1030.
Grimm, J. W., Hope, B. T., Wise, R. A., & Shaham, Y. (2001). Neuroadaptation.
Incubation of cocaine craving after withdrawal. Nature, 412(6843), 141-142. doi:
10.1038/35084134
Gronier, B., & Rasmussen, K. (1998). Activation of midbrain presumed dopaminergic
neurones by muscarinic chollinergic receptors: An in vivo electrophysiological
study in the rat. British Journal of Pharmacology, 124, 455-465.
Harris, G. C., & Aston-Jones, G. (2003). Critical role for ventral tegmental glutamate in
preference for a cocaine-conditioned environment. Neuropsychopharmacology,
28 (1 ), 73 - 76.
88 Harris, G. C., Wimmer, M., Byrne, R., & Aston-Jones, G. (2004). Glutamate-associated
plasticity in the ventral tegmental area is necessary for conditioning
environmental stimuli with morphine. Neuroscience, 129 (3 ), 841 - 847.
Hebb, D. O. (1949). The Organization of Behavior. New York: Wiley (Interscience).
Heien, M. L., & Wightman, R. M. (2006). Phasic dopamine signaling during behavior,
reward, and disease states. [Research Support, N.I.H., Extramural Review]. CNS
& neurological disorders drug targets, 5(1), 99-108.
Heimer, L., Zaborszky, L., Zahm, D. S., & Alheid, G. F. (1987). The ventral
striatopallidothalamic projection: I. The striatopallidal link originating in the
striatal parts of the olfactory tubercle. [Research Support, U.S. Gov't, P.H.S.]. The
Journal of comparative neurology, 255(4), 571-591. doi: 10.1002/cne.902550409
Henderson, Z., & Sherriff, F. E. (1991). Distribution of choline acetyltransferase
immunoreactive axons and terminals in the rat and ferret brainstem. Journal of
Comparative Neurology, 314(1), 147-163.
Hernandez, L., & Hoebel, B. G. (1988). Food reward and cocaine increase extracellular
dopamine in the nucleus accumbens as measured by microdialysis. Life Sciences,
42, 1705-1712.
Horvitz, J. C. (2000). Mesolimbocortical and nigrostriatal dopamine responses to salient
non-reward events. Neuroscience, 96(4), 651-655.
Horvitz, J. C., Stewart, J., & Jacobs, B. L. (1997). Burst activity of ventral tegmental
dopamine neurons is elicited by sensory stimuli in the awake cat. Brain Research,
759(2), 251-258.
Howland, J. G., Taepavarapruk, P., & Phillips, A. G. (2002). Glutamate receptordependent modulation of dopamine efflux in the nucleus accumbens by
basolateral, but not central, nucleus of the amygdala in rats. Journal of
Neuroscience, 22(3), 1137-1145.
Ikemoto, S. (2007). Dopamine reward circuitry: two projection systems from the ventral
midbrain to the nucleus accumbens-olfactory tubercle complex. Brain Research
Reviews, 56(1), 27-78.
Ikemoto, S., & Panksepp, J. (1996). Dissociations between appetitive and consummatory
responses by pharmacological manipulations of reward-relevant brain regions.
Behavioral Neuroscience, 110(2), 331-345.
Janak, P. H., Chang, J. Y., & Woodward, D. J. (1999). Neuronal spike activity in the
nucleus accumbens of behaving rats during ethanol self-administration. Brain
Research, 817, 172-184.
Jay, T. M. (2003). Dopamine: a potential substrate for synaptic plasticity and memory
mechanisms. Progress in Neurobiology, 69(6), 375-390.
Johnson, S. W., & North, R. A. (1992). Two types of neurone in the rat ventral tegmental
area and their synaptic inputs. Journal of Physiology, 450, 455-468.
Kandel, E. R. (2001). The molecular biology of memory storage: a dialogue between
genes and synapses. Science, 294(5544), 1030-1038.
Kelley, A. E. (1999). Neural integrative activities of nucleus accumbens subregions in
relation to learning and motivation. Psychobiology, 27(2), 198-213.
Kest, K., Cruz, I., Chen, D. H., Galaj, E., & Ranaldi, R. (2012). A food-associated CS
activates c-Fos in VTA DA neurons and elicits conditioned approach.
Behavioural brain research, 235(2), 150-157. doi: 10.1016/j.bbr.2012.07.044
89 Kiyatkin, E. A., & Gratton, A. (1994). Electrochemical monitoring of extracellular
dopamine in nucleus accumbens of rats lever-pressing for food. Brain Research,
652, 225-234.
Kiyatkin, E. A., & Rebec, G. V. (1998). Heterogeneity of ventral tegmental area neurons:
single-unit recording and iontophoresis in awake, unrestrained rats. Neuroscience,
85(4), 1285-1309.
Kiyatkin, E. A., & Stein, E. A. (1996). Conditioned changes in nucleus accumbens
dopamine signal established by intravenous cocaine in rats. Neuroscience Letters,
211(2), 73-76.
Kofman, O., & Yeomans, J. S. (1988). Cholinergic antagonists in ventral tegmentum
elevate thresholds for lateral hypothalamic and brainstem self-stimulation.
Pharmacology, Biochemistry and Behavior, 31, 547-559.
Lavin, A., Noqueira, L., Lapish, C. C., Wightman, R. M., Phillips, P. E., & Seamans, J.
(2005). Mesocortical dopamine neurons operate in distinct temporal domains
using multimodal signaling. Journal of Neuroscience, 25(20), 5013-5023.
Lewis, B. L., & O'Donnell, P. (2000). Ventral tegmental area afferents to the prefrontal
cortex maintain membrane potential 'up' states in pyramidal neurons via D(1)
dopamine receptors. Cerebral Cortex, 10(12), 1168-1175.
Lisman, J., Schulman, H., & Cline, H. (2002). The molecular basis of CaMKII function
in synaptic and behavioural memory. [Review]. Nature reviews. Neuroscience,
3(3), 175-190. doi: 10.1038/nrn753
Ljungberg, T., Apicella, P., & Schultz, W. (1992). Responses of monkey dopamine
neurons during learning of behavioral reactions. Journal of Neurophysiology,
67(1), 145-163.
Lodge, D. J., & Grace, A. A. (2006). The laterodorsal tegmentum is essential for burst
firing of ventral tegmental area dopamine neurons. Proc.Natl.Acad.Sci.U.S.A,
103(13), 5167-5172.
Luu, P., & Malenka, R. C. (2008). Spike timing-dependent long-term potentiation in
ventral tegmental area dopamine cells requires PKC. J.Neurophysiol., 100(1),
533-538.
Margolis, E. B., Lock, H., Chefer, V. I., Shippenberg, T. S., Hjelmstad, G. O., & Fields,
H. L. (2006). Kappa opioids selectively control dopaminergic neurons projecting
to the prefrontal cortex. Proc.Natl.Acad.Sci.U.S.A, 103(8), 2938-2942.
Meil, W. M., & See, R. E. (1996). Conditioned cued recovery of responding following
prolonged withdrawal from self-administered cocaine in rats: an animal model of
relapse. Behavioural pharmacology, 7(8), 754-763.
Miller, A. D., & Blaha, C. D. (2005). Midbrain muscarinic receptor mechanisms
underlying regulation of mesoaccumbens and nigrostriatal dopaminergic
transmission in the rat. European Journal of Neuroscience, 21(7), 1837-1846.
Miller, J. D., Sanghera, M. K., & German, D. C. (1981). Mesencephalic dopaminergic
unit activity in the behaviorally conditioned rat. Life Science, 29(12), 1255-1263.
Mogenson, G. J. (1987). Limbic-motor integration. Progress in Psychobiology and
Physiological Psychology, 12, 117-170.
Mogenson, G. J., Jones, D. L., & Yim, C. Y. (1980). From motivation to action:
functional interface between the limbic system and the motor system. Progress in
Neurobiology, 14, 69-97.
90 Montague, P. R., Dayan, P., & Sejnowski, T. J. (1996). A framework for mesencephalic
dopamine systems based on predictive Hebbian learning. [Comparative Study
Research Support, Non-U.S. Gov't Research Support, U.S. Gov't, P.H.S.
Review]. The Journal of neuroscience : the official journal of the Society for
Neuroscience, 16(5), 1936-1947.
Murschall, A., & Hauber, W. (2006). Inactivation of the ventral tegmental area abolished
the general excitatory influence of Pavlovian cues on instrumental performance.
Learn.Mem., 13(2), 123-126.
Naranjo, C. A., Tremblay, L. K., & Busto, U. E. (2001). The role of the brain reward
system in depression. [Research Support, Non-U.S. Gov't Review]. Progress in
neuro-psychopharmacology & biological psychiatry, 25(4), 781-823.
Nicola, S. M. (2007). The nucleus accumbens as part of a basal ganglia action selection
circuit. Psychopharmacology, 191(3), 521-550.
Nicola, S. M., & Deadwyler, S. A. (2000). Firing rate of nucleus accumbens neurons is
dopamine-dependent and reflects the timing of cocaine-seeking behavior in rats
on a progressive ratio schedule of reinforcement. Journal of Neuroscience,
20(14), 5526-5537.
Nicola, S. M., Taha, S. A., Kim, S. W., & Fields, H. L. (2005). Nucleus accumbens
dopamine release is necessary and sufficient to promote the behavioral response
to reward-predictive cues. Neuroscience, 135(4), 1025-1033.
Niznik, H. B., & van Tol, H. H. M. (1992). Dopamine receptor genes: new tools for
molecular psychiatry. Journal of Psychiatry and Neuroscience, 17, 158-180.
Nugent, F. S., Hwong, A. R., Udaka, Y., & Kauer, J. A. (2008). High-frequency afferent
stimulation induces long-term potentiation of field potentials in the ventral
tegmental area. Neuropsychopharmacology, 33(7), 1704-1712.
O'Donnell, P., & Grace, A. A. (1995). Synaptic interactions among excitatory afferents to
nucleus accumbens neurons: hippocampal gating of prefrontal cortical input.
Journal of Neuroscience, 15(5), 3622-3639.
Oakman, S. A., Faris, P. L., Kerr, P. E., Cozzari, C., & Hartman, B. K. (1995).
Distribution of pontomesencephalic cholinergic neurons projecting to substantia
nigra differs significantly from those projecting to ventral tegmental area. The
Journal of Neuroscience, 15(9), 5859-5869.
Olds, J., & Milner, P. M. (1954). Positive reinforcement produced by electrical
stimulation of septal area and other regions of rat brain. Journal of Comparative
and Physiological Psychology, 47, 419-427.
Omelchenko, N., & Sesack, S. R. (2006). Cholinergic axons in the rat ventral tegmental
area synapse preferentially onto mesoaccumbens dopamine neurons. Journal of
Comparative Neurology, 494(6), 863-875.
Overton, P. G., Richards, C. D., Berry, M. S., & Clark, D. (1999). Long-term potentiation
at excitatory amino acid synapses on midbrain dopamine neurons. Neuroreport,
10(2), 221-226.
Pan, W. X., Schmidt, R., Wickens, J. R., & Hyland, B. I. (2005). Dopamine cells respond
to predicted events during classical conditioning: evidence for eligibility traces in
the reward-learning network. Journal of Neuroscience, 25(26), 6235-6242.
Panksepp, J. (1998), Series in Affective Science Oxford University Press
91 Pfaus, J. G., Damsma, G., Wenkstern, D., & Fibiger, H. C. (1995). Sexual activity
increases dopamine transmission in the nucleus accumbens and striatum of female
rats. Brain Research, 693, 21-30.
Phillips, A. G., Atkinson, L. J., Blackburn, J. R., & Blaha, C. D. (1993). Increased
extracellular dopamine in the nucleus accumbens of the rat elicited by a
conditional stimulus for food: an electrochemical study. Canadian Journal of
Physiological Pharmacology, 71(5-6), 387-393.
Phillips, P. E., Stuber, G. D., Heien, M. L., Wightman, R. M., & Carelli, R. M. (2003).
Subsecond dopamine release promotes cocaine seeking. Nature, 422(6932), 614618.
Phillipson, O. T. (1979). Afferent projections to the ventral tegmental area of Tsai and
interfascicular nucleus: a horseradish peroxidase study in the rat. Journal of
Comparative Neurology, 187 (1 ), 117 - 143.
Rada, P. V., Mark, G. P., Yeomans, J. S., & Hoebel, B. G. (2000). Acetylcholine release
in ventral tegmental area by hypothalamic self-stimulation, eating, and drinking.
Pharmacology, Biochemistry and Behavior, 65(3), 375-379.
Ranaldi, R. (2014). Dopamine and reward seeking: the role of ventral tegmental area.
Reviews in the neurosciences. doi: 10.1515/revneuro-2014-0019
Ranaldi, R., & Beninger, R. J. (1994). The effects of systemic and intracerebral injections
of D 1 and D 2 agonists on brain stimulation reward. Brain Research, 651, 283292.
Ranaldi, R., Kest, K., Zellner, M. R., Lubelski, D., Muller, J., Cruz, Y., & Saliba, M.
(2011). The effects of VTA NMDA receptor antagonism on reward-related
learning and associated c-fos expression in forebrain. Behavioural brain research,
216(1), 424-432. doi: 10.1016/j.bbr.2010.08.026
Ranaldi, R., Pocock, D., Zereik, R., & Wise, R. A. (1999). Dopamine fluctuations in the
nucleus accumbens during maintenance, extinction, and reinstatement of
intravenous D-amphetamine self-administration. Journal of Neuroscience, 19(10),
4102-4109.
Ranaldi, R., & Roberts, D. C. S. (1996). Initiation, maintenance and extinction of cocaine
self-administration with and without conditioned reward. Psychopharmacology,
128, 89-96.
Redgrave, P., Prescott, T. J., & Gurney, K. (1999). Is the short-latency dopamine
response too short to signal reward error? Trends Neurosci., 22(4), 146-151.
Richardson, N. R., & Gratton, A. (1996). Behavior-relevant changes in nucleus
accumbens dopamine transmission elicited by food reinforcement: an
electrochemical study in rat. Journal of Neuroscience, 16(24), 8160-8169.
Robinson, T. E., & Berridge, K. C. (1993). The neural basis of drug craving: an
incentive-sensitization theory of addiction. Brain Research Reviews, 18, 247-291.
Rodriguez, J. J., Doherty, M. D., & Pickel, V. M. (2008). N-methyl-D-aspartate (NMDA)
receptors in the ventral tegmental area: subcellular distribution and colocalization
with 5-hydroxytryptamine(2A) receptors. Journal of Neuroscience Research,
60(2), 202-211.
Romo, R., & Schultz, W. (1990). Dopamine neurons of the monkey midbrain:
contingencies of responses to active touch during self-initiated arm movements.
Journal of Neurophysiology, 63(3), 592-606.
92 Rossetti, Z. L., & Wise, R. A. (1995). Electrical stimulation of the prefrontal cortex
increases extracellular glutamate in the ventral tegmental area. Society for
Neuroscience Abstracts, 21, 1378.
Salamone, J. D., Correa, M., Farrar, A., & Mingote, S. (2007). Effort-related functions of
nucleus
accumbens
dopamine
and
associated
forebrain
circuits.
Psychopharmacology, 191(3), 461-482.
Schilstrom, B., Svensson, H. M., Svensson, T. H., & Nomikos, G. G. (1998). Nicotine
and food induced dopamine release in the nucleus accumbens of the rat: putative
role of alpha7 nicotinic receptors in the ventral tegmental area. Neuroscience, 85
(4 ), 1005 - 1009.
Schultz, W. (1997). A neural substrate of prediction and reward. Science, 275, 15931599.
Schultz, W. (1998). Predictive reward signal of dopamine neurons. Journal of
Neurophysiology, 80(1), 1-27.
Schultz, W., Tremblay, L., & Hollerman, J. R. (2000). Reward processing in primate
orbitofrontal cortex and basal ganglia. [Research Support, Non-U.S. Gov't].
Cerebral cortex, 10(3), 272-284.
Semenova, S., & Markou, A. (2003). Cocaine-seeking behavior after extended cocainefree periods in rats: role of conditioned stimuli. [Research Support, Non-U.S.
Gov't Research Support, U.S. Gov't, P.H.S.]. Psychopharmacology, 168(1-2),
192-200. doi: 10.1007/s00213-002-1251-7
Sesack, S. R., Deutch, A. Y., Roth, R. H., & Bunney, B. S. (1989). Topographical
organization of the efferent projections of the medial prefrontal cortex in the rat:
an anterograde tract-tracing study with Phaseolus vulgaris leucoagglutinin.
Journal of Comparative Neurology, 290(2), 213-242.
Sesack, S. R., & Pickel, V. M. (1992). Prefrontal cortical efferents in the rat synapse on
unlableled neuronal targets of catecholamine terminals in the nucleus accumbens
septi and on dopamine neurons in the ventral tegmental area. Journal of
Comparative Neurology, 320(2), 145-160.
Sharf, R., McKelvey, J., & Ranaldi, R. (2006). Blockade of muscarinic acetylcholine
receptors in the ventral tegmental area prevents acquisition of food-rewarded
operant responding in rats. Psychopharmacology (Berl), 186(1), 113-121. doi:
10.1007/s00213-006-0352-0
Sharf, R., & Ranaldi, R. (2006a). Blockade of muscarinic acetylcholine receptors in the
ventral
tegmental
area
disrupts
food-related
learning
in
rats.
Psychopharmacology, 184, 87-94.
Sharf, R., & Ranaldi, R. (2006b). Blockade of muscarinic acetylcholine receptors in the
ventral tegmental area disrupts food-related learning in rats. Psychopharmacology
(Berl), 184(1), 87-94. doi: 10.1007/s00213-005-0235-9
Smith, Y., Charara, A., & Parent, A. (1996). Synaptic innervation of midbrain
dopaminergic neurons by glutamate-enriched terminals in the squirrel monkey.
Journal of Comparative Neurology, 364(2), 231-253.
Smith-Roe, S. L., & Kelley, A. E. (2000). Coincident activation of NMDA and dopamine
D1 receptors within the nucleus accumbens core is required for appetitive
instrumental learning. Journal of Neuroscience, 20(20), 7737-7742.
93 Sombers, L. A., Beyene, M., Carelli, R. M., & Wightman, R. M. (2009). Synaptic
overflow of dopamine in the nucleus accumbens arises from neuronal activity in
the ventral tegmental area. Journal of Neuroscience, 29(6), 1735-1742.
Spooren, W. P., Veening, J. G., Groenewegen, H. J., & Cools, A. R. (1991). Efferent
connections of the striatopallidal and amygdaloid components of the substantia
innominata in the cat: projections to the nucleus accumbens and caudate nucleus.
Neuroscience, 44(2), 431-447.
Stewart, J. (1984). Reinstatement of heroin and cocaine self-administration behavior in
the rat by intracerebral application of morphine in the ventral tegmental area.
Pharmacology, Biochemistry and Behavior, 20, 917-923.
Stewart, J., de Wit, H., & Eikelboom, R. (1984). Role of unconditioned and conditioned
drug effects in the self-administration of opiates and stimulants. Psychological
Reviews, 19, 251-268.
Stuber, G. D., Klanker, M., de, R. B., Bowers, M. S., Joosten, R. N., Feenstra, M. G., &
Bonci, A. (2008). Reward-predictive cues enhance excitatory synaptic strength
onto midbrain dopamine neurons. Science, 321(5896), 1690-1692.
Swanson, L. W. (1982). The projections of the ventral tegmental area and adjacent
regions:a combined flourescent retrograde tracer and immunofluorescence study
in the rat. Brain Research Bulletin, 9, 321-353.
Takahata, R., & Moghaddam, B. (1998). Glutamatergic regulation of basal and stimulusactivated dopamine release in the prefrontal cortex. Journal of Neurochemistry,
71, 1443-1449.
Talmi, D., Seymour, B., Dayan, P., & Dolan, R. J. (2008). Human pavlovian-instrumental
transfer. Journal of Neuroscience, 28(2), 360-368.
Testa, A., Festa, F., Ranaldi, R., Giachelia, M., Tirindelli, D., De Marco, A., . . . Cozzi,
R. (2005). A multi-biomarker analysis of DNA damage in automobile painters.
Environ Mol Mutagen, 46(3), 182-188. doi: 10.1002/em.20147
Wallace, D. M., Magnuson, D. J., & Gray, T. S. (1992). Organization of amygdaloid
projections to brainstem dopaminergic, noradrenergic, and adrenergic cell groups
in the rat. [Research Support, Non-U.S. Gov't Research Support, U.S. Gov't, NonP.H.S. Research Support, U.S. Gov't, P.H.S.]. Brain research bulletin, 28(3), 447454.
Watanabe, M. (1990). Prefrontal unit activity during associative learning in the monkey.
[Research Support, Non-U.S. Gov't]. Experimental brain research.
Experimentelle Hirnforschung. Experimentation cerebrale, 80(2), 296-309.
Westerink, B. H. C., Enrico, P., Feimann, J., & De Vries, J. B. (1998). The pharmacology
of mesocortical dopamine neurons: a dual-probe microdialysis study in the ventral
tegmental area and prefrontal cortex of the rat brain. Journal of Pharmacology
and Experimental Therapeutics, 285(1), 143-154.
White, N. M., & Milner, P. M. (1992). The psychobiology of reinforcers. Annual Reviews
in Psychology, 43, 443-471.
Wilson, C., Nomikos, G. G., Collu, M., & Fibiger, H. C. (1995). Dopaminergic correlates
of motivated behavior:importance of drive. The Journal of Neuroscience, 15(7),
5169-5178.
Wise, R. A. (1978). Catecholamine theories of reward: a critical review. Brain Research,
152, 215-247.
94 Wise, R. A. (1994). Cocaine reward and cocaine craving: the role of dopamine in
perspective. [Review]. NIDA research monograph, 145, 191-206.
Wise, R. A. (2004). Dopamine, learning and motivation. Nat Rev Neurosci, 5(6), 483494.
Wise, R. A. (2006). Role of brain dopamine in food reward and reinforcement.
Philosophical Transactions of the Royal Society B: Biological Sciences,
361(1471), 1149-1158.
Wise, R. A., & Bozarth, M. A. (1987). A psychomotor stimulant theory of addiction.
Psychological Reviews, 94(4), 469-492.
Wise, R. A., & RomprÇ, P. P. (1989). Brain dopamine and reward. Annual Reviews in
Psychology, 40, 191-225.
Wise, R. A., & Rompre, P. P. (1989). Brain dopamine and reward. [Research Support,
Non-U.S. Gov't Review]. Annual review of psychology, 40, 191-225. doi:
10.1146/annurev.ps.40.020189.001203
Wise, R. A., Spindler, J., deWit, H., & Gerber, G. J. (1978). Neuroleptic-induced
"anhedonia" in rats: pimozide blocks reward quality of food. Science, 201, 262264.
Wolterink, G., Phillips, G., Cador, M., Donselaar-Wolterink, I., Robbins, T. W., &
Everitt, B. J. (1993). Relative roles of ventral striatal D 1 and D 2 dopamine
receptors in responding with conditioned reinforcement. Psychopharmacology,
110, 355-364.
Wright, C. I., & Groenewegen, H. J. (1995). Patterns of convergence and segregation in
the medial nucleus accumbens of the rat:relationships of prefrontal
cortical,midline thalamic, and basal amygdaloid afferents. The Journal of
Comparative Neurology, 361, 383-403.
Wu, Y. N., & Johnson, S. W. (1996). Pharmacological characterization of inward current
evoked by N-methyl-D-aspartate in dopamine neurons in the rat brain slice. [In
VitroResearch Support, U.S. Gov't, P.H.S.]. The Journal of pharmacology and
experimental therapeutics, 279(2), 457-463.
Yamaguchi, T., Sheen, W., & Morales, M. (2007). Glutamatergic neurons are present in
the rat ventral tegmental area. European Journal of Neuroscience, 25(1), 106-118.
Yeomans, J. S., Mathur, A., & Tampakeras, M. (1993). Rewarding brain stimulation: role
of tegmental cholinergic neurons that activate dopamine neurons. Behavioral
Neuroscience, 107, 1077.
Yokel, R. A., & Wise, R. A. (1975). Increased lever pressing for amphetamine after
pimozide in rats: implications for a dopamine theory of reward. Science, 187,
547-549.
Yoshida, M., Yokoo, H., Mizoguchi, K., Kawahara, H., Tsuda, A., Nishikawa, T., &
Tanaka, M. (1992). Eating and drinking cause increased dopamine release in the
nucleus accumbens and ventral tegmental area in the rat: measurement by in vivo
microdialysis. Neuroscience Letters, 139 (1 ), 73 - 76.
You, Z. B., Wang, B., Zitzman, D., Azari, S., & Wise, R. A. (2007). A role for
conditioned ventral tegmental glutamate release in cocaine seeking. J Neurosci,
27(39), 10546-10555. doi: 27/39/10546 [pii] 10.1523/JNEUROSCI.2967-07.2007
You, Z. B., Wang, B., Zitzman, D., & Wise, R. A. (2008). Acetylcholine release in the
mesocorticolimbic dopamine system during cocaine seeking: conditioned and
95 unconditioned contributions to reward and motivation. J Neurosci, 28(36), 90219029. doi: 28/36/9021 [pii] 10.1523/JNEUROSCI.0694-08.2008
Young, A. M., Ahier, R. G., Upton, R. L., Joseph, M. H., & Gray, J. A. (1998). Increased
extracellular dopamine in the nucleus accumbens of the rat during associative
learning of neutral stimuli. Neuroscience, 83(4), 1175-1183.
Yun, I. A., Wakabayashi, K. T., Fields, H. L., & Nicola, S. M. (2004). The ventral
tegmental area is required for the behavioral and nucleus accumbens neuronal
firing responses to incentive cues. Journal of Neuroscience, 24 (12 ), 2923 - 2933.
Zellner, M. R., Kest, K., & Ranaldi, R. (2009a). NMDA receptor antagonism in the
ventral tegmental area impairs acquisition of reward-related learning. Behav Brain
Res,
197(2),
442-449.
doi:
S0166-4328(08)00565-2
[pii]
10.1016/j.bbr.2008.10.013
Zellner, M. R., Kest, K., & Ranaldi, R. (2009b). NMDA receptor antagonism in the
ventral tegmental area impairs acquisition of reward-related learning. Behav.Brain
Res., 197(2), 442-449.
Zellner, M. R., & Ranaldi, R. (2010). How conditioned stimuli acquire the ability to
activate VTA dopamine cells: A proposed neurobiological component of rewardrelated learning. Neuroscience & Biobehavioral Reviews, 34(5), 769-780.