* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download - CUNY Academic Works
Long-term depression wikipedia , lookup
Types of artificial neural networks wikipedia , lookup
Neural coding wikipedia , lookup
Neuroplasticity wikipedia , lookup
Aging brain wikipedia , lookup
Neural oscillation wikipedia , lookup
Central pattern generator wikipedia , lookup
Neuroesthetics wikipedia , lookup
Premovement neuronal activity wikipedia , lookup
Neurotransmitter wikipedia , lookup
Environmental enrichment wikipedia , lookup
Time perception wikipedia , lookup
Neural engineering wikipedia , lookup
Multielectrode array wikipedia , lookup
Neuroethology wikipedia , lookup
Signal transduction wikipedia , lookup
Donald O. Hebb wikipedia , lookup
Nonsynaptic plasticity wikipedia , lookup
Nervous system network models wikipedia , lookup
Psychoneuroimmunology wikipedia , lookup
Neural correlates of consciousness wikipedia , lookup
Pre-Bötzinger complex wikipedia , lookup
Synaptogenesis wikipedia , lookup
Neuroanatomy wikipedia , lookup
NMDA receptor wikipedia , lookup
Conditioned place preference wikipedia , lookup
Development of the nervous system wikipedia , lookup
Molecular neuroscience wikipedia , lookup
Metastability in the brain wikipedia , lookup
Endocannabinoid system wikipedia , lookup
Eyeblink conditioning wikipedia , lookup
Synaptic gating wikipedia , lookup
Activity-dependent plasticity wikipedia , lookup
Stimulus (physiology) wikipedia , lookup
Feature detection (nervous system) wikipedia , lookup
Neuroeconomics wikipedia , lookup
Optogenetics wikipedia , lookup
Channelrhodopsin wikipedia , lookup
City University of New York (CUNY) CUNY Academic Works Dissertations, Theses, and Capstone Projects Graduate Center 9-30-2015 The Role of the VTA NMDA Receptors, VTA DA Cells and VTA Terminal Regions in RewardRelated Learning Karen Kest Graduate Center, City University of New York How does access to this work benefit you? Let us know! Follow this and additional works at: http://academicworks.cuny.edu/gc_etds Part of the Biology Commons, and the Psychology Commons Recommended Citation Kest, Karen, "The Role of the VTA NMDA Receptors, VTA DA Cells and VTA Terminal Regions in Reward-Related Learning" (2015). CUNY Academic Works. http://academicworks.cuny.edu/gc_etds/1005 This Dissertation is brought to you by CUNY Academic Works. It has been accepted for inclusion in All Graduate Works by Year: Dissertations, Theses, and Capstone Projects by an authorized administrator of CUNY Academic Works. For more information, please contact [email protected]. THE ROLE OF THE VTA NMDA RECEPTORS, VTA DA CELLS AND THE VTA TERMINAL REGIONS IN REWARD-RELATED LEARNING by KAREN KEST A dissertation submitted to the Graduate Faculty in Psychology in partial fulfillment of the requirements for the degree of Doctor of Philosophy, The City University of New York 2015 ii © 2015 Karen Kest All Rights Reserved This manuscript has been read and accepted by the Graduate Faculty in Psychology in satisfaction of the dissertation requirement for the degree of Doctor of Philosophy ________________ Date ________________ Date Dr. Robert Ranaldi _____________________________________________ Chair of Examining Committee Dr. Joshua Brumberg _____________________________________________ Executive Officer Dr. Richard Bodnar Dr. Caroline Pytte Dr. Kenneth Carr Dr. Jon Horvitz Supervisory Committee THE CITY UNIVERSITY OF NEW YORK iii iv Abstract THE ROLE OF THE VTA NMDA RECEPTORS, VTA DA CELLS AND THE VTA TERMINAL REGIONS IN REWARD RELATED LEARNING by Karen Kest Adviser: Robert Ranaldi, Ph.D. Reward-related learning occurs when an initially neutral stimulus acquires the capacity to elicit responses similar to an unconditioned stimulus (US) with which it is associated, in which case the stimulus now functions as a conditioned stimulus (CS). The mechanisms whereby stimuli come to function as CSs are not fully understood and comprise the theme of this dissertation. We have previously proposed that coincident signals from an unconditioned and the eventual conditioned stimulus (US and CS) signals on dopamine (DA) cells of the ventral tegmental area (VTA) leads to strengthening of CS synapses, allowing the CS to acquire the ability to activate VTA DA cells on its own and elicit conditioned approach, thereby functioning as a CS. Furthermore, we proposed that this type of learning is VTA NMDA receptor dependent. This dissertation was designed to test this model by specifically testing the following hypotheses: (1) A food US will activate VTA DA cells; (2) A food-associated CS will activate VTA DA cells and cause conditioned approach; (3) Blockade of NMDA receptors in the VTA will prevent a foodassociated stimulus from acquiring the capacity to function as a CS (i.e., cause v conditioned approach) and to cause conditioned activation of the mesocorticolimbic DA system. To test the hypothesis that a US, in this case food, activates VTA DA cells, male rats were maintained at 85% of their free feeding weights for the duration of this study. Rats were exposed to an eating protocol in which the rats were able to eat or not eat food. Rat brains were then removed and immunostained for c-Fos followed by tyrosine hydroxylase (TH) to examine VTA DA cell activation. As expected, rats that ate the food demonstrated a significantly greater number of VTA DA (TH-labeled) cells expressing cFos than rats that did not receive food. To test the hypothesis that a CS, in this case a food-associated light, activates VTA DA cells, male rats were maintained at 85% of their free feeding weights for the duration of this study. Rats were trained to retrieve a food pellet after a light presentation (the CS) and then tested for the expression of the food checking response with only CS presentations. As expected, a light functioning as a CS caused a significantly greater number of VTA DA cells to express c-Fos than a light not functioning as a CS. We also hypothesized that VTA NMDA receptor stimulation is necessary for a foodassociated stimulus (CS; also a food-associated light) to elicit conditioned approach via conditioned activation of VTA DA cell terminal regions (mesocorticolimbic DA terminal regions). Rats were prepared with indwelling cannulae positioned so as to allow bilateral microinjections of AP-5 (a NMDA receptor antagonist) in the VTA and were maintained vi at 85% of their free feeding weights. Male rats were exposed to an acquisition or expression of learning protocol. Subsequently, all rat brains were processed for c-Fos labeling to examine activation of mesocorticolimbic DA terminal regions. As expected, AP-5 significantly impaired the acquisition of conditioned approach and significantly reduced the amount of c-Fos expressed in the cells in the mesocorticolimbic DA terminal regions in response to the CS. Also, AP-5 did not impair the expression of the already learned conditioned approach response. All together, the results support the model - that a CS acquires, via the VTA NMDA receptor, the capacity to cause conditioned activation of VTA DA cells and mesocorticolimbic DA terminal regions therefore eliciting conditioned approach in a manner similar to a US. vii ACKNOWLEDGEMNTS There are many people I’d like to thank for supporting me over the course of my work that have enabled me to complete this dissertation. I am especially grateful to my mentor, Dr. Robert Ranaldi. You had confidence in a young undergraduate student that approached you in your Behavioral Neuroscience class. From then till now, you have shown an unwavering bolstering in my abilities to comprehend and execute research. In addition to your constant confidence and support, you have imparted attributes that will always remain with me – in addition to training me with the necessary procedural skills I needed to conduct my research you have also trained me to write and think in clear and focused manner, as well as that patience, attention detail, and hard work are the key to success. For all this I thank you! Thank you Dan Hong Chen, Ivonne Cruz, Daniel Lubelski, Jonathan Muller and Michelle Saliba for your commitment to my doctoral research as undergraduate students. Your ability to work together, comprehension of the tasks at hand and your attention to details were contributing factors to my research results. To my husband Asher and son Avi, thank you for your effortless and endless support in me. Asher, thank you for your calm words of encouragement and advice at times I’ve felt overwhelmed. Lastly, I want to thank the members of my dissertation committee: Drs. Robert Ranaldi, Richard Bodnar, Caroline Pytte, Kenneth Carr and Jon Horvitz for their willingness to participate in this dissertation. viii Table of Contents List of Figures …………………………………………………………………………x Chapter 1. ...................................................................................................................... 1 1. What is reward-related learning? ........................................................................... 1 2. The physiology and function of DA and its role in reward-related learning ...... 2 2.a. Physiological features of DA and its receptors .................................................... 2 2.b. Functions of DA ..................................................................................................... 5 2.c. DA and its role in reward-related learning.......................................................... 8 3. A proposed Hebbian Model of Reward-Related Learning ................................. 11 4. Framework of the VTA and its DA projection sites ............................................ 12 5. VTA and reward-related learning......................................................................... 16 6. A more detailed proposed Hebbian Model of reward-related learning in the VTA .............................................................................................................................. 18 7. Hypothesis................................................................................................................ 20 Chapter 2. .................................................................................................................... 23 1. Introduction: A food-associated CS activates c-Fos in VTA DA neurons and elicits conditioned approach ...................................................................................... 23 2. Methods.................................................................................................................... 27 2.a. Subjects ................................................................................................................. 27 2.b. Apparatus ............................................................................................................. 28 2.b.i. Conditioning chambers ..................................................................................... 28 2.b.ii. Foot-shock chambers........................................................................................ 28 2.b.iii. Locomotor activity chambers ......................................................................... 28 2.c. Experiments .......................................................................................................... 29 2.c.i. Experiment 1: The effects of a US (food) on c-Fos in VTA TH-labeled cells 29 2.c.ii. Experiment 2: The effects of a CS on c-Fos in VTA TH-labeled cells ......... 29 2.c.iii. Experiment 3: The effects of reward omission on locomotor activity ........ 30 2.d. Immunohistochemistry ....................................................................................... 31 2.e. Data Analysis ........................................................................................................ 32 2.e.i. Experiment 1: The effects of a US (food) on c-Fos in VTA TH-labeled cells 33 2.e.ii. Experiment 2: The effects of a CS on c-Fos in VTA TH-labeled cells ......... 33 2.e.iii. Experiment 3: The effects of reward omission on locomotor activity ........ 33 3. Results ...................................................................................................................... 34 3.a. Experiment 1: The effects of a US (food) on c-Fos in VTA TH-labeled cells . 34 3.b. Experiment 2: The effects of a CS on c-Fos in VTA TH-labeled cells ............ 34 3.c. Experiment 3: The effects of reward omission on locomotor activity ............. 35 4. Discussion ................................................................................................................ 43 Chapter 3. .................................................................................................................... 47 ix 1. Introduction: NMDA receptor stimulation in the VTA is necessary for the acquisition of conditioned approach and the capacity of a conditioned stimulus to activate dopamine terminal regions .......................................................................... 47 2. Methods.................................................................................................................... 50 2.a. Subjects ................................................................................................................. 50 2.b. Surgical procedure............................................................................................... 51 2.c. Apparatus ............................................................................................................. 51 2.d. Microinjection procedure ................................................................................... 51 2.e. Drugs ..................................................................................................................... 52 2.f. Conditioning experiments .................................................................................... 52 2.f.i. Conditioning procedure for acquisition study ................................................. 53 2.f.ii. Conditioning procedure for expression study................................................. 54 2.g. Histology and Immunohistochemistry ............................................................... 54 2.g.i. Acquisition study ............................................................................................... 54 2.g.ii. Expression study ............................................................................................... 55 2.h. Data Analysis........................................................................................................ 56 3. Results ...................................................................................................................... 57 3.a. Histology ............................................................................................................... 57 3.b. Acquisition study ................................................................................................. 57 3.c. Expression study .................................................................................................. 58 3.d. Immunohistochemistry ....................................................................................... 59 4. Discussion ................................................................................................................ 68 Chapter 4………………………………………………………………...…………...73 1. General Discussion .................................................................................................. 72 1.a. Results ................................................................................................................... 72 1.b. The role of the VTA ............................................................................................. 73 1.c. Understanding the role of Glu ............................................................................ 75 1.d. DA activation from unconditioned and conditioned stimuli ........................... 76 1.e. The role of VTA NMDA receptors ..................................................................... 76 1.f. A detailed Hebbian model................................................................................... 77 1.g. Possible shortcomings of the proposed Hebbian model in reward-related learning ........................................................................................................................ 81 1.h. The importance of understanding the neural mechanisms underlying rewardrelated learning ........................................................................................................... 83 References…………………………………………………………………………….84 x List of Figures Figure 1. An illustration of the proposed Hebbian Model of reward-‐related learning in the VTA 22 Figure 2. VTA TH-labeled cells expressing c-Fos in the VTA for rats in the US (food pellets) and no US (no food pellets) groups 36 Figure 3: Representative VTA slices showing double-labeled TH/c-Fos cells in rats receiving US or no US. Below is an image showing a representative TH/c-Fos labeled cell (black arrow) and a representative TH-only labeled cell (white arrow) in the VTA 37 Figure 4: CS/Pre-CS food trough head entry ratios for EP and NEP groups during the 3 conditioning and CS-only test sessions 38 Figure 5: VTA TH-labeled cells expressing c-Fos in the VTA for animals in the EP and NEP groups 39 Figure 6: Representative VTA slices showing double-labeled TH/c-Fos cells in the EP and NEP groups 40 Figure 7: Locomotor counts for the groups receiving foot shock or no foot shock just prior to placement in the locomotor activity chambers 41 Figure 8: A. CS/Pre-CS food trough head entry ratios for EP and NEP groups during the 3 conditioning and CS-only test sessions. B. Locomotor counts for the EP and NEP groups tested immediately after the CS-only test session 42 Figure 9. Histological reconstruction of injection sites adapted from Paxinos and Watson (1997) 60 Figure 10. Average number of food trough head entries for each group across three conditioning and one CS-only test session of the acquisition study 62 Figure 11. Average CS/pre-CS ratios for each group across three conditioning and one CS-only test session of the acquisition study 63 Figure 12. Average number of food trough head entries for each group across seven conditioning and one CS-only test session of the expression study 64 Figure 13. Average CS/pre-CS ratios for each group across seven conditioning and one CS-only test session of the expression study 65 xi Figure 14. A. c-Fos counts in the NAcc core and shell, medial and lateral caudate and frontal cortex in rats completing the acquisition study. B. Average CS/pre-CS ratios for the two groups mentioned in A for the CS-only test session of the acquisition study 66 Figure 15. Illustrations representing c-Fos in identical portions of the NAcc core in one rat that received bilateral microinjections of 0.5 µg AP-5, and one that received vehicle, immediately preceding the CS-only test session in the acquisition study 67 1 Chapter 1. 1. What is reward-related learning? Reward-related learning is an essential adaptive function for survival in which organisms acquire behaviors that place them in contact with stimuli that function as primary rewards (unconditioned stimuli, USs; e.g., food, water), which consequently elicit approach behaviors and reinforce these behaviors. Neutral stimuli associated with primary rewards acquire the ability to act as conditioned stimuli (CSs), which can elicit conditioned approach behaviors similar to the primary rewards with which they are associated. Involved in this type of learning is the brain’s reward system, the mesocorticolimbic dopamine (DA) system, which originates in the ventral tegmental area (VTA) and projects to forebrain regions such as the nucleus accumbens (NAcc), amygdala (AMG) and prefrontal cortex (PFC). These forebrain regions will be referred to as the DA terminal regions. The neural mechanisms underlying this type of rewardrelated learning in this system are not fully understood, particularly how neutral stimuli become CSs. One possibility is that a conditioned stimulus (CS), by virtue of its contiguity with an unconditioned stimulus (US), acquires the capacity to activate the same neural substrates activated by the US and thus elicit conditioned approach behaviors similar to the US (Ranaldi, 2014; Zellner & Ranaldi, 2010). The present experiments were designed to test a particular neurobiological model focused on the VTA and the DA terminal regions for this type of learning. 2 2. The physiology and function of DA and its role in reward-related learning 2.a. Physiological features of DA and its receptors There is a large body of evidence implicating DA in reward-related learning (see (Beninger, 1989; Beninger & Miller, 1998; Wise, 2006; Wise & RomprÇ, 1989; Wise & Rompre, 1989; Zellner & Ranaldi, 2010) for reviews). Hence, a mention of the complex physiology of DA is merited. DA is a neurotransmitter in the catecholamine family that cannot cross the blood brain barrier so it is synthesized from cells within the central nervous system. DA is then transported from cytosol into synaptic vesicles by vesicular monoamine transporter 2 (VMAT2). DA remains in the vesicle until an action potential occurs and forces the vesicles to merge with the cell membrane, thus spilling DA into the synaptic cleft. Once in the synaptic cleft, DA binds to and activates postsynaptic DA receptors. DA can also bind to presynaptic DA receptors which can either excite or inhibit the DA neurons depending on their electrical potential; thus presynaptic DA receptors act as autoreceptors which result in the inhibition of DA neurotransmitter synthesis and release to help normalize DA levels. DA neurons are small, unmyelinated neurons that ascend within the mesocorticolimbic DA system. There are several dopaminergic pathways in the central nervous system involved in reward. First is the mesocorticolimbic DA system, in which DA neurons located in the VTA (A10) ascend to the DA terminal regions. DA within this system has been shown to affect neurons depending on their current neurophysiological state. For instance, DA can maintain and prolong a cell’s excited states (Lavin et al., 2005; Lewis & O'Donnell, 2000; O'Donnell & Grace, 1995). To exemplify this, Lavin and her colleagues (Lavin, et al., 2005) found that post synaptic potentials in the mPFC from VTA stimulation are not mediated by mPFC DA receptors, but rather initiated by 3 mPFC Glu receptors and maintained by mPFC DA receptors. Second is the nigrostriatal system that arises in the substantia nigra pars compacta (A9) and ascends to the neostriatum (caudate and putamen), part of the basal ganglia (BG) – a region involved in movement. Beninger and colleagues (Beninger, D'Amico, & Ranaldi, 1993) demonstrated this system’s involvement in food-reinforced operant responding by injecting trained rats with a DA antagonist in the striatum which resulted in decreased food-reinforced lever pressing. For the purpose of staying focused, only the mesocorticolimbic DA system will be discussed here. Levels of extracellular DA are modulated by tonic and phasic DA transmission. Tonic DA transmission occurs when small amounts of DA are released impartially of neuronal firing. In contrast, phasic DA transmission, or rapid burst firing, results from the firing of DA neurons (Grace & Bunney, 1984). Concentrated bursts result in greater amounts of extracellular DA. The amount of DA signal in the mesocorticolimbic DA terminal regions, time-locked to reward events, has been suggested to be a necessary input for the synaptic plasticity in this neuronal system that leads to reward-related learning (Beninger, 1983; Beninger & Ranaldi, 1994; Kelley, 1999). In behaving primates, DA neurons of the NAcc display short-latency, phasic firing to primary reward and conditioned cues associated with reward. In rats, transient DA release in terminal regions that mimics that seen during burst firing has been demonstrated during presentation of reward-related cues. Taken together, these studies suggest that phasic dopamine release is a critical mediator of reward-related processes (Heien & Wightman, 2006). DA binds to and activates postsynaptic DA receptors to mediate its physiological 4 effects on the target. DA can have long-term effects on neurons by playing a role in synaptic plasticity (Beninger & Miller, 1998; Jay, 2003). DA can act on stimulatory or inhibitory metabotropic DA receptor families (Niznik & van Tol, 1992) (all DA receptors are metabotropic). Activation of the stimulatory DA receptor families, receptors D1 and D5, results in a long-term molecular change in a neuron by triggering a conformational change in the DA receptor and therefore exposing a binding site for a G-protein. The newly formed G-protein coupled receptor (GPCR) causes displacement of a guanine diphosphate (GDP) with guanine triphosphate (GTP) on its alpha subunit, causing this alpha subunit to dissociate and, in the presence of adenosine triphosphate (ATP), form adenylyl cyclase (AC). AC converts ATP into cyclic adenosine monophosphate (cAMP; a second messenger). In the presence of cAMP the alpha subunit, along with GTP, becomes protein kinase A (PKA) (cAMP binds to the regulatory subunit of PKA allowing its catalytic subunit to become active), which can translocate to the nucleus and phosphorylates cAMP response element binding protein (Creb) 1. Creb2 (which when bound to a gene represses it) is bound to a cAMP response element (CRE; a DNA promotor region). A phosphorylated Creb1 displaces Creb2, causing activation of CRE that results in the expression of neighboring genes, in other words gene transcription. This pathway has been implicated in activity-dependent long-term sensitization in the Aplysia (Kandel, 2001) as well as activity-dependent plasticity involved in behavioral classical conditioning (Antonov, Antonova, Kandel, & Hawkins, 2001) supporting the idea that synaptic plasticity is a mechanism of learning and memory more generally. Conversely, activation of the inhibitory DA receptor families, receptors D2, D3 and D4, inhibits AC therefore preventing long-term molecular changes. For instance, D3 5 receptor mutant mice exhibit potentiated acquisition of conditioned place preference with cocaine compared to wild-type mice (Kong, Kuang, Li, & Xu, 2011). Activation of CaM kinase 2 (CaMKII; a calcium sensitive serine/therine protien kinase that when activated can phosphorylate and thus regulate Glu receptors (Barria, Muller, Derkach, Griffith, & Soderling, 1997)) in the NAcc, AMG and PFC is also potentiated in D3 receptor mutant mice compared to that in wild-type mice following conditioned place preference expression. These results support the notion that inhibitory DA receptors modulate reward-related learning induced by cocaine by inhibiting the activation of CaMKII in the brain reward circuit. DA can also have short-term effects on neurons which are usually modulatory (Greengard, 2001) and tend to increase post-synaptic excitability via coupling to the D1 family of receptors, which result in a cascade of processes leading to increased expression of glutamate (Glu) receptors. 2.b. Functions of DA The heterogeneity of effects of DA – short v. long-term consequences, the brain region in question, the relative concentration of receptor subtypes and the current state of affected neurons – accounts for the fact that a number of competing views on the function of DA exist. These perspectives include the following: (1) The anhedonia hypothesis, which emphasizes DA’s role in basic reward. Wise (Wise, Spindler, deWit, & Gerber, 1978) provided support for this hypothesis by blocking DA receptors with pimozide which resulted in blunting the rewarding effects of food. (2) The learning and reinforcement hypothesis, which emphasizes the idea that DA plays an important role in establishing and reinforcing food-seeking behaviors (Beninger, 1983; Wise, 2006). (3) The effort hypothesis, which emphasizes that the effort required in obtaining a reward is 6 DA dependent. Salamone and his colleagues (Salamone, Correa, Farrar, & Mingote, 2007) provided support for this hypothesis by blocking DA in the NAcc during foodseeking behaviors that were critically dependent upon the work requirements of the task – lever pressing schedules that have minimal work requirements are largely unaffected by blocking DA in the NAcc, whereas reinforcement schedules that have high work requirements are substantially impaired by blocking DA in the NAcc. (4) The behavioral switching hypothesis, which emphasizes that short latency firing in the mesencephalic dopaminergic neurons in response to neutral stimuli that are behaviorally significant, is DA dependent (Nicola, 2007; Redgrave, Prescott, & Gurney, 1999). Behavioral switching may underlie associative learning in that the initial dopaminergic neuron firing could represent an essential component in the process of switching behavioral selections to unexpected, behaviorally important stimuli. (5) The reward prediction error hypothesis, which emphasizes that reward prediction error is conveyed via DA itself (Ljungberg, Apicella, & Schultz, 1992; Schultz, 1998). Reward prediction error is the phasic response of DA neurons observed when an unexpected reward is presented. Specifically, Montague and colleagues (Montague, Dayan, & Sejnowski, 1996) provided support for this hypothesis by demonstrating that the release of DA from the VTA cells into cortical targets conveys information about prediction errors between the expected amount of reward and the actual reward. Under this postulate, increases in DA release indicate that the current state is better than expected whereas decreases indicate the current state is worse than expected. These responses transfer to the onset of a CS after repeated pairings with the reward. (6) The incentive-salience hypothesis, which emphasizes the role of DA in reward “wanting” as opposed to “liking” (Berridge, 2007). 7 Incentive salience is a motivational wanting attribute given by the brain to rewardpredicting stimuli. This “wanting” is unlike “liking” in that “liking” is a pleasure immediately gained from consumption or other contact with rewarding stimuli, while the “wanting” is a motivational state aimed at a salient incentive stimulus that makes it a desirable goal and thus transforming it from a sensory experience into something that commands attention, and therefore elicits approach behavior. For instance, electrophysiological studies conducted in animals suggest that DA neurons are activated in response to CSs predictive of food reward to a greater extent than when animals actually eat the food (Naranjo, Tremblay, & Busto, 2001). DA plays a role in salience of rewarding stimuli in that it assists in attaining the most rewarding stimuli to the one concerned. Each of these perspectives encompasses a particular aspect of DA function, and a comprehensive account of DA’s function may involve one or a combination of these perspectives. For instance, Schultz and his colleagues (Schultz, Tremblay, & Hollerman, 2000) recorded DA neurons situated in the mesocorticolimbic DA system during rewardrelated tasks and demonstrated that primates respond to specific aspects of reward behavior such as during the detection of a reward, reward predicting cues (i.e. CSs of predictive food reward) and reward expectation period. Berridge and Robinson (Berridge & Robinson, 1998) argued that activation of midbrain DA pathways attributes incentive salience to and approach toward CSs, and that the CS continues to reinforce incentive salience and approach as long as the reward is present. Nicola and Deadwyler (Nicola & Deadwyler, 2000) reported that neurons of the NAcc fired between, but not during, selfadministrated cocaine infusions thus supporting DA’s involvement in motivation for 8 cocaine and therefore ruling out DA’s involvement in the “liking” associated with rewarding stimuli. Altogether, these findings suggested that DA in the mesocorticolimbic system plays a role in reward-related learning, and that there may be discrepancies in the particular aspects of reward it mediates. This dissertation proposes that DA plays several roles in reward-related learning (see the next section, 2.c.) encompassing different aspects on the aforementioned perspectives. 2.c. DA and its role in reward-related learning Many studies have demonstrated that DA released from within the mesocorticolimbic DA system is strongly implicated in reward-related behaviors. In an early animal study Olds and Milner found that if animals are given the choice between food and direct electrical stimulation of various regions of the mesocorticolimbic DA system, animals preferred the direct stimulation and consequently starved (Olds & Milner, 1954). Later animal studies suggest that DA strengthens and/or reinforces associations between rewarding stimuli and responses (Kelley, 1999; White & Milner, 1992). More specifically, it has been well established that mesocorticolimbic DA plays a critical role in mediating the behavioral and neural effects of primary rewards (Robinson & Berridge, 1993; Wise, 2004; Wise, et al., 1978). Consumption of primary rewards is associated with VTA DA cell activation (Kest, Cruz, Chen, Galaj, & Ranaldi, 2012; Yoshida et al., 1992), the release of DA in terminal regions of the mesocorticolimbic DA system (J. J. Day, Roitman, Wightman, & Carelli, 2007; Hernandez & Hoebel, 1988; Pfaus, Damsma, Wenkstern, & Fibiger, 1995; Yoshida, et al., 1992) and increased approach behavior (Ikemoto, 2007). Blockade of DA neurotransmission in cortical, striatal and midbrain regions reduces or eliminates the rewarding effects of USs such as 9 food (Wise, 1978), brain stimulation and drugs of abuse (for a detailed review see (Wise, 2004)). Thus it appears that mesocorticolimbic DA mediates the effects of primary rewards. In addition to primary rewards, the mesocorticolimbic DA system also plays an important role in mediating the behavioral and neural effects of CSs. Definitively, a foodassociated CS caused increased behavioral approach and elevated levels of VTA DA cell activation (Kest, et al., 2012; A. G. Phillips, Atkinson, Blackburn, & Blaha, 1993) and elevated levels of DA in the NAcc and caudate nucleus in rats (A. G. Phillips, et al., 1993). Furthermore, presentation of CSs is associated with neural activity in the VTA and/or DA release in the DA terminal regions (see, for example, (Blackburn & Phillips, 1989; Carelli, 2000; Carelli & Ijames, 2001; Childress et al., 1999; Gratton & Wise, 1994; Janak, Chang, & Woodward, 1999; Talmi, Seymour, Dayan, & Dolan, 2008). Neuronal activity in the VTA, the source of mesolimbic DA, is also correlated with the presentations of CSs, and indeed may be a necessary condition for responding to CSs (Di Ciano & Everitt, 2004; Murschall & Hauber, 2006). This CS-induced DA activity, in the VTA and the mesocorticolimbic DA terminal regions, is associated with increased approach behaviors; increases in mesolimbic DA can reinstate extinguished lever pressing (Ranaldi, Pocock, Zereik, & Wise, 1999; Stewart, 1984) and are observed just prior to reinforced lever presses (Gratton & Wise, 1994; Kiyatkin & Gratton, 1994; Richardson & Gratton, 1996). Thus, it appears that, in reward-related learning, mesocorticolimbic DA activity not only mediates the effects of primary rewards but also comes under the control of CSs. 10 We are particularly interested in the activation of approach behaviors associated with mesocorticolimbic DA activity. There is evidence that mesocorticolimbic stimulation results in increased forward locomotion, seeking and appetitive behaviors, and that approach behavior is linked with mesocorticolimbic cell activity (Beninger, 1983; Berridge, 2007; Berridge & Robinson, 1998; Mogenson, 1987; Panksepp, 1998; Phillipson, 1979; Wise & Bozarth, 1987). Therefore, it appears that mesocorticolimbic DA plays a key role in the incentive-motivational effects of primary rewards, increasing approach and seeking (Ljungberg, et al., 1992; P. E. Phillips, Stuber, Heien, Wightman, & Carelli, 2003; Romo & Schultz, 1990; Yun, Wakabayashi, Fields, & Nicola, 2004), and therefore maximizing the animals proximity to rewards. In light of the heterogeneous effects of DA, we argue that activation of the mesocorticolimbic DA neurons enhances current motivated behavior to reward-related stimuli and facilitates associative processes underlying reward-related learning. Therefore, we believe that DA probably plays a role in eliciting approach behaviors several times in a sequence during reward-related learning. First, DA release from the VTA during the encounter with primary rewards initially spurs increased investigatory and approach behavior. Second, synaptic changes in the VTA allow neutral stimuli associated with the US to additionally recruit DA cells, augmenting DA release and approach behavior, putting the animal in greater proximity to rewards and related cues. Third, DA release at the terminal regions due to USs and CSs facilitate associative processes that underlie this type of learning. Once CS associations have been acquired, either primary rewards or CSs can continue to activate VTA cells, providing for DA release that will maintain approach behaviors and motivational processes. This suggests 11 that activation of DA pathways within the mesocorticolimbic DA system, in response to USs and CSs, will elicit approach behavior. 3. A proposed Hebbian Model of Reward-Related Learning Several neural models of reward-related learning have been developed, most of which are based on the Hebbian postulate that when a synapse is active at the same time the postsynaptic neuron is active the synapse changes in strength. Furthermore, when two synapses (for example, one associated with a US and another with a CS) are active at the same time the post-synaptic neuron is active, the convergence of the two synaptic events induces neuronal plasticity (Hebb, 1949; Kandel, 2001). We propose here, and others have elsewhere (Beninger, 1983; Kelley, 1999; Ranaldi, 2014; Testa et al., 2005; Zellner & Ranaldi, 2010), that reward-related learning involves similar kinds of neural processes. Furthermore, we propose that the VTA, the source of mesocorticolimbic DA neurons which project to terminal regions such as the NAcc, AMG and mPFC, may constitute a candidate node for where this kind of neural plasticity occurs; the VTA is strongly implicated in reward, receives neuronal afferents that may signal USs and CSs, and has been shown to undergo long term potentiation (LTP) (Bonci & Malenka, 1999; Overton, Richards, Berry, & Clark, 1999) - a form of synaptic plasticity and a putative neural mechanism of learning (Kandel, 2001). It has been suggested that CSs gain access to the same neural circuits, plausibly the mesocorticolimbic DA system, as primary rewards and therefore come to activate these circuits on their own (Bindra, 1974; Ranaldi & Beninger, 1994; Wise, 2004). Others and we have suggested that one component of this process by 3which CSs gain access to these circuits is by way of synaptic plasticity in the VTA (Bonci & Malenka, 1999; 12 Harris, Wimmer, Byrne, & Aston-Jones, 2004; Sharf & Ranaldi, 2006a; Stuber et al., 2008; Zellner, Kest, & Ranaldi, 2009a). One possibility is that a CS, by virtue of its contiguity with a US, acquires the capacity to activate VTA DA cells and hence the DA terminal regions—the same neural substrate activated by the US—, therefore causing conditioned approach and other reward-related behavior similar to the US. We propose a neurobiological model of reward-related learning where the VTA constitutes a primary site where such neural plasticity occurs (Kest, et al., 2012; Ranaldi et al., 2011; Sharf & Ranaldi, 2006a; Zellner & Ranaldi, 2010). In this model the VTA receives signals about both USs and eventual CSs. These signals converge on VTA DA neurons leading to the acquisition by the CS of the capacity to activate DA neurons, and hence DA terminal regions, on its own and elicit conditioned approach. 4. Framework of the VTA and its DA projection sites The mesocorticolimbic DA system begins with the VTA. The VTA is a group of neurons located medial to the substantia nigra, ventral to the red nucleus and between the caudal hypothalamus and brainstem reticular formation. There are different types of neurons located in the VTA which can be divided into distinct subpopulations which participate in the various dopaminergic pathways that mediate the pleasurable effects and the regulation and control of behavior in reward-related learning. The functions of these subpopulations are determined by the neurotransmitter they receive from VTA afferents and those released by VTA efferents. The neurons located in the VTA include DA (Margolis et al., 2006; Swanson, 1982), GABA (Carr & Sesack, 2000), and Glu (Lavin, et al., 2005). The VTA is partially populated (50%) with dopaminergic neurons (Margolis, et al., 2006); DA neurons are the “principal” cells of the VTA. The GABA 13 neurons are the “secondary” cells of the VTA (Johnson & North, 1992). Both DA and GABA neurons project within the VTA itself and outside the VTA (Carr & Sesack, 2000). Recent evidence demonstrates the presence of Glu neurons in the VTA that are non-dopaminergic and non-GABAergic, and may therefore constitute a third type of VTA neuron (Yamaguchi, Sheen, & Morales, 2007). The VTA releases DA in the terminal regions of the mesocorticolimbic DA system. The VTA receives a large number of inputs from various brain regions. There are glutamatergic inputs from the mPFC (Christie, Bridge, James, & Beart, 1985) (Sesack & Pickel, 1992), lateral hypothalamus (LH) (Fields, Hjelmstead, Margolis, & Nicola, 2007), bed nucleus of the stria terminalis (Georges & Aston-Jones, 2002), and the superior colliculus (Geisler & Zahm, 2005). It receives additional glutamatergic, and GABAergic information (Cornwall & Phillipson, 1988; Oakman, Faris, Kerr, Cozzari, & Hartman, 1995) from the pedunculopontine tegmental nucleus (PPTg), AMG, and the posterior lateral dorsal tegmental nucleus (LDT) (Fibiger, Phillips, & Brown, 1992). GABAergic inputs to the VTA also arise from the ventral pallidum (Geisler & Zahm, 2005) and the NAcc (Conrad & Pfaff, 1976). There are also cholinergic afferents from the PPTg and LDT (Fibiger, et al., 1992), serotonergic projections from the dorsal raphe nucleus as well as inputs from the pontine, cerebellar and medullary nuclei (Geisler & Zahm, 2005) and AMG (Wallace, Magnuson, & Gray, 1992). The diverse afferents to the VTA release a variety of neurotransmitters to specific targeted subpopulations of VTA neurons. The glutamatergic afferents from the PFC, raphe nucleus and LH target the subpopulation of VTA DA neurons which project back to the PFC (Fields, et al., 2007). Additional inputs to the VTA DA neurons include 14 cholinergic afferents (Omelchenko & Sesack, 2006). Furthermore, the LDT has glutamatergic and GABAergic afferents to the VTA which synapse onto dopaminergic and GABAergic VTA neurons which also project back to the PFC (Fields, et al., 2007). These diverse inputs mediate complex synaptic actions within the VTA which in turn determine the type of outputs leaving the VTA. Efferents of the VTA project to targeted brain regions, which receive inputs from a distinct subpopulation of VTA neurons. Dopaminergic fibers, which receive input from the PFC, LH, PPTg and LTD, target the NAcc (Floresco, West, Ash, Moore, & Grace, 2003), lateral septal area, AMG (Bjorklund & Dunnett, 2007) - specifically to the basolateral AMG (BLA) (Fallon & Ciofi, 1992; Howland, Taepavarapruk, & Phillips, 2002), mPFC (Fuxe & et al., 1974; Swanson, 1982) via the mediodorsal thalamus, and the hippocampus. There are glutamatergic (Lavin, et al., 2005) and GABAergic (Carr & Sesack, 2000; Margolis, et al., 2006) efferents, that receive input from the LTD, which project to the NAcc and PFC (Wise, 2004) . It is within these multifaceted inputs and outputs from subsets of VTA neurons that separate circuits are shaped and formed which help form the underlying mechanisms of reward-related learning. Each DA terminal region has two things in common: (1) they all receive information from the VTA and (2) they all send information back to the VTA. Efferents of the NAcc are GABAergic and project back to the VTA and the ventral pallidum (VP; (Heimer, Zaborszky, Zahm, & Alheid, 1987)). Mogenson and his colleagues (Mogenson, Jones, & Yim, 1980) proposed that the NAcc projections to the VP translate mesolimbic motivation signals into motor output, a neural circuit which may contribute to approach behaviors. The inputs and outputs of the NAcc could play an important role in behaving 15 animals to adapt to different environmental conditions. The BLA sends Glu fibers to the mPFC and NAcc (Spooren, Veening, Groenewegen, & Cools, 1991). Activation of the NAcc, from the BLA, results in activation of the VP, which then initiates motor systems involved particularly in goal-directed behaviors – an underlying theme in reward-related learning. Also, Floresco and colleagues (Floresco, Yang, Phillips, & Blaha, 1998) suggest that the glutamatergic BLA inputs to NAcc dopamine terminals synaptically facilitate or depress dopamine efflux, and these effects are independent of DA neuronal firing activity. Moreover, they also imply that changes in NAcc DA levels following presentation of reward-related stimuli may be mediated, in part, by the BLA. The glutamatergic fibers of the central nuclues of the AMG connect back to the VTA providing feedback to its DA projection site, perhaps enhancing the effects of the connections from the VTA to the BLA. The mPFC sends its glutamatergic efferents (from the pyramidal cells in the pyramidal layer) back to the VTA, the NAcc (Sesack, Deutch, Roth, & Bunney, 1989; Wright & Groenewegen, 1995) and the BLA forming a closed neural loop in the mesocorticolimbic DA system. Furthermore, stimulation of the mPFC results in Glu and DA release in the NAcc; Glu release may be directly from the mPFC pyramidal cells to the VTA and then to the NAcc (Rossetti & Wise, 1995), and the DA release may be indirect through the VTA . The role of the mPFC in this neural loop may involve the formation of the stimulus-reward associations, specifically in higher order information processing such as extracting and retaining the “associative significance” of a stimulus independent of its physical properties (Watanabe, 1990). The inputs and outputs of the mesocorticolimbic DA system, as well as the subpopulations of nuclei, are linked together and allow for communication amongst many brain regions. 16 This interconnection allows for the modification, or in other words “synaptic plasticity” including the strengthening and weakening, of incoming and outgoing signals that underlie reward-related learning. 5. VTA and reward-related learning There is evidence that the VTA receives signals about USs. The VTA receives acetylcholine (ACh) afferents from PPTg and LDT (Bolton, Cornwall, & Phillipson, 1993; Garzon, Vaughan, Uhl, Kuhar, & Pickel, 1999; Henderson & Sherriff, 1991; Oakman, et al., 1995). In the VTA, extracellular concentrations of ACh increase during eating, drinking and self-stimulation of the lateral hypothalamus (Rada, Mark, Yeomans, & Hoebel, 2000). In addition, injections of muscarinic ACh (mACh) receptor antagonists in the VTA reduce eating both in our laboratory (Sharf, McKelvey, & Ranaldi, 2006; Sharf & Ranaldi, 2006a, 2006b) and in others (Rada, et al., 2000) and reduces approach and consummatory responses for food (Ikemoto & Panksepp, 1996). In-vivo (Gronier & Rasmussen, 1998) and in-vitro (Calabresi, Lacey, & North, 1989) studies show that stimulation of mACh receptors in the VTA depolarizes presumed DA neurons and releases DA in terminal regions of the mesocorticolimbic system (A. D. Miller & Blaha, 2005; Westerink, Enrico, Feimann, & De Vries, 1998). These findings support the idea that VTA mACh receptor stimulation is involved in mediating some of the unconditioned (i.e., rewarding or incentive motivational) effects of primary rewards (USs), including food, through activation of VTA DA neurons. It may be presumed that there is a CS signal to the VTA mediated by glutamatergic afferents originating in the hindbrain and the forebrain (Geisler, Derst, Veh, & Zahm, 2007), including from regions known to process environmental stimuli 17 such as PFC, AMG, bed nucleus of the stria terminalis, and superior colliculus (Sesack & Pickel, 1992; Smith, Charara, & Parent, 1996). It is possible, then, that the VTA receives a Glu signal that is in some way related to reward-associated environmental stimuli and which might be involved in the acquisition of reward-related learning. For instance, convergence of signals from reward-associated stimuli and USs on VTA DA neurons may lead to LTP-like neural plasticity, resulting in strengthening of CS-related synapses causing conditioned activation of DA neurons and conditioned reward responding. Indeed, some evidence of this exists. Microinjections of NMDA receptor antagonists in the VTA block the development of morphine conditioned place preference (Harris, et al., 2004), the acquisition of a food-reinforced operant response (Zellner, Kest, & Ranaldi, 2009b) and the acquisition of a conditioned approach response (Ranaldi, et al., 2011; Stuber, et al., 2008). This convergent stimulation creates the conditions for LTP which is in many cases dependent on NMDA receptor stimulation (Citri & Malenka, 2007). NMDA receptors are found in the VTA (Rodriguez, Doherty, & Pickel, 2008) and LTP has been demonstrated in the VTA DA neurons where it is NMDA receptor dependent (Bonci & Malenka, 1999). NMDA receptor stimulation can result in initiation of second messenger cascades implicated in LTP and other intracellular events that lead to neurochemical and structural changes associated with learning. Such synaptic changes could result in what may initially be a weak glutamate signal (i.e., one that is not able to robustly activate VTA DA neurons) becoming one that is able to robustly activate DA cells and causing behaviorally adequate DA release. This increased DA release, triggered by strengthened CS-related synapses in the VTA, might lead to increased approach behaviors elicited by 18 CSs, increasing the likelihood of direct contact with rewards or processes downstream in DA terminal regions which will facilitate other regions involved in reward-related behaviors. 6. A more detailed proposed Hebbian Model of reward-related learning in the VTA Our proposed Hebbian model of synaptic plasticity in the acquisition of rewardrelated learning involves activity dependent changes in the strength of synaptic connections among US and CS sets of neurons, brought about by the conjoint activity of these neurons on DA neurons of the VTA. Specifically, we propose that when animals consume a reward in a given environment, US-stimulated ACh release activates VTA DA cells, leading to increased DA in mesocorticolimbic terminal regions (A. D. Miller & Blaha, 2005; Schilstrom, Svensson, Svensson, & Nomikos, 1998; Westerink, et al., 1998), which stimulates increased approach behavior (Beninger, 1983; Berridge, 2007; Mogenson, 1987; Wise & Bozarth, 1987). As the VTA DA cells are stimulated by the ACh signal (i.e., the US), they also receive relatively weak excitation from Glu afferents conveying information about environmental stimuli. This coincident stimulation of ACh and Glu receptors on VTA DA neurons induces LTP-like processes in VTA DA neurons, resulting in the previously weak synapses becoming strong enough to activate the VTA DA cells in their own right. These strengthened synapses now allow reward-associated stimuli to function as CSs, increasing the levels of approach toward rewards. Up to date, much research indicates that VTA DA activity comes under the control of CSs and suggests that this activity is a necessary component of the associative processes underlying reward-related learning. 19 On a physiological level, we propose that a US (food) stimulates ACh release which activates mACh receptors at the level of the VTA resulting in strong depolarization of VTA DA neurons; depolarization of the VTA DA cells removes a magnesium ion (Wu & Johnson, 1996) blocking the NMDA receptor channel. Removal of the magnesium block, allows for calcium to flow through the NMDA receptor in response to CS-stimulated Glu release at the level of the VTA leading to the initiation of calcium-dependent second messenger cascades. Activation of the NMDA receptors allow for an influx of intracellular calcium. High levels of calcium allow itself to bind to calmodulin (a calcium binding protein) thus forming CaM which can activate CaM dependent kinases such as CamKII (a calcium sensitive serine/threonine protein kinase) – an enzyme which causes autophosphorylation and/or phosphorylation and is crucial for LTP that underlies reward-related learning (Lisman, Schulman, & Cline, 2002). CamKII, in addition to PKA, regulates (phosphorylates) Creb, which results in gene transcription (see Figure 1 depicted from (Ranaldi, 2014)). In addition to NMDA receptor activation, mACh receptor activation also results in a long-term molecular change in VTA DA cells. Ach can trigger a conformational change in the mACh receptor therefore exposing a binding site for a G-protein. The newly formed GPCR causes displacement of a GDP with a GTP on its alpha subunit, activating phosphoinositide (PI) located in the cell membrane. PI forms phosphoinositide phosphate (PIP) and in the presence of protein kinase C (PKC), PIP is broken down into inositol triphosphate (IP3) and diglyceride (DAG); IP3 increases the amount of calcium channels in the endoplasmic reticulum and DAG is IP3 dependent – it stimulates PKC when there are high levels of Ca therefore enhancing LTP and the strengthening of future synaptic events. Thus, if US (mACh 20 receptor) and CS (NMDA receptors) stimulation at the level of the VTA occurs simultaneously, then the originally weak excitations from Glu afferents (presumably mediating reward-associated stimulus information) may become strengthened. 7. Hypothesis According to the Hebbian model coincident US (from ACh stimulation) and CS (from Glu stimulation) signals on VTA DA cells are proposed to lead to CSs to acquire the ability to activate VTA DA cells and corticolimbic DA terminal regions on their own and elicit conditioned approach, in a manner similar to the US. Furthermore, this type of learning has been proposed to be dependent on VTA NMDA receptors. To test this model of reward-related learning, we hypothesized that (1) a food US will activate VTA DA cells, (2) a food-associated CS (light) will acquire the capacity to activate VTA DA cells and cause conditioned approach, and (3) that AP-5 (2-amino-5-phosphonopentanoic acid; a competitive NMDA receptor antagonist) in the VTA will impair the acquisition, but not the expression, of conditioned approach on a behavioral level and the expression of c-Fos in DA terminal regions on a neural level. c-Fos is an amino-acid protein encoded by an immediate early gene which allows for increased expression of that particular gene. The expression of c-Fos has been shown to be an indication of neuronal activity (Dragunow & Faull, 1989), and specifically recent neuronal activity (H. E. Day, Kryskow, Nyhuis, Herlihy, & Campeau, 2008). Since c-Fos indicates recent neuronal activity, we will measure cellular activity by labeling and then quantifying c-Fos. To test the first hypothesis, rats were exposed to a food study in which we hypothesized that rats consuming the food pellets will display a significantly greater number of VTA DA (tyrosine hydroxylase (TH)-labeled) cells expressing c-Fos than rats that are not presented with food pellets. To test the second hypothesis, rats were exposed 21 to a conditioned-approach study in which we hypothesized that a light functioning as a CS will cause a significantly greater number of VTA DA cells to express c-Fos than the same light not functioning as a CS. To test the third hypothesis, rats were surgically prepared with indwelling cannulae positioned so as to allow bilateral microinjections of AP-5 or vehicle in the VTA. Rats were then exposed to either an acquisition or an expression study in which we hypothesized that AP-5 will significantly impair the acquisition of conditioned approach and significantly reduce the amount of c-Fos expressed in the mesocorticolimbic DA terminal regions in response to the CS relative to vehicle treatment. It was also expected that AP-5 would not impair the expression of the already learned conditioned approach response. Altogether our expected results support our model of reward-related learning, namely that a CS acquires, via the VTA NMDA receptor, the capacity to cause conditioned activation of VTA DA cells and mesocorticolimbic DA terminal regions therefore eliciting conditioned approach. 22 Figure 1: An illustration of the proposed Hebbian Model of reward-related learning in the VTA. Shown are the components of the proposed neural mechanisms necessary for a neutral stimulus the ability to acquire the capacity to activate a VTA DA neuron on thier own therefore acting as a conditioned stimulus that can cause a conditioned approach response. US, mAch receptor stimulation allows for activation of VTA DA neurons and then activation of the DA terminal regions that cause approach. CS, NMDA receptor stimulation, allows for the conditioned activation of VTA DA neurons and then conditioned activation of the DA terminal regions that cause conditioned approach. When there is coincident mAch and NMDA receptors stimulation in the VTA DA neurons, calcium flowing through the NMDA receptor initiates intracellular cascades resulting in long-term changes in neural activity. It is proposed that these changes in neural activity allow CS the ability to acquire the capacity to activate VTA DA neurons on their own and elicit conditioned approach. Image was prepared and displayed by Dr. Robert Ranaldi (Ranaldi, 2014). 23 Chapter 2. 1. Introduction: A food-associated CS activates c-Fos in VTA DA neurons and elicits conditioned approach Reward-related learning is an essential adaptive function for survival in which organisms acquire behaviors that place them in contact with stimuli that function as primary rewards (unconditioned stimuli, USs; e.g., food, water) – which consequently elicit approach behaviors and reinforce these behaviors. Neutral stimuli associated with primary rewards acquire the ability to act as conditioned stimuli (CSs), which can elicit conditioned approach responses similar to the primary rewards with which they are associated. The neural mechanisms underlying this type of reward-related learning are not fully understood. We propose that conditioned approach learning involves the acquisition by CSs of the capacity to cause conditioned activation of the same neural systems activated by primary rewards, thereby eliciting the conditioned response (CR). The present experiments were designed to test this model. Several neural models of reward-related learning have been developed, most of which are based on the Hebbian postulate that when a synapse is active at the same time the postsynaptic neuron is active the synapse changes in strength. Furthermore, when two synapses (for example, one activated by a US and another by a stimulus with a contiguous relation with the US) are active at the same time the postsynaptic neuron is active, the convergence of the two synaptic events induces neuronal plasticity (Hebb, 1949; Kandel, 2001). We propose here, as others have elsewhere (Beninger & Hahn, 1983; Beninger & Ranaldi, 1994; Kelley, 1999; Zellner & Ranaldi, 2010) that rewardrelated learning involves similar kinds of neural processes. Furthermore, we propose that the ventral tegmental area (VTA), the source of mesocorticolimbic dopamine (DA) 24 neurons, may constitute a candidate node for where this kind of neural plasticity occurs; the VTA, as detailed below, is strongly implicated in reward, receives neuronal afferents that may signal USs and CSs, and has been shown to undergo long term potentiation (LTP) (Bonci & Malenka, 1999; Overton, et al., 1999) - a form of neuronal plasticity and a putative neural mechanism of learning. Mesocorticolimbic DA plays a critical role in mediating the behavioral effects of primary rewards (Robinson & Berridge, 1993; Wise, 2004). It is well established that consumption of primary rewards is associated with the release of DA in terminal regions of the mesocorticolimbic DA system (J. J. Day, et al., 2007; Hernandez & Hoebel, 1988; Pfaus, et al., 1995; Yoshida, et al., 1992) and increased approach behavior (Ikemoto, 2007). Blockade of DA neurotransmission in cortical, striatal and midbrain regions reduces or eliminates the rewarding effects of USs such as food, brain stimulation and drugs of abuse (for a detailed review see (Wise, 2004; Zellner & Ranaldi, 2010). Thus it appears that mesocorticolimbic DA mediates the effects of primary rewards. In addition to primary rewards, the mesocorticolimbic DA system also plays an important role in mediating the behavioral effects of CSs (Wise, 2004; Zellner & Ranaldi, 2010). Phillips and colleagues (A. G. Phillips, et al., 1993) demonstrated that a foodassociated CS caused increased behavioral approach and elevated levels of DA in the NAcc and caudate nucleus in rats. This CS-induced DA activity is associated with increased approach behaviors; increases in mesolimbic DA can reinstate extinguished lever pressing (Ranaldi, et al., 1999; Stewart, 1984) and are observed just prior to reinforced lever presses (Gratton & Wise, 1994; Kiyatkin & Gratton, 1994; Richardson & Gratton, 1996). Thus, it appears that, in reward-related learning, mesocorticolimbic DA 25 activity comes under the control of CSs. Precisely how this happens remains to be determined. One possibility is that a CS, by virtue of its contiguity with a US, acquires the capacity to activate VTA DA neurons – the same neural substrate activated by the US –and thus causes conditioned approach and other reward-related behavior similar to the US. We propose a neurobiological model of reward-related learning where the VTA constitutes a primary site where such neural plasticity occurs (Sharf & Ranaldi, 2006a; Zellner & Ranaldi, 2010). In this model the VTA receives signals about both USs and eventual CSs that converge on DA neurons leading to the acquisition by the CS of the capacity to activate DA neurons on its own and elicit conditioned approach (the CR). There is evidence that the VTA receives signals about USs. The VTA receives acetylcholine (ACh) afferents from the pedunculopontine tegmental nucleus (PPTg) and the posterior lateral dorsal tegmental nucleus (LDT) (Bolton, et al., 1993; Garzon, et al., 1999; Henderson & Sherriff, 1991; Oakman, et al., 1995). In the VTA, extracellular concentrations of ACh increase during eating, drinking and self-stimulation of the lateral hypothalamus (Rada, et al., 2000). In addition, injections of muscarinic ACh (mACh) receptor antagonists in the VTA reduce eating both in our laboratory (Sharf, et al., 2006; Sharf & Ranaldi, 2006a, 2006b) and others (Rada, et al., 2000) and reduces approach and consummatory responses for food (Ikemoto & Panksepp, 1996). In-vivo (Gronier & Rasmussen, 1998) and in-vitro (Calabresi, et al., 1989) studies show that stimulation of mACh receptors in the VTA depolarizes presumed DA neurons and releases DA in terminal regions of the mesocorticolimbic system (A. D. Miller & Blaha, 2005; Westerink, et al., 1998). These findings support the hypothesis that VTA mACh receptor 26 stimulation is involved in mediating some of the unconditioned (i.e., rewarding or incentive motivational) effects of primary rewards (USs), including food. There is also evidence of a putative CS signal in the VTA mediated by glutamatergic afferents originating in the hindbrain and the forebrain (Geisler, et al., 2007), including from regions known to process environmental stimuli such as frontal cortical areas, amygdala, bed nucleus of the stria terminalis, superior colliculus and others (Sesack & Pickel, 1992; Smith, et al., 1996). It is possible, then, that the VTA receives glutamate (Glu) signals that are in some way related to reward-associated environmental stimuli and which might be involved in the acquisition of reward-related learning. Indeed, some evidence of this exists. Microinjections of NMDA receptor antagonists in the VTA block the development of morphine conditioned place preference (Harris, et al., 2004), the acquisition of a food-reinforced operant response (Zellner, et al., 2009b) and the acquisition of a conditioned approach response (Ranaldi, et al., 2011; Stuber, et al., 2008). Given the role of mesocorticolimbic DA in reward and the evidence that the VTA receives US- and CS-related signals, we propose that the VTA is a crucial site for neural plasticity underlying reward-related learning. In this model, conditioned approach learning occurs when a reward-associated stimulus acquires the capacity to activate VTA DA neurons and elicit conditioned approach (the CR). If this model is correct then a US (e.g., food) should activate VTA DA neurons and a CS (e.g., food-paired light) should acquire the capacity to activate VTA DA neurons and elicit conditioned approach. Recent data from Sombers and colleagues (Sombers, Beyene, Carelli, & Wightman, 2009) is in accordance with this prediction. These researchers found that a cue that 27 signaled availability of brain stimulation reward (BSR) activated VTA DA cells and initiated self-stimulation. This demonstrates that a tonic (contextual) cue that is predictive of artificial reward (i.e., BSR) ‘availability’ causes conditioned activation of VTA DA neurons. In the present experiments we tested whether or not a phasic, discrete cue associated with natural reward (food) and which elicits conditioned responses per se, can activate VTA DA cells. Specifically, we hypothesized that a food-associated CS (light) would acquire the capacity to elicit conditioned approach and activate VTA DA neurons. To quantify DA cell activation we used immunohistochemical procedures that labeled neurons with tyrosine hydroxylase (TH) to identify them as DA neurons and stained for c-Fos to identify activated neurons; cells labeled with both TH and c-Fos represented activated DA cells. 2. Methods The protocols used in the present experiments were in accordance with the National Institutes of Health Guide for Care and Use of Laboratory Animals and were approved by the Queens College Institutional Animal Care and Use Committee. 2.a. Subjects Subjects consisted of 68 male Long Evans rats, facility-bred from males and females obtained from Charles River Laboratories (Raleigh, NC), with initial free-feeding weights between 300 and 400 grams. All rats were individually housed and maintained on a 12 h light:12 h dark cycle (lights off at 0600). All experimental sessions were conducted during the dark phase in order to test the rats during their active periods. All rats had unlimited access to food (Purina rat chow) until experimental sessions started, at 28 which time access was restricted to daily rations that maintained their weights at 85% of their free-feeding values. 2.b. Apparatus 2.b.i. Conditioning chambers All behavioral testing was conducted in eight conditioning chambers each measuring 30 x 21 x 18 cm (l x w x h). One wall was equipped with a food trough and two white stimulus lights, each situated one inch above and two inches to the right or left of the food trough. Each chamber was housed in a ventilated, sound-attenuating box. A PC through a MED Associates interface controlled the chambers. 2.b.ii. Foot-shock chambers All foot-shock stimuli were administered in 8 chambers each measuring 30 x 21 x 18 cm (l x w x h). These chambers were equipped with aluminum rod floors. Each rod was connected to a constant-current aversive stimulator. Each chamber was housed in a ventilated, sound-attenuating box. A PC through a MED Associates interface controlled the foot-shock administration. 2.b.iii. Locomotor activity chambers Locomotor activity was measured in 6 open field activity monitors, each measuring 44.5 x 44.5 x 30.5 cm (l x w x h). Each chamber was equipped with 16 photoemitters/photocells along the length and 16 along the width, all 4 cm above the floor. Locomotor activity counts were registered by the photocells when adjacent beams were broken consecutively. 29 2.c. Experiments Starting one day after beginning the food restriction diet each rat was given 20 food pellets (45 mg, Bioserv, Frenchtown, NJ) in its home cage on each of three days. All rats were then tested in either a feeding (Experiment 1), a conditioned approach learning (Experiment 2) or a stress experiment (Experiment 3). The feeding experiment tested whether or not food (the US) can activate c-Fos in VTA DA [tyrosine-hydroxylase (TH)-labeled] cells. The conditioned approach experiment tested whether or not a foodassociated CS (a 3-s light presentation) can cause conditioned approach and activation of c-Fos in VTA TH-labeled cells. The stress experiment was an additional control experiment testing the possibility that reward omission may cause stress, measured as suppressed locomotor activity. 2.c.i. Experiment 1: The effects of a US (food) on c-Fos in VTA TH-labeled cells Rats were exposed to a protocol that consisted of 4 daily 10 min sessions in test chambers. For sessions 1-3 all rats were habituated to the test chambers with no US. During session 4, half the rats were given 30 food pellets in a dish to eat in the test chamber while the other half was given an empty dish (US vs. no US). Rat brains were then harvested and immunostained for c-Fos followed by TH. 2.c.ii. Experiment 2: The effects of a CS on c-Fos in VTA TH-labeled cells Rats were exposed to a protocol that consisted of 6 daily sessions. During session 1 all rats were given 20 min of magazine training in which 20 food pellets were delivered on a random time schedule. For sessions 2-4 rats were randomly assigned to one of two 30 groups, US-CS explicitly paired (EP) or US-CS not explicitly paired (NEP) groups. Each of these sessions was 60 min long. During these conditioning sessions, 30 food pellets were delivered on a random time 120-s schedule (range of 15 to 245 s). For rats in the EP group, each pellet delivery was preceded by a 3-s presentation of a light on the left side of the trough. For rats in the NEP group the light presentations (on left side of the trough) and food deliveries were not correlated. After the 3 conditioning sessions all rats received one 30-min session during which there were no light or food presentations. This was followed by a CS-only test session in which rats were presented with light presentations under the same random time schedule as in conditioning but with no food. Rat brains were then harvested and immunostained for c-Fos followed by TH. For all rats head entries during each session were counted and analyzed (see Data Analysis section below for details). 2.c.iii. Experiment 3: The effects of reward omission on locomotor activity Four groups of rats were used in this experiment. Two groups of rats were exposed to a protocol that consisted of 4 daily, 1-h sessions in the shock chambers each followed by 30 min sessions in the locomotor activity chambers. For sessions 1-3 all rats were habituated to the test chambers with no shock. After the third locomotor activity session, all rats were assigned to a shock or a no-shock group with group assignment matched to their locomotor activity levels during session 3. During session 4, the rats in the shock group received randomly delivered foot-shocks (0.8 mA; 1 s on; mean off period of 120 s) in the shock chambers while the no-shock group did not. All rats were then placed in the locomotor activity chambers for 30 min and locomotor activity was measured. 31 The other two groups were exposed to the same conditioned approach protocol described in Experiment 2 with the exception of the following (one group was exposed to the EP and the other to the NEP protocol). First, for 3 days prior to the conditioned approach procedure, rats were habituated to the locomotor activity chambers for 30 min each day. Second, immediately after the CS-only test session, rats were placed in the locomotor chambers for 30 min and locomotor activity was measured. 2.d. Immunohistochemistry Seventy-five minutes after the end of the last session in Experiments 1 and 2 all rats were anesthetized with sodium pentobarbital in preparation for perfusion. While under deep anesthesia the rats were perfused through the heart first with 0.9% saline followed by a phosphate-buffered (0.1 M) fixative containing 4% paraformaldehyde. Brains were removed from the skull and fixed with 4% paraformaldehyde overnight at 4° C and sectioned through the ventral tegmental area in the coronal plane on a vibratome. Free-floating sections (40 µm) were collected into different wells for immunocytochemistry. Sections were first washed with phosphate-buffered saline (PBS) and then blocked in 5% NGS and 0.2% Triton X-100 for 1 h. Sections were then incubated with primary antibodies (rabbit anti-CFOS 1:5000, Calbiochem) in 0.1% Triton X-100, 2.5% NGS, and PBS at 4° C overnight. Sections were rinsed several times with 2.5% NGS in PBS and then incubated in biotinylated secondary antibodies (biotinylated goat anti-rabbit; 1:200, Vector Labs, Burlingame, California) for 2 h at room temperature. Sections were rinsed several times with PBS and incubated for 1 h in an avidinhorseradish peroxidase mixture (Vector Labs, Burlingame, California). Sections were rinsed in PBS and then reacted with nickel chloride and 0.05% diaminobenzidine (DAB) 32 in the presence of 0.0015% H2O2. This procedure was completed twice, first for c-Fos and then for TH using a TH primary antibody (TH anti-TH). Sections were collected onto gelatin-coated slides, dried for several hours, and coverslipped with Cytoseal. 2.e. Data Analysis Neural activity data (from Experiments 1 and 2) consisted of c-Fos counts in VTA TH-labeled cells. To quantify VTA TH-labeled cells expressing c-Fos we identified from Paxinos and Watson (1986) brain slices for the anterior (plate 37), medial (plate 39) and posterior (plate 41) planes of the VTA that were common to all rats. VTA TH-labeled cells expressing c-Fos were counted on an Olympus (Tokyo, Japan) BX51W microscope with a motorized stage at 4x magnification. Stereo Investigator software (MicroBrightField, Williston, VT) was used to outline regions of the VTA. The numbers of VTA TH-labeled cells expressing c-Fos in each VTA region were added to arrive at a VTA total count for each rat. To prevent bias that may skew the results, all counting was conducted blind to the treatment groups. Conditioned approach data (from Experiments 2 and 3) consisted of the number of food trough head entries made during (1) a 6-s period immediately preceding the onset of the CS (pre-CS period), (2) the 3-s period during the CS, (3) a 3-s period immediately following the offset of the CS and (4) at all other times (non-CS period). For analyses the 3-s periods during and immediately following the CS were combined for a 6-s total period and referred to as the CS period. For each session the total number of head entries during the CS periods and the total number of head entries during the pre-CS periods were used to calculate the CS/pre-CS ratio. This ratio indicates the magnitude of the 33 conditioned approach response (i.e., the degree to which food trough head entries were elicited by the CS). Locomotor activity data (Experiment 3) consisted of locomotor activity counts (consecutive beam breaks) during 30 min sessions in the locomotor chambers. 2.e.i. Experiment 1: The effects of a US (food) on c-Fos in VTA TH-labeled cells A one-way analysis of variance (ANOVA) with group (US and no US) as the factor was used to analyze the amounts of VTA TH-labeled cells expressing c-Fos in the groups. 2.e.ii. Experiment 2: The effects of a CS on c-Fos in VTA TH-labeled cells A mixed-design ANOVA, with group (EP and NEP) as a between groups factor and session as a repeated measures factor was conducted on the CS/pre-CS ratio data from the 3 conditioning sessions (sessions 2-4). A one-way, between groups ANOVA was conducted on the CS/pre-CS ratio data from the CS-only test session. A one-way ANOVA, with group (EP and NEP) as the factor was conducted to analyze differences in the amount of VTA TH-labeled cells expressing c-Fos between groups. 2.e.iii. Experiment 3: The effects of reward omission on locomotor activity Foot shock groups: An independent samples, one-tailed t-test was used to compare the locomotor activity counts between the shock and no-shock groups. Conditioned approach/locomotor activity tested groups: A one-way, between groups ANOVA was conducted on the CS/pre-CS ratio data from the CS-only test 34 session. An independent samples, one-tailed t-test was used to compare the locomotor activity counts between the EP and NEP groups. Criterion for significance was p < 0.05 in all cases. 3. Results 3.a. Experiment 1: The effects of a US (food) on c-Fos in VTA TH-labeled cells The US group (i.e., rats who ate the food pellets) demonstrated a significantly greater number of VTA TH-labeled cells expressing c-Fos than the no US group (i.e., rats that did not eat food) (see Figure 2). A one-way ANOVA revealed a significant group effect (F(1,14) 12.819, p < 0.003). Figure 3 contains representative images of VTA slices, in identical regions of the VTA, showing double-labeled TH/c-Fos cells in rats receiving US or no US as well as a representative image of a double-labeled TH/c-Fos cell and a TH labeled cell in the VTA. 3.b. Experiment 2: The effects of a CS on c-Fos in VTA TH-labeled cells The EP group (i.e., rats who had the US and CS explicitly paired) demonstrated significantly more food trough head entries during the CS presentations than during no CS presentations, as indicated by significantly greater CS/pre-CS ratios, than the NEP group (i.e, rats who did not have the US and CS explicitly paired) during the 3 conditioning sessions (see Figure 4). A two way ANOVA with group and session (repeated measures) as factors showed significant group (F(1,18)=22.574,p<0.0005), session (F(1,16)=4.984,p<0.012), and session X group (F(2,36)=5.10,p<0.009) effects. Tests of simple effect of session in each group revealed that the EP group showed a significant (F(1,36)=10.032,p=.01) increase in the CS/pre-CS ratio across the 3 conditioning sessions 35 whereas the NEP group did not. The EP group also demonstrated a significantly greater CS/pre-CS ratio than the NEP group during the CS-only test session (see Figure 4). A one-way ANOVA showed a significant group effect (F(1,18)=4.288,p<0.05). The EP group demonstrated a greater number of VTA TH-labeled cells expressing c-Fos than the NEP group during the CS-only test session (see Figure 5). A one-way ANOVA showed a significant group (F(1,18)=22.460,p<0.001) effect. Figure 6 depicts representative images of VTA slices, in identical regions of the VTA, showing double-labeled TH/c-Fos cells in rats from the EP and NEP groups. 3.c. Experiment 3: The effects of reward omission on locomotor activity The group that received intermittent foot shock prior to the locomotor activity test demonstrated significantly less locomotor activity than the group that did not receive foot shock prior to this test (t(14) = -1.756, p < 0.05) (see Figure 7). Of the conditioned approach groups that were later tested in locomotor activity the EP group demonstrated significantly more food trough head entries during the CS presentations than when no CS was present, indicated by significantly greater CS/pre-CS ratios, than the NEP group during the CS-only test session (see Figure 8A). A one-way ANOVA revealed a significant group effect (F(1, 14) = 30.10, p < 0.001). Immediately following this CS-only test was the locomotor activity test, which showed virtually identical amounts of locomotor activity in the EP and NEP groups (see Figure 8B; an independent samples ttest revealed no significant effect between the groups). 36 Number of TH/c-Fos labeled cells 2500 * 2000 US no US 1500 1000 500 0 Group Figure 2: Mean (±SEM) counts of VTA TH-labeled cells expressing c-Fos in the VTA for rats in the US (food pellets) and no US (no food pellets) groups. * represents a significant group difference 37 Figure 3: Representative VTA slices showing double-labeled TH/c-Fos cells in rats receiving US and no US. Below is an image showing a representative TH/c-Fos labeled cell (black arrow) and a representative TH-only labeled cell (white arrow) in the VTA. These representative images were taken at 4x magnification that was used to quantify cell counts. 38 Figure 4: Mean (±SEM) CS/Pre-CS food trough head entry ratios for EP and NEP groups during the 3 conditioning and CS-only test sessions. * represents a significant group difference. Number of TH/c-Fos labeled cells 400 39 * 300 EP NEP 200 100 0 Group Figure 5: Mean (±SEM) counts of VTA TH-labeled cells expressing c-Fos in the VTA for animals in the EP and NEP groups. * represents a significant group difference. 40 Figure 6: Representative VTA slices showing double-labeled TH/c-Fos cells in the EP and NEP groups. These representative images were taken at 4x magnification that was used to quantify cell counts. 41 1200 Locomotor counts 1000 * 800 no shock shock 600 400 200 0 Group Figure 7: Mean (±SEM) locomotor counts for the groups receiving foot shock or no foot shock just prior to placement in the locomotor activity chambers. * represents a significant group difference. 42 A B 8 EP NEP 1600 * 1400 Locomotor counts 6 CS/pre-CS ratio EP NEP 1800 4 2 1200 1000 800 600 400 200 0 0 1 2 3 Session Test Group Figure 8: A. Mean (±SEM) CS/Pre-CS food trough head entry ratios for EP and NEP groups during the 3 conditioning and CS-only test sessions. * represents a significant group difference. B. Mean (±SEM) locomotor counts for the EP and NEP groups tested immediately after the CS-only test session. 43 4. Discussion The data from Experiment 1 demonstrated a greater number of VTA TH-labeled cells expressing c-Fos in animals that ate food than in animals that did not. These data strongly suggest that food, a US, activates VTA DA neurons. The data from Experiment 2 demonstrated that a stimulus explicitly paired (EP) with the US (food reward) produced a greater number of VTA-TH labeled cells expressing c-Fos and greater food trough head entries (the conditioned approach response) than a stimulus not explicitly paired (NEP) with the US. In Experiment 3, the fact that the EP and NEP groups showed identical levels of locomotor activity rules out the possibility that the greater VTA DA cell activation observed in the EP group of Experiment 2 was due to group differences in reward omission-induced stress. Altogether, these results strongly suggest that a foodassociated CS acquires the capacity to cause conditioned activation of VTA DA neurons. Although this experiment did not directly test the causal relation between conditioned activation of VTA DA neurons and conditioned approach learning, the finding that significantly more VTA DA cell activation was observed when the light stimulus served a behavioral CS function than when it didn’t suggests such a causal relation. Overall, the present results provide support for our neurobiological model of reward-related learning (Zellner and Ranaldi, 2010; (Ranaldi, 2014). An alternative explanation for the significant amount of VTA TH-labeled cells expressing c-Fos in Experiment 2 is that this expression is due to the light itself, apart from its role as a CS. In our conditioned approach learning experiment, the US-CS EP and US-CS NEP groups received the same number of light presentations. If c-Fos expression in these regions was due simply to light presentations then we should have observed equal amounts of c-Fos in the EP and NEP groups. The fact that we observed 44 significantly more c-Fos activation in the EP group argues that it is the CS attribute of the light stimulus, not the sensory effects per se, that is responsible for the significant enhancement of c-Fos activation in the EP group. Studies have shown that the presentation of a US or CS can translate into synaptic overflow of DA into the terminal regions of the mesocorticolimbic DA system (Fiorino, Coury, Fibiger, & Phillips, 1993); (Nicola, Taha, Kim, & Fields, 2005); (Sombers, et al., 2009), however, the manner in which this DA release is caused is unclear. It may result from DA cell activation from signals in the VTA, or from presynaptic stimulation of DA terminals. The current study suggests that conditioned approach occurs through signal activation of DA cells, at the level of the cell bodies, in the VTA. We proposed this signal is glutamatergic in nature (Sharf, et al., 2006; Sharf & Ranaldi, 2006a; Zellner & Ranaldi, 2010). If this is true then one would expect reduced responding maintained by reward-associated stimuli and reduced dopamine release in the mesolimbic DA terminal regions after blockade of glutamate signaling in the VTA and in fact this is what Sombers and colleagues found (Sombers, et al., 2009). These findings suggest a causal link between a CS’s capacity to cause activation of VTA DA cells to elicit conditioned approach. Furthermore, these findings suggest that unconditioned and conditioned food reward use similar neural mechanisms in mediating their effects on learned and motivated behavior. In reward-related learning, natural and drug rewards trigger approach behaviors primarily because they elevate mesolimbic DA levels (Wise, 2004). Under conditions of DA receptor blockade, rewards such as food (Wise, et al., 1978), brain stimulation (Fouriezos, Hansson, & Wise, 1978) and psychomotor stimulants like amphetamine 45 (Yokel & Wise, 1975) and cocaine (de Wit & Wise, 1977) fail to maintain responding in trained animals. We hypothesized that the capacity of CSs to function as such also requires that they elevate mesolimbic DA levels and we have proposed (Sharf & Ranaldi, 2006a; Zellner & Ranaldi, 2010) that their capacity to stimulate DA neurons depends on Glu neurotransmission in the VTA. For instance, You et al (You, Wang, Zitzman, Azari, & Wise, 2007) have demonstrated that exposure to a cocaine self-administration context causes Glu release in the VTA and that this CS-induced glutamatergic signal is necessary for cocaine seeking. It is also well-established that reward predictive stimuli cause burst firing of VTA DA neurons (Schultz, 1997). And although the mechanisms through which this happens have not been definitively established they likely involve Glu neurotransmission. Several brain regions, including the prefrontal cortex, amygdala, the midbrain tegmental nuclei, and the bed nucleus of the stria terminalis project directly to the VTA (Georges & Aston-Jones, 2002; Harris, et al., 2004). Most of these inputs are glutamatergic and excite VTA DA neurons (Georges & Aston-Jones, 2002; Takahata & Moghaddam, 1998) suggesting that one or more of these signals may play a necessary role in CS-induced mesolimbic DA activation in reward-related learning. In summary, we tested the hypothesis that a phasic, discrete food-associated CS can acquire the capacity to cause conditioned activation of VTA DA neurons. We found that consumption of food (presentation of a US) was associated with significant elevations in VTA TH-labeled cells expressing c-Fos. We also found that animals show a significantly greater amount of VTA TH-labeled cells expressing c-Fos in response to a food-associated light stimulus that served a behavioral CS function than in response to a 46 light stimulus that did not serve such a function. These results support our hypothesis that conditioned activation of VTA DA cells underlies the capacity of CSs to elicit CRs. 47 Chapter 3. 1. Introduction: NMDA receptor stimulation in the VTA is necessary for the acquisition of conditioned approach and the capacity of a conditioned stimulus to activate dopamine terminal regions The mesocorticolimbic dopamine (DA) system plays an important role in mediating the behavioral and neural effects of unconditioned stimuli (USs; Berridge and Robinson, 1998; Wise, 2004). Rats approaching food-related stimuli demonstrate elevated nucleus accumbens (NAcc) DA levels whereas rats not approaching do not (Bassereo et al., 2007). Furthermore, DA depletion in the NAcc, caudate and frontal cortex result in diminished rewarding effects of natural (Beninger and Ranaldi, 1993; Aberman et al., 1998) and drug rewards (Hiroi and White, 1991; McGregor and Roberts, 1993 and 1995). In addition to primary rewards, the mesocorticolimbic DA system also plays an important role in mediating the behavioral and neural effects of conditioned stimuli (CSs; Phillips et al., 1993). AP-5 in the NAcc core blocks the acquisition of conditioned approach for food reward (Di Ciano et al., 2001). Presentations of CSs are associated with neural activity or DA release in terminal regions of the mesocorticolimbic DA system, including the NAcc and prefrontal cortex (PFC; Blackburn and Phillips, 1989; Carelli, 2000, 2002; Gratton and Wise, 1994; Talmi et al., 2008). Depletions of DA using 6-hydroxydopamine lesions in the NAcc core impair Pavlovian approach to a conditioned stimulus (Parkinson, et al., 1999). Accordingly, it appears that, in rewardrelated learning mesocorticolimbic DA activity comes under the control of CSs allowing for the activation of DA terminal regions and the elicitation of a conditioned approach response. Precisely how this happens remains to be determined. One possibility is that 48 neural plasticity in the ventral tegmental area (VTA), the source of midbrain DA in the mesocorticolimbic DA system, may be involved. Several neural models of the associative properties of reward-related learning have been developed, most of which are based on the Hebbian postulate that when a synapse is active at the same time the postsynaptic neuron is active the synapse changes in strength. Furthermore, when two synapses (an unconditioned stimulus [US] and conditioned stimulus [CS]) are active at the same time a post synaptic neuron is active, the convergence of the two synaptic events induces neuronal plasticity (Hebb, 1949). We propose here, as we and others have elsewhere (Beninger, 1983; Beninger and Ranaldi 1993), that reward-related learning involves similar kinds of neural processes. Synaptic plasticity, such as long term potentiation (LTP), in the VTA may underlie the neural activity that develops during the formation of associations between rewards and neutral stimuli (see Zellner et al., 2009 and Zellner and Ranaldi, 2010 for review). Studies have found that LTP is one of the main mechanisms of learning (Kandel, 2001) and that it is VTA NMDA receptor dependant (Bonci and Malenka, 1999; Stuber et al., 2008). For the VTA to be considered a possible site where CS-US associations may be formed it must be a site where convergence of afferents carrying US signals and afferents carrying CS signals occurs. It appears that the VTA has these types of afferents in the form of acetylcholine (possible US signals) and glutamate (possible CS signals) afferents. The VTA has acetylcholine (ACh) secreting terminals of neurons originating in the pedunculopontine nucleus (PPN) and laterodorsal tegmental nuclei (LDT; Bolton, 1993; Garzon, 1999). In the VTA, extracellular concentrations of ACh increase during eating, drinking and self-stimulation of the lateral hypothalamus (Rada, 2000). In vitro 49 studies show that stimulation of metabotropic ACh receptors (mAchR) in the VTA induces a membrane depolarization in the DA neurons (Kalivas, 1993) and DA release in terminal regions of the mesolimbic system (Marchi, 1991). In addition, mACh receptor antagonists in the VTA reduce eating both in our laboratory (Sharf and Ranaldi, 2006) and others (Rada, 2000) and approach and consummatory responses for food (Ikemoto, 1996). These findings suggest that VTA mACh receptor stimulation is involved in mediating some of the US signals about rewards, including food. The VTA also receives glutamate (Glu) projections from structures ranging from the brainstem to the forebrain (Geisler et al., 2007), including the PPN (Charara 1996) and medial PFC (Smith 1996). In the VTA, extracellular concentrations of Glu increase in response to environmental stimuli. For instance, microinjections of NMDA and AMPA receptor antagonists delivered simultaneously into the VTA blocks the development of cocaine place preference (Harris and Aston-Jones, 2003) and, when administered individually, blocks the development of morphine place preference (Harris et al., 2004). Accordingly, it appears that Glu activation in the VTA is involved in drug-related learning and may therefore also be involved in reward-related learning. The VTA may receive a Glu signal that is related to environmental stimuli, possibly mediating the effects of CSs, and which might be involved in the acquisition of reward-related learning. Given this confluence of signals, of both ACh and Glu stimulation in the VTA, and the kinds of information these signals transmit, it is reasonable to suppose that these signals are integrated in some way when learning about reward-producing responses, such as conditioned approach responses. According to the Hebbian model of associative learning the convergence of ACh (the US) and Glu (the CS) at the level of the VTA can 50 lead to the CS acquiring the ability to induce DA release and thus activation of regions downstream of the VTA, the DA terminal regions, and consequently to stimulate approach. Therefore, we hypothesized that NMDA receptor stimulation in the VTA is necessary for the acquisition, but not the expression, of a CS to elicit conditioned approach and conditioned activation of DA terminal regions. In particular, we predicted that AP-5 in the VTA would impair the acquisition, but not the expression, of conditioned approach on a behavioral level and the expression of c-fos in DA terminal regions on a neural level. 2. Methods The protocols used in the present experiments were in accordance with the National Institutes of Health Guide for Care and Use of Laboratory Animals and were approved by the Queens College Institutional Animal Care and Use Committee. 2.a. Subjects Subjects consisted of 65 male Long Evans rats, facility-bred from males and females obtained from Charles River Laboratories (Raleigh, NC), with initial free-feeding weights between 375 and 450 grams at the time of surgery. All rats were individually housed and maintained on a 12 h light:12 h dark cycle (lights off at 6 AM). All experimental sessions were conducted during the dark phase in order to test the rats during their active periods. All rats had unlimited access to food (Purina rat chow) until experimental sessions began, at which time access was restricted to daily rations that maintained their weights at 85% of their free-feeding values. 51 2.b. Surgical procedure All rats received an intraperitoneal (IP) injection of atropine sulfate (0.1 ml) and were anesthetized by sodium pentobarbital (65 mg/kg). Stainless steel guide cannulae (0.635 mm outer diameter, 0.3302 mm inner diameter) were bilaterally implanted to a depth that allowed for microinjections into the VTA using the following coordinates: -5.6 mm caudal to bregma, ±2.0 mm from the midline at a 10° angle toward the midline and 8.3 mm below the surface of the skull (Paxinos and Watson, 1986). The cannulae were fixed in dental acrylic anchored to the skull by four stainless steel screws. Obturators (0.3048 mm diameter), extending 1 mm beyond the tip of the cannulae, were inserted at all times except during microinjections. 2.c. Apparatus All behavioral testing was conducted in eight conditioning chambers each measuring 30 x 21 x 18 cm (l x w x h). One wall was equipped with a food trough and two white stimulus lights, each situated one inch above and two inches to the right or left of the food trough. Each chamber was housed in a ventilated, sound-attenuating box. The chambers were controlled by a PC through a MED Associates interface. 2.d. Microinjection procedure Immediately prior to the appropriate sessions the obturator was removed from one of the guide cannulae and a stainless steel injector tube was inserted to extend 1 mm beyond the end of the guide cannula. The injector was connected by polyethylene tubing to a 10 µl Hamilton syringe (Reno, NV) preloaded with vehicle or AP-5. The compound was delivered manually over a 30-s period and the injector was kept in place for an 52 additional 60 seconds before being removed and the obturator replaced. This procedure was repeated on the contralateral side after which the animal was placed in the conditioning chamber and the session started. 2.e. Drugs AP-5 (Sigma-Aldrich, St. Louis, MO) was dissolved in 0.9% saline before the start of the experiments. Each microinjection was delivered in a volume of 0.5 µl at a dose of 0.25 or 0.5 µg/ 0.5 µl which was chosen based on our previous experiments showing that the high dose of AP-5 significantly attenuated the acquisition of foodreinforced operant responding without affecting food motivation or food reward value (Zeller et al., 2009). 2.f. Conditioning experiments One week after surgery rats began the food restriction diet to reduce their weights to 85% of their free-feeding values where they were maintained for the duration of the experiments. At least one day after food restriction began all rats were given 20 food pellets (45 mg, Bioserv, Frenchtown, NJ) in their home cages on each of three days. All rats were then tested in one of two versions of a conditioned approach paradigm. The first version, the acquisition study, was used to investigate the effects of treatment on the acquisition of conditioned approach (behavioral level) and activity in the DA terminal regions (neural level). The second version, the expression study, was used to investigate the effects of treatment on the expression of a learned conditioned approach. 53 2.f.i. Conditioning procedure for acquisition study Rats were given one 20-min magazine training session in the conditioning chambers in which 20 food pellets were delivered on a random time schedule, to allow rats to become acquainted with magazine delivery of food pellets. Rats were then randomly assigned to groups receiving bilateral intra-VTA microinjections of AP-5 or vehicle immediately before each of three daily, consecutively held 60-min conditioning sessions. During conditioning sessions, 30 food pellets were delivered on a random time 120-s schedule (range of 15 to 245 s). In a explicitly-paired (EP) group each pellet delivery was preceded by a 3-s presentation of a light on the left side of the food trough. In a non-explicitly paired (NEP) group the light presentations and food deliveries were not correlated to each other. After the 3 conditioning sessions with intra-VTA treatment, all rats were exposed to one 30-min extinction session with no treatment during which no light or food presentations were programmed. The extinction session represents goaldirected behavior and is indicative of remembered reward. This was followed by a 60min CS-only test session in which rats were presented with light presentations under the same random time schedule as in conditioning but with no further consequences (no food). All rats received intra-VTA vehicle injections prior to the CS-only test session. For all rats the number of head entries during each session was counted and analyzed (see Data analysis section for details). After the last session the rats were sacrificed and their brains were extracted and prepared for histological cannula verification and some for immunohistochemistry. 54 2.f.ii. Conditioning procedure for expression study This procedure was similar to the one described in the acquisition test but with the following differences. The number of conditioning sessions was 7 (to ensure a learned response), no microinjections were made prior to the conditioning sessions and microinjections of AP-5 or vehicle were made prior to the CS-only test session. No brains were prepared for immunohistochemistry. 2.g. Histology and Immunohistochemistry 2.g.i. Acquisition study Immediately after the end of the last session rats analyzed on a behavioral level were anesthetized with an overdose of sodium pentobarbital, perfused with 0.9% saline followed by 4% formalin, and decapitated. The brains were removed and stored in 4% formalin for at least seven days before being sectioned in the coronal plane on a cryostat, stained with cresyl violet and inspected for cannulae implantations and injection sites. All rats included in the data analysis had verified cannulae placements in VTA. Seventy five minutes after the end of the last session rats analyzed on a neural level were anesthetized with an overdose of sodium pentobarbital, perfused with a phosphate-buffered saline (PBS; 0.1 M) followed by a fixative containing 4% paraformaldehyde, and decapitated. The brains were removed and stored in 4% paraformaldehyde overnight at 4°C and sectioned through the VTA, NAcc core and shell, caudate, and frontal cortex in the coronal plane on a vibratome. Free-floating sections (40 µm) were collected into different wells for immunocytochemistry. Sections were first rinsed 3 times in PBS for 10 min each. For use with conventional light microscopy sections were incubated in 0.5% hydrogen peroxide (H2O2) and methanol in PBS for 20 55 min. Sections were rinsed 3 times in PBS for 10 min each and then blocked in 5% normal goat serum (NGS) and 0.2% Triton X-100 in PBS for 1 h. Sections were then incubated with primary antibody (rabbit anti-C-FOS 1:5000, Calbiochem) in 0.1% Triton X-100, 2.5% NGS, and PBS at 4°C overnight. Sections were rinsed 3 times in PBS for 10 min each and then incubated in biotinylated secondary antibody (biotinylated goat anti-rabbit; 1:200, Vector Labs, Burlingame, California) and 2.5% NGS in PBS for 2 h at room temperature. Sections were rinsed 3 times with PBS for 10 min each and then incubated in an avidin-horseradish peroxidase mixture (Vector Labs, Burlingame, California) for 2 h. Sections were rinsed 3 times in PBS for 10 min each and then reacted with 0.05% diaminobenzidine in the presence of 0.0015% H2O2. Sections were collected onto gelatin-coated slides, dried for several hours, and coverslipped with Cytoseal for VTA cannula verification and c-Fos quantification (black labeled c-Fos nuclei indicate activated neurons are easily identifiable and thus easily counted for statistical analyses). All rats included in the data analysis had verified cannulae placements in VTA. 2.g.ii. Expression study Immediately after the end of the last session rats were anesthetized with an overdose of sodium pentobarbital, perfused with 0.9% saline followed by 4% formalin, and decapitated. The brains were removed and stored in 4% formalin for at least seven days before being sectioned in the coronal plane on a cryostat, stained with cresyl violet and inspected for cannulae implantations and injection sites. All rats included in the data analysis had verified cannulae placements in VTA. 56 2.h. Data Analysis For all rats in the conditioning procedures the data consisted of the number of food trough head entries made during (1) a 6-s period immediately preceding the onset of the CS (pre-CS period), (2) the 3-s period during the CS, (3) a 3-s period immediately following the offset of the CS and (4) at all other times (non-CS period). For analyses the period during the CS and the period immediately following the CS were combined for a 6-s total period and referred to as the CS-period. For each session the total number of head entries during the CS periods and the total number of head entries during the pre-CS periods are used to calculate the CS/pre-CS ratio. This ratio indicates the magnitude of the conditioned approach response (re: the degree to which food trough head entries are elicited by the CS). Separate two-way, mixed-design ANOVAs, with group (dose of AP-5) as a between groups factor and session as a repeated measures factor were conducted on the CS/pre-CS ratio data from sessions 1 to 3 (acquisition) or 1 to 7 (expression) for the acquisition and expression tests, respectively. Significant interactions were followed by tests of simple effect of session in each group (dose). Separate one-way ANOVAs were conducted on the CS/pre-CS ratio data from the CS-only test session for the conditioning tests. Separate one-way ANOVAs were conducted on the total head entries from the CSonly test session for the conditioned procedures. C-Fos labeled nuclei were counted on an Olympus (Tokyo, Japan) BX51W microscope with a motorized stage at 4x magnification. Stereo Investigator software (MicroBrightField, Williston, VT) was used to outline the NAcc core and shell, medial and lateral caudate, and frontal cortex. To prevent bias that may skew the results, all counting was conducted blind to the treatment groups. Planned independent sample t- 57 tests with bonferroni adjusted p values comparing c-Fos counts in the AP-5 and vehicle groups were conducted for each brain region. 3. Results 3.a. Histology Most VTA microinjection sites were localized in the caudal portion of the VTA (5.6 to -6.04 mm posterior to bregma) with some injections occurring in the central portion (-5.2 to -5.3 mm posterior to bregma) (Figure 9). 3.b. Acquisition study Rats receiving AP-5 prior to conditioning sessions made the greatest number of food trough head entries during sessions 1 to 3 while rats receiving vehicle made the fewest [dose effect: F(3,32) = 7.625, P < .001]. Across sessions 1 to 3, the total number of head entries did not change for the vehicle group while it declined for the 0.25 µg AP5 and random control groups and increased in the 0.5 µg AP-5 group [session x dose: F(6,64) = 4.601, P < .001]. In the test session, when all rats were treated with vehicle injections, the total number of head entries was similar for all groups (Figure 10). The left panel in Figure 11 shows the ratio of CS/pre-CS food trough head entries in the three conditioning sessions. The pattern of change across sessions for this ratio was different among the different treatment groups [a two-way ANOVA revealed a significant session x dose interaction, F(6,64) = 3.747, p < .005]. The vehicle and 0.25 µg AP-5 groups both showed progressively larger CS/pre-CS ratios across sessions while for the 0.5 µg AP-5 and random control groups this ratio did not change (test of simple effect of session at each dose was < .01 for vehicle and < .001 for 0.25 µg AP-5). The 58 right panel in Figure 11 shows the CS/pre-CS ratio in the CS-only test session. The ratios for the vehicle and 0.25 µg AP-5 groups were much higher than for the 0.5 µg AP-5 and random control groups (a one-way ANOVA revealed a significant group effect; F(3,32) = 3.15, P < .05). The ratios for the vehicle and 0.25 µg AP-5 were similar and the ratios for the 0.5 µg AP-5 and random control groups were similar (LSD post hoc tests showed that each of the vehicle and 0.25 µg AP-5 groups differed significantly from each of the 0.5 µg AP-5 and random control groups, all ps < .05). 3.c. Expression study Rats receiving 0.5 µg AP-5 or vehicle prior to the CS-only test session made similar amounts of food trough head entries during sessions 1 to 7 (with no treatment) that declined across these sessions (Figure 12; analyses revealed a significant session effect, F(6,78) = 2.219, P < .05, that did not significantly interact with group). In the CSonly test session, the group receiving 0.5 µg AP-5 showed a somewhat greater number of overall head entries than the group receiving vehicle but this difference was not significant. The left panel in Figure 13 shows the ratio of CS/pre-CS food trough head entries in the seven conditioning sessions. Although both groups depicted received no treatments prior to session 1 to 7—that is, they were identical—the data are depicted separately according to the treatment that they would receive in the CS-only test session. The pattern of change in the CS/pre-CS ratio was similar for both groups across the conditioning sessions; both groups showed increasing ratios across the sessions that appeared to level off in the last three sessions or so (a two-way ANOVA revealed a significant session effect; F(6,78) = 4.438, P < .005). The right panel in Figure 13 shows 59 the ratio of CS/pre-CS food trough head entries in the CS-only test session after receiving 0.5 µg AP-5 or vehicle. Both groups showed similar ratios. 3.d. Immunohistochemistry To test the hypothesis that intra-VTA AP-5 reduction in CS/pre-CS ratios is associated with reduced activity in DA terminal regions we tested two additional groups (on a neural level) in the acquisition study, one with 0.5 µg AP-5 and the other with vehicle, both stained for c-Fos after the CS-only test. Similar to the previous groups tested (on a behavioral level) in the same conditions, the vehicle group showed significantly greater CS/pre-CS ratios than the AP-5 group during the CS-only test (Figure 14B). Accordingly, c-Fos expression was lower in all regions tested in the AP-5 group. (see Figure 14A). Our planned comparisons revealed that in the NAcc core the amount of c-Fos in the AP-5 group was significantly lower than in the vehicle group [t(11)=2.038, p < .03); t-tests for the NAcc shell, medial caudate and frontal cortex approached significance, Ps = .1 for each]. Figure 15 shows representative brain sections of identical locations in the VTA, one from a vehicle (left) and the other from an AP-5 (right) animal, showing c-Fos in the NAcc core at the same level of magnification used during counting. 60 -‐5.2 mm -‐5.6 mm -‐5.8 mm -‐6.04 mm -‐6.3 mm 61 Figure 9. Histological reconstruction of injection sites adapted from Paxinos and Watson (1997). Black circles represent the injection sites in the VTA. The numbers to the right of each section indicates the distance posterior to the bregma. 62 1000 Total head entries 800 Vehicle AP-5 0.25 AP-5 0.5 Random control 600 400 200 0 1 2 3 Test Session Figure 10. Average number of food trough head entries for each group across three conditioning and one CS-only test session of the acquisition study. Bilateral microinjections of 0.25 or 0.5 µg of AP-5 or vehicle immediately preceded conditioning sessions whereas only vehicle immediately preceded the CS-only test session. The vertical lines represent the standard error of the mean. Ratio of CS/Pre-CS head entries 8 Vehicle 0.25 µg AP-5 0.5 µg AP-5 Random Control 7 6 63 # * 5 4 + 3 + *# 2 1 0 1 2 3 CS-only Session Figure 11. Average CS/pre-CS ratios for each group across three conditioning and one CS-only test session of the acquisition study. Bilateral microinjections of 0.25 or 0.5 µg of AP-5 or vehicle immediately preceded conditioning sessions whereas only vehicle immediately preceded the CS-only session. * and # represent significant group differences in CS-only test; + represents significant increase across the three conditioning sessions. 64 1000 Total head entries 800 Vehicle AP-5 0.5 600 400 200 0 1 2 3 4 5 6 7 Test Session Figure 12. Average number of food trough head entries for each group across seven conditioning and one CS-only test session of the expression study. Bilateral microinjections of 0.5 µg of AP-5 or vehicle immediately preceded the CS-only test session. The vertical lines represent the standard error of the mean. Ratio of CS/Pre-CS head entries 10 65 Vehicle 0.5 µg AP-5 9 8 7 + 6 5 + 4 3 2 1 0 1 2 3 4 5 6 7 CS-only Session Figure 13. Average CS/pre-CS ratios for each group across seven conditioning and one CS- only test session of the expression study. Bilateral microinjections 0.5 µg of AP-5 or vehicle immediately preceded the CS-only test session. + represents significant increase across the seven conditioning sessions. 66 A B cFOS count 100 Ratio of CS/Pre-CS head entries 120 vehicle AP-5 80 60 40 20 0 * NA co NA sh Cau med Cau lat 6 5 4 * vehicle AP-5 3 2 1 0 Front cortex Brain region Figure 14. A. c-Fos counts in the NAcc core and shell, medial and lateral caudate and frontal cortex in rats completing the acquisition study. Bilateral microinjections of 0.5 µg AP-5 or vehicle immediately preceded conditioning sessions whereas only vehicle immediately preceded the CS-only test session. * represents significant group differences at that site. B. Average CS/pre-CS ratios for the two groups mentioned in A for the CSonly test session of the acquisition study. * represents significant group differences. For A and B vertical lines represent the standard error of the mean. Vehicle 67 0.5 µg AP-5 Figure 15. Illustrations representing c-Fos in identical portions of the NAcc core in one rat that received bilateral microinjections of 0.5 µg AP-5, and one that received vehicle, immediately preceding the CS-only test session in the acquisition study. Illustrations were taken with the same microscope set to the same level of 4x magnification. The white arrow indicates c-Fos. 68 4. Discussion Based on the results of the present study we can conclude that the NMDA receptors in the VTA are necessary for the acquisition, but not the expression, of a conditioned approach response and for a CS to significantly activate the NAcc core. In the acquisition study, rats that received intra-VTA injections of 0.5 µg AP-5 showed no evidence of acquiring conditioned approach in response to the 3-s light presentation preceding each pellet delivery. This was first indicated by a lack of increase in the ratio of CS/pre-CS head entries across three conditioning sessions by the 0.5 µg AP-5 and random control groups while the ratio of the 0.25 µg AP-5 and vehicle groups did increase across those sessions. More importantly, during the CS-only test session, which directly followed vehicle injection for all groups, the ratio of CS/pre-CS was significantly higher in the 0.25 µg AP-5 and vehicle groups than the 0.5 µg AP-5 group, which in turn was not significantly different from the random control group. Because the CS-only test session was conducted without any acute effects of AP-5 treatment, the lack of conditioned approach provides extremely strong evidence that NMDA receptor blockade in the VTA prevented the acquisition of learning in relation to the CS. On a similar note, in the expression study, intra-VTA injections of 0.5 µg AP-5 and vehicle administered prior to CS-only test session had no effect on conditioned approach behavior indicating that the NMDA receptor stimulation in the VTA is not necessary for the expression of conditioned approach In addition, our findings demonstrate that NMDA receptor stimulation in the VTA is necessary for CS-associated activity in DA terminal regions, specifically in the NAcc core. In the acquisition study, rats treated with intra-VTA injections of 0.5 µg AP-5 prior to conditioning sessions not only failed to acquire the conditioned approach response, but 69 also showed significantly less c-Fos expression in the NAcc core in response to the CS during the CS-only test session than rats treated with vehicle. These results suggest two things. First, that expression of CS responding involves CS induced activation of cells in the NAcc core. Second, that the neural plasticity that allows a CS to acquire the ability to elicit a response similar to that of a US and CS induced NAcc core activity both require NMDA receptor stimulation in the VTA during acquisition. All in all, our results demonstrate that NMDA receptor antagonism in the VTA leads to impairment in the acquisition of a conditioned approach response and in the acquisition of CS-induced neuronal activity. The behavioral and neural findings of this study are consistent with a previous study from our lab, as well as from others. On the behavioral level, we have previously reported that intra-VTA AP-5 blocked the acquisition of food reinforced instrumental responding but not its expression (Zellner et al., 2009). Furthermore, intra-VTA treatment with an NMDA receptor antagonist impaired the acquisition of morphine induced place preference (Harris et al, 2004). These studies support our findings that the NMDA receptors in the VTA play a role in the acquisition of reward-related learning for both natural and drug reward. On the neural level, excitotoxic lesions of the NAcc core (Parkinson et al., 2000) or depletion of DA in the NAcc using 6-hydroxydopamine (Parkinson et al., 2002) impaired the acquisition of approach to a reward-related stimulus. Altogether, these studies support our hypotheses that NMDA receptor stimulation in the VTA is necessary for the acquisition, but not the expression, of a CS to elicit conditioned approach and conditioned activation of DA terminal regions. 70 The present findings make a significant contribution to the understanding of the neural plasticity underlying reward-related learning. We propose that when rats consume a reward in a given environment, ACh transmission in the VTA activates DA cells, leading to increased DA in forebrain regions which stimulates increased approach behavior. As the VTA DA cells are stimulated by the ACh signal of reward, they also receive excitation from Glu afferents. Because of this coincident stimulation of ACh and Glu receptors, LTP occurs at the Glu synapses. Through the process of LTP, the environmental signals acquire the ability to activate VTA DA cells in their own right, increasing the levels of conditioned approach towards stimuli in the environment which are contiguously associated with the reward. From this model we can predict that NMDA receptor stimulation in the VTA and activation of DA terminal regions play a role in these proposed associative processes of reward-related learning, a prediction that is supported by the present data. The increase in head entries across the three conditioning sessions during the acquisition study emitted by the 0.5 µg AP-5 group deserves closer examination. This change in behavior suggests the possibility that some kind of learning was taking place in these rats, although we are unable to determine specifically what that might have been. It is possible these rats were acquiring some association between reward and the conditioning chamber, or the food trough, but in the absence of the ability to associate food delivery with the light presentation they were indiscriminately increasing their investigation of the food trough. It is also possible that the stimulation of food reward, in the context of an inability to link reward to any specific stimuli, resulted in heightened activity. The ability of intra-VTA AP-5 to facilitate context learning or other aspects of 71 reward-learning which are not related to discrete cues therefore merits additional investigation. The scant evidence which exists to date would argue against this being the case, at least in the case of drug administration with VTA NMDA blockade (Harris, et al., 2004) where NMDA receptor blockade is associated with impaired context conditioning; to our knowledge context-related learning with food reward has not been tested with this manipulation and therefore remains an empirical question. The results of this experiment also argue against context learning; although the total head entry data during days 1 to 3 suggest a possibility of learning, pre-CS responding on the test day was no higher than controls, which ought to have been the case if AP-5 treated rats had learned a reward-association with the environment. In any case, given that AP-5 rats on the test day with no drug pre-treatment emitted similarly low levels of head entries to vehicle-treated rats; it appears that even if some kind of learning were taking place across the training sessions, AP-5 rats were specifically impaired in being able to acquire an association with the discrete reward-paired cue. Another possibility for the increase in head entries seen in the 0.5 µg AP-5 group is that there occurred sensitization to the stimulant effects of this drug. Intra-VTA injections of 0.5 µg AP-5 causes an increase in locomotor activity (Zellner et al., 2009) which might become sensitized with repeated use as in the present study. However, a number of studies do not support this possibility and demonstrate that repeated administration of the NMDA receptor antagonists AP-5 (Licata et al., 2004) or MK-801 (Kalivas and Alesdatter, 1993) in the VTA prevents the development of sensitization. 72 Chapter 4. 1. General Discussion 1.a. Results The neural mechanisms underlying reward-related learning in the mesocorticolimbic dopamine (DA) system are not fully understood, particularly how neutral stimuli become conditioned stimuli (CSs). We proposed, based of the Hebbian model, that for this type of learning the ventral tegmental area (VTA) receives signals about both unconditional stimuli (USs) and eventual CSs which converge on DA neurons leading to the acquisition by the conditioned stimulus (CS) of the capacity to activate DA neurons, and hence DA terminal regions (specifically the nucleus accumbens, amygdala and prefrontal cortex) on its own and elicit conditioned approach. To test this model we hypothesized that (1) a food US activates VTA DA cells, (2) a food-associated CS (light) acquires the capacity to activate VTA DA cells and cause conditioned approach, and (3) that blockade of NMDA receptors in the VTA would impair the acquisition, but not the expression, of conditioned approach on a behavioral level and the expression of c-Fos in DA terminal regions on a neural level. Our results support our hypotheses. First, that food (the US) activates VTA DA neurons. Second, that a stimulus explicitly paired with the US, the conditioned stimulus (CS; light) also activates VTA DA neurons and produces a significantly greater conditioned approach response than a stimulus not explicitly paired with the US. Third, that NMDA receptor stimulation in the VTA is necessary for the acquisition, but not expression, of conditioned approach behavior and for a CS to significantly activate the nucleus accumbens (NAcc) core. Altogether, it appears that a discrete food-associated 73 CS can acquire the capacity to cause conditioned activation of VTA DA neurons, and that this is VTA NMDA receptor dependent. The results of the present studies lead us to conclude that conditioned activation of VTA DA cells underlies the capacity of CSs to elicit a conditioned approach response similar to that of a US, and that this is VTA NMDA receptor dependent. Furthermore, that the neural plasticity underlying rewardrelated learning involves CS-induced activation of DA terminal regions, specifically the NAcc core. 1.b. The role of the VTA The VTA is an essential node in the mesocorticolimbic DA system because it constitutes a node for where this kind of neuronal plasticity occurs. We (Bonci & Malenka, 1999; Harris, et al., 2004; Ranaldi, 2014; Zellner & Ranaldi, 2010) suggest that one constituent of the process by which CSs gain access to the same neuronal circuits as USs is via synaptic plasticity in which US and CS relevant cells activate the same VTA DA neuron concurrently. This convergent stimulation creates the conditions for long term potentiation (LTP) which has been demonstrated in the VTA DA neurons (Bonci & Malenka, 1999; Chen et al., 2008; Luu & Malenka, 2008) as well as to be VTA NMDA receptor dependent (Bonci & Malenka, 1999; Citri & Malenka, 2007; Nugent, Hwong, Udaka, & Kauer, 2008; Stuber, et al., 2008). We have shown that injections of an NMDA receptor antagonist directly into the VTA block the acquisition of reward-related learning during both operant (Zellner, et al., 2009b) and classical (Ranaldi, et al., 2011) conditioning tasks for natural reward (food). The same has been shown for drug reward. Harris and Aston-Jones (Harris & Aston-Jones, 2003) found that stimulation of NMDA receptors in the VTA is essential for the acquisition of cocaine conditioned place 74 preference. Therefore, we proposed here that, if the reward neurons are firing in response to muscarinic acetylcholine (ACh) receptor activation, then simultaneous relatively weak stimulation of the NMDA receptor via glutamate (Glu) signals can result in the initiation of second messenger cascades implicated in LTP (Harris, et al., 2004), as well as other intracellular events that lead to neurochemical and structural changes associated with synaptic plasticity underlying reward-related learning. Such synaptic changes could result in VTA Glu acquiring the capacity to robustly activate VTA DA cells, causing enhanced DA release. This increase in DA might lead to increased approach, increasing the likelihood of direct contact with rewards or emission of behaviors that lead to rewards or to associative processes downstream that facilitate other aspects of conditioned behaviors. The VTA acts as a node where this type of neuronal plasticity occurs because it receives substantial afferents able to provide coincident stimulation representing both USs and CSs, contains NMDA receptors that can set in motion intracellular processes that lead to neural plasticity following such coincident stimulation such as LTP. The VTA receives and is modulated by many neurotransmitters including DA, ACh, Glu, GABA, serotonin, norepinephrine, opioids and peptides including CCK and orexin. In this dissertation I focus only on the role of ACh and Glu. ACh likely mediates the primary reward signal of various USs to the VTA whereas Glu likely carries signals about environmental stimuli, which provides for the stimulation of the VTA NMDA receptor. VTA NMDA receptor stimulation mediates LTP, and is the signal by which CSs are likely to continue to activate VTA cells, and the DA terminal regions, after associative learning has taken place. 75 1.c. Understanding the role of Glu It is thought that in reward-related learning, natural and drug rewards trigger approach behaviors primarily because they elevate mesolimbic DA levels (Wise, 2004). Under conditions of DA receptor blockade, rewards such as food (Wise, et al., 1978), brain stimulation (Fouriezos, et al., 1978) and psychomotor stimulants like amphetamine (Yokel & Wise, 1975) and cocaine (de Wit & Wise, 1977) fail to maintain responding in trained animals. We hypothesized that the capacity of CSs to function as such also requires that they elevate mesolimbic DA levels and we have proposed (Ranaldi, 2014; Sharf & Ranaldi, 2006a; Zellner & Ranaldi, 2010) that their capacity to stimulate DA neurons depends on Glu neurotransmission in the VTA. On a similar note, You et al. (You, et al., 2007) have demonstrated that exposure to a cocaine self-administration context causes Glu release in the VTA and that this CS-induced glutamatergic signal is necessary for cocaine seeking. It is also well-established that reward predictive stimuli cause burst firing of VTA DA neurons (Schultz, 1997), and although the mechanisms through which this happens have not been definitively established they likely involve Glu neurotransmission. Several brain regions, including the prefrontal cortex, amygdala (AMG), the midbrain tegmental nuclei, and the bed nucleus of the stria terminalis project directly to the VTA (Georges & Aston-Jones, 2002; Harris, et al., 2004). Most of these inputs are glutamatergic and excite VTA DA neurons (Georges & Aston-Jones, 2002; Takahata & Moghaddam, 1998) suggesting that one or more of these signals may play a necessary role in CS-induced mesolimbic DA activation in reward-related learning. 76 1.d. DA activation from unconditioned and conditioned stimuli Studies have shown that the presentation of a US or CS can translate into synaptic overflow of DA into the terminal regions of the mesocorticolimbic DA system (Fiorino, et al., 1993; Sombers, et al., 2009), however, the manner in which this DA release is initiated is unclear. It may result from DA cell activation from signals in the VTA, or from presynaptic stimulation of DA terminals. The current studies suggest that conditioned approach occurs through signal activation of DA cells, at the level of the cell bodies, in the VTA. We proposed this signal is glutamatergic in nature (Sharf, et al., 2006; Sharf & Ranaldi, 2006a; Zellner & Ranaldi, 2010). If this were true then one would expect reduced responding maintained by reward-associated stimuli and reduced dopamine release in the mesolimbic DA terminal regions after blockade of glutamate signaling in the VTA and in fact this is what Sombers and colleagues found (Sombers, et al., 2009). Moreover, in a previous study we demonstrated that blockade of NMDA receptors in the VTA prevented the acquisition of conditioned approach and this was associated with significantly reduced c-Fos activation in the NAcc core in response to the (non-effective in this case) CS (Ranaldi, et al., 2011). Altogether these findings suggest a causal link between a CS’s capacity to cause activation of VTA DA cells and mesolimbic terminal regions and its capacity to elicit conditioned approach. In addition, our findings demonstrate that NMDA receptor stimulation in the VTA is necessary for CS-associated activity in DA terminal regions. 1.e. The role of VTA NMDA receptors There is an accumulation of evidence (See Chapter 4, 1.b.) that NMDA receptor stimulation in the VTA plays a necessary role in reward-related learning for both natural 77 (e.g., food) and drug reward. And altogether, these studies support our hypotheses that NMDA receptor stimulation in the VTA is necessary for the acquisition, but not the expression (Addy, Nunes, & Wickham, 2015), of a CS to elicit conditioned approach and conditioned activation of DA terminal regions. 1.f. A detailed Hebbian model We have proposed a model of reward-related learning (Sharf & Ranaldi, 2006a) (Zellner, et al., 2009a; Zellner & Ranaldi, 2010) predicated on the assumption that this learning occurs when the CS acquires the ability to activate the same neural system that produces unconditioned approach (Bindra, 1974). This is a Hebbian model that proposes that the VTA is a site where signals from reward-associated and US (i.e., primary rewards) converge onto DA cells and, through NMDA-dependent LTP, CS signals acquire the capacity to activate VTA DA cells by themselves. In the VTA, muscarinic ACh receptor stimulation has been shown repeatedly to mediate the rewarding effects of USs like food (Rada, et al., 2000; Sharf & Ranaldi, 2006a), brain stimulation reward (Kofman & Yeomans, 1988; Rada, et al., 2000; Yeomans, Mathur, & Tampakeras, 1993) and cocaine (You, Wang, Zitzman, & Wise, 2008) and therefore could serve as the US signal in this model. Also in the VTA, Glu is released from afferents of neurons originating in frontal cortex, pedunculopontine nuclei and other brain regions that are involved in processing information about environmental stimuli. Thus, the VTA is a site where convergence of US and CS signals is possible. Through the process of LTP at the Glu synapse, the environmental signals acquire the ability to activate VTA DA cells in their own right, increasing the levels of conditioned approach towards stimuli in the environment which are contiguously associated with the reward. Support for this model 78 comes from previous studies showing that blockade of muscarinic ACh receptor stimulation in the VTA blocks the acquisition, but not the expression, of food-reinforced instrumental responding (Sharf, et al., 2006) and other food-related learning (Sharf & Ranaldi, 2006a). The present findings make a significant contribution to the understanding of the neural plasticity underlying reward-related learning. Support for the role of NMDA receptor stimulation in this model comes from the present and other (Stuber, et al., 2008) findings that blockade of NMDA receptors impairs the acquisition of the conditioned approach or instrumental (Zellner, et al., 2009a) response but not the expression of this learning. Additional evidence supporting our model is that the VTA and its DA terminal regions show progressive changes in activity as neutral stimuli are paired with rewards, indicating that associative processes are taking place. Midbrain DA neurons respond to both primary rewards and CSs (see (Horvitz, 2000) for review), but responding to CSs develops over time. Generally, midbrain DA neurons fire in response to reward receipt until animals are well trained at which time responding of DA cells comes primarily under the control of CSs (Ljungberg, et al., 1992; Pan, Schmidt, Wickens, & Hyland, 2005). The present findings that intra-VTA AP-5, in addition to impairing the acquisition of conditioned approach also resulted in significantly less c-Fos in NAcc core and a trend toward less c-Fos in other DA terminal regions in response to the CS, is consistent with the hypothesis that the reward-associated stimulus failed to control behavior because it failed to acquire the capacity to activate DA cells, resulting in less cellular activation in DA terminal regions. However, some findings do not support this model. Stuber and his colleagues (Stuber, et al., 2008) have found that although conditioned approach was 79 associated with synaptic strengthening onto VTA DA cells to the CS, this LTP-like enhanced response was temporary; it developed during acquisition of conditioned approach for food but dissipated after the behavioral response stabilized. These findings suggest that VTA DA cell activation facilitates the transformation of neutral stimuli into salient reward predictive cues but is not required to maintain the US-CS association. The maintenance of the US-CS association may rely on synaptic plasticity outside the VTA and possibly in the DA terminal regions. It seems probable that NMDA receptor stimulation in the VTA is needed to produce a DA signal downstream that is itself essential for synaptic plasticity. Phasic, or burst, firing of VTA DA cells is at least partly dependent on NMDA receptor stimulation (Chergui et al., 1991), and this NMDA effect appears dependent on intact signaling from afferents originating in the laterodorsal tegmentum (Lodge & Grace, 2006). It has long been hypothesized that a DA signal in terminal regions, time-locked to reward events, serves as a necessary input for the synaptic plasticity in these regions that leads to reward-related learning (Beninger, 1983; Beninger & Ranaldi, 1994). In this model, stimuli associated with reward provide a glutamatergic signal to output neurons in DA terminal regions while primary rewards provide a DA signal to the same. This convergent stimulation would produce synaptic plasticity in these output neurons allowing reward-related stimuli to function as CSs, activating these output neurons to produce conditioned responding. Kelley and colleagues have conducted several studies demonstrating the necessity of both DA and NMDA receptor stimulation in NAcc (Smith-Roe & Kelley, 2000), prefrontal cortex (Baldwin, Sadeghian, Holahan, & Kelley, 2002) and AMG (Andrzejewski, Spencer, & Kelley, 2005) for the acquisition, but not 80 performance, of instrumental responding. But in addition to the stimulus-response type of learning that might be represented by the Kelley studies it is also possible that DA signals in terminal regions mediate the synaptic plasticity that underlies stimulus-reward associations; such plasticity may then result in amplified CS-related glutamate signals back to the VTA (You, et al., 2007) causing the CS-induced activation of DA cells (Ljungberg, et al., 1992; Pan, et al., 2005) and CS-induced DA release (Bassareo & Di Chiara, 1999; Wilson, Nomikos, Collu, & Fibiger, 1995; Wolterink et al., 1993) that is observed in reward-related learning. Definitely, Glu in the VTA has been shown to have a role in conveying information about CSs. You and colleagues (You, et al., 2007) demonstrated that instrumental responding influenced by cocaine CSs is correlated with VTA Glu release and reduced with VTA Glu blockade. Moreover, inhibition of Glu release in the VTA during heroin self-administration reduces drug-context induced reinstatement of already learned operant responding for drug (Bossert, Liu, & Shaham, 2004). These findings suggest that a route by which signals relating to CSs excite the VTA, and by which those signals come to be able to excite the VTA in and of themselves, thereby becoming CS signals, is via glutamate stimulation of the NMDA receptor. Thus in both of these cases, where phasic DA activity is necessary for stimulusresponse or stimulus-reward associative learning, if phasic DA activity is dependent on NMDA receptor stimulation in the VTA (Chergui, et al., 1991; Lodge & Grace, 2006), then antagonism of this neurochemical pathway should impair acquisition of conditioned approach learning. In the case where phasic DA activity is necessary for synaptic plasticity leading to enhanced CS-related glutamate signals to VTA, blockade of VTA NMDA receptors during learning should also result in the diminished capacity of CSs to 81 activate VTA DA cells leading to reduced cellular activity in DA terminal regions in response to CSs, a finding observed in the present study. 1.g. Possible shortcomings of the proposed Hebbian model in reward-related learning In this dissertation we surmised evidence that supports our model of rewardrelated learning. Specifically, we assessed the multitude of data demonstrating that VTA ACh provides a US signal to the VTA, that VTA Glu provides a CS signal to the VTA and coincident stimulation of the NDMA receptor is necessary for its synaptic strengthening allowing environmental stimuli to recruit DA cells on their own. Therefore, we propose that reward-related stimuli gain access to the same motivational neural circuits as primary rewards and thereafter come to activate these circuits and elicit motivational states, specifically approach behaviors, similar to the primary rewards. In light of this proposition, there are some who suppose that DA may not have a role in reward-related learning or that it plays a different role than approach. It has been demonstrated that mesocorticolimbic DA neurons actually respond to salient and novel stimuli, and furthermore, that DA neurons respond to salient events that extend beyond that of reward stimuli (Horvitz, 2000); in other words, DA neurons respond to salient and arousing changes in environmental conditions, regardless of the motivational valence of that change. Specifically, Horvitz (Horvitz, Stewart, & Jacobs, 1997) demonstrated that DA neurons in the VTA are activated by non-conditioned auditory and visual stimuli. Other studies found that cells in the VTA increase their firing to sound, tail pressing, tail pricks and spontaneous movement not associated with rewards (Kiyatkin & Rebec, 1998; J. D. Miller, Sanghera, & German, 1981). Though these findings demonstrate that the 82 VTA and its terminal regions respond to novel USs, in the absence of reinforcement, these responses have been shown to habituate (Ljungberg, et al., 1992; Schultz, 1998). In both primates (Ljungberg, et al., 1992) and rats (Pan, et al., 2005), midbrain DA neurons fire in response to reward receipt until animals are well trained and then respond only to other unexpected rewards. Furthermore, in the awake rat, a transient increase in NAcc DA to a novel stimulus disappears on subsequent sessions when not paired with reward (Kiyatkin & Stein, 1996). Though these findings don’t directly demonstrate a role for DA in approach in reward-related learning, they do show an important role in sensory or attentional processes that may be necessary in the early stages of reward-related learning, but not later on. Altogether, it appears that midbrain DA neurons respond to novel and aversive (Young, Ahier, Upton, Joseph, & Gray, 1998) events, in addition to rewards and conditioned stimuli. Given the significant amount of data assessed in this dissertation, it seems reasonable to state that CSs have to the ability to activate the same neural systems as primary rewards (USs) and can thus directly influence behavior. Additionally, it appears that after years of hypotheticals and experimentation we are currently in the process of understanding the underlying neural mechanisms of how CSs acquire this capacity. We hope that further investigation in this specific area will contribute to our basic knowledge of this important adaptive function as well as in addressing pathologies and diseases of reward-related behaviors that involve abnormal strengthening of reward-related cues. 83 1.h. The importance of understanding the neural mechanisms underlying rewardrelated learning Elucidating the neural mechanisms underlying reward-related learning is important because it adds insight into understanding pathologies, such as drug addiction, that involve this type of learning. For instance, since drug-taking is similar to other types of reinforced learning (Goldberg, 1973) it is possible that our “Hebbian-based” neural model of reward-related learning applies to drug-related learning occurring in addiction as well. Drug-associated stimuli, which cause craving (Childress, Ehrman, McLellan, & O'Brien, 1988; Ehrman, Robbins, Childress, & O'Brien, 1992), drug-seeking (Goldberg, 1975; Ranaldi & Roberts, 1996) and reinstatement of drug seeking in animals (Fuchs, Tran-Nguyen, Specio, Groff, & Neisewander, 1998; Grimm, Hope, Wise, & Shaham, 2001; Meil & See, 1996; Semenova & Markou, 2003), do so in part because of their capacity to elevate DA levels in the mesocorticolimbic DA system (Robinson & Berridge, 1993; Stewart, de Wit, & Eikelboom, 1984; Wise, 1994). It is important to note that the mesocorticolimbic DA system did not evolve to respond to drugs of abuse or drug-associated stimuli but rather to play a role in adaptive behaviors for survival such as reward-related learning. It has been suggested that cue-induced drug craving and seeking results from direct activation of ventral tegmental area DA neurons. If this is so, then strategies that can prevent drug cue signals from activating these neurons may prove beneficial in treating relapse. Thus, understanding the neurobiology underlying the role of the mesocorticolimbic DA system in reward-related learning may help address disorders and diseases (such as schizophrenia, Parkinson’s, and attention deficit hyperactivity disorder), and potential therapies or improved treatment strategies. 84 References Addy, N. A., Nunes, E. J., & Wickham, R. J. (2015). Muscarinic, but not nicotinic, acetylcholine receptor blockade in the ventral tegmental area attenuates cueinduced sucrose-seeking. Behavioural brain research, 291, 372-376. doi: 10.1016/j.bbr.2015.05.036 Andrzejewski, M. E., Spencer, R. C., & Kelley, A. E. (2005). Instrumental learning, but not performance, requires dopamine D1-receptor activation in the amygdala. Neuroscience, 135(2), 335-345. Antonov, I., Antonova, I., Kandel, E. R., & Hawkins, R. D. (2001). The contribution of activity-dependent synaptic plasticity to classical conditioning in Aplysia. [In Vitro Research Support, U.S. Gov't, P.H.S.]. The Journal of neuroscience : the official journal of the Society for Neuroscience, 21(16), 6413-6422. Baldwin, A. E., Sadeghian, K., Holahan, M. R., & Kelley, A. E. (2002). Appetitive instrumental learning is impaired by inhibition of cAMP-dependent protein kinase within the nucleus accumbens. Neurobiology of Learning and Memory, 77(1), 4462. Barria, A., Muller, D., Derkach, V., Griffith, L. C., & Soderling, T. R. (1997). Regulatory phosphorylation of AMPA-type glutamate receptors by CaM-KII during longterm potentiation. [Comment In Vitro Research Support, Non-U.S. Gov't Research Support, U.S. Gov't, P.H.S.]. Science, 276(5321), 2042-2045. Bassareo, V., & Di Chiara, G. (1999). Modulation of feeding-induced activation of mesolimbic dopamine transmission by appetitive stimuli and its relation to motivational state. European Journal of Neuroscience, 11, 4389-4397. Beninger, R. J. (1983). The role of dopamine in locomotor activity and learning. Brain Research Reviews, 6, 173-196. Beninger, R. J. (1989). Dissociating the effects of altered dopaminergic function on performance and learning. Brain Research Bulletin, 23, 365-371. Beninger, R. J., D'Amico, C. M., & Ranaldi, R. (1993). Microinjections of flupenthixol into the caudate putamen of rats produce intrasession declines in food-rewarded operant responding. Pharmacol Biochem Behav, 45(2), 343-350. doi: 00913057(93)90249-S [pii] Beninger, R. J., & Hahn, B. L. (1983). Pimozide blocks establishment but not expression of amphetamine-produced environment-specific conditioning. Science, 220, 13041306. Beninger, R. J., & Miller, R. (1998). Dopamine D1-like receptors and reward-related incentive learning. Neuroscience & Biobehavioral Reviews, 22(2), 335-345. Beninger, R. J., & Ranaldi, R. (1994). Dopaminergic agents with different mechanisms of action differentially affect responding for conditioned reward. In T. Palomo & T. Archer (Eds.), Strategies for Studying Brain Disorders. Vol. 1: Depressive, Anxiety and Drug Abuse Disorders. London: Farrand Press. (Reprinted from: IN FILE). Berridge, K. C. (2007). The debate over dopamine's role in reward: the case for incentive salience. Psychopharmacology (Berl), 191(3), 391-431. 85 Berridge, K. C., & Robinson, T. E. (1998). What is the role of dopamine in reward:hedonic impact, reward learning, or incentive salience? Brain Research Reviews, 28, 309-369. Bindra, D. (1974). A motivational view of learning, performance, and behavior modification. Psychological Reviews, 81, 199-213. Bjorklund, A., & Dunnett, S. B. (2007). Fifty years of dopamine research. [Editorial Review]. Trends in neurosciences, 30(5), 185-187. doi: 10.1016/j.tins.2007.03.004 Blackburn, J. R., & Phillips, A. G. (1989). Dopamine and preparatory behavior: II. a neurochemical analysis. Behavioral Neuroscience, 103, 15-23. Bolton, R. F., Cornwall, J., & Phillipson, O. T. (1993). Collateral axons of cholinergic pontine neurones projecting to midline, mediodorsal and parafascicular thalamic nuclei in the rat. Journal of Chemical Neuroanatomy, 6 (2 ), 101 - 114 Bonci, A., & Malenka, R. C. (1999). Properties and plasticity of excitatory synapses on dopaminergic and GABAergic cells in the ventral tegmental area. Journal of Neuroscience, 19(10), 3723-3730. Bossert, J. M., Liu, S. Y., & Shaham, Y. (2004). A role of ventral tegmental area glutamate in contextual cue-induced relapse to heroin seeking. Journal of Neuroscience, 24(47), 10726-10730. Calabresi, P., Lacey, M. G., & North, R. A. (1989). Nicotinic excitation of rat ventral tegmental neurones in vitro studied by intracellular recording. British Journal of Pharmacology, 98 (1 ), 135 - 140. Carelli, R. M. (2000). Activation of accumbens cell firing by stimuli associated with cocaine delivery during self-administration. Synapse, 35, 238-242. Carelli, R. M., & Ijames, S. G. (2001). Selective activation of accumbens neurons by cocaine-associated stimuli during a water/cocaine multiple schedule. Brain Research, 907(1-2), 156-161. Carr, D. B., & Sesack, S. R. (2000). GABA-containing neurons in the rat ventral tegmental area project to the prefrontal cortex. Synapse, 38(2), 114-123. Chen, B. T., Bowers, M. S., Martin, M., Hopf, F. W., Guillory, A. M., Carelli, R. M., . . . Bonci, A. (2008). Cocaine but not natural reward self-administration nor passive cocaine infusion produces persistent LTP in the VTA. Neuron, 59(2), 288-297. Chergui, K., Charlety, P. J., Akaoka, H., Brunet, J. L., Saunir, C. F., Buda, M., . . . Chouvet, G. (1991). Participation of NMDA receptors in spontaneous burst firing of dopaminergic mesencephalic neurons. C R Academy des Sciences III, 313(2), 139-144. Childress, A. R., Ehrman, R. N., McLellan, A. T., & O'Brien, C. P. (1988). Conditioned craving and arousal in cocaine addiction: a preliminary report. NIDA.Res.Monogr., 81, 74-80. Childress, A. R., Mozley, P. D., McElgin, W., Fitzgerald, J., Reivich, M., & O'Brien, C. P. (1999). Limbic activation during cue-induced cocaine craving. American Journal of Psychiatry, 156(1), 11-18. Christie, M. J., Bridge, S., James, L. B., & Beart, P. M. (1985). Excitotoxin lesions suggest an aspartatergic projection from rat medial prefrontal cortex to ventral tegmental area. Brain Research, 333, 169-172. 86 Citri, A., & Malenka, R. C. (2007). Synaptic plasticity: multiple forms, functions, and mechanisms. Neuropsychopharmacology, 33(1), 18-41. Conrad, L. C., & Pfaff, D. W. (1976). Autoradiographic tracing of nucleus accumbens efferents in the rat. Brain research, 113(3), 589-596. Cornwall, J., & Phillipson, O. T. (1988). Afferent projections to the dorsal thalamus of the rat as shown by retrograde lectin transport-I. the mediodorsal nucleus. Neuroscience, 24, 1035-1049. Day, H. E., Kryskow, E. M., Nyhuis, T. J., Herlihy, L., & Campeau, S. (2008). Conditioned fear inhibits c-fos mRNA expression in the central extended amygdala. [Research Support, N.I.H., Extramural]. Brain research, 1229, 137146. doi: 10.1016/j.brainres.2008.06.085 Day, J. J., Roitman, M. F., Wightman, R. M., & Carelli, R. M. (2007). Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens. [Research Support, N.I.H., Extramural]. Nature neuroscience, 10(8), 1020-1028. doi: 10.1038/nn1923 de Wit, H., & Wise, R. A. (1977). Blockade of cocaine reinforcement in rats with the dopamine receptor blocker pimozide, but not with noradrenergic blockers phentolamine and phenoxybenzamine. Canadian Journal of Psychology, 31, 195203. Di Ciano, P., & Everitt, B. J. (2004). Contribution of the ventral tegmental area to cocaine-seeking maintained by a drug-paired conditioned stimulus in rats. European Journal of Neuroscience, 19(6), 1661-1667. Dragunow, M., & Faull, R. (1989). The use of c-fos as a metabolic marker in neuronal pathway tracing. [Review]. Journal of neuroscience methods, 29(3), 261-265. Ehrman, R. N., Robbins, S. J., Childress, A. R., & O'Brien, C. P. (1992). Conditioned responses to cocaine-related stimuli in cocaine abuse patients. Psychopharmacology, 107(4), 523-529. Fallon, J. H., & Ciofi, P. (1992). Distribution of monoamines within the amygdala The Amgydala: Neurobiological Aspects of Emotion, Memory, and Mental Dysfunction (pp. 97-114). (Reprinted from: IN FILE). Fibiger, H. C., Phillips, A. G., & Brown, E. E. (1992). The neurobiology of cocaineinduced reinforcement. Ciba Foundation symposium, 166, 96-111. Fields, H. L., Hjelmstead, G. O., Margolis, E. B., & Nicola, S. M. (2007). Ventral tegmental area neurons in learned appetitive behavior and positive reinforcement. Annual Reviews in Neuroscience, 30, 289-316. Fiorino, D. F., Coury, A., Fibiger, H. C., & Phillips, A. G. (1993). Electrical stimulation of reward sites in the ventral tegmental area increases dopamine transmission in the nucleus accumbens of the rat. Behavioral Brain Research, 55, 131-141. Floresco, S. B., West, A. R., Ash, B., Moore, H., & Grace, A. A. (2003). Afferent modulation of dopamine neuron firing differentially regulates tonic and phasic dopamine transmission. Nat.Neurosci., 6(9), 968-973. Floresco, S. B., Yang, C. R., Phillips, A. G., & Blaha, C. D. (1998). Basolateral amygdala stimulation evokes glutamate receptor-dependent dopamine efflux in the nucleus accumbens of the anaesthetized rat. European Journal of Neuroscience, 10(4), 1241-1251. 87 Fouriezos, G., Hansson, P., & Wise, R. A. (1978). Neuroleptic-induced attenuation of brain stimulation reward in rats. Journal of Comparative and Physiological Psychology, 92, 661. Fuchs, R. A., Tran-Nguyen, L. T., Specio, S. E., Groff, R. S., & Neisewander, J. L. (1998). Predictive validity of the extinction/reinstatement model of drug craving. [Comparative Study Research Support, Non-U.S. Gov't Research Support, U.S. Gov't, P.H.S.]. Psychopharmacology, 135(2), 151-160. Fuxe, K., & et al. (1974). Central catecholamine neurons, behavior and neuroleptic drugs: An analysis to understand the involvement of catecholamines in schizophrenia. Journal of Psychiatric Research, 11, 151-161. Garzon, M., Vaughan, R. A., Uhl, G. R., Kuhar, M. J., & Pickel, V. M. (1999). Cholinergic axon terminals in the ventral tegmental area target a subpopulation of neurons expressing low levels of the dopamine transporter. Journal of Comparative Neurology, 410 (2 ), 197 - 210. Geisler, S., Derst, C., Veh, R. W., & Zahm, D. S. (2007). Glutamatergic afferents of the ventral tegmental area in the rat. Journal of Neuroscience, 27(21), 5730-5743. Geisler, S., & Zahm, D. S. (2005). Afferents of the ventral tegmental area in the ratanatomical substratum for integrative functions. Journal of Comparative Neurology, 490(3), 270-294. Georges, F., & Aston-Jones, G. (2002). Activation of ventral tegmental area cells by the bed nucleus of the stria terminalis: a novel excitatory amino acid input to midbrain dopamine neurons. Journal of Neuroscience, 22 (12 ), 5173 - 5187. Goldberg, S. R. (1973). Comparable behavior maintained under fixed-ratio and secondorder schedules of food presentation, cocaine injection or d-amphetamine injection in the squirrel monkey. The Journal of pharmacology and experimental therapeutics, 186(1), 18-30. Goldberg, S. R. (1975). Stimuli associated with drug injections as events that control behavior. [Research Support, U.S. Gov't, Non-P.H.S. Research Support, U.S. Gov't, P.H.S.]. Pharmacological reviews, 27(3), 325-340. Grace, A. A., & Bunney, B. S. (1984). The control of firing pattern in nigral dopamine neurons: single spike firing. J.Neurosci, 4(11), 2866-2876. Gratton, A., & Wise, R. A. (1994). Drug- and behavior-associated changes in dopaminerelated electrochemical signals during intravenous cocaine self-administration in rats. Journal of Neuroscience, 14, 4130-4146. Greengard, P. (2001). The neurobiology of slow synaptic transmission. Science, 294, 1024-1030. Grimm, J. W., Hope, B. T., Wise, R. A., & Shaham, Y. (2001). Neuroadaptation. Incubation of cocaine craving after withdrawal. Nature, 412(6843), 141-142. doi: 10.1038/35084134 Gronier, B., & Rasmussen, K. (1998). Activation of midbrain presumed dopaminergic neurones by muscarinic chollinergic receptors: An in vivo electrophysiological study in the rat. British Journal of Pharmacology, 124, 455-465. Harris, G. C., & Aston-Jones, G. (2003). Critical role for ventral tegmental glutamate in preference for a cocaine-conditioned environment. Neuropsychopharmacology, 28 (1 ), 73 - 76. 88 Harris, G. C., Wimmer, M., Byrne, R., & Aston-Jones, G. (2004). Glutamate-associated plasticity in the ventral tegmental area is necessary for conditioning environmental stimuli with morphine. Neuroscience, 129 (3 ), 841 - 847. Hebb, D. O. (1949). The Organization of Behavior. New York: Wiley (Interscience). Heien, M. L., & Wightman, R. M. (2006). Phasic dopamine signaling during behavior, reward, and disease states. [Research Support, N.I.H., Extramural Review]. CNS & neurological disorders drug targets, 5(1), 99-108. Heimer, L., Zaborszky, L., Zahm, D. S., & Alheid, G. F. (1987). The ventral striatopallidothalamic projection: I. The striatopallidal link originating in the striatal parts of the olfactory tubercle. [Research Support, U.S. Gov't, P.H.S.]. The Journal of comparative neurology, 255(4), 571-591. doi: 10.1002/cne.902550409 Henderson, Z., & Sherriff, F. E. (1991). Distribution of choline acetyltransferase immunoreactive axons and terminals in the rat and ferret brainstem. Journal of Comparative Neurology, 314(1), 147-163. Hernandez, L., & Hoebel, B. G. (1988). Food reward and cocaine increase extracellular dopamine in the nucleus accumbens as measured by microdialysis. Life Sciences, 42, 1705-1712. Horvitz, J. C. (2000). Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events. Neuroscience, 96(4), 651-655. Horvitz, J. C., Stewart, J., & Jacobs, B. L. (1997). Burst activity of ventral tegmental dopamine neurons is elicited by sensory stimuli in the awake cat. Brain Research, 759(2), 251-258. Howland, J. G., Taepavarapruk, P., & Phillips, A. G. (2002). Glutamate receptordependent modulation of dopamine efflux in the nucleus accumbens by basolateral, but not central, nucleus of the amygdala in rats. Journal of Neuroscience, 22(3), 1137-1145. Ikemoto, S. (2007). Dopamine reward circuitry: two projection systems from the ventral midbrain to the nucleus accumbens-olfactory tubercle complex. Brain Research Reviews, 56(1), 27-78. Ikemoto, S., & Panksepp, J. (1996). Dissociations between appetitive and consummatory responses by pharmacological manipulations of reward-relevant brain regions. Behavioral Neuroscience, 110(2), 331-345. Janak, P. H., Chang, J. Y., & Woodward, D. J. (1999). Neuronal spike activity in the nucleus accumbens of behaving rats during ethanol self-administration. Brain Research, 817, 172-184. Jay, T. M. (2003). Dopamine: a potential substrate for synaptic plasticity and memory mechanisms. Progress in Neurobiology, 69(6), 375-390. Johnson, S. W., & North, R. A. (1992). Two types of neurone in the rat ventral tegmental area and their synaptic inputs. Journal of Physiology, 450, 455-468. Kandel, E. R. (2001). The molecular biology of memory storage: a dialogue between genes and synapses. Science, 294(5544), 1030-1038. Kelley, A. E. (1999). Neural integrative activities of nucleus accumbens subregions in relation to learning and motivation. Psychobiology, 27(2), 198-213. Kest, K., Cruz, I., Chen, D. H., Galaj, E., & Ranaldi, R. (2012). A food-associated CS activates c-Fos in VTA DA neurons and elicits conditioned approach. Behavioural brain research, 235(2), 150-157. doi: 10.1016/j.bbr.2012.07.044 89 Kiyatkin, E. A., & Gratton, A. (1994). Electrochemical monitoring of extracellular dopamine in nucleus accumbens of rats lever-pressing for food. Brain Research, 652, 225-234. Kiyatkin, E. A., & Rebec, G. V. (1998). Heterogeneity of ventral tegmental area neurons: single-unit recording and iontophoresis in awake, unrestrained rats. Neuroscience, 85(4), 1285-1309. Kiyatkin, E. A., & Stein, E. A. (1996). Conditioned changes in nucleus accumbens dopamine signal established by intravenous cocaine in rats. Neuroscience Letters, 211(2), 73-76. Kofman, O., & Yeomans, J. S. (1988). Cholinergic antagonists in ventral tegmentum elevate thresholds for lateral hypothalamic and brainstem self-stimulation. Pharmacology, Biochemistry and Behavior, 31, 547-559. Lavin, A., Noqueira, L., Lapish, C. C., Wightman, R. M., Phillips, P. E., & Seamans, J. (2005). Mesocortical dopamine neurons operate in distinct temporal domains using multimodal signaling. Journal of Neuroscience, 25(20), 5013-5023. Lewis, B. L., & O'Donnell, P. (2000). Ventral tegmental area afferents to the prefrontal cortex maintain membrane potential 'up' states in pyramidal neurons via D(1) dopamine receptors. Cerebral Cortex, 10(12), 1168-1175. Lisman, J., Schulman, H., & Cline, H. (2002). The molecular basis of CaMKII function in synaptic and behavioural memory. [Review]. Nature reviews. Neuroscience, 3(3), 175-190. doi: 10.1038/nrn753 Ljungberg, T., Apicella, P., & Schultz, W. (1992). Responses of monkey dopamine neurons during learning of behavioral reactions. Journal of Neurophysiology, 67(1), 145-163. Lodge, D. J., & Grace, A. A. (2006). The laterodorsal tegmentum is essential for burst firing of ventral tegmental area dopamine neurons. Proc.Natl.Acad.Sci.U.S.A, 103(13), 5167-5172. Luu, P., & Malenka, R. C. (2008). Spike timing-dependent long-term potentiation in ventral tegmental area dopamine cells requires PKC. J.Neurophysiol., 100(1), 533-538. Margolis, E. B., Lock, H., Chefer, V. I., Shippenberg, T. S., Hjelmstad, G. O., & Fields, H. L. (2006). Kappa opioids selectively control dopaminergic neurons projecting to the prefrontal cortex. Proc.Natl.Acad.Sci.U.S.A, 103(8), 2938-2942. Meil, W. M., & See, R. E. (1996). Conditioned cued recovery of responding following prolonged withdrawal from self-administered cocaine in rats: an animal model of relapse. Behavioural pharmacology, 7(8), 754-763. Miller, A. D., & Blaha, C. D. (2005). Midbrain muscarinic receptor mechanisms underlying regulation of mesoaccumbens and nigrostriatal dopaminergic transmission in the rat. European Journal of Neuroscience, 21(7), 1837-1846. Miller, J. D., Sanghera, M. K., & German, D. C. (1981). Mesencephalic dopaminergic unit activity in the behaviorally conditioned rat. Life Science, 29(12), 1255-1263. Mogenson, G. J. (1987). Limbic-motor integration. Progress in Psychobiology and Physiological Psychology, 12, 117-170. Mogenson, G. J., Jones, D. L., & Yim, C. Y. (1980). From motivation to action: functional interface between the limbic system and the motor system. Progress in Neurobiology, 14, 69-97. 90 Montague, P. R., Dayan, P., & Sejnowski, T. J. (1996). A framework for mesencephalic dopamine systems based on predictive Hebbian learning. [Comparative Study Research Support, Non-U.S. Gov't Research Support, U.S. Gov't, P.H.S. Review]. The Journal of neuroscience : the official journal of the Society for Neuroscience, 16(5), 1936-1947. Murschall, A., & Hauber, W. (2006). Inactivation of the ventral tegmental area abolished the general excitatory influence of Pavlovian cues on instrumental performance. Learn.Mem., 13(2), 123-126. Naranjo, C. A., Tremblay, L. K., & Busto, U. E. (2001). The role of the brain reward system in depression. [Research Support, Non-U.S. Gov't Review]. Progress in neuro-psychopharmacology & biological psychiatry, 25(4), 781-823. Nicola, S. M. (2007). The nucleus accumbens as part of a basal ganglia action selection circuit. Psychopharmacology, 191(3), 521-550. Nicola, S. M., & Deadwyler, S. A. (2000). Firing rate of nucleus accumbens neurons is dopamine-dependent and reflects the timing of cocaine-seeking behavior in rats on a progressive ratio schedule of reinforcement. Journal of Neuroscience, 20(14), 5526-5537. Nicola, S. M., Taha, S. A., Kim, S. W., & Fields, H. L. (2005). Nucleus accumbens dopamine release is necessary and sufficient to promote the behavioral response to reward-predictive cues. Neuroscience, 135(4), 1025-1033. Niznik, H. B., & van Tol, H. H. M. (1992). Dopamine receptor genes: new tools for molecular psychiatry. Journal of Psychiatry and Neuroscience, 17, 158-180. Nugent, F. S., Hwong, A. R., Udaka, Y., & Kauer, J. A. (2008). High-frequency afferent stimulation induces long-term potentiation of field potentials in the ventral tegmental area. Neuropsychopharmacology, 33(7), 1704-1712. O'Donnell, P., & Grace, A. A. (1995). Synaptic interactions among excitatory afferents to nucleus accumbens neurons: hippocampal gating of prefrontal cortical input. Journal of Neuroscience, 15(5), 3622-3639. Oakman, S. A., Faris, P. L., Kerr, P. E., Cozzari, C., & Hartman, B. K. (1995). Distribution of pontomesencephalic cholinergic neurons projecting to substantia nigra differs significantly from those projecting to ventral tegmental area. The Journal of Neuroscience, 15(9), 5859-5869. Olds, J., & Milner, P. M. (1954). Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain. Journal of Comparative and Physiological Psychology, 47, 419-427. Omelchenko, N., & Sesack, S. R. (2006). Cholinergic axons in the rat ventral tegmental area synapse preferentially onto mesoaccumbens dopamine neurons. Journal of Comparative Neurology, 494(6), 863-875. Overton, P. G., Richards, C. D., Berry, M. S., & Clark, D. (1999). Long-term potentiation at excitatory amino acid synapses on midbrain dopamine neurons. Neuroreport, 10(2), 221-226. Pan, W. X., Schmidt, R., Wickens, J. R., & Hyland, B. I. (2005). Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. Journal of Neuroscience, 25(26), 6235-6242. Panksepp, J. (1998), Series in Affective Science Oxford University Press 91 Pfaus, J. G., Damsma, G., Wenkstern, D., & Fibiger, H. C. (1995). Sexual activity increases dopamine transmission in the nucleus accumbens and striatum of female rats. Brain Research, 693, 21-30. Phillips, A. G., Atkinson, L. J., Blackburn, J. R., & Blaha, C. D. (1993). Increased extracellular dopamine in the nucleus accumbens of the rat elicited by a conditional stimulus for food: an electrochemical study. Canadian Journal of Physiological Pharmacology, 71(5-6), 387-393. Phillips, P. E., Stuber, G. D., Heien, M. L., Wightman, R. M., & Carelli, R. M. (2003). Subsecond dopamine release promotes cocaine seeking. Nature, 422(6932), 614618. Phillipson, O. T. (1979). Afferent projections to the ventral tegmental area of Tsai and interfascicular nucleus: a horseradish peroxidase study in the rat. Journal of Comparative Neurology, 187 (1 ), 117 - 143. Rada, P. V., Mark, G. P., Yeomans, J. S., & Hoebel, B. G. (2000). Acetylcholine release in ventral tegmental area by hypothalamic self-stimulation, eating, and drinking. Pharmacology, Biochemistry and Behavior, 65(3), 375-379. Ranaldi, R. (2014). Dopamine and reward seeking: the role of ventral tegmental area. Reviews in the neurosciences. doi: 10.1515/revneuro-2014-0019 Ranaldi, R., & Beninger, R. J. (1994). The effects of systemic and intracerebral injections of D 1 and D 2 agonists on brain stimulation reward. Brain Research, 651, 283292. Ranaldi, R., Kest, K., Zellner, M. R., Lubelski, D., Muller, J., Cruz, Y., & Saliba, M. (2011). The effects of VTA NMDA receptor antagonism on reward-related learning and associated c-fos expression in forebrain. Behavioural brain research, 216(1), 424-432. doi: 10.1016/j.bbr.2010.08.026 Ranaldi, R., Pocock, D., Zereik, R., & Wise, R. A. (1999). Dopamine fluctuations in the nucleus accumbens during maintenance, extinction, and reinstatement of intravenous D-amphetamine self-administration. Journal of Neuroscience, 19(10), 4102-4109. Ranaldi, R., & Roberts, D. C. S. (1996). Initiation, maintenance and extinction of cocaine self-administration with and without conditioned reward. Psychopharmacology, 128, 89-96. Redgrave, P., Prescott, T. J., & Gurney, K. (1999). Is the short-latency dopamine response too short to signal reward error? Trends Neurosci., 22(4), 146-151. Richardson, N. R., & Gratton, A. (1996). Behavior-relevant changes in nucleus accumbens dopamine transmission elicited by food reinforcement: an electrochemical study in rat. Journal of Neuroscience, 16(24), 8160-8169. Robinson, T. E., & Berridge, K. C. (1993). The neural basis of drug craving: an incentive-sensitization theory of addiction. Brain Research Reviews, 18, 247-291. Rodriguez, J. J., Doherty, M. D., & Pickel, V. M. (2008). N-methyl-D-aspartate (NMDA) receptors in the ventral tegmental area: subcellular distribution and colocalization with 5-hydroxytryptamine(2A) receptors. Journal of Neuroscience Research, 60(2), 202-211. Romo, R., & Schultz, W. (1990). Dopamine neurons of the monkey midbrain: contingencies of responses to active touch during self-initiated arm movements. Journal of Neurophysiology, 63(3), 592-606. 92 Rossetti, Z. L., & Wise, R. A. (1995). Electrical stimulation of the prefrontal cortex increases extracellular glutamate in the ventral tegmental area. Society for Neuroscience Abstracts, 21, 1378. Salamone, J. D., Correa, M., Farrar, A., & Mingote, S. (2007). Effort-related functions of nucleus accumbens dopamine and associated forebrain circuits. Psychopharmacology, 191(3), 461-482. Schilstrom, B., Svensson, H. M., Svensson, T. H., & Nomikos, G. G. (1998). Nicotine and food induced dopamine release in the nucleus accumbens of the rat: putative role of alpha7 nicotinic receptors in the ventral tegmental area. Neuroscience, 85 (4 ), 1005 - 1009. Schultz, W. (1997). A neural substrate of prediction and reward. Science, 275, 15931599. Schultz, W. (1998). Predictive reward signal of dopamine neurons. Journal of Neurophysiology, 80(1), 1-27. Schultz, W., Tremblay, L., & Hollerman, J. R. (2000). Reward processing in primate orbitofrontal cortex and basal ganglia. [Research Support, Non-U.S. Gov't]. Cerebral cortex, 10(3), 272-284. Semenova, S., & Markou, A. (2003). Cocaine-seeking behavior after extended cocainefree periods in rats: role of conditioned stimuli. [Research Support, Non-U.S. Gov't Research Support, U.S. Gov't, P.H.S.]. Psychopharmacology, 168(1-2), 192-200. doi: 10.1007/s00213-002-1251-7 Sesack, S. R., Deutch, A. Y., Roth, R. H., & Bunney, B. S. (1989). Topographical organization of the efferent projections of the medial prefrontal cortex in the rat: an anterograde tract-tracing study with Phaseolus vulgaris leucoagglutinin. Journal of Comparative Neurology, 290(2), 213-242. Sesack, S. R., & Pickel, V. M. (1992). Prefrontal cortical efferents in the rat synapse on unlableled neuronal targets of catecholamine terminals in the nucleus accumbens septi and on dopamine neurons in the ventral tegmental area. Journal of Comparative Neurology, 320(2), 145-160. Sharf, R., McKelvey, J., & Ranaldi, R. (2006). Blockade of muscarinic acetylcholine receptors in the ventral tegmental area prevents acquisition of food-rewarded operant responding in rats. Psychopharmacology (Berl), 186(1), 113-121. doi: 10.1007/s00213-006-0352-0 Sharf, R., & Ranaldi, R. (2006a). Blockade of muscarinic acetylcholine receptors in the ventral tegmental area disrupts food-related learning in rats. Psychopharmacology, 184, 87-94. Sharf, R., & Ranaldi, R. (2006b). Blockade of muscarinic acetylcholine receptors in the ventral tegmental area disrupts food-related learning in rats. Psychopharmacology (Berl), 184(1), 87-94. doi: 10.1007/s00213-005-0235-9 Smith, Y., Charara, A., & Parent, A. (1996). Synaptic innervation of midbrain dopaminergic neurons by glutamate-enriched terminals in the squirrel monkey. Journal of Comparative Neurology, 364(2), 231-253. Smith-Roe, S. L., & Kelley, A. E. (2000). Coincident activation of NMDA and dopamine D1 receptors within the nucleus accumbens core is required for appetitive instrumental learning. Journal of Neuroscience, 20(20), 7737-7742. 93 Sombers, L. A., Beyene, M., Carelli, R. M., & Wightman, R. M. (2009). Synaptic overflow of dopamine in the nucleus accumbens arises from neuronal activity in the ventral tegmental area. Journal of Neuroscience, 29(6), 1735-1742. Spooren, W. P., Veening, J. G., Groenewegen, H. J., & Cools, A. R. (1991). Efferent connections of the striatopallidal and amygdaloid components of the substantia innominata in the cat: projections to the nucleus accumbens and caudate nucleus. Neuroscience, 44(2), 431-447. Stewart, J. (1984). Reinstatement of heroin and cocaine self-administration behavior in the rat by intracerebral application of morphine in the ventral tegmental area. Pharmacology, Biochemistry and Behavior, 20, 917-923. Stewart, J., de Wit, H., & Eikelboom, R. (1984). Role of unconditioned and conditioned drug effects in the self-administration of opiates and stimulants. Psychological Reviews, 19, 251-268. Stuber, G. D., Klanker, M., de, R. B., Bowers, M. S., Joosten, R. N., Feenstra, M. G., & Bonci, A. (2008). Reward-predictive cues enhance excitatory synaptic strength onto midbrain dopamine neurons. Science, 321(5896), 1690-1692. Swanson, L. W. (1982). The projections of the ventral tegmental area and adjacent regions:a combined flourescent retrograde tracer and immunofluorescence study in the rat. Brain Research Bulletin, 9, 321-353. Takahata, R., & Moghaddam, B. (1998). Glutamatergic regulation of basal and stimulusactivated dopamine release in the prefrontal cortex. Journal of Neurochemistry, 71, 1443-1449. Talmi, D., Seymour, B., Dayan, P., & Dolan, R. J. (2008). Human pavlovian-instrumental transfer. Journal of Neuroscience, 28(2), 360-368. Testa, A., Festa, F., Ranaldi, R., Giachelia, M., Tirindelli, D., De Marco, A., . . . Cozzi, R. (2005). A multi-biomarker analysis of DNA damage in automobile painters. Environ Mol Mutagen, 46(3), 182-188. doi: 10.1002/em.20147 Wallace, D. M., Magnuson, D. J., & Gray, T. S. (1992). Organization of amygdaloid projections to brainstem dopaminergic, noradrenergic, and adrenergic cell groups in the rat. [Research Support, Non-U.S. Gov't Research Support, U.S. Gov't, NonP.H.S. Research Support, U.S. Gov't, P.H.S.]. Brain research bulletin, 28(3), 447454. Watanabe, M. (1990). Prefrontal unit activity during associative learning in the monkey. [Research Support, Non-U.S. Gov't]. Experimental brain research. Experimentelle Hirnforschung. Experimentation cerebrale, 80(2), 296-309. Westerink, B. H. C., Enrico, P., Feimann, J., & De Vries, J. B. (1998). The pharmacology of mesocortical dopamine neurons: a dual-probe microdialysis study in the ventral tegmental area and prefrontal cortex of the rat brain. Journal of Pharmacology and Experimental Therapeutics, 285(1), 143-154. White, N. M., & Milner, P. M. (1992). The psychobiology of reinforcers. Annual Reviews in Psychology, 43, 443-471. Wilson, C., Nomikos, G. G., Collu, M., & Fibiger, H. C. (1995). Dopaminergic correlates of motivated behavior:importance of drive. The Journal of Neuroscience, 15(7), 5169-5178. Wise, R. A. (1978). Catecholamine theories of reward: a critical review. Brain Research, 152, 215-247. 94 Wise, R. A. (1994). Cocaine reward and cocaine craving: the role of dopamine in perspective. [Review]. NIDA research monograph, 145, 191-206. Wise, R. A. (2004). Dopamine, learning and motivation. Nat Rev Neurosci, 5(6), 483494. Wise, R. A. (2006). Role of brain dopamine in food reward and reinforcement. Philosophical Transactions of the Royal Society B: Biological Sciences, 361(1471), 1149-1158. Wise, R. A., & Bozarth, M. A. (1987). A psychomotor stimulant theory of addiction. Psychological Reviews, 94(4), 469-492. Wise, R. A., & RomprÇ, P. P. (1989). Brain dopamine and reward. Annual Reviews in Psychology, 40, 191-225. Wise, R. A., & Rompre, P. P. (1989). Brain dopamine and reward. [Research Support, Non-U.S. Gov't Review]. Annual review of psychology, 40, 191-225. doi: 10.1146/annurev.ps.40.020189.001203 Wise, R. A., Spindler, J., deWit, H., & Gerber, G. J. (1978). Neuroleptic-induced "anhedonia" in rats: pimozide blocks reward quality of food. Science, 201, 262264. Wolterink, G., Phillips, G., Cador, M., Donselaar-Wolterink, I., Robbins, T. W., & Everitt, B. J. (1993). Relative roles of ventral striatal D 1 and D 2 dopamine receptors in responding with conditioned reinforcement. Psychopharmacology, 110, 355-364. Wright, C. I., & Groenewegen, H. J. (1995). Patterns of convergence and segregation in the medial nucleus accumbens of the rat:relationships of prefrontal cortical,midline thalamic, and basal amygdaloid afferents. The Journal of Comparative Neurology, 361, 383-403. Wu, Y. N., & Johnson, S. W. (1996). Pharmacological characterization of inward current evoked by N-methyl-D-aspartate in dopamine neurons in the rat brain slice. [In VitroResearch Support, U.S. Gov't, P.H.S.]. The Journal of pharmacology and experimental therapeutics, 279(2), 457-463. Yamaguchi, T., Sheen, W., & Morales, M. (2007). Glutamatergic neurons are present in the rat ventral tegmental area. European Journal of Neuroscience, 25(1), 106-118. Yeomans, J. S., Mathur, A., & Tampakeras, M. (1993). Rewarding brain stimulation: role of tegmental cholinergic neurons that activate dopamine neurons. Behavioral Neuroscience, 107, 1077. Yokel, R. A., & Wise, R. A. (1975). Increased lever pressing for amphetamine after pimozide in rats: implications for a dopamine theory of reward. Science, 187, 547-549. Yoshida, M., Yokoo, H., Mizoguchi, K., Kawahara, H., Tsuda, A., Nishikawa, T., & Tanaka, M. (1992). Eating and drinking cause increased dopamine release in the nucleus accumbens and ventral tegmental area in the rat: measurement by in vivo microdialysis. Neuroscience Letters, 139 (1 ), 73 - 76. You, Z. B., Wang, B., Zitzman, D., Azari, S., & Wise, R. A. (2007). A role for conditioned ventral tegmental glutamate release in cocaine seeking. J Neurosci, 27(39), 10546-10555. doi: 27/39/10546 [pii] 10.1523/JNEUROSCI.2967-07.2007 You, Z. B., Wang, B., Zitzman, D., & Wise, R. A. (2008). Acetylcholine release in the mesocorticolimbic dopamine system during cocaine seeking: conditioned and 95 unconditioned contributions to reward and motivation. J Neurosci, 28(36), 90219029. doi: 28/36/9021 [pii] 10.1523/JNEUROSCI.0694-08.2008 Young, A. M., Ahier, R. G., Upton, R. L., Joseph, M. H., & Gray, J. A. (1998). Increased extracellular dopamine in the nucleus accumbens of the rat during associative learning of neutral stimuli. Neuroscience, 83(4), 1175-1183. Yun, I. A., Wakabayashi, K. T., Fields, H. L., & Nicola, S. M. (2004). The ventral tegmental area is required for the behavioral and nucleus accumbens neuronal firing responses to incentive cues. Journal of Neuroscience, 24 (12 ), 2923 - 2933. Zellner, M. R., Kest, K., & Ranaldi, R. (2009a). NMDA receptor antagonism in the ventral tegmental area impairs acquisition of reward-related learning. Behav Brain Res, 197(2), 442-449. doi: S0166-4328(08)00565-2 [pii] 10.1016/j.bbr.2008.10.013 Zellner, M. R., Kest, K., & Ranaldi, R. (2009b). NMDA receptor antagonism in the ventral tegmental area impairs acquisition of reward-related learning. Behav.Brain Res., 197(2), 442-449. Zellner, M. R., & Ranaldi, R. (2010). How conditioned stimuli acquire the ability to activate VTA dopamine cells: A proposed neurobiological component of rewardrelated learning. Neuroscience & Biobehavioral Reviews, 34(5), 769-780.