* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Reward loss and addiction: Opportunities for cross
Survey
Document related concepts
Metastability in the brain wikipedia , lookup
Limbic system wikipedia , lookup
Endocannabinoid system wikipedia , lookup
Behaviorism wikipedia , lookup
Parent management training wikipedia , lookup
Sexually dimorphic nucleus wikipedia , lookup
Emotional lateralization wikipedia , lookup
Neuropsychopharmacology wikipedia , lookup
Conditioned place preference wikipedia , lookup
Multi-armed bandit wikipedia , lookup
Orbitofrontal cortex wikipedia , lookup
Clinical neurochemistry wikipedia , lookup
Transcript
Pharmacology, Biochemistry and Behavior 154 (2017) 39–52 Contents lists available at ScienceDirect Pharmacology, Biochemistry and Behavior journal homepage: www.elsevier.com/locate/pharmbiochembeh Review Reward loss and addiction: Opportunities for cross-pollination Leonardo A. Ortega a,⁎, José L. Solano a, Carmen Torres b, Mauricio R. Papini c a b c Department of Psychology, Fundación Universitaria Konrad Lorenz, Bogotá, Colombia Department of Psychology, University of Jaén, Jaén 23071, Spain Department of Psychology, Texas Christian University, Fort Worth, TX 76129, USA a r t i c l e i n f o Article history: Received 24 August 2016 Received in revised form 1 February 2017 Accepted 3 February 2017 Available online 4 February 2017 Keywords: Reward loss Reward devaluation Reward omission Reward comparison Negative emotion Addiction Substance use disorder a b s t r a c t Paradigms used to study the response to and consequences of exposure to reward loss have been underutilized in approaches to the psychobiology of substance use disorders. We propose here that bringing these two areas into contact will help expanding our understanding of both reward loss and addictive behavior, hence opening up opportunities for cross-pollination. This review focuses on two lines of research that point to parallels. First, several neurochemical systems involved in addiction are also involved in the modulation of the behavioral effects of reward loss, including opioid, GABA, and dopamine receptors. Second, there are extensive overlaps in the brain circuitry underlying both reward loss and addiction. Common components of this system include, at least, the amygdala, ventral and dorsal striatum, and various prefrontal cortex regions. Four emerging avenues of research that benefit from emphasis on the common ground between reward loss and addiction are reviewed, namely, the neural circuitry involved in reward devaluation, the influence of genetic and reward history on the behavioral vulnerability and resilience, the role of competing natural rewards, and emotional self-medication. An understanding of the role of reward loss in addiction will point to a deeper understanding of the initiation and maintenance of substance use disorders. © 2017 Elsevier Inc. All rights reserved. Contents 1. 2. Reward loss and addiction: psychobiological parallels . . . . . . Effects of drug of abuse on reward loss . . . . . . . . . . . . . 2.1. Reward loss and opioids . . . . . . . . . . . . . . . . 2.2. Reward loss, conflict, and GABA anxiolytics . . . . . . . . 2.3. Reward loss and the dopamine reward signal . . . . . . . 2.4. Other receptors: cholinergic and cannabinoid receptors . . 3. Neural commonalities between reward loss and addiction . . . . 3.1. Anterior cingulate cortex (ACC) . . . . . . . . . . . . . 3.2. Orbitofrontal cortex (OFC) . . . . . . . . . . . . . . . 3.3. Medial prefrontal cortex (mPFC). . . . . . . . . . . . . 3.4. Insular cortex (IC) . . . . . . . . . . . . . . . . . . . 3.5. Striatum . . . . . . . . . . . . . . . . . . . . . . . 3.6. Lateral habenula (LHb) . . . . . . . . . . . . . . . . . 3.7. Amygdala . . . . . . . . . . . . . . . . . . . . . . . 4. Emergent topics . . . . . . . . . . . . . . . . . . . . . . . 4.1. Circuitry for reward devaluation and implications for SUDs. 4.2. Genetic vulnerability to reward loss and addiction. . . . . 4.3. Role of competing natural rewards. . . . . . . . . . . . 4.4. Reward loss and emotional self-medication . . . . . . . . 5. Concluding remarks. . . . . . . . . . . . . . . . . . . . . . Conflict of interest . . . . . . . . . . . . . . . . . . . . . . . . . Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ⁎ Corresponding author at: Departamento de Psicología, Fundación Universitaria Konrad Lorenz, Carrera 9 Bis No. 62-43, Bogotá, Colombia. E-mail address: [email protected] (L.A. Ortega). http://dx.doi.org/10.1016/j.pbb.2017.02.001 0091-3057/© 2017 Elsevier Inc. All rights reserved. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 41 41 41 42 43 43 43 43 43 44 44 44 44 45 45 46 47 48 48 49 49 49 40 L.A. Ortega et al. / Pharmacology, Biochemistry and Behavior 154 (2017) 39–52 Drug addiction is one of the most pressing and complex current social issues. The latest National Survey on Drug Use and Health (Center for Behavioral Health Statistics and Quality, 2015) estimated that during 2014, around 21.5 million people in the United States exhibited a substance use disorder (SUD), with symptoms including sustained excessive consumption, loss of use control, tolerance, craving, withdrawal, and relapse (American Psychiatric Association, 2013). As a result, the identification of psychobiological factors underlying SUDs is a major challenge for current research. A rich set of behavioral processes and experimental procedures have been suggested and used as a way to better understand the psychobiology of drug addiction, and as a strategy to improve prevention and treatment (Ahmed, 2012). The present review highlights the psychobiological mechanisms and potential applications of reward loss paradigms, an underutilized but promising research area for the comprehensive study of some of the learning, motivational, and emotional factors that contribute to the initiation and maintenance of addictive behavior. Reward loss refers to situations in which an expected appetitive reinforcer (expected on the basis of prevailing signals) is not presented or is reduced in magnitude or quality. Although reward loss does not involve the delivery of aversive stimuli, it instigates behavioral and neurophysiological responses similar to those induced by the presentation or anticipation of aversive events (Amsel, 1992; Papini and Dudley, 1997; Papini et al., 2006). Current research places a special emphasis on understanding the transition from initial drug use to compulsive drug seeking and taking (e.g., Everitt and Robbins, 2016; Kelley and Berridge, 2002; Robinson and Berridge, 2000). Research suggests that the onset, development, and establishment of compulsive drug seeking and taking are mediated by a complex set of underlying psychobiological systems (Goldstein and Volkow, 2011). The interaction of such systems controls, among others, (1) learning and motivational processes dependent on dopaminergic neurons of the mesocorticolimbic brain circuitry (Volkow and Morales, 2015; Gipson and Kalivas, 2016), (2) cognitive deficits related to dysfunctional top-down cortical control of behavior (Robbins et al., 2008), and (3) reward processing and emotional dysregulation (Egli et al., 2012; Koob, 2013; Koob and Le Moal, 2001; Koob and Volkow, 2010; Sinha, 2008). Stressful life events are also relevant to understand the initiation and development of addictive behavior (Gordon, 2002; Hayaki et al., 2005; Keyes et al., 2011). Individuals suffering from addictive disorders have problems to cope with and adjust to unexpected environmental events and uncertainty (Brevers et al., 2015; Ersche et al., 2016). These events involve not only aversive stimulation, but also negative changes in the relative value of appetitive stimulation (that is, the complete or partial loss of reward). Indeed, the most stressful events suggested by Holmes and Rahe's (1967) social readjustment rating scale involve several types of profound reward loss (Papini et al., 2006), which could underlie a wide variety of biobehavioral disorders, such as anxiety (Kessler, 1997), depression (Huston et al., 2013), and drug addiction (Hassanbeigi et al., 2013; Konopka et al., 2013; Waldrop et al., 2007). Flaherty (1996) suggested that the adaptive significance of reward loss must be viewed in terms of reward relativity. According to this notion, organisms make decisions and adjust their behavior toward one incentive as a function of their experience with different rewards. For instance, the emotional impact of rejection by a romantic partner can be best understood in the context of the value of the rewards (e.g., romantic, social, support) received before the separation (Sbarra and Hazan, 2008). The loss of reward derived from a romantic breakup triggers a variety of behavioral, physiological, affective, and cognitive changes that are dependent on brain areas involved in reward processing, emotion, and addiction (Fisher et al., 2010). In fact, that life events inducing negative emotions are related to both drug addiction and reward loss has been separately, but consistently supported by an increasing amount of clinical and experimental research (Hassanbeigi et al., 2013; Konopka et al., 2013; Papini et al., 2015; Spanagel et al., 2014). This review intends to bring focus to the reciprocal benefits that stem out of bringing together two areas of research that have proceeded largely independently: reward loss and drug addiction. Opportunities for cross-pollination of these research areas will be discussed in the following sections in relation to two points: (1) the psychobiological parallels between reward loss and addiction, centered on the effects of some drugs of abuse on reward loss paradigms and on the common components of the neural circuitry underlying reward loss and addiction, and (2) recent advances in reward loss research related to four items, namely, a hypothesis of the neural circuitry underlying reward devaluation, genetic vulnerability for addiction and reward history, drugs and competing natural rewards, and emotional self-medication based on drugs with addictive potential. 1. Reward loss and addiction: psychobiological parallels Reward loss involves the invalidation of anticipatory representations of positive beliefs and goals related to positive reinforcement (Miceli and Castelfranchi, 2015), resulting from a comparison between obtained and expected rewards (Flaherty, 1996). Reward loss is studied in the laboratory using an array of procedures that can be divided in two classes (see Table 1): Reward devaluation, in which the value of a reward is downshifted from a previous higher appetitive value to a lower, nonzero value; and reward omission, in which there is a complete withdrawal of the reward (Papini et al., 2006; Papini et al., 2015). Examples of reward devaluation procedures are consummatory successive negative contrast and instrumental successive negative contrast (Flaherty, 1996). In these tasks, behavior (whether consummatory or instrumental) is rewarded with a large reward for a number of preshift sessions and then the magnitude or quality of the reward is downshifted. This unexpected reward downshift causes either consummatory suppression or instrumental avoidance of the goal. These effects are transient. Examples of reward omission are appetitive extinction paradigms (Norris et al., 2009). In these tasks, either consummatory or instrumental, the downshift is from reward to nonreward, a change that causes the behavior to decrease to very low levels. Reward loss has been widely shown to induce an emotional response described in terms of an aversive internal state usually referred to as frustration (Amsel, 1992; Papini and Dudley, 1997), disappointment (Flaherty, 1996; Miceli and Castelfranchi, 2015), anxiety (Gray, 1987), or, more recently, psychological pain (Papini et al., 2015). This emotional state is accompanied by a set of responses that include aggressive behavior, escape responses, behavioral inhibition, release of stress hormones, and depression-related symptoms (Huston et al., 2013; Papini and Dudley, 1997). In particular, there is extensive research showing a close relationship between Table 1 Experimental procedures involving reward loss: definitions and descriptions. Reward Procedure Devaluation Consummatory successive negative contrast Devaluation Instrumental successive negative contrast Omission Consummatory extinction Omission Instrumental extinction Example After several sessions of access to 32% sucrose, the consummatory behavior of rats is suppressed when downshifted to 4% sucrose, relative to unshifted controls always receiving 4% sucrose. After receiving 12 food pellets for traversing a runway, the rat's running response deteriorates when the reward is downshifted to 1 food pellet, relative to unshifted controls always receiving 1 pellet. After several sessions of access to 32% sucrose, the consummatory behavior of rats is sharply reduced by a downshift to water. After running down a runway for 12 food pellets on either a continuous or 50% partial reinforcement schedule, the running response is reduced, usually faster after continuous than after partial reinforcement acquisition. L.A. Ortega et al. / Pharmacology, Biochemistry and Behavior 154 (2017) 39–52 reward loss and negative emotion. Both reward devaluation and omission are followed by activation of the hypothalamic-pituitary-adrenal (HPA) axis (Coe et al., 1983; Coover et al., 1971; Davis et al., 1976; Flaherty et al., 1985; Kawasaki and Iwasaki, 1997; Mitchell and Flaherty, 1998; Pecoraro et al., 2009). Additionally, the effects of reward loss are modulated by both early and late sources of negative emotion, including exposure to the random neonatal stress procedure (Ruetti et al., 2010), early handling (Cuenya et al., 2014), maternal separation (Matthews et al., 1996), social isolation during adolescence and/or adulthood (Shanab and Ralph, 1979), physical restraint (Ortega et al., 2013b), open field exposure (Justel et al., 2014), and peripheral pain (Ortega et al., 2011a), among others. Negative emotional states also affect drug addiction. Increased activity of the HPA axis facilitates drug self-administration (Piazza et al., 1991) and individual stressor reactivity is persistently disrupted by chronic drug self-administration (Mantsch et al., 2007). Emotion induced by stimuli such as foot shocks, forced swimming, or immobilization facilitates drug self-administration and reinstatement of drug seeking behavior after a drug-free period (Ahmed and Koob, 1997; Anisman and Waller, 1974; Fullgrabe et al., 2007; Goeders and Guerin, 1994; Nash and Maickel, 1985; Piazza and Le Moal, 1998; Shaham et al., 2000). Social situations such as isolation, interaction, aggression, and defeat increase drug self-administration (Alexander et al., 1981; Caldwell and Riccio, 2010; Marks-Kaufman and Lewis, 1984). Chronic stress is also a key vulnerability factor in the development and relapse of addiction (Brady and Sinha, 2005; Sinha, 2008). Overall, interactions between the stress response and reward brain systems have been proposed as the neurobiological basis for increases in the acute use and chronic abuse of drugs induced by negative emotional states (Yap and Miczek, 2008), and both become dysregulated in addicted organisms (Edwards and Koob, 2010; Egli et al., 2012; Koob and Le Moal, 2001; Spanagel et al., 2014). Similar interventions also affect and are affected by reward-loss, including restraint stress and physical pain (Jiménez-García et al., 2016; Mustaca and Papini, 2005; Ortega et al., 2011a; Ortega et al., 2013b), HPA axis activation (Pecoraro et al., 2009), postdevaluation administration of memory enhancers like corticosterone and D-cycloserine (Bentosela et al., 2006; Norris et al., 2011; Ruetti et al., 2009), dopamine release in the nucleus accumbens (Genn et al., 2004a), amygdala lesions (Kawasaki et al., 2015, submitted), dorsomedial striatum lesions (Torres et al., 2016), anterior cingulate cortex lesions (Ortega et al., 2011b), and insular cortex lesions (Lin et al., 2009), among others. As discussed below, this neurobehavioral overlap suggests a bidirectional communication between reward loss and addiction that can help us understand, for example, how a reward loss experience can increase the individual's vulnerability to develop an SUD and why addictions can make a person more reactive to an experience of loss. 41 2.1. Reward loss and opioids A role of opioid receptors on reward devaluation is suggested by studies administering opioid agonists and antagonists. Rowan and Flaherty (1987) reported that the effects of reward devaluation were diminished by morphine and also that this effect of morphine was abolished by naloxone, an opiate antagonist. Naloxone was also found to increase behavioral suppression induced by the downshift of a sucrose solution (Daniel et al., 2009; Pellegrini et al., 2005). Morphine and naloxone are nonselective opioid receptor agonist and antagonist, respectively, although they have greater affinity for mu receptors (Papini and Ortega, 2011). Further research based on the administration of selective opioid-receptor agonists and antagonists has shown that opioid receptors play specific roles in different stages of the reward devaluation experience, as proposed by Amsel's (1992) frustration theory and Flaherty's (1996) multistage model. Both theories suggest that the initial reaction to the loss is regulated by different underlying mechanisms relative to the response ensuing after some experience with the devalued reward. Consistent with these views, the selective delta opioid-receptor agonist DPDPE [D-Penicillamine2-D-Penicillamine5-enkephalin] and the antagonist naltrindole reduced and enhanced, respectively, the consummatory suppression induced by reward devaluation (Pellegrini et al., 2005; Wood et al., 2005). Interestingly, these effects were observed when these drugs were administered before the first downshift session, but not when they were administered before the second downshift session. By contrast, the kappa opioid-receptor agonist U50,488H reduced the impact of a second reward devaluation, but had no effect when administered before the first devaluation (Wood et al., 2008). Importantly, none of these drugs affected consummatory behavior in unshifted controls, thus suggesting a specific action of opioids on psychological pain derived from the loss of an expected reward, rather than on the response to the current reward. A role of opioid receptors on reward omission has also been reported. Morphine has a dose-dependent (biphasic) effect on responding during the extinction of a lever press response (Molinengo, 1964), whereas endorphins delayed appetitive extinction from an instrumental task (Le Moal et al., 1979). Conversely, naloxone facilitated food and sexual appetitive extinction, interfered with escape from a context previously associated to appetitive extinction, and accelerated extinction from ethanol-induced conditioned place preference (Cunningham et al., 1998; Holloway, 2012; Norris et al., 2009). Overall, the research reviewed above on both reward devaluation and omission tests suggests that opioid receptors modulate the intensity and time course of the response to an episode involving reward loss (Papini, 2009; Papini et al., 2015). The role of opioids as a source of SUDs can hardly be overemphasized. 2.2. Reward loss, conflict, and GABA anxiolytics 2. Effects of drug of abuse on reward loss Behavior induced by reward loss has been extensively characterized from a pharmacological perspective (Flaherty, 1996; Papini et al., 2015). Most studies have used reward devaluation, rather than omission (appetitive extinction), but there is an emerging literature that focuses on the emotional underpinnings of appetitive extinction (Mustaca et al., 2009; Papini et al., 2015). Most of the studies cited in the review that follows include appropriate controls to discard alternative explanations of drug effects in terms of actions on the processing of the absolute value of the reward, nonspecific drug effects, and sensory-motor interference (see below). Therefore, we argue that such drugs draw their addictive potential in part from their efficacy at reducing negative emotions associated to reward loss, such as frustration, disappointment, and grief. As a result, these studies can help identify with greater precision the functions of various neurochemical systems involved in reward loss and SUDs. Most drugs with anxiolytic effects modulate GABAA receptors (gamma aminobutyric acid type-A receptors) and can support addictive behavior (Lader, 1994; Tan et al., 2011). These drugs also show consistent effects on reward loss. Administration of benzodiazepine anxiolytics, which enhance GABAergic transmission by binding to a specific site in the GABAA receptor, diminishes the impact of reward devaluation on consummatory behavior, a consistent effect that has been shown to be unrelated to the appetite-stimulant effects of these drugs (Flaherty et al., 1986; Flaherty and Rowan, 1988; Mitchell et al., 2004). Interestingly, the ameliorating effect of benzodiazepine anxiolytics on reward devaluation is observed when drugs are administered before the second incentive downshift session, but not when administered before the first downshift session, suggesting that the attenuating action of benzodiazepines on reward loss requires some experience with the devalued reward (Flaherty et al., 1996; Flaherty et al., 1986; Liao and Chuang, 2003; Ortega et al., 2014a). Accordingly, when rats had access to the devalued reward for 20 min in the first postshift session (instead of 42 L.A. Ortega et al. / Pharmacology, Biochemistry and Behavior 154 (2017) 39–52 the usual 5 min), the benzodiazepine chlordiazepoxide had no effect during minutes 1–5, but it attenuated the suppressive effects of reward devaluation during minutes 6–10—the downshift effect dissipated during minutes 11–20 in saline and chlordiazepoxide groups (Flaherty et al., 1986). Additional studies support this hypothesis and exclude alternative explanations based on differences between the first and the second downshift experience in terms of dose/response functions, memory traces, or retention intervals (Flaherty et al., 1986, 1990). Whereas benzodiazepines require some experience with the devalued reward to have an effect, the barbiturate sodium amobarbital, which binds to a different site of the GABAA receptor than that of benzodiazepines, reduces the effects of reward devaluation regardless of prior reward downshift experience, that is, when administered before either the first or the second downshift session (Flaherty et al., 1982). Similar attenuating effects of GABAergic anxiolytics have been observed in reward devaluation tasks based on massed-trial and instrumental situations, in which there is no clear-cut distinction between initial and subsequent reward devaluation trials in terms of pharmacological selectivity (Flaherty and Checke, 1982; Flaherty and Driscoll, 1980; Morales et al., 1992; Phelps et al., 2015; Rosen et al., 1967; Torres et al., 1996). For example, Morales et al. (1992) trained rats in a one-way avoidance task in which a downshift from 30 s to 1 s in the safety compartment deteriorated avoidance behavior relative to an unshifted control always given 1 s in safety. The benzodiazepine diazepam eliminated this contrast effect in a dose-dependent manner. Although benzodiazepines also seem to attenuate the emotional response induced by reward omission, inconsistent results have also been reported in extinction tasks (see Ruetti and Justel, 2010). For example, chlordiazepoxide increased lever press responding (for food or electrical brain self-stimulation) and running behavior (for food) when administered during extinction sessions (Feldon and Gray, 1981; Gandelman and Trowill, 1968; Heise et al., 1970). However, Beck and Loh (1990) found that chronic administration of the benzodiazepine diazepam decreased the rate of instrumental responding and increased competing responses during the extinction in the radial arm task. Similar results were reported during extinction of lever pressing (Leslie et al., 2005, 2012; Williams et al., 1990). Similar inconsistencies were found in consummatory tasks. Benzodiazepines (e.g., diazepam, chlordiazepoxide, lorazepam) increased resistance to extinction when rats were first trained with water as the reward and then exposed to an empty sipper tube in an instrumental extinction task (Bialik et al., 1982; Soubrie et al., 1978); however, diazepam accelerated consummatory extinction after a transition from 32% sucrose to water (Justel et al., 2011). The source for this discrepancy between instrumental and consummatory effects is unclear. Ethanol results in similar anxiolytic effects to those of benzodiazepines and also modulates GABAergic function (Tan et al., 2011). Consistently, the effects of ethanol on reward loss are similar to those previously described for benzodiazepines anxiolytics. Ethanol diminished the behavioral suppression induced by reward downshift when administered before the second devaluation session, but had no effect when administered before the first one (Becker and Flaherty, 1982), and had an effect after several downshift events (Kamenetzky et al., 2008). It should be noted that chlordiazepoxide also has a reducing effect on reward devaluation during the initial session after several downshift cycles (Flaherty et al., 1996). Ethanol had an additive effect when simultaneously administered with chlordiazepoxide using ineffective individual doses (Becker and Flaherty, 1983). Like diazepam, ethanol also facilitated consummatory extinction after a transition from 32% sucrose to water (Kamenetzky et al., 2009b). The surprising effect of posttrial chlordiazepoxide administration on reward devaluation deserves special mention. As noted above, presession administration of chlordiazepoxide before the first downshift session has no effect on reward devaluation. However, postsession administration appears to disrupt memory consolidation processes for learning about the new, devalued reward (Ortega et al., 2014a, Glueck et al., 2015). As a result, the suppressive effects of reward devaluation on consummatory behavior were lengthened. Interestingly, posttrial administration of chlordiazepoxide did not affect the performance of unshifted controls, had no effect when its administration was delayed for 3 h after the session, was ineffective when animals had completely recovered from the downshift event, and did not support a conditioned taste aversion. Thus, the effect of postsession chlordiazepoxide administration on reward devaluation is selectively related to an emotionally significant reward devaluation event. GABAergic transmission is a key component of brain processes that contribute to reduce the negative effects of loss-induced negative emotions. Their activation after some experience with the loss event has taken place suggests that these receptors may be critically involved in the resolution of approach-avoidance conflicts (Flaherty, 1996). After some experience with the devalued conditions, situational cues and signals might become ambivalent, promoting both approach and avoidance of the same goal. Anxiolytics may bias the conflict in favor of approach by reducing negative emotions instigating goal avoidance (Flaherty, 1996; Gray and McNaughton, 2000). 2.3. Reward loss and the dopamine reward signal Although several experiments failed to demonstrate an action of dopaminergic drugs on reward devaluation (Flaherty, 1996; Torres et al., 1996), more recent reports suggest that studying the role of dopamine receptors in reward loss is a promising research line. First, Barr and Phillips (2002) showed that withdrawal from D-amphetamine delayed recovery from reward downshift in a consummatory task, a result interpreted as indicative of increased emotionality in withdrawn animals. Second, Nikiforuk and Popik (2009) trained rats in a progressive ratio schedule of reinforcement in which the value of the last completed response ratio (break point) was used as the dependent variable. When the initial reward, 32% sucrose, was replaced by 4% sucrose, vehicle-treated animals showed a decrease in the break point, whereas animals treated with cocaine, a dopamine reuptake inhibitor, showed little or no decrease in the break point. However, the fact that cocaine also increased responding for unshifted controls suggests a general effect of the drug on reward processing, rather than a specific action on the impact of the reward devaluation. Third, Phelps et al. (2015) trained rats to make a nose-poke response to a light cue to obtain four pellets; the reward was devalued to one pellet in repeated cycles. The indirect dopamine agonist amphetamine attenuated the reward devaluation effect, reducing response latency and increasing premature responses, whereas the D1/D2 dopamine receptor antagonist alpha-flupenthixol had the opposite effect. Although these results were interpreted as indicative of the involvement of dopamine receptors in the drive to respond for a reward, the lack of appropriate nondevalued controls makes it difficult to discard alternative explanations, including reward processing, attention, and motor control. Reward omission paradigms also seem to respond to dopaminergic drugs of abuse. First, Barker and Rebec (2016) reported that repeated cocaine administration resulted in a more negative slope for latencies to respond and a steeper slope for lever presses across days of training, when compared to controls in a reward omission task. Second, Herling et al. (1979) reported that cocaine and D-amphetamine administration in monkeys increased response rates during an operant appetitive extinction task. Third, cocaine and heroin administration prior to extinction of a Pavlovian learning task disrupted extinction performance from an overexpectation procedure (i.e., response decrement after compounding stimuli trained separately and pairing them with the same reward magnitude), but not from simple acquisition (Lucantonio et al., 2015). Finally, acute cocaine administration enhanced extinction responding when brain stimulation of dopaminergic pathways was used as reward in adult rats prenatally stressed (Gao et al., 2011). In the prenatal stress procedure, dams were periodically subject to physical restraint and exposed to a bright light. L.A. Ortega et al. / Pharmacology, Biochemistry and Behavior 154 (2017) 39–52 2.4. Other receptors: cholinergic and cannabinoid receptors The brain cholinergic system has been scarcely explored in rewardloss studies and the little available information shows inconsistent results. For example, the cholinergic agonist nicotine and the antagonist scopolamine were without effects when administered prior the first or second postshift session of a reward devaluation task (Flaherty, 1996; Genn et al., 2004b). With respect to reward omission, scopolamine increased resistance to extinction in a number of instrumental situations, including lever pressing for sucrose (McCoy, 1972) or water (Morley and Russin, 1978), but not for food (Olds, 1970). By contrast, administration of acetylcholinesterase inhibitors resulted in faster extinction after training in several tasks, such as fixed-ratio schedules (Glazer, 1972) and discrimination training (Whitehouse, 1966), and increased errors in a serial problemsolving task (Banks and Russell, 1967). Similarly, nicotine increased response rates during extinction of a lever pressing response associated to either visual stimulation or a 4% sucrose solution (Barret and Bevins, 2013). Type-1 cannabinoid (CB1) receptor agonists can significantly influence the behavioral reaction to reward devaluation. Genn et al. (2004b) found that low doses of CP-55,940 abolished consummatory suppression induced by downshifting a sucrose solution. Similarly, the CB1 receptor agonist WIN 55-212,2, administered into the basolateral amygdala, prevented the enhancing effect of a negative event (elevated platform test inducing open-space anxiety) on instrumental reward devaluation, without affecting the reaction to the reward devaluation itself (Ramot and Akirav, 2012). Reward omission phenomena have also been manipulated by cannabinoid agents, although mixed results have been reported. On the one hand, delta-9-tetrahydrocannabinol (THC) administration can facilitate instrumental extinction (Gonzalez et al., 1971), increase aggression induced by morphine withdrawal (Boshka et al., 1966), and abolish resistance to extinction induced by partial reinforcement training (Drewnowski and Gray, 1975). On the other hand, the CB1 antagonist rimonabant disrupted extinction learning in an aversive, but not in an appetitive Barnes maze conditioning task, suggesting that cannabinoid receptors are less critical for the extinction of this appetitively motivated behavior (Harloe et al., 2008). The research reviewed in this section constitutes the first step to identifying commonalities between reward loss and addiction in pharmacological terms. There is no question that many drugs with abuse potential (e.g., morphine, benzodiazepines, ethanol, psychostimulants, nicotine, cannabis) affect performance in a variety of reward-loss situations. Our hypothesis is that these modulatory drug effects of the negative emotions induced by events involving reward loss are one causal factor in the early development and later maintenance of addictive behavior. One possible avenue to pursue this connection would be to shift from drug administration to drug self-administration, especially under procedures mimicking drug intake in humans. We return to this issue in a later section on emotional self-medication. 3. Neural commonalities between reward loss and addiction The reviewed evidence on the effects of addictive drugs on reward devaluation and omission, and their underlying neurochemical mechanisms suggest parallels between reward loss and addiction at the level of neural circuitry. This section describes research on the role of several brain areas that have been shown independently to modulate reward loss and drug abuse, including the mesocorticolimbic brain circuitry and extended amygdala, typically highlighted in addiction and emotion research (Gipson and Kalivas, 2016; Koob, 2013; Koob and Volkow, 2010, 2016; Lu and Richardson, 2014; Sinha, 2008; Volkow and Morales, 2015). 3.1. Anterior cingulate cortex (ACC) Kalivas and Volkow (2005) suggested two main functions for the ACC in SUDs. First, this area would be involved in the robust motivation 43 induced by stimuli predicting drugs of abuse, compared to stimuli predicting natural rewards. Second, the ACC would be related to difficulties in the cognitive control of drug seeking. This cognitive deficit is particularly obvious in anxiety situations. For instance, Sinha et al. (2005) found that recovery of personal memories related to a negative situation was accompanied by ACC hypoactivation and striatum hyperactivation during craving in cocaine addicts, compared to controls. Based on these and additional findings, Sinha and Li (2007) suggested that dysfunction of the ACC, among other brain structures, is involved in negative emotional states induced by exposure to drug-associated stimuli, increasing the probability of relapse. Several studies suggest that the ACC is also involved in the control of behaviors induced by reward loss. Lesions of the ACC eliminated the effects of reward devaluation in an instrumental task (Gurowitz et al., 1970), retarded recovery from a consummatory reward devaluation manipulation (Ortega et al., 2011b), and resulted in impaired appetitive extinction from acquisition in a Pavlovian task (Griffin and Berry, 2004). A role of the ACC on reward loss is further suggested by reduced pCREB expression and c-Fos activation of this area following consummatory reward devaluation (Glueck et al., 2015; Pecoraro and Dallman, 2005). pCREB (phosphorylated cyclic adenosine monophosphate responsive element-binding protein) is a marker of synaptic plasticity, whereas cFos (a gene-transcription factor) is a marker of neuronal activity. This evidence is particularly relevant in this context, given the significant role played by CREB in the development of drug addiction (Nestler, 2004). 3.2. Orbitofrontal cortex (OFC) The OFC is also related to deficits in the inhibitory/executive control of drug seeking behavior, the processing of stimuli with negative emotional value, and the assignment of relative motivational value to stimuli predicting drug availability (Volkow and Baler, 2014). In particular, it has been proposed that its brain location and circuitry makes the OFC a key candidate for the integration of information about the value of future outcomes in choice situations (Schoenbaum et al., 2006). According to this view, abnormal functioning of the OFC has been related to loss of control and compulsive drug taking (Volkow and Fowler, 2000). Moreover, a role of the OFC on the interaction between drug addiction and negative emotion is suggested by the finding that its inactivation attenuates the reinstatement of cocaine self-administration induced by exposure to foot shocks, but not by priming injections of cocaine (Capriles et al., 2003). Two studies suggest a role of the OFC on reward loss. First, Pecoraro and Dallman (2005) reported increased c-Fos activation in the OFC of rats exposed to reward downshift test. Second, Ortega et al. (2013a) reported that OFC lesions reduced the behavioral suppression observed during the initial response to reward devaluation (without affecting the behavior of unshifted controls) and also eliminated the enhancement of lever pressing observed during partial reinforcement acquisition training in autoshaping (relative to continuous reinforcement training). These results were interpreted as a lesion-induced deficit in a nonassociative factor (e.g., drive) tied to reward loss. A role of the OFC in reward omission has also been suggested by recent work showing impaired appetitive extinction after functional inactivation of the OFC in rats (Panayi and Killcross, 2014), and on the observation of complex patterns of brain activation and deactivation during instrumental extinction in humans (Finger et al., 2008). 3.3. Medial prefrontal cortex (mPFC) Several possible disruptions on psychobiological processes have been associated with mPFC function under addiction states. For instance, Goldstein and Volkow (2011) suggested that the mPFC is involved in disrupted delay gratification, enhanced stress reactivity, enhanced drug-stimuli seeking, and decreased motivation for other 44 L.A. Ortega et al. / Pharmacology, Biochemistry and Behavior 154 (2017) 39–52 stimuli, all common impairments in SUDs. Interestingly, Baratta et al. (2015) found that the optogenetic inhibition of mPFC neurons in the infralimbic area mediated the effect of escapable shocks on the extinction of cocaine seeking behavior, compared to animals that experienced inescapable shocks. Additionally, studies indicate that stress hormones target mPFC neurons to produce the neuroendocrine and behavioral changes associated with drug dependence (Lu and Richardson, 2014). Evidence showing the involvement of the mPFC in reward devaluation is conflicting. Pecoraro et al. (2008) reported that mPFC lesions accelerated behavioral recovery after reward downshift; however, unshifted controls were absent in their study. By contrast, Ortega et al. (2013a) found no detectable effects of mPFC lesions on consummatory suppression induced by reward devaluation in an experiment that included unshifted controls. Molecular studies indicated enhanced pCREB expression in the prelimbic section of the mPFC (Glueck et al., 2015) and mPFC c-Fos activation following reward devaluation (Pecoraro and Dallman, 2005). With respect to reward omission, facilitation of extinction from an appetitive Pavlovian task has been reported after inactivating the prelimbic section of the mPFC (Mendoza et al., 2015). Moreover, a set of studies showed that lesions of the prelimbic mPFC enhanced reinstatement and renewal of extinguished responding in an appetitive Pavlovian task (Rhodes and Killcross, 2004, 2007). taking (Yager et al., 2015). For example, the shell division of the NAc plays a key role in mediating the acute reinforcing effects of abuse drugs, whereas the influence of drug-associated conditioned stimuli on drug-seeking behavior seems to depend mainly on the core division of the NAc and on the dorsal striatum (Everitt and Robbins, 2013). Striatal function is also related to reward loss, but in a complex way that requires more research to be fully understood. Increased pCREB expression in the dorsomedial striatum and c-Fos activation in the NAc were reported following reward devaluation (Glueck et al., 2015; Pecoraro and Dallman, 2005). There is also evidence of decreased dopamine efflux in the NAc of during reward devaluation (Genn et al., 2004a) and reward omission (Biesdorf et al., 2015). In addition, NAc lesions affected instrumental reward devaluation (Leszczuk and Flaherty, 2000) and disrupted enhanced responding after a surprising reward omission in rats (Judice-Daher and Bueno, 2013), but had no detectable effect on reward devaluation in the consummatory situation (Eagle et al., 1999; Leszczuk and Flaherty, 2000). Finally, lesions on the dorsomedial striatum did not affect consummatory reward devaluation or omission; however, such lesions eliminated the behavioral activation typical of partial reinforcement during acquisition of lever pressing in the autoshaping situation (Torres et al., 2016). The authors concluded that output from dorsomedial striatum neurons is important for enhanced motivation in reward-uncertain conditions. 3.4. Insular cortex (IC) 3.6. Lateral habenula (LHb) Some psychobiological models have incorporated IC function in the circuitry underlying SUDs. A key function of the IC is the representation of interoceptive states of the body (Craig, 2002). For example, Paulus and Stein (2006) reported that individuals with a higher tendency toward anxiety also show increased IC activation when processing salient stimuli. This is consistent with the proposal that the IC is a key brain area modulating the disrupting nature of interoceptive cues on choice processes involved in drug use and relapse (Cisler et al., 2013; Naqvi and Bechara, 2009, 2010; Volkow and Baler, 2014). Consistent with this view, smokers with insular damage were able to stop smoking without experiencing craving or relapse, unlike smokers with damage in other brain regions (Naqvi et al., 2007). Similarly, inactivation of the granular IC disrupted the reinstatement of nicotine self-administration after extinction (Forget et al., 2010). Several sources of evidence also suggest a role for the IC in reward loss. Behavioral suppression induced by reward devaluation is eliminated by IC lesions, which do not affect unshifted controls (Lin et al., 2009). This result that can be interpreted as a failure for the reward devaluation procedure to induce an emotional response in IC lesioned animals. Consistent with this finding, Pecoraro and Dallman (2005) reported IC c-Fos activation following reward devaluation. Extensive evidence suggests that the mesocorticolimbic brain system is crucial for the processing of rewards, including the encoding of reward-prediction errors characterizing reward loss. In this vein, dopamine neurons projecting from the VTA to the NAc are activated by unexpected rewards or reward-predicting stimuli, and inhibited by the omission of expected rewards (Ungless et al., 2010). This inhibition seems to depend on inputs coming from the LHb, as several electrophysiological studies suggest (e.g., Kimura et al., 2007). Furthermore, the LHb has been proposed as a detector of “worse-than-expected outcomes,” based on its key role in appetitive extinction (Friedman et al., 2010, 2011) and in decision-making under reward uncertainty conditions (Stopper and Floresco, 2014). Additional studies suggest that the LHb also regulates reactivity to inescapable shocks (Shumake and Gonzalez-Lima, 2003), anxiety responses (Pobbe and Zangrossi, 2008), depressive states (Sartorius et al., 2010), and, most importantly in the present context, addictive behavior, including drug seeking and taking, relapse, reinstatement, and extinction (Friedman et al., 2010; Haack et al., 2014; Lecca et al., 2014; Maroteaux and Mameli, 2012; Velasquez et al., 2014). 3.5. Striatum One of the key functions of the amygdala is the establishment of learned associations between events with motivational value, so that previously neutral stimuli can become signals (Everitt et al., 2003). This function has been suggested to be connected to the dysregulation of reward and stress responses underlying compulsive patterns of drug seeking and taking (Egli et al., 2012; Koob and Le Moal, 2001; Volkow and Baler, 2014). The amygdala has also been linked to reward loss. Direct evidence is suggested by the reduction and elimination of consummatory reward devaluation effects after lesions of the centromedial and lateral amygdala, respectively (Becker et al., 1984). These lesions did not affect unshifted controls, but they were rather large and not very uniform in location. Still, the results were consistent with subsequent research based on more precise methods. For example, local infusion of the benzodiazepine diazepam into the central amygdala also attenuated consummatory reward devaluation (Liao and Chuang, 2003). Furthermore, c-Fos expression was enhanced in the lateral amygdala following the first reward devaluation event (Pecoraro and Dallman, Striatal function has been widely recognized as critical for reward processing and addiction (Everitt and Robbins, 2013), although the precise way in which striatal subcomponents contribute to addictive behavior remains elusive because of the complexity of this circuit. Briefly, the striatum can be divided into two main regions: the dorsal striatum (including the dorsomedial and the dorsolateral divisions), and the ventral striatum or nucleus accumbens (NAc). As the major integration site of the circuit involving also areas of the prefrontal cortex and thalamic nuclei, the striatum receives inputs from brainstem sources such as the ventral tegmental area (VTA) and the substantia nigra, as well as from the hippocampus, amygdala, and raphe nuclei, among others. The outputs project mainly to brainstem nuclei and to the thalamus, which, in turn, projects back mainly to the prefrontal cortex (Lanciego et al., 2012). Extensive evidence suggests an involvement of the striatum in the acute and chronic effects of abuse drugs, as well as in the transition from initial drug use to compulsive drug seeking and 3.7. Amygdala L.A. Ortega et al. / Pharmacology, Biochemistry and Behavior 154 (2017) 39–52 2005) and pCREB expression was increased after the second devaluation event relative to the first one when basolateral and central nuclei were analyzed together (Glueck et al., 2015). Consistent with these findings, reversible inactivation of the centromedial amygdala during the first reward downshift event reduced consummatory reward devaluation (Kawasaki et al., 2015). Importantly, this inactivation procedure did not affect anticipatory negative contrast, a procedure involving the same rewards, but arranged in a Pavlovian-like fashion that has proven insensitive to anxiolytics (Flaherty, 1996). In anticipatory negative contrast, licking for 4% sucrose is lower if this is followed immediately by access to 32% sucrose, rather than followed by access to 4% sucrose. Furthermore, in the same experiment, centromedial amygdala inactivation also increased locomotion in the central area of a well-lit open field, an effect suggesting reduced innate fear of an open space. Based on this pattern of results, Kawasaki et al. (2015) suggested that output from the centromedial amygdala tags the event as involving negative emotion. A follow up experiment involving excitotoxic lesions of the basolateral amygdala produced a different pattern of results (Kawasaki et al., submitted). In this case, basolateral lesions also eliminated reward devaluation, but in both the consummatory successive negative contrast and in the anticipatory negative contrast situations. The lesion also impaired appetitive extinction in autoshaping, but it spared open field activity. The authors suggested that the basolateral amygdala is involved in reward comparisons, that is, the contrast between current and expected rewards. Basolateral lesions affected behavior in tasks involving transitions in reward magnitude—reward devaluation and omission—but had no effect in a task known to induce negative emotion, but unrelated to reward comparisons—the open field task. This picture is complicated by experiments with analogous amygdala manipulations, but involving reward devaluation or omission in instrumental conditioning situations. For example, the effects of instrumental reward devaluations were enhanced by lesions of the central nucleus, but reduced by lesions of the basolateral nucleus (Salinas et al., 1996). As reviewed above, both lesions eliminated the effects of reward devaluation on consummatory behavior (Kawasaki et al., 2015, submitted). Regarding reward omission, basolateral amygdala lesions or inactivation increased resistance to extinction from conditioning tasks using appetitive reinforcers (Burns et al., 1999; McLaughlin and Floresco, 2007), whereas they had reduced resistance to extinction after appetitive Pavlovian training (Kawasaki et al., submitted). The enhancing effect of surprising reward omissions on instrumental lever pressing was also impaired by large amygdala lesions that included the central and basolateral nuclei (Tavares et al., 2014). 4. Emergent topics 4.1. Circuitry for reward devaluation and implications for SUDs The preceding review makes it clear that the psychobiology of reward loss intersects with drug addiction in terms of neurotransmitters, neuromodulators, and neurocircuitry. Some of the most salient parallels involve: (1) the effects of drugs with addictive potential on reward loss (e.g., opioids, GABAergic anxiolytics, dopaminergic drugs); (2) the role of NAc neurons as reward value markers in both reward loss and drug seeking and taking, and its regulation by input from the LHb via the VTA; (3) the functions of the amygdala in reward comparisons (basolateral nucleus) and in tagging reward loss and drug withdrawal with aversive value (central nucleus); (4) the possible antagonistic functions of the ACC and IC in reward expectancy, affecting reward devaluation and extending to drug anticipation triggered by exteroceptive or interoceptive cues; (5) the disruption of top-down cognitive control in drug addiction via dysfunction in various areas of the prefrontal cortex that also affect reward loss; and (6) the role of the dorsal striatum as a possible output component connected to reward uncertainty and habitual drug seeking and taking. 45 This intersection between reward loss and drug addiction brings to the table “the usual suspects” in terms of neurotransmitters and brain areas. It is still useful to consider the brain circuitry underlying one instance of reward loss as a way of drawing attention to potential functions that may otherwise be ignored. There is little surprise in the common elements between reward loss and drug addiction, including reward expectancies and the negative emotion that ensue when such expectancies are violated. But reward loss can only have an impact on behavior when devaluations or omissions occur in the context of a reactivated reward memory. This function, called here reward comparison, is not typically invoked to understand the neurobiology of addiction (but see Grigson, 2008) or of fear conditioning, the more typical model for negative emotion (e.g., Tovote et al., 2015). The effects of reward loss on behavior are complex and to some extent depend upon the components of the situation (e.g., whether instrumental or consummatory). We believe that at this point in the development of research, there is enough information about the effects of neural manipulations on consummatory successive negative contrast (see Table 1) that it would be worth organizing the information in terms of a neural circuit. Fig. 1 presents this circuit as a testable hypothesis that contains some key elements, but it is extremely simplified in that only a fraction of the known pathways is included. Many of the elements of this circuit are familiar components of the reward system activated and also modified by the consumption of drugs in individuals suffering from an SUD. A major assumption is that the origin and maintenance of addictive behaviors relies, at least to some extent, on the emotional modulation afforded by the actions of drugs such as opioids, benzodiazepine anxiolytics, alcohol, and cocaine, among others, on this circuitry. The first level in the hypothesized circuit involves the taste-licking modal action pattern organized at the brainstem level and responsible for substance detection and consumption in rats (Jones et al., 2006; Travers et al., 1997). This circuit alone can organize licking responses to sucrose (Flynn and Grill, 1988), but it requires telencephalic influence for modulation involving taste memory and expectancies. The parabrachial nucleus (PbN) is one of the components of the taste-licking modal action pattern that bridges the two levels by connecting with the amygdala. Bilateral electrolytic lesions of the PbN have been shown to eliminate reward devaluation in the consummatory situation (Grigson et al., 1994). The second level involves circuits that receive and process taste inputs, and produce an output that modulates licking. Two functions necessary for a significant response to reward devaluation are distinguished: reward comparison and negative emotion. The proposed reward comparison mechanism centers on the BLA (Kawasaki et al., submitted), a nucleus that receives information about the current reward (4% sucrose) via the PbN-thalamus pathway and also from the reactivated memory of the expected reward (32% sucrose) via the PbN-thalamus-IC pathway. Electrophysiological recordings show that BLA neurons respond to taste stimuli (Fontanini et al., 2009), probably by receiving inputs from the PbN and/or gustatory thalamus (Bielavska and Roldan, 1996). Anatomical connections between the IC and the BLA have been widely recognized (e.g., Yamamoto et al., 1984). Thus, it is hypothesized here that lesions of the PbN (Grigson et al., 1994), gustatory thalamus (Reilly and Trifunovic, 2003), and IC (Lin et al., 2009) eliminate the effects of consummatory devaluations by disrupting the reward comparison mechanism (see Reilly and Trifunovic, 2003). Without the comparison, consummatory behavior simply adjusts to the current reward level (absolute reward value) without producing a contrast effect (relative reward value). The task of the BLA comparator is hypothesized to be to produce an output that carries information about the significance of the current-vs.-expected discrepancy. Significant discrepancies would activate the negative emotion circuit. In this circuit, output from the BLA comparator activates the CeA directly or via intercalated cell masses (Paré et al., 2004) and the CeA provides the main source of behavioral suppression during reward devaluation 46 L.A. Ortega et al. / Pharmacology, Biochemistry and Behavior 154 (2017) 39–52 Fig. 1. Common elements of the neural circuitry underlying reward devaluation in consummatory successive negative contrast. This figure is intended to highlight the major components and it is not meant to provide an exhaustive outline of the circuit. Two levels are distinguished. The input/output level can be described as the taste/licking modal action pattern. The telencephalic level modulates the functioning of the taste/licking modal action pattern and it includes two major components: reward comparison and negative emotion. Reward comparison is suggested to be the function of the BLA, which receives information about the current reward (e.g., 4% sucrose) from the PbN-thalamus pathway and also retrieves a memory of the expected reward (e.g., 32% sucrose) assumed to be stored in the IC. If the output from the BLA involves a significant negative prediction error (i.e., the current reward is less valued than the expected reward), then the CeA is engaged to inhibit the taste/licking modal action pattern. In turn, activity in the CeA is assumed to be regulated by a loop involving the NAc, dorsal striatum, LHb, and VTA, and also by a BLA-ACC connection with an inhibitory influence on the CeA. The only output included in the figure is provided by the CeA. ACC: anterior cingulate cortex. BLA: basolateral amygdala. CeA: central amygdala. IC: insular cortex. LHb: lateral habenula. NAc: nucleus accumbens. NST: nucleus of the solitary tract. PbN: parabrachial nucleus. PFC: prefrontal cortex. VTA: ventral tegmental area. (Kawasaki et al., 2015). The BLA also sends information to the NAc and the ACC (Gabbott et al., 2005). The BLA-NAc projection may be responsible for the reduction in accumbens dopamine release during the first reward devaluation event (Genn et al., 2004a). The BLA-ACC pathway may be responsible for an inhibitory influence of the ACC on the CeA. Although there is evidence of an ACC projection to the CeA in rats (McDonald, 1998), whether this pathway exerts an inhibitory influence on CeA neurons is not known to the authors. Such an inhibitory action of the ACC on CeA neurons would be consistent with the retardation of recovery from reward devaluation reported after electrolytic ACC lesions (Ortega et al., 2011b) and also with the attenuation of reward devaluation effects after diazepam infusions into the CeA (Liao and Chuang, 2003). We are also hypothesizing that an additional modulatory influence on CeA neurons involves the pathway connecting the NAc with the dorsal striatum and the LHb, which inhibits activity in the VTA. Such inhibition of VTA dopaminergic neurons may influence some unidentified area in the prefrontal cortex responsible for activating CeA output. There is evidence of a connection between the dorsal striatum and the LHb in primates (Saleem et al., 2002), but there is yet no evidence that the LHb provides a key signal for reward devaluation in the consummatory contrast situation, as it seems to do other tasks involving negative prediction error (Kimura et al., 2007; Stopper and Floresco, 2014). Thus, the net output from the CeA may be the result of excitatory afferents from the BLA and prefrontal cortex, and inhibitory afferents from the ACC. The consummatory successive negative contrast effect (see Table 1) would require that, at least initially, activation of the CeA dominates over the inhibitory cortical influences. However, with increased devaluation experience, inhibitory influences may play a key role in the recovery process. Interestingly, ACC lesions do not affect the level of consummatory suppression during the first reward devaluation session, but retard recovery from reward downshift in subsequent sessions (Ortega et al., 2011b). Output from the CeA represented in Fig. 1 is assumed to be a major (though surely not the only) influence ultimately inhibiting licking responses during consummatory reward devaluation. 4.2. Genetic vulnerability to reward loss and addiction Potential factors underlying the transition from initial drug use to compulsive drug seeking and taking vary from the genetic to the social (Anthony et al., 1994). Individual differences in aspects related to reward dysregulation are a promising research field to understand the development and probability of relapse in drug addiction (Piazza and Le Moal, 1996). Similarly, several sources confirm the presence of individual differences that have unlocked the genetic correlates of behaviors induced by reward loss. First, rats exhibit distinct profiles of recovery from an experience of reward devaluation. S. Papini et al. (2014) conducted an extensive secondary analysis of such profiles using latent-growth mixture modeling and identified three distinct recovery trajectories. The majority of rats (83% of 262 animals) exhibited the usual response suppression followed by recovery to control levels of consummatory behavior. A second group (6%) showed either little or no evidence of consummatory suppression after reward downshift. However, some animals (11%) exhibited similar suppression to the first subgroup during the initial experience with reward devaluation, but subsequently expressed little evidence of recovery from the suppressive effects of reward downshift. Interestingly, the typical and no recovery profiles did not differ in performance during the initial reaction to the reward devaluation, but in their rate of recovery. These results are consistent with previous studies showing that opioid blockage with naloxone affected animals that exhibited slow recovery from reward devaluation, but not those exhibiting fast recovery, in groups matched by their initial response to the devaluation (Pellegrini et al., 2005). Thus, the initial response to reward downshift does not predict the speed of recovery from that event. Second, individual differences in the initial reaction to and in the recovery from reward devaluation respond to selective breeding protocols (Torres and Sabariego, 2014). For example, Flaherty et al. (1994) selectively bred Sprague-Dawley rats showing large vs. small initial reaction to reward devaluation over seven generations. This selective breeding gave rise to two lines with extreme divergence in reactivity to reward L.A. Ortega et al. / Pharmacology, Biochemistry and Behavior 154 (2017) 39–52 downshift, although, unfortunately, random mating controls were not included in this study. Similarly, Ortega et al. (2014b) selectively bred Long-Evans rats based on differences in recovery rate from reward devaluation (fast vs. slow recovery). They found that the degree of behavioral suppression induced by reward downshift was reduced in the fastrecovery line after five generations, relative to a random breeding control line. By contrast, the slow-recovery line did not differ from random controls. Although the results derived from selective breeding based on reward devaluation reactivity appear promising, whether this selection protocol affects the proneness to take and abuse drugs awaits further investigation. Extensive evidence shows that strains of rats selectively bred for differences in fear, anxiety, and emotional reactivity often also differ in situations involving reward loss. This includes evidence from Maudsley reactive vs. nonreactive rats (Rowan and Flaherty, 1991), initially selected for their degree of defecation in a novel open-field situation; Syracuse high- vs. low-avoidance rats (Flaherty and Rowan, 1989), initially selected for performance in an active avoidance task; Lewis vs. Fischer rats (Brewer et al., 2016; Freet et al., 2006), inbred strains differing in terms of many traits; and spontaneously hypertensive vs. WistarKyoto rats (Bentosela and Mustaca, 2005), two divergent strains derived from the same stock. In all these cases, the more “emotional” strain exhibits increased response to reward downshift, although in no case were they evolved on the basis of this trait, with one exception: the Maudsley reactive rats actually showed smaller reaction to reward loss than the nonreactive rats (Rowan and Flaherty, 1991). Additional tests indicated that these animals were still showing the difference in emotionality in the open field as expected. Thus, the reason for these surprising results remains unknown. Some of these strains also differ in drug intake and other behavioral traits related to vulnerability to addiction. For example, Lewis rats acquire self-administration of a number of abuse drugs, including cocaine and heroin, more rapidly and at higher rates compared with Fischer rats (Picetti et al., 2012). Importantly, this divergent pattern of cocaine selfadministration correlates with differences in gene expression for endogenous neuromodulatory systems, such as opioid peptides and receptors, in specific brain regions, such as the dorsal striatum, linked to reward loss and uncertainty (Torres et al., 2016; Valenza et al., 2016; Wood et al., 2005; Wood et al., 2008). The extensive and systematic characterization of the Roman highavoidance (RHA) and low-avoidance (RLA) strains is of special importance in the present context because they have been studied in terms of both reward loss and drug addiction. Although initially selected for their good (RHA) vs. poor (RLA) two-way active avoidance performance in the 1960s, Roman rats consistently differ in situations involving drug taking (and related behavioral traits) and reward loss. For example, RHA rats show higher proneness to voluntarily self-administration of alcohol in comparison to RLA rats (Manzo et al., 2012). Similarly, only the RHA strain show behavioral sensitization in response to the repeated administration of cocaine, amphetamine, and morphine (Giorgi et al., 2007). These differences may be due to strain-dependent divergence in mesolimbic dopaminergic pathways (Tournier et al., 2013). This picture changes when RHA and RLA rats are subject to reward loss. In this case, RLA rats have been shown to be consistently more reactive than the RHA rats when exposed to reward devaluation situations, including instrumental (appetitive and aversive) and consummatory tasks (Cuenya et al., 2012; Cuenya et al., 2015; Gómez et al., 2009b; Rosas et al., 2007; Sabariego et al., 2013; Torres et al., 2005). For example, downshifts from 12 to 2 pellets in the runway (Rosas et al., 2007), from 22 to 4% sucrose in the consummatory situation (Gómez et al., 2009b), and from 30 s to 1 s in safety in one-way avoidance contrast (Torres et al., 2005) have a stronger effect on behavior in RLA rats than in RHA rats. RLA rats also show faster extinction after continuous reinforcement than after partial reinforcement, an effect that is absent in RHA rats, which tend to exhibit slow extinction of instrumental behavior (Gómez et al., 2009a; Gómez et al., 2008). Finally, 47 the most relevant evidence supporting a connection between addiction and reward loss derives from studies showing that only the more reactive RLA strain increases ethanol consumption in response to reward omission (Manzo et al., 2014). In terms of the application of reward loss research to the study of the genetic basis of drug addiction, two interesting results are worth highlighting. First, individual differences in recovery from reward devaluation are associated to differential sensitivity of opioid and GABAergic receptors (Freet et al., 2006; Ortega et al., 2014b; Pellegrini et al., 2005), suggesting a common neurochemical background underlying addictive behavior and the response to reward loss. Second, microarray research with the Roman strains suggests that individual differences in response to reward devaluation are related to differences in brain expression of genes related to drug abuse and emotion, including TAAR2, PRL, CAMKK2, CRHBP, and HOMER3, among others (Sabariego et al., 2011, 2013). These results make these genes targets for future research on the interplay between reward loss and addiction. One implication derived from the studies reviewed above, mostly centered on selectively bred strains, is that the same genes control the expression of mechanisms underlying reward loss and addiction (pleiotropic gene effects). This conclusion, however, must be taken with caution for several reasons. For example, strains selected for their response in reward loss situations were not studied in terms of addictive behavior (Flaherty et al., 1994; Ortega et al., 2014a, 2014b); vice versa, as far as the authors know, none of the strains selected on the basis of addictive behavior have been tested in reward loss situations. The Roman strains, which are well characterized in terms of both reward loss and addiction, were originally selected for their response in active avoidance learning and have subsequently been subjected to extensive inbreeding, which poses problems of its own. Inbreeding (i.e., pairing of male and female siblings) may lead to the random fixation of alleles other than those initially affected by selective breeding. Thus, an apparent correlation of phenotypes may be the result of different genes, rather than the result of the pleiotropic effects of the same genes (Crabbe et al., 1990). Support for a common genetic basis between two phenotypes would require comparisons among several inbred strains. These concerns would be minimized, although not completely eliminated, in comparisons between outbred strains specifically selected for vulnerability to reward loss or addiction. In outbred strains (i.e., pairings of nonsibling males and females), behavioral divergence is assumed to be the result of pleiotropic gene effects, with the genetic background remaining stable. However, such an assumption requires relatively large animal samples to minimize random gene fixations, a requisite rarely fulfilled in selective breeding studies with mammals. 4.3. Role of competing natural rewards An additional application of reward loss to drug addiction research was suggested by Grigson (1997, 2008). According to her view, a key component of drug addiction is the devaluation of natural incentives. Individuals suffering from an SUD tend to care less for their health, their job, and even their children. To model this effect, animals are exposed to natural rewards (e.g., sweet substances like saccharin) followed by drug administration. For instance, Grigson and Hajnal (2007) gave rats a single saccharin-morphine Pavlovian pairing and later tested the response to the saccharin alone. They found reduced saccharin intake and a blunting of dopamine levels in the NAc, as well as reduced NAc dopamine levels in response to morphine. Such suppressive effects on the consumption of various gustatory stimuli were found after pairings with a variety of drugs with addictive potential, including cocaine, amphetamine, ethanol, benzodiazepine anxiolytics, nicotine, and opioids (Grigson, 2008). There also individual differences in the suppressive effects of these pairings. For example, after saccharine-heroin pairings, some animals exhibited a pronounced reduction in saccharin intake, whereas other showed a milder effect; “high suppressors” also showed several additional behaviors related to addiction compared to “low 48 L.A. Ortega et al. / Pharmacology, Biochemistry and Behavior 154 (2017) 39–52 suppressors,” including reinstatement of heroin seeking after exposure to mild shock and a higher braking point in a progressive ratio test (Jenney et al., 2016). These results suggest that the incentive value of a natural reward (e.g., saccharin's sweetness) can be reduced by a single paring with morphine or heroin. The suppression of saccharin intake after saccharine-drug pairings is reminiscent of the anticipatory negative contrast effect described above. In this case, for example, intake of a weak saccharin solution is suppressed when it signals access to 32% sucrose, relative to a saccharin-saccharin pairing. Thus, consumption of a less sweet current reward is reduced by an expectation of a sweater reward (Flaherty, 1996). However, the availability of some rewards can also reduce drug intake. For example, access to physical exercise reduces cocaine self-administration (Smith et al., 2016) and housing in an enriched environment reduces nicotine self-administration (Gomez et al., 2015). These data show that, as in reward loss, comparison among rewards and incentive relativity play a key role in the onset, persistence, and relapse of addictive behavior. 4.4. Reward loss and emotional self-medication The ability of abuse drugs to diminish or relieve aversive emotional states induced by psychopathological conditions and drug withdrawal has been proposed as a precipitant factor for addiction. We will refer to this approach as the emotional self-medication (ESM) hypothesis, a hypothesis initially suggested by clinical evidence (Khantzian, 1985, 2013) and lately attracting experimental attention (Torres and Papini, 2016). Despite wide evidence showing that animals also consume drugs to alleviate a variety of negative emotional states, including drug withdrawal (Koob, 2013), most of these studies lack appropriate controls to test the self-medication hypothesis. Such lacking controls include: (1) Establishing that emotional activation is actually present during the episode of substance consumption; (2) Demonstrating that increased consumption is selective for substances that reduce negative emotions; (3) Showing that increased substance consumption is restricted to periods of increased emotional activation; and (4) Providing independent evidence that the consumed substance effectively reduces emotional activation (Manzo et al., 2015a, 2015b; Torres and Papini, 2016). In addition to these control issues, there is yet insufficient data exploring the breath of situations that induce ESM and the spectrum of substances that support ESM. Several studies suggest that drug taking following reward loss can be considered a type of ESM. An initial study reported that reward devaluation induced approach to stimuli that were previously associated with systemic ethanol administration (Kamenetzky et al., 2009a). However, ESM requires direct consumption, rather than simply approaching a signal previously paired with the substance. Several experiments have used a procedure involving an induction task based on a reward loss paradigm (see Table 1) followed each day by a two-bottle preference test in which anxiolytics were diluted in water to simulate typical aspects of human drug taking, which usually involves voluntary and oral consumption. Manzo et al. (2015a)observed that preference for ethanol and the anxiolytic chlordiazepoxide was enhanced subsequently to a reward devaluation experience, relative to unshifted controls and to groups given access only to water. In the same line, Manzo et al. (2014) reported that preference for ethanol following reward omission was higher in the more emotionally reactive RLA-I rats than in the less reactive RHA-I rats. Additional research on reward loss and emotional self-medication reveals an interesting interaction between genetic and experiential factors based on a comparison of partial reinforcement vs. continuous reinforcement on drug intake. Partial reinforcement has been proposed as an effective treatment for developing resistance/resilience to reward loss, given its attenuating effects over the behavioral consequences of reward devaluation and omission (Pellegrini et al., 2004). Following this rationale, Manzo et al. (2015b) conducted an experiment in which emotionally reactive RLA-I rats were exposed to instrumental extinction sessions after partial vs. continuous reinforcement acquisition training. Each session was followed by free access to ethanol vs. water in the home cage. The results indicated that rats exposed to partial reinforcement showed lower preference for ethanol consumption after reward omission than those exposed to continuous reinforcement. Thus, exposure to reward uncertainty via partial reinforcement training counteracted a genetic predisposition for emotional self-medication after decades of selective breeding. This result can open up future research on behavioral interventions that could effectively protect against drug abuse and misuse (Manzo et al., 2015b). 5. Concluding remarks Current research trends on the psychobiology of addiction have focused on a wide set of behavioral processes and related experimental procedures. However, in comparative terms, there has been a lack of attention to the role of reward loss in drug addiction. The present review makes it clear that there are extensive behavioral, neurochemical, and neurobiological parallels between the response to reward loss and the development and maintenance of drug addiction. Such parallels suggest avenues for cross-pollination between two lines of research that have proceeded relatively independently. An understanding of the psychobiology of drug addiction can be enhanced by incorporating existing findings and animal models from reward loss research. Furthermore, reward loss research suggests new avenues of scientific inquiry that are likely to shed light in our understanding of the transition from initial drug use to a compulsive set of drug seeking and taking habits. This is suggested by the reviewed research on the interaction between genetic vulnerability and reward history, the competing effects of natural rewards for the development and maintenance of drug seeking and taking, and the growing awareness of the role of emotional selfmedication in drug addiction. One area in which research on reward loss and addiction would seem to conflict with each other is in the role of habits. Habits are usually characterized as sequential behavior that occurs independently of outcome expectations and seem to be performed automatically. Under some conditions (e.g., extended training), the posttraining devaluation of an outcome such as food may not disrupt instrumental responding established by that outcome (Adams, 1982). Such habitual behavior is an important component of well-established addictive behaviors (e.g., Everitt and Robbins, 2016), but, of course, it rounds counter to reward-loss effects, which critically depend on the violation of outcome expectancies (e.g., Papini et al., 2015). Consistent with this view, a devaluation protocol consisting of pairing self-administration infusions of cocaine with LiCl (a toxin leading to gastrointestinal sickness) reduced responding established by cocaine reinforcement when devaluation occurred after short training sessions, but not when devaluation occurred after extended sessions of cocaine training (Leong et al., 2016). Although the transition from outcome-driven to outcome-independent behavior is usually presented as a sequential process dependent, for example, upon the amount of training (Packard and McGaugh, 1996), the process might be more complex than this view suggests. Habitual responding does not imply that outcome expectancies are no longer available, but may instead reflect the structure of the situation. Unpublished research suggests that animals that behave habitually when facing a single response option may show outcomedriven performance if confronted with a choice in which one option has been devalued from a large to a small reward, whereas the other option has been consistently paired with the small reward (Conrad and Papini, 2016). Under such conditions, rats show about the same responding to both options when each of them is presented in isolation, but shift away from the devalued option when given a simultaneous choice. Thus, the choice scenario may force outcome expectancies to retake control of behavior even after extended training. Whether a similar approach could be useful in the treatment of well-established SUDs remains to be explored. L.A. Ortega et al. / Pharmacology, Biochemistry and Behavior 154 (2017) 39–52 Drug addiction is one of the most complex and costly social problems, so the tools, both conceptual and empirical, derived from the study of reward loss can help the amelioration of human suffering. Because of significant parallels in the overlap of learning and motivational processes, reward devaluation and omission phenomena can provide significant inroads in the prevention and treatment of SUDs. Conflict of interest None. Acknowledgements This work was supported by Fundación Universitaria Konrad Lorenz, Colombia [to LAO and JLS]; Ministerio de Economía y Competitividad, Spain [grant number PSI-2013-44945-P; to CT]; TCU/IS, USA [grant number 66022, to MRP]. Correspondence concerning this article may be addressed to L. A. Ortega, Departmento de Psicología, Fundación Universitaria Konrad Lorenz, Bogotá, Colombia. (leonardoa.ortegam@ konradlorenz.edu.co). References Adams, C.D., 1982. Variations in the sensitivity of instrumental responding to reinforcer devaluation. Q. J. Exp. Psychol. 34B, 77–98. Ahmed, S.H., 2012. The science of making drug-addicted animals. Neuroscience 211, 107–125. Ahmed, S.H., Koob, G.F., 1997. Cocaine-but not food-seeking behavior is reinstated by stress after extinction. Psychopharmacology 132, 289–295. Alexander, B.K., Beyerstein, B.L., Hadaway, P.F., Coambs, R.B., 1981. Effect of early and later colony housing on oral ingestion of morphine in rats. Pharmacol. Biochem. Behav. 15, 571–576. American Psychiatric Association, 2013. DSM-V. Diagnostic and Statistical Manual of Mental Disorders. fifth ed. (Washington, DC). Amsel, A., 1992. Frustration Theory: An Analysis of Dispositional Learning and Memory. Cambridge University Press, Cambridge, UK. Anisman, H., Waller, T.G., 1974. Effects of inescapable shock and shock-produced conflict on self-selection of alcohol in rats. Pharmacol. Biochem. Behav. 2, 27–33. Anthony, J.C., Warner, L.A., Kessler, R.C., 1994. Comparative epidemiology of dependence on tobacco, alcohol, controlled substances, and inhalants: basic findings from the national comorbidity survey. Exp. Clin. Psychopharmacol. 2, 244–268. Banks, A., Russell, R.W., 1967. Effects of chronic reduction in acetylcholinesterase activity on serial problem solving behavior. J. Comp. Physiol. Psychol. 64, 262–267. Baratta, M.V., Pomrenze, M.B., Nakamura, S., Dolzani, S.D., Cooper, D.C., 2015. Control over stress accelerates extinction of drug seeking via prefrontal cortical activation. Neurobiology of Stress 2, 20–27. Barker, A.T., Rebec, G.V., 2016. Cocaine withdrawal alters the reward omission effect and enhances traits of negative urgency in rats across multiple days of testing. Drug Alcohol Depend. 163, S19–S24. Barr, A.M., Phillips, A.G., 2002. Increased successive negative contrast in rats withdrawn from an escalating-dose schedule of D-amphetamine. Pharmacol. Biochem. Behav. 71, 293–299. Barret, S.T., Bevins, R.A., 2013. Nicotine enhances operant responding for qualitatively distinct reinforcers under maintenance and extinction conditions. Pharmacol. Biochem. Behav. 114–115, 9–15. Beck, C.H., Loh, E.A., 1990. Reduced behavioral variability in extinction: effects of chronic treatment with the benzodiazepine, diazepam or with ethanol. Psychopharmacology 100, 323–327. Becker, H.C., Flaherty, C.F., 1982. Influence of ethanol on contrast in consummatory behavior. Psychopharmacology 77, 253–258. Becker, H.C., Flaherty, C.F., 1983. Chlordiazepoxide and ethanol additively reduce gustatory negative contrast. Psychopharmacology 80, 35–37. Becker, H.C., Jarvis, M.F., Wagner, G.C., Flaherty, C.F., 1984. Medial and lateral amygdalectomy differentially influences consummatory negative contrast. Physiol. Behav. 33, 707–712. Bentosela, M., Mustaca, A.E., 2005. Efectos del contraste sucesivo negativo consumatorio en ratas hipertensas: Una cuestión de memoria. Suma Psicológica 12, 87–100. Bentosela, M., Ruetti, E., Muzio, R.N., Mustaca, A.E., Papini, M.R., 2006. Administration of corticosterone after the first downshift trial enhances consummatory successive negative contrast. Behav. Neurosci. 120, 371–376. Bialik, R.J., Pappas, B.A., Pusztay, W., 1982. Chlordiazepoxide-induced released responding in extinction and punishment-conflict procedures is not altered by neonatal forebrain norepinephrine depletion. Pharmacol. Biochem. Behav. 16, 279–283. Bielavska, E., Roldan, G., 1996. Ipsilateral connections between the gustatory cortex, amygdala, and parabrachial nucleus are necessary for acquisition and retrieval of conditioned taste aversion in rats. Behav. Brain Res. 81, 25–31. Biesdorf, C., Wang, A.-L., Topic, B., Petri, D., Milani, H., Huston, J.P., de Souza Silva, M.A., 2015. Dopamine in the nucleus accumbens core, but not shell, increases during 49 signaled food reward and decreases during delayed extinction. Neurobiol. Learn. Mem. 123, 125–139. Boshka, S.C., Weisman, H.M., Thor, D.H., 1966. A technique for inducing aggression in rat utilizing morphine withdrawal. Psychol. Rec. 16, 541–543. Brady, K.T., Sinha, R., 2005. Co-occurring mental and substance use disorders: the neurobiological effects of chronic stress. Am. J. Psychiatr. 162, 1483–1493. Brevers, D., Bechara, A., Hermoye, L., Divano, L., Kornreich, C., Verbanck, P., Noël, X., 2015. Comfort for uncertainty in pathological gamblers: an fMRI study. Behav. Brain Res. 278, 262–270. Brewer, A., Johnson, P., Stein, J., Schlund, M., Williams, D.C., 2016. Aversive properties of negative incentive shifts in Fischer 344 and Lewis rats. Behav. Brain Res. 319, 174–180. Burns, L.H., Everitt, B.J., Robbins, T.W., 1999. Effects of excitotoxic lesions of the basolateral amygdala on conditional discrimination learning with primary and conditioned reinforcement. Behav. Brain Res. 100, 123–133. Caldwell, E.E., Riccio, D.C., 2010. Alcohol self-administration in rats: modulation by temporal parameters related to repeated mild social defeat stress. Alcohol 44, 265–274. Capriles, N., Rodaros, D., Sorge, R.E., Stewart, J., 2003. A role for the prefrontal cortex in stress-and cocaine-induced reinstatement of cocaine seeking in rats. Psychopharmacology 168, 66–74. Center for Behavioral Health Statistics and Quality, 2015. Behavioral Health Trends in the United States: Results From the 2014 National Survey on Drug Use and Health (HHS Publication No. SMA 15-4927, NSDUH Series H-50). Cisler, J.M., Elton, A., Kennedy, A.P., Young, J., Smitherman, S., James, G.A., Kilts, C.D., 2013. Altered functional connectivity of the insular cortex across prefrontal networks in cocaine addiction. Psychiatry Res. Neuroimaging 213, 39–46. Coe, C.L., Stanton, M.E., Levine, S., 1983. Adrenal responses to reinforcement and extinction: role of expectancy versus instrumental responding. Behav. Neurosci. 97, 654–657. Conrad, S.E., Papini, M.R., 2016. Action and Habit in a Reward Devaluation/paradigm. Society for Neuroscience, San Diego, CA, November. Coover, G.D., Goldman, L., Levine, S., 1971. Plasma corticosterone increases produced by extinction of operant behavior in rats. Physiol. Behav. 6, 261–263. Crabbe, J.C., Phillips, T.J., Kosobud, A., Belknap, J.K., 1990. Estimation of genetic correlation: interpretation of experiments using selectively bred and inbred animals. Alcohol. Clin. Exp. Res. 14, 141–151. Craig, A.D., 2002. How do you feel? Interoception: the sense of the physiological condition of the body. Nat. Rev. Neurosci. 3, 655–666. Cuenya, L., Sabariego, M., Donaire, R., Fernández-Teruel, A., Tobeña, A., Gómez, M.J., Mustaca, A.E., Torres, C., 2012. The effect of partial reinforcement on instrumental successive negative contrast in inbred Roman High- (RHA-I) and Low- (RLA-I) Avoidance rats. Physiol. Behav. 105, 1112–1116. Cuenya, L., Kamenetzky, G., Mustaca, A.E., 2014. Early experience and incentive relativity in adulthood. Int. J. Comp. Psychol. 27, 459–473. Cuenya, L., Sabariego, M., Donaire, R., Fernandez-Teruel, A., Torres, C., Papini, M.R., 2015. Transfer across reward devaluation tasks in inbred Roman rat strains. Learn. Motiv. 52, 22–31. Cunningham, C.L., Henderson, C.M., Bormann, N.M., 1998. Extinction of ethanol-induced conditioned place preference and conditioned place aversion: effects of naloxone. Psychopharmacology 139, 62–70. Daniel, A.M., Ortega, L.A., Papini, M.R., 2009. Role of the opioid system in incentive downshift situations. Neurobiol. Learn. Mem. 92, 439–450. Davis, H., Memmott, J., McFadden, L., Levine, S., 1976. Pituitary-adrenal activity under different appetitive extinction procedures. Physiol. Behav. 17, 687–690. Drewnowski, A., Gray, J.A., 1975. Influence of delta9-tetrahydrocannabinol on partial reinforcement effects. Psychopharmacologia 43, 233–237. Eagle, D.M., Humby, T., Howman, M., Reid-Henry, A., Dunnett, S.B., Robbins, T.W., 1999. Differential effects of ventral and regional dorsal striatal lesions on sucrose drinking and positive and negative contrast in rats. Psychobiology 27, 267–276. Edwards, S., Koob, G.F., 2010. Neurobiology of dysregulated motivational systems in drug addiction. Future Neurol. 5, 393–410. Egli, M., Koob, G.F., Edwards, S., 2012. Alcohol dependence as a chronic pain disorder. Neurosci. Biobehav. Rev. 36, 2179–2192. Ersche, K.D., Gillan, C.M., Jones, P.S., Williams, G.B., Ward, L.H., Luijten, M., Wit, S., Sahakian, B., Bullmore, E.T., Robbins, T.W., 2016. Carrots and sticks fail to change behavior in cocaine addiction. Science 352, 1468–1471. Everitt, B.J., Robbins, T.W., 2013. From the ventral to the dorsal striatum: devolving views of their roles in drug addiction. Neurosci. Biobehav. Rev. 37, 1946–1954. Everitt, B.J., Robbins, T.W., 2016. Drug addiction: updating actions to habits to compulsions ten years on. Annu. Rev. Psychol. 67, 23–50. Everitt, B.J., Cardinal, R.N., Parkinson, J.A., Robbins, T.W., 2003. Appetitive behavior: impact of amygdala-dependent mechanisms of emotional learning. Ann. N. Y. Acad. Sci. 985, 233–250. Feldon, J., Gray, J.A., 1981. The partial reinforcement extinction effect after treatment with chlordiazepoxide. Psychopharmacology 73, 269–275. Finger, E.C., Mitchell, D.G., Jones, M., Blair, R.J.R., 2008. Dissociable roles of medial orbitofrontal cortex in human operant extinction learning. NeuroImage 43, 748–755. Fisher, H.E., Brown, L.L., Aron, A., Strong, G., Mashek, D., 2010. Reward, addiction, and emotion regulation systems associated with rejection in love. J. Neurophysiol. 104, 51–60. Flaherty, C.F., 1996. Incentive Relativity. Cambridge University Press, New York. Flaherty, C.F., Checke, S., 1982. Anticipation of incentive gain. Anim. Learn. Behav. 10, 177–182. Flaherty, C.F., Driscoll, C.D., 1980. Amobarbital sodium reduces successive gustatory contrast. Psychopharmacology 69, 161–162. Flaherty, C.F., Rowan, G.A., 1988. Effect of intersolution interval, chlordiazepoxide and amphetamine on anticipatory contrast. Anim. Learn. Behav. 16, 47–52. 50 L.A. Ortega et al. / Pharmacology, Biochemistry and Behavior 154 (2017) 39–52 Flaherty, C.F., Rowan, G.A., 1989. Rats (Rattus norvegicus) selectively bred to differ in avoidance behavior also differ in response to novelty stress, in glycemic conditioning, and in reward contrast. Behav. Neural Biol. 51, 145–164. Flaherty, C.F., Becker, H.C., Driscoll, C., 1982. Conditions under which amobarbital sodium influences contrast in consummatory behavior. Physiol. Psychol. 10, 122–128. Flaherty, C.F., Becker, H.C., Pohorecky, L., 1985. Correlation of corticosterone elevation and negative contrast varies as a function of postshift day. Anim. Learn. Behav. 13, 309–314. Flaherty, C.F., Grigson, P.S., Rowan, G.A., 1986. Chlordiazepoxide and the determinants of contrast. Anim. Learn. Behav. 14, 315–321. Flaherty, C.F., Grigson, P.S., Lind, S., 1990. Chlordiazepoxide and the moderation of the initial response to reward reduction. Q. J. Exp. Psychol. 42B, 87–105. Flaherty, C.F., Krauss, K.L., Rowan, G.A., Grigson, P.S., 1994. Selective breeding for negative contrast in consummatory behavior. J. Exp. Psychol. Anim. Behav. Process. 20, 3–19. Flaherty, C.F., Clark, S., Coppotelli, C., 1996. Lack of tolerance to the contrast-reducing actions of chlordiazepoxide with repeated reward reductions. Physiol. Behav. 60, 645–652. Flynn, F.W., Grill, H.J., 1988. Intraoral intake and taste reactivity responses elicited by sucrose and sodium chloride in chronic decerebrate rats. Behav. Neurosci. 102, 934–941. Fontanini, A., Grossman, S.E., Figueroa, J.A., Katz, D.B., 2009. Distinct subtypes of basolateral amygdala taste neurons reflect palatability and reward. J. Neurosci. 29, 2486–2495. Forget, B., Pushparaj, A., Le Foll, B., 2010. Granular insular cortex inactivation as a novel therapeutic strategy for nicotine addiction. Biol. Psychiatry 68, 265–271. Freet, C.S., Tesche, J.D., Tompers, D.M., Riegel, K.E., Grigson, P.S., 2006. Lewis rats are more sensitive than Fischer rats to successive negative contrast, but less sensitive to the anxiolytic and appetite-stimulating effects of chlordiazepoxide. Pharmacol. Biochem. Behav. 85, 378–384. Friedman, A., Lax, E., Dikshtein, Y., Abraham, L., Flaumenhaft, Y., Sudai, E., Tzion, M.B., AmiAd, L., Rami, Y., Yadid, G., 2010. Electrical stimulation of the lateral habenula produces enduring inhibitory effect on cocaine seeking behavior. Neuropharmacology 59, 452–459. Friedman, A., Lax, E., Dikshtein, Y., Abraham, L., Flaumenhaft, Y., Sudai, E., Tzion, B.M., Yadid, G., 2011. Electrical stimulation of the lateral habénula produces an inhibitory effect on sucrose self-administration. Neuropharmacology 60, 381–387. Fullgrabe, M.W., Vengeliene, V., Spanagel, R., 2007. Influence of age at drinking onset on the alcohol deprivation effect and stress-induced drinking in female rats. Pharmacol. Biochem. Behav. 86, 320–326. Gabbott, P.L.A., Warner, T.A., Jays, P.R.L., Salway, P., Busby, S., 2005. Prefrontal cortex in the rat: projections to subcortical autonomic, motor, and limbic centers. J. Comp. Neurol. 492, 145–177. Gandelman, R., Trowill, J., 1968. Effects of chlordiazepoxide on ESB-reinforcement behavior and subsequent extinction. J. Comp. Physiol. Psychol. 66, 753–755. Gao, S., Suenaga, T., Oki, Y., Yukie, M., Nakahara, D., 2011. Cocaine enhances resistance to extinction of responding for brain-stimulation reward in adult prenatally stressed rats. Behav. Brain Res. 223, 411–416. Genn, R.F., Ahn, S., Phillips, A.G., 2004a. Attenuated dopamine efflux in the rat nucleus accumbens during successive negative contrast. Behav. Neurosci. 118, 869–973. Genn, R.F., Tucci, S., Parikh, S., File, S.E., 2004b. Effects of nicotine and a cannabinoid receptor agonist on negative contrast: distinction between anxiety and disappointment? Psychopharmacology 177, 93–99. Giorgi, O., Piras, G., Corda, M.G., 2007. The psychogenetically selected roman high- and low-avoidance rat lines: a model to study the individual vulnerability to drug addiction. Neurosci. Biobehav. Rev. 31, 148–163. Gipson, C.D., Kalivas, P.W., 2016. Neural basis of drug addiction. In: De Micheli, D., et al. (Eds.), Drug Abuse in Adolescence. Springer International Publishing, New York, pp. 37–56. Glazer, H.I., 1972. Physostigmine and resistance to extinction. Psychopharmacology (Berlin) 26, 387–394. Glueck, A.C., Dennis, T.S., Perrotti, L.I., Torres, C., Papini, M.R., 2015. Brain expression of pCREB in rats exposed to consummatory successive negative contrast. Neurosci. Lett. 587, 93–97. Goeders, N.E., Guerin, G.F., 1994. Non-contingent electric footshock facilitates the acquisition of intravenous cocaine self-administration in rats. Psychopharmacology 114, 63–70. Goldstein, R.Z., Volkow, N.D., 2011. Dysfunction of the prefrontal cortex in addiction: neuroimaging findings and clinical implications. Nat. Rev. Neurosci. 12, 652–669. Gómez, M.J., de la Torre, L., Callejas-Aguilera, J.E., Lerma-Cabrera, J.M., Rosas, J.M., Escarabajal, M.D., Agüero, A., Tobeña, A., Fernández-Teruel, A., Torres, C., 2008. The partial reinforcement extinction effect (PREE) in female Roman high-(RHA-I) and low-avoidance (RLA-I) rats. Behav. Brain Res. 194, 187–192. Gómez, M.J., de la Torre, L., Aguilera, J.E.C., Rosas, J.M., Escarabajal, M.D., Agüero, A., Tobeña, A., Fernández-Teruel, A., Torres, C., 2009a. Differences on extinction of an appetitive instrumental response in female inbred Roman High (RHA-I) and Low Avoidance (RLA-I) rats. Psicológica 30, 181–188. Gómez, M.J., Escarabajal, M.D., de la Torre, L., Tobeña, A., Fernández-Teruel, A., Torres, C., 2009b. Consummatory successive negative and anticipatory contrast effects in inbred Roman rats. Physiol. Behav. 97, 374–380. Gomez, M., Altomare, D., Harrod, S.B., Creek, K.E., Zhu, J., 2015. Environmental enrichment decreases nicotine self-administration and attenuates nicotine-mediated enhancement of orexin-1 receptor gene expression and ERK1/2 activity in rat prefrontal cortex. Drug Alcohol Depend. 146, e123. Gonzalez, S.C., Matsudo, V.K.R., Carlini, E.A., 1971. Effects of marihuana compounds on the fighting behaviour of Siamese fighting fish (Betta splendens). Pharmacology 6, 186–190. Gordon, H.W., 2002. Early environmental stress and biological vulnerability to drug abuse. Psychoneuroendocrinology 27, 115–126. Gray, J.A., 1987. The Psychology of Fear and Stress. University Cambridge Press, Cambridge. Gray, J.A., McNaughton, N., 2000. The Neuropsychology of Anxiety. An Enquiry into the Functions of the Septo-Hippocampal System. Oxford University Press, Oxford, UK. Griffin, A.L., Berry, S.D., 2004. Inactivation of the anterior cingulate cortex impairs extinction of rabbit jaw movement conditioning and prevents extinction-related inhibition of hippocampal activity. Learn. Mem. 11, 604–610. Grigson, P.S., 1997. Conditioned taste aversions and drugs of abuse: a reinterpretation. Behav. Neurosci. 111, 129–136. Grigson, P.S., 2008. Reward comparison: the Achilles' heel and hope for addiction. Drug Discov. Today Dis. Model. 5, 227–233. Grigson, P.S., Hajnal, A., 2007. Once is too much: conditioned changes in accumbens dopamine following a single saccharin-morphine pairing. Behav. Neurosci. 121, 1234–1242. Grigson, P.S., Spector, A.C., Norgren, R., 1994. Lesions of the pontine parabrachial nuclei eliminate successive negative contrast effects in rats. Behav. Neurosci. 108, 714–723. Gurowitz, E.M., Rosen, A.J., Tessel, R.E., 1970. Incentive shift performance in cingulectomized rats. J. Comp. Physiol. Psychol. 70, 476–481. Haack, A.K., Sheth, C., Schwager, A.L., Sinclair, M.S., Tandon, S., Taha, S.S., 2014. Lesions of the lateral habenula increase voluntary ethanol consumption an operant self-administration, block yohimbine-induced reinstatement of ethanol seeking, and attenuate ethanol-induced conditioned taste aversion. PLoS One 9, e92701. Harloe, J.P., Thorpe, A.J., Lichtman, A.H., 2008. Differential endocannabinoid regulation of extinction in appetitive and aversive Barnes maze tasks. Learn. Mem. 15, 806–809. Hassanbeigi, A., Askari, J., Hassanbeigi, D., Pourmovahed, Z., 2013. The relationship between stress and addiction. Procedia. Soc. Behav. Sci. 84, 1333–1340. Hayaki, J., Stein, M.D., Lassor, J.A., Herman, D.S., Anderson, B.J., 2005. Adversity among drug users: relationship to impulsivity. Drug Alcohol Depend. 78, 65–71. Heise, G.A., Laughlin, N., Keller, C., 1970. A behavioural and pharmacological analysis of reinforcement withdrawal. Psychopharmacology 16, 345–368. Herling, S., Downs, D.A., Woods, J.H., 1979. Cocaine, D-amphetamine, and pentobarbital effects on responding maintained by food or cocaine in rhesus monkeys. Psychopharmacology 64, 261–269. Holloway, K.S., 2012. Opioid mediation of learned sexual behavior. Socioaffective Neuroscience & Psychology 2, 14874. Holmes, T.H., Rahe, R.H., 1967. The social readjustment rating scale. J. Psychosom. Res. 11, 213–218. Huston, J.P., de Souza Silva, M.A., Komorowski, M., Schulz, D., Topic, B., 2013. Animal models of extinction-induced depression: loss of reward and its consequences. Neurosci. Biobehav. Rev. 37, 2059–2070. Jenney, C.B., Petko, J., Ebersole, B., Nzinkeu Njatcha, C.V., Uzamere, T.O., Alexander, D.N., Grigson, P.S., Levenson, R., 2016. Early avoidance of a heroin-paired taste-cue and subsequent addiction-like behavior in rats. Brain Res. Bull. 123, 61–70. Jiménez-García, A.M., Ruíz-Leyva, L., Cendán, C.M., Torres, C., Papini, M.R., Morón, I., 2016. Hypoalgesia induced by reward devaluation in rats. PLoS One 11, e0164331. Jones, L.M., Fontanini, A., Katz, D.B., 2006. Gustatory processing: a dynamic systems approach. Curr. Opin. Neurobiol. 16, 420–428. Judice-Daher, D.M., Bueno, J.L.O., 2013. Lesions of the nucleus accumbens disrupt reinforcement omission effects in rats. Behav. Brain Res. 252, 439–443. Justel, N., Mustaca, A.E., Bentosela, M., Ruetti, E., 2011. Efecto del diacepam sobre la extinción consumatoria. Revista Argentina de Ciencias del Comportamiento 3, 7–11. Justel, N., Pautasi, R., Mustaca, A.E., 2014. Proactive interference of open field on consummatory successive negative contrast. Learn. Behav. 42, 58–68. Kalivas, P.W., Volkow, N.D., 2005. The neural basis of addiction: a pathology of motivation and choice. Am. J. Psychiatr. 162, 1403–1413. Kamenetzky, G.V., Mustaca, A.E., Papini, M.R., 2008. An analysis of the anxiolytic effects of ethanol on consummatory successive negative contrast. Advances in Latin American Psychology 26, 135–144. Kamenetzky, G.V., Cuenya, L., Pedrón, V.T., Mustaca, A.E., 2009a. Frustración y respuestas a contextos asociados al etanol. Revista Mexicana de Psicología 26, 193–201. Kamenetzky, G.V., Mustaca, A.E., Pedron, V.T., Cuenya, L., Papini, M.R., 2009b. Ethanol facilitates consummatory extinction. Behav. Process. 82, 352–354. Kawasaki, K., Iwasaki, T., 1997. Corticosterone levels during extinction of runway response in rats. Life Sci. 61, 1721–1728. Kawasaki, K., Glueck, A.C., Annicchiarico, I., Papini, M.R., 2015. Function of the centromedial amygdala in reward devaluation and open-field activity. Neuroscience 303, 73–81. Kawasaki, K., Annicchiarico, I., Morón, I., Papini, M.R., 2017. Reward Loss and the Basolateral Amygdala: A Role in Reward Comparisons (Manuscript submitted for publication). Kelley, A.E., Berridge, K.C., 2002. The neuroscience of natural rewards: relevance to addictive drugs. J. Neurosci. 22, 3306–3311. Kessler, R.C., 1997. The effects of stressful life events on depression. Annu. Rev. Psychol. 48, 191–214. Keyes, K.M., Hatzenbuehler, M.L., Hasin, D.S., 2011. Stressful life experiences, alcohol consumption, and alcohol use disorders: the epidemiologic evidence for four main types of stressors. Psychopharmacology 218, 1–17. Khantzian, E.J., 1985. The self-medication hypothesis of addictive disorders: focus on heroin and cocaine dependence. Am. J. Psychiatr. 142, 1259–1264. Khantzian, E.J., 2013. Addiction as a self-regulation disorder and the role of self-medication. Addiction 108, 668–674. Kimura, M., Satoh, T., Matsumoto, N., 2007. What does the habenula tell dopamine neurons? Nat. Neurosci. 10, 677–678. Konopka, A., Pełka-Wysiecka, J., Grzywacz, A., Samochowiec, J., 2013. Psychosocial characteristics of benzodiazepine addicts compared to not addicted benzodiazepine users. Prog. Neuro-Psychopharmacol. Biol. Psychiatry 40, 229–235. L.A. Ortega et al. / Pharmacology, Biochemistry and Behavior 154 (2017) 39–52 Koob, G.F., 2013. Addiction is a reward deficit and stress surfeit disorder. Front. Psychol. 4, 1–18. Koob, G.F., Le Moal, M., 2001. Drug addiction, dysregulation of reward, and allostasis. Neuropsychopharmacology 24, 97–129. Koob, G.F., Volkow, N.D., 2010. Neurocircuitry of addiction. Neuropsychopharmacology 35, 217–238. Koob, G.F., Volkow, N.D., 2016. Neurobiology of addiction: a neurocircuitry analysis. The Lancet Psychiatry 3, 760–773. Lader, M., 1994. Anxiolytic drugs: dependence, addiction and abuse. Eur. Neuropsychopharmacol. 4, 85–91. Lanciego, J.L., Luquin, N., Obeso, J.A., 2012. Functional neuroanatomy of the basal ganglia. Cold Spring Harb. Perspect. Med. 2, 1–21. Le Moal, M., Koob, G.F., Bloom, F.E., 1979. Endorphins and extinction: differential actions on appetitive and aversive tasks. Life Sci. 24, 1631–1636. Lecca, S., Meye, F.J., Mameli, M., 2014. The lateral habenula in addiction and depression: an anatomical, synaptic and behavioral overview. Eur. J. Neurosci. 39, 1170–1178. Leong, K.-C., Berini, C.R., Ghee, S.M., Reichel, C.M., 2016. Extended cocaine-seeking produces a shift from goal-directed to habitual responding in rats. Physiol. Behav. 164, 330–335. Leslie, J.C., Shaw, D., Gregg, G., McCormick, N., Reynolds, D.S., Dawson, G.R., 2005. Effects of reinforcement schedule on facilitation of operant extinction by chlordiazepoxide. J. Exp. Anal. Behav. 84, 327–338. Leslie, J.C., Norwood, K., Kennedy, P.J., Begley, M., Shaw, D., 2012. Facilitation of extinction of operant behaviour in C57Bl/6 mice by chlordiazepoxide and D-cycloserine. Psychopharmacology 223, 223–235. Leszczuk, M.H., Flaherty, C.F., 2000. Lesions of nucleus accumbens reduce instrumental but not consummatory negative contrast in rats. Behav. Brain Res. 116, 61–79. Liao, R.M., Chuang, F.J., 2003. Differential effects of diazepam infused into the amygdala and hippocampus on negative contrast. Pharmacol. Biochem. Behav. 74, 953–960. Lin, J.Y., Roman, C., Reilly, S., 2009. Insular cortex and consummatory successive negative contrast in the rat. Behav. Neurosci. 123, 810–814. Lu, Y.-L., Richardson, H.N., 2014. Alcohol, stress hormones, and the prefrontal cortex: a proposed pathway to the dark side of addiction. Neuroscience 277, 139–151. Lucantonio, F., Kambhampati, S., Haney, R.Z., Atalayer, D., Rowland, N.E., Shaham, Y., Schoenbaum, G., 2015. Effects of prior cocaine versus morphine or heroin self-administration on extinction learning driven by overexpectation versus omission of reward. Biol. Psychiatry 77, 912–920. Mantsch, J.R., Cullinan, W.E., Tang, L.C., Baker, D.A., Katz, E.S., Hoks, M.A., Ziegler, D.R., 2007. Daily cocaine self-administration under long-access conditions augments restraint-induced increases in plasma corticosterone and impairs glucocorticoid receptor-mediated negative feedback in rats. Brain Res. 1167, 101–111. Manzo, L., Gómez, M.J., Callejas-Aguilera, J.E., Fernández-Teruel, A., Papini, M.R., Torres, C., 2012. Oral ethanol self-administration in inbred Roman high- and low-avoidance rats: gradual versus abrupt ethanol presentation. Physiol. Behav. 108, 1–5. Manzo, L., Gómez, M.J., Callejas-Aguilera, J.E., Fernández-Teruel, A., Papini, M.R., Torres, C., 2014. Anti-anxiety self-medication induced by incentive loss in rats. Physiol. Behav. 123, 86–92. Manzo, L., Donaire, R., Sabariego, M., Papini, M.R., Torres, C., 2015a. Anti-anxiety self-medication in rats: oral consumption of chlordiazepoxide and ethanol after reward devaluation. Behav. Brain Res. 278, 90–97. Manzo, L., Gómez, M., Callejas-Aguilera, J.E., Fernández-Teruel, A., Papini, M.R., Torres, C., 2015b. Partial reinforcement reduces vulnerability to anti-anxiety self-medication during appetitive extinction. Int. J. Comp. Psychol. 28, 1–8. Marks-Kaufman, R., Lewis, M.J., 1984. Early housing experience modifies morphine selfadministration and physical dependence in adult rats. Addict. Behav. 9, 235–243. Maroteaux, M., Mameli, M., 2012. Cocaine-evokes projection-specific synaptic plasticity in the lateral habenula. J. Neurosci. 32, 12641–12646. Matthews, K., Wilkinson, L., Robbins, T., 1996. Repeated maternal separation of preweanling rats attenuates behavioral responses to primary and conditioned incentives in adulthood. Physiol. Behav. 59, 99–107. McCoy, D.F., 1972. Some effect of scopolamine on acquisition and extinction performance in rats. Psychol. Rep. 30, 867–873. McDonald, A.J., 1998. Cortical pathways to the mammalian amygdala. Prog. Neurobiol. 55, 257–332. McLaughlin, R.J., Floresco, S.B., 2007. The role of different subregions of the basolateral amygdala in cue-induced reinstatement and extinction of food-seeking behavior. Neuroscience 146, 1484–1494. Mendoza, J., Sanio, C., Chaudhri, N., 2015. Inactivating the infralimbic but not prelimbic medial prefrontal cortex facilitates the extinction of appetitive Pavlovian conditioning in Long-Evans rats. Neurobiol. Learn. Mem. 118, 198–208. Miceli, M., Castelfranchi, C., 2015. Expectancy and Emotion. Oxford University Press, Oxford. Mitchell, C.P., Flaherty, C.F., 1998. Temporal dynamics of corticosterone elevation in successive negative contrast. Physiol. Behav. 64, 287–292. Mitchell, C.P., Leszczuk, M., Flaherty, C.F., 2004. Evidence for zolpidem-induced hyperphagia, but not anxiolysis, in a successive negative negative contrast paradigm. Pharmacol. Biochem. Behav. 79, 523–531. Molinengo, L., 1964. Effects of morphine on the operant behaviour in rats. Psychopharmacologia 6, 347–367. Morales, A., Torres, C., Megias, J.L., Cándido, A., Maldonado, A., 1992. Effect of diazepam on successive negative contrast in one-way avoidance learning. Pharmacol. Biochem. Behav. 43, 153–157. Morley, B.J., Russin, R., 1978. The effects of scopolamine on extinction and spontaneous recovery. Psychopharmacology (Berlin) 56, 301–304. Mustaca, A.E., Papini, M.R., 2005. Consummatory successive negative contrast induces hypoalgesia. Int. J. Comp. Psychol. 18, 255–262. 51 Mustaca, A.E., Bentosela, M., Ruetti, E., Kamenetzky, G., Cuenya, L., Justel, N., Lopez-Seal, F., Fosacheca, S., Papini, M.R., 2009. Similitudes y discrepancies en dos modelos animales de frustracion. In: Richaud, M.C., Moreno, E. (Eds.), Recientes avances en investigacion en ciencias del comportamiento Vol. 2. CIIPME-CONICET, Buenos Aires, Argentina, pp. 921–940. Naqvi, N.H., Bechara, A., 2009. The hidden island of addiction: the insula. Trends Neurosci. 32, 56–67. Naqvi, N.H., Bechara, A., 2010. The insula and drug addiction: an interoceptive view of pleasure, urges, and decision-making. Brain Struct. Funct. 214, 435–450. Naqvi, N.H., Rudrauf, D., Damasio, H., Bechara, A., 2007. Damage to the insula disrupts addiction to cigarette smoking. Science 315, 531–534. Nash, J.F., Maickel, R.P., 1985. Stress-induced consumption of ethanol by rats. Life Sci. 37, 757–765. Nestler, E.J., 2004. Molecular mechanisms of drug addiction. Neuropharmacology 47, 24–32. Nikiforuk, A., Popik, P., 2009. Antidepressants alleviate the impact of reinforcer downshift. Eur. Neuropsychopharmacol. 19, 41–48. Norris, J.N., Pérez-Acosta, A.M., Ortega, L.A., Papini, M.R., 2009. Naloxone facilitates appetitive extinction and eliminates escape from frustration. Pharmacol. Biochem. Behav. 94, 81–87. Norris, J.N., Ortega, L.A., Papini, M.R., 2011. Posttrial D-cycloserine enhances the emotional memory of an incentive downshift event. Behav. Brain Res. 223, 348–355. Olds, M.E., 1970. Comparative effects of amphetamine, scopolamine, chlordiazepoxide, and diphenylhydantoin on operant and extinction behavior with brain stimulation and food reward. Neuropharmacology 9, 519–532. Ortega, L.A., Daniel, A.M., Davis, J.B., Fuchs, P.N., Papini, M.R., 2011a. Peripheral pain enhances the effects of incentive downshifts. Learn. Motiv. 42, 203–209. Ortega, L.A., Uhelski, M., Fuchs, P.N., Papini, M.R., 2011b. Impairment of recovery from incentive downshift after lesions of the anterior cingulate cortex: emotional or cognitive deficits? Behav. Neurosci. 6, 988–995. Ortega, L.A., Glueck, A.C., Uhelski, M., Fuchs, P.N., Papini, M.R., 2013a. Role of the ventrolateral orbital cortex and medial prefrontal cortex in incentive downshift situations. Behav. Brain Res. 244, 120–129. Ortega, L.A., Prado-Rivera, M.A., Cárdenas-Poveda, D.C., McLinden, K.A., Glueck, A.C., Gutiérrez, G., Lamprea, M.R., Papini, M.R., 2013b. Tests of the aversive summation hypothesis in rats: effects of restraint stress on consummatory successive negative contrast and extinction in the Barnes maze. Learn. Motiv. 44, 159–173. Ortega, L.A., Glueck, A.C., Daniel, A.M., Prado-Rivera, M.A., White, M.M., Papini, M.R., 2014a. Memory interfering effects of chlordiazepoxide on consummatory successive negative contrast. Pharmacol. Biochem. Behav. 116, 96–106. Ortega, L.A., Norris, J., Lopez-Seal, M.F., Ramos, T., Papini, M.A., 2014b. Correlates of recovery from incentive downshift: a preliminary selective breeding study. Int. J. Comp. Psychol. 27, 18–44. Packard, M.G., McGaugh, J.L., 1996. Inactivation of hippocampus or caudate nucleus with lidocaine differentially affects expression of place and response learning. Neurobiol. Learn. Mem. 65, 65–72. Panayi, M.C., Killcross, S., 2014. Orbitofrontal cortex inactivation impairs between-but not within-session Pavlovian extinction: an associative analysis. Neurobiol. Learn. Mem. 108, 78–87. Papini, M.R., 2009. Role of opioid receptors in incentive contrast. Int. J. Comp. Psychol. 22, 170–187. Papini, M.R., Dudley, R.T., 1997. Consequences of surprising reward omissions. Rev. Gen. Psychol. 1, 175–197. Papini, M.R., Ortega, L.A., 2011. Endogenous opioids, opioid receptors, and incentive processes. In: Preedy, V.R., Watson, R.R., Martin, C.R. (Eds.), Handbook of Behavior, Food, and Nutrition. Springer, New York, pp. 1011–1019. Papini, M.R., Wood, M., Daniel, A.M., Norris, J.N., 2006. Reward loss as psychological pain. Int. J. Psychol. Psychol. Ther. 6, 189–213. Papini, S., Galatzer-Levy, I.R., Papini, M.R., 2014. Identifying profiles of recovery from reward devaluation in rats. Behav. Brain Res. 275, 212–218. Papini, M.R., Fuchs, P.N., Torres, C., 2015. Behavioral neuroscience of psychological pain. Neurosci. Biobehav. Rev. 48, 53–69. Paré, D., Quirk, G.J., Ledoux, J.E., 2004. New vistas on amygdala networks in conditioned fear. J. Neurophysiol. 92, 1–9. Paulus, M.P., Stein, M.B., 2006. An insular view of anxiety. Biol. Psychiatry 60, 383–387. Pecoraro, N., Dallman, M.F., 2005. c-Fos after incentive shifts: expectancy, incredulity, and recovery. Behav. Neurosci. 119, 366–387. Pecoraro, N., de Jong, H., Dallman, M.F., 2009. An unexpected reduction in sucrose concentration activates the HPA axis on successive post shift days without attenuation by discriminative contextual stimuli. Physiol. Behav. 96, 651–661. Pecoraro, N., de Jong, H., Ginsberg, A.B., Dallman, M.F., 2008. Lesions of the medial prefrontal cortex enhance the early phase of psychogenic fever to unexpected sucrose concentration reductions, promote recovery from negative contrast and enhance spontaneous recovery of sucrose-entrained anticipatory activity. Neuroscience 153, 901–917. Pellegrini, S., Muzio, R.N., Mustaca, A.E., Papini, M.R., 2004. Successive negative contrast after partial reinforcement in the consummatory behavior of rats. Learn. Motiv. 35, 303–321. Pellegrini, S., Wood, M., Daniel, A.M., Papini, M.R., 2005. Opioid receptors modulate recovery from consummatory successive negative contrast. Behav. Brain Res. 164, 239–249. Phelps, C.E., Mitchell, E.N., Nutt, D.J., Marston, H.M., Robinson, E.S.J., 2015. Psychopharmacological characterization of the successive negative contrast effect in rats. Psychopharmacology 232, 2697–2709. Piazza, P.V., Le Moal, M., 1996. Pathophysiological basis of vulnerability to drug abuse: role of an interaction between stress, glucocorticoids, and dopaminergic neurons. Annu. Rev. Pharmacol. Toxicol. 36, 359–378. 52 L.A. Ortega et al. / Pharmacology, Biochemistry and Behavior 154 (2017) 39–52 Piazza, P.V., Le Moal, M., 1998. The role of stress in drug self-administration. Trends Pharmacol. Sci. 19, 67–74. Piazza, P.V., Maccari, S., Deminiere, J.M., Le Moal, M., Mormede, P., Simon, H., 1991. Corticosterone levels determine individual vulnerability to amphetamine self-administration. Proceedings of the National Academy of Sciences. 88, pp. 2088–2092. Picetti, P., Caccavo, J.A., Ho, A., Kreek, M.J., 2012. Dose escalation and dose preference in extended-access heroin self-administration in Lewis and Fischer rats. Psychopharmacology (Berlin) 220, 163–172. Pobbe, R.L.H., Zangrossi, H., 2008. Involvement of the lateral habenula in the regulation of generalized anxiety- and panic-related defensive responses in rats. Life Sci. 82, 1256–1261. Ramot, A., Akirav, I., 2012. Cannabinoid receptors activation and glucocorticoid receptors deactivation in the amygdala prevent the stress-induced enhancement of a negative learning experience. Neurobiol. Learn. Mem. 97, 393–401. Reilly, S., Trifunovic, R., 2003. Gustatory thalamus lesions eliminate successive negative contrast in rats: evidence against a memory deficit. Behav. Neurosci. 117, 606–615. Rhodes, S.E., Killcross, S., 2004. Lesions of rat infralimbic cortex enhance recovery and reinstatement of an appetitive Pavlovian response. Learn. Mem. 11, 611–616. Rhodes, S.E., Killcross, S., 2007. Lesions of rat infralimbic cortex enhance renewal of extinguished appetitive Pavlovian responding. Eur. J. Neurosci. 25, 2498–2503. Robbins, T.W., Ersche, K.D., Everitt, B.J., 2008. Drug addiction and the memory systems of the brain. Ann. N. Y. Acad. Sci. 1141, 1–21. Robinson, T.E., Berridge, K.C., 2000. The psychology and neurobiology of addiction: an incentive-sensitization view. Addiction 95, 91–117. Rosas, J.M., Callejas-Aguilera, J.E., Escarabajal, M.D., Gómez, M.J., de la Torre, L., Agüero, A., Tobeña, A., Fernández-Teruel, A., Torres, C., 2007. Successive negative contrast effect in instrumental runway behaviour: a study with Roman high- (RHA) and Roman low- (RLA) avoidance rats. Behav. Brain Res. 185, 1–8. Rosen, A.J., Glass, D.H., Ison, J.R., 1967. Amobarbital sodium and instrumental performance changes following reward reduction. Psychon. Sci. 9, 129–130. Rowan, G.A., Flaherty, C.F., 1987. The effects of morphine in the consummatory contrast paradigm. Psychopharmacology 93, 51–58. Rowan, G.A., Flaherty, C.F., 1991. Behavior of maudsley reactive and nonreactive rats (Rattus norvegicus) in three consummatory contrast paradigms. J. Comp. Psychol. 105, 115–124. Ruetti, R., Justel, N., 2010. Bases neurobiológicas de la frustración. Revista Argentina de Ciencias del Comportamiento 2 (3), 45–60. Ruetti, E., Justel, N., Mustaca, A.E., Papini, M.R., 2009. Posttrial corticosterone administration enhances the effects of incentive downshift: exploring the boundaries of this effect. Behav. Neurosci. 123, 137–144. Ruetti, E., González, J.A., Justel, N., Torrecilla, M., Mustaca, A.E., 2010. Estrés neonatal y frustración. Revista Latinoamericana de Psicología 42, 279–288. Sabariego, M., Gómez, M.J., Morón, I., Torres, C., Fernández-Teruel, A., Tobeña, A., Canete, T., Martínez-Conejero, J.A., Horcajadas, J.A., Esteban, F.J., 2011. Differential gene expression between inbred Roman high-(RHA-I) and low-(RLA-I) avoidance rats. Neurosci. Lett. 504, 265–270. Sabariego, M., Morón, I., Gómez, M.J., Donaire, R., Tobeña, A., Fernández-Teruel, A., Esteban, F.J., Martínez-Conejero, J.A., Torres, C., 2013. Incentive loss and hippocampal gene expression in inbred Roman high-(RHA-I) and Roman low-(RLA-I) avoidance rats. Behav. Brain Res. 257, 62–70. Saleem, K.S., Pauls, J.M., Augath, M., Trinath, T., Prause, B.A., Hashikawa, T., Logothetis, N.K., 2002. Magnetic resonance imaging of neuronal connections in the macaque monkey. Neuron 34, 685–700. Salinas, J.A., Parent, M.B., McGaugh, J.L., 1996. Ibotenic acid lesions of the amygdala basolateral complex or central nucleus differentially effect the response to reductions in reward. Brain Res. 742, 283–293. Sartorius, A., Kiening, K.L., Kirsch, P., von Gall, C.C., Utenberg, A.W., Henn, F., MeyerLindenberg, A., 2010. Remission of mayor depression under deep brain stimulation of the lateral habenula in a therapy-refractory patient. Biol. Psychiatry 67, e9–e11. Sbarra, D.A., Hazan, C., 2008. Coregulation, dysregulation, self-regulation: an integrative analysis and empirical agenda for understanding adult attachment, separation, loss, and recovery. Personal. Soc. Psychol. Rev. 12, 141–167. Schoenbaum, G., Roesch, M.R., Stalnaker, T.A., 2006. Orbitofrontal cortex, decision-making and drug addiction. Trends Neurosci. 29, 116–124. Shaham, Y., Erb, S., Stewart, J., 2000. Stress-induced relapse to heroin and cocaine seeking in rats: a review. Brain Res. Rev. 33, 13–33. Shanab, M.E., Ralph, L., 1979. Negative contrast and partial reinforcement effects as a function of crowded rearing condition in the rat. J. Gen. Psychol. 100, 13–26. Shumake, J., Gonzalez-Lima, F., 2003. Brain systems underlying susceptibility to helplessness and depression. Behav. Cogn. Neurosci. Rev. 2, 198–221. Sinha, R., 2008. Chronic stress, drug use, and vulnerability to addiction. Ann. N. Y. Acad. Sci. 1141, 105–130. Sinha, R., Li, C.S.R., 2007. Imaging stress-and cue-induced drug and alcohol craving: association with relapse and clinical implications. Drug Alcohol Rev. 26, 25–31. Sinha, R., Lacadie, C., Skudlarski, P., Fulbright, R.K., Rounsaville, B.J., Kosten, T.R., Wexler, B.E., 2005. Neural activity associated with stress-induced cocaine craving: a functional magnetic resonance imaging study. Psychopharmacology 183, 171–180. Smith, M.A., Fronk, G.E., Zhang, H., Magee, C.P., Robinson, A.M., 2016. Acute bouts of wheel running decrease cocaine self-administration: influence of exercise output. Pharmacol. Biochem. Behav. 150–151, 94–99. Soubrie, P., Thiebot, P., Simon, P., Boissier, J.R., 1978. Benzodiazepines and behavioural effects of reward (water) omission in the rats. Psychopharmacology 59, 95–100. Spanagel, R., Noori, H.R., Heilig, M., 2014. Stress and alcohol interactions: animal studies and clinical significance. Trends Neurosci. 37, 219–227. Stopper, C.M., Floresco, S.B., 2014. What's better for me? Fundamental role for lateral habenula in promoting subjective decision biases. Nat. Neurosci. 17, 33–37. Tan, K.R., Rudolph, U., Lü, C., 2011. Hooked on benzodiazepines: GABA receptor subtypes and addiction. Trends Neurosci. 4, 188–197. Tavares, T.F., Judice-Daher, D.M., Bueno, J.L.O., 2014. Large neurotoxic amygdala lesion impairs reinforcement omission effects. Behav. Brain Res. 266, 1–6. Torres, C., Papini, M.R., 2016. Emotional self-medication and addiction. In: Preedy, V.R. (Ed.), Neuropathology of Drug Addiction and Substance Misuse Vol. 1. Elsevier, New York, pp. 71–81. Torres, C., Sabariego, M., 2014. Incentive relativity: gene-environment interactions. Int. J. Comp. Psychol. 27, 446–458. Torres, C., Morales, A., Cándido, A., Maldonado, A., 1996. Successive negative contrast in one-way avoidance: effect of thiopental sodium and chlorpromazine. Eur. J. Pharmacol. 314, 269–275. Torres, C., Cándido, A., Escarabajal, M.D., de la Torre, L., Maldonado, A., Tobeña, A., Fernández-Teruel, A., 2005. Successive negative contrast in one-way avoidance learning in female roman rats. Physiol. Behav. 85, 377–382. Torres, C., Glueck, A.C., Conrad, S.E., Moron, I., Papini, M.R., 2016. Dorsomedial striatum lesions affect adjustment to reward uncertainty, but not to reward devaluation or omission. Neuroscience 332, 13–25. Tournier, B.B., Steimer, T., Millet, P., Moulin-Sallanon, M., Vallet, P., Ibáñez, V., Ginovart, N., 2013. Innately low D2 receptor availability is associated with high novelty-seeking and enhanced behavioural sensitization to amphetamine. Int. J. Neuropsychopharmacol. 6, 1819–1834. Tovote, P., Fadok, J.P., Lüthi, A., 2015. Neuronal circuits for fear and anxiety. Nature Neuroscience Reviews 16, 317–331. Travers, J.B., Dinardo, L.A., Karimnamazi, H., 1997. Motor and premotor mechanisms of licking. Neurosci. Biobehav. Rev. 21, 631–647. Ungless, M.A., Argilli, E., Bonci, A., 2010. Effect of stress and aversion on dopamine neurons: implications for addiction. Neurosci. Biobehav. Rev. 35, 151–156. Valenza, M., Picetti, R., Yuferov, V., Butelman, E.R., Kreek, M.J., 2016. Strain and cocaine-induced differential opioid gene expression may predispose Lewis but not Fischer rats to escalate cocaine self-administration. Neuropharmacology 105, 639e650. Velasquez, K.M., Molfese, D.L., Salas, R., 2014. The role of the habenula in drug addiction. Front. Hum. Neurosci. 8, 1–10. Volkow, N.D., Baler, R.D., 2014. Addiction science: uncovering neurobiological complexity. Neuropharmacology 76, 235–249. Volkow, N.D., Fowler, J.S., 2000. Addiction, a disease of compulsion and drive: involvement of the orbitofrontal cortex. Cereb. Cortex 10, 318–325. Volkow, N.D., Morales, M., 2015. The brain on drugs: from reward to addiction. Cell 162, 712–725. Waldrop, A.E., Back, S.E., Brady, K.T., Upadhyaya, H.P., McRae, A.L., Saladin, M.E., 2007. Daily stressor sensitivity, abuse effects, and cocaine use in cocaine dependence. Addict. Behav. 32, 3015–3025. Whitehouse, J.M., 1966. The effects of physostygmine on discrimination learning. Psychopharmacology (Berlin) 9, 183–188. Williams, J., Gray, J.A., Sinden, J., Buckland, C., Rawlins, J., 1990. Effects of GABAergic drugs, fornicotomy, hippocampectomy, and septal lesions on the extinction of a discretetrial fixed ratio 5 lever-press response. Behav. Brain Res. 41, 129–150. Wood, M.D., Daniel, A.M., Papini, M.R., 2005. Selective effects of the δ-opioid receptor agonist DPDPE on consummatory successive negative contrast. Behav. Neurosci. 119, 446–454. Wood, M.D., Norris, J.N., Daniel, A.M., Papini, M.R., 2008. Trial-selective effects of U50,488H, a κ-opioid receptor agonist, on consummatory successive negative contrast. Behav. Brain Res. 193, 28–36. Yager, M.Y., Garcia, A.F., Wunsch, A.M., Ferguson, S.M., 2015. The ins and outs of the striatum: role in drug addiction. Neuroscience 301, 529–541. Yamamoto, T., Azuma, S., Kawamura, Y., 1984. Functional relations between the cortical gustatory area and the amygdala: electrophysiological and behavioral studies in rats. Exp. Brain Res. 56, 23–31. Yap, J.J., Miczek, K.A., 2008. Stress and rodent models of drug addiction: role of VTA-accumbens-PFC-amygdala circuit. Drug Discov. Today Dis. Model. 5, 259–270.