Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Q1911—QJEP(B)02b21 / Mar 3, 04 (Wed)/ [34 pages – 0 Tables – 7 Figures – 0 Footnotes – 0 Appendices]. . Centre single caption. cf. [no comma]. WTG OCR Scanned. THE QUARTERLY JOURNAL OF EXPERIMENTAL PSYCHOLOGY, 2004, 57B (2), 97–132 Experimental extinction in Pavlovian conditioning: Behavioural and neuroscience perspectives Andrew R. Delamater Brooklyn College–CUNY, New York, USA This paper reviews the behavioural and neuroscience literatures on extinction in Pavlovian conditioning with a view towards finding possible points of contact between these two often independent lines of investigation. Recent discoveries at the behavioural level indicate (1) that conditioned stimulus (CS)–unconditioned stimulus (US) associations specific in their sensory content are fully preserved during extinction, (2) that inhibitory stimulus–response associations appear to be learned during extinction, (3) that extinction is influenced by the level of activation of the US representation during nonreinforced trials, (4) that decreases in attention can influence conditioned performance during extinction, and (5) that contexts acquire an ability to modulate learning during both conditioning and extinction. Recent discoveries at the neural systems level suggest (1) that the hippocampus is important in context-specific learning during extinction, (2) that the prefrontal cortex is possibly important in long-term memory for extinction, (3) that the basolateral amygdala may be important in sustaining attention to a CS during extinction, (4) that NMDA receptors are important either in neural plasticity during extinction or by affecting the value of the US representation during extinction, and (5) that the GABAergic system may partially mediate inhibitory learning during extinction. It is concluded that both of these levels of analysis can benefit the other in the pursuit of a more comprehensive understanding of extinction. Experimental extinction often refers to the procedure of presenting a conditioned stimulus (CS) repeatedly, but in the absence of the unconditioned stimulus (US) with which it was paired previously. The effect of this procedure is to decrease the CS’s ability to evoke the conditioned response (CR) established during the initial phase in which the CS and US were paired. Extinction also commonly refers to the processes (associative or nonassociative) that are affected by the procedure of withholding the US during nonreinforced presentations of the CS. While there has been a long tradition of research directed towards an analysis of extinction as a procedure and as a process (e.g., see Mackintosh, 1974), our understanding of how extinction is best conceptualized has remained rather limited. In recent years, however, Correspondence should be addressed to Andrew R. Delamater, Psychology Department, Brooklyn College– CUNY, 2900 Bedford Ave, Brooklyn, New York 11210, USA. Email: [email protected] The author gratefully acknowledges Rob Honey, Simon Killcross, Todd Schachtman, Mark Bouton, and Vin LoLordo for providing many helpful comments on earlier versions of this manuscript. 2004 The Experimental Psychology Society http://www.tandf.co.uk/journals/pp/02724995.html DOI:10.1080/02724990344000097 98 DELAMATER there has been an upsurge of interest in extinction from multiple perspectives. A considerable amount of attention has been directed to a more in-depth associative analysis of extinction conducted almost exclusively at the behavioural level. This more in-depth analysis of extinction has been aided by the development of more refined behavioural techniques that allow us to identify with more precision the nature of the associations formed in a given learning situation, as well as how the experimental context comes to exert control over those associations. This analysis has aided tremendously our understanding of the psychological consequences of an extinction procedure. Quite independently, a large body of data has been accumulating in recent years involving delineating the neural basis of extinction. This study has been aided by the advent of refined neuroscience techniques that range from different systems-level lesion methods, to a variety of receptor-specific pharmacological and genetic manipulations, down to single cellular electrophysiological recording. While there is now a considerable amount of information concerning the neurobiological foundations of extinction, a consensus regarding the basic central nervous system mechanisms of extinction is only beginning to emerge. It is striking how truly independent these two literatures have become. There is very little crosstalk, in spite of the fact that they both are concerned with essentially overlapping sets of behavioural phenomena. The point of this paper is to attempt to begin to bridge the gap, both methodologically and conceptually, between these literatures. First, I review what I take to be the important discoveries in each of these literatures, and then I conclude by suggesting ways in which psychological and neuroscience approaches can be informative to one another in reaching an understanding of extinction. My task is made a bit easier by the existence of several recent excellent and comprehensive reviews of the psychological (Rescorla, 2001a) and neural (Davis, Falls, & Gewirtz, 2000; Myers & Davis, 2002) literatures on extinction. The present review differs from these by focusing on a more limited set of recent findings and by more explicitly inviting comparisons between these two research domains. In addition, I restrict my attention to research investigating extinction in Pavlovian preparations, though studies exist in separate literatures investigating extinction in instrumental learning situations from both behavioural and neural perspectives. BEHAVIOURAL STUDIES A. What is learned in extinction? 1. Unlearning versus new learning One important issue in the study of extinction concerns the nature of the learning that occurs during extinction. Quite early on, Pavlov (1927) entertained the possibilities that extinction results in (1) unlearning of the associations established during the original acquisition phase, and (2) new learning during extinction, but of an inhibitory nature. His research led him to favour the second alternative. However, many popular learning models since Pavlov (e.g., Rescorla & Wagner, 1972) have assumed that extinction results in unlearning of the old associations. The experimental basis for the conclusion that extinction does not result in unlearning has been rather unconvincing until recently (see also Hall, 2002; Rescorla, 2001a). Commonly PAVLOVIAN EXTINCTION 99 cited phenomena taken as evidence against unlearning, in fact, often do not include the comparisons that would permit such an evaluation. Spontaneous recovery, renewal of conditioned responding following a context shift out of the extinction context, and US reinstatement phenomena have all been discussed as requiring that extinction does not entail unlearning. However, this conclusion would only seem to apply to the strong claim that extinction results in complete unlearning of the CS–US association. In other words, the common observation of a renewal of conditioned responding when subjects are tested in a context other than where the CS had been extinguished implies that extinction did not totally abolish the CS–US association. However, a commonly underappreciated point (raised also by Rauhut, Thomas, & Ayres, 2001) is that renewal (even in an ABA design) often leaves the stimulus less able to elicit responding than in a control group lacking extinction. This comparison suggests that renewal is far from complete, in support of the view that some unlearning does take place during extinction. A surprisingly small set of studies, however, has produced results that are more consistent with the conclusion that once associations are established they are not unlearned. In a set of closely related studies, Delamater (1996) and Rescorla (1996a) examined the effects of extinction in procedures designed to assess the selectivity of the associations learned during excitatory training. In their studies, two different CSs each were associated with different reinforcing outcomes, and the selectivity of these associations was determined using either a Pavlovian-to-instrumental transfer testing procedure (Delamater, 1996) or a selective outcome devaluation procedure (Rescorla, 1996a). In both cases, it was observed that, compared to nonextinguished controls, extinction had no effect upon the specific CS–US associations acquired earlier. For instance, in the Pavlovian-to-instrumental transfer test, Delamater observed that a CS selectively biased instrumental choice in favour of an instrumental response with which it shared an outcome (see Figure 1). Moreover, extinguished and nonextinguished CSs equally controlled instrumental choice in this way. A natural conclusion based on these data is that specific associations between the CS and the sensory properties of the US, as measured by transfer and devaluation tests, are fully preserved following extinction. Delamater (1996) went on to assess the generality of this conclusion by assessing the effects of additional extinction treatments upon outcome-selective CS–US associations. It was demonstrated, for instance, that such associations were fully intact after extinction treatments where the CSs were exposed to either random or negative contingencies. Thus, it appears as though outcome-specific associations are extremely resistant to a variety of treatments designed to undermine those associations. One sort of objection directed at these findings is that extinction may have resulted in unlearning in these situations but went undetected because of insensitive response measures. Though an objection like this is always difficult to refute, it may be noted that Pavlovian-toinstrumental transfer tests have been used successfully in other contexts to detect rather small differences in associative strength between stimuli (e.g., Delamater, 1995; Rescorla, 1999, 2001b). A rather different type of preserved learning has also been reported recently. Ohyama, Gibbon, Deich, and Balsam (1999) adapted the “peak procedure” commonly employed in studies of interval timing in instrumental learning tasks to a Pavlovian autoshaping situation. They demonstrated that ring doves will show a peak rate of responding within the CS that is 100 DELAMATER Figure 1. Experimental design and data from Delamater (1996). close to the point in time where food is normally delivered. More interestingly, the temporal location of peak responding does not change throughout an extinction phase even though overall levels of responding gradually decline. Thus, it would appear as though extinction has an overall effect upon conditioned responding, but little effect upon learned temporal aspects of US delivery. Parenthetically, it may be added that this distinction between temporal aspects of conditioning and overall levels of responding is generally consistent with a rate expectancy theory of conditioning and extinction (Gallistel & Gibbon, 2000). However, that framework has difficulty with recent results (Haselgrove & Pearce, 2003) showing that CRs do not decline simply as a function of total CS exposure time during extinction. PAVLOVIAN EXTINCTION 101 While specific CS–US associations appear to be preserved after an extinction treatment, a recent study indicates that extinction has consequences for what happens when the CS and US are once again paired together. Rescorla (2001b) recently demonstrated increases in the strength of a sensory-specific CS–US association if reacquisition trials immediately followed extinction. The same additional CS–US pairings had no detectable effect on the associative strength of a nonextinguished CS. Thus, although the CS–US association itself is not diminished by extinction, the extinction treatment somehow allows the stimulus to benefit from further CS–US pairings that would otherwise be ineffective in strengthening the sensoryspecific association. This important result needs further experimental attention for it raises questions regarding the nature of the learning rules governing acquisition and extinction. For example, Rescorla’s data suggest that there exists some positive discrepancy between expected and obtained outcomes when the CS and US are once again paired together. This is puzzling given that other research indicates that the strength of the CS–US association is unaffected by extinction (Delamater, 1996; Rescorla, 1996a). Exactly how these two facts fit together is a matter for additional research. 2. Sensory and affective processes in extinction A surprisingly understudied issue is whether extinction has different influences upon associations formed between the CS and separate components of the US. Two components widely discussed originate from Konorski’s distinctions between sensory and motivational or affective properties of the US (Konorski, 1967). Contemporary theories of Pavlovian conditioning recognize the utility of this distinction (e.g., Wagner & Brandon, 1989); however, in practice it has sometimes been difficult to assess independently each type of association. Theories of this sort make the assumption that when a CS is paired with a US, separate and independent associations are formed with the sensory and affective components of the US. Though much of the work reviewed above has used response measures (such as the transfer test, for instance) that are sensitive to learning sensory-based CS–US associations, not much attention has been directed at investigating the status of associations with the affective component of the US following an extinction procedure. This problem seems like a rather important one because an extinction procedure may well have differential effects on the CS’s associations with the sensory and affective properties of the US (see also Napier, Macrae, & Kehoe, 1992). Relevant to this possibility is another aspect of the data reported by Delamater (1996). In that experiment, although extinction did not impair the CS’s ability to exert outcome-selective transfer of control over instrumental responses, it did result in a permanent reduction of conditioned magazine approach responses measured during the transfer test session (bottom panel of Figure 1). One interpretation of this finding is that during the conditioning phase the CS enters into separate associations with the affective and sensory attributes of the US, but that extinction results in unlearning of the affective association while sparing the sensory association. To explain these data one additionally needs to assume that associations with the affective properties of the US controlled the magazine approach CR, and that associations with the sensory properties of the US were responsible for the occurrence of outcome-specific transfer. Before this sort of analysis can be pursued more rigorously, however, it will become important to establish commonly recognized measures of each type of association. Ideally, one 102 DELAMATER would also like to have a means of manipulating each type of association in isolation before assessing the impact of other variables, like extinction, on each. A promising experimental paradigm for this purpose has been used recently by Blundell, Hall, and Killcross (2001). These investigators assessed the effects of basolateral amygdala lesions in rats on Pavlovianto-instrumental transfer. They observed that the normal tendency of CSs to exert selective control over instrumental responses with which they share an outcome was abolished by the lesion. However, in lesioned rats the CSs increased not only the response with which they shared an outcome but also the response with which they did not share an outcome. One interpretation of this result is that the lesion selectively abolished the sensory association, but left intact the affective association that nonselectively energized performance. If this interpretation is correct, then one could use this technique to assess the effects of extinction specifically on the association between the CS and the affective properties of the US in complete isolation from the sensory properties of the US. Unfortunately, the behavioural literature on extinction has very little approaching this level of separation between the two types of association. It is tempting to speculate that conditioned flavour preference and aversion preparations entail learned associations that are predominantly motivational in character. In both of these preparations it is possible to associate a flavour with some meaningful postingestive consequence (e.g., see Elizalde & Sclafani, 1990). To the extent that conditioned preferences or aversions established in this way are resistant to extinction, this may imply that associations with the motivational or affective properties of the US themselves are resistant to extinction. Indeed, there have been reports suggesting that it is difficult to extinguish a conditioned flavour preference established by pairings between a novel flavour and intra-gastric infusions of a carbohydrate (Perez, Lucas, & Sclafani, 1998). However, before interpreting data of this sort in terms of associations with the affective but not the sensory properties of the US, the possibility that sensory-specific signals can be generated postingestively and participate in the learning (e.g., see Perez, Ackroff, & Sclafani, 1996) would need to be eliminated. 3. Inhibitory stimulus–response associations Recent studies have demonstrated that although specific CS–US associations survive extinction fully intact the extinction procedure does have a lasting effect on conditioned responding. Specifically, consistent with Pavlov’s earlier claim, it seems to result in new learning. As will be discussed more thoroughly below, one way of characterizing this new learning is in terms of contextual modulation (e.g., Bouton & Bolles, 1985). However, another way of characterizing this new learning is in terms of the formation of an inhibitory association between the CS and the conditioned response elicited by the CS during the extinction phase. Evidence relevant to this claim comes from three sources. First, as noted above, Delamater (1996) demonstrated that although stimuli trained and then extinguished did not display a diminished capacity to exert outcome-selective transfer of control over instrumental responding, the CSs did lose their ability to evoke magazine approach CRs during extinction and during the transfer test session. Following a similar set of findings in instrumental settings (e.g., Colwill, 1991; Rescorla, 1993), Delamater (1996) interpreted this result in terms of inhibitory associations between the CS and the magazine PAVLOVIAN EXTINCTION Figure 2. 103 Experimental design from Rescorla (1997a). approach CR. Such learning would not impair the ability of the selective CS–US association to exert its influence during the Pavlovian-to-instrumental transfer test. Another finding suggestive of inhibitory stimulus–response associations in extinction was reported by Rescorla (1997a). In this study (see Figure 2), Rescorla used repeated Pavlovianto-instrumental transfer tests in extinction to ensure that different instrumental responses were made more frequently in the presence of each CS. In other words, following Pavlovian training in which each of two CSs signalled different USs, these CSs were tested in an instrumental choice situation. In these tests, each CS exerted more control over the instrumental response with which it shared a reinforcer. Rescorla gave repeated extinction test sessions until the overall levels of instrumental responding declined steadily in the presence of the Pavlovian cues. Since different instrumental responses occurred more frequently in the presence of each CS in these tests, it was hypothesized that inhibitory stimulus–response associations would develop between each CS and the more frequent instrumental response occurring in its presence. Rescorla tested for this first by retraining each instrumental response with the alternative reinforcer. At this point, the two instrumental responses had now received training with both reinforcers. Under normal circumstances, this would ensure that each Pavlovian cue would not affect the instrumental responses differentially. However, when the CSs were tested in a final transfer test, they now selectively decreased the instrumental response that previously was made more frequently in their presence. This result suggests that while the CS is undergoing extinction it can acquire an ability to selectively inhibit any response that occurs frequently in its presence. A third line of evidence supporting, albeit more indirectly, the inhibitory stimulus– response association view of extinction comes from a set of studies of spontaneous recovery by Rescorla (1996b, 1997b, 1997c). Spontaneous recovery of a magazine approach CR was examined in an appetitive situation with rats in which a CS first was associated with US1 and then with US2 in a second phase. This procedure deviates from the usual one in that the CS was paired with US2 instead of being nonreinforced during the second phase (Rescorla, 1997c). Since both USs were delivered to the same food magazine, the same magazine CR was measured to each US. Of interest was whether imposing a delay between Phase 2 training and a nonreinforced test session would influence the CR displayed to the CS. Rescorla observed that if the second phase occurred 1 week prior to the test, then the CR was augmented relative to its baseline at the end of the second phase. However, if training the CS with US2 occurred immediately prior to the test, then the CR was not augmented relative to the Phase 2 baseline. One interpretation of these observations is based on the notion of overexpectation. Specifically, during Phase 2 one could suppose that the CS continues to activate a representation of US1 while also acquiring the ability to activate a representation of US2. Since US2 is the only US presented in Phase 2, this overexpectation may result in a decrement in the CR. This decrement in the CR may not be observable immediately, however, because the CS also acquires a 104 DELAMATER new association (with US2) that should strengthen the CR. Nevertheless, if the decremental process wanes over a delay, then the new association with US2 will result in an increase in the CR. Rescorla (1997c) suggested that this decremental process was an inhibitory stimulus– response association that wanes over the delay. Note that the effect might alternatively be understood in terms of the CS developing an inhibitory association with US1 during phase 2 while being paired with US2. However, other research (e.g., Delamater, 1996) has demonstrated that associations with first and second trained USs are equally strong. Thus, an interpretation in terms of a US-independent decremental process, perhaps an inhibitory stimulus– response association, is more tenable. 4. Does extinction result in the CS becoming a net inhibitor? Interest in this question has been stimulated recently by demonstrations of slow reacquisition following extensive extinction in conditioned emotional response (CER) and conditioned taste aversion (CTA) preparations. However, the reliability of this effect as well as its interpretation in terms of extinction producing net inhibition is not well established. Bouton and Swartzentruber (1989), following an earlier study by Bouton (1986), demonstrated that after eight tone–shock pairings, 72 tone-alone extinction trials resulted in slow reacquisition of conditioned fear (measured by conditioned suppression of food-reinforced lever pressing) when additional tone–shock pairings were given. Slow reacquisition was assessed against a control group that received light–shock training and extinction prior to receiving tone–shock pairings in the test phase. Thus, compared to this novel stimulus control group reacquisition was slow (for evidence in the CTA preparation, see Hart, Bourne, & Schachtman, 1995). Two additional latent inhibition control groups were also run in the Bouton and Swartzentruber study. These groups received either 72 or 80 tone-alone trials prior to receiving tone–shock pairings in the test phase. A fact about this paper that is often overlooked is that the experimental group reacquired conditioned fear more rapidly than did these latent inhibition controls. Though the point of this paper was not to claim that extinction leaves a stimulus in a net inhibitory state, this comparison certainly complicates any claims regarding inhibition when assessed against a novel stimulus control condition. In conceptually similar experiments using the conditioned eyeblink preparation with rabbits, Napier et al. (1992) found exactly the opposite result. In the Napier et al. study rapid reacquisition of eyeblink CRs relative to a novel CS control was obtained when CS–US pairings were resumed following an extinction procedure, an explicitly unpaired procedure, a differential conditioned inhibition procedure, or a Pavlovian conditioned inhibition procedure. One important difference between these sets of results is the specific treatment given to the control groups. In the Bouton and Swartzentruber (1989) study the control group was trained and extinguished on another stimulus before being trained on the target CS. In the Napier et al. (1992) study the control groups were never exposed to any CS or US until the test. Using both of these control groups in the rat appetitive conditioning preparation, Ricker and Bouton (1996) obtained evidence of both sets of results described above. Another difference between the two sets of studies, noted by Napier et al., is that the eyeblink and CER preparations may measure quite different aspects of conditioning, and these may be differentially sensitive to extinction. For example, if the eyeblink preparation measured associations with PAVLOVIAN EXTINCTION 105 the sensory properties of the US, then the finding of rapid reacquisition is not inconsistent with earlier research showing that extinction fails to undermine sensory-specific associations (e.g., Delamater, 1996; Rescorla, 1996a). Several studies recently have used the CTA preparation to assess the putatively inhibitory status of an extinguished taste with retardation and summation tests for conditioned inhibition. Calton, Mitchell, and Schachtman (1996; also Schachtman, Threlkeld, & Meyer, 2000), for example, reported that extinction of a conditioned taste aversion resulted in slower reacquisition of the aversion than was observed in a group of subjects trained and extinguished on a different taste prior to the retardation test. As was true in the Bouton and Swartzentruber (1989) study, however, Aguado, de Brugada, and Hall (2001) demonstrated that a trained and extinguished group reacquired more rapidly than a latent inhibition control group. This result leaves one wondering whether extinction in this situation resulted in the stimulus acquiring net inhibition or more simply a form of latent inhibition. Calton et al. (1996) provided evidence for the net inhibition account by demonstrating that the trained and extinguished taste could also pass a summation test for inhibition (see Figure 3). The experimental group received saccharin–LiCl pairings followed by saccharin-alone extinction. Intake of a conditioned vinegar solution (the transfer excitor) was low in the summation test relative to intake of a solution consisting of vinegar and saccharin combined. Calton et al. used a control group that had an aversion conditioned to a coffee solution and that was then exposed to saccharin (when the experimental rats received saccharin-alone extinction). Intake of the vinegar + saccharin mixture in this control group was less than intake in the experimental group. However, interpretation of this result is complicated by the results from the Aguado et al. (2001) study that used essentially the same control group as Calton et al. (1996). Figure 3 presents a comparison of these two experimental designs. Aguado et al. found that their control group consumed less of the transfer excitor than did the extinguished Figure 3. Experimental designs from the Calton, Mitchell, and Schachtman (1996), and Aguado, de Brugada, and Hall (2001) studies. 106 DELAMATER group, a result that they interpreted in terms of generalization of an aversion from the nonextinguished taste to the transfer excitor in the control group. Intake of the transfer excitor alone in this control group was not assessed by Calton et al. In a final experiment, Aguado et al. (2001) controlled for this difference in conditioned strength to the transfer excitor and found that the control group consumed more of the compound mixture than did the extinguished group. This result is opposite to one anticipated on the basis of net inhibition accruing to the extinguished taste. In summary, the several conflicting results in the studies reported above highlight the need to exercise caution before accepting the controversial claim that extinction leaves a stimulus in a net inhibitory state. Moreover, as comparisons with a latent inhibition control make clear, the common practice of using a novel CS to assess retarded acquisition is certainly not a very conservative control procedure. B. What factors influence extinction? 1. Attentional processes in extinction The possibility that extinction is accompanied by decreases in attention to the CS has been recognized since Pavlov (1927) introduced the notion. However, it has not been until recently that experimental effort has been made to distinguish this mechanism from other potential sources of extinction. For instance, Robbins (1990) explicitly addressed the issue of whether extinction is best construed in terms of attentional decrements or the acquisition of conditioned inhibition to the CS during its nonreinforcement. He provided evidence favouring the attentional decrement view. However, an important feature of his experiment was that extinction occurred in a single session, and testing occurred at the very end of that session. Habituation-like processes (i.e., that could result in a decrease in attention to the stimulus) are likely to figure prominently when extinction is massed and performance is evaluated at the end of or soon after the session. It is unknown to what extent attentional processes play a role in studies in which extinction is more distributed across multiple sessions, and performance is measured in tests conducted in subsequent sessions. Kehoe and White (2002) recently suggested that attentional processes may be partly responsible for explaining within-session decreases and between-session increases in CRs (i.e., spontaneous recovery). One promising idea is that attentional decrements to a CS during extinction may provide a means to protect the CS from undergoing any associative loss that might otherwise occur with nonreinforcement. Subsequent restoration of attention to the CS, in turn, should restore the CS’s ability to control performance to fully functional levels. However, if extinction occurred at a time when attention to the CS is maintained, then perhaps the CS–US association would undergo an associative loss with nonreinforcement. This attractive idea, however, has not met with much success in studies designed to maintain attention to the CS during the extinction phase (e.g., Delamater, 1996; also Peck & Bouton, 1990). Thus, it seems unlikely that attentional factors explain why stimulus–outcome associations are preserved throughout extinction. Nevertheless, the observation that extinction can result in attentional decrements in some circumstances (e.g., with massed extinction sessions) suggests that there may be multiple consequences of nonreinforcernent and that different extinction procedures might differentially engage different mechanisms. PAVLOVIAN EXTINCTION 107 2. Associative decrements in extinction and US activation The studies reviewed above suggest that extinction does not undermine associations established between the CS and the sensory properties of the US. However, extinction does diminish the CR, suggesting the occurrence of an associative decrement of some sort. Above this effect was characterized in terms of extinction resulting in either (1) unlearning of the affective association, or (2) learning of an inhibitory stimulus–response association. But regardless of how one chooses to discuss the learned decrements in performance that do accompany extinction, a more general question concerns identifying the factors that promote these associative decrements in extinction. Several recent experiments, performed mainly by Rescorla (1999, 2000b, 2001c, see also Kehoe & White, 2002; Taylor & Boakes, 2002), suggest that the magnitude of the associative decrement produced by extinction is directly related to the level of activation of the US representation at the time of nonreinforcement (see also Mackintosh, 1974). This conclusion stems from four different sorts of experiment briefly reviewed below. Recently, Rescorla (2001c) examined whether the negatively accelerated extinction function reflects a negatively accelerated change in an underlying learning or performance process. He determined whether there was a greater decrement produced by nonreinforcement during the first few extinction trials than by the same amount of nonreinforcement occurring after many extinction trials. Initially, rats (or pigeons in another experiment) were trained with four CSs each paired with a food US (A+, B+, C+, D+). Two of these CSs (A–, C–) were then extinguished on each of 2 days. Subjects were eventually tested with AD and BC compound stimuli (on nonreinforced test trials). Prior to this test, however, Stimuli A and B received one session of extinction trials. If the decrements on A– and B– extinction trials in this session were equal, then no difference in responding to AD and BC should occur during the test (on the grounds that A received the same total number of extinction trials as B and C combined). Rescorla observed, however, less responding to BC than to AD, a result that fits with the idea that more of an associative decrement was produced by nonreinforcing B than by nonreinforcing A in the session prior to the test. Since A but not B had received extinction trials previously, then one can conclude that more of an associative decrement occurs during the early than during the latter trials of extinction. If one assumes that the level of activation of the US representation is reduced by extinction (an assumption that follows from Rescorla, 2001b), then the findings described above indicate that more of an associative decrement is produced when the US representation is more strongly activated (during early extinction trials) at the time of nonreinforcement. In another experiment, Rescorla (2000b) more directly manipulated the level of activation of the US representation at the time of nonreinforcement. In one condition, one auditory stimulus (A) and two visual stimuli (V1, V2) initially were each paired with a food pellet US (i.e., A+, V1+, V2+). In the extinction phase, one visual CS was extinguished in the presence of Stimulus A, and the other visual stimulus was extinguished by itself (AV1–, V2–). In a subsequent test session, V1 elicited less responding than V2. This result indicates that a greater associative decrement was produced by nonreinforcement of V1 (when the US representation was more strongly activated by the concurrent presence of A) than by nonreinforcement of V2. The generality of this result, however, is questioned by the fact that Pearce and Wilson (1991) ran essentially the same experiment and found the opposite result. 108 DELAMATER Rescorla’s experiment was a magazine approach study with rats while Pearce and Wilson’s study used an autoshaping preparation with pigeons. The critical difference here might well be related to whether the preparation is likely to generate an excitatory summation effect when two separately trained excitatory CSs are combined. This result is routinely observed in the rat appetitive conditioning preparation, though it is rarely seen in the pigeon autoshaping preparation. In any case, excitatory summation would seem to unambiguously suggest an increase in the level of activation of the US representation. Under these conditions, Rescorla (2000b) found that nonreinforcement resulted in a greater associative decrement than nonreinforcement of a stimulus presented alone. In another study of interest, Rescorla (1999) compared the impact of nonreinforcement of stimuli that were trained according to a partial or continuous reinforcement schedule. He observed that nonreinforcement of the continuously reinforced CS resulted in a greater associative decremental effect than nonreinforcement of the partially reinforced CS. One keylight was consistently paired with grain (A+) while a second keylight was paired with grain on 25% of the trials (B+/–). In a second phase, these stimuli continued to be trained as in the first phase but, in addition, two diffuse stimuli were introduced to form two nonreinforced stimulus compounds (AX– and BY–). Inhibitory learning to the X and Y stimuli then was assessed in summation tests in which these stimuli were combined with a third keylight that had earlier been paired with grain (on either 100% or 25% of the trials in different experiments). Stimulus X was more inhibitory on this measure. It seems reasonable to assume that the US representation is more strongly evoked by the continuously reinforced A stimulus than by the partially reinforced B, and so nonreinforcement of AX and BY would produce more inhibitory learning to X than Y. If so, then here is another case in which a greater associative decrement is produced by nonreinforcement in the presence of a more strongly activated US representation. A potential problem with this analysis, however, is provided by the final experiment in this report (Rescorla, 1999, Exp. 4). A blocking procedure was used to reduce the associative strength of the continuously reinforced CS in this study. During initial training, X → Y+ and Y+ trials were intermixed with Z+/– trials. Stimulus X, thus, was continuously reinforced, and Z was partially reinforced. However, because X was blocked by Stimulus Y responding to X was less than responding to Z throughout training. In a subsequent phase, additional stimuli accompanied X and Z on conditioned inhibition trials (forming AX– and BZ– compounds). In subsequent tests, A transferred its inhibitory control to a test excitor more than did B. If one were to take key peck responding as a measure of the associative strengths of X and Z, then Z’s associative strength was greater than X. This conclusion was supported by an additional choice test in which Z was chosen in preference to X when the two were presented side by side. A greater ability of X, then, to support conditioned inhibition to stimulus A would be problematic for the claim that larger associative decrements will result from nonreinforcement that accompanies a more strongly activated US representation. Further research will be necessary to determine whether there is a fundamental difference, other than US activation, between partial and continuous reinforcement procedures. One problem with interpreting these data, however, is that the serial blocking procedure may have resulted in significant inhibition of delay, effectively lowering response rates and making it difficult to use key peck responding (or choice) as a measure of US activation. PAVLOVIAN EXTINCTION 109 Finally, another experiment is relevant to the claim that greater associative decrements occur when nonreinforcement accompanies strongly activated US representations. Holland and Rescorla (1975; also Holland, 1981) used US devaluation procedures to assess the effect of reducing the US value upon the magnitude of the CR. In one study, rats were given tone–food pairings, and general activity CRs were measured. Rats for whom the US was devalued prior to an extinction test (either through food-rotation pairings or through satiation on food just prior to the extinction test) were less active to the tone throughout the test session than were rats for whom the US was not devalued. Of greater interest, however, is that the rats extinguished when the US was devalued were more active to the tone during a second extinction test conducted at a time when the US value had been restored. Thus, the associative decrement produced by the first extinction session seems less in the rats whose US value was reduced during that extinction session. In some way, the weakened US value at the time of the extinction session appears to have protected the CS from undergoing an associative decrement. This result is intriguing because it suggests a formal similarity between studies that have examined the roles of the activation levels and the value of the US representations in extinction. However, the role of context modulatory factors in the effect reported by Holland and Rescorla (1975) will need to be considered more thoroughly. On the basis of the renewal phenomenon (see below), for example, subjects extinguished while sated may be expected to recover more when tested deprived than subjects who were extinguished and tested deprived. One final point to make before leaving the topic of associative decrements accompanying nonreinforcement relates to some additional recent findings. While I have emphasized the importance of the level of activation of the US representation at the time of nonreinforcement, another factor that seems important is CS predictiveness. In a recent set of studies, Rescorla (2000a, 2001d) investigated the size of the associative decrement occurring to an individual stimulus when that stimulus is nonreinforced in compound with another stimulus. The interesting result, again found using both rat and pigeon appetitive procedures, is that the magnitude of the associative decrement occurring to a given stimulus depends upon how well nonreinforcement is predicted by the individual CS. For instance, in one study Rescorla (2000a) initially trained three stimuli as excitors for food and two stimuli as inhibitors for food (e.g., A+, C+, X+, XB–, XD–). Of interest was how much of an associative decrement would accrue to each stimulus of a nonreinforced compound that consisted of one excitor and one inhibitor (i.e., AB–). In a final test, more CRs occurred to the BC stimulus compound than to AD. This result suggests that a greater associative decrement occurred to Stimulus A than to Stimulus B during nonreinforcement of AB. The general conclusion here seems to be that if a stimulus is a poor predictor of the significant event on the trial (nonreinforcement in this case) then that stimulus will undergo a large associative change on that trial. This principle is radically different from those currently employed in most popular computational learning models (see Rescorla, 2000a, 2001d), and research of the sort reviewed in this section will undoubtedly play an important role in constraining future theoretical developments in this field. C. Contextual modulation in extinction Current research on extinction has revealed a role for the experimental context in the “modulation” of conditioned responding to a CS. For instance, it is well known that extinction is, in some manner, context-specific (for a brief survey, see Pearce & Bouton, 2001). Furthermore, a 110 DELAMATER strong case has also been made for the claim that learning that occurs during extinction is more context-specific than learning that occurs during acquisition (e.g., Bouton & Ricker, 1994). However, recent research has also indicated that under certain circumstances contexts can acquire the ability to modulate processes of acquisition as well as extinction (e.g., Hall & Honey, 1989; Harris, Jones, Bailey, & Westbrook, 2000). In many of these cases it appears that contexts can modulate responding to a CS in a manner that is unrelated to any simple associative relation that incidentally may have been established between the context and the US. In the study by Bouton and Ricker (1994), two CSs (X and Y) received fear conditioning in different contexts (e.g., Stimulus X in Context A and Stimulus Y in Context B). These CSs were then extinguished, each in its own conditioning context. Finally, X was tested in Contexts A and B. Greater renewal of conditioned fear was seen to X when it was tested in Context B (AAB renewal) than in Context A (AAA renewal). Since each context had a CS conditioned and then extinguished within it, an appeal to differences in context–US associations cannot easily apply to these results. Instead, one might argue that greater AAB renewal was the result of the acquisition memory more readily transferring across contexts than the extinction memory. If, for example, a CS-no US association is formed during extinction, and the retrieval of this association is contextually modulated, then this association may be difficult to retrieve when subjects are tested outside of the extinction context (see Bouton, 1993). An alternative interpretation of this result, however, is couched in terms of CS processing. For instance, the extinction context may form an association with the specific CS presented within it. One consequence of this is that processing of the CS should be reduced (e.g., Wagner, 1981) when the CS is presented in its associated context but not when the CS is presented in the alternate context. This sort of analysis could apply to some but not all of the data reported by Harris et al. (2000). Their first experiment demonstrated context-specific extinction using an ABB versus ABC renewal design (see Figure 4). Two stimuli (X and Y) were first subjected to fear conditioning in one context (Context 1). This was followed by an extinction phase in which each CS was extinguished in different contexts (X in Context 2 and Y in Context 3). Finally, in one group of rats one of the CSs was tested in the context in which it was extinguished (X in Context 2), and in the other group it was tested in the context in which the other CS was extinguished (X in Context 3). Thus, in the first group Stimulus X was conditioned in one context and extinguished and tested in a second context (i.e., ABB renewal), and in the second group X Figure 4. Experimental designs used by Harris, Jones, Bailey, and Westbrook (2000). PAVLOVIAN EXTINCTION 111 was conditioned, extinguished, and tested in separate contexts (i.e., ABC renewal). Conditioned freezing was greater in the ABC renewal group. It is noteworthy that Contexts 2 and 3 each had a CS extinguished in its presence. Thus, a difference between these groups in terms of any direct inhibitory context–US associations cannot easily be applied to the results (though caution is warranted by the common finding that conditioned inhibitors rarely transfer their inhibition completely to another CS, Rescorla, 1982). However, as is the case for AAB renewal an analysis of these data in terms of either contextual modulation of extinction or CS processing can readily apply. The CS processing account, however, has difficulty explaining other results. In another experiment, Harris et al. (2000) used an ABA versus ABC renewal design to demonstrate that acquisition memories can be context-specific. Fear conditioning to each of two CSs occurred in separate contexts (X in Context 1 and Y in Context 2). These CSs were then extinguished in Context 3. Both CSs were then tested in Context 1. Thus, Stimulus X had been conditioned in one context, extinguished in another, and tested in its acquisition context (ABA renewal), whereas Stimulus Y was conditioned in one context, extinguished in another, and tested in a third (ABC renewal). Greater renewal of conditioned freezing was seen to Stimulus X (i.e., greater ABA than ABC renewal). This result is inconsistent with the CS processing account as X, but not Y, was conditioned in the “A” context and consequently should have been processed less well than Stimulus Y in this context (by virtue of the context–X association). Thus, the results from both of these experiments are more parsimoniously understood in terms of some context modulation account. It may be appreciated that the CS processing account was initially applied to the finding of reduced renewal of responding when the CS was tested in its extinction context. In contrast, Harris et al. (2000) demonstrated contextual specificity in acquisition as well as extinction processes (see also Hall & Honey, 1989). It seems possible, though unlikely, that CS processing effects might be specific to extinction processes. Another intriguing, though controversial (e.g., Hall & Mondragon, 1998), aspect of this research is that still other results in the Harris et al. study (2000) suggest that acquisition memories are only context-specific when the CS has undergone extinction. In other words, a group of subjects lacking the extinction phase displayed equal levels of conditioned freezing when the CS was tested in its own conditioning context as well as when it was tested in the alternate context. It is as though when the CS undergoes extinction, the conditioning context retrospectively acquires the ability to modulate the acquisition memory (see Harris et al. for alternative interpretations). Further research will be required to better specify the mechanisms involved in demonstrating context-specific and context-independent acquisition memories. NEUROSCIENCE STUDIES A. Lesion experiments 1. Hippocampus and contextual modulation A number of studies have recently investigated the role of hippocampal and surrounding structures in acquisition, extinction, renewal, and reinstatement phenomena in Pavlovian conditioning. The literature contains many inconsistencies that make it difficult to assert with 112 DELAMATER any confidence what specific role the hippocampus has in these Pavlovian conditioning phenomena. Nevertheless, some emerging consensus can be appreciated. Stimulated by behavioural studies on contextual modulation, Bouton and his colleagues investigated the importance of the hippocampus in several extinction phenomena. Frohardt, Guarraci, and Bouton (2000; see also Wilson, Brooks, & Bouton, 1995) examined the effects of neurotoxic lesions of the hippocampus (produced by an ibotenic acid and NMDA mixture) on fear conditioning, extinction, reinstatement, and renewal in a conditioned suppression of bar pressing task. They observed that acquisition, extinction, and renewal of conditioned suppression (AAA versus ABA renewal) to a visual CS were not impaired by hippocampal lesions. However, tests of context-specific reinstatement revealed a deficit in lesioned rats. Prior to the reinstatement test, rats were given unsignalled shocks either in the same context used for acquisition, extinction, and test, or in a different, equally exposed context. Nonlesioned control subjects displayed more conditioned suppression to the CS when reinstating shocks were delivered in the same context in which testing took place than in the different context. Lesioned subjects, on the other hand, displayed equally low levels of conditioned suppression in the two groups. The authors interpreted these results by suggesting that hippocampal lesions impair the context’s ability to form simple associations with the US, while sparing the context’s ability to acquire a modulatory function. Since reinstatement is thought to depend on context–US associations, and renewal is thought to depend upon contextual modulation, this pattern of results can be understood from this perspective. In a conceptually similar study, however, Fox and Holland (1998) failed to find any evidence that ibotenic acid lesions of the hippocampus influenced reinstatement of conditioned magazine approach CRs. One important difference in the two studies is that Fox and Holland studied reinstatement in an appetitive task, whereas Frohardt et al. (2000) used an aversive task. Another difference is that Fox and Holland compared a group given unsignalled USs in the same context used for conditioning, extinction, and testing to a control group not given reinstating USs. Both lesioned and nonlesioned subjects alike displayed increased magazine approach to the CS in the reinstatement groups compared to the nonreinstatement controls. Thus, it is possible that had Fox and Holland included a context manipulation like Frohardt et al., the context-specific form of reinstatement would have been abolished in the appetitive task as well. More troubling for Frohardt et al.’s (2000) interpretation is Corcoran and Maren’s (2001) finding that renewal is sensitive to hippocampal inactivation. These investigators deactivated the dorsal hippocampus (with muscimol infusions) immediately prior to the renewal test session, whereas in the previously mentioned studies the hippocampus was lesioned prior to the start of the experiments. Corcoran and Maren compared ABC to ABB renewal of conditioned freezing. Figure 5 presents the experimental design. They observed that in control animals infused with saline prior to the test, there was more freezing to the auditory CS when it was tested outside of the extinction context (ABC renewal group) than when it was tested in the extinction context (ABB group). However, low levels of conditioned freezing were seen in both groups given muscimol infusions prior to the renewal test, as though there was good retention of extinction irrespective of test context in groups without a functioning hippocampus. Corcoran and Maren (2001) interpreted this pattern of data in terms of the dorsal hippocampus being important in establishing a modulatory function to the context. This PAVLOVIAN EXTINCTION Figure 5. 113 Experimental design from Corcoran and Maren (2001). interpretation is consistent with other data showing that hippocampal lesions or inactivation impair the contextual specificity of conditioning and of latent inhibition (Holt & Maren, 1999; Honey & Good, 1993). However, from this modulatory perspective it is not obvious why both hippocampus inactivated groups in the Corcoran and Maren study should have behaved as though the extinction memory was fully retrieved. An impairment in the context modulatory function may very well have rendered the extinction memory inaccessible, in which case we may have expected to see recovery of conditioned freezing in both groups instead of the lack of recovery in both groups. An alternative interpretation is based on the possibility that hippocampus inactivated rats failed to discriminate between the extinction and the test contexts. This account was rendered less plausible by Holt and Maren’s (1999) supplementary observation that inactivation of the dorsal hippocampus did not impair expression of a context-based discrimination in which only one of two contexts was associated with shock (see also Honey & Good, 1993). This objection notwithstanding, the Corcoran and Maren (2001) study is important in demonstrating a role for the hippocampus in renewal. This inconsistency with earlier studies was interpreted by these authors in terms of differences in the method by which the hippocampus was inactivated in the various studies. In the earlier studies, the hippocampus was lesioned prior to the experiment, and this could have allowed other structures to substitute for the role normally performed by the hippocampus. By deactivating the hippocampus just prior to the test, Corcoran and Maren may have prevented this reassignment of roles from taking place and, thus, may have illuminated more purely the functions ordinarily served by the hippocampus. There is, however, another difference between the studies. Whereas Corcoran and Maren studied ABC renewal, the earlier studies examined ABA renewal. As described above (Harris et al., 2000), these two procedures are likely to differ in the types of contextual control they engage. It seems possible, therefore, that hippocampal lesions may influence both types of contextual control function in different ways. Future research will be necessary to clarify these issues. 2. Prefrontal cortex Several studies have examined the involvement of the prefrontal cortex in extinction. Stimulated by earlier reports that extinction of conditioned freezing in rats is impaired by lesions of the ventromedial prefrontal cortex (Morgan & LeDoux, 1995; Morgan, Romanski, & LeDoux, 1993), more recent studies have suggested that this effect is highly specific anatomically (see also Morgan & LeDoux, 1999) and possibly pharmacologically. 114 DELAMATER Gewirtz, Falls, and Davis (1997) investigated the effects of ventromedial prefrontal cortex lesions upon the acquisition and extinction of conditioned freezing (measured by general activity) and fear-potentiated startle. Unlike the earlier reports, these investigators failed to find any effects of the lesions upon conditioning or extinction. Additional tests for spontaneous recovery and reinstatement were also performed in this report. There was no evidence that spontaneous recovery occurred, and there was only inconsistent evidence for a US reinstatement effect that was not influenced by the lesion. Gewirtz et al. (1997) reconciled the difference between their results and the earlier studies by suggesting that the lesions in the earlier studies may have resulted in increased asymptotic levels of conditioning that were concealed by a ceiling, but then unmasked by extinction testing. This analysis is plausible given the fact that the lesions were made before the conditioning phase in the studies by Morgan and his colleagues, but they were made after conditioning and before extinction testing in the Gewirtz et al. study. In a more recent report, Quirk, Russo, Barron, and Lebron (2000) suggested that the caudal infralimbic nucleus (IL) may play an especially important role in the effects of ventromedial prefrontal cortex (vmPFC) lesions on extinction. In this experiment, fear conditioning and extinction occurred in two sessions on a single day. Spontaneous recovery and US reinstatement tests were conducted a day later. Rats given sham lesions or electrolytic vmPFC lesions displayed equal acquisition and extinction of fear with both measures on the first day. However, there was significantly more spontaneous recovery seen in the first four trials in rats given vmPFC lesions whose IL nucleus was also significantly damaged than in sham lesioned controls and vmPFC lesioned subjects whose IL nucleus was not significantly damaged. All groups displayed equivalent reinstatement of conditioned freezing and conditioned suppression in a subsequent test. Quirk et al. (2000) interpreted this pattern of results to mean that normal within-session extinction does not depend upon a functioning vmPFC with IL nucleus, however; consolidation of within-session extinction memories does require such structures. Added support for this view comes from a recent demonstration of a negative correlation between spontaneous recovery to an extinguished fear-conditioned CS and changes in individual IL cell firing rates in response to the same CS (Milad & Quirk, 2002). These authors also demonstrated that weak electrical stimulation of individual IL neurons shortly after presentation of the CS undergoing extinction accelerated extinction of conditioned fear. Since the same weak electrical stimulation of IL cells failed to maintain instrumental bar pressing previously reinforced by food, these authors concluded that IL stimulation accelerated fear extinction because it simulated an extinction memory rather than by counterconditioning the fear CS. Other possible explanations that do not rest on the assumption that IL neurons are responsible for extinction memory consolidation may be noted, however. Since the studies by Quirk and his colleagues were concerned with spontaneous recovery, it is possible that the IL nucleus is involved in controlling other mechanisms thought to be important in this phenomenon. For instance, spontaneous recovery has also been discussed in terms of temporal contextual control (Bouton, 1991) and the involvement of attentional processes (Robbins, 1990). If IL nucleus lesions impaired either of these processes, then the present set of results may be understood without an appeal to the concept of memory consolidation. Additional research will be required to distinguish among such alternative accounts. Nevertheless, the present studies do suggest that the IL nucleus is an important structure in the extinction of conditioned fear. PAVLOVIAN EXTINCTION 115 In a rather novel approach to the study of the role of the mPFC in extinction of conditioned freezing, Herry and Garcia (2002) looked at the influence of long-term potentiation (LTP) and long-term depression (LTD) upon the maintenance of extinction in mice. Following an acquisition session in which there were several tone–shock pairings, the tone CS was extinguished in sessions that occurred shortly after LTP or LTD was induced in the mPFC by a train of high-frequency or low-frequency stimulation, respectively, of the mediodorsal thalamic nucleus (MD). Such stimulation is known to increase or decrease, respectively, mPFC field potentials in response to single pulse MD stimulation. Electrophysiological recordings in the mPFC of LTD-induced mice showed that during extinction tests there was reduced neural activity relative to baseline in response to a single pulse MD stimulation during the CS. This reduced neural activity returned to baseline levels in control animals lacking LTD stimulation, but was maintained throughout extinction testing in LTD subjects. Furthermore, whereas conditioned freezing decreased throughout extinction testing in nonLTD subjects, it was maintained at high levels in LTD subjects. A different pattern of results in this study was seen in mice extinguished following LTP induction. These mice displayed increased mPFC field potentials during a single session of CS extinction. In addition, conditioned freezing extinguished normally, and this loss of conditioned responding was retained over a 7-day extinction–test interval. Control subjects lacking the LTP treatment during the extinction phase displayed two types of response pattern. Some subjects displayed depressed mPFC activity during the extinction test, and these subjects showed spontaneous recovery over the 7-day extinction–test interval. Other control subjects, however, displayed little change in mPFC activity during nonreinforced CS trials, and like the LTP subjects they failed to show spontaneous recovery over the 7-day extinction–test interval. Perhaps this somewhat complex data set can best be summarized with the suggestion that new learning during extinction is more readily accomplished when mPFC cells are in a heightened, but not a depressed, state of activation. This conclusion, however, seems somewhat optimistic given the inconsistent patterns of results with the control subjects in the LTP study and given the likelihood that an LTD treatment might maintain conditioned freezing at high levels in part due to sensitization of this learned response. 3. Amygdala In spite of the large amount of attention directed to the amygdala in aversive conditioning, there have not been many studies examining the effects of lesions of this structure on extinction. A paper by Armony, Quirk, and LeDoux (1998) has produced some interesting findings regarding the impact of electrolytic lesions of the basolateral nucleus of the amygdala in rats upon extinction of conditioned freezing and conditioned enhanced single-unit firing patterns of cells located in temporal cortex (Te1, Te1v, & Te3). In this study, conditioning and extinction occurred on the same day. Initially, there was a baseline period in which the CS (2-s tone) and US (0.5-s, 0.5-mA shock) were presented unpaired. This was followed immediately by a period in which the two were paired, and this was followed after a 1-hour delay by an extinction period in which the CS was presented repeatedly without the US. As there was no evidence of conditioned freezing in lesioned rats it is not possible to determine whether the amygdala is important in the extinction of this response. However, the firing patterns of individual cells in the temporal cortex were altered by the conditioning protocol. Specifically, 116 DELAMATER some cells were found to be responsive immediately after CS onset, while other cells were responsive close to the time at which the US would normally occur. Cells showing early-onset sensitivity were responsive during the baseline (unpaired) phase and increased their responsivity during the conditioning phase. In nonlesioned, control rats these early-onset enhanced firing rates were unaffected by extinction, whereas in lesioned rats the firing patterns of these cells returned to baseline levels following the extinction phase. Cells showing enhanced firing rates late in the CS (near the point in time when the US would normally occur) could not be found in lesioned rats. It is tempting from this set of results to think of early-onset cells as coding enhanced attention to the stimulus, and late-onset cells as coding US expectancy (or some such construct). From this perspective, the decrease in early-onset responsiveness in lesioned rats could indicate decrements in attention to the CS as a result of extinction. Indeed, it would be impressive if dissociations even in normal animals could be found between these two neurophysiological measures. Unfortunately, it was not reported that late-onset cells in control rats showed any effect of extinction. One could imagine that a set of conditions could arise in which earlyonset, but not late-onset, responding may be influenced by extinction. Such a result would be consistent with findings discussed above that attention to a CS may wane during extinction, though the underlying CS–US association remains intact. However, more research would be required in order to make a strong case for the suggestion that these early- and late-onset responses reflect attention coding and US expectancy respectively. An additional complicating factor with this interpretation of the results from the Armony et al. (1998) study is that other research suggests that the central nucleus of the amygdala, but not the basolateral amygdala, participates in conditioning-enhanced attention to a CS in appetitive tasks (e.g., Holland & Gallagher, 1999). Of particular interest in connection with the Armony et al. data is the finding that electrolytic lesions of the central nucleus of the amygdala abolish conditioned increases in orienting to a visual CS paired repeatedly with food. The parallel between increased orienting to the CS in an appetitive task and increased early-onset cell responsiveness in the aversive conditioning task suggests that the central nucleus of the amygdala may be important in early-onset cell responsiveness as well. The implications of the appetitive conditioning data for extinction are less clear. Holland and Gallagher (1999) claim that the central nucleus of the amygdala is critical in mediating conditioned increases in attention, but not conditioned decrements in attention to a CS in various appetitive tasks. On the one hand, since the central nucleus appears to be related to attentional processing of CSs (at least in appetitive tasks), one may expect this structure to be important in extinction to the extent that changes in attention accompany extinction (e.g., Pearce & Hall, 1980). On the other hand, if decreases in attention accompany extinction (cf. Robbins, 1990), then the central nucleus of the amygdala is unlikely to play a role. B. Pharmacological manipulations 1. NMDA receptor studies A considerable amount of attention has been directed to the involvement of the NMDA receptor in extinction of aversive conditioning. The emerging consensus from this literature is that normal NMDA receptor functioning is required for normal neural plasticity to occur PAVLOVIAN EXTINCTION 117 during an extinction phase. However, while a fairly good case can be made for the importance of the NMDA receptor, the psychological mechanisms that may be involved are less clear. The general form of the experiments includes an acquisition phase followed by an extinction phase with or without some drug (e.g., an NMDA receptor antagonist or agonist), and finally a test phase without the drug. As several authors have noted, this general experimental design contains some potential problems. First, if the drug produces an internal state that has discriminative properties, then any extinction that may occur in the presence of this drug state may be specific to the drug state (so-called state-dependent learning). This could occur because, for example, the drug state acts as a different context and thus acquires the ability to modulate extinction learning much like explicit contexts do in the renewal phenomenon discussed above. Alternatively, the drug state may itself enter into an inhibitory association with the US due to the nonoccurrence of the US during the extinction phase in the presence of that drug state (Cox & Westbrook, 1994). Second, if the drug has direct effects upon how the CS is processed, then this could complicate any analysis of a drug effect on extinction. Third, if the drug has the effect of decreasing or increasing the level of activation of the US representation (evoked by the CS), then this could also complicate the analysis. For example, it was noted above that greater extinction occurs when the US representation is more strongly activated at the time of nonreinforcement. This conclusion was reached largely on the basis of behavioural studies using appetitive conditioning procedures. If similar effects occur in aversive conditioning preparations as well, then these possibilities are relevant to the interpretation of the results of several of the studies reviewed below. 1a. Effects of diminishing NMDA receptor functioning. In an extensive series of studies, Falls, Miserendino, and Davis (1992) examined the effects of infusions of the NMDA antagonist AP5 directly into the basolateral nucleus of the amygdala upon extinction of the fearpotentiated startle response in rats. This drug diminished the impact of extinction upon potentiated startle in a dose-dependent manner when measured in nondrug tests. Furthermore, the effect was anatomically and pharmacologically specific as (1) infusions of AP5 into the interpositus nucleus of the cerebellum had no effect, and (2) infusions of a non-NMDA glutamatergic antagonist into the amygdala also had no effect on extinction. However, note that this investigation cannot rule out an interpretation of the effects in terms of state dependency. Lu, Walker, and Davis (2001) provided a test for state dependency in the context of studying the impact of a mitogen-activated protein kinase (MAPK) inhibitor, PD98059, upon extinction of fear-potentiated startle. MAPK can be activated by NMDA receptor stimulation, and other work points to its involvement in acquisition of excitatory fear conditioning. In this study, fear conditioning to a visual CS was extinguished in subjects given either drug or control infusions during the extinction phase. A test for potentiated startle without infusions showed that the CS lost this ability in the control group but not in the drug-infused group. A second test was conducted under infusion conditions similar to the extinction conditions. Both groups displayed lower levels of the potentiated startle response in this test, a result interpreted in terms of some extinction occurring to the CS in both groups as a result of the first potentiated startle response test. However, because the CS in the drug-infused group still showed potentiated startle responding, whereas the non-drug group did not, the authors concluded that state-dependent learning was unlikely in this situation. It may be argued, 118 DELAMATER however, that in order to more effectively rule out a state-dependent learning interpretation a comparison between the two critical groups (one extinguished with the drug and one without the drug) should be made in a test where both are given the drug. Cox and Westbrook (1994) used a conditioned hypoalgesia preparation in rats to investigate the effects of the NMDA receptor antagonist, MK-801, upon extinction. This study included the more stringent controls for state dependency (see the design in Figure 6). All groups were initially placed on a hotplate apparatus whose temperature was set high. In a subsequent test, paw lick latencies were increased (conditioned hypoalgesia). However, subjects given extinction, which consisted of placing the subject on the hotplate repeatedly after it had been cooled down, failed to show this hypoalgesic response in a final test on the reheated hotplate. Their experimental design factorially combined type of (systemic) injection (saline or MK801) with the phase of the experiment (extinction or test) to form four groups. All groups were trained on the heated hotplate with saline injections. The test results showed that the two groups extinguished with saline displayed less conditioned hypoalgesia than the two groups extinguished with MK-801. A direct comparison between the two groups tested with MK-801 but extinguished with saline or MK-801 failed to reveal the result expected on the basis of a state-dependent learning analysis. The group extinguished with saline was less hypoalgesic than the group extinguished with MK-801. Exactly the opposite result should have been obtained if the context created by the MK-801 injection during extinction had acquired either a conditioned inhibitory or modulatory role. Baker and Azorlosa (1996) obtained essentially the same result using a conditioned lick suppression task. The final study in the Cox and Westbrook (1994) paper is also of interest. Subjects exposed for the first time to the heated hotplate were more hypoalgesic than subjects first familiarized with the hotplate unheated. Importantly, a group of rats injected with MK-801 during the familiarization phase also failed to show the hypoalgesic response characteristic of the nonfamiliarized subjects. This means that MK-801 did not interfere with normal processing of the contextual cues present during the familiarization phase. Thus, it seems unlikely that MK-801’s protective effects against extinction in this report were related to any reductions in CS processing (i.e., processing of the hotplate chamber). Similarly, an effect of MK-801 on US processing was examined in this report by comparing subjects injected with either MK-801 or saline prior to their first placement on the heated hotplate. Paw lick latencies were not different, suggesting that the drug did not diminish the magnitude of the unconditioned response as might be expected if it reduced US processing. However, it is important to note that the US activation account of the drug’s effect on extinction requires that the drug reduce activation of the US representation evoked by the CS at the time of extinction. This representation may or may not be directly tied to the unconditioned response. 1b. NMDA receptor antagonist effects on CS and/or US processing. A pair of additional studies monitored the effects of NMDA antagonists upon responding during the extinction phase as well as upon responding during a nondrug test session. The results of each of these studies are consistent with an effect of the drug on processing in either the CS or the US representation. Kehoe, Macrae, and Hutchinson (1996) used a rabbit eyeblink conditioning preparation to study the effects of systemic injections of MK-801 on extinction. They found that the drug dose-dependently decreased conditioned responding during the extinction phase, and Figure 6. Experimental design from Cox and Westbrook (1994). 119 120 DELAMATER that responding recovered in the nondrug test to a greater extent the more it had been suppressed by the drug during the extinction phase. The same pattern of results was obtained with discrete and context CSs by Lee and Kim (1998) who used a conditioned freezing preparation with rats and who administered the NMDA antagonist APV directly into the basolateral amygdala. It is noteworthy that although both of these results can be interpreted in terms of the drug reducing CS processing during the extinction phase, an effect on activation of the US representation is perhaps a more plausible explanation. First, as noted by Kehoe et al. (1996), MK801 fails to attenuate latent inhibition in the rabbit eyeblink preparation if the drug is given during the preexposure phase (Robinson, Port, & Stillwell, 1993). Moreover, Lee and Kim (1998) also reported that intra-amygdala infusions of APV reduced acquisition of conditioned freezing in rats (for a similar result with eyeblink conditioning in mice, see Takatsuki, Kawahara, Takehara, Kishimoto, & Kirino, 2001). These findings are consistent with the view that the drug reduces activation of the US representation that gets associated with the CS. Additional data support this conclusion by showing that MK-801 reduces the ability of a shock US to reinstate conditioned suppression to a previously extinguished CS (Johnson, Baker, & Azorlosa, 2000). However, caution is needed in interpreting this result because there were no controls for state-dependent effects in this report. 1c. Effects of enhancing NMDA receptor functioning. Another set of studies has demonstrated that manipulations of the NMDA receptor can facilitate extinction. Walker, Ressler, Lu, and Davis (2002) administered the NMDA agonist, DCS, systemically or into the basolateral amygdala, during extinction, and they assessed its effects in a postextinction potentiated startle test. Subjects extinguished with control injections (or infusions) displayed modest reductions in potentiated startle relative to a nonextinguished control group. However, groups extinguished with DCS showed no potentiated startle response to the CS in the postextinction test. Other data showed this effect to be dose dependent and reversible by an NMDA antagonist injected concurrently with DCS during the extinction phase. One general point to make here is that to the extent that one class of drugs—agonists—facilitates extinction, and another class of drugs—antagonists—retards extinction, such results cannot easily be understood in terms of state-dependent learning mechanisms. Of additional interest, though, is a possible interpretation of these data in terms of an effect of DCS upon activation of US representation during extinction. If DCS increased this level, then more learning during the extinction phase may be expected to occur in the group given the drug. This interpretation was rejected by Walker et al. (2002) on the grounds that there was no difference between two nonextinguished groups given a startle response test with DCS or saline. Thus, there was no direct support for the claim that DCS increased the activation level of the US representation evoked by the CS. Nevertheless, it remains possible that an effect on extinction may be a more sensitive assay of presumed differences in US activation levels than the potentiated startle measure in nonextinguished subjects. Tang, Wang, Feng, Kyin, and Tsien (2001) similarly found facilitated extinction of conditioned freezing in transgenic mice with enhanced NMDA receptor function (enhanced via the NR2B subunit transgene in forebrain neurons) compared to control mice. An important feature of this study, often overlooked, is that the transgenic mice also displayed higher levels of conditioned freezing on the first extinction test session, a result that could indicate a higher PAVLOVIAN EXTINCTION 121 degree of learning during the single CS–US pairing. If these data can be taken to mean that the effective US intensity is higher in transgenic mice, then an enhanced rate of extinction might be related to this putative difference in US effectiveness. 1d. NMDA receptors and extinction memory consolidation. One additional report has presented a set of intriguing findings regarding the impact of NMDA receptor drugs on memory for extinction. Santini, Muller, and Quirk (2001) used a conditioned suppression and conditioned freezing preparation to assess the effects of a systemically delivered NMDA antagonist (CPP) upon extinction and its recall (see Figure 7). These investigators paired a tone with shock and then presented tone-alone extinction trials later that same day. CPP or saline was injected in different groups of rats prior to the extinction trials. During extinction, subjects injected with CPP displayed less freezing and conditioned suppression although both groups extinguished at the same rate throughout this single extinction session. Of more interest was the finding that when subjects were brought back to the chamber and tested 24 hours later (without injections), subjects extinguished with CPP on the previous day displayed significantly more spontaneous recovery of conditioned freezing (nearly matching levels shown in nonextinguished control subjects). This result is reminiscent of the finding discussed above of greater spontaneous recovery in rats given prefrontal cortex lesions (Quirk et al., 2000). However, when subjects in a different experiment in the Santini et al. paper were Figure 7. Experimental design from Santini, Muller, and Quirk (2001). 122 DELAMATER tested 48 hours after extinction took place, subjects initially extinguished with CPP displayed equal retention of extinguished conditioned freezing. It is as though the extinction memory in CPP subjects required the extra day for long-term consolidation of the extinction memory to take place. This consolidation was blunted in another experiment, however, by injecting CPPextinguished subjects with CPP on the intervening day (i.e., 24 hours following extinction and 24 hours before the test). Such a treatment restored spontaneous recovery in the CPP-extinguished rats compared to those extinguished with saline injections. This set of findings is noteworthy in that any state-dependent learning interpretation would face difficulties. Similarly, although the basic observation of greater spontaneous recovery in subjects extinguished with the drug is consistent with the idea that the drug can protect the CS from extinction by decreasing activation of the US representation, this analysis has no ready way of explaining why spontaneous recovery should wane over time. Rather, the results speak to the involvement of long-term memory processes and suggest a way in which short- and long-term mechanisms of extinction might be dissociable. 2. Dopamine and GABA receptor studies There is growing interest in the use of pharmacological manipulations in the study of the involvement of dopamine and GABA in extinction, though there has not yet been much research on these topics. A pair of studies conducted by Willick and Kokkinidis (1995) and Borowski and Kokkinidis (1998) have investigated the effects of dopamine agonists (cocaine, amphetamine, and SKF 38393) upon extinction of conditioned fear-potentiated startle in rats. However, contrary to the authors’ claims, a contribution of state-dependent learning processes to these data has not been unambiguously ruled out. In the Willick and Kokkinidis study, for instance, an aversively conditioned visual CS extinguished following presession cocaine injections was shown to potentiate the startle response to a noise burst when tested without injections. In comparison, the potentiated startle response to the CS was extinguished completely in a control group that received saline injections prior to each extinction session. More critically, though, a group extinguished with cocaine and then tested with cocaine displayed a reduction in the size of the potentiated startle response (much like the control group) when comparing postacquisition to postextinction tests. Although a t test revealed this apparent decrease to be nonsignificant, the pattern of data was opposite to that obtained in the group extinguished with cocaine and tested without cocaine. The apparent interaction between these two groups was not evaluated statistically. On the basis of these results the authors concluded that state-dependent learning was an unlikely explanation of their data and so opted not to include tests for state dependency in their subsequent work with additional drugs (Borowski & Kokkinidis, 1998). Given that the pattern of results in this report was consistent with the state-dependent learning view, however, it would be prudent to examine this possibility more thoroughly. A role for dopamine neurons in mediating extinction of conditioned freezing was suggested by another recent study (Morrow, Elsworth, Rasmusson, & Roth, 1999). In this experiment, rats were given medial prefrontal cortex lesions with 6-hydroxydopamine (6OHDA),which has the effect of depleting the mPFC of DA neurons. Rats with such lesions acquired a conditioned freezing response to a tone CS normally, but displayed persistent PAVLOVIAN EXTINCTION 123 conditioned freezing to the tone in an extinction test session conducted the next day. In comparison, sham lesioned rats extinguished rapidly during this test session. This effect was seen in rats lesioned either before or after the conditioning phase. However, the effect of 6OHDA lesions was restricted to subjects trained with a high-intensity shock US (0.8 mA). Conditioning and extinction of conditioned freezing CRs in rats trained with a lower intensity shock US (0.4 mA) were not impaired by 6-OHDA lesions. This intensity-dependent effect makes it unlikely that 6-OHDA lesions produce their effect by rendering the animals generally more sensitive to the shock US, as this should have resulted in the animals trained at the weaker intensity US displaying faster acquisition. Thus, it appears that mPFC dopamine neurons may be involved in extinction processes at least when higher shock intensities are used during conditioning. Regarding the role of GABA in extinction, once again, the work of Westbrook and his colleagues is rather illuminating. Harris and Westbrook (1998) provided a set of results that fairly convincingly establishes a role for GABA in extinction of conditioned freezing. Having demonstrated that the GABA receptor inverse-agonist FG 7142 retards the loss of conditioned freezing to a CS undergoing extinction, and that this drug also reduces the expression of extinction of conditioned freezing, Harris and Westbrook then examined the influence of FG 7142 upon context-modulated extinction. Using an ABB versus ABC renewal design, Harris and Westbrook first trained a clicker with shock in one context (A) before extinguishing it in a second context (B) while also exposing subjects to a third context (C). The clicker was then tested in different groups of rats in either Context B or Context C. In rats injected with saline prior to the test, more recovery of conditioned freezing was seen to the clicker when it was tested in Context C than when it was tested in Context B. However, subjects injected with FG 7142 prior to the test displayed equal recovery of conditioned freezing to the clicker, independent of where it was tested. In other words, these subjects froze as much in the extinction context as they did when tested outside the extinction context. This result can be interpreted in terms of the drug deactivating the inhibitory GABA mechanisms engaged by extinction and thereby releasing the CS from the inhibitory effects of extinction. Another interpretation, however, is simply that the stimulus properties of the drug itself were detected as a powerful internal context change that overrode any effects of changing the external context. Alternatively, it is possible that anxiogenic effects of the drug may have increased fear to the CS in the group of rats tested in the extinction context with the drug relative to the group tested in the extinction context without the drug. Though plausible, these alternative interpretations were not supported by the results of a final experiment that failed to find a similar effect of the drug on the expression of latent inhibition. This study used the same experimental design as the extinction experiment, except that the order of the first two phases was reversed. In other words, subjects were first given nonreinforced exposures to the clicker in Context A and were also equally exposed to Context B. They then received clicker–shock pairings in Context C. The clicker was then tested in either Context A or Context B. This test session was preceded, in different groups, by an injection of saline or FG 7142. In this study, the drug failed to exert an effect during the test. Subjects injected with saline or FG 7142 froze less to the clicker when it was tested in Context A (the preexposure context) than in Context B. The results from this experiment together with the extinction study make it unlikely that the drug exerts its effect on extinction in a manner that is related to its internal stimulus or 124 DELAMATER anxiogenic properties. If so, then one would have expected similar effects in the latent inhibition and extinction studies. Given the data set, it is more plausible to suppose that extinction and latent inhibition phenomena differentially engage inhibitory mechanisms that are affected by GABA neurotransmitters. One qualification to this view comes from a study by McGaugh, Castellano, and Brioni (1990), who observed in a conditioned inactivity preparation with mice that administration of the GABA antagonist, picrotoxin, after the extinction session enhanced expression of extinction in a subsequent test. However, the results from this study are difficult to interpret because it is possible that the CS, the context, or the behaviour of the animal during the extinction session may have associated directly with the drug administered immediately after the extinction session. Thus, it is difficult to know whether the drug exerts its effects on an extinction learning process or by adding new learning of a different nature to that extinction learning process. Nevertheless, as suggested by Davis et al. (2000), it is possible that pre- and postextinction GABA manipulations have opposing effects on extinction learning mechanisms. 3. Protein synthesis studies Several recent studies have begun to investigate the effects of protein synthesis inhibitors upon learning during extinction. Other research has demonstrated a role for protein synthesis in acquisition and consolidation of CS–US associations (e.g., Nader, Schafe, & LeDoux, 2000), and this suggests that protein synthesis may also play a role in extinction. Though this idea is appealing, the empirical support for it to date is limited. In one experiment, Berman and Dudai (2001) demonstrated in rats that an insular cortex infusion of the protein synthesis inhibitor, anisomycin, just prior to (or shortly after) the first extinction trial, retarded extinction on subsequent trials of a taste aversion (previously established by a single saccharin–LiCl pairing). Unlike the shock-based aversive conditioning results noted above, the NMDA receptor antagonist, APV, and the MAPK inhibitor, PD98059, both had no effect on extinction in this study. However, the β-adrenergic receptor blocker, propranolol, did have a similar effect to anisomycin in retarding extinction of the taste aversion. These results cannot easily be interpreted, however. For example, the drugs may produce their effects either by (1) establishing an association between the taste and any putatively aversive properties of the drug, or by (2) any stimulus properties of the drugs exerting state-dependent control over extinction learning during the first trial. These possibilities were not examined. Lattal and Abel (2001) recently explored the effects of systemically administered anisomycin on the acquisition and extinction of context-shock associations in a conditioned freezing preparation with mice. These investigators found no effect of the drug on extinction, though it did interfere with acquisition of conditioned freezing when administered before the acquisition session. The drug was injected either prior to or following each of several extinction sessions. Subjects injected with anisomycin showed equivalent extinction of their freezing response and displayed equally low levels of freezing in a final test conducted without the drug as did control subjects (injected with saline). These authors concluded that the neural mechanisms underlying acquisition and extinction appear to differ, with protein synthesis being involved in the former but not the latter. However, Abel and Lattal (2001) also note that protein synthesis may be important early in extinction because other research has suggested PAVLOVIAN EXTINCTION 125 that anisomycin may interfere with consolidation of the previously learned CS–US association that would otherwise occur (Nader et al., 2000). This effect of the drug, though, appears to have more to do with consolidation than extinction learning processes, but could nonetheless complicate further analyses of extinction. SUMMARY AND CONCLUSIONS In recent years the study of extinction has led to a great deal of information concerning the basic psychological and neurophysiological bases of the phenomenon. Indeed, although these two different approaches to the study of extinction have for the most part developed independently, it is now a good time to examine these two literatures together and to determine how their respective findings might be mutually beneficial. There are, perhaps, two major impediments, however, in succeeding in this enterprise, but these should be in no way insurmountable. The first major difference between these two literatures is fairly obvious, given the review provided above. A significant amount of information regarding extinction in the behavioural studies comes from appetitive conditioning paradigms, whereas aversive conditioning preparations have been the model of choice to study the neural basis of extinction. So an obvious research area of growth would be to study the neurophysiological basis of extinction using appetitive preparations. One can only imagine that future growth in this area will lead to more insight into the functional organization of the brain as neuroscience-oriented studies are conducted that closely relate to existing behavioural findings and theory. Another major difference between the two literatures studying extinction, as I have depicted them above, is probably fairly obvious to the neuroscientist but less so to the behaviourally oriented psychologist. I am here referring to differences in the preferred level of analysis adopted by each group of scientists. Whereas psychologists and neuroscientists are, generally speaking, interested in studying basic mechanisms of a behavioural phenomenon like extinction, each type of scientist has a different view of what constitutes a basic “mechanism”. The associative theorist, for instance, is content with making statements about effects or lack thereof of a given treatment upon the underlying associative structure, with the understanding that the elements of that associative structure constitute the basic units of analysis. However, the level at which the neuroscientist operates is with underlying nervous system structures. The translation between psychological constructs and nervous system structures is not going to be easy to accomplish, but one may regard it to be an essential problem to cope with ultimately. These two levels of analysis sometimes complement one another quite well as is illustrated in the work on modulatory factors in extinction described above (e.g., Corcoran & Maren, 2001; Frohardt et al., 2000; Harris et al., 2000). However, sometimes these two levels of analysis seem to be at odds with one another. For example, in the studies investigating the role of NMDA receptors in extinction, I have emphasized the possibility that the results can be made sense of from the perspective of the NMDA antagonists or agonists influencing the activation level of the US representation. Less extinction may occur, for instance, when the drug decreases activation of the US representation during the extinction phase, but more extinction may occur should the drug have the opposite effect on US activation. This is clearly a psychological-level description of a pharmacological manipulation. A rather different approach, actually taken by many of the authors of these reports, is that NMDA receptor activation is 126 DELAMATER involved in the neural plasticity that mediates extinction. This position is agnostic with respect to what the nature of that neural plasticity is in psychological terms, but it surely represents a different way of capturing the data, which may, in fact, be accurate at the neural level. It would be difficult, however, to determine what implications, if any, such a description would have for a psychological theory of extinction other than to point out what the physical mechanisms are for instantiating an instance of learning. Nevertheless, I hope it will be appreciated that by emphasizing the psychological interpretation of these pharmacological manipulations, it will be clear that one will not be able without additional considerations to make firm claims about the effects of the manipulations on the functional aspects of neural plasticity. More concretely, before concluding, for instance, that NMDA antagonists interfere with neural plasticity in extinction, one would want to know that the drug did not exert its effect on behaviour by altering activation levels of the CS or US representations. Perhaps this will be an area where psychologists and neuroscientists together will begin to reach a better consensus on what level of analysis works best in describing the data. These two impediments to the integrative study of extinction notwithstanding, there is a wealth of interesting findings from the behavioural literature that beg for a neural analysis, and, likewise, there is a wealth of neuroscience techniques that psychologists should be able to use in order to further a psychological analysis. Among the chief behavioural results are the findings (1) that associations formed during acquisition and extinction can be modulated by context, (2) that CS–US associations specific in their sensory content are preserved in extinction, (3) that inhibitory stimulus–response associations can be learned in extinction, (4) that extinction, at least in appetitive task, is related to the activation level of the US representation during extinction, and (5) that attention can sometimes play a role in extinction. Among these, the role of context has received most attention in the neuroscience literature, though strong claims about the function of the hippocampus in extinction, for instance, seem premature (see above). This is very likely due to the almost exclusive use of aversive conditioning preparations in this literature. As more neuroscience research is devoted to the study of extinction with appetitive paradigms, then perhaps there will be greater convergence with some of the other major findings noted above. For example, perhaps brain structures will be discovered that code the formation of inhibitory stimulus–response associations during extinction, or that mediate changes in attention to the CS during extinction, or that mediate the effects of the activation level of the US representation on the decremental process in extinction. Furthermore, although extinction seems to have deleterious effects on overt performance, the underlying sensory-specific association seems fully preserved. It should be possible to find evidence for this sort of dissociation at a neural level of analysis. A number of results from the neuroscience literature on extinction may also have implications for future research at the psychological level. Most notably, it may be possible to make dissociations between psychological phenomena with neuroscience techniques that may not be possible with behavioural manipulations alone. One example of this comes from investigations of the role of the hippocampus in extinction. Bouton and his colleagues demonstrated that US reinstatement and renewal phenomena were differentially influenced by hippocampal lesions. Regardless of how one interprets what these results mean for the functioning of the hippocampus, they suggest, at the very least, that these two phenomena have important mechanistic differences, a conclusion that was not easily reached from behavioural studies alone (e.g., Bouton & King, 1986). Another example comes from studies of the role of the prefrontal cortex and of NMDA receptors in extinction. The work of Quirk and his colleagues (Milad & Quirk, 2002; Quirk et al., 2000; Santini et al., 2001) illustrates how memory consolidation (also Nader et al., 2000) may play a prominent role in extinction. Consolidation processes historically have not been very influential in associative theory, aside from those processes that occur at the time of a CS–US pairing (e.g., Wagner, 1981). However, the finding that such consolidation may occur in the absence of relevant stimulation one day after extinction (Quirk et al., 2000; Santini et al., 2001) should alert theorists to the possible importance of such memory mechanisms. There is a particularly interesting implication of the research on GABA for psychological analyses of extinction. For years, psychologists have distinguished between two types of inhibitory association in extinction (or in studies of conditioned inhibition more generally). Konorski (1948), for example, discussed the concept of conditioned inhibition in terms of inhibitory associations developing between the two neural centres corresponding to the CS and US representations, the so-called inhibitory CS–US link. Konorski (1967) later theorized that inhibition entailed the formation of new excitatory associations between neural centres that correspond to the CS and a no-US representation, the so-called excitatory CS–no-US link. It is difficult to imagine how one might distinguish in a given behavioural experiment between these two views. However, in principle, a neural analysis could allow such a distinction. Harris and Westbrook (1998) demonstrated that the GABAergic neurotransmitter system is important in extinction. Since these neurotransmitters exert an inhibitory influence on other neurons, it seems like a good candidate for instantiating the inhibitory CS–US link that Konorski first proposed. Before such a claim could be made with any confidence, however, additional research would have to establish that the neurons coding the US representation are themselves the target of these GABA-mediated effects. It should not seem so farfetched that such an analysis eventually will be possible. Finally, two other areas that offer promise in furthering a cross-fertilization in these two different, but related, disciplines include studies of attention and studies of the distinction between sensory and affective components of the association. Electrophysiological studies of extinction offer the promise that attentional factors in conditioning can be measured fairly directly (see Armony et al., 1998). One might be able to use such recordings not only as a means of locating attention circuits in the brain, but also as a way of manipulating attention to the CS. Since attention is commonly regarded as a factor that influences associative learning, this sort of control could be useful for investigations of extinction and learning more generally. In addition, a clearly understudied problem is the relationship between the effects of extinction upon associations with sensory and motivational components of the US. It will be difficult to study these types of association in isolation using a purely behavioural preparation, but, as noted above, various lesion techniques may be very helpful here (see Blundell et al., 2001). If, for example, basolateral amygdala lesions prevent associations from forming between the CS and the sensory, but not the motivational, components of the US, then this can be a powerful tool for investigating questions relating to the properties of these associations individually. Such an investigation may not be possible otherwise. In conclusion, the study of extinction continues to be a ripe area of research. Knowledge in this area has advanced considerably in recent years by, largely, independent investigations at behavioural and neuroscience levels. A number of interesting new facts have been discovered about extinction at each of these levels of analysis. What remains is the more challenging 128 DELAMATER prospect of furthering our understanding of extinction and related phenomena from a more integrative perspective (see also Myers & Davis, 2002). In this review, I have attempted to identify points at which such interaction would be of mutual benefit to both basic behavioural and neuroscientific levels of analysis. REFERENCES Abel, T., & Lattal, K. M. (2001). Molecular mechanisms of memory acquisition, consolidation and retrieval. Current Opinion in Neurobiology, 11, 180–187. Aguado, L., de Brugada, I., & Hall, G. (2001). Tests for inhibition after extinction of a conditioned stimulus in the flavour aversion procedure. Quarterly Journal of Experimental Psychology, 54B, 201–218. Armony, J. L., Quirk, G. J., & LeDoux, J. E. (1998). Differential effects of amygdala lesions on early and late plastic components of auditory cortex spike trains during fear conditioning. Journal of Neuroscience, 18, 2592–2601. Baker, J. D., & Azorlosa, J. L. (1996). The NMDA antagonist MK-801 blocks the extinction of Pavlovian fear conditioning. Behavioral Neuroscience, 110, 618–620. Berman, D. E., & Dudai, Y. (2001). Memory extinction, learning anew, and learning the new: Dissociations in the molecular machinery of learning in cortex. Science, 291, 2417–2419. Blundell, P., Hall, G., & Killcross, S. (2001). Lesions of the basolateral amygdala disrupt selective aspects of reinforcer representation in rats. Journal of Neuroscience, 21, 9018–9026. Borowski, T. B., & Kokkinidis, L. (1998). The effects of cocaine, amphetamine, and the dopamine D1 receptor agonist SKF 38393 on fear extinction as measured with potentiated startle: Implications for psychomotor stimulant psychosis. Behavioral Neuroscience, 112, 952–965. Bouton, M. E. (1986). Slow reacquisition following the extinction of conditioned suppression. Learning & Motivation, 17, 1–15. Bouton, M. E. (1991). Context and retrieval in extinction and in other examples of interference in simple associative learning. In L. Dachowski & C. F. Flaherty (Eds.), Current topics in animal learning: Brain emotion and cognition (pp. 25–53). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. Bouton, M. E. (1993). Context, time, and memory retrieval in the interference paradigms of Pavlovian learning. Psychological Bulletin, 114, 80–99. Bouton, M. E., & Bolles, R. C. (1985). Contexts, event-memories, and extinction. In P. D. Balsam & A. Tomie (Eds.), Context and learning. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. Bouton, M. E., & King, D. A. (1986). Effect of context on performance to conditioned stimuli with mixed histories of reinforcement and nonreinforcement. Journal of Experimental Psychology: Animal Behavior Processes, 12, 4–15. Bouton, M. E., & Ricker, S. T. (1994). Renewal of extinguished responding in a second context. Animal Learning & Behavior, 22, 317–324. Bouton, M. E., & Swartzentruber, D. (1989). Slow reacquisition following extinction: Context, encoding, and retrieval mechanisms. Journal of Experimental Psychology: Animal Behavior Processes, 15, 43–53. Calton, J. L., Mitchell, K. G., & Schachtman, T. R. (1996). Conditioned inhibition produced by extinction of a conditioned stimulus. Learning & Motivation, 27, 335–361. Colwill, R. M. (1991). Negative discriminative stimuli provide information about the identity of omitted responsecontingent outcomes. Animal Learning & Behavior, 19, 326–336. Corcoran, K. A., & Maren, S. (2001). Hippocampal inactivation disrupts contextual retrieval of fear memory after extinction. Journal of Neuroscience, 21, 1720–1726. Cox, J., & Westbrook, R. F. (1994). The NMDA receptor antagonist MK-801 blocks acquisition and extinction of conditioned hypoalgesic responses in the rat. Quarterly Journal of Experimental Psychology, 47B, 187–210. Davis, M., Falls, W. A., & Gewirtz, J. (2000). Neural systems involved in fear inhibition: Extinction and conditioned inhibition. In M. S. Myslobodsky & I. Weiner (Eds.), Contemporary issues in modeling psychopathology (pp. 113– 142). Boston, MA: Kluwer Academic Publishers. Delamater, A. R. (1995). Outcome-selective effects of intertrial reinforcement in a Pavlovian appetitive conditioning paradigm with rats. Animal Learning & Behavior, 23, 31–39. PAVLOVIAN EXTINCTION 129 Delamater, A. R. (1996). Effects of several extinction treatments upon the integrity of Pavlovian stimulus–outcome associations. Animal Learning & Behavior, 24, 437–449. Elizalde, G., & Sclafani, A. (1990). Flavor preferences conditioned by intragastric polycose infusions: A detailed analysis using an electronic esophagus preparation. Physiology & Behavior, 47, 63–77. Falls, W. A., Miserendino, M. J., & Davis, M. (1992). Extinction of fear-potentiated startle: Blockade by infusion of an NMDA antagonist into the amygdala. Journal of Neuroscience, 12, 854–863. Fox, G. D., & Holland, P. C. (1998). Neurotoxic hippocampal lesions fail to impair reinstatement of an appetitively conditioned response. Behavioral Neuroscience, 112, 255–260. Frohardt, R. J., Guarraci, F. A., & Bouton, M. E. (2000). The effects of neurotoxic hippocampal lesions on two effects of context after fear extinction. Behavioral Neuroscience, 114, 227–240. Gallistel, C. R., & Gibbon, J. (2000). Time, rate, and conditioning. Psychological Review, 107, 289–344. Gewirtz, J. C., Falls, W. A., & Davis, M. (1997). Normal conditioned inhibition and extinction of freezing and fearpotentiated startle following electrolytic lesions of medical prefrontal cortex in rats. Behavioral Neuroscience, 111, 712–726. Hall, G. (2002). Associative structures in Pavlovian and instrumental conditioning. In H. Pashler & R. Gallistel (Eds.), Steven’s handbook of experimental psychology; Vol 3: Learning, motivation, and emotion (3rd ed., pp. 1–45). New York: John Wiley & Sons, Inc. Hall, G., & Honey, R. C. (1989). Contextual effects in conditioning, latent inhibition, and habituation: Associative and retrieval functions of contextual cues. Journal of Experimental Psychology: Animal Behavior Processes, 15, 232– 241. Hall, G., & Mondragon, E. (1998). Contextual control as occasion setting. In N. A. Schmajuk & P. C. Holland (Eds.), Occasion setting: Associative learning and cognition in animals (pp. 199–222). Washington, DC: American Psychological Association. Harris, J. A., Jones, M. L., Bailey, G. K., & Westbrook, R. F. (2000). Contextual control over conditioned responding in an extinction paradigm. Journal of Experimental Psychology: Animal Behavior Processes, 26, 174–185. Harris, J. A., & Westbrook, R. F. (1998). Evidence that GABA transmission mediates context-specific extinction of learned fear. Psychopharmacology, 140, 105–115. Hart, J. A., Bourne, M. J., & Schachtman, T. R. (1995). Slow reacquisition of a conditioned taste aversion. Animal Learning & Behavior, 23, 297–303. Haselgrove, M., & Pearce, J. M. (2003). Facilitation of extinction by an increase or a decrease in trial duration. Journal of Experimental Psychology: Animal Behavior Processes, 29, 153–166. Herry, C., & Garcia, R. (2002). Prefrontal cortex long-term potentiation, but not longterm depression, is associated with the maintenance of extinction of learned fear in mice. Journal of Neuroscience, 22, 577–583. Holland, P. C. (1981). The effects of satiation after first- and second-order appetitive conditioning in rats. Pavlovian Journal of Biological Science, 16, 18–24. Holland, P. C., & Gallagher, M. (1999). Amygdala circuitry in attentional and representational processes. Trends in Cognitive Science, 3, 65–73. Holland, P. C., & Rescorla, R. A. (1975). The effect of two ways of devaluing the unconditioned stimulus after firstand second-order appetitive conditioning. Journal of Experimental Psychology: Animal Behavior Processes, 1, 355– 363. Holt, W., & Maren, S. (1999). Muscimol inactivation of the dorsal hippocampus impairs contextual retrieval of fear memory. Journal of Neuroscience, 19, 9054–9062. Honey, R. C., & Good, M. (1993). Selective hippocampal lesions abolish the contextual specificity of latent inhibition and conditioning. Behavioral Neuroscience, 107, 23–33. Johnson, D. M., Baker, J. D., & Azorlosa, J. L. (2000). Acquisition, extinction, and reinstatement of Pavlovian fear conditioning: The roles of the NMDA receptor and nitric oxide. Brain Research, 857, 66–70. Kehoe, E. J., Macrae, M., & Hutchinson, C. L. (1996). MK-801 protects conditioned responses from extinction in the rabbit nictitating membrane preparation. Psychobiology, 24, 127–135. Kehoe, E. J., & White, N. E. (2002). Extinction revisited: Similarities between extinction and reductions in US intensity in classical conditioning of the rabbit’s nictitating membrane response. Animal Learning & Behavior, 30, 96–111. 130 DELAMATER Konorski, J. (1948). Conditioned reflexes and neuron organization. New York: Cambridge University Press. Konorski, J. (1967). Integrative activity of the brain. Chicago: University of Chicago Press. Lattal, K. M., & Abel, T. (2001). Different requirements for protein synthesis in acquisition and extinction of spatial preferences and context-evoked fear. Journal of Neuroscience, 21, 5773–5780. Lee, H., & Kim, J. J. (1998). Amygdalar NMDA receptors are critical for new fear learning in previously fear–conditioned rats. Journal of Neuroscience, 18, 8444–8454. Lu, K. T., Walker, D. L., & Davis, M. (2001). Mitogen-activated protein kinase cascade in the basolateral nucleus of amygdala is involved in extinction of fear-potentiated startle. Journal of Neuroscience, 21, RC 162. Mackintosh, N. J. (1974). The psychology of animal learning. London: Academic Press. McGaugh, J. L., Castellano, C., & Brioni, J. (1990). Picrotoxin enhances latent extinction of conditioned fear. Behavioral Neuroscience, 104, 264–267. Milad, M. R., & Quirk, G. J. (2002). Neurons in medial prefrontal cortex signal memory for fear extinction. Nature, 420, 70–74. Morgan, M. A., & LeDoux, J. E. (1995). Differential contribution of dorsal and ventral medial prefrontal cortex to the acquisition and extinction of conditioned fear in rats. Behavioral Neuroscience, 109, 681–688. Morgan, M. A., & LeDoux, J. E. (1999). Contribution of ventrolateral prefrontal cortex to the acquisition and extinction of conditioned fear in rats. Neurobiology of Learning and Memory, 73, 244–251. Morgan, M. A., Romanski, L. M., & LeDoux, J. E. (1993). Extinction of emotional learning: Contribution of medial prefrontal cortex. Neuroscience Letters, 163, 109–113. Morrow, B. A., Elsworth, J. D., Rasmusson, A. M., & Roth, R. H. (1999). The role of mesoprefrontal dopamine neurons in the acquisition and expression of conditioned fear in the rat. Neuroscience, 92, 553–564. Myers, K. M., & Davis, M. (2002). Behavioral and neural analysis of extinction. Neuron, 36, 567–584. Nader, K., Schafe, G. E., & LeDoux, J. E. (2000). Fear memories require protein synthesis in the amygdala for reconsolidation after retrieval. Nature, 406, 722–726. Napier, R. M., Macrae, M., & Kehoe, E. J. (1992). Rapid reacquisition in conditioning of the rabbit’s nictitating membrane response. Journal of Experimental Psychology: Animal Behavior Processes, 18, 182–192. Ohyama, T., Gibbon, J., Deich, J. D., & Balsam, P. D. (1999). Temporal control during maintenance and extinction of conditioned keypecking in ring doves. Animal Learning & Behavior, 27, 89–98. Pavlov, I. P. (1927). Conditioned reflexes. Oxford: Oxford University Press. Pearce, J. M., & Bouton, M. E. (2001). Theories of associative learning in animals. Annual Review of Psychology, 52, 111–139. Pearce, J. M., & Hall, G. (1980). A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychological Review, 87, 532–552. Pearce, J. M., & Wilson, P. N. (1991). Effects of extinction with a compound conditioned stimulus. Journal of Experimental Psychology: Animal Behavior Processes, 17, 151–162. Peck, C. A., & Bouton, M. E. (1990). Context and performance in aversive-to-appetitive and appetitive-to-aversive transfer. Learning & Motivation, 21, 1–31. Perez, C., Ackroff, K., & Sclafani, A. (1996). Carbohydrate- and protein-conditioned flavor preferences: Effects of nutrient payloads. Physiology & Behavior, 59, 467–474. Perez, C., Lucas, F., & Sclafani, A. (1998). Increased flavor acceptance and preference conditioned by the postingestive actions of glucose. Physiology & Behavior, 64, 483–492. Quirk, G. J., Russo, G. K., Barron, J. L., & Lebron, K. (2000). The role of ventromedial prefrontal cortex in the recovery of extinguished fear. Journal of Neuroscience, 20, 6225–6231. Rauhut, A. S., Thomas, B. L., & Ayres, J. J. (2001). Treatments that weaken Pavlovian conditioned fear and thwart its renewal in rats: Implications for treating human phobias. Journal of Experimental Psychology: Animal Behavior Processes, 27, 99–114. Rescorla, R. A. (1982). Some consequences of associations between the excitor and the inhibitor in a conditioned inhibition paradigm. Journal of Experimental Psychology: Animal Behavior Processes, 8, 288–298. Rescorla, R. A. (1993). Inhibitory associations between S and R in extinction. Animal Learning & Behavior, 21, 327– 336. PAVLOVIAN EXTINCTION 131 Rescorla, R. A. (1996a). Preservation of Pavlovian associations through extinction. Quarterly Journal of Experimental Psychology, 49B, 245–258. Rescorla, R. A. (1996b). Spontaneous recovery after training with multiple outcomes. Animal Learning & Behavior, 24, 11–18. Rescorla, R. A. (1997a). Response-inhibition in extinction. Quarterly Journal of Experimental Psychology, 50B, 238– 252. Rescorla, R. A. (1997b). Spontaneous recovery of instrumental discriminative responding. Animal Learning & Behavior, 25, 485–497. Rescorla, R. A. (1997c). Spontaneous recovery after Pavlovian conditioning with multiple outcomes. Animal Learning & Behavior, 25, 99–107. Rescorla, R. A. (1999). Partial reinforcement reduces the associative change produced by nonreinforcement. Journal of Experimental Psychology: Animal Behavior Processes, 25, 403–414. Rescorla, R. A. (2000a). Associative changes in excitors and inhibitors differ when they are conditioned in compound. Journal of Experimental Psychology: Animal Behavior Processes, 26, 428–438. Rescorla, R. A. (2000b). Extinction can be enhanced by a concurrent excitor. Journal of Experimental Psychology: Animal Behavior Processes, 26, 251–260. Rescorla, R. A. (2001c). Are associative changes in acquisition and extinction negatively accelerated? Journal of Experimental Psychology: Animal Behavior Processes, 27, 307–315. Rescorla, R. A. (2001a). Experimental extinction. In R. R. Mowrer & S. B. Klein (Eds.), Handbook of contemporary learning theories (pp. 119–154). Mahwah, NJ: Lawrence Erlbaum Associates, Inc. Rescorla, R. A. (2001b). Retraining of extinguished Pavlovian stimuli. Journal of Experimental Psychology: Animal Behavior Processes, 7, 115–124. Rescorla, R. A. (2001d). Unequal associative changes when excitors and neutral stimuli arc conditioned in compound. Quarterly Journal of Experimental Psychology, 54B, 53–68. Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. H. P. Black & W. F. Prokasy (Eds.), Classical conditioning II: Current research and theory (pp. 64–99). New York: Appleton-Century-Crofts. Ricker, S. T., & Bouton, M. E. (1996). Reacquisition following extinction in appetitive conditioning. Animal Learning & Behavior, 24, 423–436. Robbins, S. J. (1990). Mechanisms underlying spontaneous recovery in autoshaping. Journal of Experimental Psychology: Animal Behavior Processes, 16, 235–249. Robinson, G. B., Port, R. L., & Stillwell, E. J. (1993). Latent inhibition of the classically conditioned rabbit nictitating membrane response is unaffected by the NMDA antagonist MK 801. Psychobiology 21, 120–124. Santini, E., Muller, R. U., & Quirk, G. J. (2001). Consolidation of extinction learning involves transfer from NMDAindependent to NMDA-dependent memory. Journal of Neuroscience, 21, 9009–9017. Schachtman, T. R., Threlkeld, R., & Meyer, K. (2000). Retention of conditioned inhibition produced by extinction. Learning & Motivation, 31, 283–300. Takatsuki, K., Kawahara, S., Takehara, K., Kishimoto, Y., & Kirino, Y. (2001). Effects of the noncompetitive NMDA receptor antagonist MK-801 on classical eyeblink conditioning in mice. Neuropharmacology, 41, 618– 628. Tang, Y.-P., Wang, H., Feng, R., Kyin, M., & Tsien, J. Z. (2001). Differential effects of enrichment on learning and memory function in NR2B transgenic mice. Neuropharmacology, 41, 779–790. Taylor, K. M., & Boakes, R. A. (2002). Extinction of conditioned taste aversions: Effects of concentration and overshadowing. Quarterly Journal of Experimental Psychology, 55B, 213–240. Wagner, A. R. (1981). SOP: A model of automatic memory processing in animal behavior. In N. E. M. Spear & R. R. Miller (Eds.), Information processing in animals: Memory mechanisms. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. Wagner, A. R., & Brandon, S. E. (1989). Evolution of a structured connectionist model of Pavlovian conditioning (AESOP). In S. B. Klein & R. R. Mowrer (Eds.), Contemporary learning theories: Pavlovian conditioning and the status of traditional learning theory (pp. 149–189). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. 132 DELAMATER Walker, D. L., Ressler, K. J., Lu, K. T., & Davis, M. (2002). Facilitation of conditioned fear extinction by systemic administration or intra-amygdala infusions of D-cycloserine as assessed with fear-potentiated startle in rats. Journal of Neuroscience, 22, 2343–2351. Willick, M. L., & Kokkinidis, L. (1995). Cocaine enhances the expression of fear-potentiated startle: evaluation of state-dependent extinction and the shock-sensitization of acoustic startle. Behavioral Neuroscience, 109, 929–938. Wilson, A., Brooks, D. C., & Bouton, M. E. (1995). The role of the rat hippocampal system in several effects of context in extinction. Behavioral Neuroscience, 109, 828–836. Original manuscript received 25 June 2002 Accepted revision received 28 March 2003