* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download computational and in vitro studies of persistent activity
Neural engineering wikipedia , lookup
Executive functions wikipedia , lookup
Apical dendrite wikipedia , lookup
Persistent vegetative state wikipedia , lookup
Emotion and memory wikipedia , lookup
Neuroplasticity wikipedia , lookup
Cognitive neuroscience of music wikipedia , lookup
State-dependent memory wikipedia , lookup
Electrophysiology wikipedia , lookup
Synaptogenesis wikipedia , lookup
Biology of depression wikipedia , lookup
Eyeblink conditioning wikipedia , lookup
Neurotransmitter wikipedia , lookup
Convolutional neural network wikipedia , lookup
Multielectrode array wikipedia , lookup
Neural modeling fields wikipedia , lookup
Neuroeconomics wikipedia , lookup
Molecular neuroscience wikipedia , lookup
Neural coding wikipedia , lookup
Sparse distributed memory wikipedia , lookup
Epigenetics in learning and memory wikipedia , lookup
Feature detection (nervous system) wikipedia , lookup
Channelrhodopsin wikipedia , lookup
De novo protein synthesis theory of memory formation wikipedia , lookup
Central pattern generator wikipedia , lookup
Chemical synapse wikipedia , lookup
Biological neuron model wikipedia , lookup
Premovement neuronal activity wikipedia , lookup
Development of the nervous system wikipedia , lookup
Types of artificial neural networks wikipedia , lookup
Holonomic brain theory wikipedia , lookup
Nonsynaptic plasticity wikipedia , lookup
Recurrent neural network wikipedia , lookup
Neural correlates of consciousness wikipedia , lookup
Activity-dependent plasticity wikipedia , lookup
Optogenetics wikipedia , lookup
Spike-and-wave wikipedia , lookup
Metastability in the brain wikipedia , lookup
Neural oscillation wikipedia , lookup
Pre-Bötzinger complex wikipedia , lookup
Neuropsychopharmacology wikipedia , lookup
Neuroscience 139 (2006) 135–151 COMPUTATIONAL AND IN VITRO STUDIES OF PERSISTENT ACTIVITY: EDGING TOWARDS CELLULAR AND SYNAPTIC MECHANISMS OF WORKING MEMORY ALBERT COMPTE* ical constraints. Within the attractor framework, a memorycapable system (for instance, a prefrontal neuron, or circuit) is essentially a multistable system: one system that can remain stably in a collection of possible states (for instance firing at 5 Hz, or firing at 25 Hz). These stable states are called fixed-point attractors because small perturbations to the system, while it is in these states, are followed by restorative forces that bring the system back to them. Whether the system settles in one or the other attractor depends uniquely on where external influences (for instance sensory information) transiently set the system to be, and therefore the readout of this attractor is indicative of the nature of those extinct external influences: the system remembers previous external information. Thus, in a working memory task the brain is conceptually a multistable system being switched by the sensory cues across various stable “states,” or attractors, that are univocally associated with the sensory stimulus parameters, or a categorization thereof. This idea is schematically represented in Fig. 1, where a ball representing the stimulus falls onto a surface or energy landscape, which represents the dynamical properties of our system. The ball hits the surface and then it rolls downhill until it settles in a stable configuration (attractor). If at that point we were to guess, just from that final configuration, the original location of the ball, we would make more or less accurate guesses only if our surface were of the multistable kind (Fig. 1C–E). A monostable system as in Fig. 1B cannot serve a mnemonic purpose. The conceptualization in terms of attractor dynamics spurs the distinction between two groups of persistent activity systems, one where the memories being encoded are discrete and one where the attractor states form a continuum, a “line attractor.” In Fig. 1 panels C and D would correspond to discrete persistent activity systems: the item encoded comes from a finite family of possible memories. If a stimulus is presented to the system that does not correspond to any of the attractors, the system will enter the attractor that most closely matches the stimulus, thus performing a categorization of the stimuli (panel C in Fig. 1, for instance, classifies any possible location along the bar above into three classes: left, middle, right). In contrast, panel E in Fig. 1 corresponds to a line attractor: any possible stimulus is faithfully kept in the system’s attractor memory. As it turns out, discrete and continuous attractor models of persistent activity share many cellular and synaptic mechanisms, but continuous models have some more stringent requirements regarding stability and robustness (intuitively, panel E in Fig. 1 needs to be per- Instituto de Neurociencias de Alicante, Universidad Miguel HernándezConsejo Superior de Investigaciones Científicas, 03550 Sant Joan d’Alacant, Spain Abstract—Persistent neural activity selective to features of an extinct stimulus has been identified as the neural correlate of working memory processes. The precise nature of the physiological substrate for this self-sustained activity is still unknown. In the last few years, this problem has gathered experimental together with computational neuroscientists in a quest to identify the cellular and network mechanisms involved. I introduce here the attractor theory framework within which current persistent activity computational models are built, and I then review the main physiological mechanisms that have been linked thereby to persistent activity and working memory. Open computational and physiological issues with these models are discussed, together with their potential experimental validation in current in vitro models of persistent activity. © 2005 Published by Elsevier Ltd on behalf of IBRO. Key words: attractor, reverberation, slice, NMDA, inhibition, network. Sustained neuronal firing selective to a stimulus feature no longer present in the environment is thought to be the neural correlate of working memory and it is known as persistent activity. It has been observed in many cortical areas of awake monkeys performing working memory tasks (Fuster, 1995; Goldman-Rakic, 1995). Apart from its implication in working memory, persistent activity has also been observed in the oculomotor system (for a review, see Delgado-Garcia, 2000) and in the head-direction system (for a review, see Taube and Bassett, 2003) in vertebrates. Thus, persistent activity is probably a general computational strategy developed by the nervous system for all those situations where information relevant for the organism but no longer available to the senses needs to be kept over short periods of time for immediate access and evaluation (for a review, see Major and Tank, 2004). Persistent activity has gathered great interest from the computational neuroscience community because it lends itself to a rigorous and rich conceptualization within the mathematical framework of dynamical systems theory, but it poses very significant challenges when imposing biolog*Tel: ⫹34 965919210; fax: ⫹34 965919561. E-mail address: [email protected] (A. Compte). Abbreviation: AMPAR, AMPA receptor; NMDAR, NMDA receptor; ACSF, artificial cerebro-spinal fluid; PFC, prefrontal cortex; VTA, ventral tegmental area. 0306-4522/06$30.00⫹0.00 © 2005 Published by Elsevier Ltd on behalf of IBRO. doi:10.1016/j.neuroscience.2005.06.011 135 136 A. Compte / Neuroscience 139 (2006) 135–151 Fig. 1. A toy, conceptual model is used to illustrate the relation between attractors, multistability and memories. In this model a ball falls onto a hard surface (one of panels B–E) from a continuum of possible locations and finds a gravitationally stable configuration on the surface. The surface or energy landscape summarizes all the dynamical and structural properties of the system. The surface B presents a monostable configuration, so that the ball always ends up in the same location irrespective of where it started. This particular case shows that monostable systems are not useful to encode a memory, because there is no way that one can make an accurate guess about the location the ball fell from just by looking at its steady state on the surface. Contrasting with this situation, panels C–E show multistable systems with three, nine and a continuum of attractors, respectively. Any of them allows one to “recall” more or less accurately the original position of the ball. Discrete attractors (C, D) perform a categorization of the stimulus, according to each attractor’s basin of attraction (range of locations that fall onto one same attractor). Panel C, for instance, categorizes the whole spectrum in A in just three classes: left, middle, right. Line attractors or continuous attractors (panel E), in contrast, retain faithfully the properties of the memorandum but have critical robustness issues (see Sec. 2.2). fectly flat: any deviation from horizontality, albeit small, will destroy the line attractor and memories will only be kept faithfully in time scales of the order of the network dynamics). Most possibly, this distinction between continuous and discrete persistent activity models, is also relevant in working memory. There are working memory systems that are presumably discrete in nature: like working memory for faces, or objects, or numerical digits and there are other systems that could possibly have a continuous nature: working memory for space locations (Constantinidis and Wang, 2004), or working memory for the frequency of vibrations (Romo et al., 1999). Note, however, that it might be very difficult to distinguish experimentally between closely spaced discrete memories and a true continuum (see Sec. 2.2). Still, they are distinct theoretical models and I will refer to object working memory models (see Fig. 2A, discrete attractors are reviewed by Brunel, 2003), to spatial working memory models line attractors in space or “bump attractors,” (see Fig. 3A, reviewed by Wang, 2001; Constantinidis and Wang, 2004) and to parametric working memory models (line attractors in the rate of firing, see Seung, 1996; Seung et al., 2000a; Miller et al., 2003). Now, where does the multistable system actually reside in the brain? Is it in the synapses? In the neurons? In the local cortical circuit? In highly distributed brain area networks? These questions are still a matter of intense research (Wang, 2001; Major and Tank, 2004), and computational models have helped to establish the mechanisms of both intrinsic and circuital mechanisms for persistent activity. Computational models have thus taken up either the intrinsic or the network mechanism for persistent activity generation and it is customary to confront persistent activity models based on intrinsic cellular mechanisms with persistent activity models that rely on network recurrent interactions through reverberation (Durstewitz et al., 2000b; Major and Tank, 2004). This review concentrates on cellular and synaptic mechanisms in recurrent network models, because they have been the subject of most computational research in this field so far. However, models that invoke intrinsic cellular mechanisms are now catching renewed attention (see, for instance Camperi and Wang, 1998; Loewenstein and Sompolinsky, 2003), especially after the experimental observation of intrinsic cellular multistability in cortical neurons (Egorov et al., 2002). The question that these biophysical models—whether discrete or continuous, intrinsic or recurrent—aim to answer is how the multistable property of the model is obtained in a biological situation by means of the known physiological mechanisms: synaptic receptors, intrinsic neuronal channels, proportions of excitation and inhibition, plasticity mechanisms, etc. I review in the following section some of these mechanisms and the situation in which they have been implicated by biophysical computational models of persistent activity in working memory. Recent revision articles include more specific accounts of some of these findings: a first general overview of persistent activity models was made by Durstewitz et al. (2000b); the attractor formalism of object working memory is reviewed by Brunel (2003); reverberating synaptic mechanisms are reviewed by Wang (2001); inhibition mechanisms are reviewed by Amit and Mongillo (2003a); mechanisms for line attractors by Brody et al. (2003), by Constantinidis and Wang (2004) and by Major and Tank (2004); mechanisms for dynamical stability of persistent activity by Tegner et al. (2002). Beyond the scope of this review lie some very active and related areas of research involving biophysically-based modeling accounts of working memory, like neuromodulatory effects on persistent activity circuits (see also Tanaka in this issue; Durstewitz et al., 2000a; Brunel and Wang, 2001; Durstewitz and Seamans, 2002; Navarro-Lopez et A. Compte / Neuroscience 139 (2006) 135–151 137 Fig. 2. Excitatory and inhibitory mechanisms in an attractor network model for object working memory. (A) Example of network performance during a trial. The network consists of one pool of inhibitory neurons (upper rasters, labeled I) and six pools of excitatory neurons, five of them selective to different stimuli (rasters labeled 1 through 5) and one of unselective neurons (unlabeled rasters). Within selective populations, neurons are coupled with potentiated synapses relative to synaptic couplings among cells from different populations. Because of this, when a brief stimulus (labeled “Sample”) arrives to one of the selective populations, reverberatory activity above baseline firing is maintained in the neurons of this population (thick black trace) while other populations remain close to their spontaneous firing state (gray traces). Inhibitory neurons also increase their firing slightly, becoming much more responsive at the time of memory erasure (thin black trace). Reproduced from Fig. 2 in Brunel and Wang (2001) with kind permission of Springer. (B) Bifurcation diagram for a network as the one in A. The abscissa labels the strength of coupling among neurons of one same selectivity population, relative to the strength of coupling among neurons belonging to different selectivity groups. In the y axis, the average firing rate after stimulus presentation (upper branches) or during spontaneous activity (lower branches) is plotted either from a mean-field mathematical formulation (lines) or from the simulations (symbols). Two cases are represented differing just in a rescaling of all synaptic couplings and external stimulation rates: connectivity in the rightmost case (thin lines, pluses and diamonds) is approximately 70% of that of the leftmost case (thicker lines, crosses and squares), and external stimulation is 50% stronger in the rightmost than the leftmost graphs. Notice lower persistent activity rates in the weaker coupling case. Reproduced from Fig. 6 in Brunel (2000). Copyright Taylor & Francis, Ltd. (C) Feedback inhibition controls the rates of the persistent activity state weakly, but with the cost of making it less robust and eventually destabilizing it. Bifurcation diagram (see B) as inhibition is progressively increased in the network. As inhibition increases, persistent rates decrease slightly but soon multistability is abolished. Reproduced from Fig. 7 in Brunel and Wang (2001) with kind permission of Springer. (D) Illustration of the mechanisms behind the inhibition-dominated regimen of the network in panel A. Unselective, constant external inputs to the network (labeled AMPA ext) are the primary source of inputs to the neurons of the network. Recurrent inputs are dominated by inhibition in the spontaneous state (upper panel), so that their sum (Total rec⫽AMPA rec⫹NMDA rec⫹GABA rec) is negative and suppresses the firing imposed by the external inputs. In the active state (lower panel) both recurrent excitation and recurrent inhibition increase, but excitation does it to a larger degree so that now recurrent interactions are globally excitatory on average (dashed lines indicate values in the spontaneous state, i.e. as in the upper panel). Reproduced from Fig. 3 in Brunel and Wang (2001) with kind permission of Springer. al., 2004), many-area-interaction in more involved delay tasks (Renart et al., 2001; Deco and Rolls, 2003, 2005; Deco et al., 2004), formation of persistent activity attractor networks through Hebbian learning (Mongillo et al., 2003; 138 A. Compte / Neuroscience 139 (2006) 135–151 Fig. 3. NMDA receptors help in stabilizing low-rate persistent activity in an attractor network model of spatial working memory. (A) Example of network activity during a working memory trial. Neurons are disposed along a ring (y axis, upper and lower limits wrap together to form the ring) and they are mutually connected with strength modulated according to the distance on that ring, such that excitatory cells are more strongly coupled when they lie close to each other. As time flows (x axis), neurons fire spontaneously (each dot corresponds to one spike) driven by external Poisson inputs (see Fig. 2D) until a subpopulation of cells is stimulated externally (between the two first vertical lines, labeled C). After the extinction of this external stimulus a localized “bump” of activity is actively maintained by local reverberations. This persistent activity state during the delay period (label D) is reset by means of an unselective stimulation to the whole network (period labeled R). On the right panel, the average firing rate during the delay period between cue and response is plotted. Borrowed from Fig. 2 in Compte et al. (2000) by permission of Oxford University Press. (B) Rastergrams for the delay period similar to A but with increasing AMPA participation in excitatory synaptic transmission. As NMDA contribution is diminished, oscillations develop and eventually persistent activity destabilizes. Borrowed from Fig. 2 in Tegner et al. (2002). Copyright Elsevier Science. (C) The critical aspect for stability against oscillations is the long time constant of NMDAR-mediated currents as shown here, where only the time constant of excitation is changed and oscillations appear that eventually disrupt persistent activity. Adapted from Fig. 8 in Wang (1999). Copyright 1999 by the Society for Neuroscience. (D) NMDA receptors also contribute to rate control in the active state. This is accomplished by means of the saturation properties of NMDA receptors as illustrated here. Upper panels show the input– output relationships (convex lines) for a neuron with asynchronous synaptic inputs of varying rate (x axis) through AMPA and NMDA channels, respectively. Notice that the curve in the NMDA case is much more convex than for AMPA, due to saturation. The straight line is the condition for self-consistency: the rate of the afferent inputs is the same as the output rate if this is just one of many neurons in a recurrent network in steady-state. The states that do fulfill this self-consistency are indicated by solid dots and can be translated onto the lower panels as the input– output relationship is changed through the modification of the synaptic efficacy. Notice that, because of NMDA saturation, increasing synaptic strength augments the range of multistability without increasing the rates significantly (compare with AMPA). Adapted from Fig. 5 in Wang (1999). Copyright 1999 by the Society for Neuroscience. Amit and Mongillo, 2003b; Brunel, 2003), and mechanisms for multi-item working memory (Amit et al., 2003; Amit and Mongillo, 2003a). 1: Cellular and synaptic mechanisms that have been associated with persistent activity 1.1: Strong feedback excitation in the local circuit. This is the longest standing cellular hypothesis for persistent activity. It was formulated as the possible substrate for memory, based on morphological arguments, even before persistent activity was first observed experimentally (Lorente de No, 1933; Hebb, 1949). As it goes, persistent activity would be the result of mutual excitation between neurons in a local circuit that would be self-sustained by virtue of the dense reciprocal synaptic connections existent in most brain areas, notably in the frontal lobes (Goldman-Rakic, 1995). The activity would then be self-maintained for a much longer time than any of the intrinsic or synaptic time constants. As simple as this idea is, it hides an important caveat to its actual implementation in a network of physiological neurons: how could such activity be stable? It just seems that a closed loop of excitatory inter- A. Compte / Neuroscience 139 (2006) 135–151 actions would lead to runaway excitation up to firing rates near the neuronal saturation limit (Amit, 1995; Tegner et al., 2002), which is certainly not the experimental observation. Computational models have substantiated this hypothesis by establishing the conditions for stable reverberation at low rates in neuronal networks (see next two mechanisms in this section). Early, simple models focused on laying the theoretical grounds within attractor theory and, to this end, killed biological detail (Wilson and Cowan, 1973; Amari, 1977; Hopfield, 1982; Zipser et al., 1993). Later models showed that, for the case of instantaneous synapses, the stability problem of runaway excitation in the reverberation hypothesis could be solved by introducing feedback inhibition into the picture (Amit and Treves, 1989; Buhmann, 1989; Rubin and Sompolinsky,1989; Amit and Tsodyks, 1991a,b; Amit, 1995),which also helped in stabilizing a non-silent spontaneous activity state (Amit and Brunel, 1997; Brunel, 2000). Subsequent network models have emphasized synaptic dynamics as an important aspect for the stability of reverberatory persistent activity, both in object working memory (Wang, 1999; Durstewitz et al., 2000b; Brunel and Wang, 2001; Hansel and Mato, 2001; Koulakov, 2001), in spatial working memory (Compte et al., 2000; Laing and Chow, 2001; Gutkin et al., 2001; Tegner et al., 2002) and in parametric persistent activity models (Seung et al., 2000a; Koulakov et al., 2002; Miller et al., 2003). What all these models consistently show is that the maintenance of persistent activity at low firing rates through reverberation in a local cortical network is a plausible scenario from the dynamical, computational standpoint (see examples in Fig. 2A and in Fig. 3A), provided some stabilizing physiological mechanisms are present in the circuit (see below). One important issue about the strong reverberation hypothesis for working memory is to understand how strong feedback excitation between groups of neurons selective for the same stimulus might emerge. One possible mechanism is Hebbian learning, as originally assumed by associative memory models à la Hopfield (Hopfield, 1982) and extensively studied recently in more realistic models (Mongillo et al., 2003; Amit and Mongillo, 2003b; Brunel, 2003). Another mechanism is based on the colocalization of same-selectivity neurons in a “micro-column,” which by virtue of the limited extent of axonal and dendritic arborizations would result in such strong feedback loops among neurons of equal selectivity. Several predictions, as yet untested, emerge from the attractor network picture: if it were correct, neuronal hyperpolarization should not interrupt persistent activity in a hypothetical intracellular recording in an awake behaving monkey during a working memory task, because this sustained activity is being maintained by synaptic inputs that persist as opposed to intracellular mechanisms, which would be switched off by hyperpolarization. Also, a hallmark of the synaptic reverberation hypothesis in an attractor framework is the fact that bistability emerges and disappears abruptly as parameters of the recurrent connectivity are varied (like the strength of excitatory connections, or the strength of inhibitory connections) in what is known 139 mathematically as a “bifurcation” (see diagram in Fig. 2B). This has been presented as a prediction to be tested experimentally by using incremental progressive blockade of excitatory (or inhibitory) synaptic transmission (Durstewitz et al., 2000b). Dynamically, practically all network models of persistent activity have required a high degree of asynchronous population firing for proper mnemonic function. In some models, oscillations can be accommodated during stable persistent activity, provided they are of sufficiently small amplitude. Non-synchronous firing is such a critical aspect in reverberatory persistent activity because the time course of the bulk of cortical synaptic potentials is typically much shorter (⬃10 ms) than the average interspike interval in the low-rate persistent activity state of working memory (⬃50 ms at 20 Hz). Therefore, synaptic drive to maintain autonomously a reverberatory firing state in the network cannot rely on inputs coming from neurons that fire in synchrony. An asynchronous firing state ensures sustained, almost constant, synaptic drive to neurons in the network, which can underlie their persistent activity. If such an asynchronous state is stabilized, persistent activity can be sustained even with very fast synaptic dynamics (Hansel and Mato, 2003) and this necessity for asynchrony is less stringent, the longer the time course of synaptic interactions is (see Sec. 1.3). Also, for technical reasons, asynchronous population firing is interesting mathematically for it allows the description of large-scale network simulations in terms of much simpler, and analytically tractable, meanfield descriptions (Amit and Tsodyks, 1991a; Amit and Brunel, 1997; Brunel, 2000; Renart et al., 2003a). Notice that in strongly coupled networks like the ones considered here, when the asynchronous state destabilizes it usually enters an oscillatory regimen. This underlies the observation that oscillatory network activity of large enough amplitude is generally detrimental to persistent activity stability in recurrent network models. However, oscillatory activity related to working memory has been observed experimentally in neurons of the parietal cortex (Pesaran et al., 2002) and the occipital lobe (Lee et al., 2005), but not of the prefrontal cortex (Compte et al., 2003a). One functional role has been proposed for synchronous activation of cells during persistent activity: erasure of the actively held memory (Gutkin et al., 2001). Because synchrony disrupts persistent activity, if one injects a strong pulse of current to all cells in a recurrent module that is sustaining asynchronous persistent activity, many cells will come to fire almost in synchrony and persistent activity will be aborted, resetting the network to the spontaneous, non-selective state. If this mechanism were indeed being used for memory erasure in working memory, the clear prediction to be tested is whether there is an increase of synchrony to be seen in the local field potential or in multielectrode recordings around the time when the monkey responds in a working memory task. 1.2: Strong feedback inhibition in the local circuit. As mentioned above, strong feedback inhibition was another mechanism soon brought into the picture for persistent 140 A. Compte / Neuroscience 139 (2006) 135–151 activity network models. The very convenient property of feedback inhibition is the fact that its suppressive effect on excitatory units of the network is stronger, the higher the rate of these units is. This looks like an effective rate control mechanism: the faster the excitatory cells venture to fire, the deeper their membrane voltage is pushed by feedback inhibition so they are brought back to sensible firing rates. This was shown to control the rate of persistent activity states in early network models with combined recurrent excitation and inhibition (Amit and Treves, 1989; Rubin and Sompolinsky, 1989; Buhmann, 1989). However, as a rate control mechanism, inhibition is not totally satisfactory. On the one hand, apart from reducing the rate in the active state, inhibition to excitatory cells also reduces dramatically the robustness of multistability in the network so that realistic firing rates often require very fine-tuned network connectivity (Wang, 1999). Fig. 2C shows that increasing feedback inhibition soon renders the network monostable before affecting too significantly the firing rates. The fine tuning problem in these network models is not specifically linked to inhibition. In fact, bringing the firing rate in the persistent state as close to the spontaneous rates as seen in experiments requires fine tuning irrespective of what mechanism is being used to control the rates (Wang, 1999; but see Latham and Nirenberg, 2004). Also, an important family of spiking network models that incorporate robust spontaneous activity in the network exclude a role for feedback inhibition in controlling the rate of persistent activity (Brunel, 2000; Brunel and Wang, 2001). Instead, these models use the saturation of the input– output neuronal relationship to control the rate of the persistent activity state (Brunel, 2000). Critically, these models operate in the sparse coding limit, when neurons selective for one same stimulus are just a tiny fraction of the typical size of a synaptically connected population. In similar network models, Latham and Nirenberg (2004) have argued that away from the sparse coding limit feedback inhibition keeps playing an important role in controlling the rate of the persistent activity state. Thus, the sparsity of encoding may be an aspect that determines the qualitative role of inhibition as a rate control mechanism for persistent activity. So, despite the intuitive simplicity of the relationship between feedback inhibition and neuronal rate control, inhibition is a mechanism involved in many different ways in the generation of multistability in these networks (see below) so that its effects on the network activity are not always straightforward to understand on the basis of this intuitive association (in some networks this may even be the least of the roles played by inhibition, as shown by Brunel (2000)): see for instance Fig. 2B, where a manipulation that decreases inhibition (and excitation, proportionally) makes persistent activity firing rates much lower. Apart from its suppressive effects through hyperpolarizing postsynaptic potentials, inhibition has also been implicated differently in the maintenance of highly variable, low-rate reverberatory activity. In networks where excitation and inhibition are balanced, the average synaptic current impinging on a neuron keeps it just below firing threshold on average, and it is the synaptic fluctuations that make the neuron fire. This leads to very irregular, low-rate firing patterns (Tsodyks and Sejnowski, 1995; van Vreeswijk and Sompolinsky, 1996). The addition of recurrent inhibition, therefore, is not just a means to diminish excitability through hyperpolarization, but in a balanced state it is also a way to increase current fluctuations through recurrent interactions, and this results in enhanced, highly irregular firing. It has been proposed that it is precisely this balanced regime where the low-rate persistent activity dynamics might be realizable at the network level, taking advantage of the highly fluctuating synaptic current regime generated by intracortical inhibition (Amit and Tsodyks, 1991a,b). A related issue is how to explain unselective, very low-rate (⬍5 Hz) inter-trial interval firing or “spontaneous activity” in the context of very strongly reverberatory systems that can sustain a higher rate (⬃20 Hz) persistent activity state. Especially when the two states do not differ too much in firing rate, how can spontaneous activity remain stable, not engaging the reverberatory mechanisms that are recruited by persistent activity? This would not be a problem if neurons were practically silent in non-mnemonic epochs, but this is not the case experimentally. Amit and Brunel (1997) set the conditions in which both a lowrate non-selective firing mode and a selective higher-rate mode could coexist as stable attractors of a spiking-neuron recurrent network: The net effect of recurrent synaptic currents, in the spontaneous state, must be inhibitory. The neurons are driven by external inputs from other parts of the brain, whence the silent state is not stable, and recurrent connectivity suppresses this firing through an inhibition-dominated circuit that brings it into the balanced regime discussed above (see Fig. 2D). This ensures that a low-rate, non-silent and non-selective state is stable for this network. Thus, feedback inhibition should not only be strong, but also dominant over feedback excitation in baseline conditions (Wang, 2001; Amit and Mongillo, 2003a; Latham and Nirenberg, 2004). Many network models of persistent activity have used this inhibition-dominated regime (Amit and Brunel, 1997; Brunel, 2000, 2003; Compte et al., 2000; Brunel and Wang, 2001; Tegner et al., 2002; Wang et al., 2004; Latham and Nirenberg, 2004) but other models have used an excitation-dominated circuit (Durstewitz et al., 2000a; Gutkin et al., 2001; Laing and Chow, 2001) where spontaneous activity, if present, reflects rare fluctuations in subthreshold inputs that are not sufficiently amplified by recurrent excitation to generate the activated mnemonic condition. This might explain discrepancies among the predictions of the various models. Indeed, whether the recurrent interactions are primarily excitatory or inhibitory might have a very profound impact on the dynamical stability of the attractors (Brunel, 2003). An additional, and critical, role for feedback inhibition in many of these models is the maintenance of selectivity: as persistent activity is maintained for a subset of neurons in the network, other neurons selective to different stimuli are being inhibited and prevented from entering the persistent activity state. Thus, strong cross-inhibition makes possible A. Compte / Neuroscience 139 (2006) 135–151 that the original selectivity set by the stimulus is preserved throughout the mnemonic period. If the cortical ensembles of distinct selectivity are spatially intermingled in the cortical tissue forming within-column functional networks, this translates into the need for nonselective inhibitory innervation to these different ensembles from local interneurons, as it has indeed been observed in layer 2/3 of the rat visual cortex (Yoshimura et al., 2005). On the other hand, in networks that assume a spatial organization of the selective subpopulations (columnar organization), this translates into the necessity of surround inhibition. This is a contentious issue for pyramidal cortical neurons are known to have a rich local axonal arborization which is typically larger than that of most GABAergic interneurons (Lund and Wu, 1997; Douglas and Martin, 2004). One possibility, as yet unproven, is that instead of using far-reaching monosynaptic inhibition these network models could operate equally well when di-synaptic inhibitory connections reach farther away than mono-synaptic excitation within the local circuitry associated to one unitary mnemonic module (as suggested by Melchitzky et al., 2001; Kang et al., 2003). Alternatively, possible mediators of this far-reaching inhibition mechanism are large basket cells (Somogyi et al., 1983; Markram et al., 2004). The existence of significant, functional surround inhibition remains to be demonstrated in the nervous tissue that presumably underlies persistent activity (see, for instance Gonzalez-Burgos et al., 2005). Intracortical inhibition might also be involved in the stabilization of non-synchronous firing in the network population. This is a critical stability issue in network models of persistent activity because when one incorporates the physiological dynamics of the principal cortical excitatory (AMPAR-mediated) and inhibitory (GABAA) postsynaptic currents into recurrent models of working memory, this induces a dynamical instability that makes persistent activity unstable through the development of synchronized oscillations (Wang, 1999; Tegner et al., 2002). This is so because the decay time constant of postsynaptic currents mediated by GABAA receptors is longer than that of AMPAR-mediated excitatory postsynaptic currents. A system endowed with fast positive feedback and slower negative feedback is prone to oscillatory behavior, and oscillations are detrimental to the stability of a persistent activity attractor (Wang, 1999; Compte et al., 2000; Gutkin et al., 2001). The way in which most models have proposed to overcome this difficulty is by appealing to the slower form of ionotropic excitatory transmission in the cortex: NMDA receptor (NMDAR)-mediated transmission (see Sec. 1.3). However, (Hansel and Mato, 2001) have proposed that also inhibition might have a part to play in this respect: strong inhibitory connections among inhibitory neurons in a recurrent network can break the synchrony in the population dynamics and thus stabilize persistent activity driven only by the faster form of excitation; see, however, conflicting results where an increase of inhibition to inhibitory cells made the network more vulnerable, rather than more robust, to oscillations (Tegner et al., 2002). Another use for an inhibition-dominated recurrent circuit in the context of working memory is the possibility to erase an 141 actively held memory by means of a non-selective excitatory pulse to the network (Compte et al., 2000; Brunel and Wang, 2001; Brunel, 2003). This is confronted to the more straightforward alternative of resetting the persistent activity state by means of a hyperpolarizing pulse to all neurons in the network (Wang, 1999). Because this silences the whole network, when the network is released and allowed to evolve autonomously again, it settles in the spontaneous activity, non-selective state and the previously active memory is erased. However, this implies that the resetting operation in neurons involved in the maintenance of persistent activity (presumably in the prefrontal cortex) should be characterized by suppression of firing relative to baseline. Instead, in most cases one sees strong enhancement of firing during the animal response and immediate recovery to baseline (Funahashi et al., 1989). Experimental data, therefore, suggest that the erasure signal is excitatory. An inhibition-dominated circuit provides a possible network substrate for this excitation-induced suppression of activity: in an inhibition-dominated network, a sufficiently strong pulse of excitation recruits massive intracortical inhibition that overwhelms intracortical excitation when the input is withdrawn. This results in the suppression of firing and eventually the extinction of persistent activity. This can be given a more formal and quantitative expression within attractor theory (Brunel and Wang, 2001; Brunel, 2003). As a mechanistically different scenario, an excitatory pulse can also reset a reverberatory network by inducing synchronous firing, which destabilizes persistent activity (Gutkin et al., 2001, see a more detailed discussion in Sec. 1.1). Wang et al. (2004) identified one final possible functional role for strong inhibition, based on the electrophysiological and morphological diversity of interneurons in the cortical network. In their network model of spatial working memory, three classes of interneurons are included with distinct morphological and electrophysiological properties, as suggested by experimental data. In a working memory context, this disposition serves to make mnemonic persistent activity more resistant to intervening distractors. This is accomplished by means of the disinhibition of active pyramidal neurons, when dendrite-targeting interneurons are strongly inhibited by interneuron-targeting interneurons (Wang et al., 2004). Feedback inhibition is therefore a fundamental piece in the stability of the reverberatory hypothesis for persistent activity in working memory, and it has been associated in a long list of possible roles (see also Amit and Mongillo, 2003a; Brunel, 2003): rate control for the active mnemonic firing state maintained by reverberation; generation of a fluctuating balanced state and stabilization of low-rate reverberatory activity; stabilization of the spontaneous, lowrate state by means of an inhibition-dominated recurrent connectivity; maintenance of selectivity in the network during persistent activity; reset of the persistent activity module upon sudden excitation of the whole network; promotion of asynchronous firing by means of strong inhibitory-to-inhibitory connections; and increasing the network capability to resist distractors via pyramidal cell disinhibition. 142 A. Compte / Neuroscience 139 (2006) 135–151 1.3: Recurrent excitation primarily mediated by NMDARs. Several roles have been ascribed to the NMDAR in the context of recurrent network models of working memory and persistent activity, in line with the three main features that differentiate this receptor from the AMPA receptor: longer decay time constant (approximately 100 ms), much higher affinity for glutamate (resulting in faster saturation properties), and voltage dependency of the receptor conductance in the presence of extracellular magnesium (the channel is virtually closed at hyperpolarized potentials). As anticipated in the subsection 1.2, NMDARs have been implicated in the stabilization of persistent activity against disruption by synchronous population firing (see Fig. 3). The key aspect of NMDARs here is their long decay time constant (Fig. 3C), and they can accomplish this stabilization in two different ways: On the one hand, by introducing much slower excitatory interactions the binomial fast excitation plus slow inhibition, which is responsible for synchrony generation, is weakened and asynchronous firing is promoted. On the other hand, when some degree of synchrony is present in the persistent activity state, the long decay time constant of NMDAR-mediated postsynaptic currents can bridge the gap in excitatory synaptic drive during synchronized neuronal refractory periods and maintain stable self-sustained activity (Wang, 1999, 2001; Compte et al., 2000; Tegner et al., 2002; Ermentrout, 2003). Other mechanisms have been proposed that act along the first of the strategies outlined here (stabilization of the asynchronous firing regime) (Hansel and Mato, 2001; Gutkin et al., 2001), but it seems like stable persistent activity with partially synchronous firing would require necessarily the action of NMDARs (Tegner et al., 2002; Ermentrout, 2003). Also the long decay time constant of NMDAR-mediated postsynaptic currents has been associated by computational models to the robustness problem of line attractor models (see Sec. 2.2). In order to have neurons stably maintain a continuum of firing rates, depending on what stimulus is transiently presented (line attractor), reverberatory network models have typically found that the synaptic weights of the model have to be very carefully tuned to within 1% accuracy. The accuracy required, however, if one uses AMPA instead of NMDA in recurrent connections is orders of magnitude higher (Seung, 1996; Seung et al., 2000a,b). Both of these accuracy values seem biologically unrealistic, but NMDARs seem to alleviate the problem. Note that the key aspect of NMDARs for the stabilization of persistent activity described so far is its slow dynamics. Thus, any other form of synaptically induced slow depolarization could also mediate these stabilizing effects. One candidate is, for instance, the combination of glutamatergic transmission and cholinergic neuromodulation in the oculomotor system (Navarro-Lopez et al., 2004). NMDARs can also play a role in the rate control problem (see Sec. 1.1) of stable low-rate reverberatory activity in a recurrent network. The property of high affinity for glutamate of NMDARs can be critical in this respect: if excitatory recurrence in the local network is primarily mediated by NMDARs, then their saturation properties at low rates help very significantly in avoiding runaway excitation and near-saturation firing. This is so because NMDARs would be already completely open by a presynaptic stimulation of around 30 Hz. If a presynaptic neuron ventures beyond these rates, the postsynaptic neuron will not follow because the excitatory drive will already be saturated and the state will be stabilized (Fransén and Lansner, 1995; Wang, 1999; Fig. 3D). Finally, NMDARs could also be engaging their voltage dependency for generating a bistable switch in their synapses (Lisman et al., 1998). This is a mechanism by which one avoids having to set the selectivity in the model beforehand. It relies also on a recurrent network, where reentrant excitation is able to sustain persistent activity. However, because of the voltage dependency of NMDARs, only those synapses that link a presynaptic neuron and a postsynaptic neuron that are both strongly activated by the external stimulus reach the required synaptic strength to sustain reverberation. Persistent activity remains constrained to the population of neurons that was first recruited by the stimulus, without spreading to nonimplicated neurons. A unique feature of such a circuit is that it is not preset to store a given set of memories, but just remembers whatever stimuli one presents, without requiring learning-induced synaptic modifications (which are considered to be a slow process in the time scales of this task). 1.4: Intrinsic currents. Even in the framework of recurrent network models, intrinsic membrane properties will have an effect on the behavior of the system. This has been explored in recurrent network models that include the detail of biophysical membrane channels other than synaptic receptors and spike-generating channels (Durstewitz et al., 2000b; Tegner et al., 2002; Durstewitz and Seamans, 2002; Koulakov et al., 2002; Goldman et al., 2003; Wang et al., 2004). Specific predictions include: slow calcium-dependent cationic channels (Ican) help in stabilizing persistent activity against oscillatory activity, much like NMDARs (see Sec. 1.3) (Tegner et al., 2002), dendritic channels that induce a bistable switch in the dendrite help in rendering line attractors more robust (Goldman et al., 2003), and dopamine neuromodulation of synaptic and intrinsic channels can make the network more resistant to distraction (Durstewitz et al, 2000b; Durstewitz and Seamans, 2002). In addition, experiments (see Sec. 3.1) have also suggested a role for the h-current in controlling the duration of activity network states maintained by reverberation (Shu et al., 2004). This mechanism remains to be explored in network models of persistent activity. 2: The computational challenges of recurrent models of persistent activity 2.1: High variability of firing in delay periods. As I have outlined in Sec. 1, recurrent models of persistent activity are plausible biologically and can make use of well-characterized physiological mechanisms present in the cortical circuit in order to perform their mnemonic function in a more stable and robust manner, being resistant to distractors. Recently, however, the statistical characteriza- A. Compte / Neuroscience 139 (2006) 135–151 tion of firing activity of single neurons in the prefrontal cortex of monkeys performing a working memory task has challenged this entire class of models: neurons increase the variability of their spike trains when they engage in persistent activity during working memory tasks with respect to their baseline firing (Compte et al., 2003a). This qualitative feature is very difficult to account for within current models of persistent activity, that show typically the opposite trend (Brunel and Wang, 2001; Brunel, 2003). Qualitatively, these network models produce a highly variable spike output through the balancing of excitatory and inhibitory synaptic currents in the spontaneous state (van Vreeswijk and Sompolinsky, 1996). However, the persistent activity state is obtained by sacrificing this balance locally so that a subpopulation receives more excitation and fires at relatively higher rates. The cost of this operation is a reduction of the variability of the spike trains in persistent activity since the neurons depart from the balanced state, where they fire in the fluctuation regime, and enter the drift regime of firing, much more regular. This relative reduction in variability is therefore intrinsically associated with these models. Whether matching the experimental observation of high variability during persistence implies just the necessity of adding on features to existing recurrent models (bursting properties, additional inhibitory populations, . . .) or else of developing a new framework to conceptualize persistent activity, is an open issue. The only attempt so far to address this issue in a recurrent network model of working memory proposes a microcolumnar organization for excitation and inhibition (Renart, 2000; Renart et al., 2003a). In their model, increased firing both in excitatory and inhibitory cells of a subpopulation during persistent activity occurs through an increase of current fluctuations with no change in the average current. As a result, both spontaneous activity and persistent activity would operate in a balanced state and their difference in firing rate would be due to the increase of fluctuations in the current. This translates into a persistent activity state with higher variability than the spontaneous state. However, this network requires a very precise fine tuning of connectivity parameters to operate in the way described. Recent findings on the firing properties of neurons with slow postsynaptic currents, or short membrane time constant, might shed some light onto this challenge for persistent activity network models. It has been shown that, when the synaptic time constant is much slower than the effective membrane time constant (one can label this condition “high-conductance state” because a way to achieve this regime is by intense balanced synaptic bombardment that reduces dramatically the effective membrane time constant from its resting value), the neuronal firing threshold can be set in such a way that the neuron responds only to the largest fluctuations that occur in incoming conductance-based synaptic currents (Moreno-Bote and Parga, 2004). These correspond typically to the synchronous arrival of presynaptic spikes, so that the neuron fires as a synchrony detector (Bugmann, 1991; Abeles, 1992) and the output of the neuron is more variable (it is actually bursty) than that of its inputs (see also Svirskis and Rinzel, 143 2000; Salinas and Sejnowski, 2002). If persistent activity models could make use of this regime of operation, this might provide a way in which neurons in the persistent activity state (presumably a high-conductance state) increase their variability with respect to the spontaneous state (less of a high-conductance state). A similar idea, although now relying on the slower intrinsic dynamics of dendrites, has been experimentally documented by Larkum et al. (2004). In their experiments, they show that noisy current injection in the soma produces much more regular firing patterns than the same current injection in the dendrite. This is accomplished through the involvement of the slow kinetics of calcium channels in the dendrites, suggesting a similar operation as the “high conductance” state described above (Moreno-Bote and Parga, 2004). 2.2: Robustness of line attractors. Computational models that implement a line attractor for the encoding of a continuous stimulus, as for spatial working memory or parametric working memory, suffer from a serious problem regarding their robustness: their synaptic weights need to be tuned to an absurdly precise accuracy (Brody et al., 2003). This is a serious problem in a biological system, usually characterized by a high level of noise and heterogeneity. Several approaches to this problem have been put forward. Firstly, since line attractors can be thought of as the limit of infinitely many discrete attractors (as in Fig. 1D–E), one may wonder whether the brain just gets away with a finite number of discrete attractors, rather than a real continuum (Brunel, 2003; Major and Tank, 2004; Miller et al., 2002). Discrete attractors do not suffer as severely from the robustness issue as continuous attractors. Some models have implemented this idea by endowing those discrete attractors with additional bistability mechanisms, so as to render the wells in Fig. 1D deeper, more stable (Koulakov et al., 2002; Goldman et al., 2003). Is there any evidence that the brain implements an actual line attractor rather than using closely spaced discrete attractors? There is only indirect evidence that suggests that, if not strict continuous attractors, spatial working memory uses very tightly spaced and shallow discrete attractors: subjects in spatial working memory tasks make on average more errors, the longer the delay interval is (White et al., 1994; Ploner et al., 1998), as if mnemonic activity was drifting as a random walk on a line attractor (Ben-Yishai et al., 1995; Camperi and Wang, 1998; Seung, 1996; Zhang, 1996; Compte et al., 2000). The fact that the magnitude of this average drift depends on the length of the mnemonic period indicates that it is generated in the memory-maintaining system, rather than the sensory or motor systems. Similar delay-dependent behavioral errors have been reported also for working memory in other visual experiments and in other sensory modalities (Pasternak and Greenlee, 2005). This does not invalidate the discrete attractor approximation to the line attractor (Miller et al., 2002) and it seems that a statistical analysis of correlations among the spike trains of neurons in the network during persistent activity would only help in unambiguously dis- 144 A. Compte / Neuroscience 139 (2006) 135–151 entangling this dichotomy by going to fourth order in the statistics, which is technically not feasible with currently available data (Miller, unpublished observations). A second possibility is that indeed, the system is implementing a continuous attractor and that homeostatic mechanisms in the brain compensate for heterogeneities, so that the steady state of this homeostatic system is precisely the fine-tuned condition necessary for line-attractor function. This has been shown to work with a physiologically characterized activity-dependent synaptic scaling mechanism in a spatial working memory network (Renart et al., 2003b). There, individual neurons have a target average firing rate over a long time scale and they scale up or down their synaptic inputs in order to attain it. This compensates the heterogeneities in the cellular properties of the neurons in the network and recovers the continuous attractor that these heterogeneities destroy. sive methods on awake behaving animals. This is the natural test bed for cellular mechanistic hypotheses of working memory. However, in the last five years a number of experiments in cortical slices in vitro that attempted to reproduce dynamics similar to persistent activity have suggested that there might be ways to explore the specific hypotheses of recurrent working memory models in reduced preparations. Currently, these preparations do not achieve truly persistent activation, but their activity is of an intrinsically episodic nature. This indicates that caution must be exerted when relating findings in these experiments with persistent activity, as discussed below. In this section I review what we might learn from these systems in relation to the points discussed in the previous sections, and I also speculate about where a tight relationship between computational models of persistent activity and in vitro electrophysiology could take us to in the future. 2.3: Compatibility with cellular physiology. As much as I have stressed in Sec. 1 all the physiological mechanisms that are beneficial for the purposes of a working memory module that generates persistent activity through recurrent connectivity, there are also mechanisms widely present in cortical neurons that are detrimental and usually absent (or minimized) in current network models of persistent activity. The most prominent of these mechanisms are membrane channels responsible for spike-frequency adaptation (calcium-dependent or sodium-dependent potassium channels). Although object working memory models are robust enough to accommodate a certain degree of cellular adaptation (Durstewitz et al., 2000a; Durstewitz and Seamans, 2002), neither spatial working memory models nor parametric working memory models tolerate much of it in their constituent neurons (Compte et al., 2000; Seung et al., 2000a; Tegner et al., 2002; Miller et al., 2003; Wang et al., 2004). In spatial working memory models, for instance, spike-frequency adaptation destabilizes localized persistent activity and makes the region of activity move with constant velocity along the network, as a traveling wave (Hansel and Sompolinsky, 1998; Laing and Longtin, 2001). This is a very undesirable feature that cancels all mnemonic function for the network. However, spike-frequency adaptation is ubiquitous in cortical pyramidal cells. How can one reconcile this? The only proposal so far, has pointed at the fact that enhanced noisy input from external sources to the network can stop the continuous drift of the traveling wave (Laing and Longtin, 2001). However, this is done at the cost of increasing the noisiness of the environment, which in a continuous attractor results in accentuated random drifting of the bump of activity. Another possibility is that these intrinsic currents are attenuated by some neuromodulator that is required to switch on working memory function, like for instance dopamine (Durstewitz et al., 2000a). 3.1: The slowly oscillating slice. An experiment where cortical slices are maintained in a modified ACSF (artificial cerebro-spinal fluid) solution that mimics the ionic concentrations of the cerebrospinal fluid in situ has shown that cortical slices can maintain sustained activity largely mediated by synaptic interactions over long periods of time (Sanchez-Vives and McCormick, 2000). This episodic activity manifests itself in the form of long bouts of activity (between 300 ms and 4 s) interspersed with longer periods of silence, in an oscillatory pattern very reminiscent of in vivo cortical activity during deep anesthesia or natural slow-wave sleep (see Fig. 4A). In a generalization of the concept that we have been using here, this episodic phenomenon has been presented as an example of persistent activity. Although activity is not triggered by a known stimulus, and hence firing is not strictly known to persist after its extinction (see, however, Shu et al., 2003), the long duration of active states indicates the possible implication of a nontrivial persistence mechanism. Also, it is indeed a case where the system appears to make regular transitions between two states of quasi-stability (this is described in detail in the model by Compte et al. (2003c)), so that probably multistability, the hallmark of working memory in attractor network models, underlies this phenomenon. It is nonetheless also true that the system is far from being a satisfactory experimental model for persistent activity in working memory: Firstly, oscillatory activity per se is conceptually very far from stimulus-reactive systems like the ones necessary for working memory. The slice cannot be stopped oscillating and yet conserve stimulus-triggered upstates. Upstates are generated periodically and spontaneously and therefore they tell us about the inner conditions of the system, rather than about external contingencies. Secondly, it lacks selectivity. When the slice enters an active state (“upstate”), this propagates to all neurons in the slice (Fig. 4A.4) and it does not remain restricted to a subset of them, so it does not appear to engage subpopulations selectively. Thirdly, the downstate in this preparation is practically quiescent, while spontaneous activity in cortical neurons in the alert animal is sizable and very variable (Compte et al., 2003a). In the context of attractor 3: Experimental tests for persistent activity mechanisms in vitro Experimental research on the physiological mechanisms of working memory has to be done necessarily with inva- A. Compte / Neuroscience 139 (2006) 135–151 145 Fig. 4. Illustration of in vitro experiments related to the cellular and synaptic mechanisms of persistent activity. (A) The oscillating acute slice of (Sanchez-Vives and McCormick (2000)) shows recurring events of synchronized network firing (“upstates”) as demonstrated by simultaneous extracellular (A.1) and intracellular (A.2) recordings. These upstates remain upon neuronal hyperpolarization revealing that they are generated through incoming barrages of postsynaptic potentials (A.3). Array extracellular recordings show that the upstate propagates horizontally through the slice and it recruits most neuronal tissue (A.4). Copyright by Nature. (B) In organotypic prefrontal cortex (PFC)-ventral tegmental area (VTA) co-cultures, spontaneously occurring episodes of synaptically sustained activity are also observed (B.1) in close similarity to the slowly oscillating slice (Seamans et al. (2003)). In PFC–VTA– hippocampal co-cultures, CA1 stimulation also produces prolonged firing in PFC cortical neurons (B.2 left) which is synaptically sustained through NMDA receptors (see bath application of APV in B.2 right). Voltage clamp experiments (B.2 lower graphs) reveal a slow inward current (gray line) that is blocked by NMDA antagonists. Reproduced by permission of Oxford University Press. (C) Simultaneous calcium imaging (C.1) and intracellular recording (C.2) in a mouse visual cortex slice show that in periods of increased network synchrony (two different snapshots shown in left and right columns) network activations as revealed by calcium increases (C.1 red dots) correspond to persistent firing (upstate) of neurons in the network (Cossart et al. (2003)): the neuron recorded in C.2 is indicated by the black arrow in C.1, so that when calcium signals are not increased for this cell no firing activity is observed electrophysiologically (left), while the opposite occurs when robust calcium transients are associated to the neuron (right). Copyright by Nature. (D) A hybrid experimental– computational approach to analyzing the conditions for persistent activity (Fellous and Sejnowski (2003)) shows that injecting into the recorded neuron a mixed barrage of excitatory and inhibitory background synaptic conductances in dynamic clamp, together with a short series of random excitatory and inhibitory postsynaptic potential trains following each spike of the neuron (reactive clamp, simulating the local network feedback) is able to replicate in vivo-like persistent activity (D.1). However, a minority of strongly adapting neuron were unable to fire persistently (D.2). Reproduced by permission of Oxford University Press. networks these differences can indicate that the slice network lies in a very different dynamical regime than the in vivo circuit. It is, for instance, possible that the upstate in the slice corresponds more closely to the low-rate spontaneous activity in vivo than to the persistent activity state in working memory, while the downstate is a third possible state associated with anesthesia or sleep (Steriade et al., 1999; Steriade, 2000). Still, it remains a fascinating issue to learn about the mechanisms of synaptically sustained activity in the slice and test the hypotheses formulated by attractor network models of persistent activity, with the ultimate objective of drawing conclusions to be tested in awake behaving animals. Oscillating slices in vitro, are therefore possible experimental test beds of mechanisms for unselective multistability, rate control, reset of activity, and firing statistics in sustained states; but they are not well suited to address issues like selectivity, spontaneous activity, distraction, line attractors, etc. So far, research in the slowly oscillating slice has produced a wealth of informative data related to the cellular and synaptic mechanisms of reverberatory activity (Sanchez-Vives and McCormick, 2000; Shu et al., 2003; Compte et al., 2003c; McCormick et al., 2003; Silberberg et al., 2004). On the one hand, experiments have convincingly established that these activated states are synaptically maintained through barrages of NMDAR- and AMPAR-mediated excitatory postsynaptic currents temporally superimposed on barrages of inhibitory postsynaptic 146 A. Compte / Neuroscience 139 (2006) 135–151 currents (Sanchez-Vives and McCormick, 2000; Shu et al., 2003; see also Fig. 4A.3). This is the first functional confirmation that dense reciprocal connectivity in the cortical local circuit, in near-physiological conditions, is potent enough to sustain reverberatory activity, which is all the more remarkable for in an acute cortical slice a significant fraction of connections and presynaptic partners have been drastically pruned. The functional substrate for the reverberation hypothesis of working memory is therefore present in the local cortical circuit. On the other hand, inhibition has been shown to follow excitation closely, keeping a stable degree of depolarization (Shu et al., 2003). Thus, inhibition is likely implementing the rate control mechanism that we have described in the previous section to avoid runaway excitation in reverberatory conditions. Experiments have also shown that excitatory pulses to the slice can switch off an active upstate (Shu et al., 2003), lending support either to massive inhibitory recruitment by external input during the upstate (see Sec. 1.2 above) or else to a synchrony-based reset mechanism (see Sec. 1.1 above). The mechanism that underlies the selectivity of most working memory network models is strong inhibitory inputs to neurons that do not belong to the active ensemble of the network (Sec. 1.2, but see Lisman et al., 1998). What these experiments demonstrate is that surround inhibitory connectivity in the slowly oscillating cortical slice is not strong enough to maintain localized activity, for it systematically spreads along the slice (Sanchez-Vives and McCormick, 2000). This questions the role of strong inhibitory feedback in selectivity formation. It is however possible that these surround inhibitory mechanisms require the third dimension of the cortical tissue, which is truncated in the slice; or else that neuromodulation in vivo reshapes the spatial synaptic balance with respect to the in vitro situation so as to ensure robust selectivity during persistent activity through inhibitory interactions. Alternatively, selectivity producing surround inhibitory mechanisms might be evoked only for stronger activated cortical states than seen in vitro, in line with the idea that what one sees as an activated state in these slices is to be associated rather to the spontaneous activity state than to the mnemonic state of working memory systems. As for the implication of NMDARs in the maintenance of persistent activity in the slice, current experiments are not conclusive. On the one hand, both AMPARs and NMDARs seem to be necessary for the maintenance of this activity (Sanchez-Vives and McCormick, 2000). Also, the fact that oscillatory activity in the beta– gamma band is often seen in field recordings during the upstates and not the downstates of the slow oscillations (Compte et al., 2003b) supports the implication of a long-time-constant mechanism, possibly NMDAR-mediated currents, in the stability of persistent activity based on the indirect theoretical arguments laid down above (Sec. 1.3). However, it remains to be explicitly shown in the experiment whether NMDARs participate in the reverberation of the upstate by any of the means detailed in Sec. 1.3, rather than just as bulk excitatory drive substitutable by AMPAR-mediated synaptic currents. The active periods of the slow oscillation in the in vitro preparation also reveal that synaptically sustained neuronal activity is very variable: the average coefficient of variation measured for upstate activity by (Shu et al., 2003) is similar to the one observed in vivo during working memory (Compte et al., 2003a). As explained in Sec. 2.1, generating a high coefficient of variation in the persistent activity state is a problem for current attractor models of persistent activity. Since activity in the slice is known to be based on reverberation, the fact that high variability is observed in the slice indicates that both the reverberation hypothesis and highly irregular activity must be compatible. Further experimental research will surely help us understand what kind of additional mechanisms are involved in making synaptically sustained persistent activity in this preparation so variable. 3.2: Membrane voltage bistability in organotypic cultured slices. A similar network dynamics can be observed in organotypic cultured slices (Plenz and Aertsen, 1996; Seamans et al., 2003; McCormick and Aptowicz, 2004). Because of the similarity with the activity in the slowly oscillating acute slice described above (see Fig. 4B), the same caveats are in order, regarding the usage of this experimental model to investigate the cellular and synaptic mechanisms of persistent activity (see subsection 3.1). This preparation, in contrast to the slowly oscillating slice, does not strictly correspond to the structure of an adult cortex but as a bonus its connectivity is not as truncated as in acute slices, since it grows new connections on the experimenter’s plate. Therefore, these slices might be more easily brought to reverberate than acute slices (Seamans et al., 2003). An interesting thing about this preparation is the fact that one can grow slices from different brain regions together, and as they grow they tend to connect to each other in ways similar to how they are linked in situ. This provides a good experimental tool to explore neuromodulatory mechanisms of persistent activity (Seamans et al., 2003). These experiments have suggested so far that NMDAR-mediated synaptic currents have a predominant role in the maintenance phase of the upstate (Fig. 4B.2), rather than in its initiation, and the critical aspect for this seems to be their long decay time constant (Seamans et al., 2003). 3.3: Calcium imaging and recordings in immature visual cortical slices. This in vitro preparation also evidences the spontaneous emergence of synaptically-sustained long-lasting activity in neocortical slices, but of a qualitatively different nature than the experiments above. Slices of visual cortex of juvenile mice in their late postnatal development are maintained in standard ACSF solution while calcium transients are recorded with two-photon imaging (Mao et al., 2001; Cossart et al., 2003). A few neurons in the slice are spontaneously active, and they engage occasionally in collective, synchronous firing (Fig. 4C.1). Simultaneous intracellular recordings show that during these periods of synchronous fluorescence changes, A. Compte / Neuroscience 139 (2006) 135–151 most coactive neurons undergo a membrane voltage transition to a stably elevated membrane potential (“upstate,” see Fig. 4C.2). This upstate can be as long as 30 s, and when it is suprathreshold neuronal firing is around 20 Hz. In addition, upstate transition frequency is drastically reduced when glutamate receptors are blocked and totally eliminated after additional blockade of GABAergic transmission (Cossart et al., 2003). All in all, this evidences again the ability of the neocortical circuitry to sustain activity over long time periods through synaptic reverberation. As I discussed in 3.1 above, the association of this type of activity to persistent activity in working memory is debatable because these episodic periods of increased synchrony occur spontaneously, revealing the internal dynamics of the tissue rather than informative aspects about the environment, and they are followed by periods of practically total quiescence, very rare in vivo. In addition, in this preparation GABAergic transmission appears to have an excitatory character, in contrast to in situ conditions in the cortex. However, in this case only a small fraction of neurons participates in this persistent activity as opposed to the oscillating slice experiments above, where the whole network was recruited during each upstate of the oscillation (Fig. 4A.4). The sparsity of active neural populations is a feature of persistent activity in working memory and it is the base on which stimulus selectivity can be built. These spontaneous, reverberating upstates have therefore the potentiality of being selective, which the previously discussed in vitro experiments lack (subsections 3.1 and 3.2). The mechanisms for this selectivity are still unknown. Attractor networks have typically invoked inhibition to generate selectivity (see Sec. 1.2). Alternatively, it could rely on the voltage dependency of NMDA channels, as proposed by Lisman et al. (1998). Also, it could be subserved by very sparse neuronal connectivity such that synaptically connected neurons conform essentially independent cell assemblies (in line with synfire chains: Abeles, 1991). Notice however that individual cells have been seen to participate in more than one activation ensemble (Cossart et al. (2003). Whether one of these mechanisms is actually engaged in this preparation or it is a different one that restrains persistent activity within a small subpopulation of the network, is an open issue that can be explored experimentally. 3.4: Dynamic and “reactive” clamp in the slice. Recently, a new in vitro experiment has been proposed where one can carry out hybrid experimental–theoretical investigations on the cellular and synaptic mechanisms of persistent activity (Fellous and Sejnowski, 2003). One records intracellularly from a cell in a cortical slice while simultaneously injecting two different sources of synthetically generated synaptic inputs by means of a dynamic clamp setup: on the one hand, continuous Poisson trains of excitatory and inhibitory conductance openings are simulated to represent the input that a cell in vivo presumably receives from neurons from outside the local circuit; the second source of input simulates feedback from the local circuit by injecting a short Poisson train of AMPAR, 147 NMDAR and GABAA-receptor conductance openings each time that the recorded neuron fires one action potential (“reactive clamp”). With appropriate parameters for these inputs, recorded neurons can often be made to display bistability between a quiescent state and a low-rate persistent firing state (see Fig. 4D.1; Fellous and Sejnowski, 2003). This experiment cannot test the existence of an actual connectivity or synaptic substrate for persistent activity (since both are bypassed by the simulation part of the experiment), but it can test computational hypotheses against a real neuron and determine how intrinsic mechanisms influence the dynamics. One conclusion of the experiment of Fellous and Sejnowski (2003), for instance, is that neurons with strong spike-frequency adaptation are unable to be brought into bistability (Fig. 4D.2), as models showed (see Sec. 2.3). A careful examination of the conditions for this persistent activity could help establish, for instance, how high the NMDA-to-AMPA ratio in the feedback needs to be to sustain robust persistent activity, and whether the value of the NMDA-to-AMPA ratio in different cells correlates with the magnitude of a possible intrinsic afterdepolarization (as suggested by Tegner et al. (2002)). This experiment could also be used to implement inhibition-dominated conditions in the local circuit in spontaneous activity and compare the conditions for stable persistent activity there with the case of excitation-dominated recurrence (which is the case in Fellous and Sejnowski (2003)). As it was also the case in the previous experiments discussed, persistent activity in this in vitro preparation is quite variable (though lower than in vivo), with a coefficient of variation approaching 1. Here, however, everything is known regarding connectivity and synapses, and the critical feature for this high variability is the built-in exact balance between excitation and inhibition (Tsodyks and Sejnowski, 1995; van Vreeswijk and Sompolinsky, 1996), in line with the microcolumnar model by Renart (2000). 3.5: Models and in vitro experiments. As outlined in the previous sections, several in vitro experiments are now available that can be used to investigate some of the issues discussed here, but not all, regarding the reverberatory hypothesis for persistent activity in working memory. In the coming years, this exciting field of research will see close cooperation between modeling and in vitro electrophysiology that will address some of the open issues in current theoretical models of persistent activity. I want to emphasize that these conclusions must be taken with caution, as the activity observed in vitro is episodic rather than truly persistent and their mechanisms might therefore differ. Eventually, the consistent findings of models and in vitro experiments will have to be tested against recordings in awake behaving animals in order to determine whether these reverberatory mechanisms are indeed at play in the brain during working memory tasks. In vitro experiments have already given ample evidence of the existence in the cortical circuitry of a reverberatory substrate that can underlie synaptically-based persistent activity. Some of the issues that this joint com- 148 A. Compte / Neuroscience 139 (2006) 135–151 Table 1. Reference table identifying computational issues (first column), mechanisms implicated (second column) by different biophysical models (third column) and experiments in vitro providing some support to the presence of the mechanism in the cortical circuit or to the involvement of the mechanism in addressing the corresponding issue (fourth column) Issue Mechanism Model references Supporting experiments in vitro Maintenance of persistent activity Excitatory reverberation in the local circuit Amari (1977); Hopfield (1982) Intrinsic cellular mechanismsa Booth and Rinzel (1995); Camperi and Wang (1998); Loewenstein and Sompolinsky (2003) Sanchez-Vives and McCormick (2000); Cossart et al. (2003) Egorov et al. (2002) Strong inhibition in the local circuit Amit and Treves (1989); Buhmann (1989); Rubin and Sompolinsky (1989) NMDAR saturation Fransén and Lansner (1995); Wang (1999) Wang (1999); Miller et al. (2003) Low-rate persistent activity Short-term synaptic depressiona Non-silent spontaneous activity Inhibition-dominated recurrence Amit and Treves (1989); Amit (1995); Amit and Brunel (1997) Selectivity during persistent activity Cross-inhibition NMDAR voltage-dependence Fast Hebbian learninga Amit and Brunel (1997) Lisman et al. (1998) Sandberg et al. (2003) Stability against oscillations NMDARs in recurrence Strong inhibition to i-cells Dynamics of spike generationa Slow inward current Wang (1999, 2001) Hansel and Mato (2001) Gutkin et al. (2001) Tegner et al. (2002) Reset of persistent activity External excitatory input . . . . . . recruits massive inhibition . . . synchronizes activity Seamans et al. (2003) Shu et al. (2003) Compte et al. (2000); Brunel and Wang (2001) Gutkin et al. (2001) High CV in persistent activity Fluctuation driven persistent firing with excitatory-inhibitory balance Renart (2000); Renart et al. (2003a) Line attractor robustness Homeostatic mechanisms Discrete bistable units Neuronal bistability Noisy external inputs Renart et al. (2003b) Koulakov et al. (2002); Goldman et al. (2003) Camperi and Wang (1998) Laing and Longtin (2001) Strong external noisy input Laing and Chow (2001) Stability to spike-frequency adaptation Sanchez-Vives and McCormick (2000); Shu et al. (2003); Seamans et al. (2003) Fellous and Sejnowski (2003) Entries in the table are indexed by the computational issue involved (first column), rather than by the mechanism implicated (as is done in the text) to help the reader find its way in both directions. a Not discussed in the text. putational– experimental approach should be able to settle in the future are: (1) cortical inhibition dominance: can an inhibition-dominated regimen be observed experimentally? if so, how does it compare with the models that implement it and to the excitation-dominated regimens? (2) selectivity: can strong surround inhibition be observed such that selectivity is maintained during persistent activity? (3) NMDARs: are NMDARs really indispensable for synaptically sustained activity in vitro? (4) activity variability: what is missing in network models to be able to have as variable a persistent activity as seen in vitro? CONCLUSIONS I have reviewed here the principal cellular and synaptic mechanisms of working memory that network attractor models of persistent activity have put forward to date. Most of these mechanisms revolve around three ideas: strong reverberation, strong inhibitory feedback, and NMDARmediated recurrent connections, each one playing one or several possible roles in the generation, maintenance or stability of persistent activity in working memory (see Sec. 1 and Table 1). At present, attractor models of recurrently A. Compte / Neuroscience 139 (2006) 135–151 generated persistent activity face a number of challenges, notably the structural lack of robustness of models based on line attractors and the high variability of firing observed during persistent activity in working memory tasks. Despite these weaknesses, attractor models are currently the best formulated framework for persistent activity generation and they raise very specific and testable predictions that will surely be the focus of much experimental attention in the coming years. For this reason, I have also reviewed here the current in vitro experimental models for synapticallysustained persistent activity: their strengths, weaknesses, findings and prospects (see Sec. 3) in the hope that both experimenters and modelers will find this useful in following this rapidly developing field of research. The attractor hypothesis for persistent activity has driven all computational research on persistent activity and working memory up to now, but recently there has been increased interest in the fact that typical neuronal firing in working memory tasks includes very significant temporal dynamics, and this suggests that maybe a framework beyond fixed-point attractors might be more appropriate to understand working memory (see also Durstewitz and Seamans in this issue; Brody et al., 2003; Ikegaya et al., 2004). Such a formal framework is presently lacking, but intense research in this direction is under way that will probably yield exciting insights both biologically and theoretically. Acknowledgments—I gratefully acknowledge Alfonso Renart, Maria V. Sanchez-Vives, Miguel Maravall and Alfonso Fairén for helpful comments on the manuscript. Supported by the Spanish Ministry of Education and Science, by the European Regional Development Fund, and by the Volkswagen Foundation. REFERENCES Abeles M (1991) Corticonics: Neural circuits of the cerebral cortex. Cambridge: Cambridge University Press. Abeles M (1992) Role of cortical neuron: integrator or coincidence detector? Israeli J Med Sci 18:83–92. Amari S (1977) Dynamics of pattern formation in lateral-inhibition type neural fields. Biol Cybern 27(2):77– 87. Amit D (1995) The Hebbian paradigm reintegrated: local reverberations as internal representations. Behav Brain Sci 18:617– 626. Amit DJ, Bernacchia A, Yakovlev V (2003) Multiple-object working memory: a model for behavioral performance. Cereb Cortex 13(5):435– 443. Amit DJ, Brunel N (1997) Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex. Cereb Cortex 7(3):237–252. Amit DJ, Mongillo G (2003a) Selective delay activity in the cortex: phenomena and interpretation. Cereb Cortex 13(11):1139 –1150. Amit DJ, Mongillo G (2003b) Spike-driven synaptic dynamics generating working memory states. Neural Comput 15(3):565–596. Amit DJ, Treves A (1989) Associative memory neural network with low temporal spiking rates. Proc Natl Acad Sci U S A 86(20):7871–7875. Amit DJ, Tsodyks MV (1991a) Quantitative study of attractor neural networks retrieving at low spike rates I: substrate-spikes, rates and neuronal gain. Network 2:259 –273. Amit DJ, Tsodyks MV (1991b) Quantitative study of attractor neural networks retrieving at low spike rates II: low rate retrieval in symmetric networks. Network 2:275–294. Ben-Yishai R, Bar-Or RL, Sompolinsky H (1995) Theory of orientation tuning in visual cortex. Proc Natl Acad Sci U S A 92(9):3844 –3848. 149 Booth V, Rinzel J (1995) A minimal, compartmental model for a dendritic origin of bistability of motoneuron firing patterns. J Comput Neurosci 2(4):299 –312. Brody CD, Romo R, Kepecs A (2003) Basic mechanisms for graded persistent activity: discrete attractors, continuous attractors, and dynamic representations. Curr Opin Neurobiol 13(2):204 –211. Brunel N (2000) Persistent activity and the single-cell frequency-current curve in a cortical network model. Network 11(4):261–280. Brunel N (2003) Dynamics and plasticity of stimulus-selective persistent activity in cortical network models. Cereb Cortex 13(11): 1151–1161. Brunel N, Wang XJ (2001) Effects of neuromodulation in a cortical network model of object working memory dominated by recurrent inhibition. J Comput Neurosci 11(1):63– 85. Bugmann G (1991) Summation and multiplication: two distinct operation domains of leaky integrate-and-fire neurons. Netw Comput Neural Syst 2(4):489 –509. http://stacks.iop.org/0954-898X/2/489. Buhmann J (1989) Oscillations and low firing rates in associative memory neural networks. Phys Rev A 40(7):4145– 4148. Camperi M, Wang XJ (1998) A model of visuospatial working memory in prefrontal cortex: recurrent network and cellular bistability. J Comput Neurosci 5(4):383– 405. Compte A, Brunel N, Goldman-Rakic PS, Wang XJ (2000) Synaptic mechanisms and network dynamics underlying spatial working memory in a cortical network model. Cereb Cortex 10(9):910 –923. Compte A, Constantinidis C, Tegner J, Raghavachari S, Chafee MV, Goldman-Rakic PS, Wang X-J (2003a) Temporally irregular mnemonic persistent activity in prefrontal neurons of monkeys during a delayed response task. J Neurophysiol 90(5):3441–3454. Compte A, Harvey M, Puccini GD, Descalzo VF, Reig R, Sanchez-Vives MV (2003b) Fast, synchronized oscillations during activated states in the cortical microcircuit in vitro. Program No. 921.9. 2003 Abstract Viewer/Itinerary Planner. Society for Neuroscience, Washington, DC, Online http://sfn.scholarone.com/itin2003 Compte A, Sanchez-Vives MV, McCormick DA, Wang X-J (2003c) Cellular and network mechanisms of slow oscillatory activity (⬍1 Hz) and wave propagations in a cortical network model. J Neurophysiol 89(5):2707–2725. Constantinidis C, Wang X-J (2004) A neural circuit basis for spatial working memory. Neuroscientist 10(6):553–565. Cossart R, Aronov D, Yuste R (2003) Attractor dynamics of network UP states in the neocortex. Nature 423(6937):283–288. Deco G, Rolls ET (2003) Attention and working memory: a dynamical model of neuronal activity in the prefrontal cortex. Eur J Neurosci 18(8):2374 –2390. Deco G, Rolls ET (2005) Synaptic and spiking dynamics underlying reward reversal in the orbitofrontal cortex. Cereb Cortex 15(1): 15–30. Deco G, Rolls ET, Horwitz B (2004) “What” and “where” in visual working memory: a computational neurodynamical perspective for integrating FMRI and single-neuron data. J Cogn Neurosci 16(4): 683–701. Delgado-Garcia JM (2000) Why move the eyes if we can move the head? Brain Res Bull 52(6):475– 482. Douglas RJ, Martin KAC (2004) Neuronal circuits of the neocortex. Annu Rev Neurosci 27:419 – 451. Durstewitz D, Seamans JK (2002) The computational role of dopamine D1 receptors in working memory. Neural Netw 15(4 – 6):561–572. Durstewitz D, Seamans JK (2005) Beyond bistability: The biophysics and temporal dynamics of working memory. Neuroscience Durstewitz D, Seamans JK, Sejnowski TJ (2000a) Dopamine-mediated stabilization of delay-period activity in a network model of prefrontal cortex. J Neurophysiol 83(3):1733–1750. Durstewitz D, Seamans JK, Sejnowski TJ (2000b) Neurocomputational models of working memory. Nat Neurosci 3 (Suppl):1184–1191. Egorov AV, Hamam BN, Fransen E, Hasselmo ME, Alonso AA (2002) Graded persistent activity in entorhinal cortex neurons. Nature 420(6912):173–178. 150 A. Compte / Neuroscience 139 (2006) 135–151 Ermentrout B (2003) Dynamical consequences of fast-rising, slowdecaying synapses in neuronal networks. Neural Comput 15(11): 2483–2522. Fellous J-M, Sejnowski TJ (2003) Regulation of persistent activity by background inhibition in an in vitro model of a cortical microcircuit. Cereb Cortex 13(11):1232–1241. Fransén E, Lansner A (1995) Low spiking rates in a population of mutually exciting pyramidal cells. Network 6(2):271–288. Funahashi S, Bruce CJ, Goldman-Rakic PS (1989) Mnemonic coding of visual space in the monkey’s dorsolateral prefrontal cortex. J Neurophysiol 61(2):331–349. Fuster J (1995) Memory in the cerebral cortex. Cambridge, MA: MIT Press. Goldman MS, Levine JH, Major G, Tank DW, Seung HS (2003) Robust persistent neural activity in a model integrator with multiple hysteretic dendrites per neuron. Cereb Cortex 13(11):1185–1195. Goldman-Rakic PS (1995) Cellular basis of working memory. Neuron 14(3):477– 485. Gonzalez-Burgos G, Krimer LS, Povysheva NV, Barrionuevo G, Lewis DA (2005) Functional properties of fast spiking interneurons and their synaptic connections with pyramidal cells in primate dorsolateral prefrontal cortex. J Neurophysiol 93(2):942–953. Gutkin BS, Laing CR, Colby CL, Chow CC, Ermentrout GB (2001) Turning on and off with excitation: the role of spike-timing asynchrony and synchrony in sustained neural activity. J Comput Neurosci 11(2):121–134. Hansel D, Mato G (2001) Existence and stability of persistent states in large neuronal networks. Phys Rev Lett 86(18):4175– 4178. Hansel D, Mato G (2003) Asynchronous states and the emergence of synchrony in large networks of interacting excitatory and inhibitory neurons. Neural Comput 15(1):1–56. Hansel D, Sompolinsky H (1998) Modeling feature selectivity in local cortical circuits. In: Methods in neuronal modeling: from ions to networks (Koch C, Segev I, eds), pp 499 –567. Cambridge, MA: MIT Press. Hebb D (1949) The organization of behavior. Wiley. Hopfield JJ (1982) Neural networks and physical systems with emergent collective computational abilities. Proc Natl Acad Sci U S A 79(8):2554 –2558. Ikegaya Y, Aaron G, Cossart R, Aronov D, Lampl I, Ferster D, Yuste R (2004) Synfire chains and cortical songs: temporal modules of cortical activity. Science 304(5670):559 –564. Kang K, Shelley M, Sompolinsky H (2003) Mexican hats and pinwheels in visual cortex. Proc Natl Acad Sci U S A 100(5): 2848 –2853. Koulakov AA (2001) Properties of synaptic transmission and the global stability of delayed activity states. Network 12(1):47–74. Koulakov AA, Raghavachari S, Kepecs A, Lisman JE (2002) Model for a robust neural integrator. Nat Neurosci 5(8):775–782. Laing C, Longtin A (2001) Noise-induced stabilization of bumps in systems with long-range spatial coupling. Physica D 160(3– 4): 149 –172. Laing CR, Chow CC (2001) Stationary bumps in networks of spiking neurons. Neural Comput 13(7):1473–1494. Larkum ME, Senn W, Luscher H-R (2004) Top-down dendritic input increases the gain of layer 5 pyramidal neurons. Cereb Cortex 14(10):1059 –1070. Latham PE, Nirenberg S (2004) Computing and stability in cortical networks. Neural Comput 16(7):1385–1412. Lee H, Simpson GV, Logothetis NK, Rainer G (2005) Phase locking of single neuron activity to theta oscillations during working memory in monkey extrastriate visual cortex. Neuron 45(1):147–156. Lisman JE, Fellous JM, Wang XJ (1998) A role for NMDA-receptor channels in working memory. Nat Neurosci 1(4):273–275. Loewenstein Y, Sompolinsky H (2003) Temporal integration by calcium dynamics in a model neuron. Nat Neurosci 6(9):961–967. Lorente de No R (1933) Vestibulo-ocular reflex arc. Arch Neurol Psychiatry 30:245–291. Lund JS, Wu CQ (1997) Local circuit neurons of macaque monkey striate cortex: IV. Neurons of laminae -3A. J Comp Neurol 384(1):109 –126. Major G, Tank D (2004) Persistent neural activity: prevalence and mechanisms. Curr Opin Neurobiol 14(6):675– 684. Mao BQ, Hamzei-Sichani F, Aronov D, Froemke RC, Yuste R (2001) Dynamics of spontaneous activity in neocortical slices. Neuron 32(5):883– 898. Markram H, Toledo-Rodriguez M, Wang Y, Gupta A, Silberberg G, Wu C (2004) Interneurons of the neocortical inhibitory system. Nat Rev Neurosci 5(10):793– 807. McCormick DA, Aptowicz C (2004) Organotypic cultures of cerebral cortex generate periods of persistent activity through recurrent excitation and inhibition. Program No. 973.12. 2004 Abstract Viewer/Itinerary Planner. Society for Neuroscience, Washington DC, Online http:// sfn.scholarone.com/itin2004/ McCormick DA, Shu Y, Hasenstaub A, Sanchez-Vives M, Badoual M, Bal T (2003) Persistent cortical activity: mechanisms of generation and effects on neuronal excitability. Cereb Cortex 13(11): 1219 –1231. Melchitzky DS, Gonzalez-Burgos G, Barrionuevo G, Lewis DA (2001) Synaptic targets of the intrinsic axon collaterals of supragranular pyramidal neurons in monkey prefrontal cortex. J Comp Neurol 430(2):209 –221. Miller P, Brody CD, Romo R, Wang X-J (2003) A recurrent network model of somatosensory parametric working memory in the prefrontal cortex. Cereb Cortex 13(11):1208 –1218. Miller P, Brody CD, Romo R, Wang X-J (2002) Parametric working memory: Trade-off between robust bistabilities and fine-tuning to continuous line attractors. Program No. 187.5 2002 Abstract Viewer/ Itinerary Planner. Society for Neuroscience, Washington DC, Online http://sfn.scholarone.com/itin2002 Mongillo G, Amit DJ, Brunel N (2003) Retrospective and prospective persistent activity induced by Hebbian learning in a recurrent cortical network. Eur J Neurosci 18(7):2011–2024. Moreno-Bote R, Parga N (2004) Role of synaptic filtering on the firing response of simple model neurons. Phys Rev Lett 92(2):028102. Navarro-Lopez J de D, Alvarado JC, Marquez-Ruiz J, Escudero M, Delgado-Garcia JM, Yajeya J (2004) A cholinergic synaptically triggered event participates in the generation of persistent activity necessary for eye fixation. J Neurosci 24(22):5109 –5118. Pasternak T, Greenlee MW (2005) Working memory in primate sensory systems. Nat Rev Neurosci 6(2):97–107. Pesaran B, Pezaris JS, Sahani M, Mitra PP, Andersen RA (2002) Temporal structure in neuronal activity during working memory in macaque parietal cortex. Nat Neurosci 5(8):805– 811. Plenz D, Aertsen A (1996) Neural dynamics in cortex-striatum cocultures-II. Spatiotemporal characteristics of neuronal activity. Neuroscience 70(4):893–924. Ploner CJ, Gaymard B, Rivaud S, Agid Y, Pierrot-Deseilligny C (1998) Temporal limits of spatial working memory in humans. Eur J Neurosci 10(2):794 –797. Renart A (2000) Models of multi-areal cortical processing. Ph.D. thesis, Universidad Autónoma de Madrid, Madrid, Spain. Renart A, Brunel N, Wang X-J (2003a) Mean-field theory of recurrent cortical networks: working memory circuits with irregularly spiking neurons. In: Computational neuroscience: a comprehensive approach (Feng J, ed), pp 431– 490. Boca Raton, FL: CRC Press. Renart A, Song P, Wang X-J (2003b) Robust spatial working memory through homeostatic synaptic scaling in heterogeneous cortical networks. Neuron 38(3):473– 485. Renart A, Moreno R, de la Rocha J, Parga N, Rolls ET (2001) A model of the IT-PF network in object working memory which includes balanced persistent activity and tuned inhibition. Neurocomputing 38 – 40:1525–1531. Romo R, Brody CD, Hernandez A, Lemus L (1999) Neuronal correlates of parametric working memory in the prefrontal cortex. Nature 399(6735):470 – 473. A. Compte / Neuroscience 139 (2006) 135–151 Rubin N, Sompolinsky H (1989) Neural networks with low local firing rates. Europhys Lett 10(5):465– 470. Salinas E, Sejnowski TJ (2002) Integrate-and-fire neurons driven by correlated stochastic input. Neural Comput 14(9):2111–2155. Sanchez-Vives MV, McCormick DA (2000) Cellular and network mechanisms of rhythmic recurrent activity in neocortex. Nat Neurosci 3(10):1027–1034. Sandberg A, Tegner J, Lansner A (2003) A working memory model based on fast Hebbian learning. Network 14(4):789 – 802. Seamans JK, Nogueira L, Lavin A (2003) Synaptic basis of persistent activity in prefrontal cortex in vivo and in organotypic cultures. Cereb Cortex 13(11):1242–1250. Seung HS (1996) How the brain keeps the eyes still. Proc Natl Acad Sci U S A 93(23):13339 –13344. Seung HS, Lee DD, Reis BY, Tank DW (2000a) Stability of the memory of eye position in a recurrent network of conductancebased model neurons. Neuron 26(1):259 –271. Seung HS, Lee DD, Reis BY, Tank DW (2000b) The autapse: a simple illustration of short-term analog memory storage by tuned synaptic feedback. J Comput Neurosci 9(2):171–185. Shu Y, Hasenstaub A, McCormick DA (2003) Turning on and off recurrent balanced cortical activity. Nature 423(6937):288 –293. Shu Y, Hasenstaub DA, McCormick DA (2004) Control of cortical somatodendritic communication by recurrent network activity and the h-current. Program No. 641.8. 2004 Abstract Viewer/Itinerary Planner. Society for Neuroscience, Washington, DC, Online http:// sfn.scholarone.com/itin2004/ Silberberg G, Wu C, Markram H (2004) Synaptic dynamics control the timing of neuronal excitation in the activated neocortical microcircuit. J Physiol (Lond) 556(1):19–27. http://jp.physoc.org/cgi/content/ abstract/556/1/19. Somogyi P, Kisvarday ZF, Martin KA, Whitteridge D (1983) Synaptic connections of morphologically identified and physiologically characterized large basket cells in the striate cortex of cat. Neuroscience 10(2):261–294. Steriade M (2000) Corticothalamic resonance, states of vigilance and mentation. Neuroscience 101(2):243–276. Steriade M, Timofeev I, Grenier F (1999) Intracellular activity of various neocortical cell-classes during the natural wake-sleep cycle. 151 Society for Neuroscience. 29th Annual Meeting. Miami Beach, FL. October 23–28. Session 664.14, p 1661. Svirskis G, Rinzel J (2000) Influence of temporal correlation of synaptic input on the rate and variability of firing in neurons. Biophys J 79(2):629 – 637. http://www.biophysj.org/cgi/content/abstract/79/ 2/629. Tanaka S (2005) Dopaminergic control of working memory processing and its relevance to schizofrenia: a computational circuit perspective. Neuroscience Taube JS, Bassett JP (2003) Persistent neural activity in head direction cells. Cereb Cortex 13(11):1162–1172. Tegner J, Compte A, Wang X-J (2002) The dynamical stability of reverberatory neural circuits. Biol Cybern 87(5– 6):471– 481. Tsodyks MV, Sejnowski T (1995) Rapid state switching in balanced cortical network models. Network 6:111–124. van Vreeswijk C, Sompolinsky H (1996) Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science 274(5293): 1724–1726. Wang XJ (1999) Synaptic basis of cortical persistent activity: the importance of NMDA receptors to working memory. J Neurosci 19(21):9587–9603. Wang XJ (2001) Synaptic reverberation underlying mnemonic persistent activity. Trends Neurosci 24(8):455– 463. Wang X-J, Tegner J, Constantinidis C, Goldman-Rakic PS (2004) Division of labor among distinct subtypes of inhibitory neurons in a cortical microcircuit of working memory. Proc Natl Acad Sci U S A 101(5):1368 –1373. White JM, Sparks DL, Stanford TR (1994) Saccades to remembered target locations: an analysis of systematic and variable errors. Vision Res 34(1):79 –92. Wilson HR, Cowan JD (1973) A mathematical theory of the functional dynamics of cortical and thalamic nervous tissue. Kybernetik 13(2):55– 80. Yoshimura Y, Dantzker JL, Callaway EM (2005) Excitatory cortical neurons form fine-scale functional networks. Nature 433(7028):868–873. Zhang K (1996) Representation of spatial orientation by the intrinsic dynamics of the head-direction cell ensemble: a theory. J Neurosci 16(6):2112–2126. Zipser D, Kehoe B, Littlewort G, Fuster J (1993) A spiking network model of short-term active memory. J Neurosci 13(8):3406 –3420. (Accepted 3 June 2005) (Available online 6 December 2005)