Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Wien bridge oscillator wikipedia , lookup
Resistive opto-isolator wikipedia , lookup
Power MOSFET wikipedia , lookup
Regenerative circuit wikipedia , lookup
Index of electronics articles wikipedia , lookup
Switched-mode power supply wikipedia , lookup
Schmitt trigger wikipedia , lookup
Opto-isolator wikipedia , lookup
Valve RF amplifier wikipedia , lookup
Operational amplifier wikipedia , lookup
RLC circuit wikipedia , lookup
Current mirror wikipedia , lookup
LETTER Communicated by Kwabena Boahen Modeling Short-Term Synaptic Depression in Silicon Malte Boegerhausen [email protected] Pascal Suter [email protected] Shih-Chii Liu [email protected] Institute of Neuroinformatics, University of Zurich and ETH Zurich, Winterthurerstrasse 190, CH-8057 Zurich, Switzerland We describe a model of short-term synaptic depression that is derived from a circuit implementation. The dynamics of this circuit model is similar to the dynamics of some theoretical models of short-term depression except that the recovery dynamics of the variable describing the depression is nonlinear and it also depends on the presynaptic frequency. The equations describing the steady-state and transient responses of this synaptic model are compared to the experimental results obtained from a fabricated silicon network consisting of leaky integrate-and-re neurons and different types of short-term dynamic synapses. We also show experimental data demonstrating the possible computational roles of depression. One possible role of a depressing synapse is that the input can quickly bring the neuron up to threshold when the membrane potential is close to the resting potential. 1 Introduction Short-term synaptic dynamics has been observed in different parts of the cortical system (Stratford, Tarczy-Hornoch, Martin, Bannister, & Jack, 1998; Varela et al., 1997; Markram, Wang, & Tsodyks, 1998; Reyes et al., 1998). The functionality of the short-term dynamics has been implicated in various cortical models (Senn, Segev, & Tsodyks, 1998; Chance, Nelson, & Abbott, 1998; Matveev & Wang, 2000). The properties of a network with dynamic synapses have been investigated by Tsodyks, Pawelzik, and Markram (1998), and Maass and Zador (1999) and the use of such a network in time series processing by Maass and Zador (1999) and Natschlager, Maass, and Zador (2001). The introduction of these dynamic synapses into hardware implementations of recurrent neuronal networks allows a wide range of operating regimes, especially in the case of time-varying inputs. In this work, we describe a model that was derived from a circuit implementation of short-term depression and facilitation. The dynamics of this Neural Computation 15, 331–348 (2003) c 2002 Massachusetts Institute of Technology ° 332 M. Boegerhausen, P. Suter, and S. Liu circuit, initially proposed by Rasche and Hahnloser (2001), was not analyzed in their work. Our work is presented in two parts. We rst describe the model of this synaptic circuit and compare the differential equations describing the depression with the equations of one of the theoretical models frequently used in network simulations (Abbott, Sen, Varela, & Nelson, 1997; Varela et al., 1997). We analyze the transient and steady-state responses of this synaptic model in response to inputs of different statistical distributions. Next, we compare the theoretical results with experimental responses obtained from a silicon network of leaky integrate-and-re neurons together with different short-term dynamic synapses. We also show experimental data from the chip that demonstrate the possible computational roles of depression. From these experiments, we postulate that one possible role of depression is to bring the neuron’s response up to threshold quickly if the membrane potential of the neuron was close to the resting potential. We also mapped a proposed cortical model of direction selectivity that uses depressing synapses onto this chip. The results are qualitatively similar to the results obtained in the original work (Chance et al., 1998). The similarity of the circuit responses to the responses from Abbott and colleagues’ model means that we can use these VLSI networks of integrateand-re (IF) neurons as an alternative to computer simulations of dynamical networks composed of large numbers of integrate-and-re (IF) neurons using synapses with different time constants. The outputs of such networks can also be used as an interface with wetware. An infrastructure for a reprogrammable, recongurable, multichip neuronal system is being developed along with a user-dened interface so that the system is easily accessible to a naive user. 2 Comparisons Between Models of Depression Two theoretical models describing synaptic depression and facilitation used frequently in network simulations are the phenomenological model of Tsodyks and Markram (1997) and the model from Abbott et al. (1997). Because these models are equivalent if the inactivation synaptic time constant in Tsodyks and Markram’s model is small when compared with the recovery time constant of the depression, we compare the circuit model only with the model from Abbott and colleagues. Here, we describe only the circuit model for synaptic depression. The equivalent model for facilitation is described elsewhere (Liu, in press). 2.1 Theoretical Model for Depression Model. In the model from Abbott et al. (1997), the synaptic strength is described by gD (t), where D is a variable between 0 and 1 that describes the amount of depression (D D 1 means no depression) and g is the maximum synaptic strength. The recovery Modeling Short-Term Synaptic Depression in Silicon 333 dynamics of D is td dD D 1 ¡ D, dt (2.1) where td is the recovery time of the depression. The update equation for D right after a spike at time t D tsp is C ¡ ) D dD (tsp ), D(tsp (2.2) where d (d < 1) is the amount by which D is decreased right after the spike and tsp is the time of the spike. The average steady-state value of depression for a regular spike train with a rate r is Dss D 1 ¡ e¡1/ (rtd ) . 1 ¡ de ¡1 / (rtd ) (2.3) 2.2 Circuit Model of Depressing Synapse. In this model of synaptic depression derived from the circuit implementation, the equation that describes the recovery dynamics of the depressing variable, D, is nonlinear. This nonlinearity comes about because there is no compact circuitry to implement a linear resistor whose value can be adjusted easily after fabrication. A transistor can act as a linear resistor as long as the terminal voltages satisfy certain criteria. Additional circuitry would be needed to satisfy these criteria for a wide range of voltages, and it would increase the size of the circuit. One alternative is to replace the exponential dynamics in equation 2.1 with diode dynamics, which is easily obtained with a single diode-connected transistor. The current through the diode has a nonlinear decay instead of an exponential decay in the case of a linear resistor. Because of the different dynamics, the equation describing the recovery of D (derived from the circuit in the region where a transistor operates in the subthreshold region or the current is exponential in the gate voltage of the transistor) can be formulated as dD 1k D M (1 ¡ D / ), dt (2.4) where 1 / M is the equivalent of td in equation 2.1 and k (a transistor parameter) is less than 1. The maximum value of D is 1. The update equation remains as before: C ¡ ) D dD (tsp ). D(tsp (2.5) 2.2.1 Circuit. Equations 2.4 and 2.5 are derived from the circuit in Figure 1. The operation of this circuit is described in the caption, and the detailed analysis leading to the differential equations for D is described in the 334 M. Boegerhausen, P. Suter, and S. Liu Figure 1: Schematic for a depressing synapse circuit and responses to a regular input spike train. (a) Depressing synapse circuit. The voltage Va determines the synaptic conductance g, while the synaptic term gD or Isyn is exponential in the voltage, Vx . The subcircuit consisting of transistors, M1 , M2 , and M3 controls the dynamics of Isyn. The presynaptic input goes to the gate terminal of M3 , which acts like a switch. When there is a presynaptic spike, a quantity of charge (determined by Vd ) is removed from the node Vx . In between spikes, Vx recovers to the voltage, Va , through the diode-connected transistor, M1 . When there is no spike, Vx is around Va . When the presynaptic input comes from a regular spike train, Vx decreases with each spike and recovers between spikes. It reaches a steady-state value, as shown in b. During the spike, transistor M4 turns on, and the synaptic weight current Isyn charges up the membrane potential of the neuron through the current-mirror circuit consisting of M6 , M7 , and the capacitor C2 . We can convert the Isyn current source into a synaptic current Id with some gain and a “time constant” by adjusting the voltage Vgain . The decay dynamics of Id is given by Id (t) D Isyn (t D tsp ) 1C AIsyn (tD tsp ) QT k (Vdd ¡Vgain ) / UT where QT D C2 UT and A D e . In a normal synapse circuit (i.e., without short-term dynamics), Vx is controlled by an external bias voltage. (b) Input spike train at a frequency of 20 Hz (bottom curve) and corresponding response Vx (top curve) of the circuit for Vd D 0.3 V. The diode-connected transistor M1 has nonlinear dynamics. This dynamics is discussed in the text. The recovery time of the variable D varies depending on the distance of the present value of Vx from Va . The recovery rate of D increases for a larger difference between Vx and Va . appendix. The voltage Vx in Figure 1 codes for gD. The conductance g is set by Va , while the dynamics of D is set by both Vd and Va . The time taken for the present value of Vx to return to the value of Va is determined by the current dynamics of the diode-connected transistor M1 and Va . As shown in the appendix, the recovery time constant (1 / M) of D is set by Va . Modeling Short-Term Synaptic Depression in Silicon 335 The synaptic weight is described by the current, Isyn in Figure 1a, Isyn (t) D Ion ek Vx / UT D gD (t), where g D Ion ek Va / UT is the synaptic strength, D(t) is k (Vdd ¡Vx ) / UT Irf D Iop e (2.6) k (V e dd ¡Va ) / UT Iop , Irf (t) and . The recovery time constant (1 / M) of D is set by Va Iopk ¡(1¡k ) (V ¡Va ) / UT dd ). QT e (M D The synaptic current, Id to the neuron, is then a current source Isyn , which lasts for the duration of the pulse width of the presynaptic spike. However, we can set a longer time constant for the synaptic current through Vgain . The equation describing this dependence (that is, the equation of the current in a current-mirror circuit) is given in the caption of Figure 1. It is difcult to compute a closed-form solution for equation 2.4 for any value of k (a transistor parameter which is less than 1). This value also changes under different operating conditions and between transistors fabricated in different processes. Hence, we solve for D(t) in the case of k D 0.5 given that the last spike occurred at t D t0 1 : dD / dt D M (1 ¡ D2 ) ) D(t) D D D(t 0 ) C D(t0 )e2Mt ¡ 1 C e2Mt ¡D(t0 ) C D(t0 )e2Mt C 1 C e2Mt D(t0 ) cosh (Mt) C sinh (Mt) . cosh (Mt) C D(t0 ) sinh (Mt) (2.7) When D is far from its recovered value of 1, we can approximate its recovery dynamics by dD / dt D M (irrespective of k ), and solving for D(t), we get D(t) D Mt C D(t 0 ). In this regime, D(t) follows a linear trajectory. Note that the same is true of equation 2.1 when t ¿ td . 3 Neuron Circuit The dynamics of the neuron circuit of Figure 2 is similar to that of a leaky IF neuron with a constant leak. The circuit was previously described in Indiveri (2000) and Liu et al. (2001). It is a modied version of previous designs (Mead, 1989; Van Schaik, 2001) and also includes the circuitry that models ring-rate adaptation (Boahen, 1997a, 1997b) frequently seen in pyramidal 1 Note that if k =1, the equation reduces to equation 2.1. 336 M. Boegerhausen, P. Suter, and S. Liu C1 Vb V thresh Id V leak Ileak Cm Vo Vm M5 V pw M6 Vo Vo Vm M1 Vo M2 Vca M4 M3 V refr Ipw Ca Vt Spike adaptation circuitry Figure 2: Schematic of the leaky integrate-and-re neuron circuit. In this gure, Vrefr, sets the refractory period, Vthresh sets the threshold voltage, Vpw sets the pulse width of the spike, Vleak sets the leak current Ileak, and VCa and Vt set the output spike adaptation dynamics. cells in the cortex. The equation describing the depolarization of the soma potential Vm is Cm dV m (t) D i(t) ¡ Ileak ¡ Iahp , dt Vm (t) < Vthresh , (3.1) where i(t) is the synaptic current to the soma, Ileak is the leakage current, and Iahp is the after-hyperpolarization potassium (K) current, which causes the adaptation in the ring rate of the cells. The synaptic current can be set to a point current source, that is, i(t) D Idd (t ¡tjk ), or it can have a “time constant” and gain that is determined by the current-mirror circuit consisting of M6 , M7 , and C2 in Figure 1. The time constant and gain are controlled by the Isyn (tD tsp ) voltage Vgain . The decay dynamics of Id is of the form Id (t) D AIsyn (tD tsp ) , kV dd ¡Vgain 1C QT UT where QT D C2 UT and A D e . The operation of the circuit is as follows. When Vm (t) increases above Vthresh at ts D t (ts is the time of the output spike), it increases by a step increment determined by the amplitude of the spike and the capacitive divider set by C1 and Cm . The output Vo becomes active at this time and turns on the discharging current path through transistors M5 and M6 . The time during which Vo remains high, TP , depends on the time taken for Vm to discharge below Vthresh . In this design, the pulse width TP is determined by the rate at which Vm is discharged, which in turn depends on the difference between the input current Id , the leak current Ileak, and the current Ipw . In other designs, Vm is reset immediately below Vthresh when Vo becomes active because either the input current is blocked from charging the membrane or Modeling Short-Term Synaptic Depression in Silicon 337 the current Ipw is much larger than the input current. The refractory period, TR , is determined by Vrefr , which keeps Vo high so that Id cannot charge up the membrane. The spike output is taken from the node VNo . We ignore the Iahp current in determining the ring rate dependence on the input current. The time needed for the neuron to charge up to threshold Vthresh with the input current is TI D Cm i(t)¡I . The spiking rate is given by rD leak 1 TI C TP C TR . 4 Comparison Between Models We compare the two models by looking at how D changes in response to a Poisson-distributed train whose frequency varied from 40 Hz to 1 Hz, as shown in Figure 3. We used a simple linear differential equation to describe the dynamics of the membrane potential Vm , tm dV m (t) D Ri(t) ¡ Vm (t), dt where tm is the membrane time constant and i(t) is the synaptic current. We ran an optimization algorithm on the parameters in the two models so that the least-square error between the excitatory postsynaptic potential (EPSP) outputs of both models was at a minimum. In this case, the EPSP responses were identical (see Figure 3b) and the corresponding D values (see Figure 3c) were almost identical except in the region where D was close to the maximum value. We performed the same comparison with Tsodyks and Markram’s model, and the results were similar. Hence, the circuit model can be used to describe short-term synaptic depression in a network simulation. However, the nonlinear recovery dynamics of the circuit model leads a different functional dependence of the average steady-state EPSP on the frequency of a regular input spike train. 5 Transient Response The data in the gures in the remainder of this article are obtained from a fabricated silicon network of aVLSI IF neurons of the type described in section 3 together with different types of synapses. We rst measured the transient response of the neuron when stimulated by a 10 Hz spike train through the depressing synapse. We tuned the parameters of the synapse and the leak current so that the membrane potential did not build up to threshold. These data are shown in Figure 4a for a regular train and in Figure 4b for a Poisson-distributed train. The measured EPSP amplitude, DVm can be tted with equation 2.6, DVm (t) D dt dt (Isyn (t) ¡ Ileak ) D (gD (t) ¡ Ileak ), Cm Cm (5.1) 338 M. Boegerhausen, P. Suter, and S. Liu Figure 3: Comparison between the outputs of the two models of depression. An optimization algorithm was used to determine the parameters of the models so that the least-square error between the EPSPs from the two models was at a minimum. The corresponding D distribution is shown in c. (a) Poissondistributed input with an initial frequency of 40 Hz and an end frequency of 1 Hz. (b) The EPSP responses of both models were identical. (c) The D values were almost identical except in the region when D is close to 1. Parameters used in the simulations: tm D 20 ms, d D 0.6, k D 0.7, M D 2.20 s ¡1 . where Cm is the membrane capacitance, Ileak is the leak current, dt is the pulse width of the presynaptic spike, and D(t) is computed using equation 2.7. Equation 5.1 ts the measured data well (see Figure 4). 6 Steady-State Response The equation describing the dependence of the steady-state values of D on the presynaptic frequency can easily be determined in the case of a regular spiking input of rate r by using equations 2.5 and 2.7. From these equations, we get p (¡1 C d) (1 C e2M/ r ) C 4d(¡1 C e2M/ r ) 2 C (1¡d) 2 (1 C e2M/ r )2 Dss D . (6.1) 2d(¡1 C e2M/ r ) Modeling Short-Term Synaptic Depression in Silicon 339 Figure 4: Transient responses to a (a) 10 Hz regular spike train and a (b) 10 Hz Poisson-distributed train. The input is the bottom curve of each plot. In a, the amplitude of the EPSP decreases with each incoming input spike, clearly showing the effect of synaptic depression. In b, the EPSP amplitude depends on the occurrence of the previous spike. The asterisks are the ts of the circuit model to the peak value of each EPSP. The ts give a d value of 0.79. 340 M. Boegerhausen, P. Suter, and S. Liu If we use the reduced dynamics expression dD / dt D M, we replace equation 2.7 with equation 3.1 and obtain a simpler expression for Dss : Dss D M . (1 ¡ d)r (6.2) This equation shows that the steady-state D, and, hence, the steady-state EPSP amplitude, is inversely dependent on the presynaptic rate r. The form of the curve is similar to the results obtained by Abbott et al. (1997), where the data can be tted with equation 2.3. From the chip, we measured the steady-state EPSP amplitudes using the two-input spike distributions changing over a frequency range of 3 Hz to 50 Hz in steps of 1 Hz. Each frequency interval lasted 15 s, and the EPSP amplitude was averaged in the last 5 s to obtain the steady-state value. Four separate trials were performed; the resulting mean and the variance of the measurements are shown in Figure 5 for both the regular train and the Poisson-distributed train. We tted the experimental data using equations 5.1 and 6.1. The parameters from the ts to the data in Figure 5a were then used to generate the tted curve to the data from the Poissondistributed train. The values from the ts give recovery time constants from 1 to 3 s and Dss values varying between 0.02 and 0.04. 7 Role of Synaptic Depression Different computational roles have been proposed for networks that incorporate synaptic depression. In this section, we describe some measurements that illustrate the postulated roles of depression. 7.1 Gain Control. Depressing synapses have been implicated in cortical gain control (Abbott et al., 1997). A depressing synapse acts like a transient detector to changes in frequency (or a rst derivative lter) similar to the way in which the photoreceptor responds with a higher gain to contrast and is less responsive to background illumination. A synapse with short-term depression responds equally to equal percentage rate changes in its input on different ring rates. We demonstrate the gain control mechanism of short-term depression by measuring the neuron’s response to step changes in input frequency from 10 Hz to 20 Hz to 40 Hz. Each step change is the same rate change in input frequency. These results are shown in Figure 7.1a for a regular train and in Figure 6b for a Poisson-distributed train. Each frequency epoch lasted 3 s, so the synaptic strength should have reached steady state before the next step increase in input frequency. For both gures in Figure 7.1, the top curve shows the response of the neuron when stimulated by the input (bottom curve) through a depressing synapse (top curve) and a nondepressing synapse (middle curve). We tuned the parameters of Modeling Short-Term Synaptic Depression in Silicon 341 Figure 5: Dependence of the steady-state EPSP amplitude on the input frequency for different values of depression. The data were measured from the fabricated circuit. The solid lines are ts using equations 5.1 and 6.1. The parameters for the ts are determined using the data in a. (a) Steady-state EPSP amplitude versus frequency for a regular train. From the retrieved values of d (between 0.2 and 0.6), we nd that Dss varies between 0.02 to 0.04. (b) Steadystate EPSP amplitude versus frequency for a Poisson-distributed train. 342 M. Boegerhausen, P. Suter, and S. Liu both types of synapses in Figure 6b so that the steady-state ring rate of the neuron was similar during the 20 Hz input period. Figure 6a shows clearly the transient increase in the ring rate of a neuron when stimulated through a depressing synapse right after each step increase in input frequency and the subsequent adaptation of its ring rate to a steady-state value. The steady-state ring rate of the neuron with a depressing synapse is less dependent on the absolute input frequency when compared to the ring rate of the neuron when stimulated through the nondepressing synapse. In the latter case, the ring rate of the neuron is approximately linear in the input rate. The data shown in Figure 6a were obtained using a Poisson-distributed input. One obvious difference in the responses between the depressing and nondepressing synapse is that the neuron quickly reached threshold for a 10 Hz input for the depressing-synapse case, while the neuron remained subthreshold in the nondepressing synapse case until the input increased to 20 Hz. This suggests that a depressing synapse can be used to drive a neuron quickly to threshold when the membrane potential of the neuron is far from the threshold. 7.2 Phase Advance. Another feature of the depressing synapse is the phase advance of the membrane response of the neuron. This advance increases when the depression increases (see Figure 7). This feature was exploited in a model that described the direction-selective responses of visual cortical neurons (Chance et al., 1998). In this model, the neuron was driven by the output of a set of lateral geniculate nucleus (LGN) cells through depressing synapses and through an adjacent spatially shifted set of LGN cells through nondepressing synapses. We have attempted the same experiment using a neuron on our chip by using spikes recorded from an LGN Figure 6: Facing page. Response of neuron to a step change in input frequency (bottom curve) when stimulated through a depressing synapse (top curve) and a nondepressing synapse (middle curve). The neuron was stimulated for three frequency intervals (10 Hz to 20 Hz to 40 Hz) lasting 3 s each. (a) Response of neuron using a regular spiking input. The steady-state ring rate of the neuron increased almost linearly with the input frequency when stimulated through the nondepressing synapse. In the depressing synapse curve, there is a transient increase in the neuron’s ring rate before the rate adapted to steady state. The steady-state output is less dependent on the absolute input frequency. (b) Response of neuron using a Poisson-distributed input. The parameters for both types of synapses were tuned so that the steady-state ring rates were about the same at the end of each frequency interval for both synapses. Notice that during the 10 Hz interval, the neuron quickly built up to threshold if it was stimulated through the depressing synapse, but it stayed subthreshold when it was driven through to the nondepressing synapse until the input frequency reached 20 Hz. Modeling Short-Term Synaptic Depression in Silicon 343 344 M. Boegerhausen, P. Suter, and S. Liu Figure 7: Phase advance in the membrane response for different levels of depression in the synapse. The neuron was driven through the synapse by a Poisson-distributed input depicted at the bottom of the gure. The neuron shows the largest phase advance when the synapse has the largest depression value (Vd D 0.3V has the largest d value in the plot). cell in the cat visual cortex during stimulation with a drifting sinusoidal grating (courtesy of K. Martin). An example of the direction-selective response is shown in Figure 8 (Liu, 2001). The direction-selective results were qualitatively similar to the data in Chance et al. (1998). 8 Conclusion We described a model of synaptic depression derived from a circuit implementation. This circuit model has nonlinear recovery dynamics for the depression variable in contrast to current theoretical models of dynamic synapses. However, it gives qualitatively similar results when compared to the model of Abbott and colleagues (1997). Measured data from a chip with aVLSI IF neurons and dynamic synapses show that this network can be used to simulate the responses of dynamic networks with short-term dynamical synapses. Experimental results suggest that depressing synapses can be used to drive a neuron quickly up to threshold if its membrane potential is at the resting potential. The silicon networks provide an alternative to computer simulation of spike-based processing models with different time constant synapses because they run in real time and the computational time does not scale with the size of the neuronal network. Modeling Short-Term Synaptic Depression in Silicon 345 Figure 8: Response to a 1 Hz drifting sinusoidal grating. The input spikes from the LGN cell are depicted at the bottom of the curve. The top curve shows the response of the neuron when the grating drifted in the preferred direction. The sharp excursions at the top of the potential are the output spikes of the neuron. The middle curve shows the response to the stimulus in the null direction. The membrane potential did not build up to threshold. Appendix The current equation for a p-type eld-effect transistor (pFET) in subthreshold where the bulk of the transistor is at the voltage Vdd is Iop ek (Vdd ¡Vg ) / UT (e ¡(Vdd ¡Vs ) / UT ¡ e (Vdd ¡Vd ) / UT ), where Vg is the voltage at the gate, and Vd and Vs are the voltages at the drain and source terminals of a transistor; k describes the efciency of the gate in controlling the current in the channel and is less than 1 for a transistor operating in subthreshold. The voltages in the exponent are normalized in terms of the thermal voltage, UT . The differential equation for the current in a diode-capacitor circuit (e.g., M1 and C in Figure 1) when there is no input current is given by Iop ek (Vdd ¡Vx ) / UT (e¡(Vdd ¡Va ) / UT ¡ e ¡(Vdd ¡Vx ) / UT ) D C dV x . dt (A.1) 346 M. Boegerhausen, P. Suter, and S. Liu By introducing the current variable, Irf D Iop ek (Vdd ¡Vx ) / UT , we solve for the differential, dI rf : dI rf D ¡ k UT Irf dV x ) dV x D ¡ UT dI rf . k Irf By changing variables from Vx to Irf in equation A.1, we get, QT Á 1 d k Irf 0 ! " ) D dt @e¡(Vdd ¡Va / UT By substituting D D k (V e dd ¡Va ) / UT Iop Irf Iop ¡ Irf #1/k 1 A. (A.2) into equation A.2, we get Iopk ¡(1¡k ) (V ¡Va ) / UT dD dd (1 ¡ D1/k ) D M(1 ¡ D1 /k ), e D dt QT (A.3) Iopk where M D QT e¡(1¡k ) (Vdd ¡Va ) / UT . In steady state, D D 1. The update dynamics for D can be inferred from the dynamics of Ir , which are described by Ir (tnC ) D (1 C a)Ir (tn¡ ), (A.4) where tnC is the time right after the nth spike and tn¡ is the time right before the nth spike. The a factor is determined by a voltage parameter and the pulse width of the presynaptic spike. The corresponding equation for D is D(tnC ) D dD(tn¡ ), where d is 1 1C a (A.5) in equation 2.5. 2 Acknowledgments This work was supported in part by the Swiss National Foundation Research SPP grant. We also acknowledge Kevan Martin, Pamela Baker, and Ora Ohana for many discussions on dynamic synapses. 2 In equation A.5, for d D 1, a has to be 0. But a is never zero due to leakage currents. To compute d, we rst solve for 1 C a D e (k Idt ) / QT , where I is set by the voltage parameter Vd , dt is the pulse width of the input spike, and k is a transistor parameter. The smallest I is the leakage current, so for this value, we approximate a D (k Ir0dt ) / QT . Modeling Short-Term Synaptic Depression in Silicon 347 References Abbott, L., Sen, K., Varela, J., & Nelson, S. (1997).Synaptic depression and cortical gain control. Science, 275, 220–223. Boahen, K. A. (1997a). The retinomorphic approach: Pixel-parallel adaptive amplication, ltering, and quantization. Analog Integrated Circuits and Signal Processing, 13, 53–68. Boahen, K. A. (1997b). Retinomorphic vision systems: Reverse engineering the vertebrate retina. Unpublished doctoral dissertation, California Institute of Technology, Pasadena, CA. Chance, F., Nelson, S., & Abbott, L. (1998). Synaptic depression and the temporal response characteristics of V1 cells. Journal of Neuroscience, 18, 4785–4799. Indiveri, G. (2000). Modeling selective attention using a neuromorphic aVLSI device. Neural Computation, 12, 2857–2880. Liu, S.-C. (2001, Oct. 19). Simple cortical modeling with aVLSI spiking neurons and dynamic synapses. Paper presented at the ZNZ Symposium, University of Zurich. Liu, S.-C. (in press). Analog VLSI circuits for short-term dynamic synapses. EURASIP Journal on Applied Signal Processing: Special Issue. Liu, S.-C., Kramer, J., Indiveri, G., Delbrück, T., Burg, T., & Douglas, R. (2001). Orientation-selective aVLSI spiking neurons. Neural Networks: Special Issue, 14(6/7), 629–643. Maass, W., & Zador, A. (1999). Computing and learning with dynamic synapses. In W. Maass & C. M. Bishop (Eds.), Pulsed neural networks (pp. 157–178). Cambridge, MA: MIT Press. Markram, H., Wang, Y., & Tsodyks, M. (1998). Differential signaling via the same axon of neocortical pyramidal neurons. Proc. Natl. Acad. Sci. USA, 95, 5323–5328. Matveev, V., & Wang, X. (2000). Differential short-term synaptic plasticity and transmission of complex spike trains: To depress or to facilitate? Cerebral Cortex, 10, 1143–1153. Mead, C. (1989). Analog VLSI and neural systems. Reading, MA: Addison-Wesley. Natschlager, T., Maass, W., & Zador, A. (2001). Efcient temporal processing with biologically realistic dynamic synapses. Network, 12, 75–87. Rasche, C., & Hahnloser, R. (2001). Silicon synaptic depression. Biological Cybernetics, 84, 57–62. Reyes, A., Lujan, R., Rozov, A., Burnashev, N., Somogyi, P., & Sakmann, B. (1998). Target-cell-specic facilitation and depression in neocortical circuits. Nature Neuroscience, 1, 279–285. Senn, W., Segev, I., & Tsodyks, M. (1998). Reading neuronal synchrony with depressing synapses. Neural Computation, 10, 815–819. Stratford, K., Tarczy-Hornoch, K., Martin, K., Bannister, N., & Jack, J. (1998). Excitatory synaptic inputs to spiny stellate cells in cat visual cortex. Nature, 382, 258–261. Tsodyks, M., & Markram, H. (1997). The neural code between neocortical pyramidal neurons depends on neurotransmitter release probability. Proc. Natl. Acad. Sci. USA, 94(2), 719–723. 348 M. Boegerhausen, P. Suter, and S. Liu Tsodyks, M., Pawelzik, K., & Markram, H. (1998). Neural networks with dynamic synapses. Neural Computation, 10, 821–835. Van Schaik, A. (2001). Building blocks for electronic spiking neural networks. Neural Networks, 14(6/7), 617–628. Varela, J., Sen, K., Gibson, J., Fost, J., Abbott, L., and Nelson, S. (1997). A quantitative description of short-term plasticity at excitatory synapses in layer 2/3 of rat primary visual cortex. Journal of Neuroscience, 17, 7926–7940. Received June 14, 2002; accepted July 24, 2002.