* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download DELTA-SIGMA MODULATION IN SINGLE NEURONS Mats Høvin
Flip-flop (electronics) wikipedia , lookup
Audio crossover wikipedia , lookup
Resistive opto-isolator wikipedia , lookup
405-line television system wikipedia , lookup
Telecommunication wikipedia , lookup
Regenerative circuit wikipedia , lookup
Superheterodyne receiver wikipedia , lookup
Rectiverter wikipedia , lookup
Active electronically scanned array wikipedia , lookup
Dynamic range compression wikipedia , lookup
Signal Corps (United States Army) wikipedia , lookup
Battle of the Beams wikipedia , lookup
Oscilloscope history wikipedia , lookup
Cellular repeater wikipedia , lookup
Time-to-digital converter wikipedia , lookup
Analog-to-digital converter wikipedia , lookup
Analog television wikipedia , lookup
Opto-isolator wikipedia , lookup
Phase-locked loop wikipedia , lookup
Radio transmitter design wikipedia , lookup
Valve RF amplifier wikipedia , lookup
Index of electronics articles wikipedia , lookup
DELTA-SIGMA MODULATION IN SINGLE NEURONS Mats Høvin, Dag Wisland, Yngvar Berg, Jan Tore Marienborg and Tor Sverre Lande University of Oslo, Dept. of Informatics, Gaustadalleen 23, 0316 OSLO, Norway In this paper we investigate the relationship between the pulse-coded signal representation found in biological neurons and signals represented by first-order delta-sigma bit-streams. We show that a first-order delta-sigma bit-stream is a digital equivalent to a pulse-coded neural signal. By increasing the clock frequency, a delta-sigma bit-stream can approach a pulse-coded representation with arbitrary accuracy. 1. INTRODUCTION Biological neurons communicate with each other by spikes or pulse trains. The information is not encoded by the shape of the pulses, but by the arrival time and the correlation with other pulses. This is known as pulse-frequency-coding. It is hard to know why evolution have chosen this communication concept but a very strong feature is its low sensitivity to interfering noise. Several papers have pointed out that there may be a connection between pulse- frequency-coding and ∆ − Σ noise-shaping. In [1] the authors argue for that an integrate-and-fire neuron posses a behavior similar to a ∆− Σ modulator. The relationship is not shown mathematically, but the authors presents a successfully working Hopfield network where each neuron is implemented by a ∆ − Σ modulator. In [2], [3] the concept of spatial noise-shaping is introduced. Here it is shown that by using inhibitory coupling between neighborly neurons, an increase in signal-to-noise ratio results. The resulting noise is shaped and therefore suppressed for lower frequencies. In this paper we focus on the behavior of a single integrateand-fire neuron where the accumulator is completely reset after firing. We presents a mathematical analysis showing that, by disregarding the shape of the biological pulse, the output of a first-order ∆−Σ modulator is a digital equivalent to a pulsecoded neural signal. In Chapter 2 we present a new model for pulse frequency coding based on standard frequency modulation. This model is in Chapter 3 compared to a first-order ∆−Σ modulator. In Chapter 4 different features of the model is further analyzed by comparing it to a first-order frequency delta-sigma modulator (FDSM). 2. NEURAL PULSE-CODING MODELED AS STANDARD FREQUENCY MODULATION Consider an integrate-and-fire neuron where the various synaptic input signals are integrated in the cell body (Fig. 1). When the accumulated cell voltage exceeds the action potential, the cell voltage is reset to zero and a pulse is emitted trough the axon. By looking at a single synaptic input, the axon pulse frequency will be modeled as proportional to α where α represents the input stimulus signal. α could be thought of as a low-pass filtered epresentation of the pulsecoded signal from a synapsing neuron. Since the axon pulse frequency is proportional to the input signal, the mean time between pulses will be given by K , (1) α where K is a constant modeling the strength of the synaptic input. In the following we will disregard the phase (initial condition) of the pulse-coded signal and consider Eq. 1 to represent the main feature of the neuron. In general, a frequency tα = Synapse Dendrit Nucleus Axon Axon voltage ABSTRACT ta time Fig. 1. Top: A single neuron. Bottom: The pulse frequency coded axon signal. modulated signal [4] can be represented by Z t ka(τ )dτ ), b(t) = sin( 2π (2) 0 where a(τ ) is the modulating signal and k is the frequency sensitivity. In this model the carrier frequency is included in a(τ ) assuming a(τ ) > 0. By the use of the sqr function 1:x≥0 sqr(x) = (3) 0:x<0 the frequency modulated signal is then hard-limited as shown in Fig. 2b-c. By derivating this signal with respect to time and applying the absolute value as illustrated in Fig. 2d-y we have a sequence of impulses (Dirac delta pulses) d sqr( sin(2π R t ka(τ )dτ ) ) 0 (4) y(t) = , dt where the spacing between each impulse is modulated by the input signal. The impulse sequence represents the essence of the pulse-coded signal by representing only the position of each pulse where the pulse shape is of no significance. In Fig. 3 the generation of y(t) is shown schematically. The Accumulator a(t) an q(.) Z-1 yn sampler Fig. 4. A first-order ∆ − Σ modulator. a(t) where frac(x) is the the fractional part of x, i.e. x without its integer part, the output term of Eq. 7 is already quantized and may be resolved form the quantizing function giving ! n n X X ak−1 − yk−1 . (9) yn = q b(t) c(t) d(t) k=0 y(t) By collecting the output terms on the left side, we have ! n+1 n X X yk−1 = q ak−1 , (10) ta Fig. 2. a(t): Analog input signal. b(t): FM signal. c(t): Hard limited FM signal. d(t): Derivated signal. y(t): Neural pulsefrequency-coded representation. a(t) k=0 Frequency b(t) modulator sqr(.) c(t) k=0 k=0 and by inserting this expression into Eq. 9 we arrive with ! ! n n−1 X X ak−1 − q ak−1 . (11) yn = q k=0 y(t) d(.) d(t) abs(.) dt By applying the definition of q(x) we have yn = n X Fig. 3. A model of axon pulse-coding based on impulse representa- k=0 tion. n−1 X time between impulses can by found from Eq. 4 by replacing the input signal a(t) by the constant a d sqr( sin(2πkat ) . (5) y(t) = dt Each zero-crossing of the term sin(2πkat) represents an impulse, and the time between impulses will be tx = 1/(2k) , a (6) which is in accordance with the main feature of the neuron defined by Eq. 1. 3. COMPARISON TO FIRST-ORDER ∆ − Σ MODULATION: MODEL A In Fig. 4 a standard discrete-time first-order ∆ − Σ modulator [5] is shown. The quantizer is represented by the quantizing function q(·). The output of this circuit can be expressed as ! n X (ak−1 − yk−1 ) . (7) yn = q k=0 By defining the quantizing function q(x) as def q(x) = x − frac(x), k=0 (8) n X ak−1 − frac( ak−1 ) − k=0 n−1 X ak−1 − frac( k=0 ak−1 ). (12) k=0 This may be rearranged providing the following description of the ∆ − Σ modulator output signal ! ! n n−1 X X ak−1 − frac ak−1 ). (13) yn = an − (frac k=0 k=0 We notice that the output signal is given by the input signal an plus a differentiated quantization error term. Assuming 0 ≤ an < 1, Eq. 13 may be broken into two different cases. Case I: The integer part of both sum terms in Eq. 13 are equal. In this case the frac(·) functions may be disregarded as they both subtracts the same integer from their arguments. The output will always be zero. n n−1 X X ak−1 − ak−1 ) = 0 y n = an − ( k=0 (14) k=0 Case II: The integer part of the first sum term is given by the integer part of the last sum term plus one. In this case the first frac(·) function will reduce its argument by one integer more than the last modulo function. The output will always be one n n−1 X X ak−1 − 1) − ak−1 ) = 1. yn = an − (( k=0 k=0 (15) (16) The mean number of output zero samples between each output impulse will be 1/a, and the mean time between each output impulse is Ts (17) t¯a = a where Ts is the clock period. This is in accordance with the main feature of the neuron defined by Eq. 1. Since the output of the ∆ − Σ modulator is sampled (quantized in time), we will have a maximum time-quantization error given by δtmax = 2Ts (18) By letting the clock frequency approach infinity, this error will approach zero, and by keeping the fraction Ts /a constant by simultaneously decreasing a, the ∆ − Σ output signal will approach an exact digital representation of the pulse coded signal. In Fig. 5 the input/output signals are illustrated for low clock frequencies. In Fig. 6 (top) the power spectral density of the ∆ − Σ modulator is shown for a clock frequency of 8kHz, where the average number of zeros between each impulse is 2. The input signal was a single sinusoidal signal at 12Hz. In Fig. 6 (bottom) the clock frequency is increased to 0.8MHz keeping the fraction Ts /a constant. The excess noise at high frequencies is pattern noise [5]. The modulator was simulated in C++. By doubling the oversampling ratio in a first-order ∆ − Σ modulator, the signal-to-quantization-noise-ratio will increase by 9dB. However, by keeping the fraction Ts /a constant, each doubling of the oversampling ratio will only increase the signal-to-quantization-noise-ratio by 9 − 6 = 3dB as the signal power is halved. Normalized PSD (dB) yn = a − (frac(na) − frac((n − 1)a)) 10 0 -10 -20 -30 -40 -50 -60 -70 -80 -90 1 Normalized PSD (dB) In other words; the output will be one each time the integer part of the first sum term changes. The mean time between impulses can be found form Eq. 13 by replacing the input signal an by the constant a 10 100 Frequency (Hz) 1000 0 -20 -40 -60 -80 -100 10 100 1k Frequency (Hz) 10k 100k Fig. 6. Simulated ∆ − Σ modulator output power spectral density. Top: 1/Ts =8kHz. Bottom: 1/Ts =0.8MHz ∆ − Σ modulator (FDSM) [6], [7], [8], and is mathematically equivalent to a first-order ∆−Σ modulator where the loop accumulator is replaced with a continuous-time integrator. The Differentiator a(t) Frequency modulator ZC counter w Z-1 yn w+1 Samplng clock a(t) Fig. 7. An implementation of Eq. 19. 2Ts ta y[n] Fig. 5. ∆ − Σ modulator input/output signals. 4. COMPARISON TO FREQUENCY ∆ − Σ MODULATION: MODEL B By replacing the accumulator in Eq. 11 with a continuoustime integrator we have nT (n−1)T Z s Z s 1 1 a(τ )dτ − q a(τ )dτ . (19) yn = q Ts Ts 0 0 This equation can be implemented by the circuit shown in Fig. 7. This circuit is in literature referred to as a frequency FDSM utilize the fact that the phase of a FM signal is the integral of the modulating signal (Eq. 2). By the use of an zerocross (ZC) counter the FM phase is detected and quantized providing the first term in Eq. 19. The result is then digitally differentiated producing yn . In general the ZC counter wordAnalog input D-S FM signal Frequency modulator bit-stream D clk Q D Q clk High speed clock Fig. 8. A practical FDSM. length must be infinity, but by the use of modulo wrap-around arithmetic a one-bit word-length may be used [6]. A one-bit ZC counter can be implemented by a D flip-flop, and a onebit modulo subtractor by an XOR gate resulting in the simple circuit shown in Fig. 8. This circuit is a well documented cn Differentiator a(t) b(t) Frequency modulator c(t) d[n] sqr(.) e[n] Z-1 a(t) f[n] abs(.) sampler yn c'n y'n Fig. 9. A mathematical FDSM equivalent. ∆ − Σ modulator, and it will for a proper input signal provide the same output signals as shown in Chapter III. The FDSM Fig. 11. ∆ − Σ modulator signals. a(t): Hard-limited FM signal. cn : Sampled FM signal (low speed clock frequency). yn : Output signal (low speed clock frequency). c0n : Sampled FM signal (high speed clock frequency). yn0 : Output signal (high speed clock frequency). a(t) b(t) c(t) dn en 5. CONCLUSION We have presented a new model for pulse-frequency-coding. Based on this model we have shown that a first-order ∆ − Σ bit-stream is a digital equivalent to a pulse-coded neural signal. By increasing the clock frequency, a ∆ − Σ bit-stream can approach a pulse-coded representation with arbitrary accuracy. Additive noise affecting the positions of the pulses is noise-shaped. 6. REFERENCES yn Fig. 10. ∆ − Σ signals. a(t): Analog input signal. b(t): FM signal. c(t): Hard-limited FM signal. dn : Sampled signal. en : Differentiated signal. fn : Modulator output signal. operates by first integrating the input signal by the use of a frequency modulator (Fig. 9, Fig. 10a-b). Then the FM signal is fed trough a D flip-flop which is equivalent to a sqr function followed by a signal sampler (Fig. 9, Fig. 10c-d). The resulting digital sequence is then differentiated and rectified as illustrated in Fig. 9 and Fig. 10e-y. By comparing Fig. 10 to Fig. 2 we see that the FDSM is a digital equivalent to the neural pulse code model presented in Chapter 2. The mean time between impulses will be given by Eq. 6, and as a sampled system, the maximum time-quantization error is given by Eq. 18. In Fig. 11 the output signal is shown for two different clock frequencies. We notice that as the clock frequency is increased the output signal approaches an exact digital representation of the continuous-time impulse sequence given by Eq. 4. In [8] it is shown both mathematically and by measurements that clock phase noise (jitter) is noise shaped in the FDSM. In other words, additive noise affecting the positions of the impulses will, in the resulting output power spectrum, be shifted up to higher frequencies where it can be removed by a low-pass filter. In this way the pulse-frequency-coding is a powerful signal representation due to its low sensitivity to excess noise. [1] K.F. Cheung and P.Y.H. Tang: ”Sigma-Delta Modulation Neural Networks”, Proc. IEEE Int. Conf. Neural Networks, pp. 489-493, 1993. [2] R.W. Adams: ”Spectral Noise-Shaping in Integrate-andFire Neural Networks”, Proc. IEEE Int. Conf. Neural Networks, pp. 953-958, 1997. [3] D.J. Mar, C.C. Chow, W. Gerstner, R.W. Adams and J.J. Collins: ”Noise shaping in populations of coupled model neurons”, Proc. Natl. Acad. Sci. USA, Vol. 96, pp. 10450-10555, 1999. [4] S. Haykin: Communication systems, John Wiley and Sons, 1987, pp.179-183 [5] J.C. Candy and G.C. Temes: Oversampling DeltaSigma Data Converters, IEEE Press, New York, 1992. [6] M. Høvin, A. Olsen, T. S. Lande and C. Toumazou: ”Delta-sigma Modulators using Frequency Modulated Intermediate Values”, IEEE Journal of Solid State Circuits, vol. 32, no. 1, pp. 13-22, 1997. [7] M. Høvin, T.S. Lande, D.T. Wisland and S. Kiaei:”A Triangularly Weighted Zero-Crossing Detector providing Delta-Sigma Frequency-to-Digital Conversion”, IEE Electronics Letters, vol. 33, no. 13, pp. 1121-1122, June, 1997. [8] M. Høvin, T.S. Lande, J.T. Marienborg and D.T. Wisland: ”Pattern Noise in the Frequency Delta-Sigma Modulator”, Analog Integrated Circuits and Signal Processing, vol. 26, no. 1, pp. 75-82, 2001.