Download FUNCTION REPRODUCTION WITH POLYCHRONOUS SPIKING NETWORKS

Proceedings of the 5th Annual ISC Research Symposium
ISCRS 2011
April 7, 2011, Rolla, Missouri
FUNCTION REPRODUCTION WITH POLYCHRONOUS SPIKING NETWORKS
Cameron Johnson, Ph. D. Candidate
Department of Electrical and Computer
Engineering
Real-Time Power and Intelligent Systems
Laboratory
ABSTRACT
Neural networks (NNs) are computational tools capable of
incredible nonlinear functional approximation. Commonly used
for time-series prediction, NNs face a scaling problem when
large numbers of inputs and outputs are needed. Increasingly
more neurons are required for each additional input or output to
be computed. Living brains, the inspiration for NNs, exhibit
capabilities to process absolutely enormous amounts of data all
at once. To capture this property, the next generation of NNs
utilize spiking signals in emulation of living neurons. This is
believed to lead to a solution to the scaling problem, but
introduces a whole new issue of transmitting real-world data
into the network, and reading meaningful real-world values
back out. This paper demonstrates successful encoding and
decoding of information into, through, and out of a PSN, which
demonstrates high fidelity on a time series reproduction
problem.
1. INTRODUCTION
Spiking Neural Networks (SNNs) take the inspiration that
neural networks (NNs) draw from living brains one step further
by more closely modeling the artificial neurons themselves on
the behavior of living neurons. Equations (1) and (2) show the
Izhikevich model [1] for time-dependent voltage on a given
neuron, with I serving as the input to the neuron from all
external sources (including the weighted spikes from other
neurons) and k is the discrete-time variable. The voltage itself is
v, with u serving as a damping factor when the neuron has not
spiked, and a, b, c, and d are variables that control the
frequency and regularity of the neuron‟s spikes. More
information on setting them can be found in [1]; for this paper,
the excitatory neurons are set to be “regular spiking,” and the
inhibitory neurons use inhibitory settings.
v(k  1)  0.04v 2 (k  1)  5v(k  1)...
if v  30mV

v(k )  
...  140  u (k  1)  I
(1)
if v  30mV

c

u (k  1)  a bv(k  1)  u (k  1) if v  30mV
u (k )  
(2)
if v  30mV
u (k  1)  d

G. K. Venayagamoorthy
Department of Electrical and Computer
Engineering
Director, Real-Time Power and Intelligent Systems
Laboratory
Because a primary inspiration behind exploring SNN
architectures is the observed scalability of living brains, we
borrow from [2] the delay mechanics needed to make the
Polychronous Spiking Network (PSN). In a nutshell, a PSN
naturally forms temporal clusters where individual neurons
spike only in response to specific temporal sequences applied to
multiple neurons that lead up to it. These polychronous groups
are identified by a form of unsupervised training known as
spike timing dependent plasticity (STDP), which strengthens
synaptic weights that lead to strong polychronous groups, and
weakens synaptic weights that do not.
Polychronous groups respond to different input stimuli,
and increase in number exponentially faster than the number of
neurons in the PSN. This effectively inverts the scaling
problem, because each new neuron added to the PSN
dramatically increases the theoretical potential limit of
simultaneous inputs and outputs the PSN can handle.
The difficulty of any SNN, however, is encoding and
decoding real-world data. SNNs operate on spikes, and the
spikes and their timing within the dynamic reservoir ARE the
data on which the SNN is operating. This is especially true
when the delays of the synapses are as important as they are in
a PSN. Previous papers [3, 4] have demonstrated means of
encoding real-world data into spikes and even tested these
methods‟ reversibility to prove that all of the data actually is
contained in the spike trains they generate. This paper will
explain a novel decoding scheme which translates spike timings
in a PSN into numeric outputs corresponding to individual
inputs, and a couple training methods to tune these outputs to
reproduce a time series input.
In this paper, the encoding method used (Gaussian
receptor fields, or GRF) will be briefly described, and then the
decoding and tuning of the PSN‟s outputs will be explained in
detail.
2. ENCODING AND DECODING REAL WORLD DATA
In [3, 4], different encoding schemes for SNNs are examined.
In particular, [3] outlines a uniquely spatial dual-neuron n-bit
representation (DNNR), while [4] compares this method to a
mixed spatio-temporal method, GRF, and a purely temporal
method, the Poisson rate encoding method (PREM) [5]. Each of
these is found in [4] to have all of the encoded data present in
1
Fig. 1. Block diagram of a PSN system used to fully reproduce a time series of real-valued data.
the spike trains generated by simple expedient of devising a
recovery algorithm for each method and comparing the
recovered values with the original inputs.
This section will demonstrate how to use a PSN to fully
reproduce a time series, as illustrated in Fig. 1.
matter. Each input x thus is represented by a set of spike outputs
at specified delays from the time of input presentation on two
or more of the sensory neurons, as illustrated in Fig. 2.
2.1. Encoding the Input
For the experiments done here, the GRF encoding method
was chosen. Future work will include a comparison of the GRF
method and the DNNR methods for full time series prediction
and function approximation, but PREM was found to be too
cumbersome for this purpose.
GRF, briefly, uses a set of input neurons that can be
considered to be an artificial sensory organ to transform realworld data into spikes. The neurons each have a Gaussian curve
associated with them, and the centers of these neurons‟
Gaussians are distributed between 0 and 1. Inputs are
normalized to this range in preprocessing (which is a very
simple computational step), and the intersections of that input‟s
normalized value with the Gaussian functions are translated
into response values associated with each neuron. The closer to
the center of a given receptive field the input value is, the
greater the response of the associated neuron is to that input.
f i ( x) 

 x Ci
A
2w2
Ti  1  fi  *

2
i  1, 2,..., m
i  1, 2,..., m
(3)
(4)
Equation (3), with A as the amplitude (taken to be one in
this experiment), Ci as the ith neuron‟s center and w as the width
of the receptive field, gives the response fi(x) of the neuron to
input x. Equation (4) finds a delay (between 0 and 10 ms) Ti
that is inversely proportional to the response. All delays over
nine milliseconds are discarded as too weak a response to
Fig. 2. (a) Encoding with overlapping GRF. Input values „p‟ and
„q‟ are translated into firing times for the input-neurons
encoding this input-variable. The highest stimulated neuron,
neuron 5, fires at a time close to zero, whereas less stimulated
neurons, as for instance neuron 3 and 7, fire at increasingly
later times towards tmax as shown in (b). Time delay values
generally range from 0 to 10.
The n sensory neurons (12 are used in this paper) are each
connected to 15 neurons in the PSN (inhibitory and excitatory),
2
serving to stimulate the PSN system. The PSN used in this
experiment consists of 80 excititory neurons and 20 inhibitory
neurons.
2.2. Decoding and Training the Output
Having implemented a means of passing real-world data
into the PSN, it now becomes necessary to make sense of the
spiking responses of the neurons that make up the PSN itself.
The spikes and their timings are the response of the PSN to its
input, the process of calculating important dynamics of the
data, and the processed data, itself. How, then, can these
asynchronous spikes be read and real-world values that
correspond to the real-world inputs be determined?
First, each neuron in the PSN is monitored over a period of
time after each input known as a watch window (set to 20 ms in
this experiment), and the spikes in that period are filtered
through a temporal response function and summed over the
watch window period. This generates a decimal value which
can then be passed to a synapse leading to the final output
generator.
Specifically, as demonstrated in Fig. 3, each neuron has a
normalized Gaussian function (3) associated with it. The center
of this function is located within the temporal watch window.
During a period of offline, unsupervised training, particle
swarm optimization (PSO) [6] is used to find the centers and
widths that maximize the responses of the neurons‟ watch
windows to their spiking patterns.
associated amplitude that maximizes the response to the
neuron‟s typical spike patterns. If it is prone to bursts of
multiple spikes close together, it will tend towards a narrow
width and high amplitude with the center near the most
common occurrence of these bursts in a watch window. If it is
instead prone to steady streams of spikes, its width will tend to
be broader. This offline training is typically done at the same
time as STDP maturation of the internal weights of the PSN [1,
2].
Once done, the online training of the output weights can
begin. Each of the summation symbols in Fig. 3 outputs a realworld number corresponding to every real-world input initially
fed to the GRF-based sensory organ. In this paper, two methods
are examined for combining the individual outputs of each PSN
neuron into a trained output intended to have meaning in the
real world.
The first is a very straight-forward weighted sum. Fig. 4
illustrates the weights connecting each output from the PSN to
a summation neuron, and these outputs are trained online with
simple gradient descent. As section three will show, this leads
to an interesting tendency to follow the target, and testing
reveals that it has learned this behavior, but it is not dreadfully
faithful.
Fig. 4. Simple sum output, trained with gradient descent error.
Fig. 3. Gaussian temporal filters transforming spike trains in a
watch window into a real-world numeric output.
The second method is a slightly more involved MLP
applied to the PSN outputs, used to isolate and track the
dynamics in the PSN itself and output the final value of the
system. Illustrated in Fig. 5, it requires full-fledged
backpropagation training. This paper tests both five and ten
hidden neurons in the hidden layer of the MLP, and
demonstrates very strong results illustrating the capability to
learn a PSN‟s dynamics online, and not merely force the
outputs.
The amplitude A in (3) for each of these Gaussians is a
function of the width w, as shown in (5) for each of the m
neurons. The PSO training finds the center, width, and
3
Fig. 5. MLP output tuning, trained with backpropagation.
3. Results and Discussion
To test the capability of this encoding and decoding and
training system on a PSN, (6) was used as a time series input
that was repeated multiple times. Three seconds of input are
simulated per iteration of (6). The PSN was matured (and the
PSO-based unsupervised training of the PSN‟s output was
performed) over the course of 24 simulated hours; [1, 2]
indicate that six simulated hours is the minimum maturation
time for STDP, and 24 simulated hours were given to be
absolutely thorough. Once the PSN was matured with (6) as its
constant input, its output weights were trained to reproduce the
time series represented by (6).
20 sin(2 t )
t 1


f ( x)  
20 sin(4 t )
1 t  2
(6)

20 sin(2 t )
t2

The simple summation-based output, trained via online
gradient descent, was trained for 19 iterations. The 20 th iteration
was performed with training turned off, so the output weights
were frozen. Fig. 6 illustrates that not only does the gradient
descent training lead to a behavior that is strongly influenced
by the input, but that, by the 19th iteration, the output system
has learned to reproduce it as well as it was already doing. It is
important to note that there are some output steps where the
PSN does not spike at all, and thus the decoded output would
be zero no matter what the trained output weights were. This is
alleviated by simply having any output of zero from the final
output be replaced by the prior output. As can clearly be seen,
this has no undesirable effects on the time series reproduction.
If greater accuracy is desired, further means of predicting what
the next output will be so interpolation might be performed
would need to be developed.
Fig. 7 is actually the exact same data as Fig. 6, save that
the final outputs are averaged over the last 20 outputs. This is
done to examine the trend by smoothing out the jitter.
Replacing the simple summation with a one-layer MLP
with 5 hidden sigmoidal neurons and repeating the same
experiment with a learning gain of 0.08 and a momentum gain
of 0.05 produces Fig. 8. Notice how it follows it well until the
testing run is performed, at which point the output of the
system caps at 10 and -10. Experiments with greater numbers
of hidden neurons initially seemed unpromising, but extending
the training time to 299 iterations and the hidden neuron count
to 10, while changing both learning and momentum gain to
0.01, yielded Fig. 9, which is a remarkably faithful time series
reproduction.
In all of Figs. 6-9, a) gives a raster plot of the PSN‟s actual
response during the final iteration. The top 12 neurons (101112) are the input, sensory neurons, and the sine-wave-like
pattern is due to the spatio-temporal nature of the spiking inputs
given to the network. Plot (b) in each figure represents the first
iteration, with online training active, while plot (c) is the last
iteration with online training active and plot (d) is the final
iteration, used as a testing run, during which the output weights
are frozen.
One additional noteworthy behavior with the MLP: the
first iteration has a very faithful, but delayed, tracing of the
target; this is due to training forcing the output. The noisier, but
less delayed, behavior in the later iterations (plots c and d)
demonstrate that the network actually has some idea what it‟s
doing – i.e., that it‟s learned the function.
Fig. 6. Gradient-descent trained weighted sum of the PSN
output.
4
Fig. 7. Output of Fig. 6 averaged over past 20 output values.
Demonstrates that the overall trend of the highly-noisy
weighted sum output still learned the time series.
Fig. 8. MLP decoding PSN output with 5 sigmoidal neurons in
the hidden layer.
Fig. 9. MLP output with 10 sigmoidal neurons in the hidden
layer.
4. CONCLUSION AND FUTURE WORK
A PSN has been successfully utilized for time series
reproduction in this paper. Prior work has established several
means of entering real-world values into an SNN, and the GRF
method was chosen from amongst those candidates. The major
step forward and contribution of this paper is the novel
temporal-sensitivity method used to transform the spike
patterns of the PSN neurons into numeric values, and training
output weights from those numeric outputs to generate target
values.
This is a tremendous breakthrough, enabling an onlinetrainable method of extracting dynamics from a spiking
network of any variety. Previous work with spiking neurons
typically relies on offline training and pseudoinverse
calculation; this method can adapt on the fly to changing,
unstable dynamics within the reservoir of neurons.
The next steps are to attempt static function approximation,
where the outputs are a product only of the single
corresponding input rather than a time series result based on
multiple past inputs, and then to utilize this PSN to model the
time series problem of power generator speed and voltage
dynamics – a real-world highly complex function with industry
applications for the Smart Grid.
With this breakthrough in online-trainable PSNs, this new
generation of advanced neural network architecture is ready to
begin its march into real-world application, as well as opening
5
the door to more advanced theoretical research into living brain
functionality.
9. ACKNOWLEDGEMENTS
This research has been funded in part by the Intelligent Systems
Center at the Missouri University of Science and Technology,
NSF/EFRI COPN #0836017, and a GAANN Fellowship.
10. REFERENCES
[1] Izhikevich, E., 2003, “Simple Model of Spiking
Neurons,” Vol. 14, pp. 1569-1572.
[2] Izhikevich, E., 2006, “Polychronization: Computation
with Spikes,” Neural Computation, Vol. 18, pp. 245-282.
[3] Johnson, C., Venayagamoorthy, G. K., 2010,
“Encoding Real Values into Polychronous Spiking
Networks,” Word Congress on Computational
Intelligence 2010, Barcelona, Spain, pp. 1-7.
[4] Johnson, C., Roychoudhury, S., Venayagamoorthy, G.
K., 2011, “A Reversibility Analysis of Encoding
Methods for Spiking Neural Networks,” International
Joint Conference on Neural Networks, 2011, Paris,
(submitted).
[5] Zhang, X., You, G., Chan, T., and Feng, J., 2009,
“Maximum Likelihood Decoding of Neuronal Inputs
from an Interspike Interval Distribution,” Neural
Computation, Vol. 21, pp. 3079-3105.
[6] Kennedy, J., Eberhart, R., 1995, “Particle Swarm
Optimization,” Proceedings of the IEEE International
Conference on Evolutionary Computation, pp. 69-73.
6