Download neural spike

‫سميناردرس‬
‫مباحث پيشرفته در شبكه هاي عصبي‬
‫شبکه های عصبی ايمپالس و کوپالژپالس‬
‫‪Impulse and pulse-coupled Neural Net‬‬
‫پيمان گيفانی‬
‫خرداد ‪84‬‬
‫‪1‬‬
DURING last few years we have witnessed a shift of the emphasis
in the artificial neural network community toward
spiking neural networks.
Motivated by biological discoveries,
many studies consider pulse-coupled neural
networks with spike-timing as an essential component in
information processing
by the brain.
In any study of network dynamics, there are two crucial issues
which are
2
1) what model describes spiking dynamics of
each neuron and
2) how the neurons are connected.
20 of the most prominent features of biological spiking neurons
20 of the most prominent features of biological spiking neurons. The goal of this
section is to illustrate the richness and complexity of spiking behavior of individual
neurons in response to simple pulses of dc current. What happens when only tens
(let alone billions) of such neurons are coupled together is beyond our
comprehension.
3
4
Which Model to Use for Cortical Spiking
Neurons?
To understand how the brain works, we need to combine experimental studies of
animal and human nervous systems with numerical simulation of large-scale brain
models.
As we develop such large-scale brain models consisting of spiking neurons, we must
find compromises between two seemingly mutually exclusive requirements: The
model for a single neuron must be:
1) computationally simple, yet
2) capable of producing rich firing patterns exhibited by real biological neurons.
Using biophysically accurate Hodgkin–Huxley-type models is computationally
prohibitive, since we can simulate only a handful of neurons in real time.
In contrast, using an integrate-and-fire model is computationally effective, but
the model is unrealistically simple and incapable of producing rich spiking and
bursting dynamics exhibited by cortical neurons.
5
6
Spike-timing Dynamics of Neuronal Groups
Spiking Model by Izhikevich (2003)
A neuronal network inspired by the anatomy of the
cerebral cortex was simulated to study the self-organization
of spiking neurons into neuronal groups.
The network consisted of 100 000 reentrantly interconnected
neurons exhibiting known types of cortical firing patterns,
receptor kinetics, short-term plasticity and long-term
spike-timing dependent plasticity (STDP), as well as a
distribution of axonal conduction delays.
The dynamics of the network allowed us to study the fine
temporal structure of emerging firing patterns with
millisecond resolution. We found that the interplay between
STDP and
conduction delays gave rise to the spontaneous formation of
neuronal groups — sets of strongly connected neurons
capable of firing time-locked, although not necessarily
synchronous, spikes.
Despite the noise present in the model, such groups
repeatedly generated patterns of activity with millisecond
spike-timing precision.
Exploration of the model allowed us to characterize various
group properties, including spatial distribution, size,
growth, rate of birth, lifespan, and persistence in the
presence of synaptic turnover.
7
8
Synaptic Dynamics
– Short-term Depression and
Facilitation
– Synaptic Conductance
– Long-term Synaptic Plasticity
– Spike-timing in Neuronal Groups
9
Time-locked spiking patterns
10
11
Distribution of groups on the simulated
‘cortical’ surface.
Neurons within each group are marked by
the same color.
12
self-reconstruction & Self-organization
when synaptic turnover
Even synaptic turnover, which changed all synapses in a random order during a 24
h period, did not affect the stability of spike-timing neuronal groups (yellow bins in
Fig. 11). Repetitive activation of neuronal groups combined with STDP resulted in
a self-reconstructing behavior of the model.
13
Parallel Processing
•
•
•
Concurrent Processing
Unprecedented Memory Capacity
Competition and Cooperation
between groups
14
Rhythmic Activity
Rhythmic activity in the delta frequency
range around 4 Hz. This is one of the four
fundamental types of brain waves,
sometimes called “deep sleep waves”,
because it occurs during dreamless states
of sleep, infancy, and in some brain
disorders.
As the synaptic connections evolve
according to STDP, the delta oscillations
disappear, and spiking activity of the
neurons becomes more Poissonian and
uncorrelated. After a while, gamma
frequency rhythms in the range 30-70 Hz
appear. This kind of oscillations, implicated
in cognitive tasks in humans and other
animals, play an important role in the
activation of polychronous groups.
15
Cognitive Computations
• Rate to Spike-Timing
Conversion
Neurons in the model use spike-timing code to
interact and form groups. However, the external
input from sensory organs, such retinal cells,
hair cells in cochlear, etc., arrives as the rate
code, i.e., encoded into the mean firing
frequency of spiking.
How can the network convert rates to
precise spike timings?
Open circles - excitatory neurons, black circles inhibitory neurons.
inhibitory postsynaptic potential (IPSP).
Notice that synchronized inhibitory activity
occurs during gamma frequency oscillations.
Thus, the network constantly converts rate code
to spike-timing code (and back) via gamma
rhythm. The functional implications of such a
non-stop conversion are not clear.
16
Representations of
Memories and Experience
hypothesize that polychronous groups could represent memories and experience. In the simulation
above, no coherent external input to the system was present. As a result, random groups emerge; that
is, the network generates random memories not related to any previous experience.
Persistent stimulation of the network with two spatio-temporal patterns result in emergence of
polychronous groups that represent the patterns. the groups activate whenever the patterns are
present.
17
Consciousness
When no stimulation is present, there is a spontaneous activation of
polychronous groups.
If the size of the network exceeds certain threshold, a random
activation of a few groups corresponding to a previously seen stimulus
may activate other groups corresponding to the same stimulus so that the
total number of activated groups is comparable to the number of
activated groups that occurs when the stimulus is present.
Not only such an event excludes all the other groups not related to the
stimulus from being activated, but from the network point of view, it would
be indistinguishable from the event when the stimulus is actually present.
One can say that the network “thinks” about the stimulus. A sequence of
spontaneous activations corresponding to one stimulus, then another, and
so on, may be related to the
stream of thought and primary consciousness.
18
NEURAL EXCITABILITY, SPIKING AND BURSTING
•
•
•
•
•
The brain types of cells: neurons, neuroglia,
and Schwann cells.
neurons are believed to be the key elements in
signal processing.
1011 neurons in the human brain each can
have more than 10 000 synaptic connections
with other neurons.
Neurons are slow, unreliable analog units, yet
working together they carry out highly
sophisticated computations in cognition and
control.
Action potentials play a crucial role among
the many mechanisms for communication
between neurons.
19
Synchronization and locking are ubiquitous in nature
20
In-phase Synchronization
Anti-phase Synchronization
Out-of-phase Synchronization
No Locking
Neural Excitability
•
•
•
•
•
21
Excitability is the most fundamental property of neurons allowing communication via
action potentials or spikes.
From mathematical point of view a system is excitable when small perturbations near
a rest state can cause large excursions for the solution before it returns to the rest.
Systems are excitable because they are near bifurcations from rest to oscillatory
dynamics.
The type of bifurcation determines excitable properties and hence neurocomputational features of the brain cells. Revealing these features is the most important
goal of mathematical neuroscience.
The neuron produse spikes periodically when there is a large amplitude limit cycle
attractor, which may coexist with the quiescent state.
•
Most of the bifurcations discussed here can be illustrated using a twodimensional (planar) system of the form
  x '  f ( x, y )
y '  g ( x, y )
•
•
•
22
Much insight into the behavior of such systems can be gained by
considering their nullclines.
the sets determined by the conditions f(x, y) = 0 or g(x, y) = 0.
When
nullclines are called fast and slow, respectively. Since the
language of nullclines is universal in many areas of applied mathematics
23
24
An excitable system at an
Andronov/Hopf bifurcation
possesses an important information
processing capability:
Its response to a pair (or a
sequence) of stimuli depends on the
timing between the stimuli relative
to the period of the small amplitude
damped oscillation at the
equilibrium.
25
Selective communication
and multiplexing
The same doublet may or may not elicit
response in a postsynaptic neuron depending
on its eigenfrequency.
This provides a powerful mechanism for
selective communication between such
neurons. In particular, such neurons can
multiplex send many messages via a single
transmission line.
26
Phase of the sub-threshold oscillation
27
Fast sub threshold oscillation
If a neuron exhibits fast subthreshold oscillation of its membrane potential, then
its response to a brief strong input may depend on the amplitude and timing of
the input.
If the input is weak, so that it never evokes an action potential, but can modulate
the subthreshold oscillation, by changing its phase, so that the neuron would
react dierently to a future strong pulse.
From the FM interaction theory it follows that the phase of subthreshold
oscillation can be affected only by those neurons with a certain resonant
frequency.
By changing the frequency of the subthreshold limit cycle, the neuron
can control the set of the presynaptic neurons that can modulate its dynamics.
The entire brain can rewire and regulate itself dynamically without
changing the synaptic hardware.
28
Bursters
•
•
•
When neuron activity alternates between a quiescent state and repetitive spiking,
the neuron activity is said to be bursting. It is usually caused by a slow voltage- or
calcium-dependent process that can modulate fast spiking activity.
There are two important bifurcations associated with bursting :
– Bifurcation of a quiescent state that leads to repetitive spiking .
– Bifurcation of a spiking attractor that leads to quiescence .
These bifurcations determine the type of burster and hence its neuro-computational
features.
An example of "fold/homoclinic"
(square-wave) bursting. When a slow
variable changes, the quiescent state
disappears via fold bifurcation and the
periodic spiking attractor disappears via
saddle homoclinic orbit bifurcation
29
Bursting
• So far we have considered
spiking mechanisms assuming
that all parameters of the
neuron are fixed. From now
on we drop this assumption
and consider neural systems of
the form
x '  f ( x, u )
• Fast spiking
• Slow modulation u '  g ( x, u )
where u represents slowly
changing parameters in the
system.
30
31
32
33
34
Synchronization of bursters
35
Coupling
36
Applications
• Pattern Recognition Via Synchronization in Phase-Locked Loop
Neural Networks
– a novel architecture of an oscillatory neural network that consists of phase-locked
loop (PLL) circuits. It stores and retrieves complex oscillatory patterns as
synchronized states with appropriate phase relations between neurons.
37
38
Synchronization of MEMS Resonators and
Mechanical Neurocomputing
Combination two well-known and established
concepts: microelectromechanical systems
(MEMS) and neurocomputing.
First, we consider MEMS oscillators having low
amplitude activity and we derive a simple
mathematical model that describes nonlinear
phase-locking dynamics in them.
Then, we investigate a theoretical possibility of
using MEMS oscillators to build an
oscillatory neurocomputer having
autocorrelative associative memory.
The neurocomputer stores and retrieves complex
oscillatory patterns in the form of
synchronized states with appropriate phase
relations between the oscillators. Thus, we
show that MEMS alone can be used to build a
sophisticated information
processing system (U.S. provisional patent
60/178,654).
39
PCNNW and Noise
Automatic edge and
target extraction
based on PulsedCoupled NN wavelet
theory (PCNNW)
40
‫با سپاس از تحمل شما‬
‫‪41‬‬