Download Simulation with NEST, an example of a full

Simulation with NEST, an example of a full-scale spiking
neuronal network model - seminar paper
Till Schumann, CES, 293576
Computational and Systems Neuroscience (INM-6)
1
1.1
Introduction to Computational Neuroscience
Motivation
Computational neuroscience is part of the computational biology, which, besides other methods, relies on modeling to understand various aspects of biological systems. Computational
neuroscience itself focuses on the nervous system. It is a growing field of research. With
the fast development of computer systems and the growing availability of experimental data,
computational simulations get more important. The computational power, which is available
now and will be available in the next years, allows simulations of mammalian brains. Even a
simulation of the human brain seems to be doable in the upcoming years. Modeling nervous
systems helps us to understand the functionality of the human brain. It can help us to understand different kinds of diseases like Alzheimer’s and can help to develop novel therapies.
Since the human brain is not accessible to direct experimental studies models of the brain are
essential to understand its functionality. Computational modeling allows to construct neuron
models that are based on cell-level data obtained from experiments. In contrast to the human,
brain experimental data from animal experiments are widely available. Because of related
structures these data is used to improve and validate models of the brain. In combination
with neuronal simulations these data allows a first look into the functionality of nervous systems. The paper The cell-type specific cortical microcircuit: relating structure and activity in
a full-scale spiking network model shows the usability of current models and simulation tools.
It shows that simulations using measurements from rats and cats can reproduce dynamic
behaviors of brain cells.
1.2
Anatomy of the brain
The human brain is the main part of the central nervous system which consists of the spinal
cord, sensory organs and all of the nerves that connect these organs with the rest of the body.
These organs are responsible for the control of the body and communication between its parts.
The nervous system is the most complex system of our body with respect to functionality.
It contains billions of nerve and glia cells. The nerve cells are connected via synapses to
a complex network. Electrical pulses from neuron to neuron transmit information through
the network. Glia cells help to maintain the right concentration of chemical substances in
the extracellular space around neurons and provide supporting structures for the growth of
neurons and for their spatial arrangement.
1
1.2.1
Macroscopic structure
The anatomy of the brain as depicted in Figure 1(a), shows that different parts vary in cell
density and functionality. Figure 1(a) shows a cross-section of the human brain. The outer
layer is called the gray matter, due to the color caused by the high density of nerve cells.
The white matter, which is underneath the gray matter, consists most of connection fibers of
the nerve cells. The thalamus is situated in the middle of the brain and functions as a relay
station between the sensory system and the cortical systems for cognition and motor control.
Because of the high density of nerve cells the gray matter is the main part of information
(a) A cross-section of the human
brain shows different densities of
nerve cells [11].
(b) A general map of the human brain
assigns parts of the gray matter to fuctionalities [11].
(c) The vertical structure of the gray matter
shows six layers [11].
Figure 1: The macroscopic structure of the human brain.
processing of the brain.
The number of nerve cells (neurons), the number of connections (synapses) and the structure differs from person to person. The connections of each neuron are dynamic and change
over time. Some parts of the brain can still be assigned roughly to functionality as shown in
Figure 1(b).
Having a look at the vertical structure of the cortex, the gray matter can be partitioned
in six layers as shown in figure 1(c). The cells in each layer have similarities like cell type,
connections to other layers and connections to the thalamus and other parts.
2
1.2.2
Microscopic structure
The nerve cells are tiny structures which are connected to each other. For an understanding
of the brain a deeper look at the nerve cells is necessary. There are different cell types in a
brain. They vary in structure and size. Pyramidal, spiny stellate and smooth stellate cells
occur most often. For each layer there are types which occur more frequent.
In Figure 2 a typical neuron is depicted. It contains the soma (the cell body) dendrites
and axons. Electrical pulses are transported from the dendrites to the soma. In case of a
spike an electrical impulse is forwarded through the axon. These axons are connected via
synapses to further dendrites. The electrical impulse is transmitted via a chemical reaction in
the synapse to the dendrites of connected cells. There are excitatory and inhibitory neurons.
Figure 2: Microscopic structure of a neuron. [13]
The excitatory neurons excite the following neurons, in contrast the inhibitory neurons inhibit
the following neurons. Via electrical currents the connected neurons influence the membrane
potential of each neuron. The membrane potential can be measured. As an example the
membrane potential is plotted over time in Figure 3(a). Chemical processes inside the neuron generate a spike if the membrane potential reaches a specific electrical level called the
threshold. As shown in Figure 3(a) spikes are peaks in the membrane potential.
3
(a) The plot shows the membrane potential of a neuron
over the time. The peaks are called spikes. There are
four spikes in the time span shown. The firing threshold
of the cell is at about 58 mV [11].
(b) The dot plot shows spikes of each neuron
over time. On the y-axis there are the neurons
number. The histogram in the lower panel sums
up all spikes for each time bin. [11]
Figure 3: The activity of a single neurons is displayed using its membrane potential. For
multiple neurons the information is reduced to spike timings.
In order to analyze the membrane potential more objectively it is reduced to timings of
the spikes. For multiple neuron the spike timings in a dot plot can be visualized as in Figure
3(b). One can get an overview of the activity in a whole neuronal network if the spike sums
are plotted (summed up spikes for each time bin) in a so-called histogram.
1.3
Neuron models
To understand the behavior and functionality of spiking neurons, various models have been
developed over the last years, which focus on the electrical and chemical interactions.
There are two main types of spiking neuron models: single compartment models and
multi compartment models. The single compartment models reduce the whole dentric tree,
the axon and the soma of the nerve cell to a single point. Synapse models are used as
connections between these point neurons. A range of single compartment models have been
developed, which vary in accuracy and complexity. The goal of each model is to reproduce the
spiking activity. The Hodgkin Huxley model is one of the most accurate single compartment
models available.
C V̇ = I − ḡK + n4 (V − EK + ) − ḡN a+ m3 (V − EN a+ ) − gL (V − EL )
(1)
ṅ =
n∞ (V ) − n)
τn (V )
(2)
ṁ =
m∞ (V ) − m)
τm (V )
(3)
h∞ (V ) − h)
τh (V )
(4)
ḣ =
4
(a) A picture of a pyramidal cell with soma, dendrites and cell body.
(b) The neuron can be divided into soma, dendrites
and cell body.
(c) Reducing the
neuron to a point
neuron.
Figure 4: The partioning of a neuron for a single compartment model. The dendrites are the
connection inputs of the neuron and the axons are the connection output of the neuron.
The three ordinary differential equations (ODE) consider the ion currents of sodium (ḡK + n4 (V −
EK + )), potassium (ḡN a+ m3 (V − EN a+ )) and leak (gL (V − EL )) in a synapse.
The Izhikevich and the MAT model are simplifications of the Hodgkin Huxley model [11].
Further information can be found in the neuroscience literature [9]. The simplest one is the
Integrate-and-fire model, which is based on one ODE:
τm
dν
= −ν(t) + RI(t)
dt
(5)
The equation can be solved explicitly in one step. From the perspective of computational
costs, this is very important if a large amount neurons have to be simulated. This is the case
for most complex neuronal network models. Neuronal networks are described in section 1.4.
The multi compartment models partition the dendrites, soma and axons in smaller bits.
Therefore a multi compartment model is more accurate but also more complex. Each compartment is modeled similar to a single compartment model, while the different compartments
are coupled in an electrical cable equation. Further details are available in the neuroscience
literature [9].
5
1.4
Neuronal networks
The nervous system in the human brain is a complex neuronal network. It contains around
1011 neurons and each neuron has on average 7,000 synaptic connections to other neurons.
Estimates of the total number vary between 100 to 500 trillion connections [7]. Figure 5
shows axons in the cortical tissue in a micro meter scale. It gives an idea of how complex
the neuronal networks are. The most important external drive of the neuronal network is the
Figure 5: Axons in cortical tissue [1]
thalamus (1.2.1). It is connected to several neurons in the network. Via signals from other
parts of the human body it stimulates the network and can be seen as its input. Figure 6(a)
shows Golgi-stained neurons form somatosensory cortex in the macaque monkey. It is just a
small slice of the monkey brain but gives an idea of its structure. Reducing the neurons in the
nervous system to single compartment models allows to represent it as a graph (Figure 6(b)).
The graph stores point neurons and connections with their strengths and delays. The behavior
of a neuronal network depends on the size of the inhibitory and excitatory populations and
on the connections as well as the external drives. The balanced states of these networks is
characterized by fluctuations of population activity about an attractive fixed point[14]. If
there is a fix point, such a network is called balanced network.
6
(a) Golgi-stained neurons from
somatosensory cortex in the
macaque monkey on micro meter
scale. [4]
(b) Neurons are connected to
each other.
A graph allows
to store the connection information.
(c) A neuronal network contains excitatory and inhibitory
neurons. They are connected
to each other. Additionally
there are external drives which
are connected to the populations.
Figure 6: For a point neuron model, the brain cells are reduced to a graph of point neurons.
2
Simulation tools
There are a few simulation tools for neuronal networks available. The focus of these tools
are varying to a great extent. CSIM [15], NEST [10] and NCS [3] are using mostly singlecompartment neuron models. These tools focus on the functionality of the whole neuronal
network and not the chemical processes. Neuron [5], Genesis [2] and SPLIT [6] are supporting
multi-compartment neuron models. They allow a deeper look into the processes inside each
neuron.
Neuron is used for the simulation of empirically-based models of biological neurons and neuronal circuits, especially for models with complex branched anatomy. Genesis is a general
simulation system for the realistic modeling of neuronal and biological systems and NEST is
focused on large neuronal networks with biologically realistic connectivity. In the paper The
cell-type specific cortical microcircuit: relating structure and activity in a full-scale spiking
network model. , the NEST simulator is used. Therefore it is described in more detail below.
2.1
NEST
The NEST Initiative (http://nest-initiative.org) has developed the neuronal simulation tool
NEST over the past 20 years [8]. It supports the simulation of large-scale neuronal networks
with different types of neuron models and focuses on the dynamics, size and structure of
neuronal systems rather than on the exact morphology [10]. NEST supports models of information processing in the visual or auditory cortex of mammals, models of network activity
dynamics, laminar cortical networks or balanced random networks, and models of learning
and plasticity.
The complexity of the simulations reaches form small networks computed on local machines
up to ones using the full capabilities of the world’s leading supercomputers. To cover this
big range NEST is a hybrid parallel application using threads locally and message passing
over the compute nodes. For large networks of integrate-and-fire neurons, the communication dominates the computation costs. Large scale simulations with NEST were tested on K
and JUQUEEN. K and JUQUEEN are two of the fastest supercomputers available. Besides
7
high computational power spread over a huge amount of cores they have high interconnection
speed (See Figure 7). The results of scalability benchmarks are plotted below. The maximum
compute nodes
cores per compute node
architecture
clock speed
interconnect network
interconnection speed per link
K
88,128
8
SPARC64 VIIIfx
2 GHz
six-dimensional
mesh/torus network
5 GB/s
JUQUEEN
28,672
16
IBM PowerPC A2
1.6 GHz
five-dimensional
mesh/torus network
2 GB/s
Figure 7: Main characteristics of K and JUQEEN.
ressources
(comp. time * number of virtual proc.)
neurons
number of neurons is around 109 which is at the scale of a mouse brain. To provide enough
computation power and storage for a human brain simulation computers have to be at least
a factor 100 bigger than they currently are.
JUQUEEN
K
number of virtual processes
(a) Maximum network size and corresponding run
time as a function of number of virtual processes
[12]
3
3.1
JUQUEEN
K
number of neurons
(b) Required resources to simulate a network of size
N [12]
Example of a full-scale spiking network model
Motivation
Simulation in neuroscience is an important tool to test hypothesis inspired from anatomical
and functional data. Experiments on humans are difficult to perform and even the possibilities
to do experiments with animals are restricted. However there are connection measurements
available from species like rats and cats. Exact measurements of the human brain are not yet
available on this level of detail and constructing a human brain model is thus not possible at
the moment.
In order to understand the behavior of neuronal networks statistical measurements from
cats and rats are used to build up meaningful neuronal networks. These neuronal networks
can be tested with different stimuli to understand their functionality.
8
In the following sections the paper The cell-type specific cortical microcircuit: relating
structure and activity in a full-scale spiking network model is presented which shows whether
this is possible. This paper mainly focuses on the questions: Can a meaningful network be
created from these measurements? Can we use this network to understand the functionality of
the brain? Which connectivity data and which level of abstraction are adequate to reproduce
the reported differences in cell-type specific activity?
3.2
Model
A neuronal network is constructed, which contains different layers as shown in Figure 1(c). As
described in section 1.2.1 the properties of nerve cells in one layer are similar. Cell types and
the connection patterns are related. The network is defined by 8 neuronal populations representing the excitatory and inhibitory cells in L2/3, L4, L5, L6. The cell types are taken from
neuroscientific literature. Cell type specific connectivity and activity at local cortical networks
are characterized experimentally. The characteristics can be traced back to stochastic values.
For the connection probability they assume that the synapses are randomly distributed. They
get 8 cell types with 64 connection probabilities. Furthermore a connectivity map containing
all connection probabilities is generated. The results are shown in Figure 8.
Figure 8: Model definition. Layers 2/3, 4, 5, and 6 are each represented by an excitatory
(triangles) and an inhibitory (circles) population of model neurons. Input to the populations is
represented by thalamo-cortical input targeting layers 4 and 6 and other external background
input to all populations. The model size corresponds to the cortical network under a surface
of 1mm2 . [16]
With an integrated connectivity map a full-scale spiking network model of the local cortical
microcircuit is generated. Current-based leaky integrate-and-fire model neurons with exponential synaptic currents are randomly connected with connection probabilities according to
the integrated connectivity map used. The generated model contains over 80,000 neurons and
0.3 billion synapses.
3.3
Simulation
The simulation was executed on 24 nodes compute each with two quad core AMD Opteron
2834 processors and interconnected by a 24-port Voltaire InfiniBand switch ISR9024D-M.
The resulting simulation was running close to real time.
9
3.4
Analysis
The output of the simulation are spike trains of the neurons in the layers. The spike trains
contain spike timings of neurons. A plot of the spike timings can be seen in Figure 9(a).
In the first and third column there are spike timings plotted for each neuron as a dot when
they occur. The firing rate (spikes per second) is summed in the second column for two
simulations. The fourth column shows histograms of the firing rates for each layer. These
results are compared to real values measured from cats and rats. At first the model is tested
for robustness. Therefore the external inputs are varied to compare the resulting activity
of different layers. Constant input currents and poissonian background inputs yields similar
activity. Reducing the input of specific layers produces zero activity in these layers, which is
in agreement with real measurements. Comparing frequencies of simulation and experimental
results also provide good matches for most of the layers. All in all the model predicts the
cell-type specific activity very well, if compared with data from awake animals.
(a) Dot plot of the spike timings of the neurons in
layers is in the first and third column. The second and fourth column shows the firing rates of
each layer. Excitatory neurons are visualized with
dark dot/bar/line. Inhibitory neurons are visualized with bright dot/bar/line. [16]
(b) The dots show the main activities in the network.
The arrows visualize the activation and deactivation dynamics of the network. Excitatory neurons are visualized
with dark dot/bar/line. Inhibitory neurons are visualized with bright dot/bar/line. [16]
Figure 9(b) shows the dynamics of the neuronal network. Confronted wit transient thalamic input, the model exhibits a particular propagation of activity from the input layers to the
output layers [16]. The different layers activate and deactivate each other with feed-forward
and feedback loops. The layer L4 activates L2/3. L6 deactivates L4. L2/3 activates L5 and
L2/3 deactivates L4 and so on.
The propagation pattern is comparable to in vivo experiments of awake rats [16]. The simulation successfully reproduces prominent features of cortical activity, even though just simple
point integrate-and-fire neuron models were used. This suggests that the connectivity structure is enough to reproduce the activity reliably, and the exact features of the neurons only
play a minor role[16].
References
[1] Moshe Abeles. Corticonics: Neural circuits of the cerebral cortex. Cambridge University
Press, 1991.
10
[2] David Beeman. GENESIS modeling tutorial. Brains, Minds, and Media, 1:1–44, 2005.
[3] Reno) Brain Computation Laboratory (University of Nevada. NeoCortical simulator
(www.cse.unr.edu/brain/ncs).
[4] BrainMaps.org. Description golgi-stained neurons from somatosensory cortex in the
macaque monkey, 2006.
[5] Nicholas T Carnevale and Michael L Hines. The NEURON book. Cambridge University
Press, 2006.
[6] Mikael Djurfeldt, Christopher Johansson, Örjan Ekeberg, Martin Rehn, Mikael
Lundqvist, and Anders Lansner. Massively parallel simulation of brain-scale neuronal
network models. 2005.
[7] David A Drachman. Do we have brain to spare? Neurology, 64(12):2004–2005, 2005.
[8] Jochen Martin Eppler, Abigail Morrison, Hans Ekkehard Plesser, Markus Diesmann,
Marc-Oliver Gewaltig, Moritz Helias, Susanne Kunkel, and Yury Zaytsev. 20 years of
NEST: a mature brain simulator. Technical report, Computational and Systems Neuroscience, 2013.
[9] Wulfram Gerstner and Werner M Kistler. Spiking neuron models: Single neurons, populations, plasticity. Cambridge university press, 2002.
[10] Marc-Oliver Gewaltig and Markus Diesmann. NEST (NEural Simulation Tool). Scholarpedia, 2(4):1430, 2007.
[11] INM-6. Introduction to computational neuroscience. lecture notes, 2013.
[12] Susanne Kunkel, Maximilian Schmidt, Jochen M Eppler, Hans E Plesser, Gen Masumoto,
Jun Igarashi, Shin Ishii, Tomoki Fukai, Abigail Morrison, Markus Diesmann, et al. Spiking network simulation code for petascale computers. Frontiers in neuroinformatics, 8,
2014.
[13] LadyofHats. Complete neuron cell diagram
(https://commons.wikimedia.org/wiki/File:Complete neuron cell diagram en.svg), 2007.
[14] Fereshteh Lagzi and Stefan Rotter. A Markov model for the temporal dynamics of
balanced random networks of finite size. Frontiers in computational neuroscience, 8,
2014.
[15] W. Maass, T. Natschläger, and H. Markram.
(www.lsm.tugraz.at). 2002.
CSIM: A neural circuit simulator
[16] Tobias C Potjans and Markus Diesmann. The cell-type specific cortical microcircuit:
relating structure and activity in a full-scale spiking network model. Cerebral Cortex,
24(3):785–806, 2014.
11