Download PPT - Neural Micro circuits

Parallel neural Circuit
SIMulator (PCSIM) - Tutorial
Dejan Pecevski
Institute for Theoretical Computer Science
Graz University of Technology
FIAS Theoretical Neuroscience Summer School, Frankfurt, August, 2008
Outline of the Tutorial
•
•
•
•
•
•
General Info: What is PCSIM?
Features: What is PCSIM useful for?
Network elements: neurons and synapses
Scalability: Using clusters for parallel simulation.
Python Interface
PCSIM basic operations
 Creating and connecting neurons
 Simulating
 Recording signals
 Populations and Projections
• Summary
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
2/40
Who am I?
• Ph.D. Student at the
Institute for Theoretical Computer Science,
Graz University of Technology
• One of the developers of PCSIM.
• Research Interests
 Plasticity and Learning mechanisms in neural circuits
 Spiking Neural Networks
 Simulation technologies/algorithms for biological neural networks
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
3/40
PCSIM in a nutshell
• Simulator for parallel simulation of spiking and analog neural
networks with point neuron models
• Implemented in C++, with Python and Java interfaces
• High-level definition of neural networks
• Library of built-in neuron and synapse models
• Part of FACETS standardization efforts: the pyNN interface
http://www.neuralensemble.org
• Open-source, free, released under the GPL license, web page at:
http://www.igi.tugraz.at/pcsim
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
4/40
Developers
Thomas Natschläger Ph.D.
Software Competence Center Hagenberg
Hagenberg, Austria
Dejan Pecevski DI
Institute for theoretical computer science
Graz University of Technology
Graz, Austria
Other Contributors: Klaus Schuch DI (IGI),
Christian Ernstbrunner DI (SCCH), Walter Hargassner DI (SCCH)
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
5/40
Feature Overview
• Object Oriented Design
 The user interface/view is object oriented
• General Communication System
 Analog and spiking messages
 Network elements with multiple input and output ports
 Supported in distributed/multi-threaded mode
• Flexible construction of more complex network
architectures
 Populations of simulated objects
 Projections for connectivity specification
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
6/40
Feature Overview
• Parallel simulation
 Mixed multi-thread and distributed
 Easy to use, transparent to the user
• Easily extensible
 A compilation tool for creating PCSIM extension packages
• Usable as backend component in C++, Python, Java, …
 Easier and faster setup of simulations in Python and Java
• Runs on Linux or other Unix-like platforms
 can be compiled under Windows (still not tested).
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
7/40
PCSIM high-level structure
Utilities in Python
NeuroML
Parser
Java Interface
Python Interface
C++
Network Construction Layer
Built-in neuron and
synapse types
BOOST
Simulation Engine
MPI
GSL
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
8/40
Type of models PCSIM is suitable for
• Large recurrent networks with simple point neurons
 Distributed and multi-threaded capabilities
 Efficient simulation engine in C++
 More than 105 spiking neurons and 108 synapses
• Hybrid models
 Abstract modules (filters etc.) combined together with circuits of
analog neurons and spiking neurons
 Extensible with new custom elements
 Closed loop/feedback models
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
9/40
Type of models PCSIM is suitable for
• Structured models
 Networks composed of smaller subnetworks (Populations)
 Probabilistic connectivity patterns based on neuron’s attributes
• Diversity of neurons and synapses
 Different neuron types in the neuron populations
 Specifying random distribution for parameter values
 New neuron and synapse types can be implemented
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
10/40
Examples of PCSIM usage
• Spiking neural network model
of the V1 area in the visual
cortex of mammals (Schuch &
Rasch)
• Testing the computational
performance of laminar cortical
models based on experimental
data. (Schuch & Haeusler)
• Experiments to examine the
learning capabilities of rewardmodulated spike-timingdependent plasticity
(Legenstein, Pecevski, Maass)
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
11/40
A simple model represented in PCSIM
Model equations
dV (t ) V  V (t )
C 2  0 2  I (t )  J 2
dt
R
C
dV1 (t ) V0  V1 (t )

 J1
dt
R
if Vi  VT then V  V0 , spike
s
dI (t )
  I (t )
dt
I (t )  I (t ) w after spike
• The equations can be decoupled into separate simulated elements.
• Presynaptic neurons send spikes to the synapses.
• Synapses inject currents (or modify conductances) in the
postsynaptic neuron.
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
12/40
Network element: basic building block
o u tp u t po r ts
n e t w o rk
e le m e n t
n e tw o r k
e le m e n t
in p u t po rts
• Nodes of the network (dynamical systems)
• Advanced/integrated each time step of the simulation
• Represented by class SimObject

Neurons, synapses, recorders are derived from this class
• Multiple input and output ports, either analog or spiking.
• Connections are formed from output to input ports of the same
type.
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
13/40
Neurons and Synapses as Network Elements
• Synapses as network
elements are attached to the
postsynaptic neuron
• Synapses have one input port,
no output ports
• Neurons have only one output,
no inputs
• The spiking and analog
connections can connect
neurons on different nodes
p re sy n ap tic
ne ur on
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
s y n a ps e
p o stsy na pt ic
n eu ro n
14/40
Built-in Neuron Models
Spiking Models
Input Neurons
• LifNeuron, CbLifNeuron
• PoissonInputNeuron
• SpikingInputNeuron
Leaky integrate-and-fire
• HHNeuron
Hodgkin-Huxley Neuron
• IzhiNeuron
(Izhikevich, 2004)
• aeIFNeuron
Analog Neurons
• LinearAnalogNeuron
Adaptive Exponential I&F
(Brette and Gerstner, 2005)
• LinearPoissonNeuron
Leaky integrate with
Poisson output
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
15/40
Built-in Synapse Models
• Type
 current based
 conductance based
• PSR Kernel
 exponential, alpha,
double-exponential, Square
• Short-term Plasticity
(Markram et al. 1998)
• Other Mechanisms
 NMDA
(Gabbiani et al. 1994)
 GabaB
(Mainen et al. 1994)
 Reward-modulated STDP
(Izhikevich, 2007)
 Homeostatic plasticity
(Buonomano et al. 2005)

• Spike-timing-dependent
Plasticity
 (Froemke and Dan, 2002)
 (Gütig et al, 2003)
 each pair and nearest
Ornstein-Uhlenbeck noise synapse
(Destexhe et al. 2001)
• Analog Synapse
neighbour
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
16/40
Simulation Strategy
• Clock-driven simulation with a fixed time step.
• Hybrid Integration Strategy
 Neurons are integrated every time step.
 Synapses are event driven, i.e. they are integrated only after
receiving a spike.
• Numeric Integration Algorithms
 Exponential Euler Method
 ODE solvers from GSL
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
17/40
Parallel Simulation
•
•
Each MPI process advances subset of neurons
MPI as underlying communication layer

•
spiking and analog messages
Each process can have multiple-threads
PCSIM
subnetwork
PCSIM
subnetwork
PCSIM
subnetwork
MPI
• Transparent to the user

PCSIM
subnetwork
Default round-robin distribution of the neurons
over nodes
 Custom distribution strategies
 Easy retrieval of recordings (as in singlethread case)
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
18/40
Parallel Simulation
• Why?
 Speed-up simulation of an existing model by using more
processors
 Enables simulation of larger models: no memory limit problem
 Scale up the number of neurons while keeping the simulation
time the same.
• Comparison between single-threaded and distributed
simulation
 < 104 neurons and 107 synapses on a single machine
 > 105 neurons and 108 synapses on clusters
 > 106 neurons and 1010 synapses on supercomputers?
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
19/40
Scalability of PCSIM
Hardware: 3.4 Ghz, 512 kb cache, 4GB RAM
Model:
• 50000 LifNeurons, 4.0 Hz average firing rate
• 50106 synapses, 1.5 ms transmission delay
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
20/40
Scalability of PCSIM
Hardware: 3.4 Ghz, 512 kb cache, 4GB RAM
Cuba Model:
• 5000 LifNeurons per node, 4.0 Hz average firing rate
• 103 synapses per neuron, 1.0 ms transmission delay, broad
distribution of time synaptic constants
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
21/40
Extending PCSIM
• What are extensions?

User implemented classes that are plugged into the PCSIM OO
framework
 usualy coded in C++ (can be coded also in Python)
• Compilation of additional separate Python extension packages,
without the need to recompile the main PCSIM code.
• The extension classes are automatically wrapped in Python
• Usual extensions for PCSIM

new neuron/synapse types
 other user defined network elements
 custom rules for specific network construction
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
22/40
Python Interface
What is Python? (taken from www.python.org)
• Python is a dynamic object-oriented programming language
• offers strong support for integration with other languages and tools.
• comes with extensive standard libraries (“batteries included”).
• can be learned in a few days.
• Many Python programmers report substantial productivity gains and
feel the language encourages the development of higher quality, more
maintainable code.
• many scientific packages: scipy, numpy, matplotlib, ipython, Rpy etc.
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
23/40
Python Interface
• PCSIM can be imported as a package within Python
from pypcsim import *
• The model is represented as a network object:
net = SingleThreadNetwork()
• Individual neurons and synapses are accessible as python objects.
• Population and Projection objects for high-level construction of
networks composed of populations
• Together with the other Python packages for scientific computing
provides a complete working environment for neural simulations.
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
24/40
Creating and connecting neurons
# Creates one LIF neuron with default parameter values
nrn_model = LifNeuron()
nrn_id = net.create( nrn_model )
# Creates array of 10 LIFneurons
nrn_array = net.create( LifNeuron(), 10)
# Connects the first and the second neuron in the array
syn_id = net.connect( nrn_array[0], nrn_array[1],
StaticSpikingSynapse())
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
25/40
Specifying neuron/synapse parameters
nrn = net.create(LifNeuron( Cm=2e-10,
Rm=1e8,
Vthresh=-50e-3,
Vresting=-60e-3,
Vreset=-60e-3,
Trefract=5e-3,
Vinit=-60e-3 )) ;
syn = net.connect( pre_nrn, post_nrn,
StaticSpikingSynapse( W=1e-10,
tau= 5e-3,
delay=1e-3 ))
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
26/40
Recording
# record the output spikes of a neuron
rec_spk = net.record( nrn, SpikeTimeRecorder() )
# record the membrane potential
rec_vm = net.record( nrn, “Vm”, AnalogRecorder() )
# record the weight of a synapse
rec_syn = net.record( syn, “W”, AnalogRecorder() )
# record any field in an element
rec = net.record( myElement,
“anyFieldName”, AnalogRecorder())
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
27/40
Simulate the model
# simulate the network for 1 second
net.simulate(1.0)
# or advance it for 200 time steps
net.advance(200)
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
28/40
Accessing individual neurons/synapses
# creates return an ID of the neuron
nrn_id = net.create( LifNeuron() )
# get the actual neuron object from the network
nrn_object = net.object(nrn_id)
# Manipulate the neuron object
nrn_object.Vthresh = -59e-3
# getting the recorded values from a recorder
values = list(net.object(rec_id).getRecordedValues())
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
29/40
Synapse classes nomenclature
• StaticCurrExpSynapse
(StaticSpikingSynapse)
Current/conductance based?
• DynamicCondDblExpSynapse
Short-term plasticity
Shape of PSR kernel
• StaticStdpCondAlphaSynapse
• DynamicCurrAlphaSynapse
• DynamicNMDAAlphaSynapse
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
30/40
Hello, world!
from pypcsim import *
net = SingleThreadNetwork()
pre_nrn = net.create( LifNeuron(Iinject = 5e-8) )
post_nrn = net.create( LifNeuron(Iinject = 5e-8) )
net.connect( pre_nrn, post_nrn, StaticSpikingSynapse())
rec = net.record( post_nrn, SpikeTimeRecorder() )
net.simulate(0.2)
print “spike times:”,
list(net.object(rec).getRecordedValues())
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
31/40
Specifying inputs
# Create input neuron which spikes at specific times
net.create(SpikingInputNeuron( [0.1, 0.2, 0.4]))
# Create input neuron with Poisson process spike times
net.create( PoissonInputNeuron(rate = 10, duration = 10) )
# Create analog input neuron with an
# output analog signal specified as array
net.create( AnalogInputNeuron( arange(0,10,1e-4) % 1 ) )
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
32/40
Creating and connecting many neurons
•
•
Population of 1000 neurons randomly interconnected
with 0.1 probability
Population of recorders: one for each neuron in the population
# create a population of 1000 LIF neurons
popul = SimObjectPopulation(net, LifNeuron(), 1000)
# create the projection
proj = ConnectionsProjection( popul, popul,
StaticSpikingSynapse(),
RandomConnections( 0.1 ) )
# create a recorder population
rec_popul = popul.record( SpikeTimeRecorder() )
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
33/40
Random distributions for parameter values
• Factory – an object that generates PCSIM elements
• Neuron and synapse objects (used as models for creation) are
simple clone factories
• SimObjectVariationFactory – factory that attaches random
distributions to parameter names
nrn_factory = SimObjectVariationFactory( LifNeuron() )
nrn_factory.set( “Vinit”,
NormalDistribution( -55e-3, 0.1 ) )
popul = SimObjectPopulation(net, nrn_factory, 1000)
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
34/40
Spatial heterogenous populations
•
•
Each neuron has coordinates in 3D space
The population is composed of different families of neurons
(that can be of different type)
exc_nrn = LifNeuron( Cm = 2e-10,
Inoise = 1e-10)
inh_nrn = LifNeuron( Cm = 3e-10 )
popul = SpatialFamilyPopulation(net,
[ exc_nrn, inh_nrn],
RatioBasedFamilies((4,1)
),
CuboidIntegerGrid3D(20,10,10))
exc_popul, inh_popul = popul.splitFamilies()
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
35/40
Distance dependent random connections
Creates random connections with probability
p( D)  C  e

D2
2
where D is the euclidean distance between the two neurons
proj = ConnectionsProjection( popul, popul,
StaticSpikingSynapse(),
EuclideanDistanceRandomConnections( C, lambda ) )
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
36/40
Saving recorded data
from pypcsimplus import *
# Create the recording class
r = Recordings()
# Add the recorder populations as attributes to r
r.spikes = popul1.record( SpikeTimeRecorder() )
r.vm_traces = popul2.record( “Vm”, AnalogRecorder() )
# Some additional variables to be saved
r.v = 5
# Save the Recordings class to a HDF5 file
r.saveInOneH5File(“results.h5”)
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
37/40
Running a parallel simulation
• Change the network class from SingleThreadNetwork to
either:
MultiThreadNetwork( nThreads = 4 )
 DistributedSingleThreadNetwork
 DistributedMultiThreadNetwork

• then run the script with mpirun
$ mpirun –n 10 python my_experiment.py
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
38/40
Summary
• PCSIM is a tool for simulation structured random neural networks
composed of spiking and analog point neuron models
• Has various built-in neuron and synapse models
• Supports parallel simulation: distributed and multi-threaded
• Flexible high-level construction of networks based on probabilistic
rules
• The Object-oriented framework is designed to support easy
extensions
• Can be used as a package within the Python interpreting
programming language, together with other useful scientific packages
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
39/40
PCSIM Resources
• Web page
http://www.lsm.tugraz.at/pcsim
• On Sourceforge
http://www.sourceforge.net/projects/pcsim
• Mailing list
http://sourceforge.net/mailarchive/forum.php?forum_name=pcsim-users
• User manual
http://www.lsm.tugraz.at/pcsim/usermanual/html/index.html
• C++ class reference
http://www.lsm.tugraz.at/pcsim/cppclassreference/html/hierarchy.html
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
40/40
Thank you for your attention!
Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008
41/40