Download Overview Synaptic plasticity Synaptic strength

Overview
• Forms of synaptic plasticity
• Models of spike-timing-dependent synaptic plasticity:
– phenomenological
– optimal
– biophysical
Modeling plasticity at a single
neuron and network levels
• Application of learning rules:
– modeling brain in health and disease
– neuronal robot control
Aušra Saudargienė
Vytautas Magnus University
Lithuania
1
2
3nd Baltic-Nordic Summer School on Neuroinformatics, 15-18 June 2015, Tartu, Estonia
Synaptic plasticity
Synaptic strength - weight
• Human brain consists of a trillion
(1012) neurons and a quadrillion
(1015) synapses
Presynaptically:
• Number of synaptic vesicles n
• Probability of neurotransmitter release p
Postsynaptically:
• Maximal conductance g associated with one synaptic vesicle
• Brain plasticity – the ability of the
brain to modify itself and adapt to
changing environment.
Axon terminal
Synaptic weight:
• Synaptic plasticity - modification of the synaptic strength
triggered by the neuronal activity
w=npg
Denritic spine
3
4
http://www.brainhealthandpuzzles.com/brain_plasticity.html
http://flipper.diff.org/app/items/info/4864
Forms of synaptic plasticity
Hebb‘s rule: LTP
• Short-term synaptic plasticity
When an axon of cell A is near enough to excite cell B and
repeatedly or persistently takes part in firing it, some growth
processes or metabolic change takes place in one or both
cells so that A‘s efficiency ... is increased.
Donald Hebb (1949)
– Induction: presynaptic activity, 0.5 sec
– Maintenance: 1 sec
• Long-term synaptic plasticity
– Induction: correlated pre- and postsynaptic activity, 1-3 sec
– Maintenance: several hours
A
• Long-term potentiation (LTP): Hebb’s rule
B
– LTP is faster: ~1.3 min after the onset of the stimulation protocol
t
t
• Long-term depression (LTD): Stent’s rule
A
– LTD is slower: ~3.1 min after the onset of the stimulation protocol
B
5
w
6
(O’Connor et al., 2005)
1
Spike-timing-dependent plasticity
STDP
Stent‘s rule: LTD
•
When the presynaptic axon of cell A repeatedly and
persistently fails to axcite the postsynaptic cell B while cell B
is firing under the influence of other presynaptic axons,
metabolic change takes place in one or both cells so that A‘s
efficiency ... is decreased.
Gunther Stent (1973)
Spike-timing-dependent synaptic plasticity is a form of bidirectional change
in synaptic strength that depends on the temporal order and temporal
difference of the pre- and postsynaptic activity.
Long-term
depression
A
t
Post
B
t
t
t
A
Synaptic weigh change %
Post-Pre
Pre
Long-term
potentiation
Pre-Post
Pre
t
B
T = t post − t pre > 0
T = t post − t pre < 0
w
t
Post
T = t post − t pre
7
8
(Bi&Poo, 2001)
Biophysics of
Spike-timing dependent plasticity
Induction and measuring
synaptic modifications
1. Assessment: Presynaptic action potential evokes EPSP at a postsynaptic site
Pre neuron
• Pairing pre-and postsynaptic activity at 1-5Hz for 60100sec
Pre
EPSP
t
Post
Post neuron
t
t
2. Induction: Pre- and postsynaptic neurons are activated simultaneously
• Presynaptic site:
Pre neuron
– neurotransmitter release
t
Post neuron
• Postsynaptic site:
t
3. Assesment: Presynaptic action potential evokes a stronger EPSP
– NMDAr, AMPAr channel activation
– Somatic action potential, back-propagating spike
– Voltage-gated calcium channel opening
Pre neuron
t
Post neuron
EPSP
9
10
Stuart and Sakmann, 1994
Neuronal response before
learning
Application of STDP in modelling:
healthy brain
• Neuronal response shift earlier in time (Gerstner and
Kistler, 2002)
• Formation of receptive fields in place cells (Gerstner and
Kistler, 2002)
• Development of direction selectivity in the visual system
(Honda, Urakubo, Tanaka, Kuroda, 2011)
• Temporal sequence learning (Izhikevich, 2007)
• Memory encoding and retrieval in hippocampus
(Cutsuridis et al., 2010)
11
Neuron receives trains of 5 spikes.
x1 w1
Neuron fires after 4 synapses are activated.
x2 w
2
x1
x2
t
x3
t
x4
t
x5
t
y
t
x3 w3
x4 w4
x5 w5
STDP
∆w
Weight increase
Weight decrease
y
 + −τT+
A e , T ≥ 0
∆w(T ) = 
T
 A− e τ − , T < 0

T = t post − t pre
2
Neuronal response after
learning
The first synapses generates a response
x1
x2
t
x3
t
x4
t
x5
t
y
t
Neuronal response shift earlier in time
Neuronal response
before learning
31ms
x2 w
2
and
58ms
x3 w3
after stimulus
presentation
x4 w4
x5 w5
Neuronal response
after learning
Neuron is active
y
STDP
∆w
Weight
increase
Weight decrease
Neuron is active
x1 w1
 + −τT+
A e , T ≥ 0
∆w(T ) = 
T
 A− e τ − , T < 0

T = t post − t pre
Neuron is presented
with 20 inputs
5ms
after stimulus
presentation
Gerstner and Kistler, Spiking Neuron Models. Single Neurons, Populations, Plasticity
Cambridge University Press, 2002
Desynchronization of neural
populations
Application of STDP models
Parkinson’s disease
• Parkinson’s disease is characterized by abnormally
strong, pathological neuronal synchronization
• Synaptic coupling is strong and has to be unlearned
• Parkinson’s disease – a neurogenerative disease
characterized by tremors, rigidity, slowness of
movement, stiffness.
Normal
• Electrical high frequency
deep brain stimulation of
the subthalamic nucleus is
the effective surgical
treatment for Parkinson's
disease.
Pathological
15
http://www.ocduk.org/dbs-caution-compassion-advised
Unlearning strong synaptic
coupling via STDP
Weak synaptic
coupling
Strong synaptic
coupling
16
Popovych V., Tass,P. Desynchronizing electrical and sensory coordinated reset neuromodulation. Front
Hum Neurosc 2012
Models of Spike-timing
Dependent Plasticity
• Network of Hodgkin–Huxley neurons and STDP learning rule
• Stimulation causes an unlearning of pathological
synchronization
– Weak or Strong stimulation does not have a long-lasting effect
– Intermediate strength stimulation suppresses synaptic coupling and
leads to a long- lasting desynchronization
Phenomenological – encode data and intuitions
about synaptic plasticity.
(Abbott and Song, 1999; Gerstner et al, 1996; Kempter et al, 1999; Kistler et al,
2000; Song et al. 2000; van Rossum, 2000);
Optimal – use some optimality criterion to
deduce the rules of synaptic modifications.
Synaptic coupling
(Bell and Parra, 2003; Dayan et al, 2004; Pfister et al, 2004; Toyoizumi et al,
2005);
Popovych V., Tass, P. Desynchronizing electrical and sensory coordinated reset
neuromodulation. Front Hum Neurosc 2012
Biophysical – biophysically realistic, usually based
on calcium dynamics, involve detailed
biochemical reactions to explain synaptic
plasticity.
(Bhalla and Iyengar, 1999; Karmarkar and Buonomano, 2001; Abarbanel et al,
2002; d’Alcantara et al, 2003; Shouval et al, 2002, 2005; Saudargiene et al, 2004;
Rubin et al, 2005; Badoual et al, 2006; Pi and Lisman 2008; Graupner and Brunel
2007, 2012; Clopath et al 2010; Jin et al 2013);
17
18
3
Phenomenological models
Optimal models
• Black box approach
• No explicit modelling of biophysical processes
• Used in networks – memory formation, neuronal selectivity
• Derive the optimal STDP function for a given task:
– Maximal information transfer in network of spiking
neurons – use information theory to obtain STDP curve
(Bell and Parra, 2003; Toyoizumi et al, 2005)
Pre
T
Post
T
∆ω
tpre
 + −τT+
A e , T > 0
∆w(T ) = 
T
 A− eτ − , T < 0

T = t post − t pre
– Optimal filter function - STDP shape represents the filter
to remove the noise from the signal and increase the
influence of its significant components;
∆ω
tpost
(Dayan et al, 2004)
T
(Abbott and Song, 1999; Gerstner et al, 1996; Pfister and Gerstner, 2006;
Kempter et al, 1999; Kistler et al, 2000; Song et al. 2000; van Rossum, 2000)19
Biophysical models
[Ca2+]
(Pfister et al, 2004)
20
Calcium dynamics in spines
• White box approach
• Based on Ca2+ dynamics in spine of a postsynaptic
neuron
• Used to model synaptic plasticity at a single synapse,
local mircocircuit
Neuronal
activity
– Optimal firing time of the postsynaptic neuron –
maximization of the probability of firing at the defined
times leads to the STDP function
NMDAr channels
Ca2+ pump
Ca2+ buffers
∆ω
Voltage gated Ca2+
channels
Ca2+ diffusion
(Abarbanel et al, 2002; Badoual et al, 2006; Bhalla and Iyengar, 1999; Bush and Jin, 2012;
Graupner and Brunel, 2007, 2012; d’Alcantara et al, 2003; Karmarkar and Buonomano, 2001;
Rubin et al, 2005; Shouval et al, 2002, 2005; Saudargiene et al, 2004, 2005);
21
22
Malenka and Nicoll, 1999
Calcium concentration
Calcium current through voltage-gated calcium channels:
I Ca _ VGCC = g Ca _ L m p h q (V − ECa )
Calcium current through NMDA receptor-gated channels:
I Ca _ NMDA = 0.1g NMDA (V − E NMDA )
Calcium concentration:
2+
d [Ca 2+ ]
I
[Ca0 ] − [Ca 2+ ]
= − Ca +
dt
2 Fv
τ Ca
23
Decay due to pumping, buffering, diffusion
http://synapses.clm.utexas.edu/atlas/1_4_1_17.stm
Ca2+ transients
STDP stimulation protocol
Amplitude and time course of Ca2+ transient depend
on the temporal order and temporal difference of
pre- and postsynaptic activity.
Post - Pre
LTD
Pre
Post
Pre – Post
LTP
T
Pre
T=-20
Post
[Ca2+],
µM
T
T=20
[Ca2+],
µM
LTP threshold θp
LTD threshold θd
t, ms
t, ms
24
4
Biophysical models
Calcium control hypothesis
Shouval et al, 2002, 2005
• Weight change is determined by [Ca2+] levels:
– low [Ca2+] levels – no changes
– intermediate [Ca2+] levels between θd and θp – LTD
– high [Ca2+] levels above θp – LTP
Biophysical models
Calcium control hypothesis
Shouval et al, 2002, 2005
• Weight change is proportional to the threshold function Ω([Ca]):
∆w ≅ η Ω([Ca ])
• Plasticity outcomes:
Ω([Ca])
– short positive T – LTP
– short negative and large positive T – LTD
Graupner and Brunel,252010
Biochemical mechanisms of
Spike-timing dependent plasticity
26
Biophysical models
LTP and LTD enzyme competition
• Correlated activity of pre- and postsynaptic
neurons
• Glutamate release
• Activation of NMDAr-gated channels
• Increase in calcium concentration in a spine
m+Ca2+
h + Glu
Ca2+/CaM KinaseII
Dephosporylation
of AMPA receptors
Phosporylation
of AMPA receptors
Long-term
depression
Long-term
potentiation
am
bm
K + 4Ca2+
m*
ak
bk
K*
ah *
h
bh
m * + h* + Ph
27
LTP enzyme K
LTD enzyme Ph
Ca2+
Phosphatases
Badoual et al, 2005
Ca2+
Ph*
∆w = − Ph* + K *
m activated by Ca2+
h activated by Glu
Ph activated by m* and h*
Reflects activity of
Ca2+/calmodulin-dependent protein
kinase II, activated by Ca2+
28
Kandel, Schwartz, Jessell, 2000
Ca2+/calmodulin (CaM)-dependent
protein kinase CaMKII
Binary synapse model
• Synaptic changes are all-ornone switch-like events
DOWN → UP
UP → DOWN
• CaMKII holoenzyme consists of 6 subunits.
• Each subunit can be in:
– active, phosphorylated state;
– not active, dephosphorylated state.
• Two stable states of synaptic
transmission efficacy at resting
calcium levels:
– DOWN, low synaptic strength
– UP, high synaptic strength
• CaMKII can be stable in two states at resting calcium:
– weakly phosphorylated state - DOWN
– highly phosphorylated state - UP
• Transitions:
– UP-DOWN: LTD
– DOWN-UP: LTP
• CaMKII switch in a binary synapse:
– Transition UP-DOWN: LTD
– Transition DOWN-UP: LTP
29
Graupner and Brunel, 2010
30
Lisman, Schulman & Cline, 2002
5
Biochemical networks of synaptic
plasticity: more details
Bistable synapse model based on
CaMKII activation network
• Ca2+/calmodulin C activates CaMKII
• PP1 inhibits, dephosphorylates CaMKII
• Model of CaMKII regulation, mGlu receptor and NMDA
receptor activation
– PP1 influence is increased by calcineurin activation
– PP1 influence is reduced by cyclic-AMP and PKA activation
14 differential equations!
31
32
Graupner and Brunel, 2008
Bhalla and Iyengar, 1999
STDP timing windows
Multiple forms of STDP
Hippocampus,
excitatory
synapse
• STDP depends on:
• Stimulation protocol
– Pattern of post activity (single spike, burst)
– Frequency of pairings
– Duration of stimulation
• Properties of neuron
Hippocampus,
inhibitory
synapse
Neocortex,
excitatory
synapse
– Dendritic plasticity
– Dendritic synapse location
• Neuron type
• Brain region
33
34
Buchannan and Mellor, 2010
STDP model based on kinase and
phosphatase competition
Abbott and Nelson, 2000
Diversity of STDP curves
• Recent models are able to account for the effect of spike
pattern, frequency, duration of stimulation.
• Numbers of postsynaptic spikes and duration of the stimulation
qualitatively change the STDP curve.
• Synaptic efficacy ρ: two stable states UP and DOWN
– transition UP-DOWN: LTD
– transition DOWN-UP: LTP
• Synaptic efficacy ρ
τ
LTD term
LTP term
dρ
= − ρ (1 − ρ )( ρ* − ρ ) + γ p (1 − ρ )Θ[Ca (t ) − θ p ] − γ d ρΘ[Ca (t ) − θ d ] + Noise(t )
dt
Absence of
Kinase influence
activity
[Ca2+]>Θp
Phosphotase influence
[Ca2+]>Θd
35
Graupner and Brunel, 2012
36
Graupner and Brunel, 2012
6
NEURON demo
Modeling learning in hippocampus
•
•
•
Models of LTP and LTD: review
NEURON simulator
STDP model: synaptic efficacy variables ρUP, ρDOWN
Associative learning in a CA1 microcircuit
ρUP
ρDOWN
•
•
37
Neuromodulation of STDP
117 models analyzed
Models of molecular mechanisms underlying synaptic plasticity are
classified:
–
–
–
–
Direction: LTP/LTD
Complexity: single pathways/simplified processes/signaling networks
Method: deterministic/stochastic method
Cell type
38
Dopamine pathways
• Dopamine is involved in:
• STDP depends on:
– reward-motivated behavior
– pleasure
– motor control
– pre- and postsynaptic activity
– a “third factor” – neuromodulators
dopamine, acetylcholine, etc
• Neuromodulators open the STDP gate:
– necessary for plasticity induction or
– change the shape of the STDP
• Dopamine signal mimics
reward prediction error
(reinforcement learning
theory)
• Time and spatial scale:
– Correlated pre-post ativity, STDP – local and fast, ms
– Neuromodulation – global and slow, 3-5sec
Markram H , Gerstner W and Sjöström PJ. A History of Spike-timing-dependent Plasticity Synaptic
Neuroscience 2011
39
Pawlak V, Wickens JR, Kirkwood A and Kerr JND. Timing is not everything: neuromodulation opens the
STDP gate. Frontiers in Synaptic Neuroscience 2010.
Dopamine controls STDP in
hippocampal neurons
•
Dopamine changes the shape of
the STDP window.
• LTP window is wider
• LTD converts into LTP
•
40
http://www.drugabuse.gov/publications/addiction-science/why-do-people-abuse-drugs/brain-pathways-are-affected-by-drugs-abuse
http://www.drugabuse.gov/publications/addiction-science/why-do-people-abuse-drugs/brain-pathways-are-affected-by-drugs-abuse
How do molecules affect
behaviour?
Dopamine reduces the number of
spike pairs required to induce LTP
from 60 to 10.
Pawlak et al, 2014
http://www.cartoonstock.com/newscartoons/directory/b/behaviour.asp
http://molinterv.aspetjournals.org/cgi/content/abstract/3/7/375
Izhikevich E. Solving the distal reward problem through linkage of STDP and dopamine signal. Cerebral
41
Cortex 2007
42
7
Application of STDP-like learning
rule in robotics
Learning in a navigating robot
B.Porr, F.Wörgötter 2003
Differential Hebbian learning:
dw
= µxvision y ′
dt
∆ω
T<0
vision x1
T>0
vision x1
collision x0
Collision
sensors
collision x0
Vision
sensors
T
T
T
Robot learns to navigate without
collisions
RunBot - a biped walking robot –
learns to walk on a terrain
Porr, B. and Wörgötter, F. (2003) Isotropic sequence order learning. Neural Comp., 15, 831-864.
STDP-like weight change
43
44
Manoonpong, P., Porr, B. and Woergoetter, F. (2007) Adaptive, Fast Walking in a Biped Robot under Neuronal Comtrol and
learning. Comp Biology, 3(7),1305-1311.
Final remarks
•
•
•
Final remarks
Phenomenological models apply the STDP function and
analyze the computational principles and its functional
consequences in networks.
•
Neuromudulation gates STDP and provides link from
correlation-based synaptic plasticity rules to behaviorally
based learning.
Optimal models derive optimal STDP function for a given task
– optimal firing time, maximal information transfer between
neurons, optimal filter function.
•
Link between the cellular processes and behaviour is hard to
establish. Robotics offer one of the possibilities to analyze the
functional consequences of the learning rules.
Biophysical models explain STDP using the principles of ion
channel dynamics and intracellular processes. Usually
contains a large parameter space.
•
Models contribute to understanding of brain functioning in
health and disease.
45
46
Final remarks
•
Acknowledgement
Modeling synaptic plasticity:
–
from molecules (nm) to a system level (m);
–
from fast electrical events (ms) to slow
chemical processes (min, hours);
–
from unsupervised learning to reinforcement
learning (reward-based).
47
•
•
•
•
•
Marja-Leena Linne
Bruce Graham
Arnd Roth
Florentin Worgotter
Bernd Porr
48
https://www.humanbrainproject.eu/documents/10180/17648/TheHBPReport_LR.pdf/18e5747e-10af-4bec-9806-d03aead57655
8