* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Overview Synaptic plasticity Synaptic strength
Multielectrode array wikipedia , lookup
Neural engineering wikipedia , lookup
Aging brain wikipedia , lookup
Premovement neuronal activity wikipedia , lookup
Neural oscillation wikipedia , lookup
Apical dendrite wikipedia , lookup
Eyeblink conditioning wikipedia , lookup
Biochemistry of Alzheimer's disease wikipedia , lookup
Types of artificial neural networks wikipedia , lookup
Transcranial direct-current stimulation wikipedia , lookup
Haemodynamic response wikipedia , lookup
Neuroeconomics wikipedia , lookup
Neural coding wikipedia , lookup
Endocannabinoid system wikipedia , lookup
Environmental enrichment wikipedia , lookup
Development of the nervous system wikipedia , lookup
Optogenetics wikipedia , lookup
Neuroanatomy wikipedia , lookup
Single-unit recording wikipedia , lookup
Memory consolidation wikipedia , lookup
Donald O. Hebb wikipedia , lookup
Dendritic spine wikipedia , lookup
Spike-and-wave wikipedia , lookup
Holonomic brain theory wikipedia , lookup
Neuropsychopharmacology wikipedia , lookup
Epigenetics in learning and memory wikipedia , lookup
Stimulus (physiology) wikipedia , lookup
Clinical neurochemistry wikipedia , lookup
NMDA receptor wikipedia , lookup
Pre-Bötzinger complex wikipedia , lookup
Neural modeling fields wikipedia , lookup
End-plate potential wikipedia , lookup
Neuroplasticity wikipedia , lookup
Metastability in the brain wikipedia , lookup
De novo protein synthesis theory of memory formation wikipedia , lookup
Molecular neuroscience wikipedia , lookup
Biological neuron model wikipedia , lookup
Synaptic noise wikipedia , lookup
Neuromuscular junction wikipedia , lookup
Neurotransmitter wikipedia , lookup
Long-term potentiation wikipedia , lookup
Nervous system network models wikipedia , lookup
Synaptic gating wikipedia , lookup
Long-term depression wikipedia , lookup
Synaptogenesis wikipedia , lookup
Chemical synapse wikipedia , lookup
Overview • Forms of synaptic plasticity • Models of spike-timing-dependent synaptic plasticity: – phenomenological – optimal – biophysical Modeling plasticity at a single neuron and network levels • Application of learning rules: – modeling brain in health and disease – neuronal robot control Aušra Saudargienė Vytautas Magnus University Lithuania 1 2 3nd Baltic-Nordic Summer School on Neuroinformatics, 15-18 June 2015, Tartu, Estonia Synaptic plasticity Synaptic strength - weight • Human brain consists of a trillion (1012) neurons and a quadrillion (1015) synapses Presynaptically: • Number of synaptic vesicles n • Probability of neurotransmitter release p Postsynaptically: • Maximal conductance g associated with one synaptic vesicle • Brain plasticity – the ability of the brain to modify itself and adapt to changing environment. Axon terminal Synaptic weight: • Synaptic plasticity - modification of the synaptic strength triggered by the neuronal activity w=npg Denritic spine 3 4 http://www.brainhealthandpuzzles.com/brain_plasticity.html http://flipper.diff.org/app/items/info/4864 Forms of synaptic plasticity Hebb‘s rule: LTP • Short-term synaptic plasticity When an axon of cell A is near enough to excite cell B and repeatedly or persistently takes part in firing it, some growth processes or metabolic change takes place in one or both cells so that A‘s efficiency ... is increased. Donald Hebb (1949) – Induction: presynaptic activity, 0.5 sec – Maintenance: 1 sec • Long-term synaptic plasticity – Induction: correlated pre- and postsynaptic activity, 1-3 sec – Maintenance: several hours A • Long-term potentiation (LTP): Hebb’s rule B – LTP is faster: ~1.3 min after the onset of the stimulation protocol t t • Long-term depression (LTD): Stent’s rule A – LTD is slower: ~3.1 min after the onset of the stimulation protocol B 5 w 6 (O’Connor et al., 2005) 1 Spike-timing-dependent plasticity STDP Stent‘s rule: LTD • When the presynaptic axon of cell A repeatedly and persistently fails to axcite the postsynaptic cell B while cell B is firing under the influence of other presynaptic axons, metabolic change takes place in one or both cells so that A‘s efficiency ... is decreased. Gunther Stent (1973) Spike-timing-dependent synaptic plasticity is a form of bidirectional change in synaptic strength that depends on the temporal order and temporal difference of the pre- and postsynaptic activity. Long-term depression A t Post B t t t A Synaptic weigh change % Post-Pre Pre Long-term potentiation Pre-Post Pre t B T = t post − t pre > 0 T = t post − t pre < 0 w t Post T = t post − t pre 7 8 (Bi&Poo, 2001) Biophysics of Spike-timing dependent plasticity Induction and measuring synaptic modifications 1. Assessment: Presynaptic action potential evokes EPSP at a postsynaptic site Pre neuron • Pairing pre-and postsynaptic activity at 1-5Hz for 60100sec Pre EPSP t Post Post neuron t t 2. Induction: Pre- and postsynaptic neurons are activated simultaneously • Presynaptic site: Pre neuron – neurotransmitter release t Post neuron • Postsynaptic site: t 3. Assesment: Presynaptic action potential evokes a stronger EPSP – NMDAr, AMPAr channel activation – Somatic action potential, back-propagating spike – Voltage-gated calcium channel opening Pre neuron t Post neuron EPSP 9 10 Stuart and Sakmann, 1994 Neuronal response before learning Application of STDP in modelling: healthy brain • Neuronal response shift earlier in time (Gerstner and Kistler, 2002) • Formation of receptive fields in place cells (Gerstner and Kistler, 2002) • Development of direction selectivity in the visual system (Honda, Urakubo, Tanaka, Kuroda, 2011) • Temporal sequence learning (Izhikevich, 2007) • Memory encoding and retrieval in hippocampus (Cutsuridis et al., 2010) 11 Neuron receives trains of 5 spikes. x1 w1 Neuron fires after 4 synapses are activated. x2 w 2 x1 x2 t x3 t x4 t x5 t y t x3 w3 x4 w4 x5 w5 STDP ∆w Weight increase Weight decrease y + −τT+ A e , T ≥ 0 ∆w(T ) = T A− e τ − , T < 0 T = t post − t pre 2 Neuronal response after learning The first synapses generates a response x1 x2 t x3 t x4 t x5 t y t Neuronal response shift earlier in time Neuronal response before learning 31ms x2 w 2 and 58ms x3 w3 after stimulus presentation x4 w4 x5 w5 Neuronal response after learning Neuron is active y STDP ∆w Weight increase Weight decrease Neuron is active x1 w1 + −τT+ A e , T ≥ 0 ∆w(T ) = T A− e τ − , T < 0 T = t post − t pre Neuron is presented with 20 inputs 5ms after stimulus presentation Gerstner and Kistler, Spiking Neuron Models. Single Neurons, Populations, Plasticity Cambridge University Press, 2002 Desynchronization of neural populations Application of STDP models Parkinson’s disease • Parkinson’s disease is characterized by abnormally strong, pathological neuronal synchronization • Synaptic coupling is strong and has to be unlearned • Parkinson’s disease – a neurogenerative disease characterized by tremors, rigidity, slowness of movement, stiffness. Normal • Electrical high frequency deep brain stimulation of the subthalamic nucleus is the effective surgical treatment for Parkinson's disease. Pathological 15 http://www.ocduk.org/dbs-caution-compassion-advised Unlearning strong synaptic coupling via STDP Weak synaptic coupling Strong synaptic coupling 16 Popovych V., Tass,P. Desynchronizing electrical and sensory coordinated reset neuromodulation. Front Hum Neurosc 2012 Models of Spike-timing Dependent Plasticity • Network of Hodgkin–Huxley neurons and STDP learning rule • Stimulation causes an unlearning of pathological synchronization – Weak or Strong stimulation does not have a long-lasting effect – Intermediate strength stimulation suppresses synaptic coupling and leads to a long- lasting desynchronization Phenomenological – encode data and intuitions about synaptic plasticity. (Abbott and Song, 1999; Gerstner et al, 1996; Kempter et al, 1999; Kistler et al, 2000; Song et al. 2000; van Rossum, 2000); Optimal – use some optimality criterion to deduce the rules of synaptic modifications. Synaptic coupling (Bell and Parra, 2003; Dayan et al, 2004; Pfister et al, 2004; Toyoizumi et al, 2005); Popovych V., Tass, P. Desynchronizing electrical and sensory coordinated reset neuromodulation. Front Hum Neurosc 2012 Biophysical – biophysically realistic, usually based on calcium dynamics, involve detailed biochemical reactions to explain synaptic plasticity. (Bhalla and Iyengar, 1999; Karmarkar and Buonomano, 2001; Abarbanel et al, 2002; d’Alcantara et al, 2003; Shouval et al, 2002, 2005; Saudargiene et al, 2004; Rubin et al, 2005; Badoual et al, 2006; Pi and Lisman 2008; Graupner and Brunel 2007, 2012; Clopath et al 2010; Jin et al 2013); 17 18 3 Phenomenological models Optimal models • Black box approach • No explicit modelling of biophysical processes • Used in networks – memory formation, neuronal selectivity • Derive the optimal STDP function for a given task: – Maximal information transfer in network of spiking neurons – use information theory to obtain STDP curve (Bell and Parra, 2003; Toyoizumi et al, 2005) Pre T Post T ∆ω tpre + −τT+ A e , T > 0 ∆w(T ) = T A− eτ − , T < 0 T = t post − t pre – Optimal filter function - STDP shape represents the filter to remove the noise from the signal and increase the influence of its significant components; ∆ω tpost (Dayan et al, 2004) T (Abbott and Song, 1999; Gerstner et al, 1996; Pfister and Gerstner, 2006; Kempter et al, 1999; Kistler et al, 2000; Song et al. 2000; van Rossum, 2000)19 Biophysical models [Ca2+] (Pfister et al, 2004) 20 Calcium dynamics in spines • White box approach • Based on Ca2+ dynamics in spine of a postsynaptic neuron • Used to model synaptic plasticity at a single synapse, local mircocircuit Neuronal activity – Optimal firing time of the postsynaptic neuron – maximization of the probability of firing at the defined times leads to the STDP function NMDAr channels Ca2+ pump Ca2+ buffers ∆ω Voltage gated Ca2+ channels Ca2+ diffusion (Abarbanel et al, 2002; Badoual et al, 2006; Bhalla and Iyengar, 1999; Bush and Jin, 2012; Graupner and Brunel, 2007, 2012; d’Alcantara et al, 2003; Karmarkar and Buonomano, 2001; Rubin et al, 2005; Shouval et al, 2002, 2005; Saudargiene et al, 2004, 2005); 21 22 Malenka and Nicoll, 1999 Calcium concentration Calcium current through voltage-gated calcium channels: I Ca _ VGCC = g Ca _ L m p h q (V − ECa ) Calcium current through NMDA receptor-gated channels: I Ca _ NMDA = 0.1g NMDA (V − E NMDA ) Calcium concentration: 2+ d [Ca 2+ ] I [Ca0 ] − [Ca 2+ ] = − Ca + dt 2 Fv τ Ca 23 Decay due to pumping, buffering, diffusion http://synapses.clm.utexas.edu/atlas/1_4_1_17.stm Ca2+ transients STDP stimulation protocol Amplitude and time course of Ca2+ transient depend on the temporal order and temporal difference of pre- and postsynaptic activity. Post - Pre LTD Pre Post Pre – Post LTP T Pre T=-20 Post [Ca2+], µM T T=20 [Ca2+], µM LTP threshold θp LTD threshold θd t, ms t, ms 24 4 Biophysical models Calcium control hypothesis Shouval et al, 2002, 2005 • Weight change is determined by [Ca2+] levels: – low [Ca2+] levels – no changes – intermediate [Ca2+] levels between θd and θp – LTD – high [Ca2+] levels above θp – LTP Biophysical models Calcium control hypothesis Shouval et al, 2002, 2005 • Weight change is proportional to the threshold function Ω([Ca]): ∆w ≅ η Ω([Ca ]) • Plasticity outcomes: Ω([Ca]) – short positive T – LTP – short negative and large positive T – LTD Graupner and Brunel,252010 Biochemical mechanisms of Spike-timing dependent plasticity 26 Biophysical models LTP and LTD enzyme competition • Correlated activity of pre- and postsynaptic neurons • Glutamate release • Activation of NMDAr-gated channels • Increase in calcium concentration in a spine m+Ca2+ h + Glu Ca2+/CaM KinaseII Dephosporylation of AMPA receptors Phosporylation of AMPA receptors Long-term depression Long-term potentiation am bm K + 4Ca2+ m* ak bk K* ah * h bh m * + h* + Ph 27 LTP enzyme K LTD enzyme Ph Ca2+ Phosphatases Badoual et al, 2005 Ca2+ Ph* ∆w = − Ph* + K * m activated by Ca2+ h activated by Glu Ph activated by m* and h* Reflects activity of Ca2+/calmodulin-dependent protein kinase II, activated by Ca2+ 28 Kandel, Schwartz, Jessell, 2000 Ca2+/calmodulin (CaM)-dependent protein kinase CaMKII Binary synapse model • Synaptic changes are all-ornone switch-like events DOWN → UP UP → DOWN • CaMKII holoenzyme consists of 6 subunits. • Each subunit can be in: – active, phosphorylated state; – not active, dephosphorylated state. • Two stable states of synaptic transmission efficacy at resting calcium levels: – DOWN, low synaptic strength – UP, high synaptic strength • CaMKII can be stable in two states at resting calcium: – weakly phosphorylated state - DOWN – highly phosphorylated state - UP • Transitions: – UP-DOWN: LTD – DOWN-UP: LTP • CaMKII switch in a binary synapse: – Transition UP-DOWN: LTD – Transition DOWN-UP: LTP 29 Graupner and Brunel, 2010 30 Lisman, Schulman & Cline, 2002 5 Biochemical networks of synaptic plasticity: more details Bistable synapse model based on CaMKII activation network • Ca2+/calmodulin C activates CaMKII • PP1 inhibits, dephosphorylates CaMKII • Model of CaMKII regulation, mGlu receptor and NMDA receptor activation – PP1 influence is increased by calcineurin activation – PP1 influence is reduced by cyclic-AMP and PKA activation 14 differential equations! 31 32 Graupner and Brunel, 2008 Bhalla and Iyengar, 1999 STDP timing windows Multiple forms of STDP Hippocampus, excitatory synapse • STDP depends on: • Stimulation protocol – Pattern of post activity (single spike, burst) – Frequency of pairings – Duration of stimulation • Properties of neuron Hippocampus, inhibitory synapse Neocortex, excitatory synapse – Dendritic plasticity – Dendritic synapse location • Neuron type • Brain region 33 34 Buchannan and Mellor, 2010 STDP model based on kinase and phosphatase competition Abbott and Nelson, 2000 Diversity of STDP curves • Recent models are able to account for the effect of spike pattern, frequency, duration of stimulation. • Numbers of postsynaptic spikes and duration of the stimulation qualitatively change the STDP curve. • Synaptic efficacy ρ: two stable states UP and DOWN – transition UP-DOWN: LTD – transition DOWN-UP: LTP • Synaptic efficacy ρ τ LTD term LTP term dρ = − ρ (1 − ρ )( ρ* − ρ ) + γ p (1 − ρ )Θ[Ca (t ) − θ p ] − γ d ρΘ[Ca (t ) − θ d ] + Noise(t ) dt Absence of Kinase influence activity [Ca2+]>Θp Phosphotase influence [Ca2+]>Θd 35 Graupner and Brunel, 2012 36 Graupner and Brunel, 2012 6 NEURON demo Modeling learning in hippocampus • • • Models of LTP and LTD: review NEURON simulator STDP model: synaptic efficacy variables ρUP, ρDOWN Associative learning in a CA1 microcircuit ρUP ρDOWN • • 37 Neuromodulation of STDP 117 models analyzed Models of molecular mechanisms underlying synaptic plasticity are classified: – – – – Direction: LTP/LTD Complexity: single pathways/simplified processes/signaling networks Method: deterministic/stochastic method Cell type 38 Dopamine pathways • Dopamine is involved in: • STDP depends on: – reward-motivated behavior – pleasure – motor control – pre- and postsynaptic activity – a “third factor” – neuromodulators dopamine, acetylcholine, etc • Neuromodulators open the STDP gate: – necessary for plasticity induction or – change the shape of the STDP • Dopamine signal mimics reward prediction error (reinforcement learning theory) • Time and spatial scale: – Correlated pre-post ativity, STDP – local and fast, ms – Neuromodulation – global and slow, 3-5sec Markram H , Gerstner W and Sjöström PJ. A History of Spike-timing-dependent Plasticity Synaptic Neuroscience 2011 39 Pawlak V, Wickens JR, Kirkwood A and Kerr JND. Timing is not everything: neuromodulation opens the STDP gate. Frontiers in Synaptic Neuroscience 2010. Dopamine controls STDP in hippocampal neurons • Dopamine changes the shape of the STDP window. • LTP window is wider • LTD converts into LTP • 40 http://www.drugabuse.gov/publications/addiction-science/why-do-people-abuse-drugs/brain-pathways-are-affected-by-drugs-abuse http://www.drugabuse.gov/publications/addiction-science/why-do-people-abuse-drugs/brain-pathways-are-affected-by-drugs-abuse How do molecules affect behaviour? Dopamine reduces the number of spike pairs required to induce LTP from 60 to 10. Pawlak et al, 2014 http://www.cartoonstock.com/newscartoons/directory/b/behaviour.asp http://molinterv.aspetjournals.org/cgi/content/abstract/3/7/375 Izhikevich E. Solving the distal reward problem through linkage of STDP and dopamine signal. Cerebral 41 Cortex 2007 42 7 Application of STDP-like learning rule in robotics Learning in a navigating robot B.Porr, F.Wörgötter 2003 Differential Hebbian learning: dw = µxvision y ′ dt ∆ω T<0 vision x1 T>0 vision x1 collision x0 Collision sensors collision x0 Vision sensors T T T Robot learns to navigate without collisions RunBot - a biped walking robot – learns to walk on a terrain Porr, B. and Wörgötter, F. (2003) Isotropic sequence order learning. Neural Comp., 15, 831-864. STDP-like weight change 43 44 Manoonpong, P., Porr, B. and Woergoetter, F. (2007) Adaptive, Fast Walking in a Biped Robot under Neuronal Comtrol and learning. Comp Biology, 3(7),1305-1311. Final remarks • • • Final remarks Phenomenological models apply the STDP function and analyze the computational principles and its functional consequences in networks. • Neuromudulation gates STDP and provides link from correlation-based synaptic plasticity rules to behaviorally based learning. Optimal models derive optimal STDP function for a given task – optimal firing time, maximal information transfer between neurons, optimal filter function. • Link between the cellular processes and behaviour is hard to establish. Robotics offer one of the possibilities to analyze the functional consequences of the learning rules. Biophysical models explain STDP using the principles of ion channel dynamics and intracellular processes. Usually contains a large parameter space. • Models contribute to understanding of brain functioning in health and disease. 45 46 Final remarks • Acknowledgement Modeling synaptic plasticity: – from molecules (nm) to a system level (m); – from fast electrical events (ms) to slow chemical processes (min, hours); – from unsupervised learning to reinforcement learning (reward-based). 47 • • • • • Marja-Leena Linne Bruce Graham Arnd Roth Florentin Worgotter Bernd Porr 48 https://www.humanbrainproject.eu/documents/10180/17648/TheHBPReport_LR.pdf/18e5747e-10af-4bec-9806-d03aead57655 8