Download No Slide Title

Plasticity and learning
Dayan and Abbot
Chapter 8
Introduction
• Learning occurs through synaptic plasticity
• Hebb (1949): If neuron A often contributes to the firing of
neuron B, then the synapse from A to B should be
strengthened
– Stimulus response (Pavlov)
– Converse: If neuron A does not contribute to the firing of B, the
synapse is weakened
– Hippocampus, neocortex, cerebellum
LTP and LTD at Shaffer collateral inputs to CA1
region of rat hippocampal slice. High stimulation yields
LTP. Low stimulation yields LTD.
NB: no stimulation yields no LTD
Function of learning
• Unsupervised learning (ch 10)
– Feature selection, receptive fields, density estimation
• Supervised learning (ch 7)
– Input-output mapping, feedback as teacher signal
• Reinforcement learning (ch 9)
– Feedback in terms of reward, similar to control theory
• Hebbian learning (ch 8)
– Biologically plausible + normalization
– Covariance rule for (un)supervised learning
– Occular dominance, maps
Rate model with fast time scale
• Neural activity as continuous rate, not spike train
V is output neuron, u is vector input neurons, w is
vector of weights
If tau_r small wrt learning time:
Basic Hebb rule
• V and u are functions of time. Makes dynamics hard to
solve. Alternative is to assume v,u from distribution p(v,u)
and assume p time independent.
Using v=w. u we get
Basis Hebb rule
• Hebb rule is unstable, because norm always increases
• Continuous differential equation can be simulated using
Euler scheme
Covariance rule
• Basic Hebb rule describes only LTP since u, v positive
• LTD occurs when pre-synaptic activity co-occurs with low
post synaptic activity
• Alternatively,
Covariance rule
• When
Either rule produces
Covariance rule is unstable
BCM rule
• Bienenstock, Munro, Cooper (1982):
Requires both pre and post synaptic activity for learning
For fixed threshold, the BMC rule is also unstable
BMC rule
• BMC rule can be made stable by threshold dynamics
• Tau_theta is smaller than tau_w
• BMC rule implements competition between synapses
– Strenghtening one synapse, increases the threshold, makes
strengthening of other synapses more difficult
• Such competition can also be implemented by
normalization
Synaptic normalization
• Limit the sum of weights or sum of squared weights
• Impose this constraint rigidly, or dynamically
• Two examples:
– Rigid scheme for sum of weights constraint (subtractive norm.)
– Dynamic scheme for sum of squared weights (multipl. Norm.)
Subtractive normalization
• Subtractive normalization ensures that sum w does not
change
Not clear how to implement this rule biophysically (nonlocality).
We must add a constraint that weights are non-negative.
Multiplicative normalization
• Oja rule (1982)
• The rule implements the constraint dynamically:
Unsupervised learning
• Adapting the network for a set of tasks
– Neural selectivity, receptive field
– Cortical map
• Process depends partly on neural activity and partly not
(axon growth)
• Ocular dominance
– Adult neurons favor one eye over the other (layer 4 input from
LGN)
– Neurons are clustered in bands or stripes
Single post-synaptic neuron
• We analyze Eq. 8.5
Single post-synaptic neuron
• Solution in terms of eigenvalues of Q
• Eigen values are positive, so solution explodes.
• Asymptotically
– e1 is the principle eigen direction
– Neuron projects input onto this direction:
Single post-synaptic neuron
• Example with two weights.
• Weights grow indefinite, one positive one negative. Choice
depends on initial conditions.
• Limit to [0, 1] yields different solutions depending in init
value
Single post-synaptic neuron
• Subtractive normalization. Averaging over inputs:
• Analysis in terms of eigenvectors:
– In ocular dominance e1=n/sqrt(n). W in direction of e1 has rhs
equal to zero. Ie this component of w is unaltered
– In other directions normalizing term is zero
– W asymptotically dominated by second eigenvector
Hebbian development of ocular
dominance
Subtractive normalization may solve this, since e1=n weight
grows proportional to e2=(1,-1)
Single post-synaptic neuron
• Using the Oja rule
• Show that each eigenvector of Q is solution.
• One can show that only principal eigenvector is stable.
Single post-synaptic neuron
• A: Behavior of
–
–
Unnormalized Hebbian learning
Multiplicative normalization (Oja rule)
gives w propto e1. This is similar to PCA.
• B: Shifting mean of u may yield different solution
• C: Covariance based learning corrects for mean
• Saturation constraints may alter this conclusion
Hebbian development of ocular
dominance
• Model layer 4 cell with input from two LGN cells, each
associated with different eye.
Hebbian development of
orientation selectivity
Spectral analysis also applicable to non-linear systems
Dominant eigenvector uniform.
Non-uniform receptive fields result from sub-dominant
eigenvector
Cortical receptive fields from LGN. ON-center (white) and
OFF-center (black) cells excite cortical neuron.
Multiple postsynaptic neurons
Hebbian development of ocular
dominance stripes
• A: model with right and left eye inputs drive array of
cortical neurons
• B: ocular dominance maps. Top: light and dark areas in top
and bottom cortical layer show ocular dominance in cat
primary cortex. Bottom: model of 512 neurons with
Hebbian learning
Hebbian development of ocular
dominance stripes
• Use 8.31 with W=(w+,w-) the n*2 matrix. See book.
• Subtractive normalization dw+/dt=0
• Ocular dominance pattern given by largest eig. vector of K
Hebbian development of ocular
dominance stripes
• Suppose K translation invariant
– Periodic boundary conditions simulating a patch of cortex ignoring
boundary effects
– Eigenvectors are
– Eigenvalues are Fourrier components
– Solution of learning is spatially periodic (viz. fig 8.7)
Feature based models
• Multi dimensional input (retinal location, ocular
dominance, orientation preference, ....)
• Replace input neurons by input features, W_ab is
selectivity of neuron a to feature b
– Feature u1 is location on retina in coordinates
– Feature u2 is ocularity (how much is the stimulus prefering left
over right eye), a single number
• The coupling
to neuron a describe the preferred
stimulus
• Activity of output a is
Feature based models
• Output is soft-max
• Combined with lateral averaging
– Self-organizing map (SOM)
– Elastic net
Feature based models
optical imaging shows ocularity and orientation selectivity in
macaque primary visual cortex. Dark lines are ocular
dominance boundaries, light lines are iso-orientation
contours. Pin wheel singularities, linear zones
Feature based models
• Elastic net output
• SOM, competitive Hebbian rules can produce similar output
Anti-hebbian modification
• Another way to make different outputs specialize is by
adaptive anti-Hebbian modification
• Consider Oja rule:
• Each output a will be identical
• Anti-Hebbian modification is shown at synapses from
parallel fibers to Purkinje cells in cerebellum.
• Combination yields different eigenvectors as outputs
Timing based rules
• Left: in vitro cortical slice. Right: in vivo xenopus tadpoles
• LTP when pre-synaptic spike precedes post-synaptic spike
• LTD when pre-synaptic spike follows post-synaptic spike
Timing based rules
• Simulating spike-time plasticity requires spiking neurons
• Approximate description with firing rates
• H(t) positive/negative for t positive/negative
Timing based plasticity and
prediction
• Consider array of neurons labeled by a with receptive
fields f_a(s) (dashed and solid curves)
• Timing based learning
rule. Stimulus s moves
from left to right.
Timing based plasticity and
prediction
• If a left of b, then link a to b is strengthened and link b to a
is weakened. Receptive field of neuron a is asymmetrically
deformed (A solid bold line)
• Prediction: next presentation of s(t) will activate a earlier.
• In agreement with shift of place field mean when rats run
around track (B).
Supervised Hebbian learning
• Weight decay:
• Asymptotic solution is
Classification and the Perceptron
• If output values are +/- 1 the model implements a
classifier, called the Perceptron:
– The weight vector defines a separating hyper plane:
=
gamma.
– The perceptron can solve problems that are ‘linearly separable’