Download Δt (ms)

Vidna kognicija III
Danko Nikolić
Teme
• Neurofiziološki kodovi prijenosa i obrade
informacija u vidnom sustavu
• Dva kôda za percepciju svjetline
• Problem povezivanja dijelova vidne scene u
cjelinu (tzv. binding problem)
• Uloga pažnje u pohranjivanju informacija u radno
pamćenje
• Uloga radnog pamćenja za formiranje dugoročnog
vidnog pamćenja
• Mehanizmi sinestezijskih asocijacija
New lecture
Small delays in
synchronization
→ 150 ms
→ 250 ms
The speed of vision
20 ms per stage!
40-50 ms
30-50 ms
1 spike per neuron!
20-40 ms
50-70 ms
70-90 ms
80-100 ms
Thorpe & Fabre-Thorpe (2001)
What can one spike tell us?
What can one spike tell us?
Increase in stimulus intensity
Kuffler (1953) Discharge patterns and
functional organisation of vertebrate retina,
Journal of Neurophysiology
Stimulus onset
“I don’t think Steve much liked our
abstract”.
A simulation test
Van Rulen and Thorpe (2001) Rate Coding Versus Temporal Order Coding:What the
Retinal Ganglion Cells Tell the Visual Cortex, Neural Computation, 13
Spike timing in sensory receptors
Onset latencies in somatosensation
D1
D2
D3
D4
D5
Stimulus onset
Johansson & Birznieks (2004)
Measuring small delays
• Fitting a function and taking its
maximum value for the estimate.
Cosine fit
Phase offsets can be measured with submillisecond precision
Schneider and Nikolić, Journal of Neuroscience Methods (2006).
Large networks
200 μm
200 μm
Relative firing time [ms]
Extraction of the firing sequence
Relative firing time [ms]
Nikolić, Journal of Comp. Neuroscience (2007)
Schneider, Havenith and Nikolić, Neural Computation (2006)
Schneider and Nikolić, Journal of Neuroscience Methods (2006)
Non-parametric detection of temporal order
Nikolić, Journal of Comp. Neuroscience (2007).
Example:
Stimulus
dependence
1)
2)
A
B
20 ms
Neuron ID
Neuron ID
20 ms
Time [ms]
Time [ms]
Firing sequences change dynamically
Spike timing in single neurons:
Synaptic integration
Rall (1964)
Spike timing in single neurons:
Synaptic integration
Rall (1964)
Spike timing in single neurons:
Synaptic integration
Rall (1964)
Spike timing in single neurons:
Synaptic integration
Rall (1964)
Spike timing in single neurons:
Synaptic integration
Rall (1964)
Spike timing in single neurons:
Synaptic integration
Rall (1964)
Spike timing in single neurons:
Synaptic integration
Rall (1964)
Spike timing in single neurons:
Synaptic integration
Euler & Denk (2004)
Stiefel & Sejnowski (2007)
Conclusion: Firing
sequences
• Short time delays can serves as a code for
carrying stimulus-related information that
is as reliable as is the neuronal firing rate.
• Stronger synchronization increases the
reliability of the code.
Good timing is
everything
Binding problem
Perceptual integration and organization
Hierarchical coding by extraction of feature
combinations
Grandmother cell
Problems: combinatorial explosion and novel
combinations
Combinatorial explosion: There are many different
grandmothers and each can be seen from many different
perspectives.
Novel combinations: Some grandmothers are seen for the
first time. There is no chance to learn all the possible
combination of features that make a grandmother.
Perceptual organization through synchronization of
action potentials
Split bar experiment
(Gray et. al, 1989)
Perceptual organization through synchronization of
action potentials
Synchrony at
different scales
Conflicting bar experiment
Engel, A.K., Koenig, P.& Singer, W. (1991); Kreiter, A.K. & Singer, W. (1996).
Mechanisms: Tangential connections
Higher brain areas and
awake states
An important role of attention
- In early visual areas
- In higher visual areas V4, MT.
Infero-temporal cortex and
recognition of faces
Mechanisms of synchronization
Mechanisms of detection
Attention and rates in V4
• Modulation of rate firing rate responses in V4
• Moran & Desimone, 1985
Attention and synchrony in V4
- Fries et al., 2001
- Investigated strength
of synchrony
- Spike-triggered
averages
Delay period
Stimulus period
Infero-temporal cortex
- Hirabayashi & Miyashita
- Perception of faces
Face
Non-face
- Synchronization is stronger when faces are perceived.
Attention in early visual areas
• Roelfsema et al., 2004
• Non-modulation of
synchrony in V1
Mechanisms
•
•
•
•
For large part unknown
To a high degree theoretical answers
Models, simulations
Three types of mechanism are considered:
– bottom-up
– lateral interactions
– top-down
Bottom-up
Common input
Input is not shared
Lateral interactions
Tangential connections
Top-down
Lower visual area
Higher visual area
Feedback connections
The brain as a liquidstate machine
A thinking ocean is the main character in
a famous science fiction novel by Stanislav Lem
But how can a liquid possibly
process information ?
Common mistake:
Trying to understand the brain from the perspective of the
organization of a digital computer
IBM Blue-Gene supercomputer
Amsterdam-07
•
Two obvious differences in the
organization
of computers and brains
A computer
requires a program
(it implements a Turing machine).
A brain does not have a program:
Instead it has to rely on learning.
It may be viewed as a genetically
encoded network of learning-agents.
Amsterdam-07
A literal interpretation of liquid computing
Fernando and Sojakka: Pattern recognition in a bucket: A real liquid brain,
ECAL 2003:
“This paper demonstrates that the waves produced on the surface of water can be used
as a medium for a “Liquid State Machine”. We made a bucket of water, vibrated it with
lego motors, filmed the waves with a webcam and put it through a perceptron on matlab
and got it to solve the XOR problem and do speech recognition.”
„zero“
„one“
Amsterdam-07
Why did we call this style of computing liquid computing ?
A cortical circuit is viewed here as a special case of a
Liquid State Machine
input
1
15
-5
23
31
102
0
liquid
•
•
•
•
•
•
•
•
1+15
1/-5*23
Log(15*31)
Sin(102)+Log(…
…
…
…
do it many, many
times.
readout
Synaptic weights
input
liquid
readout
Synaptic weights
primary visual cortex
•
•
•
•
Parallel recordings by
multiple Michigan probes.
Up to 48 channels.
Cat visual cortex, area 17
Anesthesia
main result
persistence of information
temporal superposition
XOR:
code invariance
correlations matter
precise spike timing
SD - jitter [ms]
a subject to noise
conclusions
- memory: information about previously shown images is available for a
prolonged period of time.
- superposition: information about previously and currently shown
stimuli is available simultaneously.
- non-linearity: information about non-linear transformations of input
properties can be extracted by linear classifiers.
- rates and timing: information is coded partially in neuronal firing
rates and partially in the precise timing of neuronal spiking activity.
- 2nd order correlation: the advantage of using additional non-linear
classification methods was limited to the use of pair-wise correlations
between neurons.
input
t
non-linear
map
readout
Neuronal Avalanches
in Vivo
European avalanche-size table
1 - Slough
Small snow slide that cannot bury a person.
length <50 m
volume <100 m³
2 - Small
Stops within the slope.
length <100 m
volume <1,000 m³
3 - Medium
Runs to the bottom of the slope.
length <1,000 m
volume <10,000 m³
4 – Large
Runs over flat areas, may reach the valley bottom.
length >1,000 m
volume >10,000 m³
P(size)
P(size)
Distribution of Avalanche-Sizes
Size of avalanches
P (size) ~
sizeα
α
Size of avalanches
log P (n) ~ α log (n) = y ~ k
x
Power law in complex systems:
• Earthquakes, forest fires, evolution of species
• Size of US cities, citations of papers
Does the brain generate neuronal avalanches with
power law statistics ?
→How do we imagine such “neuronal” avalanches?
→How do we observe them in the brain?
Neuronal Avalanche of Spikes
Size = 24 spikes
Lifetime = 14 ms
Methods
• Recording:
→ Michigan Probes
• Recording sites:
→ 3 cats: Area 17
→ 1 cat: Area 17 & Area 21
Unit
• Spontaneous activity under anesthesia
• 7 datasets → Duration: 100 – 500 s
• Units → Per dataset: 105 - 158
→ Single units: 68 - 118
→ Multi units: 26 - 50
ms
Definition of Avalanches
avalanche i + 1
(size = 3 spikes)
Unit
avalanche i
(size = 14 spikes)
Δt (ms)
active bin
blank bin
6 ms
Power Law
P (spikes)
1
0.1
0.01
0.001
0.0001
1
10
#spikes
Δt:
4 ms
Cat: Col05 – Probe1
100
Power law is independent of Δt
• Δt was varied between 1 and 10 ms
→Number of extracted avalanches decreases with larger Δt
→Number of larger avalanches increases relative to smaller ones
• Power law remains stable irrespective of Δt !
• Exponent of power law increases with Δt
Non - Power Law
Dependence on Δt → 2 cases
• Exponential-like distribution remains robust
• With small Δt → power law
/ with large Δt → exponential function
1
P (spikes)
0.1
0.01
Cat: Col11 – Probe 1
0.001
Δt = 4 ms
0.0001
1
10
# spikes
100
Exponent
-1.5
Exponent
-1.7
-1.9
-2.1
-2.3
Cat: Col05 – Probe1
-2.5
1
2
3
4
5
6
7
Δt (ms)
→ Exponent increases with larger bin-sizes
8
Exponent – Δtavg
Δtavg = mean of time intervals between spikes
→ statistical approach for optimal Δt
→ Separation and concatenation of avalanches is minimized
2.5
Δtavg
2
1.5
1
0.5
0
0
20
40
60
Δt
80
100
Exponent (Δtavg) ~ -1.8
1
P(spikes)
0.1
0.01
0.001
Col05 I (-1.84)
Col05 II (-1.87)
0.0001
Nat03 17 (-1.84)
0.00001
1
10
# Spikes
100
Lifetime
Lifetime distribution of avalanches does not follow a power law
in any of the probes, irrespective of Δt.
Cat: Col05 – Probe 1
P (lifetime)
1
0.1
1 ms
0.01
2 ms
4 ms
8 ms
0.001
1
10
Lifetime [ms]
100
Conclusions
• Neuronal avalanches defined by spikes
→ Power law in some size-distributions
→ Exponent ~ -1.8
→ No Power law in other size - distributions
→ No Power law in lifetime distributions
• Interpretation
→ Self-organized criticality (SOC)
→ Critical branching processes