Download Slide 1

Scaling-up Cortical Representations
in Fluctuation-Driven Systems
David W. McLaughlin
Courant Institute & Center for Neural Science
New York University
http://www.cims.nyu.edu/faculty/dmac/
Cold Spring Harbor -- July ‘04
In collaboration with:
  David Cai
Louis Tao
Michael Shelley
Aaditya Rangan
Lateral Connections and Orientation -- Tree Shrew
Bosking, Zhang, Schofield & Fitzpatrick
J. Neuroscience, 1997
Coarse-Grained Asymptotic
Representations
Needed for “Scale-up”
Cortical networks have a
very noisy dynamics
• Strong temporal fluctuations
• On synaptic timescale
• Fluctuation driven spiking
Experiment Observation
Fluctuations in Orientation Tuning (Cat data from Ferster’s Lab)
Ref:
Anderson, Lampl, Gillespie, Ferster
Science, 1968-72 (2000)
threshold (-65 mV)
Fluctuation-driven
spiking
(very noisy dynamics,
on the synaptic time scale)
Solid:
average
( over 72 cycles)
Dashed: 10 temporal
trajectories
•
To accurately and efficiently describe these
networks requires that fluctuations be retained in a
coarse-grained representation.
•
“Pdf ” representations –
(v,g; x,t),  = E,I
•
•
will retain fluctuations.
But will not be very efficient numerically
Needed – a reduction of the pdf representations
which retains
1. Means &
2. Variances
•
PT #1: Kinetic Theory provides this representation
Ref: Cai, Tao, Shelley & McLaughlin, PNAS, pp 7757-7762 (2004)
First, tile the cortical layer with coarse-grained (CG) patches
Kinetic Theory begins from
PDF representations
(v,g; x,t),  = E,I
• Knight & Sirovich;
• Tranchina, Nykamp & Haskell;
• First, replace the 200 neurons in this CG
cell by an effective pdf representation
• Then derive from the pdf rep, kinetic thry
• For convenience of presentation, I’ll sketch
the derivation a single CG cell, with 200
excitatory Integrate & Fire neurons
• The results extend to interacting CG cells
which include inhibition – as well as
“simple” & “complex” cells.
• N excitatory neurons (within one CG cell)
• Random coupling throughout the CG cell;
• AMPA synapses (with time scale )
 t vi = -(v – VR) – gi (v-VE)
 t gi = - gi + l f (t – tl) +
(Sa/N) l,k (t – tlk)
• N excitatory neurons (within one CG cell)
• All-to-all coupling;
• AMPA synapses (with time scale )
 t vi = -(v – VR) – gi (v-VE)
 t gi = - gi + l f (t – tl) +
(Sa/N) l,k (t – tlk)
(g,v,t)  N-1 i=1,N E{[v – vi(t)] [g – gi(t)]},
Expectation “E” over Poisson spike train
 t vi = -(v – VR) – gi (v-VE)
 t gi = - gi + l f (t – tl) + (Sa/N) l,k (t – tlk)
Evolution of pdf -- (g,v,t): (i) N>1; (ii) the total input to
each neuron is (modulated) Poisson spike trains.
t  = -1v {[(v – VR) + g (v-VE)] } + g {(g/) }
+ 0(t) [(v, g-f/, t) - (v,g,t)]
+ N m(t) [(v, g-Sa/N, t) - (v,g,t)],
0(t) = modulated rate of Poisson spike train from LGN;
m(t) = average firing rate of the neurons in the CG cell
=  J(v)(v,g; )|(v= 1) dg,
and where J(v)(v,g; ) = -{[(v – VR) + g (v-VE)] }
Kinetic Theory Begins from Moments
•
•
•
•
(g,v,t)
(g)(g,t) =  (g,v,t) dv
(v)(v,t) =  (g,v,t) dg
1(v)(v,t) =  g (g,tv) dg
where (g,v,t) = (g,tv) (v)(v,t).
t  = -1v {[(v – VR) + g (v-VE)] } + g {(g/) }
+ 0(t) [(v, g-f/, t) - (v,g,t)]
+ N m(t) [(v, g-Sa/N, t) - (v,g,t)],
Under the conditions,
N>1; f < 1; 0 f = O(1),
(i) v2(v) = 0;
(ii) 2(v) = g2
2(v) = 2(v) – (1(v))2 ,
And the Closure:
where
g2 = 0(t) f2 /(2) + m(t) (Sa)2 /(2N)
G(t) = 0(t) f + m(t) Sa
One obtains:
t (v) = -1v [(v – VR) (v) + 1(v)(v-VE) (v)]
t 1(v) = - -1[1(v) – G(t)]
+ -1{[(v – VR) + 1(v)(v-VE)] v 1(v)}
+ g2 / ((v)) v [(v-VE) (v)]
Together with a diffusion eq for (g)(g,t):
 t (g) = g {[g – G(t)]) (g)} + g2 gg (g)
Fluctuations in g are Gaussian
 t (g) = g {[g – G(t)]) (g)} + g2 gg (g)
Fluctuation-Driven Dynamics
PDF of v
Theory→
←I&F (solid)
firing rate (Hz)
N=75
N=75
σ=5msec
S=0.05
f=0.01
Fokker-Planck→
Theory→
←I&F
←Mean-driven limit (
Hard thresholding N  
):
Bistability and Hysteresis
 Network of Simple, Excitatory only
N=16!
N=16
FluctuationDriven:
Relatively Strong
Cortical Coupling:
MeanDriven:
N 
Bistability and Hysteresis
 Network of Simple, Excitatory only
N=16!
MeanDriven:
Relatively Strong
Cortical Coupling:
Computational Efficiency
• For statistical accuracy in these CG patch settings,
Kinetic Theory is 103 -- 105 more efficient than I&F;
Average firing rates
Vs
Spike-time statistics
Bursting Model:
With NMDA at all times
Potential (mV)
20
0
19 Spikes
-20
-40
-60
0
50
100
150
Time (ms)
200
250
300
No NMDA when V D >= -50
20
Potential (mV)
16 Spikes
0
-20
-40
-60
0
50
100
150
Time (ms)
200
250
300
• Coarse-grained theories involve local
averaging in both space and time.
• Hence, coarse-grained theories average
out detailed spike timing information.
• Ok for “rate codes”, but if spike-timing
statistics is to be studied, must modify the
coarse-grained approach
PT #2: Embedded point neurons will
capture these statistical firing properties
[Ref: Cai, Tao & McLaughlin, PNAS (to appear)]
• For “scale-up” – computer efficiency
• Yet maintaining statistical firing properties of multiple neurons
• Model especially relevant for biologically distinguished sparse,
strong sub-networks – perhaps such as long-range connections
• Point neurons -- embedded in, and fully interacting with, coarsegrained kinetic theory,
• Or, when kinetic theory accurate by itself, embedded as “test
neurons”
dVi  D

  Vi  D   r  Gi  ED  t  Vi  D   E  Gi  ID  t  Vi  D   I ,
dt




dGi  ED
S E D
 ED
D
i
E
 Gi  f E   t  t 
dt
NE D


 S E DB

  p   t  t '  
j


  p   t  t  
j
j
jPE D
j
jPE B
dGi  ID
S I D
 ID
D
i
I
 Gi  f I   t  T  D
dt
NI


 S I DB

  p   t  T '  
k
kPI
  p   t  T  
k
kPI
k
D
k
B
 1, simple
  E , I ;  simple, complex,  
0, complex
Poissonspiketrainst ' j  '  , T ' j  ' arereconstructed  fromtheratemE ' B N E ' B , mI ' B N I ' B .
 B

  v   U  v,  E B ,  I B  B  v  
t
v


  EB 2   v   E


1
B
B
B
B
B
  E  v   U  v,   E ,   I
 E  v  
  v   g  EB  t   B
t
v
 E E
  v  v  




  IB 2   v   I


1
B
B
B
B
B
  I  v   U  v,   E ,   I
 I  v  
  v   g  IB  t   B
t
v
 I I
  v  v  


g  EB  t   f E  0 E
B
B
 t   S E

B
mE
B
t  
g  IB  t   f I B 0 I B  t   S I B mI B  t  
S E BD
NE
S I BD
NI

D

t
e
D
t
e


t s
E
t s
I

 B
   v  



 B

v



 


  p  t  t   ds,
j
jPE D 
j
  p  t  t   ds
k
k
kPI D



2
2
B
BD
t s


S
S
t 
2

E
1  B
E
E
2
B
B
  EB  t  
fE  0E t  
mE  t  
e
p j   t  t j   ds  ,


D 

2 E 
pN E
pN E
jPE D 


2
2
B
BD
t s


S
S
t 
2

I
1  B
I
I
2
B
B
  IB  t  
fI  0I t  
mI  t  
e
pk   t  tk   ds 


D 

2 I 
pN I
pN I
kPI D 



 





dVi  D

  Vi  D   r  Gi  ED  t  Vi  D   E  Gi  ID  t  Vi  D   I ,
dt




dGi  ED
S E D
 ED
D
i
E
 Gi  f E   t  t 
dt
NE D


 S E DB

  p   t  t '  
j


  p   t  t  
j
j
jPE D
j
jPE B
dGi  ID
S I D
 ID
D
i
I
 Gi  f I   t  T  D
dt
NI


 S I DB

  p   t  T '  
k
kPI
  p   t  T  
k
kPI
k
D
k
B
 1, simple
  E , I ;  simple, complex,  
0, complex
Poissonspiketrainst ' j  '  , T ' j  ' arereconstructed  fromtheratemE ' B N E ' B , mI ' B N I ' B .
I&F vs. Embedded Network Spike Rasters
a) I&F Network: 50 “Simple” cells, 50 “Complex” cells. “Simple” cells driven at 10 Hz
b)-d) Embedded I&F Networks: b) 25 “Complex” cells replaced by single kinetic equation;
c) 25 “Simple” cells replaced by single kinetic equation; d) 25 “Simple” and 25 “Complex”
cells replaced by kinetic equations. In all panels, cells 1-50 are “Simple” and cells 51-100
are “Complex”. Rasters shown for 5 stimulus periods.
Embedded Network
Full I & F Network
Raster Plots, Cross-correlation and ISI distributions.
(Upper panels) KT of a neuronal patch with strongly coupled embedded neurons;
(Lower panels) Full I&F Network.
Shown is the sub-network, with neurons 1-6 excitatory; neurons 7-8 inhibitory;
EPSP time constant 3 ms; IPSP time constant 10 ms.
“Test neuron” within a CG Kinetic Theory
ISI distributions for two simulations: (Left) Test Neuron driven by a CG neuronal patch;
(Right) Sample Neuron in the I&F Network.
The Importance of Fluctuations
Cycle-averaged Firing Rate Curves [Shown: Exc Cmplx Pop in a 4
population model): Full I&F network (solid) , Full I&F + KT (dotted);
Full I&F coupled to Full KT but with mean only coupling (dashed).]
In both embedded cases (where the I&F units are coupled to KT),
half the simple cells are represented by Kinetic Theory
Reverse Time Correlations
• Correlates spikes against driving signal
• Triggered by spiking neuron
• Frequently used experimental technique to
get a handle on one description of the system
• P(,) – probability of a grating of orientation
, at a time  before a spike
-- or an estimate of the system’s linear
response kernel as a function of (,)
Reverse Correlation
Left: I&F Network of 128 “Simple” and 128
“Complex” cells at pinwheel center. RTC
P() for single Simple cell.
Below: Embedded Network of 128
“Simple” cells, with 128 “Complex” cells
replaced by single kinetic equation. RTC
P() for single Simple cell.
Computational Efficiency
• For statistical accuracy in these CG patch settings,
Kinetic Theory is 103 -- 105 more efficient than I&F;
• The efficiency of the embedded sub-network scales as
N2, where N = # of embedded point neurons;
(i.e. 100  20 yields 10,000 400)
Conclusions
• Kinetic Theory is a numerically efficient, and remarkably accurate,
method for “scale-up” – Ref: PNAS, pp 7757-7762 (2004)
• Kinetic Theory introduces no new free parameters into the model,
and has a large dynamic range from the rapid firing “mean-driven”
regime to a fluctuation driven regime.
• Kinetic Theory does not capture detailed “spike-timing” statistics
• Sub-networks of point neurons can be embedded within kinetic
theory to capture spike timing statistics, with a range from test
neurons to fully interacting sub-networks.
Ref: PNAS, to appear (2004)
Conclusions and Directions
• Constructing ideal network models to discern and extract possible
principles of neuronal computation and functions
Mathematical methods for analytical understanding
Search for signatures of identified mechanisms
• Mean-driven vs. fluctuation-driven kinetic theories
New closure, Fluctuation and correlation effects
Excellent agreement with the full numerical simulations
• Large-scale numerical simulations of structured networks
constrained by anatomy and other physiological observations to
compare with experiments
Structural understanding vs. data modeling
New numerical methods for scale-up --- Kinetic theory
Three Dynamic Regimes of Cortical Amplification:
1) Weak Cortical Amplification
No Bistability/Hysteresis
2) Near Critical Cortical Amplification
3) Strong Cortical Amplification
Bistability/Hysteresis
(2)
(1)
(3)
I&F
Excitatory Cells Shown
 Possible Mechanism
for Orientation Tuning of Complex Cells
Regime 2 for far-field/well-tuned Complex Cells
Regime 1 for near-pinwheel/less-tuned
Summed Effects
(2)
(1)
Summary & Conclusion
Summary Points for Coarse-Grained
Reductions needed for Scale-up
1. Neuronal networks are very noisy, with
fluctuation driven effects.
2. Temporal scale-separation emerges from network
activity.
3. Local temporal asynchony needed for the
asymptotic reduction, and it results from synaptic
failure.
4. Cortical maps -- both spatially regular and
spatially random -- tile the cortex; asymptotic
reductions must handle both.
5. Embedded neuron representations may be
needed to capture spike-timing codes and
coincidence detection.
6. PDF representations may be needed to capture
synchronized fluctuations.
Scale-up & Dynamical Issues
for Cortical Modeling of V1
• Temporal emergence of visual perception
• Role of spatial & temporal feedback -- within and
between cortical layers and regions
• Synchrony & asynchrony
• Presence (or absence) and role of oscillations
• Spike-timing vs firing rate codes
• Very noisy, fluctuation driven system
• Emergence of an activity dependent, separation of
time scales
• But often no (or little) temporal scale separation
 1
 N
    v, g E , g I , t   E 

i   v  Vi  t    g E  Gi  E t   g I  Gi  I t 

Under ASSUMPTIONS: 1) N 1  1
2) Summed intra-cortical low rate spike events become Poisson:
   g E
 

   v   r 
 v E 
 v   I 
   

g

g



E 
I 


    g     g
t
v    







E  E
I





f
  0 E  t     v, g E  E , g I , t     v, g E , g I , t  
E







f 
 0 I  t     v, g E , g I  I , t     v, g E , g I , t  
I 







S E
  pmE '  t  N E '    v, g E 
, g I , t     v, g E , g I , t  
pN E ' E
'








S I
  pmI '  t  N I '    v, g E , g I 
, t     v, g E , g I , t  
pN I ' I 
'



S '  pS '
 gI



 
 I

 

   v   r 
 v E 
 v   I 
   

g

g

 E
 I        g
t
v    






E
 
  1 

2

g

g
t



t





   E g   
  E
 E
g E  E 
E
 
 
  1 

2

g

g
t



t







  I
 I

 I
  
g I  I 
g I
 
 gE
 


  

 E
 g I
 gI



 
 I

 1 




2
g

g
t

g


t

g










  E
 E

E
 E

E 


g

E
E






 

1 

2
  g I  
g

g
t

g


t

g
  I
 I      I 
 I  
  I   
t
g I 
g I

 I 



  g E  
t
g E
g  E  t    f E 0 E  t   S E  mE '  t ,
g  I  t   f I 0 I  t   S I  mI '  t 
'
  E 2  t  
'
mE '  t  
1 
2
2
  f E  0 E  t   S E 
,
2 E 
 ' pN E ' 
  I 2  t  
mI '  t  
1  2
2
 f I  0 I  t   S I 

2 I 
 ' pN I  ' 

m  t    J  (v, g E , g I , t ) |v VT dg E dg I
0
 v   r
J   v, g E , g I , t    
 
 v E

  gE 

 

 v   I 
  g I       v, g E , g I , t 



Closures:

 E 2  v   0,
v

  I 2  v   0
v
and
 E 2  v     E 2 ,
  I  2  v     I 2
 EI  2  v    E 1  v   I 1  v 
 E
 n

 v   0

dg E  dg I g E   g E , g I | v  ,  I
n
0


0
0
 n

 v   0

dg E  dg I g I n   g E , g I | v 
0
 EI  2  v    dg E  dg I g E g I   g E , g I | v 
 E 2  v    E  2  v     E 1  v  ,
2
 I 2  v    I  2  v     I 1  v 
2


  v   U   v,  E ,  I    v  
t
v
  E 2   v   E


1
 E  v   U   v,  E ,  I   E  v  
   v   g E  t      v  v  
t
v
 E  E


  I  2   v   I


1
 I  v   U   v,  E ,  I   I  v     I  v   g I  t   
t
v
I
  v  v  
 v   r
U   v,  E ,  I   
 
g  E  t    f E 0 E  t   S E  mE '  t ,
'
  E 2  t  
mE '  t  
1 
2
2

f

t

S
  E 0 E    E 
,
2 E 
 ' pN E ' 
 v E



v



E


 





v



 





v
    



 v I


v



I


 


g  I  t   f I 0 I  t   S I  mI '  t 
'
  I 2  t  
mI '  t  
1  2
2
f

t

S
 I 0 I    I 

2 I 
 ' pN I  ' 





  v   J   v   0
t
v
1
1
1

J   v      v   r   2  g  E  t    E  E 2  t    v   E   2  g  I  t    I   I 2  t   v   I     v 




  E  E 2  t 
 I   I 2  t 
2
2 

v



v


 E
 I   v   v 
2
2




m t   J  VT 
      v, , t  ,
 '     '  v, , t 
g  E  , t    f E   0 E  , t    S E    '   mE '  ', t  d '
'
  E
2

mE '  ', t 
1 
2
2
d '
t  
  f E  0 E  t    S E    ' 
2 E 
 ' pN E '  ' 



 B  v   U  v,  E B ,  I B  B  v  
t
v


  E B 2   v   E


1
B
B
B
B
B
 E  v   U  v,  E ,  I
 E  v  

 v   g E B  t   B
t
v
 E  E
  v  v  




  I  B 2   v   I


1
B
B
B
B
B
 I  v   U  v,  E ,  I
 I  v  

 v   g  I B  t   B
t
v
 I  I
  v  v  




g  E B  t    B f E B 0 E B  t   S E B  mE ' B  t   S  E BD 
'
g  I B  t   f I  0 I
B
B
'
 t   S I  mI '  t   S
B
B
BD
 I
'

 N
'
1
I '
1
N E '
t
D
t
e
e
D

t s
I

t s
E


B
   v  




B

v



 


  p   t  t   ds,
j
'
j
'
jPE 'D
  p   t  t   ds
k
'
k
'
kPI 'D
t s

t 
2
2
2
mE ' B  t 
1 
1
E
B
B
B
BD
  E B  t  

f

t

S

S
e
p

t

t
ds
  E
,
 E




0E  
 E
j  ' 
j  ' 
D 
D
2 E 
pN
pN
'
'

jPE '
E '
E '

t s

t 
2
2
mI ' B  t 
1  B 2 B
1
I
2
B
BD
  I B  t  

S
e
p

t

t
ds
 f I  0 I  t   S I




I



 D  k  '
k  '
D 
2 I 
pN I '
'
 ' pN I  '
kPI '

2

 









dVi  D

  Vi  D   r  Gi  E D  t  Vi  D   E  Gi  I D  t  Vi  D   I ,
dt





dGi  E D
1
E
 Gi  E D   D f E D   t  t i  S E D 
D
dt

 ' N E '

 S E DB 

  p    t  t '  
j
 ' jPE 'B
'
j
 S I DB 

  p    t  T '  
 ' kPI ' B
k
'
k
  p    t  t  
j
jPE '
'
j
'
D
'
dGi  I D
1
I
 Gi  I D  f I D   t  T i  S I D 
D
dt

 ' N I '


  p    t  T  
k
'
k
'
kPI 'D
'
 1, simple
0, complex
  E , I ;  simple, complex,  
Poissonspiketrainst ' j  '  , T ' j  ' arereconstructed  fromtheratemE ' B N E ' B , mI ' B N I ' B .
Kinetic Theory for Population Dynamics
Population of interacting neurons:
dVi
  Vi   r   Gi  t  Vi   E 
dt
dG
S
 i  Gi  f   t  t i    p j  t  t j 
dt
N j 




1-p: Synaptic Failure rate
 1,with probability p,
p j  
 0,with probability1  p.
1
N
  v, g , t   E 

   v  V t     g  G t  
i
i
i
Under ASSUMPTIONS: 1) N 1  1
2) Summed intra-cortical low rate spike events become Poisson:
Kinetic Equation:

   v   r
  v, g     
t
v   
 v E

 g

 
   g



v
,
g


v
,
g











 g 
 

f 
 0  t     v, g  , t     v, g , t  
 
 

 


S
 pm  t  N    v, g 
, t     v, g , t  
pN 
 


m  t    J (v, g , t ) |v VT dg
0
pS  S

 
  v   r
  v, g    
t
v 
  
 v E

 g

 
 v   r
J  v, g , t    
 

 v r
 g

 

    v, g , t 




  1 
2  

v
,
g

g

g
t



v
,
g

   g   
 
 

g

g 






g  t   f  0  t   Sm  t  ,
1
 g t  
2
2
 2

S2
m t  ,
 f  0 t  
Np


f , N  finite
Fluctuation-Driven Dynamics
Physical Intuition:
Fluctuation-driven/Correlation between g and V
Hierarchy of Conditional Moments

 1  v    g   g | v  dg
0

 2

 v   0
g   g | v  dg
2
 v   
2
 2
 v     1  v 


 1 

  g     g  g  t     g    g 2
  g  
t
g  
g

2
g  t   f  0  t   Sm  t  ,

1  2
S2
 g t  
m  t 
 f  0 t  
2 
pN

f , N : finite
2

   v   r
  v    
t
v   
 v E

1


v



 





v






 v   r
 1
1
  v      1  v   g  t  
t

 
   v    v   E

  v  v  
 v   E     1 

1


v



     v   v  



 
   2  v E 


v


     v  



v







Closure Assumptions:
 2
 v  0
v

1  2
S2
 g t  
m  t 
 f  0 t  
2 
pN

2
2  v    g 2
Closed Equations — Reduced Kinetic Equations:

   v   r
  v    
t
v   
 v   r
 1
  v   
t
 

 v   E 

1
   v 
   v 





 v E

1


v
 


 
 g   v   E
1 1
   1

v


v

g
t

 
  
  v

  v  v  



2



v
  


Coarse-Graining in Time:
  0,
m
1
 g 2   v   E 
 g 2  g 2  v   E  

  v   g t  
  v   g  t  

 v 
  v  v   



v

v
 

1
 g 2  t  O 1
 Fluctuation Effects
 Correlation Effects
Fokker-Planck Equation:

 
  v   r
  v    
t
v  

 g 2   v   E
 

   g  t  
   
 
Flux:
 g


v






 
2
v   E 
2



  v 
v


2 
1

J  v       t  v    t     v   q 2  v   E 
  v 

v

Determination of Firing Rate:
JV VT   m  t 
For a steady state,
m can be determined implicitly
   v  dv  m  t 
VT
r
ref
1