Survey
* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project
Modeling Primary Visual Cortex (of Macaque) David W. McLaughlin Courant Institute & Center for Neural Science New York University [email protected] Santa Barbara Aug ‘01 Input Layer of V1 for Macaque Modeled at : Courant Institute of Math. Sciences & Center for Neural Science, NYU In collaboration with: Robert Shapley Michael Shelley Louis Tao Jacob Wielaard Visual Pathway: Retina --> LGN --> V1 --> Beyond Why the Primary Visual Cortex? Why the Primary Visual Cortex? Elementary processing, early in visual pathway Neurons in V1 detect elementary features of the visual scene, such as spatial frequency, direction, & orientation Why the Primary Visual Cortex? Elementary processing, early in visual pathway Neurons in V1 detect elementary features of the visual scene, such as spatial frequency, direction, & orientation Vast amount of experimental information about V1 Why the Primary Visual Cortex? Elementary processing, early in visual pathway Neurons in V1 detect elementary features of the visual scene, such as spatial frequency, direction, & orientation Vast amount of experimental information about V1 Input from LGN well understood (Shapley, Reid, …) Why the Primary Visual Cortex? Elementary processing, early in visual pathway Neurons in V1 detect elementary features of the visual scene, such as spatial frequency, direction, & orientation Vast amount of experimental information about V1 Input from LGN well understood (Shapley, Reid, …) Anatomy of V1 well understood (Lund, Callaway, ...) Why the Primary Visual Cortex? Elementary processing, early in visual pathway Neurons in V1 detect elementary features of the visual scene, such as spatial frequency, direction, & orientation Vast amount of experimental information about V1 Input from LGN well understood (Shapley, Reid, …) Anatomy of V1 well understood (Lund, Callaway, ...) The cortical region with finest spatial resolution -- Why the Primary Visual Cortex? Elementary processing, early in visual pathway Neurons in V1 detect elementary features of the visual scene, such as spatial frequency, direction, & orientation Vast amount of experimental information about V1 Input from LGN well understood (Shapley, Reid, …) Anatomy of V1 well understood (Lund, Callaway, ...) The cortical region with finest spatial resolution -Detailed visual features of input signal; Why the Primary Visual Cortex? Elementary processing, early in visual pathway Neurons in V1 detect elementary features of the visual scene, such as spatial frequency, direction, & orientation Vast amount of experimental information about V1 Input from LGN well understood (Shapley, Reid, …) Anatomy of V1 well understood (Lund, Callaway, ...) The cortical region with finest spatial resolution -Detailed visual features of input signal; Fine scale resolution available for possible representation; Our Model • A detailed, fine scale model of a layer of Primary Visual Cortex; • Realistically constrained by experimental data; Our Model • A detailed, fine scale model of a layer of Primary Visual Cortex; • Realistically constrained by experimental data; • A “max-min’’ model -in that in its construction, we attempt to make maximal use of experimental data, & minimal use of posited architectural assumptions which are not supported by direct experimental evidence (such as Hebbian wiring schemes). Overview: One Max-Min Model of V1 Overview: One Max-Min Model of V1 • A detailed fine scale model -- constrained in construction and performance by experimental data ; • Orientation selectivity & its diversity from cortico-cortical activity, with neurons more selective near pinwheels; Overview: One Max-Min Model of V1 • A detailed fine scale model -- constrained in construction and performance by experimental data ; • Orientation selectivity & its diversity from cortico-cortical activity, with neurons more selective near pinwheels; • Linearity of Simple Cells -- produced by (i) averages over spatial phase, together with cortico-cortical overbalance for inhibition; Overview: One Max-Min Model of V1 • A detailed fine scale model -- constrained in construction and performance by experimental data ; • Orientation selectivity & its diversity from cortico-cortical activity, with neurons more selective near pinwheels; • Linearity of Simple Cells -- produced by (i) averages over spatial phase, together with cortico-cortical overbalance for inhibition; • Complex Cells -- produced by weaker (and varied) LGN input, together with stronger cortical excitation; Overview: One Max-Min Model of V1 • A detailed fine scale model -- constrained in construction and performance by experimental data ; • Orientation selectivity & its diversity from cortico-cortical activity, with neurons more selective near pinwheels; • Linearity of Simple Cells -- produced by (i) averages over spatial phase, together with cortico-cortical overbalance for inhibition; • Complex Cells -- produced by weaker (and varied) LGN input, together with stronger cortical excitation; • Operates in a high conductance state -- which results from cortical activity, is consistent with experiment, and makes integration times shorter than synaptic times, an emergent separation of temporal scales with functional implications; Overview: One Max-Min Model of V1 • A detailed fine scale model -- constrained in construction and performance by experimental data ; • Orientation selectivity & its diversity from cortico-cortical activity, with neurons more selective near pinwheels; • Linearity of Simple Cells -- produced by (i) averages over spatial phase, together with cortico-cortical overbalance for inhibition; • Complex Cells -- produced by weaker (and varied) LGN input, together with stronger cortical excitation; • Operates in a high conductance state -- which results from cortical activity, is consistent with experiment, and makes integration times shorter than synaptic times, an emergent separation of temporal scales with functional implications; • Together with a coarse-grained asymptotic reduction -- which unveils cortical mechanisms, and will be used to parameterize or ``scaleup’’ to larger more global cortical models. Features of the Single Layer, Local Patch Model Features of the Single Layer, Local Patch Model • Integrate & fire, point neuron model Features of the Single Layer, Local Patch Model • Integrate & fire, point neuron model • 16,000 neurons/sq mm 12,000 excitatory, 4000 inhibitory Features of the Single Layer, Local Patch Model • Integrate & fire, point neuron model • 16,000 neurons/sq mm 12,000 excitatory, 4000 inhibitory • A patch (1 sq mm) of 4 orientation hypercolumns Features of the Single Layer, Local Patch Model • Integrate & fire, point neuron model • 16,000 neurons/sq mm 12,000 excitatory, 4000 inhibitory • A patch (1 sq mm) of 4 orientation hypercolumns • Orientation pref from convergent LGN input Features of the Single Layer, Local Patch Model • Integrate & fire, point neuron model • 16,000 neurons/sq mm 12,000 excitatory, 4000 inhibitory • A patch (1 sq mm) of 4 orientation hypercolumns • Orientation pref from convergent LGN input • Coupling architecture, set by anatomy Features of the Single Layer, Local Patch Model • Integrate & fire, point neuron model • 16,000 neurons/sq mm 12,000 excitatory, 4000 inhibitory • A patch (1 sq mm) of 4 orientation hypercolumns • Orientation pref from convergent LGN input • Coupling architecture, set by anatomy • Local connections isotropic Features of the Single Layer, Local Patch Model • Integrate & fire, point neuron model • 16,000 neurons/sq mm 12,000 excitatory, 4000 inhibitory • A patch (1 sq mm) of 4 orientation hypercolumns • Orientation pref from convergent LGN input • Coupling architecture, set by anatomy • Local connections isotropic • Excitation longer range than inhibition Features of the Single Layer, Local Patch Model • Integrate & fire, point neuron model • 16,000 neurons/sq mm 12,000 excitatory, 4000 inhibitory • A patch (1 sq mm) of 4 orientation hypercolumns • Orientation pref from convergent LGN input • Coupling architecture, set by anatomy • Local connections isotropic • Excitation longer range than inhibition • Cortical inhibition dominant Conductance Based Model = E,I vj -- membrane potential -- = Exc, Inhib -- j = 2 dim label of location on cortical layer VE & VI -- Exc & Inh Reversal Potentials Conductance Based Model = E,I Schematic of Conductances Conductance Based Model = E,I Schematic of Conductances gE(t) = gLGN(t) + gnoise(t) + gcortical(t) Conductance Based Model = E,I Schematic of Conductances gE(t) = gLGN(t) + gnoise(t) + gcortical(t) (driving term) Conductance Based Model = E,I Schematic of Conductances gE(t) = gLGN(t) + gnoise(t) + gcortical(t) (driving term) (synaptic noise) (synaptic time scale) Conductance Based Model = E,I Schematic of Conductances gE(t) = gLGN(t) + gnoise(t) + gcortical(t) (driving term) (synaptic noise) (synaptic time scale) (cortico-cortical) (LExc > LInh) (Isotropic) Conductance Based Model = E,I Schematic of Conductances gE(t) = gLGN(t) + gnoise(t) + gcortical(t) (driving term) (synaptic noise) (synaptic time scale) Inhibitory Conductances: gI(t) = gnoise(t) + gcortical(t) (cortico-cortical) (LExc > LInh) (Isotropic) Elementary Feature Detectors Individual neurons in V1 respond preferentially to elementary features of the visual scene (color, direction of motion, speed of motion, spatial wave-length). Elementary Feature Detectors Individual neurons in V1 respond preferentially to elementary features of the visual scene (color, direction of motion, speed of motion, spatial wave-length). Three important features: Elementary Feature Detectors Individual neurons in V1 respond preferentially to elementary features of the visual scene (color, direction of motion, speed of motion, spatial wave-length). Three important features: • Spatial location (receptive field of the neuron) Elementary Feature Detectors Individual neurons in V1 respond preferentially to elementary features of the visual scene (color, direction of motion, speed of motion, spatial wave-length). Three important features: • Spatial location (receptive field of the neuron) • Spatial phase (relative to receptive field center) Elementary Feature Detectors Individual neurons in V1 respond preferentially to elementary features of the visual scene (color, direction of motion, speed of motion, spatial wave-length). Three important features: • Spatial location (receptive field of the neuron) • • Spatial phase (relative to receptive field center) Orientation of edges. Grating Stimuli Standing & Drifting Two Angles: Angle of orientation -- Angle of spatial phase -- (relevant for standing gratings) Orientation Tuning Curves (Firing Rates Vs Angle of Orientation) Spikes/sec Terminology: • Orientation Preference • Orientation Selectivity Measured by “ Half-Widths” or “Peak-to-Trough” Orientation Preference Orientation Preference • Model neurons receive their orientation preference from convergent LGN input; Orientation Preference • Model neurons receive their orientation preference from convergent LGN input; • How does the orientation preference k of the kth cortical neuron depend upon the neuron’s location k = (k1, k2) in the cortical layer? Cortical Map of Orientation Preference • Optical Imaging Blasdel, 1992 ---- • Outer layers (2/3) of V1 ---- 500 • Color coded for angle of orientation preference right eye left eye Pinwheel Centers 4 Pinwheel Centers 1 mm x 1 mm Orientation Selectivity While the model neurons receive their orientation preference hardwired from convergent LGN input; they receive their orientation selectivity & diversity from cortico-cortical activity; Orientation Tuning Curves __ __ __ Cortex off Spikes/sec Ringach, Hawken & Shapley McLaughlin,Shapley,Shelley & Wielaard PNAS ‘00 Orientation Selectivity (Measured by the “circular variance’’ of the tuning curves) CV ~ 1, poorly tuned ~ 0, very selective A measure of “height-to-trough” Useful for population studies Orientation Selectivity -- Population Behavior (CV = Circular Variance of Tuning Curves) CV ~ 1, poorly tuned ~ 0, very selective Ringach, Hawken & Shapley ____ Excitatory …… Inhibitory McLaughlin,Shapley,Shelley & Wielaard PNAS ‘00 Spatial Distributions of Firing Rates and Orientation Selectivity (Relative to Locations of Pinwheel Centers) Poorly tuned Spikes/sec Selective Firing Rates Circular Variance (of Orientation Selectivity) Experimental Evidence on Spatial Distribution of Orientation Selectivity (relative to pinwheel centers) • Maldonado, Gray, Goedecke & Bonhoffer, Science ‘97 • In cat • Data converted to CV’s by M. Shelley • Selectivity is diverse • More selective (?) near pinwheels Cortical Mechanism For Spatial Distribution Of Orientation Selectivity • Discs of incoming inhibition • Radius set by axonal arbors of inh. neurons • While inhibition is “local” in cortex, • Near pinwheels, it is “global” in orientation Simple and Complex Cells Simple cells respond linearly to properties of the stimulus – a network property. In a nonlinear network, “simple” is not so“simple”. • Simple Cells : Wielaard, Shelley, McLaughlin & Shapley, to appear, J. Neural Science (2001) • Simple & Complex Cells: Tao, Shelley, McLaughlin & Shapley, in prep (2001) Simple vs Complex Cells • Simple cells respond ``linearly’’ to properties of visual stimuli -- Simple vs Complex Cells • Simple cells respond ``linearly’’ to properties of visual stimuli -(i) Follow spatial phase of standing grating Simple vs Complex Cells • Simple cells respond ``linearly’’ to properties of visual stimuli -(i) Follow spatial phase of standing grating (ii) Respond temporally at the fundamental (1st harmonic) Simple vs Complex Cells • Simple cells respond ``linearly’’ to properties of visual stimuli -(i) Follow spatial phase of standing grating (ii) Respond temporally at the fundamental (1st harmonic) • Complex cells -- phase insensitive & large second harmonics Experimental Measurements: Simple and Complex Cells : Simple Cell Phase Time Model Results : Contrast Reversal (For Optimal and Orthogonal Phase) Cortex On Membrane Potential Optimal Phase Firing Rates Orthogonal Phase Cortex Off Mechanisms by which the Model Produces Simple Cells Inputs to Cortical Cell: • From LGN (Frequency doubled at orthogonal to optimal phase) • From Other Cortical Neurons Mechanisms by which the Model Produces Simple Cells Inputs to Cortical Cell: • From LGN (Frequency doubled at orthogonal to optimal phase) • From Other Cortical Neurons (Also freq doubled, because of averaging over random phases -- whose distribution is broad (De Angelis, et al ‘99) Mechanisms by which the Model Produces Simple Cells Inputs to Cortical Cell: • From LGN (Frequency doubled at orthogonal to optimal phase) • From Other Cortical Neurons (Also freq doubled, because of averaging over random phases -- whose distribution is broad (De Angelis, et al ‘99) Mechanisms by which the Model Produces Simple Cells Inputs to Cortical Cell: • From LGN (Frequency doubled at orthogonal to optimal phase) • From Other Cortical Neurons (Also freq doubled, because of averaging over random phases -- whose distribution is broad (De Angelis, et al ‘99) Cortical Overbalance for Inhibition (Borg-Graham, et al ‘98; Hirsch, et al ‘98; Anderson, et al ‘00) Cancellation Simple Cells • Recall mechanisms which produce (linear responses of) simple cells: (i) Averaging over spatial phases in cortico-cortical terms; Simple Cells • Recall mechanisms which produce (linear responses of) simple cells: (i) Averaging over spatial phases in cortico-cortical terms; (ii) Overbalance for inhibition in cortico-cortical terms. Simple Cells • Recall mechanisms which produce (linear responses of) simple cells: (i) Averaging over spatial phases in cortico-cortical terms; (ii) Overbalance for inhibition in cortico-cortical terms. (iii) Balance produces linearity of simple cells Simple Cells • Recall mechanisms which produce (linear responses of) simple cells: (i) Averaging over spatial phases in cortico-cortical terms; (ii) Overbalance for inhibition in cortico-cortical terms. (iii) Balance produces linearity of simple cells Indeed, this balance can be broken by pharmacologically weakening inhibition -- converting simple cells to complex Expt refs -- Sillito (‘74); Fregnac and Schulz (‘99); Humphrey (‘99) Simple vs Complex Cells Continued The model also contains complex cells (but, as yet, not enough, and the complex cells are not selective enough for orientation): Simple vs Complex Cells • Recall mechanisms which produce (linear responses of) simple cells: (i) Averaging over spatial phases in cortico-cortical terms; (ii) Overbalance for inhibition in cortico-cortical terms. Simple vs Complex Cells • Recall mechanisms which produce (linear responses of) simple cells: (i) Averaging over spatial phases in cortico-cortical terms; (ii) Overbalance for inhibition in cortico-cortical terms. • Mechanisms which produce (nonlinear responses of) complex cells: Simple vs Complex Cells • Recall mechanisms which produce (linear responses of) simple cells: (i) Averaging over spatial phases in cortico-cortical terms; (ii) Overbalance for inhibition in cortico-cortical terms. • Mechanisms which produce (nonlinear responses of) complex cells: (i) Weaker (and varied) LGN input; Simple vs Complex Cells • Recall mechanisms which produce (linear responses of) simple cells: (i) Averaging over spatial phases in cortico-cortical terms; (ii) Overbalance for inhibition in cortico-cortical terms. • Mechanisms which produce (nonlinear responses of) complex cells: (i) Weaker (and varied) LGN input; (ii) Stronger cortico-cortical excitation (Abbott, et al, Nature Neural Science ‘98) Simple vs Complex Cells Continued Drifting grating stimulation Distributions of simple and complex cells Expt -- Ringach, Shapley & Hawken Model -- Tao, Shelley, McLaughlin & Shapley Expts Model ( Ringach, Shapley & Hawken) (Tao, Shelley, McL & Shapley) (Similar to earlier results of De Valois, et al) In V1, 40% Simple (Preliminary) In 4C, 55% Simple 1 mm x 1mm Local Patch of 4C 1 mm x 1mm Active Model Cortex - High Conductances Active Model Cortex - High Conductances • Background Firing Statistics ====> gBack = 2-3 gslice Active Model Cortex - High Conductances • Background Firing Statistics ====> gBack = 2-3 gslice • Active operating point ====> gAct = 2-3 gBack = 4-9 gslice Active Model Cortex - High Conductances • Background Firing Statistics ====> gBack = 2-3 gslice • Active operating point ====> gAct = 2-3 gBack = 4-9 gslice ====> gInh >> gExc Active Model Cortex - High Conductances • Background Firing Statistics ====> gBack = 2-3 gslice • Active operating point ====> gAct = 2-3 gBack = 4-9 gslice ====> gInh >> gExc • Consistent with experiment Hirsch, et al, J. Neural Sci ‘98; Borg-Graham, et al, Nature ‘98; Anderson, et al, J. Physiology ’00; Lampl, et al, Neuron ‘99 Conductances Vs Time • Drifting Gratings -- 8 Hz • Turned on at t = 1.0 sec • Cortico-cortical excitation weak relative to LGN; inhibition >> excitation Distribution of Conductance Within the Layer Sec-1 <gT> = Time Average SD(gT) = Standard Deviation Of Temporal Fluctuations Sec-1 Active Cortex - Consequences of High Conductances • Separation of time scales ; Active Cortex - Consequences of High Conductances • Separation of time scales ; • Activity induced g = gT-1 << syn (actually, 2 ms << 4 ms) Active Cortex - Consequences of High Conductances • Separation of time scales ; • Activity induced g = gT-1 << syn (actually, 2 ms << 4 ms) • Membrane potential ``instantaneously’’ tracks conductances on the synaptic time scale. Definition of Effective Reversal Potential V(t) ~ VEff(t) = [VE gEE(t) - | VI | gEI(t) ] [gT(t)]-1 Where gT(t) denotes the total conductance Conductance Based Model = E,I dv/dt = gT(t) [ v - VEff(t) ], where gT(t) denotes the total conductance, and VEff(t) = [VE gEE(t) - | VI | gEI(t) ] [gT(t)]-1 If [gT(t)] -1 << syn v VEff(t) High Conductances in Active Cortex Membrane Potential Tracks Instantaneously “Effective Reversal Potential” Active Background Effects of Scale Separation g = 2 syn ____(Red) = VEff(t) ____(Green) = V(t) g = syn g = ½ syn Active Cortex - Consequences of High Conductances Thus, with this instantaneous tracking (on the synaptic time scale), cortical activity can convert neurons from integrators to burst generators & coincidence detectors. Coarse-Grained Asymptotics Coarse-Grained Asymptotics • Using the spatial regularity of cortical maps (such as orientation preference), we “coarse grain” the cortical layer into local cells or “placquets”. Cortical Map of Orientation Preference • Optical Imaging Blasdel, 1992 ---- • Outer layers (2/3) of V1 ---- 500 • Color coded for angle of orientation preference right eye left eye Coarse-Grained Asymptotics • Using the spatial regularity of cortical maps (such as orientation preference), we “coarse grain” the cortical layer into local cells or “placquets”. • Using the separation of time scales which emerge from cortical activity, Coarse-Grained Asymptotics • Using the spatial regularity of cortical maps (such as orientation preference), we “coarse grain” the cortical layer into local cells or “placquets”. • Using the separation of time scales which emerge from cortical activity, • Together with an averaging over the irregular cortical maps (such as spatial phase) Coarse-Grained Asymptotics • Using the spatial regularity of cortical maps (such as orientation preference), we “coarse grain” the cortical layer into local cells or “placquets”. • Using the separation of time scales which emerge from cortical activity, • Together with an averaging over the irregular cortical maps (such as spatial phase) • we derive a coarse-grained description in terms of the average firing rates of neurons within each placquet Uses of Coarse-Grained Eqs Coarse-grained equations can be used to unveil the model’s mechanism for • Better selectivity near pinwheel centers Spatial Distributions of Firing Rates and Orientation Selectivity (Relative to Locations of Pinwheel Centers) Poorly tuned Spikes/sec Selective Firing Rates Circular Variance (of Orientation Selectivity) m = {F + cEE KEE * m – cEI KEI * n }+ n = {F + cIE KIE * m – cII KII * n }+ ---------------------------------------For ease, specialize : cEE = cIE = cII = 0 m = {F – cEI KEI * n }+ n = {F }+ That is, ---------------------------------------------- m = {F – cEI KEI * {F }+ }+ m() = {F () – cEI ’ KEI ( -’) {F (’) }+ }+ ----------------------------------------------m() = {F () – cEI {F () }+ }+ FARR m() = {F () – cEI {’ F (’) }+ }+ NEAR Uses of Coarse-Grained Eqs • Unveil mechanims for (i) Better selectivity near pinwheel centers Uses of Coarse-Grained Eqs • Unveil mechanims for (i) Better selectivity near pinwheel centers (ii) Balances for simple and complex cells Uses of Coarse-Grained Eqs • Unveil mechanims for (i) Better selectivity near pinwheel centers (ii) Balances for simple and complex cells • Input-output relations at high conductance One application of Coarse-Grained Equations Uses of Coarse-Grained Eqs • Unveil mechanims for (i) Better selectivity near pinwheel centers (ii) Balances for simple and complex cells • Input-output relations at high conductance • Comparison of the mechanisms and performance of distinct models of the cortex Uses of Coarse-Grained Eqs • Unveil mechanims for (i) Better selectivity near pinwheel centers (ii) Balances for simple and complex cells • Input-output relations at high conductance • Comparison of the mechanisms and performance of distinct models of the cortex • Most importantly, much faster to integrate; Uses of Coarse-Grained Eqs • Unveil mechanims for (i) Better selectivity near pinwheel centers (ii) Balances for simple and complex cells • Input-output relations at high conductance • Comparison of the mechanisms and performance of distinct models of the cortex • Most importantly, much faster to integrate; • Therefore, potential parameterizations for more global descriptions of the cortex. Conductance Based Model = E,I -- 16,000 neurons per mm2 -- Locally, connections are isotropic but -- Long range coupling is spatially heterogenous and orientation specific Lateral Connections and Orientation -- Tree Shrew Bosking, Zhang, Schofield & Fitzpatrick J. Neuroscience, 1997 Scale-up & Dynamical Issues for Cortical Modeling • Temporal emergence of visual perception • Role of temporal feedback -- within and between cortical layers and regions • Synchrony & asynchrony • Presence (or absence) and role of oscillations • Spike-timing vs firing rate codes • Very noisy, fluctuation driven system • Emergence of an activity dependent, separation of time scales • But often no (or little) temporal scale separation Summary: One Max-Min Model of V1 • A detailed fine scale model -- constrained in its construction and performance by experimental data ; • Orientation selectivity & its diversity from cortico-cortical activity, with neurons more selective near pinwheels; • Linearity of Simple Cells -- produced by (i) averages over spatial phase, together with cortico-cortical overbalance for inhibition; • Complex Cells -- produced by weaker (and varied) LGN input, together with stronger cortical excitation; • Operates in a high conductance state -- which results from cortical activity, is consistent with experiment, and makes integration times shorter than synaptic times, a separation of temporal scales with functional implications; • Together with a coarse-grained asymptotic reduction -- which unveils cortical mechanisms, and will be used to parameterize or ``scaleup’’ to larger more global cortical models.