* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Modeling working memory and decision making using generic
Emotion and memory wikipedia , lookup
Activity-dependent plasticity wikipedia , lookup
Synaptogenesis wikipedia , lookup
Time perception wikipedia , lookup
Cortical cooling wikipedia , lookup
Artificial general intelligence wikipedia , lookup
Nonsynaptic plasticity wikipedia , lookup
Binding problem wikipedia , lookup
Caridoid escape reaction wikipedia , lookup
Clinical neurochemistry wikipedia , lookup
Mirror neuron wikipedia , lookup
Psychophysics wikipedia , lookup
Neurocomputational speech processing wikipedia , lookup
State-dependent memory wikipedia , lookup
Neuroanatomy wikipedia , lookup
Neuroethology wikipedia , lookup
Mathematical model wikipedia , lookup
Pre-Bötzinger complex wikipedia , lookup
Stimulus (physiology) wikipedia , lookup
Neural oscillation wikipedia , lookup
Neural modeling fields wikipedia , lookup
Sparse distributed memory wikipedia , lookup
Premovement neuronal activity wikipedia , lookup
Holonomic brain theory wikipedia , lookup
Convolutional neural network wikipedia , lookup
Optogenetics wikipedia , lookup
Artificial neural network wikipedia , lookup
Neural correlates of consciousness wikipedia , lookup
Neuroeconomics wikipedia , lookup
Feature detection (nervous system) wikipedia , lookup
Neuropsychopharmacology wikipedia , lookup
Neural coding wikipedia , lookup
Central pattern generator wikipedia , lookup
Neural engineering wikipedia , lookup
Biological neuron model wikipedia , lookup
Metastability in the brain wikipedia , lookup
Channelrhodopsin wikipedia , lookup
Recurrent neural network wikipedia , lookup
Neural binding wikipedia , lookup
Types of artificial neural networks wikipedia , lookup
Synaptic gating wikipedia , lookup
Modeling working memory and decision making using generic neural microcircuits Prashant Joshi, Institute for Theoretical Computer Science, Technische Universität Graz, Austria Email: [email protected] | Web: www.igi.tugraz.at/joshi Synopsis Decision making is a recurring event in our day to day lives, and involves three phases: L – Initial loading of stimulus into working memory (WM), M – Maintaining the stimulus in WM for a couple of seconds, D – Making a binary decision on the arrival of second stimulus A neurocomputational model using generic neural microcircuits with feedback can integrate these three phases into a single unified framework! Outline Background & Methods Computational principles Generic neural microcircuits What is a liquid state and who are the “neural users”? Interval discrimination tasks Two-interval discrimination (TID) Delayed match-to-sample (DMS) A glimpse of biology – the role of pre-frontal cortex in decision making Modeling Results Additional Results Conclusions Computational Principles Dilemma – Analog fading memory has an upper limit on the order of tens of msec whereas working memory holds information on the order of seconds New theoretical results [Maass, Joshi, Sontag, 2005] imply that generic neural microcircuit with feedback has unexpected computational capabilities: It can emulate any dynamical system, in particular any analog computer Induces multiple co-existing “partial-attractor” states in the circuit dynamics This work demonstrates that even in presence of feedback noise, such “partial attractor” states can be held by generic neural microcircuits on the time-scales of several seconds, which is obviously a requirement for tasks involving working memory Generic Cortical Microcircuits Neurons – leaky integrate-and-fire neurons, 20% of them inhibitory, neuron a is connected to neuron b with probability C.exp(-D2(a,b)/λ2) Synapses – dynamic synapses with fixed parameters w, U, D, F chosen from distributions based on empirical data from Henry Markram’s lab What is a liquid state and who are the “neural users”? Each readout neuron receives as input a vector x(t), which has as many components as it has the pre-synaptic neurons in the circuit The ith component of x(t) results from the spiketrain of the ith pre-synaptic neuron by applying a low-pass filter, which models the low-pass filtering properties of receptors and membrane of the readout neuron We assume that a readout neuron has at time t a firing rate proportional to w.x(t) Remark: Experimental results from the labs of Nicolelis and Poggio show that such weighted sums (from neurons in visual or motor cortex) contain behavior-relevant information Interval Discrimination Task Classical experimental paradigm to study working memory and decision making Subject has to compare two stimuli (tactile, visual, auditory etc.) The experiment starts with presentation of the first stimulus Second stimulus is presented after a temporal delay Subject needs to make a binary (yes/no) decision after the presentation of the second stimuli, usually by pressing a corresponding switch Two-Interval-Discrimination Subject receives two frequencies f1 and f2 and has to decide if f1 > f2? Delayed-Match-to-Sample Cue stimulus (a color) is presented initially on the screen Subsequently two probe stimuli (one identical to cue stimulus) are presented Subject has to decide which probe stimuli has the same color as the cue stimulus A glimpse of biology – the role of pre-frontal cortex in decision making f1 > f2? Two kinds of neurons (+ and “-”) have been observed in PFC which show opposite activities is response to the above question The “+” neurons show an increase in their activity during the decision phase when the answer to the above question is “yes” The “– “neurons show an increase when this answer is “no” Figure from Machens et. Al. Science, 307:1121-1124 Modeling Results The generic neural microcircuit is used as a model for PFC Inputs are fed to the circuit with a simple form of spatial coding (moving Gaussian window) Each input is fed to a layer of 100 neurons in the circuit At each time step t, a signal-dependent noise was added to the input signal (0.0001. ρ . f(t)) ρ is a random number drawn from a Gaussian distribution with mean 0 and SD 1 f(t) is the current value of the input signal Simple linear readouts trained by linear regression are used to model the “+” or “-” neurons, and send feedback of their activity to the neural circuit Modeling Two-intervaldiscrimination 400 neurons, arranged of a 20 x 5 x 4 grid Circuit receives two external inputs (f1 and f2), and feedback from the readouts modeling the “+” and “-” neurons Total simulation time for one trial = 3.5 sec Simulation time-step = 10 msec 10 pairs of input frequencies f1 and f2 are presented for 0.5 sec each, during the L and D phases respectively Results - TID f1 = 18 Hz, f2 = 26 Hz Readouts perform exceptionally well even in the presence of feedback noise (correlation of 0.9 and above) Robustness – Setup is robust to synaptic pruning – control values (no pruning) for 10 validation runs, mean = 0.9956, SD = 0.0206 Interesting as traditional models of attractor based neural computation fail to demonstrate robustness Additional Results Decision making followed by action selection is one of the most challenging events of our daily lives Involves 4 phases – 3 for decision making (L,M, and D), and one for acting on the decision (called the A phase) Neurocomputational models of working memory (including this talk till now) fail to address how the decisions made by neurons in PFC are converted into motor commands, which are executed by the sensori-motor system? Similarly, models for computational motor control ignore the first three phases (L, M and D) A neurocomputational architecture is proposed that integrates the four phases (LMDA) involved in the process of action selection in presence of a decision Essentially this model integrates two different cortical modalities – decision making carried on by PFC, and subsequent action selection carried out by sensori-motor system A spiking neural network model is presented for the DMS task followed by an arm movement to the decided goal position Modeling DMS followed by arm movement PFC Circuit: •500 neurons, arranged of a 20 x 5 x 5 grid •Circuit receives three external inputs (Csample, Cleft, and Cright), and feedback from the readouts modeling the “left” and “right” neurons M1 Circuit: • 1000 neurons (20 x 5 x 10) • 4 external inputs (The x and y coordinates of the center of left and right probe regions) • the output of the “left” and “right” readouts from PFC circuit • feedback from readouts that predict joint angles • feedback from readouts that compute joint torques Arm Model: • Standard model of a 2-joint robot arm • Trajectory followed by the tip of the arm is generated using the minimum jerk model Simulation: • Total simulation time for one trial = 2.5 sec • Simulation time-step = 10 msec • 5 triplets of input colors Csample is presented for 0.32 seconds during the L phase, and Cleft, and Cright are presented simultaneously for 0.6 seconds, during the D phase Results – DMS + Arm Movement Movement occurs during the A phase (last 500 msec) Biologically realistic bell-shaped velocity profile observed Performance – tested over 100 validation runs: Left – mean = 0.9962, SD = 0.0018 Right – mean = 0.9942, SD = 0.0018 τ1 – mean = 0.9246, SD = 0.0247 τ2 – mean = 0.9571, SD = 0.0019 θ1 – mean = 0.9733, SD = 0.0282 θ2 – mean = 0.9887, SD = 0.0035 Results – DMS + Arm Movement Conclusions A new neurocomputational paradigm is described that uses synaptic learning mechanisms and is able to integrate the L,M,D and A phases involved in decision making followed by action selection Task independent and biologically realistic Demonstrates the ability of generic neural microcircuit models to hold “partial attractor” states Robustness to factors such as synaptic pruning and feedback noise Works for significant long time-scales in presence of noise because: Generic neural microcircuits are inherently endowed with fading memory Feedback enhances this Adding noise to teacher-feedback helps in making the target an attractor Reflects the “internal model” hypothesis References 1. Machens , C.K., Romo, R., and Brody, C.D., 2005, Flexible control of mutual inhibition: a neural model of two-interval discrimination, Science, 307:11211124 2. W. Maass, P. Joshi, and E. D. Sontag. Computational aspects of feedback in neural circuits. submitted for publication, 2006. (PDF, 1154 KB) 3. W. Maass, P. Joshi, and E. D. Sontag. Principles of real-time computing with feedback applied to cortical microcircuit models. In Advances in Neural Information Processing Systems, volume 18. MIT Press, 2006. in press. (PDF, 806 KB) 4. P.Joshi. Modeling working memory and decision making using generic neural microcircuits. In Proc. of the International Conference on Artificial Neural Networks, ICANN, 2006. in press 5. P. Joshi. From memory based decisions to decision based movements: A model of interval discrimination followed by action selection. submitted for publication, 2006