* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Neural basis of sensorimotor learning: modifying
Central pattern generator wikipedia , lookup
Environmental enrichment wikipedia , lookup
Cognitive neuroscience of music wikipedia , lookup
Time perception wikipedia , lookup
Holonomic brain theory wikipedia , lookup
Catastrophic interference wikipedia , lookup
Optogenetics wikipedia , lookup
Neuroeconomics wikipedia , lookup
Neural oscillation wikipedia , lookup
Neuroplasticity wikipedia , lookup
Artificial neural network wikipedia , lookup
Activity-dependent plasticity wikipedia , lookup
Recurrent neural network wikipedia , lookup
Donald O. Hebb wikipedia , lookup
Neuropsychopharmacology wikipedia , lookup
Embodied language processing wikipedia , lookup
Neural engineering wikipedia , lookup
Perceptual learning wikipedia , lookup
Psychological behaviorism wikipedia , lookup
Neural correlates of consciousness wikipedia , lookup
Development of the nervous system wikipedia , lookup
Types of artificial neural networks wikipedia , lookup
Learning theory (education) wikipedia , lookup
Neural modeling fields wikipedia , lookup
Premovement neuronal activity wikipedia , lookup
Nervous system network models wikipedia , lookup
CONEUR-623; NO OF PAGES 9 Available online at www.sciencedirect.com Neural basis of sensorimotor learning: modifying internal models Hagai Lalazar1 and Eilon Vaadia1,2 The neural basis of the internal models used in sensorimotor transformations is beginning to be uncovered. Sensorimotor learning involves the modification of such models. Different stages of sensory-motor processing have been explored with a continuum of experimental tasks, from learning arbitrary associations of sensory cues to movements, to adapting to altered kinematic and dynamic environments. Several groups have been studying changes in neuronal activity in cortical and subcortical areas that may be related to the acquisition and consolidation processes. We discuss the progress and challenges in understanding how these learning-related neural changes are involved in the modification of internal models, and offer future directions. Addresses 1 Department of Physiology, Faculty of Medicine and The Interdisciplinary Center for Neural Computation (ICNC), The Hebrew University of Jerusalem, Israel 2 The Jack H. Skirball Chair & Research Fund in Brain Research Corresponding author: Lalazar, Hagai ([email protected]) and Vaadia, Eilon ([email protected]) Current Opinion in Neurobiology 2008, 18:1–9 This review comes from a themed issue on Motor systems Edited by Tadashi Isa and Andrew Schwartz 0959-4388/$ – see front matter # 2008 Elsevier Ltd. All rights reserved. DOI 10.1016/j.conb.2008.11.003 Introduction Sensorimotor learning is essential for daily behavior. It is used for maintaining known behavioral abilities and for learning new skills. For example, it is used to learn how to move a cursor on a computer screen or to assign behavioral relevance to some arbitrary stimulus, such as the instruction ‘stop the car’ when the traffic light is red. Current opinion holds that sensorimotor actions are produced by transformations that utilize internal models of the mechanics of the body and the world [1] (Figure 1b). Inverse models work in the obvious direction: transforming desired goals into a plan to accomplish thema [2] a We refer here to inverse models in a general sense. Desired goals are just the input to the model, and the plan refers to the output. For example, an inverse kinematics model may map a desired target location to some kinematic plan, for example movement direction. In the case of an inverse dynamics model, plan equates with the command to the muscles or torques. www.sciencedirect.com (Figure 1a). These transformations are undoubtedly a major part of sensorimotor control. However, since action based solely on sensory feedback is slow and risky, the brain — like efficient artificial control systems — has developed the power of prediction. Predictive abilities can be achieved by using forward models that simulate the outcomes of a given plan (Figure 1a). There are compelling theoretical arguments why forward models are an essential feature of sensorimotor control, perception, and learning [3]. First, forward models can be used to give instantaneous predicted feedback that provides an estimate of the state of the controlled effector, and thus overcome the significant delays of real sensory feedback. Therefore, forward models are a necessary ingredient for the contemporary description of the sensorimotor system as an optimal feedback controller [4]. When real sensory feedback does arrive, it can be combined with such predicted feedback to compensate for limitations and noise of the sensory systems [5]. A second proposed role for forward models involves anticipating and cancelling the sensory effects of self initiated actions from the incoming sensory stimuli (reafference). Finally, during learning itself, forward models may be used to generate sensory error signals (predicted feedback minus real feedback) which can guide learning of inverse models (distal teacher) [6]. An increasing number of psychophysical experiments support the notion that humans make use of forward models (e.g. [7–10], for a review see [11]). Classic experiments on fish [12,13], bats [14], and cats [15] have demonstrated the existence of corollary discharges (the outputs of the forward models) and their effects on reafference. More recently, two elegant series of electrophysiological experiments in monkeys have culminated in showing how forward models are used in the saccadic eye movement system [16], and in sensing of head motion in the vestibular system [17]. Mulliken et al. [18] showed that during hand movements a subpopulation of neurons in the posterior parietal cortex of monkeys may encode a forward estimate of the cursor direction on the screen. They found neurons whose maximal encoding evolved too late to subserve feedforward motor processing, but too early to result from sensory feedback. The extent that prediction is used by different brain functions remains an open question. The theory of active perception proposes that in order to perceive the world, predictions can direct active exploratory movements (e.g. of the eyes, or the vibrissa [19]) and top-down attention to the informative features of the stimulus. Going even further, the proposal that perception itself is none other than the process of integrating our internally generated Current Opinion in Neurobiology 2008, 18:1–9 Please cite this article in press as: Lalazar H, Vaadia E. Neural basis of sensorimotor learning: modifying internal models, Curr Opin Neurobiol (2008), doi:10.1016/j.conb.2008.11.003 CONEUR-623; NO OF PAGES 9 2 Motor systems Figure 1 Internal models in sensorimotor processing. (a) The basic computational unit: a paired inverse and forward model. Inverse models (inverse to the direction of causality in the world) transform desired goals into a plan (see footnote a). An efference copy of the resulting plan can be transformed by forward models into a prediction of the outcome, termed corollary discharge. (b) Canonical series of sensory-motor transformations utilizing internal models. Given an abstract goal (I’m thirsty), or confronted with several choices (a glass of apple or orange juice), an action is selected (reach for glass of orange juice on table). Coordinate transformations extract the target location. Using the current state kinematics (current state of hand), the inverse kinematics computes a kinematics plan (e.g. movement direction from current endpoint position to target). Finally, using this plan and the current state of the muscles and other properties of the target (e.g. for computing the necessary grip force and prehension), the inverse dynamics computes the motor command to the muscles. Each of these inverse transformations has a matching forward model(s), which can compute predicted outcomes for that stage of processing. These predictions can be used in the various ways described in the text. (c) Internal models in active perception. The module enclosed in the dotted rectangle fits in the series of transformations of (b), as denoted. Given an abstract goal (I’m thirsty), a specific unseen object is selected (bottle of orange juice in fridge). The inverse sensory model uses knowledge of the statistics of the world to guide an active exploratory search: directing movements (eyes, hands, etc.), or attention to informative features if already looking in the right direction. For example, when facing the open fridge, the model would direct eye movements to the relevant shelf, and to the correct height to differentiate the different bottles, or attention to the relevant colors (orange markings on the bottle). This can yield a desired target location for movement (gray line) that would continue the series of transformations as in (b). In addition, the coupled forward model(s) can yield a prediction of the next sensory inputs (we have knowledge of how orange juice bottles appear in the back of the fridge). This hypothesis about what you will see can be compared with the real incoming sensory inputs, and may be the seat of perception. Note that we propose that internal sensory models are used beyond their traditional role in motor control, and therefore active sensory or active perception models may be more suitable terms. Current Opinion in Neurobiology 2008, 18:1–9 www.sciencedirect.com Please cite this article in press as: Lalazar H, Vaadia E. Neural basis of sensorimotor learning: modifying internal models, Curr Opin Neurobiol (2008), doi:10.1016/j.conb.2008.11.003 CONEUR-623; NO OF PAGES 9 Neural basis of sensorimotor learning Lalazar and Vaadia 3 predictions with the incoming sensory stream is an attractive one. In this vain, Noë argues that ‘the experience of seeing occurs when the organism masters what we call the governing laws of sensorimotor contingency’ [20]. Thus, forward models may be a more fundamental and pervasive feature of brain organization than just a component of motor control [21] (Figure 1c). Different sensorimotor learning tasks modify the relevant internal models. Depending on the context, a red light may be associated with different behaviors. Likewise, we may need to learn different sensorimotor transformations when we control a cursor by using different devices, such as a mouse, a touch-pad, or an electronic-pen. When the design of such devices merges well with our internal models, we learn to control novel devices more quickly and naturally. Neural basis of sensorimotor learning Different sensorimotor learning tasks involve different learning strategies. As a result, distinct tasks affect internal models at different stages of sensory-motor processing (Figure 1b). Here we discuss the challenges and review the progress in studying the neural basis of sensorimotor learning. Challenges In spite of the advances in sensorimotor learning theory and the many sophisticated model-driven psychophysics experiments, our understanding of how learning is actually carried out in the brain has progressed slowly. One of the fundamental problems in analyzing neural activity during learning is the inherent difficulty in breaking the closed sensory-motor loops. Since inverse and forward models are activated concurrently, they may each be represented simultaneously in neural activity within a behavioral trial [18]. Therefore, it is often difficult to dissociate predictions of forward models from ‘predictions’ which are none other than adapted inverse models [22]. It remains a major challenge to devise experiments that can tease apart the neural activity underlying inverse models from those of forward models. Another challenge is an obvious one. Since by definition learning causes behavioral changes (e.g. different arm kinematics), the correlated changes in neural activity may reflect either the learning process itself or rather just the different parameters of the movement. One approach is to compare the effects of learning on a standard task after learning, in comparison to before learning. Below we describe several studies where postlearning traces in neuronal activity have been shown after behavior has returned to prelearning levels [23,24,25]. A third challenge arises from the attempt to isolate the brain structures involved in learning the internal models www.sciencedirect.com themselves from those structures interacting with them. For example, the cerebellum has been traditionally emphasized as a key structure in motor learning [26], and more recently as a site for inverse and forward models for volitional movements and tool use. Support for this view comes from behavioral studies in cerebellar patients [27], from fMRI studies [22,28], and from physiological experiments (for a review see [29]). However, owing to the distributed nature of the cortical-cerebellar loops, it has remained difficult to decisively demarcate the locations of such models [30]. Learning arbitrary associations The simplest learning tasks are related to the classical notion that sensorimotor learning involves the generation of new associations between stimuli (S) and responses (R). Obviously we could learn to stop for a green light instead of a red one. Learning a new arbitrary association changes the categorical mapping between a given sensory cue and a known movement. The learning, however, need not affect the parameters of the movement itself. Therefore, such learning modifies early stages in sensorymotor processing (Figure 1b, right side). Owing to technical and conceptual limitations, most animal experiments were based on studying neuronal representations in the steady behavioral state. Animals were first overtrained to generate an S ! R association and only then surgery was performed and neural activity recorded. In 1991, Wise and colleagues [31] examined the dynamics of neuronal activity in premotor (PM) and primary motor cortex (M1) during the process of learning arbitrary associations. They found that most of the cells changed their firing rate in high correlation with the behavioral acquisition of an association between arbitrary visual stimuli and well-known hand movements. Chen and Wise replicated the same task in the supplementary eye field (SEF; the assumed oculomotor equivalent of PM) and found that such neurons’ preferred directions (PD’s) shifted throughout the learning until converging on their PD’s for familiar associations [32]. This might provide evidence for the learning of an inverse internal model. Since throughout the learning, the new stimulus progressively activates the neurons that encode the correct eye movement. Congruently, the prediction of saccade direction by the population vector (computed with the PD’s of the control stimuli) greatly improved with learning [33]. In addition, they found a second subpopulation of cells that were generally not task-related, yet showed enhanced firing rates only during the initial learning phase and then returned to baseline [34] (thus perhaps correlated to learning rate [35]). In a recent study, Zach et al. [25] recorded neural activity from monkeys as they learned to associate a color cue with a movement to a given location, regardless of the spatial location of the color cue. They found that during learning Current Opinion in Neurobiology 2008, 18:1–9 Please cite this article in press as: Lalazar H, Vaadia E. Neural basis of sensorimotor learning: modifying internal models, Curr Opin Neurobiol (2008), doi:10.1016/j.conb.2008.11.003 CONEUR-623; NO OF PAGES 9 4 Motor systems roughly 50% of neurons in M1 and PM cortex showed learning-related changes. After learning, in a standard task where target color was no longer relevant, most of these neurons maintained their newly acquired sensitivity to the learned colors (as opposed to control colors, not used in learning; see Figure 2). This study implies that when an arbitrary sensory feature becomes behaviorally relevant, it can shape neuronal activity both during and after learning, even in M1. Now that similar arbitrary association tasks have been repeated for a network of task-related structures, the following speculative picture is emerging. The hippocampus signals learning of new arbitrary mappings, yet without any spatial or motor components [36]. While the caudate [35] and its output through the globus pallidus (GPi) [37] may be involved in learning the mapping of the arbitrary cue to the rewarded action, and may drive learning in the prefrontal cortex [38]. Thus these areas are probably involved in the early learning phases. During late phases of learning, neurons in the putamen fire persistently but selectively throughout the trial [39], suggesting it may be involved in remembering the association just executed until its consequence becomes available. Finally, the dorsal premotor area (PMd) and SEF seem to participate in the long-term retention and recall of the associated action for each arbitrary stimulus. The postlearning changes observed in M1 support its role in consolidation [25], as do TMS studies (e.g. [40]). In recent years, the involvement of some of these brain areas in humans has been confirmed, using fMRI [41– 44] and PET [45]. Continuing to study the dynamics of neural activity during learning in several areas simultaneously is valuable for discovering the relative roles of each area and their interactions, both across and within trials. For example, both PMd and the dorsomedial putamen (a striatal target Figure 2 Neuronal changes after learning arbitrary associations. Responses of a single neuron in (a) prelearning trials, (b) during learning, and (c) postlearning. The cell showed enhanced responses when the monkey was learning to associate two colors (red and blue) with two movement directions (908 and 1358, respectively). After learning, the responses to these colors (that were not relevant anymore to task performance) remained significantly higher as compared to the prelearning responses, and in comparison to the colors not learned (in black and gray). The inset shows that this cell was not directionally tuned, and that the postlearning enhanced responses (red and blue) were evoked in all movement directions. (d) Enhanced responses during learning are correlated with postlearning responses to the learned colors. Each dot represents one cell. The x-axis shows firing rate changes for the same movement directions and target colors during learning, normalized by firing rate to these trial types prelearning. The y-axis shows firing rate changes to these target colors and movement directions postlearning, normalized by firing rates to these trial types before learning (taken from [25]). Current Opinion in Neurobiology 2008, 18:1–9 www.sciencedirect.com Please cite this article in press as: Lalazar H, Vaadia E. Neural basis of sensorimotor learning: modifying internal models, Curr Opin Neurobiol (2008), doi:10.1016/j.conb.2008.11.003 CONEUR-623; NO OF PAGES 9 Neural basis of sensorimotor learning Lalazar and Vaadia 5 of PMd output) were shown to have equivalent trial-bytrial learning dynamics [46], that arose, however, from distinct intra-trial dynamics [39]. Learning altered kinematics Learning tasks that affect the kinematics of movement alter an intermediate sensory-motor processing stage (Figure 1b, middle). In visuomotor rotation, the direction of the cursor motion relative to the direction of the (unseen) hand motion is altered. For example, in a clockwise 908 rotation, leftward hand movements cause upward cursor movements. Psychophysical [47] and electrophysiological studies in behaving primates [24,48] show that the generalization of visuomotor rotations decays rapidly for directions beyond the learned target direction [47,49]. Postlearning changes in M1, during the preparatory period, are ‘local’ to the neurons with preferred directions of the learned hand movement direction [24] (see Figure 3). During learning, changes appeared earlier in higher regions (supplementary motor area; SMA) of the purported hierarchy of motor planning areas, than in lower ones (M1) [50]. The dynamics of these changes mirrored the dynamics of the learning. The neuronal changes in SMA were correlated with the early sharp improvement in behavior, while the changes in M1 appeared only later at the slower phase of the learning curve. Other altered kinematics tasks have also been studied. Ojakangas and Ebner examined Purkinje cell activity while monkeys learned new gains for reaching movements [51,52]. Their results suggest that complex spikes encode the speed errors. Several learning tasks that have been studied in humans (e.g. prism adaptation [53]) have not yet been extended to electrophysiology. Learning altered dynamics The final stage of the series of sensorimotor transformations involves mapping some form of kinematic goal to muscle commands (Figure 1b, left side). In their pioneering studies on learning in the cerebellum, Thach and colleagues recorded from Purkinje neurons as monkeys learned to oppose altered loads in a wrist flexion-extension task (for a review see [54]). They showed that the firing rates of complex spikes changed preferentially during adaptation to the altered load. Ito et al. followed with the demonstration that these spikes can change the parallel fiber–Purkinje cell synapse by long-term depression [55]. Altered dynamics have since been often studied by having subjects learn to make reaching movements while grasping a manipulandum that produces forces on the hand. Using this paradigm, Shadmehr and Mussa-Ivaldi [56] discovered behavioral aftereffects that have served as the most compelling evidence for the existence of internal models and for their updating during learning. Aftereffects are deviations from normal performance in a standard task (e.g. curved trajectories) that are opposite to the perturbation just learned. Such aftereffects can be monitored even during the learning by catch trials — standard trials randomly interleaved amid the learning trials. However, it has been shown that the schedule of catch trials during learning can affect memory consolidation [57]. Force fields too generalize only locally, and are learned in intrinsic (joint-based) coordinates [58]. In a series of experiments, Bizzi and colleagues probed the cortical changes in neuronal activity while monkeys learned to control curl force fields (forces orthogonal to the direction of movement and proportional to the movement Figure 3 Neuronal changes after learning altered kinematics. (a) Directional tuning of a single neuron in the preparatory period. Prelearning in blue and postlearning in red. The x-axis represents the distance from the learned movement direction which was near this cell’s PD. (b) Population average of preparatory activity (normalized rates) as a function of the distance from the learned-movement direction. The blue line shows activity during the prelearning epoch, and the red line in the postlearning epoch. Note that the population of cells shows increased activity around the learned movement direction during the performance of the standard reaching task in the postlearning epoch (taken from [24]). www.sciencedirect.com Current Opinion in Neurobiology 2008, 18:1–9 Please cite this article in press as: Lalazar H, Vaadia E. Neural basis of sensorimotor learning: modifying internal models, Curr Opin Neurobiol (2008), doi:10.1016/j.conb.2008.11.003 CONEUR-623; NO OF PAGES 9 6 Motor systems speed). They showed that many M1 neurons during the movement epoch shifted their tuning while learning the force field [23]. Moreover, some neurons maintained their learned changes in a subsequent standard reaching task, while others that did not have learning-related changes did change in this final standard task. Using the same experimental paradigm, they have since studied neuronal changes in dorsal (PMd; see Figure 4a) and ventral (PMv) premotor cortex [59], SMA [60], and the cingulate motor areas (CMA’s) [61]. Learning-related changes were seen more so in M1 and SMA, less so in PMd and PMv, and not at all in CMA. In analyzing SMA neurons during learning, they showed the development of PD shifts during the preparatory activity (DT, delay time; see Figure 4b) [62]. They interpret this finding as evidence for the modifi- cation of the kinematics to dynamics transformation, that is updating of an inverse dynamics model. The interpretation of any of the physiological studies described in this review as evidence for the acquisition or consolidation of inverse models remains difficult. Deciphering the subject’s true intentions is often ambiguous (e.g. intended movement direction). At the neuronal level, we still lack the ability to identify and simultaneously record the inputs, the local circuits, and the outputs of each area involved. Because learning-related neuronal changes were found in M1 in all three types of tasks [23,24,25], one cannot exclude the possibility that internal models of dynamics, kinematics, and arbitrary associations are implemented in distributed brain areas. Figure 4 Neuronal changes after learning altered dynamics. (a) Tuning curve of a PMd cell for each condition (prelearning, during learning, and postlearning) and for two time windows (DT: delay time, MT: movement time) is plotted in blue. The PD is plotted in red. This cell was recorded with a clockwise force field. It was classified by the authors as dynamic, because its PD changes during the delay time window. The firing rate scale is 26 spikes/s (denoted by the full circle) (taken from [59]). (b) Temporal evolution of population mean PD shifts during the delay period. Trials are aligned on the target appearance (time 0 ms). Positive values on the y-axis indicate shifts of PD in the direction of the external force. Asterisks (right plot, top right corner) indicate data points significantly greater than 0. In the standard task (left plot) the collective PD remains essentially constant throughout the delay. During adaptation to the force field the collective PD is initially aligned with that recorded in the standard task and progressively shifts over the course of the delay in the direction of the external force. Only the activity preceding the go signal was included (taken from [62]). Current Opinion in Neurobiology 2008, 18:1–9 www.sciencedirect.com Please cite this article in press as: Lalazar H, Vaadia E. Neural basis of sensorimotor learning: modifying internal models, Curr Opin Neurobiol (2008), doi:10.1016/j.conb.2008.11.003 CONEUR-623; NO OF PAGES 9 Neural basis of sensorimotor learning Lalazar and Vaadia 7 Learning forward models While the discussion above focused on learning inverse models, we assume that predictive forward models are employed in such tasks as well. Therefore, some of the neural activity during a known task may be attributed to the processing by the forward model. When exposed to a learning task, these forward models may also be used to help acquire the correct inverse model (distal teacher). Moreover, during learning, the forward models themselves may be adapted and thus may account for some of the neural activity changes. The first electrophysiological evidence for the experience dependent updating of a forward model was shown by Bell [63] for electrosensation in the electric fish. An artificial external electrical stimulation was paired with the motor command recorded from the tail, while the electric organ was blocked with curare. As a result, central electrosensory neurons that first responded to the novel stimulation (experimental reafference) learned to ignore it as the association with the motor command was acquired. After stimulation ceased the sensory neurons’ response to the motor command reversed, providing evidence for the affect of the predictive model learned. Most electrophysiological studies of forward models thus far have examined effects of cancelling reafference. It appears that this case is easier to study, since the forward mapping subserves sensory processing and therefore only the motor ! sensory segment of the sensory-motor loop is studied. The neural basis of the other two proposed varieties of forward models ( predicted feedback and distal teacher) has thus far been harder to study. Those functions involve the full motor ! sensory ! motor loop, and therefore involve differentially identifying the motor plan, motor command, predicted feedback, and real feedback, in the activity of motor areas. Summary and future directions While there is a growing body of physiological data, our knowledge of how the brain learns to control sensorimotor actions is still in its infancy. Where, when, and how does the brain implement the internal models underlying the sensory-motor transformations that guide both our actions and perceptions remain open questions. To this end, theoretical models which yield experimental predictions are of key value. Specifically, computational approaches from engineering need to be converted into neural network models from which predictions about neuronal activity can be gleaned (e.g. [64]). Recently, two such models provide interesting predictions about the processes underlying sensorimotor learning. In their novel hypothesis, Rokni et al. [65] posit that the neuronal changes found after learning an altered sensorimotor environment are a combination of the learned component riding atop an ongoing random walk in the www.sciencedirect.com tuning of motor cortical neurons. They suggest that this underlying synaptic variability is not an empty by-product, but a useful injection of noise which makes the network more readily adaptable to learning new sensorimotor skills, while still maintaining the previous ones. Thus, the seemingly well-developed representation of familiar tasks that monkeys perform very stereotypically and accurately may be based on a surprisingly unstable neural representation. This noisy representation may help to overcome the tension between the encoding necessary for reliable performance and the fast plasticity needed to quickly adapt to changes. Sussillo and Abbott offer another interesting direction (D Sussillo, LF Abbott, abstract III-46, COSYNE, 2008). In their model, a chaotic recurrent network is connected to different linear readout subnetworks that can learn to produce a wide variety of output functions, one per readout subnetwork. In this architecture, each subnetwork may represent a different motor primitive. Their model predicts two distinct neuronal populations. The recurrent network would exhibit chaotic dynamics with connection weights that do not change, and the readout subnetworks would undergo plasticity during learning. Addressing such theoretical predictions using new experimental tools will aid our exploration of how the brain learns. For example, brain–machine interface (BMI) setups artificially short circuit neuronal activity to an end effector (e.g. a cursor on a screen or robotic arm [66]) via an algorithm. This creates a causal and well-controlled link between specific neurons and outputs. Recent studies have begun using this paradigm to probe the cortical changes underlying sensorimotor learning [67]. In their novel approach, Jarosiewicz et al. [68] applied a visuomotor rotation to their BMI algorithm by rotating the preferred directions of a subpopulation of neurons. To compensate for the perturbation, they found global tuning changes in the entire population, as well as, additional local changes in the rotated subpopulation. This innovative approach exemplifies how established learning paradigms can be used in novel ways to further explore the neural basis underlying sensorimotor learning. Acknowledgements We wish to thank several of our collaborators and the students in the lab; some of their results we have cited herein. Note, typo. Our thanks especially go to Dr Hagai Bergman, Dr Rony Paz, Dr Neta Zach, Dorrit Inbar, and Yael Grinvald. The study was supported in part by the Israeli Science Foundation (ISF) and the American Israeli Bi-National Foundation (BSF). References and recommended reading Papers of particular interest, published within the period of review, have been highlighted as: of special interest of outstanding interest 1. Wolpert DM, Ghahramani Z: Computational principles of movement neuroscience. Nat Neurosci 2000, 3 Suppl:1212-1217. Current Opinion in Neurobiology 2008, 18:1–9 Please cite this article in press as: Lalazar H, Vaadia E. Neural basis of sensorimotor learning: modifying internal models, Curr Opin Neurobiol (2008), doi:10.1016/j.conb.2008.11.003 CONEUR-623; NO OF PAGES 9 8 Motor systems 2. Kawato M: Internal models for motor control and trajectory planning. Curr Opin Neurobiol 1999, 9:718-727. 3. Miall RC, Wolpert DM: Forward models for physiological motor control. Neural Netw 1996, 9:1265-1279. 4. Todorov E: Optimality principles in sensorimotor control. Nat Neurosci 2004, 7:907-915. 5. Wolpert DM, Ghahramani Z, Jordan MI: An internal model for sensorimotor integration. Science 1995, 269:1880-1882. 6. Jordan MJ, Rumelhart D: Forward models: supervised learning with a distal teacher. Cognit Sci 1990, 16:307-354. 7. Flanagan JR, Vetter P, Johansson RS, Wolpert DM: Prediction precedes control in motor learning. Curr Biol 2003, 13:146-150. 8. Blakemore SJ, Frith CD, Wolpert DM: Spatio-temporal prediction modulates the perception of self-produced stimuli. J Cogn Neurosci 1999, 11:551-559. 9. Vaziri S, Diedrichsen J, Shadmehr R: Why does the brain predict sensory consequences of oculomotor commands? Optimal integration of the predicted and the actual sensory feedback. J Neurosci 2006, 26:4188-4197. 10. McIntyre J, Zago M, Berthoz A, Lacquaniti F: Does the brain model Newton’s laws? Nat Neurosci 2001, 4:693-694. 11. Davidson PR, Wolpert DM: Widespread access to predictive models in the motor system: a short review. J Neural Eng 2005, 2:S313. 12. Roberts BL, Russell IJ: The activity of lateral-line efferent neurones in stationary and swimming dogfish. J Exp Biol 1972, 57:435-448. 13. Zipser B, Bennett MV: Interaction of electrosensory and electromotor signals in lateral line lobe of a mormyrid fish. J Neurophysiol 1976, 39:713-721. 14. Suga N, Shimozawa T: Site of neural attenuation of responses to self-vocalized sounds in echolocating bats. Science 1974, 183:1211-1213. 15. Ghez C, Pisa M: Inhibition of afferent transmission in cuneate nucleus during voluntary movement in the cat. Brain Res 1972, 40:145-155. 22. Kawato M, Kuroda T, Imamizu H, Nakano E, Miyauchi S, Yoshioka T: Internal forward models in the cerebellum: fMRI study on grip force and load force coupling. Prog Brain Res 2003, 142:171-188. 23. Li CS, Padoa-Schioppa C, Bizzi E: Neuronal correlates of motor performance and motor learning in the primary motor cortex of monkeys adapting to an external force field. Neuron 2001, 30:593-607. 24. Paz R, Boraud T, Natan C, Bergman H, Vaadia E: Preparatory activity in motor cortex reflects learning of local visuomotor skills. Nat Neurosci 2003, 6:882-890. 25. Zach N, Inbar D, Grinvald Y, Bergman H, Vaadia E: Emergence of novel representations in primary motor cortex and premotor neurons during associative learning. J Neurosci 2008, 28:9545-9556. M1 and premotor neuronal responses were studied before, during, and after learning arbitrary associations between color and movement direction. This study shows that when arbitrary sensory features become behaviorally relevant, they can shape neuronal activity both during and after learning, even in M1. 26. Marr D: A theory of cerebellar cortex. J Physiol 1969, 202:437-470. 27. Tseng YW, Diedrichsen J, Krakauer JW, Shadmehr R, Bastian AJ: Sensory prediction errors drive cerebellum-dependent adaptation of reaching. J Neurophysiol 2007, 98:54-62. 28. Diedrichsen J, Criscimagna-Hemminger SE, Shadmehr R: Dissociating timing and coordination as functions of the cerebellum. J Neurosci 2007, 27:6291-6301. 29. Wolpert DM, Miall RC, Kawato M: Internal models in the cerebellum. Trends Cogn Sci 1998, 2:338-347. 30. Pasalar S, Roitman AV, Durfee WK, Ebner TJ: Force field effects on cerebellar Purkinje cell discharge with implications for internal models. Nat Neurosci 2006, 9:1404-1411. The authors tested the hypothesis that an inverse dynamics model resides in the cerebellum. They recorded Purkinje cell simple spikes while monkeys made circular tracing arm movements in two different force fields. More than 90% of neurons were found to correlate with movement kinematics but not dynamics. This result undermines the hypothesis, although this conclusion is still being debated. More research is necessary to reveal what internal models are implemented in the cerebellum, and how. 16. Sommer MA, Wurtz RH: Brain circuits for the internal monitoring of movements. Annu Rev Neurosci 2008, 31:317-338. This paper reviews a series of methodical experiments on the effects of corollary discharge from the superior colliculus to the frontal eye field (FEF) in primates. By inactivating the thalamic relay, they showed that this corollary discharge conveys a prediction of eye position for the subsequent saccade. Furthermore, this activity shifts FEF neurons’ receptive fields in anticipation of the upcoming saccade, thus suggestive of a mechanism for visual stability. These studies form some of the most compelling electrophysiological evidence for the use of forward models in primates. 32. Chen LL, Wise SP: Evolution of directional preferences in the supplementary eye field during acquisition of conditional oculomotor associations. J Neurosci 1996, 16:3067-3081. 17. Cullen KE, Roy JE: Signal processing in the vestibular system during active versus passive head movements. J Neurophysiol 2004, 91:1919-1933. 34. Chen LL, Wise SP: Neuronal activity in the supplementary eye field during acquisition of conditional oculomotor associations. J Neurophysiol 1995, 73:1101-1121. 18. Mulliken GH, Musallam S, Andersen RA: Forward estimation of movement state in posterior parietal cortex. Proc Natl Acad Sci U S A 2008, 105:8170-8177. This is the first evidence, to our knowledge, for the use of forward models during hand movements. The authors describe a subpopulation of neurons in the posterior parietal cortex that may encode a forward estimate of the cursor direction on the screen. However, because the distribution of peak response times was unimodal (Figure 3b, in their paper), it remains a challenge to delineate activity underlying forward estimates from the motor command or feedback. Improving the dissociation between target and movement directions may address this issue. Thus, futures studies are necessary to confirm this interesting result. 35. Williams ZM, Eskandar EN: Selective enhancement of associative learning by microstimulation of the anterior caudate. Nat Neurosci 2006, 9:562-568. The role of the caudate in learning arbitrary associations was investigated. Similar to the SEF, neurons were found that correlated with either the learning rate or learning curve. Moreover, when microstimulating this area the learning was expedited, yet reached the same final performance level. This is the first study that shows a causal role of the basal ganglia in learning arbitrary associations, and may reflect its role in updating the value of stimulus–response associations. 19. Kleinfeld D, Ahissar E, Diamond ME: Active sensation: insights from the rodent vibrissa sensorimotor system. Curr Opin Neurobiol 2006, 16:435-444. 20. Noe A: Action in Perception. Cambridge: MIT; 2005. 21. Crapse TB, Sommer MA: Corollary discharge across the animal kingdom. Nat Rev Neurosci 2008, 9:587-600. Current Opinion in Neurobiology 2008, 18:1–9 31. Mitz AR, Godschalk M, Wise SP: Learning-dependent neuronal activity in the premotor cortex: activity during the acquisition of conditional motor associations. J Neurosci 1991, 11:1855-1872. 33. Chen LL, Wise SP: Conditional oculomotor learning: population vectors in the supplementary eye field. J Neurophysiol 1997, 78:1166-1169. 36. Suzuki WA: Integrating associative learning signals across the brain. Hippocampus 2008, 17:842-850. 37. Inase M, Li BM, Takashima I, Iijima T: Pallidal activity is involved in visuomotor association learning in monkeys. Eur J Neurosci 2001, 14:897-901. 38. Pasupathy A, Miller EK: Different time courses of learningrelated activity in the prefrontal cortex and striatum. Nature 2005, 433:873-876. www.sciencedirect.com Please cite this article in press as: Lalazar H, Vaadia E. Neural basis of sensorimotor learning: modifying internal models, Curr Opin Neurobiol (2008), doi:10.1016/j.conb.2008.11.003 CONEUR-623; NO OF PAGES 9 Neural basis of sensorimotor learning Lalazar and Vaadia 9 39. Buch E, Brasted PJ, Wise SP: Comparison of population activity in the dorsal premotor cortex and putamen during the learning of arbitrary visuomotor mappings. Exp Brain Res 2006, 169:69-84. 40. Hadipour-Niktarash A, Lee CK, Desmond JE, Shadmehr R: Impairment of retention but not acquisition of a visuomotor skill through time-dependent disruption of primary motor cortex. J Neurosci 2007, 27:13413-13419. 41. Hanakawa T, Honda M, Zito G, Dimyan M, Hallett M: Brain activity during visuomotor behavior triggered by arbitrary and spatially constrained cues: an fMRI study in humans. Exp Brain Res 2006, 172:275-282. 42. Eliassen JC, Souza T, Sanes JN: Experience-dependent activation patterns in human brain during visual-motor associative learning. J Neurosci 2003, 23:10540-10547. 43. Toni I, Ramnani N, Josephs O, Ashburner J, Passingham RE: Learning arbitrary visuomotor associations: temporal dynamic of brain activity. NeuroImage 2001, 14:1048-1057. 44. Brovelli A, Laksiri N, Nazarian B, Meunier M, Boussaoud D: Understanding the neural computations of arbitrary visuomotor learning through fMRI and associative learning theory. Cerebral Cortex 2008, 18:1485-1495. 45. Toni I, Rushworth M, Passingham R: Neural correlates of visuomotor associations. Spatial rules compared with arbitrary rules. Exp Brain Res 2001, 141:359-369. 46. Brasted PJ, Wise SP: Comparison of learning-related neuronal activity in the dorsal premotor cortex and striatum. Eur J Neurosci 2004, 19:721-740. 47. Krakauer JW, Pine ZM, Ghilardi MF, Ghez C: Learning of visuomotor transformations for vectorial planning of reaching trajectories. J Neurosci 2000, 20:8916-8924. 48. Wise SP, Moody SL, Blomstrom KJ, Mitz AR: Changes in motor cortical activity during visuomotor adaptation. Exp Brain Res 1998, 121:285-299. 49. Ghahramani Z, Wolpert DM, Jordan MI: Generalization to local remappings of the visuomotor coordinate transformation. J Neurosci 1996, 16:7085-7096. 50. Paz R, Natan C, Boraud T, Bergman H, Vaadia E: Emerging patterns of neuronal responses in supplementary and primary motor areas during sensorimotor adaptation. J Neurosci 2005, 25:10941-10951. 51. Ojakangas CL, Ebner TJ: Purkinje cell complex and simple spike changes during a voluntary arm movement learning task in the monkey. J Neurophysiol 1992, 68:2222-2236. 52. Ojakangas CL, Ebner TJ: Purkinje cell complex spike activity during voluntary motor learning: relationship to kinematics. J Neurophysiol 1994, 72:2617-2630. 53. Cui QN, Bachus L, Knoth E, O’Neill WE, Paige GD: Eye position and cross-sensory learning both contribute to prism adaptation of auditory space. In Progress in Brain Research Using Eye Movements as an Experimental Probe of Brain function — A Symposium in Honor of Jean Büttner-Ennever, vol 171. Edited by Kennard rso, John Leigh R. Elsevier; 2008:265-270. 54. Thach WT: A role for the cerebellum in learning movement coordination. Neurobiol Learn Memory 1998, 70:177-188. www.sciencedirect.com 55. Ito M, Sakurai M, Tongroach p: Climbing fibre induced depression of both mossy fibre responsiveness and glutamate sensitivity of cerebellar Purkinje cells. J Physiol 1982, 324:113-134. 56. Shadmehr R, Mussa-Ivaldi FA: Adaptive representation of dynamics during learning of a motor task. J Neurosci 1994, 14:3208-3224. 57. Overduin SA, Richardson AG, Lane C, Bizzi E, Press DZ: Intermittent practice facilitates stable motor memories. J Neurosci 2006, 26:11888-11892. 58. Gandolfo F, Mussa-Ivaldi FA, Bizzi E: Motor learning by field approximation. Proc Natl Acad Sci U S A 1996, 93:3843-3846. 59. Xiao J, Padoa-Schioppa C, Bizzi E: Neuronal correlates of movement dynamics in the dorsal and ventral premotor area in the monkey. Exp Brain Res 2006, 168:106-119. 60. Padoa-Schioppa C, Li CSR, Bizzi E: Neuronal activity in the supplementary motor area of monkeys adapting to a new dynamic environment. J Neurophysiol 2004, 91:449-473. 61. Richardson AG, Lassi-Tucci G, Padoa-Schioppa C, Bizzi E: Neuronal activity in the cingulate motor areas during adaptation to a new dynamic environment. J Neurophysiol 2008, 99:1253-1266. 62. Padoa-Schioppa C, Li CS, Bizzi E: Neuronal correlates of kinematics-to-dynamics transformation in the supplementary motor area. Neuron 2002, 36:751-765. 63. Bell CC: An efference copy which is modified by reafferent input. Science 1981, 214:450-453. 64. Deneve S, Duhamel JR, Pouget A: Optimal sensorimotor integration in recurrent cortical networks: a neural implementation of Kalman filters. J Neurosci 2007, 27:5744-5756. 65. Rokni U, Richardson AG, Bizzi E, Seung HS: Motor learning with unstable neural representations. Neuron 2007, 54:653-666. This study offers a new theoretical perspective on the processes underlying the neuronal changes during learning. This idea is then tested by reanalyzing data from Bizzi and colleagues [61]. The importance of the study is in considering noise as a useful feature in learning, as discussed in the text. 66. Velliste M, Perel S, Spalding MC, Whitford AS, Schwartz AB: Cortical control of a prosthetic arm for self-feeding. Nature 2008, 453:1098-1101. 67. Zacksenhouse M, Lebedev MA, Carmena JM, O’Doherty JE, Henriquez C, Nicolelis MAL: Cortical modulations increase in early sessions with brain–machine interface. PLoS ONE 2007, 2:e619. 68. Jarosiewicz B, Chase SM, Fraser GW, Velliste M, Kass RE, Schwartz AB: Functional network reorganization during learning in a brain-computer interface paradigm. Proc Natl Acad Sci U S A 2008, in press The authors studied sensorimotor learning in motor cortex using the novel technology of brain–machine interface (BMI). Their innovation was to associate the activity of a subset of neurons with rotated movements. They suggest that the changes in neural activity reflect both a new global behavioral strategy and local changes in the altered subpopulation. Current Opinion in Neurobiology 2008, 18:1–9 Please cite this article in press as: Lalazar H, Vaadia E. Neural basis of sensorimotor learning: modifying internal models, Curr Opin Neurobiol (2008), doi:10.1016/j.conb.2008.11.003