Download Neural basis of sensorimotor learning: modifying

CONEUR-623; NO OF PAGES 9
Available online at www.sciencedirect.com
Neural basis of sensorimotor learning: modifying internal models
Hagai Lalazar1 and Eilon Vaadia1,2
The neural basis of the internal models used in sensorimotor
transformations is beginning to be uncovered. Sensorimotor
learning involves the modification of such models. Different
stages of sensory-motor processing have been explored with a
continuum of experimental tasks, from learning arbitrary
associations of sensory cues to movements, to adapting to
altered kinematic and dynamic environments. Several groups
have been studying changes in neuronal activity in cortical and
subcortical areas that may be related to the acquisition and
consolidation processes. We discuss the progress and
challenges in understanding how these learning-related neural
changes are involved in the modification of internal models, and
offer future directions.
Addresses
1
Department of Physiology, Faculty of Medicine and The
Interdisciplinary Center for Neural Computation (ICNC), The Hebrew
University of Jerusalem, Israel
2
The Jack H. Skirball Chair & Research Fund in Brain Research
Corresponding author: Lalazar, Hagai ([email protected]) and
Vaadia, Eilon ([email protected])
Current Opinion in Neurobiology 2008, 18:1–9
This review comes from a themed issue on
Motor systems
Edited by Tadashi Isa and Andrew Schwartz
0959-4388/$ – see front matter
# 2008 Elsevier Ltd. All rights reserved.
DOI 10.1016/j.conb.2008.11.003
Introduction
Sensorimotor learning is essential for daily behavior. It is
used for maintaining known behavioral abilities and for
learning new skills. For example, it is used to learn how to
move a cursor on a computer screen or to assign behavioral
relevance to some arbitrary stimulus, such as the instruction ‘stop the car’ when the traffic light is red. Current
opinion holds that sensorimotor actions are produced by
transformations that utilize internal models of the
mechanics of the body and the world [1] (Figure 1b).
Inverse models work in the obvious direction: transforming
desired goals into a plan to accomplish thema [2]
a
We refer here to inverse models in a general sense. Desired goals are
just the input to the model, and the plan refers to the output. For
example, an inverse kinematics model may map a desired target location
to some kinematic plan, for example movement direction. In the case of
an inverse dynamics model, plan equates with the command to the
muscles or torques.
www.sciencedirect.com
(Figure 1a). These transformations are undoubtedly a
major part of sensorimotor control. However, since action
based solely on sensory feedback is slow and risky, the
brain — like efficient artificial control systems — has
developed the power of prediction. Predictive abilities
can be achieved by using forward models that simulate
the outcomes of a given plan (Figure 1a). There are
compelling theoretical arguments why forward models
are an essential feature of sensorimotor control, perception, and learning [3]. First, forward models can be used to
give instantaneous predicted feedback that provides an
estimate of the state of the controlled effector, and thus
overcome the significant delays of real sensory feedback.
Therefore, forward models are a necessary ingredient for
the contemporary description of the sensorimotor system
as an optimal feedback controller [4]. When real sensory
feedback does arrive, it can be combined with such
predicted feedback to compensate for limitations and
noise of the sensory systems [5]. A second proposed role
for forward models involves anticipating and cancelling
the sensory effects of self initiated actions from the
incoming sensory stimuli (reafference). Finally, during
learning itself, forward models may be used to generate
sensory error signals (predicted feedback minus real feedback) which can guide learning of inverse models (distal
teacher) [6]. An increasing number of psychophysical
experiments support the notion that humans make use
of forward models (e.g. [7–10], for a review see [11]).
Classic experiments on fish [12,13], bats [14], and cats [15]
have demonstrated the existence of corollary discharges
(the outputs of the forward models) and their effects on
reafference. More recently, two elegant series of electrophysiological experiments in monkeys have culminated
in showing how forward models are used in the saccadic
eye movement system [16], and in sensing of head
motion in the vestibular system [17]. Mulliken et al.
[18] showed that during hand movements a subpopulation of neurons in the posterior parietal cortex of monkeys may encode a forward estimate of the cursor
direction on the screen. They found neurons whose
maximal encoding evolved too late to subserve feedforward motor processing, but too early to result from
sensory feedback.
The extent that prediction is used by different brain
functions remains an open question. The theory of active
perception proposes that in order to perceive the world,
predictions can direct active exploratory movements (e.g.
of the eyes, or the vibrissa [19]) and top-down attention to
the informative features of the stimulus. Going even
further, the proposal that perception itself is none other
than the process of integrating our internally generated
Current Opinion in Neurobiology 2008, 18:1–9
Please cite this article in press as: Lalazar H, Vaadia E. Neural basis of sensorimotor learning: modifying internal models, Curr Opin Neurobiol (2008), doi:10.1016/j.conb.2008.11.003
CONEUR-623; NO OF PAGES 9
2 Motor systems
Figure 1
Internal models in sensorimotor processing. (a) The basic computational unit: a paired inverse and forward model. Inverse models (inverse to the
direction of causality in the world) transform desired goals into a plan (see footnote a). An efference copy of the resulting plan can be transformed by
forward models into a prediction of the outcome, termed corollary discharge. (b) Canonical series of sensory-motor transformations utilizing internal
models. Given an abstract goal (I’m thirsty), or confronted with several choices (a glass of apple or orange juice), an action is selected (reach for glass
of orange juice on table). Coordinate transformations extract the target location. Using the current state kinematics (current state of hand), the inverse
kinematics computes a kinematics plan (e.g. movement direction from current endpoint position to target). Finally, using this plan and the current state
of the muscles and other properties of the target (e.g. for computing the necessary grip force and prehension), the inverse dynamics computes the
motor command to the muscles. Each of these inverse transformations has a matching forward model(s), which can compute predicted outcomes for
that stage of processing. These predictions can be used in the various ways described in the text. (c) Internal models in active perception. The module
enclosed in the dotted rectangle fits in the series of transformations of (b), as denoted. Given an abstract goal (I’m thirsty), a specific unseen object is
selected (bottle of orange juice in fridge). The inverse sensory model uses knowledge of the statistics of the world to guide an active exploratory
search: directing movements (eyes, hands, etc.), or attention to informative features if already looking in the right direction. For example, when facing
the open fridge, the model would direct eye movements to the relevant shelf, and to the correct height to differentiate the different bottles, or attention
to the relevant colors (orange markings on the bottle). This can yield a desired target location for movement (gray line) that would continue the series of
transformations as in (b). In addition, the coupled forward model(s) can yield a prediction of the next sensory inputs (we have knowledge of how orange
juice bottles appear in the back of the fridge). This hypothesis about what you will see can be compared with the real incoming sensory inputs, and may
be the seat of perception. Note that we propose that internal sensory models are used beyond their traditional role in motor control, and therefore
active sensory or active perception models may be more suitable terms.
Current Opinion in Neurobiology 2008, 18:1–9
www.sciencedirect.com
Please cite this article in press as: Lalazar H, Vaadia E. Neural basis of sensorimotor learning: modifying internal models, Curr Opin Neurobiol (2008), doi:10.1016/j.conb.2008.11.003
CONEUR-623; NO OF PAGES 9
Neural basis of sensorimotor learning Lalazar and Vaadia 3
predictions with the incoming sensory stream is an attractive one. In this vain, Noë argues that ‘the experience of
seeing occurs when the organism masters what we call the
governing laws of sensorimotor contingency’ [20]. Thus,
forward models may be a more fundamental and pervasive feature of brain organization than just a component of
motor control [21] (Figure 1c).
Different sensorimotor learning tasks modify the relevant
internal models. Depending on the context, a red light
may be associated with different behaviors. Likewise, we
may need to learn different sensorimotor transformations
when we control a cursor by using different devices, such
as a mouse, a touch-pad, or an electronic-pen. When the
design of such devices merges well with our internal
models, we learn to control novel devices more quickly
and naturally.
Neural basis of sensorimotor learning
Different sensorimotor learning tasks involve different
learning strategies. As a result, distinct tasks affect
internal models at different stages of sensory-motor processing (Figure 1b). Here we discuss the challenges and
review the progress in studying the neural basis of sensorimotor learning.
Challenges
In spite of the advances in sensorimotor learning theory
and the many sophisticated model-driven psychophysics
experiments, our understanding of how learning is actually carried out in the brain has progressed slowly.
One of the fundamental problems in analyzing neural
activity during learning is the inherent difficulty in breaking the closed sensory-motor loops. Since inverse and
forward models are activated concurrently, they may each
be represented simultaneously in neural activity within a
behavioral trial [18]. Therefore, it is often difficult to
dissociate predictions of forward models from ‘predictions’ which are none other than adapted inverse models
[22]. It remains a major challenge to devise experiments
that can tease apart the neural activity underlying inverse
models from those of forward models.
Another challenge is an obvious one. Since by definition
learning causes behavioral changes (e.g. different arm
kinematics), the correlated changes in neural activity
may reflect either the learning process itself or rather
just the different parameters of the movement. One
approach is to compare the effects of learning on a
standard task after learning, in comparison to before
learning. Below we describe several studies where postlearning traces in neuronal activity have been shown after
behavior has returned to prelearning levels [23,24,25].
A third challenge arises from the attempt to isolate the
brain structures involved in learning the internal models
www.sciencedirect.com
themselves from those structures interacting with them.
For example, the cerebellum has been traditionally
emphasized as a key structure in motor learning [26],
and more recently as a site for inverse and forward models
for volitional movements and tool use. Support for this
view comes from behavioral studies in cerebellar patients
[27], from fMRI studies [22,28], and from physiological
experiments (for a review see [29]). However, owing to
the distributed nature of the cortical-cerebellar loops, it
has remained difficult to decisively demarcate the
locations of such models [30].
Learning arbitrary associations
The simplest learning tasks are related to the classical
notion that sensorimotor learning involves the generation
of new associations between stimuli (S) and responses
(R). Obviously we could learn to stop for a green light
instead of a red one. Learning a new arbitrary association
changes the categorical mapping between a given sensory
cue and a known movement. The learning, however,
need not affect the parameters of the movement itself.
Therefore, such learning modifies early stages in sensorymotor processing (Figure 1b, right side).
Owing to technical and conceptual limitations, most
animal experiments were based on studying neuronal
representations in the steady behavioral state. Animals
were first overtrained to generate an S ! R association
and only then surgery was performed and neural activity
recorded. In 1991, Wise and colleagues [31] examined the
dynamics of neuronal activity in premotor (PM) and
primary motor cortex (M1) during the process of learning
arbitrary associations. They found that most of the cells
changed their firing rate in high correlation with the
behavioral acquisition of an association between arbitrary
visual stimuli and well-known hand movements. Chen
and Wise replicated the same task in the supplementary
eye field (SEF; the assumed oculomotor equivalent of
PM) and found that such neurons’ preferred directions
(PD’s) shifted throughout the learning until converging
on their PD’s for familiar associations [32]. This might
provide evidence for the learning of an inverse internal
model. Since throughout the learning, the new stimulus
progressively activates the neurons that encode the correct eye movement. Congruently, the prediction of saccade direction by the population vector (computed with
the PD’s of the control stimuli) greatly improved with
learning [33]. In addition, they found a second subpopulation of cells that were generally not task-related, yet
showed enhanced firing rates only during the initial
learning phase and then returned to baseline [34] (thus
perhaps correlated to learning rate [35]).
In a recent study, Zach et al. [25] recorded neural activity
from monkeys as they learned to associate a color cue with
a movement to a given location, regardless of the spatial
location of the color cue. They found that during learning
Current Opinion in Neurobiology 2008, 18:1–9
Please cite this article in press as: Lalazar H, Vaadia E. Neural basis of sensorimotor learning: modifying internal models, Curr Opin Neurobiol (2008), doi:10.1016/j.conb.2008.11.003
CONEUR-623; NO OF PAGES 9
4 Motor systems
roughly 50% of neurons in M1 and PM cortex showed
learning-related changes. After learning, in a standard
task where target color was no longer relevant, most of
these neurons maintained their newly acquired sensitivity to the learned colors (as opposed to control colors,
not used in learning; see Figure 2). This study implies
that when an arbitrary sensory feature becomes behaviorally relevant, it can shape neuronal activity both during
and after learning, even in M1.
Now that similar arbitrary association tasks have been
repeated for a network of task-related structures, the
following speculative picture is emerging. The hippocampus signals learning of new arbitrary mappings, yet
without any spatial or motor components [36]. While the
caudate [35] and its output through the globus pallidus
(GPi) [37] may be involved in learning the mapping of the
arbitrary cue to the rewarded action, and may drive
learning in the prefrontal cortex [38]. Thus these areas
are probably involved in the early learning phases. During
late phases of learning, neurons in the putamen fire
persistently but selectively throughout the trial [39],
suggesting it may be involved in remembering the association just executed until its consequence becomes available. Finally, the dorsal premotor area (PMd) and SEF
seem to participate in the long-term retention and recall
of the associated action for each arbitrary stimulus. The
postlearning changes observed in M1 support its role in
consolidation [25], as do TMS studies (e.g. [40]). In
recent years, the involvement of some of these brain
areas in humans has been confirmed, using fMRI [41–
44] and PET [45].
Continuing to study the dynamics of neural activity
during learning in several areas simultaneously is valuable
for discovering the relative roles of each area and their
interactions, both across and within trials. For example,
both PMd and the dorsomedial putamen (a striatal target
Figure 2
Neuronal changes after learning arbitrary associations. Responses of a single neuron in (a) prelearning trials, (b) during learning, and (c) postlearning.
The cell showed enhanced responses when the monkey was learning to associate two colors (red and blue) with two movement directions
(908 and 1358, respectively). After learning, the responses to these colors (that were not relevant anymore to task performance) remained significantly
higher as compared to the prelearning responses, and in comparison to the colors not learned (in black and gray). The inset shows that this cell
was not directionally tuned, and that the postlearning enhanced responses (red and blue) were evoked in all movement directions. (d) Enhanced
responses during learning are correlated with postlearning responses to the learned colors. Each dot represents one cell. The x-axis shows firing
rate changes for the same movement directions and target colors during learning, normalized by firing rate to these trial types prelearning.
The y-axis shows firing rate changes to these target colors and movement directions postlearning, normalized by firing rates to these trial types before
learning (taken from [25]).
Current Opinion in Neurobiology 2008, 18:1–9
www.sciencedirect.com
Please cite this article in press as: Lalazar H, Vaadia E. Neural basis of sensorimotor learning: modifying internal models, Curr Opin Neurobiol (2008), doi:10.1016/j.conb.2008.11.003
CONEUR-623; NO OF PAGES 9
Neural basis of sensorimotor learning Lalazar and Vaadia 5
of PMd output) were shown to have equivalent trial-bytrial learning dynamics [46], that arose, however, from
distinct intra-trial dynamics [39].
Learning altered kinematics
Learning tasks that affect the kinematics of movement
alter an intermediate sensory-motor processing stage
(Figure 1b, middle). In visuomotor rotation, the direction
of the cursor motion relative to the direction of the
(unseen) hand motion is altered. For example, in a clockwise 908 rotation, leftward hand movements cause
upward cursor movements. Psychophysical [47] and
electrophysiological studies in behaving primates
[24,48] show that the generalization of visuomotor
rotations decays rapidly for directions beyond the learned
target direction [47,49]. Postlearning changes in M1,
during the preparatory period, are ‘local’ to the neurons
with preferred directions of the learned hand movement
direction [24] (see Figure 3). During learning, changes
appeared earlier in higher regions (supplementary motor
area; SMA) of the purported hierarchy of motor planning
areas, than in lower ones (M1) [50]. The dynamics of
these changes mirrored the dynamics of the learning. The
neuronal changes in SMA were correlated with the early
sharp improvement in behavior, while the changes in M1
appeared only later at the slower phase of the learning
curve.
Other altered kinematics tasks have also been studied.
Ojakangas and Ebner examined Purkinje cell activity
while monkeys learned new gains for reaching movements [51,52]. Their results suggest that complex spikes
encode the speed errors. Several learning tasks that have
been studied in humans (e.g. prism adaptation [53]) have
not yet been extended to electrophysiology.
Learning altered dynamics
The final stage of the series of sensorimotor transformations involves mapping some form of kinematic goal to
muscle commands (Figure 1b, left side). In their pioneering studies on learning in the cerebellum, Thach and
colleagues recorded from Purkinje neurons as monkeys
learned to oppose altered loads in a wrist flexion-extension task (for a review see [54]). They showed that the
firing rates of complex spikes changed preferentially
during adaptation to the altered load. Ito et al. followed
with the demonstration that these spikes can change the
parallel fiber–Purkinje cell synapse by long-term depression [55].
Altered dynamics have since been often studied by having subjects learn to make reaching movements while
grasping a manipulandum that produces forces on the
hand. Using this paradigm, Shadmehr and Mussa-Ivaldi
[56] discovered behavioral aftereffects that have served as
the most compelling evidence for the existence of
internal models and for their updating during learning.
Aftereffects are deviations from normal performance in a
standard task (e.g. curved trajectories) that are opposite to
the perturbation just learned. Such aftereffects can be
monitored even during the learning by catch trials —
standard trials randomly interleaved amid the learning
trials. However, it has been shown that the schedule of
catch trials during learning can affect memory consolidation [57]. Force fields too generalize only locally, and are
learned in intrinsic (joint-based) coordinates [58].
In a series of experiments, Bizzi and colleagues probed
the cortical changes in neuronal activity while monkeys
learned to control curl force fields (forces orthogonal to the
direction of movement and proportional to the movement
Figure 3
Neuronal changes after learning altered kinematics. (a) Directional tuning of a single neuron in the preparatory period. Prelearning in blue and
postlearning in red. The x-axis represents the distance from the learned movement direction which was near this cell’s PD. (b) Population average of
preparatory activity (normalized rates) as a function of the distance from the learned-movement direction. The blue line shows activity during the
prelearning epoch, and the red line in the postlearning epoch. Note that the population of cells shows increased activity around the learned movement
direction during the performance of the standard reaching task in the postlearning epoch (taken from [24]).
www.sciencedirect.com
Current Opinion in Neurobiology 2008, 18:1–9
Please cite this article in press as: Lalazar H, Vaadia E. Neural basis of sensorimotor learning: modifying internal models, Curr Opin Neurobiol (2008), doi:10.1016/j.conb.2008.11.003
CONEUR-623; NO OF PAGES 9
6 Motor systems
speed). They showed that many M1 neurons during the
movement epoch shifted their tuning while learning the
force field [23]. Moreover, some neurons maintained their
learned changes in a subsequent standard reaching task,
while others that did not have learning-related changes
did change in this final standard task. Using the same
experimental paradigm, they have since studied neuronal
changes in dorsal (PMd; see Figure 4a) and ventral (PMv)
premotor cortex [59], SMA [60], and the cingulate motor
areas (CMA’s) [61]. Learning-related changes were seen
more so in M1 and SMA, less so in PMd and PMv, and not
at all in CMA. In analyzing SMA neurons during learning,
they showed the development of PD shifts during the
preparatory activity (DT, delay time; see Figure 4b) [62].
They interpret this finding as evidence for the modifi-
cation of the kinematics to dynamics transformation, that
is updating of an inverse dynamics model.
The interpretation of any of the physiological studies
described in this review as evidence for the acquisition or
consolidation of inverse models remains difficult. Deciphering the subject’s true intentions is often ambiguous
(e.g. intended movement direction). At the neuronal
level, we still lack the ability to identify and simultaneously record the inputs, the local circuits, and the
outputs of each area involved. Because learning-related
neuronal changes were found in M1 in all three types of
tasks [23,24,25], one cannot exclude the possibility that
internal models of dynamics, kinematics, and arbitrary
associations are implemented in distributed brain areas.
Figure 4
Neuronal changes after learning altered dynamics. (a) Tuning curve of a PMd cell for each condition (prelearning, during learning, and postlearning) and
for two time windows (DT: delay time, MT: movement time) is plotted in blue. The PD is plotted in red. This cell was recorded with a clockwise force
field. It was classified by the authors as dynamic, because its PD changes during the delay time window. The firing rate scale is 26 spikes/s (denoted
by the full circle) (taken from [59]). (b) Temporal evolution of population mean PD shifts during the delay period. Trials are aligned on the target
appearance (time 0 ms). Positive values on the y-axis indicate shifts of PD in the direction of the external force. Asterisks (right plot, top right corner)
indicate data points significantly greater than 0. In the standard task (left plot) the collective PD remains essentially constant throughout the delay.
During adaptation to the force field the collective PD is initially aligned with that recorded in the standard task and progressively shifts over the course
of the delay in the direction of the external force. Only the activity preceding the go signal was included (taken from [62]).
Current Opinion in Neurobiology 2008, 18:1–9
www.sciencedirect.com
Please cite this article in press as: Lalazar H, Vaadia E. Neural basis of sensorimotor learning: modifying internal models, Curr Opin Neurobiol (2008), doi:10.1016/j.conb.2008.11.003
CONEUR-623; NO OF PAGES 9
Neural basis of sensorimotor learning Lalazar and Vaadia 7
Learning forward models
While the discussion above focused on learning inverse
models, we assume that predictive forward models are
employed in such tasks as well. Therefore, some of the
neural activity during a known task may be attributed to
the processing by the forward model. When exposed to a
learning task, these forward models may also be used to
help acquire the correct inverse model (distal teacher).
Moreover, during learning, the forward models themselves may be adapted and thus may account for some
of the neural activity changes.
The first electrophysiological evidence for the experience
dependent updating of a forward model was shown by
Bell [63] for electrosensation in the electric fish. An
artificial external electrical stimulation was paired with
the motor command recorded from the tail, while the
electric organ was blocked with curare. As a result, central
electrosensory neurons that first responded to the novel
stimulation (experimental reafference) learned to ignore
it as the association with the motor command was
acquired. After stimulation ceased the sensory neurons’
response to the motor command reversed, providing
evidence for the affect of the predictive model learned.
Most electrophysiological studies of forward models thus
far have examined effects of cancelling reafference. It
appears that this case is easier to study, since the forward
mapping subserves sensory processing and therefore only
the motor ! sensory segment of the sensory-motor loop
is studied. The neural basis of the other two proposed
varieties of forward models ( predicted feedback and distal
teacher) has thus far been harder to study. Those functions
involve the full motor ! sensory ! motor loop, and
therefore involve differentially identifying the motor
plan, motor command, predicted feedback, and real feedback, in the activity of motor areas.
Summary and future directions
While there is a growing body of physiological data, our
knowledge of how the brain learns to control sensorimotor
actions is still in its infancy. Where, when, and how does
the brain implement the internal models underlying the
sensory-motor transformations that guide both our actions
and perceptions remain open questions. To this end,
theoretical models which yield experimental predictions
are of key value. Specifically, computational approaches
from engineering need to be converted into neural network models from which predictions about neuronal
activity can be gleaned (e.g. [64]). Recently, two such
models provide interesting predictions about the processes underlying sensorimotor learning.
In their novel hypothesis, Rokni et al. [65] posit that the
neuronal changes found after learning an altered sensorimotor environment are a combination of the learned
component riding atop an ongoing random walk in the
www.sciencedirect.com
tuning of motor cortical neurons. They suggest that this
underlying synaptic variability is not an empty by-product, but a useful injection of noise which makes the
network more readily adaptable to learning new sensorimotor skills, while still maintaining the previous ones.
Thus, the seemingly well-developed representation of
familiar tasks that monkeys perform very stereotypically
and accurately may be based on a surprisingly unstable
neural representation. This noisy representation may
help to overcome the tension between the encoding
necessary for reliable performance and the fast plasticity
needed to quickly adapt to changes.
Sussillo and Abbott offer another interesting direction (D
Sussillo, LF Abbott, abstract III-46, COSYNE, 2008). In
their model, a chaotic recurrent network is connected to
different linear readout subnetworks that can learn to
produce a wide variety of output functions, one per
readout subnetwork. In this architecture, each subnetwork may represent a different motor primitive. Their
model predicts two distinct neuronal populations. The
recurrent network would exhibit chaotic dynamics with
connection weights that do not change, and the readout
subnetworks would undergo plasticity during learning.
Addressing such theoretical predictions using new experimental tools will aid our exploration of how the brain
learns. For example, brain–machine interface (BMI) setups artificially short circuit neuronal activity to an end
effector (e.g. a cursor on a screen or robotic arm [66]) via
an algorithm. This creates a causal and well-controlled
link between specific neurons and outputs. Recent studies have begun using this paradigm to probe the cortical
changes underlying sensorimotor learning [67]. In their
novel approach, Jarosiewicz et al. [68] applied a visuomotor rotation to their BMI algorithm by rotating the
preferred directions of a subpopulation of neurons. To
compensate for the perturbation, they found global tuning changes in the entire population, as well as, additional
local changes in the rotated subpopulation. This innovative approach exemplifies how established learning paradigms can be used in novel ways to further explore the
neural basis underlying sensorimotor learning.
Acknowledgements
We wish to thank several of our collaborators and the students in the lab; some
of their results we have cited herein. Note, typo. Our thanks especially go to Dr
Hagai Bergman, Dr Rony Paz, Dr Neta Zach, Dorrit Inbar, and Yael Grinvald.
The study was supported in part by the Israeli Science Foundation (ISF)
and the American Israeli Bi-National Foundation (BSF).
References and recommended reading
Papers of particular interest, published within the period of review,
have been highlighted as:
of special interest
of outstanding interest
1.
Wolpert DM, Ghahramani Z: Computational principles of
movement neuroscience. Nat Neurosci 2000, 3 Suppl:1212-1217.
Current Opinion in Neurobiology 2008, 18:1–9
Please cite this article in press as: Lalazar H, Vaadia E. Neural basis of sensorimotor learning: modifying internal models, Curr Opin Neurobiol (2008), doi:10.1016/j.conb.2008.11.003
CONEUR-623; NO OF PAGES 9
8 Motor systems
2.
Kawato M: Internal models for motor control and trajectory
planning. Curr Opin Neurobiol 1999, 9:718-727.
3.
Miall RC, Wolpert DM: Forward models for physiological motor
control. Neural Netw 1996, 9:1265-1279.
4.
Todorov E: Optimality principles in sensorimotor control. Nat
Neurosci 2004, 7:907-915.
5.
Wolpert DM, Ghahramani Z, Jordan MI: An internal model for
sensorimotor integration. Science 1995, 269:1880-1882.
6.
Jordan MJ, Rumelhart D: Forward models: supervised learning
with a distal teacher. Cognit Sci 1990, 16:307-354.
7.
Flanagan JR, Vetter P, Johansson RS, Wolpert DM: Prediction
precedes control in motor learning. Curr Biol 2003, 13:146-150.
8.
Blakemore SJ, Frith CD, Wolpert DM: Spatio-temporal
prediction modulates the perception of self-produced stimuli.
J Cogn Neurosci 1999, 11:551-559.
9.
Vaziri S, Diedrichsen J, Shadmehr R: Why does the brain predict
sensory consequences of oculomotor commands? Optimal
integration of the predicted and the actual sensory feedback.
J Neurosci 2006, 26:4188-4197.
10. McIntyre J, Zago M, Berthoz A, Lacquaniti F: Does the brain
model Newton’s laws? Nat Neurosci 2001, 4:693-694.
11. Davidson PR, Wolpert DM: Widespread access to predictive
models in the motor system: a short review. J Neural Eng 2005,
2:S313.
12. Roberts BL, Russell IJ: The activity of lateral-line efferent
neurones in stationary and swimming dogfish. J Exp Biol 1972,
57:435-448.
13. Zipser B, Bennett MV: Interaction of electrosensory and
electromotor signals in lateral line lobe of a mormyrid fish.
J Neurophysiol 1976, 39:713-721.
14. Suga N, Shimozawa T: Site of neural attenuation of responses
to self-vocalized sounds in echolocating bats. Science 1974,
183:1211-1213.
15. Ghez C, Pisa M: Inhibition of afferent transmission in cuneate
nucleus during voluntary movement in the cat. Brain Res 1972,
40:145-155.
22. Kawato M, Kuroda T, Imamizu H, Nakano E, Miyauchi S,
Yoshioka T: Internal forward models in the cerebellum: fMRI
study on grip force and load force coupling. Prog Brain Res
2003, 142:171-188.
23. Li CS, Padoa-Schioppa C, Bizzi E: Neuronal correlates of motor
performance and motor learning in the primary motor cortex
of monkeys adapting to an external force field. Neuron 2001,
30:593-607.
24. Paz R, Boraud T, Natan C, Bergman H, Vaadia E: Preparatory
activity in motor cortex reflects learning of local visuomotor
skills. Nat Neurosci 2003, 6:882-890.
25. Zach N, Inbar D, Grinvald Y, Bergman H, Vaadia E: Emergence of
novel representations in primary motor cortex and premotor
neurons during associative learning. J Neurosci 2008,
28:9545-9556.
M1 and premotor neuronal responses were studied before, during, and
after learning arbitrary associations between color and movement direction. This study shows that when arbitrary sensory features become
behaviorally relevant, they can shape neuronal activity both during and
after learning, even in M1.
26. Marr D: A theory of cerebellar cortex. J Physiol 1969, 202:437-470.
27. Tseng YW, Diedrichsen J, Krakauer JW, Shadmehr R, Bastian AJ:
Sensory prediction errors drive cerebellum-dependent
adaptation of reaching. J Neurophysiol 2007, 98:54-62.
28. Diedrichsen J, Criscimagna-Hemminger SE, Shadmehr R:
Dissociating timing and coordination as functions of the
cerebellum. J Neurosci 2007, 27:6291-6301.
29. Wolpert DM, Miall RC, Kawato M: Internal models in the
cerebellum. Trends Cogn Sci 1998, 2:338-347.
30. Pasalar S, Roitman AV, Durfee WK, Ebner TJ: Force field effects
on cerebellar Purkinje cell discharge with implications for
internal models. Nat Neurosci 2006, 9:1404-1411.
The authors tested the hypothesis that an inverse dynamics model
resides in the cerebellum. They recorded Purkinje cell simple spikes
while monkeys made circular tracing arm movements in two different
force fields. More than 90% of neurons were found to correlate with
movement kinematics but not dynamics. This result undermines the
hypothesis, although this conclusion is still being debated. More research
is necessary to reveal what internal models are implemented in the
cerebellum, and how.
16. Sommer MA, Wurtz RH: Brain circuits for the internal
monitoring of movements. Annu Rev Neurosci 2008,
31:317-338.
This paper reviews a series of methodical experiments on the effects of
corollary discharge from the superior colliculus to the frontal eye field (FEF)
in primates. By inactivating the thalamic relay, they showed that this
corollary discharge conveys a prediction of eye position for the subsequent
saccade. Furthermore, this activity shifts FEF neurons’ receptive fields in
anticipation of the upcoming saccade, thus suggestive of a mechanism for
visual stability. These studies form some of the most compelling electrophysiological evidence for the use of forward models in primates.
32. Chen LL, Wise SP: Evolution of directional preferences in the
supplementary eye field during acquisition of conditional
oculomotor associations. J Neurosci 1996, 16:3067-3081.
17. Cullen KE, Roy JE: Signal processing in the vestibular system
during active versus passive head movements. J Neurophysiol
2004, 91:1919-1933.
34. Chen LL, Wise SP: Neuronal activity in the supplementary eye
field during acquisition of conditional oculomotor
associations. J Neurophysiol 1995, 73:1101-1121.
18. Mulliken GH, Musallam S, Andersen RA: Forward estimation of
movement state in posterior parietal cortex. Proc Natl Acad Sci
U S A 2008, 105:8170-8177.
This is the first evidence, to our knowledge, for the use of forward models
during hand movements. The authors describe a subpopulation of neurons in the posterior parietal cortex that may encode a forward estimate of
the cursor direction on the screen. However, because the distribution of
peak response times was unimodal (Figure 3b, in their paper), it remains a
challenge to delineate activity underlying forward estimates from the
motor command or feedback. Improving the dissociation between target
and movement directions may address this issue. Thus, futures studies
are necessary to confirm this interesting result.
35. Williams ZM, Eskandar EN: Selective enhancement of
associative learning by microstimulation of the anterior
caudate. Nat Neurosci 2006, 9:562-568.
The role of the caudate in learning arbitrary associations was investigated. Similar to the SEF, neurons were found that correlated with either
the learning rate or learning curve. Moreover, when microstimulating this
area the learning was expedited, yet reached the same final performance
level. This is the first study that shows a causal role of the basal ganglia in
learning arbitrary associations, and may reflect its role in updating the
value of stimulus–response associations.
19. Kleinfeld D, Ahissar E, Diamond ME: Active sensation: insights
from the rodent vibrissa sensorimotor system. Curr Opin
Neurobiol 2006, 16:435-444.
20. Noe A: Action in Perception. Cambridge: MIT; 2005.
21. Crapse TB, Sommer MA: Corollary discharge across the animal
kingdom. Nat Rev Neurosci 2008, 9:587-600.
Current Opinion in Neurobiology 2008, 18:1–9
31. Mitz AR, Godschalk M, Wise SP: Learning-dependent neuronal
activity in the premotor cortex: activity during the acquisition of
conditional motor associations. J Neurosci 1991, 11:1855-1872.
33. Chen LL, Wise SP: Conditional oculomotor learning: population
vectors in the supplementary eye field. J Neurophysiol 1997,
78:1166-1169.
36. Suzuki WA: Integrating associative learning signals across the
brain. Hippocampus 2008, 17:842-850.
37. Inase M, Li BM, Takashima I, Iijima T: Pallidal activity is involved
in visuomotor association learning in monkeys. Eur J Neurosci
2001, 14:897-901.
38. Pasupathy A, Miller EK: Different time courses of learningrelated activity in the prefrontal cortex and striatum. Nature
2005, 433:873-876.
www.sciencedirect.com
Please cite this article in press as: Lalazar H, Vaadia E. Neural basis of sensorimotor learning: modifying internal models, Curr Opin Neurobiol (2008), doi:10.1016/j.conb.2008.11.003
CONEUR-623; NO OF PAGES 9
Neural basis of sensorimotor learning Lalazar and Vaadia 9
39. Buch E, Brasted PJ, Wise SP: Comparison of population activity in
the dorsal premotor cortex and putamen during the learning of
arbitrary visuomotor mappings. Exp Brain Res 2006, 169:69-84.
40. Hadipour-Niktarash A, Lee CK, Desmond JE, Shadmehr R:
Impairment of retention but not acquisition of a visuomotor
skill through time-dependent disruption of primary motor
cortex. J Neurosci 2007, 27:13413-13419.
41. Hanakawa T, Honda M, Zito G, Dimyan M, Hallett M: Brain activity
during visuomotor behavior triggered by arbitrary and
spatially constrained cues: an fMRI study in humans. Exp Brain
Res 2006, 172:275-282.
42. Eliassen JC, Souza T, Sanes JN: Experience-dependent
activation patterns in human brain during visual-motor
associative learning. J Neurosci 2003, 23:10540-10547.
43. Toni I, Ramnani N, Josephs O, Ashburner J, Passingham RE:
Learning arbitrary visuomotor associations: temporal
dynamic of brain activity. NeuroImage 2001, 14:1048-1057.
44. Brovelli A, Laksiri N, Nazarian B, Meunier M, Boussaoud D:
Understanding the neural computations of arbitrary
visuomotor learning through fMRI and associative learning
theory. Cerebral Cortex 2008, 18:1485-1495.
45. Toni I, Rushworth M, Passingham R: Neural correlates of
visuomotor associations. Spatial rules compared with
arbitrary rules. Exp Brain Res 2001, 141:359-369.
46. Brasted PJ, Wise SP: Comparison of learning-related neuronal
activity in the dorsal premotor cortex and striatum.
Eur J Neurosci 2004, 19:721-740.
47. Krakauer JW, Pine ZM, Ghilardi MF, Ghez C: Learning of
visuomotor transformations for vectorial planning of reaching
trajectories. J Neurosci 2000, 20:8916-8924.
48. Wise SP, Moody SL, Blomstrom KJ, Mitz AR: Changes in motor
cortical activity during visuomotor adaptation. Exp Brain Res
1998, 121:285-299.
49. Ghahramani Z, Wolpert DM, Jordan MI: Generalization to local
remappings of the visuomotor coordinate transformation.
J Neurosci 1996, 16:7085-7096.
50. Paz R, Natan C, Boraud T, Bergman H, Vaadia E: Emerging
patterns of neuronal responses in supplementary and primary
motor areas during sensorimotor adaptation. J Neurosci 2005,
25:10941-10951.
51. Ojakangas CL, Ebner TJ: Purkinje cell complex and simple
spike changes during a voluntary arm movement learning task
in the monkey. J Neurophysiol 1992, 68:2222-2236.
52. Ojakangas CL, Ebner TJ: Purkinje cell complex spike activity
during voluntary motor learning: relationship to kinematics.
J Neurophysiol 1994, 72:2617-2630.
53. Cui QN, Bachus L, Knoth E, O’Neill WE, Paige GD: Eye position
and cross-sensory learning both contribute to prism
adaptation of auditory space. In Progress in Brain Research
Using Eye Movements as an Experimental Probe of Brain
function — A Symposium in Honor of Jean Büttner-Ennever, vol
171. Edited by Kennard rso, John Leigh R. Elsevier; 2008:265-270.
54. Thach WT: A role for the cerebellum in learning movement
coordination. Neurobiol Learn Memory 1998, 70:177-188.
www.sciencedirect.com
55. Ito M, Sakurai M, Tongroach p: Climbing fibre induced
depression of both mossy fibre responsiveness and glutamate
sensitivity of cerebellar Purkinje cells. J Physiol 1982,
324:113-134.
56. Shadmehr R, Mussa-Ivaldi FA: Adaptive representation of
dynamics during learning of a motor task. J Neurosci 1994,
14:3208-3224.
57. Overduin SA, Richardson AG, Lane C, Bizzi E, Press DZ:
Intermittent practice facilitates stable motor memories.
J Neurosci 2006, 26:11888-11892.
58. Gandolfo F, Mussa-Ivaldi FA, Bizzi E: Motor learning by field
approximation. Proc Natl Acad Sci U S A 1996, 93:3843-3846.
59. Xiao J, Padoa-Schioppa C, Bizzi E: Neuronal correlates of
movement dynamics in the dorsal and ventral premotor area in
the monkey. Exp Brain Res 2006, 168:106-119.
60. Padoa-Schioppa C, Li CSR, Bizzi E: Neuronal activity in the
supplementary motor area of monkeys adapting to a new
dynamic environment. J Neurophysiol 2004, 91:449-473.
61. Richardson AG, Lassi-Tucci G, Padoa-Schioppa C, Bizzi E:
Neuronal activity in the cingulate motor areas during
adaptation to a new dynamic environment. J Neurophysiol
2008, 99:1253-1266.
62. Padoa-Schioppa C, Li CS, Bizzi E: Neuronal correlates of
kinematics-to-dynamics transformation in the supplementary
motor area. Neuron 2002, 36:751-765.
63. Bell CC: An efference copy which is modified by reafferent
input. Science 1981, 214:450-453.
64. Deneve S, Duhamel JR, Pouget A: Optimal sensorimotor
integration in recurrent cortical networks: a neural
implementation of Kalman filters. J Neurosci 2007,
27:5744-5756.
65. Rokni U, Richardson AG, Bizzi E, Seung HS: Motor learning with
unstable neural representations. Neuron 2007, 54:653-666.
This study offers a new theoretical perspective on the processes underlying the neuronal changes during learning. This idea is then tested by
reanalyzing data from Bizzi and colleagues [61]. The importance of the
study is in considering noise as a useful feature in learning, as discussed
in the text.
66. Velliste M, Perel S, Spalding MC, Whitford AS, Schwartz AB:
Cortical control of a prosthetic arm for self-feeding. Nature
2008, 453:1098-1101.
67. Zacksenhouse M, Lebedev MA, Carmena JM, O’Doherty JE,
Henriquez C, Nicolelis MAL: Cortical modulations increase in
early sessions with brain–machine interface. PLoS ONE 2007,
2:e619.
68. Jarosiewicz B, Chase SM, Fraser GW, Velliste M, Kass RE,
Schwartz AB: Functional network reorganization during
learning in a brain-computer interface paradigm. Proc Natl
Acad Sci U S A 2008, in press
The authors studied sensorimotor learning in motor cortex using the novel
technology of brain–machine interface (BMI). Their innovation was to
associate the activity of a subset of neurons with rotated movements.
They suggest that the changes in neural activity reflect both a new global
behavioral strategy and local changes in the altered subpopulation.
Current Opinion in Neurobiology 2008, 18:1–9
Please cite this article in press as: Lalazar H, Vaadia E. Neural basis of sensorimotor learning: modifying internal models, Curr Opin Neurobiol (2008), doi:10.1016/j.conb.2008.11.003