Download The computational and neural basis of voluntary motor control and

Review
The computational and neural basis of
voluntary motor control and planning
Stephen H. Scott
Centre for Neuroscience Studies, Department of Biomedical and Molecular Sciences, Department of Medicine, Queen’s University,
Kingston, ON K7L 3N6, Canada
Optimal feedback control (OFC) provides a powerful tool
to interpret voluntary motor control, highlighting the
importance of sensory feedback in the control and planning of movement. Recent studies in the context of OFC
have increasingly used mechanical perturbations and
visual shifts to probe voluntary control processes. These
studies reveal the surprising sophistication of corrective
responses, which are goal-directed and exhibit knowledge of the physical properties of the limb and the
environment. These complex feedback processes appear
to be generated through transcortical feedback pathways. The research reviewed here opens and enhances
several lines of discovery, including testing whether
feedback corrections share all of the attributes associated with voluntary control, identifying how prediction
influences optimal state estimation, and importantly,
how these voluntary control processes are generated
by the highly distributed circuitry within the brain.
Changing views on the use of sensory feedback for
voluntary control
It is easy to recognize that sensory signals, such as from our
eyes, skin or muscles, help us perceive our body and the
world around us. It is harder to understand how these
same sensory signals help guide and correct our body
movements. Sensory feedback is clearly involved when
someone accidentally bumps your arm at a cocktail party
leading to rapid motor corrections to avoid spilling a drink
held in the hand. Less obvious is whether and how sensory
feedback is used continuously to adjust small errors when
we reach out to grab an object.
The importance of sensory feedback for motor control
was demonstrated over a century ago in the classic study
by Sherrington on the influence of afferent feedback on
spinal level reflexes in the cat hindlimb [1]. The development of feedback control theory in the mid 20th century led
to its application to biological control [2] and studies in the
1970s began to explore spinal and cortical feedback during
voluntary motor actions. However, several observations in
the 1980s suggested that feedback may not be very important for voluntary control [3]. In particular, it was noted
that transmission delays between the central nervous
system and the limb, on the order of 10s of milliseconds,
makes traditional servo-control unstable. As a result,
voluntary control was commonly assumed to use motor
programs, with motor cortex specifying feedforward
Corresponding author: Scott, S.H. ([email protected]).
commands to the spinal cord. Feedback was relegated to
a modest contribution at only the spinal level.
Recognition of the importance of sensory feedback for
voluntary control is in a renaissance, due to the introduction of optimal feedback control (OFC) as a model of
biological control. Here I review the use of OFC to interpret
voluntary control and the value of using perturbations
(once again) to probe the properties of the voluntary motor
system. Finally, I provide a brief sketch of how OFC-like
control may be implemented by distributed circuits in the
brain.
Optimal Feedback Control (OFC) as a theory of
voluntary control
Optimality principles have been common in motor control
with various possible objectives, such as minimizing jerk,
torque change, or the influence of noise [4]. In 2002,
Todorov and Jordan [5] proposed OFC as a theory of
voluntary control (Figure 1a). Delayed sensory feedback
that makes servo-control unstable is overcome by using
optimal state estimation, a Kalman filter integrating efference copy signals with delayed sensory feedback. The
derivation of the optimal control policy (i.e., feedback
gains) uses knowledge of the system dynamics, such as
properties of the musculoskeletal system, to achieve an
optimal balance between behavioural performance and
associated motor costs [6].
What is most impressive about these controllers is that
they capture many aspects of biological control. In particular, biological movements are highly variable and yet
always quite successful. This variability is created by noise
(neural, mechanical, and other), a known feature of sensorimotor systems [7]. Importantly, these types of controllers
correct errors/noise if they influence the goal of the task
and ignore them if they do not interfere with the goal. This
selective control of movement leads to characteristic patterns of motor variability during behavior [8–13]. Although
other models of motor control exist [14–16], they cannot
capture this behavioural-level feature of voluntary control.
One criticism of OFC as a model for biological control is
that it cannot be disproved; observed behaviour can be
predicted by modifying the cost function (this is true to a
degree). There are studies that highlight how humans do
not appear to behave optimally [17–19]. However, OFC is
best viewed as a normative model that identifies the best
possible solution for a given control problem (like Bayesian
inference for decisional processes). OFC crystalizes what
good control ought to look like, generating a wealth of
1364-6613/$ – see front matter ß 2012 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.tics.2012.09.008 Trends in Cognitive Sciences, November 2012, Vol. 16, No. 11
541
Review
[(Figure_1)TD$IG]
Trends in Cognitive Sciences November 2012, Vol. 16, No. 11
(a)
Opmal
feedback
control policy
Task
selecon
Motor
commands
Efference
copy
Opmal
state
esmaon
Sensory
feedback
(b)
Controlled plant
Basal ganglia
Spinal cord
PF
PP
PP
dPM
vPM
SMA
A5
dPM
vPM
SMA
A5
S1
A5
S1
dPM
M1
Cerebellum
TRENDS in Cognitive Sciences
Figure 1. Optimal Feedback Control and neural implementation. (a) Optimal Feedback Control Policy converts state variables to motor commands. Optimal State Estimation
uses efference copy of motor commands and sensory feedback to estimate state variables. Task Selection specifies the goal of the behaviour which then defines the
corresponding control policy. (b) Neural implementation of OFC-like voluntary control. The controlled plant is assumed to include the limb and spinal cord. The three basic
processes, task selection, control policy, and state estimation, are each generated by highly distributed circuits. Dashed boxes denote individual cortical regions and some
cortical regions are assumed to participate in two or even all three processes. Cortical connections are assumed amongst cortical regions involved in a given process. PF, PP
and S1 include multiple subdivisions not displayed for simplicity. Cortical abbreviations: PF, prefrontal; PP, posterior parietal; dPM, dorsal premotor; vPM, ventral premotor;
SMA, supplementary motor area; A5, area 5; S1, primary somatosensory; M1, primary motor. Images of spinal cord and limb adapted from [21].
experimental predictions on the general properties that
biological controllers could or should possess. In fact, observed deviations from ‘optimal behaviour’ provide extremely useful insight into the tricks and tradeoffs
inherent in biological control.
The mathematical tools used to solve optimal control
problems are limited largely to linear control problems
with some extensions for nonlinear systems [8]. Clearly,
the brain does not use the formalisms of OFC to solve
control problems. This is often used as a criticism of OFC
[20], although these alternatives inevitably have the same
problem. The assumption is that genetics and learning
provide good solutions that are OFC-like.
Perturbations as a probe of voluntary control
Because feedback is an essential feature of OFC, a key
approach to probe control is through perturbations, either
visual or mechanical. The use of mechanical perturbations
is not new – this approach has been used for many decades
to observe muscle stretch responses (Box 1). What OFC
clarifies is that control is highly specific to the ongoing
behavioural goal. Perturbations do not just elicit stereotypical motor reflexes, but also voluntary control processes
542
directly [21]. Thus, perturbations provide an important
window into voluntary control. Recent studies highlight
several important features of these rapid corrective
responses (see [22], for a detailed review).
Behavioural goal
One key feature of OFC is that errors are only corrected if
they affect the behavioural goal and are ignored if they do
not. This principle has been shown in a recent study that
compared corrective responses when subjects reached to a
circular target versus a rectangular bar (Figure 2, [23]).
Corrective responses were directed back to the circular
target, whereas responses for the rectangular bar were
redirected to a new location along the bar. Subjects took
advantage of the rectangular shape by making smaller
corrective responses closer to the perturbed position of
the hand. Differences in electromyographic (EMG) activity
for corrective responses to the circular and rectangular
targets were observed in as little as 70 ms. Further, obstacles in the environment altered these corrective responses.
Long latency responses can also be altered to maximize
overall success [24]. Blocks of predominantly resistive
perturbations resulted in an increase in perturbation
Review
Trends in Cognitive Sciences November 2012, Vol. 16, No. 11
Box 1. Electromyographic responses to a perturbation
A mechanical perturbation applied to a limb (or joint) elicits activity in
muscle afferents (and likely some cutaneous afferents), generating a
complex pattern of electromyographic (EMG) activity in muscles that
resist the perturbation (Figure I; for a review, see [22]). This pattern of
activity is generally divided into a number of phases. The earliest
response occurs from 20 to 40 ms (varies somewhat dependent on
distance of muscle from spinal cord), is called the short latency reflex,
and is generated entirely by the spinal cord. Activity after 100ms has
traditionally been viewed as ‘voluntary’, because movement-related
EMG can be generated at this time from visual or somatosensory
stimuli [40]. The time from 50 to 100 ms is termed the long latency
response and involves both spinal and supraspinal feedback.
Stretch responses to mechanical perturbations provide a simple
paradigm to study feedback corrections during voluntary control. The
location of muscle spindles (in muscles!) creates an inherent link
between their activity and motor errors, whereas other sensory
systems are only conditionally involved (which is also interesting for
this very reason). The large diameter of primary muscle-spindle
afferents makes transmission to the spinal cord relatively fast and
suggests that the motor system has put a premium on their role in
rapid feedback processing. There are several factors that need to be
considered when exploring corrective responses:
(i) OFC highlights how feedback depends on the behavioural goal,
thus it is important to define a clear behavioural goal to the
subject, both before and after the perturbation.
(ii) The task-dependent flexibility of stretch responses impacts the
types of perturbations one should apply. Motors are commonly
used to apply perturbations but can be used in two different
modes. In force control, the motor specifies a force and subjects
can counter these perturbations in order to continue to perform a
behavioural task. In motion control, the motor specifies a position
(or velocity), making it impossible to attain a behavioural goal.
Subjects may attempt to compensate, but this is difficult to
measure experimentally.
(iii) The stretch response can be quickly quenched 30 ms following
the offset of a perturbation [69]. Thus, step perturbations that
remain on to the end of the trial allow the full multi-phasic EMG
response to be observed.
(iv) Perturbation magnitude is a balance between large enough to
elicit clear EMG responses, but small enough so that subjects can
continue to perform the behavioural goal. This magnitude effect
may explain why homonymous short-latency responses were
observed in [44], whereas heteronymous responses were found
for large perturbations in [31].
responses in order to maintain task success as compared to
when such perturbations were rare.
Shifts in visual feedback (visual perturbations) provide
a parallel approach to study the use of vision for control.
The influence of target shape was highlighted in a recent
study that used visual shifts in hand position during
reaching [25]. Corrections for rectangular targets oriented
parallel to the movement path elicited larger corrective
responses than corrections for targets oriented perpendicular to the movement path. Subjects can also ignore visual
shifts in hand feedback, if required. Franklin and Wolpert
[9] found that subjects made larger corrective responses
when visual shifts remained to the end of the trial, whereas
smaller corrective responses were elicited if the visual shift
was only transiently present.
Everyday activities require us to use each limb independently (e.g., use a spoon to eat cereal) or to coordinate the
limbs together (e.g., carry a large bowl). As neural circuitry
for controlling each limb is largely confined to separate
halves of the brain, bimanual motor tasks provide an
interesting model to study task-dependent feedback [26].
(v) Background loads prior to the perturbation allow excitatory and
inhibitory responses to be observed. Short latency responses are
also difficult to observe without some background muscle
activity.
[(Box_1)TD$FIG]
Applied step-torque perturbaon
0
50
100
Joint moon
0
100
50
EMG of stretched muscle
Spinal propriocepve feedback
Corcal propriocepve feedback
Visual feedback
Perturbaon Short
onset
latency
epoch
Voluntary epoch
Long
latency
epoch
R3
R1
R2
epoch epoch epoch
0
100
50
Time (ms)
TRENDS in Cognitive Sciences
Figure I. Mechanical perturbation applied to a joint causes joint motion and
a multiphasic electromyographic response in stretched muscles.
When the two limbs are used in a bimanual motor task,
perturbations applied to one limb result in corrective
responses in the other limb [27–29]. Impressively, EMG
changes to the perturbation can be observed in less
than 100 ms for these inter-limb corrections (Figure 3,
[29–31].
The presence of goal-directed corrections touches on an
age-old debate on whether there is a desired trajectory
during tasks such as reaching. Alternative models to OFC
commonly assume that the motor system computes a desired trajectory [20]. If the goal is to put the hand at a spatial
goal, OFC suggests that there is no need for a desired
trajectory. Although there appears to be some importance
placed on initiating relatively straight movements [18], even
in congenitally blind subjects [32], corrective responses do
not return to a desired trajectory. Movements performed in a
curl force field that tends to push the hand sideways result in
over-compensation early in the movement [33]. Why should
the brain produce a greater force than is necessary to
overcome a perturbation at the start of a reaching
movement? Optimal control predicts this pattern as early
543
Review
[(Figure_2)TD$IG]
Trends in Cognitive Sciences November 2012, Vol. 16, No. 11
Hand paths
(a)
y
x
2 cm
(b)
(c)
X posion
0.35
Brachioradialis
15
R1 R2
R3
0.3
EMG (au)
Posion (m)
0.25
0.2
0.15
0.1
10
5
0.05
0
−0.05
0
200
400
600
800 1000
0
-50
0
50
100
150
Time (ms)
Time (ms)
TRENDS in Cognitive Sciences
Figure 2. Corrective responses to mechanical perturbations applied when reaching to a circle or rectangular bar. (a) Black lines are unperturbed reaching movements to each
target. Display of rectangular bar in diagram is clipped on the left, because unperturbed reaches were made roughly to the centre of the bar. Extensor perturbations were applied
to each joint just after movement onset. Corrective responses were directed back to the circular goal (blue lines), whereas corrective responses were redirected along the bar
near the perturbed hand (red lines). (b), (c) Differences in hand corrections between target shapes can be observed in just over 200 ms, with EMG differences in an elbow flexor
observable in 60 ms post-perturbation. Vertical lines denote perturbation onset and dashed lines denote separation of stretch response epochs. Adapted from [23].
compensation is advantageous for producing movements
with minimum effort [23,33].
One could say that there is a nominal trajectory that is
generated by the voluntary motor system if there were no
noise or errors. However, any noise or error generates
slight deviations from the nominal trajectory that leads
to goal-directed corrections, creating a unique trajectory on
each trial. In contrast, if the goal is to draw the letter h,
[(Figure_3)TD$IG]then the trajectory is the goal. Each individual has a
(a)
0
Task selection
The studies described above required subjects to counter a
perturbation and continue to perform a motor task. However, many studies ask subjects to generate another task
after the perturbation, such as giving instructions to
Triceps longus
Brachioradialis
50
100
0
150
Key:
(b)
0
unique way of writing the letters of the alphabet and this
transfers into related control policies to generate that
shape whether drawing using the hand, whole arm, or foot.
50
100
150
Contralateral flexion
Contralateral extension
101
50
100
96
150
Time from perturbaon onset (ms)
0
50
100
150
Time from perturbaon onset (ms)
TRENDS in Cognitive Sciences
Figure 3. Inter-limb corrective responses elicited by perturbations to a single limb. Schematics show EMG recorded from the unperturbed right arm during perturbations of
the left arm. (a) Group averages of normalized EMG in the uncoupled control task when the contralateral arm was perturbed either downward (blue traces) or upward
(orange trace). No statistical change in the either muscle was observed in this task. (b) Response in the coupled single-object task when the two hands controlled a single
bar and the subject was instructed to maintain it in a spatial target that was parallel to the body. Red vertical line indicates the earliest time at which differences in EMG were
observed between the two conditions. Adapted from [29].
544
Review
resist/let go after the perturbation. In effect, the perturbation elicits a change in the control policy, from maintaining a fixed posture to generating a rapid movement to
oppose the load for ‘resist’ or relaxation for ‘let go’. In order
to remove the ambiguity of the behavioural goal inherent
in these verbal instructions, Pruszynski et al. [34] introduced a spatial version of this task in which a perturbation
pushed the hand either into or away from a spatial goal.
Subjects can easily launch goal-directed motor actions
throughout the workspace with corresponding EMG
responses present in 60 ms. Such motor responses are
not entirely preplanned (triggered), given that the response scales with the size of the perturbation [35].
Motor reflexes, such as the stumble corrective response
or flexion/crossed-extension reflexes, are triggered
responses that interrupt an ongoing motor pattern, such
as walking [36]. These corrective responses are only observed for large perturbations or stimuli. Perturbations in
the studies described above were all relatively large in
order to generate clear EMG responses. Although subjects
were still able to attain the behavioural goal, the question
is whether these corrective responses are distinct from
voluntary control, like the stumble corrective response.
This issue was addressed by observing EMG responses
to multi-joint perturbations of varying magnitude [37].
Significant muscle responses were observed, even when
perturbations generated extremely small changes in angular motion that approached the natural range of variability observed during unperturbed trials. The use of
linear regressions found that the scaling between perturbation magnitude and EMG activity passed through the
origin. This suggests that the perturbations used in most
studies cause corrective responses that reflect feedback
processing that is also present during unperturbed voluntary motor actions.
The ability of mechanical perturbations to launch
impending motor plans provides an interesting way to probe
decisional processes. For example, a recent study examined
stretch responses as subjects judged the direction of motion
in a random dot motion display [38]. The authors probed the
state of the motor system by perturbing the arm at random
times during decision formation and found that long-latency
responses were modulated by the strength and duration of
the motion pattern prior to their decision to respond to the
stimuli. Long-latency responses can also be quickly updated
to new spatial goals [39,40]. For example, long latency
responses can be modified in as little as 100 ms following
the presentation of a spatial goal [40]. Interestingly,
responses to mechanical perturbations applied 100 ms after
the shift in the target during reaching also result in a
corresponding change in the corrective response, including
modulation of short latency spinal responses [39]. Thus,
shifts in the behavioural goal lead to a shift in movement
direction and corresponding changes in feedback corrections, both now directed towards the new spatial goal. These
studies highlight the intimate link between decisional processes and motor control [41,42].
Limb and environment
The physics of limb movement are complex as the activity
of a muscle spanning a joint can lead to motion at multiple
Trends in Cognitive Sciences November 2012, Vol. 16, No. 11
joints. The voluntary motor system considers these interactions [43], but what about feedback corrections? This
question was addressed by applying loads to the shoulder
and elbow, yielding particular motion patterns at these two
joints [44]. In one experiment, pure shoulder torque or pure
elbow torque perturbations were applied, resulting in the
same shoulder motion but different elbow motion. Perturbation-related activity of mono-articular shoulder muscles
was examined to see whether they reflected the shoulder
motion or the underlying shoulder torque. The short-latency response was the same in both conditions, indicating
that it responded only to local joint motion. In contrast, the
long-latency response considered the motion at both joints,
generating a larger response for the shoulder torque perturbation than the elbow torque perturbation. In a second
experiment, combined shoulder and elbow torque perturbations were applied generating no motion at the shoulder
and either flexion or extension motion at the elbow
(Figure 4a). If the stretch response of shoulder muscles
account for the limb’s mechanical properties, they should
still respond to this perturbation, even though the joint is
not moving. Again, short-latency response of the monoarticular shoulder extensor muscle (posterior deltoids)
mirrored the local joint motion. Importantly, there was a
robust long-latency response that appropriately countered
the underlying torque perturbation (Figure 4b). Knowledge of the physics of the limb is also used for online
corrections during reaching for mechanical perturbations
[45], visual jumps in target location [46], and even motor
planning [47].
The physical environment alters the forces necessary to
move our bodies, whether it is an object we are holding or
the viscous forces generated when moving underwater.
There is a large literature demonstrating that the voluntary motor system can quickly adapt to changes in the
visual or mechanical environment [4,48]. Evidence is now
mounting that corrective responses are also altered during
these adaptations. Burdet et al. [49] and others [50,51]
imposed a diverging force field on reaching movements and
found that, with practice, corrective responses during the
reach increased along an axis that was parallel to the
diverging field. When people reached in a curl field in
which the forces pushed the hand in only one direction,
perturbation responses increased and reflected the properties of the load [52–55]. If a perturbing force is present
only for a portion of the movement, long latency responses
are modified even before the hand enters that region
[56,57]. Adaptation to novel loads result in changes in long
latency responses prior to movement during the preparatory period [58]. If the force field perturbs the hand, but this
perturbation is irrelevant to task success (i.e., the cursor
moves to the target in any case), corrective responses are
decreased during the reach [59]. Finally, visual feedback
corrections are also modified when adapting to novel mechanical loads [60].
Neural implementation
Spinal and transcortical feedback
As a theory to describe behaviour, OFC is agnostic as to
how voluntary motor control is generated by the spinal
cord and brain. The spinal cord provides the first level of
545
Review
[(Figure_4)TD$IG]
Trends in Cognitive Sciences November 2012, Vol. 16, No. 11
(c)
(a)
te
80
te
Excitatory
shoulder
torque
ts
40
ts
No shoulder Moon
Inhibitory
shoulder Torque
0
x091007a
Perturbaon
onset
(d)
(b)
60
Perturbaon
onset
Post. deltoid
EMG
R1
R2/R3
1au
Firing rate (Hz)
50
40
Responding to
underlying torque
30
20
10
0
50ms
∼50 100
Time (ms)
200
TRENDS in Cognitive Sciences
Figure 4. Long-latency response accounts for limb dynamics. (a) Subjects were instructed to maintain their hand at a central target and step-torque perturbations were
randomly applied to both the shoulder and elbow (flexor torques at both joints, red, or extensor torques at both joints, blue). Shoulder and elbow torque magnitudes and
arm geometry were chosen so that they caused substantial elbow motion but minimal shoulder motion. (b) Muscle activity aligned on perturbation onset. Note that even
though the shoulder muscle was neither stretched nor shortened by the mechanical perturbation, there is still a robust long-latency response (both excitatory and
inhibitory) in the shoulder extensor muscle. Adapted from [44]. (c), (d) Response of an exemplar shoulder-like M1 neuron (c) and the population of shoulder-like neurons (d)
to a mechanical perturbation that causes pure elbow motion. Same color scheme as in (a). Note that the shoulder-like neurons respond to the underlying shoulder torque,
even though the local motion information from the shoulder is ambiguous. Adapted from [71].
feedback processing. In many species, spinal circuits support sophisticated control, such as scratching, wiping, and
basic locomotor patterns [36,61,62]. Analyses of these
movements highlight many of the key characteristics predicted by OFC, notably success with variability. The spinal
cord also provides phase-dependent feedback in humans
during cyclical tasks such as locomotion and even hand
cycling [63,64]. Recent work also highlights the importance
of spinal circuits for transmitting descending commands
during voluntary motor actions [65], although the specific
computational processes provided at this level remains
unknown.
The short-latency stretch response is generated only by
spinal feedback. Its size is relatively small and can only be
modulated over days or weeks with training [66]. Shortlatency spinal responses are modified by background load
due to the effects of motoneuron recruitment, but this is not
necessarily a beneficial quality [67]. Sensory gating also
occurs at the spinal cord prior to and during movement
[68]. Stretch responses can be quickly halted 30 ms after
perturbations are removed, demonstrating a powerful inhibitory capability of the spinal cord to quench stretch
responses [69].
Although spinal feedback certainly contributes to longlatency responses, a transcortical feedback pathway starts
to contribute at this time. Several studies have demonstrated that TMS used on primary motor cortex influences
546
feedback corrections during the long-latency time period
[54,56,70,71]. In non-human primates, many studies in the
1970s and early 1980s demonstrated how primary motor
cortex (M1) neurons respond to mechanical perturbations
or passive limb motion (for reviews, see [3,72]). A classic
study by Evarts and Tanji [73] examined neural processing
in M1 in a variant of the intervene/do not intervene task by
training monkeys to respond to a mechanical perturbation
by either pulling or pushing the perturbing handle. Early
activity in M1 (20-50 ms post-perturbation) reflected the
applied perturbation, but was quickly followed by the
appropriate motor response at 50 ms.
M1 neurons are directionally tuned to mechanical perturbations [74] in the same manner that activity is broadly
tuned to various parameters of movement during reaching
[75] or load-related activity during posture [76]. The impact of sensory feedback on M1 activity has also recently
been demonstrated by comparing how encoding models of
M1 activity depend on the ongoing movements of the limb
[77]. In this study, an encoding model was generated from
M1 activity when the monkey made reaching movements
to spatial targets. Subsequent performance of the monkey
on the use of the model to move a cursor to spatial targets
was best when the monkey was also moving its hand to the
spatial targets, but degraded somewhat when the limb
remained stationary and was worst when the limb was
randomly moved around the workspace. Thus, changes in
Review
afferent signals influenced ongoing activity in M1. The
impact of sensory feedback on M1 de-emphasizes ideas
of neural coding and emphasizes neural dynamics when
interpreting M1 function [78].
A recent study examined whether the transcortical
pathway through M1 considers the mechanical properties
of the limb [71] using the paradigm developed by Kurtzer
and colleagues [44]. Unlike muscles that have clear anatomical actions, the study identified shoulder and elbowrelated M1 neurons based on their response to steady-state
motor outputs as the monkey countered various combinations of shoulder and elbow loads [74]. The earliest response, 20 to 50 ms after perturbation onset, did not
distinguish between the various loading conditions, similar to the default response observed in Evarts and Tanji
[73]. However, at 50ms the shoulder-related neurons rapidly increased their firing for the appropriate load at the
shoulder, approximately 20 ms before shoulder muscles
appropriately responded to the applied load (Figure 4c
and d). Thus, in M1 (or elsewhere in this pathway), elbow
and shoulder motion are integrated to identify the underlying muscular torque that will counter the applied load.
Some models of voluntary control focus attention on
spinal feedback [14,15]. However, the observation that
virtually all sophisticated stretch responses occur during
the long- and not the short-latency time period suggests
that supraspinal feedback must be important for voluntary
control. A shift to higher motor centres may have occurred
as the complexity and breadth of motor skills increased
through evolution. Spinal circuits may provide good control, but only for a limited range of behaviours, such as
quadrupedal locomotion, scratching, and other stereotyped
behaviours. As animals, notably primates, began to increase their breadth of motor skills, such as goal-directed
reaching for objects in the environment and social grooming, and increased the use of vision for control, the voluntary motor system may have required a more general,
highly-flexible approach to control, increasing the size
and importance of corticocerebellar circuits. One reason
for this shift to supraspinal control may reflect the importance of prediction in optimal state estimation, because the
use of efference copies for control requires knowledge of the
properties of the limb, contact forces, and prior experience
(but see, [79]).
Mapping OFC onto brain circuits
OFC-like control is speculated as being generated by cortical and sub-cortical brain circuits [21,80]. Figure 1b provides an overview of the putative contributions of various
brain regions to voluntary control. It is at best a sketch that
will evolve as experimentation helps to delineate further
the contribution of different brain regions to control. The
three basic processes, task selection, control policy, and
state estimation, are each generated by highly distributed
cortical circuits. The basal ganglia participate in task
selection through connections with associated cortical
regions and the cerebellum participates in optimal state
estimation through associated connections with cortex
[81–83]. The diagram highlights that many cortical regions
support each process and a given cortical region likely
participates in two or even all three processes. Interactions
Trends in Cognitive Sciences November 2012, Vol. 16, No. 11
between the three processes can occur between connected
cortical regions within a given process or within a cortical
region that is involved in multiple processes. There are also
direct projections from basal ganglia and cerebellum to M1.
From the perspective of supraspinal circuits, the spinal
cord may be viewed as part of the peripheral motor apparatus modifying the force-length and force-velocity relationships of the limb, simplifying the properties of the
musculoskeletal system [84]. More complex spinal processes certainly exist. However, it seems unlikely that all the
complexities observed for spinal processing during cyclical
behaviours of the hindlimb of non-primates are engaged
during voluntary motor actions in the upper limb of
humans. Regardless, the prediction is that supraspinal
circuits recognize and compensate for spinal complexities
much like they recognize and compensate for properties of
the musculoskeletal system.
The model emphasizes the distributed nature of feedback control much like decisional processes are presumed
to involve both frontal and parietal circuits [42,85]. Transcortical feedback for stretch responses has been assumed to
reflect a relatively simple pathway of sensory transmission
from primary somatosensory cortex to primary motor cortex. However, state estimation is complex, involving the
conversion of efferent copy signals into estimates of the
present position of the limb, as well as integration of
delayed sensory feedback from visual, muscle, and cutaneous afferents. These processes likely involve both cerebellar and parietal cortical regions. A key challenge for
research is to understand how different feedback pathways
contribute to state estimation during online control. Sensorimotor integration has been examined predominantly
during motor planning before the initiation of movement.
Does sensory integration observed prior to movement reflect how such processes occur during motor action? Are all
signals integrated in a single representation that represents the present state of the system or is each signal
processed individually and signals only converge in motor
areas such as M1?
Present models emphasize a serial organization across
cortical regions with descending control principally generated by M1. However, M1 provides only 40% of the axons in
the corticospinal tract [86]. Premotor and somatosensory
cortex also directly influence spinal processing. Given that
the spinal cord is viewed in the present model as part of the
plant, one can consider operations such as modifying sensory processing in the spinal cord and modifications of
gamma motoneuron activity as essential features of control. We need a broader view of control beyond the generation of EMG patterns.
Concluding remarks
The use of OFC to interpret voluntary movement has
emphasized the importance of sensory feedback for control.
Visual and mechanical perturbations during motor behaviour constitute an important scientific approach to probe
the sophistication of feedback processing, providing a window into voluntary control processes. Recent studies reveal
that corrective responses are highly adaptable based on the
behavioural goal and consider the many complexities inherent in multi-joint movements. The basic computational
547
Review
Box 2. Questions for future research
How is OFC-like control generated without the mathematical
formalisms? The brain clearly does not use the mathematical
formalisms of OFC and a major challenge is to understand how
learning and evolution can generate OFC-like behavior.
How is OFC-like control generated by the brain? Figure 1b
identifies which brain regions may contribute to a specific
process, but not how. State estimation is a complex process
involving forward models to estimate the present state from
efference copy and delayed sensory signals, and integration
across these multiple sources of information. How do the many
feedback pathways in the brain generate state estimation? Even a
single sensory modality has multiple feedback pathways or
processes [87].
What is the role of spinal feedback during voluntary control?
Spinal feedback is clearly important in cyclical behaviours, but
appears to be more limited during voluntary control. The ability to
record spinal neurons in non-human primates during voluntary
behaviour provides an important method to address this question
[88,89]. Also important to consider is evidence of predictive
signals in the firing patterns of muscle spindle afferents during
human movement [90], which suggests that this sensory system
provides more than just muscle length and velocity feedback.
processes in OFC – task selection, control policy and state
estimation – provide an interesting way to interpret the
contributions of brain regions to voluntary control, guiding
future research on the neural basis of voluntary motor
action (Box 2).
Acknowledgements
This work was supported by grants from the National Science and
Engineering Research Council of Canada, Canadian Institutes of Health
Research (CIHR), and a GlaxoSmithKlein-CIHR chair in Neurosciences.
The author would like to thank members of the Limb Lab for helpful
feedback on this article.
References
1 Sherrington, C.S. (1910) Flexion-reflex of the limb, crossed extensionreflex, and reflex stepping and standing. J. Physiol. 40, 28–121
2 Powers, W.T. (2008) Living Control Systems III: The Fact of Control,
Benchmark Publications
3 Scott, S.H. (2008) Inconvenient truths about neural processing in
primary motor cortex. J. Physiol. 586, 1217–1224
4 Franklin, D.W. and Wolpert, D.M. (2011) Computational mechanisms
of sensorimotor control. Neuron 72, 425–442
5 Todorov, E. and Jordan, M.I. (2002) Optimal feedback control as a
theory of motor coordination. Nat. Neurosci. 5, 1226–1235
6 Todorov, E. (2006) Optimal control theory. In Bayesian Brain:
Probabilistic Approaches to Neural Coding (Doya, K. et al., eds), pp.
269–298, MIT Press
7 Faisal, A.A. et al. (2008) Noise in the nervous system. Nat. Rev.
Neurosci. 9, 292–303
8 Li, W. and Todorov, E. (2007) Iterative linearization methods for
approximately optimal control and estimation of non-linear
stochastic systems. Int. J. Control 80, 1439–1453
9 Franklin, D.W. and Wolpert, D.M. (2008) Specificity of reflex
adaptation for task-relevant variability. J. Neurosci. 28, 14165–14175
10 Nagengast, A.J. et al. (2009) Optimal control predicts human
performance on objects with internal degrees of freedom. PLoS
Comput. Biol. 5, e1000419
11 Valero-Cuevas, F.J. et al. (2009) Structured variability of muscle
activations supports the minimal intervention principle of motor
control. J. Neurophysiol. 102, 59–68
12 Stevenson, I.H. et al. (2009) Bayesian integration and non-linear
feedback control in a full-body motor task. PLoS Comput. Biol. 5,
e1000629
13 Dingwell, J.B. et al. (2010) Do humans optimally exploit redundancy to
control step variability in walking? PLoS Comput. Biol. 6, e1000856
548
Trends in Cognitive Sciences November 2012, Vol. 16, No. 11
14 Feldman, A.G. and Levin, M.F. (2009) The equilibrium-point
hypothesis–past, present and future. Adv. Exp. Med. Biol. 629, 699–
726
15 Loeb, G.E. (2012) Optimal isn’t good enough. Biol. Cybern. http://
dx.doi.org/10.1007/s00422-012-0514-6
16 Friston, K. (2011) What is optimal about motor control? Neuron 72,
488–498
17 Hudson, T.E. et al. (2010) Compensation for changing motor
uncertainty. PLoS Comput. Biol. 4, e1000982
18 Kistemaker, D.A. et al. (2010) The central nervous system does not
minimize energy cost in arm movements. J. Neurophysiol. 104, 2985–
2994
19 De Rugy, A. et al. (2012) Muscle coordination is habitual rather than
optimal. J. Neurosci. 23, 7384–7391
20 Mohan, V. and Morasso, P. (2011) Passive motion paradigm: an
alternative to optimal control. Front. Neurobot. 5, 4
21 Scott, S.H. (2004) Optimal feedback control and the neural basis of
volitional motor control. Nat. Rev. Neurosci. 5, 534–546
22 Pruszynski, J.A. and Scott, S.H. (2012) Optimal feedback control and
the long-latency stretch response. Exp. Brain Res. 218, 341–359
23 Nashed, J.Y. et al. (2012) Influence of the behavioural goal and
environmental obstacles on rapid feedback responses. J.
Neurophysiol. 108, 999–1009
24 Cheng, E.J. and Loeb, G.E. (2008) On the use of musculoskeletal
models to interpret motor control strategies from performance data.
J. Neural Eng. 5, 232–253
25 Knill, D.C. et al. (2011) Flexible, task-dependent use of sensory
feedback to control hand movements. J. Neurosci. 31, 1219–1237
26 Diedrichsen, J. et al. (2010) The coordination of movement: optimal
feedback control and beyond. Trends Cogn. Sci. 14, 31–39
27 Diedrichsen, J. (2007) Optimal task-dependent changes of bimanual
feedback control and adaptation. Curr. Biol. 17, 1675–1679
28 Diedrichsen, J. and Gush, S. (2009) Reversal of bimanual feedback
responses with changes in task goal. J. Neurophysiol. 101, 283–288
29 Dimitriou, M. et al. (2012) Task-dependent coordination of rapid
bimanual motor responses. J. Neurophysiol. 107, 890–901
30 Mutha, P.K. and Sainburg, R.L. (2009) Shared bimanual tasks
elicit bimanual reflexes during movement. J. Neurophysiol. 102,
3142–3155
31 Manning, C.D. et al. (2012) Proprioceptive reaction times and longlatency reflexes in humans. Exp. Brain Res. 221, 155–166
32 Sergio, L.H. and Scott, S.H. (1998) Hand and joint paths during
reaching movements with and without vision. Exp. Brain Res. 122,
157–164
33 Izawa, J. et al. (2008) Motor adaptation as a process of reoptimization.
J. Neurosci. 28, 2883–2891
34 Pruszynski, J.A. et al. (2008) Rapid motor responses are appropriately
tuned to the metrics of a visuospatial task. J. Neurophysiol. 100, 224–
238
35 Pruszynski, J.A. et al. (2011) The long-latency reflex is composed of at
least two functionally independent processes. J. Neurophysiol. 106,
449–459
36 Rossignol, S. et al. (2006) Dynamic sensorimotor interactions in
locomotion. Physiol. Rev. 86, 89–154
37 Crevecoeur, F. et al. (2012) Fast corrective responses are evoked by
perturbations approaching the natural variability of posture and
movement tasks. J. Neurosphysiol. 107, 2821–2832
38 Selen, L.P.J. et al. (2012) Deliberation in the motor system: reflex gains
track evolving evidence leading to a decision. J. Neurosci. 32, 2276–
2286
39 Mutha, P.K. et al. (2008) Visual modulation of proprioceptive reflexes
during movement. Brain Res. 1246, 4–69
40 Yang, L. et al. (2011) Rapid motor responses quickly integrate
visuospatial task constraints. Exp. Brain Res. 211, 231–242
41 Wolpert, D.M. and Landy, M.S. (2012) Motor control is decision-making.
Curr. Opin. Neurobiol. http://dx.doi.org/10.1016/j.conb.2012.05.003
42 Cisek, P. (2012) Making decisions through a distributed consensus.
Curr. Opin. Neurobiol. http://dx.doi.org/10.1016/j.conb.2012.05.007
43 Gribble, P.L. and Ostry, D.J. (1999) Compensation for interaction
torques during single- and multijoint limb movements. J.
Neurophysiol. 82, 2310–2326
44 Kurtzer, I.L. et al. (2008) Long-latency reflexes of the human arm
reflect an internal model of limb dynamics. Curr. Biol. 18, 449–453
Review
45 Kurtzer, I. et al. (2009) Long-latency responses during reaching
account for the mechanical interaction between the shoulder and
elbow joints. J. Neurophysiol. 102, 3004–3015
46 Gritsenko, V. et al. (2009) Integration of predictive feedforward and
sensory feedback signals for online control of visually guided
movement. J. Neurophysiol. 102, 914–930
47 Cos, I. et al. (2011) The influence of predicted arm biomechanics on
decision making. J. Neurophysiol. 105, 3022–3033
48 Shmeulof, L. and Krakauer, J.W. (2011) Are we ready for a natural
history of motor learning? Neuron 72, 469–476
49 Burdet, E. et al. (2001) The central nervous system stabilizes unstable
dynamics by learning optimal impedance. Nature 414, 446–449
50 Franklin, D.W. et al. (2007) Endpoint stiffness of the arm is
directionally tuned to instability in the environment. J. Neurosci.
27, 7705–7716
51 Wong, J. et al. (2009) The influence of visual perturbations on the
neural control of limb stiffness. J. Neurophysiol. 101, 246–257
52 Wang, T. et al. (2001) Learning the dynamics of reaching movements
results in the modification of arm impedance and long-latency
perturbation responses. Biol. Cybern. 85, 437–448
53 Wagner, M.J. and Smith, M.A. (2008) Shared internal models for
feedforward and feedback control. J. Neurosci. 28, 10663–10673
54 Shemmell, J. et al. (2009) The differential role of motor cortex in stretch
reflex modulation induced by changes in environmental mechanics and
verbal instruction. J. Neurosci. 29, 13255–13263
55 Krutky, M.A. et al. (2010) Interactions between limb and
environmental mechanics influence stretch reflex sensitivity in the
human arm. J. Neurophysiol. 103, 429–440
56 Kimura, T. et al. (2006) Transcranial magnetic stimulation over
sensorimotor cortex disrupts anticipatory reflex gain modulation for
skilled action. J. Neurosci. 26, 9272–9281
57 Kimura, T. and Gomi, H. (2009) Temporal development of anticipatory
reflex modulation to dynamical interactions during arm movement. J.
Neurophysiol. 102, 2220–2231
58 Ahmadi-Pajouh, M.A. et al. (2012) Preparing to reach: selecting an
adaptive long-latency feedback controller. J. Neurosci. 32, 9537–9545
59 Scheidt, R.A. et al. (2005) Interaction of visual and proprioceptive
feedback during adaptation of human reaching movements. J.
Neurophysiol. 93, 3200–3213
60 Franklin, S. et al. (2012) Visuomotor feedback gains upregulate during
the learning of novel dynamics. J. Neurophysiol. 108, 467–478
61 Stein, P.S. (2005) Neuronal control of turtle hindlimb motor rhythms.
J. Comp. Physiol. A 191, 213–229
62 Frigon, A. (2012) Central pattern generators of the mammalian spinal
cord. Neuroscientist 18, 56–69
63 Zehr, E.P. and Duysens, J. (2004) Regulation of arm and leg movement
during human locomotion. Neuroscientist 10, 347–361
64 Zehr, E.P. et al. (2012) Neural control of rhythmic arm cycling after
stroke. J. Neurophysiol. 108, 891–905
65 Alstermark, B. and Isa, T. (2012) Circuits for skilled reaching and
grasping. Annu. Rev. Neurosci. 35, 559–5788
66 Wolpaw, J.R. (2010) What can the spinal cord teach us about learning
and memory? Neuroscientist 16, 532–549
67 Pruszynski, J.A. et al. (2009) Temporal evolution of ‘Automatic GainScaling’. J. Neurophysiol. 102, 992–1003
68 Seki, K. and Fetz, E.E. (2012) Gating of sensory input at spinal and
cortical levels during preparation and execution of voluntary
movement. J. Neurosci. 32, 890–902
Trends in Cognitive Sciences November 2012, Vol. 16, No. 11
69 Kurtzer, I. et al. (2010) Long-latency and voluntary responses to an arm
displacement can be rapidly attenuated by perturbation offset. J.
Neurophysiol. 103, 3195–3204
70 Perreault, E.J. et al. (2008) Interactions with compliant loads alter
stretch reflex gains but not intermuscular coordination. J.
Neurophysiol. 99, 2101–2113
71 Pruszynski, J.A. et al. (2011) Primary motor cortex underlies multijoint integration for fast feedback control. Nature 478, 387–390
72 Hatsopoulos, N.G. and Suminski, A.J. (2011) Sensing with the motor
cortex. Neuron 72, 477–487
73 Evarts, E.V. and Tanji, J. (1976) Reflex and intended responses in
motor cortex pyramidal tract neurons of monkey. J. Neurophysiol. 39,
1069–1080
74 Herter, T.M. et al. (2009) Comparison of neural responses in primary
motor cortex to transient and continuous loads during posture. J.
Neurophysiol. 101, 150–163
75 Kalaska, J.F. (2009) From intention to action: motor cortex and the
control of reaching movements. Adv. Exp. Med. Biol. 629, 139–178
76 Herter, T.M. et al. (2007) Characterization of torque-related activity in
primary motor cortex during a multi-joint postural task. J.
Neurophysiol. 97, 2887–2899
77 Suminski, A.J. et al. (2010) Incorporating feedback from multiple
sensory modalities enhances brainmachine interface control. J.
Neurosci. 30, 16777–16787
78 Churchland, M.M. et al. (2012) Neural population dynamics during
reaching. Nature 487, 51–56
79 Hantman, A.W. and Jessell, T.M. (2010) Clarke’s column neurons as
the focus of a corticospinal corollary circuit. Nat. Neurosci. 13, 1233–
1239
80 Shadmehr, R. and Krakauer, J.W. (2008) A computational
neuroanatomy for motor control. Exp. Brain Res. 185, 359–381
81 Strick, P.L. et al. (2009) Cerebellum and nonmotor function. Annu. Rev.
Neurosci. 32, 413–434
82 Bostan, A.C. and Strick, P.L. (2010) The cerebellum and basal ganglia
are interconnected. Neuropsychol. Rev. 20, 261–270
83 Prevosto, V. et al. (2010) Cerebellar inputs to intraparietal cortex areas
LIP and MIP: functional frameworks for adaptive control of eye
movements, reaching, and arm/eye/head movement coordination.
Cereb. Cortex 20, 214–228
84 Nichols, T.R. and Houk, J.C. (1976) Improvement in linearity and
regulation of stiffness that results from actions of stretch reflexes. J.
Neurophysiol. 39, 119–142
85 Cisek, P. and Kalaska, J.F. (2010) Neural mechanisms for interacting
with a world full of action choices. Annu. Rev. Neurosci. 33, 269–298
86 Porter, R. and Lemon, R.N. (1993) Corticospinal Function and
Voluntary Movement, Oxford University Press
87 Gritsenko, V. and Kalaska, J.F. (2010) Rapid online correction is
selectively suppressed during movement with a visuomotor
transformation. J. Neurophysiol. 104, 3084–3104
88 Takei, T. and Seki, K. (2010) Spinal interneurons facilitate coactivation
of hand muscles during a precision grip task in monkeys. J. Neurosci.
30, 17041–17050
89 Shalit, U. et al. (2012) Descending systems translate transient cortical
commands into a sustained muscle activation signal. Cereb. Cortex 22,
1904–1914
90 Dimitriou, M. and Edin, B.B. (2010) Human muscle spindles act as
forward sensory models. Curr. Biol. 20, 1763–1767
549