Download n e w s a n d ...

npg
© 2015 Nature America, Inc. All rights reserved.
news and views
the usage of specific splice sites. Specifically,
alternative splice isoforms upregulated after
cocaine were more likely to be associated
with increased 5hmC at the corresponding
splice site, whereas 5hmC at splice sites
was more likely to be reduced for isoforms
downregulated after cocaine (Fig. 1).
These data provide functional evidence
that 5hmC may regulate splice site usage,
which will be an important area for future
investigation.
The last question the authors asked is
whether changes in 5hmC contribute to the
persistent changes in NAc physiology that
are both induced by chronic cocaine and
relevant to addiction. The authors found
a significant global correlation between
genes that showed increased 5hmC following repeated cocaine and those that showed
enhanced steady-state expression 24 h after
withdrawal from chronic cocaine. The correlation with increased 5hmC was even stronger for genes that were induced by a cocaine
challenge. These data therefore indicate
that 5hmC levels not only reflect current
transcriptional states, but also predict the
potential for genes to turn on in response to
a future stimulus. Finally, the authors demonstrated that, at least for a subset of genes,
both mRNA induction and cocaine-induced
changes in 5hmC can persist for at least 1
month after the cessation of cocaine exposure. Thus, rather than being just an intermediate in the demethylation of DNA, these
data support a model of 5hmC as a meaningful epigenetic mark of its own, with potential
functions in the maintenance of transcriptional memory.
This work by Feng et al.1 underscores the
importance of epigenetic mechanisms of chromatin regulation in the long-lasting changes
in neuronal gene expression that are induced
by chronic cocaine. Furthermore, their findings demonstrate the power of genome-level
sequencing techniques to open new windows
of understanding into the mechanisms of
neuronal adaptation. The challenge for the
future will be to distill the detailed chromatin
landscape revealed here into a set of principles
for gene regulation that will better link
molecular mechanism via cellular function to
the maladaptive circuit changes that underlie
drug addiction.
COMPETING FINANCIAL INTERESTS
The author declares no competing financial interests.
1. Feng, J. et al. Nat. Neurosci. 18, 536–544
(2015).
2. Lister, R. et al. Science 341, 1237905 (2013).
3. Kriaucionis, S. & Heintz, N. Science 324, 929–930
(2009).
4. Tahiliani, M. et al. Science 324, 930–935
(2009).
5. Koh, K.P. et al. Cell Stem Cell 8, 200–213 (2011).
6. Ito, S. et al. Nature 466, 1129–1133 (2010).
7. Dawlaty, M.M. et al. Dev. Cell 24, 310–323
(2013).
8. Szulwach, K.E. et al. Nat. Neurosci. 14, 1607–1616
(2011).
9. Rudenko, A. et al. Neuron 79, 1109–1122
(2013).
10.Li, X. et al. Proc. Natl. Acad. Sci. USA 111,
7120–7125 (2014).
11.Kaas, G.A. et al. Neuron 79, 1086–1093
(2013).
12.Wen, L. et al. Genome Biol. 15, R49 (2014).
Carrot or stick in motor learning
Dagmar Sternad & Konrad Paul Körding
A study shows that reward and punishment have distinct influences on motor adaptation. Punishing mistakes
accelerates adaptation, whereas rewarding good behavior improves retention.
We both love salsa dancing, but learning
salsa is not easy. When one partner misses
a step, the other may punish him with a
frown, but when he masters a new move, her
praise rewards him—or does it make him
complacent? Carrot or stick: the manner
by which reward and punishment affects
motor learning is a long-standing question
in education, sports, therapy and beyond.
In this issue of Nature Neuroscience, Galea
et al.1 address this question using a simple reaching task in a perturbed visual
environment.
Dagmar Sternad is in the Departments of Biology,
Electrical and Computer Engineering, and Physics,
and the Center for the Interdisciplinary Research on
Complex Systems, Northeastern University, Boston,
Massachusetts, USA, and Konrad Paul Körding
is in the Sensory Motor Performance Program,
Rehabilitation Institute of Chicago, Chicago, Illinois,
USA, and the Departments of Physical Medicine
and Rehabilitation, and Physiology, Northwestern
University, Chicago, Illinois, USA.
e-mail: [email protected] or [email protected]
480
The authors build on previous studies that
have shown the crucial importance of reward
on retention2,3, but they now contrast the
effect of reward with that of punishment by
differentiating their effects on acquisition rate
and retention. This study examined participants moving a cursor to targets displayed on
a screen, steering with their hand movements
hidden from view. To create a learning challenge, the cursor position was rotated by 30
degrees and participants had to practice to
successfully reach the target. This exercise is
similar to moving a computer mouse when you
turn it upside down: a challenge that one masters with practice. The specific question of this
paper was how money received for good performance (reward) or lost for bad performance
(punishment) would affect the rate of learning
and the retention of the acquired performance.
The authors found that punishment accelerated the rate of adaptation, whereas reward
improved retention of the new mapping.
This study is at the crossroads of at least
three research disciplines that have examined the consequences of motivational and
error feedback on motor performance:
neuroscience, computational science and, of
course, psychology (Fig. 1). These fields have
approached this issue in different ways and
each can inform and motivate future directions in motor control.
In psychology, reward and punishment
have long been recognized as instrumental for
learning. As early as 1898, Edward Thorndike’s
law of effect stated that if a response leads to a
“satisfying state of affairs” it will be strengthened and, conversely, if it leads to unpleasant
consequences it will be weakened4. The thesis
that reward is a better motivator than punishment was also at the core of B.F. Skinner’s
principle of reinforcement. Operant conditioning developed systematic reinforcement
schedules to enhance learning and thereby
shape behavior5. However, social psychologists
have also reminded us that human nature is
far more nuanced and more than a collection
of systematically reinforced associations. Many
studies have highlighted the mediating effects
of emotions, such as threat, anxiety, pride or
shame, on behavior. Invoking stereotypes, such
as inferior performance of females in mathematics or athletics, lowers test performance.
volume 18 | number 4 | april 2015 nature neuroscience
news and views
Neuroscience
Computational science
≈
≈
Motor learning
≈
npg
© 2015 Nature America, Inc. All rights reserved.
Psychology
Figure 1 Motor control is at the intersection of three fields: psychology, neuroscience and
computational science. The metaphors for understanding human behavior differ across fields:
neuroscience focuses on the brain, computational science stresses the robot analogy, and psychology
addresses behavioral, cognitive and social processes.
There are also inter-individual differences:
some individuals thrive under competitive
stress, whereas others choke under pressure6.
Motivation, risk and reward, as well as individual differences, critically affect learning.
Psychology has therefore highlighted
many facets of learning that research on motor
control has thus far neglected7. Galea et al.1
now present a simple approach for probing the
motivational effect of two long-known factors
affecting learning and retention: reward and
punishment. Taking a closer look at the various
psychological determinants and incorporating
them into experiments and learning models
may be an important direction for future
motor control research.
Much research in neuroscience has focused
on the brain areas and neurons involved in the
various aspects of motor learning, including
reward and punishment. For example,
we know from imaging studies that many
cortical and subcortical structures are affected
by positive and negative reinforcement8.
For example, some neurons in the ventral tegmental area can be tuned to reward and others
to punishment9. With its focus on mechanism,
neuroscience provides a knowledge base
for formulating models and contributes to our
understanding that reward and punishment
have multifaceted effects on motor learning.
Neuroscience offers many opportunities for
the behavioral study of motor learning, as it
reveals targets for interventions and mechanisms to interpret behavior. In this spirit, the
study by Galea et al.1 attributes reward-based
learning to the motor cortex and learning
enhanced by negative reinforcement to the
cerebellum. The interpretation of cerebellar
learning from negative feedback, however, is very
indirect, and it was recently shown that errors
can overpower rewards10. Although Galea et al.
provide an example, there is great potential
for further connections between behavior
and neuroscience. First, targeted interventions
via transcranial magnetic stimulation can
reveal the contribution of the respective brain
area. Lesion studies can establish causality.
For example, if lesioning the cerebellum
largely abolishes fast eye movement adaptation, then that would be evidence of its causal
role11. Second, extending current mathematical models of plasticity in the nervous system
promises to make precise testable predictions
of behavior. It is important for motor control
to take input from neuroscience and match
the current simple learning models to the new
insights from the nervous system.
In computational science, humans are typically viewed as problem solvers. For a given
task, the human learns to compute the optimal solution, just as we would program a
robot to perform this task. This perspective
has led scientists to employ algorithms such
as Kalman filters to describe processes underlying motor learning12. Describing humans
as optimal problem solvers implies that the
outcome, such as winning the jackpot or paying penalties, should be irrelevant for motor
learning. After all, reward and punishment
contain, in principle, the same information.
Consequently, modeling approaches have
ignored, if not rejected, the influence of factors such as reward and punishment. Learning
is regarded as an iterative process of correcting
errors and optimizing cost functions, such as
energy, effort or time spent. Computational
nature neuroscience volume 18 | number 4 | april 2015
motor control has produced a range of models that explain data, but psychological factors
clearly cannot be ignored.
Computational science can contribute a
broad set of modeling and data analysis techniques to motor control. The simple, highly
standardized task of reaching in a remapped
virtual environment affords precise manipulations, measurements and mathematical
modeling. The manageable nature of the data
facilitates differentiating the effects of positive
and negative reinforcement during learning
and retention. There is an opportunity to further disentangle how these effects interact with
other known forms of motor learning. Can the
findings be described by interacting explicit
and implicit processes13? How does motivation affect learning over different time­scales14?
How can we construct models that apply across
a broad set of experiments15? What is the effect
of a given reward size on the stability of motor
memories? Computational motor control is
now poised to start exploring such additional
determinants for motor learning.
Coming back to salsa dancing: should he
praise her successful spin? Should she frown
when he does not finish the turn in time?
The study by Galea et al.1 suggests an answer,
although the simple reaching movement of this
study is worlds away from the sophistication of
a dance move. Studying a skill as complex as
salsa is even more challenging than dancing it.
Can motor control rise to the challenge?
COMPETING FINANCIAL INTERESTS
The authors declare no competing financial interests.
1. Galea, J.M., Mallia, E., Rothwell, J. & Diedrichsen, J.
Nat. Neurosci. 18, 597–602 (2015).
2. Shmuelof, L. et al. J. Neurosci. 32, 14617–14621
(2012).
3. Izawa, J. & Shadmehr, R. PLoS Comput. Biol. 7,
e1002012 (2011).
4. Thorndike, E.L. Pyschol. Monogr. Gen. Appl. 2, 1–109
(1898).
5. Skinner, B.F. The Behavior of Organisms: an
Experimental Analysis (Appleton-Century, 1938).
6. Beilock, S.L. & Gray, R. in Handbook of Sport
Psychology (eds. Tenenbaum, G. & Eklund, R.C.)
425–444 (John Wiley & Sons, 2007).
7. Huber, M.E., Seitchik, A., Brown, A., Sternad, D. &
Harkins, S. J. Exp. Psychol. Hum. Percept. Perform.
published online, doi:10.1037/xhp0000039
(23 February 2015).
8. Delgado, M.R. Ann. NY Acad. Sci. 1104, 70–88
(2007).
9. Cohen, J.Y., Haesler, S., Vong, L., Lowell, B.B. &
Uchida, N. Nature 482, 85–88 (2012).
10.Mazzoni, P. & Krakauer, J.W. J. Neurosci. 26,
3642–3645 (2006).
11.Barash, S. et al. J. Neurosci. 19, 10931–10939
(1999).
12.Wei, K. & Kording, K. J. Neurophysiol. 101, 655–664
(2009).
13.Taylor, J.A., Krakauer, J.W. & Ivry, R.B. J. Neurosci.
34, 3023–3032 (2014).
14.Park, S.-W., Dijkstra, T.M.A. & Sternad, D. Front.
Comput. Neurosci. 7, 111 (2013).
15.Walker, B. & Kording, K. PLoS ONE 8, e78747
(2013).
481