Download PDF

Writing Memories with Light-Addressable
Reinforcement Circuitry
Adam Claridge-Chang,1,3 Robert D. Roorda,1 Eleftheria Vrontou,1 Lucas Sjulson,1,4 Haiyan Li,2,5 Jay Hirsh,2
and Gero Miesenböck1,*
1Department
of Physiology, Anatomy and Genetics, University of Oxford, Parks Road, Oxford OX1 3PT, UK
of Biology, University of Virginia, Gilmer Hall, Charlottesville, VA 22903, USA
3Present address: Wellcome Trust Centre for Human Genetics, University of Oxford, Roosevelt Drive, Oxford OX3 7BN, UK
4Present address: Department of Psychiatry, New York University School of Medicine, 550 First Avenue, New York, NY 10016, USA
5Present address: Department of Psychiatry, University of California, 401 Parnassus Avenue, San Francisco, CA 94143, USA
*Correspondence: [email protected]
DOI 10.1016/j.cell.2009.08.034
2Department
SUMMARY
Dopaminergic neurons are thought to drive learning
by signaling changes in the expectations of salient
events, such as rewards or punishments. Olfactory
conditioning in Drosophila requires direct dopamine
action on intrinsic mushroom body neurons, the
likely storage sites of olfactory memories. Neither
the cellular sources of the conditioning dopamine
nor its precise postsynaptic targets are known. By
optically controlling genetically circumscribed subsets of dopaminergic neurons in the behaving fly,
we have mapped the origin of aversive reinforcement
signals to the PPL1 cluster of 12 dopaminergic cells.
PPL1 projections target restricted domains in the
vertical lobes and heel of the mushroom body. Artificially evoked activity in a small number of identifiable
cells thus suffices for programming behaviorally
meaningful memories. The delineation of core reinforcement circuitry is an essential first step in dissecting the neural mechanisms that compute and
represent valuations, store associations, and guide
actions.
INTRODUCTION
Having to decide, moment by moment, what to do next is the
price of motility. Mobile agents must continuously evaluate their
circumstances and choose actions based on predicted consequences. Intelligence subserving these decisions is found even
in the simplest organisms. The flagellar motors of E. coli, for
example, are coupled to chemosensors via a biochemical circuit
that enables the bacterium to chase nutrients and avoid toxins
(Berg, 2004).
Fruit flies, too, are attracted to some chemicals and repelled
by others. But in contrast to E. coli’s hardwired responses,
a fly’s reactions to chemical signals are plastic: the valence of
most odors is neither innate nor invariant but influenced by experience. When a scent is paired repeatedly with electric foot
shock, it acquires persistent negative valence—an aversive
memory is formed (Quinn et al., 1974; Tully and Quinn, 1985).
Omitting electric shocks during further odor presentations gradually restores the odor’s original hedonic valence—the aversive
memory is extinguished (Quinn et al., 1974; Tully and Quinn,
1985). The fly thus keeps a record of its experience, which it
uses to inform its actions.
Olfactory-driven action choices in Drosophila require two brain
centers, the lateral protocerebrum and the mushroom body. Of
these, only the mushroom body has been implicated in response
plasticity (Heisenberg et al., 1985; de Belle and Heisenberg,
1994; Zars et al., 2000; McGuire et al., 2003). Olfactory learning
depends acutely on cyclic AMP (cAMP) signaling in the intrinsic
mushroom body neurons (Zars et al., 2000; McGuire et al., 2003),
called Kenyon cells (KCs). An increase in cAMP levels coincident
with odor-evoked activity is thought to modify KC output
synapses (Dubnau et al., 2001; McGuire et al., 2001; Heisenberg,
2003; Davis, 2005; Keene and Waddell, 2007). The calcium/
calmodulin-dependent adenylyl cyclase encoded by the rutabaga gene (Livingstone et al., 1984; Levin et al., 1992) has
been proposed to function as a logic gate integrating sensory
and reinforcement signals (Levin et al., 1992; Heisenberg,
2003; Davis, 2005; Keene and Waddell, 2007). cAMP production
by this enzyme is thought to be regulated (Abrams et al., 1991;
Levin et al., 1992) by calcium influx (due to odor-evoked KC
depolarization) AND coupling to active GS protein (due to reinforcing dopamine action [Schwaerzel et al., 2003] on receptors
expressed by KCs [Kim et al., 2007]).
Dopamine’s role as a putative aversive reinforcer in fly olfactory learning mirrors, but with reversed polarity, its rewarding
role in mammals (Wise and Rompre, 1989; Schultz et al.,
1997). While the evidence implicating dopamine as an aversive
reinforcement signal in Drosophila is substantial (Schwaerzel
et al., 2003; Schroll et al., 2006; Kim et al., 2007), many mechanistic questions remain. First, the sources of the conditioning
dopamine among the 200–300 dopaminergic neurons in the
central fly brain (Budnik and White, 1988) are undefined; dopaminergic neurons might even communicate by volume transmission, not the localized activity of specific synapses. Second,
although experiments in larvae point to an instructive role
in memory formation (Schroll et al., 2006), it is unconfirmed
Cell 139, 405–415, October 16, 2009 ª2009 Elsevier Inc. 405
Figure 1. Pavlovian Olfactory Conditioning
Positions in a behavioral chamber (horizontal dimension) as a function of time (vertical dimension) of 20 Canton-S flies choosing between MCH (blue) and OCT
(orange). The traces are sorted by untrained preference. Bar graphs on the right indicate population averages of decisions in favor of the left and right chamber
halves before and after conditioning (‘‘pre’’ and ‘‘post’’), in the presence of odors (colored bars) or air (white and gray bars).
(A) Mock conditioning without electric shock preserves individual pretraining preferences.
(B) Pairing the presentation of MCH with electric shock causes conditioned avoidance of MCH.
whether dopamine acts in an instructive or merely permissive
capacity in the much better characterized adult olfactory system.
Third, neither the synaptic targets of dopaminergic projections in
the mushroom body nor the effects of dopamine on the physiology of these cells are known. Here, we show that genetically
targeted optical activation (Zemelman et al., 2002; Lima and
Miesenböck, 2005; Sjulson and Miesenböck, 2008) of dopaminergic neurons is, in itself, sufficient for writing aversive olfactory
memories. The origin of the conditioning dopamine is not the
entire population of dopaminergic neurons but a specific cluster
of 12 cells. The axonal projections of these neurons target exclusively mushroom body neurites in the vertical lobes and heel.
Aversive dopamine signals thus act on restricted domains within
a compartmentalized memory system.
RESULTS
Drosophila’s ability to learn and remember has been extensively
probed by training and analyzing groups of flies in the olfactory T
maze (Quinn et al., 1974; Tully and Quinn, 1985). Though statistically powerful, this population assay suffers from several disadvantages: it is blind to individual behavioral variation and its
406 Cell 139, 405–415, October 16, 2009 ª2009 Elsevier Inc.
physiological causes, it may report collective influences on individual decision making (Quinn et al., 1974; Couzin, 2009), and it
does not allow the animal’s behavior to control the rate and
timing of reinforcement. To overcome these drawbacks, we designed an assay in which the odor choices of single flies could be
monitored and altered. Olfactory preference was measured by
tracking a fly’s movements in two air streams converging from
opposite ends of a narrow 50 mm chamber (Figures S1–S3 available online). Flies paced the full length of the chamber in the
absence of added odors but restricted their movements according to preference when odors were introduced (Figure 1).
Untrained odor preferences varied widely among individuals
but fluctuated little when the same flies were tested repeatedly
(p < 0.0001, permutation test; Figures 1A and 2A); untrained preferences are thus individual invariants.
Individual odor preferences, however, could be modified by
Pavlovian shock conditioning (Figure 1B). Training cycles consisted of two epochs in random order: a 1 min presentation of
3-octanol (OCT) without shock and a 1 min presentation of 4methylcyclohexanol (MCH) with 12 electric shocks at 60 V (Tully
and Quinn, 1985). Two such training cycles caused profound
changes in behavior: animals exhibiting an untrained bias in
other than temporal coincidences have a significant effect. One
possibility is that stimulus and action representations are bound
together more tightly, facilitating correct action selection during
recall (Brembs and Heisenberg, 2000; Gallistel and Gibbon,
2000).
Figure 2. Performance of Individually Trained Flies
(A) Percentage of time spent in OCT before and after mock conditioning (left)
and Pavlovian training against MCH (right). **p < 0.0001; permutation test.
The graphs summarize data from Figures 1A and 1B; black lines connect
data points corresponding to the same individual.
(B) Canton-S flies trained against MCH (column b) or OCT (column d) avoid the
shock-associated odor, but trained rutabaga2080 flies (column c) perform indistinguishably from mock-conditioned Canton-S animals (column a). p < 0.0001;
Kruskal-Wallis ANOVA; **, significantly different from mock-conditioned
controls in post hoc comparison (n = 20 flies per condition; means ± SEM).
(C) Persistence of memory (n = 20 flies per time point; means ± SEM).
favor of MCH reversed their preference; animals lacking an
untrained MCH bias acquired a preference for OCT (Figures 1B
and 2A). Across individuals, the rank order of preference was
preserved even after training (p = 0.03, permutation test;
Figure 2A), suggesting that olfactory conditioning operates on
top of the individually variable base valence of a scent.
Individually trained flies exhibited many of the characteristics
seen in population learning (Tully and Quinn, 1985). First, aversive memories could be formed against both MCH and OCT
(Figure 2B, columns b and d), yielding an approximate performance index (PI) (Quinn et al., 1974; Tully and Quinn, 1985) of
0.68 (half PIs 0.54 for anti-OCT learning and 0.81 for anti-MCH
learning). Second, flies carrying the rutabaga2080 mutation,
which impairs a calcium/calmodulin-dependent adenylyl cyclase (Livingstone et al., 1984; Levin et al., 1992), failed to learn
(Figure 2B, column c). Third, memories persisted for at least
3 hr (Figure 2C).
In contrast to populations in a T maze, individuals could also
be trained under automated closed-loop conditions. Here, a
fly’s actions—its entry into and exit from one of two odor
streams—controlled the delivery of electric shock (Figures 3A
and 3B). Learning in this regime is action contingent but distinct
from pure operant conditioning in the strictest sense (Brembs
and Heisenberg, 2000; Brembs and Plendl, 2008), as the animal
is guided by predictive sensory cues and subjected to sustained,
quasi-Pavlovian punishment if it lingers in the reinforced odor.
Action-contingent olfactory learning depended, like Pavlovian
olfactory conditioning (Figure 2B) and some forms of visually
cued learning (Liu et al., 2006; Brembs and Plendl, 2008), on a
functional rutabaga gene (Figure 3C, column b), but it required
less reinforcement to achieve the same level of performance
as Pavlovian training (Figure 3D). This ‘‘operant advantage’’
puts a nuance on the prevalent model of olfactory learning, which
proposes that aversive conditioning is entirely a result of coincident inputs to KCs from the sensory and reinforcement pathways. That action-contingent training yields better performance
despite fewer coincident stimuli (Figure 3D) suggests that factors
Optical Implantation of Memory
A forceful extension of these experiments is to replace the
external reinforcer—electric foot shock—with direct manipulations of the brain’s internal valuation systems. If electric foot
shock drives olfactory learning through a final common path of
dopaminergic inputs to the mushroom bodies, it should be
possible to write memories directly by activating these inputs
in the presence of odors. To test whether this would result in conditioned odor avoidance, we used a fly’s entry into one odor
stream to trigger a laser pulse that opened, via release of ATP
from a previously microinjected caged precursor, ATP-gated
P2X2 channels expressed selectively in circumscribed groups
of cells (Zemelman et al., 2003; Lima and Miesenböck, 2005).
When targeted optical activation of dopaminergic neurons in
TH-GAL4:UAS-P2X2 flies (Friggi-Grelin et al., 2003; Lima and
Miesenböck, 2005) was made contingent upon entry into one
odor, an aversive memory specific to that odor indeed formed
(Figures 4 and 5A, columns a and b). The performance of flies instructed via light-evoked dopamine release matched that of
animals trained conventionally via electric shock (Figure 5A).
The high efficacy of dopaminergic instruction, combined with
the absence of short-term memory when dopaminergic synapses are blocked during training (Schwaerzel et al., 2003),
argues that signaling from dopaminergic neurons is the primary
means by which aversive odor associations are stored.
Common molecular and timing requirements for conditioning
suggest that reinforcement via light-evoked dopamine release
and electric shock draw on the same neural mechanisms: both
forms of learning depend on the adenylyl cyclase encoded by
the rutabaga gene (Figure 3C, column b, and Figure 5A, column
c), and both require that the delivery of the optical or electrical
reinforcer follows the reinforced olfactory choice behavior in
time (Figure 3C, column c, and Figure 5A, column d). When
this contingency was broken (by delivering the same average
number of 20.3 laser pulses that are consumed during 4 min of
effective training, but at random times rather than in an actioncontingent fashion), olfactory preference remained unchanged
(Figure 5A, column d).
Pairing an odor with photorelease of ATP in the absence of
P2X2 expression, or with laser illumination alone (not shown),
induced only minimal changes in preference (Figure 5A, columns
f–h). These changes are in all likelihood due to a small intrinsically
aversive effect of intense ultraviolet irradiation.
Sources of Dopaminergic Reinforcement Signals
The dual roles of dopaminergic neurons in regulating movement
(Lima and Miesenböck, 2005) and providing reinforcement
(Figures 4 and 5) suggest a functional subdivision of the dopaminergic system. It is not known how this division of labor is realized at the level of individual cells. Rendering subsets of dopaminergic neurons light addressable opens a route into this problem:
if function segregates along anatomical boundaries, it should be
Cell 139, 405–415, October 16, 2009 ª2009 Elsevier Inc. 407
Figure 3. Action-Contingent Olfactory Conditioning
(A) Positions in a behavioral chamber (horizontal dimension) as a function of time (vertical dimension) of 20 Canton-S flies choosing between MCH (blue) and OCT
(orange). The traces are sorted by untrained preference. During four 1 min training periods, entries into MCH are punished by electric shock. Bar graphs on the
right indicate population averages of decisions in favor of the left and right chamber halves before and after conditioning (‘‘pre’’ and ‘‘post’’), in the presence of
odors (colored bars) or air (white and gray bars). The MAT file used to generate this figure can be downloaded for further analysis (Supplemental MAT file).
(B) Locomotion traces at an expanded scale of ten individuals (see corresponding numbers in A) during four epochs of action-contingent conditioning. A red tick
mark to the left of a trace indicates the delivery of one electric shock. Animals receive 2–17 shocks during training; the selected individuals represent the minimum
(trace 0) and nine deciles (traces 1–9) in the frequency distribution of shock consumption. Fast learners (traces 0–4) tend to consume reinforcement only during the
first two training epochs, whereas slow learners (traces 5–9) are reinforced throughout.
(C) Effective conditioning requires a functional rutabaga gene product (column b) and contingency between olfactory choice behavior and electric shock; learning
does not occur when this contingency is broken by randomizing reinforcement (column c). p < 0.0001; Kruskal-Wallis ANOVA; **, significantly different from
Canton-S animals in post hoc comparison (n = 20 flies per condition; means ± SEM).
(D) Comparison of the performance of Canton-S flies after 2 and 4 min of Pavlovian and action-contingent training, normalized to the number of electric shocks
consumed. p < 0.0001; Kruskal-Wallis ANOVA; **, significantly different from 2 min of Pavlovian conditioning in post hoc comparison (n = 20 flies per condition;
means ± SEM).
possible genetically to separate reinforcement from locomotor
circuits.
The dopa decarboxylase enhancer fragment carried by the
HL9-GAL4 line marks a subset of dopaminergic neurons that
differs from TH-GAL4 (Figure 6 and Table S1). Whereas
TH-GAL4 runs in six of the seven paired dopaminergic clusters
(Budnik and White, 1988) in the central fly brain (the exception
being the paired anterior medial cluster [PAM]; Figures 6A and
6C and Table S1), HL9-GAL4 shows majority coverage of PAM
neurons (Figures 6B and 6D), but minority or no expression in
both paired posterior lateral clusters (PPL1 and PPL2) and two
of the three paired posterior medial clusters (PPM1 and PPM3)
(Figures 6E and 6F and Table S1). Projections from TH-GAL4
and HL9-GAL4 neurons appear to target preferentially the
vertical and horizontal mushroom body lobes, respectively
(Figures 6G and 6H; Movies S1 and S2).
408 Cell 139, 405–415, October 16, 2009 ª2009 Elsevier Inc.
Dopa decarboxylase is also expressed in serotonergic (5-HT)
cells (Johnson et al., 1989). To exclude a confound due to serotonergic neurons that might potentially be captured by one or
another of the GAL4 lines used in our experiments, we increased
5-HT levels 11-fold above baseline by feeding the 5-HT
precursor 5-hydroxytryptophan (5-HTP; 3 days of 5-HTP treatment raised the mean 5-HT content per head from 0.20 to
2.27 pmol). Tests of olfactory memory revealed identical performance of 5-HTP-treated and untreated animals (avoidance
change 58.16 ± 4.22 versus 62.77 ± 3.45% in the presence
and absence of 5-HTP, respectively; n = 99–105 flies per condition; means ± SEM; p = 0.20, permutation test). Dramatic
changes in 5-HT levels thus have no effect on aversive olfactory
conditioning, allowing us to ascribe driver line effects on learning
to differential transgene expression in subsets of dopaminergic
neurons.
Figure 4. Optical Implantation of Memory
(A) Examples of conditioned odor avoidance in
TH-GAL4:UAS-P2X2 flies after genetically targeted
photostimulation of dopaminergic neurons. Positions
in a behavioral chamber (horizontal dimension) as
a function of time (vertical dimension) of ten flies
choosing between MCH (blue) and OCT (orange). The
traces are sorted by untrained preference. During
four 1 min training periods, entries into MCH activate
10 ms laser pulses. Laser pulses are repeated at
0.2 Hz while the fly remains in the reinforcement
zone. Note the conditioned avoidance of MCH (blue)
after training.
(B) Bar graphs indicate population averages (n = 68
flies) of decisions in favor of the left and right chamber
halves before and after conditioning (‘‘pre’’ and
‘‘post’’), in the presence of odors (colored bars) or air
(white and gray bars).
When neurons labeled by either the TH-GAL4 or the
HL9-GAL4 driver were silenced by inducible overexpression
(McGuire et al., 2003) of the inwardly rectifying potassium channel
Kir2.1 (Baines et al., 2001), spontaneous locomotor activity
dropped to <25% of baseline (Figure 5C, columns b and d). In
striking contrast to these equally pronounced locomotor effects,
activity in TH-GAL4 neurons, but not in HL9-GAL4 neurons, was
necessary (Figure 5B, compare columns b and d) and sufficient
(Figure 5A, compare columns a and e) for instructing aversive
memories. The two driver lines thus differentiate between two
subsets of dopaminergic neurons: TH-GAL4 labels cells involved in both locomotor control and reinforcement, whereas
HL9-GAL4 labels only cells involved in locomotor control.
Because HL9-GAL4 drives expression in more dopaminergic
neurons than TH-GAL4 (76 versus 51 cells; Figure 6 and Table
S1), manipulations of HL9-GAL4 neurons are expected to have
a larger impact on extracellular dopamine levels than manipulations of TH-GAL4 neurons. Yet neither photostimulation (Figure 5A, column e) nor silencing (Figure 5B, column d) of all
HL9-GAL4 cells had a detectable effect on memory. Reinforcement is thus not due to volume transmission of dopamine; rather,
it requires a specific circuit formed by a specific subset of
neurons that is missing in HL9-GAL4.
Targets of Dopaminergic Reinforcement Signals
Which of the four dopaminergic cell clusters captured by THGAL4 but absent from HL9-GAL4 (PPL1, PPL2, PPM1, and
PPM3; Figure 6 and Table S1) are part of this circuit? Because
dopamine must act directly on receptors expressed by KCs of
the mushroom body to provide reinforcement (Kim et al.,
2007), we are able to constrain the four candidate clusters further
by their projection patterns. To highlight individual cell clusters,
dopaminergic neurons were biosynthetically loaded with a
photoactivatable variant of GFP (PA-GFP) (Patterson and Lippincott-Schwartz, 2002), which was subsequently switched on in
identified somata by two-photon photoconversion (Datta et al.,
2008). The fluorescent marker filled the illuminated neurons by
diffusion, permitting their arborizations to be visualized against
Figure 5. Sources of Dopaminergic Reinforcement Signals
(A) Action-contingent photoactivation of P2X2 in
dopaminergic neurons under TH-GAL4 control
produces conditioned odor avoidance (columns
a and b). Optically reinforced flies achieve the
same level of performance as animals trained
conventionally via electric shock (horizontal
shaded band; mean ± SEM). Effective conditioning requires a functional rutabaga gene
product (column c) and contingency between
olfactory choice behavior and optically evoked
dopamine release; learning does not occur when
this contingency is broken by randomizing reinforcement (column d). Activation of P2X2 in dopaminergic neurons under HL9-GAL4 control
(column e) or ATP uncaging in flies lacking P2X2
expression (columns f–h) are equally ineffective. p < 0.0001; Kruskal-Wallis ANOVA; **, significantly different from electric shock conditioning in post hoc comparison (n = 20–68 flies per condition; means ± SEM).
(B) Temperature-induced expression of Kir2.1 in dopaminergic neurons under TH-GAL4 control (dark gray columns), but not under HL9-GAL4 control (medium
gray columns), blocks action-contingent conditioning (column b). p = 0.0062; Kruskal-Wallis ANOVA; **, significantly different from permissive temperature in post
hoc comparison (n = 19–58 flies per condition; means ± SEM).
(C) Temperature-induced expression of Kir2.1 in dopaminergic neurons, under either TH-GAL4 control (dark gray columns) or HL9-GAL4 control (medium gray
columns), inhibits locomotion (columns b and d). p < 0.0001; Kruskal-Wallis ANOVA; **, significantly different from permissive temperature in post hoc comparison (n = 82–120 flies per condition; means ± SEM).
Cell 139, 405–415, October 16, 2009 ª2009 Elsevier Inc. 409
Figure 6. Anatomy of Two Functionally Distinct Sets of Dopaminergic Neurons
(A and B) TH-GAL4 (top row) and HL9-GAL4 (bottom row) mark distinct but partially overlapping clusters of dopaminergic neurons.
(C–F) Maximum intensity projections of confocal sections reveal seven paired neuronal clusters expressing tyrosine hydroxylase in the central brain (red pie
charts in A and B, cell numbers in parentheses; see Table S1 for statistics); the fractions of neurons coexpressing mCD8-GFP in the two GAL4 lines are indicated
in green. Neuropil was stained with nc82 antibodies (blue), dopaminergic neurons with antibodies against tyrosine hydroxylase (red), and mCD8-GFP-expressing
neurons with antibodies against mCD8 (green). The scale bar represents 50 mm.
(G and H) Maximum intensity projections of confocal sections through the mushroom body. KCs express the mb247-DsRed transgene; dopaminergic projections
are labeled by mCD8-GFP. The scale bar represents 10 mm.
DsRed-counterstained (Riemensperger et al., 2005) mushroom
bodies.
Of the four candidate clusters present in the TH-GAL4 line, only
PPL1 neurons were found to target the mushroom body lobes
(Figure 7), forcing the conclusion that this cluster of 12 cells
contains the source of aversive reinforcement. PPM1 and PPM2
neurons, which were not discriminated further, ramify extensively
in a region posterior to the mushroom bodies (Figure 7E), PPM3
neurons innervate the central complex (Figure 7F), and PPL2
neurons elaborate two principal branches: one extending into
undefined synaptic neuropil posterior to the mushroom body,
and another traced to the vicinity of the PPL1 cluster (Figure 7C).
PPL1 somata sit immediately lateral of the mushroom body
calyx and project a bundle of neurites medially between
peduncle and vertical lobe (Figure 7A). Here, the neurons elaborate dense (possibly dendritic) ramifications that remain confined
to the same hemisphere. Long-range fibers extend bilaterally to
the tips and stalks of the vertical (a and a0 ) lobes, the heel, and
the peduncle of the mushroom body and arborize within these
regions (Figure 7A). Close inspection of confocal image stacks
reveals an additional PPL1 projection to the central complex.
The targets of PPL1 neurons thus include structures implicated
in both olfactory (Zars et al., 2000; McGuire et al., 2003; Krashes
et al., 2007; Wang et al., 2008) and visual (Liu et al., 2006)
memory.
Notably, PPL1 neurons are not the sole sources of dopaminergic input to the mushroom body lobes. A second dopami-
410 Cell 139, 405–415, October 16, 2009 ª2009 Elsevier Inc.
nergic projection originates from cells in the PAM cluster and
terminates bilaterally in the medial portions of the horizontal (b)
lobes (Figure 7B). The function of these inputs is currently
unknown, but it is safe to say that they play no role in shortterm olfactory learning, as comprehensive manipulations of
PAM neuron activity via the HL9-GAL4 driver are without consequence for memory (Figures 5A and 5B).
DISCUSSION
Nervous systems transform sensory signals and internal states
into actions. Learning is a higher-order process by which this
transformation is altered, producing different actions from the
same initial state. Associative learning, by definition, uses an
inherently valued stimulus, such as pain, to modify behavioral
responses to sensory cues. When flies form aversive olfactory
memories, they associate at least two sets of external inputs,
conveyed by the olfactory and reinforcement pathways. As the
presumed sites of memory storage, KCs lie at the intersection
of these neural pathways. Signals from odorant receptors reach
KCs, which are third-order olfactory neurons, via an extensively
characterized sensory processing stream (Heisenberg, 2003;
Keene and Waddell, 2007). By contrast, little is known, after
more than 30 years of research, about the neural circuits that
transmit nociceptive inputs from peripheral sensors and transform them into reinforcement signals.
Figure 7. Projections
Neurons
of
Dopaminergic
Three-dimensional reconstructions of PA-GFPlabeled dopaminergic arborizations (green) and
mb247-DsRed expressing KCs (gray). The expression of PA-GFP is controlled by TH-GAL4 (A, C–F)
or HL9-GAL4 (B). Only PPL1 neurons (A, captured
by TH-GAL4) and PAM neurons (B, captured by
HL9-GAL4) innervate the mushroom body lobes.
The two cell clusters target the vertical and horizontal lobes, respectively. In each panel, a righthanded coordinate system near the left mushroom
body indicates the anterior (A), lateral (L), and
dorsal (D) directions.
We do know that dopamine is an essential mediator of aversive reinforcement in adult flies (Schwaerzel et al., 2003;
Kim et al., 2007) and that its broad release can substitute for
aversive salt conditioning in larvae (Schroll et al., 2006). The
data presented here advance our understanding from this purely
pharmacological level of analysis to a resolution of the relevant
circuitry. Dopaminergic neurons display a remarkable degree
of anatomical target specificity and functional specialization.
Of the seven paired cell clusters in the central fly brain, only
two (PAM and PPL1) innervate the mushroom body lobes, where
their projections are segregated into discrete, nonoverlapping
domains (Figure 7). Of these two cell clusters, only one (PPL1)
carries aversive reinforcement information. Artificial manipulation of these reinforcement signals is, in itself, sufficient for
reprogramming a fly’s olfactory choice behavior (Figures 4 and
5). Neurons of the PPL1 cluster thus appear to link the brain’s
valuation systems, where neural measures of attractiveness or
aversiveness are constructed, to memory systems that associate these measures with predictive sensory cues.
Presynaptic Inputs to PPL1 Neurons:
Learning Algorithms
In primates, midbrain dopaminergic neurons signal not the absolute magnitudes of rewards but the extent to which these
rewards are unpredicted, and it is this ‘‘prediction error’’ that is
thought to control synaptic weight changes during learning
(Schultz et al., 1997). The algorithm ensures that the learning process terminates, as it should, when an animal has understood
a regularity in its environment: accurate prediction reduces the
error signal to zero, and no further adjustments to the animal’s model of the environment are made.
Comparing the responses of PPL1
neurons to predictable and unpredictable
punishment will provide a clear test of
whether olfactory learning in flies is also
driven by prediction error: if PPL1 neurons
encode error signals, their responses to
aversive stimuli that are forecast by learned
olfactory cues are expected to be attenuated or abolished. Previous attempts to
perform such a test, by imaging calcium
signals in dopaminergic projections to the
mushroom bodies (Riemensperger et al., 2005), proved inconclusive, perhaps due to an inability to focus on what we only now
appreciate are the memory-relevant PPL1 neurons.
Measuring the physiological responses of these neurons may
uncover further functional heterogeneity within the PPL1 cluster
and thereby extend the resolution limit imposed by currently
available genetic tools. For example, it is conceivable that reinforcement signals are only emitted by some subset of PPL1
neurons, or in the extreme, a single cell.
The ability to pin reinforcement to a small, identified set of
dopaminergic neurons provides a starting point for the dissection of upstream circuits that regulate the activity of these cells.
Engineers (Kalman, 1960), researchers in machine learning
(Samuel, 1959; Holland, 1986; Sutton and Barto, 1998), and
psychologists (Rescorla and Wagner, 1972) have developed
models of adaptive behavior in which error signals play a central
role; they have also designed efficient algorithms for their
computation (Sutton and Barto, 1998). Ironically, however, the
biological systems that provided the inspiration for these models
are still poorly understood. The classic temporal difference (TD)
algorithms of machine learning suggest that error signals are
calculated by a bootstrapping method, which assigns credit by
comparing successive predictions (Sutton and Barto, 1998). A
variety of neuromorphic circuits for computing such signals
have been proposed (Houk et al., 1995; Montague et al., 1995;
Schultz et al., 1997; Wörgötter and Porr, 2005), but these remain
speculative until actual biological examples are solved. The
simplicity of the fly, which like other invertebrates (Hammer,
1993; Brembs et al., 2002) employs only a handful of cells to
Cell 139, 405–415, October 16, 2009 ª2009 Elsevier Inc. 411
convey reinforcement, raises hopes that the mechanisms
controlling PPL1 activity are coming within reach.
Postsynaptic Targets of PPL1 Neurons: Memory Circuits
High-resolution mapping of the instructive PPL1 terminations
within their mushroom body target domains will help disclose
how olfactory memories are organized, written, and read.
Most, if not all, memory devices are constructed from four circuit
components: data storage locations, an addressing system that
activates these locations for reading or writing, data input lines,
and data output lines (Kanerva, 1988). In the case of the mushroom bodies, there is every expectation (though not yet formal
proof) that information is stored in the weights of synapses
between KCs and presently unidentified mushroom body output
neurons (Dubnau et al., 2001; McGuire et al., 2001; Heisenberg,
2003; Davis, 2005; Keene and Waddell, 2007). The addressing
system that activates these storage locations is odor-evoked
KC activity patterns. Memories are written when a data input
line—identified here as a PPL1 projection—releases dopamine
onto active storage locations. The ensuing surge in cAMP
production is thought to strengthen connections between the
active KC ensemble and mushroom body output neurons (Levin
et al., 1992; Heisenberg, 2003; Davis, 2005; Keene and Waddell,
2007). Memories are retrieved when the addressing system
reactivates these data locations—that is, when a fly re-encounters a learned odor—and a mushroom body output neuron reads
the contents of memory by pooling its synaptic inputs.
A key detail of this memory architecture is that data input and
output lines supply the same storage locations in matched pairs
(Kanerva, 1988). This has two important implications. First, the
synaptic terminals of PPL1 projections mark the locations of input
to the associative machinery. Although much effort has been
devoted to identifying the mushroom body subdomains in which
associative memories first form, different studies have reached
apparently irreconcilable conclusions (Zars et al., 2000; Pascual
and Preat, 2001; McGuire et al., 2003; Akalal et al., 2006; Krashes
et al., 2007; Wang et al., 2008). The restricted distribution of PPL1
neurites within the mushroom bodies limits the potential sites of
short-term plasticity to the vertical lobes, the heel, and the
peduncle and can thus further focus attempts to localize and
monitor the changes accompanying memory formation. The
ability to write behaviorally relevant memories at will, simply by
activating a small set of dopaminergic neurons, opens the possibility of performing the same manipulation in preparations where
memory formation can be observed directly.
Second, in order to retrieve information from memory, the
dendritic arbors of mushroom body output neurons must visit
the same storage locations as the dopaminergic data input lines.
This anatomical constraint will facilitate the search for these
currently unknown circuit elements, which are expected to
hold the key to understanding how remembered experience
biases action choice.
EXPERIMENTAL PROCEDURES
Fly Strains
Transgenic strains were backcrossed to Canton-S for more than four generations. For photostimulation experiments, balanced UAS-P2X2 animals (Lima
412 Cell 139, 405–415, October 16, 2009 ª2009 Elsevier Inc.
and Miesenböck, 2005) were crossed with balanced driver lines TH-GAL4
(Friggi-Grelin et al., 2003) or HL9-GAL4 to yield UAS-P2X2/CyO ; GAL4driver/TMSb males. These males were paired with Canton-S or rutabaga2080
virgins to generate experimental animals. The HL9-GAL4 line carries an insertion of the GAL4 coding sequence into a fragment of the dopa decarboxylase
(Ddc) gene. Genomic Ddc sequence from bp 2702 (numbering relative to
transcription start site) to bp +970 in exon B was fused to yeast GAL4 and
flanked by the hsp70 termination/polyA region and the Ddc exon D 30 untranslated region (UTR) and 30 flanking region (bp +3521 to +4643). The initiating
AUG in Ddc exon B was mutated to prevent interference with the translation
of GAL4.
Where indicated, 5-HT levels were elevated by growing flies on food
containing 50 mM 5-HTP and 25 mg/100 ml ascorbic acid for 3 days. The
effectiveness of treatment was verified amperometrically (LC-4B detector,
Bioanalytical Systems) after fractionating frozen head homogenates by
reverse-phase HPLC (Jennings et al., 2006).
To deliver caged ATP (Lima and Miesenböck, 2005), flies were briefly iced
and injected (Nanoject II, Drummond) through the ocelli with 9.2 nl of 60 mM
Molecular
P3-[1-(4,5-dimethoxy-2-nitrophenyl)ethyl]-ATP (DMNPE-ATP;
Probes) in artificial hemolymph (5 mM Na-HEPES [pH 7.3], 115 mM NaCl,
5 mM KCl, 2 mM CaCl2, 8 mM MgCl2, 4 mM NaHCO3, 1 mM NaH2PO4, 5 mM
trehalose, 10 mM sucrose). Injected flies were allowed to recover for >5 min
and used within 30 min.
For temperature-induced silencing of neuronal activity, balanced tubPGAL80ts flies were crossed with a balanced UAS-Kir2.1 line (Baines et al.,
2001) to yield tubP-GAL80ts/CyO ; UAS-Kir2.1/TMSb offspring, which were
paired with TH-GAL4 or HL9-GAL4. Experimental animals were maintained
at the permissive temperature of 20 C, at which expression of the Kir2.1 potassium channel is repressed by GAL80ts (McGuire et al., 2003). Channel expression was induced by shifting flies for 16 hr to the restrictive temperature of
30 C, at which GAL80ts inactivates (McGuire et al., 2003).
Expression patterns were visualized by crossing GAL4 driver lines to
responder strains carrying one copy of a UAS-mCD8-GFP transgene or two
copies of a UAS-PA-GFP transgene (Datta et al., 2008); KCs expressed
mb247-DsRed (Riemensperger et al., 2005).
Training Apparatus and Software
Olfactory choice behavior was evaluated and conditioned in custom-built
single-fly chambers (Figure S1A). The chambers, designed to confine movement to one dimension, measured 50 mm in length, 5 mm in width (sufficient
for flies to turn freely), and 1.3 mm in height (to prevent flies from walking on
side walls and thereby escape electric shock). The chambers were fabricated
from clear polycarbonate to allow tracking of the animals’ backlit silhouettes
in side-on video images. Floors and ceilings consisted of printed circuit boards
(PCBs) with 1 mm electrodes spaced at 1 mm intervals. Air/odor mixtures
entered the chambers through two inlets (5.0 3 0.3 mm) at the left and right
ends and exited through four outlets (5.0 3 0.3 mm) at the midpoint. As in
a traditional T maze, two converging air/odor streams meet at central outflow
vents, defining a narrow (5 3 5 mm) choice zone. To eliminate any spatial
bias, odor streams were randomly alternated between the left and right
chamber halves during test, training, and retest.
In electric shock conditioning experiments (Figure S1B), 20 chambers were
arranged into two side-by-side stacks by slotting their metal air/odor inlet
tubes into gas distribution manifolds; the gas tubes also served as electrical
leads connecting the PCBs via solid-state relays (Fairchild HSR312L) to a
60 V source. The array was backlit by 940 nm LEDs (TSAL6100, Vishay) with
plastic diffusers and viewed by a Sunell SN-425M CCD camera equipped
with a Navitar1212 lens and connected to a video acquisition board (PCI1409, National Instruments). A virtual instrument written in LabVIEW 7.1
(National Instruments) extracted position data from video images and
controlled gas flow and the solid-state relays. Fly positions were calculated
by subtracting the current video frame from a continuously updated background image that retained the brightest pixels from the previous frame,
setting negative pixels in the difference image to zero, and deriving the center
coordinates of the silhouettes with the centroid location function. Electric
shocks of 1.25 s duration were delivered at fixed 0.2 Hz frequencies, either
during predetermined training intervals (Pavlovian conditioning) or when the
virtual instrument detected entry of the fly into the reinforced odor stream
(action-contingent conditioning).
In optical conditioning experiments (Figure S1C), four flies were tested
simultaneously in four chambers with 50 3 5 mm apertures in the top and
bottom cover plates of the assemblies; the PCB floors and ceilings were replaced with quartz glass windows. The chambers were arranged flat on a clear
acrylic platform and backilluminated by 940 nm LEDs (TSAL6100, Vishay)
through plastic diffusers. The chambers were viewed from above by a Sunell
SN-425M CCD camera equipped with a Navitar Zoom7000 lens. A virtual
instrument written in LabVIEW 7.1 (National Instruments) extracted position
data from video images and controlled gas flow and laser exposures. Optical
stimuli were delivered by a Q-switched, frequency-tripled Nd:YVO4 laser emitting at 355 nm (DPSS Lasers, model 3520-30). The beam was switched and
intensity-modulated with an acousto-optic deflector (AOD; IntraAction model
ASN-802832 with ME-802 driver), directed into a pair of scan mirrors (GSI
Lumnonics VM2000 with MiniSAX servo controllers), and merged with the
observation path by a UV cold mirror (CVI). Upon entry of a fly into the reinforced odor stream, its coordinates were automatically fed to the scan mirrors
to position the beam, and the Q switch and AOD were activated for 10 ms,
delivering a light pulse of 0.08 mJ/mm2. Optical stimuli were repeated at
0.2 Hz while the fly remained in the reinforcement zone. During such intervals,
the scan mirrors were repositioned every 100 ms.
The photostimulation conditions used here are milder than those in our
earlier experiments (Lima and Miesenböck, 2005). Previously, flies were injected with 41 nl caged ATP and exposed to 10 mJ/mm2 of light (four 150 ms
pulses at 17 mW/mm2). In the current protocol, flies were injected with 9.2 nl
caged ATP. Individual light pulses delivered optical energies of 0.08 mJ/
mm2; flies consumed an average of 20.3 such pulses during training. The
mean cumulative optical energy delivered is therefore 1.62 mJ/mm2, corresponding to 16% of that used previously. Conditioned behavior was assayed
>5 min after photostimulation, when any potential acute locomotor effects of
dopaminergic activation have subsided (Lima and Miesenböck, 2005).
Odors
Compressed air was passed through desiccant and activated charcoal before
manifolding into mass flow controllers (MFCs; CMOSens PerformanceLine,
Sensirion). Two 5000 ml capacity MFCs controlled left and right arm carrier
air flows; six 500 ml capacity MFCs regulated OCT, MCH, and compensatory
air flows in the left and right chamber arms, respectively. Flow-controlled
carrier air was rehumidified by passage through a gas washing tube containing
distilled water; stimulus streams were drawn through vials filled with pure liquid
odorants. Measurements with a ppbRAE photoionization detector (PID)
showed that the vials delivered saturated vapor at flow rates of up to
360 ml/min for MCH and 480 ml/min for OCT. For determination of the range
of appropriate experimental odor concentrations, PID measurements were
made from a conventional T maze odor cup (Tully and Quinn, 1985; Connolly
and Tully, 1998); these corresponded to 14% MCH and 11% OCT. We
adjusted these odor concentrations slightly to shift the untrained preference
toward the odor to be made aversive; this yielded more stable baselines. Aversive training against MCH used 5% MCH versus 12% OCT, while aversive
training against OCT used 10% MCH versus 8% OCT. Flow rate differences
were compensated with air. The combined air/odor flow rates were 100 ml/
min per half chamber, roughly corresponding to the air speed experienced
by a fly in flight (David, 1979).
Data Analysis
Experimental data were logged to ASCII text files and analyzed offline in MATLAB 7.4 (The MathWorks; see the Supplemental MAT file for an example). Odor
preference was quantified as the percentage of decisions in favor of odor A
during a test interval (Figures S2–S4). A decision was counted every time
a fly entered and exited the central choice zone. Entry into odor A from the
choice zone, either as the result of a reversal or a traversal, was tallied in favor
of that odor’s choice percentage: (decisionsodorA/decisionsinterval) 3 100%.
Learning was scored by subtracting the posttraining preference from the
pretraining preference, giving a percentage change. The results of aversive
conditioning are reported as negative preference—that is, positive avoidance—changes. Test intervals lasted 2 min; memory was assayed after
a 5 min rest period after conditioning. Only data from flies making at least
one choice per test interval were analyzed.
Pairwise hypotheses were evaluated by permutation test. Kruskal-Wallis
ANOVA and post hoc analysis using Tukey’s honestly significant difference
criterion were used to test hypotheses involving multiple groups.
Immunofluorescence Microscopy
Synaptic neuropil was decorated with monoclonal antibody nc82 (Developmental Studies Hybridoma Bank) and goat anti-mouse Alexa 568 conjugate
(Molecular Probes). Tyrosine hydroxylase was detected with antiserum
Ab152 (Chemicon, 1: 100) and goat anti-rabbit Alexa 635 conjugate (Molecular
Probes). Mouse CD8 expressed from the UAS-mCD8-GFP transgene was
stained with rat anti-mCD8a antibody (Invitrogen Caltag, 1: 100) and goat
anti-rat Alexa 488 conjugate (Molecular Probes). Stained brains were embedded in FocusClear and MountClear (CelExplorer Labs) and imaged on
a Zeiss LSM710 laser-scanning confocal microscope.
Two-Photon Photoconversion of PA-GFP
Fly brains were dissected in artificial hemolymph and positioned under
a custom-built two-photon microscope equipped with a Chameleon Ultra II
laser (Coherent) and Hamamatsu H7422-40 GAsP detectors. Dopaminergic
cell bodies expressing two copies of a UAS-PA-GFP transgene (Datta et al.,
2008) under TH-GAL4 or HL9-GAL4 control emitted faint basal fluorescence
when excited at 925 nm. Individual cell clusters could be identified by their
anatomical position relative to the mushroom bodies, which expressed
mb247-DsRed (Riemensperger et al., 2005). To photoconvert PA-GFP to its
intensely fluorescent form (Patterson and Lippincott-Schwartz, 2002; Datta
et al., 2008), a small volume of tissue containing the dopaminergic cell cluster
of interest was illuminated with 5–30 mW of laser power at 710 nm in a scan
configuration that delivered 9–30 mJ of optical energy per mm3 voxel. Following
a 1 hr waiting period to allow diffusion of PA-GFP, brains were recovered, fixed
with 4% (w/v) paraformaldehyde in PBS containing 0.1% (w/v) Triton X-100,
embedded in FocusClear and MountClear (CelExplorer Labs), and imaged
on a Zeiss LSM710 laser-scanning confocal microscope. Three-dimensional
models were rendered in Volocity 5.0.3 (Improvision).
SUPPLEMENTAL DATA
Supplemental Data include four figures, one table, two movies, and one
MAT file and can be found with this article online at http://www.cell.com/
supplemental/S0092-8674(09)01104-0.
ACKNOWLEDGMENTS
We thank Trevor Sharp and Paul Overton for 5-HT measurements, Simon
Reeve for experiments on rutabaga mutants, Richard Axel, Michael Bate,
Ronald Davis, André Fiala, and Scott Waddell for fly strains, and Shankar
Srinivas for access to a confocal microscope. This work was supported by
grants from the Medical Research Council (G.M.), the National Institutes of
Health (G.M. and J.H.), the Office of Naval Research (G.M.), the Dana Foundation (G.M.), and the European Molecular Biology Organization (E.V.).
Received: April 6, 2009
Revised: May 25, 2009
Accepted: August 17, 2009
Published: October 15, 2009
REFERENCES
Abrams, T.W., Karl, K.A., and Kandel, E.R. (1991). Biochemical studies of stimulus convergence during classical conditioning in Aplysia: dual regulation of
adenylate cyclase by Ca2+/calmodulin and transmitter. J. Neurosci. 11,
2655–2665.
Akalal, D.B., Wilson, C.F., Zong, L., Tanaka, N.K., Ito, K., and Davis, R.L.
(2006). Roles for Drosophila mushroom body neurons in olfactory learning
and memory. Learn. Mem. 13, 659–668.
Cell 139, 405–415, October 16, 2009 ª2009 Elsevier Inc. 413
Baines, R.A., Uhler, J.P., Thompson, A., Sweeney, S.T., and Bate, M. (2001).
Altered electrical properties in Drosophila neurons developing without
synaptic transmission. J. Neurosci. 21, 1523–1531.
Kanerva, P. (1988). Sparse Distributed Memory (Cambridge, MA: MIT Press).
Berg, H.C. (2004). E. coli in Motion (New York: Springer).
Kim, Y.C., Lee, H.G., and Han, K.A. (2007). D1 dopamine receptor dDA1 is
required in the mushroom body neurons for aversive and appetitive learning
in Drosophila. J. Neurosci. 27, 7640–7647.
Brembs, B., and Heisenberg, M. (2000). The operant and the classical in conditioned orientation of Drosophila melanogaster at the flight simulator. Learn.
Mem. 7, 104–115.
Brembs, B., and Plendl, W. (2008). Double dissociation of PKC and AC manipulations on operant and classical learning in Drosophila. Curr. Biol. 18,
1168–1171.
Brembs, B., Lorenzetti, F.D., Reyes, F.D., Baxter, D.A., and Byrne, J.H. (2002).
Operant reward learning in Aplysia: neuronal correlates and mechanisms.
Science 296, 1706–1709.
Budnik, V., and White, K. (1988). Catecholamine-containing neurons in
Drosophila melanogaster: distribution and development. J. Comp. Neurol.
268, 400–413.
Connolly, J.B., and Tully, T. (1998). Behaviour, learning, and memory. In
Drosophila. A Practical Approach, D.B. Roberts, ed. (Oxford: Oxford University
Press), pp. 265–317.
Couzin, I.D. (2009). Collective cognition in animal groups. Trends Cogn. Sci.
13, 36–43.
Datta, S.R., Vasconcelos, M.L., Ruta, V., Luo, S., Wong, A., Demir, E., Flores,
J., Balonze, K., Dickson, B.J., and Axel, R. (2008). The Drosophila pheromone
cVA activates a sexually dimorphic neural circuit. Nature 452, 473–477.
David, C.T. (1979). Optomotor control of speed and height by free-flying
Drosophila. J. Exp. Biol. 82, 389–392.
Keene, A.C., and Waddell, S. (2007). Drosophila olfactory memory: single
genes to complex neural circuits. Nat. Rev. Neurosci. 8, 341–354.
Krashes, M.J., Keene, A.C., Leung, B., Armstrong, J.D., and Waddell, S.
(2007). Sequential use of mushroom body neuron subsets during Drosophila
odor memory processing. Neuron 53, 103–115.
Levin, L.R., Han, P.L., Hwang, P.M., Feinstein, P.G., Davis, R.L., and Reed,
R.R. (1992). The Drosophila learning and memory gene rutabaga encodes
a Ca2+/calmodulin-responsive adenylyl cyclase. Cell 68, 479–489.
Lima, S.Q., and Miesenböck, G. (2005). Remote control of behavior through
genetically targeted photostimulation of neurons. Cell 121, 141–152.
Liu, G., Seiler, H., Wen, A., Zars, T., Ito, K., Wolf, R., Heisenberg, M., and Liu, L.
(2006). Distinct memory traces for two visual features in the Drosophila brain.
Nature 439, 551–556.
Livingstone, M.S., Sziber, P.P., and Quinn, W.G. (1984). Loss of calcium/
calmodulin responsiveness in adenylate cyclase of rutabaga, a Drosophila
learning mutant. Cell 37, 205–215.
McGuire, S.E., Le, P.T., and Davis, R.L. (2001). The role of Drosophila mushroom body signaling in olfactory memory. Science 293, 1330–1333.
McGuire, S.E., Le, P.T., Osborn, A.J., Matsumoto, K., and Davis, R.L. (2003).
Spatiotemporal rescue of memory dysfunction in Drosophila. Science 302,
1765–1768.
Davis, R.L. (2005). Olfactory memory formation in Drosophila: from molecular
to systems neuroscience. Annu. Rev. Neurosci. 28, 275–302.
Montague, P.R., Dayan, P., Person, C., and Sejnowski, T.J. (1995). Bee
foraging in uncertain environments using predictive hebbian learning. Nature
377, 725–728.
de Belle, J.S., and Heisenberg, M. (1994). Associative odor learning in
Drosophila abolished by chemical ablation of mushroom bodies. Science
263, 692–695.
Pascual, A., and Preat, T. (2001). Localization of long-term memory within the
Drosophila mushroom body. Science 294, 1115–1117.
Dubnau, J., Grady, L., Kitamoto, T., and Tully, T. (2001). Disruption of neurotransmission in Drosophila mushroom body blocks retrieval but not acquisition
of memory. Nature 411, 476–480.
Friggi-Grelin, F., Coulom, H., Meller, M., Gomez, D., Hirsh, J., and Birman, S.
(2003). Targeted gene expression in Drosophila dopaminergic cells using
regulatory sequences from tyrosine hydroxylase. J. Neurobiol. 54, 618–627.
Gallistel, C.R., and Gibbon, J. (2000). Time, rate, and conditioning. Psychol.
Rev. 107, 289–344.
Hammer, M. (1993). An identified neuron mediates the unconditioned stimulus
in associative olfactory learning in honeybees. Nature 366, 59–63.
Heisenberg, M. (2003). Mushroom body memoir: from maps to models. Nat.
Rev. Neurosci. 4, 266–275.
Heisenberg, M., Borst, A., Wagner, S., and Byers, D. (1985). Drosophila mushroom body mutants are deficient in olfactory learning. J. Neurogenet. 2, 1–30.
Holland, J.H. (1986). Escaping brittleness: the possibilities of general purpose
learning algorithms applied to parallel rule-based systems. In Machine
Learning, R.S. Michalski, J.G. Carbonell, and T.M. Mitchell, eds. (Los Altos,
CA: Morgan Kaufmann), pp. 593–623.
Houk, J.C., Adams, J.L., and Barto, A.G. (1995). A model of how the basal
ganglia generate and use neural signals that predict reinforcement. In Models
of Information Processing in the Basal Ganglia, J.C. Houk, J.L. Davis, and D.G.
Beiser, eds. (Cambridge, MA: MIT Press), pp. 249–270.
Jennings, K.A., Loder, M.K., Sheward, W.J., Pei, Q., Deacon, R.M., Benson,
M.A., Olverman, H.J., Hastie, N.D., Harmar, A.J., Shen, S., et al. (2006).
Increased expression of the 5-HT transporter confers a low-anxiety phenotype
linked to decreased 5-HT transmission. J. Neurosci. 26, 8955–8964.
Johnson, W.A., McCormick, C.A., Bray, S.J., and Hirsh, J. (1989). A neuronspecific enhancer of the Drosophila dopa decarboxylase gene. Genes Dev.
3, 676–686.
Kalman, R.E. (1960). A new approach to linear filtering and prediction problems. J. Basic Eng. Trans. ASME 82, 35–45.
414 Cell 139, 405–415, October 16, 2009 ª2009 Elsevier Inc.
Patterson, G.H., and Lippincott-Schwartz, J. (2002). A photoactivatable GFP
for selective photolabeling of proteins and cells. Science 297, 1873–1877.
Quinn, W.G., Harris, W.A., and Benzer, S. (1974). Conditioned behavior in
Drosophila melanogaster. Proc. Natl. Acad. Sci. USA 71, 708–712.
Rescorla, R.A., and Wagner, A.R. (1972). A theory of Pavlovian conditioning:
variations in the effectiveness of reinforcement and non-reinforcement. In
Classical Conditioning II: Current Research and Theory, A.H. Black and W.F.
Prokasy, eds. (New York: Appleton-Century-Crofts), pp. 64–69.
Riemensperger, T., Völler, T., Stock, P., Buchner, E., and Fiala, A. (2005).
Punishment prediction by dopaminergic neurons in Drosophila. Curr. Biol.
15, 1953–1960.
Samuel, A.L. (1959). Some studies in machine learning using the game of
checkers. IBM J. Res. Dev. 3, 211–229.
Schroll, C., Riemensperger, T., Bucher, D., Ehmer, J., Völler, T., Erbguth, K.,
Gerber, B., Hendel, T., Nagel, G., Buchner, E., et al. (2006). Light-induced activation of distinct modulatory neurons triggers appetitive or aversive learning in
Drosophila larvae. Curr. Biol. 16, 1741–1747.
Schultz, W., Dayan, P., and Montague, P.R. (1997). A neural substrate of
prediction and reward. Science 275, 1593–1599.
Schwaerzel, M., Monastirioti, M., Scholz, H., Friggi-Grelin, F., Birman, S., and
Heisenberg, M. (2003). Dopamine and octopamine differentiate between
aversive and appetitive olfactory memories in Drosophila. J. Neurosci. 23,
10495–10502.
Sjulson, L., and Miesenböck, G. (2008). Photocontrol of neural activity:
biophysical mechanisms and performance in vivo. Chem. Rev. 108,
1588–1602.
Sutton, R.S., and Barto, A.G. (1998). Reinforcement Learning (Cambridge, MA:
MIT Press).
Tully, T., and Quinn, W.G. (1985). Classical conditioning and retention in
normal and mutant Drosophila melanogaster. J. Comp. Physiol. [A] 157,
263–277.
Wang, Y., Mamiya, A., Chiang, A.S., and Zhong, Y. (2008). Imaging of an early
memory trace in the Drosophila mushroom body. J. Neurosci. 28, 4368–4376.
Zars, T., Fischer, M., Schulz, R., and Heisenberg, M. (2000). Localization of
a short-term memory in Drosophila. Science 288, 672–675.
Wise, R.A., and Rompre, P.P. (1989). Brain dopamine and reward. Annu. Rev.
Psychol. 40, 191–225.
Zemelman, B.V., Lee, G.A., Ng, M., and Miesenböck, G. (2002). Selective photostimulation of genetically chARGed neurons. Neuron 33, 15–22.
Wörgötter, F., and Porr, B. (2005). Temporal sequence learning, prediction,
and control: a review of different models and their relation to biological mechanisms. Neural Comput. 17, 245–319.
Zemelman, B.V., Nesnas, N., Lee, G.A., and Miesenböck, G. (2003). Photochemical gating of heterologous ion channels: Remote control over genetically
designated populations of neurons. Proc. Natl. Acad. Sci. USA 100, 1352–1357.
Cell 139, 405–415, October 16, 2009 ª2009 Elsevier Inc. 415