Download Neural correlates of a decision in the dorsolateral prefrontal cortex of

© 1999 Nature America Inc. • http://neurosci.nature.com
articles
Neural correlates of a decision in
the dorsolateral prefrontal cortex
of the macaque
Jong-Nam Kim and Michael N. Shadlen
Department of Physiology and Biophysics and Regional Primate Research Center, University of Washington Medical School, Box 357290,
Seattle, Washington 98195-7290, USA
© 1999 Nature America Inc. • http://neurosci.nature.com
Correspondence should be addressed to M.N.S. ([email protected])
To make a visual discrimination, the brain must extract relevant information from the retina,
represent appropriate variables in the visual cortex and read out this representation to decide which
of two or more alternatives is more likely. We recorded from neurons in the dorsolateral prefrontal
cortex (areas 8 and 46) of the rhesus monkey while it performed a motion discrimination task. The
monkey indicated its judgment of direction by making appropriate eye movements. As the monkey
viewed the motion stimulus, the neural response predicted the monkey’s subsequent gaze shift,
hence its judgment of direction. The response comprised a mixture of high-level oculomotor signals
and weaker visual sensory signals that reflected the strength and direction of motion. This combination of sensory integration and motor planning could reflect the conversion of visual motion
information into a categorical decision about direction and thus give insight into the neural computations behind a simple cognitive act.
The brain uses sensory information to form interpretations and
decisions that guide behavior. Such interpretations often outlast the fleeting sensory impressions on which they are based,
so that sensory input can motivate subsequent behavior. To study
this process, we trained rhesus monkeys to discriminate the
direction of motion in a dynamic random dot display1. The difficulty of the task was controlled by varying the fraction of coherently moving dots. At high motion coherences, the animal can
commit to an action the moment the stimulus is seen. In contrast, near psychophysical threshold, the monkey must base its
direction judgments on weak sensory signals perceived over hundreds of milliseconds1.
The extrastriate visual cortex (areas MT and MST) contains
the neural representation of visual motion that allows the monkey
to perform this demanding task1–4. The direction-selective neurons in these areas represent the evidence favoring one direction
or another, but this evidence must be interpreted to reach a decision. This distinction between sensory evidence and decision can
be appreciated by imposing a delay between motion viewing and
behavioral response. Direction-selective neurons stop responding when the motion stimulus is absent5, whereas neurons that
encode a decision must maintain their activity when the visual
motion is no longer present, until the animal responds.
The dorsolateral prefrontal cortex (PFC) is felt to be critical
for tasks with a delay between instruction and execution6,7. During the delay, many PFC neurons show sustained discharge, which
is often selective for a particular object or location8–11. We studied
neurons with sustained activity through the memory/delay period before an eye movement to a restricted region of the visual
field, termed the neural response field (RF). We hypothesized that
such neurons may be involved in linking the sensory evidence in
visual cortex to a behavioral plan to shift the gaze. To test this, we
176
arranged the motion-discrimination task so that one of two
response targets appeared in the neuron’s RF (Fig. 1a). The monkey was trained to make a delayed eye movement to one or the
other response target, depending on the direction of random dot
motion. We found that the activity of many PFC neurons reveals
the monkey’s intention to make a saccade to one or the other target. The time course and intensity of neural activity seem to represent the formation of a decision about motion direction.
RESULTS
We studied 88 neurons in the frontal eye field (FEF) and the posterior third of the principal sulcus (PS) region (areas 8Ar and 46)
that responded selectively when the monkey planned a saccade
to a region of space (the RF). For example, the PS neuron shown
in Fig. 2 responded during the delay preceding saccades to targets that appeared up and to the right, but not down and to the
left of the fixation point (Fig. 2a and b).
To determine this neuron’s behavior during a perceptual decision, we monitored its discharge while the monkey judged the
direction of random dot motion. The random dots appeared in
a five degree aperture outside the neuron’s RF (see Methods).
Motion direction was toward or away from the RF, and motion
strength was varied to span psychophysical threshold. After a delay,
the monkey indicated its direction judgment by making an eye
movement to one of the two targets. If the motion was up-right,
the monkey was rewarded for choosing the target inside the RF.
Conversely, if the motion was down-left, the monkey was rewarded for choosing the other target, outside of the RF.
This neuron’s response predicted the monkey’s decision. The
response was larger on trials in which the monkey’s eyes moved
toward the RF (Fig. 2c, e and g) and attenuated when they moved
away from the RF (Fig. 2d, f and h). The spike discharge from this
nature neuroscience • volume 2 no 2 • february 1999
© 1999 Nature America Inc. • http://neurosci.nature.com
Fig. 1. Behavioral tasks and neua
ron locations. (a, b) Direction-discrimination task. The monkey
gazed at the fixation point for 350
ms. Then two targets appeared,
one of which was in the neural
response field (RF, shaded). After
200–300 ms, the random dot kinemotion delay saccade
b
matogram appeared between the
targets and outside the RF. The
direction of motion was toward
one of the two targets. Motion
strength was varied from trial to
trial by adjusting the percentage of
coherently moving dots. After 1 s,
e
m
the random dots were turned off,
Ti
leaving only the fixation point and
targets. After 0.5–1.5 s, the fixation point was extinguished, signaling the monkey to indicate its
choice by shifting its gaze to one of
the targets. The monkey was
rewarded for choosing the target
along the direction of random dot
c
motion, or randomly when there
d
was no net motion (0% coherflash
delay
saccade
ence). T1 and T2, saccade targets;
FP, fixation point. (c) Average psychometric function for 88 experiments. Error bars are standard
deviations of the proportion of
correct choices. (d) Memory-saccade task used to screen neurons.
A target was flashed (100 ms) at a
random location in the visual field.
The monkey maintained fixation
Motion strength (% coherence)
through a variable delay until the
fixation point was extinguished.
The monkey was then required to
shift its gaze to the remembered
location of the flashed target.
e
(e) Location of the recording cylinf
der in a schematic diagram of the
rhesus monkey brain. (f) Magnetic
resonance imaging. Fast spin-echo,
short-T1, inversion-recovery scan
through the electrode grid of monkey S. This is one of a series of
images obtained in the coronal
plane (slice thickness 1.5 mm). The
recording grid was filled with sterile saline to reveal the angle and
location of electrode guide tubes in the coronal plane. A second series was obtained in the sagittal plane to determine the position and angle
of guide tubes in the anterior–posterior direction. The section shows the arcuate sulcus and prearcuate gyrus at the caudal end of the principal sulcus (ps, principal sulcus; as, arcuate sulcus).
Probability correct (mean ± stdev)
© 1999 Nature America Inc. • http://neurosci.nature.com
articles
neuron began to reveal the monkey’s decision as early as 200–300 ms
after the onset of random dot motion and remained informative
until the saccade. Moreover, the response was modulated more
strongly when the task was easier: the neuron discharged more
intensely when the monkey viewed coherent motion toward the RF
(up-right) and attenuated more profoundly to coherent motion
away from the RF. Thus, the response reflected not only the monkey’s impending eye movement, but also the sensory input that
determined it. This mixture of visual-sensory and visuomotor
response properties is thought to occur at the nexus of sensory-tomotor conversion12–14 where the decision is computed.
nature neuroscience • volume 2 no 2 • february 1999
Although this pattern of response was common in the FEF
and PS region, we also encountered many neurons that modulated their activity only during the delay after the random dot
motion was turned off, presumably after the monkey had reached
its decision. This activity (Fig. 3a) clearly predicted the monkey’s
plan to look to the target in the RF on the memory-saccade task.
However, during the motion-discrimination task, the response
did not reveal the monkey’s decision until the delay (Fig. 3c
and d). During motion viewing, this neuron failed to signal the
direction of the next eye movement. Such responses may reflect
motor preparation, but they provide little insight into the deci177
© 1999 Nature America Inc. • http://neurosci.nature.com
Fig. 2. Response of a principalis neuron during the
motion-discrimination and memory-saccade tasks.
The response field of the neuron is shown in gray.
The arrow indicates the direction of the monkey’s
saccade at the end of the trial. (a, b) Response on
memory saccades to targets in (a) and out (b) of
the RF. The time axis is broken to align the
response to two events: target appearance and
saccade initiation. (c–h) Response during the
motion-discrimination task. Responses are aligned
to the onset of random dot motion and to the
time of the saccade. The left column (c, e, g)
depicts trials in which the monkey decided the
motion was toward the RF. The right column (d, f,
h) shows trials in which the monkey decided that
the direction was away from the RF, leading to a
saccade to the target outside the RF. Three motion
strengths are shown (sub-, near- and supra-threshold). For the 0% coherence stimulus, there is no
net direction of motion. Only correct discrimination trials are shown for nonzero motion
strengths. For clarity, only 10 trials are shown in
the rasters. The response preceding the onset of
motion was associated with fixation and the
appearance of the saccade targets.
a
b
c
d
sion-making process. Unless the monkey
makes eye movements capriciously — a pose
sibility precluded by the well behaved psychometric function (Fig. 1c) — the decision
must form during motion viewing.
For each neuron, we computed an index
of predictive activity using the responses
measured during motion viewing and durg
ing the delay. The index measures the
response distribution overlap from trials in
which the monkey judged motion as
toward versus away from the RF and estimates the probability that an ideal observer could predict the monkey’s decision
based on the spike rate (Methods). For
example, an index of 0.5 indicates a chance
association (complete overlap of response
distributions associated with the two choices), whereas an index of 1 indicates perfect correspondence
between predicted and observed choices (no overlap).
Most neurons (76/88, 86%) predicted the monkey’s choice
reliably during motion viewing or the delay or both (Fig. 4; predictive index > 0.5 and p < 0.01 in at least one epoch). Sixty
percent (53/88) predicted the monkey’s decision reliably during motion viewing (index > 0.5 and p < 0.01) and could reflect
the association between motion processing and behavioral
response. However, about 1/4 of the neurons failed to indicate
the monkeys’ choices until the delay (9/26 neurons in the FEF
and 10/62 in the posterior PS region), which is too late for them
to be involved in the decision process.
Some neurons (11%, 1 FEF, 9 PS) failed to predict the monkey’s response at any time during the discrimination task (values
near 0.5 on both axes, p > 0.01). Five neurons (1 FEF, 4 PS) reliably
predicted the monkey’s choices during motion viewing but in a
pattern opposite to their response on the memory-saccade task
(predictive index < 0.5, p < 0.01). The activity of these five neurons
diminished when the monkey chose the target in the RF, but none
retained this reversed activity pattern during the delay. We tend-
f
h
Spikes/s
© 1999 Nature America Inc. • http://neurosci.nature.com
articles
178
ed to encounter similar patterns of predictive activity in the FEF
and PS region (Fig. 4, p > 0.08, two-dimensional KolmogorovSmirnov test15), which may result from our strategy of sampling
neurons of similar type in both areas.
Time course of predictive activity
To perform above chance on this task, the monkey must decide
about the motion direction based on the evidence acquired
during motion viewing. Those neurons that predicted the monkey’s subsequent eye movement during motion viewing may
therefore divulge properties of the decision process itself, the
linkage between sensory processing and saccade planning. A
neural correlate of the decision process is likely to reflect both
the outcome of the decision and the quality of the evidence
upon which it is founded. Moreover, neurons linking sensory
evidence to a behavioral response should undergo a temporal
transformation in their activity as the evidence produces a categorical answer. Early responses might reflect properties of the
sensory stimulus (the strength of the evidence). Responses later
in the decision process should reflect a stereotyped outcome,
nature neuroscience • volume 2 no 2 • february 1999
© 1999 Nature America Inc. • http://neurosci.nature.com
articles
Fig. 3. Response of a frontal eye field neuron during the motion-discrimination and
memory-saccade tasks. (a, b) Response on
memory-guided saccades to targets flashed
briefly inside or outside the response field.
This neuron had a prominent visual
response at the onset of the target and a
sustained response when the monkey
planned an eye movement up-leftward.
(c, d) Response on the motion-discrimination task. The poststimulus time histogram
includes all trials in which the monkey
chose the correct direction. The response
was stronger when the monkey judged the
motion to be toward the RF, but the effect
was not apparent until the delay (lower
arrows). Responses are aligned to the onset
of random dot motion and to the saccade
initiation. The transient response at the
time of motion onset was caused by the
appearance of saccade targets.
Movement/memory
response field
b
c
d
regardless of whether the evidence was strong or weak, and
whether it was interpreted correctly or incorrectly. These properties are evident in the PFC response.
Motion strength affected the response of many PFC neurons
(Fig. 5a and b). For this analysis, we selected neurons that predicted the monkey’s subsequent choice during motion viewing
(n = 53 neurons with predictive index > 0.5 and a significant
permutation test, p < 0.01). For each neuron, we computed the
average spike rate during motion viewing (from 200–800 ms
after the onset of dot motion) and normalized the spike rate to
the mean. We applied this procedure separately for the two
directions of motion and restricted the analysis to correct choices. For motion toward the RF, the degree of response enhancement varied by 12.5% across the range of motion strengths
(~3.2 spikes/s; Fig. 5a). For motion away from the RF, the
degree of suppression varied by 22% across the range of motion
strengths (~4.3 spikes/s; Fig. 5b).
This normalization procedure removes the main determinant
of the response magnitude for these neurons: whether an eye
movement is ultimately made to the RF or away from it. The effect
of motion strength is therefore relatively subtle. The regression
analysis was statistically significant (p = 0.0032 and p < 0.00001
for Fig. 5a and b, respectively) and was unaffected by the incorporation of eye movement descriptors such as saccadic latency,
amplitude, duration and velocity (see Methods). This last point
implies that subtle variations in the actual saccade produced in
each trial of the experiment does not explain the variation in neural response found as a function of task difficulty.
We represented the evolution of predictive activity during
motion discrimination by calculating the predictive index in 250ms epochs beginning 500 ms before the onset of random dot
motion and ending just after the saccade (Fig. 5c). We computed
the index separately for each of the 53 neurons that predicted the
monkey’s behavior during motion viewing (as above), using only
nature neuroscience • volume 2 no 2 • february 1999
correct choices at each of six motion strengths. On average, the
response began to predict the monkey’s decision 100–200 ms after
onset of the random dot motion. Later in the trial, the response
predicted the monkey’s choice with greater fidelity, reaching a maximum during the delay, just before the eye movement. Although
each point represents the predictive activity from just 250 ms of
spike discharge, the curves look like cumulative functions.
Predictive index delay period
© 1999 Nature America Inc. • http://neurosci.nature.com
Spikes/s
a
Predictive index motion viewing period
Fig. 4. Predictive activity for 88 neurons during motion viewing and the
delay. The predictive index approximates the accuracy with which one
could guess monkey’s decision based on the spike discharge measured
during motion viewing or the delay (Methods). Values larger than 0.5
imply greater accuracy in predicting the monkey’s decision from the
neural response. The histograms summarize the distribution of these
indices; shading indicates a significant departure from 0.5 (p < 0.01 by
permutation test; Methods). Blue circles, FEF; red circles, PS (area 8Ar
and Walker area 46). The open symbols indicate the neurons shown in
Figs. 2 (red) and 3 (blue).
179
© 1999 Nature America Inc. • http://neurosci.nature.com
a
b
Normalized response
Fig. 5. Effect of motion strength on the magnitude and time
of the prefrontal response. (a, b) Effect of stimulus strength
on average response during motion viewing for 53 neurons
with statistically significant predictive indices. (a) For decisions favoring motion toward the RF, the response was larger
to stronger random dot motion. The ordinate represents the
response strength relative to the mean for all decisions
toward the RF. Filled points represent the mean ± standard
error of the normalized response. (b) For decisions favoring
motion away from the RF, the response was more suppressed to stronger motion stimuli. Conventions are the
same as in (a) except that the response is normalized to the
mean of each neuron’s response associated with choices outside the RF. (c) The predictive power of the response was
computed in 250-ms epochs whose midpoint is plotted on
the time axis. Each point represents the probability of correctly predicting the monkey’s choice from 250 ms of spike
discharge. Curves represent the averaged probabilities from
53 neurons with predictive activity during motion viewing.
The neurons predicted the monkey’s choice sooner and
more reliably when the motion was stronger.
Motion strength (% coherence)
c
Saccade
Probability (mean)
© 1999 Nature America Inc. • http://neurosci.nature.com
articles
The emergence of predictive activity on the most
difficult motion condition (no coherent motion) indicates that the neurons primarily encode variables that
pertain to the monkey’s behavioral response. On the
other hand, when the motion was stronger (that is, easier), the response predicted the monkey’s choice sooner and better than when the motion was weaker. This
effect was subtle for individual neurons but highly reliable across the population (p < 0.00001, likelihood ratio
test using modified probit analysis; Eq. 3 in Methods).
It reflects a stronger and more consistent rise in the
response when motion was toward the RF and a more
profound attenuation of the discharge when motion
was toward the target outside the neuron’s RF (Fig. 2).
This response pattern could represent the accumulation of
motion information from the extrastriate cortex toward a plateau.
Indeed, some neurons responded to the direction of random dot
motion during passive viewing, when the monkey was not
engaged in the discrimination task (Fig. 6). The weak direction
bias observed in this control experiment favored motion toward
the RF, suggesting that instructive visual signals reach the PFC
even without a task.
Errors
Could passive visual responses explain our finding of predictive
responses during motion viewing of the discrimination task? This
seems unlikely because the neurons were predictive when the
motion strength was negligible, as in the 0% coherent motion
condition (Figs. 2c and d and 5). The neural response is dominated by the direction that the monkey judges, and hence the
direction of the planned saccade. This point is further supported
by examining error trials, in which the motion direction and the
saccade choice are opposed. The response was larger when the
monkey chose the target in the RF, whether or not this was based
on a proper interpretation of the random dot motion (Fig. 7).
Thus, the response was dominated by what the monkey planned
to do, rather than by what the monkey saw.
There was, however, a subtle difference between correct and
erroneous decisions. On average, neurons modulated their
response less strongly on the error trials. During motion viewing,
the response was weaker when the monkey’s choice of the target in
180
Time (s)
the RF was erroneous as compared to correct (p = 0.001; t-test of
mean normalized response), and the response was less attenuated
on erroneous decisions favoring motion away from the RF
(p < 0.00001). Thus, the response cannot be explained solely by
the behavioral outcome. The observation is consistent with the
idea that the neural response reflects the accumulation of sensory ‘evidence’ toward a decision. As shown in the next section, this
is because the strength of neural signals favoring the wrong direction (for example, the responses of rightward motion sensors
when the direction is actually leftward) are unlikely to exceed by
much the neural signals that would favor the correct direction.
Theory
Our results are consistent with the idea that neurons in the dorsolateral PFC compare sensory signals from the extrastriate cortex that favor motion toward or away from the response field.
Accumulated spike counts from pools of neurons in areas MT
and MST can account for the monkey’s sensitivity and trial-totrial choices on the random dot motion task16. We therefore
expect that neurons involved in the decision process must accumulate spikes from groups of sensory neurons and make the
appropriate comparison, and that the neural computations
underlying a decision probably involve integration in time.
The sensory signals that inform the monkey’s choices on a
near-threshold discrimination (Fig. 8) are presumed to be the
accumulated spike counts from pools of noisy and weakly correlated direction sensors in the extrastriate visual cortex16. These
nature neuroscience • volume 2 no 2 • february 1999
© 1999 Nature America Inc. • http://neurosci.nature.com
a
b
Direction away from response field
articles
Fig. 6. Response to random dot motion during passive viewing. (a) Direction-tuning function for one neuron. This neuron’s RF was near the upper
vertical meridian at an eccentricity of 10 degrees. The random dots were shown in a five-degree aperture centered on the fixation point, outside the
RF of the neuron. The response histograms are displayed to indicate the direction of random dot motion. Responses are aligned to motion onset. The
polar plot shows the mean response calculated during viewing. The response was largest when motion was toward the RF. (b) Summary data from 10
neurons tested during passive fixation, comparing responses to passively viewed motion toward or away from the RF. Filled symbols denote neurons
with significant predictive activity during motion viewing on discrimination trials (done in a separate block). These neurons had a stronger direction
bias (points farther from diagonal; p < 0.001, ROC area comparison and permutation test; see Methods). Error bars represent standard error.
with weaker evidence. At the near-threshold motion strength in
accumulations constitute the evidence for a ‘rightward’ or ‘leftthis example, approximately 24% of the trials would result in
ward’ choice. Imagine a PFC neuron whose RF is situated so that
errors. For erroneous rightward choices (Fig. 8f), the difference
a rightward decision increases its response. According to our
signals on error trials tend to be small (compare Fig. 8d and f).
scheme, this neuron compares the rightward cumulant to the leftThe expected mean is 0.84 units of signal standard deviation.
ward cumulant. The monkey chooses right when this difference is
This analysis could explain the subtle difference in the neural
positive and left when it is negative. The magnitude of the differresponse that we observed on correct versus error trials (Fig. 7).
ence represents the strength of the evidence favoring a decision.
This same analysis can be extended across the range of stimuFigure 8a shows idealized distributions of pooled responses
lus strengths (Fig. 8g). At each motion strength, there are four
from leftward and rightward sensors to a near-threshold rightpossible outcomes: two directions times two choices. At all motion
ward motion stimulus. The size of the responses is expressed in
strengths, whenever the monkey chooses rightward — correct or
units of standard deviation (σ). Notice that the distribution of
not — the evidence for rightward motion (Fig. 8g) exceeds the
pooled rightward signals exceeds that of the leftward signals by
evidence for leftward. However, the strength of the evidence
1 σ on average (that is, the index of discriminability, d´, equals 1),
increases with stronger motion (larger values of d´), leading to
consistent with a motion strength that would support 76% correct choices. When the stimulus direction is leftward, the leftward signals are
larger on average (Fig. 8b).
This analysis permits us to assess
Fig. 7. Comparison of the responses on
the strength of the evidence associated
error and correct trials. The responses from
with each of four types of decisions as
53 neurons with predictive activity during
seen from the point of view of a
motion viewing were used to construct these
‘choose right’ neuron in the PFC (Fig.
curves, using all non-zero motion strengths.
8c–f). When motion is rightward and
Responses were normalized to the mean of
each neuron’s response over the 600-ms
the correct choice follows (Fig. 8d), the
epoch indicated by the gray bar. The black
strength of the evidence favoring rightcurves depict trials in which the monkey
ward is positive and large on average.
judged the direction as toward the RF, culmiThe expected mean difference between
nating in a saccade to the target in the RF
rightward and leftward sensory signals
(T1); the gray curves indicate decisions
is 1.6 units of signal standard deviaresulting in a saccadic eye movement to the
tion. When motion is leftward and the
target outside the RF (T2). Error trials are
correct choice follows, the evidence
indicated by the dashed curves. The promifavoring a rightward choice is negative
nent response that begins before the onset of
motion was caused by the presentation of
and large (Fig. 8e; expected mean,
Time from motion onset
the saccade targets.
–1.6 σ). The error trials are associated
Response (mean normalized)
© 1999 Nature America Inc. • http://neurosci.nature.com
Direction toward response field
nature neuroscience • volume 2 no 2 • february 1999
181
© 1999 Nature America Inc. • http://neurosci.nature.com
articles
Distribution of signals from ‘rightward’ and ‘leftward’ sensors
Motion is rightward (d´ = 1)
Motion is leftward (d´ = 1)
b
Distribution of difference signals received by a ‘choose right’ neuron
c
Wrong ‘left’ choice
SRight–SLeft (σ)
d
Correct ‘right’ choice
SRight–SLeft (σ)
e
Correct ‘left’ choice
SRight–SLeft (σ)
f
Wrong ‘right’ choice
SRight–SLeft (σ)
Fig. 8. Theoretical basis for variation in the strength of evidence associated with direction judgg
ments. The graphs trace the flow of information from populations of opponent direction sensors to
a neuron in the PFC that would increase its response in association with a rightward decision. The
input to the PFC neuron represents the evidence that motion is rightward, in the form of the difference between rightward and leftward sensory signals. Four scenarios are depicted: motion is either
leftward or rightward, and the choice is either correct or erroneous. (a) Activity of the leftward and
rightward motion sensors when the true direction is rightward. The surface shows the joint probability distribution of a pair of signals from rightward and leftward sensors. The two signals should be
interpreted as the population average responses from many neurons. The graph depicts the situation
at psychophysical threshold when the rightward signal is one standard deviation larger than the leftward signal, on average (projected bell-shaped curves). The points below the surface are 100 ranStimulus strength (d´)
dom samples from the overlying distribution. Each point is labeled according to the resulting
decision (blue, rightward decision; red, leftward decision). The majority of samples fall on the side of
the main diagonal that indicates that the rightward signal exceeds the leftward signal. In this plot, the rightward choices are correct. (b) Activity of the
motion sensors when the true direction is leftward. Same conventions as in (a). The leftward sensors produce the larger signal, on average. In this plot,
the leftward choices (red) are correct. (c–f) Frequency histograms of the difference in sensory signals between rightward and leftward sensors. The differences are tabulated separately for the two directions of motion and the two decisions. The histograms are shown under the simulated points from
which they were obtained in (a) and (b). The difference, SRight – SLeft, is positive when the decision is rightward (blue) and negative when the decision is
leftward (red). Notice that there are more samples corresponding to correct choices (d and e), and the difference signals tend to be larger positive and
negative values. (g) Average difference signal, SRight – SLeft, accompanying correct and incorrect choices for a range of motion strengths spanning the
psychometric function. The d´ = 1 case corresponds to a motion strength supporting 76% correct choices (~10% coherence), as illustrated in (a–f). The
theoretical means for (c–f) are shown. When d´ = 0, there is no net motion direction (0% coherence), and there is no distinction between correct and
error trials. When d´ = 2, there are very few errors (≥ 25% coherence). The difference signals associated with correct choices attain greater positive
and negative values when the motion is stronger. The opposite trend is predicted for error trials.
Evidence for ‘rightward’ (σ)
© 1999 Nature America Inc. • http://neurosci.nature.com
a
larger responses when the decision is rightward and greater suppression when the decision is leftward. The increasing separation
of the solid curves in Fig. 8g helps explain the increase in predictive index as a function of motion strength (see Fig. 5). The analysis also predicts that the responses associated with errors should
diminish at higher motion strengths. We cannot evaluate this prediction because errors occur rarely for strong motion.
Finally, this analysis helps to explain the relatively subtle
variation in response that we observed in the PFC as a function
of motion strength. Across the entire psychometric function,
182
the ‘evidence for rightward’ changes by only one unit of signal
standard deviation for correct choices that lead to the same
behavioral response (Fig. 8g). The unit, σ, can be related to the
responses of MT neurons. At a d´ value of 1, the response of
pooled rightward-preferring MT neurons exceeds the response
of pooled leftward-preferring MT neurons by one standard
deviation of the pooled values. Recordings from area MT show
that at a motion strength of 10% (corresponding to d´~1),
rightward- and leftward-preferring neurons differ in their
responses by ~5 spikes/s1,17. According to the analysis (Fig. 8g),
nature neuroscience • volume 2 no 2 • february 1999
© 1999 Nature America Inc. • http://neurosci.nature.com
articles
the monkey’s judgments at the weakest motion strength (0%
coherence) are based on evidence of ± 1.13 σ, or ~5 spikes/s
from the average MT neuron. We do not know how to convert
this value to a spike rate in the PFC; in the example we are pursuing, all rightward choices are associated with a high spike rate.
However, as the motion strength increases to span the monkey’s psychometric function, the strength of the evidence
increases (or decreases) by only another unit of σ or ~5 spikes/s.
This value is comparable to the change in PFC response that
we observed across the range of motion strengths used in our
experiments (see Fig. 5a, b and legend).
© 1999 Nature America Inc. • http://neurosci.nature.com
DISCUSSION
The dorsolateral prefrontal cortex is important in linking sensation to action, especially when the linkage involves a delay6,7.
Our study provides a glimpse of this linkage in the activity of
individual PFC neurons. We used a task in which delayed saccades were instructed by a direction judgment. By requiring
the animal to make difficult judgments near psychophysical
threshold, we were able to characterize neural activity during
a period in which sensory processing gradually gave way to a
categorical choice, what we have termed a decision process.
Our results demonstrate a gradual unfolding of neural activity, which we interpret as a correlate of the monkey’s decision.
To find a neural correlate of the monkey’s decision about
the direction of motion, we searched for neurons that signaled
the monkey’s commitment to a particular behavioral option.
We therefore studied neurons in the FEF and principalis region
that showed sustained activity during an oculomotor delayedresponse task. These neurons could be said to lie toward the
‘motor’ side of a sensation–action continuum 14, linking the
visual cortex with the motor system 18.
The sustained response of PFC neurons has been interpreted as a neural correlate of short-term memory for spatial
location19,20, but we do not know if such neural activity represents the instruction (for example, location of a spatial cue)
or the behavioral plan to move the eyes to that location. Our
results suggest that the response may conflate these representations. Most neurons modulated their response in association with all the key ingredients of the discrimination task:
the appearance of a target within the response field, the direction and strength of random dot motion and the direction of
a planned saccade. Our analysis showed that early in the task,
during formation of the decision, the modulation in neural
response varies parametrically with the strength of visual
motion (Fig. 5). By the end of this process, many neurons in
the PFC reflect the monkey’s choice. These observations suggest that the PFC represents not simply the final outcome of
sensory processing but the conversion of an analog motion
representation to a binary decision variable.
This conclusion must be viewed cautiously because we do
not know the moment that the monkey decides the direction
of motion, or indeed if there is such a discrete moment. It is
possible that on different trials, the monkey reaches its decisions at different times and the neural responses change shortly thereafter to reveal the intended action. The average
response from many trials might give the appearance of a
gradual evolution of the monkey’s plan (Fig. 5c). If the decision were to occur sooner, on average, for easier discriminations, then this could explain the parametric variation in
response magnitude that we observed with different motion
strengths (Fig. 5a and b). We have looked for discrete changes
in the firing patterns of our neurons using a previously
nature neuroscience • volume 2 no 2 • february 1999
described algorithm21, but we have failed to find evidence for
such abrupt transitions. Simultaneous recordings from two
or more neurons could, in principle, facilitate detection of
such abrupt changes in firing pattern.
Our observations support an emerging view that the distinction between sensory and motor systems may be blurred
within the association areas of the cerebral cortex18,22,23. In
many association areas, neural responses are predicted by
stimulus qualities as well as motor preparation. It is tempting
to regard the former as instructing the latter. For example,
FEF neurons represent features of a visual stimuli that instruct
an eye movement in a visual search task24. Neurons in area 46
respond at the moment an instruction is given to guide a
future eye movement11. Even neural responses in the primary
and supplementary motor cortex reflect not only the planned
behavior but the sensory cue that instructs that action12,25–31.
Any brain region containing signals related to both sensory processing and motor preparation may be involved in the
conversion of the former to the latter. For most tasks, however, there is insufficient time to study the process of conversion: the moment the instruction is received, the animal can
prepare an action. In the threshold discrimination task that
we used, the monkey cannot process the instruction instantly. The monkey, like human subjects, benefits from the temporal accumulation of information 1,32 (J.D. Roitman &
M.N.S., Soc. Neurosci. Abstr. 24, 262, 1998). Our results suggest
that prefrontal neurons may do more than hold information
in short-term memory. They seem to be involved in the accumulation (that is, integration) and comparison of sensory
streams toward a categorical outcome or behavioral plan.
This does not imply that the neurons we recorded are
directly responsible for deciding direction. They may reflect
the computation made at another site in the brain. In fact, the
neurons described here respond similarly to neurons in the
lateral intraparietal area (LIP) and the superior colliculus33,34
during the same motion discrimination task. This is not surprising because area LIP, the FEF and the posterior principalis
region are strongly connected with each other35–38. Both LIP
and the dorsolateral PFC seem to be activated during similar
delayed eye movement tasks39, and both areas project to the
superior colliculus36,40–42. The relative importance of these
brain regions will need to be addressed with reversible inactivation or simultaneous recording. Meanwhile, the activity in
the prefrontal and inferior parietal cortex reveals something
about the computations that may underlie the linkage between
sensory data, interpretation and behavioral planning. We propose that such neurons compute the time integral of sensory
‘evidence’ toward a plateau state.
Neurons whose dominant mode of response signals the plan
to enact a behavior must be influenced by sensory signals. The
existence of such a link comes as no surprise, but the mixture
of signals on single neurons, reflecting motor planning and
sensory ‘strength’, constrains a view of the brain’s logical architecture. It seems inconsistent with a central executive function
that interprets the sensory data, declares an interpretation and
recruits circuitry to enact a response. Instead, it supports a
view of brain organization that would recruit premotor circuitry in the interest of several potential actions while querying
sensory streams for evidence to select the appropriate one.
METHODS
Electrophysiology. We recorded from 88 neurons in the frontal eye
field (FEF, areas 8Ac and 45a) and posterior principalis region (areas
8Ar and Walker area 46) of two rhesus monkeys trained on a random
183
© 1999 Nature America Inc. • http://neurosci.nature.com
articles
© 1999 Nature America Inc. • http://neurosci.nature.com
dot motion discrimination task 1. Monkeys were implanted with an
eye coil, head-holding device and recording cylinder suitable for magnetic resonance imaging (MRI; Crist Instrument Co., Damascus, Maryland). The recording cylinder was placed over the arcuate sulcus and
the posterior third of the principal sulcus (PS; Fig. 1e). Sterile guide
tubes were placed through a plastic grid (Crist Instruments) to introduce tungsten/glass microelectrodes to the surface of the dura mater.
The grid array was visualized in situ by MRI and registered with the
anatomy (Fig. 1f). We used the MRI to identify the FEF in the anterior bank of the arcuate sulcus and to distinguish the area principalis
from more caudal portions of the prearcuate gyrus. Action potentials
were identified using a dual-voltage, time-window discriminator (Bak
Electronics, Germantown, Maryland) and stored on computer with 1ms precision43. All training, surgery and experimental procedures complied with the National Institutes of Health Guide for the Care and
Use of Laboratory Animals and were approved by the University of
Washington Animal Care Committee.
Electrical stimulation. To aid in identifying recording sites within the
frontal eye field, we attempted to elicit saccades using the electrical
microstimulation protocol described 44. We classified sites as ‘low
threshold’ if we could elicit a fixed-vector saccade with a stimulating
current less than 50 µA. We classified cells as in or out of the FEF
based on their location in the anterior bank of the arcuate sulcus and
their proximity to low-threshold stimulation sites. This procedure
unequivocally classified most neurons.
normalized spike rate and C is the rank motion strength (0 to 5). For
each trial, we also extracted five descriptors of the saccade: its latency, amplitude, accuracy, maximal speed and duration. We included
these factors along with motion strength in a multivariate regression
analysis, fitting the model,
Y = β0 + β1C + β2LAT + β3AMP + β4ACC + β5VMAX + β6DUR + ε
Analysis of predictive activity. For several analyses, we computed an
index that describes the association between neural response and the
monkey’s decision. This index, the probability that the neural
response associated with one behavioral choice exceeds the neural
response associated with the other behavioral choice, approximates
the ability of the experimenter to predict the monkey’s behavior from
the neural response. Denoting the responses associated with the two
choices by {r1} and {r2}, this is the joint probability of observing r1 = k
and r2 < k, over all possible values of k:
∞
∫ Pr(r = k)Pr(r <k)dk
= ∫ Pr(r = k) [∫ Pr(r =µ)dµ]dk
Predictive index =
1
-∞ ∞
Data analysis. Responses were analyzed off-line using custom software. To combine data from several neurons, we first normalized
each neuron’s response using the mean spike rate in an epoch from
200–800 ms after the onset of random dot motion. This epoch corresponds to motion viewing, when the monkey decides the direction of motion and after the prominent transient response to the
onset of saccade targets.
The effect of motion strength on the neural response was estimated using a linear regression model, Y = β0 + β1C + ε where Y is the
184
2
k
1
-∞
Behavioral tasks. Neurons were selected using a delayed-saccade task
that relies on working memory (Fig. 1d)8,45,46. We studied neurons
that responded during the delay (memory) period preceding saccades
to a restricted region of the visual field. We refer to this region as the
neural response field (RF) to remain agnostic as to function. Many
neurons also responded transiently at the onset of movement or to the
onset of saccade targets, but this was not a criterion in their selection.
Similar selection criteria were used in a related study of area LIP33.
In the motion-discrimination task (Fig. 1a and b), two response
targets appeared. One target was in the RF of the neuron; the random
dot kinematogram and the second response target appeared outside
the RF. After a brief pause (200–300 ms), the random dot motion was
displayed for 1 s. The direction of motion was toward one or the other
target. Both the direction of motion and the fraction of coherently
moving dots were randomized. After a delay, the monkey indicated its
direction judgment by making an eye movement to one of the two
response targets. The monkey was trained to associate rightward
motion with the target to the right of the dots aperture, upward motion
with a target above the aperture, and so forth. The monkey received a
liquid reward for correct responses (and on half of the trials in which
the 0% coherent motion was shown). In all experiments, the monkeys
showed a smooth improvement in performance as a function of
motion strength (Fig. 1c). The average threshold motion strength supporting 81% correct choices was 12.9% (range 1.5 to 34.1%; standard
deviation, 5.2%), which is typical of highly trained rhesus monkeys
in similar studies1,16.
We occasionally held neurons long enough to do a passive-fixation
control experiment (Fig. 6). No saccade targets appeared on this block
of trials, and the monkey was rewarded simply for maintaining fixation. Random dot motion (51.2% coherence) appeared in the same
location as in the discrimination block and moved toward the RF or
in one to seven directions away from the RF.
(1)
The fit to equation 1 allowed us to test whether motion strength
affects the neural response in a manner that cannot be attributed to
variation in saccadic eye movements. This is a test of the null hypothesis, β 1 = 0, which is done by comparing the extra sum of squares
with and without this regressor and computing an F ratio47.
(2)
2
-∞
Equation 2 can be estimated by computing the area under a receiver-operator-characteristic (ROC) curve obtained from the two
response distributions1,48.
The distribution of the predictive index under the null hypothesis
is typically not normal. We used a permutation test to estimate the
probability that the measured index would be observed under the
null hypothesis of a random association between neural and behavioral response (that is, the true index is 0.5). Each neural response
and each behavioral response were randomly associated to construct
the two distributions, {r1´} and {r2´}, which therefore contain the same
number of observations as the original {r1} and {r2}. The predictive
index was computed using Equation 2, and the distribution of its
absolute difference from 0.5 was estimated using 1000 permutations
of the data. The probability of obtaining the measured index under
the null hupothesis is the fraction of this distribution exceeding the
absolute value of the measured index’s difference from 0.5.
The same procedure was used to measure of the time course of predictive activity (Fig. 5c) by computing the predictive index from spike
counts obtained from 250-ms epochs at intervals designated with
respect to the onset random dot motion or the monkey’s saccadic eye
movement. The resulting sigmoid functions are well fit by the scaled
integral of a Gaussian function of time:
p(t) = a0 + a1C + (b0 + b1C)Φ(t),
where
t
[τ–(µ0 + µ1C]2
–
1
Φ(t) =
e 2[σ0 + σ1C]2 dτ
√2 π (σ0 + σ1C) -∞
∫
(3)
Notice that Φ(t) is a cumulative Gaussian function of time, parameterized by its mean and standard deviation: the mean determines the
position of the sigmoid from left to right, and the standard deviation
determines its slope. We modeled the baseline values, a; the scaling
terms, b; and the sigmoid parameters, µ and σ, as linear functions of the
motion strength, C. We fit equation 3 to the measured predictive indices
using a simplex algorithm to find the maximum likelihood solution
(assuming binomially distributed error). The null hypothesis that
motion strength does not affect the shape of these sigmoid functions
was tested by fitting the model with a1 = b1 = µ1 = σ1 = 0 and comparing likelihoods under the full and reduced models (likelihood ratio
test, df = 4)49. We refer to this method as a modified probit analysis.
nature neuroscience • volume 2 no 2 • february 1999
© 1999 Nature America Inc. • http://neurosci.nature.com
articles
ACKNOWLEDGEMENTS
© 1999 Nature America Inc. • http://neurosci.nature.com
We thank Melissa Mihali for animal training and technical support. We also
thank Joshua Gold, Greg Horwitz, Mark Mazurek, Bill Newsome, Jeff Schall
and Kirk Thompson for helpful suggestions on the manuscript. This research
was supported by RR00166, EY11378 and the McKnight Foundation.
25.
26.
RECEIVED 24 AUGUST: ACCEPTED 23 DECEMBER 1998
27.
1. Britten, K. H., Shadlen, M. N., Newsome, W. T. & Movshon, J. A. The
analysis of visual motion: a comparison of neuronal and psychophysical
performance. J. Neurosci. 12, 4745–4765 (1992).
2. Newsome, W. & Pare, E. A selective impairment of motion perception
following lesions of the middle temporal visual area (MT). J. Neurosci. 8,
2201–2211 (1988).
3. Salzman, C. D., Murasugi, C. M., Britten, K. H. & Newsome, W. T.
Microstimulation in visual area MT: Effects on direction discrimination
performance. J. Neurosci. 12, 2331–2355 (1992).
4. Celebrini, S. & Newsome, W. T. Neuronal and psychophysical sensitivity
to motion signals in extrastriate area MST of the macaque monkey. J.
Neurosci. 14, 4109–4124 (1994).
5. Seidemann, E., Zohary, E. & Newsome, W. T. Temporal gating of neural
signals during performance of a visual discrimination task. Nature 394,
72–75 (1998).
6. Jacobsen, C. F. Functions of frontal association area in primates. Arch.
Neurol. Psychiatry 33, 558–569 (1935).
7. Fuster, J. The Prefrontal Cortex (Raven, New York, 1989).
8. Funahashi, S., Bruce, C. & Goldman-Rakic, P. Mnemonic coding of visual
space in the monkey’s dorsolateral prefrontal cortex. J. Neurophysiol. 61,
331–349 (1989).
9. Wilson, F. A., Scalaidhe, S. P. & Goldman-Rakic, P. S. Dissociation of
object and spatial processing domains in primate prefrontal cortex.
Science 260, 1955–1958 (1993).
10. Miller, E. K., Erickson, C. A. & Desimone, R. Neural mechanisms of visual
working memory in prefrontal cortex of the macaque. J. Neurosci. 16,
5154–5167 (1996).
11. Hasegawa, R., Sawaguchi, T. & Kubota, K. Monkey prefrontal neuronal
activity coding the forthcoming saccade in an oculomotor delayed
matching-to-sample task. J. Neurophysiol. 79, 322–334 (1998).
12. Zhang, J., Riehle, A., Requin, J. & Kornblum, S. Dynamics of single
neuron activity in monkey primary motor cortex related to sensorimotor
transformation. J. Neurosci. 17, 2227–2246 (1997).
13. Riehle, A., Kornblum, S. & Requin, J. Neuronal coding of stimulusresponse association rules in the motor cortex. Neuroreport 5, 2462–2464
(1994).
14. Leon, M. I. & Shadlen, M. N. Exploring the neurophysiology of decisions.
Neuron 21, 669–672 (1998).
15. Press, W. H., Teukolsky, S. A., Vetterling, W. T. & Flannery, B. P. Numerical
Recipes in C. The Art of Scientific Computing (Cambridge Univ. Press,
Cambridge, 1992).
16. Shadlen, M. N., Britten, K. H., Newsome, W. T. & Movshon, J. A. A
computational analysis of the relationship between neuronal and
behavioral responses to visual motion. J. Neurosci. 16, 1486–1510 (1996).
17. Britten, K. H., Shadlen, M. N., Newsome, W. T. & Movshon, J. A.
Responses of neurons in macaque MT to stochastic motion signals. Vis.
Neurosci. 10, 1157–1169 (1993).
18. Goodale, M. A. Visuomotor control: Where does vision end and action
begin? Curr. Biol. 8, R489–491 (1998).
19. Goldman-Rakic, P. in Handbook of Physiology, Sec I, The Nervous System
(ed. Plum, F.) 373–417 (Am. Physiol. Soc., Bethesda, MD, 1987).
20. Funahashi, S., Bruce, C. & Goldman-Rakic, P. Dorsolateral prefrontal
lesions and oculomotor delayed-response performance: evidence for
mnemonic “scotomas”. J. Neurosci. 13, 1479–1497 (1993).
21. Commenges, D., Seal, J. & Pinatel, F. Inference about a change point in
experimental neurophysiology. Math. Biosci. 80, 81–108 (1986).
22. Seal, J. & Commenges, D. A quantitative analysis of stimulus- and
movement-related responses in the posterior parietal cortex of the
monkey. Exp. Brain Res. 58, 144–153 (1985).
23. Seal, J. Sensory and motor functions of the superior parietal cortex of the
monkey as revealed by single-neuron recordings. Brain Behav. Evol. 33,
113–117 (1989).
24. Thompson, K. G., Bichot, N. P. & Schall, J. D. Dissociation of visual
28.
nature neuroscience • volume 2 no 2 • february 1999
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
48.
49.
discrimination from saccade programming in macaque frontal eye field.
J. Neurophysiol. 77, 1046–1050 (1997).
di Pellegrino, G. & Wise, S. Visuospatial versus visuomotor activity in the
premotor and prefrontal cortex of a primate. J. Neurosci. 13, 1227–1243
(1993).
Salinas, E. & Romo, R. Conversion of sensory signals into motor
commands in primary motor cortex. J. Neurosci. 18, 499–512 (1998).
Romo, R., Ruiz, S., Crespo, P., Zainos, A. & Merchant, H. Representation
of tactile signals in primate supplementary motor area. J. Neurophysiol.
70, 2690–2694 (1993).
Shen, L. & Alexander, G. E. Neural correlates of a spatial sensory-tomotor transformation in primary motor cortex. J. Neurophysiol. 77,
1171–1194 (1997).
Shen, L. & Alexander, G. E. Preferential representation of instructed
target location versus limb trajectory in dorsal premotor area. J.
Neurophysiol. 77, 1195–1212 (1997).
Alexander, G. E. & Crutcher, M. D. Neural representations of the target
(goal) of visually guided arm movements in three motor areas of the
monkey. J. Neurophysiol. 64, 164–178 (1990).
Quintana, J. & Fuster, J. Mnemonic and predictive functions of cortical
neurons in a memory task. Neuroreport 3, 721–724 (1992).
Watson, A. B. Probability summation over time. Vision Res. 19, 515–522
(1979).
Shadlen, M. N. & Newsome, W. T. Motion perception: seeing and
deciding. Proc. Natl. Acad. Sci. USA 93, 628–633 (1996).
Newsome, W. T. Deciding about motion: linking perception to action. J.
Comp. Physiol. A 181, 5–12 (1997).
Stanton, G. B., Bruce, C. J. & Goldberg, M. E. Topography of projections
to posterior cortical areas from the macaque frontal eye fields. J. Comp.
Neurol. 353, 291–305 (1995).
Lynch, J. C., Graybiel, A. M. & Lobeck, L. J. The differential projection of
two cytoarchitectonic subregions of the inferior parietal lobule of
macaque upon the deep layers of the superior colliculus. J. Comp. Neurol.
235, 241–254 (1985).
Huerta, M. F., Krubitzer, L. A. & Kaas, J. H. Frontal eye field as defined by
intracortical microstimulation in squirrel monkeys, owl monkeys, and
macaque monkeys. II. Cortical connections. J. Comp. Neurol. 265,
332–361 (1987).
Schall, J. D., Morel, A., King, D. J. & Bullier, J. Topography of visual cortex
connections with frontal eye field in macaque: convergence and
segregation of processing streams. J. Neurosci. 15, 4464–4487 (1995).
Friedman, H. & Goldman-Rakic, P. Coactivation of prefrontal cortex and
inferior parietal cortex in working memory tasks revealed by 2DG
functional mapping in the rhesus monkey. J. Neurosci. 14, 2775–2788
(1994).
Huerta, M. F., Krubitzer, L. A. & Kaas, J. H. Frontal eye field as defined by
intracortical microstimulation in squirrel monkeys, owl monkeys, and
macaque monkeys: I. Subcortical connections. J. Comp. Neurol. 253,
415–439 (1986).
Segraves, M. A. & Goldberg, M. E. Functional properties of corticotectal
neurons in the monkey’s frontal eye field. J. Neurophysiol. 58, 1387–1419
(1987).
Stanton, G. B., Goldberg, M. E. & Bruce, C. J. Frontal eye field efferents in
the macaque monkey: I. Subcortical pathways and topography of striatal
and thalamic terminal fields. J. Comp. Neurol. 271, 473–492 (1988).
Hays, A. V., Richmond, B. J. & Optican, L. M. A UNIX-based multiple
process system for real-time data acquisition and control. WESCON Conf.
Proc. 2, 1–10 (1982).
Bruce, C. J., Goldberg, M. E., Bushnell, M. C. & Stanton, G. B. Primate
frontal eye fields. II. Physiological and anatomical correlates of
electrically evoked eye movements. J. Neurophysiol. 54, 714–734 (1985).
Gnadt, J. W. & Andersen, R. A. Memory related motor planning activity
in posterior parietal cortex of monkey. Exp. Brain Res. 70, 216–220
(1988).
Hikosaka, O., Sakamoto, M. & Usui, S. Functional properties of monkey
caudate neurons. III. Activities related to expectation of target and
reward. J. Neurophysiol. 61, 814–832 (1989).
Draper, N. & Smith, H. Applied Regresion Analysis, Second Edition (Wiley,
New York, 1966).
Green, D. M. & Swets, J. A. Signal Detection Theory and Psychophysics
(Wiley, New York, 1966).
Mood, A. M., Grabill, F. A. & Boes, D. C. Introduction to the Theory of
Statistics (McGraw-Hill, New York, 1963).
185