Download Abstract of “Primate frontal eye fields mediate spatial attention in

Abstract of “Primate frontal eye fields mediate spatial attention in covert visual search”
by Ilya E. Monosov Ph.D., Brown University, May 2010.
Visual spatial attention serves to select locations of interest in the visual field and enhances the cortical
representation of objects at those locations. Previous studies suggest that neural activity in primate frontal
eye fields (FEF) is involved in the spatial selection of salient stimuli in complex visual environments for
eye movements and spatial attention. Here, we explore the origin of the spatial selection signal in FEF and
its relationship to measures of covert spatial attention. We compare the timing of spatial selection for the
location of the target in two simultaneously recorded cortical signals: local field potentials (LFPs) and
spikes. LFPs are thought to represent synaptic input, while spiking activity is the output, of the area around
the electrode tip. We found that spatial selectivity identifying the location of the target in the visual search
appeared in the spikes about 30 ms before it appeared in the LFPs. This suggests that the spatial selection
signal is computed locally in FEF from spatially non selective inputs. Additionally, we show that the
magnitude of spatial selection in FEF is related to behavioral measures of attention during the time period
in which the stimulus is being processed by the visual system. This relationship shows that FEF is directly
involved in spatial attention.
Primate frontal eye fields mediate
spatial attention in covert visual
search
by
Ilya E. Monosov
B.S., University of California San Diego, 2004
A dissertation submitted in partial fulfillment of the
requirements for the Degree of Doctor of Philosophy in the
Division of Biology and Medicine
Providence, Rhode Island
May, 2010
© Copyright 2009 by Ilya E. Monosov
This dissertation by Ilya E. Monosov is accepted in its present form by the Division of
Biology and Medicine as satisfying the dissertation requirement of the degree of Doctor
of Philosophy.
Date__________
___________________________________
Kirk G. Thompson, Ph.D., Advisor
Laboratory of Sensorimotor Research
National Eye Institute
National Institutes of Health
Recommended to the Graduate Council
Date__________
__________________________________
David L. Sheinberg, Ph.D., Reader
Dept. of Neuroscience
Brown University
Date__________
__________________________________
Robert H. Wurtz, Ph.D., Reader
Laboratory of Sensorimotor Research
National Eye Institute
National Institutes of Health
Date__________
__________________________________
Jeffrey D. Schall, Ph.D., Reader
Dept. of Psychology
Vanderbilt University
Approved by the Graduate Council
Date__________
__________________________________
Shelia Bonde, Ph.D.
Dean of the Graduate School
iii
Curriculum Vitae
Ilya E. Monosov, BS, MS
DOB: 01-13-1978
Contact Information
Laboratory of Sensorimotor Research
2a50, Bldg 49, National Eye Institute, NIH
Bethesda, MD, 20891
Email: [email protected]
Education
2000-2004 B.S. Biology, UCSD. Graduated with Provost’s Honors
2004-2005 M.S. Architecture, New School Of Architecture (San Diego, CA)
Thesis: Cognitive Neuroscience and Architecture: Interdisciplinary Research for K-6 School
Design.
2005-current PhD Candidate in Neuroscience, Brown University-NIH partnership
Teaching Experience
2004 – Teaching Assistant. Course: Molecular Basis for Disease. Taught by Immo Scheffler at
UCSD.
2005 – Teacher. Course: Environmental Biology for Architects at New School of Architecture.
San Diego, CA.
2005 – Guest lecturer in UCSD music department (invited by Professor Charles Curtis).
Professional Memberships
Society for Neuroscience
Frogpeak Music Composer’s Collective
Peer-Reviewed Publications
iv
(1) Monosov IE, Traverse JC, Thompson KG. Measurements of simultaneously recorded spiking
activity and local field potentials suggest that spatial selection emerges in the frontal eye field.
Neuron. 2008 Feb 28;57(4):614-25.
(2) Traverse JC, Monosov IE, Zhou Y, Thompson KG. A perceptual representation in the frontal
eye field during covert visual search that is more reliable than the behavioral report. Eur J
Neurosci. 2008 Dec;28(12):2542-9.
Posters
(1) Monosov IE, Trageser JC, Thompson KG. A comparison of the time course and spatial tuning
of attention in evoked and induced LFP responses, and spikes in the primate frontal eye field.
Society for Neuroscience 2006.
Professional Talks
(1) Monosov IE, Trageser JC, Xu P, Thompson KG. The effects of frontal eye field inactivation
in a cued covert visual search task. Society for Neuroscience, 2008.
(2) Monosov IE, Trageser JC, Thompson KG. Spatial selection in the frontal eye field predicts
accurate object recognition. Society for Neuroscience, 2007.
(3) Monosov IE, Bellugi U. William’s Syndrome and Music Creativity. Salk Institute for
Biological Studies, 2005.
(4) Monosov IE. K-6 Design: Classroom and Brain development. Academy of Neuroscience for
Architecture workshop for architects and neuroscientists, 2005.
v
Preface
“Trying to understand perception by studying only neurons is like trying to understand bird flight by
studying only feathers: it just cannot be done. In order to understand bird flight, we have to understand
aerodynamics; only then do the structure of the feathers and the different shapes of birds’ wings make
sense.” (Marr, 1982)
Visual spatial attention serves to select locations of interest in the visual field and enhances the
cortical representation of objects at those locations. Previous studies suggest that neural activity in primate
frontal eye fields (FEF) is involved in the spatial selection of salient stimuli in complex visual
environments for eye movements and spatial attention. The main goal of the following thesis is to
understand the temporal dynamics of spatial selection and to understand the relationship of this measurable
neural signal to behavioral measures of spatial attention. Chapter 1 outlines the hypothesis of this thesis and
provides the background that is to guide the experiments outlined in Chapter 2 and 3. I discuss how spatial
attention has been measured behaviorally and what is already known about the neurophysiology of spatial
selection in the primate frontal eye fields (FEF). My hypothesis is that 1) spatial selection is computed in
FEF from non-spatial inputs and 2) that the magnitude of spatial selection in FEF is directly related to
behavioral measures of spatial attention in covert visual search. In Chapter 2, I present the results of
experiments that demonstrate that FEF is directly involved in the generation of a spatially selective signal
that indicates the presence of a target amongst distractors in a complex visual scene. In Chapter 3, I show
the results of an experiment that suggests that there is a direct relationship between behavioral measures of
spatial attention and the spatial selection signal in FEF. Chapter 5 is a summary and outline of the main
results of the thesis. It also includes several proposals for future inquiry.
vi
Acknowledgements
This thesis was completed in the Laboratory of Sensorimotor Research at the National Institutes of
Health (NIH) and Brown University between June, 2005 and May, 2009. It was made possible by the
Brown University Graduate Partnerships Program (GPP) with the NIH. I would like to thank my advisor
Kirk G. Thompson for the enormous amount of support, encouragement, and training he has provided. I
would also like to thank my co-advisor David L. Sheinberg for providing a great deal of expertise and
support. I hope to repay their kindness with a long and successful collaborative relationship. I would like to
thank the members of my thesis committee, Robert H. Wurtz and Jeffrey D. Schall, for helpful discussions
and comments. I would like to thank my parents, Edward and Anna Monosov for a lifetime of
encouragement and love; and Elizabeth P. Weber for the love and kindness she has given me on a daily
basis.
vii
Table of Contents
Curriculum Vitae ........................................................................................................................................iv
Preface ........................................................................................................................................................vi
Acknowledgements ....................................................................................................................................vii
Table of Contents........................................................................................................................................viii
List of Figures ............................................................................................................................................x
Chapter 1: Introduction...............................................................................................................................1
1.1 The correlates of spatial attention: behavior ....................................................................................1
1.2 Spatial selection in the primate frontal eye fields ............................................................................1
1.3 The correlates of spatial attention: neurophysiology........................................................................3
1.4 Hypothesis and Outline.....................................................................................................................4
Chapter 2: The frontal eye field converts feature-related information into a categorical representation of the
target’s location...........................................................................................................................................6
2.1 Introduction ......................................................................................................................................6
2.2 Methods ............................................................................................................................................8
2.3 Results ..................................................................................................................................................20
2.4 Discussion.........................................................................................................................................32
Chapter 3: Spatial selective signals in the frontal eye fields mediate spatial attention and predict accuracy in
a covert visual search task that requires object identification.....................................................................38
3.1 Introduction ......................................................................................................................................38
3.2 Methods ............................................................................................................................................39
3.3 Results ..............................................................................................................................................44
3.4 Discussion.........................................................................................................................................62
Chapter 5: Summary and Conclusions .......................................................................................................67
5.1 Overview ..............................................................................................................................................67
5.2 Future directions ...................................................................................................................................69
viii
Bibliography................................................................................................................................................71
ix
List of Figures
Figure 2.1: The behavioral tasks................................................................................................................9
Figure 2.2: Spatial tuning analysis of spiking activity recorded at the same site as the LFP shown in Figure
2.3...........................................................................................................................................................11
Figure 2.3: Spatial tuning analysis of the LFP response recorded at the same site as the spiking activity
shown in Figure 2.2.................................................................................................................................13
Figure 2.4: Visual response latency analysis of the spike (left) and LFP (right) responses recorded during
the covert visual search task....................................................................................................................14
Figure 2.5: Testing the effects of signal filtering on the determination of selection time.........................19
Figure 2.6: Population results from the covert visual search tasks shown separately for the two
monkeys..................................................................................................................................................23
Figure 2.7: Population results from the covert visual search task at each recording site combined across the
two monkeys...........................................................................................................................................25
Figure 2.8: The relationship of selection time in LFPs and spikes to LFP visual response latency..........28
Figure 2.9: Comparisons of spatial tuning in spiking activity and LFP responses recorded in the memoryguided saccade and covert visual search tasks........................................................................................31
Figure 3.1: Task and behavior....................................................................................................................44
Figure 3.2: Activity of a single FEF neuron during correct target present trials.......................................48
Figure 3.3: Relationship between cue-related spatial selectivity and behavioral measures of spatial attention
for monkey C (open symbols) and for monkey B (solid symbols) ........................................................50
Figure 3.4: Population analysis of activity during correct and error valid cue trials.................................52
Figure 3.5: Population analysis of activity during correct and error invalid cue trials..............................54
Figure 3.6: Population analysis of activity during correct and error neutral cue trials..............................56
Figure 3.7: A comparison of population average activity during (a) correct rejection trials and (b) invalid
cue miss trials.............................................................................................................................................58
Figure 3.8: The relationship between the SI and the speed of target identification...................................60
x
Chapter 1
Introduction
1.1 The correlates of spatial attention: behavior
To function in a complex and constantly changing environment, it is important to select and filter
out irrelevant information and to attend to information that is of benefit and interest. Generally, attention is
thought to include the following components: working memory, competitive selection, top-down signal
sensitivity control, and saliency filters, which select sensory information for the processing of stimuli that
are relevant for behavior (Knudsen, 2007). Visual spatial attention serves to select locations of interest in
the visual field and enhances the cortical representation of objects at those locations. This type of attention
is necessary for recognition of objects in complicated natural environments (Sheinberg and Logothetis,
2001; Rensink, 2002). In psychology experiments, spatial attention is typically measured as improvements
in performance accuracy and decreases in reaction time for detection and discrimination of stimuli
presented at the attended location (Pashler 1998). This thesis will concentrate on exploring the neural basis
of spatial attention in the primate frontal eye fields.
1.2 Spatial selection in the primate frontal eye fields
The visual system is thought to be divided into two hierarchical processing streams (Ungerleider
and Mishkin, 1982). The dorsal visual stream, or the ‘where’ stream, is concerned with coding space, while
the ventral visual stream, or the “what” stream, is thought to process the visual scene and extract meaning
from the environment, for example regarding object identity. The frontal eye field (FEF) is an important
site of convergence in the visual system. The FEF lies at the end of the dorsal “where” visual processing
stream. It was first observed by David Ferrier in the late 1880’s when he discovered that electrical
1
2
stimulation of the macaque prefrontal cortex elicited saccadic eye movements. FEF is located in the
prefrontal cortex in the rostral bank of the arcuate sulcus of macaques and receives input from dorsal
stream visual areas MT, MST, an LIP; ventral stream visual areas V4, TEO, and TE; and from the
supplementary eye field and prefrontal areas 46 and 12 (Schall, 2009). Therefore, FEF is in a unique
position to signal the presence of a behaviorally relevant stimulus as that stimulus becomes the target for a
saccade.
FEF is retinotopically organized and has a map of visual field eccentricity (Schall, 2009). Many
functionally different types of neurons have been identified in FEF. Movement neurons begin to fire
approximately 50ms before the initiation of a saccadic eye movement (Bruce and Goldberg, 1985;
Thompson et al., 1996). These neurons are thought to influence eye movements through projections to the
superior colliculus and basal ganglia (Stanton et al., 1988; Schall et al., 1995; Schall, 2009). Visually
responsive neurons in FEF respond when a target is placed in their receptive field. Their activity selects the
salient target amongst distractors in visual search tasks without eye movements (Thompson et al, 2005).
From now on when I discuss FEF activity, I will be referring to visually responsive FEF neurons. Some of
these neurons also have eye movement related activity (these are termed visuomotor neurons). We do not
discuss the activity of purely movement neurons because movement related activity in FEF is suppressed
during the covert visual search tasks used in the thesis (Thompson et al., 2005). Examples of visually
responsive FEF neurons are shown in Chapters 2 and 3.
Though FEF receives input from the ventral visual pathway, its visually responsive activity is
generally not selective for the identity of objects or their specific features; though a few exceptions have
been reported. Generally, if a given visual feature is important to the subject, FEF activity will identify its
location in space, and not the feature itself (Thompson and Bichot, 2005). For example, if for a given
behavior motion becomes more important than the color red, FEF activity will identify the location of the
motion amongst red stimuli, conversely, if a color identifies a target that is to guide a behavior, FEF will
select the location of the color. Therefore, FEF acts like the final stage of dorsal processing, forming a
saliency map of space (Thompson and Bichot, 2005), where higher activity is correlated with the
importance of a given location. It is thought that this map of visual salience guides visual spatial attention
3
(Thompson et al., 2005; Schafer and Moore, 2007) and eye movements (Schall, 1995; Thompson et al.,
1996).
Understanding the time course and origin of the explicit spatial selection signal in FEF is key to
understanding the neural computations that underlie it. This thesis explores the origins of the spatial
selection signal in FEF and presents evidence that this signal is directly involved in the top-down control of
spatial attention.
1.3 The correlates of spatial attention: neurophysiology
The neurophysiological literature defines visual spatial attention as an enhanced processing of
visual features used to guide a decision process (Desimone and Duncan, 1995; Maunsell and Treue, 2006;
Knudsen, 2007). Attentional effects have been demonstrated in the neural responses of V1, V4, MT, and IT
in experiments that combined neural recordings and attention-demanding behavior (Reynolds and Chelazzi,
2004). For example in V4, neuronal firing rates are enhanced when the animal pays attention to a
behaviorally relevant stimulus, and suppressed when the animal attends away from that same stimulus
(Reynolds and Desimone, 2000).
It is widely accepted that the spatially selective signals in the frontal eye field (FEF) are associated
with the planning and execution of saccadic eye movements (Goldberg and Segraves 1989; Schall and
Thompson 1999; Tehovnik et al. 2000). In humans, functional imaging studies show that the FEF is part
of what is referred to as the fronto-parietal attention network and is active during the allocation of attention
with and without eye movements (Beauchamp et al. 2001; Corbetta and Shulman 2002; Kincade et al.
2005; Bressler et al. 2008; Kelley et al. 2008). Transcranial magnetic stimulation over the FEF modulates
perceptual performance in covert attention tasks (Grosbras and Paus 2002; Muggleton et al. 2003; Smith et
al. 2005), and also modulates visual activity in extrastriate visual cortex (Silvanto et al. 2006; Taylor et al.
2007).
There is a growing body of evidence that FEF plays a causal role in directing covert spatial
attention (Awh et al. 2006). In monkeys, electrical microstimulation of FEF enhances perception (Moore
and Fallah 2001; Moore and Fallah 2004; Schafer and Moore 2007) and produces enhanced responses in
extrastriate visual cortex that resemble the effects of covert spatial attention (Moore and Armstrong 2003;
4
Armstrong and Moore 2007). Inactivation of FEF disrupts target detection during covert visual search
(Wardak et al. 2006) Neuron recordings in monkeys have shown that the activity of visually responsive
FEF neurons identifies the location of an attended visual stimulus, even in the absence of eye movements
(Kodaka et al. 1997; Murthy et al. 2001; Sato and Schall 2003; Thompson et al. 2005b). But thus far, there
has not been an experiment relating FEF activity and behavioral measures of spatial attention. This thesis
directly assesses the relationship between reaction time and performance accuracy in a cued covert visual
search task in order to establish a link between FEF activity and spatial attention.
1.4 Hypothesis and Outline
In the following chapters, we will explore the origin of spatial selection in FEF and its relationship
to spatial attention. The general hypothesis that guides this thesis is that FEF is one of the sources of spatial
selection signals in the brain, and that the magnitude of these signals is correlated with reaction time and
performance accuracy in covert visual search.
In Chapter 2, we will explore the origin of spatial selection in FEF during covert visual search.
This work is an extension of previous work that showed that the activity of visually responsive FEF
neurons selects the location of the behaviorally relevant stimulus amongst distractors in covert visual
search before the monkeys’ made their response (Thompson et al., 2005). In this study, local field
potentials (LFP) and spiking activity were recorded while the monkeys performed a covert visual search
and a memory guided saccade tasks. It has been proposed that LFPs represent the sum of dendritic synaptic
activity in an area that spans less than 1mm of tissue around the tip of the electrode (Logothetis et al., 2007;
Katzner et al., 2009), while spiking activity represents the output of neurons being recorded. Therefore, we
will compare the time course of non-selective visually evoked signals in FEF LFPs and spiking activity.
We will show that visually evoked activity was detectable first in LFPs and then in spiking activity. This is
consistent with the hypothesis that LFPs represent input from local processing and from other brain
regions. Next, we will compare the time course of spatial selection in spiking activity and LFPs and show
that while the non-selective visually evoked activity was first detectable in LFPs and then spikes, spatial
selection was first detectable in spiking activity and 30ms later in LFPs. We suggest that this finding
5
supports the hypothesis that FEF converts spatially non-selective inputs to a categorical representation of
target location, and therefore that FEF is a source of spatial selection in the brain.
In Chapter 3, we will explore the influence of spatial selection in FEF on visual processing by
employing a cued covert visual search task that requires the monkeys to report the presence and identity of
a stimulus amongst distractors. We will fill the missing piece in a growing body of evidence that suggests
that FEF is directly involved in shifting covert spatial attention and enhancing visual processing of taskrelevant stimuli. Importantly, we will show that the magnitude of spatial selection in FEF predicts
accuracy and response time of object identification only during the time when the stimulus is being
processed by the visual system.
Chapter 2
The frontal eye field converts non-spatial visual
information into a categorical representation of the
target’s location
The frontal eye field (FEF) participates in selecting the location of behaviorally relevant stimuli for guiding
attention and eye movements. We simultaneously recorded local field potentials (LFPs) and spiking
activity in the FEF of monkeys performing memory-guided saccade and covert visual search tasks. We
compared visual latencies and the time course of spatially selective responses in LFPs and spiking activity.
Consistent with the view that LFPs represent synaptic input, visual responses appeared first in the LFPs
followed by visual responses in the spiking activity. However, spatially selective activity identifying the
location of the target in the visual search array appeared in the spikes about 30 ms before it appeared in the
LFPs. Because LFPs reflect dendritic input and spikes measure neuronal output in a local brain region, this
temporal relationship suggests that spatial selection necessary for attention and eye movements is compute
locally in FEF from non-spatially selective inputs. †
2.1 Introduction
Visual spatial selection describes the process that guides spatial attention (Serences and Yantis,
2006) and selectively couples perception to action (Allport, 1987). Understanding the time course of this
process is key to understanding the neural computations that underlie it. Typically, this question has been
addressed by analyzing event-related brain potentials (ERPs) recorded from scalp electrodes in humans
† The bulk of this chapter is taken from a previously published manuscript: Monosov IE, Trageser JC, Thompson KG (2008) Measurements of
simultaneously recorded spiking activity and local field potentials suggest that spatial selection emerges in the frontal eye field. Neuron 57(4):614-25.
6
7
(Hillyard and Anllo-Vento, 1998; Luck et al., 2000) and neuronal spiking activity in behaving primates
(Schall and Thompson, 1999). In visual search studies, in which subjects are required to discriminate a
target among distractors, human ERPs (Luck and Hillyard, 1994) and single units recorded in primate
frontal eye field (FEF) (Thompson et al., 1996; Sato et al., 2001), lateral intraparietal area (Ipata et al.,
2006; Thomas and Pare, 2007) and superior colliculus (McPeek and Keller, 2002) exhibit an initial period
of non-selective activation followed by a discrimination process that identifies the location of the target in
the search array.
Local field potentials (LFPs) are electrical potentials recorded with an electrode positioned in the
brain. The LFP signal represents the summed synaptic activity occurring near the tip of the electrode. It is
a combined measure of local processing and synaptic inputs from other brain regions regardless of whether
or not spikes are generated (Mitzdorf, 1985, 1987; Juergens et al., 1999; Cruikshank et al., 2002; Kaur et
al., 2004; Logothetis and Wandell, 2004; Kreiman et al., 2006; Nielsen et al., 2006; Chen et al., 2007). In
contrast, spiking activity represents the results of local neural processing and is the output signal, from the
neurons near the tip of the electrode, to local circuits or other brain regions. Although both LFPs and
spiking activity have been used to measure the time course of spatial attention processes, the relationship
between these neurophysiological signals is still unclear. Analysis of concurrently recorded LFP and
spiking activity can shed light on how sensory representations in dendritic input are transformed into
cognitive signals such as spatial selection (Kreiman et al., 2006; Nielsen et al., 2006).
The FEF is a brain area in monkeys and humans that participates in the visual spatial selection
process (Schall and Thompson, 1999; Pessoa et al., 2003; Awh et al., 2006; Serences and Yantis, 2007).
The spatial selection process localizes behaviorally important objects in a complex visual scene and is
necessary for guiding visual attention and goal directed behaviors. It was previously shown that spiking
activity in monkey FEF reflects the locus of spatial attention during covert visual search tasks in the
absence of eye movements (Thompson et al., 2005a). During the collection of these neuronal spiking data,
LFPs were also recorded simultaneously from the same electrodes. The goals of this study were to
determine whether LFP responses were spatially selective, and if so, to compare the time course and spatial
tuning of the spatially selective signals in neuronal spiking activity and LFP responses.
8
We found that in the covert visual search task both the LFPs and the spiking activity exhibited
initial non-selective visual responses that evolved into significant spatial tuning in the time period before
the monkeys’ behavioral report. The directional tuning of the spatially selective responses in the visual
search task matched the directional tuning of the visually evoked responses to a single visual stimulus in
the memory-guided saccade task. Although the initial visual responses appeared first in the LFP signals in
both tasks, the spatially selective responses in the visual search task appeared first in the spiking activity.
These results suggest that during visual search, spatial selectivity is generated in FEF from non-spatially
selective inputs.
2.2 Methods
Data collection
The data were collected from two male monkeys (Macaca mulatta) weighing 8 kg (monkey S) and
6.5 kg (monkey C). All surgical and experimental protocols were approved by the National Eye Institute
Animal Care and Use Committee and complied with the National Institutes of Health Guide for the Care
and Use of Laboratory Animals.
The surgical procedures, behavioral control, and visual stimulation techniques have been
described previously (Thompson et al., 2005b). The single unit spiking activity analyzed in this study is
the same as in the previous study (Thompson et al., 2005b). Often two or three units were recorded
simultaneously on one electrode and sorted offline. For this study, all the single units recorded at each site
were combined to represent the overall spiking activity at each recording site.
The local field potentials (LFPs) were recorded simultaneously on the same glass insulated
tungsten electrodes as the spikes using a Plexon data acquisition system (Plexon Inc.). The impedance of
the head-stage was 40 MΩ at 1 kHz. Electrode impedance ranged from 0.5 to 1.5 MΩ. A stainless steel
guide tube resting on the surface of the dura served as the reference. The signals were amplified and
filtered between 154 Hz and 8.8 kHz to obtain spike data. LFP signals were digitized and sampled at 1
kHz after filtering the electrode signal between 3 Hz and 88 Hz. Analog eye position and lever position
signals were digitized and sampled at 1 kHz.
9
Figure 2.1: The behavioral tasks.
(A) The memory-guided saccade task. After the monkey fixated on a central spot, a peripheral stimulus identical to the fixation spot
was flashed for 50 ms randomly at one of the six or eight locations matching the stimulus locations in the covert visual search task.
After a delay, the fixation spot disappeared, and the monkey was rewarded for making a saccade to the remembered target location.
(B) The covert visual search tasks. After the monkey grasped the lever in the vertical position, a small fixation cross appeared. After
fixating the central cross, a search array appeared in which one of the stimuli was different. Monkey S was rewarded for turning the
lever in the same direction as a different-colored stimulus in relation to the fixation cross. Monkey C was rewarded for turning the
lever in the same direction as the gap in the C target stimulus regardless of its location in the search array. The depiction of the lever
at the bottom shows the correct behavioral responses for the example trials shown in the search displays.
Behavioral tasks
At each recording site monkeys performed a memory-guided saccade task (Figure 2.1 A) and one
of two visual search tasks (Figure 2.1 B) in separate blocks of trials as described in a previous report
(Thompson et al., 2005b). In the memory-guided saccade task, after the monkey fixated on a 0.3° diameter
gray spot for 400-800 ms, an identical gray spot was flashed for 50 ms at one of six or eight isoeccentric
peripheral target locations spaced equally around the central fixation spot. The eccentricity was adjusted so
10
that at least one of the stimulus locations was inside the receptive field of the neuron being recorded. The
eccentricities of the stimuli ranged between 8° and 12° across recording sessions depending on receptive
field location. Monkeys were required to maintain fixation on the central fixation spot for a random period
ranging from 800 to 1400 ms. After the fixation spot disappeared, the monkeys were rewarded for making
a saccade to the remembered target location.
In the covert visual search tasks, monkeys initiated a trial by grasping a lever and holding it in a
vertical position. Once the lever was within 10° of vertical, a small central yellow fixation cross (0.3°)
appeared. After fixating the cross for 400 to 800 ms, a search array appeared that was made up of a target
randomly placed at one of the locations used in the memory-guided saccade task and distractors at the
remaining locations. Each of the search array stimuli subtended 1.5° of visual angle. The monkeys were
rewarded for maintaining fixation on the central cross and making the correct lever turn (> 15° from
vertical) within 2 s after search array presentation; in practice, the monkeys nearly always turned the lever
to the physical limit of 35° from vertical. If the monkey broke fixation on the central cross, released the
lever, or made an incorrect lever turn the trial was aborted immediately. The reward was given after a
correct lever turn; however, the search array remained on for an additional 250-500 ms to probe for latent
saccade plans. The monkeys did not tend to make saccades to the target of the search array after obtaining
the reward (Thompson et al., 2005b).
Monkey S was trained to report the location of the color singleton target of the search array
(Figure. 2.1 B, upper). The luminance of the stimuli was determined by a Minolta Ca-100 color analyzer.
The stimuli were isoluminant green and red discs. The target could be either green or red, but within a
block of trials the color of the target and distractors did not change. The singleton target appeared
randomly at one of six stimulus locations, three to the left and three to the right of the fixation cross. A
correct response was a lever turn to the left or right corresponding to the location of the target stimulus
relative to the fixation cross.
In order to verify that the direction of the lever turn did not affect FEF activity during the task,
Monkey C was trained to report the orientation of a C among O distractors (Figure 2.1 B, lower). The
stimuli were gray rings with one of them having a 0.5° gap randomly on the left or right. The C target
appeared randomly at one of eight locations positioned around the fixation cross. A correct response was a
11
lever turn to the left or right corresponding to the location of the gap in the C target regardless of its
location in the search array.
Data analysis
The LFP signal is a continuous measure of brain activity. A comparable measure of spiking
activity was obtained by convolving each spike with a function that resembles an EPSP (Thompson et al.,
1996). With this method, each spike exerts influence only forward in time and represents the post-synaptic
consequences of spiking activity. The resulting spike density function reflects the onset of spiking activity
at a 1 millisecond time resolution and is comparable to the onset of activity measured in the LFP signal.
Examples of the EPSP spike density functions are shown in Figure 2.2 A (see below). Next we describe
the analytical methods used to determine the time course of visual activation and spatial selection, and
characterize the spatial tuning of spiking activity and LFP responses recorded during the memory-guided
saccade and covert visual search tasks.
Figure 2.2: Spatial tuning analysis of spiking activity recorded at the same site as the LFP shown in
Figure 2.3.
(A) Spike density functions, derived from a filter resembling an EPSP, are plotted above tick marks representing times of action
potentials for three representative trials. (B) The average target-aligned spiking activity at each target location from the memoryguided saccade task (gray) and the covert visual search task (black). The box-whisker plot in each panel indicates the median, quartiles
and range of reaction times in the covert visual search task. The neuron’s preferred target direction (60°) corresponds to the filled
12
circle in the search array at the center. (C) The superimposed average activity for each target position from the memory-guided
saccade task (left) and the visual search task (right). The thick line represents the average activity on trials when the target was at the
preferred spatial location. (D) The p-value (ANOVA) at each millisecond in the memory-guided saccade task (left) and in the visual
search task (right) that estimates the probability that the spiking activity did not vary across target locations. The black triangle at the
bottom of the plot marks the selection time (vertical dotted line: memory-guided = 70 ms, visual search = 128 ms) which was defined
as the first millisecond that the p-value crossed p = 0.05 (horizontal dotted line), only if it continued past p = 0 001 and p < 0.05 for
more than 20 of the next 25 milliseconds. (E) The spatially selective response measured from 50 – 300 ms following the target flash in
the memory-guided saccade task (left), and from 100 – 300 ms following the time of search array presentation (right) as a function of
target direction. The time ranges for measuring spatial tuning are indicated by black bars in (C). The parameters of the best fit
Gaussian curve from the memory guided saccade task (left) are B = 40.26 sp/s, R = 36.54 sp/s, Φ = 64.42°, and Tφ = 38.25°; and from
the covert visual search task (right) are B = 45.75 sp/s, R = 55.36 sp/s, Φ = 63.22°, and Tφ = 45.02°.
Selection time
The time course of spatial selectivity in the LFP and spiking activity was determined with an
analysis of variance (ANOVA) at each millisecond following the target flash in the memory-guided
saccade task and the presentation of the search array in the visual search tasks (Figures 2.2 and 2.3). The
running ANOVA estimated the probability (p) at each millisecond that the response did not vary across
target locations. Figures 2.2 and 2.3 illustrate the time course analysis for the spiking activity (Figure 2.2)
and the LFP response (Figure 2.3) recorded concurrently at a single site. The selection times of the spiking
activity and the LFP response were determined separately and were defined as the first millisecond that the
p-value dropped below the 0.05 level only if it continued past the 0.001 level and remained below the 0.05
level for more than 20 of the next 25 milliseconds. To obtain the earliest possible selection times, a
threshold of p = 0.05 was used. However, a threshold of p = 0 01 did not alter the temporal relationship
between the selection times of the LFP and spiking activity. Again, the important point is that the same
statistical analysis and threshold was used to determine selection times in the LFP and spiking activity in
the memory-guided saccade task and in the visual search tasks. Our threshold for determination of the time
of selection (see above) also insured that our results were reliable.
13
In Figures 2.2D and 2.3D, p-values obtained from the running ANOVA are plotted as a function
of time on a log axis from 1 to 10-10 for spikes (Figure 2.2D) and LFPs (Figure 2.3D) recorded concurrently
at a single site during the memory-guided saccade task (left) and the visual search task (right).
It is important to note that selection time measured in the memory-guided saccade task is
qualitatively different from that measured in the visual search task. In the memory-guided saccade task a
single target stimulus is presented alone and evokes a different initial response across target locations.
Therefore, selection time in the memory-guided saccade task corresponds to the initial visual response
latency to a single visual stimulus. In the visual search task, however, selection time measures the first
time that the responses to the target of the search array are different from the responses to the distractors.
As previously shown for spiking activity (Thompson et al., 1996), and now we demonstrate for LFPs, the
initial visually evoked responses in FEF during visual search do not distinguish the target from the
distractors. Therefore we used a different method to determine visual response latency in the visual search
task.
Figure 2.3: Spatial tuning analysis of the LFP response recorded at the same site as the spiking
activity shown in Figure 2.2.
(A) The LFP responses on three representative visual search trials. (B) The average target-aligned LFP response in the memoryguided saccade task (gray) and in the covert visual search task (black) sorted by target location. (C) The superimposed average LFP
14
response for each target position from the memory guided saccade task (left) and the covert visual search task (right). (D) The p-value
(ANOVA) at each millisecond that estimates the probability that the LFP response did not vary across target locations. The selection
time of the LFP response at this recording site is 69 ms for the memory-guided saccade task and 142 ms for the visual search task. (E)
The spatially selective response measured from 100 – 200 ms following the target flash in the memory-guided saccade task (left), and
from 180 – 300 ms following the time of search array presentation (right) as a function of target direction. The time interval used for
determining the spatial tuning of the LFP response was the interval that exhibited the most variability in the ANOVA analysis shown
in D. The parameters of the best fit Gaussian curve from the memory guided saccade task (left) are B = -4.59, R = -33.14, Φ = 43.05°,
and Tφ = 59.24°; and from the covert visual search task (right) are B = 19.59, R = -16.33, Φ = 64.05°, and Tφ = 62.91°.
Visual response latency during visual search
A paired t-test was performed across all correct trials comparing the average LFP and spiking activity
during the 50 ms preceding the appearance of the search array on each trial to the activity at each
millisecond following the appearance of the search array. The visual response latency was defined as the
first time that the p-value dropped below the 0.01 level only if it continued past the 0.001 level and
remained below the 0.01 level for more than 20 of the next 25 milliseconds. The same threshold was used
for determining visual response latencies in the LFP and spiking activity recording during the visual search
task. Figure 2.4 (see below) illustrates how we measured the initial visual response latencies of the spiking
activity and the LFP response recorded simultaneously during the visual search task.
Figure 2.4: Visual response latency analysis of the spike (left) and LFP (right) responses recorded
during the covert visual search task.
15
The activity is from the same recording session as shown in Figures 2.2 and 2.3. (A) The average spike density function constructed
by convolving each spike with a kernel that resembles an EPSP and averaging across all trials. (B) The p-value (paired t-test) at each
millisecond that estimates the probability that the spiking activity is equal to the baseline activity (measured from -50 to 0 ms). The
visual response latency was defined as the first time that the p-value crossed p = 0.01 (horizontal dotted lines), only if it continued past
p = 0 001 and p < 0.01 for more than 20 of the next 25 milliseconds. The spiking visual response latency in the covert visual search
task at this recording site is 63 ms (black triangles and vertical dotted lines). (C) The average LFP signal across all trials. (D) The pvalue (paired t-test) at each millisecond that estimates the probability that the LFP signal is equal to the baseline signal (measured
from -50 to 0 ms). The LFP visual response latency is 53 ms.
Spatial tuning
To describe the variation in the spiking and LFP responses with the location of the singleton
target, the response averaged over a time interval was fit with a Gaussian function of the form
A(φ) = B + R·exp(-½[( φ-Φ)/ Tφ]2),
where activation (A) as a function of meridional direction (φ) depends on the baseline response (B), peak
response (R), optimum direction (Φ), and tuning width (Tφ). Previous reports have shown that this function
effectively characterizes the spatial pattern of FEF spiking activity (Bruce and Goldberg, 1985; Schall et
al., 1995a; Schall et al., 2004). The best fit Gaussian curve was obtained for the average activity measured
over a time range following visual stimulus presentation. For spiking activity, the time range was from 50
ms to 300 ms for the memory-guided saccade task, and from 100 ms to 300 ms for the visual search task.
These time intervals were used because it encompassed the period of spatial selectivity observed across the
data (Thompson et al., 2005b). For the LFP response in the memory-guided saccade task, the time range
was from 100 ms to 200 ms because this interval encompassed a strong spatially selective negative-going
deflection observed across all the LFP recordings (see Figure 2.3C, left panel). For the LFP response in the
visual search task, it was necessary to determine the appropriate time interval individually for the different
recording sites. This is because a spatially selective response could emerge in a positive or in a negative
difference in the LFP signal. Therefore, to determine the spatial tuning of the LFP signal, we made the
assumption that the preferred direction was in the visual hemifield contralateral to the brain hemisphere in
which the LFP signals were recorded. This was reasonable because the right visual hemifield is
represented in the left hemisphere of the brain and the left visual hemifield is represented in the right
hemisphere of the brain. In some of the LFP recordings, spatial tuning was evident in positive tuning during
16
one time interval and in negative tuning during another time interval that was separated by a non-selective
period during which time the polarity of the spatial tuning switched. The time interval we used for
determining the directional tuning was the interval that exhibited the strongest spatial selectivity in the
running ANOVA analysis described above because it was most reliable.
Selection time was defined as the first time that there was significant spatial tuning (see above). In
the covert visual search task, the earliest spatial tuning in the LFP from 18 of the recording sites was shortlived and positively tuned. In each of these 18 recordings there was also a later, longer lasting and
negatively tuned response that was more significant statistically. The LFP shown in Figure 2.3 is one
example of this pattern. On average in these 18 recordings the polarity of the tuning was opposite in the
early and late periods and the responses during the early period were more variable than during the late
period, but the preferred target directions were nearly identical. Across the population, there was a strong
correlation between the preferred target directions from the early and late time periods (p = 0.005). The
preferred target directions of 16 of the 18 LFPs (89%) were separated by less than the distance between
adjacent target locations (60° for Monkey S and 45° for Monkey C).
Signal Variability and the Reliability of Timing Measurements
The accuracy of the timing relationships between LFPs and spikes presented in this study depends
on the reliability of the signals and analysis methods. If the amount of variability was different for spikes
and LFP signals, then the results of the ANOVA analysis comparing the timing relationships between the
spikes and LFPs are questionable. Therefore we compared the variability of the LFP responses and spiking
activity at each recording site by calculating a variability index (VI) for each signal, a ratio of two standard
deviations that measures signal variability across trials relative to the variability across target positions.
The numerator of the VI is the standard deviation across all trials measured at each millisecond and
averaged over the first 50 ms following search array presentation, and the denominator is the standard
deviation across the mean activity for each target position averaged over the same time period. The VI for
spikes (6.8 ± 0.4) was not significantly different from the VI for LFPs (6.6 ± 0.4) (paired t-test; p = 0.6).
The results were the same when the initial non-selective visual response was included by calculating VI
from the time of search array presentation to the measured selection time. This indicates that the later
17
selection times in the LFPs than in the spiking activity determined from the ANOVA analysis was not a
false result caused by more variability in the LFP data than in the spike data.
Effects of Signal Filtering
We were concerned that the signal filtering during data acquisition distorted the recorded LFP and
artificially delayed the selection times measured in the LFPs relative to spikes (Nelson et al., 2008), so we
directly tested whether signal distortions during data acquisition could have affected the results of our
study. This test is illustrated for two recording sessions in Figure 2.5. First we appended 100 ms of a 40
Hz sine wave to the beginning and end of an actual LFP signal recorded in the covert visual search task.
The appended sine waves were used to align temporally the original LFP signal to signals re-recorded in
the same Plexon data acquisition system (Plexon Inc.) used to record the original LFP (Figure 2.5A). The
original LFP signals with the appended sine waves were converted into “.wav” sound files in Matlab (The
MathWorks, Inc). We used a portable battery powered audio compact disc player (Panasonic Model No.
SL-CT582V) to feed the .wav file signal into a beaker of 0.9% saline in which we placed a 1 MΩ metal
electrode used in the neural recordings. The electrode was connected to the filtered (3.3 – 88 Hz) Plexon
LFP input channel. An unfiltered signal was simultaneously recorded on a separate a/d card (National
Instruments) in the Plexon system normally used to record eye position. Figure 2.5A shows that the
continuous voltage signal recorded on the Plexon analog input channel from the portable audio player
effectively reproduced the original LFP signal. (We tried playing the .wav file from various laptop
computers, but the sound cards attenuated the lower frequencies.) We replaced the actual LFP signal in the
original data file with the re-recorded LFP signals and performed the same time course analysis. Figure
2.5B shows the analysis performed on the .wav signal recorded on the unfiltered analog input channels.
One of the original LFP signals shown was the same one as in Figure 2.3. The average stimulus related LFP
signals for each target position are nearly identical to the originals, and the ANOVA time course analysis
generated the exact same selection times as the originals. This shows that the .wav file and audio signals
reliably reproduced the originally recorded LFP signals. Figure 2.5C shows the analysis performed on the
.wav signals recorded on the filtered LFP input through a 1 MΩ metal electrode in 0.9% saline. There were
18
some distortions of the average stimulus related signals that are apparent when compared to the original
LFP signals, and the significance levels of the ANOVA analysis were lessened. But the important result
was that the measurement of selection time was not appreciably affected. We performed this test on the
LFP signals from 5 separate recording sessions (3 from monkey S and 2 from monkey C). The selection
times determined from the .wav file signals recorded on the LFP input channel ranged from 1 ms before to
2 ms after the selection times determined from the .wav file signal recorded on the analog input channel. In
summary, the results were the same from all 5 sessions – the signal filtering during data acquisition
somewhat altered the shape of the evoked LFP response, but did not affect the determination of selection
time.
19
Figure 2.5: Testing the effects of signal filtering on the determination of selection time.
(A) Voltage signals from the original LFP recording (red line) and audio .wav file signals re-recorded through an unfiltered analog
20
input channel (dark blue) and the filtered LFP input channel. (light blue). A 40 Hz sine wave was appended to the original before
creating the .wav file to temporally align the re-recorded signals. (B) The superimposed average .wav signals recorded through the
analog input channels for two LFP signals from two different monkeys (top), and results of the ANOVA time course analysis
(bottom). The selection times determined from these re-recorded signals are the same as from the original LFP signals (left – 142 ms,
right – 105 ms). (C) The superimposed average .wav signals recorded through the Plexon LFP input channel for the two LFP
signals (top), and results of the ANOVA time course analysis (bottom). The selection times determined from these re-recorded signals
are nearly the same as from the original LFP signals (left – 144 ms, right – 104 ms).
2.3 Results
Spiking activity and LFP responses were recorded concurrently on single electrodes inserted into
the FEF of two monkeys in 43 separate recording sessions. The monkeys performed a memory-guided
saccade task (Figure 2.1A) and one of two covert visual search tasks (Figure 2.1B). In the covert visual
search tasks the monkeys made a manual lever turn as the behavioral report. Monkey S was required to
report the location of the singleton target in the search array (20 recording sites) and monkey C was
required to report the orientation of the C among Os in the search array (23 recording sites). Single neuron
activity recorded with this task was described previously (Thompson et al., 2005b). For this study we
combined the activity from simultaneously recorded single neurons into a single representation of spiking
activity at each recording site. The primary aim of this study was to compare the times that a spatially
selective response first appeared in the LFPs and spikes in the covert visual search task. We refer to this
time as the selection time. For the data collected at a recording site to be included in the study, there must
have been measurable visual response onset latencies in both the LFPs and spikes, and a measurable
selection time in the visual search task for either the LFP response or the spiking activity. In addition, the
visual response latencies and selection times must have occurred before the average reaction time of the
session. Over all sessions, lever turn reaction times averaged 284 ms for monkey S and 297 ms for monkey
C.
There were strong correlations between the directional tuning of the spatially selective responses
in the LFPs and spikes within and across the visual search task and the memory-guided saccade task, which
is consistent with a functional relationship between the LFPs and spikes (see Figure 2.9 below). But first
21
we describe the results of the time course analysis which is blind to the preferred target directions of the
two signals.
Visual response latencies and spatial selection times of LFPs and spikes
The spiking activity and LFP signals recorded simultaneously at each recording site were analyzed
using the same methods to obtain the visual response onset latencies and the time of spatial selection
measured from the time of search array presentation. The temporal relationship between initial visual
response latencies measured in LFPs and spikes was the same across the two tasks, even though visual
response latencies were measured using different visual stimuli and different measurement methods in the
memory-guided saccade and visual search tasks (see METHODS). The initial visual response occurred
earlier in the LFPs than in the spikes. For the memory-guided saccade task, the average ± SE selection
time was 63.4 ± 3.2 ms for LFPs, and 72.8 ± 4.3 ms for spikes (paired t-test: p < 0.001) . For the visual
search task, the average ± SE onset latency was 56.5 ± 2.4 ms for LFPs, and 71.8 ± 4.0 ms for spikes (p <
0.001). There were also strong correlations between the selection times obtained from the memory-guided
saccade task and the visual response onset latencies obtained from the visual search task at each recording
site (LFPs: r = 0.48, p = 0.001; spikes: r = 0.78, p < 0.001). Because we were interested comparing visual
onset times to spatial selection times in visual search, in this study we will focus mostly on results obtained
in the visual search tasks.
Cumulative distributions of onset latencies and selection times measured in the visual search task
are shown separately for the two monkeys in Figure 2.6A and B. Visual response latencies were obtained
for the spiking activity and the LFP response from all 43 recording sites. For spiking activity, the average
± SE onset latency was 68.4 ± 3.5 ms for monkey S, and 74.7 ± 3.3 ms for monkey C. For the LFP
response, the average ± SE onset latency was 53.6 ± 1.0 ms for monkey S, and 59.0 ± 0.9 ms for monkey
C. An ANOVA that factored the monkey and response measure revealed a significant difference in
response latencies between the 2 monkeys (p = 0.02), and between spiking activity and LFP response (p <
0.001) with no interaction between monkey and activity measure (p = 0.86).
Selection times in the visual search task were obtained for spiking activity from 38 (88.4%)
recording sites and for the LFP response from all 43 recording sites. For spiking activity, the average
selection time was 124.6 ± 5.1 ms for monkey S, and 113.0 ± 6.2 ms for monkey C. For the LFP response,
22
the average selection time was 155.2 ± 6.3 ms for monkey S, and 133.3 ± 7.1 ms for monkey C. An
ANOVA revealed a significant difference in the selection times in the visual search task between the 2
monkeys (p = 0.01), and between spiking activity and LFP response (p < 0.001) with no interaction
between monkey and the activity measure (p = 0.43).
The differences in visual response latencies and selection times between the two monkeys may be
due to individual differences or to the different visual stimuli used in two different visual search tasks in the
two monkeys. It has previously been shown that a search for a gap in a C among Os is very easy (Treisman
and Gormican, 1988) and the visual system may be able to resolve a single gap in a circle faster than it can
resolve a color difference in a search array. Nevertheless, the important result is the absence of significant
interaction between monkeys performing different visual search tasks and the measured timing differences
between LFPs and spikes. This means that in spite of the individual differences, the temporal relationships
between LFPs and spikes were the same in the two monkeys.
23
Figure 2.6: Population results from the covert visual search tasks shown separately for the two
monkeys.
(A) Cumulative distributions of visual response latencies and spatial selection times for all recording sites in monkeys S performing
the ‘location’ version of the covert visual search task. The average ± SE times from left to right were 53.6 ± 1.0 ms for LFP visual
latencies (thin dotted line; median = 53.5), 68.4 ± 3.5 ms for spike visual latencies (thin solid line; median = 65), 124.6 ± 5.1 ms for
spike selection times (thick solid line; median = 119), and 155.2 ± 6.3 ms for LFP selection times (thick dotted line; median = 152.5).
(B) The same as (A) but for monkey C performing the ‘identity’ version of the covert visual search task. The average ± SE times from
left to right were 59.0 ± 0.9 ms for LFP visual latencies (median = 59 ms), 74.7 ± 3.3 ms for spike visual latencies (median = 72 ms),
113.0 ± 6.2 ms for spike selection times (median = 102.5 ms), and 133.3 ± 7.1 ms for LFP selection times (median = 129 ms).
(C and D) The percentage of recording sites showing significant modulation at each millisecond following the presentation of the
search array in the monkey S (C), and monkey C (D). The plots were smoothed using a running window of 5 ms for easier viewing.
The line types correspond to those in A and B.
24
To validate the results in Figures 2.6A and 2.6B, we plotted the percentage of recording sites
showing significant modulation at each millisecond following the presentation of the search array
separately for monkey S (Figure 2.6C) and monkey C (Figure 2.6D). For both monkeys, significant visual
responses are evident in the LFPs before the spikes and significant spatially selective responses are evident
in the spikes before the LFPs. Because the relationships between spiking activity and LFP responses were
the same for both monkeys, the data from the two monkeys are combined in the following analyses.
25
Figure 2.7. Population results from the covert visual search task at each recording site combined
across the two monkeys.
(A) Visual response latencies in the covert visual search task of the LFP responses (open squares) and spikes (filled diamonds) at each
recording site sorted by the visual response latency of the spikes. LFP and spike visual response latencies were obtained from all 43
recording sites. The histogram shows the distribution of LFP visual response latency relative to spike visual response latency obtained
across all recording sites (LFP – spikes; mean = -15 ± 2.2 ms). Similar results were obtained from the selection times measured in the
26
memory-guided saccade task (see Figure 2.7C). (B) Selection times in the covert visual search task of the LFP responses (open and
filled circles) and spikes (filled triangles) at each recording site sorted by the selection time of the spikes. LFP and spike selection
times were obtained from 38 recording sites. The histogram shows the distribution of LFP selection time relative to spike selection
time obtained across all recording sites (LFP – spikes; mean = 24.7 ± 5.0 ms). The filled circles in the scatter plot and filled bars in
the histogram represent the 10 recording sites that the spatial tuning of the LFP and spikes differed by more than 40° of visual angle
(see Fig. 2.9B). (C) Selection times in the memory-guided saccade task measured from the LFP responses (open squares) and spikes
(filled diamonds) at each recording site sorted by the selection time of the spikes (N = 42). The histogram shows the distribution of
LFP selection time relative to spike selection time across all recording sites (LFP – spikes; mean = -9.9 ± 2.5 ms). Compare to results
in Figure 2.7A.
We compared the response latencies and selection times measured from the spiking activity and
LFP responses recorded simultaneously at individual recording sites during the visual search task (Figure
2.6 and 2.7). Significant positive correlations between spiking activity and LFP responses for onset
latencies (r = 0.46; p = 0.002), and for selection times (r = 0.51, p = 0.001) support the claim that spiking
activity and LFP responses are related. Spiking activity and LFP response onset latencies for each
recording site are plotted in Figure 2.7A, and selection times are plotted in Figure 2.7B. In both plots, the
times from each site are sorted according to the time measured in the spiking activity and a histogram
shows the distribution of differences between the times obtained from the LFPs and spikes. For nearly all
(41/43 = 95%) of the recording sites, the measured response onset latency was earlier in the LFP response
than in the spiking activity. On average, the LFP visual response began 15.3 ± 2.2 ms earlier than the spike
visual response. The visual latencies of the LFP responses varied less than the spike responses. As a
consequence, the difference between visual onset latency measured in the spikes and in the LFP increased
with increasing spike response latency. Nevertheless, even the recording sites with the earliest spike
responses had LFP response latencies that were significantly earlier. For the quartile of recording sites with
the earliest spike visual response (range: 48 – 60 ms), the LFP visual response began on average 2.3 ± 0.7
ms earlier than the spike visual response (paired t-test: p = 0.01). The earlier initial visual onsets in the
LFP signal is consistent with the expected result that feedforward visual inputs in postsynaptic potentials
precedes the visual evoked spiking activity (Schroeder et al., 1998).
Selection times in the visual search tasks were obtained from all 43 sites for the LFP response and
from 38 sites for spiking activity. For the 38 recording sites with selection times from both measures,
selection times occurred later in the LFP response than in the spiking activity for 84% (32/38) of the
27
recording sites and differed, on average, by 24.7 ± 5.0 ms (Figure 2.7B). However, at 10 recording sites,
the spatial tuning of the LFP response and spiking activity differed by more than 40° of visual angle (see
Figure 2.9B); these are indicated in Figure 2.7B by the filled circles in the scatter plot and shaded bars in
the histogram. It is possible that at these recording sites the LFP response and spiking activity were less
related to each other than at the sites in which the spatial tuning of the two signals corresponds. When
these 10 sessions were removed from the analysis, the selection times occurred later in the LFP response
than in the spiking activity at 93% (26/28) of the recording sites and differed, on average, by 31.5 ± 5.1 ms.
For comparison we also compared the selection times for LFPs and spiking activity in the
memory-guided saccade task (Figure 2.7C). Selection time in the memory-guided saccade task measures
visual response latency because it identifies the first time that the responses differed across target locations
for a single visual stimulus presented alone. It corresponds to the visual response latency measured in the
covert visual search task, and across the recording sites the two measures were strongly correlated for both
spikes (Pearson’s r = 0.78, p < 0.001) and LFPs (r = 0.60, p < 0 001). Just like the visual response latencies
measured in the visual search task (Figure 2.7A), the selection times measured in the memory-guided
saccade task were earlier (9.9 ± 2.5 ms) for LFPs than for spikes (Figure 2.7C). The similarity in the
results across the tasks and analysis methods is an indication that the later LFP selection times in the visual
search task are not due to the quality of the LFP signal or the analysis methods is the result of the selection
time analysis in the memory-guided saccade task. Overall, approximately 2.5 times more trials were
included in the analysis of visual search data (an average of 290 trials per session) than were included in the
analysis of the memory-guided saccade data (an average of 110 trials per session). Because variability
affects an ANOVA more when there are fewer samples, it should have affected the selection time results
from the memory-guided saccade task more than from the visual search tasks. But in the memory-guided
saccade task, the selection times were on average 9.9 ± 2.5 ms earlier in the LFPs than in the spikes (Figure
2.7C). This shows that the later LFP selection times observed in the visual search task (Figure 2.7B) are
unlikely due to the analysis method.
28
Figure 2.8. The relationship of selection time in LFPs and spikes to LFP visual response latency.
The symbols representing the different times are the same as in Fig. 2.7. The visual response latencies and selection times across all
recording sites are sorted by increasing LFP visual response latency. Each of the data points plot the average for a group of eight
sorted recording sites. Consecutive data points represent the average of eight recording sites after shifting the averaging window by
one. The statistical comparisons are shown at the top (large symbols). The averages ± SE of the response latencies and selection times
are plotted after dividing the recording sites into 2 groups based on LFP visual response latency. The recording sites with LFP visual
response latencies between 48-55 ms were assigned to the ‘early’ group (N = 22; LFP visual response latencies = 51.7 ± 0.5 ms; spike
visual response latencies = 65.0 ± 3.3 ms; spike selection times = 109.5 ± 4.7 ms; LFP selection times = 145.4 ± 6.9 ms), and
recording sites with visual response latencies between 56-67 ms to the ‘late’ group (N=21; LFP visual response latencies = 61.5 ± 0.8
ms; spike visual response latencies = 79.0 ± 2.9 ms; spike selection times = 126.3 ± 6.6 ms; LFP selection times = 141.5 ± 7.6 ms).
Relationship of LFP visual response latency to selection time
Studies have shown that the earliest visual response latencies of LFPs recorded in dorsal stream
areas of visual cortex are in cortical layer 4 which corresponds to the feedforward projection of visual
inputs (Chen et al., 2007; Schroeder et al., 1998). We hypothesized that if the inputs to FEF from visual
29
cortex were spatially selective, they would be evident first at the recording sites with the earliest LFP visual
response latencies. Therefore, we examined whether LFP visual response latencies were related to times of
spatial selection (Figure 2.8). It should be noted that this analysis does not establish the cortical layer of the
recording sites, but it is motivated by the assumption that recording sites in FEF with earlier visually
evoked LFP activity are functionally closer to the feedforward visual input from visual cortex.
Surprisingly, this was not the case. To visualize the data we plotted how the spike visual response latencies
and selection times, and LFP selection times changed with increasing LFP visual response latency (Figure
2.8). For statistical analysis, the recording sites were divided into two groups based on LFP visual response
latency measured in the visual search task. The sites with LFP visual response latencies between 48 and 55
ms were assigned to the ‘early’ group (N = 22), and sites with latencies between 56 and 67 ms to the ‘late’
group (N=21). The large symbols in Figure 2.8 indicate the average ± SE of each group. The spike visual
response latencies differed significantly across the ‘early’ (65.0 ± 3.3 ms) and ‘late’ groups (79.0 ± 2.9 ms)
(t-test: p = 0.003). This is consistent with the result that LFP and spike visual latencies were positively
correlated. LFP response selection times did not differ significantly between the ‘early’ (145.4 ± 6.9 ms)
and ‘late’ (141.5 ± 7.6 ms) groups (p = 0.7). For the spiking activity, the selection times of the ‘early’
(109.5 ± 4.7 ms) and ‘late’ (126.3 ± 6.6 ms) groups were marginally different (p = 0.04). The surprising
result that the recording sites with the earliest LFP visual response latencies exhibited the earliest spike
selection times and the latest LFP selection times suggests that spatial selectivity first appears in the spiking
output of neurons closest to the feedforward input from the visual cortex. The difference between spike and
LFP selection times in the ‘early’ group was highly significant (paired t-test: p < 10-5). For the ‘late’ group,
the difference between the LFP and spike selection times did not reach statistical significance (p = 0.07).
We also divided the recording sessions into ‘early’ and ‘late’ groups based on the ‘selection times’
measured in the data collected from the memory-guided saccade task which were recorded in a separate
block of trials in each session. Note that ‘selection time’ for the memory-guided saccade data is
determined using the exact same analysis method as for visual search data but actually measures the visual
onset latency to a single visual stimulus. The results were statistically identical to those shown in Figure
2.8.
30
Comparison of spatial tuning
The variation of spatially selective LFP and spiking responses with target direction in the
memory-guided saccade and visual search tasks was characterized with Gaussian functions (Figures 2.2E
and 3E). The spatial parameters of the best-fit Gaussian curves provide estimates of the preferred direction
and spatial extent of the LFP and spiking response fields. Details of the spatial tuning analysis are provided
in the METHODS. There were no differences in the directional tuning measures between the two
monkeys. The preferred direction was provided by the optimum direction (Φ) parameter. The preferred
tuning directions of the spiking activity and LFP responses in the memory-guided saccade and visual search
tasks were compared by taking the angle difference between the two measures. Angle differences can
range from -180° to +180°. Figure 2.9 shows the distributions of angle differences between the preferred
target directions of the LFPs and spikes for the memory-guided saccade and visual search tasks (Figures
2.9A and B), and between the preferred target directions obtained from the memory-guide saccade and
visual search tasks for spikes and LFPs (Figures 2.9C and D). All the distributions are peaked near 0°
(Rayleigh test; p < 0.001). An analysis that measures the correlation between two circular variables
(Mardia and Jupp, 2000) showed that there were strong correlations between the preferred directions
obtained from LFPs and spikes in the memory-guided saccade task (Figure 2.9A; p < 10-9) and the visual
search task (Figure 2.9B; p = 0.001). There were also strong correlations between the preferred directions
obtained across the two tasks for both spikes (Figure 2.9C; p < 10-8) and LFPs (Figure 2.9D, p = 0.001). In
summary, there were overall strong correlations between the directional tuning of the LFP and spike
response fields across the memory-guided saccade task in which a visual stimulus is presented alone and
the covert visual search tasks in which the target must be identified among distractors.
31
Figure 2.9. Comparisons of spatial tuning in spiking activity and LFP responses recorded in the
memory-guided saccade and covert visual search tasks.
(A-D) The distributions of the differences in the preferred target directions measured from the spiking activity and LFP responses for
the recording sites that exhibited significant spatial tuning in the spiking activity. LFP responses exhibited significant spatial tuning at
all 43 recording sites in both the memory-guided saccade and covert visual search tasks. Angle differences can range from -180° to
+180°. All of the distributions are peaked near 0° (Rayleigh test; p < 0.001) A circular correlation analysis (Mardia and Jupp, 2000)
showed that the preferred target directions are significantly correlated between: (A) LFPs and spikes recorded in the memory-guided
32
saccade task (N = 42; p < 10-9). (B) LFPs and spikes recorded in the covert visual search tasks (N = 38; p = 0.001). (C) Spikes
recorded in the memory-guided saccade task and spikes recorded in the covert visual search tasks (N = 37; p < 10-8). (D) LFPs
recorded in the memory-guided saccade task and LFPs recorded in the covert visual search tasks (N = 43; p = 0.001). (E) The
distribution of tuning widths of the LFP (open bars) and spike (filled bars) responses in the memory-guided saccade task. Tuning
width was defined as the standard deviation (Tφ) parameter of the best fit Gaussian curves. The average response field width is 38.7 ±
3.7° for spiking activity, and 64.0 ± 3.0° for LFP responses; and the two distributions differ significantly (paired t-test; p < 10-7).
(F) The distribution of tuning widths of the LFP and spike responses in the covert visual search tasks. The average response field
width is 26.5 ± 2.6° for spiking activity, and 44.7 ± 3.3° for LFP responses (paired t-test; p < 10-4).
The tuning widths of the LFP and spike response fields were estimated by the standard deviation
(Tφ) parameter of the best fit Gaussian curves. The distributions of tuning widths (in polar angle
coordinates) for the single visual stimulus in the memory-guided saccade task and for the target among
distractors in the search tasks are shown in Figures 2.9E and 9F, respectively. For the memory guided
saccade task, the average response field width is 38.7 ± 3.7° for spiking activity, and 64.0 ± 3.0° for LFP
responses. For the visual search tasks, the average response field width is 26.5 ± 2.6° for spiking activity,
and 44.7 ± 3.3° for LFP responses. The results of an ANOVA showed that LFP tuning widths were
significantly larger than spike tuning widths (p < 0.001), and the tuning widths of responses in the memoryguided saccade task were significantly larger than in the visual search task (p < 0.001). When converted to
visual field angles according to the law of cosines, the width of receptive fields for a single visual stimulus
alone averaged 6.5 ± 0.4° for spikes and 10.4 ± 0.4° for LFPs; and receptive fields in the visual search task
averaged 4.6 ± 0.4° for spikes and 7.4 ± 0.5° for LFPs. The sizes of receptive fields of the spiking activity
and the narrower spatial tuning in the visual search task than for single targets are similar to previous
reports that used comparable methods (Schall et al., 1995a; Schall et al., 2004). This is important because
it shows that our data is consistent with previous reports and suggests that the results of this study are
reproducible in slightly different visual search tasks.
2.4 Discussion
We show for the first time that LFPs in FEF exhibit visually evoked responses that are spatially
selective; they identify the location of a target presented alone in a memory-guided saccade task and
33
identify the location of a behaviorally important stimulus during covert visual search in the absence of eye
movements. We compared the LFP responses to the single unit activity recorded concurrently on the same
electrodes (Thompson et al., 2005b). In the covert visual search task, both the LFPs and spikes exhibited a
short latency non-spatially selective visual response followed by a selective response that identified the
location of the behaviorally relevant stimulus that instructed the monkey to manually turn a lever to the left
or right. The spatial selectivity for the behaviorally relevant target in the visual search task appeared in the
spiking activity before the LFP response. This result is especially intriguing because it suggests that a
cognitive representation identifying the location of behaviorally important visual stimuli is computed in the
FEF from non-spatially selective inputs (Thompson and Bichot, 2005; Thompson et al., 2005a).
The spatial tuning for target location was consistent across tasks and across LFPs and spikes at
each recording site, but was generally broader in the LFP signal than in the spikes. Previous spike vs. LFP
comparisons either used full field visual stimulation (e.g. Chen et al., 2007; Logothetis et al. 2001) or
placed visual stimuli based on the spatial extent of the spike receptive fields (Fries et al., 2001; Liu and
Newsome, 2006; Pesaran et al., 2002). We are not aware of any study that compared the spatial extent of
visual responses of LFPs to that of spikes recorded on the same electrode. But the broader spatial tuning in
LFPs than in the spikes is consistent with the view that LFPs reflect synaptic activity over a larger area of
cortex than is reflected in the spiking output of a few localized neurons (Kreiman et al., 2006; Liu and
Newsome, 2006; Logothetis et al., 2007; Logothetis and Wandell, 2004; Mitzdorf, 1985; Mitzdorf, 1987).
Nevertheless, the overall strong correlations of spatial tuning between the LFP responses and spiking
activity when a target was presented alone and when presented among distractors indicate that the LFP and
spike signals originate from the same region of FEF.
In a recent study, Buschman and Miller (2007) compared the time course of spatially selective
spiking activity recorded simultaneously in FEF and the lateral intraparietal area (LIP), an area that is
interconnected with FEF, in monkeys performing visual search tasks. Their results suggest that spatial
attention signals appear first in the FEF during top-down attention and first in LIP during bottom-up
attention. The implication is that visually driven attention signals flow from LIP to FEF and cognitively
driven attention signals flow from FEF to LIP. Although simultaneous spike recordings can be used to
34
compare signals in interconnected areas, this experimental method does not address whether or how
different brain areas influence each other or how synaptic inputs are transformed into spiking outputs in a
given area. In addition, the results of Buschman and Miller (2007) have been called into question mainly
due the difficulty in knowing whether the neurons recorded in LIP and FEF in that study were those that
received input from or influenced activity in the other brain area (Schall et al., 2007). The combined LFPspike analysis described in this study may be able to address some of these unresolved issues.
Combined analysis of LFP and spiking activity can provide information about computations that
cannot be obtained when these signals are considered separately (Kreiman et al., 2006; Nielsen et al.,
2006). In the cerebral cortex, there is strong evidence that the LFP is a mass signal that is primarily
influenced by the excitatory postsynaptic potentials of dendrites (Chen et al., 2007; Cruikshank et al., 2002;
Juergens et al., 1999; Kaur et al., 2004; Kreiman et al., 2006; Logothetis and Wandell, 2004; Mitzdorf,
1985; Mitzdorf, 1987; Nielsen et al., 2006), and thus reflects inputs from other brain regions as well as
local neural processes mediated by interneurons. Spiking activity reflects local processing and the outputs
of neurons to other brain regions. Simultaneous LFP and spike recordings provide a way to compare the
dendritic input to the spiking output, which is required to understand the transformation of neural signals
from one processing stage to the next. In general, brain areas where cognitive functions are computed
should show response modulations in the spiking activity of single units before they appear in the LFP.
Whereas, the brain areas that receive this information from other areas should show response modulations
first in the LFP, or simultaneously in the LFP and spiking activity (Nielsen et al., 2006). In this study, we
specifically examined the transformation of a non-selective visual representation of items in a search array
into a cognitive signal that identifies the location of the behaviorally relevant target stimulus.
The FEF is an important site of convergence in the visual system (Jouve et al., 1998; Schall, 1997;
Schall et al., 1995b; Vezoli et al., 2004). The FEF receives retinotopically organized input from dorsal
stream visual areas MT, MST, and LIP; ventral stream visual areas V4, TEO, and TE; and from the
supplementary eye field and prefrontal areas 46 and 12. The dorsal stream innervation is most likely
responsible for the fast non-selective initial visual responses we measured in the LFP and spikes (Bisley et
al., 2004; Chen et al., 2007; Pouget et al., 2005; Schmolesky et al., 1998). The latencies of the initial
visually-evoked LFP and spike responses were correlated, appearing in the LFP signals about 15 ms before
35
the spikes in the visual search tasks, and about 10 ms before the spikes in the memory-guided saccade task.
At the recording sites with the earliest spike latencies, the LFP latency was about 2 ms earlier. The earlier
visually-evoked modulation in the LFP than in the spike response is consistent with studies in visual cortex
(Logothetis et al., 2001; Schroeder et al., 1998), and with the hypothesis that the LFP signal reflects
synaptic input and indicates that the initial visual response was relayed to the FEF from other brain areas.
The reverse temporal relationship was found in the visual search data when we compared the time
course of spatial selectivity in the LFP response and spiking activity. Following the initial non-selective
visual response, a spatially selective signal identifying the location of the search array target emerged first
in the spiking activity, and then in the LFP signal about 30 ms later. The earlier spatially selective signal in
the spiking activity suggests that the representation of the location of the behaviorally relevant target
stimulus is computed within the FEF rather than relayed from other brain areas.
This temporal difference could not have been produced by LFP signal distortion (see
METHODS). Our tests showed only a 1-2 ms delay due to our recording methods. Another indication that
this was not a serious problem was the earlier LFP visual onset latencies measured in the visual search
tasks (see Figure 2.7A) and earlier LFP selection times in the memory-guided saccade tasks (see Figure
2.7C). If signal filtering caused the delay of the selection times in the visual search LFP signals, it should
have also caused delay of selection times in the memory-guided saccade LFP signals – but this was not the
case.
The alternative interpretation is that some modulations in synaptic activity cannot be detected in
event related brain potentials using the methods we employed in this study. It is possible that FEF generates
the strong spatially selective spiking signals by amplifying weak differences in the synaptic inputs.
Although the exact nature of the input signals to FEF is currently unknown, they must contain information
about the visual stimuli, and differences between them. Our results suggest that computations in FEF
convert these differences into a strong categorical representation identifying the target location, regardless
of the visual feature that differentiates the target from distractors. Consistent with this view, in our study
we used two different classes of visual features, color and shape, and we obtained the same results.
The recording sites with the earliest LFP visual response latencies tended to have the earliest
spatial selection times in the spiking activity. In dorsal stream visual areas of monkey cortex, LFPs
36
recorded in lamina 4 have the shortest visual response latencies due to feedforward input from lower areas
(Chen et al., 2007; Schroeder et al., 1998). We therefore made the reasonable assumption that the FEF
recording sites with the earliest LFP visual response latencies were functionally closer to the feedforward
inputs. Although we cannot identify the cortical layers we were recording from, the results depicted in Fig.
2.8 suggest that spatial selectivity in FEF originates first in neurons near the feedforward input and then is
distributed to the functionally more distant regions in FEF via local connections or feedback from other
areas. Consistent with this view, at the recording sites with the latest LFP visual response latencies, the
selection times measured in the LFP and spikes did not differ significantly. This is the first evidence for
such functional architecture and further studies are clearly needed test this hypothesis.
Our results suggest that spatial selectivity during a pop-out covert visual task is generated in FEF
from non-spatially selective inputs. A few studies have examined the relationships between LFP and
spiking responses in other areas. In area MT, for example, Liu and Newsome (2006) found that tuning for
motion direction and speed in LFP responses are highly correlated with that of spike activity. In
inferotemporal (IT) cortex, Krieman et al. (2006) showed a simultaneous time course of object selectivity
in LFP responses and spiking activity. A study by Nielsen et al. (2006) showed that spikes and LFPs in IT
exhibited learned object selectivity and the modulation of LFP responses, but not spiking activity, grew
stronger from posterior to anterior IT. Because LFP modulation reflects the synaptic input, they concluded
that learned object selectivity was encoded first in posterior IT and then transmitted to anterior IT. Only
one study, conducted in area V4, has compared the spatial selection process measured in LFPs and spikes
during visual search (Bichot et al., 2005). In that study, spatially selective responses appeared in the LFP
and spikes at the same time. Although it was not specifically addressed in that study, the simultaneous
modulation in LFP and spikes suggests that the spatial selectivity was present in the inputs.
The combined analysis of LFPs and spikes promises to provide useful information for
understanding computations in the brain. Also, LFPs recorded in monkeys can be an important link
between monkey single unit data and human EEG and imaging data (Logothetis and Wandell, 2004;
Woodman et al., 2007). For example, the spatially selective LFP response we report could be related to the
attention-related modulations observed in human EEG recordings during visual search (Luck and Hillyard,
1994). Single units, LFPs and EEG recordings provide high temporal resolution. It is more difficult,
37
however, to localize the source of the computations reflected in EEG recordings than in the other two
signals. EEGs recorded from scalp electrodes reflect the post-synaptic potentials summed over a large
region of the brain that could include many areas that are related to spatial vision. The FEF is just one of
the potential sources of the spatially selective signals necessary for spatial attention (Pessoa et al., 2003;
Serences and Yantis, 2006). Further work is needed to determine the relationships between LFPs and spikes
within and between the many regions of the brain involved in spatial attention.
Chapter 3
Spatial selective signals in the frontal eye fields mediate
spatial attention and predict accuracy in a covert visual
search task that requires object identification
Relating spatial attention to neural activity requires an ability to create behavioral tasks that allow the
separation of spatial attention from other cognitive factors and a potential target area in the brain from
which to obtain the neuronal data. In this chapter, we explore evidence from our experiments that combine
neurophysiology and behavior in order to demonstrate that FEF is one of the sources of the spatially
selective signal in the brain. We explore the time course of the spatial selection in FEF in local field
potentials (LFP) and neural spiking activity.
3.1 Introduction
Visual spatial attention is a critical component of normal vision and is necessary for the
recognition of objects in natural environments (Sheinberg and Logothetis, 2001; Rensink, 2002). It
enhances the representation of visual information at selected peripheral locations thereby improving
detection and discrimination (Maunsell and Cook, 2002).
It is widely accepted that the spatially selective signals in the frontal eye field (FEF) are
associated with the planning and execution of saccadic eye movements (Goldberg and Segraves, 1989;
Hanes et al., 1998; Schall and Thompson, 1999; Tehovnik et al., 2000). A growing body of evidence
suggests that FEF also plays a causal role in directing covert spatial attention (Awh et al., 2006). Human
fMRI studies show that the FEF is active during the allocation of attention with and without eye
movements (Beauchamp et al., 2001; Corbetta and Shulman, 2002; Kincade et al., 2005; Bressler et al.,
2008; Kelley et al., 2008). Transcranial magnetic stimulation over FEF modulates perceptual
38
39
performance in covert attention tasks (Grosbras and Paus, 2002; Muggleton et al., 2003; Smith et al.,
2005), and also modulates visual activity in extrastriate visual cortex (Silvanto et al., 2006; Taylor et al.,
2007; Morishima et al., 2009). In monkeys, electrical microstimulation of FEF enhances perception
(Moore and Fallah, 2001, , 2004; Schafer and Moore, 2007) and produces enhanced visual responses in
extrastriate visual cortex that resemble the effects of covert spatial attention (Moore and Armstrong,
2003; Armstrong and Moore, 2007; Ekstrom et al., 2008). Inactivation of FEF disrupts target detection
during covert visual search (Wardak et al., 2006) Neuron recordings in monkeys have shown that the
activity of visually responsive FEF neurons identifies the location of an attended visual stimulus, even in
the absence of eye movements (Kodaka et al., 1997; Murthy et al., 2001; Sato and Schall, 2003;
Thompson et al., 2005a). Conspicuously absent from the evidence linking FEF to covert spatial attention,
however, is data that directly link neuronal activity in FEF to behavioral measures of attention.
In this study we compare the activity of neurons during trials with differing behavioral outcomes
to determine whether and when neural signals in FEF predict the monkeys’ accuracy and speed of
identifying a target object during visual search. The results support the hypothesis that FEF activity plays
a causal role in directing covert spatial attention and enhances the processing of a stimulus to be
identified as it is being processed by the visual system.
3.2 Methods
Data collection
The data were collected from two male monkeys (Macaca mulatta) weighing 6.5 kg (monkey C)
and 7.8 kg (monkey B). All surgical and experimental protocols were approved by the National Eye
Institute Animal Care and Use Committee and complied with the National Institutes of Health Guide for the
Care and Use of Laboratory Animals. Sterile surgery was performed under ketamine and isofluorane
anesthesia to place a head-holding device, a plastic recording chamber over the frontal eye field, and a
scleral search coil. Frontal eye field was localized within the recording chamber using low current
microstimulation (< 50 µA) to evoke saccades, and by the presence of saccade-related movement neurons
(Bruce and Goldberg, 1985). Neuronal recordings were made in the left FEF in monkey C and in the right
40
FEF in monkey B. Recording sites were confirmed to be in the rostral bank of the arcuate sulcus by MRI
in both monkeys and histologically in monkey C.
Visual stimulation and behavioral control was done by a computer running REX (Hays et al.,
1982). Visual stimuli were presented on a computer monitor (26 cm X 21 cm, 1024 X 768 pixel resolution,
60 Hz frame rate) viewed at a distance of 57 cm. In each recording session, a single tungsten
microelectrode (Frederick Haer, Bowdoin, ME) was inserted into the FEF by a motorized microdrive under
computer control. Action potential waveforms were digitized, and saved using a computer-based data
acquisition system (Plexon). Often two or three units were recorded simultaneously. Offline spike sorting
separated single units based on the size and shape of the spike waveforms. Analog eye position and lever
position signals were digitized and sampled at 1 kHz.
Behavioral training and tasks
Monkeys were seated in a primate chair with head fixed. Using operant conditioning with positive
reinforcement, the monkeys were trained to perform a memory-guided saccade task and a covert visual
search task. The two tasks were run in separate blocks of trials.
The memory-guided saccade task was used to map the spatial extent of each neuron’s response
field (Bruce and Goldberg, 1985). After the monkey fixated a 0.3° diameter grey spot on a black
background for 400-800 ms, an identical spot was flashed for 50 ms at a peripheral location. The monkeys
were required to maintain fixation on the central spot for a random interval ranging from 800 to 1400 ms.
After the central spot disappeared, the monkeys were rewarded for making a saccade to the remembered
location of the target. Once gaze shifted, the target reappeared to provide feedback and a fixation target for
the monkeys.
After mapping the neuron’s response field with the memory-guided saccade task, the monkeys
performed the cued covert visual search task illustrated in Figure 3.1a. A lever that could be turned left or
right of vertical was attached to the front of the chair within easy reach of the monkey. When no force was
applied to the lever, a spring automatically returned it to the vertical position. Although they were free to
use either hand to grasp and turn the lever, both monkeys were exclusively right-handed.
After grasping the lever and positioning it within 10° of vertical, a small (0.3°) central yellow
fixation cross appeared on a black background. The monkeys were required to maintain eye position
41
within 2° of the central fixation cross until the reward. After fixating the central cross for a random interval
(400-800 ms), a cue array was presented that was comprised of eight 2° diameter isoluminant rings spaced
equally around the fixation cross. The rings were isoeccentric and adjusted so that at least one ring was
inside the neuron’s response field. Response field eccentricities of recorded neurons ranged between 8°
and 12° of visual angle. On about 75% of trials one of the rings was red and the other seven rings were
green. On the remaining 25% of trials, all the rings were green. The position of the red ring was
randomized from trial to trial and served as a spatial cue for the most probable location of the visual search
target to be identified.
After a cue-target onset asynchrony (CTOA) lasting between 0 and 470 ms one item of the search
array appeared simultaneously inside each of the colored rings. All items of the search array were gray in
color and 1° in diameter. The target of the search array was a leftward or rightward oriented Landolt C
target with a 0.5° gap on the left or right. The Landolt C target appeared on about 75% of trials (range:
66% to 83% across sessions). The monkeys’ task was to report the presence and identity of a Landolt C
target with a leftward or rightward lever turn that corresponded to the location of the gap in the Landolt C
target. The monkeys were rewarded for making the correct lever turn (> 15° from vertical) within 2
seconds following search array presentation; in practice, the monkeys nearly always turned the lever to the
limit of 35° from vertical. If the monkey broke fixation at any time during the trial, released the lever, or
made an incorrect lever turn, the trial was immediately aborted. In about 25% of trials (ranging from 17%
to 34% across sessions) no target and only distractors were presented and the monkeys were rewarded for
releasing the lever. Overall, about 46% of target present trials were valid cue trials; the Landolt C target
appeared inside the red ring (range from 31% to 55%). About 29% of target present trials were invalid cue
trials; the Landolt C target appeared inside one of the three green rings opposite the red ring (range from
14% to 39%). Therefore, on invalid cue trials each of the three target locations opposite the red ring had
about a 10% probability of containing the C target. The remaining 25% of target present trials were neutral
cue trials; all the rings in the cue array were green.
Combining the probabilities of cue or neutral cue, target or no target trial, and valid or invalid cue,
the approximate percentages of each trial condition shown in Figure 3.1a within a recording session are as
follows: valid cue trials – 34%, invalid cue trials – 22%, neutral cue target trials – 19%, cue no target
42
trials – 19%, and neutral cue no target trials – 6%.
Although the task was the same for the two monkeys, there were some differences in the visual
stimuli. Data was collected first from monkey C. In this monkey, we tested various CTOAs between 0 and
470 ms. We found that reaction time was longest for 0 ms CTOAs, and reaction times did not differ across
trials with CTOAs greater than 100ms (Fig. 3.1b). For monkey B we used only three CTOAs: a short
CTOA of 33 ms, a medium CTOA of 167 ms, and a long CTOA of 300 ms. The luminance of the search
array and were also different between the two monkeys. Luminance was measured using a Minolta CA100 spectrophotometer. For monkey C, the luminance of all the visual stimuli was 3.7 cd/m2 on a dark
gray background of 0.5 cd//m2. During practice sessions with monkey B it became evident that the task
was too easy. Therefore, to generate enough errors for analysis it was necessary to make the task more
difficult. We did this by decreasing the luminance of the search array stimuli to 0.2 cd/m2 on a black
background of 0.02 cd/m2. In addition, to make the task more difficult for monkey B, we altered the shape
of the distractors. For monkey C the distractors of the search array were gray rings with no gaps, for
monkey B they were upward or downward oriented Landolt Cs as illustrated in Figure 3.1. Finally, to
provide further encouragement to monkey B to direct attention to the red ring, on about 70% of valid cue
trial the search array stimuli were presented for only 50 ms. The behavioral and neurophysiological results
were the same from trials in which the search array was presented briefly and from trials in which the
search array remained on and were combined for the behavioral and neurophysiological analyses. In all
invalid cue and neutral cue trials the search array remained on the screen for the entire trial. The results
from 30 recording sessions are included in this study (monkey C: 10 sessions; monkey B: 20 sessions).
Data analysis
A lever turn was defined as turn > 15° from vertical. The beginning and end of each lever turn
was defined as the beginning and end of the monotonic change in lever position before and after the 15°
threshold was reached. The time of the beginning of the lever turn on each trial was used as the reaction
time for that trial. The only trials included in the analysis where those in which the monkeys maintained
fixation on the central fixation cross until after the behavioral report of lever turn or lever release.
Spike density functions were calculated for each trial by convolving spike times with a Gaussian
filter (σ = 10 ms). For the analysis of neural activity, we compared activity on trials when the target of the
43
search array was in the response field to activity on trials when only distractors were in the response field.
For invalid cue trials, distractor related activity was from trials in which the red cue stimulus appeared in
the response field. To be included in any of the analyses in this study a neuron had to exhibit significant
spatially selective activity before the lever turn on correct trials and have at least 4 trials for each trial
condition being compared.
The time course and magnitude of spatially selective activity was quantified by calculating the
area under the receiver operating characteristic (ROC) curve at each millisecond from activity aligned on
the time of search array presentation. This method has been described in detail in previous reports
(Thompson et al., 1996; Thompson et al., 2005b). The ROC area measures the separation of 2 distributions
normalized to values between 0 and 1 and corresponds to the probability of an ideal observer correctly
identifying which distribution a sample belongs (Green and Swets, 1966). An ROC area of 0.5 indicates
that the 2 distributions of activity are completely indistinguishable. ROC areas of 0 and 1 are equivalent
statistically; both indicate that the 2 distributions are completely separate. For each neuron in this study,
the ROC analysis was used to compare the activity during trials in which the search array target was
presented in the RF was compared to the activity during trials in which the distractor was presented in the
RF. As a convention, the analysis was structures so that ROC area values > 0.5 indicate that the activity
during trials that the target was in the RF was greater than the activity during trials that distractors were in
the RF.
Only those neurons that exhibited spatially selective activity in the cued covert search task were
included in this study. A neuron was determined to exhibit spatially selective activity if the average
activity during correct valid cue trials was significantly greater in trials in which the target was in the RF
than in trials in which a distractor was in the RF during the time interval between 150 ms following the
appearance of the search array and the median reaction time. Statistical significance was determined using
a t-test (p < 0.05). To be included in an analysis, there must have been at least 4 trials for each of the trial
conditions being compared. In addition, the ROC area comparing target and distractor related activity on
correct valid cue trials had to reach a value of 0.6.
44
3.3 Results
We recorded single unit activity in the FEF of two monkeys performing a cued covert visual
search task (Fig. 3.1a). The monkeys’ task was to maintain fixation on the central cross and report the
presence and identity of a peripheral leftward or rightward oriented gray Landolt C in the search array with
a leftward or rightward lever turn, respectively, or to report target absence by releasing the lever In about
75% of trials a popout red ring among green rings in a cue array identified the most probable target
location. In the remaining neutral cue trials, all the rings in the cue array were green.
Figure 3.1: Task and behavior.
a, The cued covert visual search task. After the monkey grasped the lever in the vertical position and fixated a small central fixation
stimulus a cue array appeared. Cue trials – In about 75% of trials one of the rings was red and the rest were green. Neutral cue trials
–In about 25% of trials all the rings were green. After a variable cue-target onset asynchrony (CTOA) ranging from 0 to 470 ms a
search array appeared. The target of the search array was a leftward or rightward oriented Landolt C. The location of the red ring
identified most probable target location, but in a minority of trials the cue was invalid – the target C appeared in one of the 3 green
rings directly opposite the red ring. The monkeys were required to maintain fixation at the center and were rewarded for turning the
lever in the same direction as the leftward or rightward oriented C target and for releasing the lever if there was no target. Reaction
time (RT) was measured from search array presentation to the time of the monkeys’ behavioral report. The temporal structure of the
task is illustrated at the bottom. The approximate percentages of each trial condition are indicated in parentheses. b, Cue-related
behavior. Average RT and percent correct ± SEM across recording sessions is shown for monkey C (left) and for monkey B (right)
from valid cue trials (red bars), invalid cue trials (green bars), and neutral cue trials (white bars). Percent correct performance from
45
no target trials is shown for each monkey (gray bars). For Monkey C we grouped trials into 4 groups based on similar CTOAs. For
Monkey B all recording sessions included the same three CTOAs: 33 ms, 167 ms, and 300 ms. For each monkey, the average RT
and percent correct across all trials is shown at the far right. c, Average reaction times of monkey C (top) and monkey B (bottom) in
correct and error trials for each trial condition. The color conventions are the same as in b. Errors in target present trials are
separated by behavior into lever turn errors and lever release errors.
The critical manipulations in this study were whether or not there was a red cue ring and the
spatial relationship between the red cue ring and the Landolt C target. For each monkey we determined
whether reaction time and performance accuracy varied with whether or not there was a spatial cue, with
cue validity, and with duration of the cue-target onset asynchrony (CTOA)(Fig. 3.1b). The behavioral
results were similar across CTOAs. The only exception was that reaction times were longer for the
shortest CTOAs (0 ms for monkey C and 33 ms for monkey B). For all CTOAs, reaction times were
shorter and performance accuracy was better on valid cue trials compared to invalid cue trials. For
reaction time, an ANOVA that factored the type of trial (valid cue trial vs. invalid cue trial) and CTOA
group revealed a significant difference in reaction times between valid and invalid cue trials (p = 0.005
for Monkey C, p<.001 for Monkey B), and across CTOA groups (p<.001 for both monkeys) due to the
longer reaction times for the shortest CTOAs. When the shortest CTOA trials were removed, there was
no significant difference in reaction time across the remaining CTOA groups for either monkey. There
was a significant difference in accuracy across valid cue and invalid cue trials (monkey C: p = 0.04;
monkey B: p<.00001), with no difference across CTOA groups for either monkey. There were no
significant interactions between CTOA and trial type for reaction time or performance accuracy.
The behavioral results from neutral cue trials tended to be intermediate between valid and
invalid cue trials for monkey C, and similar to the results from invalid cue trials for monkey B.
Performance accuracy on no target trials is included in Figure 3.1b. Performance accuracy did not differ
significantly across CTOAs for either monkey. For all trial types the performance accuracy was well
above chance for a three alternative choice task (33%) which indicates that the monkeys were using the
stimuli in the search array to guide their behavioral reports. In addition, the improvement of performance
on valid cue trials relative to invalid and neutral cue trials indicates that the monkeys used the red ring as
a spatial cue to guide attention.
Average reaction times from correct and error trials across all recording sessions and CTOAs are
46
compared in Figure 3.1c. There were two types of errors in trials in which a target was present. The
monkey could turn the lever in the wrong direction indicating that the C target was present but incorrectly
identifying its direction or the monkey could release the lever indicating that the target was not present.
On no target trials, an error was a lever turn indicating the presence of a target. There are two main
results when comparing reaction times from correct and error trials. First, reaction times from error trials
tended to be slightly longer than from correct trials suggesting that errors were not due to a speedaccuracy tradeoff. Second, reaction times from correct and error lever release trials, in which the
monkeys reported the absence of a target, were approximately double that of lever turn trials in which the
monkeys reported the presence of a target. This doubling of reaction times in trials that the target was not
found is indicative of an inefficient, serial, self-terminating visual search (Treisman, 1988; Chun and
Wolfe, 1996).
Neuron activity
The focus of this study is the relationship between spatially selective activity in FEF and visual
perception as indexed by the monkeys’ behavioral report of target identity. Therefore, we included only
those neurons that exhibited significant spatially selective activity for the location of the C target before
the lever turn in correct valid cue trials. We recorded enough trials from a total of 49 FEF neurons that
met this criterion (19 from monkey C and 30 from monkey B). As in a previous study, all of the neurons
with spatially selective activity during the covert search task were visually responsive (Thompson et al.,
2005a).
Figure 3.2 shows the activity during correct target present trials from a representative FEF
neuron recorded in monkey C performing the cued covert visual search task with three CTOAs. This
neuron exhibited an initial visual response at about 80 ms following the cue array presentation that did
not discriminate the red cue ring from the green rings. After the initial non-selective response, the
neuron’s activity identified the location of the red ring by exhibiting greater activity when a red ring was
in the response field (RF) than when a green ring was in the RF. During valid cue trials (left column),
after the appearance of the search array the neuron maintained spatial selectivity for the C target until the
monkey’s behavioral report. During invalid cue trials (middle column), after the presentation of the
search array, the spatial selectivity switched to identify the location of the C target inside a green ring in
47
the RF before the monkey’s behavioral report. During neutral cue trials (right column) the spatially
selective response emerged after the appearance of the search array and identified the location of the C
target.
The time course and magnitude of spatial selectivity in the cued covert search task were
quantified for each FEF neuron using an ROC analysis (Thompson et al., 1996; Thompson et al., 2005b).
Examples of this analysis are shown in Figure 3.2b for the activity recorded on 235 ms CTOA trials (last
row of Figure 3.2a). The area under the ROC curve quantifies the difference between two distributions
normalized to values between 0 and 1 with 0.5 indicating completely overlapping distributions. Values of
0 and 1 indicate completely non-overlapping distributions. The magnitude of spatial selectivity is
indexed in the distance the ROC area is from 0.5; an ROC value of n is equivalent in magnitude to 1 – n.
The analysis was structured so that ROC values greater than 0.5 indicate spatial selectivity toward the
target location and ROC values less than 0.5 indicate spatial selectivity away from the target location.
For convenience we will refer to the ROC area as the selectivity index (SI).
Before the visual response to the cue array, the SI was close to 0.5 indicating equivalent baseline
activity. Following the visual response to the cue array the activity for the red ring stimulus during valid
and invalid cue trials grew to higher levels than the activity for the green ring stimuli. This emerging
difference in activity is reflected in a change in the SI that begins around 100 ms following the
presentation of the cue array. Due to the way the ROC analysis was structured, the initial change in the
SI is toward 1 on valid cue trials indicating that the spatial selectivity was directed toward the future
target location and toward 0 on invalid cue trials indicating that the spatial selectivity was directed away
from the future target location. In the neutral cue trials all 8 rings were green. The probability of the
target to be identified appearing in anyone of the green rings was 1/8. Therefore, during these trials the SI
remained around 0.5 indicating equivalent activity before the presentation of the search array. After the
presentation of the search array, but before the behavioral report, the SI across all trial conditions became
greater than 0.5 reflecting the selection of the C target location. The switch in spatial selectivity during
invalid cue trials from the cue location to the target location is indexed by the changing SI in time from
values less than 0.5 following the cue array to values greater than 0.5 following the target array. The
pattern of activity exhibited by the example neuron shown in Figure 3.2 was present in the activity of all
48
49 neurons (see Figs 3.4-3.6) included in this study.
Figure 3.2: Activity of a single FEF neuron during correct target present trials.
a, Average activity during valid cue trials (left column), invalid cue trials (middle column) and neutral cue trials (right column)
separated by cue-target onset asynchrony (CTOA) in different rows. The CTOAs of the trials in each row are indicated at the left.
Activity is aligned on the time of the search array presentation and ends at the median reaction time of each trial condition. The time
of cue array presentation is marked by a black triangle below each plot. The task illustrations at the top of the figure provide a key
to the color and line types of the neuronal activity traces. The color corresponds to the color of the cue array stimulus in the
response field (RF) (red: red ring, green: green ring). The line type differentiates activity from trials in which the target C (solid
line) or a distractor stimulus (dotted line) appeared in the RF. b, Analysis of the change in spatial selectivity in time for activity
from trials with 235 ms CTOAs (bottom row in a). The selectivity index (SI) is the area under the receiver operating characteristic
(ROC) curve computed at each millisecond from the activity occurring before the lever turn on individual trials. SI values greater
than 0.5 indicate selectivity toward the target location and SI values less than 0.5 indicate selectivity away from the target location.
A recent study has shown that some FEF neurons can exhibit shape selective responses (Peng et
al., 2008). Therefore we examined whether there was any evidence of a preference for a leftward or
rightward oriented C targets. For each neuron we compared the firing rates from valid cue trials with
leftward C target in the receptive field to those with rightward C targets in the receptive field. The
average firing rate measured between 100ms after search array onset to median reaction time did not
49
differ significantly for 94% (46/49) of neurons (t-test, p > 0.05). Although 3 neurons exhibited a slight
significant difference in firing rate (p < 0.05), the difference never exceeded 10%. The key result was
that all 49 neurons exhibited a strong spatially selective response for both leftward and rightward oriented
C targets. Therefore, we conclude that the neurons recorded in this study did not exhibit object-related
selectivity for the C target, but instead represented the location of the target to be identified.
Relationship of cue-related spatial selectivity to behavioral measures of spatial attention
The relationship between neuronal activity and perceptual performance averaged across
recording sessions is shown in Figure 3.3. For each neuron we quantified the cue-related selectivity
during valid, invalid, and neutral cue trials as the average SI measured from 50 ms before to 50 ms after
the appearance of the search array. This period of time captures the activation related specifically to the
cue array during the 100 ms just prior to the beginning of activity related to the search array. Only trials
with CTOAs greater than 100 ms were included in this analysis because shorter CTOAs do not allow for
the opportunity for cue related responses to develop before the search arrays appears (see Fig. 3.2a, top
row). Figure 3.3a plots, for the two monkeys separately, average performance accuracy across recording
sessions as a function of average SI for each trial condition from all neurons. Error trials were included
in the SI calculations because performance accuracy is a measure that includes correct and incorrect
behavioral reports. Figure 3.3b plots reaction time from correct trials averaged across recording sessions
as a function of average SI calculated from correct trials. Performance accuracy was best and reaction
time was fastest when the cue related spatial selectivity was directed toward the future target location
during valid cue trials. The worst performance accuracy and slowest reaction times were from trials
when the cue-related selectivity was directed away from the future target location during invalid cue
trials. During neutral cue trials there was not a spatially selective response for the cue array and the
behavioral measures were intermediate. There was a weak spatial bias in correct neutral cue trials (Fig.
3.3b) that will be addressed below (see Fig. 3.6). A Pearson correlation analysis using the ROC areas and
behavioral measures obtained from individual recording sessions revealed significant positive
correlations for accuracy (monkey C: r = 0.30, p = 0.02; monkey B: r = 0.50, p < 0.001) and significant
negative correlations for reaction time (monkey C: r = -0.32, p = 0.02; monkey B: r = -0.51, p < 0.001).
50
Figure 3.3: Relationship between cue-related spatial selectivity and behavioral measures of spatial
attention for monkey C (open symbols) and for monkey B (solid symbols).
For each neuron the cue-related SI was calculated for each trial condition as the average SI between 50ms before to 50ms after the
appearance of the search array. Only trials in which the CTOA was > 100ms were included. a, Each point plots the average percent
correct performance across recording sessions as a function of average SI from each trial condition (circles: valid cue trials,
triangles: neutral cue trials, squares: invalid cue trials. The error bars are SEM across recording sessions for percent correct
performance and SEM across neurons for SI. Correct and error trials were included in the analysis. The correlation across individual
neurons and recording sessions is significant for both monkeys (monkey C: r = 0.30, p = 0.02; monkey B: r = 0.50, p < 0.001). b,
Each point plots the average of median reaction times in correct trials across recording sessions as a function of average SI in
correct trials across trial conditions. Conventions are the same as in a. The correlation across individual neurons and recording
sessions is significant for both monkeys (monkey C: r = -0.32, p = 0.02; monkey B: r = -0.51, p < 0.001).
These results are consistent with the hypothesis that FEF neurons track the monkey’s allocation
of spatial attention in the task. The activity first identifies the location of the attentional cue, and this
activity is generally predictive of overall behavioral tendencies across trial conditions. In addition,
following the appearance of the target array, the spatially selective activation identifies the location of the
target object to be identified. Notably, during invalid cue trials the spatially selective response shifts
51
away from the popout cue stimulus and towards the target before the monkey identifies the target with a
lever turn. These observations, however, do not demonstrate a direct relationship between spatially
selective activity in FEF and an enhancement of perceptual processing on a trial-by-trial basis. In the
following analyses we directly examine the relationship between FEF activity and the monkeys’ ability to
detect and identify the target of the search array as indexed by their behavioral report.
Accuracy of target identification
Only trials with CTOAs greater than 100 ms were included in this analysis because shorter
CTOAs do not allow for the analysis of cue related responses. Because the monkeys’ behavior across the
remaining longer CTOAs were not different they are combined. Also, the neuronal activity results from the
two monkeys were not different, so they are combined.
Valid cue trials. Figure 3.4 shows an analysis of the population activity during valid cue trials. The C target
appeared inside the red ring. In Figures 3.4a-c the population average activity during trials in which the red
cue ring and the C target were presented in the RF is compared to the population average activity during
trials in which a green ring and distractor were presented in the RF. Shown in Figure 3.4 are plots of the
average activity in correct trials (Fig. 3.4a), error trials in which the monkeys made an incorrect lever turn
(Fig. 3.4b), and error trials in which the monkeys released the lever (Fig. 3.4c) from the 17 neurons that had
enough trials for the analysis.
52
Figure 3.4: Population analysis of activity during correct and error valid cue trials.
Color and line conventions are the same as in Figure 3.2. a-c, Pooled average activity in (a) correct trials, (b) lever turn error trials,
and (c) lever release error trials for the 17 neurons with enough trials for the analysis of all three trial conditions. Activity is aligned
on the time of the search array appearance and ends at the average of median reaction times across the included sessions. The range
of median reaction times across recording sessions from correct valid cue trials is indicated by the bracket above the activity plots.
d, Pooled average SI from correct (solid line) and error trials (dotted line). All error trials were combined to obtain a greater
number of neurons with enough trials to analyze (N = 29). SEM at each millisecond is indicated by the width of the magenta
(correct trials) and cyan (error trials) shading. The horizontal black bar above the time axis marks periods of significantly greater
spatial selectivity in correct trials than in error trials (paired t-test, p < 0.05). e, Average activity between 150 ms after search array
presentation and median reaction time in correct trials from the 17 neurons shown in a-c. The approximate time range of the
activity analysis is shown by the gray bars in a-c. Median reaction time varies across recording sessions; the vertical line within the
RT bracket represents the median reaction time on correct trials across recording sessions. Error bars in e indicate SEM across
neurons. Solid red bars show the average activity during trials in which the red cue ring and target C were in the neurons’ RF.
Dotted green bars show the activity during trials in which the green ring and distractors were in the neurons’ RF. The average
activity on correct trials, lever turn error trials (turn) and lever release error trials (release) are shown separately. The magnitude of
target related activity varied significantly across the three trial types (Friedman test for related samples, p = 0.014). Distractorrelated activity did not vary significantly across the three trial types (Friedman test, p = 0.33).
Before the appearance of the search array there is greater activity for the red cue ring than for the
green rings in both correct and error trials. After search array presentation, the spatially selective response
continues in both correct and error trials. During the error trials in which the monkeys released the lever,
thereby reporting that no target was present, the spatially selective response eventually disappears before
53
the behavioral report (Fig. 3.4c).
Reaction times were faster in correct trials than in error trials (Fig. 3.1c). Reaction times from
correct trials, therefore, provide an endpoint time to look for difference in FEF activity that is predictive of
the monkeys’ accuracy of reporting target identity. In other words, to influence target identification,
modulations of activity must occur before the monkeys typically would make a correct report of target
identity. For each neuron, the time course of the SI was calculated from correct and error trials up to the
median reaction time from correct trials. To include the maximum number of neurons in the analysis, the
two types of error trials were combined. Two main points are evident in Figure 3.4d, which compares the
population average SI from correct and error valid cue trials. First, the spatially selective responses for the
red cue in correct and error trials occurring before the appearance of the search array do not differ. Second,
the spatially selective responses in correct and error trials become significantly different between 143 ms
and 271 ms after the search array appears.
The two types of error trials are qualitatively different. Lever turn errors indicate that the monkey
detected a target C but incorrectly identified its direction and lever release errors indicate that the monkey
did not detect the presence of a target C in the search array. We examined whether there are differences of
activity for the two types of errors. Figure 3.4e compares the magnitude of neuronal activity in correct and
error valid cue trials before the time the monkeys’ made a correct behavioral report. For each neuron we
calculated the average activity for each trial type from 150 ms after the presentation of the search array to
the median reaction time from correct trials. The neurons exhibited significantly greater activity for the
target C in the RF than for distractors in the RF during correct and error trials (paired t-test, p < 0.001).
However, the magnitude of target related activity varied significantly across the three trial types (Friedman
test for related samples, p = 0.014) and was ranked according to the correctness of the behavioral report.
The highest activity was during correct trials, intermediate activity was during the partially correct trials in
which the monkeys correctly reported that the C was present but incorrectly reported its direction, and the
lowest activity was during error trials in which the monkey reported that the target was absent. Distractorrelated activity did not vary significantly across the three trial types (Friedman test, p = 0.33).
54
Invalid cue trials
Figure 3.5: Population analysis of activity during correct and error invalid cue trials.
Figure conventions are the same as in Figure 3.4. Color and line conventions are the same as in Figure 3.2. a-c, Pooled average
activity in (a) correct trials, (b) lever turn error trials, and (c) lever release error trials for the 22 neurons with enough trials for the
analysis of all three trial conditions. d, Pooled average SI from correct (solid line) and error trials (dotted line). The horizontal
black bar above the time axis marks periods of significantly greater spatial selectivity in correct trials than in error trials (paired ttest, p < 0.05). e, Average activity between 150 ms after search array presentation and median reaction time in correct trials from the
22 neurons shown in a-c. Error bars in indicate SEM across neurons. The magnitude of target related activity (solid green bars)
varied significantly across the three trial types (Friedman test for related samples, p < 0.001). Distractor related activity (dotted red
bars) did not vary significantly across the three trial types (Friedman test, p = 0.14).
Figure 3.5 shows the analysis of the population activity during invalid cue trials. The target
appeared in one of the three green rings opposite the red ring. Activity before the appearance of the search
array was selective for the location of the red cue ring in correct and in error trials (Figs. 3.5a-c). The
magnitude of the cue-related selectivity is the same as in valid cue trials because before the appearance of
the search array the monkeys have the same expectation of target location. Also, the cue-related spatially
selective activity in correct trials was not different from that in error trials. After search array presentation,
the spatially selective responses in correct and error trials become markedly different. In correct trials,
55
there is a clear switch of spatial selectivity from the location of the red ring to the location of the C target.
This switch in spatial selectivity during correct trials is quantified Figure 3.5d. The SI shifts from values
less than 0.5 before and immediately after search array onset to values greater than 0.5 after about 150 ms
following search array onset. In error trials, the shift of spatial selectivity away from the red cue was
sluggish and by the time that the monkeys made a lever turn in correct trials, the activity in error trials only
weakly identified the location of the C target. The SI from correct trials became significantly greater than
the SI from error trials at 136 ms following search array presentation and continued past the median
reaction time of correct trials.
Figure 3.5e compares the magnitude of neuronal activity during correct trials, lever turn error trials
and lever release error trials in the time period between 150 ms following the presentation of the search
array to the median RT of correct trials. During this time the neurons exhibited greater activity for the C
target in the RF than for distractors in the RF in correct trials (paired t-test, p = 0.003), but not in error
trials. As in valid cue trials, the magnitude of target related activity varied significantly across the three
trial types (Friedman test for related samples, p < 0.001) and was ranked according to the correctness of the
behavioral report. Distractor-related activity did not vary significantly across the three trial types
(Friedman test, p = 0.14).
Neutral cue trials
56
Figure 3.6: Population analysis of activity during correct and error neutral cue trials.
Figure conventions are the same as in Figure 3.4. Color and line conventions are the same as in Figure 3.2. a-c, Pooled average
activity in (a) correct trials, (b) lever turn error trials, and (c) lever release error trials for the 32 neurons with enough trials for the
analysis of all three trial conditions. d, Pooled average SI from correct (solid line) and error trials (dotted line). The horizontal
black bar above the time axis marks periods of significantly greater spatial selectivity in correct trials than in error trials (paired ttest, p < 0.05). e, Average activity between 150 ms after search array presentation and median reaction time in correct trials from the
32 neurons shown in a-c. Error bars in indicate SEM across neurons. The magnitude of target related activity (solid green bars)
varied significantly across the three trial types (Friedman test for related samples, p < 0.001). Distractor related activity (dotted
green bars) did not vary significantly across the three trial types (Friedman test, p = 0.7).
Figure 3.6 shows the analysis of the population activity during neutral cue trials in which there
was no spatial cue, all the rings in the cue array were green. There was not a red cue ring in the cue array
and before the appearance of the search array the activity was not spatially selective (Figs. 3.6a-c). Spatial
selectivity emerged after the presentation of the search array and identified the location of the C target.
Like the results from valid and invalid cue trials the SI from correct trials was greater than the SI from error
trials during the time in which the search array was present (Fig. 3.6d). However, unlike the results from
valid and invalid cue trials, the SI before and around the appearance of the search array was slightly yet
significantly (p < 0.05) greater during correct trials than during error trials. This result suggests that in the
absence of an exogenous spatial cue the monkeys anticipate the location of the target and this anticipation
57
is reflected in FEF activity. When he guesses correctly, as indexed by a slightly higher SI, he is more likely
to be correct. It should be noted, however, that this weak anticipatory bias was associated with
performance accuracy but not with reaction time differences across trials (see Fig. 3.8).
Figure 3.6e compares the magnitude of neuronal activity during correct trials, lever turn error trials
and lever release error trials in the time period between 150 ms following the presentation of the search
array to the median RT of correct trials. During this time in correct neutral cue trials the neurons exhibited
greater activity for the C target in the RF than for distractors in the RF (Fig. 3.6a; paired t-test, p < 0.001).
During error trials in which the monkey made an incorrect lever turn the difference was marginally
significant (Fig. 3.6b; p = 0.05) and during error trials in which the monkey released the lever the activity
did not differ significantly (Fig. 3.6c; p = 0.4). As in valid and invalid cue trials, the magnitude of target
related activity varied across the three trial types (Friedman test for related samples, p = 0.001) and was
ranked according to the correctness of the behavioral report. Distractor-related activity did not vary
significantly across the three trial types (Friedman test, p = 0.7).
58
Target absent trials
Figure 3.7: A comparison of population average activity during (a) correct rejection trials and (b)
invalid cue miss trials.
In both types of trials the monkey reported target absence by releasing the lever and reaction times (RT) did not differ across trial
conditions (Wilcoxon rank sum test; p= 0.9). a, Pooled average activity during correct rejection trials. The activity during trials in
which the red ring and a distractor were presented in the RF (red dotted line) is compared to the activity in which a green ring and a
distractor were presented in the RF (green dotted line). Activity is aligned on the time of search array presentation. The brackets
above the plot show the range of median RTs during individual recording sessions. The vertical line within the RT bracket indicates
the median RT on correct rejection trials across recording sessions. b, Pooled average activity during invalid cue miss trials.
Conventions are the same as in Figure 3.5c. c, A comparison of the pooled average SI in correct rejection trials (solid line, magenta
shading) and invalid cue miss trials (dotted line, cyan shading). SI values less than 0.5 indicate selection of the cue stimulus and SI
values greater than 0.5 from invalid cue trials indicate selection of the target stimulus.
No target appeared in approximately 25% of trials, all the search array stimuli were distractors.
The monkeys were rewarded for reporting target absence by releasing the lever. We identify these trials as
correct rejections (Green and Swets, 1966). The average reaction time of correct rejection trials was
approximately double that of correct target present trials (see Fig. 3.1c). There is a spatially selective
59
response for the location of the red cue stimulus during correct rejection trials Figure 3.7a. Following the
presentation of the search array the spatially selective response weakens and eventually disappears before
the monkeys release the lever to report target absence. We hypothesize that the reduction of the spatially
selective response corresponds to the monkeys shifting attention away from the red cue stimulus to find the
C target. To examine the relationship between spatially selective activity in FEF and the monkeys’
perceptual judgment of target presence, we compared the activity from correct rejections trials to that from
invalid cue misses – trials in which the target appeared but was not identified or detected (Fig. 3.7b). The
monkeys’ behavioral report was the same; the monkeys released the lever reporting target absence.
Reaction times of misses were not statistically different from reaction times of correct rejections (Wilcoxon
rank sum test; p= 0.9) (see Fig. 3.1c). The similar reaction times lend confidence to the assumption that
when a monkey incorrectly released the lever on a target present trial he was accurately reporting his
percept that the target was absent.
The activity during invalid cue miss trials is similar to that during correct rejection trials.
Following the appearance of the search array, the spatial selectivity for the cue location disappears and the
SI returns to values near 0.5. However, during miss trials the SI moves toward 0.5 at a faster rate and ends
at slightly higher values than during correct rejection trials. The average SI from correct rejection trials and
miss trials measured between 150 ms following the search array and the median reaction time of correct
rejections differ significantly (p = 0.01). Therefore it appears that the presence of a target stimulus
influences the shift in the spatially selective response away from the red cue stimulus. But the most
important result is that after the appearance of the search array a spatially selective response for the target
location never develops. This result provides neurophysiological evidence that a visual spatial selection
process must occur before an object can be detected and identified.
Speed of target identification
Manual reaction times have classically been used to measure the benefits and costs of allocating
spatial attention (Posner, 1980; Pashler, 1998). The idea behind this practice is that directed attention
speeds perceptual processing at the cued location thus leading to faster reaction times. Therefore, we
reasoned that if FEF activity corresponds to the allocation of attention, then some aspect of the spatially
selective response should be related to the time it takes for the monkey to report the identity of the target
60
stimulus. To test this hypothesis we separated correct trials from each recording session into fast reaction
time groups (fastest 35% of the trials) and a slow reaction time groups (slowest 35%) for each trial
condition and calculated the SI as before to quantify the time course and magnitude of spatial selectivity.
Figure 3.8: The relationship between the SI and the speed of target identification.
a-c, Pooled average SI from correct valid cue trials (a), invalid cue trials (b), and neutral cue trials (c) separated in fast reaction
time (fastest 35% of trials – solid line, magenta shading) and slow reaction time (slowest 35% of trials – dotted line, cyan shading)
trial groups. All plots end at the average of the median RTs across the included sessions. The ranges of median reaction times from
the fast RT groups are shown by the brackets above each plot and the vertical line represents the median RT across recording
sessions. The shading around the average SI traces represents the SEM across neurons. The horizontal black bars above the time
axis indicates periods in which the SI was significantly different between fast and slow RT groups (paired t-test, p < 0.05). d and e,
The relationship between the time of spatial selection and the speed of target identification during invalid cue trials. The selectivity
switch time is defined as the time the SI crosses 0.5 during invalid cue trials – the time that the spatially selective activity switches
from the red cue ring to the C target. d, Each point of the scatter plot compares the selectivity switch time from fast RT and slow
RT trials from a single neuron (N = 24). The average selectivity switch time from monkey C neurons (open symbols) is 127 ± 7 ms
for fast RT trials and 170 ± 18 ms for slow RT trials. The average selectivity switch time from monkey B neurons (solid symbols)
is 186 ± 11 ms for fast RT trials and 227 ± 13 ms for slow RT trials. e, Distribution of the percentage of change in reaction time
accounted for by the change in selectivity switch time for single neurons. The average across all neurons is 21 ± 4 %.
Figures 3.8a, b and c plot the average SI from the fast and slow reaction time groups for correct
61
valid cue trials (Fig. 3.8a), correct invalid cue trials (Fig. 3.8b), and neutral cue trials (Fig. 3.8c). The
critical question is whether there are differences in the SI before the monkey reports the identity of the
target in fast reaction time trials that could account for the difference in reaction times across the two
groups of trials.
The consistent result for all three trial conditions is that the selectivity for the C target location, as
indexed by the SI, is greater in fast reaction time trials than in slow reaction time trials. The greatest
difference in magnitude of spatial selectivity between fast and slow reaction time trials occurs during the
time in which the visual system is processing the target of the search array. During valid cue (Fig. 3.8a) and
neutral cue (Fig. 3.8c) trials there were a few brief significant differences in the cue related activity prior to
the appearance of the search array. But these differences, although significant statistically, were not
consistently greater for fast reaction time trials and, therefore, cannot account for the differences in reaction
times across trials. Across all three trial conditions there was a greater spatially selective response in fast
reaction time trials beginning around 150 ms and ending around 250 ms following the presentation of the
search array (reaction times of fast trials averaged around 300 ms). This result suggests that the magnitude
of the spatial selection in FEF influences the speed of the visual processing leading to target identification.
Some of the difference in reaction time across trials could be due to differences in the speed of the
visual selection process rather than the magnitude of selection. We probed this relationship by examining
the time of the switch of spatial selectivity during invalid cue trials from the location of red cue ring to the
location of the C target. For each neuron, we first smoothed the SI values in time with a 100 ms boxcar
filter. Then we identified the time that the smoothed ROC area crossed 0.5 after the presentation of the
target search array on invalid cue trials. We termed this time the selectivity switch time and it is indicated
Figure 3.8b. The scatter plot in Figure 3.8d compares the selectivity switch times in fast reaction time trials
and slow reaction time trials from all neurons with enough correct invalid cue trials for the analysis (N=24).
For monkey C, the average ± SEM selectivity switch time was 127 ± 7 ms in fast trials and 170 ± 18 ms in
slow trials. For monkey B, the average selectivity switch time was 186 ± 11 ms in fast trials and 227 ± 13
ms in slow trials. For both monkeys the selectivity switch time in fast and slow trials differed significantly
(paired t-test; p < 0.001).
If the variability in reaction time is simply related to when the target is selected, then the increase
62
in target switch time should be the same as the increase in reaction time. To test this, for each neuron we
calculated the percentage change in reaction time accounted for by the change in selectivity switch time.
On average, the percentage of the increase of reaction time accounted for by the increase of selectivity
switch time was 21 ± 4 %. Therefore, about 20% of the variability in reaction time to accurately report the
identity of the target during invalid cue trials can be attributed to the time of visual selection as measured in
FEF. Other contributing factors include the difference in the magnitude of spatial selection as well as
variability in motor processes that occur after target identification.
3.4 Discussion
The fundamental goal of this study was to directly link neuronal activity to subjective perception
as indexed by a behavioral report. This was done by determining whether the spatially selective responses
of FEF neurons are related to behavioral measures of spatial attention in a cued object identification task.
We used a color popout stimulus as an exogenous spatial cue to inform the monkey of the most probable
location of a target to be identified. If the target was not at the cue location (invalid cue trials), or if there
was no spatial cue (neutral cue trials), the monkeys had to covertly search for the target by endogenously
(i.e., willfully) shifting attention to non-cued locations. The doubling of reaction time in no target trials to
report target absence indicates that the search for the target was inefficient and effortful (Treisman, 1988;
Chun and Wolfe, 1996). The results of this study show that the magnitude of the spatially selective
response in FEF occurring specifically during the time that the target stimulus is being processed by the
visual system predicts the accuracy and speed of target identification.
In a previous report we showed that the activity of FEF visually responsive neurons identifies the
location of the visual search target in monkeys performing easy popout visual search tasks in the absence of
eye movements (Thompson et al., 2005a). We hypothesized that the spatially selective activity
corresponded to the allocation of covert spatial attention. But this was an inferred relationship based on
psychophysical reports showing that the oddball targets of popout search arrays automatically capture
attention (Egeth and Yantis, 1997; Turatto and Galfano, 2000). In the current study we confirm that there
is a relationship between the spatially selective activity in FEF for a color popout singleton stimulus and
behavioral measures of attention obtained simultaneously in the same monkeys. Overall, the highest
63
proportion of correct trials and the fastest reaction times were observed in valid cue trials in which the
spatially selective responses to the singleton target of the popout cue array matched the location of the
target to be identified (see Fig. 3.3). This pattern of results is consistent with behavioral studies of spatial
attention showing improvements in perceptual performance at the attended location at a cost to perceptual
performance at non-attended locations (Posner, 1980; Pashler, 1998). This correlation, however, only
demonstrates an overall relationship between neuronal activity and perceptual performance and it is entirely
predictable based on the results of the previously cited psychophysical and neurophysiological studies. In
other words, the same relationship would be observed if the behavioral data and neurophysiological data
were obtained at different times or from different subjects.
To directly examine the link between FEF and the efficacy of visual processing, in the current
study we correlated the activity of FEF neurons with the perceptual performance of the monkeys on a trialby trial basis. We asked whether and when in the course of a covert visual search task do differences in
neural activity correspond to differences in perception as indexed by a behavioral report. The results
demonstrate a clear relationship between the strength of the spatially selective signal in FEF and the
accuracy and speed of reporting the identity of a target in a cued covert visual search task in the absence of
eye movements. The correlation between neural activity and the behavioral measures of perceptual
performance occurred predominantly between 100 ms following the presentation of the target to be
identified and the initiation of the behavioral report. This is the period of time in which the processing
related to target detection and identification is taking place in the visual system.
The task used in this study requires the monkey to detect, recognize and identify a visual object.
As in a previous report (Thompson et al., 2005a), we found no evidence that FEF neurons could identify
whether the C target was leftward or rightward. In the primate visual system, there is strong evidence the
representation and analysis of visual objects takes place primarily in the temporal lobes along the ventral
visual processing stream (Logothetis and Sheinberg, 1996; Tanaka, 1997). Studies have shown that spatial
attention is necessary for object recognition in natural scenes (Rensink, 2002), and for object selective
responses of neurons in inferior temporal cortex during visual search which begin about 150 ms after
search array appearance (Chelazzi et al., 1998). Sheinberg and Logothetis (2001) showed that during
active scanning of complex visual scenes with saccadic eye movements, object selective neurons in inferior
64
temporal cortex become active specifically before the saccades used to fixate the target object of the search.
These object selective responses in IT cortex that occur specifically before saccades that bring the object to
the fovea are likely related to the shift of spatial attention that precedes saccades (Hoffman and
Subramaniam, 1995; Kowler et al., 1995; Deubel and Schneider, 1996).
There is growing support for the premotor theory of attention (Rizzolatti et al., 1987) that
hypothesizes that presaccadic signals from areas like the frontal eye field are a source of the spatial
attention signals that enhance visual processing (Awh et al., 2006). Hamker (2005), specifically proposed
a model in which spatially selective signals from FEF play an important role in object recognition. In a
previous study (Thompson et al., 2005a) and in this study we show that visually responsive FEF neurons
exhibit spatially selective responses in the absence of eye movements and, therefore, are a potential source
covert spatial attention. In an elegant series of experiments, Moore and colleagues have shown that weak
electrical microstimulation of FEF using currents below the threshold to evoke saccades increases the
monkeys’ perceptual sensitivity at the location in visual space represented by the neurons at the stimulated
site (Moore and Fallah, 2001, , 2004; Schafer and Moore, 2007), and modulates visual responses in
extrastriate visual cortex in a manner similar to the effects of directed visual attention (Moore and
Armstrong, 2003; Armstrong et al., 2006; Armstrong and Moore, 2007). A particularly relevant result from
the study by Moore and Armstrong (2003) is that FEF stimulation modulates activity in area V4 only when
the neuron in the visual cortex is being driven by a stimulus in its receptive field. A recent fMRI study in
monkeys showed that FEF stimulation modulates activity throughout the visual cortex (Ekstrom et al.,
2008). In the current study we demonstrate that spatially selective activity in FEF is correlated with the
accuracy and reaction time of object recognition during covert visual search specifically during the period
of time in which neurons in the visual cortex are processing the object to be identified. Together these
results provide strong converging evidence that FEF activity plays an important role in ongoing visual
processing by providing a spatial signal that selects and enhances the representation of behaviorally
relevant objects.
Human psychophysical experiments have manipulated perceptual task difficulty to demonstrate
that visual processing efficiency is directly related to the amount of attentional effort (Lavie and Tsal, 1994;
Urbach and Spitzer, 1995). Single-unit recording studies have shown that representations of visual stimuli
65
in the visual cortex of monkeys are stronger with increased task demands (Spitzer et al., 1988; Spitzer and
Richmond, 1991; Boudreau et al., 2006). Boudreau et al. (2006) specifically showed that neuronal
modulations related to task difficulty in area V4 are correlated with the magnitude of spatial attention as
measured with changes in the ability of the monkeys to detect small changes in a visual stimulus.
Therefore, a neuronal source of spatial attention should exhibit activity that is related to different degrees of
attention as measured by the ability to process visual stimuli. Additionally, the results of recent
psychophysical experiments in humans performing covert visual search tasks suggest that spatial
attentional selection is a necessary prerequisite for object identification (Evans and Treisman, 2005; Zenon
et al., 2008). Therefore, the level of neural activity in an area that directs spatial attention should predict
whether or not an object is identified. Our results show that FEF activity exhibits these properties. The
magnitude of target related activity in FEF is correlated with the efficiency of processing the target
stimulus. For all 3 trials conditions – valid, invalid, and neutral cue trials – target location related activity
in FEF was correlated with the accuracy of the behavioral report. The highest level of activity was
associated with correct object identification. Activity below this level can support partial processing
leading to target detection, but not identification. And when the spatially selective activity in FEF does not
reach a minimum threshold level, the monkey does not detect the target.
In summary, there are two main results of this study. First, FEF exhibits spatially selective
activity for an exogenous spatial cue in the absence of eye movements that is correlated with the overall
behavioral benefits and costs of directed covert spatial attention across trial conditions that vary in cue
validity. This result is consistent with the view that FEF is a part of a distributed spatial attention network
that includes LIP and the superior colliculus, brain structures that exhibit similar patterns of spatially
selective responses for the target in popout visual search arrays (McPeek and Keller, 2002; Constantinidis
and Steinmetz, 2005; Thomas and Pare, 2007). But trial-to-trial variability in cue-related spatial selection
before the appearance of the target is not correlated with the efficacy of visual processing within trial
conditions. The second main result is that trial-to trial variability in the accuracy and reaction time of
visual object recognition is correlated with the strength of the spatially selective signal after the appearance
of the search array, primarily during the time in which the processing related to target detection and
identification is taking place along the ventral visual processing stream. This result indicates that FEF
66
activity exerts a top-down modulatory influence on ongoing visual processing and is consistent with the
hypothesis that it is a source of covert spatial attention needed for object perception (Hamker, 2005).
Chapter 4
Summary and Conclusions
The world is ever-changing and infinitely complex. Our brain’s sensory systems are designed to rapidly
detect relevant cues, which then are used to guide our behavior. In a covert visual search task, I showed that
neuronal activity in the frontal eye fields selects the locations of salient targets that need to be identified to
guide the subject’s behavior to obtain reward. The magnitude of spatial selection in the frontal eye fields
for the location of the target to be identified predicts accuracy and the speed of identification of that target.
The explicit spatial signal observed in the activity of FEF neurons is likely computed from spatially nonselective inputs at the level of FEF and is one of the sources of top-down spatial attention.
4.1 Overview
The main goal of the thesis was to establish a link between spatial selection in FEF and spatial
attention, and to explore the origin of spatial selection in the brain during covert visual search. In Chapter 2,
we compared the timing of spatial selection in LFPs and spiking activity while the monkeys covertly
searched for the task relevant target. We found that the timing of spatial selection in spiking activity
preceded the timing of spatial selection in LFPs by approximately 30 milliseconds; we argued that this
suggests that FEF is one of the sources of spatial selection in the brain. Though until this thesis, the
relationship between LFPs and spiking responses has not been studied in FEF, it has been studied in other
brain areas. For example, in inferotemporal (IT) cortex the time course of object selectivity in spiking
activity and LFPs has been shown to be nearly simultaneous (Krieman et al., 2006). A study by Nielsen et
al. (2006) showed that spikes and LFPs in IT exhibited learned object selectivity, and that the modulation
of LFP responses, but not spiking activity, grew stronger from posterior to anterior IT. Because LFPs likely
contain inputs from other brain areas, they concluded that the learned object selectivity was encoded first in
posterior IT and then transmitted to anterior IT. In the experiments outlined in Chapter 2, we showed that
67
68
non-selective stimulus evoked responses in FEF are first detectable in LFPs and then in the spiking activity.
We then showed that at the same recording sites, during the same recording sessions, spatial selectivity
emerges first in the spiking activity and 30ms later in the LFPs. This approach allowed us to verify that 1)
input from the visual cortex is strongly represented in LFPs and 2) to show that FEF is likely one of the
cortical sources for spatial selectivity during covert visual search.
The experiments in Chapter 3 show that spatial selectivity in FEF is related to behavioral measures of
attention in a cued object identification task. Behaviorally, spatial attention is measured as a speeding of
reaction time and an improvement in performance accuracy in visual processing tasks at the attended
location (Pashler 1998). We showed that these behavioral measures are correlated to the variability in the
magnitude of spatial selection in FEF. First, we reconfirmed the results of previous studies demonstrating
that spatially selective activity corresponds to the location of spatial attention (e.g. Thompson et al., 2005,
Zhou and Thompson, 2008). However, previous studies have only inferred a general relationship between
attention and spatial selection. Instead of inferring the relationship between measures of attention and the
overall levels of neuronal activity across different task conditions, we correlated neuronal activity to
measures of attention on a trial by trial basis. We tested whether and when differences in neural activity
correspond to differences in perception as indexed by the subjects’ behavioral report. We showed that the
correlation between neural activity and the behavioral measures of perceptual performance occurred
predominantly between 100 ms following the presentation of the target to be identified and the initiation of
the behavioral report. The magnitude of cue related selectivity did not differ on correct and error trials
within each trial condition. The findings presented in Chapter 3 therefore suggest that FEF spatial
selectivity is helpful to visual processing in the period of time in which the processing related to target
detection and identification is taking place in the visual system. Additionally, when there was no target
related selection the monkeys reported that the target was absent. A larger magnitude of selection resulted
in a report of target’s presence, though even a greater magnitude of selection was correlated with a correct
identification of a target. These results suggest that there is a threshold of spatial selection that is required
for object identification. The results presented in this thesis demonstrate that FEF is one of the cortical
sources of spatial attention.
69
4.2 Future Directions
One of the biggest mysteries in cognitive science is how the brain locates and identifies objects in
a complex and ever-changing world. One alternative is that it relies on a low-level feature analysis to guide
attention to the most probable location of a target and then commits resources to the identification of the
objects at that location. Another alternative is that the brain gathers information about the possible identity
of the objects in the visual scene and then guides attention to the location of the most behaviorally relevant
object. The results of Chapter 2 and 3 suggest that FEF is one of the sources of spatial attention and that its
activity plays a direct role in visual processing. While the results of this thesis add to our general
understanding of cortical processing and the role of FEF activity in the CNS, we did not directly record the
activity of neurons in a brain area that is involved in processing the features of the target (e.g. V2, MT, V4,
IT, etc) to directly ascertain the effect of FEF activity on visual processing. To (1) directly assess the
relationship between FEF activity and visual processing and (2) to test if spatial selection precedes object
identification, we are currently simultaneously recording single neurons in FEF and inferior temporal
cortex (IT), an area crucial for object identification. Thus far, our findings are consistent with the results of
Chapter 3 and models of attention (Hamker, 2005) that propose that spatial selection is required to bind
features of an object to create a representation of the object's identity (recognition).
Many areas in the brain exhibit spatially selective properties (e.g. lateral intraparietal area (LIP),
7a, superior colliculus (SC), and dorsolateral prefrontal cortex (dlPFC)). What are the different
contributions these brain regions make to our ability to attend? Schaffer and Moore (2007) electrically
stimulated FEF in an experiment that tested the contribution of FEF activity to eye movement preparation
and to the enhancement of saliency of visual stimuli. They showed that artificial FEF activation influences
perception of a stimulus and not the eye movement plan to the location of that same stimulus. A study by
Buschman and Miller (2007) suggests that FEF is more involved in top-down (or cognitively driven) spatial
attention and LIP is more involved in bottom-up (or stimulus driven) attention. In line with the hypothesis
that FEF is a source of top-down spatial selection, Hanes and Wurtz (2001) showed that information flows
from FEF to SC and from SC to the pre-motor neurons in the brainstem (Hanes and Wurtz, 2001) and that
pre-saccadic spatial selection in FEF does not originate from the SC (Berman, et al., 2009). Simultaneous
recording of spikes and LFPs in LIP, SC, and FEF would help to quantify the time course of spatial
70
selection across multiple brain regions and therefore shed light on the complex multi-area interactions that
take place during spatial selection of targets for eye movements and visual processing. One possibility is
that in a top-down attention task, spatial selection in LIP (or SC) would first be detectable in LFPs rather
than spikes, while in FEF spatial selection would be first detectable in spikes and later in the LFPs, as
reported in Chapter 2 of this thesis. Such a finding would strengthen the hypothesis that FEF is one of the
sources of spatial selection in the brain. But regardless of the outcome, the proposed experiment would add
to our understanding of the neural mechanisms of spatial selection.
In general, a combination of methods such as direct manipulation (e.g. electrical and optical
stimulation, inactivation), optical imaging, neural recording, and fMRI will be required to unravel the
complex circuitry that is responsible for the primate’s ability to attend to stimuli and select targets for
visual processing.
Bibliography
Allport A (1987) Selection for action: Some behavioral and neurophysiological considerations of attention
and action. In: Perspectives on Perception and Action (Heuer H, Sanders AF, eds), pp 395-419.
Hillsdale, NJ: Lawrence Erlbaum Associates.
Armstrong KM, Moore T (2007) Rapid enhancement of visual cortical response discriminability by
microstimulation of the frontal eye field. Proc Natl Acad Sci U S A 104:9499-9504.
Armstrong KM, Fitzgerald JK, Moore T (2006) Changes in visual receptive fields with microstimulation of
frontal cortex. Neuron 50:791-798.
Awh E, Armstrong KM, Moore T (2006) Visual and oculomotor selection: links, causes and implications
for spatial attention. Trends Cogn Sci 10:124-130.
Beauchamp MS, Petit L, Ellmore TM, Ingeholm J, Haxby JV (2001) A parametric fMRI study of overt and
covert shifts of visuospatial attention. Neuroimage 14:310-321.
Bichot NP, Rossi AF, Desimone R (2005) Parallel and serial neural mechanisms for visual search in
macaque area V4. Science 308:529-534.
Bisley JW, Krishna BS, Goldberg ME (2004) A rapid and precise on-response in posterior parietal cortex.
Journal of Neuroscience 24:1833-1838.
Boudreau CE, Williford TH, Maunsell JH (2006) Effects of task difficulty and target likelihood in area V4
of macaque monkeys. J Neurophysiol 96:2377-2387.
Bressler SL, Tang W, Sylvester CM, Shulman GL, Corbetta M (2008) Top-down control of human visual
cortex by frontal and parietal cortex in anticipatory visual spatial attention. J Neurosci 28:1005610061.
Bruce CJ, Goldberg ME (1985) Primate frontal eye fields: I. Single neurons discharging before saccades. J
Neurophysiol 53:603-635.
Buschman TJ, Miller EK (2007) Top-down versus bottom-up control of attention in the prefrontal and
posterior parietal cortices. Science 315:1860-1862.
Chelazzi L, Duncan J, Miller EK, Desimone R (1998) Responses of neurons in inferior temporal cortex
during memory-guided visual search. J Neurophysiol 80:2918-2940.
Chen CM, Lakatos P, Shah AS, Mehta AD, Givre SJ, Javitt DC, Schroeder CE (2007) Functional Anatomy
and Interaction of Fast and Slow Visual Pathways in Macaque Monkeys. Cereb Cortex 17:15611569.
Chun MM, Wolfe JM (1996) Just say no: how are visual searches terminated when there is no target
present? Cognit Psychol 30:39-78.
Constantinidis C, Steinmetz MA (2005) Posterior parietal cortex automatically encodes the location of
salient stimuli. J Neurosci 25:233-238.
Corbetta M, Shulman GL (2002) Control of goal-directed and stimulus-driven attention in the brain. Nat
Rev Neurosci 3:201-215.
Cruikshank SJ, Rose HJ, Metherate R (2002) Auditory thalamocortical synaptic transmission in vitro. J
Neurophysiol 87:361-384.
Deubel H, Schneider WX (1996) Saccade target selection and object recognition: evidence for a common
attentional mechanism. Vision Res 36:1827-1837.
Egeth HE, Yantis S (1997) Visual attention: control, representation, and time course. Annu Rev Psychol
48:269-297.
Ekstrom LB, Roelfsema PR, Arsenault JT, Bonmassar G, Vanduffel W (2008) Bottom-up dependent gating
of frontal signals in early visual cortex. Science 321:414-417.
Evans KK, Treisman A (2005) Perception of objects in natural scenes: is it really attention free? J Exp
Psychol Hum Percept Perform 31:1476-1492.
Fries P, Reynolds JH, Rorie AE, Desimone R (2001) Modulation of oscillatory neuronal synchronization
by selective visual attention. Science 291:1560-1563.
Goldberg ME, Segraves MA (1989) The visual and frontal cortices. In: The Neurobiology of Saccadic Eye
Movements, Reviews of Oculomotor Research, Vol. III (Wurtz RH, Goldberg ME, eds), pp 283313. Amsterdam: Elsevier.
71
72
Green DM, Swets JA (1966) Signal Detection Theory and Psychophysics. New York: Wiley.
Grosbras MH, Paus T (2002) Transcranial magnetic stimulation of the human frontal eye field: Effects on
visual perception and attention. Journal of Cognitive Neuroscience 14:1109-1120.
Hamker FH (2005) The reentry hypothesis: the putative interaction of the frontal eye field, ventrolateral
prefrontal cortex, and areas V4, IT for attention and eye movement. Cereb Cortex 15:431-447.
Hanes DP, Patterson WF, Schall JD (1998) Role of frontal eye fields in countermanding saccades: Visual,
movement, and fixation activity. Journal of Neurophysiology 79:817-834.
Hays AV, Richmond BJ, Optican LM (1982) A UNIX-based multiple process system for real-time data
acquisition and control. WESCON Conference Proceedings 2:1-10.
Hillyard SA, Anllo-Vento L (1998) Event-related brain potentials in the study of visual selective attention.
Proc Natl Acad Sci U S A 95:781-787.
Hoffman JE, Subramaniam B (1995) The role of visual attention in saccadic eye movements. Percept
Psychophys 57:787-795.
Ipata AE, Gee AL, Goldberg ME, Bisley JW (2006) Activity in the lateral intraparietal area predicts the
goal and latency of saccades in a free-viewing visual search task. J Neurosci 26:3656-3661.
Jouve B, Rosenstiehl P, Imbert M (1998) A mathematical approach to the connectivity between the cortical
visual areas of the macaque monkey. Cereb Cortex 8:28-39.
Juergens E, Guettler A, Eckhorn R (1999) Visual stimulation elicits locked and induced gamma oscillations
in monkey intracortical- and EEG-potentials, but not in human EEG. Exp Brain Res 129:247-259.
Kaur S, Lazar R, Metherate R (2004) Intracortical pathways determine breadth of subthreshold frequency
receptive fields in primary auditory cortex. J Neurophysiol 91:2551-2567.
Kelley TA, Serences JT, Giesbrecht B, Yantis S (2008) Cortical mechanisms for shifting and holding
visuospatial attention. Cereb Cortex 18:114-125.
Kincade JM, Abrams RA, Astafiev SV, Shulman GL, Corbetta M (2005) An event-related functional
magnetic resonance imaging study of voluntary and stimulus-driven orienting of attention. J
Neurosci 25:4593-4604.
Kodaka Y, Mikami A, Kubota K (1997) Neuronal activity in the frontal eye field of the monkey is
modulated while attention is focused on to a stimulus in the peripheral visual field, irrespective of
eye movement. Neurosci Res 28:291-298.
Kowler E, Anderson E, Dosher B, Blaser E (1995) The role of attention in the programming of saccades.
Vision Res 35:1897-1916.
Kreiman G, Hung CP, Kraskov A, Quiroga RQ, Poggio T, DiCarlo JJ (2006) Object selectivity of local
field potentials and spikes in the macaque inferior temporal cortex. Neuron 49:433-445.
Lavie N, Tsal Y (1994) Perceptual load as a major determinant of the locus of selection in visual attention.
Percept Psychophys 56:183-197.
Liu J, Newsome WT (2006) Local field potential in cortical area MT: stimulus tuning and behavioral
correlations. J Neurosci 26:7779-7790.
Logothetis NK, Sheinberg DL (1996) Visual object recognition. Annu Rev Neurosci 19:577-621.
Logothetis NK, Wandell BA (2004) Interpreting the BOLD signal. Annu Rev Physiol 66:735-769.
Logothetis NK, Kayser C, Oeltermann A (2007) In vivo measurement of cortical impedance spectrum in
monkeys: implications for signal propagation. Neuron 55:809-823.
Logothetis NK, Pauls J, Augath M, Trinath T, Oeltermann A (2001) Neurophysiological investigation of
the basis of the fMRI signal. Nature 412:150-157.
Luck SJ, Hillyard SA (1994) Electrophysiological correlates of feature analysis during visual search.
Psychophysiology 31:291-308.
Luck SJ, Woodman GF, Vogel EK (2000) Event-related potential studies of attention. Trends Cogn Sci
4:432-440.
Mardia KV, Jupp PE (2000) Directional Statistics. Chichester: John Wiley & Sons Ltd.
Maunsell JH, Cook EP (2002) The role of attention in visual processing. Philos Trans R Soc Lond B Biol
Sci 357:1063-1072.
McPeek RM, Keller EL (2002) Saccade target selection in the superior colliculus during a visual search
task. Journal of Neurophysiology 88:2019-2034.
Mitzdorf U (1985) Current source-density method and application in cat cerebral cortex: investigation of
evoked potentials and EEG phenomena. Physiol Rev 65:37-100.
Mitzdorf U (1987) Properties of the evoked potential generators: current source-density analysis of visually
evoked potentials in the cat cortex. Int J Neurosci 33:33-59.
73
Monosov IE, Trageser JC, Thompson KG (2006) A comparison of the spatial tuning and time course of
attention-related signals in the primate frontal eye field. Soc Neurosci Abstr 32:439.415.
Monosov IE, Trageser JC, Thompson KG (2008) Measurements of simultaneously recorded spiking
activity and local field potentials suggest that spatial selection emerges in the frontal eye field.
Neuron 57(4):614-25.
Moore T, Fallah M (2001) Control of eye movements and spatial attention. Proc Natl Acad Sci U S A
98:1273-1276.
Moore T, Armstrong KM (2003) Selective gating of visual signals by microstimulation of frontal cortex.
Nature 421:370-373.
Moore T, Fallah M (2004) Microstimulation of the frontal eye field and its effects on covert spatial
attention. Journal of Neurophysiology 91:152-162.
Morishima Y, Akaishi R, Yamada Y, Okuda J, Toma K, Sakai K (2009) Task-specific signal transmission
from prefrontal cortex in visual selective attention. Nat Neurosci 12:85-91.
Muggleton NG, Juan CH, Cowey A, Walsh V (2003) Human frontal eye fields and visual search. Journal of
Neurophysiology 89:3340-3343.
Murthy A, Thompson KG, Schall JD (2001) Dynamic dissociation of visual selection from saccade
programming in frontal eye field. J Neurophysiol 86:2634-2637.
Nelson MJ, Pouget P, Schall JD (2006) Effect of electrode impedence on measurement of local field
potentials in the supplementary eye field. Soc for Neurosci Abstr 32:835.814.
Nielsen KJ, Logothetis NK, Rainer G (2006) Dissociation between local field potentials and spiking
activity in macaque inferior temporal cortex reveals diagnosticity-based encoding of complex
objects. J Neurosci 26:9639-9645.
Pashler HE (1998) The Psychology of Attention. Cambridge, MA: MIT Press.
Peng X, Sereno ME, Silva AK, Lehky SR, Sereno AB (2008) Shape selectivity in primate frontal eye field.
J Neurophysiol 100:796-814.
Pesaran B, Pezaris JS, Sahani M, Mitra PP, Andersen RA (2002) Temporal structure in neuronal activity
during working memory in macaque parietal cortex. Nature Neuroscience 5:805-811.
Pessoa L, Kastner S, Ungerleider LG (2003) Neuroimaging studies of attention: From modulation of
sensory processing to top-down control. Journal of Neuroscience 23:3990-3998.
Posner MI (1980) Orienting of attention. Q J Exp Psychol 32:3-25.
Pouget P, Emeric EE, Stuphorn V, Reis K, Schall JD (2005) Chronometry of visual responses in frontal eye
field, supplementary eye field, and anterior cingulate cortex. J Neurophysiol 94:2086-2092.
Rensink RA (2002) Change detection. Annual Review of Psychology 53:245-277.
Reynolds JH, Desimone R (2003) Interacting roles of attention and visual salience in V4.
Neuron 37(5):853-63.
Reynolds JH, Pasternak T, Desimone R (2000) Attention increases sensitivity of V4
neurons.Neuron 26(3):703-14.
Reynolds JH, Chelazzi L (2004). Attentional modulation of visual processing. Annu Rev
Neurosci 27:611-47.
Rizzolatti G, Riggio L, Dascola I, Umilta C (1987) Reorienting attention across the horizontal and vertical
meridians: evidence in favor of a premotor theory of attention. Neuropsychologia 25:31-40.
Sato T, Murthy A, Thompson KG, Schall JD (2001) Search efficiency but not response interference affects
visual selection in frontal eye field. Neuron 30:583-591.
Sato TR, Schall JD (2003) Effects of stimulus-response compatibility on neural selection in frontal eye
field. Neuron 38:637-648.
Schafer RJ, Moore T (2007) Attention governs action in the primate frontal eye field. Neuron 56:541-551.
Schall JD (1997) Visuomotor areas of the frontal lobe. In: Cerebral Cortex (Rockland K, Kaas JH, Peters
A, eds), pp 527-638. New York: Plenum Press.
Schall JD, Thompson KG (1999) Neural selection and control of visually guided eye movements. Ann Rev
Neurosci 22:241-259.
Schall JD, Pare M, Woodman GF (2007) Comment on "Top-down versus bottom-up control of attention in
the prefrontal and posterior parietal cortices". Science 318:44; author reply 44.
Schall JD, Morel A, King DJ, Bullier J (1995a) Topography of visual cortex connections with frontal eye
field in macaque: Convergence and segregation of processing streams. J Neurosci 15:4464-4487.
Schall JD, Hanes DP, Thompson KG, King DJ (1995b) Saccade target selection in frontal eye field of
macaque. I. Visual and premovement activation. J Neurosci 15:6905-6918.
74
Schall JD, Sato TR, Thompson KG, Vaughn AA, Juan CH (2004) Effects of search efficiency on surround
suppression during visual selection in frontal eye field. J Neurophysiol 91:2765-2769.
Schmolesky MT, Wang Y, Hanes DP, Thompson KG, Leutgeb S, Schall JD, Leventhal AG (1998) Signal
timing across the macaque visual system. J Neurophysiol 79:3272-3278.
Schroeder CE, Mehta AD, Givre SJ (1998) A spatiotemporal profile of visual system activation revealed by
current source density analysis in the awake macaque. Cereb Cortex 8:575-592.
Serences JT, Yantis S (2006) Selective visual attention and perceptual coherence. Trends Cogn Sci 10:3845.
Serences JT, Yantis S (2007) Spatially Selective Representations of Voluntary and Stimulus-Driven
Attentional Priority in Human Occipital, Parietal, and Frontal Cortex. Cereb Cortex 17:284-293.
Sheinberg DL, Logothetis NK (2001) Noticing familiar objects in real world scenes: the role of temporal
cortical neurons in natural vision. J Neurosci 21:1340-1350.
Silvanto J, Lavie N, Walsh V (2006) Stimulation of the human frontal eye fields modulates sensitivity of
extrastriate visual cortex. J Neurophysiol 96:941-945.
Smith DT, Jackson SR, Rorden C (2005) Transcranial magnetic stimulation of the left human frontal eye
fields eliminates the cost of invalid endogenous cues. Neuropsychologia 43:1288-1296.
Spitzer H, Richmond BJ (1991) Task difficulty: ignoring, attending to, and discriminating a visual stimulus
yield progressively more activity in inferior temporal neurons. Exp Brain Res 83:340-348.
Spitzer H, Desimone R, Moran J (1988) Increased attention enhances both behavioral and neuronal
performance. Science 240:338-340.
Tanaka K (1997) Mechanisms of visual object recognition: monkey and human studies. Curr Opin
Neurobiol 7:523-529.
Taylor PC, Nobre AC, Rushworth MF (2007) FEF TMS affects visual cortical activity. Cereb Cortex
17:391-399.
Tehovnik EJ, Sommer MA, Chou IH, Slocum WM, Schiller PH (2000) Eye fields in the frontal lobes of
primates. Brain Res Brain Res Rev 32:413-448.
Thomas NW, Pare M (2007) Temporal Processing of Saccade Targets in Parietal Cortex Area LIP During
Visual Search. J Neurophysiol 97:942-947.
Thompson KG, Bichot NP (2005) A visual salience map in the primate frontal eye field. Prog Brain Res
147:251-262.
Thompson KG, Biscoe KL, Sato TR (2005a) Neuronal basis of covert spatial attention in the frontal eye
field. J Neurosci 25:9479-9487.
Thompson KG, Bichot NP, Sato TR (2005b) Frontal eye field activity before visual search errors reveals
the integration of bottom-up and top-down salience. J Neurophysiol 93:337-351.
Thompson KG, Hanes DP, Bichot NP, Schall JD (1996) Perceptual and motor processing stages identified
in the activity of macaque frontal eye field neurons during visual search. J Neurophysiol 76:40404055.
Treisman A (1988) Features and objects: the fourteenth Bartlett memorial lecture. Q J Exp Psychol A
40:201-237.
Treisman A, Gormican S (1988) Feature analysis in early vision: evidence from search asymmetries.
Psychol Rev 95:15-48.
Turatto M, Galfano G (2000) Color, form and luminance capture attention in visual search. Vision Res
40:1639-1643.
Urbach D, Spitzer H (1995) Attentional effort modulated by task difficulty. Vision Res 35:2169-2177.
Vezoli J, Falchier A, Jouve B, Knoblauch K, Young M, Kennedy H (2004) Quantitative analysis of
connectivity in the visual cortex: extracting function from structure. Neuroscientist 10:476-482.
Wardak C, Ibos G, Duhamel JR, Olivier E (2006) Contribution of the monkey frontal eye field to covert
visual attention. J Neurosci 26:4228-4235.
Woodman GF, Kang MS, Rossi AF, Schall JD (2007) Nonhuman primate event-related potentials indexing
covert shifts of attention. Proc Natl Acad Sci U S A 104:15111-15116.
Zenon A, Ben Hamed S, Duhamel JR, Olivier E (2008) Spatial and temporal dynamics of attentional
guidance during inefficient visual search. PLoS ONE 3:e2219.