Download 1 Integration of Retinal Disparity and Fixation Distance

Articles in PresS. J Neurophysiol (February 11, 2004). 10.1152/jn.00712.2003
1
Integration of Retinal Disparity and Fixation Distance-Related Signals toward an Egocentric
Coding of Distance in the Posterior Parietal Cortex of Primates
Aldo Genovesio and Stefano Ferraina
Department of Human Physiology and Pharmacology, University of Rome ‘La Sapienza’, Piazzale
Aldo Moro 5, 00185 Rome, Italy.
Correspondence should be addressed to Stefano Ferraina ([email protected]).
Phone: +39 06 49910306; Fax: +39 06 49910936
Running Title: Egocentric coding of distance in parietal cortex
Number of words in the abstract: 168
Number of text pages: 42
Number of figures: 14
Number of tables: 1
Copyright (c) 2004 by the American Physiological Society.
2
Abstract
For those movements that are directed towards objects located in extrapersonal space, is preliminary
that visual inputs are remapped from a retinal coordinate system to a body-centered one. The posterior
parietal cortex (PPC) most likely integrates retinal and extraretinal information to determine the
egocentric distance of an object located in the three-dimensional (3-D) space. This determination
requires both a retinal disparity signal and a parallel estimate of the fixation distance. We recorded
from the lateral intraparietal area (LIP) to see if single neurons respond to both vergence angle and
retinal disparity and if these two signals are integrated to encode egocentric distance. Monkeys were
trained to make saccades to real targets in 3-D space. When both fixation distance and disparity of
visual stimuli were varied, the disparity tuning of individual neurons display a fixation distance
modulation. We propose that the observed modulation contributes to a spatial coding domain
intermediate between retinal and egocentric, because the disparity tuning shifts in a systematic way
with changes in fixation distance.
Key words: Vergence; disparity; eye movements; distance computation; lateral intraparietal area.
3
Introduction
Much of primate motor behaviour is manipulation of objects and other movements directed at targets in
the immediate extrapersonal space. Generating useful movements requires a body-centered estimate of
the objects’ locations. To a very large degree, localization is provided through visual input. Locating
objects in relation to the body with visual input requires a retinal disparity signal to provide a measure
of the object’s distance from the plane of fixation (Poggio 1995; Cumming and De Angelis 2001). If
this object distance information is combined with an estimate of fixation distance, the brain has
sufficient information to calculate an egocentric distance (Pouget and Sejnowski 1994). There are
three important cues that can provide the needed estimate of fixation distance: the extraretinal signals
vergence angle and accommodation, and the vertical disparity (Foley 1980). Vertical disparity is only
used by the visual system for objects larger than 20° of visual angle (Cumming et al. 1991; Rogers and
Bradshaw 1993), so it can be neglected in cases where smaller targets are viewed.
Data from
psychophysical experiments suggest that the vergence angle is the most important cue used for the
computation of fixation distance (Hofsten von 1976; Foley 1980).
Area LIP, in the lateral bank of the intraparietal sulcus (IPS; Fig. 1) is important for attention
(Gottlieb et al. 1998; Colby and Goldberg 1999), target selection, coordinate transformations for the
representation of extrapersonal space (Zhang and Barash 2000), decision processes (Platt and Glimcher
1999) and intention to move (Andersen et al. 1997; Snyder et al. 1997). It is an obvious candidate for a
site where retinal signals and extraretinal signals might be combined (Andersen et al. 1990b; Bremmer
et al. 1997a). Neurons of area LIP are modulated by retinal disparity (Gnadt and Mays 1995; Gnadt
and Beyer 1998; Ferraina et al. 2002). These disparity signals are relayed from LIP to both the frontal
eye field (FEF; Ferraina et al. 2000, 2002) and the intermediate layers of the superior colliculus (SC;
Gnadt and Beyer 1998; Ferraina et al. 2002): two important nodes in the network that controls saccadic
eye movements (Wurtz and Goldberg 1989; Wurtz et al. 2001). The presence of disparity-sensitive
4
neurons, characterized by the presence of a wide range of disparity sensitivity (Gnadt and Mays 1995;
Gnadt and Beyer 1998; Ferraina et al. 2002), indicates that these pathways may be used to generate eye
movements to targets located away from the fixation plane (see also Poggio 1995). In addition to the
retinal disparity signal that provides a measure of the distance to the fixation plane, there is also
activity in the PPC that reflects the fixation distance of a target (Sakata et al. 1980). Missing from the
literature, however, is the demonstration that these two signals are combined in a way that could
provide an estimate of egocentric distance. The present study was designed to investigate the presence
and to describe the form of such interaction in area LIP. We find that the interaction assumes the form
of a partial shift that is suitable for a role of the area in the process of visuomotor transformation from a
retinal to a body centered frame of reference. The results of such integration would be available to
motor areas of the frontal lobe that receive parietal anatomical connections, both directly from LIP
(Schall 1997) or through areas, such as the parietal MDP, 7m, 7a, AIP, that receive input from LIP
(Colby et al. 1988; Cavada and Goldman-Rakic 1989; Andersen et al. 1990a; Blatt et al. 1990;
Nakamura et al. 2001), and in turn, project to both dorsal and ventral premotor areas (Caminiti et al.
1996; Rizzolatti and Luppino 2001).
A brief report of this work has appeared previously (Ferraina and Genovesio 2001).
Material and Methods
Animals.
Two rhesus monkeys (Macaca mulatta) were studied using general procedures identical to those
recently described (Ferraina et al. 2002) and outlined here.
Animal care, housing, and surgical
procedures were in accordance with European guidelines on the use of animals in research (European
Community Council Directive 86/609/ECC).
5
In each monkey, under general anaesthesia (isofluorane), a recording cylinder was implanted
centered at stereotaxic coordinates P 5.0 – L 12.0 mm to allow recordings from area LIP. Binocular
scleral search coils were also implanted. During the experiments, the monkey sat in a primate chair
with its head restrained and faced two red light-emitting diodes (LEDs) positioned by robotic arms
(CRS Robotics, Burlington, Canada, see Fig. 2A). In order to prevent the LEDs from illuminating the
robot arms, the central target diameter was 0.5 mm and formed by the tip of a fibre optic that transmits
the light produced from the LED and helps on reducing the light diffusion. The peripheral target was a
< 0.5° diameter spot. All tasks were performed in total darkness and the room was illuminated during
the inter-trial epoch to avoid dark adaptation.
Behavioural tasks.
The animals’ workspace was formed by eight isovergence surfaces (Fig. 2B), passing through the
rotational centers of the two eyes and through all the potential targets subtended by the same vergence
angle, calculated by taking into account the interocular distance of the two monkeys (36 mm and 34
mm respectively) and separated in depth by steps of 1° of vergence angle. The closest isovergence
surface corresponded to 13° of vergence angle, and the farthest to 6° of vergence angle. Accordingly,
the closest fixation point (FP) was located at 158 mm and the farthest FP was located at 349 mm from
the monkey (149 mm and 324 mm were the corresponding values of fixation distances for the second
monkey). The studied fixation distance range was chosen in the proximal space, roughly corresponding
to the monkey’s arm length, because that is where the vergence angle expresses most of its power.
Rightward and upward rotations of the eyes were treated as positive. Vergence angle was computed as
left eye position minus right eye position (convergence thus being positive). Conjugate eye position
was calculated as the average between left and right eye position. Calibration was obtained by
requiring the monkeys to fixate sequentially five LEDs, one central and the others positioned in order
6
to form an angle of -10 and +10° both in the vertical and in the horizontal axis of the isovergence
surface corresponding to 10° of vergence angle. During calibration, the centre LED was aligned with
the eye that was being calibrated, while the other eye was kept covered. The same procedure was then
repeated for the other eye.
Monkeys were trained in a delayed saccade task with targets in 3-D space. Each trial started
with the presentation of a central FP and, after 500-800 ms of fixation, a peripheral visual target
appeared and remained on until the end of the trial. The depth of the FP and peripheral targets varied
between isovergence surfaces and from trial to trial. After 800-1200 ms delay period, the monkey was
required to make a saccade to the target within 500 ms. A memory version of the task was interleaved
in a block design; data obtained using this task will be part of a future report. Recording electrodes
were advanced down the lateral bank of the IPS while the monkeys performed the visual delayed
saccadic task. During this preliminary exploration, eye movements were made to a two-dimensional
array of targets at different elevation and eccentricity along the isovergence surface of the FP
corresponding to 10° of vergence angle. Each isolated neuron was first studied in this task to determine
the locus of maximal neural activity in the response field. Target eccentricity was tested from 10° to
20°; target elevation was tested from –20° to 20° in steps of 10°.
With the target centered in the response field, the fixation distance modulation was explored
while the monkeys made conjugate eye movements within isovergence surfaces (Fig. 2B; left). The
disparity modulation was explored while the monkeys made disjunctive eye movements (Fig. 2B;
right). The FP was maintained at one locus while the targets were arranged at different relative
positions in the 3-D workspace; the targets were arranged to produce either positive or negative
differences between the initial and the final vergence angle. The version component of the movement,
corresponding to the mean eccentricity, was never modified (Fig. 2C). Conjugate and disjunctive eye
movements trials were intermingled within blocks. In most of the neurons tested, data were obtained
7
during disjunctive eye movements that started from three fixation distances, corresponding to 6°, 10°
and 13° of vergence angle.
In a subgroup of neurons we obtained data from disjunctive eye
movements starting from six fixation distances. For each condition tested, both conjugate and
disjunctive eye movements were repeated for at least 5 trials (up to a maximum of 10 trials). Monkeys
were required to fixate within a 2° (diameter) spherical window. The control was maintained separated
for each eye. An eye movement was considered successful if each eye landed within a 3° (diameter)
spherical window centered on the target. The accuracy and consistency from trial to trial of the
vergence behaviour was evaluated by offline inspection. More than 85% of the trials resulted in a
vergence error of less than 0.3°. All trials with vergence errors above 0.5° were excluded from further
analysis.
Data analysis.
Rasters of neuronal discharges were aligned on specific behavioural events. Raw spike counts were
used to measure neuronal activity during successive epochs of the tasks. Several scalar activity values
were defined for various epochs: 1) “fixation activity,” the mean discharge rate 500 ms to 200 ms
before target presentation, 2) “visual activity,” the mean discharge rate from 70 to 170 ms after
appearance of the target, 3) “delay activity,” the mean discharge rate of the last 300 ms of the delay
epoch, ending when the fixation point disappears, 4) “presaccadic activity,” the mean discharge rate
100 ms before saccade initiation. The choice of the analysis windows originated both from offline
inspection of the data (for example, the beginning of the visual activity period was obtained from the
average latency observed) as well as from the choice to obtain activities uncontaminated, as much as
possible, by concurrent signals (for example, the end of the fixation period was excluded from the
fixation activity because coincident with the robot approaching the target position). Our initial database
of LIP neurons (152/171 of the recorded neurons) was based on the presence of significant visual or
8
visuomotor modulation in the task used for the initial definition of the response field. Visual and delay
activities were regarded as significantly modulated if they differed statistically from the fixation
activity. Presaccadic activity was considered significant if it was statistically different from the delay
activity. In our sample, all the modulation observed was corresponding to an increase of neural
discharge.
We performed a Shapiro-Wilk test of normality on each data set. Within-sample comparisons
used either paired Student's t-tests or nonparametric Wilcoxon signed-rank tests. For group
comparisons, we used either an analysis of variance (ANOVA) or the nonparametric Kruskal-Wallis
ANOVA, depending upon the sample normality. For all statistical tests p < 0.05 was set as significance
level.
Recording sites.
Area LIP was identified physiologically in the IPS by the abundance of neurons with significant visual
responses and saccade-related activity. The presence of delay activity maintained in the memory
version of the delayed saccade task (Gnadt and Andersen 1988; Barash et al. 1991) was used as an
additional criterion for locating area LIP. In the first animal, some of the penetrations (Fig. 1) were
made using electrodes labelled with fluorescent dyes (DiCarlo et al. 1996). Subsequent standard
histological procedures confirmed that the recorded neurons were located in the middle third of the
lateral bank of the IPS. The second monkey is currently involved in a different experimental protocol
and thus the histological material is not yet available.
9
Results
Behavioral data
As described previously (Collewijn et al. 1988; Maxwell and King 1992), saccadic eye movements
between isovergence targets produce a transient deviation from conjugacy, with an initial divergence
followed by a convergence that brings the eyes back toward the target. Our behavioural data confirm
this observation (Fig. 3A). Despite these variations, we will refer to these eye movements as
‘conjugate’ to differentiate them from the disjunctive eye movements that are produced when targets
are located at a depth different from the FP (Fig. 3B). In these disjunctive eye movements, a vergence
component is introduced to the versional component of eye movement in order to allow the correct
refixation in depth (Carpenter 1988; Erkelens et al. 1989). The vergence component could be positive
or negative and depends on the disparity of the stimulus.
During binocular fixation, the fixation point is supposed to be accurately projected onto the
centers of the foveae in each eye. However, is well known that even in the best attempt to fixate, the
eyes are never stationary (Motter and Poggio 1984) and vergence errors can occur: the visual axes may
intersect in front or behind the fixation point, and result in an exo or eso fixation disparity, respectively
(Ogle 1967; Collewijn et al. 1988; Erkelens et al. 1989). Furthermore, a relation of the fixation
disparity with the fixation distance has been described with a clear pattern of increasing
underconvergence (eso fixation disparity) for increasingly near fixation (Jaschinski-Kruza 1993;
Jaschinski, 2001). This pattern is evident in the data obtained in both of our monkeys, as shown in the
examples of Figure 4. Fixation disparity gradually becomes more positive for near fixation distances.
In our data set, we observed always such linear trend that could be quantified by a linear regression
trough the single observations. The mean value for all the slopes obtained from significant (p < 0.05)
linear regressions was 2.97 min arc (SD +/- 1.1) and indicates the average effect of the fixation
distance on fixation disparity. Since this effect slightly interferes with the fixation distance modulation
10
of the neural activity, both the vergence angle and the target disparity in all of the analysis have been
adjusted accordingly.
Neurons in the lateral intraparietal area are modulated both by fixation distance changes and retinal
disparity.
One hundred and fifty-two neurons (116 from monkey B, 36 from monkey L) were analysed while the
monkey was required to make conjugate eye movements along eight different isovergence surfaces.
For the most part, we will report results obtained during the delay-period, since activity during this
period should be related to visuomotor transformations (Andersen et al. 1990b, 1997). The results
obtained from the analysis of the other two experimental epochs, visual and presaccadic activities, are
reported in Table 1.
The neural activity of 45 (29.6 %) of the LIP neurons studied was significantly modulated
(ANOVA; p < 0.05) by fixation distance changes of vergence angle. Figure 5A shows a typical
example of a neuron with fixation distance modulation. The modulation is still evident after correction
of the vergence values for fixation disparity (Fig. 5A, right). For this neuron, neural activity increases
during the preparation for eye movement to target located along an isovergence surface when the
movement is made at closer fixation distances. For most of the fixation distance-modulated neurons
(40/45; 88.9%), the neural discharge shows a monotonic increase toward near or far fixation distances.
More than 2/3 of these neurons (29/40; 72.5 %) showed a near-space preference. A small number
(5/45; 11.1%) of fixation distance-modulated neurons discharged maximally at an intermediate fixation
distance, and these were excluded from further analyses.
In order to see if a disparity signal could be found in neurons with fixation distance-related
modulation, all 152 neurons tested for conjugate eye movements were also tested during the execution
of disjunctive eye movements. Fixation distance was maintained constant and monkeys were required
11
to execute both the corresponding conjugate eye movement, as well as divergence and convergence eye
movements. The accompanying vergence component corresponded to the retinal disparity of the
targets and varied in steps of one degree. Figure 5B shows the resulting neural modulation for the same
neuron as depicted in Figure 5A. This neuron preferred positive (crossed) disparities and was classified
as a near-neuron according to Poggio’s classification (Poggio and Fisher 1977). Figure 5B right (black
dots), shows the modulation after correction for fixation disparity. Furthermore, this plot shows that the
modulation observed for the fixation distance can only be partially explained by the retinal disparity
modulation. In fact, the white dots in Figure 5B show the discharge rate of the neuron when tested
during conjugate eye movements vs. the corresponding average retinal disparity. It is evident from the
plot that changes in retinal disparity can only account at most for a small component of the changes in
the response rate.
At the population level, a significant modulation (ANOVA; p < 0.05) obtained by disparity
changes was observed for 61.2 % (93/152) of the neurons tested. Sixty-one neurons were classified as
near and 32 as far.
Thirty-eight of the neurons modulated by the fixation distance in the conjugated eye movements
were also modulated by disparity changes. Only 7 neurons were sensitive only to changes in the
fixation distance. All told, 65.8% (100/152) of the neurons studied were modulated by at least one of
the two signals (Fig. 6).
Alignment of disparity and vergence related signals at single neuron level.
The effect of fixation distance on neural activities was quantified in those neurons showing a linear
trend by fitting the data to a linear model. A significant linear fit (p < 0.05) was found in 85% of these
neurons (34/40), and most (32/34) passed the Goodness-of-fit-test for a linear model (p > 0.05). The
model was fitted to the data after correction for fixation disparity. For each replication, the fixation
12
disparity was calculated and then subtracted to the corresponding vergence angle value. Figure 7A
shows the obtained regression line for the same neuron depicted in Figure 5A. The slope value of 3.66
indicates that the fixation distance modulates neural activity by 3.66 spikes per degree change in
vergence angle. Similarly, we obtained a quantification of the effect of the changes in disparity on the
neural activity by fitting the data to a cubic model. This model was used because we often observed
that neurons discharged maximally at intermediate values of disparity in the range tested. Model was
fitted to the true disparity after correction for the fixation disparity calculated for each single
replication. In order to avoid the floor effect of zero neural activity at non-preferred disparities, we
fitted the model only to the data obtained at the preferred disparity range. Further, we used the data
obtained for movements starting from the limits of the workspace to maximize the range of disparities
included in the index. For neurons preferring the positive disparity range, we used the data obtained
from convergence eye movement starting from the isovergence surface corresponding to 6° of
vergence angle (Fig. 7B), while for neurons preferring the negative disparity range, we used data
obtained from divergence eye movement starting from the isovergence surface corresponding to 13° of
vergence angle. A significant (p < 0.05) fit to the cubic model was found in the 72 % of the neurons
(67/93) displaying a modulation for disparity changes. From the tuning function of these neurons, we
derived the value of neural activity corresponding to the maximum modulation and the corresponding
value in disparity. Armed with these values, we calculated a Zero Index = (DMAX –DZERO)/n; where
DMAX is the neural activity corresponding to the maximum, DZERO is the neural activity corresponding
to 0° of disparity, and n is the disparity value corresponding to the maximum of the tuning curve. The
sign of n is positive for crossed disparities and negative for uncrossed disparities; as a consequence, a
positive index refers to neurons preferring the positive disparities range and vice versa. This index is
the degree to which disparity changes can modulate neural activity in the preferred range. Figure 7C
shows an example of the application of this index to the neuron of Figure 7B. For this neuron, retinal
13
disparity modulates the firing frequency by 6.57 spikes per degree of change in retinal disparity.
Figures 7D-F show the corresponding indices obtained for a neuron modulated by negative disparities
and preferring far fixation distances during conjugate eye movements. Figure 8 shows the distribution
of indices (slope and Zero Index) obtained at the population level for the neurons modulated (ANOVA;
p < 0.05) by the fixation distance (A) and by disparity (B) changes, respectively. The median value of
slope and Zero Index, expressed in absolute values, were 2.7 and 4.4 respectively.
For the group of neurons modulated by both signals (ANOVA) and that displayed both a
significant linear fit for fixation distance and a significant cubic fit for disparity (32), we quantitatively
explored their interaction effect by using the two modulation indices described. What emerges is that
the modulation observed during disjunctive movements directed to targets located at different
disparities provides information consistent with the fixation distance-modulation observed in the
conjugate eye movements (Fig. 9A). In fact, the neurons increasing their discharge rates for near
fixation-distances during conjugate eye movements (with a positive slope in the linear regression)
showed the strongest activity for crossed disparities (during convergence disjunctive eye movements),
while those neurons that discharged preferentially for far fixation-distances during conjugate eye
movements (negative slope) showed the strongest activity for uncrossed disparities (divergence
disjunctive eye movements).
The same group of neurons, modulated by both signals, have been used to further confirm that
only a small component of our results can be explained by retinal disparity changes. For each neuron,
we calculated, from the slope of the linear regression, the modulation obtained during conjugate eye
movements in the range of the fixation distances tested. For example, the neuron in Figure 7A displays
a modulation of 25.6 spikes for 7° of changes in the vergence angle (from 6 to 13 degrees of vergence
angle). This value was contrasted to the estimated value of the maximal modulation due to the fixation
disparity changes. This estimate was obtained by using the fitting of the data collected during
14
disjunctive eye movements. From the fitting the modulation was calculated in the range of 0.5° of
fixation disparity, which corresponds to the maximum value present in our data after the correction
procedure applied to the vergence angle errors (see Behavioural Tasks session of Materials and
Methods). As a result, using the fitting of Fig. 7B, we could say that the same neuron displays a
modulation of 2.1 spikes for a fixation disparity of 0.5° that is 12 times lower than the modulation for
the fixation distance. Figure 9B shows these two values contrasted for all the neurons. The fitting for
the disparity data presented in this figure was obtained using a fourth order model in order to improve
the estimate. It is evident that the modulation observed during changes in fixation distance is always
stronger than the modulation that could arise from retinal disparity changes, even assuming the
maximum range of fixation disparity.
Overall, our results provide evidence that neuronal activity in parietal area LIP is sensitive to
both the vergence angle and the retinal disparity of the target, and that these two signals, when present
in the same neuron, tend to be aligned in an apparently functional way.
Integration of retinal disparity and vergence signals in LIP neurons. Contribution to egocentric
representation of targets in 3-D space.
If a neuron is coding egocentric distance rather than disparity per se, one should expect a modulation of
its disparity tuning with vergence. Figure 10A shows the effect of the fixation distance changes in the
disparity value of targets located at the same retinal eccentricity and different distances from the
subject eyes. The disparity value of an unmoving target in space becomes more positive as the
vergence angle decreases. Figure 10B shows two hypothetical neurons with broadly modulated
disparity tuning and an efficient shift of the disparity tuning, when tested at different fixation distances,
toward an egocentric coding of the targets. One neuron increases its activity with positive disparities
(black lines) and the other increases firing with negative disparities (grey lines). In each case, the shift
15
is equal in intensity, but of opposite sign, to the vergence angle changes. This relationship assures that
the discharge rate matches the discharge rate for the same unmodified egocentric distance of the target.
The overall effect, a positive shift, is similar for both (convergence and divergence) idealized neurons.
The presence of a coding for the egocentric distance is confirmed by the perfect overlap of the different
curves obtained for the different fixation distances, when the same data are reported as function of the
distance of the targets from the subject (Fig. 10C).
Using the above, idealized model, we tested the interaction between disparity and vergencerelated signals for those LIP neurons (n=51/152; 33.6 %) with sufficient data: at least five replications
during the execution of disjunctive eye movements made from six different fixation distances. As
shown in the three example neurons of Figure 11, the disparity related signals were modulated by
changes in the fixation distance. Both the vergence angle and the disparity values were those obtained
by taking into account the fixation disparity for each single data point. The power of the resulting
interaction is different for different neurons. In a sense the neurons differed from one another in their
capacity to contribute to the computation of egocentric distance to the targets. The first of the
convergence neurons (Fig. 11A) showed a clear shift in disparity tuning, producing well-separated
curves for the different fixation distances. The effect is less evident in both of the other convergence
neuron (Figs. 11B and 11E) and in the divergence neuron (Figs. 11C and 11F).
As evident from the data shown in Fig. 11, in the evaluation of the effect of the vergence angle
on the disparity tuning was often difficult to discriminate the contribute of the horizontal shift effect
from that of a gain effect. Our first tentative to approach this issue was to compare the results obtained
by fitting the data obtained from those neurons that show a significant modulation (ANOVA) for
disparity in at least one fixation distance (n= 38/51; 74.5%), to two different models:
Z = a0 + a1(x + sy)
(1);
16
Z = a0 + (xgy)
(2);
in both models, Z is the firing rate, x the disparity, y the vergence angle; in equation (1) s
represents the parameter that takes into account the horizontal shift; in equation (2) g represents the
parameter that takes into account the gain effect. In order to compare the results, we used the adjusted
R2 (R2a) that balance the cost of using and comparing models with a different number of parameters:
R2a = [(n-1) R2 – k] / (n-1-k)
(3);
where k is the number of regression parameters in the model and n is the number of data points.
Therefore, the R2a is always a reduced amount of R2. Figure 12A shows the results for the nineteen
disparity modulated neurons with a value of R2a > 0.5 in at least one of the two models (Fig. 12A).
This prevalence of the R2a obtained using the shift model suggests a stronger effect of the shift that
need to be confirmed by further evidences. In this direction, a different approach was to look at the
results obtained by fitting the data to a ‘composite’ model:
Z = a0 + a1[(x + sy) + (xgy)]
(4);
This model corresponds to a combination of the two previous models expressed in equation 1
and 2 and permits to estimate the relative contributions of the parameters s and g in determining the
regression accuracy. Fig. 12B shows the distribution of the p values obtained for s and g for the same
group of neurons of 12A. The smaller the value of p, the less likely the parameter is actually zero. For
example, if p = 0.01, there is a 1% chance that the actual parameter is zero. If p = 0.95, there is a 95%
17
chance that the actual parameter value is zero. In cases like this latter, the parameter in question can be
removed from the model without affecting the regression accuracy. Fig. 12B shows that the p value for
the parameters s is always < 0.05, the same is true for the parameters g only for two neurons.
Therefore, we concluded that even if the gain effect is able to explain partially the effect produced by
by the changes in vergence angle on the disparity tuning, the principal effect is a shift effect.
To explore quantitatively the predominant shift effect at the population level, taking into
account the observation that disparity tuning, in our data, was better described by a cubic model, the
same data were fitted to the model:
Z = a0 + a1(x+sy) + a2(x+sy)2 + a3(x+sy)3
(5)
Equation (5) amounts to a linear interaction between a cubic model for the disparity modulation and a
linear model for the fixation distance modulation. Z is the firing rate, x the disparity, y the vergence
angle, and the parameter s is the interaction term between the cubic and linear models. Parameter s is a
‘shift parameter’ related to the linear shift of the disparity tuning as a function of the vergence angle.
When s = 0, the firing rate does not depend on the vergence angle, and thus there is no shift as the
vergence angle is varied. When s = 1, the activity moves 1° towards positive disparity as the vergence
angle decreases by 1°.
Figure 13 (A-C) shows three-dimensional plots of the model for the activity obtained during the
delay period for the same neurons and data depicted in Figure 11. The amount of shift is graphically
represented by the displacement of the isodischarge color regions of the plot toward more positive
disparity values as the vergence angle decreases. The figure shows that the shift was positive (i.e.
toward positive disparities) for both the convergence and the divergence neurons.
18
Figure 14A shows the distribution of the significant parameter s (p < 0.05) obtained from the
regression model applied to the population of disparity modulated neurons (28/38; 73.7%). The greater
part of these neurons displays a positive shift value with a median value of 0.4 (arrow in the figure). In
other words, as a result of the partial shift these neurons are able ‘to compensate’ for about 3°, on
average, when the vergence angle changes by 7°. Finally we compared the regression model for the
data obtained for neurons that are disparity modulated both in the visual and in the delay epochs
(35/51). For those neurons that have a significant parameter s (p < 0.05) during the visual period
(23/35; 65.7%), the results show a median value that was 0.42, that is a value not significantly different
from that obtained for the delay epoch (t test; p = 0.19). The only difference was obtained by
comparing the R2 of the model (Fig. 14B), indicating that the interaction between disparity and
fixation-distance related signals is better expressed in the delay epoch (t test; p < 0.05). This implies a
major role for this epoch in the neural processing of visuomotor transformations.
The observed co-linearity between disparity sensitivity and vergence angle-related modulation
in the first experiment is still evident in the color plot. In fact, the isodischarge regions of the
convergence neurons (Figs. 13A and 13B) reveal how the shift determines the reduction of the firing
frequency for the isovergence movements, corresponding to zero degrees in the disparity axis, as the
vergence angle decreases. The opposite effect is equally clear for the divergence neuron (Fig. 13C).
What initially appears to be a gain modulation effect based on the vergence-related modulation
during conjugate eye movements, is, in fact, a shift effect when several disparities are considered
together. In the range of disparity tested in the present experiment, this shift never changes the
disparity-selective category (near vs. far). The two populations remain segregated with a group of ‘near
neurons’ computing the visuomotor transformations for the closer part of the extrapersonal space and a
group of ‘far neurons’ working similarly for the targets farther out.
19
Discussion
The present study is the first to demonstrate that individual LIP neurons combine visual disparity
signals with fixation distance information (the vergence angle) in a manner that can be used to
determine 3-D egocentric distance to an object in space. This work validates the hypothesis that a
combination of retinal information about target position and extraretinal information about the position
of the eyes in the head are calculated by the central nervous system, and supports the theory that these
calculations compute the position of visual targets in head-centered coordinates (Zipser and Andersen
1988). Our results show that fixation distance modulates only a subpopulation (38/93; 40.9 %) of the
disparity related neurons. The remaining disparity related neurons are presumably involved in the
representation of eye-centered coordinates used in the PPC (for a review see Andersen et al. 1997).
Previous studies in area LIP (Brotchie et al. 1995), as well evidence from humans (Brotchie et al.
2003), show that LIP integrates visual signals with head-position signals. Thus, LIP is likely to contain
multiple coordinate systems, and our results support, rather than conflict with this view.
Although it is clear that LIP has a visual representation, it has been difficult to distinguish
experimentally movement planning and attention. One hypothesis is that it codes the intention to make
a saccade to a stimulus (Andersen et al 1997; Snyder et al. 1997) and the second is that it provides a
map of salience stimuli in the visual world without specifying any motor plan (Colby and Goldberg
1999; Powell and Goldberg 2000; Gottlieb et al. 1998). The stronger evidence for a role of LIP as a
part of the system for planning eye movements is the greater delay period activity of LIP neurons for
upcoming eye movements than of simultaneously generated arm movements to a different location
(Snyder et al. 1997). In addition, the presence of neurons in LIP that show non-spatial saccade-specific
activation (Dickinson et al 2003), challenges the view that LIP is involved only in representing salient
spatial location. In this study, in addition to cells with an effector-specific activation, there was another
group of cells showing spatial selective responses that could be thought as attention responses or as
20
default plans for eye movements. Therefore, it seems unlikely that fully formed planned movement are
specified in LIP, instead the intended movements activity more likely represents early plans at the
intermediate stages of the sensory motor transformations.
Our tasks are not designed to distinguish whether the activity during the delay period is related
to attention to the target or to a motor plan because the visual attention is pinned to the spatial location
of the eye movement target, but the concept that the activity in posterior parietal cortex is related to the
sensory-motor transformations (Andersen et al. 1990b; Andersen et al 1997; Zhang and Barash, 2000)
is generally accepted. What is not clear is whether the representation of the extrapersonal space could
subserve a range of behaviours not limited to saccadic eye movements, including selective visual
processing through its connection with visual areas (Baizer et al 1991). In either case, LIP must
integrate stimuli in different modalities and provide to the areas which it projects its analysis of the 3D
extrapersonal space and/or an early eye movement plan.
Relationship to previous studies.
Previous studies have demonstrated vergence angle modulation of PPC neurons in both humans
(Hasebe et al. 1999; Kapoula et al. 2001) and monkeys (Gnadt 1992). Our data shows that the majority
(72.5 %) of the vergence angle modulated neurons prefer near target locations, and their activities
increase gradually as the fixation distance decreases. This agrees with earlier recordings in PPC (area
7a) by Sakata and colleagues during a visual fixation and a visual tracking task (Sakata et al. 1980,
1985). Distance modulation of visual related activity has also been found in V1 (Trotter et al. 1996;
Gonzalez and Perez 1998), and in V2 and V4 (Dobbins et al. 1998; Rosenbluth and Allman 2002)
areas. The influence of the extraretinal signals on the neuronal activity has been often described as a
‘gain field effect’ (Andersen et al. 1990b; Bremmer et al. 1997a; Salinas and Thier 2000). The
interaction between the disparity signal and the fixation distance, in the form of a gain effect, has been
21
previously observed in area V1 (Trotter et al. 1996; Gonzalez and Perez 1998) and in a small number
of the neurons tested in area MT (Roy et al. 1992). In all of these prior studies the monkeys' vergence
angle was not measured. A control experiment using prisms (Trotter et al. 1996) supported the
involvement of the vergence angle in the fixation distance modulation of area V1. Vergence angle has
been measured by Cumming and Parker (1999). These authors found very little effect of the vergence
angle on the disparity tuning in area V1. It should be noted that Cumming and Parker explored a range
of fixation distance (50-175 cm), corresponding to changes in vergence of 1-3.5°, different from the
range tested by Trotter and colleagues (2-10°), that found the maximal effect at 20 cm. Accordingly,
we decided to explore the vergence influence on neural activity in the proximal space (from 15 to 35
cm from the monkeys eyes and from 6° to 13° of vergence angle) where the vergence system expresses
most of its power.
In the first experiment we found a strong spatial colinearity between the disparity category
(near vs. far) and the modulation exerted by extraretinal signals. Neurons discharging preferentially
during a divergence eye movement to targets located at negative disparities displayed a parallel
positive linear trend for increasing fixation distances when the targets were maintained at constant
disparity. Neurons preferring convergence movements displayed exactly the opposite trend. In area
LIP, a similar spatial colinearity of different signals has been found during smooth pursuit, between the
preferred direction of movement and the eye position gain field (Bremmer et al. 1997a). The same
authors observed the lack of such alignment in both areas MT and MST (Bremmer et al. 1997b). In
contrast, an alignment between retinal signals and eye position signals has been found in area V3a
(Galletti and Battaglini 1989) that, similarly to MST and MT, projects to LIP (Blatt et al. 1990). The
importance of the spatial colinearity in the process of coordinate transformations has been validated by
computational studies (Xing et al. 2000). Comparing different models, the authors found a prevalence
of colinearity between receptive fields and gain fields only when the neural networks were trained to
22
operate transformations from eye-centered to body-centered coordinates. In the hidden units of the
models presented by Xing and colleagues (2000) the colinearity was associated with partial shifts of the
receptive fields following changes in eye position. A similar colinearity of signals and presence of a
partial shift was evident for the disparity tuning in the LIP neurons of our study during vergence angle
changes.
In summary, the unit properties we have demonstrated here fit well with both prior recording
studies and the theoretical framework surrounding the role of LIP in spatial mapping.
Partial shift modulation of retinal disparity.
When the disparity signal is kept constant, i.e. during the execution of conjugate eye movements to the
preferred eccentricity, the emerging modulation of the neural activity for changes of vergence angle is
similar to the previously described gain modulation. When the two signals were both varied, a
proportion of LIP neurons displayed a disparity modulation that was not fixed but showed a partial
shift, as a function of the fixation distance changes. This is the predicted behaviour of neurons using an
intermediate coding between an eye and an egocentric coordinate system. Different computational
models (Pouget and Sejnowski 1994; Mitchel and Zipser 2001) indicate how this partial shift could be
used for the egocentric distance calculation. A preliminary report for LIP neurons (Gnadt and Mays
1991) described that the gain of the response, but not the disparity tuning, is modulated as a function of
the vergence angle. Further, Lehky et al. (1990) have shown that their neural network trained to
determine distance develops gain fields for vergence and disparity (see also Pouget and Sejnowski
1994). Our data show that both effects are present in area LIP with a prevalence of the shift effect. It is
difficult to establish the relative contribute of the two effects because these neurons display a broad
tuning for disparity; the widths of the tuning curves increase as the best disparities of the response
peaks increase.
Pouget et al. (2002) showed that partially shifting receptive fields emerge in a
23
recurrent network designed to compute bi-directional coordinate transformations between eye- and
head-centered coordinates. They argue that a prerequisite for the appearing of a partial shift is the
convergence in a multimodal area of at least two inputs in distinct frames of reference. Partially
shifting receptive fields would be a consequence of having a neural network that can perform
multidirectional computations such as comparisons between sensory modalities, sensorimotor
transformations and inverse transformations from motor output to sensory coordinates (Pouget et al.
2002). LIP, as a site of convergence of visual and auditory signals, with the latter codified in both eyeand head-centered coordinates (Stricanne et al. 1996), would offer the fundamental conditions for the
appearance of a partial shift. It remains to be verified whether the neurons showing a partial shift are
well suited to the alignment of auditory and visual maps. Such integration might be possible since it
has been shown that auditory responses in area LIP are associated with visual responses in the same
neuron (Linden et al. 1999; Grunewald et al. 1999). There are other examples of intermediate coding of
different coordinate systems that appear as partial shifts. These have been reported in multimodal areas
outside of LIP: the ventral intraparietal area (Duhamel et al. 1997), the ventral premotor cortex
(Graziano et al. 1997), the parietal reach region (Cohen et al. 2000), and the SC (Jay and Sparks 1987,
Hartline et al. 1995).
Our results strongly suggest that LIP participates in the encoding of egocentric visual spatial
information through a distributed network of broadly tuned disparity neurons that display a partial shift
in their tuning in the direction opposed to the changes in vergence angle. These neurons do not change
their disparity category preference because they contribuite to the computation of different portions of
the extrapersonal space (near vs. far).
In this model intermediate coding contributes to the
representation of space in different coordinate frames, eye-, or ego-centered (Snyder et al. 1998). The
resulting computation is available to all motor systems receiving a posterior parietal input both directly
from LIP (Schall 1997) and indirectly through LIP parieto-parietal connections (Colby et al. 1988;
24
Cavada and Goldman-Rakic 1989; Andersen et al. 1990a; Blatt et al. 1990; Nakamura et al. 2001). For
example, connection between the parietal grasp-related area AIP and area LIP (Nakamura et al. 2001)
may contribute to the computation of target location relative to the hand. Just as importantly, LIP
connections through the SC (Paré and Wurtz 1997; Gnadt and Beyer 1998; Ferraina et al. 2002) could
supply a body-centered representation of visual targets used by the gaze-independent reaching related
neurons in the SC (Stuphorn et al. 2000).
25
Acknowledgements
We are grateful to R Caminiti for his support throughout this research, to RH Wurtz, M Paré and Y
Trotter for their critical comments to a preliminary draft of this manuscript and to AR Mitz for aid with
editing. The study was partially supported by funds from the Ministry of Scientific and Technological
Research of Italy.
26
References
Andersen RA, Asanuma C, Essik G, and Siegel RM. Cortico-cortical connections of anatomically
and physiologically defined subdivisions within the inferior parietal lobule. J Comp Neurol 296: 65113, 1990a.
Andersen RA, Bracewell, RM, Barash S, Gnadt JW, and Fogassi L. Eye position effects on visual,
memory, and saccade-related activity in areas LIP and 7A of macaque. J Neurosci 10: 1176-1196,
1990b.
Andersen RA, Snyder LH, Bradley DC, and Xing J. Multimodal representation of space in the
posterior parietal cortex and its use in planning movements. Annu Rev Neurosci 20: 303–330, 1997.
Baizer JS, Ungerleider LG, Desimone R. Organization of visual inputs to the inferior temporal and
posterior parietal cortex in macaques. J Neurosci 11: 168–190, 1991.
Barash S, Bracewell RM, Fogassi L, Gnadt JW, and Andersen RA. Saccade-related activity in the
lateral intraparietal area. I. Temporal properties; comparison with area 7a. J Neurophysiol 66: 10951108, 1991.
Blatt GJ, Andersen RA, and Stoner GR. Visual receptive field organization and cortico-cortical
connections of the intraparietal area (area LIP) in the macaque. J Comp Neurol 299: 421-445, 1990.
27
Bremmer F, Distler C, and Hoffmann KP. Eye position effects in monkey cortex. II. Pursuit- and
fixation-related activity in posterior parietal areas LIP and 7A. J Neurophysiol 77: 962-977, 1997a.
Bremmer F, Ilg UJ, Thiele A, Distler C, and Hoffmann KP. Eye position effects in monkey cortex.
I. Visual and pursuit-related activity in extrastriate areas MT and MST. J Neurophysiol 77: 944-961,
1997b.
Brotchie PR, Andersen RA, Snyder LH, and Goodman, SJ. Head position signals used by parietal
neurons to encode locations of visual stimuli. Nature 375: 232-235, 1995.
Brotchie PR, Lee MB, Chen DY, Lourensz M, Jackson G, and Bradley WG. Head position
modulates activity in the human parietal eye fields. Neuroimage 18: 178-84, 2003.
Caminiti R, Ferraina S, and Johnson, PB. The sources of visual information to the primate frontal
lobe: a novel role for the superior parietal lobule. Cereb Cortex 6: 319-328, 1996.
Carpenter RHS. Movements of the eyes Cambridge: Gonville and Caius, 1988.
Cavada C, and Goldman-Rakic PS. Posterior parietal cortex in rhesus monkey: Evidence for
segregated corticortical networks linking sensory and limbic areas with the frontal lobe. J Comp Neurol
287: 422-445, 1989.
Cohen YE, and Andersen RA. Reaches to sounds encoded in an eye-centered reference frame.
Neuron 27: 647-652, 2000.
28
Cohen YE, and Andersen RA. Common reference frame for movement plans in the posterior parietal
cortex. Nat Rev Neurosci 3: 553-562, 2002.
Colby CL, Gattass R, Olson A, and Gross CG. Topographical organization of cortical afferents to
extrastriate visual area PO in the macaque: a dual tracer study. J Comp Neurol 269: 392-413, 1988.
Colby CL, and Goldberg ME. Space and attention in parietal cortex. Annu Rev Neurosci 23: 319–
349, 1999.
Collewijn H, Erkelens CJ, and Steinman RM. Binocular co-ordination of human horizontal saccadic
eye movements. J Physiol 404: 157-182, 1988.
Cumming BG, Johnston EB, and Parker AJ. Vertical disparities and perception of threedimensional shape. Nature 349: 411-413, 1991.
Cumming BG, and Parker AJ. Binocular neurons in V1 of awake monkeys are selective for absolute,
not relative, disparity. J Neurosci 19: 5602-5618, 1999.
Cumming BG, and De Angelis GC. The Physiology of stereopsis. Annu Rev Neurosci 24: 203-238,
2001.
DiCarlo JJ, Lane, JW, Hsiao SS, and Johnson KO. Marking microelectrode penetrations with
fluorescent dyes. J Neurosci Methods 64: 75-81, 1996.
29
Dickinson AR, Calton JL, and Snyder LH. Nonspatial saccade-specific activation in area LIP of
monkey parietal cortex. J Neurophysiol 90: 2460-4, 2003.
Dobbins AC, Jeo RM, Fiser, J, and Allman JM. Distance modulation of neural activity in the visual
cortex. Science 281: 552-555, 1998.
Duhamel JR, Bremmer F, BenHamed S, and Graf W. Spatial invariance of visual receptive fields in
parietal cortex neurons. Nature 389: 845-848, 1997.
Erkelens CJ, Steinman RM, and Collewijn H. Ocular vergence under natural conditions. II. Gaze
shifts between real targets differing in distance and direction. Proc R Soc Lond B Biol Sci 236: 441465, 1989.
Ferraina S, and Genovesio A. Saccades to real targets in 3D space: Influences of vergence in the
lateral intraparietal area. Soc Neurosci Abstr 28: 575.3, 2001.
Ferraina S, Paré M, and Wurtz RH. Disparity sensitivity of frontal eye field neurons. J Neurophysiol
83: 625-629, 2000.
Ferraina S, Paré M, and Wurtz RH. Comparison of cortico-cortical and cortico-collicular signals for
the generation of saccadic eye movements. J Neurophysiol 87: 845-858, 2002.
Foley JM. Binocular distance perception. Psychological Review 87: 411-434, 1980.
30
Galletti C, and Battaglini PP. Gaze-dependent visual neurons in area V3A of monkey prestriate
cortex. J Neurosci 9: 1112-1125, 1989.
Gnadt JW. Area LIP: Three dimensional space and visual to oculomotor transformation. Behav Brain
Sci 15: 745-746, 1992.
Gnadt JW, and Andersen RA. Memory related motor planning activity in posterior parietal cortex of
macaque. Exp Brain Res 70: 216-220, 1988.
Gnadt JW, and Beyer J. Eye movements in depth: what does the monkey's parietal cortex tell the
superior colliculus? Neuroreport 9: 233-238, 1998.
Gnadt JW, and Mays LE. Depth tuning in area LIP by disparity and accommodative cues. Soc
Neurosci Abstr 17: 1113.
Gnadt JW, and Mays LE. Neurons in monkey parietal LIP are tuned for eye-movement parameters in
three-dimensional space. J Neurophysiol 73: 280-297, 1995.
Gonzalez F, and Perez R. Modulation of cell responses to horizontal disparities by ocular vergence in
the visual cortex of the awake Macaca mulatta monkey. Neurosci Lett 245: 101-104, 1998.
Gottlieb JP, Kusunoki M, and Goldberg ME. The representation of visual salience in monkey
parietal cortex. Nature 391: 481-4, 1998.
31
Graziano MS, Hu XT, and Gross CG. Visuospatial properties of ventral premotor cortex. J
Neurophysiol 77: 2268-2292, 1997.
Grunewald A, Linden JF, and Andersen RA. Responses to auditory stimuli in macaque lateral
intraparietal area. I. Effects of training. J Neurophysiol 82: 330-342, 1999.
Hartline PH., Vimal RL, King AJ, Kurylo DD, and Northmore DP. Effects of eye position on
auditory localization and neural representation of space in superior colliculus of cats. Exp Brain Res
104: 402-408, 1995.
Hasebe H, Oyamada H, Kinomura S, Kawashima R, Ouchi Y, Nobezawa S, Tsukada H,
Yoshikawa E, Ukai K, Takada R, Takagi M, Abe H, Fukuda H, and Bando T. Human cortical
areas activated in relation to vergence eye movements-a PET study. Neuroimage 10: 200-208, 1999.
Hofsten C von. The role of convergence in visual space perception. Vision Res 16: 193-198, 1976.
Jay, MF, and Sparks DL. Sensorimotor integration in the primate superior colliculus. II. Coordinates
of auditory signals. J Neurophysiol 57: 35-55, 1987.
Jaschinski-Kruza W. Fixation disparity at different viewing distances of a visual display unit.
Ophthal Physiol Opt 13: 27-34, 1993.
32
Jaschinski W. Fixation disparity and accommodation for stimuli closer and more distant than
oculomotor tonic positions. Vision Res 41: 923-933, 2001.
Kapoula Z, Isotalo E, Muri RM., Bucci MP, and Rivaud-Pechoux, S. Effects of transcranial
magnetic stimulation of the posterior parietal cortex on saccades and vergence. Neuroreport 12: 40414046, 2001.
Lehky SR, Pouget A, and Sejnowski TJ. Neural models of binocular depth perception. Cold. Spring.
Harb. Symp. Quant Biol 55: 765-777, 1990.
Linden JF, Grunewald A, and Andersen RA. Responses to auditory stimuli in macaque lateral
intraparietal area. II. Behavioral modulation. J Neurophysiol 82: 343-358, 1999.
Maxwell JS, and King WM. Dynamics and efficacy of saccade-facilitated vergence eye movements in
monkeys. J Neurophysiol 68: 1248-1260, 1992.
Mitchel J, and Zipser D. A model of visual-spatial memory across saccades. Vision Res 41: 15751592, 2001.
Motter BC, and Poggio GF. Binocular fixation in the rhesus monkey: spatial and temporal
characteristics. Exp Brain Res 54: 304-14, 1984.
33
Nakamura H, Kuroda T, Wakita M, Kusunoki M, Kato A, Mikami A, Sakata H, and Itoh K.
From three-dimensional space vision to prehensile hand movements: the lateral intraparietal area links
the area V3A and the anterior intraparietal area in macaques. J Neurosci 21: 8174-8187, 2001.
Ogle KN, Martens TG, and Dyer JA. Oculomotor Imbalance in Binocular Vision and Fixation
Disparity. Philadelphia: Lea and Febiger, 1967.
Paré M, and Wurtz RH. Monkey posterior parietal cortex neurons antidromically activated from
superior colliculus. J Neurophysiol 78: 3493-3497, 1997.
Platt ML, and Glimcher PW. Neural correlates of decision variables in parietal cortex. Nature 400:
233-8, 1999.
Poggio GF. Mechanisms of stereopsis in monkey visual cortex. Cereb Cortex 5: 193–204, 1995.
Poggio GF, and Fischer B. Binocular interaction and depth sensitivity in striate and prestriate cortex
of behaving rhesus monkey. J Neurophysiol 40: 1392-405, 1977.
Pouget A, and Sejnowski TJ. A neural model of the cortical representation of egocentric distance.
Cereb Cortex 4: 314-329, 1994.
Pouget A, Deneve S, and Duhamel JR. A computational perspective on the neural basis of
multisensory spatial representations. Nat Rev Neurosci 3: 741-747, 2002.
34
Powell KD, and Goldberg MA. Response of neurons in the lateral intraparietal area to a distractor
flashed during the delay period of a memory-guided saccade. J Neurophysiol 84: 301-310, 2000.
Rizzolatti G, and Luppino G. The cortical motor system. Neuron 31: 889-901, 2001.
Rogers BJ, and Bradshaw MF. Vertical disparities, differential perspective and binocular stereopsis.
Nature 361: 253-255, 1993.
Rosenbluth D, and Allman JM. The effect of gaze angle and fixation distance on the responses of
neurons in V1, V2, and V4. Neuron 33: 143-149, 2002.
Roy JP, Komatsu H, and Wurtz RH. Disparity sensitivity of neurons in monkey extrastriate area
MST. J Neurosci 12: 2478-2492, 1992.
Salinas E, and Thier P. Gain modulation: a major computational principle of the central nervous
system. Neuron 27; 15-21, 2000.
Sakata H, Shibutani H, and Kawano K. Spatial properties of visual fixation neurons in posterior
parietal association cortex of the monkey. J Neurophysiol 43: 1654-1672, 1980.
Sakata H, Shibutani H, Kawano K, and Harrington, TL. Neural mechanisms of space vision in the
parietal association cortex of the monkey. Vision Res 25: 453-463, 1985.
35
Schall JD. Visuomotor areas of the frontal lobe. In: Cerebral Cortex, edited by Rockland K, Kaas JH,
and Peters A, New York: Plenum, 1997, p. 527–638.
Snyder LH, Batista AP, and Andersen, RA. Coding of intention in the posterior parietal cortex.
Nature 386: 167-70, 1997.
Snyder LH, Grieve KL, Brotchie P, and Andersen, RA. Separate body- and world-referenced
representations of visual space in parietal cortex. Nature 394: 887-891, 1998.
Stricanne B, Andersen RA, and Mazzoni P. Eye-centered, head-centered, and intermediate coding of
remembered sound locations in area LIP. J Neurophysiol 76: 2071-2076, 1996.
Stuphorn V, Bauswein E, and Hoffmann KP. Neurons in the primate superior colliculus coding for
arm movements in gaze-related coordinates. J Neurophysiol 83: 1283-1299, 2000.
Trotter Y, Celebrini S, Stricanne B, Thorpe S, and Imbert M. Neural processing of stereopsis as a
function of viewing distance in primate visual cortical area V1. J Neurophysiol 76: 2872-2885, 1996.
Wurtz RH, and Goldberg ME. The Neurobiology of Saccadic Eye Movements, Reviews of
Oculomotor Research, Vol. III. Amsterdam: Elsevier, 1989.
Wurtz RH, Sommer MA, Paré M, and Ferraina S. Signal transformations from cerebral cortex to
superior colliculus for the generation of saccades. Vision Res 41: 3399-3412. 2001.
36
Xing J, and Andersen RA. Models of the posterior parietal cortex which perform multimodal
integration and represent space in several coordinate frames. J Cogn Neurosci 12: 601-614, 2000.
Zhang M, and Barash S. Neuronal switching of sensorimotor transformations for antisaccades.
Nature 408: 971-5, 2000.
Zipser D, and Andersen RA. A back-propagation programmed network that simulates response
properties of a subset of posterior parietal neurons. Nature 331: 679-684, 1988.
37
Figure legends
Figure 1. Sample penetration in the lateral bank of the intraparietal sulcus.
Coronal section with the reconstruction of a representative microelectrode penetration in area LIP, at
the level indicated by the broken line in the brain figurine. Scale bar is 10 mm. AS, arcuate sulcus; CS,
central sulcus; IPS, intraparietal sulcus; STS superior temporal sulcus; CGS cingulate sulcus.
Figure 2. Schematics of the work space arrangement.
(A) Two robot arms have been used to obtain a three-dimensional workspace with the ability to have
the fixation point (FP) and target (T) located at different position in depths. (B) Two dimensional
representations of the eight isovergence surfaces (IVG) used in the experiment. Conjugate eye
movements are obtained when both FP and T are in the same IVG (left part). These movements are
directed to a target located at disparity zero from the FP. The influence of vergence angle on neuronal
modulation has been tested by varying the fixation distance and maintaining the disparity constant. IVS
indicates the hyperboles of equal version, corresponding to the locations where targets maintain the
same retinal eccentricity. Disjunctive eye movements are made when FP and target are in different
IVGs (right part; vectors 2 and 3). These movements are directed to a target located to a retinal
disparity equal to the difference in vergence angle required. When the target is closer than the FP the
obtained movement is a convergence eye movement. A divergence eye movement is obtained when the
target is farther than the FP. (C) Eye record data for conjugate eye movements (1) and disjunctive eye
movements with changes of 2° in divergence (2) or convergence (3) direction. The conjugate eye
position (C; grey lines) and the vergence position (V; black lines) are shown as a function of time and
are aligned with the beginning of the saccadic eye movement (S-on; vertical line). Zero lines and
calibration bars are shown for both conjugate eye position and vergence position.
38
Figure 3. Details of vergence position obtained during conjugate and disjunctive eye movements.
Vergence position data (5 replications each) obtained in one of the two monkeys for conjugate eye
movements (A) and disjunctive eye movements (B). Conjugate eye movements were obtained for each
of the eight IVG (A); disjunctive eye movements were obtained for movements starting from the IVG
corresponding to 10° of vergence angle and directed to peripheral targets requiring changes of the
vergence angle both in convergence or divergence direction (B). The schematic figures in A and B
shows only some of the corresponding vector movements. The vergence position (black lines) is shown
as a function of time and is aligned with the beginning of the saccadic eye movement (S-on; vertical
line). Horizontal line in each plot corresponds to 10° of vergence angle. Other convention and symbols
as in Figure 2.
Figure 4. Fixation disparity as a function of viewing distance.
(A) Scatter plot of the vergence error for each required vergence value. Data from monkey B. (B)
Similar data obtained from monkey L. In both cases, the regression line is indicated together with the
slope value.
Figure 5. Response properties of a typical neuron modulated both by changes in the fixation distance
and disparity.
(A) Neural response during conjugate eye movements made starting from different fixation distances,
corresponding to different vergence angle values (from 13° near to 6° far), and directed to targets
located at the identical retinal eccentricity, as schematized in the left figurine. The right part of the
figure shows the corresponding fixation distance tuning during the delay epoch. Mean ± SE discharge
rate are shown for each average value of vergence, obtained after correction for vergence error
39
(fixation disparity) of the single trials. (B) For the same neuron as A, neural response during
disjunctive eye movement starting from the fixation distance corresponding to 10° of vergence angle
and directed to targets located at the identical retinal eccentricity but different disparity. The right part
of the figure shows (black dots) the corresponding disparity tuning during the delay epoch. Mean ± SE
discharge rate are shown for each average value of disparity, obtained after correction for the fixation
disparity of the single trials. White dots report, for comparison, the average discharge rate observed
during conjugate eye movements vs. the corresponding retinal disparity introduced by fixation distance
changes. The rasters are aligned to the target presentation (T-on) or the fixation point off (FP-off).
Thick bar indicates the delay epoch (DLY) used in the analysis, arrow indicates the approximate
beginning of eye movement. Scale bar is 100 ms.
Figure 6. LIP neurons that had significant changes in activity during the delayed-saccade tasks in
relation to vergence angle or disparity changes.
Bar graph showing the percent of neurons significantly modulated by different signals during the delay
epoch of the task. VA, neurons modulated only by vergence angle changes; DISP, neurons modulated
only by disparity changes; VA + DISP, neurons modulated by both signals.
Figure 7. Linear and cubic models for data obtained during conjugate and disjunctive eye movements.
(A-B) Scatter plot of the average neural activity during the delay epoch of the task vs. the different
vergence angles (A) and disparities (B). Black lines in A and B indicate the linear and cubic regression
line respectively. Data are for the same convergence neuron of Figure 5. (C) Example of calculation of
the Zero Index (see methods for details) from the curve obtained in Figure 7B. (D-F) Fitting data and
calculation of Zero Index shown for a divergence neuron. Other convention and symbols as in Figure 5.
40
Figure 8. Distribution of fixation distance and disparity modulation indices.
(A) Slope index for the fixation distance modulated neurons fitted to the linear model. Positive values
indicate neurons that linearly increase their discharge rate toward near fixation distances. (B) Zero
Index for the disparity modulated neurons fitted to the cubic model. Positive values correspond to
neurons preferring positive disparity.
Figure 9. Fixation distance-modulation during conjugate saccades correlated to the modulation
obtained for disjunctive movements.
(A) Scatter plot of the Zero Index values obtained from the cubic model fitting to disparity modulation
vs. the slope values obtained from the linear regression model fitted to neural activities during
conjugate eye movements. Data are for those neurons modulated by both disparity and vergence
signals. Interrupted lines separate positive from negative values. A positive slope indicates the presence
of a linear trend for near fixation distances. A positive Zero Index indicates a preference for positive
disparities. The values of R2 and the function for the linear regression fitting the distribution are
indicated. If the results were all explained by retinal disparity, these data should lie around a line with a
slope of about 20 [1/(average slope of fixation disparity curves]. (B) Scatter plot of the modulation
extracted from the linear regression model for 7° of fixation distance changes vs the fixation disparity
modulation estimated from the disparity tuning in the range of 0.5°. The linear regressions used for the
calculation of the fixation disparity slope were all passing the Goodness-of-fit-test for a linear model (p
> 0.05).
Figure 10. Idealized disparity-tuned neurons that code for egocentric distance of the targets.
(A) Example of targets located at the same retinal eccentricity and at different distances from the
subject. The disparity value of the targets changes when the FP moves between different fixation
41
distances (F4 near to F7 far) and becomes more positive as the vergence angle decreases. (B) Grey and
black lines indicate, for each fixation distance (F4 to F7), the disparity-tuning curve for an idealized
divergence and an idealized convergence neuron respectively. Horizontal line indicates how the same
value in neural activity, after vergence angle changes, corresponds to different disparity values. (C)
Data displayed as function of the distance of the targets from the body. Because the neuron is coding
for distance, the different curves overlap on a single curve.
Figure 11. Fixation distance-modulation of neural activity.
(A-C) Each panel displays for two convergence and one divergence neurons the average discharge rate
for each value of pure disparity (after correction for fixation disparity) and the corresponding tuning
curves obtained at different fixation distances by fitting the data, obtained in each single trial, to a cubic
model. The code for each fixation distance is indicated.
Figure 12. Different contribution of shift and gain effects in the interaction between disparity and
vergence angle.
(A) Scatter plot of the adjusted R2 values obtained by fitting the data to the shift and the gain model
separately. The line indicates the equality line. (B) Distribution of the p values for the parameters s and
g obtained by fitting the data to the combined model. The smaller the value of p, the less likely the
parameter is actually zero, in other words p is a measure of the possibility to remove the parameter
from the model without affecting the regression accuracy.
Figure 13. Fixation distance-modulation of disparity tuning. Regression model for vergence angle and
disparity tuning interaction.
42
(A-C) Three-dimensional color plots of the model fitting disparity and vergence angle interaction for
the same two convergence neurons and the divergence neuron of Figure 11.
The slope in the
isodischarge regions is indicative of the shift value obtained. Other conventions and symbols as in
Figure 11.
Figure 14. Distribution of the shift values obtained by fitting the data with the non-linear model.
(A) Shift values sorted by the R2 of the fitting. (B) Scatter plot of the R2 obtained from those neurons
modulated by disparity both in the visual and in the delay epochs. Interrupted line indicates the equality
line.
Table 1. Number of neurons modulated for disparity and vergence angle for each analysis epoch.
___________________________________________________________________________________
Epochs
__________________________________________________
VIS
DLY
SAC
___________________________________________________________________________________
Vergence
Disparity
Vergence &
Disparity
Anova
51/152 (33.6%)
45/152 (29.6%)
22/152 (14.5%)
Linear regression
42/51 (82.4%)
34/45 (75.6%)
19/22 (86.4%)
Anova
109/152 (71.7%)
93/152 (61.2%)
90/152 (59.2%)
Cubic regression
75/109 (68.8%)
67/93 (72%)
66/90 (73.3%)
36/152 (23.7%)
38/152 (25%)
19/152 (12.5%)
AS
CGS
CS
IPS
STS
IPS
Genovesio and Ferraina, Fig 1
A
T
FP
B
FP
T
IVG
IVG
1
IVS
2
T
IVS
FP
3
C
C
1
2
3
0
10°
V
0
10°
100ms
Genovesio and Ferraina, Fig 2
A
B
FP
T
T
IVG
IVG
IVS
IVS
15
100 ms
10
Verg angle
Verg angle
FP
15
10
5
S-on
5
S-on
Genovesio and Ferraina, Fig 3
B
60
Fixation disparity (min arc)
Fixation disparity (min arc)
A
Slope = 2.4
40
20
0
-20
-40
-60
6
7
8
9
10 11 12 13
vergence angle (°)
60
Slope = 3.5
40
20
0
-20
-40
-60
6
7
8
9
10 11 12 13
vergence angle (°)
Genovesio and Ferraina, Fig 4
T-on
40
6
6°
Discharge rate (sp/s)
35
7
13°
Vergence angle (°)
A
FP-off
8
9
10
11
30
25
20
15
10
5
12
0
6
Conjugate
eye movements
100 ms
T-on
10
11
12
13
FP-off
40
-3
35
Disparity (°)
-2
-1
0
1
2
Disjunctive
eye movements
9
Vergence angle (°)
6°
13°
8
DLY
Discharge rate (sp/s)
B
7
13
30
25
20
15
10
5
0
3
-3
100 ms
DLY
-2
-1
0
1
2
3
Disparity (°)
Genovesio and Ferraina, Fig 5
60
% of Neurons
n=100
50
40
30
20
10
VA
VA + DISP
DISP
Genovesio and Ferraina, Fig 6
Discharge rate (sp/s)
13°
30
13°
20
10
6
7
8
E
7°
Discharge rate (sp/s)
50
0°
30
20
13°
0°
2
3
5
4
6
9 10 11 12 13
R2=0.7
50
40
30
20
DZERO
n
3
4
5
6
Disparity (°)
7
Disparity (°)
60
Discharge rate (sp/s)
DMAX
30
2
8
-7 -6 -5 -4 -3 -2 -1 0
Disjunctive
eye movements
Divergence
ZI= 6.57
40
}
Discharge rate (sp/s)
7
F
50
1
7
50
DMAX
ZI= -7.93
40
30
20
10
DZERO
n
}
1
Disparity (°)
0
10
10
0
10
20
60
-7°
R =0.5
Disjunctive
eye movements
Convergence
20
30
Vergence angle (°)
10
C
40
6
2
40
R2=0.5
Slope= -2.50
50
Conjugate
eye movements
9 10 11 12 13
Vergence angle (°)
6°
60
6°
40
Conjugate
eye movements
B
D
R2=0.6
Slope= 3.66
Discharge rate (sp/s)
50
6°
Discharge rate (sp/s)
A
-7 -6 -5 -4 -3 -2 -1 0
Disparity (°)
Genovesio and Ferraina, Fig 7
n = 32
6
5
4
3
2
B
12
n = 67
10
8
6
4
2
1
0
Slope (sp/°)
16
12
8
4
0
-4
-8
0
-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6
-12
7
14
A
-16
Number of neurons
8
Number of neurons
9
Zero Index (sp/°)
Genovesio and Ferraina, Fig 8
A
B
n = 32
Zero Index (sp/°)
10
5
0
2
R = 0.6
ZI = 1.77(Slope) - 1.28
-5
-10
-15
-15
-10
-5
0
5
Slope (sp/°)
10
15
Fixation Distance Modulation (sp/s)
15
45
n = 32
35
25
15
5
5
15
25
35
Fixation Disparity Modulation (sp/s)
Genovesio and Ferraina, Fig 9
45
A
-4
-3
F5
F4
+3
-1
F7
+5
+4
B
+6
C
70
F7
F4
70
F7
F4
Discharge rate (sp/s)
Discharge rate (sp/s)
-2
F6
60
50
40
30
20
10
F7
F4
F4
F7
60
50
40
30
20
10
0
0
-6
-4
-2
0
Disparity (°)
2
4
6
0
100
200
300
Distance (mm)
Genovesio and Ferraina, Fig 10
A
40
13°
11°
10°
9°
8°
6°
Discharge rate (sp/s)
35
30
25
20
15
10
5
0
-4
-3
-2
-1
0
1
2
3
4
2
3
4
2
3
4
Disparity (°)
B
Discharge rate (sp/s)
80
70
60
50
40
30
20
10
0
-4
-3
-2
-1
0
1
Disparity (°)
C
Discharge rate (sp/s)
50
45
40
35
30
25
20
15
10
5
0
-4
-3
-2
-1
0
1
Disparity (°)
Genovesio and Ferraina, Fig 11
A
1
B
0.05
n = 19
0.04
p shift
R2a shift
0.8
0.6
0.03
0.4
0.02
0.2
0.01
n = 19
0.2
0.4
2
0.6
R a gain
0.8
1
0.2
0.4
0.6
0.8
1
p gain
Genovesio and Ferraina, Fig 12
A
Discharge rate (sp/s)
40
30
30
20
10
0
10
13 °)
12 le (
10 ang
8
e
6 genc
r
Ve
40
20
4
2 3
0 1
1
-2
rity (°)
-4 -3
Dispa
R2=0.6
Shift=0.65
B
Discharge rate (sp/s)
80
60
60
40
20
0
20
13 °)
12 le (
10 ang
8
e
6 genc
r
Ve
80
40
3 4
1 2
0
-2 -1 rity (°)
-4 -3
Dispa
R2=0.8
Shift=0.28
Discharge rate (sp/s)
C
50
30
20
10
0
30
20
10
13 °)
12 le (
10 ang
8
e
6 genc
r
Ve
40
40
3 4
1 2
0
1
-2 rity (°)
-4 -3
Dispa 2
R =0.6
Shift=0.35
Genovesio and Ferraina, Fig 13
B
1
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
n = 28
n = 23
0.8
0.6
0.4
2
R delay epoch
R
2
A
0.2
0
-1.5
-1
-0.5
0
shift
0.5
1
1.5
0
0.2
0.4
0.6
0.8
1
2
R visual epoch
Genovesio and Ferraina, Fig 14