Download The Biological Bases of Time-to

Time-to-Contact, Advances in Psychology series
Heiko Hecht & Geert J. P. Savelsbergh (Eds.)
2003 Amsterdam: Elsevier - North-Holland
CHAPTER 2:
The Biological Bases of Time-to-collision
Computation
Barrie J. Frost
Queen’s University, Kingston, Ontario, Canada
Hongjin Sun
McMaster University, Hamilton, Ontario, Canada
ABSTRACT
We begin the chapter by arguing that there may be several neural mechanisms
that have evolved for computing time-to-collision (TTC) information as a way
of controlling different classes of action. We then focus on single unit mechanisms responsible for processing the impending collision of a moving object
towards a stationary observer. After discussing TTC processing in the invertebrate visual system, we describe our own work involving neurons in the pigeon
nucleus rotundus that respond exclusively to visual information relating to objects that are approaching on a direct collision course, but not to visual information simulating movement towards those same stationary objects. Based on the
recorded neuronal responses to various manipulations of the stimuli, we classified these looming sensitive neurons into three different types of looming detectors based on the temporal differences in neuronal responding relative to the
moment of collision. We also described quantitative models for these looming
detectors as a way of explaining their physiological response properties.
14
Barrie J. Frost and Hongjin Sun
1. Introduction
Information about the time-to-collision or time-to-contact (TTC) has
important consequences for the survival of countless species and for their skilled
interaction with both the inanimate and animate objects in their environments.
As a consequence it appears very probable that there may be several neural
mechanisms that have evolved to compute TTC information to control different
classes of action, and even different mechanisms within the same animal for
different functions. For example, it appears unlikely that mechanisms that have
evolved in birds for avoidance of rapidly approaching objects such as predators,
where critical and rapid evasive maneuvers are required, are the same mechanisms that control pinpoint landing on branches. In the former case the motion of
the approaching predator will primarily determine TTC, whereas in the latter it
is only the self-motion of the animal approaching the stationary branch that determines TTC. Of course there may be many instances where both self-motion
and motion of another animal determine TTC. For an excellent review that puts
TTC in a much broader context the reader is referred to a paper by Cutting,
Vishton and Braren (1995).
One way that may help conceptualize these factors is to subdivide the
primary stimulus determinants of TTC on the one hand, and the general nature
of responses controlled by the information on the other, and place these in a
simple 2 x 2 table as illustrated in Table 1. Here we have divided the world simply into stationary and moving objects on the vertical axis, and
Source of Looming
Stimulus
Behavioural Output
Self-motion towards
stationary objects
1
•
•
•
Insect’s landing
Bird’s landing
Human or gerbil’s
approaching towards
target
2
•
•
Avoiding obstacles
Avoiding cliffs and
drop offs
Moving objects (towards and away from
the observer)
3
•
•
•
•
Pursuit – prey capture
Pursuing mates
Flock formation
Ball catching
4
•
•
Predator avoidance
Avoiding aggressive
encounters
Approach
Avoidance
Table 1: Situations Requiring TTC Information
the behavioural responses into approach and avoidance on the other. Examples of TTC studies falling in cell 1 (Stationary objects/Approach behaviour) are the landing response of the milkweed bug, Oncopeltus fas-
The Biological Bases of Time-to-collision Computation
15
ciatus (Coggshall, 1972) and the fly (Wagner, 1982). The aerodynamic
folding of gannet wings just prior to their entry into water (Lee & Reddish, 1981), birds landing on stationary perches (Lee, Davies, Green &
Van der Weel, 1993) or human subjects braking to avoid collision with
stationary barriers (Sun & Frost, 1997) or gerbil behaviour of running towards
target (Sun, Carey & Goodale, 1992) are other examples that fall into this category.
Examples of behaviour falling in cell 2 (Stationary objects/Avoidance)
would involve negotiating paths through a cluttered environment where obstacles have to be avoided. This might include steering around barriers, and avoiding holes or sudden drop offs. There appear to be few studies of TTC detection
in this category, but Cutting et al.’s (1995) study of path interceptions may be
relevant.
Prey capture by predators and pursuit chasing during mating could well
satisfy entry into cell 3 (Moving objects/Approach), although not all studies of
this behaviour have focused on TTC information. Ball catching behaviour, and
batting in sports seem also to be appropriate exemplars of this category. Escape
from rapidly approaching predators or threatening rivals in territorial mating
would be prime example of cell 4 (Moving objects/Avoidance).
Throughout the animal kingdom the sight of a rapidly approaching object almost universally signals danger and elicits an escape or avoidance response. When confronted with such a looming stimulus, the visual system must
determine precisely the 3D flight path, and compute the TTC of the object, to
provide the information necessary for eliciting and controlling the appropriate
evasive action (Fishman & Tallaroco, 1961; Schiff et al., 1962; Schiff, 1965;
Hayes & Saiff, 1967; Tronick, 1967; Bower et al., 1970; Ball & Tronick, 1971;
Dill, 1974; Ingle & Shook, 1983; Yonas & Granrud, 1985). Our own work on
neurons in the pigeon nucleus rotundus of pigeons clearly fits in this category
because these neurons respond only to the direct collision course of approaching
objects (Wang & Frost, 1992, Sun & Frost, 1998), and not to simulation of the
movement of pigeons toward the same stationary objects (Sun & Frost, submitted). Also the work on locust looming detectors would fit this category because
of the demonstrated elicitation of jumping and flying by the same expanding
stimulus patterns that optimally excites the Lobula Giant Movement Detector
(LGMD) and the Descending Contralateral Movement detector (DCMD) neurons (Rind & Simmons, 1992, 1999; Hatsopoulos, Gabbiani, Laurent, 1995).
Of course it should be remembered that the necessity to compute TTC
first requires that any object or surface be indeed on a collision course if the
present path of the observing or approaching animal is maintained. Gibson
(1979) in his classic work on ecological optics suggests that symmetrical expansion of the images of objects specifies direct approach along a course that will
ultimately result in collision with continuous motion. The advantage of using
16
Barrie J. Frost and Hongjin Sun
this strategy is that it can be computed using monocular information alone. It is
possible that animals with well-developed binocular stereoscopic visual systems,
and subpopulations of neurons that respond to stereoscopic motion directions
specifying object-motion paths directly toward the animal, might also be used
for TTC computations. In this work and his other writings, Gibson also made the
clear distinction between collisions with stationary objects occasioned by the
motion of the observer (row 1 in Table 1) and other cases where it is the approaching object’s motion itself that will result in collision if it continues along
this path (row 2 in Table 1).
In this chapter we will focus primarily on research that falls in cell 4
simply because it appears that most of the empirical studies about neural mechanisms of TTC have used stimulus arrangements that simulate events that fall into
this category, that is, a rapidly approaching object on a direct collision course
with the observing animal, and which might therefore require some sort of evasive action or avoidance response on the part of the animal. In the other category
of object motion where the observing animal is trying to arrange a collision with
the moving object such as the prey capture behaviour of dragonflies (Olberg,
Worthington & Venator, 2000), similar processing mechanisms may occur.
2. TTC in the invertebrate visual system
Flying insects have long been used as model systems because they exhibit spectacular aerial performance and accomplish this with relatively simple
neural computational mechanisms. Moreover since the same neurons can be
identified from animal to animal the neural circuitry is often amenable to analysis. Two such neurons, LGMD and DCMD in the locust, that are synaptically
linked, have been shown to be selectively responsive to approaching objects
(Rind & Simmons, 1992; Rind, 1997; Hatsopoulos et al., 1995). Neurons that
respond to changes in depth have also been found in optic lobes of the hawk
moth, Manduca sexta (Wicklein & Strausfeld, 2000), but these may be examples
of neurons computing approach and recession for the control of self motion, in
this case controlling the hovering flight in front of flowers during nectar collection, rather than for the computation of TTC.
Because the DCMD neurons can be readily recorded extracellularly,
have very large receptive fields, and respond well to the movement of objects,
they have been studied extensively for many years. Schlotterer (1977) was the
first to use approaching stimuli to show that DCMD neurons were more responsive to approaching objects than other 2D patterns of movement. Rind and her
colleagues, and Laurent and his colleagues have extensively studied these neurons using a variety of stimuli and confirmed that symmetrical expansion gener-
The Biological Bases of Time-to-collision Computation
17
ated by an approaching stimulus object is the critical stimulus variable that optimally fires these cells. The allocation of the LGMD - DCMD neurons to cell 4
of our schema presented in Table 1 is justified by their connection to pre-motor
interneurons and motor-neurons known to be involved in flying and jumping
(Burrows & Rowell, 1973; Pearson et al., 1980; Simmons, 1980). This is further
supported by the studies of Robertson and Johnson (1993a, 1993b) who have
shown in tethered, flying locusts, that approaching objects elicit a steering
avoidance response when the approaching object reaches a critical angular size,
thus indicating that some thresholding probably occurs in this pathway.
What are the critical features of a symmetrically expanding image that
these locust DCMD neurons are responding to that generates their specificity to
approaching objects? From an analysis of the several possible cues available in
the monocular image Rind and her colleagues (Simmons & Rind, 1992; Rind &
Simmons, 1992) have shown that these neurons do not register changes in overall luminance since they respond in a similar manner when light objects approach as when dark objects approach, and their responses were much smaller to
sudden luminance change per se. Divergence of two lines moving in opposite
directions, to partially represent the opposite contours of a symmetrically expanding object, also did not adequately stimulate DCMD neurons, but increasing
the amount of edges in the Receptive Field (RF) and increasing the velocity of
edges appeared to be the critical trigger features. Judge and Rind (1997; see also
Rind & Simmons 1999) have shown that these locust looming sensitive neurons
are very tightly tuned to the direct collision course.
Stimulation of the locust retina in one area suppresses LGMD response
to a second stimulus presented elsewhere in the visual field, thus indicating there
are lateral inhibitory mechanisms operating. Indeed if the appropriate experiments were to be performed one might well find the RF characteristics are similar to those found in the tectofugal or collicular-pulvinar pathway of vertebrates
where a directionally specific, double opponent RF organization occurs to ensure that these cells respond to moving objects, and not to the large patterns of
optic flow produced by the animal's self motion (Frost 1978; Frost, Scilley &
Wong, 1981; Frost, Cavanagh & Morgan, 1988, Sun, Zhao, Southall & Xu,
2002). From a functional point of view this also implies that the LGMD neurons
might be interested in approaching objects that fall into cell 4 of our matrix, and
not in the locust's approach toward stationary features in its environment.
According to Rind and Simmons (1999) the specificity of the LGMD for
approaching images is generated by a “critical race over the dendrites of the
LGMD in the optic lobe”. The two competitive forces in the race are the excitation produced by the moving edges of the expanding image, and lateral inhibition mediated by neurons in the medulla also synapse onto the LGMD. Rind and
Bramwell (1996) have produced a neural network model which seems to support
this view and have also shown through electron microscopy that the anatomical
18
Barrie J. Frost and Hongjin Sun
arrangement of presynaptic connections to LGMD are compatible with this interpretation.
Hatopoulus, Gabbiani and Laurent (1995) have also investigated the
LGMD of locusts, and shown that this neuron fires with an increasing rate as an
object approaches, then peaks, and drops off just before collision occurs. They
have shown that the responses are typically brisker for fast moving or smaller
objects, but the peak firing rate does not appears to solely depend on the approach speed or object size. They describe the peak as always exhibiting a constant latency after the time at which the object reaches a fixed angular threshold
size on the eye (Gabbiani, et al, 1999). These authors have suggested that the
behaviour of the LGMD is best described by the following equation:
f(t) = C × θ ′(t ) × e-αθ (t)
(1)
Here, θ is visual angular subtense, C is a proportionality constant, and is
a constant for a particular neuron. In contrast to Rind and her associates view,
these authors in recent papers (Gabbiani, et al, 2001; Gabbiani et al., 2002) have
suggested that the LGMD post-synaptically multiplies an excitatory and inhibitory input via two different parts of LGMD neuron’s dendritic tree. In order to
provide evidence in support of this model these authors (Gabbiani et al., 2002)
have selectively activated and deactivated pre-and post synaptic inhibition, and
have found that it is post-synaptic inhibition that plays a critical role in shaping
the temporal response profile of these neurons, and this indicates that the multiplication takes place within the LGMD neuron itself. These findings are noteworthy for two reasons: in the first place they show in a detailed way how these
computations which provide information about looming object are accomplished
within the neural machinery of the LGMD and its synaptic connections, and
secondly they provide one of the first pieces of clear evidence for how multiplication (and division) is accomplished in the nervous system.
3. Neurons that compute tau in the pigeon brain
A number of behavioral studies have revealed that the tectofugal pathway in vertebrates might be involved in processing the visual information necessary for generating such escape or avoidance action. Electrical stimulation experiments indicated that the anuran optic tectum is involved in triggering both
prey-catching and also various kinds of avoidance behaviours (Ewert, 1984), and
ablation of the optic tectum resulted in abolition of all visually guided preycatching and visual avoidance behaviour (Bechterew, 1984, cf: GrüsserCornehls, 1984). Electrical or chemical stimulation of the superior colliculus in
The Biological Bases of Time-to-collision Computation
19
rats also results in defensive-like reactions (Redgrave et al., 1981; Sahibzada et
al., 1986; Dean et al., 1988) and is associated with large increases in blood pressure and heart rate (Keay et al., 1988). Pigeons with bilateral lesions of the optic
tectum or/and the nucleus rotundus not only showed substantial impairment in
intensity, colour, and pattern discrimination, but also exhibited severe deficits in
visually guided orientation, escape or avoidance behaviour (Hodos & Karten,
1966; Hodos, 1969; Hodos & Bonbright, 1974; Jarvis, 1974; Bessette & Hodos,
1989). Wild rats with collicular lesions may ignore an approaching human
(Blanchard et al., 1981) and similar results have been reported in hamsters and
gerbils (Ellard & Goodale, 1986; Northmore et al., 1988). This evidence provides a vivid illustration of the importance of the tectofugal pathway in guiding
orientation, detecting approaching objects, and generating escape or avoidance
behaviours.
Over the years several investigators have claimed that they have encountered cells that respond specifically to objects approaching the eye on a direct
collision course. For example, as early as 1976 Grüsser and Grüsser-Cornehls
(1976) and later Ewert (1984) reported that some frog and toad tectal neurons
respond vigorously to stimuli moving on paths directly towards the eye. However from these early studies many of the appropriate controls were not performed to conclusively exclude the possibility that these neurons were not simply responding to some aspect of the lateral motion of an approaching stimulus.
It should be remembered that as an approaching image expands, obviously there
will be 2D motion of the edges of the object and its textures and if the expansion
is placed asymmetrically over a standard 2D directionally specific motion it
could artifactually stimulate the neuron to give a false impression it is responsive
to approaching stimuli. We also had encountered cells we thought were responding to the direct approach path of moving objects in 1983, but it was only when
we had extremely well-controlled stimuli, which we could systematically vary in
their simulated 3D paths, that we could finally convince ourselves that these
neurons were indeed coding some aspect of 3D motion.
In 1992 Wang and Frost showed that some neurons located in the dorsal
posterior regions of the nucleus rotundus of pigeons responded specifically to
the direct approach direction of a soccer ball pattern. Using the 3D imaging capabilities of a Silicon Graphics computer we were able to move this soccer-ball
stimulus in any trajectory in 3D space. By performing very time consuming 3D
tuning curves on these cells we were able to show that this rotundal subpopulation would only respond when the soccer ball stimulus was on a direct
collision course with the bird’s head. We chose a soccer ball because the spaceaverage mean luminance did not change as this stimulus expanded and contracted in size (especially when the object moved against a stationary background with the same texture pattern), and it provided many moving and expanding/contracting elements that might be necessary for these neurons to re-
20
Barrie J. Frost and Hongjin Sun
spond. Earlier studies, and often some more recent ones, use a simple expanding
square or circle where changes in luminance obviously occur concurrently with
the expansion/contraction of stimuli, and this necessitates several other controls
to rule out this variable as the major contributor to the responsiveness of the
neuron. Also other studies have not specifically performed 3D tuning curves to
quantify the true directional tuning of neurons of this type.
Figure 1 shows the typical 3D tuning curves of one of these rotundal neurons.
Figure 1: A. A soccer-ball-like stimulus pattern consisting of black and white panels, was
moved along simulated 3D trajectories 45° apart in spherical coordinates. The diagram illustrates the 4 planes along which stimuli were moved. B. A typical single neuron from the
nucleus rotundus of pigeons exhibiting clear selectivity for a looming visual stimulus. Firing rate is plotted for the different directions of motion of the soccer-ball stimulus in 3D
space. Each direction of motion was presented 5 times in a randomly interleaved sequence,
and the values plotted represent the mean number of spikes for each 3D direction. Note
that in the standard X-Y (tangent screen plane, or Azimuth = 90°) plot, there is no indication of directional preference, and firing rate is quite low. However, for the 0°azimuthal
plane (Z-axis) there is a strong preference for stimuli directly approaching the bird (0°).
Polar tuning plots for directions specifying the azimuthal 45° and 135° planes likewise
show no strong preference for any direction. Thus it is only the direct collision course or
looming direction that produces an increased response in these neurons, and this pattern of
activity was typical of the 27 neurons studied in the dorsal posterior area of the nucleus.
The Biological Bases of Time-to-collision Computation
21
Even with 26 directions, 45 degrees apart these are relatively crudetuning
curves, so in a few cases we have used a much narrower range of directions after
having first performed the broad 3D tuning and found them to be very tightly
tuned indeed (Sun & Frost, submitted). In fact the half-width and half-height of
detailed tuning curves like those illustrated in Figure 2 is about 3 degrees, where
the rotation is around a point halfway along the
Figure 2: Fine tuning of two cells located in the pigeon dorsal nucleus rotundus. First
these cells were presented with a soccer ball stimulus that moved in 26 directions 45 degrees apart in 3D space, and they only responded to the direct collision course direction.
The graphs shown here are the fine grained tuning curves and show that when the soccer
ball, which traveled along a simulated path of 15 meters, was rotated by small amounts
each time passing through the center of the path, the cells reduced their firing substantially.
The few degrees of rotation of the path indicate that now the soccer ball would travel in a
“near miss” and not collide with the bird.
simulated 15 meter path taken by the approaching stimulus. This means in simple terms that a stimulus that depicts a very “near miss” of the bird’s head will
only fire the neurons minimally, and one that is a clear “miss” will not evoke
any response at all.
Perhaps the most important defining character of these neurons’ responses, in addition to their sensitivity to the direct collision course direction,
was the constant time they fired before collision, irrespective of the size of the
simulated approaching soccer-ball, or of its approach velocity (Wang & Frost,
1992). This indicated to us that these neurons might well be computing the opti-
22
Barrie J. Frost and Hongjin Sun
cal variable tau that had been suggested by Lee (1976) to provide important
information about the TTC with approaching objects.
In our original paper (Wang & Frost, 1992) we reported that there were
a variety of times before collision that the population of neurons exhibiting these
characteristics showed. This can be seen in Figure 3A.
Figure 3: A. Distribution of different response onset time for 27 looming cells from the
dorsal posterior zone of nucleus rotundus of pigeons. Although different cells exhibit different values of TTC, individual cells (B) show remarkable consistency even when velocity or size of stimulus is varied.
But for a particular neuron, the variation in its response onset was remarkably
constant (see Figure 3B) on repeated trials and with stimuli of different sizes and
velocities. This variation in the population is precisely what is needed if other
factors, such as recognition of what the incoming object is, can jointly influence
the time selected to perform escape responses of different sorts, each of which
may have characteristic time requirements for their optimal deployment.
The Biological Bases of Time-to-collision Computation
23
Clearly the characteristics of these rotundal neurons suggested to us that
they might be involved among other things in predator avoidance and thus fall in
to cell 4 of our classification system shown in Table 1. To provide some evidence for this we tested a few birds under a lightened anesthesia at the conclusion of their recording session. Under deep anesthesia no electromyelograhic
signals (EMGs) are obtained from animals, but as the anesthetic is lightened and
clearly before any pain stimuli can be experienced, it is possible to obtain good
clear muscle responses. These responses do not result in any overt movement of
the animal, but can be very useful in indicating what major muscle groups might
be involved in a response system normally associated with a stimulus. In this
case we recorded from the large pectoralis muscles that power the wings for
flight. When we presented the approaching soccer-ball stimulus under these
conditions we found that first the tau cells responded with their characteristic
maintained burst of firing, then the pectoralis EMGs occurred some 200 milliseconds later, and then finally the heart rate went more slowly up to levels near
300% of the resting rate. These responses again were incredibly specific, and
only occurred when the soccer-ball was on a direct collision course with the
bird. Near misses and directions 180 degrees away showed no increased EMGs
or increases in heart rate. Data typical of these experimental findings can be seen
in Figure 4. Although only correlative, we feel this constellation of activity in
these tau neurons, and the increased wing EMG and heart rate are indicative of a
flight response elicited by the rapidly approaching ball.
In a more detailed recent study Sun and Frost (1998) have again confirmed the presence of a population of neurons in nucleus rotundus of pigeons
that only respond when the soccer-ball stimulus is on a direct collision course
with the bird’s head. Additionally, we found that these neurons only responded
when our computer simulated the approach of a moving object towards the bird
(stimuli falling into cell 4 of the matrix), and not when the complex stimulus
pattern was configured to simulate the bird moving towards the same stationary
soccer-ball (stimuli falling into cell 1, or possibly 2 of the matrix) (Sun & Frost,
submitted).
24
Barrie J. Frost and Hongjin Sun
Figure 4: Heart rate and pectoralis muscle EMGs recorded concurrently with single cell
response rate from a looming selective rotundal neuron. Note that the "looming cell" begins firing first, then the muscle response occurs, and then heart rate increases dramatically
when the soccer-ball looms toward the bird (A). B. No responses occur when the ball
moves along the same path but in the opposite direction directly away from the bird. Bars
under the visual response histograms indicate the duration of the visual stimulus. Data collection for the looming-selective neuron was terminated with stimulus offset. The neuronal
data and EMGs represent the summed activity over 5 sweeps of the stimulus whereas the
heart rate data represent the means and standard errors for the same 5 sweeps. Simulated
size of stimulus was 30 cm, path length 15 m, and velocity 375 m/s.
To do this we placed the soccer ball against a stationary background, which consisted of a checkerboard pattern. When the soccer-ball was moved in a trajectory
towards the bird (symmetrical expansion) while the background remained stationary, the neurons responded in the typical way and identically to the case
where no background was present. However, when the background was moved
along the same trajectory as the soccer-ball, so as to show a similar but delayed
expanding pattern, the neurons did not respond. This latter configuration formed
the precise simulation of the bird approaching a stationary soccer-ball that remained a constant distance in front of the background “brick wall”. The stimulus
conditions simulating a soccer ball approaching the bird, and the bird approaching a stationary soccer ball are shown in Figure 5.
The Biological Bases of Time-to-collision Computation
25
Figure 5: Schematic diagram of stimulus conditions presented to pigeons. The left portion
of the figure illustrates the kind of object movement and its background in the simulated
display, while the right portion represents the image change on the screen for the corresponding simulation illustrated on the left. In A, the soccer ball object is presented against
a blank background, and the “path” simulates direct approach toward the bird. In B, the
same looming object is presented against a stationary textured checkerboard background.
In C, both object and textured background move (at the same speed) toward the bird,
which simulates the bird’s self-motion toward a stationary soccer ball. Note that the expansion pattern of motion of the object is identical in the three conditions.
It must be emphasized that the expansion pattern of the soccer-ball was identical
in these to two cases of moving object and moving bird simulations, yet in the
former the cells responded vigorously, while in the latter they were essentially
silent. Figure 6 shows the responses of a tau neuron to an approaching soccerball, and also a simulation of the bird approaching a stationary soccer-ball where
the rate of expansion is identical in both cases.
26
Barrie J. Frost and Hongjin Sun
Figure 6: Comparison of the responses Peri-Stimulus Time Histograms (PSTHs) for a single tau neuron to a series of stimuli (soccer-ball) of varying sizes swept along the direct
collision course path toward the bird. Responses are the sum of 5 sweeps and are referenced to time zero, which is the time when the stimulus would have contacted the bird.
The looming object was presented against a white non-textured background (A), a stationary textured checkerboard background (object-motion) (B), and a looming background
moving at the same speed behind the object (C), as shown in Figure 5. The latter condition
simulated the approach of the animal toward the stationary ball and background (selfinduced motion). Responses were similar to those produced by the looming object against
a blank background and a stationary checkerboard background. The magnitude of responses (maximal firing rate) were similar across different object sizes. Note that the neuron did not fire to the self-motion display, even though the soccer ball’s image was expanding in the same way in B and C. This implies this tau neuron is exclusively selective
for “object motion in depth”. The simulated path for the object was 15 m in length and the
simulated object size varied from a diameter of 10 cm to 50 cm. Velocity was 375 cm/s.
The Biological Bases of Time-to-collision Computation
27
We have also found that not all of the neurons in nucleus rotundus appear to be
computing the tau function (Sun & Frost, 1998). Histological examination indicated that those neurons were distributed in a larger anatomical region (dorsal
rotundus) as opposed to dorsal posterior rotundus in our earlier discovery by
Wang and Frost (1992). In fact, half of the neurons seem to be clearly responding in this fashion, that is, they suddenly start firing at a particular and constant
time before the collision event and maintain this high firing rate throughout the
remainder of the approach sequence. Roughly a quarter of the neurons that show
selectivity to an approaching object show a response that begins earlier for larger
objects, or soccer-ball stimuli approaching at slower velocities. In detailed
mathematical arguments and quantification of the timing of the response, Sun
and Frost (1998) show that these neurons are computing the rate of expansion,
rho, of the approaching object. Finally, the remaining quarter of the looming
neurons appear to be computing the very same function which best describes the
locust looming detector (Hatsopoulos et al., 1995; Gabbiani et al., 1999). An
example of the response patterns of each of these three classes of neuron is
shown in Figure 7. Sun and Frost (1998) show that on several multidimensional
plots these three classes of neurons form very distinct and tight clusters which
indicate that there is not some simple underlying continuum that we have arbitrarily divided into three separate groups, but that these are genuine types of
neurons each computing the following three functions.
(1) Rho
ρ (t ) = θ ′(t)
(2)
(2) Tau
τ (t) ≈
θ (t)
θ ′(t )
(3)
(3) Eta
η(t) = C × θ ′(t ) × e-αθ (t)
(4)
28
Barrie J. Frost and Hongjin Sun
Neuron a
(τ)
Neuron b
(ρ)
Neuron c
(η)
Figure 7: Based on the differences in the time course of the neuronal responses relative to
the moment of collision, the looming sensitive neurons in nucleus rotundus have been
classified into three distinct classes. This figure shows the response pattern (PSTHs) for a
typical neuron in each of the three classes (neuron a, b, and c for tau, rho, and etc respectively) to a series of stimuli (a simulated moving sphere with a soccer-ball pattern) of varying sizes (A) and of varying velocities (B), moving along the direct collision course path
toward the bird. Responses are the sum of 5 trials and are referenced to time zero, which is
the time when the stimulus would have collided with the bird. The simulated path was 15
m in length. In (A) velocity for neuron b was 375 cm/s and for neurons a and c was 500
cm/s. In (B), object size was 30 cm for all three neurons. Note that for the neuron a, the
timing of the response remains invariant despite substantial changes in size and velocity,
whereas for neuron b and neuron c, the timing depends on object size and velocity, with
larger or slower objects evoking an earlier response.
The Biological Bases of Time-to-collision Computation
29
An example of such clustering is shown in Figure 8 taken from Sun and Frost
(1998).
Figure 8: Quantitative examination of the timing of the response for the population of nucleus rotundus looming-sensitive neurons when presented with approaching objects that
varied in size or velocity. The variances (standard deviation) of Tconset were plotted along
the x axis, and the average drop-off in firing rates at the time of collision, relative to the response peak (%), were plotted along y axis. The data points are clustered in three separate
regions, therefore this population of neurons can be classified into three distinct groups ( ○
tau neuron, ▲ rho neuron, ● etc neuron).
B
B
What is rather amazing about these findings is that all looming cells are
accounted for. Each rotundal neuron that responds specifically to an object approaching on a collision course with the bird is either a rho neuron, a tau neuron
or an eta neuron. Usually in single unit recording studies there are “junk” or
“intermediate” categories required for neurons that don’t seem to fit the major
classifications that apply to the other cells. But here we have no residual cells to
account for!
30
Barrie J. Frost and Hongjin Sun
The function of the tau neurons is quite clear; they can provide the
animal with useful information about the TTC of the object that is approaching on a direct collision course. The rho cells obviously compute
rate of expansion information that is required to compute tau, that is, the
denominator in the tau equation. Also the eta function also contains a rate
of expansion term and rho neurons could provide essential input into this
computation also. Indeed the multiplication and division required in these
three equations that describe the total set of looming detectors we have
found in the pigeon nucleus rotundus may well be performed by biophysical operations similar to those described in the recent papers by Peña
and Konishi (2001) and Gabbiani et al. (2002). In this latter paper Gabbiani et al. (2002) suggest that the eta operation is implemented within single LGMD neurons by exponentiation of the sum of a positive excitatory
postsynaptic potential representing the logarithm of angular velocity, and
a negative postsynaptic potential representing angular size.
4. Model
We have developed neuronal models to explain the physiological response properties of our pigeon looming sensitive neurons. These models were
created based on the physiological responses (both qualitative and quantitative
data) recorded to various stimulus conditions (including direct manipulation of
various optical variables that could be specified by a looming object). The models also take into account the physiological response properties and anatomical
connection of the optic tectum that sends a major input to nucleus rotundus.
For these rotundal looming detectors, the RF could be composed of a
radial arrangement of concentric arrays of RFs of simple local motion detectors
(possibly tectal neurons), with the centre of expansion overlapping with the centre of RF radial layout. These tectal neurons would respond to movements that
are oriented radially from the centre of the concentric array and they then converge onto rotundal looming detectors. This arrangement is shown in Figure 9.
The Biological Bases of Time-to-collision Computation
31
Figure 9: General model of receptive field (RF) organization of rotundal looming sensitive
neurons. The RF is composed of a number RF subunits, each of which corresponds to the
RF of a tectal local motion sensitive neuron. These tectal RFs are arranged on the circumference of a series of concentric circles (or rings) with different radii. The converging input from each concentric ring of RF subunits enable both spatial and temporal summation
to signal symmetrical image expansion. Note that this general model explains qualitatively
the vigorous firing of rotundal looming neurons to symmetrical image expansion.
The spatial and temporal summation of the activity of these small RFs
could provide a basis for the rotundal neurons’ large receptive field size, the
strong directional selectivity for motion along the direct collision course (indicated by symmetrical expansion from a stationary centre). The opponentdirection centre-surround organization of these tectal units (Frost & Nakayama,
1983) could contribute to the silence of these rotundal neurons to stimulation
with a self-motion display, in which local segments of the background move in
the same direction as that of the object in the adjacent region across the object
boundary.
A series of quantitative models were also formulated to account for the
specific properties of the time course of rotundal neurons response to impending
collision. When an object approaches the eye, the visual angular subtense θ(t)
and rate of change of visual angle θ'(t) form the basic building blocks for the
32
Barrie J. Frost and Hongjin Sun
calculation of those optical variables that could signal impending collision. For
example, TTC can be signaled by the optic variable tau, which can be specified
by the ratio of θ' (t) over θ(t) (equal to 1/tau). One way to generate this ratio is
through response to the instantaneous values of θ'(t) and θ(t) individually by two
sets of neurons, and then through neuronal interaction to generate the ratio. Alternatively, if the instantaneous value of θ'(t) can be encoded (perhaps through
the velocity selectivity of local motion detectors, e.g. tectal neurons), then visual
angle θ(t) could be registered through the spatial location of the RF of the responding local motion detector relative to the centre of the concentric array of
RFs of these local motion detectors. With the centre of image expansion overlapping with the centre of such a concentric array of RFs, a fixed ratio of θ'(t)
over θ(t) could be hardwired in the brain to create a threshold for the optical
variable 1/tau, so that Rts neurons would only start to fire when the predefined
ratio increase to a fixed level. Consequently, such neurons would only fire when
TTC decrease to a threshold level as the object approaches (see Figure 10).
A
B
A'
preferred
velocity
A
radius of the circle of subunits
Figure 10: Computation of tau. In this model, rate of expansion can be signalled by the
spatial summation of the ring of subunits, each with a certain local velocity selectivity (indicated by the size of arrow in A). The instantaneous visual angle could be registered by
the distance of this ring of RFs from the centre of the circle at the same time (radius of expansion). To calculate tau, the preferred velocity of each local tectal unit should be linearly
related to the radius of the concentric ring of RF subunits located at the same distance
(shown in B).
A similar quantitative model of the other two types of rotundal cells has
been provided in a recently submitted paper (Sun & Frost, submitted). Our models include alternative one and two component versions where size encoding is
realized either from intrinsic pattern of RF organization (one component model)
The Biological Bases of Time-to-collision Computation
33
or from the simultaneous coding of visual angle in spatial overlapping neurons,
and then converges onto the rotundal neuron.
These neuronal models provide explanations for the various physiological responses found in different classes of rotundal looming detectors. Not only
are they potentially informative for understanding the neuronal coding of motion
in depth, but these models could provide important insights for robotics and
machine vision.
A neural net model
In an interesting recent paper Karanka-Ahonen, Luque-Ruiz and LópezZamora (2002) present the results of a neural network employing backpropagation, implemented in T learn, that learned to predict collision of objects
of different sizes that moved towards it. The network consisted of 3 layers with
47 nodes, 40 of which were input nodes, three were hidden nodes, one was the
output node and there were three context nodes. The nodes of each layer were
completely connected to those of the preceding layer and connections from the
context nodes to the context nodes were one to one and reciprocal. After training
the network was tested with new objects that differed in size and distance from
the original training set and it was found that 75% of collisions were correctly
predicted. Interestingly, the behaviour of the hidden units appeared to be very
similar to some of the units we found in the pigeon nucleus rotundus, in that the
output units predictive response came earlier for larger objects, like our eta cells.
5. Conclusions
In this chapter we suggest there may be several different classes of neurons that are each specialized to compute TTC information for different subsets
of tasks critical for the survival of animals in their natural environments. We
then examine one class of these neurons in detail and find there may be several
subroutines required before the tau or eta computations can be performed. What
is interesting is that there appears to be convergence in solutions between looming sensitive neurons found in invertebrates and vertebrates in that the eta function describes well some of the neurons in both locust and pigeon visual systems. The functional significance of having both tau and eta neurons is still not
obvious, although it is provocative that learning networks appear to give rise to
both types of hidden neurons.
34
Barrie J. Frost and Hongjin Sun
REFERENCES
Ball, W. & Tronick, E. (1971). Infant responses to impending collision. Science, 171, 818-820.
Bechterew, W. (1884). Über die Function der Vierhügel. Pflügers Arch, 33, 413.
Bessette, B. B. & Hodos, W. (1989). Intensity, colour, and pattern discrimination deficits after
lesions of the core and belt regions of the ectostriatum. Visual Neuroscience, 2, 27-34.
Blanchard, D. C., William, G., Lee, E. M. C. & Blanchard, R. J. (1981). Taming of wild rattus
norvegicus by lesions of the mesencephalis central gray. Physiological Psychology, 9,
157-163.
Bower, T. G. R., Broughton, J. M. & Moore, M. K. (1970). Infant responses to approaching objects: an indicator of response to distal variables. Perception and Psychophysiology, 9,
193-196.
Burrows, M. & Rowell, C. H. F. (1973). Connections between descending visual interneurons and
metathoracic motoneurons in the locust. Journal of Comparative Physiology, 85, 221234.
Coggshall, J. C. (1972). The landing response and visual processing in the milkweed bug, Oncopeltus fasciatus. Journal of Experimental Biology, 57, 401-413.
Cutting, J. E., Vishton, P. M. & Braren, P. A. (1995). How we avoid collisions with stationary and
moving obstacles. Psychological Review, 102, 627-651.
Dean, P., Mitchell, I. J. & Redgrave, P. (1988). Response resembling defensive behaviour produced by microinjection of glutamate into superior colliculus of rats. Neuroscience, 24,
501-510.
Dill, L. M. (1974). The escape response of the zebra danio (Brachydanio rerio). I. The stimulus
for escape. Animal Behaviour, 22, 771-722.
Ellard, C. G. & Goodale, M. A. (1986). The role of the predorsal bundle in head and body movements elicited by electrical stimulation of the superior colliculus in the Mongolian gerbil. Experimental Brain Research, 71, 307-319.
Ewert, J.-P. (1984) Tectal mechanisms that underlie prey-catching and avoidance behaviours in
toads. In H. Vanegas (Ed.), Comparative neurology of the optic tectum (pp. 247-416).
New York, NY:Plenum Press.
Fishman, R. & Tallaroco, R. B. (1961). Studies of visual depth perception. II. Avoidance reaction
as an indicator response in chicks. Perceptual and Motor Skills, 12, 251-257.
Frost, B. J. (1978). Moving background patterns alter directionally specific responses of pigeon
tectal neurons. Brain Research, 151, 599-603.
Frost, B. J., Cavanagh, P. & Morgan, B. (1988). Deep tectal cells in pigeons respond to kinematograms. Journal of Comparative Physiology, 162, 639-647.
Frost, B. J. & Nakayama, K. (1983). Single visual neurons code opposing motion independent of
direction. Science, 220, 744-745.
Frost, B. J., Scilley, P. L. & Wong, S. C. P. (1981). Moving background patterns reveal double
opponency of directionally specific pigeon tectal neurons. Experimental Brain Research,
43, 173-185.
The Biological Bases of Time-to-collision Computation
35
Gabbiani, F., Krapp, H. G., Koch, C. & Laurent, G. (2002). Multiplicative computation in a looming-sensitive neuron. Nature, 420, 320-324.
Gabbiani, F., Krapp, H. G., & Laurent, G. (1999). Computation of object approach by a widefield, motion- sensitive neuron. Journal of Neuroscience, 19, 1122-1141.
Gabbiani, F., Mo, C. H., & Laurent, G. (2001). Invariance of angular threshold computation in a
wide-field looming-sensitive neuron. Journal of Neuroscience, 21, 314-329.
Gibson, J. J. (1979). The ecological approach to visual perception (Houghton Mifflin, Boston).
Grüsser, O. J. & Grüsser-Cornehls, U. (1976). Neurophysiology of the anuran visual system. In R.
Llinas & W. Precht (eds.), Frog neurobiology. (pp. 297-385). New York, NY: Plenum
Press.
Hatsopoulos, N., Gabbiani, F. & Laurent, G. (1995) Elementary computation of object approach
by a wide-field visual neuron. Science 270, 1000-1003.
Hayes, W. N. & Saiff, E. I. (1967). Visual alarm reactions in turtles. Animal Behaviour, 15, 102108.
Hodos, W. (1969). Color discrimination deficits after lesions of the nucleus rotundus in pigeons.
Brain, Behavior and Evolution, 2, 185-200.
Hodos, W. and Bonbright, J. C. (1974). Intensity difference thresholds in pigeons after lesions of
the tectofugal and thalamofugal visual pathways. Journal of Comparative and Physiological Psychology, 87, 1013-1031.
Hodos, W. & Karten, H. J. (1966). Brightness and pattern discrimination deficits in the pigeon
after lesions of nucleus rotundus. Experimental Brain Research, 2, 151-167.
Ingle, D. J. & Shook, B. L. (1983). Action-oriented approaches to visuo-spatial brain functions. In
D. Ingle, M. Jeannerod & D. Lee (Eds.). Brain mechanisms of spatial vision. (pp. 229258). Dordrecht:Matinus Nijhoft.
Jarvis, C. D. (1974). Visual discrimination and spatial location deficits after lesions of the tectofugal pathway in pigeons. Brain, Behaviour and Evolution, 9, 195-228.
Judge, S. & Rind, F. C. (1997). The Locust DCMD, a movement-detecting neurone tightly tuned
to collision trajectories. Journal of Experimental Biology, 200, 2209-2216.
Karanka-Ahonen, J. T., Luque-Ruiz, D. & López-Zamora, M. (2002). A neural network that acquires the capacity of predicting collision from the increase in the retinal size of objects.
XIV Congreso de la Sociedad Espaňola de Psicologia Comparada.
Keay, K. A., Redgrave, P. & Dean, P. (1988). Cardiovascular and respiratory changes elicited by
stimulation of rat superior colliculus. Brain Research Bulletin, 20, 13-26.
Lee, D. N. (1976). A theory of visual control of braking based on information about time-tocollision. Perception 5, 437-459.
Lee, D. N., Davies, M. N. O., Green, P. R. & Van der Weel, F. R. (1993). Visual control of velocity of approach by pigeon when landing. Journal of Experimental Biology, 180, 85-104.
Lee, D. N. & Reddish, P. E. (1981). Plummeting gannets: a paradigm of ecological optics. Nature,
293, 293-294.
Northmore, D. P. M., Levine, E. S. & Schneider, G. E. (1988). Behaviour evoked by electrical
stimulation of the hamster superior colliculus. Experimental Brain Research, 73, 595605.
36
Barrie J. Frost and Hongjin Sun
Olberg, R. M., Worthington, A. H. & Venator, K. R. (2000). Prey pursuit and interception in
dragonflies. Journal of Comparative Physiology A, 186, 155-162.
Pearson, K. G., Heitler, W. J. & Steeves, J. D. (1980). Triggering of locust jump by mutimodal
inhibitory interneurons. Journal of Neurophysiology, 43, 257-278.
Peña, J. L. & Konishi, M. (2001). Auditory spatial receptive field s created by multiplication.
Science, 292, 249-252.
Redgrave, P., Dean, P., Souki, W. & Lewis, G. (1981). Gnawing and changes in reactivity produced by microinjections of picrotoxin into the superior colliculus of rats. Psychopharmacology, 75, 198-203.
Rind, F. C. (1997). Collision avoidance: from the locust eye to a seeing machine. In M.V. Srinivasan & S. Venkatesh (Eds), From Living Eyes to Seeing Machines (pp 105-125). Oxford:Oxford University Press.
Rind, F. C. & Bramwell, D. I. (1996). Neural network based on the input organization of an identified neuron signaling impeding collision. Journal of Neurophysiology, 75, 967-985.
Rind, F. C. & Simmons, P. J. (1992). Orthopteran DCMD neuron: A reevaluation of responses to
moving objects. I. Selective responses to approaching objects. Journal of Neurophysiology, 68, 1654-1666.
Rind, F. C. & Simmons, P. J. (1999). Seeing what is coming: building collision-sensitive neurones. Trends in Neurosciences, 22, 215-220.
Robertson, R. M. & Johnson, A. G. (1993a). Retinal image size triggers obstacle avoidance in
flying locusts. Naturwissenschaften, 80, 176-178.
Robertson, R. M. & Johnson, A. G. (1993b). Collision avoidance of flying locusts: Steering
torques and behaviour. Journal of Experimental Biology, 183, 35-60.
Sahibzada, N., Dean, P. & Redgrave, P. (1986). Movements resembling orientation or
avoidance elicited by electrical stimulation of superior colliculus in rats. Journal of Neuroscience, 6, 723-733.
Schiff, W. (1965). Perception of impeding collision: A study of visually directed avoidant behaviour. Psychological Monographs: General and Applied, 79, 1-26.
Schiff, W., Caviness, J. A. & Gibson, J. J. (1962). Persistent fear responses in rhesus monkeys to
the optical stimulus of ‘looming’. Science, 136, 982-983.
Schlotterer, G. R. (1977). Response of the locust descending movement detector neuron to rapidly
approaching and withdrawing visual stimuli. Canadian Journal of Zoology, 55, 13721376.
Simmons, P. (1980). Connexions between a movement-detecting interneurone and flight motoneurones of a locust. Journal of Experimental Biology, 86, 87-97.
Simmons, P. J. & Rind, F. C. (1992). Orthopteran DCMD neuron: A reevaluation of responses to
moving objects. II. Critical cues for detecting approaching objects. Journal of Neurophysiology, 68, 1667-1682.
Sun, H.-J., Carey, D. P. & Goodale, M. A. (1992). A mammalian model of optic-flow utilization in
the control of locomotion. Experimental Brain Research, 91, 171-175.
Sun, H.-J. & Frost, B. J. (1997). The effect of image expansion on a human target-directed locomotion task tested in virtual reality. Society for Neuroscience Abstracts,23
The Biological Bases of Time-to-collision Computation
37
Sun, H.-J. & Frost, B. J. (1998). Computation of different optical variables of looming objects in
pigeon nucleus rotundus neurons. Nature Neuroscience, 1, 296-263.
Sun, H.-J., Zhao, J., Southall, T. L., & Xu, B. (2002). Contextual influences on the directional
responses of tectal cells in pigeons. Visual Neuroscience, 19, 133-144.
Tronick, E. (1967). Approach response of domestic chicks to an optical display. Journal of Comparative and Physiological Psychology, 64, 529-531.
Wang, Y. & Frost, B. J. (1992) Time to collision is signalled by neurons in the nucleus rotundus of
pigeons. Nature, 356, 236-238.
Wagner, H. (1982). Flow-field variables trigger landing in flies. Nature, 297, 147-148.
Wicklein, M. & Strausfeld, N. J. (2000). Organization and significance of neurons that detect
change of visual depth in the hawk moth Manduca sexta. Journal of Comparative Neurology, 424, 356-376.
Yonas, A. & Granrud, C. (1985). The development of sensitivity to kinetic binocular and pictorial
depth information in human infants. In D. Ingle, M. Jeannerod, & D. Lee (Eds.), Brain
Mechanisms and Spatial Vision (pp. 113-145). Amsterdam: Martinus Nijhoff Press.
38
Barrie J. Frost and Hongjin Sun
SUBJECT INDEX
Action.......................................................36
Auditory ...................................................37
motion 14, 16, 17, 19, 20, 25, 26, 30, 31,
32, 33, 35, 36
tau 22, 23, 25, 26, 27, 28, 29, 30, 32, 33
Flow .........................................................38
optic flow ............................................17
Information...............................................14
Looming ...................................................14
Tau ..................................................... 18, 27
local ........................................ 30, 31, 32
Time to collision ......................................38
trajectory ............................................ 19, 24
Velocity....................................................26