Download Frog Vision

Frog Vision
Template matching as a strategy for
seeing (ok if have small number of
things to see)
Template matching in spiders?
Template matching in frogs?
The frog’s visual parameter space
PSY305
Lecture 4
1
JV Stone
Primate Vision
The richness of a representation depends on
what it is to be used for.
Simple organisms construct simple representations.
2
Jumping Spider
Recall from PSY241 that the jumping spider has eight eyes, each of which is a
lens/camera type eye such as our own. Most of these eyes have fish-eye like lenses,
giving them a wide field of view but blurry vision. The two forward looking eyes,
though, have very high resolution (almost as good as a cat).
The jumping spider uses its low-resolution, wide field-of view eyes to detect the
presence of moving objects and prey, and then it quickly reorients its body to image the
object of interest with its high-resolution eyes (akin to foveating an object).
The jumping spider is capable of discriminating its prey (flies) from mates (other
spiders) using vision alone.
3
Jumping Spider
• Mate recognition is achieved via template matching. Eyes
are fixed in carapace, and cannot move, but retinae can.
• Each retina has )-shaped region, so that two retina together
yield )( or X-shaped region in visual field. The two retinae
move in a conjugate manner (i.e. movements of both eyes are
the same).
• Retinae are scanned over image to see if object is a mate
Land (1969).
4
Jumping Spider
Evokes
courtship
Evokes
Prey capture
Drees (1952) cited in Land (1969)
5
What the frog’s eye tells the
frog’s brain (Lettvin Maturana,
McCulloch and Pitts, 1959)
• Seminal paper.
• McCulloch and Pitts produced seminal 1943
connectionist paper, so were accustomed to
thinking about building models.
McCulloch, W. S. and Pitts, W. H. (1943).
A logical calculus of the ideas immanent in nervous activity.
Bulletin of Mathematical Biophysics, 5:115-133.
6
Frog Brain
View from above
Optic tectum
Side vew
Optic tectum
7
Front
Tail
Frog Visual System
• No fovea.
• Retina projects to superior colliculus, also called
tectum.
• Requires movement to see food.
• 1 million rods and cones project to 0.5 million
retinal ganglion cells. These project to tectum.
– (human has 126M receptors and 1M ganglion cells
which project to LGN)
8
Frog Visual System
Eye
Optic tectum
or
Superior colliculus
Spinal chord
Optic nerve
Eyes have overlapping views in tiny region of scene (unlike humans).
Complete cross-over of optic nerve at chiasm.
9
Method
• Frog faces inside of hemisphere 14 inches
in diameter.
• Stimulus moved about with magnets on
outside of hemisphere.
Stimulus
on inside of
hemisphere
10
Optic angle and image size
• The size of the image of an object on the retina is proportional
to the angle subtended by that object.
Eye
Same sized
retinal images
Retinal image
1 degree
• An optic angle of one degree can be subtended by a small
nearby object or by a large distant object.
• Sun or moon = 0.5 degree
• Fingernail at arm’s length = 1 degree.
11
Four Concentric Ganglion Cell
Receptive Field (RF) Types
1 Sustained contrast detectors (2 degrees, unmyelinated)
2 Net convexity detectors (1-3 degrees, unmyelinated)
3 Moving edge detectors (12 degrees, myelinated)
4 Net dimming detectors (15 degrees, myelinated)
30 times as many of (1,2) as (3,4) - reflects RF size.
Types 1-4 reflect depth in tectum (with 1 at surface).
Myelinated fibres transmit information more rapidly than
unmyelinated fibres - consistent with detection of
12
static/changing contrast above.
1 Sustained Contrast Detectors
• Figure 2.
• Unmyelinated, 2 degrees RF size.
• Sustained response to any static contrast
edge.
• Response to moving edge in one direction
only (2a top trace).
• Not respond to general changes in
illumination.
13
1 Sustained Contrast Detectors
are Sensitive to Motion Direction
3 degree disc moved across RF in one direction, then the other.
Direction of motion of disc across RF
Motion
Luminance
Within RF
Action
potentials
Figure 2a in paper
Time
14
1 Sustained Contrast Detectors
Response to Static Edge
3 degree disc moved into RF and stopped => sustained response.
Sustained
firing
Luminance
Within RF
Time
Figure 2b in paper
Timebase = 50ms
15
2 Net Convexity Detectors: bug
detectors?
• Figure 3. So-called bug detectors.
• Unmyelinated.
• Transient response to <3 degree moving disc.
Response small if <1 degree.
• Response large if motion is ‘jerky’ (like a fly?).
• No response to moving straight edge.
• No response to whole-field motion of array of
dots, unless one dot moves wrt whole-field
motion (i.e. responds to moving dot only if motion
is relative to whole-field). Whole-field motion can
be caused by frog moving.
16
2 Net Convexity Detectors
Responses to Moving Disc and Edge
Figure 3e and 3f in paper.
Luminance in RF (up => dim, here)
Time
Response to 1 degree disc
at 3 speeds (left to right).
Response to straight edge
at 2 speeds (no response).
Also respond to static 1 degree disc (figure 3a).
17
3 Moving-Edge Detectors
•
•
•
•
See figure 4.
12 degree RF.
Responds only if edge (bar) is moving.
Firing rate increases with speed of edge.
18
3 Moving-Edge Detectors
Time
Transient
Firing
From figure 4e in paper.
Luminance within RF (up => dim).
19
4 Net Dimming Detectors
• See Figure 5 in paper.
• 15 degree RF size
• Sustained response to general (lack of)
illumination
Illumination
Level
(up => dimmer)
Firing
Figure 5d in paper.
Dark
20
Recoding the Retinal Image
• Different cell types encode different types of
information.
• Output of any single cell is inherently ambiguous
in terms of what it signals about the retinal image
(NEXT SLIDE).
• For example, may need population of bug detector
cells to signal exact location of moving dot - see
population coding.
• May also need to take account of output of
moving edge detectors - these provide contextual
information …
21
Cell’s Output Does Not Specify
Exactly What is in Cell’s RF
A spot of light here or here gives identical
input to neuron and identical firing rate.
Edge of retina
22
What a tectal neuron ‘sees’
• “Consider the dendrite of a tectal cell
extending up through four sheets” (p257),
where each sheet of neurons encodes one of
four different types of feature (i.e.
parameter) in the same retinal location.
• Each retinal stimulus generates a different
signature, depending on how it stimulates
neurons in each of the of 4 sheets.
23
What a tectal neuron ‘sees’ if
tectum had only two layers
Image on retina
Receptive field of bug detector
Retinotopic map
Bug detector sheet
Edge detector sheet
Path of tectal dendrite
through tectum
24
Definition: Population and
Assembly Coding
• Population coding: If a population of cells all
respond to the same parameters then their
collective output is called a population code.
– For example, all cells within a tectal sheet respond to
same parameters => population code.
• Assembly coding: If two cell populations respond
to different parameters then their collective output
is called an assembly code.
– For example, cells in different tectal sheets respond to
different parameters => assembly code.
25
Recoding the Image
• (A little speculative connectionism).
26
What a tectal neuron would ‘see’
if tectum had only two layers
To spinal chord
Tectal neuron
+1
Bug
detecting
neuron
-1
Synaptic
strength
Edge detecting neuron
Retinal image
27
What a tectal neuron would ‘see’
if tectum had only two layers
•
•
•
•
•
Assume that the tectal neuron has a connection strength of +1 to the
bug detector and -1 to the edge detector.
If bug detector neuron is firing (denote this as +9) and edge detector is
too (also +9) then may not infer a bug is present (because bug
detector’s output is a ‘false alarm’).
This would yield a total input to the tectal neuron of 1x9 + (-1x9) = 0.
But if bug detector neuron is firing but edge detector is not firing then
may infer a bug is present.
This would yield a total input to the tectal neuron of
+1
-1
+1x9 + (-1x0) = +9, and the tectal neuron would fire,
and the frog would stick out its tongue.
28
From Image Space to Feature
(or Parameter) Space
• Degree of bugness == output of bug detector
• Degree of edgeness == output of moving-edge
detector
• Any image stimulates each detector to different
extent, and therefore represents different point in
2D bug-edge parameter space.
• Thus detectors implement recoding of retinal
image space into useful 2D feature space. If have
three detector types then parameter space is 3D.
29
2D Parameter Space
Degree of bugness
Very like a bug
Unlike an edge
A bit like an edge and a bug
x
Very like a contrast edge
Unlike a bug
x
x
Degree of Edgeness
Each point in parameter space represents the shape of a retinal
stimulus presented to the same retinal location.
30
Tectal Neuron’s RF in 2D Parameter Space
Degree of bugness
Very like a bug
Unlike an edge
x
x
x
Degree of Edgeness
Each tectal neuron
responds according to
output of retinal neurons
which detect ‘lower order’
parameters (bug and
edges), so the RF of the
tectal neuron can be
defined as a region or
parameter field (yellow
disc) in parameter space
which corresponds to bugand-not-edge features.
31
Two types of feature detector
neurons define a 2D feature space
Degree of bugness
N=7
•
1
x
2
3
4
5
6
7
•
x
x
Degree of Edgeness
•
•
Require M=Nk neurons to tile 2D
parameter space. If k=2 and N=7
(as here) then need M= N2=49
tectal neurons to tile 2D
parameter space.
Can’t draw 4D feature space, but
we know that M=Nk neurons are
required to code each of k
parameters over N values of each
parameter.
Therefore require M=Nk to
tile 4D parameter space.
Would need N4=74=2401 tectal cells
per retinal location to ‘tile’ 4D space.
32
Summary 1
• The four tectal sheets of neurons essentially
provide a recoding of the retinal image.
• The retinal image is specified in terms of
luminance at each receptor - this description is
redundant and not useful to frog.
• Tectal neurons recode each small region on retina
in terms of 4 basic features or parameters, so we
have 4 different average firing rates per retinal
location.
• Unlike the retinal receptor outputs, the outputs of
these four feature detectors are useful. Taken
together they make up the frog’s representation of
its visual world.
33
Summary 2
• Each point on retina represented in 4D
parameter space.
• Bug detector probably insufficient to signal
presence of bug, but requires contextual
information from other 3 cell types
(parameters).
• Bug detector is easily fooled …
34
References
Essential
• Lettvin, J.Y., Maturana, H.R., McCulloch, W.S., and Pitts, W.H.,
“What the Frog’s Eye Tells the Frog’s Brain”, Proc. Inst. Radio Engr.
47:1940-1951, 1959 (supplied as handout).
Background Reading:
• Land, MF, Structure of the retinae of the principal eyes of jumping
spiders (Salticidae: dendryphantinae) in relation to visual optics, J Exp
Biol 1969 51: 443-470.
• Land, MF, Movements of the retinae of jumping spiders (Salticidae:
dendryphantinae) in response to visual stimuli, J Exp Biol 1969 51:
471-493.
• Land, MF, Animal Eyes, Oxford University Press, 2002.
35