Download Chapter 10-Perceiving Depth and Size

Chapter 10-Perceiving Depth and Size
How does the visual system locate objects in three dimensions?
Perceiving position in two dimensions – vertical and horizontal – is easy.
Assume all objects are at the same distance from the observer, so that the brain only has to determine
horizontal and vertical locations.
Neural
activity
corresponding
to location of
t on retina.
x
t
Two objects differing in
vertical position
Neural activity
corresponding to
location of x on
retina.
Cortex
Retina
Position of objects in space nicely corresponds to location of activity on the retina and to location of activity
in the visual cortex. So differences in location of activity in the brain nicely signal differences in object
location in vertical and horizontal dimensions.
This means that location of activity in the cortex could serve as an indicator of spatial location of objects if
all objects were the same distance from us.
Perception of location in 3 dimensions – a problem.
Now assume that objects can vary in distance from the observer – x and y in the figure below.
Neural activity
for t
x
y
t
Neural
activity for
both x and y
Retina
Points x and y project to exactly the same place on the retina. So they’ll also project to the same neurons in
area V1. So how are we able to know that they are in different places in space?
So the issue is: How does the brain identify the location of objects in 3 dimensions when the location of
activity on the two-dimensional retina gives an ambiguous indication of location?
Perception of Depth- 1
5/1/2017
Cue theory A General Theory of perception of depth / distance G9 p 228
Depth is synthesized or figured out through the combination by brain processes of many imperfect cues.
People with this perspective believe that the visual world contains many indicators of distance.
However, each single indicator by itself ambiguously represents distance of an object.
So, each of the many cues is an imperfect indicator, giving a little bit of information about distance, but
not the whole picture.
People holding this perspective believe that the many ambiguous or imperfect indicators must be
synthesized by brain processes into a perception of distance by higher order cortical processes.
So the key aspects of this view are
1) Individual cues for distance are ambiguous. No one cue gives a perfect indication of distance.
2) The ambiguous cues are integrated/synthesized by higher order cortical processes.
3) The perception of distance is an inference based on the integration/synthesis of many cues.
A byproduct of this view is the realization that our brain is continually performing computations of
which we are completely unaware – distance computations in this example.
Table of the 15 generally recognized Cues . . .
Monocular (available to 1 eye)
Oculomotor
Accommodation
Occlusion
Relative height
Static – Size/Position based Familiar Size
Relative Size
Texture Gradients
Linear Perspective
Atmospheric Perspective
Static - Lighting based
Shading
Cast Shadows
Motion parallax
Dynamic
Optic flow
Deletion/Accretion
Neural
Perception of Depth- 2
Binocular (requires 2 eyes)
Convergence
Binocular Disparity
5/1/2017
Oculomotor (Eye Muscle) Cues –
Accommodation –
Changing the shape of the lens to keep attended-to object in focus
A cue available to either eye alone, so it’s a monocular cue
Convergence –
Changing the directions at which the two eyes point to keep the image of the
attended-to object at corresponding points in the two retinas.
Issue: What exactly is the signal? Two possibilities . . .
1) signals from muscles of the eye after they’ve contracted or expanded are the cues, or
2) copies of signals sent to the eye muscles are the cues. (Called corollary discharges – recall the Corollary
Discharge Theory of movement perception)
Motor area
2. Copy of signal sent to eye muscles
Signals to eye muscles
Sensory area
1. Signal from eye muscles
The signals actually used are probably corollary discharge signals.
Both convergence and accommodation give depth information for objects up to about 6 feet from us.
G9 p233 Table 10.1.
Perception of Depth- 3
5/1/2017
Static Size/Position Based Cues (All monocular)
1. Static Cues . . . Available in a painting or regular photograph.
Partial Occlusion – If the image of one object occludes
(prevents us from seeing) part of another object, the occluding
object is perceived as being closer.
Relative height -
Below the horizon: Objects lower in the visual field (1,2,3 below) are
perceived as closer.
Above the horizon, those higher (4,5) in the visual field are perceived as
farther away.
4
1
2
5
3
Familiar size - Comparison of the size of the retinal image of familiar objects at typical distances
allows us to judge their distance in unfamiliar situations.
Relative size – Comparison of retinal sizes of images of objects known to be equal in physical size
allows us to judge their relative distances from us.
Texture gradients – The larger /rougher the texture the closer the object
Linear perspective – Apparent convergence of parallel lines
Perception of Depth- 4
5/1/2017
Static – Lighting Based Cues (All monocular)
Atmospheric perspective – Distant objects appear cloudier than close-up objects.
Shading – Darker objects appear to be farther away
Cast shadows – Y1 p 199 .
A major piece of information to take away from this is that these stimulus characteristics are
automatically integrated in the formation of the experience of distance. So there is brain circuitry
doing this all the time.
Have you thanked your brain recently?
Some videos related to depth perception
*Monocular depth cues: http://www.youtube.com/watch?v=IgkzNaCSOYA
Student project; on a field; illustrate using their placement on the grassy field
*Pitting cues for depth perception against each other: http://www.youtube.com/watch?v=TeyL0tDXQw0
Simple, short; Cute
Perception of Depth- 5
5/1/2017
Dynamic Cues
A. Motion / movement parallax
The differences in speeds of movement of images of objects on the retina as those objects move
at the same physical speed.
Objects which appear to move more rapidly across the field of vision are judged to be closer.
A monocular cue. Requires only one eye.
Requires that the observer be able to perceive motion. This means that some people may not be able
to perceive depth generated through motion parallax because they can’t perceive motion.
Demonstration
Good: http://psych.hanover.edu/krantz/MotionParallax/MotionParallax.html
Neural Correlates of Motion Parallax
There is mounting evidence that there are neurons or neuron clusters that respond to motion parallax.
.
“ . . .many neurons in the middle temporal area (area MT) signal the sign of depth (near
versus far) from motion parallax in the absence of other depth cues. To achieve this,
neurons must combine visual motion with extra-retinal (non-visual) signals related to the
animal's movement. Our findings suggest a new neural substrate for depth perception
and demonstrate a robust interaction of visual and non-visual cues in area MT. Combined
with previous studies that implicate area MT in depth perception based on binocular
disparities, our results suggest that area MT contains a more general representation of
three-dimensional space that makes use of multiple cues . .. .”
Perception of Depth- 6
5/1/2017
2. Optic Flow – the relative motions of objects and surfaces as you move forward or backward
through a scene.
Play m4h01018 Optic flow demo.mp4 in
C:\Users\Michael\Desktop\Desktop folders\Class Videos\M4H01017 Optic flow demo.MP4
Note that objects in the center of the visual scene move very little.
Objects in the periphery move a lot.
Automatic neural processing integrates the different rates and directions of movement of pieces
of the visual scene into a continuous experience of movement through the 3rd dimension.
3. Deletion and Accretion
Deletion – the gradual occlusion of an object as it moves behind a closer object
Accretion – the gradual uncovering of an object as it comes from behind a closer object.
Play C:\Users\Michael\Desktop\Desktop folders\Class Videos\Deletion Accretion.m2ts
G9 p 233 Table 10.1 – Ranges of Effectiveness of Selected Depth Cues
Perception of Depth- 7
5/1/2017
Binocular Disparity
Disparity:
Differences between the images of the visual scene on the left and the right retina.
The left eye “sees” a slightly different scene than the right eye.
Stereoscopic Image – A picture with two views of the same scene. On the left is the view that would be
seen by one eye. On the right is the view that would be seen by the other eye.
Stereopsis:
the perception of depth and 3-dimensional structure obtained on the basis of visual
information deriving from two eyes by individuals with normally developed binocular vision.
Example of a Stereoscopic image
The left image is the right eye view and the right image is the left eye view.
They’re reversed to make it easier to fuse them by simply crossing your eyes.
Stereoscopic Image
Right Eye View
Left Eye View
Note that the differences between the images are not huge. You must inspect the two figures to discover the
small differences.
Yet the visual system integrates the two views into a single experience of the scene in which the
differences between the two views have been translated into differences in distance. Ask an auto engineer
designing automobile “vision” systems how difficult it is to do this.
(A simple demonstration of the different images seen by each eye can be obtained by holding a finger up in
front of your eyes and viewing a scene alternatively with the left eye and then the right eye. )
Perception of Depth- 8
5/1/2017
Optical Details of Binocular Disparity G9 p 236
Horopter: An arc surrounding us with the following characteristic: The images of all objects on the
horopter are projected to corresponding points on the retina.
All images on the horopter project to corresponding points on the two retinas.
So A and A’, B and B’, and C and C’ are three pairs of corresponding images.
B
C
A
Horopter
Images of objects beyond the fixation point project to disparate points.
Images of objects closer than the fixation point also project to disparate points.
So A and ~A and C and ~C are pairs of disparate points on the retinas.
C
B
Horopter
e
A
~A
~C
Perception of Depth- 9
5/1/2017
How are the features of binocular images matched? G9 p 241
Binocular neurons: 1000s of neurons respond to simultaneous stimulation of both eyes and do not
respond if only one eye is stimulated.
Many such neurons respond only when corresponding images (zero disparity) are displayed.
Others, however, respond when disparate images are displayed.
Receives input from
corresponding points.
Receives input from
corresponding points.
Perception of Depth- 10
Receives input from
disparate points.
5/1/2017
Ways of creating binocular disparity: Presenting different images to each eye. How Avatar could
have been presented. G9 p 239
There are several ways of presenting separate, different images to each eye. The goal is to create a situation
in which the left eye receives only the left-eye image and the right eye receives only the right-eye image.
1. The natural way. View any scene with the two eyes open. The left eye image is slightly different from
the right eye scene by virtue of the fact that the eyes are separated. Each eye sees only its own scene.
2. Use a stereoscope to view a stereoscopic image. The stereoscope is a device which holds two images
(usually photos) and forces the left eye to see one image and the right eye to see the other.
Demonstrate this.
3. Use polarized lenses to view two superimposed images. (The IMAX / Avatar method.)
The left eye view is projected using vertically polarized light. The right eye view is projected using
horizontally polarized light. Both views are projected at the same place in the visual field.
The left eye is covered with a vertically polarized filter. This allows only the vertically polarized light to
strike the left eye. Conversely, the right eye is covered with a horizontally polarized filter, allowing only
horizontally polarized light to strike the right eye.
This process results in the left eye “seeing” only the left-eye view and the right eye “seeing” only the right
eye view.
4. Use colored classes to view
anaglyph.
3D(that’s anaglyph)
The left-eye view is created using a predominantly blue image. The right eye view is created using a
predominantly red image.
The left eye is covered with a red lens. This reflects the predominantly red image but passes the
predominantly blue image, allowing the left eye to “see” the blue image. Vice versa for the right eye,
covered by a blue lens.
5. Use special surfaces that allow one image to be viewed by the left eye and another image to be viewed
by the right eye.
View from the top
L
R
L R
LR
L R
L R
L
R
L
R
L R
L
R
L R
L R
6. View a “backwards” stereoscopic image with crossed eyes so that the left eye and right eye double
images converge to one. See image on page 9 of these notes.
7. Create a regular stereoscopic image with diverged eyes, with the left eye view to the left and the right
eye view to the right. Diverge the eyes so that the left eye and right eye double images are experiences as
one image. I can’t do this.
8. Use specially created glasses electronically synchronized with the display. Used in 3-D TVs now for
sale. The left eye glass becomes clear and the right eye glass is made opaque for 1/30 sec while the left eye
image is displayed. Then the left eye glass become opaque and the right eye glass is made clear the next
1/30 sec as the right eye image is displayed.
Perception of Depth- 11
5/1/2017
Creating anaglyph 3D
You need colored glasses for these demonstrations. Red lens over the left eye. Blue lens over the right.
The red lens blocks the red image and passes the blue – sending the blue image to the left eye.
The blue lens blocks the blue image and passes the red – sending the red image to the right eye.
Visual Scene
Lenses
Eyes
1. Two squares. Left eye ~ Red lens
Right eye ~ Blue lens
You should see the square in front of the page.
2. Now reverse the lenses: Left eye: Blue lens
Right eye: Red lens
You should now see the square behind the page.
3. Text: Left eye ~ Red lens
Right eye ~ Blue lens
The center rectangle of text should appear in front of the page.
Now
is the
Now
is the
timetime
for for
all all
good
good
menmen
to to
come
to the
come
to the
aid aid
of their
party.
of their
party.
Perception of Depth- 12
5/1/2017
Gorilla at large: (1954 Movie starring Cameron
Mitchell, Lee J Cobb, Anne Bancroft, Raymond
Burr, and Lee Marvin)
Left eye: Red lens Right eye: Blue lens
http://www.youtube.com/watch?v=nMT0c5
myBc4
Gorilla at Large (1954) Theatrical Trailer
See also VL 10.5, 10.6
10.5 has example pictures.
10.6 has stick figure shapes.
Perception of Depth- 13
5/1/2017
Where is the object in Stereopsis?
Question: Does either eye alone have to be able to recognize the objects that differ in depth for stereopsis to
occur?
Skipped in Su 2016
Two possibilities . . .
1. Object Recognition First. Object recognition occurs for the image in the left eye. Object recognition
occurs for the image in the right eye. The two sets of objects are then matched up somewhere in the
brain.
2. Pieces First. Selected features – edges, colors, of the two views of the object, but not the actual objects are matched and then object recognition occurs once for the matched features.
The answer was provided in the 1960s by Bela Julesz who created . . .
Random Dot Stereograms.- stereoscopic images in which the objects that differ in depth were not visible
to either eye alone.
Consider the following square . . .
The object
It can be seen by either eye. It can be identified. But now consider the same square embedded in a larger
square
The object is in this larger
square.
Where is the original square? It’s there, because I cut it out of the larger square and pasted it onto this page.
But neither eye, nor both together can identify the object in question – the square.
Perception of Depth- 14
5/1/2017
Now
consider
following. The square object above is in both sides of the following figure.
Skipped
in Suthe
2016
But the above square is offset slightly in the left view as opposed to the right view – by about ½ mm.
When viewing the two larger squares normally, most people cannot see the smaller square within each.
But if the left half of the above figure is presented to the left eye and the right half to the right eye,
most people with normal binocular vision can identify the smaller square within the larger squares and they
will see it at a slightly different distance than the larger squares.
Since 1) the object in the top figure is not seen with either eye alone, and 2) the object IS seen when both
eyes are used, the conclusion is that little pieces of the images in each eye are matched up and then
objection recognition occurs on the matched pieces. So possibility 2 above is correct.
The above display requires a stereo viewer. The anaglyph below, which is based on the same principle can
be viewed with colored lenses. Note, however, that in the anaglyph, you can identify the object that will be
displaced with either eye. That’s because the elements of the object (the digits) are so large.
R4
L4 33 66 55 33 22
55 44 33 22 33 88 44 33 22
22
44
66
22 11 99
44 33 22 11 22 11
55 44
22 4545 2 2 1 1 3434
55
12
12
55 66
33
6 65 5 8 82 2 1 1 8 8 55
33 77
11
98
98 6 6 5 5 3 3 7 7
44
22 22
11 33 44 55 22 4 4 7 7 5 5
99 33
55 11 22
21
21
55 33 9 9 3 3 5 5 3 3 8 8
33 88 77
22 33
78
78 88 44 55 22
11
11 55 56
56
22 77
99 88 11
22 22
66 88
77
11
55 66 22
33 44
88 99 33
55 44 55 99 85
85 77 00 99
22
99 88
44 55
77 66 22 33 55 11
Perception of Depth- 15
5/1/2017
Perception of Size G9 p 243
Most obvious visual scene cues for object size is retinal image of the object.
Little
arrow
Big
arrow
Large Object has large retinal image
Small object has small retinal image.
The problem with retinal image size as an indicator of size of the external object is that the retinal image
size also depends on distance of the object.
So viewing the large object at a distance makes the retinal image of the large object as small as the retinal of
the small object viewed close up.
This means that retinal image size depends on BOTH object size and object distance.
So retinal image size won’t be a very good indicator of the actual size of an object.
So, how do we perceive size in the face of the confound illustrated here?
Do we judge distance before judging size?
Perception of Depth- 16
5/1/2017
The Holway and Boring Experiment to decide between Size -> Distance vs Distance -> Size. G9 p 243
Comparison Circles
In Left Hallway
The picture here represents the
fact that the diameter of the
comparisons circle could be
varied by the participant until
its perceived sized matched that
of the test circle.
Observer’s
chair
T1
T2
T3
T4
Test Circles in Right Hallway – only 1 was presented on each trial
Participants adjusted the comparison circle to match the perceived size of the test circle.
The dashed lines drawn on the outsides of the test circles are to illustrate that the test circles all subtended
the same visual angle.
The observer was 1) shown a test circle and then 2) asked to adjust the size of the Comparison circle so that
it was the same as the size as the test circle. The adjustable comparison circle was always at the same
distance from the observer.
Key Results
1. When cues for distance – binocular disparity, texture, linear perspective, etc - were present,
observers made the comparison circle the same size as the test circle – small comparison circle if T1 was the
test, large comparison circle if T4 was the test.
2. When cues for distance were not present – the hallway was dark with only a light shining on the
test circle and a light shining on the adjustable circle, observers adjusted the size of the comparisons
circle to the same visual angle as that subtended by the test circle. So the comparison was set the same for
all test circles.
That is, regardless of whether the test was T1, T2, T3, or T4, four clearly different-sized circles, observers
kept the comparison circle at the same size if there were no cues for distance. So distance before size.
Perception of Depth- 17
5/1/2017
Other evidence that distance judgments affect size judgments: Effects of judged distances on sizes of
afterimages.
An afterimage has a fixed retinal size.
Viewing an afterimage on a sheet of paper held close to yourself results in an
apparently small image.
But viewing the same afterimage on a far-away wall results in an apparently large image.
Note that the actual size of the afterimage on the retina is the
same.
So the conclusion from the Holway and Boring experiment and many many others is that we judge distance
first, and then use the distance judgments to estimate size.
We have a continuously running Automatic Distance Judging System (ADJS) that automatically judges
the distance of every object in the visual scene and reports it – like the automatic reporting of equipment on
airplanes – ACARS (Aircraft Communications Addressing and Reporting System )
The Distance Judging System sends information on distance to the Size Judging System.
Perception of Depth- 18
5/1/2017
Emmert’s Law – Quantification of the Distance -> Size Judgment relationship
Generally, our estimate of the size of an object, S, is determined by both
R: the size of the object’s image on the retina, and
D: the distance of the object from us.
Generally: S = R x D. This is the essence of Emmert’s Law
Perceived Size = Retinal image Size times Perceived Distance.
The law probably works like this . . .
We have a size judging module – a collection of neurons whose activity gives us the experience of how big
an object is.
That module receives information from the eye on the size of the object’s image on the retina.
The module also receives input from a Distance judging module, that has combined all of the distance
information –all of the monocular and binocular information to yield an estimate of the Distance of the
object.
The size judging module combined those two pieces of information to arrive at the perception of size.
R: Size on Retina
S=RxD: Estimate of Size
“Wow, that’s big.”
D: Estimate of Distance
For a given judged Distance, the bigger the retinal image, the bigger the experience Size.
And for a given Retinal size, the bigger the judged distance, D, the bigger the experienced Size.
Perception of Depth- 19
5/1/2017
Distance Judgments lie!!!: Illusions of size and distance
G9 p 248
The automatic size-perception adjustment resulting from distance judgments creates some famous illusions
The Ponzo illusion Note that both linear perspective, relative size, and texture combine.
The Illusion: Most people perceive the top object in the illusion as being larger.
It is not larger – it is exactly the same size as the bottom object.
The Explanation:
The Retinal Images are the same.
But the linear perspective cues for distance cause the Distance Judging Mechanism to report that the top
object is farther away.
The Size Judging Mechanism concludes, “Since the Rs are equal, but it appears that D for the top
object is bigger than D for the bottom object, my conclusion is that the top object must be larger than the
bottom object.”
Perception of Depth- 20
5/1/2017
The Moon illusion
The Zenith Moon – the
moon up in the sky
The Horizon Moon – the moon
low in the sky, just above the
horizon
The Illusion: Most people perceive the horizon moon as being larger.
It is not larger – it is exactly the same size as the zenith moon.
The Explanation:
The Rs are equal.
The objects on the horizon (houses, trees, etc) provide familiar-size cues for distance, causing the
Distance Judging Mechanism to record that the D for horizon moon is larger than D for the zenith moon.
That is, the Distance Judging Mechanism reports that the Horizon Moon is farther away.
The Size Judging Mechanism concludes, “Since the retinal images are the same, but DJM says that
the horizon moon D is larger, my conclusion is that the horizon moon must be larger than the zenith moon.”
Perception of Depth- 21
5/1/2017
The Müller-Lyer Illusion
The Illusion: Most people say that the line with “wings out” is longer than the line with “wings in”.
It is not longer – both vertical lines are exactly the same length.
The Size-Distance Explanation:
The Rs are equal.
The line with “wings in” is like the near corner of a cube. This causes the Distance Judging
Mechanism to record that its D is smaller.
The line with “wings out” is like the far corner of a room. This causes the Distance Judging
Mechanism to record that its D is larger.
The Size Judging Mechanism concludes, “Since the retinal images are the same, but DJM says that
the “wings in” line’s D is small and the “wings out” D is large, the “wings out” line must be the
longer of the two lines.
Perception of Depth- 22
5/1/2017