Download Cognition and Perception as Interactive Activation

Perception and Thought
as Constraint Satisfaction Processes
Jay McClelland
Symsys 100
April 27, 2010
Perception as Constraint
Satisfaction
• For the first figure, one
tends to see it a
nothing until the
‘solution’ emerges.
– The pieces and the
whole simultaneously
support each other
• For the second, there
are two alternative
solutions, each
involving a reinterpretation of every
part of the larger whole.
– Again the parts and the
whole are mutually
consistent
Ubiquity of the Constraint Satisfaction
Problem
• In sentence processing
– I saw the grand canyon flying to New York
– I saw the sheep grazing in the field
• In comprehension
– Margie was sitting on the front steps when she heard the
familiar jingle of the “Good Humor” truck. She
remembered her birthday money and ran into the house.
• In reaching, grasping, typing…
Puzzles and Problems that Can
Involve Constraint Satisfaction
•
Find a word that you can combine with each of the next three words to make a compound word:
Pine, crab, tree
•
Make a single word from the phrase below:
A rope ends it
•
The two string problem:
–
–
You are in an office. There is a desk with a printer, some paper, a stapler, binder clips, pens
and pencils. Two strings are hanging from a ceiling, about 12 feet apart in a room with a
workbench. You can’t reach the second one while holding the first.
What can you do to bring them both together?
•
You know how much gold weighs per cubic centimeter, and you want to test whether the king’s
golden crown is pure gold. But you don’t know how many cc’s of gold are in the crown. How can
you find out?
•
‘Seeing’ Mate in 3(?)
How Geniuses Think1
• Barwise and Etchemendy, two Stanford logicians, agree
– they believe that nearly all great discoveries by logicians and
mathematicians arise through imagery
– after they make a discovery, they then attempt to verify it formally
• they attempt to develop a proof for their insight
• whereas the insight may have happened quickly, the proof may take
years to develop
– once a proof has been established, it is typically reported in a
publication that says nothing about the imagery that led to the
discovery
1. Slide from L. Barsalou, Stanford Psychology Ph.D.
Finding Perceptual Solutions
• It appears that our brains can search for
alternative solutions until one pops out.
• How are such solutions found?
– One answer is that the process occurs
through a noisy, interactive activation
process.
The interactive Activation Model: a Gradual Mutual
Constraint Satisfaction Process
•
Units represent hypotheses about
the visual input at several levels
and positions.
– Features
– Letters
– Words
•
Connections code contingent
relations:
– Excitatory connections for
consistent relations
– Inhibitory connections for
inconsistent relations
– Lateral inhibition for competition
among mutually inconsistent
possibilities within levels.
•
Connections run in both directions
– So that the network tends to evolve
toward a state of activation in
which everything is consistent.
Interactive Activation Simultaneously
Identifies Words and Letters
•
Stimulus input comes first to letter
level, but as it builds up, it starts to
influence the word level.
•
Letter input from all four positions
makes work the most active word
unit (there is no word worr).
•
Although the bottom up input to
the letter level supports K and R
equally in the fourth letter position,
feedback from the word level
supports K, causing it to become
more active, and lateral inhibition
then suppresses activation of R.
Goodness and Constraint Satisfaction
•
Consider a network with symmetric connections,
i.e. for all pairs of units i, j:
wij = wji
•
Provide external input ei to some of the units.
•
Define the Goodness of a state of the network as:
G(s) = Si>jwijaiaj + Sieiai
•
As the network settles it tends toward states of
higher goodness
•
Examples:
–
–
–
Three-unit network
Necker Cube Network
Interactive activation network
•
Noise in the settling process allows networks to
jump out of local Goodness maxima.
•
If we gradually reduce the noisiness, we can
guarantee finding the best solution.
The Relationship Between Goodness
and Probability in the Boltzmann
Machine
•
A ‘Boltzmann Machine’ is a stochastic neural
network in which units’ activations are set to
0 or 1 with probability:
P(ai = 1) = eneti/T/(1+eneti/T)
•
When T is large, the Boltzmann machine can
jump from state to state easily.
•
If one gradually reduces T, one can reach an
equilibrium in which the probability of being
in a particular state is given by:
-1
0
1
-1
0
1
P(s) = eG(s)/T/(Ss’ eG(s’)/T)
•
Here s indexes one state and s’ ranges over
all possible states.
More on Goodness and Probability
•
If the weights, biases, and external inputs are set appropriately, then the
quantity
eG(s)/T/(Ss’ eG(s’)/T)
represents the posterior probability that the state corresponds to the correct
interpretation of the input.
•
It follows that:
– When T = 1 the probability of being in a state matches the probability that the
state is the correct one.
– When T = 0, only the best state (or the N equally good states) are possible
•
In short, neural networks are capable of achieving optimal perceptual
inference over entire ensembles of hypotheses, if the weights, biases, and
external inputs are set appropriately.
Interactivity in the Brain
• Bidirectional Connectivity
• Interactions between MT
and ‘lower’ visual areas
• Subjective Contours in V1
• Distributed Constraint
Satisfaction in Binocular
Rivalry
Effect of Cooling MT
on neural activation in lower visual areas
•
Investigated effects of cooling
MT on neuronal responses in
V1, V2, and V3 to a bar on a
background grid of lower
contrast.
•
MT cooling typically produces
a reversible reduction in firing
rate to V1/V2/V3 cells’
preferred stimulus (figure).
•
Top down effect is greatest for
stimuli of low contrast. If the
stimulus is easy to see, topdown influence from MT has
little effect.
Response decrease due to
cooling in MT
Lee & Nguyen (PNAS, 2001,
98, 1907-1911)
• They asked the question:
Do V1 neurons participate in
the formation of a
representation of the illusory
contour seen in the upper panel
(but not in the lower panel)?
• They recorded from neurons in
V1 tuned to the illusory line
segment, and varied the
position of the illusory segment
with respect to the most
responsive position of the
neuron.
Response to the illusory contour is found at
precisely the expected location.
Temporal Response to Real and Illusory
Contours
Neuron’s receptive field falls right
over the middle of the real or illusory
line defining the bottom edge of the square
• The patterns seen in the
physiology are
comparable to those
seen in the interactive
activation model in that
the effect of direct input
is manifest first, followed
somewhat later by
contextual influences,
presumably mediated in
the physiology by
neurons sensitive to the
overall configuration of
display elements.
direct
context
Distributed Alternation
of Brain Activity in
Binocular Rivalry
Models of Binocular Rivalry
• Binocular rivalry can be explained by
constraint satisfaction models in which
neurons and/or connections that are active
gradually become fatigued.
• As a result the state that happens to dominate
initially eventually weakens, and is replaced
by the alternative state.
• This state eventually weakens, and the other
neurons/connections recover in the meantime,
so that the first state will re-activate as the
second one weakens.
• And the process continues…