Download Attention acts as visual glue

Attention acts as visual glue
SUMMARY
When you gaze at a bowl of fruit, why don't the bananas occasionally look purple, the grapes look
orange, the plums look yellow or the oranges look green?
This question isn't as nonsensical as it may sound. When your brain processes the information
coming from your eyes, it stores the information about an object's shape in one place and
information about its color in another. So it’s something of a miracle that the shapes and colors of
each fruit are combined seamlessly into distinct objects when you look at them.
A brain-mapping study, published in the Aug. 6 issue of the Proceedings of the National Academy
of Sciences1, provides new support for the theory that attention is the glue that cements visual
information together as people scan complex scenes.
By David F. Salisbury
Aug. 9, 2002
RESEARCH DETAILS
For some time neuroscientists have known that the brain breaks down the stream of visual
information coming from the eyes into different elements and processes them in different areas.
Color information is handled in one area; shape information in another; and motion in yet another.
Since this discovery, the outstanding question has been exactly how the brain recombines these
different types of visual information to produce an apparently seamless vision of the external
world. Vision researchers call this the “binding problem” and it is currently the subject of
considerable scientific controversy.
Now the results of a brain mapping experiment designed and performed by René Marois,
assistant professor of psychology at Vanderbilt; John C. Gore, who recently moved from Yale to
become a Chancellor’s University Professor at Vanderbilt; and Yale graduate student Keith M.
Shafritz provide significant new support for the theory that attention is the glue that cements
visual information together as people scan complex visual scenes.
“There are more than a dozen places in the brain involved with processing visual information,
each specializing in information with slightly different attributes,” says Marois. “Some specialize in
processing color, some specialize in processing shape, while others specialize in movement.
These areas are not clustered together, but distributed widely around the back of the brain.”
Flies have compound eyes made up of hundreds of individual eyes. But we don’t have
compound eyes, we have a compound brain instead.
–René Marois
1
The role of the parietal cortex in visual feature binding; Proceedings of the National
Academy of Sciences; http://www.pnas.org/cgi/content/abstract/152694799v1
(subscription required)
-1-
Attention acts as visual glue
There are two leading theories about how the brain reintegrates visual information.
One view proposes that the neurons in the scattered areas are bound together in a way that
allows them to act simultaneously. When you look at a banana, the neurons that store information
about the banana’s shape fire simultaneously with the neurons in a different region of the brain
that store information about the banana’s color. It is the direct functional interaction between
neurons located in different visual areas that binds together an object’s numerous visual
properties.
In the 1980’s, Anne M. Triesman at Princeton and her colleagues advanced an alternative
mechanism. She proposed that visual binding is mediated by the parietal cortex, an area of the
brain known to be involved in spatial attention. She suggested that the act of focusing one’s
attention on an object’s spatial location provides the key that binds the different types of visual
information together. If an apple is sitting on the table in front of a woman, then her brain,
specifically the parietal cortex, associates the information about its color and shape with its
location and uses the spatial information to bind together the visual information whenever she
focuses her attention on the apple.
The thing I like about the binding problem is that it’s a hidden problem. Our visual world
seems so seamless. Everything is so organized. But, as soon as we start peering into the
brain, we see that, whoa, this is way complicated. There are all these areas processing
different visual attributes. How we put it all back together, that’s the issue.
–René Marois
The description of a patient who, following a brain injury in the parietal lobe, had difficulty
associating colors with more than one object at a time gave Marois the idea for the basic
experiment.2 When the person was presented with objects one at a time, he had no problem
properly pairing their shapes and colors. When presented with two or more objects at the same
time, however, he often mismatched the color of one object with the shape of another.
So Marois designed a series of trials that asked subjects to concentrate on the shape only, the
color only or both shape and color of pairs of objects displayed on a computer screen while their
brain activity was monitored using the technique called functional MRI. The researchers
presented these pairs to the individuals either sequentially in the same location or simultaneously
at different locations and recorded the areas in the brain that were most active.
“The purpose of our study was really to test the attention theory as strongly as we could,” says
Marois. “I was actually surprised that it worked because we had to adopt such stringent testing
conditions.”
Despite their stringency, the tests showed that activity in the parietal region increased significantly
whenever the individuals were presented with more than one object at the same time.
“This provides strong evidence in favor of the theory that spatial attention is the binding glue that
the brain uses to integrate visual objects whenever it is presented with more than one object at
the same time, which is most of the time,” says Marois.
While the study results support the attention theory, they do not rule out other mechanisms. “In
fact,” he adds, “it is practically certain that the brain uses several mechanisms to solve this
fascinating problem.”
The project was funded by a grant from the National Institute of Neurological Disorders and
Stroke.
The name for this kind of injury is Balint’s syndrome. For more information about this and other injuries to
the parietal lobe, go to the web page of the Centre for Neuro Skills
[http://www.neuroskills.com/index.html?main=tbi/bparieta.shtml]
2
-2-
Attention acts as visual glue
Additional information
The role of the parietal cortex in visual feature binding
Proceedings of the National Academy of Sciences
http://www.pnas.org/cgi/content/abstract/152694799v1
(subscription required)
COLOR & SHAPE TRIALS
Functional Magnetic Resonance Imaging
Functional magnetic resonance imaging (fMRI) is a powerful remote sensing method that allows
researchers to map levels of brain activity without interfering in its activity or doing any harm.
Normal MRI uses a large, circular magnet to establish the magnetic bearings of a number of the
atomic nuclei in the body. The atoms are then excited by radio waves, causing them to vibrate
and give off weak radio waves in turn. These waves are measured and converted into images of
body tissues. Because different atoms vibrate at different frequencies, the technique can
differentiate between body parts with different chemical make-ups, such as bone, blood and
muscle.
FMRI uses the same hardware to provide a picture of the brain's ever changing activity rather
than its static structure. It does this by tracking brain blood flow. The more active a brain area is
the more blood flows to it. So, fMRI provides a moment-by-moment movie of brain activity.
Click to go to an animated tour of an MRI scanner on the PBS web site
[http://www.pbs.org/wnet/brain/scanning/mri.html]
Visual binding trials
In order to determine whether the spatial attention network in the parietal lobe is involved in visual
feature binding, Marois and his colleagues designed a series of simple mental tasks for subjects
to perform while the activity of their brains was being recorded.
The tasks consisted of showing them two simple, colored geometric objects on a screen,
displaying a multi-colored mask slide, showing them a final object and asking them to judge
whether the final object, called the probe, was the same as or different from either of the initial
objects. The two initial objects were displayed either sequentially in exactly the same location or
side-by-side.
In different trials the subjects were asked to compare the objects by their shape only, by their
color only or by both shape and color.
These trials are simulated below. There are four series each consisting of three trials. In each trial
you are shown two objects, a color mask and then a “probe” object. The mental task you are
asked to perform is to determine whether the probe object is the same or different than one of the
initial objects.
In one series you are asked to compare the objects by color only; in another you are asked to
compare the objects by shape only and in the two other series you are asked to compare them by
both shape and color. In one of the shape-and-color series the initial objects are shown
simultaneously and in the other series they are shown sequentially.
Each of trial series is accompanied by an fMRI scan of the back of the brain that shows the level
of brain activity recorded while subjects perform the task involved. Levels of heightened activity
are shown and are color coded according to the type of activity involved. Shape-processing
activity is colored green; color-processing activity is colored red; and activity in the attention
network in the parietal lobe is colored yellow.
-3-
Attention acts as visual glue
Note that your parietal cortex increases in activity only when you are performing the conjunction
task with simultaneously presented objects.
BIOGRAPHICAL SKETCH
René Marois grew up in a small town named Rimouski in northeast Quebec. All of his family still
lives there. He did well in school and so he enrolled in the University of Quebec at Montreal
(UQAM).
“As an undergraduate, I was interested in geophysics,” Marois says. “I had even signed up as a
geophysics major. But my roommate was a psychology major and he was taking a course in
neuropsychology. He had this textbook about the mind and brain. I looked at it and fell in love
with the question of how does the brain work and how does it yield the mind.”
This encounter convinced Marois that he wanted to study neurobiology. But there was a problem.
UQAM did not offer the major. So he had to transfer to McGill University. It was nearby, but all the
courses were taught in English.
"I grew up speaking French, so learning English was a big change. I had to learn the language at
McGill while I completed my undergraduate studies," he says. "So my first two years were very
difficult."
After graduating from McGill, he went on to get a Master’s at Dalhousie University in Halifax and
a doctorate at Yale. There he worked with the marine snail Aplysia studying the cellular basis of
learning. “The attraction of this model system was its simplicity. But I found out that it was not that
simple and, anyway, I was more interested in humans.”
As he was finishing up his doctoral thesis, the technique of functional Magnetic Resonance
Imaging (fMRI) was just coming available. “To me, this seemed like a godsend technique that
allows us to really start asking interesting questions about how the human brain works,” Marois
says, adding. “I haven’t looked back since.”
RELATED LINKS
Other research by René Marois
Virtual reality of synesthesia
[http://exploration.vanderbilt.edu/news/news_synesthesia.htm]
Other research involving vision and the brain
Scientists detail how brain regulates sensory information
[http://exploration.vanderbilt.edu/news/news_senses.htm]
Different parts of the brain handle fantasy and reality
[http://exploration.vanderbilt.edu/news/news_gauthier.htm]
Differences in brain usage among Braille readers shed new light on the relationship
between thought and language
[http://exploration.vanderbilt.edu/news/news_braille.htm]
New clues to the location of visual consciousness
[http://exploration.vanderbilt.edu/news/news_visual.htm]
-4-
Attention acts as visual glue
-5-