Download class_2015_readinglist

1. Introduction to the visual system: retina, LGN, cortex, retinotopy, orientation
selectivity, ocular dominance,
Readings: (Hubel and Wiesel 1977) (Sereno, Dale et al. 1995) (Livingstone and Hubel 1988)
2. General architecture of the visual system.
Readings: (Bartels and Zeki 2005), (Olsen, Bortone et al. 2012), (Srihasam, Vincent et al. 2014)
3. Stereopsis and motion perception
Readings: (Ohzawa, DeAngelis et al. 1990), (Anderson 1994) (Britten, Shadlen et al. 1992)
(Pack, Berezovskii et al. 2001)
4. Object recognition 1
Readings: (Kanwisher, McDermott et al. 1997), (Pasupathy, 2002), (Bushnell, 2011)
5. Object recognition 2
Readings: (Kanwisher, McDermott et al. 1997), (Hung, Kreiman et al. 2005), (Kriegeskorte, Mur
et al. 2008), (Konkle and Oliva 2012)
6. Visual attention
Readings: (Kastner, Pinsk et al. 1999) (Cohen and Maunsell 2009) (Saalmann, Pinsk et al. 2012)
7. Visual search and visually-guided action
Readings: (Bichot, Rossi et al. 2005) (Resulaj, Kiani et al. 2009) (Nelissen, Luppino et al. 2005)
(Mante, Sussillo et al. 2013)
8. Consciousness
Readings: (Tong 2003), (Dehaene and Changeux 2011) (Wilke, Logothetis et al. 2003)
Anderson, B. L. (1994). "The role of partial occlusion in stereopsis." Nature 367(6461): 365-368.
Models of stereopsis typically assume that all the information about stereoscopic depth is
contained in the disparity field, that is, the positional differences of image features that
arise from surfaces visible to both eyes. But such models have difficulty in resolving
image regions containing occlusions, because a portion of the occluded surface is visible
to only one of the two eyes ('half-occlusions'). Here I present displays revealing an
unexpected relationship between interocular differences in image position and occluding
contours. The partial occlusion of contours can give rise to both horizontal and vertical
image differences that are not disparities. The results show that the visual system
interprets these image differences as signalling the presence of occluding contours. Even
when a single line segment serves as a binocular target, subjective contours form that can
appear both oriented and in depth. These local subjective contours have a strong tendency
to interact cooperatively and form global contours not present in the monocular images.
These and other findings show that stereoscopic processing actively decomposes vertical
and horizontal image differences into disparities and half-occlusions. The two sources of
information are complementary: while disparity provides relative depth information
about surface features visible to both eyes, half-occlusions provide information to
segment the visual world into coherent objects at object boundaries.
Bartels, A. and S. Zeki (2005). "The chronoarchitecture of the cerebral cortex." Philos Trans R
Soc Lond B Biol Sci 360(1456): 733-750.
We review here a new approach to mapping the human cerebral cortex into distinct
subdivisions. Unlike cytoarchitecture or traditional functional imaging, it does not rely on
specific anatomical markers or functional hypotheses. Instead, we propose that the unique
activity time course (ATC) of each cortical subdivision, elicited during natural
conditions, acts as a temporal fingerprint that can be used to segregate cortical
subdivisions, map their spatial extent, and reveal their functional and potentially
anatomical connectivity. We argue that since the modular organisation of the brain and its
connectivity evolved and developed in natural conditions, these are optimal for revealing
its organisation. We review the concepts, methodology and first results of this approach,
relying on data obtained with functional magnetic resonance imaging (fMRI) when
volunteers viewed traditional stimuli or a James Bond movie. Independent component
analysis (ICA) was used to identify voxels belonging to distinct functional subdivisions,
based on their differential spatio-temporal fingerprints. Many more regions could be
segregated during natural viewing, demonstrating that the complexity of natural stimuli
leads to more differential responses in more functional modules. We demonstrate that, in
a single experiment, a multitude of distinct regions can be identified across the whole
brain, even within the visual cortex, including areas V1, V4 and V5. This differentiation
is based entirely on the differential ATCs of different areas during natural viewing.
Distinct areas can therefore be identified without any a priori hypothesis about their
function or spatial location. The areas we identified corresponded anatomically across
subjects, and their ATCs showed highly area-specific inter-subject correlations.
Furthermore, natural conditions led to a significant de-correlation of interregional ATCs
compared to rest, indicating an increase in regional specificity during natural conditions.
In contrast, the correlation between ATCs of distant regions of known substantial
anatomical connections increased and reflected their known anatomical connectivity
pattern. We demonstrate this using the example of the language network involving
Broca's and Wernicke's area and homologous areas in the two hemispheres. In
conclusion, this new approach to brain mapping may not only serve to identify novel
functional subdivisions, but to reveal their connectivity as well.
Bichot, N. P., et al. (2005). "Parallel and serial neural mechanisms for visual search in macaque
area V4." Science 308(5721): 529-534.
To find a target object in a crowded scene, a face in a crowd for example, the visual
system might turn the neural representation of each object on and off in a serial fashion,
testing each representation against a template of the target item. Alternatively, it might
allow the processing of all objects in parallel but bias activity in favor of those neurons
that represent critical features of the target, until the target emerges from the background.
To test these possibilities, we recorded neurons in area V4 of monkeys freely scanning a
complex array to find a target defined by color, shape, or both. Throughout the period of
searching, neurons gave enhanced responses and synchronized their activity in the
gamma range whenever a preferred stimulus in their receptive field matched a feature of
the target, as predicted by parallel models. Neurons also gave enhanced responses to
candidate targets that were selected for saccades, or foveation, reflecting a serial
component of visual search. Thus, serial and parallel mechanisms of response
enhancement and neural synchrony work together to identify objects in a scene. To find a
target object in a crowded scene, a face in a crowd for example, the visual system might
turn the neural representation of each object on and off in a serial fashion, testing each
representation against a template of the target item. Alternatively, it might allow the
processing of all objects in parallel but bias activity in favor of those neurons that
represent critical features of the target, until the target emerges from the background. To
test these possibilities, we recorded neurons in area V4 of monkeys freely scanning a
complex array to find a target defined by color, shape, or both. Throughout the period of
searching, neurons gave enhanced responses and synchronized their activity in the
gamma range whenever a preferred stimulus in their receptive field matched a feature of
the target, as predicted by parallel models. Neurons also gave enhanced responses to
candidate targets that were selected for saccades, or foveation, reflecting a serial
component of visual search. Thus, serial and parallel mechanisms of response
enhancement and neural synchrony work together to identify objects in a scene.
Britten, K. H., et al. (1992). "The analysis of visual motion: a comparison of neuronal and
psychophysical performance." J Neurosci 12(12): 4745-4765.
We compared the ability of psychophysical observers and single cortical neurons to
discriminate weak motion signals in a stochastic visual display. All data were obtained
from rhesus monkeys trained to perform a direction discrimination task near
psychophysical threshold. The conditions for such a comparison were ideal in that both
psychophysical and physiological data were obtained in the same animals, on the same
sets of trials, and using the same visual display. In addition, the psychophysical task was
tailored in each experiment to the physiological properties of the neuron under study; the
visual display was matched to each neuron's preference for size, speed, and direction of
motion. Under these conditions, the sensitivity of most MT neurons was very similar to
the psychophysical sensitivity of the animal observers. In fact, the responses of single
neurons typically provided a satisfactory account of both absolute psychophysical
threshold and the shape of the psychometric function relating performance to the strength
of the motion signal. Thus, psychophysical decisions in our task are likely to be based
upon a relatively small number of neural signals. These signals could be carried by a
small number of neurons if the responses of the pooled neurons are statistically
independent. Alternatively, the signals may be carried by a much larger pool of neurons if
their responses are partially intercorrelated.
Cohen, M. R. and J. H. Maunsell (2009). "Attention improves performance primarily by
reducing interneuronal correlations." Nat Neurosci 12(12): 1594-1600.
Visual attention can improve behavioral performance by allowing observers to focus on
the important information in a complex scene. Attention also typically increases the firing
rates of cortical sensory neurons. Rate increases improve the signal-to-noise ratio of
individual neurons, and this improvement has been assumed to underlie attention-related
improvements in behavior. We recorded dozens of neurons simultaneously in visual area
V4 and found that changes in single neurons accounted for only a small fraction of the
improvement in the sensitivity of the population. Instead, over 80% of the attentional
improvement in the population signal was caused by decreases in the correlations
between the trial-to-trial fluctuations in the responses of pairs of neurons. These results
suggest that the representation of sensory information in populations of neurons and the
way attention affects the sensitivity of the population may only be understood by
considering the interactions between neurons.
Dehaene, S. and J. P. Changeux (2011). "Experimental and theoretical approaches to conscious
processing." Neuron 70(2): 200-227.
Recent experimental studies and theoretical models have begun to address the challenge
of establishing a causal link between subjective conscious experience and measurable
neuronal activity. The present review focuses on the well-delimited issue of how an
external or internal piece of information goes beyond nonconscious processing and gains
access to conscious processing, a transition characterized by the existence of a reportable
subjective experience. Converging neuroimaging and neurophysiological data, acquired
during minimal experimental contrasts between conscious and nonconscious processing,
point to objective neural measures of conscious access: late amplification of relevant
sensory activity, long-distance cortico-cortical synchronization at beta and gamma
frequencies, and "ignition" of a large-scale prefronto-parietal network. We compare these
findings to current theoretical models of conscious processing, including the Global
Neuronal Workspace (GNW) model according to which conscious access occurs when
incoming information is made globally available to multiple brain systems through a
network of neurons with long-range axons densely distributed in prefrontal, parietotemporal, and cingulate cortices. The clinical implications of these results for general
anesthesia, coma, vegetative state, and schizophrenia are discussed.
Hubel, D. H. and T. N. Wiesel (1977). "Ferrier lecture. Functional architecture of macaque
monkey visual cortex." Proc R Soc Lond B Biol Sci 198(1130): 1-59.
Hung, C. P., et al. (2005). "Fast readout of object identity from macaque inferior temporal
cortex." Science 310(5749): 863-866.
Understanding the brain computations leading to object recognition requires quantitative
characterization of the information represented in inferior temporal (IT) cortex. We used
a biologically plausible, classifier-based readout technique to investigate the neural
coding of selectivity and invariance at the IT population level. The activity of small
neuronal populations (approximately 100 randomly selected cells) over very short time
intervals (as small as 12.5 milliseconds) contained unexpectedly accurate and robust
information about both object "identity" and "category." This information generalized
over a range of object positions and scales, even for novel objects. Coarse information
about position and scale could also be read out from the same population.
Kanwisher, N., et al. (1997). "The fusiform face area: a module in human extrastriate cortex
specialized for face perception." J Neurosci 17(11): 4302-4311.
Using functional magnetic resonance imaging (fMRI), we found an area in the fusiform
gyrus in 12 of the 15 subjects tested that was significantly more active when the subjects
viewed faces than when they viewed assorted common objects. This face activation was
used to define a specific region of interest individually for each subject, within which
several new tests of face specificity were run. In each of five subjects tested, the
predefined candidate "face area" also responded significantly more strongly to passive
viewing of (1) intact than scrambled two-tone faces, (2) full front-view face photos than
front-view photos of houses, and (in a different set of five subjects) (3) three-quarterview face photos (with hair concealed) than photos of human hands; it also responded
more strongly during (4) a consecutive matching task performed on three-quarter-view
faces versus hands. Our technique of running multiple tests applied to the same region
defined functionally within individual subjects provides a solution to two common
problems in functional imaging: (1) the requirement to correct for multiple statistical
comparisons and (2) the inevitable ambiguity in the interpretation of any study in which
only two or three conditions are compared. Our data allow us to reject alternative
accounts of the function of the fusiform face area (area "FF") that appeal to visual
attention, subordinate-level classification, or general processing of any animate or human
forms, demonstrating that this region is selectively involved in the perception of faces.
Kastner, S., et al. (1999). "Increased activity in human visual cortex during directed attention in
the absence of visual stimulation." Neuron 22(4): 751-761.
When subjects direct attention to a particular location in a visual scene, responses in the
visual cortex to stimuli presented at that location are enhanced, and the suppressive
influences of nearby distractors are reduced. What is the top-down signal that modulates
the response to an attended versus an unattended stimulus? Here, we demonstrate
increased activity related to attention in the absence of visual stimulation in extrastriate
cortex when subjects covertly directed attention to a peripheral location expecting the
onset of visual stimuli. Frontal and parietal areas showed a stronger signal increase
during this expectation than did visual areas. The increased activity in visual cortex in the
absence of visual stimulation may reflect a top-down bias of neural signals in favor of the
attended location, which derives from a fronto-parietal network.
Konkle, T. and A. Oliva (2012). "A real-world size organization of object responses in
occipitotemporal cortex." Neuron 74(6): 1114-1124.
While there are selective regions of occipitotemporal cortex that respond to faces, letters,
and bodies, the large-scale neural organization of most object categories remains
unknown. Here, we find that object representations can be differentiated along the ventral
temporal cortex by their real-world size. In a functional neuroimaging experiment,
observers were shown pictures of big and small real-world objects (e.g., table, bathtub;
paperclip, cup), presented at the same retinal size. We observed a consistent medial-tolateral organization of big and small object preferences in the ventral temporal cortex,
mirrored along the lateral surface. Regions in the lateral-occipital, inferotemporal, and
parahippocampal cortices showed strong peaks of differential real-world size selectivity
and maintained these preferences over changes in retinal size and in mental imagery.
These data demonstrate that the real-world size of objects can provide insight into the
spatial topography of object representation.
Kriegeskorte, N., et al. (2008). "Matching categorical object representations in inferior temporal
cortex of man and monkey." Neuron 60(6): 1126-1141.
Inferior temporal (IT) object representations have been intensively studied in monkeys
and humans, but representations of the same particular objects have never been compared
between the species. Moreover, IT's role in categorization is not well understood. Here,
we presented monkeys and humans with the same images of real-world objects and
measured the IT response pattern elicited by each image. In order to relate the
representations between the species and to computational models, we compare responsepattern dissimilarity matrices. IT response patterns form category clusters, which match
between man and monkey. The clusters correspond to animate and inanimate objects;
within the animate objects, faces and bodies form subclusters. Within each category, IT
distinguishes individual exemplars, and the within-category exemplar similarities also
match between the species. Our findings suggest that primate IT across species may host
a common code, which combines a categorical and a continuous representation of
objects.
Livingstone, M. and D. Hubel (1988). "Segregation of form, color, movement, and depth:
anatomy, physiology, and perception." Science 240(4853): 740-749.
Anatomical and physiological observations in monkeys indicate that the primate visual
system consists of several separate and independent subdivisions that analyze different
aspects of the same retinal image: cells in cortical visual areas 1 and 2 and higher visual
areas are segregated into three interdigitating subdivisions that differ in their selectivity
for color, stereopsis, movement, and orientation. The pathways selective for form and
color seem to be derived mainly from the parvocellular geniculate subdivisions, the
depth- and movement-selective components from the magnocellular. At lower levels, in
the retina and in the geniculate, cells in these two subdivisions differ in their color
selectivity, contrast sensitivity, temporal properties, and spatial resolution. These major
differences in the properties of cells at lower levels in each of the subdivisions led to the
prediction that different visual functions, such as color, depth, movement, and form
perception, should exhibit corresponding differences. Human perceptual experiments are
remarkably consistent with these predictions. Moreover, perceptual experiments can be
designed to ask which subdivisions of the system are responsible for particular visual
abilities, such as figure/ground discrimination or perception of depth from perspective or
relative movement--functions that might be difficult to deduce from single-cell response
properties.
Mante, V., et al. (2013). "Context-dependent computation by recurrent dynamics in prefrontal
cortex." Nature 503(7474): 78-84.
Prefrontal cortex is thought to have a fundamental role in flexible, context-dependent
behaviour, but the exact nature of the computations underlying this role remains largely
unknown. In particular, individual prefrontal neurons often generate remarkably complex
responses that defy deep understanding of their contribution to behaviour. Here we study
prefrontal cortex activity in macaque monkeys trained to flexibly select and integrate
noisy sensory inputs towards a choice. We find that the observed complexity and
functional roles of single neurons are readily understood in the framework of a dynamical
process unfolding at the level of the population. The population dynamics can be
reproduced by a trained recurrent neural network, which suggests a previously unknown
mechanism for selection and integration of task-relevant inputs. This mechanism
indicates that selection and integration are two aspects of a single dynamical process
unfolding within the same prefrontal circuits, and potentially provides a novel, general
framework for understanding context-dependent computations.
Nelissen, K., et al. (2005). "Observing others: multiple action representation in the frontal lobe."
Science 310(5746): 332-336.
Observation of actions performed by others activates monkey ventral premotor cortex,
where action meaning, but not object identity, is coded. In a functional MRI (fMRI)
study, we investigated whether other monkey frontal areas respond to actions performed
by others. Observation of a hand grasping objects activated four frontal areas: rostral F5
and areas 45B, 45A, and 46. Observation of an individual grasping an object also
activated caudal F5, which indicates different degrees of action abstraction in F5.
Observation of shapes activated area 45, but not premotor F5. Convergence of object and
action information in area 45 may be important for full comprehension of actions.
Ohzawa, I., et al. (1990). "Stereoscopic depth discrimination in the visual cortex: neurons ideally
suited as disparity detectors." Science 249(4972): 1037-1041.
The possibility has been explored that a subset of physiologically identifiable cells in the
visual cortex is especially suited for the processing of stereoscopic depth information.
First, characteristics of a disparity detector that would be useful for such processing were
outlined. Then, a method was devised by which detailed binocular response data were
obtained from cortical cells. In addition, a model of the disparity detector was developed
that includes a plausible hierarchical arrangement of cortical cells. Data from the cells
compare well with the requirements for the archetypal disparity detector and are in
excellent agreement with the predictions of the model. These results demonstrate that a
specific type of cortical neuron exhibits the desired characteristics of a disparity detector.
Olsen, S. R., et al. (2012). "Gain control by layer six in cortical circuits of vision." Nature
483(7387): 47-52.
After entering the cerebral cortex, sensory information spreads through six different
horizontal neuronal layers that are interconnected by vertical axonal projections. It is
believed that through these projections layers can influence each other's response to
sensory stimuli, but the specific role that each layer has in cortical processing is still
poorly understood. Here we show that layer six in the primary visual cortex of the mouse
has a crucial role in controlling the gain of visually evoked activity in neurons of the
upper layers without changing their tuning to orientation. This gain modulation results
from the coordinated action of layer six intracortical projections to superficial layers and
deep projections to the thalamus, with a substantial role of the intracortical circuit. This
study establishes layer six as a major mediator of cortical gain modulation and suggests
that it could be a node through which convergent inputs from several brain areas can
regulate the earliest steps of cortical visual processing.
Pack, C. C., et al. (2001). "Dynamic properties of neurons in cortical area MT in alert and
anaesthetized macaque monkeys." Nature 414(6866): 905-908.
In order to see the world with high spatial acuity, an animal must sample the visual image
with many detectors that restrict their analyses to extremely small regions of space. The
visual cortex must then integrate the information from these localized receptive fields to
obtain a more global picture of the surrounding environment. We studied this process in
single neurons within the middle temporal visual area (MT) of macaques using stimuli
that produced conflicting local and global information about stimulus motion. Neuronal
responses in alert animals initially reflected predominantly the ambiguous local motion
features, but gradually converged to an unambiguous global representation. When the
same animals were anaesthetized, the integration of local motion signals was markedly
impaired even though neuronal responses remained vigorous and directional tuning
characteristics were intact. Our results suggest that anaesthesia preferentially affects the
visual processing responsible for integrating local signals into a global visual
representation.
Resulaj, A., et al. (2009). "Changes of mind in decision-making." Nature 461(7261): 263-266.
A decision is a commitment to a proposition or plan of action based on evidence and the
expected costs and benefits associated with the outcome. Progress in a variety of fields
has led to a quantitative understanding of the mechanisms that evaluate evidence and
reach a decision. Several formalisms propose that a representation of noisy evidence is
evaluated against a criterion to produce a decision. Without additional evidence,
however, these formalisms fail to explain why a decision-maker would change their
mind. Here we extend a model, developed to account for both the timing and the
accuracy of the initial decision, to explain subsequent changes of mind. Subjects made
decisions about a noisy visual stimulus, which they indicated by moving a handle.
Although they received no additional information after initiating their movement, their
hand trajectories betrayed a change of mind in some trials. We propose that noisy
evidence is accumulated over time until it reaches a criterion level, or bound, which
determines the initial decision, and that the brain exploits information that is in the
processing pipeline when the initial decision is made to subsequently either reverse or
reaffirm the initial decision. The model explains both the frequency of changes of mind
as well as their dependence on both task difficulty and whether the initial decision was
accurate or erroneous. The theoretical and experimental findings advance the
understanding of decision-making to the highly flexible and cognitive acts of vacillation
and self-correction.
Saalmann, Y. B., et al. (2012). "The pulvinar regulates information transmission between cortical
areas based on attention demands." Science 337(6095): 753-756.
Sereno, M. I., et al. (1995). "Borders of multiple visual areas in humans revealed by functional
magnetic resonance imaging." Science 268(5212): 889-893.
The borders of human visual areas V1, V2, VP, V3, and V4 were precisely and
noninvasively determined. Functional magnetic resonance images were recorded during
phase-encoded retinal stimulation. This volume data set was then sampled with a cortical
surface reconstruction, making it possible to calculate the local visual field sign (mirror
image versus non-mirror image representation). This method automatically and
objectively outlines area borders because adjacent areas often have the opposite field
sign. Cortical magnification factor curves for striate and extrastriate cortical areas were
determined, which showed that human visual areas have a greater emphasis on the centerof-gaze than their counterparts in monkeys. Retinotopically organized visual areas in
humans extend anteriorly to overlap several areas previously shown to be activated by
written words.
Srihasam, K., et al. (2014). "Novel domain formation reveals proto-architecture in
inferotemporal cortex." Nat Neurosci 17(12): 1776-1783.
Primate inferotemporal cortex is subdivided into domains for biologically important
categories, such as faces, bodies and scenes, as well as domains for culturally entrained
categories, such as text or buildings. These domains are in stereotyped locations in most
humans and monkeys. To ask what determines the locations of such domains, we
intensively trained seven juvenile monkeys to recognize three distinct sets of shapes.
After training, the monkeys developed regions that were selectively responsive to each
trained set. The location of each specialization was similar across monkeys, despite
differences in training order. This indicates that the location of training effects does not
depend on function or expertise, but rather on some kind of proto-organization. We
explore the possibility that this proto-organization is retinotopic or shape-based.
Tong, F. (2003). "Primary visual cortex and visual awareness." Nat Rev Neurosci 4(3): 219-229.
The primary visual cortex (V1) is probably the best characterized area of primate cortex,
but whether this region contributes directly to conscious visual experience is
controversial. Early neurophysiological and neuroimaging studies found that visual
awareness was best correlated with neural activity in extrastriate visual areas, but recent
studies have found similarly powerful effects in V1. Lesion and inactivation studies have
provided further evidence that V1 might be necessary for conscious perception. Whereas
hierarchical models propose that damage to V1 simply disrupts the flow of information to
extrastriate areas that are crucial for awareness, interactive models propose that recurrent
connections between V1 and higher areas form functional circuits that support awareness.
Further investigation into V1 and its interactions with higher areas might uncover
fundamental aspects of the neural basis of visual awareness.
Wilke, M., et al. (2003). "Generalized flash suppression of salient visual targets." Neuron 39(6):
1043-1052.
A pattern of light striking the retina of an alert observer is normally readily perceived.
While a handful of conditions exist in which even salient visual stimuli can be rendered
invisible, the mechanisms underlying such suppression remain poorly understood. Here,
we describe experiments using a novel stimulation sequence that gives rise to the sudden
and reliable subjective disappearance of a wide range of visual patterns. We found that a
parafoveal target immediately vanished from perception following the abrupt onset of a
surrounding texture. The probability of disappearance was influenced by the ocular
configuration of the target and surround, as well as their spatial separation. In addition,
suppression was critically dependent upon several hundred milliseconds of stimulusspecific adaptation. These findings demonstrate that the all-or-none disappearance of a
salient visual target, which is reminiscent of a high-level selection process, is inextricably
linked to topographic stimulus representations, presumably in the early visual cortex.