Download sensorimotor contingencies

Sensory modalities
Different approaches
O’Regan, J. K., Noe, A., A sensorimotor account of
vision and visual consciousness
Behavioral and Brain sciences (2001), 24:5
Unsignificant internal architecture
• There is no internal representations or architecture
to explain the conscious quality of different
sensations:
– From the point of view of the brain there is nothing
that in itself differentiates nervous influx coming from
retinal, haptic, proprioceptive, auditory, and other
senses
• Even if the size, shape, firing patterns, places wqwhere
neurons are localized in the cortex differ, this does not, in itself
confer them with any particular visual, olfactory, motor or
other perceptual qualities
Sensorimotor contingencies
• What does explain the differences between sensations is
the structure of the rules governing the sensory changes
produced by various motor actions = sensorimotor
contingencies
– Within different sensory modalities, sensorimotor contingencies
differ because they are subject to different invariance properties
– Each sensory modality is defined by a particular structure of the
rules governing sensoru changes produced by movement
• Sensory modalities are modes of exploration of the world
mediated by knowledge of sensorimotor contingencies
Classes of sensorimotor contingency
• We can destinguish between 2 classes of sensorimotor
contingency:
– Sensorimotor contingencies determined by the character of the
visual apparatus (the eye is a kind of sphere, etc.)
• They are the fundamental aspect of visual sensation (because they are
independent of any interpretation or categorization of objects)
– Sensorimotor contingencies determined by the visual attributes of
the object, or geometric invariants (characteristics of the object that
doesn’t change when the eye moves)
• They are the basis of visual perception
Visual sensorimotor contingencies: visual
percepts
• Movements of the eyes:
– When the eyes rotate, the sensory stimulation of the retina shifts and distorts in
a very particular way, determined by the size of the eye movement, the shape of
the retina, the nature of the ocular optics: contours shift and the curvature of the
lines changes, the distribution of information on the retin changes (because of
the difference of distribution of photoreceptors in central and perripheral
vision)
– When the body moves forwards, the flow pattern on the retina is an expanding
one; when moving backwards a contracting one
– When the eyes close during blinks, the stimulation changes drastically
becoming uniform
• Constraints of visual-type exploration/visual attributes of objects:
– The retinal image of an object only provides a frontal view of the object
– When one moves around the object parts of the object appear and disappear
– We can only apprehend an object from a definite distance (retinal projection
depends on distance)
– Color and brightness of the objects change when one moves around the object
or lighting conditions change
Perception of visual shape
• Exemple: Patiens born blind and operated:
• A patient cited by Helmholtz is surprised that a coin should so
dramatically change its shape when rotated
• A patient of Cheseldon was surprised by the change in extent of an
object in fonction of the distance
• « The visual quality of shape is precisely the set of all
potential distortions that the shape undergoes when it is
moved relative to us, or when we move relative to it.
• Although it is an infinite set, the brain can abstract from
this set a series of laws, and it is this set of laws which
codes shape »
Other sensorimotor contingecies: auditory
percepts
• Auditory sensorimotor contingencies are not affected by
eye movements or blinks, but they are affected in a special
way by head movements:
– rotations of the head change the temporal asynchrony between the
ears
– movements of the head in the direction of the sound source affect
the amplitude but not the frequency of the input
• Tactile exploration of an object obeys different
sensorimotor contingencies:
– One doesn’t touch an object from a point of view
– The tactile aspect of an object doesn’t change with lighting
conditions
Knowledge of sensorimotor laws
Summary: « what differentiate the senses are not the neural pathways but the
laws obeyed by the sensorimotor contingencies associates with these
senses »
– Sensorimotor contingencies are distinctive for each sensory modality
– The laws of sensorimotor contingency are determined by the fact that the
exploration is being made by a particular apparatus and exercised on specific
characters of the object
– The brain distinguishes between the senses on the basis of the different
sensorimotor laws a percept respond to
• Two conditions must be achieved to perceive in a certain modality:
– Presence of the conditions for the two kind of sensorimotor contingency laws
– Active master of the laws
• Third condition for visual (sensory) awareness:
– The master of the laws must be comprised in plannings, thougs, linguistic
behavior
A practical form of knowledge
• The mastery of the rules of sensorimotor contingency is not something we
possess a propositional knowledge about
– We are not able to describe in detail how it is distorted a seen surface when we
rotate the eyes
• The deviation from the laws that our brain normally extract causes an
impression that something unusual is happening and let us see the
distortions
– When we put on a new pair of glasses we see distortions for a while because
the movement of the eyes provokes displacements of unusual amplitude
• The knowledge of the laws is a predictive or anticipatory knowledge:
« Perceptual experience is a mode of activity involving practical knowledge
about currently possible behaviors and associated sensory consequences »
– When one sees or touch a bottle there is a knowledge in his nervous system
about what he would feel or see if he move his hands or eyes upwards or
downwards the bottle, and it is this anticipation that tells him that he is
touching a bottle and not a pitcher
Some phenomena explained within
sensorimotor contingency approach
• Tactile visual sensory substitution
– In experiences with TVSS it is only when the observer is allowed
to move the videocamera that identification becomes possible and
that that he begins to « see » something as externally localized, in
the space before him (also if the stimulus array is on his back)
– If an experience in one sensory modality derives from the « rules
that govern action-related changes in sensory input » visual
experience could be obtainable by other sensory channels when the
brain extracts the same invariants in the structure of the
senorimotor contingencies
– So subjects with TVSS see and not just « see »: the fact that the
stimulus is provided by the skin is irrelevant, provided that the
stimulation obeyes the required sensorimotor laws (of course the
laws are not exactly the same as seeing with the eyes, as seeing in
the dark is not the same thing that seein with full light)
Multimodal interactions
• Synesthesia and other multimodal substitutions
and interactions as ventriloquism effect, McGurk
effect and others:
– The experience associated with a modality exists only
within the context of other senses available to the
organism : senses are not really separated
• Although sensory modalities have their own specificities dues
to the particularities of sensors and of sensorimotor
contingencies, when there are systematic correlations and
common sensorimotor contingencies, interactions between the
senses are to be expected
• Anyway, the mentioned specificities define the particular
experience associated with each sense
Binding problem
• The binding problem is a pseudo problem
• Neuroanatomists believe that the visual system is composed of numerous sub-systems or
modules which are independent and often localized in different parts of the brain; so they
raise the question of where and how the different features ultimately come together to
produce a unified perception of reality as we experience; they suggest some solutions:
• The grand-mother cell in which single cells at least combine information
• The synchronized oscillation of the separate cortical areas which are analysing different
components of the information
• The binding problem can be analysed under two issues:
– Temporal unit or coherence:
– scientists think that, because we have the impression that we see simultaneously all the
attributes when we identify an object, then the information about these attributes must be bound
together synchronously.
• It is a fallacy: physical synchrony is not necessary for having a synchronous experience
because it is not the « internal representation » which counts for the coherence of the
experience. « What counts for a temporally coherent experience is the fact is a thing we
are doing, and we are doing it now »
– Conceptual unit or coherence:
• The fact that object attributes seem to be part of a single object doesn’t require them to
be « represented » in a unified way, for example at a single location of the brain, or by a
single process; and if they are represented in a spatially or temporally way this doesn’t
explain the perceptual coherence. « What explains the conceptual unity of experience is
the fact that experience is a thing we are doing and we are doing it with respect to a
conceptually unified external world »
Qualia
• Qualia doesn’exist: the differences in the qualitative
character of perceptual experiences correspond to
differences in the character of the relevant sensorimotor
contingencies
– The difference between driving a Porche and driving a tank is in
the different things we do driving them, in the different practical
knowledge about driving them
– The difference between seein and smelling a red flower consists in
the different patterns of sensorimotor contingencies that governs
perceptual encounte with each:
• To experience the redness of an object or to feel to drive a Porche is to
know what would happen if the light changes or if the accelerator is
pressed down
Some problems with multimodality and
intermodality: is a sensorimotor account complete?
•
•
•
•
The fact that the laws of sensorimotor contingency are different from a modality to
another, doesn’t indicate the necessity of a kind of binding, even if not an internal
binding, but a description of the way in which different sensorimotor contingencies
become an object instead of two objects?
Supposed that we accept there is no need of binding, what about the need for
coherence: could a system act in a world that is not coherent? Don’t we need
mechanism to warranty coherence? These mechanisms could act on information or
on sensorimotor contingencies, in order to set an accord between contingencies?
Are there laws that rule « multisensory contingency »? Are these rules effective on
perception? ie regarding the relation between vision and touch in prehension:
complex rules for a motor act that define a sort of sensorimotor act?
In tactile visual substitution, ie, how can we say that the laws of sensorimotor
contingency are analogous with eyes or tactile substitution? Performances are they
better if the device is placed on the front of the subject instead of on the back (in
order to move the camera as he moves his head)? Visibly, not