Download ling411-16 - Rice University

Ling 411 – 16
Language and Perception
Language and Perception: 3 Topics
 The representation of phonological
information needed for perception of
syllables
 Phonological recognition vs phonological
perception
 Influence of language on non-linguistic
perception
Language and Perception: Topics
 The representation of phonological
information needed for perception of
syllables
 Phonological recognition vs phonological
perception
 Influence of language on non-linguistic
perception
Demisyllables [di, de, da, du]
F1 and F2
For [de]
It is unlikely that [d] is represented as a unit in perception
Different sizes in cortical column structure
 Minicolumn
• The smallest unit
• 70-110 neurons
 Functional column
• Variable size – depends on experience
• Intermediate between minicolumn and
maxicolumn
 Maxicolumn (a.k.a. column)
• 100 to a few hundred minicolumns
 Hypercolumn
• Several contiguous maxicolumns
Functional columns: a hypothesis
 The minicolumns within a maxicolumn
respond to a common set of features
 Functional columns are intermediate in size
between minicolumns and maxicolumns
 Different functional columns within a
maxicolumn are distinct because of nonshared additional features
• Shared within the functional column
• Not shared with the rest of the maxicolumn
Mountcastle: “The neurons of a [maxi]column have
certain sets of static and dynamic properties in
common, upon which others that may differ are
superimposed.”
Similarly..
 Neurons of a hypercolumn may have similar
response features, upon which others that
differ may be superimposed
 Result is maxicolumns in the hypercolumn
sharing certain basic features while
differing with respect to others
 Such maxicolumns may be further
subdivided into functional columns on the
basis of additional features
 That is, this columnar structure is
representing categories and subcategories
Hypercolums: Modules of maxicolumns
A homotypical
area in the
temporal lobe
of a macaque
monkey
Category
(hypercolumn)
Subcategory
(can be further
subdivided)
Perceptual subcategories and
columnar subdivisions of larger columns
 Nodal specificity applies for maxicolumns
as well as for minicolumns
 The adjacency hypothesis likewise applies
to larger categories and columns
• Adjacency applies for adjacent maxicolumns
 Subcategories of a category have similar
function
• Therefore their cardinal nodes should be in
adjacent locations
Functional columns in phonological recognition
A hypothesis
 Demisyllable (e.g. /de-/) activates a
maxicolumn
 Different functional columns within the
maxicolumn for syllables with this
demisyllable
• /ded/, /deb/, /det/, /dek/, /den/, /del/
Functional columns in phonological recognition
A hypothesis
deb
[de-]
ded
det
de-
den
del
dek
A maxicolumn
(ca. 100 minicolumns)
Divided into
functional columns
(Note that all
respond to /de-/)
Phonological hypercolumns (a hypothesis)
 Maybe we have
• Hypercolumn of contiguous maxicolumns for /e/
• With maxicolumns for /de-/, /be-/, etc.
• Each such maxicolumn subdivided into
functional columns for different finals
 /det/, /ded/, /den/, /deb/, /dem/. /dek/
 N.B.: This is a hypothesis, not proven
• But there is indirect evidence
• Maybe someday soon we’ll be able to test with
sensitive brain imaging
Adjacent maxicolumns in phonological
cortex?
Hypercolum
de-
te-
be-
pe-
ge-
ke-
A module of six
contiguous
maxicolumns
Each of these
maxicolumns is
divided into
functional columns
Note that the entire module responds to [-e-]
Adjacent maxicolumns in phonological
cortex?
deb
ded
det
de-
den
de-
te-
be-
pe-
ge-
ke-
del
dek
The entire maxicolumn
responds to [de-]
The entire module
responds to [-e-]
A module of six
contiguous
maxicolumns
REVIEW
Functional columns in phonological recognition:
A hypothesis
 Demisyllable (e.g. /de-/) activates a
maxicolumn
 Different functional columns within the
maxicolumn for syllables with this
demisyllable
• /ded/, /deb/, /det/, /dek/, /den/, /del/
Learning phonological distinctions:
A hypothesis
deb
ded
det
de-
de-
te-
be-
pe-
ge-
ke-
den
del
dek
3. The maxicolumn
gets divided into
functional columns
1. In learning,
this hypercolumn
gets established
first, responding
to [-e-]
2. It gets subdivided
into maxicolumns for
demisyllables
Remaining problems – lateral inhibition
 When a hypercolumn is first recruited, no
lateral inhibition among its internal
subdivisions
 Later, when finer distinctions are learned,
they get reinforced by lateral inhibition
 Problem: How does this work?
Indirect evidence for the hypothesis
 Fits the structural organization
demonstrated in monkey vision
 Cortical structure and function have a high
degree of uniformity
 MEG is able to pick up different locations in
Wernicke’s area for different vowels
• MEG can only detect activity of at least 10,000
contiguous apical dendrites (Papanicolaou)
 Requires perhaps at least 250 adjacent
minicolumns
 The size of a maxicolumn or hypercolumn
Language and Perception: Topics
 The representation of phonological
information needed for perception of
syllables
 Phonological recognition vs phonological
perception
 Influence of language on non-linguistic
perception
Perception – Refining a simple-minded view
1. Not just bottom-up
• Top-down processing fills in unsensed details
2. Not confined to a single perceptual modality
• The McGurk effect
Visual input affects auditory perception
Conceptual structure affects auditory perception

•
3. Not even confined to posterior cortex
• Can also use motor neurons


Experiment: left hand or right hand?
Mirror neurons
Perception – Refining a simple-minded view
1. Not just bottom-up
• Top-down processing fills in unsensed details
2. Not confined to a single perceptual modality
• The McGurk effect
Visual input affects auditory perception
Conceptual structure affects auditory perception

•
3. Not even confined to posterior cortex
• Can also use motor neurons


Experiment: left hand or right hand?
Mirror neurons
Top-down processing in perception
Node for CUP in
conceptual area
for drinking
vessels
Conceptual and perceptual
information
CUP
T
MADE OF GLASS
SHORT
CERAMIC
HAS HANDLE
Properties
Visual properties
are in occipital
and lower
temporal areas
Bidirectional processing and inference
These connections
are bidirectional
CUP
T
MADE OF GLASS
SHORT
CERAMIC
HANDLE
Bidirectional processing and inference
Thought process:
CUP
T
SHORT
HANDLE
1. The cardinal
concept node is
activated by a subset
of its property nodes
2. Feed-backward
processing activates
other property nodes
Consequence:
We “apprehend”
properties that are not
actually present in the
sensory input
Perception – Refining a simple-minded view
1. Perception is not just bottom-up
•
Top-down processing fills in unsensed details
2. It is not confined to a single perceptual
modality
•
•
The McGurk effect
 Visual input affects auditory perception
Conceptual structure affects auditory
perception
3. It is not even confined to posterior cortex
• Can also use motor neurons
 Motor activation in speech perception
 Mirror neurons
The McGurk Effect
http://www.youtube.com/watch?v=aFPtc8BVdJk
 Acoustic syllable [ba] presented to subjects
 with visual presentation of articulatory
gestures for [ga]
 Subjects typically heard [da] or [ga]
 “Evidence has accumulated that visual
speech modifies activity in the auditory
cortex, even in the primary auditory cortex.”
Mikko Sams (2006)
Perception depends mainly on
cortical structures already
present before sensory input
“Perception is hallucination
constrained by sensory data”
Shepherd
Perception – Refining a simple-minded view
1. Not just bottom-up
• Top-down processing fills in unsensed details
2. Not confined to a single perceptual modality
• The McGurk effect
Visual input affects auditory perception
Conceptual structure affects auditory perception

•
3. Not even confined to posterior cortex
• Can also use motor neurons


Experiment: left hand or right hand?
Mirror neurons
Left hand or right hand?
Left hand or right hand?
Left hand or right hand?
Left hand or right hand?
Left or right hand?




Imaging experiment
Subjects were shown pictures of one hand
Asked to identify: left or right
Functional imaging showed increased CBF in
hand area of motor cortex
Peter Fox, ca. 2000
Motor structures in perception
 The left-hand vs. right-hand experiment
 ‘Mirror neurons’ in motor cortex
 Articulation as aid to phonological
perception
 Articulation in reading
 Motor activity in listening to music
 Watching an athletic event
Mirror Neurons
 NY Times: “One mystery remains: What
makes them so smart?” (Jan. 10, 2006)
 Answer: They are not smart in themselves
• Their apparent smartness is a result of their
•
position: at top of a hierarchy
Compare:
 The general of an army
 The head of a business
 Similarly, high-level conceptual nodes
• The “grandmother node”
Mirror Neurons
 What makes mirror neurons appear to be
special?
• Ans.: They receive input from visual perception
• The superior longitudinal fasciculus
 Connects visual perception to motor areas
 How can a motor neuron receive perceptual
input?
• Motor neurons are supposed to operate top•
down
Answer: bidirectional processing
 They also receive perceptual information
• Bottom-up processing
Superior Longitudinal Fasciculus
From O. D. Creutzfeldt, Cortex Cerebri (1995)
Are some neurons “smarter” than others?
 Claim: A grandmother node would have
to be very smart
• Identifies very complex object
• Even in many varieties
 Alternative: the head of a hierarchy
• It is the hierarchy as a whole that has
•
those ‘smarts’
Similarly, mirror neurons
 They get visual input since they are
connected to visual areas
• Superior longitudinal fasciculus
Implications of hierarchical organization
 Nodes at a high level in a hierarchy may
give the appearance of being very “smart”
 This appearance is a consequence of their
position — at top of hierarchy
 As the top node in a hierarchy, a node has
the processing power of the whole
hierarchy
• Grandmother nodes
• Mirror neurons
• Compare:
 The general of an army
 The head of a business organization
Multi-Modal Perception
1. Perception is not just bottom-up
•
Top-down processing fills in unsensed details
2. It is not confined to a single perceptual
modality
•
•
The McGurk effect
 Visual input affects auditory perception
Conceptual structure affects auditory
perception
3. It is not even confined to posterior cortex
• Can also use motor neurons
 Motor activation in speech perception
 Mirror neurons
A terminological problem
 We need to distinguish
• Perception narrowly conceived
•
 The basic process of recognition
 Single perceptual modality
 Bottom-up processing
 No motor involvement
Perception broadly conceived
 Two different terms needed
• Recognition (a.k.a. ‘microperception’)
•
 Bottom-up process in a single perceptual
modality
Perception (the broad conception) (a.k.a.
‘macroperception’)
Microperception and macroperception
 Microperception
• A.k.a. recognition
• The local process of integrating features
• Performed in one perceptual modality
• Bottom-up
 Macroperception
• The overall process of perception
• Uses multiple modalities
• Uses top-down processing
Perceptual structures in motor production
 Perceptual structure is used in two ways
Planning (e.g. visualizing while painting)
2. Monitoring
1.
 Examples
•
•
•
•
Phonological recognition in speech production
 Cf. Wernicke’s aphasia
Painting
Musical production
Baseball, soccer, tennis, etc.
Language and Perception: Topics
 The representation of phonological
information needed for perception of
syllables
 Phonological recognition vs phonological
perception
 Influence of language on non-linguistic
perception
Different languages categorize
differently
Examples:






Grammatical gender
Rocks
Plural vs. distributive
Time as metaphorical space
Bells
Time
Grammatical Gender




English: None
French: Masculine, Feminine
German: Masculine, Feminine, Neuter
Oneida: Masculine
Feminine-Zoic
Feminine-Inanimate
Genders in Oneida (Iroquoian)
 Masculine
 Feminine-Zoic
• Women from puberty to menopause
• Animals
 Feminine-Inanimate
• Little girls
• Old women
• Inanimate objects
Rocks
English
rock
rocks
tyhpi
Monachi
tyhtyhpi
Bells – English and French






cloche
clochette
sonnette
grelot
timbre
glas
(of a church)
(on a cow)
(of a door)
(of a sleigh)
(on a desk)
(to announce a death)
The Diversity of ‘Time’
 What time is it?
 How many times did
you go to France?
 She spends a lot of
time in front of the
mirror.
 In the time of the
Crusades…
 At that time, she was
sick.
 Quelle heure est-il?
 Combien de fois es-tu
allé en France?
 Elle passe beaucoup de
temps devant le miroir.
 A l’epoque des
Croidades…
 A ce moment-là, elle
était malade.
Language and (Visual) Perception
Phonological
Production
Language
Phonological
Perception
Object
Categories
Vision
N.B.: These connections are bidirectional
Review
Bidirectional connections
 An established finding from neuroanatomy
 Most corticocortical connections are
bidirectional
 It’s not because the connecting nerve
fibers (axons) are themselves bidirectional
 It’s because we find different but roughly
parallel fibers going in opposite directions
Recent experiments of Kay et al.
 Experiments at UC Berkeley
• Color perception: do difference in color naming
across languages influence color perception?
 Main finding:
• Lateralized influence of language on perception
• Response time faster for between-category
•
discrimination – especially for RVF presentation
A left hemisphere (RVF) phenomenon
green
blue
end