* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download ling411-16 - Rice University
Neuropsychopharmacology wikipedia , lookup
Neural oscillation wikipedia , lookup
Convolutional neural network wikipedia , lookup
Neuroanatomy wikipedia , lookup
Neural coding wikipedia , lookup
Neurocomputational speech processing wikipedia , lookup
Caridoid escape reaction wikipedia , lookup
Dual consciousness wikipedia , lookup
Perceptual learning wikipedia , lookup
Nervous system network models wikipedia , lookup
Binding problem wikipedia , lookup
Optogenetics wikipedia , lookup
Central pattern generator wikipedia , lookup
Sensory substitution wikipedia , lookup
Neural modeling fields wikipedia , lookup
Channelrhodopsin wikipedia , lookup
Top-down and bottom-up design wikipedia , lookup
Mirror neuron wikipedia , lookup
Embodied language processing wikipedia , lookup
Sensory cue wikipedia , lookup
Synaptic gating wikipedia , lookup
Neural correlates of consciousness wikipedia , lookup
Emotion perception wikipedia , lookup
Speech perception wikipedia , lookup
Premovement neuronal activity wikipedia , lookup
Neuroesthetics wikipedia , lookup
Embodied cognitive science wikipedia , lookup
Music psychology wikipedia , lookup
Cognitive neuroscience of music wikipedia , lookup
Ling 411 – 16 Language and Perception Language and Perception: 3 Topics The representation of phonological information needed for perception of syllables Phonological recognition vs phonological perception Influence of language on non-linguistic perception Language and Perception: Topics The representation of phonological information needed for perception of syllables Phonological recognition vs phonological perception Influence of language on non-linguistic perception Demisyllables [di, de, da, du] F1 and F2 For [de] It is unlikely that [d] is represented as a unit in perception Different sizes in cortical column structure Minicolumn • The smallest unit • 70-110 neurons Functional column • Variable size – depends on experience • Intermediate between minicolumn and maxicolumn Maxicolumn (a.k.a. column) • 100 to a few hundred minicolumns Hypercolumn • Several contiguous maxicolumns Functional columns: a hypothesis The minicolumns within a maxicolumn respond to a common set of features Functional columns are intermediate in size between minicolumns and maxicolumns Different functional columns within a maxicolumn are distinct because of nonshared additional features • Shared within the functional column • Not shared with the rest of the maxicolumn Mountcastle: “The neurons of a [maxi]column have certain sets of static and dynamic properties in common, upon which others that may differ are superimposed.” Similarly.. Neurons of a hypercolumn may have similar response features, upon which others that differ may be superimposed Result is maxicolumns in the hypercolumn sharing certain basic features while differing with respect to others Such maxicolumns may be further subdivided into functional columns on the basis of additional features That is, this columnar structure is representing categories and subcategories Hypercolums: Modules of maxicolumns A homotypical area in the temporal lobe of a macaque monkey Category (hypercolumn) Subcategory (can be further subdivided) Perceptual subcategories and columnar subdivisions of larger columns Nodal specificity applies for maxicolumns as well as for minicolumns The adjacency hypothesis likewise applies to larger categories and columns • Adjacency applies for adjacent maxicolumns Subcategories of a category have similar function • Therefore their cardinal nodes should be in adjacent locations Functional columns in phonological recognition A hypothesis Demisyllable (e.g. /de-/) activates a maxicolumn Different functional columns within the maxicolumn for syllables with this demisyllable • /ded/, /deb/, /det/, /dek/, /den/, /del/ Functional columns in phonological recognition A hypothesis deb [de-] ded det de- den del dek A maxicolumn (ca. 100 minicolumns) Divided into functional columns (Note that all respond to /de-/) Phonological hypercolumns (a hypothesis) Maybe we have • Hypercolumn of contiguous maxicolumns for /e/ • With maxicolumns for /de-/, /be-/, etc. • Each such maxicolumn subdivided into functional columns for different finals /det/, /ded/, /den/, /deb/, /dem/. /dek/ N.B.: This is a hypothesis, not proven • But there is indirect evidence • Maybe someday soon we’ll be able to test with sensitive brain imaging Adjacent maxicolumns in phonological cortex? Hypercolum de- te- be- pe- ge- ke- A module of six contiguous maxicolumns Each of these maxicolumns is divided into functional columns Note that the entire module responds to [-e-] Adjacent maxicolumns in phonological cortex? deb ded det de- den de- te- be- pe- ge- ke- del dek The entire maxicolumn responds to [de-] The entire module responds to [-e-] A module of six contiguous maxicolumns REVIEW Functional columns in phonological recognition: A hypothesis Demisyllable (e.g. /de-/) activates a maxicolumn Different functional columns within the maxicolumn for syllables with this demisyllable • /ded/, /deb/, /det/, /dek/, /den/, /del/ Learning phonological distinctions: A hypothesis deb ded det de- de- te- be- pe- ge- ke- den del dek 3. The maxicolumn gets divided into functional columns 1. In learning, this hypercolumn gets established first, responding to [-e-] 2. It gets subdivided into maxicolumns for demisyllables Remaining problems – lateral inhibition When a hypercolumn is first recruited, no lateral inhibition among its internal subdivisions Later, when finer distinctions are learned, they get reinforced by lateral inhibition Problem: How does this work? Indirect evidence for the hypothesis Fits the structural organization demonstrated in monkey vision Cortical structure and function have a high degree of uniformity MEG is able to pick up different locations in Wernicke’s area for different vowels • MEG can only detect activity of at least 10,000 contiguous apical dendrites (Papanicolaou) Requires perhaps at least 250 adjacent minicolumns The size of a maxicolumn or hypercolumn Language and Perception: Topics The representation of phonological information needed for perception of syllables Phonological recognition vs phonological perception Influence of language on non-linguistic perception Perception – Refining a simple-minded view 1. Not just bottom-up • Top-down processing fills in unsensed details 2. Not confined to a single perceptual modality • The McGurk effect Visual input affects auditory perception Conceptual structure affects auditory perception • 3. Not even confined to posterior cortex • Can also use motor neurons Experiment: left hand or right hand? Mirror neurons Perception – Refining a simple-minded view 1. Not just bottom-up • Top-down processing fills in unsensed details 2. Not confined to a single perceptual modality • The McGurk effect Visual input affects auditory perception Conceptual structure affects auditory perception • 3. Not even confined to posterior cortex • Can also use motor neurons Experiment: left hand or right hand? Mirror neurons Top-down processing in perception Node for CUP in conceptual area for drinking vessels Conceptual and perceptual information CUP T MADE OF GLASS SHORT CERAMIC HAS HANDLE Properties Visual properties are in occipital and lower temporal areas Bidirectional processing and inference These connections are bidirectional CUP T MADE OF GLASS SHORT CERAMIC HANDLE Bidirectional processing and inference Thought process: CUP T SHORT HANDLE 1. The cardinal concept node is activated by a subset of its property nodes 2. Feed-backward processing activates other property nodes Consequence: We “apprehend” properties that are not actually present in the sensory input Perception – Refining a simple-minded view 1. Perception is not just bottom-up • Top-down processing fills in unsensed details 2. It is not confined to a single perceptual modality • • The McGurk effect Visual input affects auditory perception Conceptual structure affects auditory perception 3. It is not even confined to posterior cortex • Can also use motor neurons Motor activation in speech perception Mirror neurons The McGurk Effect http://www.youtube.com/watch?v=aFPtc8BVdJk Acoustic syllable [ba] presented to subjects with visual presentation of articulatory gestures for [ga] Subjects typically heard [da] or [ga] “Evidence has accumulated that visual speech modifies activity in the auditory cortex, even in the primary auditory cortex.” Mikko Sams (2006) Perception depends mainly on cortical structures already present before sensory input “Perception is hallucination constrained by sensory data” Shepherd Perception – Refining a simple-minded view 1. Not just bottom-up • Top-down processing fills in unsensed details 2. Not confined to a single perceptual modality • The McGurk effect Visual input affects auditory perception Conceptual structure affects auditory perception • 3. Not even confined to posterior cortex • Can also use motor neurons Experiment: left hand or right hand? Mirror neurons Left hand or right hand? Left hand or right hand? Left hand or right hand? Left hand or right hand? Left or right hand? Imaging experiment Subjects were shown pictures of one hand Asked to identify: left or right Functional imaging showed increased CBF in hand area of motor cortex Peter Fox, ca. 2000 Motor structures in perception The left-hand vs. right-hand experiment ‘Mirror neurons’ in motor cortex Articulation as aid to phonological perception Articulation in reading Motor activity in listening to music Watching an athletic event Mirror Neurons NY Times: “One mystery remains: What makes them so smart?” (Jan. 10, 2006) Answer: They are not smart in themselves • Their apparent smartness is a result of their • position: at top of a hierarchy Compare: The general of an army The head of a business Similarly, high-level conceptual nodes • The “grandmother node” Mirror Neurons What makes mirror neurons appear to be special? • Ans.: They receive input from visual perception • The superior longitudinal fasciculus Connects visual perception to motor areas How can a motor neuron receive perceptual input? • Motor neurons are supposed to operate top• down Answer: bidirectional processing They also receive perceptual information • Bottom-up processing Superior Longitudinal Fasciculus From O. D. Creutzfeldt, Cortex Cerebri (1995) Are some neurons “smarter” than others? Claim: A grandmother node would have to be very smart • Identifies very complex object • Even in many varieties Alternative: the head of a hierarchy • It is the hierarchy as a whole that has • those ‘smarts’ Similarly, mirror neurons They get visual input since they are connected to visual areas • Superior longitudinal fasciculus Implications of hierarchical organization Nodes at a high level in a hierarchy may give the appearance of being very “smart” This appearance is a consequence of their position — at top of hierarchy As the top node in a hierarchy, a node has the processing power of the whole hierarchy • Grandmother nodes • Mirror neurons • Compare: The general of an army The head of a business organization Multi-Modal Perception 1. Perception is not just bottom-up • Top-down processing fills in unsensed details 2. It is not confined to a single perceptual modality • • The McGurk effect Visual input affects auditory perception Conceptual structure affects auditory perception 3. It is not even confined to posterior cortex • Can also use motor neurons Motor activation in speech perception Mirror neurons A terminological problem We need to distinguish • Perception narrowly conceived • The basic process of recognition Single perceptual modality Bottom-up processing No motor involvement Perception broadly conceived Two different terms needed • Recognition (a.k.a. ‘microperception’) • Bottom-up process in a single perceptual modality Perception (the broad conception) (a.k.a. ‘macroperception’) Microperception and macroperception Microperception • A.k.a. recognition • The local process of integrating features • Performed in one perceptual modality • Bottom-up Macroperception • The overall process of perception • Uses multiple modalities • Uses top-down processing Perceptual structures in motor production Perceptual structure is used in two ways Planning (e.g. visualizing while painting) 2. Monitoring 1. Examples • • • • Phonological recognition in speech production Cf. Wernicke’s aphasia Painting Musical production Baseball, soccer, tennis, etc. Language and Perception: Topics The representation of phonological information needed for perception of syllables Phonological recognition vs phonological perception Influence of language on non-linguistic perception Different languages categorize differently Examples: Grammatical gender Rocks Plural vs. distributive Time as metaphorical space Bells Time Grammatical Gender English: None French: Masculine, Feminine German: Masculine, Feminine, Neuter Oneida: Masculine Feminine-Zoic Feminine-Inanimate Genders in Oneida (Iroquoian) Masculine Feminine-Zoic • Women from puberty to menopause • Animals Feminine-Inanimate • Little girls • Old women • Inanimate objects Rocks English rock rocks tyhpi Monachi tyhtyhpi Bells – English and French cloche clochette sonnette grelot timbre glas (of a church) (on a cow) (of a door) (of a sleigh) (on a desk) (to announce a death) The Diversity of ‘Time’ What time is it? How many times did you go to France? She spends a lot of time in front of the mirror. In the time of the Crusades… At that time, she was sick. Quelle heure est-il? Combien de fois es-tu allé en France? Elle passe beaucoup de temps devant le miroir. A l’epoque des Croidades… A ce moment-là, elle était malade. Language and (Visual) Perception Phonological Production Language Phonological Perception Object Categories Vision N.B.: These connections are bidirectional Review Bidirectional connections An established finding from neuroanatomy Most corticocortical connections are bidirectional It’s not because the connecting nerve fibers (axons) are themselves bidirectional It’s because we find different but roughly parallel fibers going in opposite directions Recent experiments of Kay et al. Experiments at UC Berkeley • Color perception: do difference in color naming across languages influence color perception? Main finding: • Lateralized influence of language on perception • Response time faster for between-category • discrimination – especially for RVF presentation A left hemisphere (RVF) phenomenon green blue end