Download Semantics and Categories

Conceptual Processing and
Semantics
(also more on the Mirror Neuron/Action
Perception System)
A.P. Saygin Cogs 172 UCSD
The Nature of Conceptual Representations
• How is semantics organized?
• Sensory/Functional
– Visual properties in a visual subsystem, auditory properties
in an auditory subsystem, and so on…
• Domain-specific
– Organized by domains such as animals, plants, faces…
Independent of sensory and motor representations.
Semantic Categories
Specific deficit for living things
– JBR, SBR, KB, ING (Warrington & McCarthy, 1983;
Warrington & Shallice, 1984); TOB - auditory
(McCarthy & Warrington, 1988)
– Tasks used
• Producing definitions
• Matching names and pictures
• Picture naming
Elizabeth Warrington
Semantic Categories
Specific deficit for artifacts/non-living things
– YOT (Warrington & McCarthy, 1987)
– Tasks
• Matching a spoken name to one of several pictures
(among an array of choices)
Semantic Categories and Deficits
Specific deficit for artifacts/non-living things
– YOT (Warrington & McCarthy, 1987)
– Task: Matching a spoken name to one of several
pictures (among an array of choices)
Can be more specific (e.g., musical instruments,
food)
– Tranel, Damasio & Damasio (1997): Meta-analysis
of category specific deficits
Semantic Categories and Deficits
• Patient EW (Caramazza & Shelton)
• Naming test
– 100% Fruits & vegetables
– 92% Tools
– 34% Animals
• Other tasks as well
PS: Caramazza overheard saying “The lesion can be in the
toe!” – interested in the theory, not the neural bases
Living
Nonliving
Animals
Fruits/
Vegetables
X
X
Artifacts
Warrington & Shallice (1984), Silveri
& Gainotti (1988)
X
X
Warrington & McCarthy (1983),
Sheridan & Humphreys (1993)
Hart & Gordon (1992), Caramazza &
Shelton (1998)
X
X
X
X
Cases
Hillis & Caramazza (1991)
Hart, Berndt & Caramazza (1985),
Farah & Wallace (1992)
X
Not observed (so far)
X: Disproportionate impairment
See readings Caramazza and Martin et al. for more examples
How are living things and artifacts
different?
Note: Warrington et al. did not necessarily view the
patient deficits as evidence for domain-specificity
Instead discussed modality-specificity of semantic system
Visual vs. Functional
We may distinguish living things more by visual properties
- How they look, how they move, colour, etc.
Artifact knowledge more functional
- What is it for? How is it used?
Important to make sure the task does not confound the studies
However, Caramazza argues, there are patients whose deficits reveal
category effects even when the tasks require functional judgments.
(Visual judgments: Do ducks have long ears? Functional judgment: Are
roses given on Valentine’s day?)
Does this argument convince you? Even if the judgment is
functional, could the results be consistent with the concept
requiring sensory (here, visual) representations to be used?
Could this explain the patient data?
Living
NonLiving
Visual
Functional
Visual
Functional
EW
67
74
96
99
GR
55
58
91
84
FM
73
69
96
96
JEN
42
51
75
83
DB
69
69
100
100
RC
69
70
79
90
Patient
Emergent Category Structure
Categories can “emerge” from a system that isn not
domain-specific.
Semantic space is “lumpy” – i.e., things that move in a
certain way tend to be made of similar materials.
Items that share many properties ~ highly correlated
properties will cluster in the brain
What is the difference?
- Domain-specific vs. emergent categories
Alex Martin et al.
•
•
•
•
•
Sensorimotor/Functional model
Distributed networks
Embodiment
Animate objects –visual properties
Tools – functional properties
Sensory/Functional Account
• Evidence for modality effects
– Patients who cannot recognize objects in one
modality, but can in another (Shallice, 1987;
McCarthy & Warrington, 1988; Saffran &
Schwartz, 1994)
Neuroimaging Studies in the Healthy Brain
Silently naming
animal drawings:
occipital
Silently naming
tool drawings:
motor
Verbalizing the
action: Motor and
superior temporal
areas
Verbalizing the
color: Ventral
areas near regions
involved in color
perception
See Martin, Ungerleider & Haxby chapter
– This is pretty old for neuroimaging and
methods have advanced since then but
these studies illustrate the general points
(& the debate is ongoing)
Viewing and naming manipulable objects activates left premotor and
parietal cortex (Chao & Martin, 2000)
Why?
We talked about mirror neuron theory (and biological motion).
A brief overview again
Understanding Others
Theory
3rd person (“other”) actions: Often visually perceived
1st person (“self”) actions: Rarely visually perceived
Yet we are able to know what we perceive (Barresi & Moore,
1996, BBS)
Neuroscience
Visual hypothesis – Based on a visual analysis of the elements
(body parts, objects, motion, etc). No sensorimotor
involvement is required
Simulation hypothesis – Analysis-by-synthesis. We map the
visual representation onto our own sensorimotor
representations
Perception and Action
Frontal and posterior cortical regions are heavily interconnected
“Networks”
Mirror Neurons
Frontal area F5 in the macaque
(Rizzolatti lab, ca 1996)
Later also found in parietal cortex
Neuroimaging in humans: The mirror
neuron system
Inferior frontal cortex
Inferior parietal cortex
Superior temporal sulcus (STS)
Theory
An individual can understand others’ actions by mapping the visual
representation of the observed action onto his/her sensorimotor
representation of the same action, thus using his/her own embodied
experience of the world.
“An action is understood when its observation causes the motor
system of the observer to ‘resonate’ ” (Rizzolatti, Fogassi, & Gallese,
2001).
Mirror Neuron System
Seen as a possible neural resource for linking between self and
other, and for empathy, and social cognition.
The specific properties of the system and the possible involvement
of the mirror neuron system in brain disorders (from autism to
psychiatric disorders) is currently of interest in cognitive
neuroscience.
Mirror Neuron System
Seen as a possible neural resource for linking between self and
other, empathy, and social cognition.
That’s important stuff. So mirror neurons have been popular!
The involvement of the mirror neuron system in associated
disorders (from autism to psychiatric disorders) is currently of
intense interest.
The mirror neuron system is:
- Not a silver bullet
- Have sometimes been overhyped
- Functional properties still need to be specified
Embodiment
We can be more agnostic about “mirror neurons” per se
From mirror neurons to theories of perceptual and conceptual
processing…
Mirror-like responses:
Empathy for pain
Pain observation increases activity in
affective pain processing areas
Singer et al., 2004, Science
Mirror-like responses: Disgust
or olfactory
stimulation
Wicker et al., 2003, Neuron
Mirror-like responses:
Listening to and producing speech
UCLA/UCSD Study
Listen to monosyllables /pa/ /gi/
Produce monosyllables /pa/ /gi/
Overlap found in premotor cortex.
Note: This is a more superior area
than Broca’s area.
Wilson et al., 2004, Nature Neurosci.
Link to Semantics
Activations for tools Chao & Martin could reflect automatic
engagement of action/tool-use related systems
Frontal ~ canonical neurons (manipulable objects)
Sensorimotor/Functional Theory of
Semantics
Semantic representations are related to sensorimotor properties
Automatic activation of semantic representations
Imaging, patient, TMS and behavioral studies
Color - ventral temporal anterior to color perception areas
Motion - lateral temporal lobe
– Superior: biological motion
– Middle: artifact motion
Actions: Motor, premotor, parietal areas
Modulations in sensory or motor regions can be evoked using
linguistic stimuli
Action words (Hauk, Johnsrude, & Pulvermuller, 2004; Tettamanti et
al., 2005); odour-related words (Gonzalez et al., 2006)
Somatotopy in action observation
Buccino et al 2001, Eur. J Neurosci
Somatotopy in Language Semantics
Foot action words
Hand action words
Mouth action words
Hauk, et al., 2004, Neuron
Motion Semantics and Vision
FMRI study – we localized motion-sensitive area V5/MT+ and
face-sensitive area FFA
Motion sentences: “I drove from Modesto to Fresno”
Static sentences: “Modesto and Fresno are in California”
Fictive motion sentences: Use of a motion verb when there is
no physical occurrence of motion (Talmy 1996, 2000)
“The highway runs from Modesto to Fresno”
“The mountain range goes from Mexico to Canada”
Saygin, McCullough, Alac & Emmorey, 2010, J. Cognitive Neuroscience
Readings: Caramazza, 1999 and
Martin, Ungerleider & Haxby, 1999
What do you think?