Download ppt

Linguistics 001, Spring 2009
Neuroimaging: Syntax
Syntax in the brain
• As we saw in the last lecture and in the
readings, it has been proposed that ‘syntax’
resides in Broca’s area
• This finding connects in some ways with what
is found in the aphasia literature, although the
situation is quite complicated
• The reading moves beyond this basic picture
in many ways by looking at imaging of syntax
• Today: Introduce functional Magnetic
Resonance Imaging (fMRI), and look at a few
experiments that involve syntax
Spatial and Temporal Dimensions
Imaging techniques differ in spatial and temporal
sensitivity.
Technique
Spatial
Temporal
ERP
10-20mm
1msec
MEG
5-10mm
1msec
PET
<5mm
30sec
fMRI
<2mm
~1sec
Basics of fMRI
• Uses extremely strong magnetic field
• Measures changes in local blood flow/blood
oxygenation
• Excellent spatial resolution
• Hemodynamic (= blood-related) response is
slow in comparison with neuronal activation.
Procedure
• Subjects lie within the bore of the
magnet, with heads fixed
• Experimenters acquire structural
images of the subjects’ brains
• Functional images, showing changes in
blood flow/oxygenation, show
“activation” in different areas of the
subject’s brain
Further Details
• During changes in neuronal activity, there are
local changes in the amount of oxygen in the
tissue
• The changes in blood oxygenation are
detected by the scanner
• The resulting changes are an indirect
measure of neuronal activity (indirect
because the hemodynamics are related to the
firing)
Cortical Activity and Blood-Flow
Cortical activation results in a local increase in oxygenated blood, and blood flow, without an increase
in oxygen consumption.
Illustration
Areas in which neuronal activity occurs show
patterns of activation which the scanner detects.
The image is paired
with a graph showing
the blood flow in that
region by
experimental
condition (moving vs.
stationary dots in this
case).
Plan
• We’ll look at two early studies that use
fMRI to study aspects of syntax
• While these studies have been followed
by many others, they give an idea of the
kinds of questions that are addressed in
the functional neuroimaging of language
Study 1
• Just, Carpenter, Keller, Eddy, and
Thulborn (1996), Science 274
• Cortical activation and sentential
complexity
Basics
• Question posed: Do computationally
more demanding sentences result in
more brain activation than less complex
sentences?
• Focusing on areas known to be involved
in language, i.e. Wernicke’s and Broca’s
Stimuli
• Three types of sentences, with the
same number of words:
– Active: The reporter attacked the senator
and admitted the error.
– Subject relative: The reporter that t
attacked the senator admitted the error.
– Object Relative: The reporter that the
senator attacked t admitted the error.
Control
• Something to control for some of the
visual components of the sentencereading part
• Consonant strings:
– Pws ntkgqrfm zjkjrng kwtdc sbfght swn
mrjbxq kgt mxbtq
Idea
• The three different types of sentences
have been studied extensively in the
processing literature
– They increase in difficulty
– The reason for this appears to have to do
with the position of the trace, when there is
one
Design and Task
• Sentences were arranged in sets of four
to five of the same type
• Following each sentence, subjects
answered ‘True’ or ‘False’ to a
comprehension probe:
– “The reporter attacked the senator, True or
False?”
Subject Performance
• Reaction time and error rate data were
collected for most of the subjects
• Analysis of the behavioral data showed
these indices of complexity to increase
monotonically in the expected way with
the increasingly complex sentencetypes
Images
The analysis looked at the number of activated voxels by
Condition; more of these are present from left to right.
NOTE: Left is Right and Right is Left in these images
Cortical Activation
Interpretation
• It is suggested that Wernicke’s is involved in
something other than purely lexical matters
(since these are constant across conditions)
• Broca’s area is involved in something as well
(perhaps syntax; it’s hard to tell)
• Right homologues are implicated in
processing as well
• Overall: The brain’s response to increased
processing demands is to recruit more tissue
in each area in a network of cortical areas
Study 2
• Dapretto and Bookheimer (1999),
Neuron 24
• fMRI study, attempting to dissociate
syntax and semantics in sentence
comprehension
Motivation
• A number of prior studies showing
activation in Broca’s area during tasks
of different syntactic complexity
• The idea that this does not establish
anything about exclusivity– perhaps
Broca’s area would also be modulated
by increasing demands along another,
non-syntactic dimension
Design Basics
• Auditory presentation of pairs of
sentences
• Subjects had to decide whether or not
the meaning of the two sentences
differed
• Sentence pairs were classified as
semantic or syntactic; in each case,
there were ‘same’ and ‘different’ types
Conditions
• SEMANTIC: Pairs of identical
sentences, in which one word was
replaced either by a synonym or a
different word.
• SYNTACTIC: Sentences in the pair
were in different voices (active vs.
passive) or had word-order differences
Examples: Semantics
• Same
– The lawyer questioned the witness.
– The attorney questioned the witness.
• Different
– The man was attacked by the doberman.
– The man was attacked by the pitbull.
Examples: Syntax
• Same
– The policeman arrested the thief.
– The thief was arrested by the policeman.
• Different
– The teacher was outsmarted by the
student.
– The teacher outsmarted the student.
Justification of Stimuli
• In the semantic condition: Judgment is
supposed to rely on single word meanings
• In the syntactic condition: Judgment requires
computation and comparison of two syntactic
structures
• Authors note that both kinds of processing
are present in each task, and expect the two
conditions to differ in relative ‘weight’ of
syntax and semantics
Images
Syntax vs.
Rest
Semantics vs.
Rest
Results (General)
• Many areas associated with language
are found to be active in this experiment
• What about the syntax vs. semantics
comparison?
Syntax vs. Semantics
Syntax (A): Task-specific activity
centered in the (lower) pars
opercularis (BA 44).
Semantics (B): Task-specific
activity in the pars orbitalis (BA 47).
Brodmann Areas
Interpretation
• Authors
– Syntax: BA 44 is particularly involved in syntactic
processing
– Semantics: BA 47 is selectively involved in the
processing of lexico-semantic information
• Spend some time thinking about how strong this
kind of conclusion could be, given what
happened in the experiment etc.
• I.e., the conditions both involve syntax and
semantics; moreover, the tasks are not
thoroughly understood; moreover, the behavioral
data are not reliable, etc.
Wrapping up
• Broca’s area (and other language areas) can
be modulated in different tasks
• How this connects with the idea of
specialization is complicated, as discussed in
the reading
• Functional neuroimaging is impressive
technologically, but the connections with
cognitive hypotheses are only starting to
become clear
Space/Time/Brain
• Prospects for future theories of linguistic
computation in the brain
– Integration of spatial and temporal
information
– Correlation between linguistic ontology and
neurological ontology
– Bringing the neurological theories “up to
speed” with the categories from the
representational analyses of language
Linguistics/Neurolinguistics
• If you want to know about the analysis of
sentences/sounds/words, etc., you consult linguistic
theory
• If you want to know about structures in the brain,
cells relevant to brain activity, etc., you consult
neurology.
• What role is there for Neurolinguistics of the type that
we have been studying? What are this area’s results,
and prospects?
Results
• Clinically, results showing that (parts of)
language in the broad sense are correlated
with certain brain areas are useful
• The next step is to ask two related further
questions, and see what has to be done to
answer them:
– What can looking at the brain tell us about the
complex internal structure of language?
– What can looking at language tell us about the
complex internal structure of the brain?
Specialization Question
• Specialization Question: What does it mean for a
specific brain region to be specialized for some
(linguistic) computation?
• I.e., we have seen many descriptions of the different
areas of cortex in the brain
• Brodmann areas:
Specialization, Cont.
• To answer the specialization question, we
need to know
– Why some areas compute some things, and not
others
– What this means neurobiologically
– How ‘flexible’ the correlations are
• Then we can ask if there is any explanatory
relationship between claims like “syntax
happens in region X because this region has
certain properties etc..”
Granularity Question
• Linguistic theory and Neurolinguistic theory operate
at different levels of granularity
• I.e. we had lots of detailed analyses in the discussion
of phonology, morphology, etc.
• For a variety of reasons, many neuroimaging studies
ask questions about “syntax”, or “semantics”;
extremely coarse by comparison (remember that our
intro lectures subdivided “syntax” and “semantics”)
Granularity, Cont.
• When we talk about e.g. syntax, we know
computationally that many things are going
on; this is not a single “task”:
–
–
–
–
Creation of tree structures
Linearization of tree structures
Movement of constituents
Etc.
• In order to bring neurolinguistics into line with
linguistic theory, the brain has to be
investigated in terms of the linguistic
categories that seem to be important
Conclusion
• The future of neurolinguistics depends on the
development of unifiying hypotheses that crosses
these domains
• The way we’ve been looking at this involves
integrating linguistics and neurolinguistics by using
linguistic categories to explore the brain
• There’s not much “back and forth” at present