Download The neural representation of plural discourse entities

Brain & Language 137 (2014) 130–141
Contents lists available at ScienceDirect
Brain & Language
journal homepage: www.elsevier.com/locate/b&l
The neural representation of plural discourse entities
Timothy W. Boiteau a,⇑, Eric Bowers a, Veena A. Nair b, Amit Almor a
a
b
University of South Carolina, USA
Dept. of Radiology, School of Medicine and Public Health, University of Wisconsin Madison, USA
a r t i c l e
i n f o
Article history:
Accepted 6 August 2014
Keywords:
Discourse processing
Reference
Parietal lobes
fMRI
Neuroimaging
Plurals
Conjunction
a b s t r a c t
Little is known about the underlying neural structures that mediate the generation and tracking of discourse referents. In two functional magnetic resonance imaging experiments, we examined the neural
structures involved in generating and maintaining the representations of multiple referents. Experiment
1 used two-sentence discourses with singular and plural conditions linking back to single or conjoined
subjects. In Experiment 2, conjunction type was manipulated in order to keep the number of discourse
entities constant across the discourse. Both experiments found greater activation in the superior parietal
lobule bilaterally for plural entities relative to singular entities in Experiment 1 and for unconjoined
plural entities relative to conjoined plural entities in Experiment 2. This parietal activation suggests that
referring to multiple entities evokes multiple representations that need to be integrated and tracked. We
discuss these findings in terms of psycholinguistic theories of multiple referent representations.
Ó 2014 Elsevier Inc. All rights reserved.
1. Introduction
Successful language comprehension requires that listeners and
readers relate incoming information to previously processed linguistic input (Grosz, Joshi, & Weinstein, 1995; Sanford & Garrod,
1981). The relation between new and previously mentioned information is often established through anaphora, i.e., the subsequent
reference to a previously mentioned referent (Ariel, 1990; van Dijk
& Kintsch, 1983). The influx of new information in a discourse
makes it important for the comprehender to track references
quickly and efficiently, and this entails generating and maintaining
detailed representations of mentioned referents so that they can be
integrated into the discourse model and linked to subsequent
repeated references. The salience—i.e., prominence in the discourse
model—of a given entity affects how easily this entity can be
accessed from memory, and thus the representation and tracking
of references requires the management of memory resources
(Almor, 1999; Almor & Nair, 2007; Sanford & Garrod, 1981). Our
focus in this study is on the differences between singular and plural references. Specifically, we investigate how such plural references are represented in the brain. Before describing our
experiments, we will first review theories of plural anaphora,
which are based mostly on findings from behavioral studies. Then
⇑ Corresponding author. Address: Psychology Department, University of South
Carolina, 1512 Pendleton St., Barnwell College, Columbia, SC 29208, USA.
E-mail address: [email protected] (T.W. Boiteau).
http://dx.doi.org/10.1016/j.bandl.2014.08.003
0093-934X/Ó 2014 Elsevier Inc. All rights reserved.
we will review the extant fMRI literature on reference processing
in order to clarify our predictions.
Plural noun phrases (NPs) can be introduced into text and represented in the discourse model in several ways (Albrecht &
Clifton, 1998; Moxey, Sanford, Sturt, & Morrow, 2004; Patson &
Ferreira, 2009; Sanford & Lockhart, 1990). They can be introduced
as a regular plural NP (e.g., the students), as a quantified NP (e.g., the
two students), or as a conjoined NP (e.g., the student and the teacher). Sanford and Lockhart (1990) showed that the form of introduction into the discourse influences the likelihood of subsequent
plural reference. For example, conjoined NPs consisting of similar
singular forms like the student and the teacher are more likely to
lead to plural grouping than conjoined NPs consisting of mismatching singular forms like Jim and the teacher. In addition to
the use of a conjoined NP, having discourse entities participate in
a joint activity similarly facilitates subsequent plural pronominal
reference (Gelormini-Lezama & Almor, 2013; Moxey, Sanford,
Wood, & Gintner, 2011).
At least three different theories have been proposed to explain
how plural NPs are stored in memory and processed during discourse comprehension. According to one theory (Sanford &
Lockhart, 1990), plural referents are represented and accessed in
memory as an assemblage, or a collective group or entity (i.e.,
the group is more readily accessible than individual members).
Albrecht and Clifton (1998) found that when a conjoined NP is broken apart by a singular reference (e.g., Stan and Pam went to the
store. She bought milk), readers take longer to read the second sentence than when the second sentence makes a singular reference
T.W. Boiteau et al. / Brain & Language 137 (2014) 130–141
referring to a singular antecedent (e.g., Pam went to the store. She
bought milk). Albrecht and Clifton interpreted this delay as reflecting a ‘‘conjunction cost’’ that is incurred when a conjoined NP has
to be broken in order to access one of its components. This led
Albrecht and Clifton to conclude that conjoined NPs are represented as a single entity or assemblage.
According to a second theory (Johnson-Laird, 1983), plural referents are individuated and represented as distinct atomic tokens.
Kaup, Kelter, and Habel (2002) provided evidence that under certain conditions plural entities are represented individually. However, these types of representations are elicited mainly by the
use of partitioning plural NPs such as most of the orphans or
both-phrases. Furthermore, atomic tokens seem to be formed as
opposed to an assemblage when features of some of the members
of a plural entity are specified (Patson & Warren, 2011). Importantly, the atomic-token view can be used to explain Albrecht
and Clifton’s results, where singular reference to a conjoined entity
results in longer reading times not due to a conjunction cost, but
rather as a result of the increased difficulty of retrieving a referent
from memory due to the larger number of potential referents in the
discourse model.
A third theory, combining the principles of the previous two,
suggests that conjoined NPs are under certain conditions represented as a set of individuals or tokens within an assemblage. This
type of representation is known as a Complex Reference Object
(CRO) and can be understood to consist of a representation of both
a single entity as well as of the individuals within the entity
(Barker, 1992; Moxey et al., 2004; Moxey et al., 2011). In an eyetracking study, Patson and Ferreira (2009) found that participants’
parsing strategies of sentences with reciprocal verbs (e.g., wrestle)
were affected by the type of plural NP in the preceding sentence
(i.e., conjoined NPs: the trainer and the vet; or definite plural
descriptions: the trainers) used in the first clause of the sentence.
For example: The trainer and the vet/The trainers were near the
swamp. While they wrestled the alligator watched them closely. The
key measure in this study was reading times at the disambiguating
region (watched in this case), which indicated whether or not the
participants were garden-pathed. They found that reading times
on the disambiguating region were shorter when the preceding
sentence contained a conjoined NP than when it contained a definite plural description. The authors interpreted this as evidence of
CRO formation in which the representations of the individuals
made the reciprocal interpretation of the verb more likely than
the transitive one. They therefore argued that conjoined NPs favor
representation in a CRO.
As of yet none of these theories of plural anaphora representation have been tested with the use of neuroimaging techniques,
and therefore it is difficult to generate hypotheses about the neural
structures involved in such representations. However, there have
been a few studies on reference processing that may illuminate
this discussion. In addition, plural reference, though a linguistic
process, is analogous to other mental operations that are worth
considering.
Most pertinent to our present study is Almor, Smith, Bonilha,
Fridriksson, and Rorden (2007), who used functional magnetic resonance imaging (fMRI) to study reference processing. They measured participants’ hemodynamic responses as they were reading
discourses in which repeated reference was made with either a
repeated name (e.g., Susan is really into animals. The other day
Susan gave Betsy a pet hamster.) or a pronoun (e.g., Susan is really
into animals. The other day she gave Betsy a pet hamster.). Participants showed greater activation in temporal and parietal areas
(specifically bilateral intraparietal sulcus (IPS), the superior parietal lobule (SPL; BA 7 in particular), and precuneus) in the repeated
name condition than in the pronoun condition. Because temporal
regions are known to be involved in memory processes
131
(Bookheimer, 2002), Almor et al. attributed the activation in these
regions to the activation of multiple memory representations as a
result of reading the repeated names. On the other hand, SPL
regions have been argued to be involved in the maintenance and
integration of multiple representations (Dehaene, Piazza, Pinel, &
Cohen, 2003), leading to the conclusion that the parietal activation
represents the formation, maintenance and integration of multiple
representations resulting from the reading of the repeated names.
Inferences regarding these regions will be discussed in more detail
below.
Nieuwland, Petersson, and Van Berkum (2007) similarly used
fMRI to investigate the neural representation of reference processing (in addition to semantic coherence). This study contained three
conditions of interest to the current discussion: referential ambiguity (e.g., Ronald told Frank that he . . .), failure (Rose told Emily that
he . . .), and coherence (Ronald told Emily that he . . .). Compared to
referential coherence, ambiguity led to fronto-parietal activation,
perhaps indicating that participants were engaged in a decisionmaking process to establish reference. On the other hand, failure
compared to coherence, resulted in bilateral parietal activation
(including among other areas BA 7), which may be related to an
inference resulting in the formation of a new representation (i.e.,
assuming the anaphor is referring to an unspecified third party)
or morpho-syntactic error. In a post hoc analysis the authors
divided the subjects into two groups based on self-reported interpretation of sentences in this condition (either a new referent or
referential failure). They found that the group interpreting these
sentences as referring to a new referent relative to the referential
failure group showed greater activation in left BA8 and right dosolateral BA 9/46. Although this would suggest that introducing a
new referent into the discourse evokes frontal activity, an alternative explanation is that this reflects explicit decision making in the
new referent group. Indeed, in another fMRI study McMillan, Clark,
Gunawardena, Ryant, and Grossman (2012) found widespread
frontal activity associated with the greater decision-making
demands of ambiguous pronominal reference than non-ambiguous
pronominal reference. Additionally, the interpretation that
Nieuwland et al.’s (2007) parietal activity findings reflect introducing a new referent into the discourse sits well with Almor et al.’s
(2007) findings, where the repeated name (associated with the
temporary addition of a new discourse entity before it is resolved
as being coreferential; Gordon, Grosz, & Gilliom, 1993) is associated with bilateral parietal activation of a similar nature.
The parietal activation in both studies, and especially the SPL,
which is not often associated with non-spatial language processing, is of special interest because it suggests that language recruits
areas that are specialized for the management of spatial representations for the task of reference tracking in discourse. For example:
Beauchamp, Petit, Ellmore, Ingeholm, and Haxby (2001) found that
covert shifts of attention activated regions of IPS extending into the
SPL; Corbetta, Miezin, Shulman, and Petersen (1993) found activation in BA 7 bilaterally during a spatial attention shifting task;
Culham et al. (1998), using a multiple-object tracking task, found
activation both in bilateral IPS and BA 7 (see also, Farah, Wong,
Monheit, & Morrow, 1989; Shimozaki et al., 2003; Wojuciulik &
Kanwisher, 1999). Related to spatial processing and our discussion
of plurality, the IPS and SPL have been associated with numerical
processing (e.g., Hubbard, Piazza, Pinel, & Dehaene, 2005). Indeed,
Dehaene et al. (2003) posited that the function of the SPL (specifically posterior BA 7) is to orient attention along the mental number
line (proposed to be within the IPS) and make numerical comparisons (see also, Harvey, Klein, Petridou, & Dumoulin, 2013; Piazza &
Izard, 2009). Thus, taking these two related bodies of literature into
account, we would expect that if indeed number and space are
important to processing plurals, both the IPS and BA 7 would be
activated, depending on the nature of the discourse model. First,
132
T.W. Boiteau et al. / Brain & Language 137 (2014) 130–141
consider that plurality is at its core a linguistic numerical device. In
the case of plural pronouns, plural markings are used to draw
attention to the quantity of entities being referred to (in the case
of English, one or more than one). Thus, we might expect that in
discourses that involve plural compared to singular entities both
the IPS and posterior BA 7 would be active, indicating the need
of recruiting these resources to track multiple discourse entities.
On the other hand, in discourses with plural entities that are represented in different ways (i.e., different levels of saliency), only
posterior BA 7 would be active.
The results of Almor et al. (2007) and potentially Nieuwland
et al. (2007), while not testing the nature of plural representation,
may nevertheless be able to help us make predictions about the
neural signature of maintaining multiple discourse entities in the
mental model. Both the repeated name condition and the referential failure condition from Almor and Nieuwland’s respective studies, led to bilateral SPL activation. There are at least two potential
reasons for this. First, parietal activation in these conditions may
be a result of overall lack of discourse coherence. Second, it may
be a result of the addition of a new referent into the mental model.
In the case of the repeated name condition, the repeated name is
very briefly represented as a new discourse entity (Gordon &
Hendrick, 1998), and in the referential failure condition, the pronoun is inferred to refer to a third party. Using plural NPs can help
us resolve these two possibilities. If we compare a coherent condition with one discourse entity to a coherent condition with two
discourse entities and observe a similar parietal signature, then
we can rule out the first possibility. If, however, we do not observe
such a parietal signature, then we can rule out the second possibility in favor of the first one.
In addition, observing patterns of parietal activation could help
ascertain the nature of the plural representation in the mental
model. For example, if an assemblage without individuated tokens
is formed, there should not be any difference between one-entity
and two-entity discourses. However, if either a CRO or atomic
tokens are created instead of an assemblage, we would indeed see
a difference, and such a difference would be apparent in superior
parietal regions, if our hypothesis is correct. Experiment 1 aims to
test this prediction. In order to differentiate between the atomic
token and the CRO theories of plural representation, the number
of overall discourse entities would need to be held constant, but
the likelihood of being grouped together could be manipulated,
for example, by using a conjoined noun phrase vs. using a with
adverbial (Moxey et al., 2004). If a with adverbial results in greater
parietal activation than a conjoined noun phrase, we could conclude
that the conjoined noun phrase is represented more efficiently as a
CRO (as opposed to two atomic tokens), whereas the with phrase
leads to storing the discourse entities separately. Experiment 2 then
aims to clarify the likely representations created by plural NPs by
manipulating how multiple entities are combined in the discourse.
Crucially, in this experiment we hold the overall number of discourse entities constant, manipulating method of conjunction.
Overall, by examining the neural activation associated with plural
referents, the present work aims to inform general theories of reference as well as theories of plural reference more specifically.
2. Experiment 1
In this experiment we compare discourses with singular pronominal reference to the subject of a previous sentence (Singular
condition) to plural pronominal reference to a conjoined subject
NP of the previous sentence (Plural condition). We hypothesize
that the Plural condition (in line with the findings from both
Almor et al., 2007 and Nieuwland et al., 2007) will lead to greater
activation than the Singular condition in the parietal lobe, and
more specifically the SPL. This activation will be the result of
increased processing load associated with maintaining a more
complex representation of the subject. We also hypothesize that
the Singular condition will not lead to any more activation than
the Plural condition in these parietal regions.
2.1. Methods
2.1.1. Participants
Twenty right-handed participants (11 male, 9 female; ages 18–
26) were enrolled in this study. All participants were native speakers of English with normal or corrected-to-normal visual acuity
and no history of neurological problems (self-reported). Participants gave their informed consent to participate in this research
under the guidelines of the University of South Carolina Institutional Review Board. Data from one participant were excluded
from the study on the basis of excessive head motion.
2.1.2. Procedure
Participants underwent one 16-min fMRI scanning session after
an explanation of the task. Participants were asked to read silently
sentences that were presented using a back-projection mirror
located at the end of the scanner bore. Each trial consisted of
two sentences presented one at a time with each sentence displayed in isolation for 3960 ms with no gap between sentences.
The duration between trials varied from 4500 to 7680 ms, reducing
the correlation in hemodynamic responses between sentences
(i.e., the start of one item was not precisely phase-locked to the
start of the previous sentence). Sentence 1 introduced either one
or two people who were the grammatical subject in both
sentences. In sentence 2, one or both entities were referred to
using either a singular or plural pronoun. Because one of the present study’s aims was to assess whether the parietal activation
observed by Almor et al. (2007) was related to the representation
of multiple referents, we followed the design of that study by
introducing referents with proper names. Table 1 shows a sample
item in both conditions. Note that number of words was held constant across conditions through the addition of adjectives or adverbials to sentence 1. The session contained 54 experimental trials
mixed with 18 filler trials for a total of 72 trials. The filler trials
were also composed of two sentences, but these trials used varied
syntactic and discourse structures. The role of the filler trials was
to reduce participants’ expectancy for specific sentence structures,
especially pronouns referring to one or two people. The order of
the experimental and filler trials was randomized for each participant. The condition in which each item was shown to participants
was randomly selected with the constraints that each participant
saw the same number of items in each condition and that across
all participants each item appeared the same number of times in
each condition. Following about one third of the trials, participants
were given a simple ‘yes’ or ‘no’ question regarding the trial they
had just read. Thus each participant responded to 24 comprehension questions. Question duration was 4500 ms, and participants
indicated their response by pressing a button on a Psychology Software Tools, Inc. BrainLogics Fiber Optic Button Response System
(Pittsburgh, Pennsylvania, USA) glove placed on their left hand.
Participants used the thumb button to indicate a ‘yes’ response
and the index finger button to indicate a ‘no’ response. The comprehension questions probed information from the trial’s text
and allowed us to assess whether or not the participant was
engaged in the task. All participants responded accurately to at
least 70% of the questions, and therefore the data from all the participants were analyzed.
Scanning was performed on a 3T Siemens Trio scanner (Munich,
Germany) equipped with a 12-element head coil. A total of 322
echo-planar imaging (EPI) volumes (36 axial slices; 3.2 mm thick,
T.W. Boiteau et al. / Brain & Language 137 (2014) 130–141
Harvard-Oxford Cortical Structural Atlas, Juelich Histological Atlas,
and MNI Structural Atlas.
Table 1
A sample item in both conditions from Experiment 1.
Condition
Sentence 1
Sentence 2
Singular
Jeremy did some work on the house
next door.
Jeremy and Roger did some work on
the house.
He fixed the window and
the door.
They fixed the window and
the door.
Plural
Question
133
Were the stairs fixed?
with no gap between slices) were acquired during the fMRI session.
These scans used TR = 2.5 s; TE = 30 ms; matrix = 64 64 voxels;
flip angle = 90°, 208 208 mm FOV.
2.1.3. fMRI analyses
Preprocessing and statistical analyses of EPI data were carried
out using FEAT (fMRI Expert Analysis Tool) Version 5.91, part of
FSL 4.0 (fMRI Software Library, www.fmrib.ox.ac.uk/fsl). Lowerlevel analyses implemented motion correction, preserving the
skull, spatial smoothing using a Gaussian kernel of FWHM 8 mm,
mean-based intensity normalization of all volumes by the same
factor and high-pass temporal filtering (Gaussian-weighted LSF
straight line fitting, with r = 50.0 s). Time-series statistical analysis
was carried out with local autocorrelation correction. All EPI
images were normalized in Montreal Neurological Institute (MNI)
space. Each participant’s cortical activity was measured for the
duration of both sentences for the two different experimental conditions: Singular or Plural. Because we were interested in differences between discourses that had either one or two referents,
and because both sentence 1 and sentence 2 differed across conditions in number of referents, we contrasted in our model the activation associated with reading both sentences together in each
condition. Note that while it would have been interesting to separate the activation of each sentences in the model by including a
separate regressor for each sentence in each condition, this would
have resulted in a rank deficient design matrix due to the fact that
each version of sentence 1 was always followed by the same version of sentence 2 making a design matrix based on such regressors
highly inefficient. The results of this experiment should therefore
be interpreted as informative about the combined activation associated with both the generation of the original representation of
the referents and the subsequent anaphoric reference to these referents. While still informative about the discourse activations
associated with plural references, the results will not distinguish
between the activation associated with the generation of plural
representations and the activation associated with the repeated
references to these representations. We address this issue directly
in Experiment 2. In addition to the regressors for the two experimental conditions, the model included separate regressors for filler
items, questions, and the fixation immediately following questions
when participants responded to the questions. The fixations
between items were used as the implicit baseline and were therefore not specified explicitly in the model. For each participant, contrasts were set to evaluate the mean activation related to each
condition (Singular or Plural) and to compare the activation
between the two conditions. The group mean cortical activation
(higher-level analysis) was then calculated by investigating the
mean activation for each participant across different trials, and
the mean activation between different participants. Z-statistic
images were generated using a cluster threshold of Z > 2.3 and a
(corrected using Gaussian random field theory) cluster significance
threshold of p < 0.05. FSLview was utilized to display mean statistical maps on a standard brain template, and the anatomical
location of significant activation was confirmed with the
2.2. Results
We identified brain areas that showed significant increase in
signal during one condition compared to the other condition. The
Plural condition led to more activation than the Singular condition
in the parietal lobe bilaterally, specifically the precuneus, SPL,
anterior IPS, left angular and supramarginal gyri, and occipital
regions such as the right fusiform gyrus, right lingual gyrus, and
left lateral occipital cortex. In addition, there was bilateral activation in the cerebellum (Fig. 1; see Table 2 for a list of local cluster
activation peaks). No areas were significantly more active in the
Singular condition than the Plural condition.
2.3. Discussion
The observed parietal activation in the present study is informative about the representation and processing of conjoined NPs during language comprehension. In particular, this activation can help
distinguish between claims that conjoined NPs are represented as
an assemblage, a set of individual entities, or a CRO in which both
the conjoined entity and the individual component entities can be
easily accessed. Albrecht and Clifton (1998) argued that conjoined
NPs are represented as a single entity or assemblage. In contrast,
both Moxey et al. (2004) and Patson and Ferreira (2009) argued
that conjoined NPs are represented as CROs in which both the conjoined entity and each of the conjoined entities are accessible. The
observed parietal activation in the current study suggests that conjoined NPs are not represented as an assemblage but perhaps as a
CRO or atomic tokens (Johnson-Laird, 1983). Experiment 2 will
help differentiate between the CRO and atomic-token views.
Specifically, the increased activation observed in the precuneus
and SPL for the Plural condition over the Singular condition suggests that when multiple referents are introduced and referred to
in coherent discourse, multiple representations are formed and
maintained in the mental model. The similarity in parietal activation in this study and Almor et al.’s (2007) implies that these patterns of activation reflect the formation, manipulation, and
integration of multiple representations in the discourse rather than
differences in coherence between conditions. Within the parietal
lobes, subregions may be involved in different aspects of the maintenance and manipulation of these representations. We will discuss this possibility after Experiment 2.
Fig. 1. Brain areas that were more activated in the Plural compared to the Singular
condition in Experiment 1 (threshold: Z > 2.3, corrected cluster threshold p < 0.05).
Images were created using MRIcroGL (http://www.mccauslandcenter.sc.edu/
mricrogl/).
134
T.W. Boiteau et al. / Brain & Language 137 (2014) 130–141
Table 2
Brain areas that were more activated in the Plural condition than in the Singular condition (threshold: Z > 2.3, cluster threshold p (corrected for multiple comparisons) = 0.05).
Location is based on Harvard-Oxford Cortical Structural Atlas, Juelich Histological Atlas, and MNI Structural Atlas.
Size
Z
MNI spatial coordinates
x
y
Side
Location
z
26,864
3.86
3.71
3.71
3.71
3.67
3.65
46
14
14
12
40
41
70
86
86
88
81
79
40
16
18
21
41
41
Left
Right
Right
Right
Left
Right
Cerebellum
Occipital fusiform gyrus, V3v
Occipital fusiform gyrus
Occipital fusiform gyrus
Cerebellum
Cerebellum
26,271
3.51
3.44
3.44
3.35
3.17
3.17
24
17
27
32
25
35
64
61
74
72
64
54
30
30
42
42
28
24
Right
Left
Right
Right
Left
Left
Precuneus, anterior intraparietal sulcus
Precuneus
Lateral superior occipital cortex, superior parietal lobule 7P
Lateral superior occipital cortex, inferior parietal PGp
Lateral superior occipital cortex, optic radiation, anterior intraparietal sulcus
Angular gyrus, optic radiation, anterior intraparietal sulcus
Activation in the supramarginal gyrus likely reflects the cost of
accessing multiple lexical representations from memory (Gow,
2012). In the Plural condition, upon encountering they, participants
were accessing the stored lexical tokens from sentence 1. In the
Singular condition, participants only had to retrieve one token
from memory. Thus, the increased activation in the Plural condition may reflect accessing a greater number of lexical tokens from
memory.
Occipital activation is potentially a result of the general increase
in orthographic processing in the Plural condition, which included
an additional referent than the Singular condition (e.g., Dehaene, Le
Clec, Poline, Le Bihan, & Cohen, 2002;Yeatman, Rauschecker, &
Wandell, 2013). Although the Plural condition did not contain
more words than the Singular condition, it did have two capitalized
proper names compared to the one in the Singular condition,
which might have resulted in increased focus on orthographic
aspects of the words. The adverbials or adjectives at the end of
the first sentence in the Singular condition might have not drawn
as much attention, and thus might have been processed only cursorily. Experiment 2 addresses this potential concern by matching
the number of discourse entities in the different conditions, while
still keeping sentences between conditions identical in length.
3. Experiment 2
There are two other possible explanations of the Experiment 1
results which do not involve the construction and maintenance
of a CRO: 1. more discourse entities results in increased parietal
activation; 2. the conjunction of discourse entities in sentence 1
results in said activation. Experiment 1 did not allow us to tease
apart these two explanations as the number of referents and conjunction were confounded in this experiment. Thus, in Experiment
2 we aim to resolve this issue by manipulating the complexity of
the discourse model representation while holding the number of
mentioned entities constant. Following Sanford and Lockhart
(1990) and Moxey et al. (2004), items in this experiment introduced two referents in the first sentence that were either conjoined and appeared as a complex NP subject (e.g., John and Mary
. . .; Conjoined condition) or as collaborating in the same activity
via a with-phrase adjunct (e.g., John . . . with Mary; Unconjoined
condition). We expect that and-phrase antecedents will lead to
the formation of a CRO with the assemblage in focus while withphrase antecedents will lead to a CRO with the individuated entities more in focus. In order to ascertain whether both a conjoined
entity and individual referent representations are maintained in
either the Conjoined or Unconjoined condition, we included the
pronoun manipulation of sentence 2, as in Experiment 1 except
that here this resulted in a 2 2 design. If only an assemblage is
maintained in the discourse model, or if the assemblage is more
salient than the representations of the individual entities in the
CRO, then subsequent singular pronominal reference will result
in a processing cost. If both a conjoined entity and individual reference objects are maintained and have comparable discourse salience, singular pronominal reference will not lead to such a
processing cost. In this experiment, we expect that in the Unconjoined condition, atomic tokens will be generated, and in the Conjoined condition, a CRO will be generated, resulting in a more
efficient representation of the discourse entities. This hypothesis
thus leads to the following specific predictions: (1) the Unconjoined condition will lead to greater activation in the SPL than
the Conjoined condition, as in this condition the discourse model
is less efficient: (2) the Conjoined condition will not lead to any
greater parietal activation than the Unconjoined condition because
in the Conjoined condition the assemblage is in focus at the
expense of the individual representations, while in the Unconjoined condition, the individuals are in focus; (3) the left inferior
parietal lobule (IPL) will be more activated in the Plural condition
than the Singular condition, as the Plural Condition will result in
the activation of a richer lexical representation (just as it was in
E1). Finally, although our hypothesis does not make any a priori
predictions for an interaction effect, the presence or absence an
interaction would be informative about the relative salience of
the conjoined entity and the representation of the first-mentioned
discourse referent.
Unlike in Experiment 1, in this experiment the two conditions
of sentence 1 were crossed with the two conditions of sentence
2, which allows us to examine the activation associated with each
sentence separately, thus differentiating between activation associated with the generation of the discourse representations and
activation associated with the further processing of these
representations.
3.1. Methods
3.1.1. Participants
Twenty-eight right-handed participants (9 male, 19 female;
ages 18–27) were enrolled in this study. All participants were
native speakers of English with normal or corrected-to-normal
visual acuity and no history of neurological problems (selfreported). Participants gave their informed consent to participate
in this research under the guidelines of the University of South
Carolina Institutional Review Board. Data from two participants
were excluded from the study on the basis of excessive head
motion (>3 mm).
T.W. Boiteau et al. / Brain & Language 137 (2014) 130–141
Table 3
A sample item from Experiment 2 in each condition.
Condition
Sentence 1
Sentence 2
Conjoined
Singular
Conjoined
Plural
Unconjoined
Singular
Unconjoined
Plural
Jeremy and Lucy did some work
on the house.
Jeremy and Lucy did some work
on the house.
Jeremy did some work with Lucy
on the house.
Jeremy did some work with Lucy
on the house.
He fixed the window
and the door.
They fixed the window
and the door.
He fixed the window
and the door.
They fixed the window
and the door.
Question
Were the stairs fixed?
3.1.2. Procedure
Each participant underwent one 18-min session of fMRI scanning after an explanation of the task. Participants were asked to
read silently sentences that were presented using a back-projection mirror located at the end of the scanner bore. Each trial consisted of two sentences presented one at a time, with each
sentence displayed in isolation for 3960 ms, and no gap between
sentences. The duration between trials varied from 4500 to
7660 ms, reducing the correlation in hemodynamic responses
between sentences (i.e., the start of one item was not precisely
phase-locked to the start of the previous sentence). Sentence 1
introduced two characters. In the Conjoined condition both characters were the subject of a conjoined and-phrase, and in the Unconjoined condition the subject appeared alone while the partner
appeared later in the sentence as the head of a with-phrase adjunct.
In sentence 2, the first-mentioned or both entities together were
referred to using either a singular or plural pronoun respectively.
This resulted in four possible unique combinations for each item.
Table 3 shows a sample item in each condition. The session contained 80 experimental trials mixed with 20 filler trials for a total
of 100 trials. The filler trials were also composed of two sentences,
but these trials used varied structures. The role of the filler trials
was to reduce participants’ expectancy for specific sentence structures, especially pronouns referring to one or two people. The order
of the experimental and filler trials was randomized for each participant. The condition in which each item was shown to participants was randomly selected with the constraints that each
participant saw the same number of items in each condition, that
across all participants each item appeared the same number of
times in each condition, and that each participant saw each item
in only one condition. Following about one third of the trials, participants were given a simple ‘yes’ or ‘no’ question regarding the
trial they had just read. Question duration and response method
was identical to Experiment 1. The comprehension questions
probed information from the trial’s text and allowed us to assess
whether or not the participant was engaged in the task. All participants responded accurately to at least 70% of the questions and
therefore the data from all the participants were analyzed.
The scanner and scanning details were identical to Experiment 1.
3.1.3. fMRI analyses
Preprocessing and statistical analyses of EPI data were carried
out using FEAT (FMRI Expert Analysis Tool) Version 5.91, part of
FSL 4.0 (FMRI Software Library, www.fmrib.ox.ac.uk/fsl). Lowerlevel analyses implemented motion correction, preserving the
skull, spatial smoothing using a Gaussian kernel of FWHM 8 mm,
mean-based intensity normalization of all volumes by the same
factor and high-pass temporal filtering (Gaussian-weighted LSF
straight line fitting, with r = 50.0 s). Time-series statistical analysis
was carried out with local autocorrelation correction. All EPI
images were normalized in MNI space.
135
Unlike in Experiment 1, here the two versions of sentence 1 and
the two versions of sentence 2 were not confounded, thus allowing
us to model each participant’s cortical activity separately for the
two sentences. Thus, the model included two regressors for the first
3960 ms of the trial corresponding to the two versions of sentence 1
(conjoined and unconjoined) and four regressors for each of the
3960–7920 ms in the trial corresponding to the four conditions of
sentence 2 (single pronoun following a conjoined reference, single
pronoun following unconjoined reference, plural pronoun following conjoined reference, and plural pronoun following unconjoined
reference). Although these four regressors are not fully independent
of the two sentence 1 regressors, they nevertheless allow us to analyze the interaction of the factors in the activation associated with
reading sentence 2, bearing in mind that any effects involving conjunction type may have originated in the reading of either sentence
1 or sentence 2.1 There were also three additional regressors for filler
items, questions, and fixations immediately following questions
when participants responded to the questions. The fixations between
items were used as the implicit baseline and were therefore not specified explicitly in the model. For each participant, two contrasts were
set to compare the activation of the two sentence 1 conditions
(Conjoined > Unconjoined and Unconjoined > Conjoined) and two
contrasts were set to compare the activation of the singular and
plural sentence 2 conditions (Singular > Plural and Plural > Singular).
In addition, two interaction contrasts were set in order to test the
interaction of the two factors (UnconjoinedSingular + ConjoinedPlural > UnconjoinedPlural + ConjoinedSingular and UnconjoinedPlural +
ConjoinedSingular > UnconjoinedSingular + ConjoinedPlural). We also
tested for the main effect of conjoined vs. unconjoined for sentence 2
(Conjoined > Unconjoined and Unconjoined > Conjoined). Note that
due to the correlation between this contrast and the sentence 1 contrasts, care must be taken in interpreting the results of these contrasts.
The group mean cortical activation (higher-level analysis) was
then calculated by investigating the mean activation for each participant across different trials, and the mean activation between
different participants. Z-statistic images were generated using a
cluster threshold of Z > 2.3 and a cluster significance threshold of
p < 0.05 (corrected using Gaussian random field theory). FSLview
was utilized to display mean statistical maps on a standard brain
template, and the anatomical location of significant activation
was confirmed with the Harvard-Oxford Cortical Structural Atlas,
Juelich Histological Atlas, and MNI Structural Atlas. In addition to
the high level contrasts, we also performed region of interest analyses (ROI) using the Juelich Histological Atlas in FSL’s Featquery
tool on specific areas we found active in Experiment 1. These
regions were in the left and right SPL and the left and right IPL.
We also examined classical language areas in the left inferior frontal gyrus (IFG). Our reason for examining the latter area is to better
distinguish differences that are likely due to syntactic factors from
those likely reflecting discourse processes (Table 4 provides a list of
all ROIs examined).
3.2. Results
For sentence 1, we contrasted the two conditions (Conjoined
and Unconjoined). In the whole-brain analyses, we found no activation associated with Conjoined greater than Unconjoined or
Unconjoined greater than Conjoined.
For the sentence 1 ROIs, we only report significant effects. Consult Fig. 2 for mask locations and Fig. 3 for a graphical summary of
1
Note that this is necessarily a problem with discourse processing designs, in
which context impacts later processing of linguistic stimuli. In addition to the abovereported analyses, we also modeled the data using two regressors for sentence 1 and
two regressors for sentence 2. This did not lead to any additional findings than the
ones reported in this paper.
136
T.W. Boiteau et al. / Brain & Language 137 (2014) 130–141
Table 4
List of all ROIs by hemisphere examined in E2.
Left
Right
Hemisphere
Anterior intra-parietal sulcus hIP1
Anterior intra-parietal sulcus hIP2
Anterior intra-parietal sulcus hIP3
BA 44
BA 45
Inferior parietal lobule PF
Inferior parietal lobule PFcm
Inferior parietal lobule PFm
Inferior parietal lobule PFop
Inferior parietal lobule PFt
Inferior parietal lobule Pga
Inferior parietal lobule Pgp
Superior parietal lobule 5Ci
Superior parietal lobule 5L
Superior parietal lobule 5M
Superior parietal lobule 7A
Superior parietal lobule 7M
Superior parietal lobule 7P
Superior parietal lobule 7PC
Anterior intra-parietal sulcus hIP1
Anterior intra-parietal sulcus hIP2
Anterior intra-parietal sulcus hIP3
–
–
Inferior parietal lobule PF
Inferior parietal lobule PFcm
Inferior parietal lobule PFm
Inferior parietal lobule PFop
Inferior parietal lobule PFt
Inferior parietal lobule Pga
Inferior parietal lobule Pgp
Superior parietal lobule 5Ci
Superior parietal lobule 5L
Superior parietal lobule 5M
Superior parietal lobule 7A
Superior parietal lobule 7M
Superior parietal lobule 7P
Superior parietal lobule 7PC
Unconjoined Singular greater than Conjoined Plural, and Conjoined
Singular plus Unconjoined Plural greater than Conjoined Plural
plus Unconjoined Singular (Fig. 4 and Table 5). The Conjoined Singular greater than Conjoined Plural contrast found activation in the
right occipital pole, right lateral occipital cortex, and right fusiform
gyrus. The Unconjoined Plural greater than Conjoined Plural contrast found activation in the left occipital pole, left lateral superior
occipital cortex, and left fusiform gyrus. The Unconjoined Singular
greater than Conjoined Plural contrast found activation in the right
lateral inferior occipital cortex and right fusiform gyrus. Finally, the
interaction effect, Conjoined Singular plus Unconjoined Plural
greater than Conjoined Plural plus Unconjoined Singular, found
activation in the left occipital pole, left lateral occipital cortex,
and left lateral inferior occipital cortex.
Next, we report the results of the ROIs. For the sake of space, we
only report those effects that were significant. Consult Fig. 5 for
mask locations and Fig. 6 for a graphical summary of all analyses.
We found a main effect of Conjunction in left 7P (posterior SPL),
F(1, 25) = 12.63, p = .002, but no effect of Number and no interaction. There was also a main effect of Conjunction in right 7P,
F(1, 25) = 8.24, p = .01, again with no effect of Number nor an interaction between the two variables. In left 7M (medial SPL) there was
again a main effect of Conjunction, F(1, 25) = 13.02, p = .001, no
effect of Number and no interaction. In right 7 M there was a main
effect of Conjunction, F(1, 25) = 7.09, p = .01, no effect of Number
and no interaction. In left 7A (anterior SPL) there was again a main
effect of Conjunction, F(1, 25) = 5.47, p = .03, no main effect of
Number and no interaction. Finally, there was a main effect of
Number in left PFcm (anterior SMG) with greater activation in
the Plural than the Singular condition, F(1, 25) = 5.05, p = .03, but
no effect of Conjunction and no interaction. All the Conjunction
effects were such that there was greater activation in the Unconjoined than in the Conjoined condition. Analyses of left IFG regions
did not reveal any significant differences in activation.3
3.3. Discussion
Fig. 2. Location of masks for sentence 1 ROIs from Experiment 2 that yielded
significant activation (and their contralateral homologues). Regions in cyan are left
and right Pga. Regions in yellow are left and right 7M.
the analyses. We found a main effect of Conjunction in right 7P
(posterior SPL), F(1, 25) = 4.81, p = .04, with Conjoined leading to
more activation than Unconjoined. There was also a marginal effect
in the left homologue of this region, F(1, 25) = 3.69, p = .07, again
with Conjoined leading to more activation than Unconjoined. We
also found a main effect in the right Pga (angular gyrus),
F(1, 25) = 4.39, p = .05, with Conjoined leading to more activation
than Unconjoined, but there was no effect in the left homologue,
F < 1. There were no effects observed in the left IFG.2
For sentence 2, 4 of the contrasts turned up significant patterns
of activation. These contrasts were Conjoined Singular greater than
Conjoined Plural, Unconjoined Plural greater than Conjoined Plural,
2
Although these contrasts were preplanned based on the findings from Experiment
1 and similar patterns of activity in other neuroimaging reference studies (e.g., Almor
et al., 2007), if we were to apply a correction for multiple comparisons, none of the
observed effects would be statistically significant using the Bonferroni method
(a = .001515).
While the whole-brain analyses of sentence 1 found no effects,
the ROIs showed that in fact the Conjoined condition did result in
greater activation than the Unconjoined condition. We interpret
this as an effect of assemblage or CRO construction, which occurs
mainly in the case of conjoined structures. This shows that to some
extent the findings from Experiment 1 are due to the initial construction of these complex plural representations. It is interesting
to note that left hemisphere activation was minimal, with more
right parietal structures being involved in this process of plural
entity construction.
The high-level contrasts of sentence 2 only revealed activation
in early occipital regions, replicating Experiment 1 and likely
reflecting orthographic differences between conditions rather than
discourse level processing (Cohen et al., 2000). These occipital
regions have also been implicated in color, face, and object recognition (Kanwisher, McDermott, & Chun, 1997; Lueck et al., 1989;
Malach et al., 1995), so in addition to the possibility that activity
here is a result of differences in lexical processing, it also might
be the case that the activity reflects different patterns of imagery
evoked in reading the discourses. An interesting pattern in three
of these contrasts is that everything compared to Conjoined Plural
turned up these early occipital activations. It is possible that in
general, Conjoined Plural is the easiest or most natural of these discourse structures to process, and that the other versions involve
3
Again, these comparisons were planned based on previous results, but if we were
to apply a Bonferroni correction (a = .001515), only the right 7M in the Conjunction
analysis would be considered significant and the left 7P in the Conjunction analysis
would be borderline.
137
T.W. Boiteau et al. / Brain & Language 137 (2014) 130–141
7M
0.15
Le
Pga
Le
Right
*
0.12
0.09
Right
0.03
0
Con
Uncon
-0.03
0.06
-0.06
0.03
-0.09
0
-0.12
-0.03
-0.15
Con
Uncon
*
Fig. 3. Results of the sentence 1 ROI analyses from Experiment 2, showing percent BOLD signal change on the y-axis by hemisphere and condition on the x-axis, dividing each
ROI into a separate panel. Bars indicate standard error of the mean. Significant contrasts are marked with brackets and asterisks.
Fig. 4. Brain areas more activated in (a) Conjoined Singular than Conjoined Plural, (b) Unconjoined Plural than Conjoined Plural, (c) Unconjoined Singular than Conjoined
Plural, and (d) Conjoined Singular + Unconjoined Plural than Conjoined Plural + Unconjoined Singular in Experiment 2 (threshold: Z > 2.3, corrected cluster threshold
p < 0.05). Images were created using MRIcroGL (http://www.mccauslandcenter.sc.edu/mricrogl/).
slightly more complex situations to visualize. Alternatively, it
might be due to encountering the unexpected pronoun. Indeed,
using magnetoencephalography (MEG), Dikker, Rabagliati,
Farmer, and Pylkkänen (2010) found early occipital regions activated in violations of expected word forms. While that study was
only measuring activation associated with a single word using
MEG and ours looks at discourse-level processes using fMRI, the
interaction effect we observed can be understood along similar
lines. Both Conjoined Singular and Unconjoined Plural are the less
felicitous combinations, as a conjoined subject typically will be
referred to with a plural pronoun and a subject participating with
a with-phrase partner will normally be referred to subsequently
with a singular pronoun (Moxey et al., 2004). Therefore, the fact
that we see occipital activation when these two cases are compared with the more felicitous versions (i.e., Conjoined Plural plus
Unconjoined Singular) would seem to suggest the occipital region’s
sensitivity to these violations. In other words, the different sentence 1 conjunctions led participants to anticipate different types
of pronominal reference in the subsequent sentence, and when
these expectations were violated, the result was increased activation in these occipital regions. Whether these findings reflect the
orthographic differences between the conditions or violations of
expectancies, our main focus here is on the parietal activations,
which we discuss next.
Notably, the high-level contrasts of sentence 2 did not reveal
any activation in the SPL. The ROIs of sentence 2, however, revealed
that indeed conjunction type impacts the degree of accessibility of
referents in the discourse. While previous literature has argued
that and-phrase conjunctions bind entities together such that
accessing those bound individuated representations results in an
unbinding cost (Albrecht & Clifton, 1998), we show here that this
description is somewhat oversimplified. The Unconjoined condition consistently led to greater SPL activation than the Conjoined
condition, which we attribute to the greater effort resulting from
maintaining a more complex discourse representation. Again,
these findings are in line with Almor et al. (2007), showing that
the brain recruits parietal regions in order to maintain discourselevel representations. The lack of any interaction in these regions
138
T.W. Boiteau et al. / Brain & Language 137 (2014) 130–141
Table 5
Brain areas that were more activated in each pairwise comparison (threshold: Z > 2.3, cluster threshold p (corrected for multiple comparisons) = 0.05). Location is based on
Harvard-Oxford Cortical Structural Atlas, Juelich Histological Atlas, and MNI Structural Atlas.
Size
Z
MNI spatial coordinates
x
y
Side
Location
z
Conjoined Singular > Conjoined Plural
1386
4.16
3.1
3.04
2.9
2.9
2.84
26
48
38
2
2
58
100
86
66
106
104
72
4
8
12
12
12
8
Right
Right
Right
Right
Right
Right
Occipital pole, V1
Lateral occipital cortex, V4
Occipital fusiform gyrus, V4
Occipital pole
Occipital pole
Lateral occipital cortex
Unconjoined Plural > Conjoined Plural
2393
3.68
3.62
3.34
3.33
3.31
3.26
36
18
34
36
30
4
80
76
88
70
86
98
20
14
20
18
10
4
Left
Left
Left
Left
Left
Left
Occipital fusiform gyrus, V4
Occipital fusiform gyrus, V4
Lateral occipital cortex
Occipital fusiform gyrus, V4
Lateral superior occipital cortex
Occipital pole, V1
Unconjoined Singular > Conjoined Plural
1340
3.75
3.35
3.35
3.35
3.18
2.92
58
42
44
26
48
28
70
66
74
84
70
70
4
16
14
16
14
14
Right
Right
Right
Right
Right
Right
Lateral inferior occipital cortex
Occipital fusiform gyrus, V4
Lateral inferior occipital cortex, V4
Occipital fusiform gyrus, V3v
Lateral inferior occipital cortex, V4
Occipital fusiform gyrus, V4
10
8
8
10
10
20
Left
Left
Left
Left
Left
Left
occipital pole
Occipital pole, V2
Occipital pole, V3v
Occipital pole, V2
Lateral occipital cortex, V4
Lateral inferior occipital cortex
Conjoined Singular + Unconjoined Plural > Conjoined Plural + Unconjoined Singular
1151
3.48
26
106
3.47
22
104
3.38
32
96
3.37
30
100
3.09
44
88
3.04
36
88
Fig. 5. Location of masks for sentence 2 ROIs from Experiment 2 that yielded
significant activation (and their contralateral homologues). Regions in green are left
and right 7A. Regions in yellow are left and right 7M. Regions in blue are left and
right 7P. Regions in red are left and right PFcm.
suggests that both the conjoined entity and the individuated representation of the first-mentioned referent were equally accessible
from memory in Conjoined and Unconjoined conditions, which is
not surprising given that only two entities existed in the discourse
and they were sharing agency. The activation in the left IPL for Plural greater than Singular is in line with the results from Experiment
1 and suggests a more complex lexical activation induced by
accessing both referents from memory (Gow, 2012). It is interesting to note that no differences in IPS activation were observed in
this experiment, whereas we did find such activation in Experiment 1. This may indicate this region’s involvement in tracking
multiple discourse entities, much as it has been found to be
involved in multiple object tracking (Culham et al., 1998). Importantly, there were no differences (in this or the previous experiment) in the classic language areas (i.e., left IFG), reducing the
likelihood of a syntactic confound.
The difference in findings from the sentence 1 and sentence 2
ROIs is highly informative. The greater superior parietal activation
in sentence 1 in the Conjoined condition indicates the effort
involved in constructing a discourse model that includes an assemblage or a CRO. With the reservation that the activation differences
due to conjunction type were not fully independent in sentences 1
and 2, the greater superior parietal activation in sentence 2 in the
Unconjoined condition may show the greater complexity of the
discourse model in that condition while processing this sentence.
In other words, the initial increased effort in constructing the discourse model (i.e., constructing an assemblage or CRO) results in a
more efficient mental representation later on in the discourse. This
highlights the power simple differences in syntactic structure can
have on the construction of the mental model.
4. General discussion
We have reported two experiments that tested the effect of
number and conjunction type on the formation and accessibility
of discourse representations. The findings of these two studies suggest that when multiple referents are introduced and referred to in
coherent discourse, multiple representations are formed. These
representations are overlapping, such that collaborative actions
performed by distinctly identified individuals (as opposed to plural
definite entities; Patson & Ferreira, 2009) will produce a CRO under
certain conditions. This finding makes an important contribution
139
T.W. Boiteau et al. / Brain & Language 137 (2014) 130–141
7A
7M
Le
0.05
Con
Right
Uncon
Con
Le
Uncon
0.06
0.02
Con
Right
Uncon
Singular
-0.04
Singular
-0.06
-0.1
-0.1
-0.14
-0.13
Plural
-0.02
-0.07
*
-0.18
*
*
7P
PFcm
Le
Con
Right
Uncon
Con
Le
Uncon
0.02
Plural
Right
Singular
Plural
Singular
0
0.05
Plural
0
Singular
-0.05
Con
-0.02
Uncon
-0.04
-0.06
-0.1
-0.15
-0.2
Uncon
0.02
Plural
-0.01
0.1
Con
*
*
*
-0.08
-0.1
Fig. 6. Results of the sentence 2 ROI analyses from Experiment 2, showing percent BOLD signal change on the y-axis by hemisphere and condition on the x-axis, dividing each
ROI into a separate panel. Bars indicate standard error of the mean. Significant contrasts are marked with brackets and asterisks.
to the literature on plural processing, which so far has largely
ignored the possibility that the relative saliency of the CRO and
the entities that comprise it may vary under different conditions
(in our case, manner of introduction into the discourse). This conclusion also lends support to the finding that parietal activation
observed in both the Almor et al. (2007) and Nieuwland et al.
(2007) studies was due to the integration and manipulation of
multiple representations rather than differences in coherence
between conditions.
Previous research has shown that when multiple referents are
encountered in discourse under conditions of shared agency, an
assemblage token is generated and stored in the discourse model
along with the representations of the individual referents
(Barker, 1992; Moxey et al., 2004). Patson and Warren (2011)
showed that the degree of differentiation (i.e., semantic properties)
between discourse entities impacts the structure of a CRO, creating
pointers to specific referents. Here we show that the accessibility
of discourse referents in memory and the salience of the plural
entity are impacted by how referents are introduced into the
discourse (i.e., syntactic factors). In the case of a single-referent discourse, one entity is maintained in the mental model. In plural-referent discourse, two entities are maintained, but the manner in
which they are represented is impacted by conjunction type (among
other factors). In the case of a conjoined noun phrase, a CRO is generated, with pointers to the individual referents. In the case of an
unconjoined noun phrase, two individual representations are
introduced and maintained, and the result of our experiments
show that maintaining these separate representations is more
taxing on attentional resources (see Table 6). The result as we
move from singular to plural-conjoined and plural-unconjoined is
an increasingly complex (and less efficiently maintained) discourse
model. This is apparent by the increased involvement of parietal
regions as the discourse model grows in complexity.
Our results are informative about other theories of plural processing. According to the assemblage account of plural reference
(Albrecht & Clifton, 1998), plural conjoined references result in a
representation of the conjoined NP such that any subsequent reference to one of its components incurs a ‘‘conjunction cost.’’ The
results of our first experiment are incompatible with this theory
because if the conjoined NPs were represented as an assemblage,
as Albrecht and Clifton argue, there should not have been differences in parietal activation between the conditions.
Our findings are also incompatible with the atomic-token view
of plural references (Johnson-Laird, 1983). This view can account
for the results of Experiment 1, in that the higher activation in
the Plural than in the Singular conditions may reflect either the
cost associated with creating a conjoined referent as an antecedent
for the plural pronoun, or simply maintaining a greater number of
entities in the discourse model. Nevertheless, this view cannot
account for the results of Experiment 2, in which we found greater
activation in the SPL for the Unconjoined than for the Conjoined
conditions, regardless of the reference form.
Table 6
Schematic depiction of discourse representations from sample items in Experiment 1 and 2.
Condition
Singular
Plural Conjoined
Plural Unconjoined
Example
Mental representation
John went to the store.
John and Mary went to the store.
John went to the store with Mary.
John
John
John
Mary
Mary
140
T.W. Boiteau et al. / Brain & Language 137 (2014) 130–141
Our results are largely compatible with the CRO account of
Moxey et al. (2004). Here, we focused on the difference in the form
of referent introduction as a factor influencing how these representations are stored and maintained. However, there is substantial
research showing the role of other factors as well. For example, collaborative verbs such as the ones used in Moxey et al. (2011) are
known to increase the likelihood of future plural reference. We
therefore predict that these verbs are more likely to produce an
effect similar to the one we observed in our with-phrase conditions
(i.e., competition between conjoined and individual referents) than
other verbs. Similarly, Moxey et al. found that when referents are
performing the same roles in different locations they are less likely
to be referred to via the use of a plural reference. More generally,
we predict that any information in the discourse that would lead
to the separation between characters in the discourse model would
reduce the salience of the conjoined representation. Moreover, we
predict that the more entities in the discourse model, the larger the
processing load. Thus, a discourse consisting of two entities will
lead to more activation than one, three will lead to greater
activation than two, and so on until reaching the point of memory
saturation. Following Almor et al. (2007), we predict that this
difficulty will be greater when differences in saliency between
the representations are small than when they are large, and that
this difficulty will be reflected in activation of the SPL (for an alternative view on the neural correlates of introducing new referents
into the discourse model, see Nieuwland, 2014 and Nieuwland
et al., 2007).
One possible explanation for the results observed in these
experiments is that the observed activation in IPS and BA 7 reflect
generally greater attentional demands by plurals compared to singular, or unconjoined vs. conjoined plural discourse (e.g.,
Behrmann, Geng, & Shomstein, 2004). In the absence of any theoretical grounds to expect such differences in attentional demands,
it is unclear whether this explanation can be supported, and if it
can, whether it would provide any insight into the underlying processes and representations. An alternative explanation, and one
that is line with other theoretical accounts both from behavioral
and neuroimaging studies (e.g., Almor & Nair, 2007; Almor et al.,
2007; Dehaene et al., 2003; Gordon & Hendrick, 1998; Hubbard
et al., 2005) is that these regions, which have been implicated separately in spatial attention, numerical processing and the interaction between the two processes, and more specifically with
tracking multiple objects in space (Culham & Kanwisher, 2001;
Culham et al., 1998; Harvey et al., 2013; Piazza & Izard, 2009), were
recruited by language to help support the tracking and representation of referents in discourse. This mechanism is utilized for tracking all kinds of referents, singular and plural, and is taxed by
referential load induced by both the number of tracked referents
and their differentiability in levels of saliency.
In summary, we have provided evidence for the CRO, an account
of plural reference processing. We have shown that the representation of multiple referents in the discourse model recruits parietal
regions associated with spatial and numerical processing, increasing the number of entities in the mental model leads to an increase
in processing load in these regions, and this in turn is modulated
by the saliency of these entities, which is impacted by syntactic
and pragmatic factors.
Acknowledgments
This research was partially funded by NIH R21AG030445 and
NSF BCS0822617 grants.
The authors wish to thank Chris Rorden for help with ROI analyses and Kat Wilson and Geetanjali Pathak for help with data
collection.
References
Albrecht, J., & Clifton, C. (1998). Accessing singular antecedents in conjoined
phrases. Memory and Cognition, 26(3), 599–610.
Almor, A. (1999). Noun-phrase anaphora and focus: The informational load
hypothesis. Psychology Review, 106, 748–765.
Almor, A., & Nair, V. A. (2007). The form of referential expressions in discourse.
Language and Linguistics Compass, 1(1–2), 84–99.
Almor, A., Smith, D., Bonilha, L., Fridriksson, J., & Rorden, C. (2007). What is in a
name? Spatial brain circuits are used to track discourse references. NeuroReport,
18(12), 1215–1219.
Ariel, M. (1990). Accessing noun-phrase antecedents. London; New York: Routledge.
Barker, C. (1992). Group terms in English: Representing groups as atoms. Journal of
Semantics, 9, 69–93.
Beauchamp, M. S., Petit, L., Ellmore, T. M., Ingeholm, J., & Haxby, J. V. (2001). A
parametric fMRI study of overt and covert shifts of visuospatial attention.
Neuroimage, 14, 310–321.
Behrmann, M., Geng, J. J., & Shomstein, S. (2004). Parietal cortex and attention.
Current Opinion in Neurobiology, 14, 212–217.
Bookheimer, S. (2002). Functional MRI of language: New approaches to
understanding the cortical organization of semantic processing. Annual
Review of Neuroscience, 25, 151–188.
Cohen, L., Dehaene, S., Naccache, L., Lehéricy, S., Dehaene-Lambertz, G., Hénaff, M.A., et al. (2000). The visual word form area: Spatial and temporal
characterization of an initial stage of reading in normal subjects and posterior
split-brain patients. Brain, 123, 291–307.
Corbetta, M., Miezin, F. M., Shulman, G. L., & Petersen, S. E. (1993). A PET study of
visuospatial attention. The Journal of Neuroscience, 13(3), 1202–1226.
Culham, J. C., Brandt, S. A., Cavanagh, P., Kanwisher, N. G., Dale, A. M., & Tootell, R. B.
H. (1998). Cortical fMRI activation produced by attentive tracking of moving
targets. Journal of Neurophysiology, 80(5), 2657–2670.
Culham, J., & Kanwisher, N. (2001). Neuroimaging of cognitive functions in human
parietal cortex. Current Opinion in Neurobiology, 11, 157–163.
Dehaene, S., Le Clec, H. G., Poline, J. B., Le Bihan, D., & Cohen, L. (2002). The visual
word form area: A prelexical representation of visual words in the fusiform
gyrus. Neuroreport, 13(3), 321–325.
Dehaene, S., Piazza, M., Pinel, P., & Cohen, L. (2003). Three parietal circuits for
number processing. Cognitive Neuropsychology, 20, 487–506.
Dikker, S., Rabagliati, H., Farmer, T. A., & Pylkkänen, L. (2010). Early occipital
sensitivity to syntactic category is based on form typicality. Psychological
Science, 21(5), 629–634.
Farah, M. J., Wong, A. B., Monheit, M. A., & Morrow, L. A. (1989). Parietal lobe
mechanisms of spatial attention: Modality-specific or supramodal?
Neuropsychologia, 27(4), 461–470.
Gelormini-Lezama, C., & Almor, A. (2013). Singular and plural pronominal reference
in Spanish. Journal of Psycholinguistic Research.
Gordon, P. C., Grosz, B. J., & Gilliom, L. A. (1993). Pronouns, names, and the centering
of attention in discourse. Cognitive Science, 17, 311–347.
Gordon, P. C., & Hendrick, R. (1998). The representation and processing of
coreference in discourse. Cognitive Science, 22(4), 389–424.
Gow, D. W. Jr., (2012). The cortical organization of lexical knowledge: A dual lexicon
model of spoken language processing. Brain and Language, 121, 273–288.
Grosz, B. J., Joshi, A. K., & Weinstein, S. (1995). Centering: A framework for modeling
the local coherence of discourse. Computational Linguistics, 21, 203–226.
Harvey, B., Klein, B., Petridou, N., & Dumoulin, S. (2013). Topographic representation of
numerosity in human parietal lobe. Perception. EVCP Abstract Supplement, p. 241.
Hubbard, E. M., Piazza, M., Pinel, P., & Dehaene, S. (2005). Interactions between
number and space in parietal cortex. Nature Reviews Neuroscience, 6(6),
435–448.
Johnson-Laird, P. N. (1983). Mental models. Cambridge: Cambridge University Press.
Kanwisher, N., McDermott, J., & Chun, M. M. (1997). The fusiform face area: A
module in human extrastriate cortex specialized for face perception. The Journal
of Neuroscience, 17(11), 4302–4311.
Kaup, B., Kelter, S., & Habel, C. (2002). Representing referents of plural expressions
and resolving plurals anaphors. Language and Cognitive Processes, 17(4), 405–450.
Lueck, C. J., Zeki, S., Friston, K. J., Deiber, M.-P., Cope, P., Cunningham, V. J., et al.
(1989). The colour centre in the cerebral cortex of man. Nature, 340, 386–389.
Malach, R., Reppas, J. B., Benson, R. R., Kwong, K. K., Jiang, H., Kennedy, W. A., et al.
(1995). Object-related activity revealed by functional magnetic resonance
imaging in human occipital cortex. Proceedings of the National Academy of
Science USA, 8135–8139.
McMillan, C. T., Clark, R., Gunawardena, D., Ryant, N., & Grossman, M. (2012). FMRI
evidence for strategic decision-making during resolution of pronoun reference.
Neuropsychologia, 50(5), 674–687.
Moxey, L. M., Sanford, A. J., Sturt, P., & Morrow, L. I. (2004). Constraints on the
formation of plural reference objects: The influence of role, conjunction, and
type of description. Journal of Memory and Language, 51, 346–364.
Moxey, L. M., Sanford, A. J., Wood, A. I., & Gintner, L. M. N. (2011). When do we use
‘‘they’’ to refer to two individuals? Scenario mapping as a basis for equivalence.
Language and Cognitive Processes, 26(1), 79–120.
Nieuwland, M. S. (2014). ‘‘Who’s he?’’ Event-related brain potentials and unbound
pronouns. Journal of Memory and Language, 76, 1–28.
Nieuwland, M. S., Petersson, K. M., & Van Berkum, J. J. A. (2007). On sense and
reference: Examining the functional neuroanatomy of referential processing.
NeuroImage, 37, 993–1004.
T.W. Boiteau et al. / Brain & Language 137 (2014) 130–141
Patson, N. D., & Ferreira, F. (2009). Conceptual plural information is used to guide
early parsing decisions: Evidence from garden-path sentences with reciprocal
verbs. Journal of Memory and Language, 60, 464–486.
Patson, N. D., & Warren, T. (2011). Building complex reference objects from dual
sets. Journal of Memory and Language, 64, 443–459.
Piazza, M., & Izard, V. (2009). How humans count: Numerosity and the parietal
cortex. Neuroscientist, 15(3), 261–273.
Sanford, A. J., & Garrod, S. C. (1981). Understanding written language. Chichester:
John Wiley & Sons.
Sanford, A. J., & Lockhart, F. (1990). Description types and methods of conjoining as
factors influencing plural anaphors: A continuation study of focus. Journal of
Semantics, 7, 365–378.
141
Shimozaki, S. S., Hayhoe, M. M., Zelinsky, G. J., Weinstein, A., Merigan, W. H., &
Ballard, D. H. (2003). Effect of parietal lobe lesions on saccade targeting and
spatial memory in a naturalistic visual search task. Neuropsychologia, 41,
1365–1386.
van Dijk, T. A., & Kintsch, W. (1983). Strategies of discourse comprehension. New York:
Academic Press.
Wojuciulik, E., & Kanwisher, N. (1999). The generality of parietal involvement in
visual attention. Neuron, 23, 747–764.
Yeatman, J. D., Rauschecker, A. M., & Wandell, B. A. (2013). Anatomy of the visual
word form area: Adjacent cortical circuits and long-range white matter
connections. Brain and Language, 125, 146–155.