Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Neuroeconomics wikipedia , lookup
Aging brain wikipedia , lookup
Affective neuroscience wikipedia , lookup
Neurophilosophy wikipedia , lookup
Neurolinguistics wikipedia , lookup
Neuroanatomy of memory wikipedia , lookup
Cognitive neuroscience of music wikipedia , lookup
Neuroesthetics wikipedia , lookup
Brain & Language 137 (2014) 130–141 Contents lists available at ScienceDirect Brain & Language journal homepage: www.elsevier.com/locate/b&l The neural representation of plural discourse entities Timothy W. Boiteau a,⇑, Eric Bowers a, Veena A. Nair b, Amit Almor a a b University of South Carolina, USA Dept. of Radiology, School of Medicine and Public Health, University of Wisconsin Madison, USA a r t i c l e i n f o Article history: Accepted 6 August 2014 Keywords: Discourse processing Reference Parietal lobes fMRI Neuroimaging Plurals Conjunction a b s t r a c t Little is known about the underlying neural structures that mediate the generation and tracking of discourse referents. In two functional magnetic resonance imaging experiments, we examined the neural structures involved in generating and maintaining the representations of multiple referents. Experiment 1 used two-sentence discourses with singular and plural conditions linking back to single or conjoined subjects. In Experiment 2, conjunction type was manipulated in order to keep the number of discourse entities constant across the discourse. Both experiments found greater activation in the superior parietal lobule bilaterally for plural entities relative to singular entities in Experiment 1 and for unconjoined plural entities relative to conjoined plural entities in Experiment 2. This parietal activation suggests that referring to multiple entities evokes multiple representations that need to be integrated and tracked. We discuss these findings in terms of psycholinguistic theories of multiple referent representations. Ó 2014 Elsevier Inc. All rights reserved. 1. Introduction Successful language comprehension requires that listeners and readers relate incoming information to previously processed linguistic input (Grosz, Joshi, & Weinstein, 1995; Sanford & Garrod, 1981). The relation between new and previously mentioned information is often established through anaphora, i.e., the subsequent reference to a previously mentioned referent (Ariel, 1990; van Dijk & Kintsch, 1983). The influx of new information in a discourse makes it important for the comprehender to track references quickly and efficiently, and this entails generating and maintaining detailed representations of mentioned referents so that they can be integrated into the discourse model and linked to subsequent repeated references. The salience—i.e., prominence in the discourse model—of a given entity affects how easily this entity can be accessed from memory, and thus the representation and tracking of references requires the management of memory resources (Almor, 1999; Almor & Nair, 2007; Sanford & Garrod, 1981). Our focus in this study is on the differences between singular and plural references. Specifically, we investigate how such plural references are represented in the brain. Before describing our experiments, we will first review theories of plural anaphora, which are based mostly on findings from behavioral studies. Then ⇑ Corresponding author. Address: Psychology Department, University of South Carolina, 1512 Pendleton St., Barnwell College, Columbia, SC 29208, USA. E-mail address: [email protected] (T.W. Boiteau). http://dx.doi.org/10.1016/j.bandl.2014.08.003 0093-934X/Ó 2014 Elsevier Inc. All rights reserved. we will review the extant fMRI literature on reference processing in order to clarify our predictions. Plural noun phrases (NPs) can be introduced into text and represented in the discourse model in several ways (Albrecht & Clifton, 1998; Moxey, Sanford, Sturt, & Morrow, 2004; Patson & Ferreira, 2009; Sanford & Lockhart, 1990). They can be introduced as a regular plural NP (e.g., the students), as a quantified NP (e.g., the two students), or as a conjoined NP (e.g., the student and the teacher). Sanford and Lockhart (1990) showed that the form of introduction into the discourse influences the likelihood of subsequent plural reference. For example, conjoined NPs consisting of similar singular forms like the student and the teacher are more likely to lead to plural grouping than conjoined NPs consisting of mismatching singular forms like Jim and the teacher. In addition to the use of a conjoined NP, having discourse entities participate in a joint activity similarly facilitates subsequent plural pronominal reference (Gelormini-Lezama & Almor, 2013; Moxey, Sanford, Wood, & Gintner, 2011). At least three different theories have been proposed to explain how plural NPs are stored in memory and processed during discourse comprehension. According to one theory (Sanford & Lockhart, 1990), plural referents are represented and accessed in memory as an assemblage, or a collective group or entity (i.e., the group is more readily accessible than individual members). Albrecht and Clifton (1998) found that when a conjoined NP is broken apart by a singular reference (e.g., Stan and Pam went to the store. She bought milk), readers take longer to read the second sentence than when the second sentence makes a singular reference T.W. Boiteau et al. / Brain & Language 137 (2014) 130–141 referring to a singular antecedent (e.g., Pam went to the store. She bought milk). Albrecht and Clifton interpreted this delay as reflecting a ‘‘conjunction cost’’ that is incurred when a conjoined NP has to be broken in order to access one of its components. This led Albrecht and Clifton to conclude that conjoined NPs are represented as a single entity or assemblage. According to a second theory (Johnson-Laird, 1983), plural referents are individuated and represented as distinct atomic tokens. Kaup, Kelter, and Habel (2002) provided evidence that under certain conditions plural entities are represented individually. However, these types of representations are elicited mainly by the use of partitioning plural NPs such as most of the orphans or both-phrases. Furthermore, atomic tokens seem to be formed as opposed to an assemblage when features of some of the members of a plural entity are specified (Patson & Warren, 2011). Importantly, the atomic-token view can be used to explain Albrecht and Clifton’s results, where singular reference to a conjoined entity results in longer reading times not due to a conjunction cost, but rather as a result of the increased difficulty of retrieving a referent from memory due to the larger number of potential referents in the discourse model. A third theory, combining the principles of the previous two, suggests that conjoined NPs are under certain conditions represented as a set of individuals or tokens within an assemblage. This type of representation is known as a Complex Reference Object (CRO) and can be understood to consist of a representation of both a single entity as well as of the individuals within the entity (Barker, 1992; Moxey et al., 2004; Moxey et al., 2011). In an eyetracking study, Patson and Ferreira (2009) found that participants’ parsing strategies of sentences with reciprocal verbs (e.g., wrestle) were affected by the type of plural NP in the preceding sentence (i.e., conjoined NPs: the trainer and the vet; or definite plural descriptions: the trainers) used in the first clause of the sentence. For example: The trainer and the vet/The trainers were near the swamp. While they wrestled the alligator watched them closely. The key measure in this study was reading times at the disambiguating region (watched in this case), which indicated whether or not the participants were garden-pathed. They found that reading times on the disambiguating region were shorter when the preceding sentence contained a conjoined NP than when it contained a definite plural description. The authors interpreted this as evidence of CRO formation in which the representations of the individuals made the reciprocal interpretation of the verb more likely than the transitive one. They therefore argued that conjoined NPs favor representation in a CRO. As of yet none of these theories of plural anaphora representation have been tested with the use of neuroimaging techniques, and therefore it is difficult to generate hypotheses about the neural structures involved in such representations. However, there have been a few studies on reference processing that may illuminate this discussion. In addition, plural reference, though a linguistic process, is analogous to other mental operations that are worth considering. Most pertinent to our present study is Almor, Smith, Bonilha, Fridriksson, and Rorden (2007), who used functional magnetic resonance imaging (fMRI) to study reference processing. They measured participants’ hemodynamic responses as they were reading discourses in which repeated reference was made with either a repeated name (e.g., Susan is really into animals. The other day Susan gave Betsy a pet hamster.) or a pronoun (e.g., Susan is really into animals. The other day she gave Betsy a pet hamster.). Participants showed greater activation in temporal and parietal areas (specifically bilateral intraparietal sulcus (IPS), the superior parietal lobule (SPL; BA 7 in particular), and precuneus) in the repeated name condition than in the pronoun condition. Because temporal regions are known to be involved in memory processes 131 (Bookheimer, 2002), Almor et al. attributed the activation in these regions to the activation of multiple memory representations as a result of reading the repeated names. On the other hand, SPL regions have been argued to be involved in the maintenance and integration of multiple representations (Dehaene, Piazza, Pinel, & Cohen, 2003), leading to the conclusion that the parietal activation represents the formation, maintenance and integration of multiple representations resulting from the reading of the repeated names. Inferences regarding these regions will be discussed in more detail below. Nieuwland, Petersson, and Van Berkum (2007) similarly used fMRI to investigate the neural representation of reference processing (in addition to semantic coherence). This study contained three conditions of interest to the current discussion: referential ambiguity (e.g., Ronald told Frank that he . . .), failure (Rose told Emily that he . . .), and coherence (Ronald told Emily that he . . .). Compared to referential coherence, ambiguity led to fronto-parietal activation, perhaps indicating that participants were engaged in a decisionmaking process to establish reference. On the other hand, failure compared to coherence, resulted in bilateral parietal activation (including among other areas BA 7), which may be related to an inference resulting in the formation of a new representation (i.e., assuming the anaphor is referring to an unspecified third party) or morpho-syntactic error. In a post hoc analysis the authors divided the subjects into two groups based on self-reported interpretation of sentences in this condition (either a new referent or referential failure). They found that the group interpreting these sentences as referring to a new referent relative to the referential failure group showed greater activation in left BA8 and right dosolateral BA 9/46. Although this would suggest that introducing a new referent into the discourse evokes frontal activity, an alternative explanation is that this reflects explicit decision making in the new referent group. Indeed, in another fMRI study McMillan, Clark, Gunawardena, Ryant, and Grossman (2012) found widespread frontal activity associated with the greater decision-making demands of ambiguous pronominal reference than non-ambiguous pronominal reference. Additionally, the interpretation that Nieuwland et al.’s (2007) parietal activity findings reflect introducing a new referent into the discourse sits well with Almor et al.’s (2007) findings, where the repeated name (associated with the temporary addition of a new discourse entity before it is resolved as being coreferential; Gordon, Grosz, & Gilliom, 1993) is associated with bilateral parietal activation of a similar nature. The parietal activation in both studies, and especially the SPL, which is not often associated with non-spatial language processing, is of special interest because it suggests that language recruits areas that are specialized for the management of spatial representations for the task of reference tracking in discourse. For example: Beauchamp, Petit, Ellmore, Ingeholm, and Haxby (2001) found that covert shifts of attention activated regions of IPS extending into the SPL; Corbetta, Miezin, Shulman, and Petersen (1993) found activation in BA 7 bilaterally during a spatial attention shifting task; Culham et al. (1998), using a multiple-object tracking task, found activation both in bilateral IPS and BA 7 (see also, Farah, Wong, Monheit, & Morrow, 1989; Shimozaki et al., 2003; Wojuciulik & Kanwisher, 1999). Related to spatial processing and our discussion of plurality, the IPS and SPL have been associated with numerical processing (e.g., Hubbard, Piazza, Pinel, & Dehaene, 2005). Indeed, Dehaene et al. (2003) posited that the function of the SPL (specifically posterior BA 7) is to orient attention along the mental number line (proposed to be within the IPS) and make numerical comparisons (see also, Harvey, Klein, Petridou, & Dumoulin, 2013; Piazza & Izard, 2009). Thus, taking these two related bodies of literature into account, we would expect that if indeed number and space are important to processing plurals, both the IPS and BA 7 would be activated, depending on the nature of the discourse model. First, 132 T.W. Boiteau et al. / Brain & Language 137 (2014) 130–141 consider that plurality is at its core a linguistic numerical device. In the case of plural pronouns, plural markings are used to draw attention to the quantity of entities being referred to (in the case of English, one or more than one). Thus, we might expect that in discourses that involve plural compared to singular entities both the IPS and posterior BA 7 would be active, indicating the need of recruiting these resources to track multiple discourse entities. On the other hand, in discourses with plural entities that are represented in different ways (i.e., different levels of saliency), only posterior BA 7 would be active. The results of Almor et al. (2007) and potentially Nieuwland et al. (2007), while not testing the nature of plural representation, may nevertheless be able to help us make predictions about the neural signature of maintaining multiple discourse entities in the mental model. Both the repeated name condition and the referential failure condition from Almor and Nieuwland’s respective studies, led to bilateral SPL activation. There are at least two potential reasons for this. First, parietal activation in these conditions may be a result of overall lack of discourse coherence. Second, it may be a result of the addition of a new referent into the mental model. In the case of the repeated name condition, the repeated name is very briefly represented as a new discourse entity (Gordon & Hendrick, 1998), and in the referential failure condition, the pronoun is inferred to refer to a third party. Using plural NPs can help us resolve these two possibilities. If we compare a coherent condition with one discourse entity to a coherent condition with two discourse entities and observe a similar parietal signature, then we can rule out the first possibility. If, however, we do not observe such a parietal signature, then we can rule out the second possibility in favor of the first one. In addition, observing patterns of parietal activation could help ascertain the nature of the plural representation in the mental model. For example, if an assemblage without individuated tokens is formed, there should not be any difference between one-entity and two-entity discourses. However, if either a CRO or atomic tokens are created instead of an assemblage, we would indeed see a difference, and such a difference would be apparent in superior parietal regions, if our hypothesis is correct. Experiment 1 aims to test this prediction. In order to differentiate between the atomic token and the CRO theories of plural representation, the number of overall discourse entities would need to be held constant, but the likelihood of being grouped together could be manipulated, for example, by using a conjoined noun phrase vs. using a with adverbial (Moxey et al., 2004). If a with adverbial results in greater parietal activation than a conjoined noun phrase, we could conclude that the conjoined noun phrase is represented more efficiently as a CRO (as opposed to two atomic tokens), whereas the with phrase leads to storing the discourse entities separately. Experiment 2 then aims to clarify the likely representations created by plural NPs by manipulating how multiple entities are combined in the discourse. Crucially, in this experiment we hold the overall number of discourse entities constant, manipulating method of conjunction. Overall, by examining the neural activation associated with plural referents, the present work aims to inform general theories of reference as well as theories of plural reference more specifically. 2. Experiment 1 In this experiment we compare discourses with singular pronominal reference to the subject of a previous sentence (Singular condition) to plural pronominal reference to a conjoined subject NP of the previous sentence (Plural condition). We hypothesize that the Plural condition (in line with the findings from both Almor et al., 2007 and Nieuwland et al., 2007) will lead to greater activation than the Singular condition in the parietal lobe, and more specifically the SPL. This activation will be the result of increased processing load associated with maintaining a more complex representation of the subject. We also hypothesize that the Singular condition will not lead to any more activation than the Plural condition in these parietal regions. 2.1. Methods 2.1.1. Participants Twenty right-handed participants (11 male, 9 female; ages 18– 26) were enrolled in this study. All participants were native speakers of English with normal or corrected-to-normal visual acuity and no history of neurological problems (self-reported). Participants gave their informed consent to participate in this research under the guidelines of the University of South Carolina Institutional Review Board. Data from one participant were excluded from the study on the basis of excessive head motion. 2.1.2. Procedure Participants underwent one 16-min fMRI scanning session after an explanation of the task. Participants were asked to read silently sentences that were presented using a back-projection mirror located at the end of the scanner bore. Each trial consisted of two sentences presented one at a time with each sentence displayed in isolation for 3960 ms with no gap between sentences. The duration between trials varied from 4500 to 7680 ms, reducing the correlation in hemodynamic responses between sentences (i.e., the start of one item was not precisely phase-locked to the start of the previous sentence). Sentence 1 introduced either one or two people who were the grammatical subject in both sentences. In sentence 2, one or both entities were referred to using either a singular or plural pronoun. Because one of the present study’s aims was to assess whether the parietal activation observed by Almor et al. (2007) was related to the representation of multiple referents, we followed the design of that study by introducing referents with proper names. Table 1 shows a sample item in both conditions. Note that number of words was held constant across conditions through the addition of adjectives or adverbials to sentence 1. The session contained 54 experimental trials mixed with 18 filler trials for a total of 72 trials. The filler trials were also composed of two sentences, but these trials used varied syntactic and discourse structures. The role of the filler trials was to reduce participants’ expectancy for specific sentence structures, especially pronouns referring to one or two people. The order of the experimental and filler trials was randomized for each participant. The condition in which each item was shown to participants was randomly selected with the constraints that each participant saw the same number of items in each condition and that across all participants each item appeared the same number of times in each condition. Following about one third of the trials, participants were given a simple ‘yes’ or ‘no’ question regarding the trial they had just read. Thus each participant responded to 24 comprehension questions. Question duration was 4500 ms, and participants indicated their response by pressing a button on a Psychology Software Tools, Inc. BrainLogics Fiber Optic Button Response System (Pittsburgh, Pennsylvania, USA) glove placed on their left hand. Participants used the thumb button to indicate a ‘yes’ response and the index finger button to indicate a ‘no’ response. The comprehension questions probed information from the trial’s text and allowed us to assess whether or not the participant was engaged in the task. All participants responded accurately to at least 70% of the questions, and therefore the data from all the participants were analyzed. Scanning was performed on a 3T Siemens Trio scanner (Munich, Germany) equipped with a 12-element head coil. A total of 322 echo-planar imaging (EPI) volumes (36 axial slices; 3.2 mm thick, T.W. Boiteau et al. / Brain & Language 137 (2014) 130–141 Harvard-Oxford Cortical Structural Atlas, Juelich Histological Atlas, and MNI Structural Atlas. Table 1 A sample item in both conditions from Experiment 1. Condition Sentence 1 Sentence 2 Singular Jeremy did some work on the house next door. Jeremy and Roger did some work on the house. He fixed the window and the door. They fixed the window and the door. Plural Question 133 Were the stairs fixed? with no gap between slices) were acquired during the fMRI session. These scans used TR = 2.5 s; TE = 30 ms; matrix = 64 64 voxels; flip angle = 90°, 208 208 mm FOV. 2.1.3. fMRI analyses Preprocessing and statistical analyses of EPI data were carried out using FEAT (fMRI Expert Analysis Tool) Version 5.91, part of FSL 4.0 (fMRI Software Library, www.fmrib.ox.ac.uk/fsl). Lowerlevel analyses implemented motion correction, preserving the skull, spatial smoothing using a Gaussian kernel of FWHM 8 mm, mean-based intensity normalization of all volumes by the same factor and high-pass temporal filtering (Gaussian-weighted LSF straight line fitting, with r = 50.0 s). Time-series statistical analysis was carried out with local autocorrelation correction. All EPI images were normalized in Montreal Neurological Institute (MNI) space. Each participant’s cortical activity was measured for the duration of both sentences for the two different experimental conditions: Singular or Plural. Because we were interested in differences between discourses that had either one or two referents, and because both sentence 1 and sentence 2 differed across conditions in number of referents, we contrasted in our model the activation associated with reading both sentences together in each condition. Note that while it would have been interesting to separate the activation of each sentences in the model by including a separate regressor for each sentence in each condition, this would have resulted in a rank deficient design matrix due to the fact that each version of sentence 1 was always followed by the same version of sentence 2 making a design matrix based on such regressors highly inefficient. The results of this experiment should therefore be interpreted as informative about the combined activation associated with both the generation of the original representation of the referents and the subsequent anaphoric reference to these referents. While still informative about the discourse activations associated with plural references, the results will not distinguish between the activation associated with the generation of plural representations and the activation associated with the repeated references to these representations. We address this issue directly in Experiment 2. In addition to the regressors for the two experimental conditions, the model included separate regressors for filler items, questions, and the fixation immediately following questions when participants responded to the questions. The fixations between items were used as the implicit baseline and were therefore not specified explicitly in the model. For each participant, contrasts were set to evaluate the mean activation related to each condition (Singular or Plural) and to compare the activation between the two conditions. The group mean cortical activation (higher-level analysis) was then calculated by investigating the mean activation for each participant across different trials, and the mean activation between different participants. Z-statistic images were generated using a cluster threshold of Z > 2.3 and a (corrected using Gaussian random field theory) cluster significance threshold of p < 0.05. FSLview was utilized to display mean statistical maps on a standard brain template, and the anatomical location of significant activation was confirmed with the 2.2. Results We identified brain areas that showed significant increase in signal during one condition compared to the other condition. The Plural condition led to more activation than the Singular condition in the parietal lobe bilaterally, specifically the precuneus, SPL, anterior IPS, left angular and supramarginal gyri, and occipital regions such as the right fusiform gyrus, right lingual gyrus, and left lateral occipital cortex. In addition, there was bilateral activation in the cerebellum (Fig. 1; see Table 2 for a list of local cluster activation peaks). No areas were significantly more active in the Singular condition than the Plural condition. 2.3. Discussion The observed parietal activation in the present study is informative about the representation and processing of conjoined NPs during language comprehension. In particular, this activation can help distinguish between claims that conjoined NPs are represented as an assemblage, a set of individual entities, or a CRO in which both the conjoined entity and the individual component entities can be easily accessed. Albrecht and Clifton (1998) argued that conjoined NPs are represented as a single entity or assemblage. In contrast, both Moxey et al. (2004) and Patson and Ferreira (2009) argued that conjoined NPs are represented as CROs in which both the conjoined entity and each of the conjoined entities are accessible. The observed parietal activation in the current study suggests that conjoined NPs are not represented as an assemblage but perhaps as a CRO or atomic tokens (Johnson-Laird, 1983). Experiment 2 will help differentiate between the CRO and atomic-token views. Specifically, the increased activation observed in the precuneus and SPL for the Plural condition over the Singular condition suggests that when multiple referents are introduced and referred to in coherent discourse, multiple representations are formed and maintained in the mental model. The similarity in parietal activation in this study and Almor et al.’s (2007) implies that these patterns of activation reflect the formation, manipulation, and integration of multiple representations in the discourse rather than differences in coherence between conditions. Within the parietal lobes, subregions may be involved in different aspects of the maintenance and manipulation of these representations. We will discuss this possibility after Experiment 2. Fig. 1. Brain areas that were more activated in the Plural compared to the Singular condition in Experiment 1 (threshold: Z > 2.3, corrected cluster threshold p < 0.05). Images were created using MRIcroGL (http://www.mccauslandcenter.sc.edu/ mricrogl/). 134 T.W. Boiteau et al. / Brain & Language 137 (2014) 130–141 Table 2 Brain areas that were more activated in the Plural condition than in the Singular condition (threshold: Z > 2.3, cluster threshold p (corrected for multiple comparisons) = 0.05). Location is based on Harvard-Oxford Cortical Structural Atlas, Juelich Histological Atlas, and MNI Structural Atlas. Size Z MNI spatial coordinates x y Side Location z 26,864 3.86 3.71 3.71 3.71 3.67 3.65 46 14 14 12 40 41 70 86 86 88 81 79 40 16 18 21 41 41 Left Right Right Right Left Right Cerebellum Occipital fusiform gyrus, V3v Occipital fusiform gyrus Occipital fusiform gyrus Cerebellum Cerebellum 26,271 3.51 3.44 3.44 3.35 3.17 3.17 24 17 27 32 25 35 64 61 74 72 64 54 30 30 42 42 28 24 Right Left Right Right Left Left Precuneus, anterior intraparietal sulcus Precuneus Lateral superior occipital cortex, superior parietal lobule 7P Lateral superior occipital cortex, inferior parietal PGp Lateral superior occipital cortex, optic radiation, anterior intraparietal sulcus Angular gyrus, optic radiation, anterior intraparietal sulcus Activation in the supramarginal gyrus likely reflects the cost of accessing multiple lexical representations from memory (Gow, 2012). In the Plural condition, upon encountering they, participants were accessing the stored lexical tokens from sentence 1. In the Singular condition, participants only had to retrieve one token from memory. Thus, the increased activation in the Plural condition may reflect accessing a greater number of lexical tokens from memory. Occipital activation is potentially a result of the general increase in orthographic processing in the Plural condition, which included an additional referent than the Singular condition (e.g., Dehaene, Le Clec, Poline, Le Bihan, & Cohen, 2002;Yeatman, Rauschecker, & Wandell, 2013). Although the Plural condition did not contain more words than the Singular condition, it did have two capitalized proper names compared to the one in the Singular condition, which might have resulted in increased focus on orthographic aspects of the words. The adverbials or adjectives at the end of the first sentence in the Singular condition might have not drawn as much attention, and thus might have been processed only cursorily. Experiment 2 addresses this potential concern by matching the number of discourse entities in the different conditions, while still keeping sentences between conditions identical in length. 3. Experiment 2 There are two other possible explanations of the Experiment 1 results which do not involve the construction and maintenance of a CRO: 1. more discourse entities results in increased parietal activation; 2. the conjunction of discourse entities in sentence 1 results in said activation. Experiment 1 did not allow us to tease apart these two explanations as the number of referents and conjunction were confounded in this experiment. Thus, in Experiment 2 we aim to resolve this issue by manipulating the complexity of the discourse model representation while holding the number of mentioned entities constant. Following Sanford and Lockhart (1990) and Moxey et al. (2004), items in this experiment introduced two referents in the first sentence that were either conjoined and appeared as a complex NP subject (e.g., John and Mary . . .; Conjoined condition) or as collaborating in the same activity via a with-phrase adjunct (e.g., John . . . with Mary; Unconjoined condition). We expect that and-phrase antecedents will lead to the formation of a CRO with the assemblage in focus while withphrase antecedents will lead to a CRO with the individuated entities more in focus. In order to ascertain whether both a conjoined entity and individual referent representations are maintained in either the Conjoined or Unconjoined condition, we included the pronoun manipulation of sentence 2, as in Experiment 1 except that here this resulted in a 2 2 design. If only an assemblage is maintained in the discourse model, or if the assemblage is more salient than the representations of the individual entities in the CRO, then subsequent singular pronominal reference will result in a processing cost. If both a conjoined entity and individual reference objects are maintained and have comparable discourse salience, singular pronominal reference will not lead to such a processing cost. In this experiment, we expect that in the Unconjoined condition, atomic tokens will be generated, and in the Conjoined condition, a CRO will be generated, resulting in a more efficient representation of the discourse entities. This hypothesis thus leads to the following specific predictions: (1) the Unconjoined condition will lead to greater activation in the SPL than the Conjoined condition, as in this condition the discourse model is less efficient: (2) the Conjoined condition will not lead to any greater parietal activation than the Unconjoined condition because in the Conjoined condition the assemblage is in focus at the expense of the individual representations, while in the Unconjoined condition, the individuals are in focus; (3) the left inferior parietal lobule (IPL) will be more activated in the Plural condition than the Singular condition, as the Plural Condition will result in the activation of a richer lexical representation (just as it was in E1). Finally, although our hypothesis does not make any a priori predictions for an interaction effect, the presence or absence an interaction would be informative about the relative salience of the conjoined entity and the representation of the first-mentioned discourse referent. Unlike in Experiment 1, in this experiment the two conditions of sentence 1 were crossed with the two conditions of sentence 2, which allows us to examine the activation associated with each sentence separately, thus differentiating between activation associated with the generation of the discourse representations and activation associated with the further processing of these representations. 3.1. Methods 3.1.1. Participants Twenty-eight right-handed participants (9 male, 19 female; ages 18–27) were enrolled in this study. All participants were native speakers of English with normal or corrected-to-normal visual acuity and no history of neurological problems (selfreported). Participants gave their informed consent to participate in this research under the guidelines of the University of South Carolina Institutional Review Board. Data from two participants were excluded from the study on the basis of excessive head motion (>3 mm). T.W. Boiteau et al. / Brain & Language 137 (2014) 130–141 Table 3 A sample item from Experiment 2 in each condition. Condition Sentence 1 Sentence 2 Conjoined Singular Conjoined Plural Unconjoined Singular Unconjoined Plural Jeremy and Lucy did some work on the house. Jeremy and Lucy did some work on the house. Jeremy did some work with Lucy on the house. Jeremy did some work with Lucy on the house. He fixed the window and the door. They fixed the window and the door. He fixed the window and the door. They fixed the window and the door. Question Were the stairs fixed? 3.1.2. Procedure Each participant underwent one 18-min session of fMRI scanning after an explanation of the task. Participants were asked to read silently sentences that were presented using a back-projection mirror located at the end of the scanner bore. Each trial consisted of two sentences presented one at a time, with each sentence displayed in isolation for 3960 ms, and no gap between sentences. The duration between trials varied from 4500 to 7660 ms, reducing the correlation in hemodynamic responses between sentences (i.e., the start of one item was not precisely phase-locked to the start of the previous sentence). Sentence 1 introduced two characters. In the Conjoined condition both characters were the subject of a conjoined and-phrase, and in the Unconjoined condition the subject appeared alone while the partner appeared later in the sentence as the head of a with-phrase adjunct. In sentence 2, the first-mentioned or both entities together were referred to using either a singular or plural pronoun respectively. This resulted in four possible unique combinations for each item. Table 3 shows a sample item in each condition. The session contained 80 experimental trials mixed with 20 filler trials for a total of 100 trials. The filler trials were also composed of two sentences, but these trials used varied structures. The role of the filler trials was to reduce participants’ expectancy for specific sentence structures, especially pronouns referring to one or two people. The order of the experimental and filler trials was randomized for each participant. The condition in which each item was shown to participants was randomly selected with the constraints that each participant saw the same number of items in each condition, that across all participants each item appeared the same number of times in each condition, and that each participant saw each item in only one condition. Following about one third of the trials, participants were given a simple ‘yes’ or ‘no’ question regarding the trial they had just read. Question duration and response method was identical to Experiment 1. The comprehension questions probed information from the trial’s text and allowed us to assess whether or not the participant was engaged in the task. All participants responded accurately to at least 70% of the questions and therefore the data from all the participants were analyzed. The scanner and scanning details were identical to Experiment 1. 3.1.3. fMRI analyses Preprocessing and statistical analyses of EPI data were carried out using FEAT (FMRI Expert Analysis Tool) Version 5.91, part of FSL 4.0 (FMRI Software Library, www.fmrib.ox.ac.uk/fsl). Lowerlevel analyses implemented motion correction, preserving the skull, spatial smoothing using a Gaussian kernel of FWHM 8 mm, mean-based intensity normalization of all volumes by the same factor and high-pass temporal filtering (Gaussian-weighted LSF straight line fitting, with r = 50.0 s). Time-series statistical analysis was carried out with local autocorrelation correction. All EPI images were normalized in MNI space. 135 Unlike in Experiment 1, here the two versions of sentence 1 and the two versions of sentence 2 were not confounded, thus allowing us to model each participant’s cortical activity separately for the two sentences. Thus, the model included two regressors for the first 3960 ms of the trial corresponding to the two versions of sentence 1 (conjoined and unconjoined) and four regressors for each of the 3960–7920 ms in the trial corresponding to the four conditions of sentence 2 (single pronoun following a conjoined reference, single pronoun following unconjoined reference, plural pronoun following conjoined reference, and plural pronoun following unconjoined reference). Although these four regressors are not fully independent of the two sentence 1 regressors, they nevertheless allow us to analyze the interaction of the factors in the activation associated with reading sentence 2, bearing in mind that any effects involving conjunction type may have originated in the reading of either sentence 1 or sentence 2.1 There were also three additional regressors for filler items, questions, and fixations immediately following questions when participants responded to the questions. The fixations between items were used as the implicit baseline and were therefore not specified explicitly in the model. For each participant, two contrasts were set to compare the activation of the two sentence 1 conditions (Conjoined > Unconjoined and Unconjoined > Conjoined) and two contrasts were set to compare the activation of the singular and plural sentence 2 conditions (Singular > Plural and Plural > Singular). In addition, two interaction contrasts were set in order to test the interaction of the two factors (UnconjoinedSingular + ConjoinedPlural > UnconjoinedPlural + ConjoinedSingular and UnconjoinedPlural + ConjoinedSingular > UnconjoinedSingular + ConjoinedPlural). We also tested for the main effect of conjoined vs. unconjoined for sentence 2 (Conjoined > Unconjoined and Unconjoined > Conjoined). Note that due to the correlation between this contrast and the sentence 1 contrasts, care must be taken in interpreting the results of these contrasts. The group mean cortical activation (higher-level analysis) was then calculated by investigating the mean activation for each participant across different trials, and the mean activation between different participants. Z-statistic images were generated using a cluster threshold of Z > 2.3 and a cluster significance threshold of p < 0.05 (corrected using Gaussian random field theory). FSLview was utilized to display mean statistical maps on a standard brain template, and the anatomical location of significant activation was confirmed with the Harvard-Oxford Cortical Structural Atlas, Juelich Histological Atlas, and MNI Structural Atlas. In addition to the high level contrasts, we also performed region of interest analyses (ROI) using the Juelich Histological Atlas in FSL’s Featquery tool on specific areas we found active in Experiment 1. These regions were in the left and right SPL and the left and right IPL. We also examined classical language areas in the left inferior frontal gyrus (IFG). Our reason for examining the latter area is to better distinguish differences that are likely due to syntactic factors from those likely reflecting discourse processes (Table 4 provides a list of all ROIs examined). 3.2. Results For sentence 1, we contrasted the two conditions (Conjoined and Unconjoined). In the whole-brain analyses, we found no activation associated with Conjoined greater than Unconjoined or Unconjoined greater than Conjoined. For the sentence 1 ROIs, we only report significant effects. Consult Fig. 2 for mask locations and Fig. 3 for a graphical summary of 1 Note that this is necessarily a problem with discourse processing designs, in which context impacts later processing of linguistic stimuli. In addition to the abovereported analyses, we also modeled the data using two regressors for sentence 1 and two regressors for sentence 2. This did not lead to any additional findings than the ones reported in this paper. 136 T.W. Boiteau et al. / Brain & Language 137 (2014) 130–141 Table 4 List of all ROIs by hemisphere examined in E2. Left Right Hemisphere Anterior intra-parietal sulcus hIP1 Anterior intra-parietal sulcus hIP2 Anterior intra-parietal sulcus hIP3 BA 44 BA 45 Inferior parietal lobule PF Inferior parietal lobule PFcm Inferior parietal lobule PFm Inferior parietal lobule PFop Inferior parietal lobule PFt Inferior parietal lobule Pga Inferior parietal lobule Pgp Superior parietal lobule 5Ci Superior parietal lobule 5L Superior parietal lobule 5M Superior parietal lobule 7A Superior parietal lobule 7M Superior parietal lobule 7P Superior parietal lobule 7PC Anterior intra-parietal sulcus hIP1 Anterior intra-parietal sulcus hIP2 Anterior intra-parietal sulcus hIP3 – – Inferior parietal lobule PF Inferior parietal lobule PFcm Inferior parietal lobule PFm Inferior parietal lobule PFop Inferior parietal lobule PFt Inferior parietal lobule Pga Inferior parietal lobule Pgp Superior parietal lobule 5Ci Superior parietal lobule 5L Superior parietal lobule 5M Superior parietal lobule 7A Superior parietal lobule 7M Superior parietal lobule 7P Superior parietal lobule 7PC Unconjoined Singular greater than Conjoined Plural, and Conjoined Singular plus Unconjoined Plural greater than Conjoined Plural plus Unconjoined Singular (Fig. 4 and Table 5). The Conjoined Singular greater than Conjoined Plural contrast found activation in the right occipital pole, right lateral occipital cortex, and right fusiform gyrus. The Unconjoined Plural greater than Conjoined Plural contrast found activation in the left occipital pole, left lateral superior occipital cortex, and left fusiform gyrus. The Unconjoined Singular greater than Conjoined Plural contrast found activation in the right lateral inferior occipital cortex and right fusiform gyrus. Finally, the interaction effect, Conjoined Singular plus Unconjoined Plural greater than Conjoined Plural plus Unconjoined Singular, found activation in the left occipital pole, left lateral occipital cortex, and left lateral inferior occipital cortex. Next, we report the results of the ROIs. For the sake of space, we only report those effects that were significant. Consult Fig. 5 for mask locations and Fig. 6 for a graphical summary of all analyses. We found a main effect of Conjunction in left 7P (posterior SPL), F(1, 25) = 12.63, p = .002, but no effect of Number and no interaction. There was also a main effect of Conjunction in right 7P, F(1, 25) = 8.24, p = .01, again with no effect of Number nor an interaction between the two variables. In left 7M (medial SPL) there was again a main effect of Conjunction, F(1, 25) = 13.02, p = .001, no effect of Number and no interaction. In right 7 M there was a main effect of Conjunction, F(1, 25) = 7.09, p = .01, no effect of Number and no interaction. In left 7A (anterior SPL) there was again a main effect of Conjunction, F(1, 25) = 5.47, p = .03, no main effect of Number and no interaction. Finally, there was a main effect of Number in left PFcm (anterior SMG) with greater activation in the Plural than the Singular condition, F(1, 25) = 5.05, p = .03, but no effect of Conjunction and no interaction. All the Conjunction effects were such that there was greater activation in the Unconjoined than in the Conjoined condition. Analyses of left IFG regions did not reveal any significant differences in activation.3 3.3. Discussion Fig. 2. Location of masks for sentence 1 ROIs from Experiment 2 that yielded significant activation (and their contralateral homologues). Regions in cyan are left and right Pga. Regions in yellow are left and right 7M. the analyses. We found a main effect of Conjunction in right 7P (posterior SPL), F(1, 25) = 4.81, p = .04, with Conjoined leading to more activation than Unconjoined. There was also a marginal effect in the left homologue of this region, F(1, 25) = 3.69, p = .07, again with Conjoined leading to more activation than Unconjoined. We also found a main effect in the right Pga (angular gyrus), F(1, 25) = 4.39, p = .05, with Conjoined leading to more activation than Unconjoined, but there was no effect in the left homologue, F < 1. There were no effects observed in the left IFG.2 For sentence 2, 4 of the contrasts turned up significant patterns of activation. These contrasts were Conjoined Singular greater than Conjoined Plural, Unconjoined Plural greater than Conjoined Plural, 2 Although these contrasts were preplanned based on the findings from Experiment 1 and similar patterns of activity in other neuroimaging reference studies (e.g., Almor et al., 2007), if we were to apply a correction for multiple comparisons, none of the observed effects would be statistically significant using the Bonferroni method (a = .001515). While the whole-brain analyses of sentence 1 found no effects, the ROIs showed that in fact the Conjoined condition did result in greater activation than the Unconjoined condition. We interpret this as an effect of assemblage or CRO construction, which occurs mainly in the case of conjoined structures. This shows that to some extent the findings from Experiment 1 are due to the initial construction of these complex plural representations. It is interesting to note that left hemisphere activation was minimal, with more right parietal structures being involved in this process of plural entity construction. The high-level contrasts of sentence 2 only revealed activation in early occipital regions, replicating Experiment 1 and likely reflecting orthographic differences between conditions rather than discourse level processing (Cohen et al., 2000). These occipital regions have also been implicated in color, face, and object recognition (Kanwisher, McDermott, & Chun, 1997; Lueck et al., 1989; Malach et al., 1995), so in addition to the possibility that activity here is a result of differences in lexical processing, it also might be the case that the activity reflects different patterns of imagery evoked in reading the discourses. An interesting pattern in three of these contrasts is that everything compared to Conjoined Plural turned up these early occipital activations. It is possible that in general, Conjoined Plural is the easiest or most natural of these discourse structures to process, and that the other versions involve 3 Again, these comparisons were planned based on previous results, but if we were to apply a Bonferroni correction (a = .001515), only the right 7M in the Conjunction analysis would be considered significant and the left 7P in the Conjunction analysis would be borderline. 137 T.W. Boiteau et al. / Brain & Language 137 (2014) 130–141 7M 0.15 Le Pga Le Right * 0.12 0.09 Right 0.03 0 Con Uncon -0.03 0.06 -0.06 0.03 -0.09 0 -0.12 -0.03 -0.15 Con Uncon * Fig. 3. Results of the sentence 1 ROI analyses from Experiment 2, showing percent BOLD signal change on the y-axis by hemisphere and condition on the x-axis, dividing each ROI into a separate panel. Bars indicate standard error of the mean. Significant contrasts are marked with brackets and asterisks. Fig. 4. Brain areas more activated in (a) Conjoined Singular than Conjoined Plural, (b) Unconjoined Plural than Conjoined Plural, (c) Unconjoined Singular than Conjoined Plural, and (d) Conjoined Singular + Unconjoined Plural than Conjoined Plural + Unconjoined Singular in Experiment 2 (threshold: Z > 2.3, corrected cluster threshold p < 0.05). Images were created using MRIcroGL (http://www.mccauslandcenter.sc.edu/mricrogl/). slightly more complex situations to visualize. Alternatively, it might be due to encountering the unexpected pronoun. Indeed, using magnetoencephalography (MEG), Dikker, Rabagliati, Farmer, and Pylkkänen (2010) found early occipital regions activated in violations of expected word forms. While that study was only measuring activation associated with a single word using MEG and ours looks at discourse-level processes using fMRI, the interaction effect we observed can be understood along similar lines. Both Conjoined Singular and Unconjoined Plural are the less felicitous combinations, as a conjoined subject typically will be referred to with a plural pronoun and a subject participating with a with-phrase partner will normally be referred to subsequently with a singular pronoun (Moxey et al., 2004). Therefore, the fact that we see occipital activation when these two cases are compared with the more felicitous versions (i.e., Conjoined Plural plus Unconjoined Singular) would seem to suggest the occipital region’s sensitivity to these violations. In other words, the different sentence 1 conjunctions led participants to anticipate different types of pronominal reference in the subsequent sentence, and when these expectations were violated, the result was increased activation in these occipital regions. Whether these findings reflect the orthographic differences between the conditions or violations of expectancies, our main focus here is on the parietal activations, which we discuss next. Notably, the high-level contrasts of sentence 2 did not reveal any activation in the SPL. The ROIs of sentence 2, however, revealed that indeed conjunction type impacts the degree of accessibility of referents in the discourse. While previous literature has argued that and-phrase conjunctions bind entities together such that accessing those bound individuated representations results in an unbinding cost (Albrecht & Clifton, 1998), we show here that this description is somewhat oversimplified. The Unconjoined condition consistently led to greater SPL activation than the Conjoined condition, which we attribute to the greater effort resulting from maintaining a more complex discourse representation. Again, these findings are in line with Almor et al. (2007), showing that the brain recruits parietal regions in order to maintain discourselevel representations. The lack of any interaction in these regions 138 T.W. Boiteau et al. / Brain & Language 137 (2014) 130–141 Table 5 Brain areas that were more activated in each pairwise comparison (threshold: Z > 2.3, cluster threshold p (corrected for multiple comparisons) = 0.05). Location is based on Harvard-Oxford Cortical Structural Atlas, Juelich Histological Atlas, and MNI Structural Atlas. Size Z MNI spatial coordinates x y Side Location z Conjoined Singular > Conjoined Plural 1386 4.16 3.1 3.04 2.9 2.9 2.84 26 48 38 2 2 58 100 86 66 106 104 72 4 8 12 12 12 8 Right Right Right Right Right Right Occipital pole, V1 Lateral occipital cortex, V4 Occipital fusiform gyrus, V4 Occipital pole Occipital pole Lateral occipital cortex Unconjoined Plural > Conjoined Plural 2393 3.68 3.62 3.34 3.33 3.31 3.26 36 18 34 36 30 4 80 76 88 70 86 98 20 14 20 18 10 4 Left Left Left Left Left Left Occipital fusiform gyrus, V4 Occipital fusiform gyrus, V4 Lateral occipital cortex Occipital fusiform gyrus, V4 Lateral superior occipital cortex Occipital pole, V1 Unconjoined Singular > Conjoined Plural 1340 3.75 3.35 3.35 3.35 3.18 2.92 58 42 44 26 48 28 70 66 74 84 70 70 4 16 14 16 14 14 Right Right Right Right Right Right Lateral inferior occipital cortex Occipital fusiform gyrus, V4 Lateral inferior occipital cortex, V4 Occipital fusiform gyrus, V3v Lateral inferior occipital cortex, V4 Occipital fusiform gyrus, V4 10 8 8 10 10 20 Left Left Left Left Left Left occipital pole Occipital pole, V2 Occipital pole, V3v Occipital pole, V2 Lateral occipital cortex, V4 Lateral inferior occipital cortex Conjoined Singular + Unconjoined Plural > Conjoined Plural + Unconjoined Singular 1151 3.48 26 106 3.47 22 104 3.38 32 96 3.37 30 100 3.09 44 88 3.04 36 88 Fig. 5. Location of masks for sentence 2 ROIs from Experiment 2 that yielded significant activation (and their contralateral homologues). Regions in green are left and right 7A. Regions in yellow are left and right 7M. Regions in blue are left and right 7P. Regions in red are left and right PFcm. suggests that both the conjoined entity and the individuated representation of the first-mentioned referent were equally accessible from memory in Conjoined and Unconjoined conditions, which is not surprising given that only two entities existed in the discourse and they were sharing agency. The activation in the left IPL for Plural greater than Singular is in line with the results from Experiment 1 and suggests a more complex lexical activation induced by accessing both referents from memory (Gow, 2012). It is interesting to note that no differences in IPS activation were observed in this experiment, whereas we did find such activation in Experiment 1. This may indicate this region’s involvement in tracking multiple discourse entities, much as it has been found to be involved in multiple object tracking (Culham et al., 1998). Importantly, there were no differences (in this or the previous experiment) in the classic language areas (i.e., left IFG), reducing the likelihood of a syntactic confound. The difference in findings from the sentence 1 and sentence 2 ROIs is highly informative. The greater superior parietal activation in sentence 1 in the Conjoined condition indicates the effort involved in constructing a discourse model that includes an assemblage or a CRO. With the reservation that the activation differences due to conjunction type were not fully independent in sentences 1 and 2, the greater superior parietal activation in sentence 2 in the Unconjoined condition may show the greater complexity of the discourse model in that condition while processing this sentence. In other words, the initial increased effort in constructing the discourse model (i.e., constructing an assemblage or CRO) results in a more efficient mental representation later on in the discourse. This highlights the power simple differences in syntactic structure can have on the construction of the mental model. 4. General discussion We have reported two experiments that tested the effect of number and conjunction type on the formation and accessibility of discourse representations. The findings of these two studies suggest that when multiple referents are introduced and referred to in coherent discourse, multiple representations are formed. These representations are overlapping, such that collaborative actions performed by distinctly identified individuals (as opposed to plural definite entities; Patson & Ferreira, 2009) will produce a CRO under certain conditions. This finding makes an important contribution 139 T.W. Boiteau et al. / Brain & Language 137 (2014) 130–141 7A 7M Le 0.05 Con Right Uncon Con Le Uncon 0.06 0.02 Con Right Uncon Singular -0.04 Singular -0.06 -0.1 -0.1 -0.14 -0.13 Plural -0.02 -0.07 * -0.18 * * 7P PFcm Le Con Right Uncon Con Le Uncon 0.02 Plural Right Singular Plural Singular 0 0.05 Plural 0 Singular -0.05 Con -0.02 Uncon -0.04 -0.06 -0.1 -0.15 -0.2 Uncon 0.02 Plural -0.01 0.1 Con * * * -0.08 -0.1 Fig. 6. Results of the sentence 2 ROI analyses from Experiment 2, showing percent BOLD signal change on the y-axis by hemisphere and condition on the x-axis, dividing each ROI into a separate panel. Bars indicate standard error of the mean. Significant contrasts are marked with brackets and asterisks. to the literature on plural processing, which so far has largely ignored the possibility that the relative saliency of the CRO and the entities that comprise it may vary under different conditions (in our case, manner of introduction into the discourse). This conclusion also lends support to the finding that parietal activation observed in both the Almor et al. (2007) and Nieuwland et al. (2007) studies was due to the integration and manipulation of multiple representations rather than differences in coherence between conditions. Previous research has shown that when multiple referents are encountered in discourse under conditions of shared agency, an assemblage token is generated and stored in the discourse model along with the representations of the individual referents (Barker, 1992; Moxey et al., 2004). Patson and Warren (2011) showed that the degree of differentiation (i.e., semantic properties) between discourse entities impacts the structure of a CRO, creating pointers to specific referents. Here we show that the accessibility of discourse referents in memory and the salience of the plural entity are impacted by how referents are introduced into the discourse (i.e., syntactic factors). In the case of a single-referent discourse, one entity is maintained in the mental model. In plural-referent discourse, two entities are maintained, but the manner in which they are represented is impacted by conjunction type (among other factors). In the case of a conjoined noun phrase, a CRO is generated, with pointers to the individual referents. In the case of an unconjoined noun phrase, two individual representations are introduced and maintained, and the result of our experiments show that maintaining these separate representations is more taxing on attentional resources (see Table 6). The result as we move from singular to plural-conjoined and plural-unconjoined is an increasingly complex (and less efficiently maintained) discourse model. This is apparent by the increased involvement of parietal regions as the discourse model grows in complexity. Our results are informative about other theories of plural processing. According to the assemblage account of plural reference (Albrecht & Clifton, 1998), plural conjoined references result in a representation of the conjoined NP such that any subsequent reference to one of its components incurs a ‘‘conjunction cost.’’ The results of our first experiment are incompatible with this theory because if the conjoined NPs were represented as an assemblage, as Albrecht and Clifton argue, there should not have been differences in parietal activation between the conditions. Our findings are also incompatible with the atomic-token view of plural references (Johnson-Laird, 1983). This view can account for the results of Experiment 1, in that the higher activation in the Plural than in the Singular conditions may reflect either the cost associated with creating a conjoined referent as an antecedent for the plural pronoun, or simply maintaining a greater number of entities in the discourse model. Nevertheless, this view cannot account for the results of Experiment 2, in which we found greater activation in the SPL for the Unconjoined than for the Conjoined conditions, regardless of the reference form. Table 6 Schematic depiction of discourse representations from sample items in Experiment 1 and 2. Condition Singular Plural Conjoined Plural Unconjoined Example Mental representation John went to the store. John and Mary went to the store. John went to the store with Mary. John John John Mary Mary 140 T.W. Boiteau et al. / Brain & Language 137 (2014) 130–141 Our results are largely compatible with the CRO account of Moxey et al. (2004). Here, we focused on the difference in the form of referent introduction as a factor influencing how these representations are stored and maintained. However, there is substantial research showing the role of other factors as well. For example, collaborative verbs such as the ones used in Moxey et al. (2011) are known to increase the likelihood of future plural reference. We therefore predict that these verbs are more likely to produce an effect similar to the one we observed in our with-phrase conditions (i.e., competition between conjoined and individual referents) than other verbs. Similarly, Moxey et al. found that when referents are performing the same roles in different locations they are less likely to be referred to via the use of a plural reference. More generally, we predict that any information in the discourse that would lead to the separation between characters in the discourse model would reduce the salience of the conjoined representation. Moreover, we predict that the more entities in the discourse model, the larger the processing load. Thus, a discourse consisting of two entities will lead to more activation than one, three will lead to greater activation than two, and so on until reaching the point of memory saturation. Following Almor et al. (2007), we predict that this difficulty will be greater when differences in saliency between the representations are small than when they are large, and that this difficulty will be reflected in activation of the SPL (for an alternative view on the neural correlates of introducing new referents into the discourse model, see Nieuwland, 2014 and Nieuwland et al., 2007). One possible explanation for the results observed in these experiments is that the observed activation in IPS and BA 7 reflect generally greater attentional demands by plurals compared to singular, or unconjoined vs. conjoined plural discourse (e.g., Behrmann, Geng, & Shomstein, 2004). In the absence of any theoretical grounds to expect such differences in attentional demands, it is unclear whether this explanation can be supported, and if it can, whether it would provide any insight into the underlying processes and representations. An alternative explanation, and one that is line with other theoretical accounts both from behavioral and neuroimaging studies (e.g., Almor & Nair, 2007; Almor et al., 2007; Dehaene et al., 2003; Gordon & Hendrick, 1998; Hubbard et al., 2005) is that these regions, which have been implicated separately in spatial attention, numerical processing and the interaction between the two processes, and more specifically with tracking multiple objects in space (Culham & Kanwisher, 2001; Culham et al., 1998; Harvey et al., 2013; Piazza & Izard, 2009), were recruited by language to help support the tracking and representation of referents in discourse. This mechanism is utilized for tracking all kinds of referents, singular and plural, and is taxed by referential load induced by both the number of tracked referents and their differentiability in levels of saliency. In summary, we have provided evidence for the CRO, an account of plural reference processing. We have shown that the representation of multiple referents in the discourse model recruits parietal regions associated with spatial and numerical processing, increasing the number of entities in the mental model leads to an increase in processing load in these regions, and this in turn is modulated by the saliency of these entities, which is impacted by syntactic and pragmatic factors. Acknowledgments This research was partially funded by NIH R21AG030445 and NSF BCS0822617 grants. The authors wish to thank Chris Rorden for help with ROI analyses and Kat Wilson and Geetanjali Pathak for help with data collection. References Albrecht, J., & Clifton, C. (1998). Accessing singular antecedents in conjoined phrases. Memory and Cognition, 26(3), 599–610. Almor, A. (1999). Noun-phrase anaphora and focus: The informational load hypothesis. Psychology Review, 106, 748–765. Almor, A., & Nair, V. A. (2007). The form of referential expressions in discourse. Language and Linguistics Compass, 1(1–2), 84–99. Almor, A., Smith, D., Bonilha, L., Fridriksson, J., & Rorden, C. (2007). What is in a name? Spatial brain circuits are used to track discourse references. NeuroReport, 18(12), 1215–1219. Ariel, M. (1990). Accessing noun-phrase antecedents. London; New York: Routledge. Barker, C. (1992). Group terms in English: Representing groups as atoms. Journal of Semantics, 9, 69–93. Beauchamp, M. S., Petit, L., Ellmore, T. M., Ingeholm, J., & Haxby, J. V. (2001). A parametric fMRI study of overt and covert shifts of visuospatial attention. Neuroimage, 14, 310–321. Behrmann, M., Geng, J. J., & Shomstein, S. (2004). Parietal cortex and attention. Current Opinion in Neurobiology, 14, 212–217. Bookheimer, S. (2002). Functional MRI of language: New approaches to understanding the cortical organization of semantic processing. Annual Review of Neuroscience, 25, 151–188. Cohen, L., Dehaene, S., Naccache, L., Lehéricy, S., Dehaene-Lambertz, G., Hénaff, M.A., et al. (2000). The visual word form area: Spatial and temporal characterization of an initial stage of reading in normal subjects and posterior split-brain patients. Brain, 123, 291–307. Corbetta, M., Miezin, F. M., Shulman, G. L., & Petersen, S. E. (1993). A PET study of visuospatial attention. The Journal of Neuroscience, 13(3), 1202–1226. Culham, J. C., Brandt, S. A., Cavanagh, P., Kanwisher, N. G., Dale, A. M., & Tootell, R. B. H. (1998). Cortical fMRI activation produced by attentive tracking of moving targets. Journal of Neurophysiology, 80(5), 2657–2670. Culham, J., & Kanwisher, N. (2001). Neuroimaging of cognitive functions in human parietal cortex. Current Opinion in Neurobiology, 11, 157–163. Dehaene, S., Le Clec, H. G., Poline, J. B., Le Bihan, D., & Cohen, L. (2002). The visual word form area: A prelexical representation of visual words in the fusiform gyrus. Neuroreport, 13(3), 321–325. Dehaene, S., Piazza, M., Pinel, P., & Cohen, L. (2003). Three parietal circuits for number processing. Cognitive Neuropsychology, 20, 487–506. Dikker, S., Rabagliati, H., Farmer, T. A., & Pylkkänen, L. (2010). Early occipital sensitivity to syntactic category is based on form typicality. Psychological Science, 21(5), 629–634. Farah, M. J., Wong, A. B., Monheit, M. A., & Morrow, L. A. (1989). Parietal lobe mechanisms of spatial attention: Modality-specific or supramodal? Neuropsychologia, 27(4), 461–470. Gelormini-Lezama, C., & Almor, A. (2013). Singular and plural pronominal reference in Spanish. Journal of Psycholinguistic Research. Gordon, P. C., Grosz, B. J., & Gilliom, L. A. (1993). Pronouns, names, and the centering of attention in discourse. Cognitive Science, 17, 311–347. Gordon, P. C., & Hendrick, R. (1998). The representation and processing of coreference in discourse. Cognitive Science, 22(4), 389–424. Gow, D. W. Jr., (2012). The cortical organization of lexical knowledge: A dual lexicon model of spoken language processing. Brain and Language, 121, 273–288. Grosz, B. J., Joshi, A. K., & Weinstein, S. (1995). Centering: A framework for modeling the local coherence of discourse. Computational Linguistics, 21, 203–226. Harvey, B., Klein, B., Petridou, N., & Dumoulin, S. (2013). Topographic representation of numerosity in human parietal lobe. Perception. EVCP Abstract Supplement, p. 241. Hubbard, E. M., Piazza, M., Pinel, P., & Dehaene, S. (2005). Interactions between number and space in parietal cortex. Nature Reviews Neuroscience, 6(6), 435–448. Johnson-Laird, P. N. (1983). Mental models. Cambridge: Cambridge University Press. Kanwisher, N., McDermott, J., & Chun, M. M. (1997). The fusiform face area: A module in human extrastriate cortex specialized for face perception. The Journal of Neuroscience, 17(11), 4302–4311. Kaup, B., Kelter, S., & Habel, C. (2002). Representing referents of plural expressions and resolving plurals anaphors. Language and Cognitive Processes, 17(4), 405–450. Lueck, C. J., Zeki, S., Friston, K. J., Deiber, M.-P., Cope, P., Cunningham, V. J., et al. (1989). The colour centre in the cerebral cortex of man. Nature, 340, 386–389. Malach, R., Reppas, J. B., Benson, R. R., Kwong, K. K., Jiang, H., Kennedy, W. A., et al. (1995). Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. Proceedings of the National Academy of Science USA, 8135–8139. McMillan, C. T., Clark, R., Gunawardena, D., Ryant, N., & Grossman, M. (2012). FMRI evidence for strategic decision-making during resolution of pronoun reference. Neuropsychologia, 50(5), 674–687. Moxey, L. M., Sanford, A. J., Sturt, P., & Morrow, L. I. (2004). Constraints on the formation of plural reference objects: The influence of role, conjunction, and type of description. Journal of Memory and Language, 51, 346–364. Moxey, L. M., Sanford, A. J., Wood, A. I., & Gintner, L. M. N. (2011). When do we use ‘‘they’’ to refer to two individuals? Scenario mapping as a basis for equivalence. Language and Cognitive Processes, 26(1), 79–120. Nieuwland, M. S. (2014). ‘‘Who’s he?’’ Event-related brain potentials and unbound pronouns. Journal of Memory and Language, 76, 1–28. Nieuwland, M. S., Petersson, K. M., & Van Berkum, J. J. A. (2007). On sense and reference: Examining the functional neuroanatomy of referential processing. NeuroImage, 37, 993–1004. T.W. Boiteau et al. / Brain & Language 137 (2014) 130–141 Patson, N. D., & Ferreira, F. (2009). Conceptual plural information is used to guide early parsing decisions: Evidence from garden-path sentences with reciprocal verbs. Journal of Memory and Language, 60, 464–486. Patson, N. D., & Warren, T. (2011). Building complex reference objects from dual sets. Journal of Memory and Language, 64, 443–459. Piazza, M., & Izard, V. (2009). How humans count: Numerosity and the parietal cortex. Neuroscientist, 15(3), 261–273. Sanford, A. J., & Garrod, S. C. (1981). Understanding written language. Chichester: John Wiley & Sons. Sanford, A. J., & Lockhart, F. (1990). Description types and methods of conjoining as factors influencing plural anaphors: A continuation study of focus. Journal of Semantics, 7, 365–378. 141 Shimozaki, S. S., Hayhoe, M. M., Zelinsky, G. J., Weinstein, A., Merigan, W. H., & Ballard, D. H. (2003). Effect of parietal lobe lesions on saccade targeting and spatial memory in a naturalistic visual search task. Neuropsychologia, 41, 1365–1386. van Dijk, T. A., & Kintsch, W. (1983). Strategies of discourse comprehension. New York: Academic Press. Wojuciulik, E., & Kanwisher, N. (1999). The generality of parietal involvement in visual attention. Neuron, 23, 747–764. Yeatman, J. D., Rauschecker, A. M., & Wandell, B. A. (2013). Anatomy of the visual word form area: Adjacent cortical circuits and long-range white matter connections. Brain and Language, 125, 146–155.