Download Seeing and Saying - Dept. of Psychology (internal)

Seeing and Saying
Griffin, Z. M. & Bock, K. 2000. What the eyes say about
speaking. Psychological Science 11:4. 274-279.
Griffin, Z. M. 2004. The eyes are right when the mouth is
wrong. Psychological Science 15:12. 814-821.
What the eyes say about speaking
Griffin & Bock 2000
●
●
Time course of sentence formulation was studied by
monitoring eye movements.
Does event apprehension precede sentence formulation?
●
●
If so, “sentence production consists of a wholistic
conceptualization followed by the sequential expression of
linguistic constituents”. (Wundt-Lashley view)
If not, “sentences are the sums of their parts, originating in
sequential associations among individual concepts that are
outwardly manifested as a series of words.” (Paul's view)
Griffin & Bock 2000: Methodology
●
Four groups of participants
●
●
●
●
How much and what kind of
Extemporaneous speech (20) sentence formulation precedes
extemporanous speech
Prepared speech (12)
To estimate amount and kind of
Patient-detection (8)
viewing needed for event
apprehension
Inspection (8)
To detect whether picture
elements were more salient than
others
Griffin & Bock 2000: Methodology
original
role-traded
mirror images
Modal active sentences on 84.3% of
the trials
“victims”
Modal passives or other patientsubjects on 83.3% of trials
Role-traded versions: Modal actives
on 85.4%
Griffin & Bock 2000: Results
1690 ms
336 ms
456 ms
Griffin & Bock 2000: Results
Subjects:
646 ms vs. 179 ms
Objects:
244 ms vs. 812 ms
Griffin & Bock 2000: Results
472 ms
336 ms
●
●
For both groups, the divergence marks the beginning of sustained attention to
the subject region.
“The probability of initially fixating regions associated with subjects, objects, or
actions did not differ significantly.”
●
Strong dependencies between eye fixations and sentence elements.
●
Support for Wundt-Lashley hypothesis about sentence production
Griffin & Bock 2000: Results
Formulation and execution
●
Eye-voice span (last gaze to onset of speech)
●
Subject = 902 ms
●
Object = 932 ms
●
Similar to the 910 ms in picture naming tasks for isolated objects
●
●
●
Extemporaneous speakers incrementally select and phonologically encode
nouns.
Fixations to objects in the prepared-speech group were as long as those to
subjects in the extemporaneous group (890 vs. 824 ms).
Disfluencies in prepared speech were shorter than in extemporaneous speech.
Griffin & Bock 2000: Conclusions
●
Apprehension precedes formulation → wholistic process of conceptualization
●
●
●
Prior to initiating the process of sentence formulation, speakers have
inspected events well enough to code them.
Systematic temporal linkage between eye movements and the contents of
spoken utterances
Focus only on fluent utterances
The eyes are right when the mouth is wrong
Griffin 2004
●
●
●
Longer gazes for objects with uncommon names or multiple names.
Speakers tend to look at objects while preparing names, but not
when articulating them (see figure 4).
What is the relationship between eye gazes and speech errors?
●
●
●
“Speech errors may be associated with gazing too briefly at intended
objects or too long at other objects while preparing speech”. (p.815)
Shorter gazes due to omission of name check
Are speakers' gazes prior to errors distinct from their gazes prior to
correct noun productions?
Griffin 2004: Data selection
●
●
41 full or partial speech errors from 33 participants in different eyetracking experiments.
Speech errors:
●
●
●
Speakers signaled the error lexically (“oops”, “no”)
Speakers interrupted their own speech (only if they resumed speaking
with different word-initial sounds).
These eye movement measures were compared to those associated
with correct names (by speaker -in similar positions- and item).
●
●
For disfluent speech errors, eye movement measures were calculated
separately for fluent and disfluent correct names.
Age was also taken into account (errors naming the same object made
by older or younger speakers were treated as two different items).
Griffin 2004: Data treatment
Gaze time
before error's onset
Gaze time
after error's onset
http://homepage.psy.utexas.edu/homepage/faculty/griffin/Demos.html
Griffin 2004: Results
Before name onset
●
●
●
No difference in mean number of gazes to objects before the onset of
errors and correct names or in eye-voice spans (about 1 s)
Longer gazes to intended objects before errors were due to the fact that
objects that elicited errors were time-consuming to name in general.
Gaze shifting typically occurs 100-300 ms before articulating the name.
●
Correct names: 100 ms
●
Errors: Speakers continued to fixate the objects.
Griffin 2004: Results
After name onset
●
Interruptions occurred 519 ms (93-2,608) after the onset of the error.
●
Corrections were uttered 544 ms (40-2,439) later.
●
Does gazing at the object help the speaker detect the error?
●
●
Interruptions do not occur earlier when the speakers continue fixating the
intended object.
Speakers gazed at objects for twice as long after the onsets of errors than
after the onsets of correct names (p<.002).
●
●
This time was positively correlated with the latency from interruption to
correction (r=0.50, p<.005).
No differences between duration of gazes after the onsets of corrections and
after onset of correct names
Griffin 2004: Discussion
●
●
Speech errors of this type may not be systematically due to shorter
gazes on referents (i.e., shorter word-preparation times).
Word-substitution errors may reflect problems related to linguistic
processes rather than extralinguistic ones such as visual attention.
●
●
●
At a conceptual level, speakers know what they want to say.
Speech errors do not seem to involve omitting or rushing through
word-production processes.
Gazing at referents while preparing their names does not ensure
that the names will be correct.
●
In word-substitution errors, gazes may be more informative about
speakers' intentions than speech.