Download File4

Hand trajectories reveal hidden
cognitive states
Ken Nakayamaa
Joo-Hyun Songb
Matthew Finkbeinerc
Alfonso Caramazzaa
aHarvard
University
bSmith-Kettlewell Eye Research Institute
cMacquarie University
today’s talk
Suggest there are new
opportunities to study a wide
range of internal cognitive states
from the detailed study of motor
behavior
(Spivey et al., for language processing)
Remarkable power of psychophysical
and behavioral methods
•
•
•
•
•
Some examples
Wertheimer, 1912
Hecht, Shlaer and Pirenne, 1943
Julesz 1960s
Sperling 1960s
input
Early processing
Later processing
Cognition/decision
Motor plan
input
Early processing
fMRI
Later processing
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
COGNITIVE
EEG/MEG
NEUROSCIENCE
Motor plan
Cognition/decision
TMS
input
Reaching
Early processing
Later processing
Cognition/decision
Motor plan
Measure RT
And accuracy
PLUS
Full details of trajectory
Reveals otherwise
hidden states
Visually-guided pointing
.
Song and Nakayama (VSS 05,06)
QuickTime™ and a
Sorenson Video 3 decompressor
are needed to see this picture.
Hand movement
recording:
120Hz with Polhemus
Fastrak
• Recording
3-D movement
of finger tip
Polhemus system: Sample rate (120 Hz)
Two recent studies
• I. Early hand trajectories reflect an underlying
number representation
– Song and Nakayama, Cognition (in press)
• II. pointing trajectories reveal influence of
unseen words
– Finkbeiner, Song, Nakayama, and Caramazza,
Visual Cognition (in press)
Mental number line
Numeric distance effect
In numeric comparison task
if digital number line
3 vs. 5, 13 vs. 5
IF X>5
respond “greater”
Else
respond “lesser”
X
Same response to
3 vs. 5 &13 vs. 5
BUT, RT and error rates systematically decrease as the
numerical distance between two numbers increases.
Supporting “Analog” number line
Moyer and Landauer (1967)
• Characteristics of number
representations are mainly examined
with discrete responses such as
reaction time and accuracy.
• In the current study, to map invisible
internal cognitive processes of numeric
comparison in spatial domain over
time, we measured trajectories.
X-Y trajectories
• Systematic shift of initial trajectories towards a hypothetical position
on a number line intermediate between the numeral 1 and 5 positions
Mean trajectory deviation
• ≈ 480 ms after target onset
• ≈ 150 ms after the movement onset
Endpoint deviation
• Continuous internal processes for number
representations
• Early detection of mental numeric line than
previous tasks
• Kinematic measurements: More reliable and
robust with small numbers of subjects
• I. Early hand trajectories reflect an underlying
number representation
– Song and Nakayama, Cognition (in press)
• II. Pointing trajectories reveal influence of
unseen words
– Finkbeiner, Song, Nakayama, and Caramazza,
Visual Cognition (in press)
linkage between invisible text
and actions?
Goodale and Milner
Dorsal
Vs
Ventral
Point to color of the word
spinach
neutral
incongruent
########
500 ms
30ms
10 ms
Until Response
boy
KMDKS
spinach
########
500
red
30
10
2000
KMDKS
spinach
congruent
########
500
green
30
10
2000
KMDKS
spinach
neutral
incongruent
########
500 ms
30ms
10 ms
Until Response
boy
KMDKS
spinach
########
500
red
30
10
2000
KMDKS
spinach
congruent
########
500
green
30
10
2000
KMDKS
spinach
5
4.5
Horizontal movement (in)
4
congruent
neutral
3.5
3
2.5
incongruent
2
1.5
1
0.5
100 msec after movement onset
0
Time after movement initiation
Replication with varying prime duration
• Despite the unavailability of the invisible
stimulus to visual awareness, the congruity
of the prime stimulus with the target
stimulus had a significant effect on
participants’ pointing trajectories.
• Masked unseen word processing extends
down to include the formulation of overt
motor responses
input
Reaching
Early processing
Later processing
Cognition/decision
Motor plan
Full details of trajectory
LEAKAGE
Suggest there are new
opportunities to study a wide
range of internal cognitive
states from the detailed study
of motor behavior