Download Goldsmith

Words of a feather
flock together
John Goldsmith (Linguistics and Computer Science)
Work done with:
Jackson Lee (Linguistics)
Simon Jacobs (RCC)
Stephen Fitz (Computer Science)
Looking at texts: documents
●
Texts contain a great deal of structure.
●
That structure can be divided into:
●
●
–
Structure of the content;
–
Structure of the language.
Look at a series of books in French: what they share
in common is the structure of French.
Look at a series of translations of Harry Potter and
the Sorcerer's Stone: what they share in common is
the structure of the content.
We care about the structure of the
language (today)
●
●
Structure of a language includes a lexicon: the
words of the language.
The order in which words appear is largely
determined by the syntax of the language.
Word categories
●
Noun
●
Verb
●
Participle
●
Article
●
Pronoun
●
Preposition
●
Adverb
●
Conjunction
undue
undue
mental
undue
mental
valid
undue
mental
prison
valid
undue
mental
prison
equitable
valid
silly
silly
bagel
silly
eggs
bagel
silly
eggs
bagel
Mary
silly
eggs
crazy
bagel
Mary
silly
eggs
crazy
bagel
Mary
scarf
silly
eggs
crazy
slide-rule
bagel
Mary
scarf
silly
eggs
crazy
slide-rule
bagel
Mary
scarf
Hyde Park
silly
eggs
crazy
slide-rule
bagel
Mary
scarf
Hyde Park
bookcases
silly
eggs
crazy
slide-rule
bagel
Mary
scarf
Hyde Park
cool
bookcases
silly
eggs
crazy
bagel
Mary
scarf
Hyde Park
cool
stomach
slide-rule
bookcases
silly
eggs
crazy
bagel
Mary
scarf
Hyde Park
cool
stomach
M
slide-rule
bookcases
silly
eggs
crazy
bagel
Mary
scarf
Hyde Park
cool
stomach
M
slide-rule
bookcases
Barack
silly
eggs
crazy
bagel
Mary
scarf
Hyde Park
cool
stomach
M
cables
slide-rule
bookcases
Barack
Rheummate-ly
Rheummate-ly
jump
Rheummate-ly
sneakers
jump
Rheummate-ly
sneakers
Mitt
Romney
jump
Rheummate-ly
sneakers
Mitt
Romney
Kenwood
jump
Rheummate-ly
sneakers
Mitt
Romney
Kenwood
Igbo
jump
Rheummate-ly
sneakers
jump
Mitt
Romney
Kenwood
Igbo
Soap
Rheummate-ly
sneakers
jump
Mitt
Romney
Kenwood
Igbo
Soap
differentiate
Rheummate-ly
sneakers
jump
Mitt
Romney
Kenwood
Igbo
Soap
differentiate
pogosticks
Rheummate-ly
sneakers
jump
Mitt
Romney
Kenwood
Igbo
Soap
differentiate
GoogleMine
pogosticks
Rheummate-ly
sneakers
jump
Mitt
Romney
Kenwood
Igbo
Soap
differentiate
GoogleMine Rheummate-ly
pogosticks
Rheummate-ly
sneakers
jump
Mitt
Romney
Kenwood
Igbo
Soap
differentiate
GoogleMine Rheummate-ly
crowd-licking
pogosticks
Rheummate-ly
jump
Mitt
Romney
sneakers
Kenwood
Igbo
Soap
differentiate
GoogleMine Rheummate-ly
crowd-licking
pogosticks
taxi business
Rheummate-ly
jump
Mitt
Romney
sneakers
Kenwood
Igbo
Soap
differentiate
GoogleMine Rheummate-ly
crowd-licking
thinkify
pogosticks
taxi business
Empirically, not so obvious:
'the' versus 'a'
Left
of
in
to
on
and
for
at
with
that
from
by
is
as
into
was
:
all
In
the
Right
9629
5567
3433
2315
2143
1766
1511
1481
1392
1352
1315
793
698
673
531
466
424
412
same
f rst
other
most
United
world
*j
time
two
new
way
only
end
last
next
state
whole
old
a
Left
601
547
388
380
367
338
264
243
232
231
226
207
204
202
185
182
178
171
of
in
as
with
is
was
for
to
and
on
by
be
at
have
had
like
from
such
Right
1467
1321
914
879
872
783
740
655
615
476
413
318
256
256
246
242
229
222
few
little
man
new
good
small
great
very
long
number
large
year
single
moment
lot
more
time
matter
344
292
253
248
221
161
153
138
135
117
110
99
99
86
86
84
79
79
masculine singular nouns
feminine
singular nouns
xiisiecle
cities
simple past verbs
infinitives
years
prenominal
modifiers
plural
nouns
French
English
33
Dynamic view: English color codes
Verbs: ‘bare’ verb (jump)
Verbs: past tense(jumped, bought)
Verbs: auxiliary (should, can)
Prepositions (from, to, up, down
Adjectives
Cities
Nouns
35
red
blue
green
aqua
purple
gray
pink
Dynamic view: French color codes
Infinitives
red
Prepositions
light blue
Past participles blue
Adjectives
purple
Cities
gray
Masculine nouns pink
Feminine nouns light green
Inflected verbs light gray
36
Finding regular and irregular
past tense forms
founded
formed
discovered
named
carried
established
assumed
considered
initiated
called
influenced
placed
caused
joined
created
adopted
supported
forced
brought
provided
ordered
37
played
achieved
38
39