Download Document

Natural Language Processing
Vasile Rus
http://www.cs.memphis.edu/~vrus/teaching/nlp
Outline
• Logic Form
Problem Description

There is need for Knowledge Bases
E.g.: Question Answering
1.
find the answer to
Q471: What year did Hitler die?
in a collection of documents
A: “Hitler committed suicide in 1945”
2.

how would one justify that it is the right answer: using world knowledge
suicide – {kill yourself}
kill – {cause to die}
Create intelligent interfaces to databases:
E.g.: Where can I eat Italian food?
Or: I'd like some pizza for dinner. Where can I go?
How to Build Knowledge Bases?
• Manually
– building common sense knowledge bases
– see Cyc, Open Mind Common Sense
• Automatically
–from open text
–from dictionaries like WordNet
Logic Form Representation
• What representation to use?
• Logic Form (LF) is a knowledge
representation introduced by Jerry
Hobbs (1983)
• Logic form is a first-order representation
based on natural language
First Order Representations
• Fulfil the five main desiderata for
representing meaning:
1. Verifiability:
– Does Maharani serve vegetarian food?
– Serves(Maharani, vegetarian food)
– A representation that can be used to
match a proposition against a knowledge
base
First Order Representations
2. Unambiguous representations:
I would like to eat someplace close to UofM.
= eat in a place near UofM
= eat a place
Get rid of ambiguity by assigning a sense to
words, or by adding additional information that
rules out ambiguity.
A representation should be free of ambiguity.
First Order Representations
3. Canonical Form
–Does Maharani serve vegetarian food?
–Are vegetarian dishes served at
Maharani?
–Do they have vegetarian food at
Maharani?
–Texts that have the same meaning should
have the same representation.
First Order Representations
4. Inference and Variables
– The ability to draw inferences from the
representations
– Serves(x, Vegetarian Food) -->
EatAt(Vegetarians, x)
5. Expresiveness
– Representations should be expressive
enough to handle a wide range of subjects.
Induction, Abduction
• Use FOP for automatic reasoning
• How?
• Induction
• Abduction
Logic Form Transformations
• First order representations
–have the characteristics of FOP
• Add some extra information (e.g. POS, word
sense)
• Derived automatically from text, starting with
parse trees
• Used for automatic construction of
knowledge bases:
–e.g. Starting with WordNet
WordNet as a Source of World
Knowledge
• WordNet, developed at Princeton by Prof.
Miller, is an electronic semantic network
whose main element is the synset
– synset – a set of synonym words that define a concept
• E.g.: {cocoa, chocolate, hot chocolate}
• a word may belong to more than one synset
• WordNet contains synsets for four parts of
speech: noun, verb, adjective and adverb
WordNet
• synsets are related to each other via a set of
relations: hypernymy (ISA),
hyponymy(reverseISA), cause, entailment,
meronymy(PART-OF) and others.
• hypernymy is the most important relation which
organizes concepts in a hierarchy (see next
slide)
• adjectives and adverbs are organized in clusters
based on similarity and antonymy relations
WordNet glosses
• Each synset includes a small textual definition
and one or more examples that form a gloss.
• E.g.:
–{suicide:n#1} – {killing yourself}
–{kill:v#1} – {cause to die}
–{extremity, appendage, member} – {an
external body part that projects from the
body “it is important to keep the extremities
warm”}
• Glosses are a rich source of world knowledge
• Can transform glosses into a computational
representation
Logic Form Representation
• A predicate is a concatenation of the
morpheme’s base form, part of speech
and WordNet semantic sense
– morpheme:POS#sense(list_of_arguments)
• There are two types of arguments:
– x – for entities
– e – for events
• The position of the arguments is
important
– verb:v#sense(e, subject, direct_object,
indirect_object)
– preposition(head, prepositional_object)
Logic Form Representation
• A predicate is generated for each noun,
verb, adjective and adverb
• Complex nominals are represented using
the predicate nn:
– e.g.: “goat hair” – nn(x1, x2, x3) & goat(x2) & hair(x3)
• The logic form of a sentence is the
conjunction of individual predicates
An Example
• {lawbreaker, violator}: (someone who
breaks the law every day)
• Someone:n#1(x1) & break:v#6(e1, x1, x2;
x3) & law:n#1(x2) & day:n#1(x3)
Part of Speech
Categorial
Information
WordNet sense
Subject
Direct object
Semantic
Information
Functional
Information
Logic Form Notation (cont’d)
• Ignores: plurals and sets, verb tenses,
auxiliaries, negation, quantifiers, comparatives
• Consequence:
–Glosses with comparatives can not be fully
transformed in logic forms
• The original notation does not handle special
cases of postmodifiers (modifiers placed after
modifee) respectively relative adverbs (where,
when, how, why)
Comparatives
• {tower}: (structure taller than its
diameter)
• taller/JJR modifies structure or
diameter? Both?
• Solution: introduce a relation
between structure and diameter
• LF: structure(x1) & taller(x1, x2) &
diameter(x2)
Postmodifiers
• {achromatic_lens}: (a compound lens
system that forms an image free from
chromatic_aberration)
• Free is a modifier of image ?
• What is the prepositional head of
from ?
• Solution: free_from – NEW predicate
• LF: image(x1) & free_from(x1, x2) &
chromatic_aberration(x2)
Relative Adverbs
• {airdock}: (a large building at an airport
where aircraft can be stored)
• Equivalent to: (aircraft can be stored in a
large building at an airport)
• LF: large(x1) & building(x1) & at(x1, x2)
& airport(x2) & where(x1, e1) &
aircraft(x3) & store(e1, x4, x3)
Logic Form Identification
• Take advantage of the structural
information embedded in a parse tree
S
NP VP-PASS
NP VP-ACT
S -> NP VP
NP
VP
Architecture
Preprocess
(Extract Defs, Tokenize)
Direct object
POS Tag
Subject
Parse
LF
Transformer
Example of Logic Form
NP
NP
DT
VP
NN
VBN
PP
NP
IN
DT
NN
a
monastery
ruled
by
an
abbot
monastery:n(x1) rule:v(e1, x2, x1) abbot:n(x2)
Logic Form Derivation
• Take advantage of the syntactic information
from the parser
• For each grammar rule derive one or more LF
identification rules
Identification Rules
Grammar Rule
Rule
Phrase
Synset
NP  DT NN
Noun/NN  noun(x)
(NP (a/DT monastery/NP))
{abbey:n#3}
VP  VP PP
Verb(e, -, -)/VP-PASS by/PP(-,x) 
(VP (ruled/VBN by/PP))
{abbey:n#3}
verb(e,x, -) & by(e,x)
NP
DT
VP
NN
VP
PP
Building a Knowledge Base
from WordNet
• We parsed all glosses and extract all
grammar rules embedded in the parse trees
Part of speech
Rules
Noun
5,392
Verb
1,837
Adjectives
1,958
Adverbs
Total
639
9,826
• The grammar is large
• If we consider that a grammar rule can map
in more than one LF rules the effort to
analyse and implement all of them would be
tremendous
Coverage issue
• Group the grammar rules by the non terminal
on the Left Hand Side (LHS) and notice that
the most frequent rules for some class cover
most of the occurrences of rules belonging to
that class
Occurrences
Unique Rules
Coverage of top ten
Base NP
33,643
857
69%
NP
11,408
244
95%
VP
19,415
450
70%
PP
12,315
40
99%
S
14,740
35
99%
Phrase on the LHS
of Grammar Rule
Coverage issue (cont’d)
• Two phases:
–Phase 1: develop LF rules for most frequent
rules and ignore the others
–Phase 2: select more valuable rules
• The accuracy of each LF rule is almost perfect
• The performance issue is mainly about how
many glosses are entirely transformed into LF
– i.e. how many glosses the selected grammar rules
fully map into LF
Reduce the number of
candidate grammar rules (1)
• Selected grammar rules for baseNPs (non-recursive
NPs) have only a coverage of 69%
• Selected grammar rules for VPs have only 70%
coverage
• Before selecting rules for baseNPs we make some
transformations to reduce more complex ones to
simpler ones
NP
NP
•
NP
DT
NN CC
NP
NN
DT
a
CC
ruler or institution
a
NN
NN
ruler or institution
Reduce the number of
candidate grammar rules (2)
• Base NPs:
–Determiners are ignored (an increase of
11% in coverage for selected grammar rules
for base NPs)
–Plurals are ignored
–Everything in a prenominal position plays
the role of a modifier
Base NP rule
NP  DT JJ NN|NNS|NNP|NNPS
NP  DT VBG NN|NNS|NNP|NNPS
NP  DT VBN NN| NNS | NNP|NNPS
Map grammar rules into LF
rules
• Selected grammar rules map into one or more
Logic Form rules
• Case 1: grammar rule is mapped into one LF
rule
– Grammar rule: PP -> IN NP
– LFT: prep(_, x)  prep(_, x) & headNP(x)
Map grammar rules into LF rules
• Case 2: grammar rule is mapped into one or
more LF rules:
– Grammar rule: VP -> VP PP
– LFT 1: verb(e, x1, _)  verb-PASS(e,x1, _) & prepBy(e, x1)
– LFT 2: verb(e, _, x2)  verb-PASS(e, _, x2) & prepnonBy(e, x2)
– To differentiate among the two cases we use two
features:
• The mood of the VP: active or passive
• The type of preposition: by or non-by
Question Answering
Application
• Given a question and an answer the task is to select
the answer from a set of candidate answers and to
automatically justify that the answer is the right
answer
• Ideal case: all the keywords from the question
together with their syntactic relationship exist in the
answer
– Question: What year did Hitler die?
– Perfect Answer: Hitler died in 1945.
• Real case:
– Real Answer: Hitler committed suicide in 1945.
– Requires extra resources to link suicide to die: use WordNet
as a knowledge base
From Logic Forms to Axioms
• WordNet glosses transformed into
axioms, to enable automated reasoning
• Specific rules to derive axioms for each
part of speech:
– Nouns: the noun definition consists of a genus and differentia.
The generic axiom is: concept(x)  genus(x) & differentia(x).
• E.g.: abbey(x1)  monastery(x1) & rule(e1, x2, x1) & abbot(x2)
– Verbs: are more trickier as some syntactic functional changes
can occur from the left hand side to the right hand side
• E.g.: kill:v#1(e1, x1, x2, x3)  cause(e2, x1, e3, x3) & die(e3, x2)
From Logic Forms to Axioms
– Adjectives: they borrow a virtual argument
representing the head they modify
• E.g.: american:a#1(x1)  of(x1, x2) &
United_States_Of_America(x2)
– Adverbs: the argument of an adverb borrows
a virtual event argument as they usually
modify an event
• E.g: fast:r#1(e1)  quickly:r#1(e1)
Summary
• Logic Form
Next Time
• Applications