Download Natural Language Processing Course

Natural Language Processing
Heshaam Feili
http://mehr.sharif.edu/~hfaili
July 2003
Session Agenda

Artificial Intelligence

Natural Language Processing

History of NLP

Applications of NLP
Natural language Processing
(Heshaam Feili – July 2003)
2
AI Concepts and Definitions



Encompasses Many Definitions
AI Involves Studying Human
Thought Processes
Representing Thought Processes on
Machines
Natural language Processing
(Heshaam Feili – July 2003)
3
Artificial Intelligence



Behavior by a machine that, if
performed by a human being, would be
considered intelligent
“…study of how to make computers do
things at which, at the moment, people
are better” (Rich and Knight [1991])
Theory of how the human mind works
(Mark Fox)
Natural language Processing
(Heshaam Feili – July 2003)
4
AI Objectives



Make machines smarter (primary
goal)
Understand what intelligence is
(Nobel Laureate purpose)
Make machines more useful
(practical purpose)
(Winston and Prendergast [1984])
Natural language Processing
(Heshaam Feili – July 2003)
5
Signs of Intelligence




Learn or understand from
experience
Make sense out of ambiguous or
contradictory messages
Respond quickly and successfully
to new situations
Use reasoning to solve problems
Natural language Processing
(Heshaam Feili – July 2003)
6
More Signs of Intelligence





Deal with perplexing situations
Understand and infer in ordinary,
rational ways
Apply knowledge to manipulate
the environment
Think and reason
Recognize the relative importance
of different elements in a situation
Natural language Processing
(Heshaam Feili – July 2003)
7
Turing Test for Intelligence
A computer can be considered to be
smart only when a human
interviewer, “conversing” with
both an unseen human being and
an unseen computer, can not
determine which is which
Natural language Processing
(Heshaam Feili – July 2003)
8
Symbolic Processing


Use Symbols to Represent Problem
Concepts
Apply Various Strategies and Rules
to Manipulate these Concepts
Natural language Processing
(Heshaam Feili – July 2003)
9
AI Represents Knowledge as
Sets of Symbols
A symbol is a string of characters that
stands for some real-world concept
Examples




Product
Defendant
0.8
Chocolate
Natural language Processing
(Heshaam Feili – July 2003)
10
Symbol Structures
(Relationships)



(DEFECTIVE product)
(EQUAL (LIABILITY defendant) 0.8)
tastes_good (chocolate).
Natural language Processing
(Heshaam Feili – July 2003)
11

AI Programs Manipulate Symbols to Solve
Problems

Symbols and Symbol Structures Form
Knowledge Representation

Artificial Intelligence Dealings Primarily with
Symbolic, Nonalgorithmic Problem- Solving
Methods
Natural language Processing
(Heshaam Feili – July 2003)
12
AI Computing




Based on symbolic representation and
manipulation
A symbol is a letter, word, or number
representing objects, processes, and their
relationships
Objects can be people, things, ideas, concepts,
events, or statements of fact
Creates a symbolic knowledge base
Natural language Processing
(Heshaam Feili – July 2003)
13
AI Computing (cont’d)



Manipulates symbols to generate advice
AI reasons or infers with the knowledge base
by search and pattern matching
Hunts for answers (via algorithms)
Natural language Processing
(Heshaam Feili – July 2003)
14
Major AI Areas

Expert Systems

Natural Language Processing

Speech Understanding
Robotics and Sensory Systems
Computer Vision and Scene Recognition
Intelligent Computer-Aided Instruction
Neural Computing




Natural language Processing
(Heshaam Feili – July 2003)
15
Additional AI Areas





News Summarization
Language Translation
Fuzzy Logic
Genetic Algorithms
Intelligent Software Agents
Natural language Processing
(Heshaam Feili – July 2003)
16
NLP ?




Natural Language is one of fundamental
aspects of human behaviors.
One of the final aim of humancomputer communication.
Provide easy interaction with computer
Make computer to understand texts.
Natural language Processing
(Heshaam Feili – July 2003)
Major Disciplines Studying
Language
Discipline
Typical Problem
Linguists
How do words from phrases and
sentences?
Psycholinguists
How do people identify the
structure of sentences?
Philosophers
What is meaning and how do
words and sentences acquires
it?
Computational
Linguists
How is the structure of
sentences identified?
Natural language Processing
(Heshaam Feili – July 2003)
Interaction Level


The level that computer and human
interact.
NL used for make Interaction level near
to human.
Graphical UI
Command-line
NL UI
Human
Computer
Interaction level
Natural language Processing
(Heshaam Feili – July 2003)
Natural Language Processing
(NLP)

Natural language processing concerns the development of
computational models of aspects of human language
processing such as :





Reading and interpreting a textbook
Writing a letter
Holding a conversation
Translating a document
Searching for useful information
Such models are useful in order to write computer programs to
perform useful tasks involving language processing and in
order to develop a better understanding of human
communication.
Natural language Processing
(Heshaam Feili – July 2003)
20
Other Titles




The most common titles, apart from
Natural Language Processing include:
Automatic Language Processing
Computational Linguistics
Natural Language Understanding
Natural language Processing
(Heshaam Feili – July 2003)
21
Computational Lingusitics


This is the application of computers to the
scientific study of human language.
This definition suggests that there are
connections with Cognitive Science, that is
to say, the study of how humans produce
and understand language.
Natural language Processing
(Heshaam Feili – July 2003)
22
Computational Lingusitics

Historically, Computational
Linguistics has been associated with
work in Generative Linguistics and
formerly included the study of formal
languages (eg finite state automata)
and programming languages.
Natural language Processing
(Heshaam Feili – July 2003)
Natural Language Understanding


Distinguish a particular approach to
Natural Language Processing.
The people using this title tend to lay
much emphasis on the meaning of the
language being processed, in
particular getting the computer to
respond to the input in an apparently
intelligent fashion.
Natural language Processing
(Heshaam Feili – July 2003)
24
Natural Language
Understanding

At one time, those who belonged to
the Natural Language Understanding
camp avoided the use of any syntactic
processing, but textbooks that bear
this title now include significant
sections on syntactic processing,
which suggests that the edge of the
title has been rather blunted. (For
instance, see Allen (1987; part 1).
Natural language Processing
(Heshaam Feili – July 2003)
NLP History (1)


The first recognizable NLP
application was a dictionary lookup system developed at Birkbeck
College, London in 1948.
NLP from 1966-1980

Augmented Transition Networks

Case Grammar
Natural language Processing
(Heshaam Feili – July 2003)
26
NLP History (2)

NLP from 1966-1980

Semantic representations



Schank and his workers introduced the notion of
Conceptual Dependency, a method of expressing
language in terms of semantic primitives. Systems were
written which included no syntactic processing.
QuillianÕs work on memory introduced the idea of the
semantic network, which has been used in varying
forms for knowledge representation in many systems.
William Woods used the idea of procedural semantics to
act as an intermediate representation between a
language processing system and a database system.
Natural language Processing
(Heshaam Feili – July 2003)
27
NLP History (3)

The key systems were:



LUNAR: A database interface system that used ATNs and
Woods' Procedural Semantics.
LIFER/LADDER: One of the most impressive of NLP
systems. It was designed as a natural language interface to
a database of information about US Navy ships.
NLP from 1980 - 1990
- Grammar Formalisms

NLP from 1990- now
- Multilinguality and Multimodality
Natural language Processing
(Heshaam Feili – July 2003)
28
NLP Applications

Applications can be classified in different
ways, e.g. medium/modality; depth of
analysis; degree of interaction

Text-based applications

NL Understanding

Dialogue Systems

Multimodal
Natural language Processing
(Heshaam Feili – July 2003)
29
Text-based Applications
Processing of written texts such as books, news, papers,
reports:


Finding appropriate documents on certain topics from a
text database
Extracting information from messages, articles, Web
pages, etc.
Natural language Processing
(Heshaam Feili – July 2003)
30
Text-based Applications


Translating documents from one language
to another
Text summarization
Note: Not all such applications require NLP
Keyword based techniques can used for
identifying particular subject areas, e.g.
legal, financial, etc.
Natural language Processing
(Heshaam Feili – July 2003)
NL Understanding
Other kinds of request require a deeper level of analysis
Find me all articles concerning car accidents involving more
than two cars in Malta during the first half of 2001
Here the system must extract enough information to
determine whether the article meets the criterion defined
by the query.
Natural language Processing
(Heshaam Feili – July 2003)
32
NL- Understanding

A crucial characteristic of an
understanding system is that it can
compute some representation of the
information that can be used for later
inference
A crucial question for an NLP system is
how much understanding is necessary
to achieve the purpose of the system.
Natural language Processing
(Heshaam Feili – July 2003)
Dialogue-based Applications
Dialogue-based applications involve man-machine
communication


NL database query systems
Automated customer services, e.g. banking
services
Natural language Processing
(Heshaam Feili – July 2003)
34
Multimodal Applications
Involve two or more modalities of communication

Text

Speech

Gesture

Image
Text  speech
Speech  text

Multimodal document generation

Spoken translation systems

Spoken dialogue systems
Natural language Processing
(Heshaam Feili – July 2003)
35
Natural language Processing
(Heshaam Feili – July 2003)