Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Technological singularity wikipedia , lookup
Knowledge representation and reasoning wikipedia , lookup
Embodied cognitive science wikipedia , lookup
Embodied language processing wikipedia , lookup
Philosophy of artificial intelligence wikipedia , lookup
Ethics of artificial intelligence wikipedia , lookup
History of artificial intelligence wikipedia , lookup
Intelligence explosion wikipedia , lookup
Existential risk from artificial general intelligence wikipedia , lookup
Chapter 15 Natural Language Processing (cont) 323-670 Artificial Intelligence ดร.วิภาดา เวทย์ ประสิทธิ์ ภาควิชาวิทยาการคอมพิวเตอร์ คณะวิทยาศาสตร์ มหาวิทยาลัยสงขลานครินทร์ NLP Problems Figure 15.1 P. 378 English sentences are incomplete descriptions of the information that are intended to convey. The same expression means different things in different context. No natural language program can be complete because of new words, expression, and meaning can be generated quite freely. There are lots of ways to say the same thing. 323-670 Artificial Intelligence Lecture36-40 Page 2 NLP Problems 1) Processing written text – using lexical, syntactic, and semantic knowledge of the language – the require real world information 2) Processing spoken language – using all information needed above – plus additional knowledge about phonology – handle ambiguities in speech 323-670 Artificial Intelligence Lecture36-40 Page 3 NLP Natural Language processing Language translation / multilingual translation Language understanding – Figure 14.5 p. 365 Interaction among component – Figure 14.6 p. 366 A speech Waveform 323-670 Artificial Intelligence Lecture36-40 Page 4 Step in NLP 1) Morphological Analysis 2) Syntactic Analysis 3) Semantic Analysis 4) Discourse Integration 5) Pragmatic Analysis – boundaries between these five phrases are often fuzzy. 323-670 Artificial Intelligence Lecture36-40 Page 5 1. Morphological Analysis Individual words are analyzed into components Nonword tokens such as punctuation are separated from the words I want to print Bill’s .int file. file extension proper noun possessive suffix 323-670 Artificial Intelligence Lecture36-40 Page 6 2. Syntactic Analysis linear sequence of words are transformed into structures show how words relate to each other English syntactic analyzer If do not pass the syntactic analyzer reject (Boy the go to store the) 323-670 Artificial Intelligence Lecture36-40 Page 7 2. Syntactic Analysis Example of syntactic analysis – Figure 15.2 p. 382 RM2, RM5, RM5 A knowledge base Fragment – Figure 15.3 p. 383 – User073, F1, Printing, File_Structure, Waiting – Mental Event/ Physical Event Animate/Event Partial meaning for a sentence – Figure 15.4 p. 384 323-670 Artificial Intelligence Lecture36-40 Page 8 3. Semantic Analysis the structures created by the syntactic analyser are assign meanings mapping between the syntactic structure and objects in the task domain If no mapping reject (colorless green ideas sleep furiously) 1) It must map individual words into appropriate objects in the knowledge base or database. 2) It must create the correct structures to correspond to the meanings of the individual words combine with each other. 323-670 Artificial Intelligence Lecture36-40 Page 9 4. Discourse Integration the meaning of the individual sentence may depend on the sentences that precede it and may influence the meanings of the sentences that follow it. (Ex. John want it.) “It” depends on the previous sentence. Current user who type word “I” is – User068 = Susan_Black We get F1 with filename in /wsmith/ directory 323-670 Artificial Intelligence Lecture36-40 Page 10 5. Pragmatic Analysis The structure representing what was said is reinterpreted to determine what was actually meant. (Ex. Do you know what time it is?) we should understand what to do.... Understand to decide what to do as a result Representing the intended meaning – Figure 15.5 P. 385 323-670 Artificial Intelligence Lecture36-40 Page 11 Syntactic Processing Top-down Parsing – Begin with start symbol and apply the grammar rules forward until the symbols at the terminals of the tree correspond to the components of the sentence being parsed. Bottom-up Parsing – Begin with the sentence to be parsed and apply the grammar rules backward until a single tree whose terminals are the words of the sentence and whose top node is the start symbol has been produced. 323-670 Artificial Intelligence Lecture36-40 Page 13 ATN : Augmented Transition Network similar to finite state machine – Figure 15.8 p.392 An ATN network – Figure 15.9 p.3923An ATN Grammar in List Form – sentence “The long file has printed.” – S NP Q1 AUX Q3 V Q4 (F) halt – NP Det Q6 Adj Q6 N Q7 (F) (S DCL (NP (FILE (LONG) DEFINITE)) HAS 323-670 Artificial Intelligence (VP PRINTED)) Lecture36-40 Page 14