Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Artificial Intelligence Academic year 2016/2017 Giorgio Fumera http://pralab.diee.unica.it [email protected] Pattern Recognition and Applications Lab Department of Electrical and Electronic Engineering University of Cagliari Outline Part I: Historical notes Part II: Solving Problems by Searching Uninformed Search Informed search Part III: Knowledge-based Systems Logical Languages Expert systems Part IV: The Lisp Language Part V: Machine Learning Decision Trees Neural Networks Part IV The Lisp Language Programming paradigms Many high-level programming languages have been devised since the 1950s, each one for a more or less specific purpose: I “number crunching”: FORTRAN (1957, first high-level language) I artificial intelligence (symbol manipulation): Lisp (1958) I business: COBOL (1959) I system programming: C (1972) I data bases: SQL (early 1970s) I ... One of the main distinguishing features: programming paradigms I procedural (imperative) I declarative Procedural languages Main characteristics: I Von Neumann’s computing paradigm I program: a sequence of instructions expressing actions to be executed Examples: I machine language, Assembly I FORTRAN, Pascal, C, Python, etc. I object-oriented languages: SmallTalk, C++, Java, etc. Declarative languages Main characteristics: I higher abstraction level than procedural languages I programs: recursive expressions, representing what to obtain, not how Examples: I logical languages: SQL, Prolog I functional languages: Lisp, Erlang, Haskell, OCaml, Scheme Characteristics of procedural languages Computational model: I explicit manipulation of the content of memory cells (variables) I control structures: conditional execution, iteration Pros: I most programmers are used to procedural thinking I program structure is close to executor structure (hardware): efficient translation and execution Cons: I side effects: explicit alteration of memory content (machine state) I understanding programs and verifying correctness is difficult Drawbacks of procedural languages: examples 1. Are these C statements equivalent? w = x + f(y); w = f(y) + x; Not necessarily: x can be a global variable modified by f. This is an example of side effect: the outcome of a statement depends on its context. Drawbacks of procedural languages: examples 2. What is the operation carried out by the following C procedure? void mystery (int n) { int i,j; boolean b; for (i = 2; i <= n; i++) { j = 2; b = true; while (b == true && j <= i/2) if (i % j != 0) j++; else b = false; if (b == true) printf (“%d ”, i); } } Difficult to understand, due to nested loops and shared variables. Drawbacks of procedural languages: examples 3. The previous C procedure prints all prime numbers from 2 to its argument. Is that true? It is not easy to verify that a program behaves how it is supposed (or claimed) to, for the same reasons above. Characteristics of (ideal) functional languages Program structure: recursive expressions (including function calls) which are evaluated to produce a value I mathematics-like style I no variables I recursion takes the place of iteration Pros: I I I absence of variables ∆ no side effects referential transparency: the value of expressions is context-independent (like mathematical expressions) it is easier to understand and verify the correctness of programs Cons: I thinking recursively may be non-intuitive I recursion requires higher processing cost (memory and time) Functional-style programming: an example A C function for computing the factorial of a natural number: I procedural style: int factorial (int n) { int i, fact = 1; for (i=2; i <= n; i++) fact = fact * i; return fact; } I functional style (no explicit alteration of variables by assignment, recursion used instead of iteration): int factorial (int n) { if (n == 0) return 1; else return n * factorial (n - 1); } Characteristics of real programming languages The only pure procedural language: machine language / Assembly. All the other existing languages combine to various degrees procedural and functional features for taking advantage of both. A hypothetical procedural–functional line: machine language, Assembly FORTRAN pure imperative languages C Python Lisp "pure" Lisp pure functional languages Functional languages: some industrial applications I Lisp: expert systems, Apple Macintosh, simulation, astronomy, embedded languages (AutoLisp), rapid prototyping I Erlang: Ericsson, T-Mobile, Facebook, EDF I OCaml: financial analysis, industrial robot programming, embedded software analysis I Haskell: aerospace systems, hardware design, Web programming A short history of Lisp Lisp is a functional, interpreted language invented by John McCarthy, one of the AI’s fathers I 1956 (Dartmouth Workshop) – 1958: key ideas – mathematical neatness in a practical programming language – computing with symbolic expressions rather than numbers – main data structure: list (e.g., symbolic expressions) hence the name: List processing – main list operations expressed as a few built-in functions – recursion (function composition) and conditional expression to form more complex functions – representing Lisp programs as Lisp data I 1958–1962: first implementations, application to AI problems I 1962–1965: spreading into a variety of computers and dialects I 1970s: main dialects: MacLisp, InterLisp, Scheme I 1970s: Lisp Machines, minicomputers designed to run Lisp I 1984: Common Lisp dialect becomes the ANSI standard Summary I Lisp programs, expressions, and main data types I Expressions: atoms (numbers and symbols) and lists (function calls, special expressions) Main built-in functions I – mathematics: +, -, *, /, sqrt, exp, cos, . . . – list processing: cons, first, rest, . . . – predicates: equal, atom, listp, null I Main special expressions: quote; and, or, not; cond I User-defined Lisp functions: the special expression defun Procedural features I – the special expression setq (binding symbols to values) – I/O functions: format, read – user-defined data structures: the special expressions defstruct and setf Lisp programs and expressions Program: set of recursive calls to expressions (functional style). Expression are evaluated by the interpreter, and return a value. Lisp expressions can be written I in the Listener window (the interactive intepreter): their value is immediately evaluated and printed to screen I in the editor window: they will be evaluated on user’s request Two kinds of expressions: I atoms: either numbers or symbols I lists: ordered sequences of (recursively) atoms and lists, enclosed in round brackets Expressions: atoms Main atomic expressions: I I numbers, which evaluate to themselves; e.g.: 12, -5, +12.67, 2e-5, 3/5 complex numbers: #C(1 -2) (denotes 1 ≠ 2i) symbols: any sequence of characters excluding brackets and spaces, that is not a number; e.g.: blob, a-symbol, +, 2d, b^2-4*a*c Only the following symbol expressions are legal: I predefined symbols, which evaluate to themselves, e.g.: t and nil, used to represent the Boolean values True and False, and (nil) an empty list I symbols previously bound to a value (e.g., using setq), which is returned as the result of their evaluation Expressions: lists A list is an ordered sequence of zero or more atoms or (nested) lists, enclosed in round brackets, and separated by spaces. Examples: I (1 2 3) I (a b) I (+ 1 2) I (1 xyz (w 4) (4 (t g (f)) qwerty)) I () (the empty list, equivalent to the atom nil) Expressions: lists Legal list expressions: I I the empty list () function calls: – the first element must be a symbol associated to the name of a built-in or user-defined function – the other elements (if any) must be expressions (either atoms or lists), whose values are passed to the function as arguments I special expressions: – each one has an ad hoc evaluation rule – the first element must be a symbol associated to the name of a built-in or user-defined special expression – every other element (if any) must be an expression (either an atom or a list); either the expression itself or its value is passed as the argument, depending on the special expression at hand Main built-in functions Mathematical functions: +, -, *, /, =, sqrt, cos, sin, exp, . . . Examples: I (+ 1 2) I (+ 1 2 3 4) I (* (- 2 3) 5) I (= 3 (+ 2 1)) I (sqrt -1) Main built-in functions List processing functions are the Lisp core. Three main functions for recursive list processing: I I I (cons <expression> <list>) returns a list obtained by adding the value of <expression> to the front of the value of the expression <list>, which must evaluate to a list; e.g.: (cons 1 nil) æ (1) (cons 1 (cons 2 nil)) æ (1 2) (first <list>) (also named car) returns the first element of a list, e.g.: (first (cons 1 (cons 2 nil)) æ 1 (rest <list>) (also named cdr) returns a list without its first element, e.g.: (rest (cons 1 (cons 2 nil)) æ (2) Any other operation on lists can be obtained by a suitable, recursive combination of cons, first and rest. Lisp data types Lisp data are represented as Lisp expressions: this is one of the main distinctive features from other languages. Every Lisp value is either an atom (number or symbol) or a list. Rationale: many AI applications involve abstract data structures which can be represented as symbols and lists of symbols, e.g.: I sentences in logical languages: (Prime Two), (GreatherThan Two One), (FatherOf John Mary) I symbolic mathematical expressions:s (integral (power x two)) for x 2 I trees, e.g.: (A (B ((D) (E))) (C)) A B D C E Atoms and lists: expressions or data? How to distinguish a Lisp expression (to be evaluated) from a Lisp data (not to be evaluated)? An example: how to construct the list (Prime Two)? (cons Prime (cons Two nil)) does not work: since cons is a function, Prime and Two are treated as expressions to be evaluated. The special expression quote The special expression quote allows one to distinguish Lisp expressions from Lisp data. (quote <expression>) returns the <expression> itself, not the result of its evaluation. An example: the list (Prime Two) can be constructed by I (cons (quote Prime) (cons (quote Two) nil)) I (quote (Prime Two)) Since quote is often used in Lisp programs, the shorthand notation ’<expression> has been introduced, e.g.: I ’x is equivalent to (quote x) I ’(Prime Two) is equivalent to (quote (Prime Two)) Main built-in functions I (equal <e1> <e2>) compares the values of two expressions I (atom <expr>) checks whether the value of <expr> is an atom (this is true also for the empty list ()) I (listp <expr>) checks whether the value of <expr> is a list (this is true also for the symbol nil) I (null <expr>) checks whether the value of <expr> is the empty list (this is true also for the symbol nil) I (second <expr>), (third <expr>), . . . : returns the second, third, . . . element of a list I (nth <n> <list>) returns the n-th element of a list (n = 0 denotes the first element) I (list <expr1> <expr2> ...) returns a list made up of the values of <expr1>, <expr2>, . . . I (length <list>) evaluates the length of a list I (append <list1> <list2> ...) concatenates lists Main built-in special expressions I (and <e1> <e2> ...) checks whether none of the arguments evaluates to nil; if so, it returns the value of the last expression, otherwise it returns nil I (or <e1> <e2> ...) checks whether at least one of the arguments does not evaluate to nil; if so, it returns the value of the first such expression, otherwise it returns nil I (not <expr>) checks whether the value of <expr> is different from nil The cond special expression This expression implements a set of nested conditional expressions: if <test1> then evaluate <expr1> else if <test2> then evaluate <expr2> else if <test3> then evaluate <expr3> ... Syntax: (cond (<test1> <expr1>) (<test2> <expr2>) (<test3> <expr3>) ... ) The cond special expression Semantics: I <test1>, <test2>, . . . are considered as conditional expressions: their logical value is false if they evaluate to nil, it is true otherwise I <expr1>, <expr2>, . . . are any Lisp expressions I the interpreter evaluates in sequence <test1>, <test2>, . . . , until it finds one, say <testk>, whose value is not nil; then the corresponding <exprk>is evaluated and its value is returned as the value of the whole cond expression I if all the conditions <test1>, <test2>, . . . evaluate to nil, the cond expression returns nil I the expression (symbol) t can be used as the last <test> expression, to implement a default condition The cond special expression Examples: I (cond ((> 2 1) t)) checks whether 2 is greater than 1 I (cond ((= 1 (- 2 1)) (list ’a ’b)) ((equal ’x (first ’(x y))) 0) (t ’z)) if (2 ≠ 1) = 1, return the list (a b); otherwise, if the first element of the list (x y) is the symbol x, return 0; otherwise return the symbol z Defining new Lisp functions Built-in functions and special expressions (including cond and quote) allow one to define new Lisp functions. This requires the special expression defun. Syntax: (defun <function-name> (<arg1> <arg2> ...) <expression>) (cont.) Defining new Lisp functions (cont.) I <function-name> must be a symbol (not a predefined one like t or nil) I <arg1>, <arg2>, . . . must be symbols that will be bound to the arguments of the function when it is called, i.e., the arguments become the values of the corresponding symbols I <expression> is the function body: it can be any Lisp expression, which can use the symbols <arg1>, <arg2>, . . . as expressions to be evaluated I the value returned by the function call is defined as the value of <expression> Note that in a pure functional language the function body is made up of a single expression: since no variables can be used, the value of any expression but the last one would be lost. Example In functional languages, functions are intrinsically recursive: the expression in their body can contain recursive calls to special expressions or other functions (including itself). When defining recursive functions (that call themselves), care must be taken to include a terminal condition to stop recursion. An example: the factorial function: (defun factorial (n) (cond ((= n 0) 1) (t (* n (factorial (- n 1)))))) Other Lisp functions I I Evaluating Lisp expressions: the built-in function eval. Examples: – (eval 1) æ 1 – (eval ’(+ 1 2)) æ 3 – after (setq x ’(+ 1 2)): (eval ’x) æ (+ 1 2), (eval x) æ 3 Applying functions to arguments using their symbol names (useful, e.g., to pass functions as arguments of other functions): the built-in functions apply and funcall. Examples: (defun my-function (f x y) (funcall f x y)) (my-function ’+ 1 2) æ 3 (my-function ’list 1 2) æ (1 2) Structure of Lisp programs Lisp programs are usually made up of several function definitions. A program is invoked by calling one of its functions (analogous to the main function of C programs). Procedural features in Lisp: binding values to symbols For the sake of efficiency, and to ease the definition of certain programs, procedural features have been introduced in Lisp. Variables are implemented in Lisp by binding symbols to values (note that this happens implicitly for function arguments). The special expression setq can be used to this aim. Syntax: (setq <symbol> <expression>) I <symbol> is the desired symbol (it is not evaluated), which is bound to the value of <expression> I subsequent evaluations of <symbol> produce the value of <expression> I setq returns the value of <expression> Procedural features in Lisp: binding values to symbols Examples: I (setq x 1) binds the symbol x to the value of the expression 1, which is again 1: from now on, the expression x will evaluate to 1 I (setq y ’(a b c)) binds the symbol y to the value of the expression ’(a b c), which is the list (a b c): from now on, the expression y will evaluate to (a b c) Procedural features in Lisp: local variables Local variables can be used in function definitions, by enclosing the function body inside the special expression let (it can be used only inside function definitions). This also allows the function body to be a sequence of expressions. Syntax: (defun <function-name> (<arg1> <arg2> ...) (let (<var1> <var2> ...) <expression1> <expression2> ... )) I <var1>, <var2>, . . . must be symbols: they will be bound to values (through setq), and can be used by <expression1>, . . . I there is no conflict with symbols having the same names outside the function (if any) I the value returned by let, and thus by the function, is the one of the last expression inside let Procedural features in Lisp: local variables An example: a function that checks if a list has zero, one or more arguments, and returns the symbols zero, one and many, respectively, by using length. Without using local variables, length should be called twice: (defun how-many-elements (l) (cond ((= (length l) 0) ’zero) ((= (length l) 1) ’one) (t ’many))) Using local variables, length can be called only once: (defun how-many-elements (l) (let (n) (setq n (length l)) (cond ((= n 0) ’zero) ((= n 1) ’one) (t ’many)))) Other Lisp features I Other built-in, atomic data types (self-evaluating symbols): – characters, e.g.: #\A – strings, e.g.: "Lisp" – ... I I User-defined data types: record-structures (analogous to struct’s in C language) made up of slots (fields), defined by the special expression defstruct (every instance is an atom) I/O functions (procedural features, produce side effects): – format (printing to screen) – read (reading from the keyboard)