Download Artificial Intelligence - Academic year 2016/2017

Artificial Intelligence
Academic year 2016/2017
Giorgio Fumera
http://pralab.diee.unica.it
[email protected]
Pattern Recognition and Applications Lab
Department of Electrical and Electronic Engineering
University of Cagliari
Outline
Part I: Historical notes
Part II: Solving Problems by Searching
Uninformed Search
Informed search
Part III: Knowledge-based Systems
Logical Languages
Expert systems
Part IV: The Lisp Language
Part V: Machine Learning
Decision Trees
Neural Networks
Part IV
The Lisp Language
Programming paradigms
Many high-level programming languages have been devised since
the 1950s, each one for a more or less specific purpose:
I
“number crunching”: FORTRAN (1957, first high-level
language)
I
artificial intelligence (symbol manipulation): Lisp (1958)
I
business: COBOL (1959)
I
system programming: C (1972)
I
data bases: SQL (early 1970s)
I
...
One of the main distinguishing features: programming paradigms
I
procedural (imperative)
I
declarative
Procedural languages
Main characteristics:
I
Von Neumann’s computing paradigm
I
program: a sequence of instructions expressing actions to
be executed
Examples:
I
machine language, Assembly
I
FORTRAN, Pascal, C, Python, etc.
I
object-oriented languages: SmallTalk, C++, Java, etc.
Declarative languages
Main characteristics:
I
higher abstraction level than procedural languages
I
programs: recursive expressions, representing what to
obtain, not how
Examples:
I
logical languages: SQL, Prolog
I
functional languages: Lisp, Erlang, Haskell, OCaml, Scheme
Characteristics of procedural languages
Computational model:
I
explicit manipulation of the content of memory cells
(variables)
I
control structures: conditional execution, iteration
Pros:
I
most programmers are used to procedural thinking
I
program structure is close to executor structure (hardware):
efficient translation and execution
Cons:
I
side effects: explicit alteration of memory content (machine
state)
I
understanding programs and verifying correctness is difficult
Drawbacks of procedural languages: examples
1. Are these C statements equivalent?
w = x + f(y);
w = f(y) + x;
Not necessarily: x can be a global variable modified by f.
This is an example of side effect: the outcome of a statement
depends on its context.
Drawbacks of procedural languages: examples
2. What is the operation carried out by the following C
procedure?
void mystery (int n) {
int i,j; boolean b;
for (i = 2; i <= n; i++) {
j = 2; b = true;
while (b == true && j <= i/2)
if (i % j != 0) j++;
else b = false;
if (b == true) printf (“%d ”, i);
}
}
Difficult to understand, due to nested loops and shared
variables.
Drawbacks of procedural languages: examples
3. The previous C procedure prints all prime numbers from 2 to
its argument. Is that true?
It is not easy to verify that a program behaves how it is
supposed (or claimed) to, for the same reasons above.
Characteristics of (ideal) functional languages
Program structure: recursive expressions (including function calls)
which are evaluated to produce a value
I
mathematics-like style
I
no variables
I
recursion takes the place of iteration
Pros:
I
I
I
absence of variables ∆ no side effects
referential transparency: the value of expressions is
context-independent (like mathematical expressions)
it is easier to understand and verify the correctness of programs
Cons:
I
thinking recursively may be non-intuitive
I
recursion requires higher processing cost (memory and time)
Functional-style programming: an example
A C function for computing the factorial of a natural number:
I
procedural style:
int factorial (int n) {
int i, fact = 1;
for (i=2; i <= n; i++) fact = fact * i;
return fact;
}
I
functional style (no explicit alteration of variables by
assignment, recursion used instead of iteration):
int factorial (int n) {
if (n == 0) return 1;
else return n * factorial (n - 1);
}
Characteristics of real programming languages
The only pure procedural language: machine language / Assembly.
All the other existing languages combine to various degrees
procedural and functional features for taking advantage of both.
A hypothetical procedural–functional line:
machine language,
Assembly
FORTRAN
pure imperative languages
C
Python
Lisp
"pure" Lisp
pure functional languages
Functional languages: some industrial applications
I
Lisp: expert systems, Apple Macintosh, simulation,
astronomy, embedded languages (AutoLisp), rapid prototyping
I
Erlang: Ericsson, T-Mobile, Facebook, EDF
I
OCaml: financial analysis, industrial robot programming,
embedded software analysis
I
Haskell: aerospace systems, hardware design, Web
programming
A short history of Lisp
Lisp is a functional, interpreted language invented by John McCarthy, one
of the AI’s fathers
I
1956 (Dartmouth Workshop) – 1958: key ideas
– mathematical neatness in a practical programming language
– computing with symbolic expressions rather than numbers
– main data structure: list (e.g., symbolic expressions)
hence the name: List processing
– main list operations expressed as a few built-in functions
– recursion (function composition) and conditional expression to
form more complex functions
– representing Lisp programs as Lisp data
I
1958–1962: first implementations, application to AI problems
I
1962–1965: spreading into a variety of computers and dialects
I
1970s: main dialects: MacLisp, InterLisp, Scheme
I
1970s: Lisp Machines, minicomputers designed to run Lisp
I
1984: Common Lisp dialect becomes the ANSI standard
Summary
I
Lisp programs, expressions, and main data types
I
Expressions: atoms (numbers and symbols) and lists (function
calls, special expressions)
Main built-in functions
I
– mathematics: +, -, *, /, sqrt, exp, cos, . . .
– list processing: cons, first, rest, . . .
– predicates: equal, atom, listp, null
I
Main special expressions: quote; and, or, not; cond
I
User-defined Lisp functions: the special expression defun
Procedural features
I
– the special expression setq (binding symbols to values)
– I/O functions: format, read
– user-defined data structures: the special expressions
defstruct and setf
Lisp programs and expressions
Program: set of recursive calls to expressions (functional style).
Expression are evaluated by the interpreter, and return a value.
Lisp expressions can be written
I
in the Listener window (the interactive intepreter): their value
is immediately evaluated and printed to screen
I
in the editor window: they will be evaluated on user’s request
Two kinds of expressions:
I
atoms: either numbers or symbols
I
lists: ordered sequences of (recursively) atoms and lists,
enclosed in round brackets
Expressions: atoms
Main atomic expressions:
I
I
numbers, which evaluate to themselves; e.g.:
12, -5, +12.67, 2e-5, 3/5
complex numbers: #C(1 -2) (denotes 1 ≠ 2i)
symbols: any sequence of characters excluding brackets and
spaces, that is not a number; e.g.:
blob, a-symbol, +, 2d, b^2-4*a*c
Only the following symbol expressions are legal:
I
predefined symbols, which evaluate to themselves, e.g.: t and
nil, used to represent the Boolean values True and False, and
(nil) an empty list
I
symbols previously bound to a value (e.g., using setq), which
is returned as the result of their evaluation
Expressions: lists
A list is an ordered sequence of zero or more atoms or (nested)
lists, enclosed in round brackets, and separated by spaces.
Examples:
I
(1 2 3)
I
(a b)
I
(+ 1 2)
I
(1 xyz (w 4) (4 (t g (f)) qwerty))
I
() (the empty list, equivalent to the atom nil)
Expressions: lists
Legal list expressions:
I
I
the empty list ()
function calls:
– the first element must be a symbol associated to the name of
a built-in or user-defined function
– the other elements (if any) must be expressions (either atoms
or lists), whose values are passed to the function as arguments
I
special expressions:
– each one has an ad hoc evaluation rule
– the first element must be a symbol associated to the name of
a built-in or user-defined special expression
– every other element (if any) must be an expression (either an
atom or a list); either the expression itself or its value is passed
as the argument, depending on the special expression at hand
Main built-in functions
Mathematical functions: +, -, *, /, =, sqrt, cos, sin, exp, . . .
Examples:
I
(+ 1 2)
I
(+ 1 2 3 4)
I
(* (- 2 3) 5)
I
(= 3 (+ 2 1))
I
(sqrt -1)
Main built-in functions
List processing functions are the Lisp core.
Three main functions for recursive list processing:
I
I
I
(cons <expression> <list>)
returns a list obtained by adding the value of <expression>
to the front of the value of the expression <list>, which
must evaluate to a list; e.g.:
(cons 1 nil) æ (1)
(cons 1 (cons 2 nil)) æ (1 2)
(first <list>) (also named car)
returns the first element of a list, e.g.:
(first (cons 1 (cons 2 nil)) æ 1
(rest <list>) (also named cdr)
returns a list without its first element, e.g.:
(rest (cons 1 (cons 2 nil)) æ (2)
Any other operation on lists can be obtained by a suitable,
recursive combination of cons, first and rest.
Lisp data types
Lisp data are represented as Lisp expressions: this is one of the
main distinctive features from other languages.
Every Lisp value is either an atom (number or symbol) or a list.
Rationale: many AI applications involve abstract data structures
which can be represented as symbols and lists of symbols, e.g.:
I
sentences in logical languages: (Prime Two),
(GreatherThan Two One), (FatherOf John Mary)
I
symbolic mathematical expressions:s
(integral (power x two)) for x 2
I
trees, e.g.: (A (B ((D) (E))) (C))
A
B
D
C
E
Atoms and lists: expressions or data?
How to distinguish a Lisp expression (to be evaluated) from a Lisp
data (not to be evaluated)?
An example: how to construct the list (Prime Two)?
(cons Prime (cons Two nil)) does not work: since cons is a
function, Prime and Two are treated as expressions to be evaluated.
The special expression quote
The special expression quote allows one to distinguish Lisp
expressions from Lisp data.
(quote <expression>) returns the <expression> itself, not the
result of its evaluation.
An example: the list (Prime Two) can be constructed by
I
(cons (quote Prime) (cons (quote Two) nil))
I
(quote (Prime Two))
Since quote is often used in Lisp programs, the shorthand notation
’<expression> has been introduced, e.g.:
I
’x is equivalent to (quote x)
I
’(Prime Two) is equivalent to (quote (Prime Two))
Main built-in functions
I
(equal <e1> <e2>) compares the values of two expressions
I
(atom <expr>) checks whether the value of <expr> is an atom
(this is true also for the empty list ())
I
(listp <expr>) checks whether the value of <expr> is a list (this
is true also for the symbol nil)
I
(null <expr>) checks whether the value of <expr> is the empty
list (this is true also for the symbol nil)
I
(second <expr>), (third <expr>), . . . : returns the second,
third, . . . element of a list
I
(nth <n> <list>) returns the n-th element of a list (n = 0
denotes the first element)
I
(list <expr1> <expr2> ...) returns a list made up of the
values of <expr1>, <expr2>, . . .
I
(length <list>) evaluates the length of a list
I
(append <list1> <list2> ...) concatenates lists
Main built-in special expressions
I
(and <e1> <e2> ...) checks whether none of the
arguments evaluates to nil; if so, it returns the value of the
last expression, otherwise it returns nil
I
(or <e1> <e2> ...) checks whether at least one of the
arguments does not evaluate to nil; if so, it returns the value
of the first such expression, otherwise it returns nil
I
(not <expr>) checks whether the value of <expr> is
different from nil
The cond special expression
This expression implements a set of nested conditional expressions:
if <test1> then evaluate <expr1>
else if <test2> then evaluate <expr2>
else if <test3> then evaluate <expr3>
...
Syntax:
(cond (<test1> <expr1>)
(<test2> <expr2>)
(<test3> <expr3>)
... )
The cond special expression
Semantics:
I
<test1>, <test2>, . . . are considered as conditional
expressions: their logical value is false if they evaluate to nil,
it is true otherwise
I
<expr1>, <expr2>, . . . are any Lisp expressions
I
the interpreter evaluates in sequence <test1>, <test2>, . . . ,
until it finds one, say <testk>, whose value is not nil; then
the corresponding <exprk>is evaluated and its value is
returned as the value of the whole cond expression
I
if all the conditions <test1>, <test2>, . . . evaluate to nil,
the cond expression returns nil
I
the expression (symbol) t can be used as the last <test>
expression, to implement a default condition
The cond special expression
Examples:
I
(cond ((> 2 1) t))
checks whether 2 is greater than 1
I
(cond ((= 1 (- 2 1)) (list ’a ’b))
((equal ’x (first ’(x y))) 0)
(t ’z))
if (2 ≠ 1) = 1, return the list (a b); otherwise, if the first
element of the list (x y) is the symbol x, return 0; otherwise
return the symbol z
Defining new Lisp functions
Built-in functions and special expressions (including cond and
quote) allow one to define new Lisp functions.
This requires the special expression defun.
Syntax:
(defun <function-name> (<arg1> <arg2> ...)
<expression>)
(cont.)
Defining new Lisp functions
(cont.)
I
<function-name> must be a symbol (not a predefined one
like t or nil)
I
<arg1>, <arg2>, . . . must be symbols that will be bound to
the arguments of the function when it is called, i.e., the
arguments become the values of the corresponding symbols
I
<expression> is the function body: it can be any Lisp
expression, which can use the symbols <arg1>, <arg2>, . . . as
expressions to be evaluated
I
the value returned by the function call is defined as the value
of <expression>
Note that in a pure functional language the function body is made
up of a single expression: since no variables can be used, the value
of any expression but the last one would be lost.
Example
In functional languages, functions are intrinsically recursive: the
expression in their body can contain recursive calls to special
expressions or other functions (including itself).
When defining recursive functions (that call themselves), care must
be taken to include a terminal condition to stop recursion.
An example: the factorial function:
(defun factorial (n)
(cond ((= n 0) 1)
(t (* n (factorial (- n 1))))))
Other Lisp functions
I
I
Evaluating Lisp expressions: the built-in function eval.
Examples:
– (eval 1) æ 1
– (eval ’(+ 1 2)) æ 3
– after (setq x ’(+ 1 2)):
(eval ’x) æ (+ 1 2), (eval x) æ 3
Applying functions to arguments using their symbol names
(useful, e.g., to pass functions as arguments of other
functions): the built-in functions apply and funcall.
Examples:
(defun my-function (f x y) (funcall f x y))
(my-function ’+ 1 2) æ 3
(my-function ’list 1 2) æ (1 2)
Structure of Lisp programs
Lisp programs are usually made up of several function definitions.
A program is invoked by calling one of its functions (analogous to
the main function of C programs).
Procedural features in Lisp: binding values to symbols
For the sake of efficiency, and to ease the definition of certain
programs, procedural features have been introduced in Lisp.
Variables are implemented in Lisp by binding symbols to values
(note that this happens implicitly for function arguments).
The special expression setq can be used to this aim.
Syntax: (setq <symbol> <expression>)
I
<symbol> is the desired symbol (it is not evaluated), which is
bound to the value of <expression>
I
subsequent evaluations of <symbol> produce the value of
<expression>
I
setq returns the value of <expression>
Procedural features in Lisp: binding values to symbols
Examples:
I
(setq x 1)
binds the symbol x to the value of the expression 1, which is
again 1: from now on, the expression x will evaluate to 1
I
(setq y ’(a b c))
binds the symbol y to the value of the expression ’(a b c),
which is the list (a b c): from now on, the expression y will
evaluate to (a b c)
Procedural features in Lisp: local variables
Local variables can be used in function definitions, by enclosing the
function body inside the special expression let (it can be used only
inside function definitions).
This also allows the function body to be a sequence of expressions.
Syntax:
(defun <function-name> (<arg1> <arg2> ...)
(let (<var1> <var2> ...)
<expression1>
<expression2>
... ))
I
<var1>, <var2>, . . . must be symbols: they will be bound to values
(through setq), and can be used by <expression1>, . . .
I
there is no conflict with symbols having the same names outside
the function (if any)
I
the value returned by let, and thus by the function, is the one of
the last expression inside let
Procedural features in Lisp: local variables
An example: a function that checks if a list has zero, one or more
arguments, and returns the symbols zero, one and many, respectively, by
using length.
Without using local variables, length should be called twice:
(defun how-many-elements (l)
(cond ((= (length l) 0) ’zero)
((= (length l) 1) ’one)
(t ’many)))
Using local variables, length can be called only once:
(defun how-many-elements (l)
(let (n)
(setq n (length l))
(cond ((= n 0) ’zero)
((= n 1) ’one)
(t ’many))))
Other Lisp features
I
Other built-in, atomic data types (self-evaluating symbols):
– characters, e.g.: #\A
– strings, e.g.: "Lisp"
– ...
I
I
User-defined data types: record-structures (analogous to
struct’s in C language) made up of slots (fields), defined by
the special expression defstruct (every instance is an atom)
I/O functions (procedural features, produce side effects):
– format (printing to screen)
– read (reading from the keyboard)