Download How the electronic mind can emulate the human mind: some

How the electronic mind can
emulate the human mind: some
applications of Artificial
Intelligence
7th International Interdisciplinary Seminar
Luca Arcara, Federico Cassoli, Mattia Ferrini
Politecnico di Milano – Campus of Como
1
Agenda
Introduction
Expert Systems
Neural Networks
A sample application
Genetic Algorithms
2
Introduction
What is Artificial Intelligence?
Systems that think like humans
Systems that act like humans
Systems that think rationally
Systems that act rationally
3
Acting humanly – the Turing test
Described in “Computing machinery and
intelligence” – Turing (1950)
“Can machines think?” becomes “Can machines
behave intelligently?”
Operational test for intelligent behavior: the
Imitation Game.
4
Acting humanly – the Turing test
Suggested major components of AI




Knowledge
Reasoning
Language understanding
Learning
Problem: Turing test is not reproducible,
constructive or apt to mathematical
analysis
5
Subjects linked to A. I.
Philosophy
Logic, methods of reasoning
Mind as physical system
Foundations of learning,
language, rationality
Mathematics
Formal representation and proof
Algorithms, computation,
(un)decidability, (in)tractability
Probability
Psychology
Adaptation
Phenomena of perception and
motor control
Economics
Formal theory of rational
decisions
Linguistics
Knowledge representation
Grammar
Neuroscience
Plastic physical substrate for
mental activity
Control theory
Stability of systems
Simple optimal agent designs
6
State of the art
Artificial Intelligence
Expert systems
…
Neural networks
Genetic algorithms
7
Expert systems
Def: software systems simulating expert-like
decision making while keeping knowledge
separate from the reasoning mechanism.
Replace human expert decision making when
not available
Assist human expert when integrating various
decisions
Provides a user with:


an appropriate hypothesis;
methodology for knowledge storage and reuse.
8
Expert systems architecture
Expert’s
knowledge –
facts and rules
Inference engine
Uses the k.b. to
infer new facts
and produce
solutions
Update module
Explanation module
Simplifies the
user’s
interaction
User interface
Knowledge base
Lets the user
change the
k.b.
Tells the user the
steps that
produced the
solution
User
9
Rule-Based Systems
Knowledge in the form of if condition
then effect rules
Example:
here  fine
not here  absent
absent and not seen  at home
absent and seen  in the building
in the building  fine
at home and not holiday  sick
here and holiday  sick
?
? here  no
? seen  no
? holiday  no
sick
? here  yes
fine
? here  yes
? holiday  yes
sick
10
Expert systems vs. conventional programs
Aspect
Expert systems
Conventional programs
Paradigm
Heuristical rules, exploration of
the space of states
Algorithms, explicit predefined steps
Approach
Declarative
Procedural
Data
manipulated
Knowledge, often rules
Vectors and matrixes of
data
Control system
Inferential engine separated
from the knowledge base
Data and information
integrated with programs
User interface
Highly interactive, usually
questions and answers
No standard typology
Explanation
capability
It presents the steps that led to
the proposed solution
Not available
Learning
capability
Present
Not available
11
Applications
Interpretation
Diagnosis
Monitoring
Planning and scheduling
Forecasting
Project and configuration
12
Neural Networks
Def: mathematical models that try to emulate the
human nervous system.
Final target of neural networks is to simulate the
process of learning of the human brain, so that it
can interact with the external environment
without human help, except for the creation.
The first models were developed by W.
McCulloch and W. Pitts in 1943, with their
manifest: “A logical calculus of the ideas
immanent in nervous activity”.
13
Brain and Neurons
General Structure of a Neuron
Learning
dendrites
axon
nucleus
synapses
14
The structure of neural networks
Artificial neural networks are typically composed
of interconnected units which serve as a model
for neurons.
The synapse is modelled by a modifiable weight
associated with each particular connection.
Each unit converts the pattern of incoming
activities that it receives into a single outgoing
activity that it sends to other units.
 First: biased weighted sum
 Second: transfer function
15
How do they work?
1
0.19 + 0.88 + 0 - 0.8 = 0.27
x
0.19
0.88
x
0.4 + 0 - 0.73
= -0.33
1
x
1
0.4
0
0
-0.13
x
0
x
0
0
-0.13 + 0 - 0.82 = -0.95
0
Example of a two levels network
16
Learning
After a neural network has been created it can
be trained using one of the supervised learning
algorithms (an example is back propagation),
which uses the data to adjust the network's
weights and thresholds so as to minimize the
error in its predictions on the training set.
17
Why are they useful?
If the network is properly trained, it can model
the (unknown) function that relates the input
variables to the output variables, and can
subsequently be used to make predictions
where the output is not known.
They are based on the concept that often (not
always), it is possible to teach to a mathematical
system some laws that we did not know before,
only by letting it analyze a lot of real cases.
18
Applications
Common fields of application are when the
statistical analysis of all the problem’s variables
is too difficult or expensive (at the calculation
level), but overall is not clear beforehand what
kind of deterministic relationships there are
between the different variables.






OCR (Optical character recognition)
Diagnosis
Control of industrial productions quality
Recognition of potentially dangerous molecules (using
“electronic noses”)
Engine management
Control of robots
19
A sample application (1)
Problem:
We want to create a neural network that is able
to determine if one binary number with 4
figures is even.
20
A sample application (2)
We use a network with:



4 input nodes;
2 hidden nodes;
1 output node (1 if the number is even, 0 if it
is odd).
21
A sample application (3)
Training Data:
22
A sample application (4)
DEMO
To solve the problem we will use a program
that was produced at the Laboratory for
Computational Intelligence at the
University of British Columbia.
23
GAs Definition – The idea
Genetic algorithms are based on a biological
metaphor: they view learning as a competition among
a population of evolving candidate problem solutions.
So a GA is a probabilistic optimization algorithm that
makes use of a population of test solutions which
artificially reproduce through operations analogous to
gene transfer in sexual reproduction.
24
History
Genetic Algorithms (GAs) originated from the
studies conducted by John H. Holland and his
colleagues at the University of Michigan.
Holland’s book “Adaptation in Natural and
Artificial Systems”, published in 1975, is
generally acknowledged as the beginning of the
research of genetic algorithms.
25
Definitions(1) – Chromosome
A chromosome, a collection of genes, represents a
possible solution of the problem.
ENCODING EXAMPLES
Binary encoding – A chromosome is a collection of bits
Tree encoding - In the tree
encoding every chromosome is a tree
of some objects, such as functions or
commands in a programming
language ( i.e. LISP )
26
Definitions(2) – Fitness Function
Fitness Function
A fitness function evaluates the ability of
a candidate solution to solve the given
problem.
27
Definitions(3) – Crossover
Crossover operates on selected genes from parent
chromosomes and creates new offspring.
EXAMPLE – SINGLE POINT CROSSOVER WITH BINARY ENCODING
FATHER - 101100
101
OFFSPRING
MOTHER - 010111
111
28
Definitions(4) – Mutation
The chromosome is randomly mutated to prevent premature
convergence upon a local maximum.
It’s a further techique through wich a GA explores the
solution space: mutation gives an extra-probability to
every possible solution of the problem out of the finite
population of solutions generated by the GA.
Mutation should not occur very often, because then GA will in
fact change to random search.
29
Outline of a basic GA: Flowchart
30
Parameters(1)
Crossover probability ( Pc ): how often crossover will be
performed. If there are too few chromosomes, GAs have few
possibilities to perform crossover and only a small part of search
space is explored. On the other hand, if there are too many
chromosomes, GA slow down. Crossover probability is usually
beetween 0.4 and 0.7.
Mutation probability ( Pm ): how often parts of chromosome will
be mutated. Mutation should not occur very often, because then
GA will in fact change to random search. Tipical Pm is 0.010.001.
Population size: how many chromosomes are in a population (in
one generation). Good population size is reported to be about
20-100.
31
Parameters(2)
Elitism Number: Elitism is the name of the method that
first copies the best chromosome (or few best
chromosomes) to the new population. Elitism can rapidly
increase the performance of GA, because it prevents a
loss of the best found solution. Elitism number specifies
how many chromosomes copy directly in the new
population.
32
GAs vs. Ad-hoc approach
Genetic Approach is a rather brutal approach,
requiring large amounts of processing power, but
with the immense advantage of supplying
solutions to things we don't know how to solve,
or don't know how to solve quickly. In fact no
knowledge of how to solve the problem is
needed BUT you need to be able to encode the
chromosome and design the fitness function.
This means implementation relies on a problemindependent "engine”.
33
What are GAs used for ?
GAs are excellent for all tasks requiring optimization and are highly effective in any
situation where many inputs (variables) interact to produce a large number of
possible outputs (solutions).
Some example situations are:
Optimization such as data fitting, clustering, trend spotting, path finding, ordering.
Management: Distribution, scheduling, project management, courier routing,
container packing, task assignment, time tables.
Financial: Portfolio balancing, budgeting, forecasting, investment analysis and
payment scheduling.
Data Mining
……………….
34