Download Artificial Intelligence - Academic year 2016/2017

Artificial Intelligence
Academic year 2016/2017
Giorgio Fumera
http://pralab.diee.unica.it
[email protected]
Pattern Recognition and Applications Lab
Department of Electrical and Electronic Engineering
University of Cagliari
Outline
Part I: Historical notes
Part II: Solving Problems by Searching
Uninformed Search
Informed search
Part III: Knowledge-based Systems
Logical Languages
Expert systems
Part IV: The Lisp Language
Part V: Machine Learning
Decision Trees
Neural Networks
Part I
Historical Notes
Artificial Intelligence
A discipline born in the 1950s.
Goal: building “intelligent” machines.
A goal envisaged since a long time before, e.g.:
I
Leonardo da Vinci’s robot
knight (about 1495)
I
Automaton chess player
(“Mechanical Turk”, late
18th century)
I
science fiction
What is “intelligence”?
Broad definition: a set of capabilities that allow humans to
I
learn
I
think
I
understand
I
communicate
I
be self-conscious
I
build abstract models of the world
I
plan
I
adapt to novel external conditions
I
...
Some of these capabilities are exhibited also by animals (e.g.,
associative memory, reacting to stimuli, communicating).
Early investigations: clues from humans
Goal: understanding human intelligence.
I
High-level manifestation: rationality
Logic: Aristotle (4th cent. bc), G.W. Leibniz (17th-18th
cent.), G. Boole (19th cent.), etc.
I
High-level manifestation: behavior and mind
Psychology and cognitive science (since 19th cent.)
I
Low-level biological support: the brain
Neuroanatomy and Neurophysiology (since 19th cent.):
McCulloch and Pitt’s model of neuron (1943), D.O. Hebb’s
theory on neurons as basic units of thought, etc.
Early investigations: clues from technology
Goal: building machines capable of autonomously performing some
task.
I
Automata (e.g., jacquard loom, 1804)
Cybernetics (feedback and control):
N. Wiener, W.R. Ashby (1940s)
I
Statistics and probability as tools to deal
with uncertainty in reasoning and
decision-making: T. Bayes (18th cent.);
K. Pearson, R.A. Fisher, A. Wald,
J. Neyman (late 19th – 20th cent.)
Early investigations: clues from technology
The computer:
I
precursors: mechanical devices: B. Pascal and G.W. Leibniz (17th
cent.), C. Babbage’s analytical engine (19th cent.)
I
from engineering: electromechanical and electronic devices
(1940s): K. Zuse, J.P. Eckert, J.W. Mauchly, J. von Neumann
I
from logic and mathematics: computational theory, the foundation
of computer science: A.M. Turing, A. Church (1930s)
Early investigations: “thinking” computers
Electronic computers: the right tool for building intelligent
machines?
Alan M. Turing’s (1912-1954) main contributions:
I
first investigations into the nature of
computing
I
the “logical computing machine” (Turing
machine): a universal computer
I
envisioning intelligent computers:
Computing Machinery and Intelligence,
Mind, Vol. LIX, No. 263, 433–460, 1950;
operational definition of intelligence: the
Turing Test
The birth of AI
1950s: many researchers from different disciplines investigate
intelligence under two main viewpoints:
I
understanding human intelligence
I
building intelligent machines
Dartmouth College Workshop (USA, summer 1956): official birth
of AI, as a discipline aimed at building intelligent machines.
The founders: J. McCarthy, M. Minsky, A. Newell, H. Simon,
C. Shannon, O. Selfridge, R. Solomonoff, . . .
Early explorations: 1950s and 1960s
Goals: identifying specific tasks that require intelligence, and
figuring out how to get machines to do them.
Great interest in mimicking high-level human thought and mental
abilities, e.g.:
I
reasoning
I
understanding natural language
I
understanding images
and some related low-level abilities:
I
recognizing speech sounds
I
distinguishing objects in images
I
reading cursive script
But: how do humans do that?
Early explorations: 1950s and 1960s
Starting point: toy problems, and some real-world ones
I
game playing 15-puzzle, checkers, chess, etc.
I
theorem proving
I
natural language processing (NLP)
I
recognizing objects in images
Early explorations: 1950s and 1960s
First approaches and tools:
I symbolic (high-level) approach: based on the assumption
that the essence of intelligence is symbol processing
–
–
–
–
I
heuristic search
syntax analysis/generation
symbolic knowledge representation (symbols, lists)
symbolic knowledge processing: new programming languages
(LISP, etc.)
non-symbolic (low-level) approach
– pattern recognition
– artificial neural networks
Heuristic search methods
Symbol processing approach, applied to:
I
game playing: 15-puzzle, checkers, chess (the “Drosophila of AI”)
I
geometric analogy problems
I
theorem proving
I
mechanizing problem solving: A. Newell and H. Simon’s General
Problem Solver (1959)
Heuristic search methods
Common approach:
I
knowledge representation: lists of symbols
(main feature of the LISP language, 1958–, J. McCarthy)
I
search methods: search tree, heuristics
Natural language processing
Symbol processing approach.
Goals: understanding, generating, translating natural language.
Linguistic levels:
I
morphology: word parts (e.g.: walking = walk + -ing)
I
syntax (grammar): rules that define well-formed sentences
(e.g.: John hit the ball: Yes; ball the hit John: No)
I
semantics: meaning of a sentence
I
pragmatics: context and background knowledge, e.g.:
John went to the bank
John threw the ball to the window and broke it
John threw the glass to the wall and broke it
Natural language processing
Main focus of early research: syntactic level.
Seminal work: N. Chomsky, Syntactic Structures, 1957.
Grammar definition: syntax rules for analyzing/generating
sentences; main tool: parse tree.
Applications:
I
question answering (original goal: computer interfaces)
I
machine translation: early optimism, but a very hard task
Pattern recognition
Goal: classifying input signals (images, sounds, electronic signals,
etc.) into one of several categories.
First problems: image classification.
Early efforts: optical character recognition (OCR).
Main approaches:
I
I
template matching
learning
1. image pre-processing (noise filtering, line thickening, edge
enhancement, ...)
2. feature extraction
3. classification “rules” learnt from examples
Artificial neural networks
Non-symbolic (low-level), connectionist approach.
F. Rosenblatt (psychologist): perceptron (1957–) as a potential
model of human learning, cognition and memory:
I
network of McCulloch-Pitts’ neural elements
I
learning algorithm for adjusting connection weights from
examples
First applications: pattern recognition
I
OCR
I
aerial image recognition
Great expansion: mid–1960s to early 1980s
From toy/lab problems to real-world and commercial applications:
I
computer vision
I
mobile robots
I
game playing
I
speech recognition, NLP
I
knowledge representation and reasoning
Research funding:
I
DARPA’s Strategic Computing Program (USA)
I
Fifth Generation Computer Systems (Japan)
I
ESPRIT (Europe)
Great expansion: mid–1960s to early 1980s
Computer vision:
I
Summer Vision project (MIT, 1966)
I
low-level, hierarchical image processing (hints from biology); image
filters; line, corner, surface detection; 3D reconstruction
I
early application: guiding a robot arm to manipulate blocks
I
high-level vision: finding objects in scenes (templates, parts)
I
two main approaches emerge:
– whole scene reconstruction: difficult
– perceiving to guide robot action (purposive vision): easier
Great expansion: mid–1960s to early 1980s
Mobile robots:
I
sensors, actuators, computer vision, environment modeling,
planning
I
route finding: heuristic search, A* algorithm
I
first autonomous vehicles
Great expansion: mid–1960s to early 1980s
Game playing: progress in chess
I
programs that attain human-level (not master) capability
I
investigations into human ability: accumulated knowledge vs
massive search
Great expansion: mid–1960s to early 1980s
NLP: greater success on less ambitious goals
I
improvements in grammars
I
machine translation with humans in the loop
I
declarative (logical languages, inference algorithms) vs procedural
(“hard-wired”) knowledge
I
dialog systems (1971, T. Winograd’s SHRDLU: blocks’ world,
procedural knowledge)
I
speech recognition (easy), and understanding (difficult)
Great expansion: mid–1960s to early 1980s
Knowledge representation and reasoning:
I
consulting / decision support / expert systems
main idea: solving domain-specific problems by embedding
expert knowledge in the form of IF-THEN rules
I
applications: chemistry, medical diagnosis, geology, military;
since 1990s: business
Great expansion: mid–1960s to early 1980s
Summing up, until the 70s AI research is mainly based on the
symbol processing conception of human intelligence
I
main approach to AI: mimicking high-level human abilities
through heuristic search and symbolic processing (“good,
old-fashioned AI”, GOFAI)
I
many successful applications through a pragmatic approach in
specific tasks . . .
I
. . . but very limited achievements with respect to early
expectations for a “general” AI
Mid 1980s: the “AI winter”
Real-world tasks require much more “intelligence” than that
achievable by heuristic search and symbolic processing (GOFAI).
Two main issues emerge:
I
computational complexity: combinatorial explosion
I
need of a large body of knowledge (including common sense)
Also the non-symbolic, connectionist approach (artificial neural
networks) exhibits limitations.
Main consequences:
I
drop of interest in AI
I
scaling back AI’s goals
I
cut of research funds
Mid–1980s to 1990s: technical and theoretical advances
New results in several fields, solid theoretical foundations from:
I
mathematics
I
statistics and probability theory
I
control engineering
Concrete progress in real-world tasks (albeit still far from initial
expectations).
Mid–1980s to 1990s: technical and theoretical advances
Knowledge representation and reasoning:
I
I
new paradigms, e.g.; fuzzy logic, soft computing (inspired by
human mind)
semantic networks, ontologies; e.g.:
– WordNet, http://wordnet.princeton.edu
– BabelNet, http://babelnet.org
I
probabilistic reasoning to overcome the limits of logic
(probabilistic graphical models, Bayesian networks, learning)
Search algorithms: evolutionary approach, genetic algorithms
(inspired by evolution).
Mid–1980s to 1990s: technical and theoretical advances
The rise of machine learning:
I
huge amount of data in digital form become available
I
main idea: automatically inferring knowledge (patterns, rules,
etc.) from data instead of eliciting it from domain experts
I
data analysis methods: data mining, etc.
I
theoretical foundations: statistics
I
novel techniques: inductive logic programming, decision trees,
resurgence of ANNs (1986: back-propagation algorithm),
support vector machines, ensemble methods, etc.
I
applications to many fields: computer vision, natural language
processing, etc.
Mid–1980s to 1990s: technical and theoretical advances
Computer vision:
I
two main approaches persist: scene analysis, purposive vision
I
main achievements: surface, depth; tracking, object
recognition
I
fruitful exchanges with research on animal/human vision
I
novel techniques: hierarchical models, ANNs, deep neural
networks
I
extensive application of machine learning techniques
Mid–1980s to 1990s: technical and theoretical advances
Intelligent Agent architectures:
I
sensor networks
I
autonomous, cooperating robots; emergent behavior
I
the intelligent agent paradigm
A toy (?) example: soccer-playing robots
Mid 1990s – today: main achievements
Original goal (building intelligent machines): still far-reaching.
But: successes in real-world problems, many commercial
applications, many startup companies.
Examples:
I
games: master-level programs in checkers, chess and (very
recently) Go
I
robots: driverless automobiles, space vehicles
I
pervasive applications: home, cars, route finding in maps
(search algorithms), recommender systems (machine learning,
social/collaborative filtering), characters in video games, ...
I
automated (high-frequency) trading
I
medicine
I
business rule management systems
I
translating systems
I
computer vision (face recognition, ...)
Summary of the main approaches to AI
Four distinct approaches to build intelligent machines:
Mind/
thinking
Behavior
Human performance
Systems that think
like humans
(cognitive modeling approach)
Systems that act
like humans
(Turing test approach)
Rationality
Systems that think
rationally
(“law of thought” approach)
Systems that act
rationally
(rational agent approach)
Rational agents (acting rationally): the most general approach,
amenable to scientific/technological development (though, maybe,
not useful enough for understanding human intelligence)
A snapshot of current AI research
Associations:
I
Association for the Advancement of Artificial Intelligence (AAAI)
www.aaai.org
I
European Coordinating Committee for Artificial Intelligence
(ECCAI), www.eccai.org
I
Italian Association for Artificial Intelligence (AI*IA)
www.aixia.it
Conferences:
I
Int. Joint Conf. on Artificial Intelligence, ijcai.org
Scientific journals:
I
Artificial Intelligence
www.journals.elsevier.com/artificial-intelligence
I
J. of Artificial Intelligence Research, www.jair.org
A snapshot of current AI research
Research topics:
I
Machine learning
I
Knowledge representation and reasoning
I
Reasoning under uncertainty or imprecision
I
Natural language
I
Vision
I
Pattern recognition
I
Heuristic search
I
Intelligent robotics
I
Multi-agent systems
I
Planning
I
...
Philosophical issues
A long-standing question: Can machines be intelligent?
Two main hypotheses:
I
Weak AI: machines can emulate intelligence (act intelligently)
I
Strong AI: machines can be intelligent
(if they act intelligently, they are intelligent, e.g.: Turing test)
Another long-standing question: Is (human) mind a machine?
Philosophical issues
Some arguments raised against Weak AI:
I
machines can never do . . . (make mistakes, learn from
experience, have a sense of humor, enjoy ice creams, . . . )
I
machines are formal systems, and formal systems cannot
establish the truth of every mathematical sentence (Gödel
incompleteness theorem), whereas humans (in principle) can
I
human behavior can not be captured by a set of rules
A. Turing’s viewpoint (Mind, 1950):
Can a machine think? is an ill-posed question.
Consider this one: Can a machine fly/swim?
I
airplanes “fly”, but not as birds “fly”
I
ships “swim” in Russian, not in English or in Italian...
Philosophical issues
Some issues about Strong AI:
I
even if machines can emulate intelligence, they cannot be
self-conscious
I
relationship between mental states and body (brain) states
(free will, consciousness, intentions): dualism (R. Descartes,
17th cent.) vs materialism (“brains cause mind”)
I
a machine running the “right” program (e.g., for natural
language understanding) does not necessarily have a mind
(the Chinese room thought experiment, J. Searle, 1980)
I
intelligence is an emerging behavior that can be only
supported by biological brains
Ethical issues
I
Even if we could build intelligent machines, should we?
I
Consequences on humans: loss of jobs, loss of the sense of
being unique, end of human race, etc.
I
Accountability (e.g., driverless cars)
I
...
Ethical concerns are now re-flourishing, as many believe that
human-level AI is now in reach (e.g., current focus of the Future of
Life Institute, http://futureoflife.org).
Some recent projects
Human Brain Project (EU)
https://www.humanbrainproject.eu
Overall goal: understanding the human brain and its
diseases, and emulating its computational capabilities
RoboLaw (EU)
http://www.robolaw.eu
Regulating Emerging Robotic Technologies in Europe:
Robotics facing Law and Ethics