Download CS 561: Artificial Intelligence CS 561: Artificial Intelligence

CS 561: Artificial Intelligence
„
„
„
Instructor: Prof.
Prof Hadi Moradi,
Moradi
[email protected]
Lectures: M-Th 09:00-10:40, OHE136
Office hours: MW 2:30 – 4:00 pm,
SAL310,
Or by
O
b appointment
i
TAs: Jeong-Yoon Lee
„
„
„
„
„
SAL 112
Office hours: TTH 1:00-2:30
Email: [email protected]
CS 561: Artificial Intelligence
„
Course web page:
„
„
„
„
„
„
http://www-scf.usc.edu/~csci561a
Up to date information, lecture notes
Relevant dates, links, etc.
Also you may check http://den.usc.edu
Class format: two sections of 45 minutes
Course material:
„
[AIMA] Artificial Intelligence: A Modern
Approach, by Stuart Russell and Peter Norvig.
2nd edition
1
CS 561: Artificial Intelligence
„
„
Course overview: foundations of symbolic
intelligent systems. Agents, search, problem
solving, logic, representation, reasoning,
symbolic programming, probabilistic
reasoning, and robotics.
Prerequisites: CS 455x, i.e.,
„
programming principles, discrete mathematics
for computing, software design and software
engineering concepts. Some knowledge of
C/C++ for some programming assignments.
CS 561: Artificial Intelligence
„
„
„
„
„
Grading:
25% for midterm
25% for final
40% for homeworks and projects
10% for
f Quizzes
Q i
2
Practical issues
„
Class list: use learn.usc.edu
learn usc edu
„
„
„
Login with your USC username and
password
If CSCI561A is not listed as your courses,
notify
ot y the
t e TA.
Submissions: See class web page under
Assignments
submit -user csci561 -tag HW3
HW3.tar.gz
Administrative Issues
„
Midterm 1: 7/26/10 9:00 - 10:40pm
„
Midterm 2: 8/10/10 9:00 - 10:40pm
See also the class web page:
http://den usc edu/
http://den.usc.edu/
3
Why study AI?
Search engines
Science
Medicine/
Diagnosis
Labor
Appliances
What else?
Humanoid Robots: From
Honda to Sony
Walk
Turn
http://world.honda.com/robot/
Stairs
4
Sony AIBO
movie1
http://www.aibo.com
Natural Language Question Answering
http://aimovie.warnerbros.com
http://www.ai.mit.edu/projects/infolab/
5
Robot Teams
USC robotics Lab
Modular robots self re-assembly.
What is AI?
The exciting new effort to make “The
The study of mental faculties
computers thinks … machine
through the use of computational
with minds, in the full and
models” (Charniak et al. 1985)
literal sense” (Haugeland 1985)
“The art of creating machines
that perform functions that
require intelligence when
performed by people”
(Kurzweil, 1990)
A field of study that seeks to
explain and emulate intelligent
behavior in terms of
computational processes”
(Schalkol, 1990)
6
AI – The Bigger Picture
?
Computer Science
Philosophy
p y
Artificial Intelligence
Cognitive Science
(Psychology)
Robotics
(Engineering)
Neuroscience
(Biology)
?
Acting Humanly: The Turing
Test
„
Alan Turing
Turing'ss 1950 article Computing Machinery
and Intelligence discussed conditions for
considering a machine to be intelligent
7
Acting Humanly: The Turing
Test
What tasks require AI?
„
„
“AI
AI is the science and engineering of
making intelligent machines which can
perform tasks that require intelligence
when performed by humans …”
What tasks require AI?
8
What tasks require AI?
„
Tasks that require
q
AI:
„
„
„
„
„
„
„
„
„
„
Solving a differential equation
Brain surgery
Inventing stuff
Playing Jeopardy
Playing Wheel of Fortune
What about walking?
What about grabbing stuff?
What about pulling your hand away from fire?
What about watching TV?
What about day dreaming?
Acting Humanly: The Full
Turing Test
• Problem:
9
What would a computer need
to pass the Turing test?
„
„
„
„
Communication:
Memory:
Reasoning:
Learning:
What would a computer need
to pass the Turing test?
„
Sensing:
„
M t control
Motor
t l (total
(t t l ttest):
t)
10
Thinking Humanly: Cognitive
Science
„
1960 “Cognitive
Cognitive Revolution
Revolution”::
information-processing psychology
replaced behaviorism
Thinking Humanly: Cognitive
Science
„
Cognitive science and modeling the activities
of the brain
„
„
What level of abstraction? “Knowledge” or
“Circuits”?
How to validate models?
11
Thinking Rationally: Laws of
Thought
„
Aristotle (~
( 450 B.C.) attempted to codify
“right thinking”
„
What are correct arguments/thought
processes?
Thinking Rationally: Laws of
Thought
„
Problems:
12
Acting Rationally: The Rational
Agent
„
„
Rational behavior: Doing the right thing!
Provides the most general view of AI
because it includes:
Acting Rationally: The Rational
Agent
„
Advantages:
13
How to achieve AI?
„
How is AI research done?
„
„
Theoretical
Experimental
How to achieve AI?
„
There are two main lines of research:
„
„
„
Biological, study humans and imitate their
psychology or physiology.
phenomenal, study and formalize common sense
facts about the world and the problems that the
world presents to the achievement of goals.
The two approaches interact to some extent,
and both should eventually succeed. It is a
race, but both racers seem to be walking.
[John McCarthy]
14
Branches of AI
„
„
„
„
„
„
„
„
„
Logical AI
Search
Natural language processing
pattern recognition
Knowledge representation
Inference From some facts,, others can be inferred.
Automated reasoning
Learning from experience
Planning To generate a strategy for achieving some
goal
AI Prehistory
15
Brief History of AI
Thinking Rattionally:Laws of T
Thought
Ancient Times
M iddle Age
384 B.C.
1200
- Aristotle
- Logic: The science of knowing.
Ramon Lull
Ars Magnus: a rule-based device to
model man's behavior and nature
- Empiricism
Explanation of processes
- Gottfried Leibniz
- 1st system of formal logic
-
Renaissance
17 th Century
18 th Century
19 th Century
Next time
implement links
Rene Descartes
Dualism
1845
- Charles Babbage
- Analytical Engine
- George Boole
- Formalization of the Laws of Logic
-
1879-1903
Early 20th
Century
1910-1912
-
Gottlob Frege
First-order predicate calculus
Russel-Whitehead
Principia Mathematica
Bertrand Russel
1931
- Kurt Godel
- Incompleteness Theorem of Logic
-
Roots of AI in Science:
„
„
„
„
„
„
„
Aristotle(b.384-): syllogism – formal reasoning
Ramon Lull (b.1235): Ars Magna – a machine capable of
answering all questions
Rene Descartes (1596): mind / body separation (dualism);
"cogito ergo sum“
Wilhelm Liebniz (1646-1716): a mechanical concept
generator;; "materialism"
g
Charles Babbage(1792-1871), Ada Lovelace (1815-1860):
Analytical Engine – a general-purpose calculator
George Boole(1815-1864): logic algebras - logical encoding
and calculation of thoughts
Gottlob Frege(1848-1925): predicate calculus
16
Birth of Artificial Intelligence
1940-1956
1942
1943
1945
Greaat Expectations
1949
-
ENIAC :First digital computer
Mc Culloch and Pitts
Artificial neural network
J. Von Neumman
Modern computer architecture
Claude Shannon
Use of heuristics to solve complex
problems
1950
- Alan M.Turing
- Computing Machinery and
- Intelligence:
Turing Test
1955
- Herbert Simon,Alan Newell
- 1st AI program:Logic Theorist
-
1956
- Dartmouth Conference
-
Herbert Simon
The Beginning of AI
„
„
„
„
McCulloch & Pitts
„ developed theory of artificial neurons (precursor to
ANN's) – 1943
Alan Turing – "Can Machines Think?"
„ the turing test (1950)
„ the turing machine
Marvin Minsky & Dean Edmonds
„ first ANN constructed, 1951
John McCarthy
„ convened the Dartmouth conference that coined the
term artificial intelligence (AI) (1956) and set the
research agenda
„ symbolic AI
„ connectionism
st AI language
„ LISP (list processing) 1958 1
17
The Rise of AI
1957- 1960’s
1958
1960
1961
Growingg Disenchantment
1962
1965
- John McCarthy.
- LISP
-
Marvin Minsky
Theory of Frames
Herbert Simon,Alan Newell
GPS:General Problem Solver
Herbert Simon
Frank Rosenblatt
Perceptron:
Learning in Neural Networks
- L
Lotfi
tfi A.
A Zadeh
Zd h
Fuzyy Logic
Fuzzy Sets
-
1968
Joseph Weizenbaum
ELIZA: simulates diagnosis by a
psychiatrist.
1969
- Marvin Minsky,Seymour Papert
- Limitations of Perceptrons
S. Papert
An Optimistic Start
„
In the 50's, 60's and early 70's, much exciting
progress was being made in AI:
Chess
„
„
The Logic Theorist
„
„
Feigenbaum, Buchanan, Lederberg, 1969
SHRDLU – NLP (Blocks World)
„
„
Joseph Weizenbaum, 1966
DENDRAL – Knowledge-Based System
„
„
Arthur Samuels, 1959
Eliza - NLP
„
„
Alan Newell, Cliff Shaw, Herb Simon, 1957
Checkers (Machine Learning)
„
„
Claude Shannon, 1950
Terry Winnograd, 1972
GPS (General Problem Solver)
„
Alan Newell & Herb Simon, 1972
18
The 70’s
Bi th and
Birth
d Rise
Ri of
fE
Expertt S
Systems
t
1970-mid 1980’s
1973
1974
-
1975
Alain Colmerauer
PROLOG
Paul Werbos
Neural Networks
Back Propagation Law
E. Feigenbaum, R. Lindsay.
Dendral
E.FeigenBaum
Edward Shortliffe
MYCIN
19761980
R. Duda, P.Hart, P. Barnett
PROSPECTOR: The first commercial
Expert System
1982
John McDermott
XCON – "Expert Configurer
P.Hart
The Plateau
In the 70's, AI researchers began to discover that the
problem wasn't as easy as it looked!
„
The Frame Problem
„
L k of
Lack
f Common
C
Sense
S
Reasoning
R
i
„
Combinatorial Explosion
„
The Gap – "Toy" vs. "Real" worlds
„
„
Perceptrons, by Minsky & Papert (1969) – proved
limitations of perceptron networks and acted to limit
significant research in the 70
70'ss
Lighthill Report – 1973: curtailed research funding in
British Universities
AI developed a reputation as "over-hyped" and
unrealistic
19
1982
Rebirth of Arttificial Neural Netw
works
Commercializa
ation of Expert Systtems
The 80’s
- John Hopfield
- Hopfield Networks
-
1982
1986
Teuvo Kohonen
self-organising feature maps for speech
recognitizion
T
Sejnowski
S j
ki
- Terrence
- NETTalk
Rumerhalt,McMelland
Neural Networks
Rediscovering of Back-Propagation
Learning
1987
- Marvin Minsky
- The Society of Minds
-
Fuzzy Appliances
1989
- Dean Pomerleau
- ALVINN
-
Commercial Success
Despite it's
it s reputation as "over
over-hyped
hyped", certain
AI applications became very successful during
the 70's – 80's:
•
Expert Systems
•
Industrial Robotics
•
Planning & Scheduling Applications
AI became a $2,000,000,000 industry by 1988
20
Nowadays…
- Major advances in all areas of AI, with
- significant demonstrations
-
Early 90’s
Late 90’s
1995
Birth of Intelligent Systems
1997
The Deep Blue chess program beats
Garry Kasparov
- Web crawlers
- AI-based information extraction
- programs
Intelligent Room and Emotional
Agents at MIT's AI Lab
2000-
Interactive robot pets
The Nomad robot
The Gartner Hype Curve
„
Interest in AI followed this pattern
pattern,
typical of the hype surrounding new
technologies
21
AI State of the art
„
Have the following been achieved by AI?
„
„
„
„
„
„
„
„
„
World-class chess playing
Playing table tennis
Cross-country driving
Solving mathematical problems
Discover and prove mathematical theories
Engage in a meaningful conversation
Understand spoken language
Observe and understand human emotions
…
Types of expertise
(with examples)
Deep
cognitive
skills
Judgmental High-level
skills
social skills
Highly
creative
Musician
Senior
manager
Analytical
Mathemati i
ician
Economist, Social
programmer scientist
i ti t
Typist
Strictly
procedural
Driver
Author, poet
Social
worker
22
A driving example: Grand
Challenge
„
Goal:
Artificial Intelligence
Applications
Artificial
Intelligence
Cognitive
Science
Applications
•Expert Systems
•Fuzzy Logic
•Genetic Algorithms
•Neural Networks
Robotics
Applications
•Visual Perceptions
•Locomotion
•Navigation
•Tactility
Natural
Interface
Applications
•Natural Language
•Speech Recognition
•Multisensory Interface
•Virtual Reality
23
AI Application Areas in Business
Neural Networks
Fuzzy Logic Systems
Genetic Algorithms
Virtual Reality
y
AI Application
Areas in
Business
Intelligent Agents
Expert Systems
Components of Expert Systems
The Expert System
Expert
Advice
User
User
IInterface
t f
Programs
Inference
E i
Engine
Program
Knowledge
K
l d
Base
Workstation
Expert System Development
Knowledge
Engineering
Knowledge
Acquisition
Program
Workstation
Expert and/or
Knowledge Engineer
24
Expert System Applications
Decision Management
Diagnostic/Troubleshooting
Maintenance/Scheduling
Design/Configuration
Major
Application
Categories
of Expert Systems
Selection/Classification
Process Monitoring/Control
Course Overview
General Introduction
„
Introduction. [AIMA Ch 1] Course Schedule.
Homeworks, exams and grading. Course
material, TAs and office hours. Why study AI?
What is AI? The Turing test. Rationality.
Branches of AI. Research disciplines
connected to and at the foundation of AI.
Brief history of AI. Challenges for the future.
Overview of class syllabus.
25
Agent
effectors
sensors
Course Overview
General Introduction
„
Intelligent Agents. [AIMA Ch 2] What is
an intelligent agent? Examples. Doing the right
thing (rational action). Performance measure.
Autonomy. Environment and agent design.
Structure of agents
agents. Agent types
types. Reflex agents
agents.
Reactive agents. Reflex agents with state.
Goal-based agents. Utility-based agents. Mobile
agents. Information agents.
Course Overview (cont.)
„
Problem solving and search.
[AIMA Ch 3]
„
„
„
„
„
„
„
measuring problem.
Types of problems.
More examples.
Basic idea behind search algorithms.
Complexity.
Combinatorial explosion and NP
completeness.
Polynomial hierarchy.
3l
5l
9l
Using these 3 buckets,
measure 7 liters of water.
Traveling salesperson problem
26
Course Overview (cont.)
How can we solve complex problems?
„
Uninformed search. [AIMA Ch
3]
„
„
„
„
„
„
Depth-first.
Breadth-first.
Uniform-cost.
Depth-limited.
Iterative deepening.
Examples. Properties.
3l
5l
9l
Using these 3 buckets,
measure 7 liters of water.
Traveling salesperson problem
Course Overview (cont.)
How can we solve complex
p
p
problems?
„
Informed search. [AIMA Ch 4]
„
„
„
„
„
„
Best-first.
A* search.
Heuristics.
Hill climbing.
Problem of local extrema.
Simulated annealing.
Traveling salesperson problem
27
Course Overview (cont.)
Practical applications
of search
„
Constraint Satisfaction
[AIMA Ch 5]
„
„
Backtracking
g
Local search
Course Overview (cont.)
Practical applications
of search
„
Game playing
[AIMA Ch 6]
„
„
„
„
The minimax algorithm.
g
Resource limitations.
Aplha-beta pruning.
Elements of chance and
non-deterministic games.
tic-tac-toe
28
Course Overview (cont.)
Towards intelligent agents
„
Agents that reason
logically 1
[AIMA Ch 7]
„
„
„
Knowledge-based
agents.
Logic and
representation.
Propositional (boolean)
logic.
wumpus world
Course Overview (cont.)
Towards intelligent agents
„
Agents that reason
logically 2.
[AIMA Ch 7]
„
„
„
„
Inference in
propositional
ii
l logic.
l i
Syntax.
Semantics.
Examples.
wumpus world
29
Course Overview (cont.)
Building
u d g knowledge-based
o edge ased
agents: 1st Order Logic
„
First-order logic 1. [AIMA Ch 8]
„
„
„
„
„
„
„
Syntax.
Semantics.
Atomic sentences.
sentences
Complex sentences.
Quantifiers.
FOL knowledge base.
Situation calculus.
Course Overview (cont.)
Building knowledge
knowledgebased agents: 1st
Order Logic
„
First-order logic 2.
[AIMA Ch 9]
„
„
„
Describing actions.
Planning.
Action sequences.
30
Course Overview (cont.)
Reasoning Logically
„
Inference in first-order logic.
[AIMA Ch 9]
„
„
„
„
Proofs.
Unification
Unification.
Generalized modus ponens.
Forward and backward chaining.
Example of
backward chaining
Course Overview (cont.)
Representing and Organizing
Knowledge
„
Building a knowledge base.
[AIMA Ch 10]
„
„
„
„
Knowledge bases.
Vocabulary and rules.
Ontologies
Organizing knowledge.
An ontology for the sports domain
31
Course Overview (cont.)
Systems
y
that can Plan
Future Behavior
„
Planning.[AIMA Ch 11]
„
„
„
„
Definition and goals.
Basic representations for
planning.
l
i
Situation space and plan space.
Examples.
Course Overview (cont.)
Learning
g from Observation
„
Decision Trees
[AIMA 18]
„
„
„
„
Introduction to decision trees.
Information theory.
Constructing DT.
Examples.
32
Course Overview (cont.)
Expert
p
Systems
y
„
Probabilities + Bayesian Networks
[AIMA 13 + 14]
„
„
„
„
„
Basics of probability theory
Bayesian rule.
Conditional
d
l reasoning.
Bayesian Networks.
Reasoning under uncertainty
Course Overview (cont.)
Statistical Learning
g Methods
„
Neural Networks.
[AIMA 20]
„
„
„
„
Human brain structure
Neuron and activation function.
Forward and backward propagations.
Examples.
33
Course Overview (cont.)
Logical
g
Reasoning
g in the
Presence of Uncertainty
„
Fuzzy logic
[Handout]
„
„
„
„
Center of gravity
Introduction to fuzzyy logic.
g
Linguistic Hedges.
Fuzzy inference.
Examples.
Center of largest area
Course Overview (cont.)
Machine Learning
g
„
Genetic Algorithms
[Handout + AIMA 4]
„
„
„
Genetic algorithm approach.
Mutation, Crossover, Fitness function.
Examples.
34
Course Overview (cont.)
What challenges
g remain?
„
Towards intelligent machines.
[AIMA Ch 25]
„
The challenge of robots:
„
„
„
„
„
„
with what we have learned,
what hard problems remain to be solved?
Different types of robots.
Tasks that robots are for.
Parts of robots. Architectures.
Configuration spaces.
robotics@USC
Course Overview (cont.)
What challenges remain?
„
Overview and summary. [all of the
above]
„
„
What have we learned.
learned
Where do we go from here?
robotics@USC
35
Outlook
„
„
AI is a very exciting area right now.
now
This course will teach you the
foundations.
36