Download animated version

Course Overview
 What is AI?
 What are the Major Challenges?
 What are the Main Techniques?
 Where are we failing, and why?
Part I:
Introduce you to
what’s happening in
Artificial Intelligence

Done
 Step back and look at the Science
Part II:
Step back and look at the History of AI Give you an
appreciation for
the big picture
 What are the Major Schools of Thought?
 What of the Future?
 Why it is a
grand challenge
AI Developments from 1963 – 1970
Micro-Worlds
Minsky’s Students’ progress at MIT…
 STUDENT program

Bill's father's uncle is twice as old as Bill's father.
Two years from now Bill's father will be three
times as old as Bill. The sum of their ages is 92.
Find Bill's age.
 Could not solve standard problems in a textbook
 But could solve most if they were rewritten for it
Minsky’s Students’ progress at MIT…
 SIR program








Every boy is a person.
I UNDERSTAND.
A finger is part of a hand.
I UNDERSTAND.
Each person has two hands.
THE ABOVE SENTENCE IS AMBIGUOUS—
PLEASE REPHRASE IT.
There are two hands on each person.
I UNDERSTAND.
Minsky’s Students’ progress at MIT…
How many fingers does John have?
THE ABOVE SENTENCE IS AMBIGUOUS—BUT I
ASSUME “HAS” MEANS “HAS AS PARTS”. I DON’T
KNOW WHETHER FINGER IS PART OF JOHN.
John is a boy.
I UNDERSTAND.
How many fingers does john have?
HOW MANY FINGERS PER HAND?
Every hand has 5 fingers.
I UNDERSTAND.
How many fingers does john have?
THE ANSWER IS 10.
Minsky’s Students’ progress at MIT…
 SIR program



Worked in very constrained domain
E.g. “x times y”
“the number of times I went to the movies”
Minsky’s Students’ progress at MIT…
 Computer vision problem

No idea how hard it was

Minsky hired Gerry Sussman, a first year undergraduate student

Told him to build a computer vision system

As a Summer project!
 Eventually hired a team to work on vision

Invented “Blocks Micro World”

Regular square and rectangular blocks and pyramids

Successful in recognising objects based on edges
 Robotics



Built robot hand to grab and manipulate blocks
Took a lot of work
They felt it was similar in complexity to
NASA’s Apollo moon-landing program
Stanford’s
Shakey
Stanford’s Shakey




Moved around specially built rooms
Moved boxes around
Used STRIPS language
Preconditions

ONTABLE(X), HANDEMPTY, CLEAR(X)
 Delete List

ONTABLE(X) , HANDEMPTY
 Add list

HOLDING(X)
 Life Magazine


“the first electronic person”
“could travel around the moon for
months at a time without a single
beep of direction from the earth”
MIT’s SHRDLU
MIT’s SHRDLU
 Pick up a red block
 OK
 Find a block which is taller than the one you are holding and put it into the
box.
 By "it," I assume that you mean the block which is taller than the one I
am holding. OK.
 What does the box contain?
 The blue pyramid and the blue box
 What is the pyramid supported by?
 the box
 How many blocks are not in the box?
 four of them
 Is at least one of them narrowed than the one which I told you to pick up?
 yes, the red cube
 Is it supported?
 yes, by the table
MIT’s SHRDLU
 Pick up a red block
 OK
 Find a block which is taller than the one you are holding and put it into the
box.
 By "it," I assume that you mean the block which is taller than the one I
am holding. OK.
 What does the box contain? people said:
 The blue pyramid and the blue box
 What is the pyramid supported by?
 the box
 How many blocks are not in the box?
 four of them
 Is at least one of them narrowed than the one which I told you to pick up?
 yes, the red cube
 Is it supported?
 yes, by the table
AI is finally here
“A digital computer will be the world's chess champion
unless the rules bar it from competition;
A digital computer will discover and prove an important
new mathematical theorem;
A digital computer will write music that will be accepted by
critics as possessing considerable aesthetic value;
Most theories in psychology will take the form of computer
programs, or of qualitative statements about the
characteristics of computer programs..”
Herbert Simon, 1957
“A digital computer will be the world's chess champion
unless the rules bar it from competition;
A digital computer will discover and prove an important
new mathematical theorem;
A digital computer will write music that will be accepted by
critics as possessing considerable aesthetic value;
Most theories in psychology will take the form of computer
programs, or of qualitative statements about the
characteristics of computer programs..”
Herbert Simon, 1957
“A digital computer will be the world's chess champion
unless the rules bar it from competition;
A digital computer will discover and prove an important
new mathematical theorem;
A digital computer will write music that will be accepted by
critics as possessing considerable aesthetic value;
Most theories in psychology will take the form of computer
programs, or of qualitative statements about the
characteristics of computer programs..”
Herbert Simon, 1957
“A digital computer will be the world's chess champion
unless the rules bar it from competition;
A digital computer will discover and prove an important
new mathematical theorem;
A digital computer will write music that will be accepted by
critics as possessing considerable aesthetic value;
Most theories in psychology will take the form of computer
programs, or of qualitative statements about the
characteristics of computer programs..”
Herbert Simon, 1957
“… only moderate extrapolation is required from the
capacities of current programs already in existence to
achieve the additional problem-solving power needed for such
simulation.”
Herbert Simon, 1958
“Machines will be capable,
within twenty years,
of doing any work that a man can do.”
Herbert Simon, 1965
Herb Simon’s Predictions…
Computer chess
champ
Discover important
maths theorem
Compose music
Most Psychology
theories are programs
Simon reality
said
1967
1967
1967
1967
Herb Simon’s Predictions…
Computer chess
champ
Discover important
maths theorem
Simon reality
said
1967 Probably 1997-2007
Still close contests
1967
Compose music
1967
Most Psychology
theories are programs
1967
Herb Simon’s Predictions…
Computer chess
champ
Discover important
maths theorem
Simon reality
said
1967 Probably 1997-2007
Still close contests
1967 Not really, but HR is
probably pretty close
(by Simon Colton, c. 2000)
Compose music
1967
Most Psychology
theories are programs
1967
Herb Simon’s Predictions…
Computer chess
champ
Discover important
maths theorem
Simon reality
said
1967 Probably 1997-2007
Still close contests
1967 Not really, but HR is
probably pretty close
(by Simon Colton, c. 2000)
Compose music
1967 Possibly in the 80’s
(depends on how aesthetic it
needs to be)
Most Psychology
theories are programs
1967
Herb Simon’s Predictions…
Computer chess
champ
Discover important
maths theorem
Simon reality
said
1967 Probably 1997-2007
Still close contests
1967 Not really, but HR is
probably pretty close
(by Simon Colton, c. 2000)
Compose music
1967 Possibly in the 80’s
(depends on how aesthetic it
needs to be)
Most Psychology
theories are programs
1967 Probably not yet
Summarising AI Developments up to 1970
 Success in micro worlds
 The domains used were constrained
 Also, human intelligence used to abstract problems
 Monkey – chair – banana
 Textbook problems for STUDENT program
 Seemed a reasonable way to do science…
 like physicists’ carefully controlled experiments
 They thought it would be possible to expand the domains
 SHRDLU author thought approach could easily be extended
 … this proved not to be the case
 Discovered that a vast amount of commonsense knowledge
is needed for the most basic tasks in real world
 Note again the first law
 E.g. STUDENT or Robot arm to move blocks
“Artificial intelligence has done well in tightly
constrained domains.
Winograd [SHRDLU], for example, astonished everyone
with the expertise of his blocks-world natural
language.
Extending this kind of ability to larger worlds has
not proved straightforward, however…
The time has come to treat the problems involved as
central issues.”
Patrick H. Winston, 1976
What about all that military money?
 US National research council and military had put
millions into Machine Translation
 Wanted to translate many Russian texts (cold war)
 Thought it should be easy
– like computers could translate WWII codes
 1966 “Any task that requires real understanding of
natural language is too difficult for a computer”
- Bar-Hillel
 1966 Funding ended
 This didn’t affect Stanford, MIT, CMU
 They weren’t doing translation
What about all that military money?




Meanwhile at Stanford, MIT, CMU…
DARPA started looking for results…
Had funded speech understanding system at Carnegie Mellon
System worked by constraining grammar, with 1000 words




Academics were pleased with their progress
… but Was not very useful…
User had to keep guessing what way to say something to be accepted
More troublesome for military personnel to use than menu system
 MIT’s SHRDLU did not extend to larger domains
 Hit commonsense knowledge problem
Example of Commonsense Knowledge Problem
 Tried to build a system to understand children’s stories.
 Fred was going to the store. Today was Jack’s
birthday and Fred was going to get a present.
 Can a system answer questions on the story?
 Why is Fred going to the store?
 Who is the present for?
 Why is Fred buying a present?
 Cannot answer questions without extra (commonsense)
knowledge…
 Objects got at stores are usually bought.
 Presents are often bought at stores.
 If a person is having a birthday, he is likely to get presents.
What about all that military money?




Meanwhile at Stanford, MIT, CMU…
DARPA started looking for results…
Had funded speech understanding system at Carnegie Mellon
System worked by constraining grammar, with 1000 words




Academics were pleased with their progress
… but Was not very useful…
User had to keep guessing what way to say something to be accepted
More troublesome for military personnel to use than menu system
 MIT’s SHRDLU did not extend to larger domains
 Hit commonsense knowledge problem
 Stanford’s “Shakey” was not fooling DARPA people
 Had high probability of failure on any action
 Could not reliably do a sequence of actions
 1974 Stanford, MIT, CMU all got funding cut to almost nothing
Meanwhile in UK…
 1973 Lighthill report commissioned by Research Council…
“in no part of the field have discoveries made so far
produced the major impact that was then promised”
 Category A: advanced automation or applications
 approves of it in principle
 Category C: studies of central nervous system
computer modeling for neurophysiology and psychology
 approves of it in principle
 Category B: “building robots” and “bridge” between A&C
 Does not approve
 No place for exploring intelligent information processing for its
own sake
 No money for AI 
Chatterbots: ELIZA






Criticism of AI from within…
1966 written by Weizenbaum (MIT)
simple parsing + substitution of key words into canned phrases
E.g. for family terms: “tell me more about your …”
Chose psychiatrist because he could get away with a lot
“Tell me more about streetlights”
 Weizenbaum wanted to show that an AI program could easily
appear intelligent
 … but there was not much going on inside
 Weizenbaum said AI takes rational logic view
because the light is better there
 He believed that there was more to human int…
Minsky and Papert’s “Perceptrons”
 More criticism of AI from within…
 There had been great excitement about neural networks
 1958 Rosenblatt had an article in Science magazine, titled
“Human Brains Replaced?”
 Rosenblatt said the perceptron could
“tell the difference between a dog and a cat”
 1969 Minsky and Papert published book “Perceptrons”
 Mathematically proved that there were some simple things that
perceptrons could never learn
 For example: XOR
 Killed off neural networks work for 10 years (funding stopped)
 Rosenblatt died in a boating accident in 1969
1970’s…
The AI Winter set in
Expert Systems (1970s)







Researchers realised “Knowledge” is the key…
Computer memory was starting to get larger
DENDRAL was first expert system
To help organic chemists identify unknown organic molecules
Embodied expert chemists’ knowledge as IF THEN rules
Searched a tree of possibilities
System was successful
 MYCIN system
 diagnose infectious blood diseases
 recommend antibiotics and dosage
 Performed well 65% correct – better than non-specialist doctors
 Not used (ethical concerns)
Commercial Expert Systems
 1978 XCON by John McDermott of CMU
 Assist in ordering DEC's VAX computer systems
 Automatically selecting components based on customer requirements
 Went into use in 1980 in DEC's plant
 By 1986, had processed 80,000 orders
 Achieved 95-98% accuracy
 Saved DEC $25M a year
 No need to give customers free components when technicians made
errors
 Speeded assembly process
 Increased customer satisfaction
Micro Worlds had problems to scale up to the
real world and commonsense knowledge
Expert Systems didn’t have a problem…
Expertise is a sort of Micro-World
More Developments in the 1970s
 The emergence of Neats and Scruffies
 Roger Schank introduced scripts to help story understanding
 His programs had to act like humans… sometimes making
illogical assumptions
 He called his approach “scruffy” to distinguish from McCarthy
 Neats required everything to be logical
 Consistent, provably correct, clear cut, guaranteed stability (learning)
 Scruffies want something that works
 Don’t care how! Will use any technique, even some logic
 Procedural, without mathematical proof
 Aaron Sloman’s view:
 Scruffiness is inevitable
http://www.cs.bham.ac.uk/research/projects/cogaff/sloman.scruffy.ai.pdf
More Developments in the 1970s
The Neats Vs. The Scruffies
 Neats complain about Scruffies:
 programs are complex, ill-structured
 not possible to explain or predict their behaviour
 not possible to prove that they do what they are intended to do.
 Scruffies complain about Neats:




Neats’ programs only work on toy domains
get bogged down with real world complexity
Use fancy mathematical techniques just to show how smart they are
…but no increase in understanding of any interesting phenomenon
 Minsky also moved away from logic
 Came up with “Frames” for knowledge
 Meanwhile in the Neat camp…
1972 Prolog developed (Edinburgh and Marseille)
 Prolog gained popularity outside US
 LISP dominant in US
On to the 1980s
Developments in the 1980s
 Great commercial success for expert systems
 Neural Networks back on the scene





1986 Parallel Distributed Processing
By Rumelhart and psychologist McClelland
Neural networks become commercially successful in 1990s…
optical character recognition
speech recognition
 Another “school of thought” division
 Symbolic Vs. Sub-symbolic (aka non-symbolic)
 Symbolic means
 Typically logic, symbols are high level concepts, monkey-chair-banana
 Also called GOFAI (Good Old Fashioned AI)
 Sub-symbolic means (still has symbols, but lower level symbols)
 Neural networks (aka connectionist)
 Fuzzy systems
 Evolutionary computation
Developments in the 1980s
 1984 Doug Lenat – Cyc





Attack commonsense knowledge problem directly
Massive knowledge base
All facts that the average person knows
Expected to take only two person-centuries
Nowhere near success in 2007
Developments in the 1980s
 Autonomous Land Vehicle (ALV)







Military project started in 1983
25 million funding per year
Robot wanderer, travel over rugged terrain
By 1987 – clear that goals were far away
Longest ever off road outing: 600 yards at 2mph
1989 Pentagon ended the project
(although 20 years later technology is capable)
 In general: in the 1980s Funding returned…
 …but boom-bust cycle again
 AI did not meet expectations again
 Mini-winter 1987-1993
 1991 First Gulf War
 AI made substantial contribution… but not exactly where expected
 E.g. in scheduling, arranging supplies
More Modern Trends…
 1987 Rodney Brooks “Intelligence Without Representation”
 “Embodied Intelligence” idea
 Belief that intelligence must have a real body
 interacting with the real world
 needs to perceive, move and survive in the world
 Reject work on simulations as oversimplifying the problem
 Often try to build animal level intelligences first
More Modern Trends…
 Developmental Robotics (also called Epigenetic Robotics)
 Follows Turing’s idea:
“Instead of trying to produce a programme to simulate
the adult mind, why not rather try to produce one which
simulates the child's? If this were then subjected to an
appropriate course of education, one would obtain the
adult brain.”
 http://www.youtube.com/watch?v=FPw9z2aaTa8
 Social Interaction and Imitation approaches
 Believe that human infants learn a lot by imitating
 Also human infants learn what is important



Mother points to interesting objects
Infant looks to mother’s face to see if she approves
Infant responds to emotion in mother’s voice
 Strong AI
The Strong / Weak Division
 Trying to build a system that is equal or better than a human, on
general tasks
 e.g. Weizenbaum’s view (MIT): The goal of strong AI is
“nothing less than to build a machine on the model of man, a
robot that is to have its childhood, to learn language as a child
does, to gain its knowledge of the world by sensing the world
through its own organs, and ultimately to contemplate the
whole domain of human thought.”
 Weak AI (also called “applied AI”)
 Building useful applications,
usually restricted to a particular domain, specific tasks
 e.g. an autonomous vehicle, or a speech recognition system
 Most people work on weak AI
 Many people wonder why more people don’t work on strong AI
 Perhaps the history of AI answers this question…
 Because people working on strong AI don’t get anywhere!
“A year spent in artificial intelligence is
enough to make one believe in God.”
Alan Perlis
Up to present day… what happened?
 AI fragmented into sub-disciplines…
(From AAAI 2008 conference)










Agents (includes Multi-Agent Systems)
Cognitive modeling and human interaction
Commonsense reasoning
Constraint satisfaction
Evolutionary computation
Game playing and interactive entertainment
Information integration and extraction
Knowledge acquisition and ontologies
Knowledge representation and reasoning
Machine learning and data mining
Up to present day… what happened?









Machine learning and data mining
Model-based systems
Natural language processing
Planning and scheduling
Probabilistic reasoning
Robotics
Search
Semantic web
Vision and perception
Up to present day… what happened?
 2006 Minsky complained:
 central problems, like commonsense reasoning, neglected
 majority of researchers pursued commercial applications
 e.g. commercial applications of neural nets or genetic
algorithms
"So the question is
why we didn't get HAL in 2001?
I think the answer is
I believe we could have."