Download animated version

“The main lesson of thirty-five years of AI research is that
the hard problems are easy and the easy problems are hard.
The mental abilities of a four-year-old that we take for
granted – recognizing a face, lifting a pencil, walking
across a room, answering a question – in fact solve some of
the hardest engineering problems ever conceived.... ”
Steven Pinker (Linguist / Psychologist)
“As the new generation of intelligent devices appears, it will
be the stock analysts and petrochemical engineers and
parole board members who are in danger of being replaced
by machines. The gardeners, receptionists, and cooks are
secure in their jobs for decades to come.”
Steven Pinker (Linguist / Psychologist)
 Pinker says we’re successful on “hard” problems, but not the “easy”
 We can say more:
More and more progress on the “hard” problems seems to be
taking us no closer to solving the “easy” ones
 We’re able to tackle specific specialist problems,
i.e. Engineer a solution to a specialist problem
 But the more we go into them,
the further we get from the original goal of AI

(“original goal” = AI as good as a human)
 Like language moving more shallow than deep
 We move more to specific techniques,

but gain no insight into general intelligence
 What about general purpose AI?
 Pinker says we’re successful on “hard” problems, but not the “easy”
 We can say more:
More and more progress on the “hard” problems seems to be
taking us no closer to solving the “easy” ones
Summing up 50 years’ progress in AI
(From Part I of Course)
 We’re able to tackle specific specialist problems,
i.e. Engineer a solution to a specialist problem
 But the more we go into them,
the further we get from the original goal of AI

(“original goal” = AI as good as a human)
 Like language moving more shallow than deep
 We move more to specific techniques,

but gain no insight into general intelligence
 What about general purpose AI?
Course Overview
 What is AI?
 What are the Major Challenges?
 What are the Main Techniques?
Part I:
Introduce you to
what’s happening in
Artificial Intelligence
 Where are we failing, and why?

Done
 Step back and look at the Science
 Step back and look at the History of AI
 What are the Major Schools of Thought?
 What of the Future?
Part II:
Give you an
appreciation for
the big picture
 Why it is a
grand challenge
Course Overview
 What is AI?
 What are the Major Challenges?
 What are the Main Techniques?
Part I:
Introduce you to
what’s happening in
Artificial Intelligence
 Where are we failing, and why?

Done
Step back and look at the Science
 Step back and look at the History of AI
 What are the Major Schools of Thought?
 What of the Future?
Part II:
Give you an
appreciation for
the big picture
 Why it is a
grand challenge
“AI is an Engineering discipline
built on an unfinished Science.”
Matt Ginsberg, 1995
reported in SIGART bulletin
Vol 6, No.2 April 1995
The Science and Engineering of AI
 AI has an Engineering aspect and a Science aspect
 Engineering:
 Build stuff that works, serves a practical function
 Physical:
– Bridge, Aeroplane
 Information Processing:
– Translation system, Autonomous Vehicle
 Science:
 Discovery of knowledge; general truths and laws
 Physical:
– Mechanical forces, stresses, tension, material strength,
aeronautics
 Information Processing: We have some science …
– Speed of certain routines (Computer Science)
– Limits and abilities of certain learning algorithms
 … but we would really like a “Science of Intelligence”
The Science and Engineering of AI
 Good Engineering should rest on a solid scientific foundation
 AI’s foundation looks a bit shaky…
 Consider something like bridge building:
 Science exists, can make it strong enough to hold a certain load,
know how many pillars/cables etc. needed
 Similar for Aeroplanes. Science also exists, how many engines,
power, aerodynamic shape etc.
 What about AI problems?
 For Machine Translation: only have science for some subtasks:
parsing, n-gram language model
 For Natural Language Understanding: not even sure how to describe
the problem!
 Yet more ambitious:
What we really want to build is something intelligent
 What about the Science of Intelligence?
 AI seems obsessed with better and better engineering
“Chess is the Drosophila of artificial intelligence.”
John McCarthy
Drosophila
 Drosophila = Fruit Fly
 Drosophila Melanogaster heavily
used in research in genetics
 Small, easy to grow in laboratory
 Short generation time (two weeks)
 Only four pairs of chromosomes:
easy to study
 Genome sequenced in 2000
 Some say Chess is Drosophila of AI
 Easy to study
 Studied a lot
“Chess is the Drosophila of artificial intelligence.
However, computer chess has developed much as
genetics might have if the geneticists had
concentrated their efforts starting in 1910 on
breeding racing Drosophila.
We would have some science, but mainly we would
have very fast fruit flies.”
John McCarthy
“Chess is the Drosophila of artificial intelligence.
However, computer chess has developed much as
genetics might have if the geneticists had
concentrated their efforts starting in 1910 on
breeding racing Drosophila.
We would have some science, but mainly we would
have very fast fruit flies.”
John McCarthy
AI seems obsessed with better and better engineering
Where is the Science of Intelligence?…
What is Intelligence?
 There is no widely agreed-upon scientific definition of intelligence
 Try some dictionary definitions…
 Understand world, reason about it
 Able to use knowledge to manipulate it ( to achieve any desired end)
 Profit from experience (i.e. not static, improving all the time, learning)
 There seems to be an internal aspect
 Understand, reason
 Difficult to come up with a precise definition for what this is
 What constitutes adequate “understanding”?
 Tied up with human “meaning” of things in world
 There seems to be an external aspect
 Manipulate the world
 Difficult to come up with a precise definition for what this is
 Manipulate what exactly? And manipulate it in what way?
 Tied up with external objects/forces/relationships in the world
 We would like some clear abstract theory of “processing information”
 Not tied up with human meanings of internal processes
 Not tied up with external world objects
What is Artificial Intelligence?
 (See have the AI guys done any better for a definition)
 Definitions tied up with internal processes:
 To automate ” …activities that we associate with human thinking, activities such
as decision making, problem solving, learning...”(Bellman,1978)
 “The exciting new effort to make computers think … machines with minds”
(Haugeland, 1985)
 ”The study of mental faculties through the use of computational models.”
(Charniak and McDermott, 1985)
 ”The study of computations that make it possible to perceive, reason, and act.”
(Winston, 1992)
 Definitions tied up with external world objects:
 ”The art of creating machines that perform functions that require intelligence
when performed by people.” (Kurzweil, 1990)
 ”The study of how to make computers do things at which, at the moment, people
are better.” (Rich and Knight, 1991)
 ”AI . . . Is concerned with intelligent behavior in artifacts.” (Nilsson, 1998)
 AI definitions still tied up with poorly defined external or internal stuff
 AI definitions bring in a new aspect:
 Explicit mention of humans
 Not very helpful!
What is Artificial Intelligence?
 (See have the AI guys done any better for a definition)
 Definitions tied up with internal processes:
 To automate ” …activities that we associate with human thinking, activities such
as decision making, problem solving, learning...”(Bellman,1978)
 “The exciting new effort to make computers think … machines with minds”
(Haugeland, 1985)
 ”The study of mental faculties through the use of computational models.”
(Charniak and McDermott, 1985)
 ”The study of computations that make it possible to perceive, reason, and act.”
(Winston, 1992)
 Definitions tied up with external world objects:
 ”The art of creating machines that perform functions that require intelligence
when performed by people.” (Kurzweil, 1990)
 ”The study of how to make computers do things at which, at the moment, people
are better.” (Rich and Knight, 1991)
 ”AI . . . Is concerned with intelligent behavior in artifacts.” (Nilsson, 1998)
 AI definitions still tied up with poorly defined external or internal stuff
 AI definitions bring in a new aspect:
 Explicit mention of humans
 Not very helpful!
Towards a Scientific Definition of Intelligence
 What would a precise definition of intelligence look like?
 Can expect it to be similar to the definition of
communication
 Also a human activity, very complicated with lots of human
meaning
 …But can be studied purely abstractly as a mathematical
problem
 Possibly a good example for AI because
 Both are about processing information
 Unlike Physics/Chemistry/Biology
where theories are about physical objects/forces/processes
“The fundamental problem of communication is that of
reproducing at one point either exactly or approximately a
message selected at another point.
Frequently the messages have meaning; that is they refer to or
are correlated according to some system with certain physical or
conceptual entities. These semantic aspects of communication
are irrelevant to the engineering problem.
The significant aspect is that the actual message is one selected
from a set of possible messages.
The system must be designed to operate for each possible
selection, not just the one which will actually be chosen since this
is unknown at the time of design.”
Claude Shannon,
“A mathematical theory of communication”,
1948
“The fundamental problem of communication is that of
reproducing at one point either exactly or approximately a
message selected at another point.
Frequently the messages have meaning; that is they refer to or
are correlated according to some system with certain physical or
conceptual entities. These semantic aspects of communication
are irrelevant to the engineering problem.
The significant aspect is that the actual message is one selected
from a set of possible messages.
The system must be designed to operate for each possible
selection, not just the one which will actually be chosen since this
is unknown at the time of design.”
Claude Shannon,
“A mathematical theory of communication”,
1948
What if there is no theory?
 Maybe there is no “clean” theory of Intelligence
 Maybe it’s just some stuff that happens in your head
 Gravity, Electromagnetism, Light, Motions of planets, etc.
all have “clean” theories
 …but there’s no reason why intelligence must have a clean theory
 Human intelligence evolved over millions of years
 Could well be just a messy load of neuron wiring that is intelligence
 David Marr (1945-1980) described “Type 1” and “Type 2” theories…
Marr’s “Personal View”
 Two types of theory
 Type 1 “clean” theories
 Clear what and how
 What: Clear description of what input needs to get transformed to
what output
 Different programs (how) could solve the same computational
problem (what)
 Type 2 “messy” theories
 Problem is solved by the simultaneous action of a considerable
number of different processes,
 whose interaction is its own simplest description
 There is no reason why all theories should be Type 1
(Marr acknowledges that it is not a pure dichotomy
a spectrum of possibilities exists in between 1&2)
Marr’s “Personal View”
 Progress in AI can consist in
1. Isolate an information processing problem
2. Formulate a computational theory for it (what)
3. Construct a program that implements it (how)
 Example
1. Find shape from shading in an image
2. Mathematical description of how input related to output
3. Working program
 Part 2 tells you what and explains why


This never needs to be reformulated
Like a result in mathematics, or hard natural sciences
 Part 3 tells you how (often many options)
Marr’s “Personal View”
 Progress in AI can consist in
1. Isolate an information processing problem
2. Formulate a computational theory for it

tells you what and explains why it’s important
3. Construct a program that implements it

tells you how
 Marr criticises “Mimicry”

Behaviour:


Mimic some aspect of human behaviour
(chatterbot, IF-THEN rules)
Structure:

Mimic some aspect of low level structures (neurons)
 Problem is they are studying “how” (3)
without any clear idea of “what and why” (2)
Marr’s “Personal View”
Marr criticises “Mimicry”


Behaviour:


Structure:





Mimic some aspect of human behaviour
(chatterbot, IF-THEN rules)
Mimic some aspect of low level structures (neurons)
Problem is they are studying “how” (3)
without any clear idea of “what and why” (2)
No need to copy flapping or feathers to fly
Need to study “what” flight is
Not “how” bird is built
Marr’s “Personal View”

But remember, the breakdown only works for Type 1 theories
1.
Isolate an information processing problem
2.
Formulate a computational theory for it (what)
3.
Construct a program that implements it (how)

For Type 2 what and how are tangled

Some dangers…

Going for Type 2 theories when Type 1 exist

Can get something that works,

But sheds no light on the Type 1 theory if there is one

(?) Maybe this is what AI has been doing (Part I of this course)

Looking for Type 1 theory when the problem is messier

Type 1 theory might approximate a real Type 2 process

Might be refusing to see the reality because there seems to be a nice
elegant theory (which is wrong)
What if there is no Type 1 theory?
 Some science would help the Engineering effort of building systems
 …but if science is hard to formulate, then…
 Why not just keep building stuff that works, serves a practical function?
 We have seen from Part I of course…
 There seem to be severe limits on what we can do by building specific
systems
– Natural Language Understanding
– Recognising objects in vision
– Adapting old knowledge to new problems
– Having commonsense
 Doesn’t look like we are getting closer to general solutions
 Even if we can’t find a clean Type 1 theory for all of intelligence
 It might still be worthwhile to take a more scientific approach
 Rather than Engineering all the time
Where to Find Inspiration?
 It looks like we should step back from specific problems
 Diagnosing diseases, recognising vowels, playing chess,
recognising faces…
 We should also step back from specific techniques
 Search, logic, neural network, genetic algorithm…
 We need to look at the big picture of what intelligence is
 Where can we get some hints about this?
 Cognitive Science…
 But always bear in mind that we want to be clear about
what we are doing and why.
 We don’t want to mimic behaviour or structure for its own sake
Cognitive Science
Definition:
“the scientific study either of mind or of intelligence”
 Essential Questions
 What is intelligence?
 How is it possible to model it computationally?
 Takes ideas from






Psychology
Philosophy
Linguistics
Neuroscience
Artificial Intelligence / Computer Science
Maybe also minor contributions from:
 Anthropology, Sociology, Emotion studies,
Animal Cognition, Evolution
Cognitive Science
Definition:
“the scientific study either of mind or of intelligence”
 Essential Questions
 What is intelligence?
 How is it possible to model it computationally?
 Takes ideas from






Psychology
Philosophy
Linguistics
Neuroscience
Artificial Intelligence / Computer Science
Maybe also minor contributions from:
 Anthropology, Sociology, Emotion studies,
Animal Cognition, Evolution
Origins of Cognitive Science
 Psychology of the early 20th century was dominated by “behaviourism”
 Everything should be treated as a behaviour
 “…purely objective experimental branch of natural science.”
- John B. Watson
 Goal: prediction and control of behaviour
 “Introspection forms no essential part of its methods” - John B. Watson
 Should not have to describe things in terms of “hypothetical” internals
 Such as the “mind”
 “Consciousness” not an appropriate question for scientific inquiry
 This changed around 1950s
 Partly as a result of investigations in Artificial Intelligence,
partly changing trends - Chomsky
 People started talking about
 Theories of mind
 Internal representations
 Computational procedures
 Term “Cognitive Science” born in 1973
 Came out of AI - Christopher Longuet-Higgins comment on “Lighthill report”
Cognitive Science – Information Processing
 Cognitive Science views the mind as an information processing system
 This is also called the computational view
 From this perspective: a human mind’s activity consists of





Receive information
Store information
Retrieve information
Transmit information
Transform information
 Example: a musician improvising





Listen to many tunes
Remember them
Find similarities
Come up with rules that say what sounds good together
Use those rules in real-time while playing
Understanding Information Processing Systems
1.
We attribute non-behavioural properties to the system




2.
Representation: information in the system can represent real things


o
o
3.
We say that it has a purpose, goals or desires
We say that it has internal beliefs and knowledge and competence
We attribute meaning to its external behaviour and internal information
We treat other humans like this all the time, call it folk psychology
For example: symbols could represent objects and relationships
This would allow a clear separation of what and how
Alternatively: it could be a messy representation
what and how tangled together
It has procedures for processing information





We call these procedures algorithms in computer speak
Describes how it does what it does
A clear set of steps that need to be followed
Like the recipe for making a cake
Like the instructions for long multiplication