Download The Big Test - Rensselaer Polytechnic Institute

Selmer Bringsjord, Bettina Schimanski
Dept. of Cognitive Science / Dept. of Computer Science
RAIR (Rensselaer Artificial Intelligence and Reasoning) Lab
Rensselaer Polytechnic Institute
Troy, NY 12180 USA
[email protected] ; [email protected]
www.rpi.edu/~brings ; www.bettinaschimanski.com
www.cs.rpi.edu/~schimb/peri
IJCAI 2003 Conference
August 15, 2003
Can you
solve this
puzzle?
S/S
Could you solve it in under one
second?
S/S
AnAAnswer
to: What
is AI?
New Kind
of AI
Assume the ‘A’ part isn’t the problem: we know what
an artifact is.
There is no agreement on what human intelligence is.
- Two notorious conferences. See The g Factor.
But we can agree that one great success story of
psychology is testing, and prediction on the basis of
it. (The Big Test)
Psychometric AI offers a simple but radical answer:
– (Naïve definition): AI is the field devoted to building
intelligent artificial agents, i.e., agents capable of solid
performance on intelligence tests.
S/B
Don’t confuse this with: “Some human is
intelligent…”
Psychometric AI is not completely
new
Early roots: PAI was
implicitly entertained
with Evan’s 1968
ANALOGY program
However, PAI can not
be based on tests
which consist solely of
geometric analogies
B/B
Objection 1
“If one were offered a machine purported to be intelligent,
what would be an appropriate method of evaluating this
claim? The most obvious approach might be to give the
machine an IQ test … However, [good performance on
tasks seen in IQ tests would not] be completely satisfactory
because the machine would have to be specially prepared
for any specific task that it was asked to perform. The task
could not be described to the machine in a normal
conversation (verbal or written) if the specific nature of the
task was not already programmed into the machine. Such
considerations led many people to believe that the ability to
communicate freely using some form of natural language is
an essential attribute of an intelligent entity.” (Fischler &
Firschein 1990, p. 12)
B/B
Our Rebuttal
As we’ve already implied, PERI will be able to
solve many puzzles, without being preengineered.
Have built theorem prover-based agents to
infallibly crack not only geometric analogies, but
RPM items they have never seen before
Humans will communicate with PERI in natural
language
Tests of mental ability = much more than IQ
tests
B/S
Intelligence Tests: Narrow vs. Broad
Spearman’s
view of intelligence
S/S
Thurstone’s view of
intelligence
Narrow Test:
Correct!
S/B
RPM
=========== start of search ===========
given clause #1: (wt=2) 10 [] R1(a11).
given clause #2: (wt=2) 11 [] R1(a12).
given clause #3: (wt=2) 12 [] R1(a13).
...
given clause #4: (wt=2) 13 [] R2(a21).
given clause #278: (wt=16) 287
[para_into,64.3.1,3.3.1] R2(x)| -R3(a23)|
-EmptyBar(y)| -R3(x)| -EmptyBar(x)| -T(a23)| R3(y)| -T(y).
given clause #279: (wt=16) 288
[para_into,65.3.1,8.3.1] R2(x)| -R3(a23)|
-StripedBar(y)| -R3(x)| -StripedBar(x)| EmptyBar(a23)| -R3(y)|
-EmptyBar(y).
Search stopped by max_seconds option.
============ end of search ============
Broad Test: WAIS
Wechsler Adult Intelligent Scale
WAIS includes many sub-tests
B/B
Broad Test: WAIS
Wechsler Adult Intelligent Scale
WAIS includes many sub-tests
The Comprehension sub-test is so difficult
it could be a motivator for the CYC dream
-Deals with ordinary conversation
Ex: “Why are the tires of
automobiles made of rubber,
rather than, say, plastic?”
B/B
Broad Test: WAIS
Wechsler Adult Intelligent Scale
WAIS includes many sub-tests
Picture Arrangement
- Deals with many mental facets
B/B
Broad Test: WAIS
Wechsler Adult Intelligent Scale
WAIS includes many sub-tests
Block Design
- PERI has already
cracked this
B/B
Broad Test: WAIS
Wechsler Adult Intelligent Scale
WAIS includes many sub-tests
PERI should specifically solve all of the WAIS
- From there, we move to all other established tests
B/B
Improved Definition of PAI
Psychometric AI is the field devoted to
building information-processing entities
capable of at least solid performance on
all established, validated tests of
intelligence and mental ability, a class of
tests that includes IQ tests, tests of
reasoning, of creativity, mechanical ability,
and so on.
B/B
PERI
Pscyhometric Experimental Robotic Intelligence
Scorbot-ER IX
Sony B&W XC55 Video
Camera
Cognex MVS-8100M
Frame Grabber
Dragon Naturally
Speaking Software
NL (CARMEL & RealPro?)
BH8-260 BarrettHand
Dexterous 3-Finger
Grasper System
B/B
Block Design Task
(Not actual blocks from WAIS)
Place blocks together so that all edges match in color
PERI can solve it in under one second of CPU time and not just ones
from the WAIS but any Block Design task given to it
B/B
(Blocks courtesy of Binary Arts Corporation)
PERI in action
B/S
Objection 2
Isn’t PAI is idiosyncratic?
– PAI is a generalization of a longstanding
answer to the “What is AI?” question
– Is the answer that appeals to the TT and its
relatives
– AI is the field devoted to building artificial
entities able to pass TTT
– Tests are clean and crisp – have determinate
starts and ends, yielding concrete verdicts
– PAI extends and clarifies Turing’s approach
S/S
Objections 3 & 4
Don’t TT and TTT subsume PAI?
– PAI uses Divide and Conquer
– All topics are fair game in TT and TTT
– Both are hugely ambitious goals
AI has Applications…?
– An agent able to pass these tests will have
the capacity to provide the desired
applications
S/S
Objections 5 & 6
But PAI will only tap logicist AI?
– PAI will unite logicist and sub-symbolic
approaches
But AI researchers don’t do PAI?
– PAI is descriptive and prescriptive
– Developers will inevitably modularize their
challenges, building sub-agents and negotiate
tests associated with these modules
– AI applications probably themselves
constitute miniature tests
S/S
Objections 7 & 8
Behaving like humans vs. doing well on IQ tests
– Create “smart” agents and then “de-smart” them
later
Performing well on “any possibly existing”
psychometric test makes PAI ill-defined
– The challenge is to create an agent with the
capacity to crack tests he has never seen before
– These tests must be validated via ordinary
statistico-mathematical standards used in
psychometrics
S/B
Narratological Reasoning &
Thwarting Terrorism…
Terrorists struggle to make stories real.
Their behavior can be anticipated, and thus
thwarted.
We need computers that can imagine future
events in a (twisted) narrative.
Threat anticipation (Picture Arrangement)
B/B
Predictive power done quickly ….
Picture Arrangement
B/B
Picture Arrangement
B/B
B/S
S/S
RAIR Web and R&D
Advanced Synthetic
Characters
MARMML
PERI
Savant
PAI
Slate
CDs
Super Teaching
Future of PAI / Questions?
Efforts to crack IQ tests should lead to
systems able to generate new items for
other established tests, and new tests
May help determine what tests a machine
can solve, and those it has more trouble
than a human = security
Once successful: Is PERI a genius or does
human intelligence go well beyond what can
be tested for?
Divides Strong and Weak AI
S/S