Download Artificial general intelligence

Artificial general intelligence
(AGI)
“building thinking machines”
© 2007 General Intelligence Research Group
AGI vs “narrow” AI
• examples of narrow AI:
– face recognition
– spam filtering
– data mining
– Google
Common objections
•
•
•
•
•
•
intelligence is not well-defined
it’s too hard
computing power is not there yet
no unifying theory of AI
we don’t understand the brain
etc…
All this is bull shit!
AI pioneers
• Alan Turing (1912-1954)
• John von Neumann (1903-1957)
John McCarthy (1927-) Marvin Minsky (1927-)
Implications of AGI
• complete automation
• ethical issues
• “Technological Singularity”
Vernor Vinge (1944-)
Ray Kurzweil (1948-)
Representative AGI projects
•
•
•
•
•
•
•
•
Cyc
Soar, ACT-R
Polyscheme
LIDA
SNePS
AIXI
OSCAR
NARS
•
•
•
•
•
•
Novamente
Cog
CAM-Brain
HTM
SAIL
a2i2
and many more….
(listed by Pei Wang)
Cyc
• most-funded AI project in
history ($10s of millions)
• based on predicate logic
• complete ontology
• millions of facts, concepts
Doug Lenat (1950-)
Soar
• Allen Newell (1927-1992)
John E Laird
• based on production rules & rete algorithm
• learning – “chunking”
Novamente
• Ben Goertzel (1966-)
• probabilistic logic based on
“uncertain probabilities”
• graph-based
knowledge representation
• genetic algorithms
for learning
• robot living in virtual reality
• 2007 book:
Artificial General Intelligence
NARS
•
•
•
•
Non-Axiomatic Reasoning System
Pei Wang
can learn from experience
work with insufficient
knowledge and resources
• unified cognition:
reasoning, learning,
planning, etc…
• 2006 book:
Rigid Flexibility
SNePS
•
•
•
•
Semantic Network Processing System
Stuart C Shapiro
extends first-order logic
belief revision /
assumption-based
truth maintenance
• natural language
understanding
AIXI
• Marcus Hutter
• highly abstract
• based on Kolmogorov
complexity theory
• KC is incomputable
• learning may take
forever!
Polyscheme
• Nick Cassimatis
• integrates multiple methods
of representation, reasoning,
and problem-solving
• procedural substrate
• not “one model”
CAM-brain
•
•
•
•
•
Hugo de Garis (1947-)
neural network
evolvable hardware
cellular automata
currently at
Wuhan University
SAIL
• John Weng
 neural network-based
• navigates and learns from environment
autonomously
Jeff Hawkins (1957-)
• inventor of “Palm Pilot”
• founded Redwood
Neuroscience Institute
• 2005 book:
On Intelligence
• HTM (Hierarchical
Temporal Memory)
• neurally-inspired
Braininspired
AI
visual cortex
Wiring of 6-layer cortex
Neurally-inspired AI
• feedforward neural network
• Jeff Hawkins’ approach
• problem:
invariant recognition:
translation,
rotation,
scaling
Statistical learning
•
•
•
•
takes place in a vector space
requires many examples
target = manifold
difficult to learn
concepts with
variables
eg:
On(apple,table),
On(car,road), etc…
“Spatial” pattern recognition
ANN,
SVM,
PCA,
Clustering,
etc…
Logic-based vision
• visual features  logical representation
Logical-vision example
Quadrilateral() :∃e1:edge
∃e2:edge
∃e3:edge
∃e4:edge
∃v1:vertex
∃v2:vertex
∃v3:vertex
∃v4:vertex
Connects(e1,v1,v2) ^
Connects(e2,v2,v3) ^
Connects(e3,v3,v4) ^
Connects(e4,v4,v1)
“Syntactic” pattern recognition
predicate logic formula:
 featurei relation1(feature1, feature2, …) ^
relation2(feature3, feature4, …) ^
…
Spatial interpretation?
Logic-based AI
Avoid reinventing the wheel!
Logic-based AI
• first-order predicate logic (Prolog)
• common objections:
“brittle”
“rigid”
“binary” “not numerical”
“just a theorem prover”
• probabilistic / fuzzy logic
• non-deductive mechanisms
eg: abduction, induction
Modules
•
•
•
•
•
•
•
perception (eg vision)
pattern recognition
inference
natural language
learning
truth maintenance
planning
Architecture
Pattern recognition
• “neural characteristics” “soft computing”
• Prolog:
chair(X) :- leg1, leg2, leg3, leg4, seat, back,
horizontal(seat), vertical(back),...
leg1
leg2
leg3
leg4
… ...
fuzzy values
chair
Pattern recognition
– “chairs”
more chairs
still more chairs
Pattern recognition
• how humans recognize “concepts”?
• [Michalski 1989] “2-tiered approach”
rule-based vs instance-based
• Prolog:
chair :- chair1
chair :- chair2
chair :- chair3
...
chair :- (rule for general chair)
Probabilistic logic
• classical resolution [JA Robinson 1965]
• Bayesian networks [eg Judea Pearl]
Resolution algorithm
• try to resolve formulas repeatedly until no
more can be resolved
PVQ
~P V R
QVR
Bayesian network
• propositional
First-order Bayes net
• [Peter Norvig & Stuart • [David Poole 2003]
Russell 2003]
• [Manfred Jaeger 1997]
• [Kathryn B Laskey 2006]
etc…
BeltStatus(belt)
RoomTemp(room)
EngineStatus(machine)
Bayesian vs classical logic
• Conditional Probability Table (CPT)
classical
Bayesian
(A ^ B)
A
B
C
A
T
T
F
F
B
T
F
T
F
C
1.0
0.0
0.0
0.0
A
T
T
F
F
B
T
F
T
F
C
T
F
F
F
KBMC
• Knowledge-Based Model Construction
• [Wellman et al 1992]
• generate Bayesian networks “on-the-fly” to
answer specific queries
KB
KBMC example
KBMC example
Belief bases vs belief sets
• belief set = Cn( belief base )
set of consequences
• belief sets are too large to manipulate
• for AGI, must use belief base
Fuzzy logic
• “John’s girl friend is probably very pretty”
• fuzziness  probability
• Lotfi Zadeh (1921-)
1965 fuzzy sets
1973 fuzzy logic
Confidence
• Example:
A. 10 girls,
5 have long hair
B. 1000 girls, 500 have long hair
p = 0.5
but A and B are not the same
B has higher confidence
• used in Pei Wang’s NARS logic
Probabilistic-fuzzy inference
( P, C, Z )n  ( P, C, Z )
x1
x2
...
probability
confidence
fuzziness
Ps and Zs can be point-valued or interval-valued
Probability intervals
• Example:
marry  fool [p = 0.8]
! marry  loser [p = 0.7]
p( fool V loser ) =
0.7 + 0.1 * p( marry )
 [ 0.7, 0.8 ]
unknown
Conditional probability table (CPT)
• All permutations of fuzzy values
• Or, store in a
“distribution-free”
format?
a
z1
z2
z3
z4
…
b
…
…
…
…
…
C
(P1, C1, Z1)
(P2, C2, Z2)
(P3, C3, Z3)
(P4, C4, Z4)
…
“Rules of thought”
• “If cats have claws,
and Juney is a cat,
then Juney has claws.”
•  P,x,y P(x) ^ isa(y,x)  P(y)
• modus ponens: P, P  Q  Q
• syllogisms
reasoning
deduction
induction
retroduction
abduction
Abduction
• “finding explanations”
• eg glass is wet  it was raining
• algorithm:
reverse of deduction (eg resolution)
• very high complexity
0
(within the arithmetical complexity class  2 )
Abduction algorithm
Induction vs abduction
• abduction: answer = ground literals
eg “grass is wet”  “it was raining”
• induction: answer = general formulae
eg daughter(X,Y) :- father(Y,X) ^ female(Y)
Induction
• learning general patterns statistically
• ILP (Inductive Logic Programming)
[Stephen Muggleton]
1990s
Induction example
Given data:
male(mary) = false
female(mary) = true
mother(mary, louise) = true
father(mary, bob) = true
daughter(bob, mary) = true
 daughter(X,Y) :- father(Y,X) ^ female(Y)
Natural language
• unifying framework
• language = knowledge-based inference
• [Jerry R Hobbs] “Abduction as Interpretation”
eg “The Boston office called.”
• “apple pie” “door knob” “street hawker”
• all we need is a lot of rules
• can inductively learn the rules
Belief maintenance
•
•
•
•
Truth Maintenance System (TMS)
belief revision
to attain “consistency”
avoid “cognitive dissonance”
Truth maintenance
justifications
Belief revision
• “Epistemic entrenchment”
• [Mary-Anne Williams
1995]
Belief Base
…
…
6
5
4
entrenchment
ranking
3
2
1
0
“Click” feeling
Perhaps an effect of successful inference,
abduction, or belief revision?
Paraconsistency
• holding 2 contradictory beliefs in the
knowledge base at the same time
Associative memory
•
•
•
•
knowledge base = database
special indexing to allow associative recall
hard disk = long-term memory
RAM = working memory
Planning
Conclusions
• “neural” is problematic
• “blank slate” is problematic
• “logic-based” is very promising
Agenda
for Logic-based AI
1. design probabilistic-fuzzy logic
2. develop algorithms for:
– abduction
– belief maintenance
3. acquire common sense knowledge
“Web 2.0”-style collaboration
•
•
•
•
branching
voting
commercial
problem: too few members
Thank you
•
[Aliseda 2006] Abductive Reasoning: Logical Investigations into Discovery and
Explanation. Synthese Library Series vol 330, Springer
•
[Antoniou 1997] Nonmonotonic Reasoning, MIT Press
•
[Cussens 2001] Integrating probabilistic and logical reasoning. In David
Corfield and Jon Williamson eds Foundations of Bayesianism, volume 24 of
Applied Logic Series, pages 241-260. Kluwer, Dordrecht
[2000 Flach & Kakas eds] Induction and Abduction, Springer Applied Logic
Series #18
[Haddawy 1994] Generating Bayesian networks from probability logic
knowledge, in Proceedings of the 10th conference on uncertainty in AI, 1994.
[Hobbs 200?] Abduction as Interpretation
[Jaeger 1997] Relational Bayesian networks. In Proceedings of the 13th Annual
Conference on Uncertainty in AI (UAI-97), p266-273, San Francisco, CA, 1997,
Morgan Kaufman Publishers
[Kakas, Kowalski, Toni 1992] Abductive Logic Programming, Journal of Logic
and Computation 2(6):719-770.
http://citeseer.ist.psu.edu/kakas93abductive.html
[Laskey 2006] MEBN: A logic for open-world probabilistic reasoning. GMU C4I
Center Technical Report C4I-06-01. George Mason Univ, USA.
[Milch & Russell 2007] First-Order Probabilistic Languages: Into the Unknown In ILP:
•
•
•
•
•
•
•
Proceedings of the 16th International Conference on Inductive Logic Programming.
Berlin: Springer
• [Michalski 1989] Two-tiered concept meaning, inferential matching, and
conceptual cohesiveness. In Vosniadou & Ortony eds, Similarity and
analogical reasoning, p122-145. Cambridge University Press, New York.
• [Muggleton 1996] Stochastic logic programs. In de Raedt, ed, Advances in
Inductive Logic Programming, p254-264, IOS Press 1996.
• [Ngo, Haddawy, & Helwig 1995] A theoretical framework for contextsensitive temporal probability model construction with application to plan
projection. In Proceedings of the 11th Annual Conference on Uncertainty
in Artificial Intelligence (UAI-95), p419-426, Montreal, Quebec, Canada.
• [Norvig & Russell 2003] Artificial Intelligence: A Modern Approach,
Prentice Hall.
• [Poole 1993] Probabilistic horn abduction and Bayesian networks, Artificial
Intelligence, 64(1), 81-129, 1993
• [Poole 2003] First-order probabilistic inference, Proc, IJCAI-03, Acapulco,
August 2003, p985-991
• [Wellman, Breese, Goldman 1992] From knowledge bases to decision
models. Knowledge Engineering Review 7(1): 35-52
• [Williams 1995] Changing nonmonotonic reasoning inference relations, in
Proceedings of the second world conference on the fundamentals of AI,
469-482, Ankgor, Paris, 1995