Download INTRODUCTION TO COMPUTATIONAL INTELLIGENCE

INTRODUCTION TO
COMPUTATIONAL INTELLIGENCE
Lin Shang
Dept. of Computer Science and Technology
Nanjing University
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
Introduction to
Computational Intelligence
n
From AI to CI
n
n
n
n
n
n
n
What is intelligence ?
What is AI ?
What is CI ?
Different views of Computational Intelligence
History of Computational Intelligence
Computational Intelligence Paradigms
Conferences and Journals
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
From AI to CI
n
What is intelligence?
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
From AI to CI
n
Characteristics of Intelligence
n
Perception
n
Action
n
Reasoning
n
Problem-solving
n
LearningandAdaptation
n
Sociality
n
Creativity
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
From AI to CI
n
What is AI ?
AI prehistory
n
Philosophy
n
Mathematics
n
Economics
Logic, methods of reasoning, mind as physical system, foundations
of learning, language, rationality
Formal representation and proof algorithms, computation,
(un)decidability (in)tractability, probability
utility, decision theory
n
Neuroscience
physical substrate for mental activity
n
Psychology
phenomena of perception and motor control, experimental techniques
n
n
Computer
building fast computers
Science
Control theory design systems that maximize an objective function over time
n
Linguistics
knowledge representation, grammar
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
From AI to CI
Abridged history of AI
n
n
n
n
n
n
n
n
n
n
n
n
1943
1950
1956
1952-69
1950s
McCulloch & Pitts: Boolean circuit model of brain
Turing's "Computing Machinery and Intelligence"
Dartmouth meeting: "Artificial Intelligence" adopted
Great expectations
Early AI programs, including Samuel's checkers program, Newell & Simon's
Logic Theorist, Gelernter's Geometry Engine
1965
Robinson's complete algorithm for logical reasoning
1966-73 AI discovers computational complexity, Neural network research almost
disappears
1969-79 Early development of knowledge-based systems
1980AI becomes an industry
1986Neural networks return to popularity
1990Novelle AI and intelligent agents
1995Evolutionary Computation, Swarm Intelligence, and New Generation
Computers
16/2/29
AI:Introduction
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
6
Spring
2016
1943:McCulloch & Pitts: Boolean circuit
model of brain
MP
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
1950: Turing test
n
When does a system behave intelligently?
n Turing (1950) Computing Machinery and Intelligence
n Operational test of intelligence: imitation game
n
Requires the collaboration of major components of AI:
knowledge, reasoning, language understanding,
learning, …
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
2011:Cleverbot
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
1956:Dartmouth meeting: "Artificial
Intelligence" adopted
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
Ideas for AI
n
Learning
n
“child machine”
n
Symbolic AI
n
Connectionism
n
Nouvelle AI
n
Evolutionary Computation
n
n
“artificial life”
Computational Swarm Intelligence
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
Symbolic AI
Physical Symbol System Hypothesis of Newell and Simon
n
n
the processing of structures of symbols by a digital computer is
sufficient to produce artificial intelligence
n
the processing of structures of symbols by the human
brain is
the basis of human intelligence
n
it remains an open question whether the Physical Symbol System
Hypothesis is true or false
n
Top-down strategy
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
n
n
Problem-sloving à Expert System à Knowledge
Engineering
- Search, Representation, Reasoning
Problems
- Frame problem (CYC, Go…..)
- Substituting large amounts of
computation for understanding
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
Connectionism
n
n
The mechanisms of brains are very different in
detail from those in computers
how brains work? à Bottom-up strategy
Natural Neural
Network
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
Connectionism
n
A brief history
M-P neuron (McCulloch & Pitts) à Perceptron
(Rosenblatt) à Hopfield Model, B-P Learning Method
(Rumelhart & McClelland) à…Deep Learning
n
Applications
Recognition, Vision, Business, Medical, …….
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
Evolutionary Computation
n
Biological evolution
To produce an enormous variety of
living organisms closely suited to
different sets of needs in different
environments.
n
Simulated evolution
By modeling those processes of
biological evolution on computers, it
turns out that we can sometimes get
the computers to evolve solutions to
problems.
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
n
Genetic Algorithm
Use strings of symbols to encode solutions to problems,
like strings of molecules in DNA. Transforming and
recombining portions of strings enables an evolutionary
computation to search for good solutions, partly analogous
to biological evolution.
n
Genetic Programming
Extends these ideas to automatic programming by using
structures which are better suited to the problem than
strings are.
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
n
Evolutionary Strategy
Use natural problem-dependent representations, and primarily mutation
and selection as search operators. Mutation is normally performed by
adding a normally distributed random value to each vector component.
The step size or mutation strength is often governed by selfadaptation. The selection in evolution strategies is deterministic and
only based on the fitness rankings, not on the actual fitness values.
n
Evolutionary Programming
Harder to distinguish from evolutionary strategies. Its main variation
operator is mutation; members of the population are viewed as part
of a specific species rather than members of the same species
therefore each parent generates an offspring
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
Artificial Life (Alife)
n
Artificial Life is the study of man-made systems that
exhibit behaviors characteristic of natural living
systems. It complements the traditional biological
sciences concerned with the analysis of living
organisms by attempting to synthesize life-like behaviors
within computers and other artificial media. By
extending the empirical foundation upon which
biology is based beyond the carbon-chain life that has
evolved on Earth, Artificial Life can contribute to
theoretical biology by locating life-as-we-know-it within
the larger picture of life-as-it-could-be."
Chris Langton (in Proc. of first Alife conference)
Ref:
http://www.cogs.susx.ac.uk/users/inmanh/easy/alife09/lectures.html
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
Artificial Life and Evolutionary
Life, as it is… and might have been
Origin of Life
FromVirgil
Griffith,
Google Tech
Talk- 2007
Today
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
Example: Forming body plans with evolution
n
n
n
n
n
Node specifies part type, joint, and range
of movement
Edges specify the joints between parts
Population?
n Graphs of nodes and edges
Selection?
n Ability to perform some task (walking,
jumping, etc.)
Mutation?
n Node types change/new nodes
grafted on
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
Computational Swarm Intelligence
n
Intelligence is often considered a property of individuals.
Are we social because we are intelligent or are we
intelligent because we are social?
n
- Intelligence can emerge from social interaction.
Emergent behaviour – when a group behaves in ways that were
not ”programmed” into its members.
n
Swarm intelligence
n
- simulated social interaction
- emergent collective intelligence of groups of simple agents
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
Computational Tools
n
Multi-Agent Systems
- a system composed of multiple interacting intelligent agents.
- application including computer games, networks, transportation,
logistics, and etc.
n
Ant Colony Optimization
- 1991 (Dorigo)
- mostly for combinatorial optimization
n
Particle Swarm Optimization
- 1995 (Kennedy & Eberhart)
- more general optimization technique
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
Fuzzy Systems
n
Zadeh 1965
Fuzzy Sets
n
Fuzzy Logic
n
Fuzzy Systems
n
Successfully applied to control systems, home appliances,…
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
Computational Intelligence
•CI has been based roughly on Darwinian theory and biological
analogies.(….)
•Artificial Intelligence -----Computational Intelligence
•The following chapters go into this in more detail
(Neural Network, Evolutionary Computing, Fuzzy Systems)
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
History of Computational Intelligence
n
n
n
n
n
First use of the term (in its current context) by James
Bezdek in 1992
First IEEE World Congress on Computational Intelligence in
Orlando in 1994
First CI text in 1996
Second IEEE World Congress on CI in Anchorage in 1998
Subsequent World Congresses in Hawaii (2002),
Vancouver (2006)…next in Hong Kong (2008)…Beijing
(2014)
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
Historical View of Computational
Intelligence
n
n
“Computational Intelligence” was used in the title of a
journal in Canada starting in 1980s, but not meaning
what we now mean by the term
First paper using term published by Bezdek in 1992 in
Int. Jour. Approximate Reasoning.
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
Bezdek’s 1994 Definition of CI
A system is computationally intelligent when it: deals
only with numerical (low-level) data, has a pattern
recognition component, does not use knowledge in the
AI sense; and additionally, when it (begins to) exhibit (i)
computational adaptivity; (ii) computational fault
tolerance; (iii) speed approaching human-like turnaround,
and (iv) error rates that approximate human
performance.
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
Bezdek’s 1994 Definition of CI
Bezdek’s 1994 Definition of CI
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
History
n
n
n
n
n
n
n
1975
1977
1983
1992
1995
1999
2001
Genetic Algorithm by Holland
Tabu Search by Glover
Simulated Annealing by Kirkpatrick
Ant Colony Optimization by Dorigo
Particle Swar Optimizationby Kennedy and Eberhart
Predatory Search by Linhares
Bacterial Foraging Algorithm by Passino
www.themegallery.com
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
Computational Intelligence
n
n
n
n
n
n
n
Fuzzy Computing (FC)
Neural Computing(NC)
Evolutionary Computing(EC)
Probabilistic Computing(PC)
Granular Computing(GrC)
Rough Computing(RC)
…
Soft Computing
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
Computational Intelligence
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
Wecansay?
n
“CI is a subset of AI”
n
“CI is not a subset of AI, there is an overlap between AI
and CI”
n
In general, CI ≠AI
n
Motivation: “CI”à?
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
Conferences and Publications
n
n
IEEE Computational Intelligence Society
http://ieee-cis.org
Publications
6 transactions
IEEE Transactions on Neural Networks
IEEE Transactions on Fuzzy Systems
IEEE Transactions on Evolutionary Computation
IEEE Transactions on Autonomous Mental Development
IEEE Transactions on Computational Intelligence and AI in Games
IEEE/ACM Transactions on Computational Biology and Bioinformatics
IEEE Transactions on Information Forensics and Security
n 1 magazines
IEEE Computation Intelligence Magazine
n
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
IEEE Computational Intelligence Society
Prof. Gary G.Yen
President
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
Publications
（IEEE Transactions on Neural Networks）
n
n
n
science and technology of neural networks
disclose significant technical knowledge, exploratory
developments, and applications of neural networks from
biology to software to hardware.
Emphasis is on artificial neural networks.
www.themegallery.com
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
Publications
（IEEE Transactions on Fuzzy Systems）
n
n
n
the theory, design and applications of fuzzy systems
ranging form hardware to software.
Emphasis will be given to engineering applications.
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
Publications
（IEEE Transactions on Fuzzy Systems）
n
n
n
the theory, design and applications of evolutionary
computation
emphasis given to engineering systems and scientific
applications
evolutionary optimization, machine learning, intelligent
systems design, image processing and machine vision,
pattern recognition, evolutionary neurocomputing,
evolutionary fuzzy systems, applications in biomedicine and
biochemistry, robotics and control, mathematical modelling,
civil, chemecal, aeronautical, and industrial engineering
applications
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
Publications
（IEEE Transactions on Autonomous
Mental Development
n
n
co-sponsored by the IEEE Computational Intelligence Society
and the IEEE Consumer Electronics Society
technically co-sponsored by the IEEE Computer Society, the
IEEE Engineering in Medicine and Biology Society, and the
IEEE Robotics and Automation Society
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
Publications
（IEEE Transactions on Computational
Intelligence and AI in Games)
n
co-sponsored by IEEE CI Society the IEEE Computer Society,
the IEEE Consumer Electronics Society and the IEEE Sensors
Council
www.themegallery.com
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
Publications
（IEEE/ACM Transactions on
Computational Biology and Bioinformatics)
n
n
n
the algorithmic, mathematical, statistical, and computational
methods that are central to bioinformatics and
computational biology
the development and testing of optimization of biological
databases
important biological results that are obtained from the use
of these method, programs, and databases
www.themegallery.com
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
Publications
（IEEE Transactions on on Information
Forensics and Security)
n
n
aims to provide a unified focus for archival research on the
fundamental contributions and the mathematics behind
information forensics, information security, surveillance
system applications that incorporate these features
www.themegallery.com
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
Publications
（IEEE Computation Intelligence Magazine)
n
n
aims to introduce the readers to all areas of computational
intelligence design and applications
specific emphasis on applications oriented developments,
successful industrial implementations, design tools,
technology reviews, computational intelligence education,
and applied research
www.themegallery.com
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
WCCI(World Conference on CI)
www.themegallery.com
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
CEC (IEEE Congress on Evolutionay Compuation)
n
n
n
n
n
CEC
CEC
CEC
CEC
CEC
2013,
2011,
2009,
2007,
2005,
Cancun, Mexico, 20-23 June
New Orleans USA, 5-8 June
Trondheim Norway, 18-21 May
Singapore, 23-26, July
Edinburgh U.K.,2-5 Sept.
www.themegallery.com
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
People: Lotfi A. Zadeh
www.themegallery.com
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
People: Lawrence J. Fogel
www.themegallery.com
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
People: John Henry Holland
www.themegallery.com
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
People: Marco Dorigo
www.themegallery.com
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
People: Yao Xin(姚新)
www.themegallery.com
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
People: Eberhart Russell
www.themegallery.com
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016
Conclusion
n
From AI to CI
n
NN, EC and FS
n
History
n
Conferences and journals
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University
Spring
2016