Download evolutionary computation

Data Evolution in
General Formal Problem Statement
A General Approach to Evolutionary Problem Solving,
Implemented in the Computer Language GENETICA
Background
CLASSICAL PROBLEM SOLVING
According to the general notion of "problem solving" there is a set of states (blue dots) …
CLASSICAL PROBLEM SOLVING
… within which is defined a subset of states (red dots) having some desired properties.
Search Space
The former set—structured by an
appropriate search procedure—is
called "the search space"…
Solution Space
… while the subset is
called "the solution space".
The objective of a problem solving procedure is to reach an element of the solution space, given the search space.
CLASSICAL PROBLEM SOLVING
Confirmation
This is the case of "confirmation" problems,
i.e. problems where a state either belongs
to the solution space or not.
CLASSICAL PROBLEM SOLVING
In another class of problems, membership in the solution space is considered as quantitative: all the elements of the
search space belong to the solution space, each one having a different membership value which expresses the
acceptability of the element as a solution. Then the problem becomes one of finding a state having as high
membership value as possible.
Optimization
This is the case of the
optimization problems
Although the current Artificial Intelligence (AI) research has focused on different issues, such as "behavioral"
intelligence, adaptability, cellular or connectionist systems, evolutionary systems etc, the classical notion of "problem
solving" remains valid as—in its most abstract and general form—it is encountered in virtually any AI system. For
instance, adaptability can be conceived as convergence to goal-states, while "behaviors" can be viewed as states
within a search space.
Due to the critical role of "problem solving" in various AI methods, the development of AI systems can be effectively
supported by general problem solving tools. Such tools should offer both expressive and creative power. The former
has been traditionally dealt with by means of formal languages, within the field of symbolic AI, while the latter—viewed
as performance in searching within complex and "unpredictable" search spaces—has been extensively investigated
within the field of Evolutionary Computation (EC). As any AI system, even if it is not "algorithmic" in nature, is
subjected to a formal description (otherwise it could not be developed in the computer), while several forms of
"creative" behavior could be based on (or enhanced by) evolutionary dynamics, the development of AI systems could
be effectively supported by tools that integrate formal expression and evolution methods. According to this direction,
the computer language GENETICA is proposed here as a general tool for the development of AI systems.
First we are going to identify some key-features of the original approaches to "expressive" and "creative" problem
solving, respectively represented by symbolic AI and EC methods.
SYMBOLIC ARTIFICIAL INTELLIGENCE
A major class of problem solving systems falls into the framework of symbolic Artificial Intelligence
Physical symbol structures
Classical AI problem-solving systems are based on "physical symbol structures".
Symbols
Symbolic Language
Relations
Processes
Search
Such systems include symbols, relations and "processes",
where symbols and relations form a symbolic language, while
AI languages include predicate logic,
which makes
Thepossible
strong
to use
is any
expressive
expression
power,
which
concerning
allows countable
"processes"
generate
thepoint
elements
oflogical
the search
space.
classes in the problem representation. Given this
reduction
representation,
of the search
AI problem
spacesolvers
through
scan
the the
incorporation
search space
of
until a solution is reached. A solution is a "symbol
problem
structure"
specific
that confirms
knowledge
theinproblem
the problem
statement.
formulation.
Expressive power
Exhaustive Search : Complexity limitations
A weak point of classical AI problem-solving
However,
is that
thisexhaustive
approach search
reducessuffers
"creativity",
from the
i.e."combinatorial
the potential of the AI
Low
Creativity
explosion" effect which could make solution
system
practically
to create
innovative
solutions
unpredictable
in cannot
terms of
existing
Furthermore,
AI inefficient.
systems
based
on symbol
structures
cope
with
problem specific
knowledge.
optimization
problems
where the search space is continuous.
EVOLUTIONARY COMPUTATION
Other problem solving systems include optimization methods based on either numerical
techniques or stochastic search. Within this framework, Evolutionary Computation systems
exhibit a remarkable combination of generality of application and effectiveness.
Fitness evaluation
Evolutionary problem solving methods evolve elements of the search space aiming to the
optimization of "fitness": a magnitude representing the "desirable attribute" of a solution.
in a population of individuals
A "population" of individuals, i.e. formal representations of potential solutions, is evolved…
subjected to genetic operations
…as "genetic operations" affect individuals, while emerging high-fitness individuals
substitute lower-fitness ones in the population.
High computational efficiency
The strong point of evolutionary problem-solving is computational efficiency: evolution may
result in solutions, in cases where this would be practically inefficient by exhaustive search.
High creativity
Computationaleven
Unfortunately,
efficiency
if the supports
implementation
creativity,
of problem-specific
as the need for search
knowledge
space
is reduction
desirable (in
order to problem-specific
through
achieve a more accurate
knowledge
problem
becomes
statement
less intensive.
or a more effective solution
procedure) it is often very hard to be attained.
This
is because
problem statement
is not supported
by afor
universal
formal language
Furthermore,
thethe
quantitative
nature of "fitness"
makes hard
these methods
to cope (such as
predicate
logic).
As a consecuence,
with general
confirmation
problems. evolutionary methods tend to be highly "problem-specific".
Low expressive power
EVOLUTIONARY COMPUTATION
Major fields of investigation within the framework of Evolutionary Computation are
Genetic Algorithms (GAs) and Genetic Programming (GP)
Genetic Algorithms
Binary strings
Classical GAs represent to-be-evolved objects as fixed-length
binary strings, the latter viewed as genotypes.
EVOLUTIONARY COMPUTATION
Consider a population of genotypes, each one presented here as a colored dot.
Colors represent fitness values: higher fitness genotypes (biased to red) are shown at the top of
the oval shape, while lower fitness genotypes (biased to blue) are shown at the bottom.
11001010
10010001
Each genotype is a binary string
110
10010110
A new genotype could emerge by
the alteration of a limited number of
bits within a string.
Such alterations constitute "genetic
operations" on genotypes, known
as "mutations".
EVOLUTIONARY COMPUTATION
11000001
10011010
Homologous bits of different
genotypes can be interchanged.
This is a genetic operation,
named "crossover", which
results in new genotypes.
EVOLUTIONARY COMPUTATION
The application of either
"crossover" or "mutation" on high
fitnesss genotypes may result in
higher fitness genotypes.
As thisgenotypes
Such
…substituting
processlow-fitness
isare
repeated,
introduced
the
in the population…
genotypes.
population
is successively enriched
in higher-fitness genotypes.
EVOLUTIONARY COMPUTATION
Genetic Algorithms
Binary strings
As binary strings have very limited expressive power…
Tree-structured
genotypes
…the expressive power of genetic methods has been enhanced
by more sophisticated representations, such as tree-structured
genotypes, which reflect domain-specific knowledge.
Genetic Programming
Computer programs
Genetic Programming (GP) considers computer programs as the
objects to be evolved. This largely improves expressiveness, as
the expressive power of a computer language allows to
represent virtually anything as a computer program.
EVOLUTIONARY COMPUTATION
Consider the syntax structure of a computer program (schematically represented here as a predicate logic expression).
((P AND Q) OR (
(P AND Q) OR (
x (R))) AND (NOT (F))
x (R))
P AND Q
P
Q
NOT (F)
x (R)
R
F
EVOLUTIONARY COMPUTATION
((P AND Q) OR (
x (R))) AND (NOT (F))
(P OR Q) AND R
NOT (F)
P OR Q
P
Q
R
F
EVOLUTIONARY COMPUTATION
Parts of this structure can be interchanged with syntactically compatible parts of the syntax structure
GP program.
methods use
operations
to an
evolve
a population
of computeroperation.
programs,
of another computer
Thissuch
can be
viewed as
extension
of the "crossover"
where
thestructure
fitness ofcan
each
is calculated
by of
thethe
program
execution.
Partial regeneration
of the
beprogram
viewed as
an extension
"mutation"
operation.
((P OR Q) AND R) AND (NOT (F))
(P OR Q) AND R
P OR Q
P
Q
NOT (F)
R
F
EVOLUTIONARY COMPUTATION
Although the programming language of a GP application makes possible high expressiveness in solution
representation, GP does not provide an equally effective representation of the problem to be solved: in classical
GP the user is only allowed to define the elements of the programming language (i.e. the function set and the
terminal set), the fitness function, and potentially some aspects of the program architecture. Implementation of
problem-specific knowledge has been attempted through specifications of the solution representation language,
which
in highly problem-specific
methods.
A classresults
of evolutionary
methods—someGP
of them
using GP techniques—consider abstract objects, which reflect
Evolution in formal logic
the
syntax
of a fixed
predicate logic expression,
asquantitative
the objects nature
to be evolved.
The which
logicalseverely
expression
can
be
Another
limitation
of expressiveness
concerns the
of "fitness",
limits
the
viewed as of
a problem
whiletothe
objects to problems.
be evolved can be viewed as solution instances. In this case
possibility
a generalstatement,
GP approach
confirmation
the problem representation is supported by the expressive power of predicate logic.
Evolutionary machine learning
Evolutionary abstract interpretations
Evolutionary theorem proving
Applications have been developed within the fields of machine learning, abstract interpretation and theorem
proving. These applications exhibit high expressiveness in problem representation, while they can cope with
confirmation problems. However, they are not appropriate for general problem solving tasks, as their application
domain is limited, while they do not exhibit creativity, as they are strictly based on existing knowledge.
SOME REMARKS ON COMPUTER
AIDED PROBLEM SOLVING
Under a general conceptualization, problem-solving in the computer can be viewed as a process of
construction and evaluation of data structures during the program execution. Construction of different data
structures creates the search space while evaluation selects the elements of the solution space.
Elementary decisions
Selection
method
Evaluation
method
Countable classes
Deterministic
Logical
Continuous classes
Stochastic
Numerical
Selection in
The elementary decisions leading to data
Selection may be either deterministic
Data structures may be
construction concern selection of an
(e.g. exhaustive enumeration of
evaluated either in a
element of either a countable class (which
potential solutions in AI systems) or
Boolean manner
A general application of evolutionary heuristics in problem solving could be achieved if stochastic elementary decisions within
is represented either as a list, an array or
stochastic (e.g. random selections
(confirmation criteria) or
(arbitrarily complex) formal expressions could be treated as genes, while either logical or numerical evaluation of (arbitrarily
an integer interval) or a continuous class
encountered in stochastic optimization
in a numerical manner
complex) data structures—the latter viewed as potential solutions—could be formulated as a fitness function.
(which is represented as a real interval).
or genetic methods).
(optimization criteria).
Continued in Section_2.PPS