Download Evolutionary Algorithms

Chapter ML: V
V. Evolutionary Algorithms
q Biological Foundations
q Computation Scheme
q Recombination
q Mutation
q Selection
q Parameter Control
ML: V-1
Evolutionary Algorithms
c BUBECK 2013
Biological Foundations
Evolution Principle
ML: V-2
q
Population of individuals
q
Genetic encoding of properties of individuals
q
Gene: stretch of DNA or RNA which describes functional proteins or
regulates cellular processes
q
Recombination and mutation
q
Survival of the fittest / natural selection (Herbert Spencer, Charles Darwin)
Evolutionary Algorithms
c BUBECK 2013
Biological Foundations
Human Genome
q
Except for the gametes (egg and sperm), human cells are diploid, i.e.
contain two homologous sets of 23 chromosomes each (one from the
mother and one from the father).
q
By meiosis, a special type of cell division, haploid gametes are produced
from a diploid cell through recombination of the parent genomes.
Daughter
Nuclei II
Daughter
Nuclei
Interphase
Meiosis I
Homologous
Chromosomes
Meiosis II
Source: NIH (public domain)
ML: V-3
Evolutionary Algorithms
c BUBECK 2013
Biological Foundations
Human Genome
ML: V-4
q
The haploid human DNA contains approximately 3 billion base pairs.
q
The unfolded haploid human DNA has a total length of about 1 meter.
q
Only about 5% of the human DNA (~ 22000 genes) is protein-encoding,
the remainder is non-coding DNA.
q
The amount of biologically functional sequences within the non-coding DNA
is unclear.
Evolutionary Algorithms
c BUBECK 2013
Computation Scheme
Evolutionary algorithms are biologically-inspired optimization algorithms based on
a stochastic generate-and-test search approach.
Definition 1 (Objective Function, Minimization / Maximization Problem)
For an arbitrary search space X and a given objective function fX : X → R, the
minimization problem with respect to fX and X is to find an element x∗ ∈ X such
that fX (x∗) ≤ fX (x) for all x ∈ X .
The analogous maximization problem is to find x∗ ∈ X with fX (x∗) ≥ fX (x) ∀x ∈ X .
ML: V-5
Evolutionary Algorithms
c BUBECK 2013
Computation Scheme
Genetic Representation
An evolutionary algorithm does not directly operate on an arbitrary search space X
(phenotype space), but on a more structured genotype space X.
Phenotype Space
Encoding /
Representation
Decoding
Genotype Space
01011010001...
Genotypes are evaluated by a fitness function f : X → R which corresponds to the
objective function fX of the decoded phenotype, i.e. f (x) = fX (decode(x)).
ML: V-6
Evolutionary Algorithms
c BUBECK 2013
Remarks:
q Typical genotype spaces are {0, 1}N (bit vectors) and RN with fixed dimension N ∈ N.
q Every element in the genotype space should represent a valid solution, which may be
difficult to achieve (Standard example: Traveling Salesman Problem).
Additional techniques exist to handle or prevent infeasible genotypes, e.g.:
– penalty functions to reduce the fitness of invalid solutions,
– preserving feasibility by operator design and initialization,
– repairing of infeasible genotypes.
ML: V-7
Evolutionary Algorithms
c BUBECK 2013
Computation Scheme
Algorithm Outline
Input:
f : X → R. Fitness function.
µ. Population size.
λ. Offspring size.
ρ. Number of parents.
Output:
x. Fittest individual of the last generation.
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
ML: V-8
FOR i = 1 to µ DO xi = choose(X) // init individuals
t = 0, P (0) = {x1 , ..., xµ }
REPEAT
P0 = ∅
FOR i = 1 to λ DO // create λ children
{x1 , ..., xρ } = select_parents(ρ, P (t), f )
x0 = recombine({x1 , ..., xρ })
x0 = mutate(x0 )
P 0 = P 0 ∪ {x0 }
ENDDO
P (t + 1) = select_population(µ, P 0 ∪ P (t), f ) // select µ survivors
t=t+1
UNTIL (t > tmax )
RETURN argmaxx∈P (t) f (x)
Evolutionary Algorithms
c BUBECK 2013
Remarks:
q This general optimization technique can be turned into a powerful machine learning
approach. Then X must be a suitable representation of the hypothesis space of the
learning problem, and f must be an appropriate performance measure.
q The fitness f is typically not given as an explicit function, but can be evaluated for specific
individuals on demand (black-box optimization).
q The evaluation of f may be expensive even for a single individual. Repetitive evaluation of
the same individual should be avoided by caching of fitness values.
q P (t) and P 0 can be multisets.
q The individuals themselves are immutable. It is only possible to add modified individuals to
the population and remove others in exchange (the population size is typically fixed).
ML: V-9
Evolutionary Algorithms
c BUBECK 2013
Computation Scheme
Initialization and Termination
The initial population can be chosen by one or a combination of the following
techniques:
q
random drawing from the (feasible) genotype space,
q
problem specific seeding heuristics which generate initial approximations,
q
reusing (parts of) the population from previous runs of the algorithm on
similar input.
Additional termination conditions are possible, e.g.:
q
stagnation of fitness over k generations,
q
insufficient population diversity,
q
sufficiently high fitness, if optimal value or upper bound is known.
Problem: premature convergence may occur.
ML: V-10
Evolutionary Algorithms
c BUBECK 2013
Recombination
One-Point Crossover
q
Split parents at a randomly chosen position p ∈ {1, ..., N − 1}.
q
Combine head from one parent with tail from another parent.
q
ML: V-11
0101101001
0101101101
1010001101
1010001001
Can be generalized to n-Point Crossover.
Evolutionary Algorithms
c BUBECK 2013
Recombination
Uniform Crossover
q
ML: V-12
Randomly decide for each gene in the genome from which parent to copy it.
0100111001
0001011000
1011001100
1110101101
Evolutionary Algorithms
c BUBECK 2013
Recombination
Intermediary Recombination
q
For real-valued genes, take the arithmetic mean of the corresponding genes
in all parents:
ρ
1X k
ni :=
ei
ρ
k=1
for parents e1 = (e11, ..., e1N ), ..., eρ = (eρ1 , ..., eρN ) with genes eki ∈ R.
q
ML: V-13
Can be generalized to a weighted mean.
Evolutionary Algorithms
c BUBECK 2013
Recombination
Building Block Hypothesis
The building block hypothesis suggests that improved solutions can be assembled
from partial solutions which are aggregated in relatively small code blocks within
the genome.
Recombination allows merging favorable blocks and genetic repair of defective
blocks (Beyer, 2002).
ML: V-14
Evolutionary Algorithms
c BUBECK 2013
Mutation
Motivation
Recombination alone is in general not sufficient to explore the whole search space.
We need mutation as a main source of genetic variation with the following
properties:
q
Changes must be random and unbiased to avoid genetic drift.
q
Every (feasible) element in the genotype space must be reachable from
every other element by repeated mutation.
q
Mutation strength should be adjustable to allow problem-specific and
progress-based adaptation.
ML: V-15
Evolutionary Algorithms
c BUBECK 2013
Mutation
Bitwise Mutation
q
Flip each bit in a binary genome with probability pm.
0101101001
0101101001
0001101001
...
flip with
probability 𝑝𝑚
ML: V-16
Evolutionary Algorithms
flip with
probability 𝑝𝑚
flip with
probability 𝑝𝑚
c BUBECK 2013
Remarks:
1
q Frequently chosen values for pm are 0.001 ≤ pm ≤ 0.01, pm = N1 or pm = c · λ·N
with a small
constant c for bit vectors of dimension N and offspring size λ.
q Adaptation pm (t) to the generation count t can be useful (focus initially on exploration and
later turn to exploitation).
q Bitwise mutation can be generalized to discrete (nominal or ordinal) domains by replacing
with probability pm each gene with a randomly selected value from the corresponding
domain.
ML: V-17
Evolutionary Algorithms
c BUBECK 2013
Mutation
Mutation for Permutations
For permutations, replacement of genes with arbitrary values is typically
problematic.
Alternatives:
q
swapping the values of two randomly chosen genes,
q
inversion or scrambling of the position of genes within a randomly chosen
subsequence of the genome,
q
etc.
ML: V-18
Evolutionary Algorithms
c BUBECK 2013
Mutation
Gaussian Mutation
q
For real-valued genes, add a random real value from a Gaussian distribution
N (0, 1):
0.4
0.3
0.2
0.1
–4
–2
0
2
4
x
q
ML: V-19
Mutation strength can be controlled through the variance.
Evolutionary Algorithms
c BUBECK 2013
Selection
Motivation
Without selection, evolutionary algorithms would degenerate to parallel random
walks (with some bias from recombination).
General selection principles:
q
Individuals with high fitness are more likely to become parents and to
survive.
q
Allowing also some “weak” individuals to reproduce and to survive may
avoid local optima.
ML: V-20
Evolutionary Algorithms
c BUBECK 2013
Selection
Fitness-proportional Selection
q
Make k random draws (k = ρ for parent selection and k = µ for survivor
selection). In each draw, individual x is selected from the source population
P 0 with probability p(x) proportional to its fitness:
f (x)
0
x0 ∈P f (x )
p(x) = P
ML: V-21
Evolutionary Algorithms
c BUBECK 2013
Selection
Tournament Selection
q
Perform µ tournament rounds (with replacement):
– Randomly draw ξ individuals x01, ..., x0ξ from the source population P 0 with
equal probabilities.
– Select the individual x0 with x0 = argmaxx∈{x0 ,...,x0 }f (x), i.e. the fittest in the
1
ξ
tournament.
ML: V-22
Evolutionary Algorithms
c BUBECK 2013
Remarks:
q Tournament selection can be applied immediately to situations where actual fitness values
are unknown, but only fitness ranks are available.
q Instead of selecting the fittest individual in a tournament, fitness-proportional selection with
respect to the current tournament can be used.
q Another possible modification for survivor selection is elitism: always keep the fittest
individual(s).
ML: V-23
Evolutionary Algorithms
c BUBECK 2013
Selection
Plus and Comma Selection
q
For the survivor selection after offspring creation, different possibilities arise
for the selection source:
– survivor selection only from the offspring P 0:
P (t + 1) = select_population(µ, P 0, f ) (comma selection)
– survivor selection from the offspring P 0 and the current generation P (t):
P (t + 1) = select_population(µ, P 0 ∪ P (t), f ) (plus selection)
– survivor selection from the offspring P 0 and the last κ generations
P (t), ..., P (t − κ + 1):
P (t + 1) = select_population(µ, P 0 ∪ P (t) ∪ ... ∪ P (t − κ + 1), f )
ML: V-24
Evolutionary Algorithms
c BUBECK 2013
Parameter Control
Taxonomy of Parameter Selection Techniques
q
static parameter selection (parameters are chosen before the actual run):
– by hand (e.g. by expert knowledge or by experiments with small problem
instances)
– by suitable optimization techniques (e.g. meta evolution)
q
dynamic parameter selection:
t
– by deterministic rules (e.g. pm(t) := pmax − tmax
(pmax − pmin))
– by predefined adaptation to the search progress (e.g. by lowering the
mutation rate if only few mutants survive)
– by self-adaptation (genetic coding of parameters as part of the
individuals)
ML: V-25
Evolutionary Algorithms
c BUBECK 2013
Literature
q
A. Eiben and J. Smith.
Introduction to Evolutionary Computing.
Springer, 2007.
q
I. Gerdes, F. Klawonn, and R. Kruse.
Evolutionäre Algorithmen.
Vieweg, 2004.
q
O. Kramer.
Computational Intelligence - Eine Einführung.
Springer, 2009.
ML: V-26
Evolutionary Algorithms
c BUBECK 2013