Download Evolutionary Computation - School of Computer Science

Artificial Intelligence Search
Methodologies
Dr Rong Qu
School of Computer Science
University of Nottingham
Nottingham, NG8 1BB, UK
[email protected]
Evolutionary Computation
EAs - framework

Based on the idea of evolution
Population
Selection
Parents
Reproduction
Replacement
Offsprings
Metaphor
Optimization
Evolution
Problem solving
Individual
Solution
Fitness
Objective function
Environment
Optimization problem
Metaheuristics – From Design to Implementation pp. 199 (Talbi, 2009)
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
EAs - framework

Template of an evolutionary algorithm
Algorithm. Template of an evolutionary algorithm
Generate(P(0)); //initial population
t = 0;
While not Termination_Criterion(P(t)) Do
Evaluate(P(t));
P’(t)
= Selection(P(t));
P’(t)
= Reproduction(P’(t)); Evaluate(P’(t));
P(t+1) = Replace(P(t), P’(t));
t = t+1;
End While
Output Best individual or best population found
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Genetic Algorithms
Charles Darwin
1809 - 1882
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Genetic Algorithms
1859
Origin of the Species
Survival of the Fittest
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Genetic Algorithms
1975
Genetic Algorithms
Artificial Survival of the Fittest
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Genetic Algorithms
1989
Genetic Algorithms
Foundations and Applications
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Genetic Algorithms


Compared to exhaustive tree search techniques,
modern heuristic algorithms quickly converge to suboptimal solutions by searching a small fraction of the
search space
Simulates the Darwinian evolution process in computer
programs.


Populations evolve by selective pressures, mating between
individuals, and alterations such as mutations.
Genetic operators evolve solutions in the current population
to create a new population
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Genetic Algorithms
Flowchart (20 generations)
Generate Initial
Population
n=1
Population
Generation 'n'
Selection
Crossover
Population
Mutate
Population
Final
Population
n=n+1
n<20?
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Genetic Algorithms


Evolution of population (a set of solutions) into a new
population, using operators: reproduction, mutation
and crossover
Can be implemented as three modules: the
evaluation, the population and the reproduction
modules
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Genetic Algorithm Operators
Crossover
1
1
2
2
A percentage of the population
is selected for breeding and
assigned random mates.
A random crossover point is
selected and a pair of new
solutions are produced
Adapted from Prof Graham Kendall’s G5BAIM
lecture example
One Point Crossover in
Genetic Algorithms
Uniform Crossover in
Genetic Algorithms
Partially Matched
Crossover (PMX) in
Genetic Algorithms
© Graham Kendall
© Graham Kendall
© Graham Kendall
[email protected]
[email protected]
[email protected]
http://cs.nott.ac.uk/~gxk
http://cs.nott.ac.uk/~gxk
http://cs.nott.ac.uk/~gxk
Genetic Algorithm Operators
Mutation
In a small percentage of the population random
changes are made.
May be designed to conduct more complex
operations (local search) accordingly.
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Genetic Algorithm Example I
Maximise f(x) = x^3 - 60 * x^2 + 900 * x +100
0 <= x <= 31
Crossover probability, PC = 1.0
Mutation probability, PM = 0.0
x can be represented using five binary digits
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
100
941
1668
2287
2804
3225
3556
3803
3972
4069
4100
4071
3988
3857
3684
3475
3236
2973
2692
2399
2100
1801
1508
1227
964
725
516
343
212
129
100
f(x) = x^3 - 60x^2 + 900x + 100
Max : x = 10
4500
4000
3500
3000
2500
2000
1500
1000
500
0
1
3
5
7
9 11 13 15 17 19 21 23 25 27 29 31 33 35 37
Adapted from Dr Graham Kendall’s G5BAIM lecture example
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Genetic Algorithm Example I
Maximise f(x) = x^3 - 60 * x^2 + 900 * x +100
0 <= x <= 31
Generating random initial population
chromosome
binary string
x
f(x)
P1
11100
28
212
P2
01111
15
3475
P3
10111
23
1227
P4
00100
4
2804
Dr. R Qu
Total
7718
Average
1929.50
G54STA/G54SDA – Evolutionary Computing
Genetic Algorithm Example I
Maximise f(x) = x^3 - 60 * x^2 + 900 * x +100
0 <= x <= 31
Choose two parents (roulette selection)
Roulette wheel
Parents
1227
P3
3475
P2
One point crossover (random position 1)
P3
1
0
1
1
1
P4
0
0
1
0
0
P2
0
1
1
1
1
P2
0
1
1
1
1
C1
1
1
1
1
1
C3
0
0
1
1
1
C2
0
0
1
1
1
C4
0
1
1
0
0
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Genetic Algorithm Example I
Maximise f(x) = x^3 - 60 * x^2 + 900 * x +100
0 <= x <= 31
New generation
chromosome
binary string
x
f(x)
P1
11111
31
131
P2
00111
7
3803
P3
00111
7
3803
P4
01100
12
3889
Dr. R Qu
Total
11735
Average
2933.75
G54STA/G54SDA – Evolutionary Computing
Genetic Algorithm Example I
Maximise f(x) = x^3 - 60 * x^2 + 900 * x +100
0 <= x <= 31
Two generations
chromosome binary string
x
f(x)
chromosome binary string x
f(x)
P1
11100
28
212
P1
11111
31
131
P2
01111
15
3475
P2
00111
7
3803
P3
10111
23
1227
P3
00111
7
3803
P4
00100
4
2804
P4
01100
12
3889
Total
7718
Total
11735
Average
1929.50
Average
2933.75
Mutation
What problem do you see with the populations?
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Genetic Algorithm Example II
Traveling Salesman Problem (TSP)
a number of cities
costs of traveling between cities
a traveling sales man needs to visit all these cities
exactly once and return to the starting city
What’s the cheapest route?
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Genetic Algorithm Example II
Traveling Salesman Problem
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Genetic Algorithm Example II
Initial generation
P1
5
8
1
…
…
84
32
27
54
67
6.5
P2
78
81
27
…
…
…
9
11
7
44
24
7.8
P30
8
1
7
…
9
16
36
24
19
6.0
…
Any idea of other ways to generate the initial
population?
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Genetic Algorithm Example II
Choose pairs of parents
P30
8
1
7
…
…
9
16
36
24
19
6.0
P2
78
81
27
…
…
9
11
7
44
24
7.8
Crossover
C1
8
1
7
…
…
9
11
13
7
44
24
5.9
C2
78
81
27
…
…
9
16
36
24
19
6.2
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Genetic Algorithm Example II
Next generation
P1
8
1
7
…
…
9
11
7
44
24
5.9
P2
78
81
27
…
…
…
9
16
36
24
19
6.2
P30
7
8
2
…
…
5
10
76
4
79
6.0
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Genetic Algorithm Example II
Traveling Salesman Problem
No. of cities: 100
Population size: 30
Cost: 6.37
Generation: 88
Dr. R Qu
Cost: 6.34
Generation: 1100
G54STA/G54SDA – Evolutionary Computing
Genetic Algorithm Example III
Applying
Genetic Algorithms
to Personnel Scheduling
Personnel scheduling in healthcare is usually a very
complex operation which has a profound effect upon
the efficient usage of expensive resources.
E.K. Burke, T. Curtois, R. Qu and G. Vanden Berghe. "A Time Predefined Variable Depth Search for Nurse
Rostering". INFORMS Journal on Computing, 2012. doi: 10.1287/ijoc.1120.0510
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Genetic Algorithm Example III
A number of nurses
A number of shifts each day
A set of constraints
• shift coverage
• one shift per day
• resting time
• workload per month
• consecutive shifts
• working weekends
•…
Genetic Algorithm Example III
Genetic Algorithm
- Initial population
- construct rosters
- repair infeasible ones
Dr. R Qu
G54SAI – Evolutionary Algorithms
26
Genetic Algorithm Example III
Genetic Algorithm
- Select parents
- Recombine rows in the two rosters
- repair infeasible ones
+
Genetic Algorithm Example III
Genetic Algorithm
- Mutation
- Local optimiser
Dr. R Qu
G54SAI – Evolutionary Algorithms
28
Genetic Algorithm Example III
Population Size
Crossover Probability
Mutation Probability
Local Optimiser
Dr. R Qu
50
0.7
0.001
ON
G54STA/G54SDA – Evolutionary Computing
GA - performance

There are a number of factors which affect the
performance of a genetic algorithm







The size of the population
The initial population
Selection pressure (elitism, tournament)
The cross-over probability
The mutation probability
Defining convergence
Local optimisation
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
GA - Conclusions



Survival of the fittest, the most fit of a particular
generation (the best possible answer) are used as
parents for the next generation.
GAs are a very powerful tool, allowing searching for
solutions when there are no other feasible means to do
so.
More advanced evolutionary algorithms emerged in the
last two decades.
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Evolutionary Algorithms



Evaluation is responsible for evaluating the fitness of
individuals in the population
It is the only part of the GA that has any knowledge
about the specific problem. The rest of the GA modules
are simply operating on chromosomes with no
information about the problem.
A different evaluation module is needed for each
problem.
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Evolutionary Algorithms



Conceivable solutions are represented, the 'fittest' will
have the greatest chance of reproducing.
Successful properties will therefore be favourably
represented within the population of solutions.
The population is successively 'optimised' after a
number of generations.
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Evolutionary Algorithms






Representation
Population
Objective function
Selection
Replacement
Stopping
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Estimation of Distribution
Algorithms (EDA)
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
EDA Flowchart (20 generations)
Initial Population
Generation n = 1
Selection of
better individuals
Build probability
model
Sample model to generate
new individuals
Final
Population
New Population
n=n+1
n<20?
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
EDA

A recent class of population based metaheuristics

Uses probability distribution of the population


Statistical information extracted from the population
used to generate the next population
Probability model (vector)

Replaces the genetic operators to evolve the population
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing

Template of an EDA
Algorithm. Template of an EDA
t = 1;
Generate randomly a population of n individuals
Initialize a probability model Q(x);
While not Termination_Criterion Do
Create P(t) a population of n individuals by sampling from Q(x);
Evaluate(P(t));
Select m individuals from P(t) using a selection method;
Update the probability model Q(x) using the selected individuals;
t = t+1;
End While
Output Best individual or best population found
Adapted from Metaheuristics – From Design to Implementation pp. 223 (Talbi, 2009)
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing

Template of a PBIL
Algorithm. Template of a PBIL algorithm
Initial distribution D = (0.5, 0.5, …, 0.5);
Repeat
// Generate the population
For each individual Xi
If r < Dj (r uniform in [0,1]) Then xij = 1
Else xij = 0;
End For
Evaluate and sort the population;
Update the distribution D = (1-α)D + αXbest;
Until Stopping criteria
Adapted from Metaheuristics – From Design to Implementation pp. 224 (Talbi, 2009)
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
PBIL Example I
Maximise f(x) = x^3 - 60 * x^2 + 900 * x +100
0 <= x <= 31
x represented using 5 binary digits
Probability Vector
0.5 0.5 0.5 0.5 0.5
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
Dr. R Qu
100
941
1668
2287
2804
3225
3556
3803
3972
4069
4100
4071
3988
3857
3684
3475
3236
2973
2692
2399
2100
1801
1508
1227
964
725
516
343
212
129
100
f(x) = x^3 - 60x^2 + 900x + 100
Max : x = 10
4500
4000
3500
3000
2500
2000
1500
1000
500
0
1
3
5
7
9 11 13 15 17 19 21 23 25 27 29 31 33 35 37
G54STA/G54SDA – Evolutionary Computing
PBIL Example I
Maximise f(x) = x^3 - 60 * x^2 + 900 * x +100
0 <= x <= 31
Generating initial population based on Probability Vector
0.5 0.5 0.5 0.5 0.5
Evaluate the population
individuals
binary string
X
f(X)
P1
11100
28
212
P2
01111
15
3475
P3
10111
23
1227
P4
00100
4
2804
Average
Dr. R Qu
1929.50
G54STA/G54SDA – Evolutionary Computing
PBIL Example I
Maximise f(x) = x^3 - 60 * x^2 + 900 * x +100
0 <= x <= 31
Probability Vector
Update the PV based on Xbest
0.5 0.5 0.5 0.5 0.5
Probability Vector
0.45 0.65 0.65 0.65 0.65
D = (1-α)D + αXbest
individuals
binary string
X
f(X)
P1
11100
28
212
P2
01111
15
3475
P3
10111
23
1227
P4
00100
4
2804
Average
Dr. R Qu
1929.50
G54STA/G54SDA – Evolutionary Computing
PBIL Example I
Maximise f(x) = x^3 - 60 * x^2 + 900 * x +100
0 <= x <= 31
Generating new population based on Probability Vector
0.45 0.65 0.65 0.65 0.65
Probability Vector
…
…
…
…
…
D = (1-α)D + αXbest
individuals
binary string
x
f(x)
P1
01011
11
4071
P2
00111
7
3803
P3
01101
13
3857
P4
01110
11
3684
Average
3853.75
Mutation in GA vs. Randomness in PBIL
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
PBIL Example II
Portfolio Selection Problem
Given a set of assets (stocks, etc.) and their associated
return and risk
Mathworks
What’s the best investment
plan (i.e. which assets to
choose to achieve the highest
return and lowest risk)?
44
PBIL Example II
Portfolio Selection Problem
Modern Portfolio Theory
Nobel memorial prize winner Harry Markowitz
Diversification: “don’t
put all your eggs into
one basket”
Which assets to choose?
How much to invest to
each asset?
Real world constraints
Asset Allocation and Portfolio
Optimization, Mathworks
45
PBIL Example II
Portfolio Selection Problem
Which assets to choose?
A binary vector of size n
n: number of assets
How much to invest to each asset?
A real vector of size n, n: number of assets
Additional constraints
Cardinality: a limited number of assets
Boundary: a lower and upper bound of investment to the
assets
K. Lwin, R. Qu. "Hybrid Algorithm for Constrained Portfolio Selection Problem". Applied
Intelligence, 39(2): 251-266, 2013. doi: 10.1007/s10489-012-0411-7
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
PBIL Example II
Portfolio Selection Problem
Evaluation function
Return: combined return based on historical data
Risk: combined risk (variance of return) of all chosen assets
Constraint handling
Cardinality
Boundaries
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Evolutionary Algorithm Applications
Combinatorial optimisation problems
• bin packing problems
• vehicle routing problems
• job shop scheduling
Evolutionary Algorithm Applications
Combinatorial optimisation problems
• portfolio optimization
• multimedia multicast routing
• knapsack problem
APPENDIX
Other Evolutionary Algorithms
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
ANT ALGORITHMS
Ants are practically blind but they still manage to find their way to and
from food. How do they do it?
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Ant Algorithms

Ant systems are a population based
approach. In this respect it is similar to
genetic algorithms

There is a population of ants, with each ant
finding a solution and then communicating
with the other ants
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Ant Algorithms
H
B
G
F
E
D
A
C
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Ant Algorithms
d=1
F
d=1
D
C
E
d=1
Dr. R Qu
d=0.5
d=0.5
B
d=1
A
G54STA/G54SDA – Evolutionary Computing
Ant Algorithms

Time, t, is discrete

At each time unit an ant moves a distance, d of 1

Once an ant moved, it lays down 1 unit of
pheromone

At t = 0, there is no pheromone on any edge
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Ant Algorithms
1
E
1
F
1
D 0.5
C
0.5
B
1
A
16 ants are moving from
A - F and another 16 are
moving from F - A
Dr. R Qu
At t=1 there will be 16 ants at B
and 16 ants at D.
At t=2 there will be 8 ants at D
and 8 ants at B. There will be 16
ants at E
The intensities on the edges will
be as follows
FD = 16, AB = 16, BE = 8, ED =
8, BC = 16 and CD = 16
G54STA/G54SDA – Evolutionary Computing
Ant Algorithms


We need to allow the ants to explore paths and
follow the best paths with some probability in
proportion to the intensity of the pheromone trail
We do not want them simply to follow the route with
the highest amount of pheromone on it, else our
search will quickly settle on a sub-optimal (and
probably very sub-optimal) solution
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Ant Algorithms


The probability of an ant following a certain
route is a function, not only of the
pheromone intensity but also a function of
what the ant can see (visibility)
The pheromone trail must not build
unbounded. Therefore, we need
“evaporation”
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Ant Algorithms – initial ideas

Dorigo (1996)



Based on real world phenomena
Ants, despite almost blind, are able to find their
way to the food source using the shortest route
If an obstacle is placed, ants have to decide which
way to take around the obstacle.
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Ant Algorithms – initial ideas

Dorigo (1996)



Initially there is a 50-50 probability as to which
way they will turn
Assume one route is shorter than the other
Ants taking the shorter route will arrive at a point
on the other side of the obstacle before the ants
which take the longer route.
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Ant Algorithms – initial ideas

Dorigo (1996)



As ants walk they deposit pheromone trail.
Ants have taken shorter route will have already
laid trail
So ants from the other direction are more likely to
follow that route with deposit of pheromone.
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Ant Algorithms – initial ideas

Dorigo (1996)



Over a period of time, the shortest route will have
high levels of pheromone.
The quantity of pheromones accumulates faster
on the shorter path than on the longer one
There is positive feedback which reinforces that
behaviors so that the more ants follow a particular
route, the more desirable it becomes.
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Ant Algorithms
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Ant Algorithms - TSP




At the start of the algorithm, one ant is placed in
each city
Time, t, is discrete. t(0) marks the start of the
algorithm. At t+1 every ant will have moved to a new
city
Assuming that the TSP is represented as a fully
connected graph, each edge has an intensity of trail
on it. This represents the pheromone trail laid by the
ants
Let Ti,j(t) represent the intensity of trail edge (i,j) at
time t
Variations have been tested by Dorigo
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Ant Algorithms - TSP



An ant decides which town to move to next, with a
probability that is based on the distance to that city
AND the amount of trail intensity on the
connecting edge
The distance to the next town, is known as the
visibility, nij, defined as 1/dij, dij is the distance
between cities i and j.
At each time unit evaporation takes place, at a rate
p, a value between 0 and 1
Variations have been tested by Dorigo
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Ant Algorithms - TSP


In order to stop ants visiting the same city in
the same tour a data structure, Tabu, is
maintained
Tabuk is defined as the list for the kth ant and
it holds the cities that have already been
visited
Variations have been tested by Dorigo
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Ant Algorithms - TSP
After each ant tour the trail intensity on edge (i,j) is
updated using the following formula
Tij (t + n) = p . Tij(t) + ΔTij
T ij
k
 Q if the kth ant uses edge(i, j ) in its tour
 Lk
(between time t and t  n)


otherwise
0
Q is a constant
Lk is the tour length of the kth ant
p is the evaporation coefficient
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Ant Algorithms - TSP
Transition
Probability



k
[Tij(t )] . [nij]

pij (t )   k  allowedk [Tik(t )]



0

if j  allowed k
. [n
ik
]
otherwise
where  and  are control parameters that control the relative
importance of trail versus visibility
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Ant Algorithms - TSP
Left: Trail distribution at the beginning;
Right: Trail distribution after 100 cycles. (Dorigo et al., 1996)
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing
Ant Algorithms - Applications






Travelling Salesman Problem (TSP)
Facility Layout Problem
Vehicle Routing
Stock Cutting
…
Marco Dorigo maintains a page devoted to the
subject at


http://iridia.ulb.ac.be/~mdorigo/ACO/ACO.html
contains information about ant algorithms as well as links to
the main papers published on the subject
Dr. R Qu
G54STA/G54SDA – Evolutionary Computing