Download Population Based Algorithms - School of Computer Science

Artificial Intelligence Search
Methodologies
Dr Rong Qu
School of Computer Science
University of Nottingham
Nottingham, NG8 1BB, UK
[email protected]
Population Based Algorithms
Optimisation Problems: Methods

Meta-heuristics


Guide an underlying heuristic/search to escape
from being trapped in a local optima and to
explore better areas of the solution space
Single solution approaches


Population based approaches

Konstanz, May 2014
Simulated Annealing, Tabu Search, Variable
Neighbourhood Search, etc.;
Genetic algorithm, Memetic algorithm, Ant
Algorithms, Particle Swarm Intelligence, etc.;
AI Search Algorithms – Population Based
2
Population Based Algorithms

Local search



Concerning only one solution at a particular time
during the search
Search is very much restricted to local regions, so
called local
Population based algorithms

Konstanz, May 2014
concern a population of solutions at a time
AI Search Algorithms – Population Based
3
Charles Darwin
1809 - 1882
GENETIC ALGORITHMS
Konstanz, May 2014
AI Search Algorithms – Population Based
4
GA Algorithm – basic idea

Based on survival of the fittest



Developed extensively by John Holland in
mid 70’s
Three modules


Algorithm uses terms from genetics: population,
chromosome and gene
the evaluation module, the population module and
the reproduction module
Solutions (individuals) often coded as bit
strings
Konstanz, May 2014
AI Search Algorithms – Population Based
5
GA Algorithm – basic idea
1859
Origin of the Species
Survival of the Fittest
Konstanz, May 2014
AI Search Algorithms – Population Based
6
GA Algorithm – basic idea
1975
Genetic Algorithms
Artificial Survival of the Fittest
Konstanz, May 2014
AI Search Algorithms – Population Based
7
GA Algorithm – basic idea
1989
Genetic Algorithms
Foundations and Applications
Konstanz, May 2014
AI Search Algorithms – Population Based
8
GA Algorithm – basic steps



Initial population
Evaluations on individuals
Breeding



Choose suitable parents (proportion to evaluation
rating)
Produce two offspring (Probability of breeding)
Mutation
Domain knowledge – evaluation function
Konstanz, May 2014
AI Search Algorithms – Population Based
9
GA Algorithm – basic steps
1. Initialise a population of chromosomes Ci
2. Evaluate each Ci (individual) in the population
Create new C by using Ci in the current population (using
crossover and mutation)
Delete members of the existing population to make way for
the new members
Evaluate the new members and insert them into the
population
Repeat (evolve) until some termination condition is reached
(normally based on time or number of populations produced)
3. Return the best Ci as the solution
Konstanz, May 2014
AI Search Algorithms – Population Based
10
GA Algorithm – basic steps
Generate Initial
Population
n=1
Population
Generation 'n'
Selection
Crossover
Population
Mutate
Population
Final
Population
n=n+1
n<20?
Konstanz, May 2014
AI Search Algorithms – Population Based
11
GA Algorithm – encoding
The decision variables of a problem are
normally encoded into a finite length string
This could be a binary string or a list of integers
For example :
0 1 1 0 1 1 0 1 0 or
5 1 4 2 3 4 1
We could also represent
numbers as coloured boxes
Konstanz, May 2014
AI Search Algorithms – Population Based
12
Evaluation Module



Responsible for evaluating a chromosome
Only part of the GA that has domain knowledge.
The rest of the GA modules are simply operating
on (typically) bit strings with no information
about the problem
A different evaluation module is needed for each
problem
Konstanz, May 2014
AI Search Algorithms – Population Based
13
Population Module

Responsible for maintaining the population

Initialization





Random
Known Solutions
Heuristics
Population Size
Elitism
Konstanz, May 2014
AI Search Algorithms – Population Based
14
Population Module

Deletion



Konstanz, May 2014
Delete-All : replaces all the members of the
current population by the same number of
chromosomes created
Steady-State : replaces n old members by n new
members.
But: replace the worst, random or the parents
individuals?
Steady-State-No-Duplicates : Same as steadystate but also checks that no duplicate
chromosomes are added to the population.
AI Search Algorithms – Population Based
15
Reproduction Module

Parent selection

Fitness techniques

Crossover & mutation
Konstanz, May 2014
AI Search Algorithms – Population Based
16
Parent Selection

Roulette Wheel Selection


Select the parents with a probability in proportion
to their fitness
Tournament

Konstanz, May 2014
Select two individuals at random. The individual
with the highest evaluation becomes the parent.
Repeat to find a second parent
AI Search Algorithms – Population Based
17
Fitness Techniques
• Fitness-Is-Evaluation : Simply use the fitness
of the chromosome equal to its evaluation
• Linear Normalization : The chromosomes are
sorted by decreasing the evaluation value.
Then the chromosomes are assigned a fitness
value that starts with a constant value and
decreases linearly. The initial value and the
decrement are parameters to the techniques
Konstanz, May 2014
AI Search Algorithms – Population Based
18
Crossover Operators
One Point Crossover in
Genetic Algorithms
Uniform Crossover in
Genetic Algorithms
© Graham Kendall
© Graham Kendall
[email protected]
[email protected]
http://cs.nott.ac.uk/~gxk
http://cs.nott.ac.uk/~gxk
Order based crossover
Cycle crossover
Partially matched crossover
Konstanz, May 2014
AI Search Algorithms – Population Based
19
Mutation
• A method of ensuring premature convergence
does not occur
• Usually set to a small value
• Dynamic mutation and crossover rates
Konstanz, May 2014
AI Search Algorithms – Population Based
20
Example I
•
•
•
•
•
Crossover probability, PC = 1.0
Mutation probability, PM = 0.0
Maximise f(x) = x3 - 60 * x2 + 900 * x +100
0 <= x >= 31
x can be represented using five binary digits
Konstanz, May 2014
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
100
941
1668
2287
2804
3225
3556
3803
3972
4069
4100
4071
3988
3857
3684
3475
3236
2973
2692
2399
2100
1801
1508
1227
964
725
516
343
212
129
100
f(x) = x^3 - 60x^2 + 900x + 100
Max : x = 10
4500
4000
3500
3000
2500
2000
1500
1000
500
0
AI Search Algorithms – Population Based
1
3
5
7
9 11 13 15 17 19 21 23 25 27 29 31 33 35 37
21
Example I
• Generate random initial individuals
Maximise f(x) = x^3 - 60 * x^2 + 900 * x +100 (0 <= x <= 31)
chromosome
P1
P2
P3
P4
Konstanz, May 2014
binary string
11100
01111
10111
00100
Total
Average
AI Search Algorithms – Population Based
x
28
15
23
4
f(x)
212
3475
1227
2804
7718
1929.50
22
Example I
• Choose Parents, using roulette wheel selection
• Crossover point, 1, is chosen randomly
Roulette wheel
4116
Parents
P3
1915
P2
P3
P2
1
0
0
1
1
1
1
1
1
1
P4
P2
0
0
0
1
1
1
0
1
0
1
C1
C2
1
0
1
0
1
1
1
1
1
1
C3
C4
0
0
0
1
1
1
1
0
1
0
Konstanz, May 2014
AI Search Algorithms – Population Based
23
Example I
New generation
Maximise f(x) = x^3 - 60 * x^2 + 900 * x +100 (0 <= x <= 31)
chromosome
P1
P2
binary string
11111
00111
x
31
7
f(x)
131
3803
P3
P4
00111
01100
Total
7
12
3803
3889
11735
Average
Konstanz, May 2014
AI Search Algorithms – Population Based
2933.75
24
Example I
Two generations
what chance is there of finding
the global optimum?
Maximise f(x) = x^3 - 60 * x^2 + 900 * x +100 (0 <= x <= 31)
chromosome
binary
string
x
f(x)
chromosome
binary
string
x
f(x)
P1
11100
28
212
P1
11111
31
131
P2
01111
15
3475
P2
00111
7
3803
P3
10111
23
1227
P3
00111
7
3803
P4
00100
4
2804
P4
01100
12
3889
Total
7718
Total
11735
Average
1929.50
Average
2933.75
Mutation
What problem do you see with the populations?
Konstanz, May 2014
AI Search Algorithms – Population Based
25
GA - performance

There are a number of factors which affect the
performance of a genetic algorithm







The size of the population
The initial population
Selection pressure (elitism, tournament)
The cross-over probability
The mutation probability
Defining convergence
Local optimisation
Konstanz, May 2014
AI Search Algorithms – Population Based
26
GA - applications

Combinatorial optimisation problems



Konstanz, May 2012
Cutting and packing problems
vehicle routing problems
job shop scheduling
AI Search Algorithms – Population Based
27
GA - applications

Combinatorial optimisation problems



Konstanz, May 2012
portfolio optimization
multimedia multicast routing
knapsack problem
AI Search Algorithms – Population Based
28
ANT ALGORITHMS
Ants are practically blind but they still manage to find their way to and
from food. How do they do it?
Konstanz, May 2014
AI Search Algorithms – Population Based
29
Ant Algorithms

Ant systems are a population based
approach. In this respect it is similar to
genetic algorithms

There is a population of ants, with each ant
finding a solution and then communicating
with the other ants
Konstanz, May 2014
AI Search Algorithms – Population Based
30
Ant Algorithms
H
B
G
F
E
D
A
C
Konstanz, May 2014
AI Search Algorithms – Population Based
31
Ant Algorithms
d=1
F
d=1
D
d=0.5
C
E
d=1
Konstanz, May 2014
d=0.5
B
d=1
A
AI Search Algorithms – Population Based
32
Ant Algorithms

Time, t, is discrete

At each time unit an ant moves a distance, d of 1

Once an ant moved, it lays down 1 unit of
pheromone

At t = 0, there is no pheromone on any edge
Konstanz, May 2014
AI Search Algorithms – Population Based
33
Ant Algorithms
1
E
1
F
1
D 0.5
C
0.5
B
1
A
16 ants are moving from
A - F and another 16 are
moving from F - A
Konstanz, May 2014
At t=1 there will be 16 ants at B
and 16 ants at D.
At t=2 there will be 8 ants at D
and 8 ants at B. There will be 16
ants at E
The intensities on the edges will
be as follows
FD = 16, AB = 16, BE = 8, ED =
8, BC = 16 and CD = 16
AI Search Algorithms – Population Based
34
Ant Algorithms


We need to allow the ants to explore paths
and follow the best paths with some
probability in proportion to the intensity of
the pheromone trail
We do not want them simply to follow the
route with the highest amount of pheromone
on it, else our search will quickly settle on a
sub-optimal (and probably very sub-optimal)
solution
Konstanz, May 2014
AI Search Algorithms – Population Based
35
Ant Algorithms


The probability of an ant following a certain
route is a function, not only of the
pheromone intensity but also a function of
what the ant can see (visibility)
The pheromone trail must not build
unbounded. Therefore, we need
“evaporation”
Konstanz, May 2014
AI Search Algorithms – Population Based
36
Ant Algorithms – initial ideas

Dorigo (1996)



Konstanz, May 2014
Based on real world phenomena
Ants, despite almost blind, are able to find
their way to the food source using the
shortest route
If an obstacle is placed, ants have to
decide which way to take around the
obstacle.
AI Search Algorithms – Population Based
37
Ant Algorithms – initial ideas

Dorigo (1996)



Konstanz, May 2014
Initially there is a 50-50 probability as to
which way they will turn
Assume one route is shorter than the other
Ants taking the shorter route will arrive at
a point on the other side of the obstacle
before the ants which take the longer
route.
AI Search Algorithms – Population Based
38
Ant Algorithms – initial ideas

Dorigo (1996)



Konstanz, May 2014
As ants walk they deposit pheromone trail.
Ants have taken shorter route will have
already laid trail
So ants from the other direction are more
likely to follow that route with deposit of
pheromone.
AI Search Algorithms – Population Based
39
Ant Algorithms – initial ideas

Dorigo (1996)



Konstanz, May 2014
Over a period of time, the shortest route
will have high levels of pheromone.
The quantity of pheromones accumulates
faster on the shorter path than on the
longer one
There is positive feedback which reinforces
that behaviors so that the more ants follow
a particular route, the more desirable it
becomes.
AI Search Algorithms – Population Based
40
Ant Algorithms
Konstanz, May 2014
AI Search Algorithms – Population Based
41
Ant Algorithms - TSP




At the start of the algorithm, one ant is
placed in each city
Time, t, is discrete. t(0) marks the start of the
algorithm. At t+1 every ant will have moved
to a new city
Assuming that the TSP is represented as a
fully connected graph, each edge has an
intensity of trail on it. This represents the
pheromone trail laid by the ants
Let Ti,j(t) represent the intensity of trail edge
(i,j) at time t
Variations have been tested by Dorigo
Konstanz, May 2014
AI Search Algorithms – Population Based
42
Ant Algorithms - TSP



An ant decides which town to move to next,
with a probability that is based on the
distance to that city AND the amount of
trail intensity on the connecting edge
The distance to the next town, is known as
the visibility, nij, defined as 1/dij, dij is the
distance between cities i and j.
At each time unit evaporation takes place, at
a rate p, a value between 0 and 1
Variations have been tested by Dorigo
Konstanz, May 2014
AI Search Algorithms – Population Based
43
Ant Algorithms - TSP


In order to stop ants visiting the same city in
the same tour a data structure, Tabu, is
maintained
Tabuk is defined as the list for the kth ant and
it holds the cities that have already been
visited
Variations have been tested by Dorigo
Konstanz, May 2014
AI Search Algorithms – Population Based
44
Ant Algorithms - TSP
After each ant tour the trail intensity on edge (i,j)
is updated using the following formula
Tij (t + n) = p . Tij(t) + ΔTij
T ij
k
 Q if the kth ant uses edge(i, j ) in its tour
 Lk
(between time t and t  n)


otherwise
0
Q is a constant
Lk is the tour length of the kth ant
p is the evaporation coefficient
Konstanz, May 2014
AI Search Algorithms – Population Based
45
Ant Algorithms - TSP
Transition
Probability



k
[Tij(t )] . [nij]

pij (t )   k  allowedk [Tik(t )]



0

if j  allowed k
. [n
ik
]
otherwise
where  and  are control parameters that control
the relative importance of trail versus visibility
Konstanz, May 2014
AI Search Algorithms – Population Based
46
Ant Algorithms - TSP
Left: Trail distribution at the beginning;
Right: Trail distribution after 100 cycles. (Dorigo et al., 1996)
Konstanz, May 2014
AI Search Algorithms – Population Based
47
Ant Algorithms - Applications






Travelling Salesman Problem (TSP)
Facility Layout Problem
Vehicle Routing
Stock Cutting
…
Marco Dorigo maintains a page devoted to the
subject at


Konstanz, May 2014
http://iridia.ulb.ac.be/~mdorigo/ACO/ACO.html
contains information about ant algorithms as well as links to
the main papers published on the subject
AI Search Algorithms – Population Based
48
APPENDIX
Examples of Genetic Algorithms
Konstanz, May 2014
AI Search Algorithms – Population Based
49
Genetic Algorithm Example II
Traveling Salesman Problem
a number of cities
costs of traveling between cities
a traveling sales man needs to visit all these cities
exactly once and return to the starting city
What’s the cheapest route?
Konstanz, May 2014
AI Search Algorithms – Population Based
50
Genetic Algorithm Example II
Traveling Salesman Problem
Konstanz, May 2014
AI Search Algorithms – Population Based
51
Genetic Algorithm Example II
Initial generation
P1
5
8
1
…
…
84
32
27
54
67
6.5
P2
78
81
27
…
…
9
11
7
44
24
7.8
…
…
9
16
36
24
19
6.0
…
P30
8
1
7
Any idea of other ways to generate the initial
population?
Konstanz, May 2014
AI Search Algorithms – Population Based
52
Genetic Algorithm Example II
Choose pairs of parents
P30
8
1
7
…
…
9
16
36
24
19
6.0
P2
78
81
27
…
…
9
11
7
44
24
7.8
Crossover
C1
8
1
7
…
…
9
11
13
7
44
24
5.9
C2
78
81
27
…
…
9
16
36
24
19
6.2
Konstanz, May 2014
AI Search Algorithms – Population Based
53
Genetic Algorithm Example II
Next generation
P1
8
1
7
…
…
9
11
7
44
24
5.9
P2
78
81
27
…
…
9
16
36
24
19
6.2
P2
7
8
2
…
…
5
10
76
4
79
6.0
Konstanz, May 2014
…
AI Search Algorithms – Population Based
54
Genetic Algorithm Example II
Traveling Salesman Problem
No. of cities: 100
Population size: 30
Cost: 6.37
Generation: 88
Konstanz, May 2014
AI Search Algorithms – Population Based
Cost: 6.34
Generation: 1100
55
Genetic Algorithm Example III
Many applications of genetic algorithms.
Best suited to problems where the efficient solutions
are not already known.
Strength of GA's: ability to heuristically search for
solutions when all else fails.
If you can represent the solutions to the problem
in a suitable format, such as a series of 1's and 0's,
then the GA will do the rest.
Konstanz, May 2014
AI Search Algorithms – Population Based
56
Genetic Algorithm Example III
Applying
Genetic Algorithms
to Personnel Scheduling
Personnel scheduling in healthcare
is usually a very complex operation
which has a profound effect upon the
efficient usage of expensive resources.
Konstanz, May 2014
AI Search Algorithms – Population Based
57
Genetic Algorithm Example III
A number of nurses
A number of shifts each day
A set of constraints
Konstanz, May 2012
AI Search Algorithms – Population Based
shift coverage
one shift per day
resting time
workload per
month
consecutive
shifts
working
weekends
…
58
Genetic Algorithm Example III
Konstanz, May 2012
AI Search Algorithms – Population Based
59
Genetic Algorithm Example III
Genetic Algorithm
- Initial population
- construct rosters
- repair infeasible ones
Konstanz, May 2012
AI Search Algorithms – Population Based
60
Genetic Algorithm Example III
Genetic Algorithm
- Select parents
- Recombine rows in the two rosters
- repair infeasible ones
+
Konstanz, May 2012
AI Search Algorithms – Population Based
61
Genetic Algorithm Example III
Genetic Algorithm
- Mutation
- Local optimiser
Genetic Algorithm Example III
Population Size
50
Crossover Probability 0.7
Mutation Probability 0.001
Local Optimiser
ON
Konstanz, May 2014
AI Search Algorithms – Population Based
63