Download POEMS

Iterative Prototype Optimisation
with Evolved Improvement Steps
Jiří Kubalík and Jan Faigl
Czech Technical University
Department of Cybernetics
Technicka 2, 166 27, Prague, Czech Republic
email: {kubalik,xfaigl}@labe.felk.cvut.cz
Motivation

Typically,
evolutionary optimisation framework considers an evolution of a
population of candidate solutions to the problem at hand
 candidate solution encodes a complete solution (a complete set of
problem control parameters, a complete schedule in JSP, a complete
tour for TSP, etc.)
 the optimal or well-fit solution is hard to find for large problem
instances


Here,
EA does not handle the solved problem as a whole
 EA is employed within the iterative optimisation framework
 its role is to evolve the best modification of the current solution
prototype in each iteration

‹#›
Iter. Prototype Opt. with Evolved Imp. Steps
Outline of POEMS algorithm
generate(Prototype);
repeat
BestSequence  run_EA(Prototype);
if(apply_to(BestSequence, Prototype) is_better_than Prototype);
then Prototype  apply_to(BestSequence, Prototype);
until(POEMS termination condition);
return Prototype;

‹#›
Prototype initialised randomly or using some problem-specific knowledge
Iter. Prototype Opt. with Evolved Imp. Steps
Implementation Issues

Action Sequence Representation

Actions – set of primitive actions
 determines the explorative power of the algorithm
 linear chromosomes of maximal length MaxGenes << ProblemSize
 gene = (action_type, parameters)
 nop (no operation)
 void action with no effect on the prototype, regardless of the values
of its parameters
 one or more nop actions allowed in a chromosome
 variable effective length of chromosomes
 At least one action must be active
Ex.: MaxGenes =5, Actions ={action1, action2, action3}
Ch = (action3, 0.8),(nop, 0.7),(action2, 0.2),(nop, 0.5),(action2, 0.2)
‹#›
Iter. Prototype Opt. with Evolved Imp. Steps
Implementation Issues

Genetic Operators

Tournament selection

Crossover – generalised uniform
 Given two parental chromosomes, any combination of their actions
forms a valid offspring
Ex.:
Par1 = (action3, 0.8),(nop, 0.7),(action1, 0.2),(nop, 0.5),(action2, 0.2)
Par2 = (nop, 0.8),(action1, 0.8),(action3, 0.6),(nop, 0.5),(nop, 0.5)
Off1 = (action1, 0.2),(action3, 0.6),(nop, 0.7),(action1, 0.2),(nop, 0.5)
Off2 = (nop, 0.8),(action2, 0.2),(action3, 0.6),(nop, 0.5),(action_1, 0.8)

Mutation – action_type or parameters changed (1 gene per sequence)

Optionally, multiple duplicates of the same genes are eliminated
Off1 = (action1, 0.2),(action3, 0.6),(nop, 0.7),(action1, 0.2),(nop, 0.5)
Off1’ = (action1, 0.2),(action3, 0.6),(nop, 0.7),(nop, 0.2),(nop, 0.5)
‹#›
Iter. Prototype Opt. with Evolved Imp. Steps
Implementation Issues

Evaluation criterion
maximizes improvement of the prototype
 penalizes sequences that do not change the prototype


Evolutionary Model
EA should converge fast, since it runs many times through the whole run
 Generational / Steady-state

‹#›
Iter. Prototype Opt. with Evolved Imp. Steps
Standard Generational EA
initialize(OldPop);
BestSequence  best_of(OldPop);
repeat
NewPop  BestSequence;
// elitism
repeat
Parents  select(OldPop);
Children  cross_over(Parents);
mutate(Children);
evaluate(Children);
NewPop  Children;
until(NewPop is completed);
BestSequence = best_of(NewPop);
switch(OldPop, NewPop);
until(EA termination condition);
return BestSequence;
‹#›
Iter. Prototype Opt. with Evolved Imp. Steps
Steady-state EA
initialize(Population);
repeat
Parents  select(Population);
Children  cross_over(Parents);
mutate(Children);
evaluate(Children);
Replacement = find replacement(Population);
Population[Replacement]  Child1;
Replacement = find replacement(Population);
Population[Replacement]  Child2;
until(EA termination condition);
BestSequence  best of(Population);
return BestSequence;
‹#›
Iter. Prototype Opt. with Evolved Imp. Steps
TSP: Experimental Setup

Primitive Actions
 move(city1, city2) moves city1 right after city2 in the tour
 invert(city1, city2) inverts a subtour between city1 and city2
 swap(city1, city2) swaps city1 and city2

POEMS configuration:





chromosome length: 10
population size: Cities ×2
EA evaluations: 10.000
POEMS evaluations: Cities ×1000
Other algorithms
parameters set as recommended
 use the same number of fitness evaluations, if possible

‹#›
Iter. Prototype Opt. with Evolved Imp. Steps
Heuristic used for tour initialisation

Only links between cities from specified neighborhood are
considered when generating




‹#›
the prototype in POEMS,
tours form the initial population (E-R), and
the initial tour for 2-opt.
No heuristic used for initialising the ring of neurons for SOM
Iter. Prototype Opt. with Evolved Imp. Steps
TSP: Test Datasets

TSP Data – uniformly sampled points
100 cities
200 cities
1000 cities
500 cities
‹#›
2000 cities
Iter. Prototype Opt. with Evolved Imp. Steps
POEMS: Random vs. Heuristic Init.

100 cities
4738.5  831.9
1029.4  818.6
100 cities
100 cities
1050
4500
1000
fitness
fitness
3500
2500
900
1500
850
500
800
0
‹#›
950
25000
50000
75000
fitness evaluations
100000
0
25000
50000
fitness evaluations
POEMS
E-R
SOM
2-opt
Mean
818.6
935.1
830.4
853.5
StDev
14.5
27.1
13.0
18.2
BestPath
786.1
884.1
811.7
797.3
75000
100000
Iter. Prototype Opt. with Evolved Imp. Steps
Random vs. Heuristic initialization

200 cities
9574.2  1190.1
1357.4  1132.9
200 cities
200 cities
11000
1400
1350
9000
fitness
fitness
1300
7000
5000
1250
1200
3000
1150
1000
1100
0
‹#›
50000
100000
150000
fitness evaluations
200000
0
50000
100000
150000
fitness evaluations
POEMS
E-R
SOM
2-opt
Mean
1132.9
1406.1
1155.4
1196.1
StDev
16.2
54.6
12.4
24.1
BestPath
1098.1
1289.1
1124.7
1149.2
200000
Iter. Prototype Opt. with Evolved Imp. Steps
Random vs. Heuristic initialization

500 cities
25520.1  6234.9
2134.1  1746.3
500 cities
500 cities
2200
25000
2100
fitness
fitness
20000
15000
2000
1900
10000
1800
5000
1700
0
100000
200000
300000
400000
500000
0
fitness evaluations
‹#›
100000
200000
300000
fitness evaluations
POEMS
E-R
SOM
2-opt
Mean
1746.3
2753.3
1776.0
1866.8
StDev
13.5
66.4
14.0
123.1
BestPath
1718.0
2666.6
1751.2
1772.9
400000
500000
Iter. Prototype Opt. with Evolved Imp. Steps
Random vs. Heuristic initialization

1000 cities
51117.9  17997.0
2946.8  2523.0
1000 cities
1000 cities
55000
3000
2900
2800
fitness
fitness
45000
35000
25000
2600
2500
15000
0
‹#›
2700
250000
500000
750000
fitness evaluations
1000000
0
250000
500000
750000
1000000
fitness evaluations
POEMS
E-R
SOM
2-opt
Mean
2523.0
3799
2533.0
2650.1
StDev
15.0
91.4
12.0
160.0
BestPath
2491.6
3678.8
2508.4
2572.1
Iter. Prototype Opt. with Evolved Imp. Steps
Random vs. Heuristic initialization

2000 cities
102331.6  34878.0
4344.5  3692.2
2000 cities
2000 cities
4500
100000
4300
fitness
fitness
80000
60000
4100
3900
40000
3700
3500
20000
0
‹#›
500000
1000000
fitness evaluations
1500000
2000000
0
500000
1000000
1500000
2000000
fitness evaluations
POEMS
E-R
SOM
2-opt
Mean
3692.2
5983.5
3725.3
3908.1
StDev
20.4
95.1
14.9
122.8
BestPath
3655.1
5875.8
3695.6
3789.2
Iter. Prototype Opt. with Evolved Imp. Steps
Example of POEMS execution: TSP
Initial tour 965.134
after iteration 1
after iteration 2
after iteration 3
Final tour 824.8
‹#›
Iter. Prototype Opt. with Evolved Imp. Steps
Example of Evolved Solution
Initial prototype of length 4403
‹#›
Initial prototype of length 3655
Iter. Prototype Opt. with Evolved Imp. Steps
POEMS using Single Elem. Function
‹#›
action_type
Mean
StDev
BestPath
invert
2554.7
13.3
2526.2
move
2689.1
20.1
2653.6
swap
2824.8
31.5
2775.1
all
2523.0
15.0
2491.6
Iter. Prototype Opt. with Evolved Imp. Steps
Binary String Opt.: Test Problems

OneMax – counts 1s in a chromosome



Ch_length = 100
Ch_length = 4 × 2 × 12
F(x’1, x’2) < 0.001 is optimum
F103




Ch_length = 5 × 2 × 10
F(x’1, x’2) < 0.001 is optimum
Hierarchical If and only If

Ch_length = 128 bits
Royal Road Problem

‹#›
Deceptive function – 100 bits
Rosenbrock



Ch_length = 96 bits
Iter. Prototype Opt. with Evolved Imp. Steps
Binary String Opt.: Experimental Setup

Primitive Actions


invert(gene) … inverts the specified gene of the prototype
Linkage
tight – groups of dependent genes form compact clusters within the
chromosome
 loose – dependent genes are loosely coupled within the chromosome


POEMS configuration:






chromosome length: 10
population size: 200
EA evaluations: 3.000
POEMS evaluations: 106
Evolutionary model: generational, steady-state
Other algorithms
parameters set as recommended
 use the same number of fitness evaluations, if possible

‹#›
Iter. Prototype Opt. with Evolved Imp. Steps
Results: Loose Linkage
Problem
OneMax
DF3
Rosenbrock
F103
RR
H-IFF
‹#›
Alg.
Mean
StDev
#Succ
When
GBAM
100
0
50
10847
POEMS
100
0
50
12168
GBAM
728.3
6.9
0
-
POEMS
750
0
50
307820
GALCO
0.36
0.26
0
-
POEMS
0.029
0.14
17
516988
SGA
0.019
0.0156
0
-
0.0061
1
769400
POEMS 0.0063
GALCO
43.5
8.6
0
-
POEMS
6.7
9.7
0
-
GBAM
625.6
38.1
0
-
POEMS
580.8
15.2
0
-
Iter. Prototype Opt. with Evolved Imp. Steps
Real-valued Opt.: Implementation Issues

Primitive Actions



add(value) … adds value (maxi  xi) to the respective prototype
variable xi, where maxi is an upper bound of variable xi.
 value is initialised from interval (0.0, 1.0).
sub(value) … subtracts value (xi  mini) from xi.
sample_right(value) … picks a new value of the respective prototype
variable xi, from interval (xi , maxi).
 samples the interval (xi , maxi) non-linearly preferring smaller
changes to the larger ones.
 sample_left(value) … picks a new value of the respective prototype
variable xi, from interval (mini , xi).

Chromosome structure – an ordered list of D actions, each of them
operating on the respective prototype variable, where D is a dimension of
the problem.
‹#›
Iter. Prototype Opt. with Evolved Imp. Steps
Real-valued Opt.: Implementation Issues

Reuse of the evolved code

Initial population of EA in later iterations a portion of action from the
previous EA
 to keep the search direction that appeared beneficial in the past,
 to reuse the unexploited yet possibly useful material evolved in the
last iteration.

POEMS configuration:
chromosome length: D
 population size: 200
 EA evaluations: 1500 (10D), 2000 (30D)
 POEMS evaluations: D×104

‹#›
Iter. Prototype Opt. with Evolved Imp. Steps
Real-valued Opt.: Experiments

10D and 30D functions collected for the Special Session on
Real-parameter Optimization of the IEEE Congress on
Evolutionary Computation 2005.
25 functions in 4 groups
 P. N. Suganthan, N. Hansen, J. J. Liang, K. Deb, Y.-P. Chen, A. Auger,
and S. Tiwari, “Problem definitions and evaluation criteria for the CEC
2005 Special Session on Real-Parameter Optimization”, tech. rep.,
Nanyang Technological University, Singapore, May 2005.
http://www.ntu.edu.sg/home/epnsugan


POEMS compared to the algorithms presented at the Special Sesion.

Observations:



‹#›
POEMS exhibits about-average performance among the compared alg.
It is competitive to most of the algorithms on multimodal prob.
Improvement step size tends to decrease during the course of the run
Iter. Prototype Opt. with Evolved Imp. Steps
F1 Shifted Sphere Function
‹#›
Iter. Prototype Opt. with Evolved Imp. Steps
F9 Shifted Rastrigin’s Function
‹#›
Iter. Prototype Opt. with Evolved Imp. Steps
13 Shifted Expanded Griewank’s plus
Rosenbrock’s Function (F8F2)
‹#›
Iter. Prototype Opt. with Evolved Imp. Steps
23 Non-Continuous Rotated Hybrid
Composition Function 3
‹#›
Iter. Prototype Opt. with Evolved Imp. Steps
Conclusions

Pros and Cons


General optimisation approach for combinatorial, discrete, and realvalued optimisation problems
Linkage independent
 Inability to treat problems with building blocks of bigger size
 Local (point-to-point) search approach – prone to get stuck in a local


optimum
The latter stage of the iterational algorithm is the harder it is to find any
improvement modification ( local search)
Scope of possible applications
Rescheduling – insert a new order into the current schedule
 SAT problem – dependencies among variables are necessary for
effective searching for the optimum solution (but not known a priori)

‹#›
Iter. Prototype Opt. with Evolved Imp. Steps