Download Evolutionary Computation

EVOLUTIONARY COMPUTATION
-1-
EVOLUTIONARY COMPUTATION
BOOKS:
Christian Jacob (2001) Illustrating Evolutionary
Computation with Mathematica. Morgan Kaufmann.
Peter J. Bentley (2001) DIGITAL BIOLOGY “Evolution
chapter” (3).
EXTRA READING:
David Goldberg (1989) Genetic Algorithms in Search,
Optimization and Machine Learning. Addison-Wesley.
Melanie Mitchell (1996) An Introduction to Genetic Algorithms.
MIT Press.
© 2001 Peter J. Bentley. Slide for UCL M.Sc. Evolutionary Computation course.
EVOLUTIONARY COMPUTATION
-2-
Evolutionary Computation
Evolution can modify genes and organisms, words and
languages.
It can also modify binary digits.
© 2001 Peter J. Bentley. Slide for UCL M.Sc. Evolutionary Computation course.
EVOLUTIONARY COMPUTATION
-3-
If we set up a population of individuals in a computer,
allow some to be selected by some criterion
and allow those individuals to produce children
which inherit features from their parents with some
variation
and then select those according to the criterion and so
on…
…you get evolution in a computer.
© 2001 Peter J. Bentley. Slide for UCL M.Sc. Evolutionary Computation course.
EVOLUTIONARY COMPUTATION
-4-
Evolutionary computation is the field of computer science
(related to artificial intelligence or artificial life) that studies
programs that evolve solutions to problems.
Evolutionary algorithms come in many different varieties,
but underneath the differences, they are all the same.
They all have reproduction with inheritance, variation and
selection.
© 2001 Peter J. Bentley. Slide for UCL M.Sc. Evolutionary Computation course.
EVOLUTIONARY COMPUTATION
-5-
There are four major types of evolutionary algorithm
commonly used by researchers and in industry:
Genetic Algorithms (GAs)
Genetic Programming (GP)
Evolutionary Strategies (ES)
Evolutionary Programming (EP)
GAs, ES and EP were all independently ‘discovered’ by
researchers in the 1960s. GP was created in the early
1990s, and is a specialised type of GA.
Of them GAs and GP are perhaps the most commonly
used, so this course will focus on these.
© 2001 Peter J. Bentley. Slide for UCL M.Sc. Evolutionary Computation course.
EVOLUTIONARY COMPUTATION
-6-
What do we evolve?
Genetic algorithms (GAs) were first created by John
Holland in the 1960s.
They only became commonly used by researchers in the
late 1980s and 1990s.
Today hundreds of researchers have applied GAs to
countless problems.
Applications include: fraud detection systems, robot
controllers, schedulers, design of architecture,
composition of music, and even investigations of how life
began.
We’ll look at three here, to give you an idea of what can be
achieved:
 Axelrod’s Iterated Prisoner’s Dilemma
 Karl Sims’ evolved “virtual creatures”.
 Steven Rooke’s evolutionary art
© 2001 Peter J. Bentley. Slide for UCL M.Sc. Evolutionary Computation course.
EVOLUTIONARY COMPUTATION
-7-
Axelrod’s Iterated Prisoner’s Dilemma
The Iterated Prisoner’s Dilemma is a little game that is
used to represent hundreds of similar situations in real life.
It can be used to model reciprocal bird preening
behaviour, countries trading together, and many other
systems.
The prisoner’s dilemma itself is very simple:
Imagine two prisoners are being interrogated in separate
rooms.
Each has a choice: either cooperate with his partner and
tell the same story, or defect and turn the other in. And
neither knows what the other will do.
© 2001 Peter J. Bentley. Slide for UCL M.Sc. Evolutionary Computation course.
EVOLUTIONARY COMPUTATION
-8-
So there are four possible outcomes:
Prisoner 2 defects
Prisoner 2 cooperates
Prisoner 1 defects
P1:
1
P2:
1
P1:
5
P2:
0
Prisoner 1 cooperates
P1:
0
P2:
5
P1:
3
P2:
3
If they both defect and turn each other in, they’ll both do
badly.
If Prisoner 1 defects, but Prisoner 2 cooperates, then
Prisoner 1 will do very well, and Prisoner 2 will do very
badly.
Likewise, if Prisoner 2 defects, but Prisoner 1 cooperates,
then Prisoner 2 will do very well, and Prisoner 1 will do
very badly.
Finally, if they both cooperate and tell the same story, both
will do OK (but not as well as they could have done if one
had defected).
The dilemma is thus: what should each prisoner do?
© 2001 Peter J. Bentley. Slide for UCL M.Sc. Evolutionary Computation course.
EVOLUTIONARY COMPUTATION
-9-
The Iterated Prisoner’s Dilemma is exactly the same
situation, played over and over again.
When you play the same game a number of times, it is
possible to build up knowledge about your opponent.
For example, if he/she always seems to defect, then your
best bet is to defect as well.
But if he/she always seems to cooperate, then perhaps
you should try defecting and go for a higher payoff.
The problem is thus: how do you build a playing strategy
which best makes use of your knowledge of your
opponent’s previous moves?
© 2001 Peter J. Bentley. Slide for UCL M.Sc. Evolutionary Computation course.
EVOLUTIONARY COMPUTATION
- 10 -
Robert Axelrod held several competitions, where people
had to write computer programs that played each other.
He discovered that one of the best strategies was “Tit for
Tat”.
This strategy always began by cooperating, but then
simply played the previous move of the other player.
So if your opponent cooperated, you cooperate in the next
game.
If your opponent defected, you defect in the next game,
and so on.
© 2001 Peter J. Bentley. Slide for UCL M.Sc. Evolutionary Computation course.
EVOLUTIONARY COMPUTATION
- 11 -
Axelrod then did something more interesting.
He used a genetic algorithm to evolve strategies, and
made them play all of the other programs submitted to the
competition.
Evolution selected those strategies that were better able
to win the IPD, and made them have more children.
Because the children inherited genetic material from their
parents (binary genes), and because these genes were
randomly mutated now and again, they resembled their
parents with some variation.
Selection with inheritance and variation caused better and
better IPD strategies to evolve.
The result was a strategy that behaved in a very similar
way to Tit for Tat – evolution agreed with Axelrod’s
previous finding.
© 2001 Peter J. Bentley. Slide for UCL M.Sc. Evolutionary Computation course.
EVOLUTIONARY COMPUTATION
- 12 -
Other researchers continued his research.
Instead of using a GA to evolve a strategy that would win
when playing a set of fixed strategies, researchers have
co-evolved strategies that play each other.
What kind of strategy almost always evolved?
Co-operation from both players.
Can you work out why?
(We’ll return to this in later lectures.)
© 2001 Peter J. Bentley. Slide for UCL M.Sc. Evolutionary Computation course.
EVOLUTIONARY COMPUTATION
- 13 -
Karl Sims’ virtual creatures
(chapter 13 in Evolutionary Design by Computers edited by Peter J Bentley)
Karl is an artist and specialist in computer graphics –
today he develops special effects for movies.
Karl had the idea of using a genetic algorithm rather like
evolution in nature: use it to evolve creatures that could
move themselves around in an environment.
This, of course, was very complicated. He needed a
model of the world including forces, moments, inertia,
gravity, friction, water-resistance, and light.
He also needed to set up “digital DNA” or genetic
encodings that would be modified by the GA
The genetic encoding would define the neural networks of
the creatures (their brains) and their morphologies (their
bodies).
© 2001 Peter J. Bentley. Slide for UCL M.Sc. Evolutionary Computation course.
EVOLUTIONARY COMPUTATION
- 14 -
Karl “asked” his genetic algorithm to evolve creatures that
were good at moving and that could follow a light source.
So the faster they could propel themselves (or the better
they could find the light), the more children they had.
As usual, the children inherited genes from their parents,
and there was variation caused by random mutations.
The results caused a huge impact on computer scientists,
biologists (and the media at the time).
© 2001 Peter J. Bentley. Slide for UCL M.Sc. Evolutionary Computation course.
EVOLUTIONARY COMPUTATION
- 15 -
Turtle-like creatures evolved, that could propel themselves
through the water with flippers and follow a red light:
(stills from Sims’ movie)
© 2001 Peter J. Bentley. Slide for UCL M.Sc. Evolutionary Computation course.
EVOLUTIONARY COMPUTATION
- 16 -
Creatures walked over the ground like inch-worms:
Snake-like creatures evolved, that undulated through the
water:
© 2001 Peter J. Bentley. Slide for UCL M.Sc. Evolutionary Computation course.
EVOLUTIONARY COMPUTATION
- 17 -
Some more snake-like creatures:
© 2001 Peter J. Bentley. Slide for UCL M.Sc. Evolutionary Computation course.
EVOLUTIONARY COMPUTATION
- 18 -
As well as having complex bodies, all of Karl’s virtual
creatures had immensely complicated neural networks,
enabling them to move and sense within their
environments.
If you want to play with a similar system, look for Jeffery
Ventrella’s free software “Gene Pool” or “Darwin Pond” at
www.ventrella.com – it evolves 2D water creatures in real
time on your PC.
© 2001 Peter J. Bentley. Slide for UCL M.Sc. Evolutionary Computation course.
EVOLUTIONARY COMPUTATION
- 19 -
Steven Rooke’s evolutionary art
(chapter 10 in Creative Evolutionary Systems edited by Peter J Bentley and
David W Corne)
As mentioned earlier, genetic programming (GP) is a
newer type of genetic algorithm.
The idea of genetic programming comes from ideas of
automatic programming, or enabling computers to
program themselves.
GP does not evolve game-playing strategies or virtual
creatures – it evolves computer programs.
© 2001 Peter J. Bentley. Slide for UCL M.Sc. Evolutionary Computation course.
EVOLUTIONARY COMPUTATION
- 20 -
Normally, a GP system is given a series of exemplars –
examples of what a program should output for different
inputs.
GP then has to go away and evolve the right set of
instructions to make a computer program, which when run,
produces a matching set of outputs for the different inputs.
As before, fitter individuals (programs) have more children,
the children inherit genes from their parents with some
random mutations.
And evolution writes novel computer programs.
© 2001 Peter J. Bentley. Slide for UCL M.Sc. Evolutionary Computation course.
EVOLUTIONARY COMPUTATION
- 21 -
Steven Rooke had the idea of getting a computer to
generate art.
Rather than write a program which just displayed a pretty
picture (e.g. a fractal), Steven wanted a program that
could generate pictures that he liked.
So he used genetic programming to evolve computer
programs that displayed pictures.
© 2001 Peter J. Bentley. Slide for UCL M.Sc. Evolutionary Computation course.
EVOLUTIONARY COMPUTATION
- 22 -
To judge how effective the programs were as they
evolved, he simply judged how much he liked the pictures
they produced.
A nicer picture (in his opinion) meant that the
corresponding program was awarded a higher fitness
score.
The programs with better fitnesses had more children, the
children inherited genes from their parents, etc, etc.
The evolved result was a computer program that
generated a very nice (or at least interesting) image.
And a different program (and hence a different image) was
evolved every time the GP system was run.
© 2001 Peter J. Bentley. Slide for UCL M.Sc. Evolutionary Computation course.
EVOLUTIONARY COMPUTATION
- 23 -
Here’s what some looked like:
"In the Beginning" © 1996 Steven Rooke 1996, 427 nodes, fractals are blackbox.
"Skaters" © 1997 Steven Rooke, 81 nodes, fractals are black-box.
© 2001 Peter J. Bentley. Slide for UCL M.Sc. Evolutionary Computation course.
EVOLUTIONARY COMPUTATION
- 24 -
"Pearly Gates", © 1999 Steven Rooke, true genetic fractal.
© 2001 Peter J. Bentley. Slide for UCL M.Sc. Evolutionary Computation course.