Download DXNN: Evolving Complex Organisms in Complex Environments

DXNN: Evolving Complex Organisms in Complex Environments Using a Novel TWEANN System
Gene Sher
Independent Researcher
[email protected]
ABSTRACT
and diversity. Finally, DXNN's simple built-in feature selection
system allows it to evolve links to new sensors and actuators as
they become available, making DXNN a perfect fit for the
complex artificial life (ALife) and robotics problems.
The goal of this paper is twofold. First, to briefly present a novel
type of memetic algorithm based Topology and Weight Evolving
Artificial Neural Network (TWEANN) system called DXNN,
among whose numerous novel features are: a simple and
database friendly tuple based NN encoding method, a two phase
neuroevolutionary approach which produces high diversity
populations, a new “Targeted Tuning Phase” aimed at dealing
with “the curse of dimensionality”, and a new Random Intensity
Mutation (RIM) method that removes the need for cross-over
algorithms. Second, to discuss the excellent experimental results
of applying DXNN to co-evolutionary artificial life simulations.
2.
DXNN ALGORITHM
The DXNN Platform's learning algorithm is divided into two
main phases. The “Tuning Phase”, during which the NNs
interact with the environment or some problem, and undergo
parametric mutations. Followed by the “Topological Mutation
Phase”, during which mutational operators are applied to
DXNN, affecting its topology, and the various parameters of the
system in general. Afterwards, the fit NNs and their offspring
are again applied to the problem domain, repeating the cycle.
Categories and Subject Descriptors
I.2.6 [Computing Methodologies]: Artificial Intelligence –
Learning, Connectionism and neural nets.
3.
3.1
NOVEL FEATURES
Tuning Phase Features
General Terms
3.1.1
Targeted Tuning
Algorithms
Rather than blindly perturbing the weights of randomly selected
neurons, DXNN concentrates only on the most recently
topologically augmented elements; the neurons added or
modified within the last 2 generations. This pool of neurons is
called New Generation Neurons (NGN).
Keywords
DXNN, Neuroevolution, Memetic Algorithm, Artificial Life.
1.
INTRODUCTION
3.1.2
Parametric Random Intensity Mutation
Using uniform distribution for all selections, the system selects
a random number of neurons from the NGN. Afterwards, for
each selected neuron, a random number of weights are selected
and perturbed with a magnitude between (-PI/2 and PI/2).
Neural Networks (NNs) are universal function approximators
capable of modeling complex mappings between inputs and
outputs. As the problem posed to the NN increases in
complexity, the complexity of the NN must also increase, and
with it the difficulty of its training. Topology and Weight
Evolving Artificial Neural Networks (TWEANN) are NN
systems that are able to not only learn through weight
optimization, but also through growing, expanding, and evolving
problem-appropriate topological structures.
3.1.3
Dynamic Tuning Time
After each neuron is perturbed, the NN is once again applied to
the problem. Depending on the resulting fitness, the new
weights are either saved, or reverted back to the previously
higher fitness weights. Afterwards, P-RIM is once more applied
to the NN. P-RIM is applied until the NN consecutively fails to
improve its fitness at least: X = B + sqrt(TotWeights in NGN)
number of times, where B is set by the researcher. In this
manner, the computational power expanded on tuning is
proportional to NN complexity, and proximity to local optima.
Yet still there are a number of problems that continue to plague
even the most advanced of TWEANN systems: NEAT[2],
CoSyNe[3], HyperNEAT[4], EANT[5], and EANT2[6]. Among
these problems are the neural bloating, premature convergence,
low population diversity, and the curse of dimensionality.
The novel memetic algorithm based TWEANN, Discover &
eXplore Neural Network (DXNN)[1], briefly discussed in this
paper, presents features that allow it to overcome the above
mentioned adversities. DXNN allows us to further accelerate the
production of topological solutions to various problems, while at
the same time consistently producing much more compact NNs
when compared to even the most advanced of the mentioned
TWEANN systems. Through its two phase approach to
neuroevolution and with the utilization of a Targeted Tuning
Phase, DXNN produces populations of ever increasing fitness
3.2
Topological Mutation Phase Features
3.2.1
DXNN Selection Algorithm
After all NNs have finished with their Tuning Phase, the
population enters the topological mutation phase. The organisms
are first sorted based on fitness within their own specie groups.
After the lowest scoring 50% of the NNs within each specie is
removed, every NN is allowed to create Y number of offspring,
where Y is inversely proportional to the NN size, such that all
things being equal, the simpler NNs are allowed to create larger
number of offspring than their more complex cousins. A feature
that prevents unnecessary NN bloating.
Copyright is held by the author/owner(s).
GECCO'11, July 12-16, 2011, Dublin, Ireland.
ACM 978-1-4503-0690-4/11/07.
149
3.2.2
Topological Random Intensity Mutation
The Artificial Life experiments were of 4 types. Simple food
gathering, where the “Prey” robots moved around and had to
learn to eat green plants. Dangerous food gathering, where the
robots had to eat green plants and avoid black poisonous plants
scattered amongst the plants. A Predator vs. Prey simulation,
where the organisms were separated into 2 morphologically
different species, and then co-evolved. In this simulation the
prey gained energy by eating plants, and the predators energized
by eating the prey. The fitness ranking was done for each specie
separately, yielding morphologically local competitions. Finally,
a wall navigation robot simulation was conducted, where the
food was scattered on a 2d plane, walls were added, and the prey
were seeded with only the distance sensors. In this experiment
the robots evolved new connections to color sensors, and learned
to navigate around the walls to gather the food.
Like the parametric version (P-RIM), a Topological version of
the RIM is used during the topological mutation phase. To every
offspring, an X number of topological mutations is applied,
where X is a random number between 1 and the square root of
neurons in the parent NN. The mutation operators are uniformly
chosen from the following list: Add Neuron in parallel, add
Neuron in series, add a connection between two neurons, change
activation function, change neural plasticity function, and finally
Add Sensor/Actuator, where a new random sensor or actuator is
chosen from an available list and then connected to the NN.
3.3
Discussion
Targeted tuning allows DXNN to concentrate on elaborating and
complexifying the NN, rather than randomly rearranging and
optimizing all pieces in hope it produces a fit system. This
approach is similar to the way natural evolution works, the
neocortex for example has evolved on top of the more
primordial systems, working in parallel with these ancient
structures. In the same way, targeted tunning molds the newly
added structures to work in conjunction with the already existing
ones, rather than re-optimizing everything randomly.
In all simulations, the robots learned to gather food within the
first 5000 evaluations. In the predator-prey simulations, the
behaviors became particularly organic, the predators learned to
push plants around, baiting the prey towards the plants, and then
attacking them. Recordings of these 2d simulations were
uploaded to YouTube: www.youtube.com/user/DXNNsystem.
7.
The selection algorithm takes into account NN size and
complexity, and the fact that it takes less resources to produce
small NNs, than it does larger ones. In this manner the DXNN
platform allows for larger structures to win over the more
concise topologies, only if they add comparable fitness gains.
Finally, because of the two phase approach, high population
diversity is always maintained. After the topological mutation
phase, almost every NN has a different topology due to every
offspring being created through a Topological RIM.
4.
DPB BENCHMARK RESULTS
When DXNN was applied to the Double Pole Balancing (DPB)
benchmarks, it outperformed other TWEANNs, and produced
exceptionally concise, 2-3 neuron, solutions. The detailed results
of these benchmarks & oblation experiments are found in [1].
5.
8.
REFERENCES
[1] Sher, G., (preprint): DXNN Platform: Shedding the
Biological Inefficiencies. arXiv:1011.6022v3 [cs.NE]
POPULATION DIVERSITY
[2] Stanley, K.O., Miikkulainen, R.: Evolving neural networks
through augmenting topologies. Evolutionary Computation
10(2) (2002) 99–127
A NN is considered topologically different from the others if it
has either a different number of input or output connections, a
different number of neurons in total, or a different set of
activation functions. When diversity was calculated for DXNN
every 500 evaluations, over hundreds of generations when
applied to DPB and ALife, the populations consistently
maintained greater than 70% diversity in every simulation run.
This means that every 2 in 3 NN differ topologically from one
another. Due to the 2 phase approach, DXNN Platform simply
does not allow low diversity, or premature convergence, the top
50% of the population will always be diverse, and the created
offspring will always differ from their parents due to T-RIM,
thus diversity is naturally maintained.
6.
CONCLUSION
In this paper I briefly discussed the DXNN Platform, a novel
memetic algorithm based Topology and Weight Evolving
Artificial Neural Network system. DXNN utilizes a database
friendly, tuple based, and human readable genotype encoding.
The system demonstrated its superior performance on a number
of problems, and consistently produced exceptional population
diversity profiles. Finally, DXNN was applied to various ALife
experiments, including food foraging and predator-prey
simulations, where it evolved interesting, and clever organisms
that learned to exploit its environment rapidly and with ease. A
detailed discussion of DXNN's architecture, its features, and
experimental results is available in [1].
[3] Faustino Gomez, Jurgen Schmidhuber, Risto Miikkulainen:
Accelerated Neural Evolution through Cooperatively
Coevolved Synapses. Journal of Machine Learning
Research 9 (2008) 937-965
[4] Gauci, J., Stanley, K.O.: Generating large-scale neural
networks through discovering geometric regularities. In:
Proceedings of the 9th annual conference on Genetic and
evolutionary computation. pp. 997–1004. ACM, New York,
NY (2007)
[5] Kassahun, Y., Sommer, G.: Efficient reinforcement
learning through evolutionary acquisition of neural
topologies. In: Proceedings of the 13th European
Symposium on Artificial Neural Networks (ESANN 2005),
Bruges, Belgium (2005) 259–266
ARTIFICIAL LIFE RESULTS
Using a modified continues selection algorithm, DXNN was also
applied to the 2d artificial life simulations. In these simulations
the NNs controlled 2d robots with differential drives, ability to
attack other robots, and ability to eat plants. In this simulated
environment, every move the robot makes costs energy, if the
robot loses all its energy, it dies. A robot also dies if it is eaten
by a predator robot, or when it gets too old (after 20000 steps).
[6] Nils T Siebel and Sommer, G.: Evolutionary reinforcement
learning of artificial neural networks. International Journal
of Hybrid Intelligent Systems 4(3): 171-183, October 2007.
150