Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Embodied cognitive science wikipedia , lookup
Existential risk from artificial general intelligence wikipedia , lookup
Catastrophic interference wikipedia , lookup
Gene expression programming wikipedia , lookup
History of artificial intelligence wikipedia , lookup
Convolutional neural network wikipedia , lookup
DXNN: Evolving Complex Organisms in Complex Environments Using a Novel TWEANN System Gene Sher Independent Researcher [email protected] ABSTRACT and diversity. Finally, DXNN's simple built-in feature selection system allows it to evolve links to new sensors and actuators as they become available, making DXNN a perfect fit for the complex artificial life (ALife) and robotics problems. The goal of this paper is twofold. First, to briefly present a novel type of memetic algorithm based Topology and Weight Evolving Artificial Neural Network (TWEANN) system called DXNN, among whose numerous novel features are: a simple and database friendly tuple based NN encoding method, a two phase neuroevolutionary approach which produces high diversity populations, a new “Targeted Tuning Phase” aimed at dealing with “the curse of dimensionality”, and a new Random Intensity Mutation (RIM) method that removes the need for cross-over algorithms. Second, to discuss the excellent experimental results of applying DXNN to co-evolutionary artificial life simulations. 2. DXNN ALGORITHM The DXNN Platform's learning algorithm is divided into two main phases. The “Tuning Phase”, during which the NNs interact with the environment or some problem, and undergo parametric mutations. Followed by the “Topological Mutation Phase”, during which mutational operators are applied to DXNN, affecting its topology, and the various parameters of the system in general. Afterwards, the fit NNs and their offspring are again applied to the problem domain, repeating the cycle. Categories and Subject Descriptors I.2.6 [Computing Methodologies]: Artificial Intelligence – Learning, Connectionism and neural nets. 3. 3.1 NOVEL FEATURES Tuning Phase Features General Terms 3.1.1 Targeted Tuning Algorithms Rather than blindly perturbing the weights of randomly selected neurons, DXNN concentrates only on the most recently topologically augmented elements; the neurons added or modified within the last 2 generations. This pool of neurons is called New Generation Neurons (NGN). Keywords DXNN, Neuroevolution, Memetic Algorithm, Artificial Life. 1. INTRODUCTION 3.1.2 Parametric Random Intensity Mutation Using uniform distribution for all selections, the system selects a random number of neurons from the NGN. Afterwards, for each selected neuron, a random number of weights are selected and perturbed with a magnitude between (-PI/2 and PI/2). Neural Networks (NNs) are universal function approximators capable of modeling complex mappings between inputs and outputs. As the problem posed to the NN increases in complexity, the complexity of the NN must also increase, and with it the difficulty of its training. Topology and Weight Evolving Artificial Neural Networks (TWEANN) are NN systems that are able to not only learn through weight optimization, but also through growing, expanding, and evolving problem-appropriate topological structures. 3.1.3 Dynamic Tuning Time After each neuron is perturbed, the NN is once again applied to the problem. Depending on the resulting fitness, the new weights are either saved, or reverted back to the previously higher fitness weights. Afterwards, P-RIM is once more applied to the NN. P-RIM is applied until the NN consecutively fails to improve its fitness at least: X = B + sqrt(TotWeights in NGN) number of times, where B is set by the researcher. In this manner, the computational power expanded on tuning is proportional to NN complexity, and proximity to local optima. Yet still there are a number of problems that continue to plague even the most advanced of TWEANN systems: NEAT[2], CoSyNe[3], HyperNEAT[4], EANT[5], and EANT2[6]. Among these problems are the neural bloating, premature convergence, low population diversity, and the curse of dimensionality. The novel memetic algorithm based TWEANN, Discover & eXplore Neural Network (DXNN)[1], briefly discussed in this paper, presents features that allow it to overcome the above mentioned adversities. DXNN allows us to further accelerate the production of topological solutions to various problems, while at the same time consistently producing much more compact NNs when compared to even the most advanced of the mentioned TWEANN systems. Through its two phase approach to neuroevolution and with the utilization of a Targeted Tuning Phase, DXNN produces populations of ever increasing fitness 3.2 Topological Mutation Phase Features 3.2.1 DXNN Selection Algorithm After all NNs have finished with their Tuning Phase, the population enters the topological mutation phase. The organisms are first sorted based on fitness within their own specie groups. After the lowest scoring 50% of the NNs within each specie is removed, every NN is allowed to create Y number of offspring, where Y is inversely proportional to the NN size, such that all things being equal, the simpler NNs are allowed to create larger number of offspring than their more complex cousins. A feature that prevents unnecessary NN bloating. Copyright is held by the author/owner(s). GECCO'11, July 12-16, 2011, Dublin, Ireland. ACM 978-1-4503-0690-4/11/07. 149 3.2.2 Topological Random Intensity Mutation The Artificial Life experiments were of 4 types. Simple food gathering, where the “Prey” robots moved around and had to learn to eat green plants. Dangerous food gathering, where the robots had to eat green plants and avoid black poisonous plants scattered amongst the plants. A Predator vs. Prey simulation, where the organisms were separated into 2 morphologically different species, and then co-evolved. In this simulation the prey gained energy by eating plants, and the predators energized by eating the prey. The fitness ranking was done for each specie separately, yielding morphologically local competitions. Finally, a wall navigation robot simulation was conducted, where the food was scattered on a 2d plane, walls were added, and the prey were seeded with only the distance sensors. In this experiment the robots evolved new connections to color sensors, and learned to navigate around the walls to gather the food. Like the parametric version (P-RIM), a Topological version of the RIM is used during the topological mutation phase. To every offspring, an X number of topological mutations is applied, where X is a random number between 1 and the square root of neurons in the parent NN. The mutation operators are uniformly chosen from the following list: Add Neuron in parallel, add Neuron in series, add a connection between two neurons, change activation function, change neural plasticity function, and finally Add Sensor/Actuator, where a new random sensor or actuator is chosen from an available list and then connected to the NN. 3.3 Discussion Targeted tuning allows DXNN to concentrate on elaborating and complexifying the NN, rather than randomly rearranging and optimizing all pieces in hope it produces a fit system. This approach is similar to the way natural evolution works, the neocortex for example has evolved on top of the more primordial systems, working in parallel with these ancient structures. In the same way, targeted tunning molds the newly added structures to work in conjunction with the already existing ones, rather than re-optimizing everything randomly. In all simulations, the robots learned to gather food within the first 5000 evaluations. In the predator-prey simulations, the behaviors became particularly organic, the predators learned to push plants around, baiting the prey towards the plants, and then attacking them. Recordings of these 2d simulations were uploaded to YouTube: www.youtube.com/user/DXNNsystem. 7. The selection algorithm takes into account NN size and complexity, and the fact that it takes less resources to produce small NNs, than it does larger ones. In this manner the DXNN platform allows for larger structures to win over the more concise topologies, only if they add comparable fitness gains. Finally, because of the two phase approach, high population diversity is always maintained. After the topological mutation phase, almost every NN has a different topology due to every offspring being created through a Topological RIM. 4. DPB BENCHMARK RESULTS When DXNN was applied to the Double Pole Balancing (DPB) benchmarks, it outperformed other TWEANNs, and produced exceptionally concise, 2-3 neuron, solutions. The detailed results of these benchmarks & oblation experiments are found in [1]. 5. 8. REFERENCES [1] Sher, G., (preprint): DXNN Platform: Shedding the Biological Inefficiencies. arXiv:1011.6022v3 [cs.NE] POPULATION DIVERSITY [2] Stanley, K.O., Miikkulainen, R.: Evolving neural networks through augmenting topologies. Evolutionary Computation 10(2) (2002) 99–127 A NN is considered topologically different from the others if it has either a different number of input or output connections, a different number of neurons in total, or a different set of activation functions. When diversity was calculated for DXNN every 500 evaluations, over hundreds of generations when applied to DPB and ALife, the populations consistently maintained greater than 70% diversity in every simulation run. This means that every 2 in 3 NN differ topologically from one another. Due to the 2 phase approach, DXNN Platform simply does not allow low diversity, or premature convergence, the top 50% of the population will always be diverse, and the created offspring will always differ from their parents due to T-RIM, thus diversity is naturally maintained. 6. CONCLUSION In this paper I briefly discussed the DXNN Platform, a novel memetic algorithm based Topology and Weight Evolving Artificial Neural Network system. DXNN utilizes a database friendly, tuple based, and human readable genotype encoding. The system demonstrated its superior performance on a number of problems, and consistently produced exceptional population diversity profiles. Finally, DXNN was applied to various ALife experiments, including food foraging and predator-prey simulations, where it evolved interesting, and clever organisms that learned to exploit its environment rapidly and with ease. A detailed discussion of DXNN's architecture, its features, and experimental results is available in [1]. [3] Faustino Gomez, Jurgen Schmidhuber, Risto Miikkulainen: Accelerated Neural Evolution through Cooperatively Coevolved Synapses. Journal of Machine Learning Research 9 (2008) 937-965 [4] Gauci, J., Stanley, K.O.: Generating large-scale neural networks through discovering geometric regularities. In: Proceedings of the 9th annual conference on Genetic and evolutionary computation. pp. 997–1004. ACM, New York, NY (2007) [5] Kassahun, Y., Sommer, G.: Efficient reinforcement learning through evolutionary acquisition of neural topologies. In: Proceedings of the 13th European Symposium on Artificial Neural Networks (ESANN 2005), Bruges, Belgium (2005) 259–266 ARTIFICIAL LIFE RESULTS Using a modified continues selection algorithm, DXNN was also applied to the 2d artificial life simulations. In these simulations the NNs controlled 2d robots with differential drives, ability to attack other robots, and ability to eat plants. In this simulated environment, every move the robot makes costs energy, if the robot loses all its energy, it dies. A robot also dies if it is eaten by a predator robot, or when it gets too old (after 20000 steps). [6] Nils T Siebel and Sommer, G.: Evolutionary reinforcement learning of artificial neural networks. International Journal of Hybrid Intelligent Systems 4(3): 171-183, October 2007. 150