Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
CAP6938 Neuroevolution and Developmental Encoding Overview of NEAT and CPPNs Dr. Kenneth Stanley August 28, 2006 Why an Overview? • • • • Preparation for declaring projects Get an idea what is possible Sufficient background without details Technical details coming later in semester NEAT: NeuroEvoluton of Augmenting Topologies • Evolutionary algorithm applied to evolving artificial neural networks (ANNs) • Evolution (i.e. genetic algorithms) is a way to optimize ANN topology and weights • NEAT evolves increasingly complex ANNs • Able to solve difficult control and decisionmaking tasks Evolution:Survival of the Roundest Gen 1 Select as parents Gen 2 Select as parents Gen 3 Champ! NeuroEvolution • Evolve ANNs • Evaluated for performance on a task – – – – – – – Controlling a vehicle or robot Drawing a picture Recognizing an image Warning about danger Playing a board game Playing a video game Playing music • The fittest survive and reproduce ANNs can Sense and React NEAT Features • NEAT is unique: Networks complexify over evolution Crash Warning Champ • So they can find the right complexity for the task • NEAT solves problems with systems of diverse networks using historical marking and speciation • A first step in evolving complex systems NEAT Applications • NEAT makes new applications possible and has set performance records Double Pole Balancing Record Vehicle Warnings Keepaway Record NERO: Real-time Neuroevolution in a Video Game Robot Duel Go Hopper Not Just About Control and Classification Interactive Picture Evolution Interactive Drum Pattern Evolution Interactive Particle Effect Evolution Interactive Chord Progression Evolution Guitar Effect-Pedal Emulation Non-neural NEAT • ANNs aren’t the only thing that can complexify • Anything with arbitrary structure can potentially be evolved – Cellular Automata – Lego Structures – FSMs – L-systems L-System Plant Geometry Generator by Hung-Wen Chen What Can’t NEAT Do? • “Regular” NEAT is a direct encoding • It doesn’t encode regularities or symmetries • Regularity and symmetry are believed to be essential to representing and searching complex structures: gene reuse • Giant complex structures in nature are composed of patterns • Patterns have regularities • Indirect encodings capture such information efficiently Indirect Encoding • Most are developmental encodings – Based on embryo to adult development – “Artificial Embryogeny” – This is how encoding works in nature • Many have been proposed • None close to natural complexity Proposed Indirect Encodings What’s missing? • None implement complexification as in NEAT: maybe genes can be added and marked like in NEAT – For example: L-system NEAT? • Something’s still missing… A Recent Discovery • Drawn by a NEAT-evolved neural network!? Compositional Pattern Producing Networks (CPPNs) • Networks of functions composed to produce patterns with regularities • An abstraction of natural development without developmental simulation • Complexifying repeating patterns, regularities, symmetries, repetition with variation, etc. CPPN-NEAT • • • • A new kind of indirect encoding Elegantly captures regularities Convert patterns into phenotypes An entirely new research area Exploiting Regularity Without Development by Kenneth O. Stanley, To appear in: Proceedings of the AAAI Fall Symposium on Developmental Systems. Meno Park, CA: AAAI Press, 2006 (8 pages) Music is a Pattern over Time • A CPPN is a pattern generator that can produce repetition with variation, just like music • A CPPN can produce several related patterns from an underlying simpler pattern (multiple instruments) • Music generation is a promising demonstration Homework for 8/30/06 • Neural network weight-learning algorithms Fausett pp. 39-80 (in Chapter 2) Fausett pp. 289-316 (in Chapter 6) Chapter 1 of Sutton (on RL): Online Book ch.1 Optional: Kaelbling RL survey paper • Remember proposals/partners due 9/11