* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Evolutionary Learning
Survey
Document related concepts
Designer baby wikipedia , lookup
Heritability of IQ wikipedia , lookup
Public health genomics wikipedia , lookup
Genetic testing wikipedia , lookup
Adaptive evolution in the human genome wikipedia , lookup
Genetic engineering wikipedia , lookup
Group selection wikipedia , lookup
Polymorphism (biology) wikipedia , lookup
Human genetic variation wikipedia , lookup
Dual inheritance theory wikipedia , lookup
Genome (book) wikipedia , lookup
Genetic drift wikipedia , lookup
Gene expression programming wikipedia , lookup
Microevolution wikipedia , lookup
Transcript
Evolutionary Computation and beyond Elhanan Borenstein May. 2004 www.cs.tau.ac.il/~borens Evolution vs. Learning Evolution Learning Process: Random (?) Directional (?) Level: Population Individual Entity: Genotype Phenotype Time Scale: Generations Lifetime Operators: Mutation, Crossover Gradient Descent Search: Global Local Problems: Slow / Random Initial conditions Talk Outline Evolution – An Inevitable Process Genetic Algorithms Motivation Techniques & Pseudo Code Evolution as a Stochastic Search Process Fitness Landscapes Enhanced Genetic Algorithms Artificial Life and Evolutionary Autonomous Agents Life as it could be… Definition & Motivation Putting is All Together Experiments Demo Evolution – An inevitable Process Evolution – An inevitable Process Any system, that embeds the following three processes, will inevitably undergo evolution: Reproduction (inheritance) Initial Population Next Generation Selection Variation Reproduction Selection Variation A process of selective reproduction and substitution in a population of individuals. Evolution – An inevitable Process Just a few examples may include: Domestic Animals Breeding Life on Earth Strategies (Financial…) Ideas, Beliefs, knowledge Evolutionary Game Theory culture memes Evolutionary Computation If it works so well in nature… why not using it to solve computational problems… Many problems in computer science (and mathematics) have the following characteristics: We are searching for the optimal solution… The number of possible solutions (search space) is huge. We do not have an analytical method to reach the optimal solution… but… given a solution, we can evaluate its quality. Evolution is a search process !!! For Example … Genetic Algorithms Each possible solution will be represented (coded) as a string of characters (genome) For example: LPMRSBN (London Paris Madrid Rome Lisbon Berlin Monaco) Generate a random collection (population) of solutions: LPMRSBN, MSBRNLP, NBSMPLR, MRLNSBP, … Evaluate the quality (fitness) of each solution Generate the next generation of solutions: Delete (kill) all bad solutions. Duplicate (reproduce) a number of copies of each remaining (surviving) solution. Insert random modifications (mutations) to the newly create solutions. Repeat (life goes on) until the optimal solution was found. Genetic Algorithm – Pseudo Code Initialize population: P Generate p hypotheses at random Evaluate: For each h in P, compute Fitness(h) While [max Fitness(h) < threshold] do Select: Probabilistically select p/2 pairs of hypotheses from P. Pr(hi) = Fitness(hi) / Σ Fitness(hi) Reproduction: Produce two offspring for each pair. Add to Ps With probability pc apply crossover operator mutate: For each gene, with probability pm, apply mutation operator Update: P Ps Evaluated: For each h in P, compute Fitness(h) return Genetic Algorithm – Pseudo Code Initialize population: P Generate p hypotheses at random Evaluate: For each h in P, compute Fitness(h) While [max Fitness(h) < threshold] do Select: Probabilistically select p/2 pairs of hypotheses from P. Pr(hi) = Fitness(hi) / Σ Fitness(hi) Reproduction: Produce two offspring for each pair. Add to Ps With probability pc apply crossover operator mutate: For each gene, with probability pm, apply mutation operator Update: P Ps Evaluated: For each h in P, compute Fitness(h) return Population Genotype vs. Phenotype Representation 2.31 1.02 -0.28 -7.93 6.32 Coding Direct Developmental Stochastic 1.87 -1.72 2.46 0.45 011 101 100 111 001 010 011 111 110 Genetic Algorithm – Pseudo Code Initialize population: P Generate p hypotheses at random Evaluate: For each h in P, compute Fitness(h) While [max Fitness(h) < threshold] do Select: Probabilistically select p/2 pairs of hypotheses from P. Pr(hi) = Fitness(hi) / Σ Fitness(hi) Reproduction: Produce two offspring for each pair. Add to Ps With probability pc apply crossover operator mutate: For each gene, with probability pm, apply mutation operator Update: P Ps Evaluated: For each h in P, compute Fitness(h) return Evaluation – The fitness function Reliable assessment Average fitness Scaling (rank-based, linear, quadratic, Baltzmann) Tournament Genetic Algorithm – Pseudo Code Initialize population: P Generate p hypotheses at random Evaluate: For each h in P, compute Fitness(h) While [max Fitness(h) < threshold] do Select: Probabilistically select p/2 pairs of hypotheses from P. Pr(hi) = Fitness(hi) / Σ Fitness(hi) Reproduction: Produce two offspring for each pair. Add to Ps With probability pc apply crossover operator mutate: For each gene, with probability pm, apply mutation operator Update: P Ps Evaluated: For each h in P, compute Fitness(h) return Termination Criteria Maximal Fitness Convergence of the population No improvement Maximal # of generations Genetic Algorithm – Pseudo Code Initialize population: P Generate p hypotheses at random Evaluate: For each h in P, compute Fitness(h) While [max Fitness(h) < threshold] do Select: Probabilistically select p/2 pairs of hypotheses from P. Pr(hi) = Fitness(hi) / Σ Fitness(hi) Reproduction: Produce two offspring for each pair. Add to Ps With probability pc apply crossover operator mutate: For each gene, with probability pm, apply mutation operator Update: P Ps Evaluated: For each h in P, compute Fitness(h) return Selection (survival of the fittest) A stochastic process !!! Roulette wheel selection Tournament Competition Elitism Genetic Algorithm – Pseudo Code Initialize population: P Generate p hypotheses at random Evaluate: For each h in P, compute Fitness(h) While [max Fitness(h) < threshold] do Select: Probabilistically select p/2 pairs of hypotheses from P. Pr(hi) = Fitness(hi) / Σ Fitness(hi) Reproduction: Produce two offspring for each pair. Add to Ps With probability pc apply crossover operator mutate: For each gene, with probability pm, apply mutation operator Update: P Ps Evaluated: For each h in P, compute Fitness(h) return Reproduction Sexual vs. Asexual Haploid vs. Diploid Genetic Algorithm – Pseudo Code Initialize population: P Generate p hypotheses at random Evaluate: For each h in P, compute Fitness(h) While [max Fitness(h) < threshold] do Select: Probabilistically select p/2 pairs of hypotheses from P. Pr(hi) = Fitness(hi) / Σ Fitness(hi) Reproduction: Produce two offspring for each pair. Add to Ps With probability pc apply crossover operator mutate: For each gene, with probability pm, apply mutation operator Update: P Ps Evaluated: For each h in P, compute Fitness(h) return Genetic Operators - Variation Mutation: 011 101 100 111 001 010 011 111 110 011 101 101 111 001 010 011 111 110 Crossover (one point/two points): 011 101 100 111 001 010 011 111 110 110 100 110 000 100 101 100 111 001 011 101 100 111 001 010 100 111 001 110 100 110 000 100 101 011 111 110 Genetic Algorithm – Pseudo Code Determining which of the numerous extensions and variations of Genetic Algorithms matches a specific task is still very much a matter of art and experience. Genotype/Phenotype Coding A few notes to bear in mind about genetics and evolutionary computation: Most evolutionary computation applications, implement some sort of distinction between genotypes and phenotypes, in an analogous manner to biological evolution. While the genotype is the one that transferred across generations, it is the phenotype that goes trough the selection process. We work mostly in a strict Darwinian framework, where lifetime adaptations are not inherited. The Genotype to Phenotype coding may have a dramatic effect on the evolutionary search !!! Evolution – A Stochastic Search The Fitness Landscape Fitness Value Optimal Solution Evolution – A Stochastic Search Fitness Value Illustration – Initial Population Evolution – A Stochastic Search Fitness Value Illustration – Natural Selection Evolution – A Stochastic Search Fitness Value Illustration – Reproduction & Variation Evolution – A Stochastic Search Fitness Value Illustration – The Next Generation Evolution – A Stochastic Search Which properties of the fitness landscape affect the evolutionary search? Size!!! Incremental Ruggedness Local optima Size of basins of attraction B A Genetic Configuration C Genetic Configuration D Genetic Configuration Genetic Configuration Evolution – A Stochastic Search Can we control these properties? Yes (to some degree) !!! Genetic Representation Dimensionality Neighborhood function Building blocks Stochastic encoding Fitness Function Smooth Monotone / incremental Resolution Much is still dependent on the Task !!! Genetic Configuration Enhancing Genetic Algorithms How can we push the evolutionary process forward? Taking another lesson from nature Co-evolution: host-parasite / predator-pray Red queen effect (“You have to run faster and faster just to stay in the same place!”). Incremental evolution Lifetime learning: Darwinian framework vs Lamarckian framework. Social Learning Cultural evolution Life As It Could Be A Short Introduction to Artificial Life and Evolutionary Autonomous Agents A Multidisciplinary Research Biology Computer Science Neuroscience Mathematics Physics Robotics Artificial Life Psychology Animal Behavior Medical Research Artificial Life (ALife) - Definition “Artificial Life (ALife) is the study of man-made systems that exhibit behaviors characteristic of natural systems. It complements the traditional biological science concerned with the analysis of living organisms by attempting to synthesize lifelike behaviors within computers and other artificial media. …by extending the empirical foundation upon which biology is based beyond the carbon-chain life that has evolved on Earth… ALife can contribute to theoretical biology by locating life-as-we-know-it within the large picture of life-as-it-could-be.” Chris Langton Artificial Life (ALife) - Motivation ALife motivation is twofold: Computational Create better artificial systems that are inspired by biological systems Biological Gain a better understanding of biological systems Modeling Complex Systems What advantages are there in artificial models? Full access to all data (all vs. tens, all vs. fossil record) Time resolution (milliseconds vs. years) Intervention (all and any vs. few) But…will our findings in such models (systems) apply to real natural systems? How accurate should our model be? Which parameters should be included? :על הדיוק במדע שמפתו של מחוז אחד,המפות למידת שלמות כזו- באימפריה זו הגיעה אומנות כתיבת... ברבות.השתרעה על פני עיר שלמה ואילו מפתה של האימפריה כולה – על פני המחוז כולו ענק אלה את התושבים וועדות כותבי המפות החלו להכין את-הימים לא סיפקו עוד מפות ... והיא זהה עמה בכל נקודה ונקודה,מפת האימפריה שגודלה כגודל האימפריה עצמה חורחה לואיס בורחס Artificial Neural Networks (ANN) Artificial Neural Network: w6 w1 w21 w2 Input w3 Output w8 Such neural network can be thought of as a “black-box” that compute some function of the input. Different assignment of the synaptic weights will result in a different function. Autonomous Agents w w 2 w 1 2 6 w 1 w 3 w 8 Such artificial creatures may be either real autonomous robots or simulated autonomous agents. Developing the Agent’s Brain So…we have manage to construct a good model for a neural-network brain, controlling an artificial agent… But…we are back to square one!!! How will we develop the agent’s brain and get “working” and interesting brain models? There are two approaches we can take (both inspired again by biological process). Lifetime Learning “Genetic” Evolution Evolutionary Autonomous Agents Create a first random generation Insert the “brain” into an agent and let it operate in the world 100 brains (random genome) Evaluate each “brain” in the (virtual) world fitness value Allow the successful agent to reproduce mutations, crossovers A new generation is created Evolutionary Autonomous Agents Agents developed by evolution and imitation >> Evolutionary Autonomous Agents Agent in a fire-fighting contest Evolutionary Autonomous Agents One of the main objectives of our study is to develop methodologies for neuroscience research. We thus need to develop more complex / interesting brains (neural networks) !!! Complex Task Evolutionary Autonomous Agents Evolutionary Autonomous Agents How would you solve this task? Evolutionary Autonomous Agents S10: a forager-explorer agent S10: a forager-explorer agent Generation 200 Generation 900 >> >> Emergent Phenomena !!! Evolutionary Autonomous Agents Analyzing the emergent neural network: S10: a forager-explorer agent Command neuron clamp to 0 >> Much work is done to put forward analytical methods that could be applied to biological neuroscience research !! Summary Evolutionary computation is a powerful tool that can be used to “solve” a wide range of tasks. It is still hard to provide rules of thumb to match the numerous extensions and variations of Genetic Algorithms to a specific task. Artificial Life and biological system modeling (in-silico biology) is a growing and exiting field that can shed new light on natural biological systems. Questions ?