Download Lecture 2. Co-Evolution

Lecture 2. Co-Evolution (II)
 학습목표
공진화와 관련된 다양한 방법론을 이해하고, 응용예
를 통한 실제 적용가능성을 점검한다.
Outline
Review of the last lecture
Diploid gene representation
Parallel evolutionary algorithms (fine-grained): Local
selection, Recombination
Different types of co-evolution
Inter-population co-evolution: An example in design,
Knowledge discovery (data mining), Interactive evolution
Intra-population co-evolution: Co-evolving a backgammon
player, Taking two interwound spirals apart by co-evolution,
Iterated prisoner’s dilemma
Summary and overviews of other related work
Why co-evolutionary learning
More examples and open issues
Different Types of Co-Evolution
Based on the number of population involved:
Inter-population co-evolution
Two or more populations
Intra-population co-evolution
Within a single population
Based on the relationship among individuals
Competitive co-evolution
Individuals compete for higher fitness to solve a
problem (often a dynamic problem)
Cooperative co-evolution
Individuals cooperate with other in order to solve a
problem
Co-Evolution in Design (1)
Many design problems do not have a fixed goal or a fixed set of
specifications
People do change minds
…
Current
Problem
k
evolve
provide
fitness
provide
fitness
…
Current
Problem
k+1
Current
Solution
k
evolve
…
provide
fitness
Current
Solution
k+1
…
Co-Evolution in Design (2)
 Task: floor plan design
A candidate solution
dining ensuite bed1
kitchen room
wc
corridor
lounge hall bed3
bed2
Adjacency graph (representing requirements)
dining
kitchen
lounge
wc
bed1
corridor
hall
bed3
bed2
ensuite
Co-Evolution in Design (3)
 Representation
Solution space
genotype
phenotype
mapping
1
4
(pen movement)
2
3
mapping
1 2 3 4 5 45
5
(floor plan)
Problem (specification) space
genotype
A
mapping
A B C D E BC CD
B
E
(adjacency)
D
C
A
B E
C
D
Co-Evolution in Design (4)
Fitness evaluation
Solution space / population
Fitness = (Basic initial requirements) + (current best problem)
Problem space: Fitness = (current best solution)
What does “best” mean in this case?
Answer: How well an individual matches a floor plan (or adjacency graph)
provide
fitness
best
best
solutions
(floor plans)
provide
fitness
problems
(adjacency graphs)
Knowledge Discovery (Data Mining)
One application: fraud detection
We want to discover all kinds of frauds, but we do not really
know what they are (what kind of patterns they have)
In general, we want to find something interesting without
knowing what “interesting” actually means
corporate
database
rules
top performing
rules
evolving
new
interestingness
analysis
human ranking
Interestingness is co-evolving with interesting rules
Interactive Evolution
Evolution with a human being in the loop
Often used in creative design or creative problem solving
May be time-consuming
Case1: no co-evolution
population
fitness evaluation
by human
replacement
genetic operation
Case 2: co-evolution
computer
programs
fitness
evaluation
human
beings
Intra-Population Co-Evolution
There is only one population
However, fitness of one individual depends on other individuals
in the population
Example 1: Playing backgammon
TD-Gammon
a grand master level computer program based on NN
It learned by self-playing
Self-playing = Co-evolution
Is it because machine learning algorithms or co-evolution?
Co-Evolving Backgammon Players (1)
Simple neural network without any fancy learning algorithm
except for hill-climbing
Simple EA with population size 1, hardly an EA!
Simple mutation with Gaussian noise
No recombination at all
Task = evolve an NN that plays backgammon
Co-Evolving Backgammon Players (2)
Task = evolve an NN that plays backgammon
197-20-1 feed-forward fully connected NN
Initial weights were 0’s
1. Let the initial NN be NNk, k  0
2. Generate a mutant challenger of NNk
w’ij = wij + G(0, s)
3. If NN’k is beaten by NNk, NNk+1 = NNk
Else NNk+1 = NNk*0.95 + 0.05*NN’k
4. k  k+1, goto step 2
Performance: Winning 40% of the games against PUBEVAL
after 100,000 generations
strong program trained by experts
Separating Interwound Two Spirals
Task: Given 194 training points, learn to separate two spirals
Very tough problem for machine learning algorithms, e.g.,
decision trees, neural networks, …
There was an attempt to evolve a solution to it. But the
solution generalized poorly
Competitive co-evolution based on covering really helps
Fitness evaluation without co-evolution: number of test
cases correctly classified
Fitness evaluation with co-evolution: based on pair-wise
competition, it depends only on the number of test cases
correctly classified but NOT covered by its opponent
Iterated Prisoner’s Dilemma
Non-zero sum, non-cooperative games
The 2 player version
Player B
Player A
C
D
3
5
C 3
0
1
D 5 0 1
The purpose here is not to find the optimal solution for some
simplified conditions, but to study how to find it
Fitness evaluation
Entirely determined by the total payoff obtained through
playing against each other
The initial population was generated at random
Why Co-Evolution
We do not know the fitness function
There are too many cases to test in order to obtain a fitness
value. Co-evolution can be used to FOCUS search in the most
important area
The problem is inherently changing in time
Increase and maintain diversity
Self-learning
More Examples of Co-Evolution
Discovering CA rules (using coverage again)
Computer-aided learning (students + software tutors)
Robot morphology and control
Character recognition
Chess playing
International coffee market prices
…
Open Issue
Forgetting (also known as the Red Queen effect): co-evolution
does not have a good memory at present
Mediocre stable state: individuals learn to co-exist with each
other and do not want to explore the search space any more
Incremental evolution: continuous improvement without
forgetting
References
J. Poon and M.L. Maher, “Emergent behavior in co-
evolutionary design,” Artificial Intelligence in Design’96, J.
Gero (ed.), Kluwer Academic.
J.B. Pollack, A.D. Blair & M. Lund, “Coevolution of a
backgammon player,” Proc. Of the Fifth Alife, May 1996.
H. Juille and J.B. Pollack, “Dynamics of co-evolutionary
learning,” Proc. Of the 4th Int. Conf. on Simulation of Adaptive
Behaviro, Sept. 1996, MIT Press, pp. 526~534
Homework #1
주제: Diversity 유지를 위한 speciation 방법 구현 및 실험
마감일: 9/30
내용:
5가지 평가함수(De Jong function 1~5)에 2가지 종분화 방법(Fitness
sharing, Crowding)을 적용하여 진화동안의 diversity 변화를 조사한
다
Bonus
Sorting algorithm/problem에 공진화와 종분화를 이용하기
Othello 게임에 공진화와 종분화를 이용하기