Download Spacetime Constraints Revisited

Spacetime Constraints Revisited
Joe Marks
J. Thomas Ngo
Using genetic algorithms to find solutions
to spacetime constraint problems in 2D
Difficulties with SC problems
• Multimodality
– exponential number of possible solutions
• Search space discontinuities
– small changes in actuators leads to large
changes in trajectory
Why Genetic Algorithms?
• Traditional SC algorithms
– Initial trajectory needed to work on
• Initial trajectory is done by human hand - “coarse”
• Initial trajectory is locally optimized
• Genetic Algorithms
– Discovers solutions “from scratch”
– Cover global optimization problem
– Can find “novel” solutions that were not
imagined
Overview of GA in SC problems
• All trajectories are described as behaviors
• Behaviors generated by parametric
algorithms (nothing smart here)
• Evolutionary computation chooses behavior
parameters to find optimal solutions
– can find multiple effective solutions
• Better solutions (genomes) bred further
• Worse solutions bred-out of gene pool
GA algorithm for 2D simulation
(Ngo and Marks)
• Dynamics module
– simulates a physics driven world for testing
creatures
• Behavior module
– generates behavior using parameterized
algorithm
• Search module
– uses GA to choose values for parameters
– generates near-optimal solutions
Dynamics Module
• Most CPU intensive part of this algorithm
• Simulating simplified physics
– Based on work by Hahn
– rate of change of internal degrees of creature
controlled, rather than calculating torques
– friction and slippage simulated upon contact
Behavior Module
• Behaviors generated by stimulus-response
(SR) control algorithm
– causes instinctive reflexes to conditions
• conditions are stimulus functions over senses
• reflexes are responses to conditions
– avoids traditional use of forces
– behaviors selected in search module
– no learning or planning algorithms involved
Stimulus functions
• Scalar functions based on sense variables
–
–
–
–
state of joint angles
force between body endpoints and floor
vertical velocity of center of mass
height from floor of center of mass
• Contain parameters that are determined by
GA-based search module
• Stimulus functions exhibit sensitive regions
– locus of points for which function is positive
– important notion during mutation
Response
• Change in shape in reaction to a condition
– Conducted on highest valued positive stimulus
function
– Change of creature’s actual shape to a target
shape
– Change kept smooth by damped motion
equations
• Target shape may change during a response
– Creature must respond to another stimulus
• Response is active for several time steps
Search Module
• Use of GA to pick near optimal behaviors
• Parallel processing of solutions
– each processor handles one genome solution
per generation
do parallel
Randomize genome
end do
for generation = 1 to number_of_generations
do parallel
Evaluate genome
Select mate from another processor
Cross genome with mate
Mutate genome
end do
end for
Randomization
• Initial parameters chosen at random
• Favorable initial gene pool formed
– hill climbing algorithm used to find good initial
solutions to populate gene pool
– each solution mutated, re-evaluated four times
• mutation skewed towards multi-step solutions
– best of five solutions on each processor used
• final population is non-random, skewed
Evaluation
• One of the most important aspects of GA
• Net horizontal distance covered by center of
mass in given time
– Sometimes this encourages “cheating”
– e.g., leap head-first rather than walking
• To encourage walking, modify criteria
– e.g., net horizontal distance covered by
midpoint between two feet - i.e., use your feet!
Mate Selection
• Only local mating permitted
– maintains diversity, handles multimodality
– mate chosen on a random walk of 10 steps
• if better solution than self found, mate is chosen
• otherwise, current solution stays single
– produces large local colonies of good genes
• larger colonies use up more processing power
• Convergence - when one colony dominates
Crossover
• Linear crossover not seen as meaningful
– certain parameters should migrate as groups
• More structured, specialized crossover used
– two (of ten) SR pairs taken from self
– six (of ten) SR pairs taken from mate
– one SR pair created with stimuli and response
taken from each parent, respectively
– one SR pair created with numbers taken by
random from parents
Mutation
• Mutation algorithm tailored for SR method
• One SR pair in genome is subject to creep
– each parameter in pair is slightly changed
• One SR pair is randomized from scratch
– but one corner of sensitive regions of new
stimulus function coincides with original sensespace trajectory
• keeps new functions from either dominating or not
having any effect.
Results
• Hardware, software used
– 4,096 processor Thinking Machines CM-2
– Code written in C*
• Creatures used
– 5 rod figures; 50 time steps; 100 generations
• Five-rod Fred
• Mr. Star-man
• Beryl Biped
– importance of adapting criteria
• 30 to 60 minutes of computation used