Download Genetic Operators: Mutation

CS 416
Artificial Intelligence
Lecture 25
AI Applications
Final Exam
Saturday, May 6th at 7:00 pm
Should only conflict with French finals
It will not be cumulative
NN Assignment
Due May 2nd
Some general artificial neural
network (ANN) info
• The entire network is a function g( inputs ) = outputs
– These functions frequently have sigmoids in them
– These functions are frequently differentiable
– These functions have coefficients (weights)
• Backpropagation networks are simply ways to tune the
coefficients of a function so it produces desired output
Function approximation
Consider fitting a line to data
• Coefficients: slope and y-intercept
• Training data: some samples
• Use least-squares fit
y
This is what an ANN does
x
Function approximation
A function of two inputs…
• Fit a smooth
curve to the
available
data
– Quadratic
– Cubic
– nth-order
– ANN!
Curve fitting
• A neural network should be able to generate the
input/output pairs from the training data
• You’d like for it to be smooth (and well-behaved) in the
voids between the training data
• There are risks of over fitting the data
When using ANNs
• Sometimes the output layer feeds back into the input
layer – recurrent neural networks
• The backpropagation will tune the weights
• You determine the topology
– Different topologies have different training outcomes
(consider overfitting)
– Sometimes a genetic algorithm is used to explore
the space of neural network topologies
What is the Purpose of NN?
To create an Artificial Intelligence, or
• Although not an invalid purpose, many people in the AI
community think neural networks do not provide anything
that cannot be obtained through other techniques
– It is hard to unravel the “intelligence” behind why the
ANN works
To study how the human brain works?
• Ironically, those studying neural networks with this in mind
are more likely to contribute to the previous purpose
Some Brain Facts
• Contains ~100,000,000,000 neurons
• Hippocampus CA3 region contains
~3,000,000 neurons
• Each neuron is connected to ~10,000 other
neurons
• ~ (1015)(1015) connections!
• Consumes ~20-30% of the body’s energy
• Contains about 2% of the body’s mass
Karl Sims
Evolved Virtual Creatures (SIGGRAPH 1994)
Evolving 3D Morphology and Behavior by
Competition (ALIFE
1994)
Motivation
Automatically generate physically
simulated characters without fully
understanding the procedures used to
build and control them
• Morphology
• Control
Creating animation from scratch
Typical Scenario
• 20 seconds of animation
• 17 controlled DOFs
• (20 * 30 * 17) = 10,200 torque values to determine
• No approximations to best answer are known
– Spacetime constraints would use initial guess to find
local maximum
Creating animation from scratch
Searching for 10,200 unknowns
• We cannot prescribe initial values with confidence
• We can prescribe some behaviors and principles we
think the motion will demonstrate
– Insight: the movements at some DOFs are functions
of movements at other DOFs
 Sun and Metaxas – Automating
Gait Generation, SIGGRAPH 01
So we’re not really working from
scratch
Searching for 10,200 unknowns
• How to incorporate our motion heuristics into search?
– Simulated annealing?
– Neural networks?
– Genetic algorithms?
But we don’t know the
morphology
The previous example assumed a known
morphology
• We’re searching for best architecture AND best control
simultaneously!
Our search space just grew to an unknown
degree!
Genetic Algorithms
Imitate the processes of evolution and natural
selection in biological world to
generate/evolve the best solution
(not to scale)
Genetic Algorithms
Inject user knowledge
• Objective function
• Genetic representation
– Let’s search for collection of joints
– Let’s not search for n * 30 * 20 unknown joint
torques
– Instead, search for n equations that are logical
combinations of virtual sensors of creature state
– These equations are reused for all timesteps
Sensibility of GA for this
problem
Motion principles described as heuristics
• Minimize angular disturbances by swinging legs in
opposite directions
• Maximize balance by keeping center of mass above
platform of support
• Kinematics of arm are similar to kinematics of leg
So let’s find those heuristics and permit them
to be shared across limbs
Genetic Representation
Defined by creature morphology and
neurology
Morphology: “The form and structure of an
organism or one of its parts”
Neurology (Nervous system): “The system of
cells, tissues, and organs that regulates the
body's responses to internal and external
stimuli”
Creature Morphology
Articulated threedimension rigid body
parts
Represented as a
directed graph of nodes
and connections
Each node describes one
body part
• Many parameters to a node
Creature Neurology
Comprised of:
• Sensors
• Neurons
• Effectors
Sensors
Each sensor is at a different part of the body
• It measures
– That body part
– The world relative to that body part
• Types
– Joint angle sensor
– Contact sensors
– Photo sensors
Neurons
Virtual brain: Function based on sensor
input
Provides output effector values
Effectors
Each effector controls a degree of
freedom in a joint
Each effector’s value is applied as forces
or torques
Only exert simulated muscle force
Connected to neuron or sensor
Neural System
Local neural system:
• Neural system generated along with morphological system
• Connection allowed between local neural systems
Global neural system:
• Neurons not associated with a specific node
– Useful heuristic for creating global timing mechanisms
• Can be connected with local neural systems
Neural System
• Sum, product, divide,
threshold, >, sign, min, abs, if,
interpolate, sine, cosine, atan,
log, exponent, sigmoid,
integrate, differentiate,
smooth, memory, wave,
sawtooth
Physical Simulation
• Featherstone’s articulated body method
• Runge-Kutta-Fehlberg
– Adaptive timestep typically 1-5 steps per 1/30
second
• Bounding boxes to reduce collision checks
– Connected pairs can interpenetrate but cannot
rotate entirely through one another
• Impulses and springs used for collision corrections
• Viscosity used for underwater
Genetic Operators: Mutation
Alter internal node parameters
Dimensions- determine the physical shape of the
part
Joint type- number of DOF for each joint and
movement allowed for each DOF
Joint limits- point where spring forces will be
exerted for each DOF
Recursive limit
Set of local neurons
Set of connections to other nodes
• Position, orientation, scale and reflection relative to
parent
Genetic Operators: Mutation
Add a new node at random
Leg
Segment
Body
Segment
Head
Segment
Genetic Operators: Mutation
Parameter of each connection can
change
45
90
Leg
Segment
Body
Segment
Genetic Operators: Mutation
Add/remove connections
Leg
Segment
Body
Segment
Head
Segment
Genetic Operators: Mutation
Delete unconnected nodes
Leg
Segment
Body
Segment
Head
Segment
Genetic Operators: Crossover
Genetic Operators: Grafting
Start
Generate random population of n
suitable solutions for the problem
• Random generation of nodes and connections
• Use existing genotype to seed function
• Create genotype manually to seed function
Fitness
Swimming, walking, jumping
• Distance traveled by COM/unit of time
Following
• Speed at which creature moves towards target
Competition
• Pairs of individuals compete for control over a cube
• Final distance of the creature from cube and
opponents final distance from cube
New population
•
Survival ratio: 1/5 of 300
•
40% mutation, 30% crossover, 30%
grafting
•
Children from crossover and grafting
are sometimes mutated
Results
Different runs converged on characters
with different evolved characteristics
Specify a User Preference using the
Algorithm
• User can specify a preference over multiple runs
of the simulation
• User can perform the natural selection
Results
Video
Advantages
Generate characters without specification
or knowledge about exactly how they
work
Less likely to get stuck on a local solution
since we are searching on an encoding
Easy to parallelize
Disadvantages
The characters could produce incorrect behavior
that could go undetected because of the lack
of understanding of the underlying algorithm
Process is not very reusable
• Can reuse some of the components of the
implementation, for example, the neuron language
Genetic Algorithms sometimes require a lot of
tweaking to work right
Disadvantages
Difficult to obtain ideal size for the neuron
language, node types and parameters (joint
types and parameters and connection
parameters), sensor and effector types
i.e. Difficult to obtain best size for the search
space
• If the size is too small, then there aren't enough variations
in the population
• If the size is too large, then changes in isolated parts of
the creature do not have a big effect on the behavior or
structure
NeuroAnimator
R. Grzeszczuk et
al. SIGGRAPH 1998
Motivation
Physical Simulation
• Produces very realistic output
• Highly automated
• Computationally expensive
• Difficult to control (author controllers)
Motivation
What is it about simulations?
• (Differential) equations of motion are specified
– The equations may be brittle
• Small changes in state at time (t) may result in
large changes at time (t+1)
• Integration is required and error is possible
• Time steps may be small
Emulation
What if we could replace the “simulation”
• Replacing the equations of motion?
• Replacing the numerical integration?
• Both?
Emulation
Replacing simulation with neural network
• What is gained?
• What is lost?
Reminder about neural networks
They are a universal function approximator
• What function are we approximating?
– Equations of motion
• What do they look like?
– f(st) = st+1
• Can neural networks learn this?
Artificial Neurons
“learning”
• Transfer functions, weights, topologies
Artificial Neurons
“learning”
• Training requires data
– Underfitting – have lots of neurons
– Overfitting - 8-10 times as many examples are used
as there are weights in the network
• Backprop to tune the weights
The emulation
A simulation  neural network
•
st+Dt = sim (st, ut, ft)
•
st+Dt = NF (st, ut, ft)
– Timestep of ANN is (much) larger
Some initial concerns
Basic ANN needs some changes in setup
• Mapping from st to st+Dt is nonlinear and has a great
range
– Position and velocity can vary from +/- inf
– Sigmoid can capture nonlinearities, but its range is
limited
• Could use many sigmoids and shift/scale them to
cover the space
Some initial concerns
Changing basic ANN setup
• Learn Dst instead of st+1
• st+Dt = st + Dst
• Some variables are invariant to world position
– Normalize those variables to always be local
coordinates
Some initial concerns
Changing basic ANN setup
• Normalize
– Variables with larger magnitudes have bigger impact
on ANN output even though they may have smaller
impacts on simulation behavior
– Transform all variables to zero-mean, SD = 1
Some initial concerns
Learning with monolithic networks is tough
• Should the neurons affecting neck be influenced by
lower leg?
• Can we partition the variables a priori and learn
multiple ANNs?
Some initial concerns
Sampling simulation states
• Uniform sampling across all inputs is desirable?
– It’s too complex anyways
– Focus on what is typically encountered
– Grab data points in situ while simulation is running
Training
Examples later will demonstrate the ability to
learn the dynamics
Control
Hard to control physical simulations
• Typically trial-and-error
– Markov Chain Monte Carlo (MCMC) by Chenney
– Sims’ creatures
– Simulated annealing
• Having gradient is useful
Control
ANNs are differentiable
• Find best control decisions for simulation
Performance
• Neural network: O(n2), n = state parameter size
• Simulation: O (n3)
• Delta t for ANN is 50 times larger than simulation
Computation Time
Error
Takeaway
• ANNs can learn simulation behaviors
– Normalization, hierarchical networks, localization
were all required
– Training was limited to space of typical behavior
• Control is easier to learn with ANNs because they are
analytically differentiable