Download Evolutionary Robotics

 Introduction and Motivation
 Artificial Evolution
 Neural Network Controller
 Morphology
 Evolution through Simulation
 Subsumption Architecture
 Conclusion
Can a computer ultimately
augment or replace human
invention?
“I visualize a time when we will be to
robots what dogs are to humans, and
I'm rooting for the machines.”
Claude Shannon
Primary Goal
To develop automatic methods for creating
intelligent autonomous robot controllers
Definition (from Wikipedia)
Evolutionary robotics (ER) is a
methodology that uses evolutionary
computation to develop controllers for
autonomous robots.
 Start with a simple model/models (first
generation)
 Selection/survival: Select those models(species)
which passes fitness test.
 Variation/mutation: Perform some variations (or
recombination) to the selected models.
Engineering students learn about existing solutions
and techniques for well-defined, relatively simple
problems, and then – through practice – are expected
to improve and combine these to create larger, more
complex systems.
How is this synthesis process done? We do not know,
but we cloak it with the term “creativity”.
Is a probabilistic technique for
solving computational
problems
• Can be reduced to finding
good paths through graphs.
• Ants (initially) wander
randomly, and upon finding
food return to their colony
while laying down pheromone
(chemical excretion) trails.
•
If other ants find such a path, they are likely not to keep travelling
at random, but to instead follow the trail, returning and
reinforcing it if they eventually find food.
• Robotic systems and the environments into
which they are placed
• Increase in complexity, the difficulty of
programming
• Their control systems to respond appropriately,
increases to the point where it becomes
impracticable to foresee every possible state of
the robot in its environment.
• Find robot controllers for behaviours which are
hard to implement by hand
 Brains of robots
 Controls the action of actuators
based on the sensor inputs.
 Body of Robot
 Each robot controller competes in an environment
to perform the task for which the robots are being
evolved.
 Robots are evolved to perform tasks requiring
some level of intelligence,.
 Evolution is initialized by creating a population of
randomly configured robots.
 The controllers in the better performing robots are
selected, altered and propagated in a repeating
process that mimics natural evolution.
 Initiation – Initial population may be random, blank, or
seeded with some prior knowledge in the form of solutions
we think are good starting points.
 Selection - measure the performance of each solution
in the population. The performance, termed fitness in
evolutionary terminology, captures the merit of the design
with respect to some target performance we are seeking
as the designers.
 Variation - We select better solutions (parents) and use
them to create a new generation of solutions (offspring).
The offspring are variations of the parents, created
through variation operators like mutation and
recombination. The process is repeated generation after
generation until good solutions are found.
An overview of a typical evolutionary robotics training cycle
http://www.nelsonrobotics.org/evolutionary_robotics_web/
Behavior-Based Robotics &
ER
manipulate the world
build maps
sensors
explore
actuators
Behavior-based
robotics
[Brooks, 1986]
avoid hitting things
locomote
?
?
sensors
?
?
?
actuators
Evolutionary robotics


  V 1  v 1  i 
fitness function
G en. 0
test
reproduce
and m utate
select
genotype-to-phenotype
mapping
test
G en. 1
select
… … … ..
G en. n
reproduce
and m utate
test




Input connected
to sensors
Output to motors
or actuators
Recurrent Neural
Network
Feed forward
Neural Networ
motors
Selector
Neurons
left m.
right m.
pick-up
release
IR-sensors & BL-sensor
IR-sensors & BL-sensor
B
A
Output
Neurons
motors
IR-Sensors
LB-Sensor
motors (a)
motors (b)
[Nolfi, 1997]
IR-sensors & BL-sensor
IR-sensors & BL-sensor
C
D
 Based on Stimulus-Response rules.
 Schultz et al. demonstrated the evolution of rulebased controllers for learning complex robotic
behaviors by evolving the behavior for a shepherd
robot to coerce a sheep robot into a corral.
 Morphology (body) of robots can be evolved like
controllers.
 Body of robots made of small units which can be
detached/ attached/ reconfigured.
 Alternatively, robots can be produced using
rapid prototyping equipments.
 In nature species evolve body and behavior
complementarily.
 Starting with some basic structure, robots can be
trained some simple behavior.
 To learn more complex behavior, morphology is
evolved to suit it to learn the complex behavior.
 Starting with simple structures (tadpoles),
robots were gradually evolved to adopt more
complex shape (frog) as they learned to walk.
 They learned to walk
more rapidly as
compared to ones with
fixed body forms.
 Evolving control systems in simulation provide a
solution to prevent hardware damage and provides
a faster evaluation than the real time speed.
 But, simulation has its own drawbacks.
 Noise from external environment, imprecise
readings and motor control
 Experiments conducted with high levels of noise –
Solutions may evolve that rely on the excessive
noise.
 Include noise in the simulation of robot.
 This evolves controller for a more general robot
structure.
 Example: To make simulation of robot learning to
walk (bipedal), use mass distribution as: m(x, y,
z)+ ∆m.
 Example: To model air pressure use P + ∆P.
 Start with basic crude modeling of the robot body.
 Develop controller for that model.
 If it fails for real robot, evolve model so that
schism between robot and simulate model
abridges.
 Ultimately simulation is very close to reality and
controller evolved for that simulation does justice
with real robot also.
Refining
model
Initial
crude
model
Test
unsuccessful
Test
Successful
Robot
Evolved
Testing
controller
for actual
robot
Controller
evolved
Evolving
controller
on model
 Complicated intelligent behaviors broken into
simple behavior modules
 Organized in layers which implement a particular
goal
 Higher layer subsumes the goals of layers below
 Drawback : Whole is not always just the sum of
its parts
Building a Tower of Blocks
Move Block
Hold Block
Avoid Obstacles
Place Block
Place on the top
Maintain Balance
 As an alternative to the hand design, Evolutionary
Robotics has generated a great deal of exuberance.
 But can the ER techniques produce a net saving of
human effort when applied to complex robotics
domain.
 How to map continuous inputs and outputs to
discrete state variables, establishing appropriate
intermediate and goal states, determining how
many production rules are required
 Current ER systems are often very simple and
could be easily bettered by hand-coded
solutions.
 Many control-system evaluations achieve
satisfactory results in unacceptable runtimes
 The robot may also enter into dangerous states
in which its hardware could be damaged
 Evolutionary algorithms are used to generate
control algorithms using the Darwinian
principle of survival-of-the-fittest.
 Most of the work has focused on the
development of control systems for
autonomous mobile robots. Some researchers
have used the techniques to evolve robotic
hardware configurations.
Like tadpoles become frogs, change their body
forms while learning how to walk, Robots have
been evolved, spending less time in “infant”
tadpole-like forms and more time in “adult”
four-legged forms.
 To overcome the dependence on unreliable
hardware and poor battery performance,
control systems have been evolved through
simulation.
 Evolutionary techniques hold the potential to
solve many difficult problems in robotics which
defy simple conventional approaches, but
present many challenges as well.
Human intelligence stems from evolution and
adaptation with the environment
ER raise an important question:
Can robots too evolve like
humans in future?
 Evolutionary Robotics and Open-Ended Design Automation --
Hod Lipson, Cornell University
 Wikipedia
 Challenges and Opportunities of Evolutionary Robotics -- D. A.
Sofge, M. A. Potter, M. D. Bugajska, A. C. Schultz
 Evolutionary Robotics -- Stefano Nolfi, Dario Floreano
 Evolutionary Robotics: From Algorithms To Implementations --
Lingfeng Wang, Kay Chen Tan, Chee Meng Chew
 http://www.authorstream.com/Presentation/rajansiva-440202-
robotics/
 Brooks’ Subsumption Architecture by T. Ryan Fitz-Gibbon