Download Robot Control using Genetic Algorithms

Algoritmos Genéticos aplicados
em Machine Learning
Controle de um Robo
(em inglês)
1
Robot Control using
Genetic Algorithms
2
Summary
• Introduction
– Robot Control
– Khepera Simulator
• Genetic Model for Path Planning
– Chromosome Representation
– Evaluation Function
– Case Studies
• Conclusions
3
The Robot Controller Problem
• Given a robot and a description of an environment,
provide commands (motor speeds) to the robot, in
order to achieve a path between two specified
locations, which is collision-free and satisfies certain
optimisation criteria.
(xf, yf)
(xi, yi)
4
Optimisation Criteria
• Robot should:
– attempt near-optimal paths
– avoid obstacles
– perform straight motion
• Controller should be independent of:
– the robot’s environment
– target location
5
The Khepera Simulator
• Freeware mobile robot simulator (designed by Olivier
Michel, University of Nice Sophia-Antipolis)
• User designed worlds
• Control algorithms can be written in C/C++
• Robot’s position and angle reading
S2
S3
S1
• 8 sensors (S0-S7): [0, 1023]
• 2 motors (M1, M2): [-10, +10]
S0
S4
Front identification
M1
M2
S7
S66
S5
Simulator Readings:
sensors, position and angle
Y
S2
S1
S0
Robot’s World
S3
S4
Front identification
M1
S5
1000
M2
S2
S3
S1
S7
S6
S0
y
S0-S7: [0, 1023]
obstacle not
detected
S4
Front identification
M1
M2
S7
S6
S5
angle of the robot with the world
 : [-, ]
obstacle very
closed
0
x
1000
7
X
Control Mode
• To evolve the robot’s attitudes as it
interacts with the environment
• Each robot action determines:
– how well the controller performs with
respect a given task;
– the next input stimuli to the controller.
• The controller should learn as the robot
interacts with the environment
8
Controller Model
Khepera
Simulator
Motor 2
Motor 1
Genetic
Algorithm
evolves robot’s
attitudes
Sensors
Position
Robot’s Angle
Goal Location
9
Proposed Model
based on human behavior
IF
Obstacle detected
THEN
Avoid collision, forget target
ELSE
Straight to the target according to the
target direction
END
10
Sensors Reading Simplification
Sleft
S1
S2
S3
S0
Sright
S4
S5
Sleft = ( S0 + S1 + S2 ) / 3
Sright = ( S3 + S4 + S5 ) / 3
Sback = ( S6 + S7 ) / 2
S7
S6
Sback
11
Determining the Target Direction
Direction =

arrival position: (xf ,yf)



 = /2
Goal point to
left side
Goal point
behind

Goal point in
front
Goal point to
right side
-1
(x,y)
-
 = tan [(yf - y)/(xf - x)]



12

Model
IF ((Sleft > L) or (Sright > L) or (Sback > L))
THEN
Obstacle detected, avoid collision, forget target
Proximity-sensor = highest value (Sleft, Sright, Sback)
ELSE
Obstacle not detected (collision-free), straight to the target
Target direction =  - 
END
L=collision threshold=900
13
Genetic Algorithm Modelling
•
•
•
•
•
•
Problem
Chromosome Representation
Evaluation Function
Genetic Operators
Techniques
Parameters
14
Chromosome Representation
Which speed should be imposed to each motor in
each situation the robot is?
Target Direction(collision free)
M1,M2
M1,M2
Front
Left
1
2
M1,M2
M1,M2
Right Back
3
4
Obstacle detected
M1,M2
M1,M2
Left
Right
5
6
M1,M2
Back
7
Attitudes (Genes)
15
Evaluation Function
• Main objectives:
– (V) speed: as high as possible
– (D) straight motion: same motor speed for M1 e M2
– (A) action: reach a target and avoid obstacles
• Calculated based on the contribution for
each gene [1,7], at each step.
7
F   Vi * Di * Ai 
i 1
0  Vi  1;0  Di  1;0  Ai  1
16
Improving the Target Direction
Model
4 possible target directions
8 possible target directions
/2
Â2 - Â1
/4
3/4
/4
3/4
Target at LEFT
Target Behind
Target in FRONT
/2
0
Target at RIGHT
-3/4
- /4
First Target Direction Model
-3/4
-/4
-/2
23
Chromosome Representations
Cada estado corresponde a uma única atitude (par de velocidades
Target Direction (collision free)
detected
M1 e M2),
e cada atitude corresponde a um Obstacle
gene do
cromossoma.
M1, M2
Front
M1, M2
Left
M1, M2
Right
M1, M2
Back
M1, M2
Left
M1, M2
Right
M1, M2
Back
Chromosome with 7 attitude Genes
Target Direction (collision free)
Obstacle detected
M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2
Front
Front
Left
Left
Right
Right
Back
Back
Left
Right Back
left
right
front
back
front
back
left
right
Chromosome with 11 attitude Genes
24
Genetic Algorithm
•
•
•
•
•
•
•
•
•
•
Integer chromosome
Population Size =100
Generations = 50
Crossover Rate = 80 %
Mutation Rate = 4%
Roulette Wheel Reproduction
Elitism
Linear scaling of fitness
300 Evaluation Steps for each chromosome
Average of 25 Experiments
25
Genetic Algorithm Performance
7 Genes Chromosome
26
Genetic Algorithm Performance
7 Genes Chromosome
Average of Best Chromosomes
in 25 experiments
4,5
4
3,5
2,5
2
1,5
1
0,5
Number of Generations
49
47
45
43
41
39
37
35
33
31
29
27
25
23
21
19
17
15
13
11
9
7
5
3
0
1
FITNESS
3
27
Genetic Algorithm Performance
11 Genes Chromosome
Best Chromosomes
in 1 experiment
5
4,5
4
3
2,5
2
1,5
1
0,5
Number of Generations
49
47
45
43
41
39
37
35
33
31
29
27
25
23
21
19
17
15
13
11
9
7
5
3
0
1
FITNESS
3,5
28
Genetic Algorithm Performance
11 Genes Chromosome
Average of Best Chromosomes
in 25 experiments
5
4,5
4
3
2,5
2
1,5
1
0,5
Number of Generations
49
47
45
43
41
39
37
35
33
31
29
27
25
23
21
19
17
15
13
11
9
7
5
3
0
1
FITNESS
3,5
29
Paths Achieved in World 1
Case
Study1 1
SITUAÇÃO
7 Genes Chromosome
CROMOSSOMA DE 7 GENES
11 Genes Chromosome
CROMOSSOMA DE 11 GENES
30
Paths Achieved in World 1
CaseSITUAÇÃO
Study 42
DE 7 GENES
7CROMOSSOMA
Genes Chromosome
CROMOSSOMA
DE 11 GENES
11
Genes Chromosome
31
Speed Comparison
6000
5000
4000
11 Genes Chromosome
7 Genes Chromosome
Number
of Steps
3000
2000
1000
0
1
2
3
Case Studies
4
5
32
Paths Achieved in World 2
CaseSITUAÇÃO
Study 11
7 Genes Chromosome
CROMOSSOMA DE 7 GENES
11
Genes Chromosome
CROMOSSOMA DE 11 GENES
33
Paths Achieved in World 2
SITUAÇÃO
2 2
Case
Study
7 Genes Chromosome
CROMOSSOMA
DE 7 GENES
11
Genes Chromosome
CROMOSSSOMA
DE 11 GENES
34
Speed Comparison
4500
4000
3500
3000
Number
of Steps
11 Genes Chromosome
2500
7 Genes Chromosome
2000
1500
1000
500
0
1
2
Case Studies
3
35
Speed Comparison (%)
214,29%
Case Study 1
Case Study 2
Case Study 3
84,62%
40,00%
36
Paths Achieved in World 3
CaseSITUAÇÃO
Study 11
7CROMOSSOMA
Genes Chromosome
DE 7 GENES
11CROMOSSOMA
Genes Chromosome
DE 11 GENES
37
Paths Achieved in World 3
CaseSITUAÇÃO
Study 22
7 Genes Chromosome
CROMOSSOMA
DE 7 GENES
11CROMOSSOMA
Genes Chromosome
DE 11 GENES
38
Paths Achieved in World 3
SITUAÇÃO
Case
Study 33
CROMOSSOMA
DE 7 GENES
7 Genes Chromosome
DE 11 GENES
11CROMOSSOMA
Genes Chromosome
39
Paths Achieved in World 3
CaseSITUAÇÃO
Study 44
7 Genes Chromosome
CROMOSSOMA DE 7 GENES
11 Genes Chromosome
CROMOSSOMA DE 11 GENES
40
Speed Comparison
Case Study 1
Case Study 2
Case Study 3
Case Study 4
Speed Comparison – World 3
7 Genes Cromosome
11 Genes Cromosome
target not reached
729
1124
618
target not reached
target not reached
target not reached
1655
41
Conclusions
• A simple GA was able to gradually evolve the robot control
• The robot achieved near optimal path towards the goal,
avoiding obstacles
• Retraining is not necessary when the environment changes
• Controller improved performance with the 11 genes model
• The robot has no memory about previous unsuccessful paths
and may get lost
• Other tasks can be included in the model (e.g. energy supply)
• Chromosome codification is limited for few robot’s situations
42