Download 2015_0001_Vision for Curie and soccer

If you have any
questions please
feel free to
interrupt me
The vision system
for Marie Curie
Main Tasks in our system
Image recognition
Machine Learning
Control of robot’s behavior
Environment
Marie Curie will be communicating with
Schroedinger’s Cat robot
Interaction of Marie Curie and Cat
1.
2.
3.
4.
5.
Marie Curie does not change its main
coordinates, she can only move her hands,
head and legs, but she remains attached to the
desk.
Cat can move freely in the area of the stage.
Cat should not bump into Marie or furniture of
the lab.
Marie should know where the cat is located and
look to him
Cat should know where Marie is located and
talk to her.
Ideal view of the ceiling camera
Black
curtain
shelf
equipment
Marie Curie equipment
equipment
There will be a Kinect camera looking from
the ceiling to the stage
The role of the ceiling camera
1.
2.
3.
4.
5.
The camera will be attached to the ceiling or
will be in some position very high, as high as we
can.
We have done something similar but the robots
were small.
The camera should know x,y coordinates of every
robot and its orientation (pose)
Marie Curie does not change its main
coordinates, so it is easy
Cat is fast so we have to track the triplet
(x, y, )
There is nothing like that in
Disneyland
All behaviors of robots in commercial theatres
are strictly scripted, robots move on rails, they
cannot make an error.
In our case we have interaction, improvisation,
and robots are subject to noisy behaviors.
This task is more similar to robot soccer than to
existing robot theatres in the world.
There will be another camera
looking to faces of the audience
We will call it the
human-control
camera or a front
camera
The role of the front camera
1.
The camera is attached to the wall near the
glass window of the theatre, looking towards
humans located in the corridor.
2. This camera will look at the audience
3. There are several goals of having this camera
1.
2.
3.
4.
Recognizing (x, y, ) of every person that looks
at the performance (perhaps not more than 5).
Recognizing the emotion on the faces of these
people.
Recognizing their gestures with hands and legs,
full bodies and faces.
Use these data to control the behavior of the
robots, songs selected, slides selected, lights
and other effects.
There is nothing like that in
Disneyland
All behaviors of robots in Disneyland
are strictly scripted.
Rarely humans can change robots’
behaviors.
This is a new task for our team
Marek Perkowski has never done anything like
this before
Perhaps nobody in the world has done
something like this.
This is good as we are doing something new.
Hopefully we have done something similar and
have a good experience from the past.
We were doing ROBOT SOCCER.
We will try now to use our past experience and
theory for a new task.
Ideal view of the ceiling camera
Black
curtain
shelf
equipment
Marie Curie equipment
equipment
This is ideal, in reality the image will be much distorted
with noise and lightning and geometry
Y axis
c
Black
curtain
Yc
Xc
shelf
equipment
Marie Curie equipment
equipment
X axis
The idea of
Robot Soccer
3. Robot Soccer and Similar
Tasks
• Robot Soccer Competition
–
–
–
–
RoboCup
FIRA
Remote controlled systems
Autonomous robots
• Clustering
3.1 Robot Soccer
“RoboCup is an international joint project to promote AI,
robotics, and related fields.
It is an attempt to foster AI and
intelligent robotics research by providing a standard problem
where a wide range of technologies can be integrated and
examined.
RoboCup chose to use the soccer game as a central
topic of research, aiming at innovations to be applied for
socially significant problems and industries.
The ultimate goal
of the RoboCup project is: By 2050, develop a team of fully
autonomous humanoid robots that can win against the human
world champion team in soccer.” [RoboCup 1998]
As you see, it is difficult to approximate every robot with a
rectangle.
It will be even more difficult in our case.
Overhead Vision
• Our goal is to start
with Overhead
Vision (Ceiling
Camera) and check
how it will work.
• We may move to
more cameras if
necessary.
Local Vision
Design Criteria for robot soccer
• Controller Hardware: Enable on-board image
processing
–
–
–
–
Interface to digital camera
Incorporate graphics LCD
Incorporate user buttons
Wireless communication between robots
• Sensors: Allow variety of additional sensors:
– Shaft encoders
– Infra-red distance measurement sensors
– Compass module
• Software: Flexibility to accommodate for
different robot equipment
– Operating system RoBIOS
– Hardware description table HDT
What is AI?
Research in Artificial Intelligence (AI) includes:




design of intelligent machines
formalization of the notions of intelligence and
rational behavior
understanding mechanisms of intelligence
interaction of humans and intelligent machines.
Objectives of AI
Engineering : costruct intelligent machines
Scientific : understand what is intelligence.
Can a robot do these?
Understand?
Simulate its environment?
Act rationally?
Collaborate and compete?
Display emotions?
A bold claim:
A team of Robots will beat the FIFA World Cup
champions by 2050!
RoboCup - Aim
”pushing the state-of-the-art”
”By mid-21st century, a team of fully
autonomous humanoid robot soccer
players shall win the soccer game,
comply with the official rule of the FIFA,
against the winner of the most recent
World Cup.
TO BOLDLY GO WHERE MAN HAS GONE
BEFORE (cf. Star Trek)
Formalised Testbed
Do you really believe that a team of Robots
could beat the FIFA World Cup champions by
2050?
By all accounts this may sound overly ambitious.
In fact, if you compare this goal to other ground breaking
achievements it is not ambitious at all.
The Wright brothers' first airplane was launched and 50 years
later man landed on the moon.
Even more recently Deep Blue the computer programmed to
play chess, played chess grand master Garry Kasparov and
won -- roughly 50 years after the deployment of the first
computer.
It's a long time.
Think what has happened since 1950.
Power of AI
Is the following AI?
In 1997 a computer, Deep Blue, won a chess
match with world champion Kasparov.



Accident?
IBM paid Kasparov to loose?
Brute force with no intelligence?
So, what is intelligence?
Simulation
Turing test (1950)
Chess versus soccer robot
Environment
State Change
Info. accessibility
Sensor Readings
Control
Chess
Static
Turn taking
Complete
Symbolic
Central
RoboCup
Dynamic
Real time
Incomplete
Non-symbolic
Distributed
Difference of domain characteristics between computer chess and soccer
robots
Intelligent Agents
Agents are situated


Perception of environment
Execution of actions
Agents can communicate and collaborate


they can differ
than can compete and be more or less
egoistic/altruistic
The agents have:



objectives,
communications,
intentions.
A New Approach
Professor
Kim from
KAIST
The founder of
Robot Soccer and
FIRA president
Two organizations:
1. FIRA (earlier)
2. RoboCup (larger)
Four Blocks in two PCBs (Printed Circuit Boards)




Micro-controller (upper PCB)
Communication module (upper PCB)
Motor and driving circuits (lower PCB)
Power (lower PCB)
top view
front view
side view
Importance of Robot
Soccer
Communication
Cooperation
Coordination
Learning
Competence
Real Time
Robot Soccer Evolution
Computer simulations
Wheeled brainless
robots
Wheeled autonomous
robots
Legged autonomous
robots
Robot Soccer Purpose
“The number one goal of [robot
soccer] is not winning or losing, but
accumulating diverse technology.”

- Mr. Dao (Senior VP of Sony
Corporation).
FIRA
category
MiroSot
3 robots on 1 team
Size : 7.5cm * 7.5cm *
7.5cm
Ball : orange golf ball
Playground : black wooden
rectangular playground

(150cm * 130cm * 5cm)
Vision : global vision system

(more than 2m above
playground)
Experimental Setup
of the Vision System
Control panel
FIRA
category
NaroSot
5 robots on 1 team
Size : 4cm * 4cm * 5.5cm
Ball : orange table-tennis ball
Playground , Vision : same as Mirosot
FIRA
category
KheperaSot
3 robots on 1 team
Ball : yellow tennis ball
Playground : green playground (105cm * 68cm * 20cm)
Robot : Khepera Robot
Vision : K213 Vision Turret
FIRA
category
RoboSot
3 robots on 1 team
Size : 15cm * 15cm * 30cm
Ball : red roller-hockey ball
Playground : black wooden rectangular playground
(220cm * 150cm * 30cm)
Vision : on the robot
Under preparation
SmallSize
League
Small-Size League (F-180)
Field: 2.7 m x 1.5 m
Size
Area : 18cm rule (fit inside in 18cm diameter cylinder)
Height : 15cm (global vision), 22.5cm (otherwise)
teams of autonomous small size robot
play soccer game on a field equivalent
to a ping-pong table.
Each team consists of 5 robots.
Small size league
The field is the size and color of a Ping Pong table
orange golf ball
Robots
move at
speeds
as high
as 2
meters/s
econd

Global
vision
is
allowed
Robot Soccer Initiative
Vision system
Host
computer
Communication
System
Host
computer
Communication
System
Robots on the
playing field
“Brainless” System
Basic Architecture for Robot Soccer Systems
Vision System
Vision : global vision system
(more than 3m above ground)
Each
team
has its
own
camera
and PC
Small-Size League
20 minutes, 2 breaks
Real Robot Small-Size League Competition
MiddleSize
League
Middle-size Real Robot League
(F-2000): Local VISION

The field is the size and color of a 3 x 3 arrangement of Ping Pong tables
(9-3 5-meter field)

Each team consists of 5 robots playing with a Futsal-4 ball (4 players,
one goal-keeper)

Larger (50 centimeters in diameter) robots

Global vision is not allowed.
 Each robot has its own vision system

Goals are colored

Field is surrounded by walls to allow for distributed localization through
robot sensing

Rule structure based on the official FIFA rules
Medium size league
Teams of autonomous mid size robots
Real Robot Middle-Size League Competition
Ball : red small soccer
ball (FIFA standard size 4
or 5)
Playground : green
playground (10m * 7m *
0.5m)
Medium Size League
Medium Size League
Robots can be heterogenous
Middle-Size
League
Sony
Legged
Robot
League
Sony Legged Robot League
3 robots on 1 team (including the goalkeeper).
Robot : AIBO ERS-110 (provided by Sony)
No communication, autonomous robots, software
only. Legged Robot League. 2.8 m x 1.8 m
2 players and 1 goal-keeper in a team
Sony Legged Robot League
Is played on a field, approx 3x2 meter
Sony develops the robots, and provides a
interface for the programming of the robots.
•No Hardware
modification is
allowed
Playing time is
10 minutes per
half, with a 10
minute break at
halftime
Do different Robots have
different personalities?
Some teams have robots with very
different capabilities.
But it is hard to think of them as having
personalities;

rather the robots have different playing
styles.
Early Sony prototype

Robot movements closely mirror those of
animals
•The winner is the team
that scores the most
goals.
• In the event of a tie, a
sudden death penalty
kick competition will
determine the winner
The Legged Robot League
The Legged Robot League
If opposing teams'
robots are damaged or
play is excessively
rough (whether
intentional or not),
penalties may be
assessed to the
offending robot
Humanoid
League
Starting 2002, the humanoid league
Humanoid League
Bi-Ped League
(Humanoid)


Australia
Japan
Where is the science in
these robot
competitions?
Global vision
Local vision
Other sensors
Cooperation
Sensor fusion
Strategy
Learning
Sensors and Actuators for
Robot Soccer
Local and Global VISION
Sensors for Robot Soccer
• Shaft Encoders
– PI controller to maintain wheel speed
– PI controller to maintain path curvature
– Dead reckoning for vehicle position + orientation
• Infrared Distance Measurement
– Avoid Collision
– Navigate and map unknown environment
– Update internal position in known environment
• Compass
– Update orientation independent of shaft encoders
– Fault-tolerance in case robot gets pushed or wheels slip
Sensors for Robot Soccer
• Digital Camera
– Low resolution, 60x80 pixels, 24bit color (Braunl)
– Color or shape recognition
• Communication
– Sharing information among robots
– Receiving commands from human operator
VISION: Color Detection
• In robot soccer, objects are color coded:





ball,
goals,
opponents,
team mates,
walls, etc.
Teach ball and goal color (hue) before starting the
game
Match colors in HSI space
→ Better in changing lighting conditions
Role of Vision
Brain-on-board
system
Robots
The robots have functions such as velocity control, position control, obstacle
avoidance, etc.
Host computer
The host computer processes vision data and calculates next behaviors of robots
according to strategies and sends commands to the robots using RF modem.
2.2 Robot-based system
Distributed system
Intelligent part is implemented
in the robots.
Suitable when the large number of agents exist
Complex and expensive
Need communication among robots
Role of
vision
Robot-based
system
Robots
The robots decide their own behavior autonomously using the received vision
data, own sensor data and strategies.
Host computer
The host computer processes only vision data
can be considered as a kind of sensor.
System
Comparisons
Merits
Demerits

Remote-brainless
system

Robot -based
system



Low cost
Easy to develop
Suitable for many agents
Can use local information



Brain-on-board
system

Suitable to modularize

Cannot use local sensors
High computing power
& fast sampling time
Complex and expensive
robots.
Hard to build the system
Risk of inconsistent
property between host
computer and robot system
Research purpose





Vision system
Multi-agent theory
Robot system
Multi-agent system
development
Robot-based and
vision-based systems
VXD: role of color
Initialization


Click ‘Load VXD’ in the Initialize group box
Click ‘Start Grab’
Configuration







‘Load Conf.’: load a configuration file
‘Save Conf.’: save current configuration
‘Set Robot Size’: set the robot size in number of pixels
‘Set Pixel Size’: set the size of each color (ball, team,
robot, opponent) patch in number of pixels
‘Set Boundary’: set the field boundary on the screen
‘Change Color’: change the color setting of each color patch
‘Set Color’: set the range of tolerance of each color
Subsystems and Vision
Serial Port

Select the serial communication port
Home Goal

Select the home side on the screen
Find Objects

Check the box of which you like to find on the field
Initial Position: tell the vision system the initial position
of each object


E.g.) for the ball
i) turn on the radio button of ‘Ball’
ii) place the mouse on the ball and press the left button
Repeat above procedure for another object
Commands for Vision
Select Situation

The situation in which the game is about to start
Command



Click ‘Ready’: the vision system starts finding the objects
on the field
Click ‘Start’ : the vision system starts sending commands
to the robots
Click ‘Stop’ : the vision system stops finding objects
and sending commands
Tasks for us
How to organize the ceiling camera
system?
How to describe (learn?) the shapes of
robots?
How to find the (x,y,) for each robot?
How to modify the scripted behavior when
the triplets for each robot are known?
How to design the interactive behaviors?
Task for Robot Theatre Team
For next week
Write a half-page essay about the
vision system that we discussed today.
Use you knowledge from other lecture
of today.
Add your imagination and crazy ideas
about what the robots should see and
know for our particular scene of Marie
Curie and Schroedinger’s Cat.