Download AIBO monocular depth perception through

AIBO monocular depth perception through optical flow.
Overview
DECIS (Delft Cooperation on Intelligent Systems) has
joined forces with several Dutch universities to investigate
collaborative robot behavior. Together a total of 14 AIBO
robot dogs have been purchased for research and to play in
the RoboCup soccer competition. The AIBO is a versatile
platform that helps explore new directions. A major
challenge is the perception of the environment via robot
vision. An interesting and promising direction in robot
vision is depth perception through real time optical flow
calculations. This technology will allow the AIBO to extract depth information from the
scene viewed by its (single) onboard camera. The depth information can be used directly
for getting a 3D sense of the environment useful for navigation, obstacle detection, etc.
The aim is to implement existing optical flow algorithms in the AIBO software and
explore new possibilities on this real-time robot platform. A possible real life test case is
the Dutch AIBO soccer team; can AIBO depth perception lead to better performance in
the soccer field?
Introduction
Optical flow
Depth perception can be achieved through a number of optical cues including optical
flow. Essentially information about an object’s position can be discovered by slightly
changing the viewing position. The images from the different viewpoints are compared
and using an algorithm depth information about the viewed scene can be calculated.
Monocular depth perception through optical flow is the application of this process
continuously while traveling through a space. Nearer points in the space will flow by
quicker than far away points. To visualize this, imagine looking out the side window
while driving on a highway; nearby trees move by fast while far away trees move by
rather slowly. This reveals their relative depth with respect to the viewer. Their absolute
distance to the viewer can be calculated if the viewer motion is known precisely. The
interesting property of this technique is that depth information about the viewed scene is
generated with a single camera simply moving through it.
Examples of optical flow
Below is an image from a car driving along a road. This image is one of a sequence from
which the optical flow is calculated. The dots indicate the points that are being compared
to the previous image and the lines indicate the corresponding vector of optical flow in
those points. Note that the nearby points have a greater amount of flow (thus larger
vector) than far away points. In this scene it is clear that optical flow can be used to get a
sense of self-motion and at the same time see depth in the scene. For example the trees to
the left are nearby in contrast to the open space on the right except for the small bush
1
generating some optical flow. If an object were on the road nearby it would be detected
because it would generate relatively much optical flow, like the white stripe on the
middle of the road.
(source: http://www.ail.cs.gunma-u.ac.jp/~ohta/3-D.html)
In the above example the scene is static; the only moving object is the observer/camera
traveling through the environment. The reversed situation is also possible; in the example
below the moving cars can be detected by a static camera because of the optic flow they
generate in the otherwise static scene. This is another application of the same principle.
More problematic would be scenes where the observer is moving in a dynamic
environment because the optical flow generated by moving objects will interfere with the
optical flow generated by the moving observer.
(source : http://woodworm.cs.uml.edu/~rprice/ep/kuehne/)
Project description
Goal
The goal of this research is to investigate whether real-time optical flow depth perception
is a viable and useful enhancement for AIBO vision. Existing algorithms and techniques
can be used to obtain depth information in the moving scene. The research challenge is
2
the implementation of the algorithms in the AIBO and testing the capabilities in an
application domain.
Project phases
The project phases depend on the difficulties encountered and the progress made. The
first phases concern the implementation and the last phase deals with using the depth
perception for an application. It should be expected that there is a fair amount of overlap
between the phases and they are intended only to give an overview.
The first phase of the project consists of an assessment of the AIBO system in
combination with optical flow processing. For example, the algorithms can be run on the
AIBO or on a PC connected to the AIBO over wireless LAN. The choice will depend on
required and available processing power, required image quality and frame rate, and the
latency of the network.
The second phase of the project addresses the problems associated with getting the
optical flow algorithms to output decent depth information. These problems will be
related to the AIBO platform, its camera and environment lighting conditions. If needed a
combination might have to be made with for example the AIBO range finder to get more
precise depth information when needed. One can also imagine a depth perceptioncalibration of vision using the range finder.
The third phase of the project will focus on using the depth perception to carry out a task
in the environment. There are a number of interesting possibilities open for the student to
choose from. Some examples are:
- 3D mapping of the environment for navigation purposes.
- Object / obstacle detection and avoidance
- Judging if small obstacles can be climbed over and implementing climbing
routines (ROBORescue application)
- When the project progresses quickly collaboration with the Dutch AIBO soccer
team is possible. This is described below.
AIBO soccer Dutch AIBO team
Working close with the Dutch AIBO team is
advised. There is a lot of AIBO expertise in
this group and perception in a soccer game is
a real-life challenge. Depth perception in the
soccer field could give the Dutch AIBO team
the advantage it needs to win the RoboCup
2006 competition. It could mean an
advantage in the following areas:
- An AIBO needs to locate itself on the
field. More precise, quicker
establishment of its own position
3
-
could mean better reaction times and higher score rate.
In a known environment, like the Robocup AIBO soccer field, an attempt can be
made to detect moving objects in the environment based on optical flow.
Optical flow might allow an AIBO to better perceive the motion of the ball,
increasing the chances of interception and improving its aim.
Associated research
-
-
At the faculty of aerospace engineering at the TUDelft work is being done on
developing optical flow depth perception techniques for Unmanned Aerial
Vehicles (UAVs) vision. This group is also interested in the AIBO vision and
collaborates with DECIS on UAV research.
Dutch Aibo Team, http://www.dutchaiboteam.nl/
Student qualities
The student should have affinity with problem solving, programming, computer vision,
and robotics.
The student should be willing to collaborate with other researchers in Utrecht, Delft and
other cities on AIBO research.
Contacts
Graduation professor:
Henk Corporaal
Technische Universiteit Eindhoven
[email protected]
Mentor:
Hamed Fatemi
Technische Universiteit Eindhoven
[email protected]
Project Coordinator:
Matthijs Amelink
DECIS lab
[email protected]
4