Download IT and knowledge management for robotics - Neuron

INTERNATIONAL SCIENTIFIC CONFERENCE
„IN LOOK DAYS 2011“
13. – 14. OCTOBER 2011
AULA MAXIMA, TECHNICAL UNIVERSITY OF KOŠICE
SLOVAKIA
ZÁKLADNÝ KONCEPT IT A ZNALOSTNÉHO MANAŽMENTU PRE OBLASŤ
ROBOTIKY V BUDÚCNOSTI
BASIC CONCEPT OF IT AND KNOWLEDGE MANAGEMENT FOR
FUTURE ROBOTICS
MÁRIA VIRČÍKOVÁ1, PETER SINČÁK2
1
ING. MÁRIA VIRČÍKOVÁ, KATEDRA KYBERNETIKY A UMELEJ INTELIGENCIE, FAKULTA ELEKTROTECHNIKY A
INFORMATIKY, TECHNICKÁ UNIVERZITA V KOŠICIACH, SLOVENSKÁ REPUBLIKA, [email protected]
2
PROF. ING. PETER SINČÁK, CSC. , KATEDRA KYBERNETIKY A UMELEJ INTELIGENCIE, FAKULTA ELEKTROTECHNIKY A
INFORMATIKY, TECHNICKÁ UNIVERZITA V KOŠICIACH, SLOVENSKÁ REPUBLIKA, [email protected]
MÁRIA VIRČÍKOVÁ1, PETER SINČÁK2 (EN)
1
ING. MÁRIA VIRČÍKOVÁ, DEPARTMENT OF CYBERNETICS AND ARTIFICIAL INTELLIGENCE, FACULTY OF ELECTRICAL
ENGINEERING AND INFORMATICS, TECHNICAL UNIVERSITY OF KOSICE, SLOVAKIA, [email protected]
2
PROF. ING. PETER SINČÁK, CSC. , DEPARTMENT OF CYBERNETICS AND ARTIFICIAL INTELLIGENCE, FACULTY OF
ELECTRICAL ENGINEERING AND INFORMATICS, TECHNICAL UNIVERSITY OF KOSICE, SLOVAKIA
[email protected]
Abstrakt (MATERINSKÝ JAZYK):
Článok popisuje multi-agentový system pre vytvorenie siete pre roboty. Hlavná výhoda spočíva v tom, že ak sa určitú
znalosť naučí jeden robot zo skupiny (siete) robotov, zdieľa túto znalosť s ostatnými robotmi v sieti. Úvod vysvetľuje
koncept sieťových robotov a ich aplikácie. Predstavujeme aplikácie vyvinuté v Centre pre inteligentné technológie na
Katedre kybernetiky a umelej inteligencie TUKE – rozpoznávanie obrazu a povelov. Detailnejšie popisujeme náš
prístup učenia robotov pozorovaním človeka, v ktorom sa robot učí od človeka pohyby alebo gestá.
Klúčové slová (materinský jazyk): kolaborácia človek-robot, multi-agentový system, sieťové
robotické systémy
Abstract (EN)
This paper describes a multi-agent system for creating a network for robots. The main advantage
is that if some knowledge is learned by one robot from the group (network) of robots, it shares this knowledge with
other robots in the network. The introduction explains the concept of networked robots and their applications. We
introduce applications developed at the Center for Intelligent Technologies at the Department of Cybernetics and
Artificial Intelligence TUKE- object and command recognition. In more detail we describe our approach to robot
learning by observing humans, in which the robot learns from human movements or gestures.
Key words (EN): human-robot collaboration, multi-agent system, networked robot systems
Strana 1
1 Introduction
Robots are functioning in environments performing tasks that require them to coordinate with other
robots, cooperate with humans and act on information derived from multiple sensors. The specific
objective of the field called Human-robot interaction is developing a series of tools to have a robust
communication interface between robots and human users.
As described in our previous work [1], in many cases, these human users, robots and sensors are
not collocated and the coordination and communication happens through a network.
It a relatively young research field, still quite dynamic and open to ideas and concepts coming
from other research areas. As reported in European Robotics Research Network [2], Europe holds a strong
position in this field, and it could become a world leader in the near future if enough support is provided.
They mention these possible application fields, regarded as viable in the next ten years:
• Elderly care, where a network robotic system (NRS) can assist elderly people in their personal
homes or retirement homes, with the goal to provide physical and cognitive support, to facilitate
communication with and monitoring by remote relatives and care givers, and to detect and respond to
emergencies.
• Security applications and intelligent buildings, where a NRS can track and classify the behavior
of people, hazardous situations or threatening acts can be detected, and actions can be decided to maintain
a safe status.
• Networked service robot systems, for applications such as trash collection, delivery and logistics,
both in public spaces (e.g., city streets) and in private workplaces (e.g., factory floors).
• Flexible automation and collaborative manufacturing, where heterogeneous, cooperative robotic
systems are expected to set forth the future of networked control in industrial settings.
• Wide-scale environmental monitoring, by deploying autonomous sensor networks with the
ability to self-deploy, self-reconfigure, and self-repair, capable of monitoring in a largely unsupervised
way large environments for pollutions, environmental threats, and other hazardous situations.
• Cooperative search and rescue, where NRS actively search for people or objects, and eventually
support rescue personnel in dealing with emergencies.
Networked robots allow multiple robots and auxiliary entities to perform tasks that are well
beyond the abilities of a single robot. Cooperation entails more than one entity working toward a common
goal while coordination implies a relationship between entities that ensures efficiency or harmony.
Communication between entities is fundamental to both cooperation and coordination and hence the
central role of the network.
2 Networked robots
This new technology has being denominated Network Robot Systems and, as said in [3], includes the
following elements:
1. Physical embodiment: Any networked robot system has to have at least a physical robot which
incorporates hardware and software capabilities
2. Autonomous capabilities: A physical robot must have autonomous capabilities to be considered as
a basic element of a networked robot system.
3. Network-based cooperation: The robots, environment sensors and humans must communicate and
cooperate through a network.
4. Environment sensors and actuators: Besides the sensors of the robots, the environment must
include other sensors, such as vision cameras and laser range finders, and other actuators, such as speakers
and flickers
Strana 2
5. Human-robot interaction: In order to consider a system as networked robot system, the system
must have a human-robot related activity.
FIG. 1 – HIERARCHICAL SYSTEM ARCHITECTURE: THE INFORMATION FLOW BETWEEN SEVERAL HUMAN TELEOPERATOR REMOTE ROBOTS AND A
HUMAN (SOURCE: [4])
It seems that the basic concept of creating Intelligent Systems will be based on the following
principles:
1. Networked entities with defined hierarchy, fast mobile connection will be absolutely needed and
for majority of applications in business and industry.
2. Knowledge sharing defined policy - domain oriented knowledge will be great value.
3. Integration of approaches into standard services provided for clients.
4. Appropriate division of processes for client (Agent) and server WILKI is based on application
domain.
5. World Incremental Knowledge Integrator and its implementation is under discussion using Cloud
Technology or hybrid of Public and Private Cloud Approach.
With the ideas discussed above, we created a Multi-agent Serving System (MASS). MASS system
is written in C# programming language. It can be used for different tasks, not only from the field of
artificial intelligence. The system is based on client-server architecture, where clients ask the server and
the server responds. Basic building blocks of the system are called plugins. Individual plugins can be
connected to each other, and thereby build a functional algorithm (ActualSystem-AS) which consists of
these building blocks. The system provides a graphical representation of individual system components,
and thus the easier visual control. The server can run several AS simultaneously, each with a different
functionality. Every AS can be connected to several clients at the same time.
Strana 3
FIG. 2 – MASS SYSTEM BASED ON CLIENT-SERVER ARCHITECTURE
We can say that client-server architecture is a multiagent system. In the case that various plug-ins
are programmed for command recognition and a client learns to recognize some particular word, all other
clients are able to recognize it also. If one client learns to recognize some object, all other clients
recognize it.
3 Cooperation between network robots and human beings
Much research has been done on humans interacting with robots but especially the interaction between
human beings and a group of connected robots provides interesting aspects. The general objective of our
work is the development of new ways of cooperation between networked robots and human beings, which
to date has been seldom explored in the field of Human-Robot Interaction.
The IEEE Society of Robotics and Automation’s Technical Committee on Networked Robots
made the following definition of Networked Robots:
A ‘networked robot’ is a robotic device connected to a communications network such as the
Internet or LAN. The network could be wired or wireless, and based on any of a variety of
protocols such as TCP, UDP, or 802.11.
Many new applications are now being developed ranging from automation to exploration.
There are two subclasses of Networked Robots:
1. Tele-operated, where human supervisors send commands and receive feedback via the
network. Such systems support research, education, and public awareness by making valuable
resources accessible to broad audiences;
2. Autonomous, where robots and sensors Exchange data via the network. In such systems, the
sensor network extends the effective sensing range of the robots, allowing them to
communicate with each other over long distances to coordinate their activity. The robots in
turn can deploy, repair, and maintain the sensor network to increase its longevity, and utility.
A broad challenge is to develop a science base that couples communication to control to
enable such new capabilities.’
Strana 4
Human interaction with a multiple number of robots increases complexity, but can also be a
source of benefits when considering the exploitation of the complementarities of the robots for perception
and actuation. A team of agents should be able to learn collective behaviors, such as strategies to pursue
their goals in the environment, in the face of competitors. Learning in multirobot systems is affected by
specific challenges like multiple goals, noisy perception and actions, and inconsistencies in the internal
states and in environment models between the individual robots.
The first application and also the guiding application of the Multi-Agent Serving System MASS
was the object recognition. It was developed by T. Reiff and can be better studied in [5]. The second step
was a command recognition - at the date it contains a database of verbal commands in different languages.
The command recognition was developed by Z. Fedor – more information can be found in [6].
The key advantage of our system is if a user teaches a robot some word in some language /an
object, all other robots which are part of the multi-robotic system are able to recognize this word /the
object. Object recognition and command recognition are important components of the Human-Robot
Interactive System in general, where a human user can teach one robot to recognize an object visually and
all the robots within the networked group learn it or can give verbal commands to a single robot, or to a
multi-robotic system. Also, in human-human teamwork, sharing information through verbal and nonverbal channels plays an important role in coordinating joint activity and we think this will be the case for
human-robot teams as well.
`
Our present work consists in teaching robots new gestures and movements. We are developing
a tele-robotic approach, in which human shows to robot how it should perform a motion or a gesture.
Our challenge was to create a system in which an inexperienced human teacher could develop
new motion tasks via interaction process. Robots acquire knowledge of what to do and how to do it from
observing human demonstrations. The algorithm calculates positions of each joint of human and maps it to
the robot´s joint.
FIG. 3 – IN OUR TELE-ROBOTIC APPROACH WE MAP USER’S MOVEMENTS/GESTURES TO ROBOT
Teaching through demonstrations requires bidirectional communication between a human user
and a robotic student. The key question is how to map a teacher action to an agent action. These agent
actions in our experiments are the motor capabilities of the robots. We mimic some user movements using
Microsoft Kinect sensor device, a horizontal bar connected to a small base with a motorized pivot and is
designed to be positioned lengthwise above or below the video display. The device features an RGB
camera and depth sensor among other capabilities. Kinect can recognize 48 points of the human body and
in we extract this information while he is moving in time. We obtain a vector containing information
about the position of each point in space and map it to the related joint of the humanoid robot Nao (Fig. 3).
Kinect provides between others a full—body 3D motion capture capability. The hardware emits
and infrared light which rebounds off object in the room. After processing the rebounding light, the
system differentiates between people and other objects. The system gives the body a virtual skeleton with
moving joints so the machine can follow user movements. From programmer´s point of view, rather than
having access to the pixels of a flat image, or even the depth information from the raw Kinect data, we
have a description of users as a series of joints in space. For many applications, especially gesture-based
interfaces and motion capture, this is exactly the data which we need to get started.
Strana 5
FIG. 4 – HUMANOID ROBOT NAO
We work with humanoid robot Nao which has a human—like appearance and various sensors for
interacting with humans. It is nowadays commonly used humanoid platform for education environment. It
is a medium-sized robot available mainly for universities and laboratories for research and education
purposes. The creator is a French company, Aldebaran Robotics (http://www.aldebaran-robotics.com/).
In community of artificial intelligence, by humanoid we mean not only a robot that has the
physical appearance of a human, but also has a fairly advanced artificial intelligence, allowing it to
reasonably approximate human behavior and interactions. As our robot increasingly resemble human in
shape, we map his servos to the points of human body. This way we model different robot´s motor skills
via its interaction with several human users—we personalize robot´s behaviors according to the interaction
of the specific human user. The algorithms used in our approach are discussed in the paper [7].
FIG. 5 –HOW THE SENSOR DEVICE KINECT DETECTS A PERSON (BLUE) AND THE NAO ROBOT (GREEN)
MASS is not trying to upload the functionality on the robot but it is using the robot as an
input/output device while the processing is on a normal computer. With this approach a supported robot is
only a plugin, i.e. small library with functions, which has a functionality to get video, audio, sensor data,
etc. from the robot and the functionality to control a robot movement.
Strana 6
4 Conclusion
This work presented a multi-agent system, in which we implemented object recognition and
command recognition. Our present work consists in tele-robotics approach and teaching robots
gestures or any kinds of movements.
A major challenge is to create robot networks that are proactive and anticipate our needs and commands
rather than reacting (with delays) to human commands. We think it is very important for a middleware to
be free and open source. Therefore anyone can try the web interface of the system, read the wiki pages,
download the installation package or find the MASS repository address at http://brain.fei.tuke.sk. There
are also links to some of our demonstration videos with MASS.
Acknowledgement
This work is the result of the project implementation: Development of the Center of Information and
Communication Technologies for Knowledge Systems (ITMS project code: 26220120030) supported by
the Research & Development Operational Program funded by the ERDF.
References
[1] SINČÁK, P., REIFF, T., VIRCIKOVA, M.: Towards Intelligent Systems With Inceremental Ability. In:
Quo Vadis Intelligent Systems. ISBN 978-80-8086-173-5, 2011, pp. 58-64.
[2] SAFFIOTTI, A., LIM, P.: European Robotics Research Network: Two Hot Issues in Cooperative
Robotics: Network Robot Systems, and Formal Models and Methods for Cooperation. A white paper
from the EURON Special Interest Group on Cooperative Robotics. (April 30, 2008).
[3] SANFELIU, N., HAGITA., A. SAFFIOTTI.: Network Robot Systems. In: Special Issue: Network Robot
Systems, Robotics and Autonomous Systems, 2008.
[4] SCHILLING, K.: Networked Robots: Research challenges. In: ETSI Networked Mobile Wirelles
Robotics Workshop. Hannover, 2010.
[5] REIFF, T., SINČÁK, P.: Multi-Agent Serving System For Intelligent Technologies. In: ICCC, 2008.
[6] FEDOR, Z., SINČÁK, P.: AIBO talking procedure in multi languages based on incremental learning
approach. In: SAMI 2009 : 7th International Symposium on Applied Machine Intelligence and
Informatics : January 30-31, 2009, Herľany, Slovakia. - S.l. : IEEE, 2009.
[7] VIRCIKOVA, M., SINČÁK, P.: Integration of Verbal and Non Verbal Human-Robot Interactive System.
In: Humanoids 2011 IEEE Conference. October 26-28, Slovenia. (In Press)
Strana 7