Download 2. TOURBOT On-board interface - FORTH-ICS

PROJECT DELIVERABLE D10
TOURBOT On-board Interface
Shared-cost RTD
Project acronym: TOURBOT
Project full title: Interactive Museum Tele-presence Through
Robotic Avatars
Contract Number: IST-1999-12643
Key Action:
3
Action Line:
3-2-3
TOURBOT
On-board Interface
Project
Deliverable
TOURBOT: Interactive Museum Tele-presence Through Robotic Avatars
Project Deliverable D10: TOURBOT On-board Interface
Date Produced:
March 26, 2001
Authors:
Maria Roussou and George Kamarinos
Contents
1. Introduction .................................................................................................................. 3
2. TOURBOT On-board interface .................................................................................... 5
(a) Selection area ......................................................................................................... 7
(b) User control information ..................................................................................... 10
(c) Exhibit information.............................................................................................. 10
(d) Exhibition floor plan ............................................................................................ 11
3. Interface Implementation ........................................................................................... 13
Page 2
TOURBOT
On-board Interface
Project
Deliverable
1. Introduction
The goal of TOURBOT is the development of an interactive tour-guide robot able to
provide individual access to museums’ exhibits and cultural heritage over the Internet as
well as unique on-site guidance. TOURBOT operates as the remote user’s physical
presence in the museum by accepting commands over the Web that direct it to move in its
workspace and visit the specified exhibits. At the same time, TOURBOT acts as an onsite museum guide by accepting commands given directly through its on-board screen.
More specifically, the objectives of the project are the following:
(1) develop a robotic avatar with advanced navigation capabilities that will be able to
move (semi)autonomously in the museum’s premises,
(2) develop appropriate interfaces to the robotic avatar that will realize distant-user’s
telepresence, i.e. facilitate scene observation through the avatar’s eyes,
(3) facilitate personalized and choice-driven access to information about museum
exhibits, and
(4) enable on-site, interactive museum tour-guides.
The last three objectives are directly related with the way in which the robotic avatar will
communicate with the users, both the Web-visitors and the on-site visitors of the
museum. In both cases, the main channel of communication between the robot and the
remote or local users is the interface. Both categories of users require an individually
designed and developed interface. For the sake of uniform presentation media, similar
layouts and design concepts have been adopted for both interfaces. The on-board
interface is based on a LCD touch screen. The web-interface uses browser technology in
order to conform to current standards on web-access.
The design of the interface has been based on a number of principles:
Page 3
TOURBOT
On-board Interface
Project
Deliverable
 Intuition and ease of use: the average user is expected not to be a computer specialist.
Therefore, an intuitive and user friendly interface is of utmost importance for the
overall acceptability and success of the system.
 Compact design and provision of feedback: major operations and information to the
user are provided in a single window in order to facilitate complete overview of the
application at run time. Moreover, appropriate feedback is provided to the user for
awareness purposes.
 Provision of the appropriate information: the interface has to provide different types
of information to the user in a consistent manner. To achieve that, the screen
(interface) is divided in conceptual sectors, each of which corresponds to a (sub)
window that provides relevant information.
 Control of the robotic avatar: only simple and intuitive forms of control over the robot
are provided, in order to avoid complicated control sequences and/or decisions from
the user side.
Effort has been made so that the design of the interfaces complies fully with the above
principles.
Page 4
TOURBOT
On-board Interface
Project
Deliverable
2. TOURBOT On-board interface
The on-board interface to the TOURBOT system is the main access front that enables the
physically present user(s) to interact with TOURBOT. The purpose of the on-board
interface is to enable two main functions:
(a) control over the robot’s motion (navigation towards a goal point), and
(b) user interaction and presentation to the users of multimedia information regarding the
exhibits.
The interface should cater for both these functions through the same screens; hence the
users must be able to receive information about the exhibits that they visit by controlling
the robot’s motion in space.
The on-board interface is organized into the following sections (see Fig. 1):
(a) A general selection area
(b) A window where information to the user is constantly displayed
(c) An exhibit information area
(d) An exhibition floor plan window
Section (c), although not particularly required, is provided to further enhance the
informational purpose of the robot. A description of each section follows.
Page 5
TOURBOT
On-board Interface
Project
Deliverable
Fig. 1. A general layout of the various sections of the on-board interface.
Page 6
TOURBOT
On-board Interface
Project
Deliverable
(a) Selection area
The user selection area, located on the left side of the screen, is the area that provides the
user with information regarding the ways that the robot may be directed through the
exhibition. Through the on-board interface, the user is provided with two different
methods of visiting the exhibition: through the use of predetermined routes and through
direct selection of specific exhibits from a list (see Fig. 2). The selection area is where the
user selects the touring method and prompts the robot to begin its journey.
Fig. 2. TOURBOT On-board interface: tour selection options.
Page 7
TOURBOT
On-board Interface
Project
Deliverable
The two methods of access to the exhibition are always listed as two separate buttons
(‘TOURS’, ‘EXHIBITS’) placed on the upper part of the selection area.
When the user selects ‘TOURS’, a list of different tour options to choose from is
displayed in the selection area (see Fig. 3). These are predetermined paths that the robot
follows and can be semantically specified according to different criteria. For instance,
tours can be specified by the exhibition curators as “specialists’ tour”, by the museum
visitors as the most “popular tour” (by observing visitor preferences), as a “quick tour”
by defining the best possible route, etc.
Fig. 3. TOURBOT On-board interface: predetermined tour selection.
Page 8
TOURBOT
On-board Interface
Project
Deliverable
When the user selects the button ‘EXHIBITS’, a listing of the titles of different exhibits is
displayed. The user may select any exhibit and thus command the robot to choose the
best possible way to reach that exhibit (see Fig. 4). The interface will display information
about every exhibit that the robot approaches on its way to the chosen exhibit. At the
same time, a yellow dot on the floor plan indicates the robot’s path and current position.
Fig. 4. The TOURBOT On-board interface listing the exhibits that can be visited.
The on-board interface of TOURBOT does not provide the option to choose a point or
exhibit directly from the floor plan of the exhibition space (not a meaningful operation
for the on-site user). However, the floor plan is always visible (and takes up a larger area
Page 9
TOURBOT
On-board Interface
Project
Deliverable
on the interface for better visibility). The user can thus observe at any time the position of
the robot as it navigates through the exhibition. The colored boxes on the floor plan
indicate the position of each exhibited object.
(b) User control information
A user control information message box is always visible on the on-board interface to
display the current connection information. This panel lists messages to the user
indicating the number of web users that are waiting in the queue to control the robot and
the time required until the on-site user can take control of the robot (see Fig. 6). Other
messages are also displayed such as information about the location of the robot or the
exhibit that is being approached.
(c) Exhibit information
A basic function of every museum guide is to describe and explain the exhibits to the
visitors. TOURBOT's Web interface uses multimedia content, text images and other, to
provide information about the exhibit. A human guide uses speech and gesture to guide
visitors through an exhibition. TOURBOT's on-board interface attempts to somewhat
combine both approaches by using a text to speech feature in order to provide
information about the exhibits. Additionally, the on-board screen also presents written
information and images to the user through its interface in order to further enhance the
informational quality of the guided tour.
The multimedia information pertinent to the exhibits of the three user sites has been
compiled for all three sites. The material (data) is in html format and is structured using
the following convention on the structure of the directory tree:

one directory for each museum (FHW, MUSBON, BYZMUS),

one sub-directory for each showcase/exhibit, following the numbering that appears on
the user interaction map,

sub-directories for images, text, sound, and video.
Page 10
TOURBOT
On-board Interface
Project
Deliverable
(d) Exhibition floor plan
The floor plan of the museum is always displayed on the screen while a simple graphical
representation of the robot's position in space is dynamically displayed on the plan,
providing an overview of where the robot is located at any given moment.
The exhibition floor plan relates to the robotic system, which needs information about the
exhibition area, in order to be able to navigate safely and reliably in it. This information
is provided in the form of a map of the environment, which the robot uses to keep track
of its position within the environment. The maps used internally by the robot's navigation
system are not particularly useful for user interaction since they are not intuitive for the
average user. Therefore, the maps used for user interaction are CAD maps that resemble
well-known floor plans. These plans are aligned with the grid maps, e.g. they have the
same size and can also be used to indicate the robot’s position to the user; additionally,
they are annotated with information regarding the position of the exhibits. Figure 5 shows
the maps for user interaction for the three sites. The navigation maps (machine oriented)
are clearly probabilistic in nature and they show several exits from the exhibition area,
which the robot will never use. The latter might confuse the user and are omitted in the
interaction map.
Unlike the web interface of Tourbot where the user is able to select and exhibit or point
on the floor map and direct the robot to that location, the on-board interface does not
provide this capability for usability reasons (due to the small size of the boxes it would be
difficult to select them on a touch screen).
Page 11
TOURBOT
On-board Interface
Project
Deliverable
Figure 5. Maps for user interaction; (top left) Byzantine and Christian Museum of
Athens, (top right) Foundation of the Hellenic World, (bottom) Deutsches Museum Bonn.
The on-board interface does not include links of general interest to the user, as it is a local
kiosk and not a web browser.
Page 12
TOURBOT
On-board Interface
Project
Deliverable
3. Interface Implementation
The interface system (server) handles communication between the web and on-board user
interfaces and the main navigation system of the robot, handling all requests from the
user to the robot and vice versa. In other words, the interface module serves as the main
integrator between the robot navigation module and the user control. The interface
system handles both the web and the on-board interfaces, outputting the relevant
information for each interface where appropriate.
After thorough consideration of available technologies it was decided to implement the
interface module using the Java programming language and the client-server approach, as
appropriate for coding the communication and presentation structures implemented in
this module. The web pages were created using HTML, Dynamic HTML, and Javascript.
The interface system contains Java applets on the web page, a main Java application
module that acts as a server, and the HTTP server (see Fig. 8).
The Java applets correspond to the various sections of the interface as these were
described in Fig.1. A main Java applet on the web page (the client) opens a connection
with the Java application that acts as the server. All necessary information (the location
of the robot, messages to the user, control input from the user etc.) is communicated
throughout this connection. Inter-applet communication takes place between the main
applet and all other applets in the same web page in order for them to use the main
connection to the server and to be synchronized. The user selection applet, for instance,
communicates to the main applet all the user choices, such as a specific exhibit or a
specific tour. This information is then sent to the server that in turn processes it and sends
it to the robot navigation system. Open communication links are needed between the
applets and the Java application module to convey relevant information (e.g. location of
the robot and commands to the robot). These modules are connected via a TCP/IP socket.
The currently implemented messages that can be sent by the applets to the robot via the
Java server include:
-
a request to start the robot
-
a request to go to a specific exhibit as selected on the exhibit list (goto exhibit No._ )
Page 13
TOURBOT
On-board Interface
-
Project
Deliverable
a request to follow a specific tour as selected on the tour list (goto tour No. _ )
The Java application is connected as a client to the robot navigation module (server)
using a TCP/IP socket connection. In other words, the Java application acts as a client to
the robot navigation system and as a server to the applets module.
The Java server performs multiple functions for both the web and the on-board interfaces
simultaneously. The Java application manages the user connections to the system,
whether these come from the web or the on-board interface. A priority protocol has been
established to better manage and integrate on-line and on-site users. Thus, on predetermined days and times, priority is given by the Java server to either on-site or web
visitors, depending on exhibition visiting hours or other factors. The Java application
creates and manages the queue of user requests for connections accordingly and activates
the first user in the queue when the time has come. The duration of each user’s
connection is also decided and handled by the server, which sends the appropriate
messages to activate or terminate connections.
Page 14
TOURBOT
On-board Interface
Project
Deliverable
Figure 8. Structure of the on-board interface implementation
Page 15
TOURBOT
On-board Interface
Project
Deliverable
As a client, the Java application constantly receives the robot coordinate information
from the robot navigation module and conveys it to the applets. Additionally, the robot
navigation module can send information regarding the exhibit it has arrived at and other
messages concerning the state of the robot.
The HTTP server runs on the same machine as the Java server for security reasons since
the applets are allowed to open a socket connection only back to the hostname from
which they were loaded. Thus, the Java server acts as an intermediate stage between the
web serving and the final output, integrating all modules together.
Page 16