Download Paper 1: A network test bed for highly mobile aerial nodes

A Full-Scale Testbed for UAV and Ground Mobile Ad Hoc Networks
Timothy X Brown, Sheetalkumar Doshi,
Sushant Jadhav, Roshan-George Thekkekunel
Interdisciplinary Telecommunications
Electrical and Computer Engineering
University of Colorado
Boulder, CO, 80309-0530
[email protected]
July 31, 2004
Abstract
1
INTRODUCTION:
From November 2003 and July 2004 we developed a full-scale physical testbed to
study the performance of a mixed network of fixed, ground vehicle, and unmanned aerial
vehicle (UAV) mounted wireless ad hoc network nodes, where the nodes are based on
IEEE 802.11b wireless LAN cards. The goals of this test bed are three-fold. First, we
wish to demonstrate the viability of such ad hoc networks to provide useful
communication services. Second we want a tool for analyzing ad hoc networks in highly
dynamic outdoor environments. Third we want to study the effects of the inclusion of
UAVs in the network and how they can be used to further increase the network efficiency
on the whole.
In order to achieve these goals, we built a large-scale testbed with multihop
communication distances up to 5km. The routing protocol is the Dynamic Source
Routing (DSR) [reference] protocol implemented via Click [ref] in the Linux operating
system. A monitoring agent runs on each node to record data on packets passing through
the router. A robust monitoring collection architecture was developed to collect and view
this data in real time. The data is also archived to allow post processing or viewing via a
web-based visualization tool. The software runs on Linux laptops, and has been ported to
other architectures including PDAs and a low-cost single board computer. The
combination of DSR running on the single board computer with an 802.11 card and an
RF amplifier we denote a mesh network radio (MNR). Some MNR have been packaged
in environmental enclosures for outdoor use. Others are mounted in small UAVs that
have been built for this purpose.
This paper describes the testbed design in detail and our preliminary data. A fullscale series of tests are planned through October of this year which will provide a
complete picture and will be included in the final version of the paper. The next section
describes the motivation for building a full-scale testbed.
2
WHY HARDWARE TESTING IS IMPORTANT
The vast majority of mobile ad hoc network (MANET) research has been based on
theory and simulation. This testbed project has been labor intensive developing the
networking components, preparing the physical testbed, and executing the experiments.
Such efforts are needed in addition to simulation for several reasons:
 It is difficult to model all the interactions, timing conditions, and effects of
distributed peer-to-peer agents each running within a computer operating system,
interacting through a complex adaptive wireless interface, over a highly variable
radio environment. A hardware implementation exposes all of these issues.
 The process of building the hardware test bed is instructive to learn the real
challenges and limitations of MANETs.
 The data collected can improve modeling for other researchers.
 The hardware and software developed can be used as a basis for commercial,
military, and scientific applications of MANETs.
There have been other test bed implementations <list here briefly each paper e.g. an
early outdoor testbed was the CMU testbed for DSR [ref,ref,ref]. Other testbeds have
focused on indoor environments [refUppsala,…]. Yet others have focused on various
levels of environment emulation [ourwork, some others you can get from the EWANNT
paper].>
Our work adds several dimensions on these prior works. First, the testbed is large
scale with distances spanning several kilometers. The larger distances enables better
study of multihop behavior in natural environments and better shows the ability of
MANETs to enable communication in environments which lack other means of
communication. Second, we include aerial vehicles. UAVs are more than simply a faster
node or a third dimension. They are qualitatively different nodes. The high position in the
air means a UAV has better coverage and lower path loss than other nodes on the ground.
UAV antenna pattern quickly tilts as the plane banks and turns causing radio dynamics
beyond simple changes in position. The UAV is also a harsher environment with high
vibration, large mechanical shocks, and occasional power interruptions. Other works
have considered UAVs in MANETS [Gerla Paper], but, only in simulated environments
and for large high-altitude UAVs. Third, we added detailed monitoring data collection.
The data is placed into a database for post analysis and a web-based visualization tool
will enable the MANET community to view and analyze the data for research and
education purposes. The next section describes the testbed architecture.
3
PRIOR WORK ON MANET:
<the references here will go into the above section and this section can be
eliminated. Sushant, these are good summaries of the papers which you may want to use
in other papers like your thesis.>
Ad-hoc networks have been a subject of immense research, including the
applications to large UAVs. Much of these investigations have been in theory and
simulation with very few attempts at real world implementations of this technology. This
section of the paper discusses previous related work done on MANET test beds.
Over the past few years there has been extensive research on the simulation of ad
hoc networks all across the world. Researchers from the University of California, LA [8]
have simulated a MANET with UAVs using LANDMARK routing. Their design is based
on a hierarchical routing scheme that is aimed at reducing the hops between source and
destination nodes and provides support to large-scale military wireless networks. The
paper describes a test bed that was simulated using a Parsec/GloMoSim platform. Using
this test bed the researchers have successfully demonstrated the improved performance of
a network that utilizes flat LANMAR routing and physical network hierarchy comprising
of ad hoc ground radio subnets, point to point wireless long haul backbones, and
Unmanned Air Vehicles (UAVs). Another paper titled “Designing and Implementing A
Mobile Ad hoc Network Test bed”[11] deals with an operational ad hoc networking test
bed that allows the study of various ad hoc routing in a laboratory environment with
emphasis on low cost. Much of the research done is based on software assisted emulation
wherein each node runs unicast and multicast routing protocols on a dynamic-on-demand
basis. Their work on the test bed implementation is supposed to serve as a helpful
precursor to the real world deployment of a full scale ad hoc network.
All of the work done on MANETS that has been mentioned in this section so far is
based purely on simulations. Real world experiments on the other hand throw a
completely different challenge for researchers as variables such as radio interference,
weather, terrain, node mobility, etc. can immensely alter the results of studies based on
simulations.
Of the few real world implementations for MANET, one of the most productive
ones has been performed by researchers at CMU [5,6,7]. They have constructed and
tested a tes tbed comprising of 2 stationary nodes, 5 mobile nodes that drive around the
test site and a mobile node that enters and leaves the test site with a speed of about 25 to
40 km/h. The separation for the fixed nodes is a distance of about 700m. In contrast to
their test bed, our goal is to monitor the MANET performance with the inclusion of
UAVs in the network. The purpose of our experimentation is described by two broad
scenarios. In the first scenario, an ad hoc network of ground nodes is disconnected
because of distance and/or terrain. The UAV, with a better view of the nodes, maintains
connectivity as an ad hoc relay. In the second scenario, a small UAV because of power
and payload constraints has limited communication range which in turn may limit
operational range. Ad hoc relaying between multiple UAVs extends communication
range.
Another paper that describes a real world implementation of MANET is A Largescale Test bed for Reproducible Ad hoc Protocol Evaluations [10]. The paper introduces
APE or an Ad hoc Protocol Evaluation test bed that the researchers have created to
perform reproducible experiments to assess several different routing protocols in a real
world scenario. Their test bed included three interconnected buildings on campus that
allowed a distance of 174 meters for tests, and up to 37 nodes.
These papers and many other that describe work related to simulated ad hoc
networks discuss various aspects of MANETs. Some are aimed at introducing new
wireless protocols, while some have developed test beds to test and compare various ad
hoc protocols under different physical and simulated conditions. What makes AUGNet so
special is that it is a real world implementation of an ad hoc network that comprises of
fixed, mobile, and most importantly aerial nodes. Our goal is to study the effects of the
inclusion of UAVs in the network and how they can be used to further increase the
network efficiency on the whole.
4
THE AUGNET TESTBED:
To better describe our test bed, we have broken it down to four areas which include
the physical infrastructure, radio hardware and software, UAV platform and control, and
data collection and monitoring.
4.1
Physical Infrastructure
The physical infrastructure includes the test site, buildings, power, network
connectivity, and UAV flight takeoff and landing facilities. Our test site is the Table
Mountain Radio Quiet Zone (NRQZ) which is located 15km north of the University of
Colorado’s Boulder campus. The dimensions of the site are 2.5km by 3.5km area with a
north south alignment and with a site area of approximately 7 sq. km. Most of the site is
flat and level with steep sides dropping to the level of the surrounding plain. The top is
unobstructed for long distance communication while conversely the radio range can be
easily limited by placing the radio below the top on the steep sides. An aerial photograph
of the site is shown in Figure 1. A map of the site showing facilities is in Figure 2.
Figure 1: Aerial photograph of the Table Mountain test site looking west.
Figure 2: Table Mountain facilities
The test site buildings are connected via a fiber optic network and the site itself has
a T1 backhaul to the Internet. The UAV airfield has been prepared at the north side of the
site. It is approximately 100m long and 12m wide with a NNE by SSW alignment.
4.2
Radio Hardware and Software:
Three areas of the radio hardware and software are described: the Mesh Network
Radio, MNR packages, and base performance.
4.2.1 Mesh network radio (MNR)
The mesh network radio consists of ad hoc routing software and a hardware
platform to support the software. The software is Linux based and runs on any Linux
platform with sufficient storage and memory. The software is designed to work with any
Ethernet network interface supported by Linux. Most network interfaces are abstracted to
appear as an Ethernet interface to the operating system. Therefore the software works
with wired Ethernet and wireless 802.11, 802.11a, 802.11b, and 802.11g. Only 802.11b
has been tested since this is the focus of the test bed research.
The hardware can be any platform that supports Linux and 802.11b. A specific
hardware platform was developed for this testing. The platform uses commercial off the
shelf components and is shown in Figure . It is comprised of a Soekris single board
computer, Orinoco 802.11b card, a Fidelity-Comtech bidirectional amplifier with up to
1W output, and a GPS. The GPS is included for monitoring purposes and is not necessary
for the ad hoc routing protocol. The GPS updates a file with time and position estimates
at 1 sec intervals. The components can fit in a box of dimensions 16cm x 16cm x 5cm.
The MNR can operate in the temperature range from 0-60 °C with no cooling fan. Power
is 12V and provided either directly or via power over Ethernet.
All software resides on a compact flash (CF) card. Each CF card is identical except
for an IP address that defines a node. When powered on, the operating system boots from
an image stored on the CF, a RAM file system is created, and the software operates only
in RAM. This mode of operation was chosen for three reasons. First, the CF card has a
limited read/write life and this minimizes accesses to the card. Second, the hardware may
be turned off at any time due to power interruptions and this avoids file system corruption
due to partial file writes. Third, CF cards can be exchanged as needed between nodes
since no operational state is stored on the card.
Figure 3: Mesh Network Radio (MNR) (left). MNR mounted in environmental
enclosure for vehicle or fixed ground mounting (center). MNR mounted in UAV
(foreground right)
4.2.2 MNR packages:
The MNR can be mounted in different packages. For fixed ground sites and on
vehicles, the MNR is mounted in the environmental enclosure shown in Figure 3. For the
UAV, the MNR components are mounted on rails inside the UAV as shown in Figure 3.
In addition to these versions, the ad hoc software can be operated on standard laptops
running Linux. Such a node is used for the test bed gateway described in a later section.
The software has also been ported to an IPaq PDA. For monitoring purposes, these nodes
need to be provided with either a GPS or the ability to report their current location if they
are at a fixed position.
4.3
UAV Platform and Control
The UAV airframe is built from a carbon fiber monocoque construction. The
wingspan is 2.4m and maximum takeoff weight is 15kg. The maximum payload is 5kg.
Cruising speed is 90km/hr. The maximum measured speed is 190km/hr. The current fuel
tank supports 45min of operation. Payload and avionics power is provided by batteries.
The plane is controlled using standard 72MHz analog radio control (R/C). This limits
operation to within a kilometer of the airfield.1 Development of the plane continues on
several fronts. The fuel tank will be increased for longer flights. A plane waypoint flying
system is being installed based on a 900MHz digital packet control link. This enables
flight paths to be programmed and modified via computer taking the burden of direct
control away from a human operator. This will enable longer range and longer duration
flights.
Interference tests were performed between the two R/C radios and the 802.11
payload. No interaction between the 2.4GHz and other bands could be measured with a
spectrum analyzer or with a 802.11b WLAN monitoring tool. Direct measures of packet
loss and throughput were unaffected by the coexistence the systems. The 72MHz R/C is
affected when its antenna is placed within 10cm of the single board computer. The R/C
antenna was mounted well away from the computer.
4.4
Data Collection and Monitoring
Monitoring packets, like data packets from the nodes are directed to any one of the
fixed nodes in the network. The fixed nodes in turn send these packets to the test bed
gateway and on to the monitoring server via the internet. Remote monitoring capabilities
allow us to remotely access data obtained, and to monitor the test site and activity on a
real time basis.
The monitoring server provides several functions. Firstly it serves to archive the
monitoring data. This archived data provides additional functions like producing derived
statistics. For example, the number of packets lost on a link is determined by observing
the number of packets sent from the transmit radio on the link and observing the number
successfully received at the receiver radio of the link. The difference between these
values represents the loss rate. The monitor server also provides a playback function
where different aspects of an experiment can be replayed by remote users for analysis
purposes. The data is stored in a mySQL based database at the monitoring server. The
playback feature uses a Java interface that enables users to analyze the data from a web
brower from anywhere on the internet. A screen of the interface is shown below.
5
NETWORKING DESCRIPTION:
Our wireless communications network comprises of fixed, mobile, and aerial nodes
that relay packets to a Remote monitoring server via fixed gateways on the test bed.
Local area connectivity is made available with units supporting 802.11b wireless
transmission and mesh network routing. Each node in the network runs the Dynamic
Source Routing (DSR) protocol at the network layer while communicating with other
nodes via 802.11b at layer 2. We have implemented the DSR protocol ourselves using the
Click Modular Router [Ref]. The reason we chose DSR over other ad-hoc protocols like
AODV or TORA is because it’s routing is On-demand. In On-demand source routing
which unlike Table driven routing where there are periodic updates sent from
neighboring routers and not a continuous -------- like protocols such as ------------ where
routing messages are exchanged periodically between neighboring nodes. In On-demand
routing a traffic source only seeks a route to a destination when it has to send data. The
biggest advantage of this feature is that the nodes do not waste bandwidth in an effort to
1
The ability of the operator to safely perceive plane orientation rather than the radio limits the range.
Operator fatigue is a factor that limits flight duration.
establish routes they may never use. In DSR, when a node needs to send a packet, it
initiates a route request process among all the nodes in the network to establish a route.
DSR additionally uses source routing whereby each packet contains the ip addresses of
the nodes a packet must pass through. Each node or MNR collects the network data that
includes packets sent, packets received and packets discarded due to congestion induced
overflow. This data is tagged with the GPS time and location and sent out at 10 second
intervals. This interval ensures that the impact of the monitoring packets on other
network traffic is minimal.
6
SIMULATIONS OF EXPECTED PERFORMANCE
7
ANECDOTAL RESULTS:
This section details some of the major problems/issues we faced while conducting
various tests.
Issues:
1. To obtain a 5-hop scenario for the throughput and latency tests it was important to
locate spots on the test site to place the ground nodes such that none of the
intermediate nodes between the source and destination MNRs were skipped and
hence a 5 hop route was guaranteed. The issue was of even greater relevance as we
had to ensure that we were maintaining the maximal throughput possible while
scouting for the ideal node locations.
2. Power sources for nodes: During early experimentation, the power was drawn off car
batteries to the nodes via a power over ethernet unit . The biggest drawback with this
setup was that every time the ignition for the car was turned off the node turned off as
well. A more viable option that we have now adopted is Lead acid batteries that run
off 12volts. They provide better stability, are easily portable, and ideal for our tests as
the nodes can now be placed at isolated spots that may not be reachable by cars.
3. Another problem with using cars as node mounts as we noticed was that they offer a
large metallic area that results in greater RF reflection/refraction that in turn reduces
the RF potential of the radios considerably.
4. In terms of data collection and the monitoring software the biggest challenge was the
design of the 2 layer monitoring system. The problem required a method for storing
data and method for displaying near-real-time results. The solution involved the
integration of two independent layers, namely data storage and real-time database
querying. Java was used in the first layer to parse UDP data packets and store
relevant information in a relational database when testing was taking place. Java
applet and jsp/servlet technologies were used in the monitoring layer to take
advantage of common web GUI features in addition to the JDBC interface with
mySQL, allowing users to easily pull near-real-time test data from the database and
display updates in a graphical and intuitive manner.
7.1
far
Test Bed Measurements:
This section details the measurements obtained from tests conducted at the site so
7.2
Baseline Ground Network:
There are several factors that were measured to assess the MNR performance and
are listed below. These are completed in the lab or at the test site as appropriate in order
to establish expected network behavior.
Measures of Performance
Data Throughput
Latency (communication delay)
Jitter (delay variation)
Communication Availability
Remote Connectivity
Hardware Reliability
Range
The range between ground nodes was measured using one MNR placed 2m above
the ground and the Berkeley Variatronics Yellow Jacket held 2m above the ground. The
MNR was placed at building B9 and was transmitting short ping packets to a node inside
of B9. The Yellow Jacket was placed at various places around the test site and measured
the received signal strength of packets. The experiment was repeated with the MNR
placed on the ground so that its antenna was 0.2m above the ground (but, the Yellow
Jacket still held 2m above the ground). The results are shown in Error! Reference
source not found.. From this data we see that the range is about 400m when the node is
placed on the ground and 1200m when placed 2m above the ground.
Five nodes named MNR-1 through MNR-5 were setup on the testbed, mounted 2
meters above the ground. For the range tests, the nodes had to be setup at locations where
the throughput for a given link would be ideal while ensuring that the packets from a
source to a given destination were routed by each of the intermediate nodes and none of
them were skipped. From our range tests we have observed two types of ideal ranges. In
terms of basic connectivity, tested by sending “pings” to each of the nodes in the network
the Connectivity Range was found to be 1800 meters between two consecutive nodes.
However in terms of throughput, this range was far large and resulted in frequent stalls in
the file transfer. The distance between the nodes was thus decreased till a maximal
Throughput range of 1200m was observed.
The following graph describes the results obtained for the Performance vs. number
of hops test. Data was transferred between nodes MNR-1 and MNR-2, then between
between MNR-1 and MNR-3, and so on to the five hop transfer between MNR-1 and
MNR-6. The throughput and latency measurements as a function of the number of hops
were thus recorded, the results of the same are listed in table ___
Graph.
Measure
Boot time (sec)
Network formation time (sec)
Result
92 secs
552 secs
Remarks
MNR/Laptop
6 MNRs at test site
Ground level range:
Connectivity
Throughput
Air-to-ground range
Delay for a ground network (ms)
1 Hop
2 Hop
3 Hop
4 Hop
5 Hop
throughput (bps) for the ground network:
1 Hop
2 Hops
3 Hops
4 Hops
5 Hops
throughput (pkt/sec) with UAV as one of
the nodes in the network
1 Hop
2 Hops
3 Hops
4 Hops
8
CONCLUSION:
9
ACKNOWLEDGEMENTS
1800 m
1200 m
3900 m
11.279 ms
18.742 ms
22.512 ms
36.237 ms
40.891 ms
At test site, MNRs placed 6ft.
from the ground
at test site 2m and 150m
above the ground
At test site in a 6 node
scenario.
At test site as a function of
hops
1494 Kbps
521 Kbps
280 Kbps
198 Kbps
131 Kbps
At test site with 4 MNRs and
a UAV node
1246 Kbps
534 Kbps
234 Kbps
193 Kbps
This work has been supported by the L3-Comcept Corporation. The UAV planes
were built and operated by the faculty and students of the Aerospace Engineering
Sciences Department including Brian Argrow, Tom Bateman, Cory Dixon, Jack Elston,
and Phillip Nies. Thanks to ITS/NTIA for granting permission to use the Table Mountain
National Radio Quiet Zone and to John Ewan from ITS for supporting our operations on
the testbed.
10 REFERENCES:
1. Timothy X Brown, Brian Argrow, Cory Dixon, Sheetalkumar Doshi, Philip Nies, “Ad
hoc UAV-Ground Network(AUGNet)
2. Timothy X Brown, Brian Argrow, Sheetalkumar Doshi, Gerald Jones, “Test Bed for a
Wireless Network on Small UAVs”
3. Henrik Lundgren, David Lundberg, Johan Nielsen, Erik Nordstrom, Christian
Tschudin, “A Large-scale Testbed for Reproducible Ad hoc Protocol Evaluations”.
4. Sagar Sanghani, Timothy X Brown, Shweta Bhandare, Sheetalkumar Doshi,
“EWANT: The Emulated Wireless Ad Hoc Network Testbed”
5. David A. Maltz, Josh Broch, and David B. Johnson, “Experiences Designing and
Building a Multi-Hop Wireless Ad Hoc Network Testbed”
6. David A. Maltz, Josh Broch, and David B. Johnson. Quantitative Lessons From a
Full-Scale Multi-Hop Wireless Ad Hoc Network Testbed
7. Qifa Ke, David A. Maltz, and David B. Johnson. Emulation of Multi-Hop Wireless
Ad Hoc Networks.
8. Kaixin Xu, Xiaoyan Hong, Mario Gerla, Henry Ly, Daniel Lihui Gu, “LANDMARK
Routing In Large Wireless Battlefield Networks Using UAVs”
9. Yongguang Zhang, Wei Li, “ An integrated Environment for Testing Mobile Ad-Hoc
Networks”
10. James T. Kaba, Douglas R. Raichle, “Testbed on a Desktop: Strategies and
Techniques to Support Multi-hop MANET Routing Protocol Development”
11. Zhigang Jin, Bing Liang, Yantai Shu, Oliver W.W.Yang, “Designing and
Implementing A Mobile Ad hoc Network Testbed”