Download Adopting Ideas from Interplanetary Networking for Sensor

Adopting Ideas from Interplanetary Networking
for Sensor Network Applications
Andrew Parker, UCLA
Scott Burleigh, JPL
Richard Guy, UCLA
Deborah Estrin, UCLA
11/12/04
CENS Seminar
1
Interplanetary Network Vision
• Picture of sensor networks in space
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
* V. Cerf, InterPlanetary Internet, 2004
11/12/04
CENS Seminar
2
Why We Care
11/12/04
CENS Seminar
3
Outline
•
•
•
•
•
•
11/12/04
Mexico Seismic Array Deployment: Set Up
Interplanetary Networking: Set Up & Relevance
Delay Tolerant Networking (DTN): Architecture
DTN vs. Email
Mexico Deployment: DTN Approach
Modular DTN Architecture for Sensor Networks
CENS Seminar
4
Mexico Deployment
• People: Paul Davis, Deborah
Estrin, Richard Guy, Martin Lukac,
John Wallace, Monica Kohler,
Ramesh Govindan, Igor Stubailo,
Allen Husker, Katie Mika, John
Propst, Sam Irvine, Jeremy Elson,
et al.
•
•
•
•
Seismic array of 50 nodes
5 km apart
Spanning 250 km
Through jungle, mountain,
and urban environments
• Targeting end of Q1 2005
11/12/04
CENS Seminar
5
Data Story
•
CENS Data
Communications
Controller
•
Q330 24 bit Digitizer
Guralp Seismometer
•
•
– Satellite uplink
(expensive)
– Data mule (graduate
student)
Stargate
•
11/12/04
CENS Seminar
6
Every bit is important.
Can’t lose data.
24 bits / sample, 100
samples / sec, 3
channels = 900
bytes/sec, 3.24 MB/hour,
1.2-1.8 MB/s
compressed, per Node
1 GB Flash will hold 20
days of compressed data
Conventional methods of
retrieval
Is there a better way?
A Better Way: Networking the Array
• Goal: Multihop data from the edge into Mexico City
• Mexico deployment is different from typical sensor network
applications
– Every bit of data is important
– Radio connectivity is directional
• Due to distance, power, and regulations, we’re using directional
antennas
– Routing is over a linear topology
– Radio communication is not the greatest power consumer
11/12/04
CENS Seminar
7
Braided String
•
•
Introduces some path redundancy
802.11b
– 200 mw radio (SMC PCMCIA card)
•
Yagi antenna
–
–
–
–
–
•
20 dbi
30 degree spread
Tested with 2-way splitter
5 - 10 km in LA area
4 Mb/s - 150 kb/s
Parabolic antenna
–
–
–
–
–
11/12/04
24 dbi
5 - 10 degree spread
4 way splitter
Tested to ~ 25 km
Closer to 1 Mb/s
CENS Seminar
8
Routing is Hard to Do Right
• Possible approach: Static Routes - topology is linear and nodes
are immobile
• Problem: Still as brittle as a single string -- can’t back track
around breaks
• Possible approach: Use AODV, DSR, Roofnet, etc.
• Problem: Flapping links will make establishing end-to-end routes
nearly impossible
• What to do, what to do…?
11/12/04
CENS Seminar
9
Outline
•
•
•
•
•
•
11/12/04
Mexico Seismic Array Deployment: Set Up
Interplanetary Networking: Set Up & Relevance
Delay Tolerant Networking (DTN): Architecture
DTN vs. Email
Mexico Deployment: DTN Approach
Modular DTN Architecture for Sensor Networks
CENS Seminar
10
Interplanetary Networking: Motes in Space
• Huge transmission
delays
– Several seconds to
the Moon
– 5 - 20 minutes to
Mars
– 1 hour to Jupiter
– 7 hours to Pluto
• Lossy links
• Long disconnects
– Sometimes
predictable
11/12/04
CENS Seminar
11
Mars Communication
http://www.astrosurf.com/lombry/qsl-mars-communication3.htm
11/12/04
CENS Seminar
12
Mars Connectivity
QuickTime™ and a
Cinepak decompressor
are needed to see this picture.
11/12/04
CENS Seminar
13
Oops!
11/12/04
CENS Seminar
14
Properties of Interplanetary Networking
•Links are lossy and high delay
•End to end paths
unreliable
•Interactive / chatty
protocols break
•Need to communicate
across varying network
technologies, including
non-IP networks
–Very high round trip times,
–Even for single hop
•Links are often disrupted
–Node mobility
–Node powered down
•Asymmetric links
•Asymmetric node capabilities
11/12/04
CENS Seminar
15
Delay Tolerant Network Architecture
• Messaging service (Like Email)
– Insulates applications from
network behavior
– Asynchronous, deferred
transmission
– Non-interactive end-to-end
You’v
eGot
DTN!
11/12/04
CENS Seminar
16
Delay Tolerant Networking
• Bundles are Basic Unit of Transport
– Written as files onto persistent store
– Resilient across server / node restarts
• Custody Transfer of Bundles
– Represents a QOS agreement: custodian tries real hard to transfer
custody and not delete until this has happened
– Conceptually moves the “end” of an end to end transaction
– Custody is asserted, rather than given.
• Overlay routing of Bundles among Custodians
– Custodians may be separated by intermediate nodes.
11/12/04
CENS Seminar
17
Delay Tolerant, Not Run-Over-by-a-Bus Tolerant
VS
• Bad things can still happen with Custody Transfer
• Brings up the question of trust. When should a node trust
another more than it itself?
• Risk vs. Resource trade-off
11/12/04
CENS Seminar
18
Bundle Agents and Clients
•
Client nodes register with Bundle Agents to send and receive Bundles
on their behalf
– Email analogy: Pop or IMAP client connecting to a server
– Based on the destination name, delivery is made to the corresponding
Bundle Agent.
– At this point it’s considered to be done. Again, similar to Email.
•
Naming scheme allows for late binding and separation of names from
nodes
– Determination of which node(s) receive the message may be deferred
•
Connects different domains via gateways
– Late binding names allow different domains to communicate (through
gateways).
– Connects loosely coupled “Internets”
– Official DTN specification uses Tuples:
• (Interdomain Label, Opaque Intradomain Label)
11/12/04
CENS Seminar
19
Outline
•
•
•
•
•
•
11/12/04
Mexico Seismic Array Deployment: Set Up
Interplanetary Networking: Set Up & Relevance
Delay Tolerant Networking (DTN): Architecture
DTN vs. Email
Mexico Deployment: DTN Approach
Modular DTN Architecture for Sensor Networks
CENS Seminar
20
DTN vs. Email
DTN? Email?
What’s the difference?
Professor Culler
11/12/04
CENS Seminar
21
DTN vs. Email
Biggist Difference:
DTN Overlay
DTN is able to make progress
towards the destination, even when
no contemporaneous route exists
Cool
Professor Culler
11/12/04
CENS Seminar
22
Outline
•
•
•
•
•
•
11/12/04
Mexico Seismic Array Deployment: Set Up
Interplanetary Networking: Set Up & Relevance
Delay Tolerant Networking (DTN): Architecture
DTN vs. Email
Mexico Deployment: DTN Approach
Modular DTN Architecture for Sensor Networks
CENS Seminar
23
Mexico Deployment: Routing
• DTN has the “Tuple” address: (Interdomain, Intradomain)
– Doesn’t really apply to Mexico
– Single domain
• Hybrid Routing Approach
– You know where you want to get to, but not how
– Overlay static on top of mesh routing
• Using Roofnet
– link-quality aware
– minmizes number of transmissions
– Use mesh routing to tell you the next hop towards the furthest
reachable downstream candidate
11/12/04
CENS Seminar
24
Hybrid Examples
•
•
•
•
11/12/04
CENS Seminar
Normal case
Simple break
Partition
Reconnect
25
Hybrid Examples
•
•
•
•
11/12/04
CENS Seminar
Normal case
Simple break
Partition
Reconnect
26
Hybrid Examples
•
•
•
•
11/12/04
CENS Seminar
Normal case
Simple break
Partition
Reconnect
27
Hybrid Examples
•
•
•
•
11/12/04
CENS Seminar
Normal case
Simple break
Partition
Reconnect
28
Hybrid Examples
•
•
•
•
11/12/04
CENS Seminar
Normal case
Simple break
Partition
Reconnect
29
Mexico Deployment: Handling Data
• DTN suggests exchanging Bundles (Files)
• Divide data into one-hour segments
• Augment with meta-data
– To, From, Data, Size, etc.
•
•
•
•
11/12/04
1.2 - 1.8 MB compressed
Store as files on disk (even on intermediate hops)
Easy to recover from server and node restarts
Human manageable
CENS Seminar
30
Node Software Architecture
Upstream
Nodes
Duiker
CENS NODE
Inbox
Drafts
Bundle
Bundle
Bundle
Data File
Data File
Data File
Bundle Receiver
Bundle Sender
Downstream
Node
11/12/04
Bundle
Forwarder
Outbox
Bundle
Bundle
Bundle
CENS Seminar
Seismic Data
(Q330)
31
Routing
Data
Managing Space
• Nodes are equipped with 2 - 4 GB of storage: 40 - 80 days of
data
• A Bundle is deleted when ACK is received from sink for locally
generated data
– ACK is just another bundle
– Application-level ACK
• When space gets low
– Node refuses to accept transient data (“route around me”). Better
than accept and drop.
– Delete transient data when space gets low
• No intermediate custody transfer
• This policy delays the deletion of original data the longest
11/12/04
CENS Seminar
32
Transmission Priority
• Transmission priority affects performance
– Want to avoid starvation
• Priority can be based on:
–
–
–
–
–
11/12/04
Age
Transient vs. Local
Number of hops traversed
Number of times sent
Some other measure of how hard it was to get this far
CENS Seminar
33
What about Off the Shelf DTN?
•
•
•
•
There is an official Bundle specification
There are 1 1/2 reference implementations of the Bundle specification
Why not use it?
Because…
– Only reason why Mars worked, and only way Mexico will work, is because
scenario specific information was used. You MUST do this.
– It’s impossible to do a good job in all scenarios
– The more generic you try to be, the more it looks like flooding.
– The more options and switches you support, the more brittle and complex it
becomes
• See Sendmail configuration file
•
11/12/04
How do we remain flexible and customizable, yet stay SIMPLE?
CENS Seminar
34
A Modular DTN Software Architecture
•
•
•
•
•
Like Click
Modular software IP router
Easy to plug modules together
Easy to create new modules
Results in an optimized router
// Declare three elements…
src :: FromDevice(eth0);
ctr :: Counter;
sink :: Discard;
// .. Connect them together
src -> ctr;
ctr -> sink;
11/12/04
CENS Seminar
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
35
Click’s Elegance
•
Click modules consume and
produce the same data structure
– IP packets
•
•
Same is nice and simple. Easy to
compose modules if they
input/output the same thing
IP is nice too (as opposed to
something obscure or too
generic)
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
– Strong foundation in
specifications
– Many compliant implementations
as examples
– People are already familiar with
certain features and behaviors of
IP routers
11/12/04
CENS Seminar
36
Click and DTN - High Level
Click
•
•
•
•
11/12/04
DTN
•
•
IP packets
IP specifications
Lots of existing IP routers
Lots of specific IP
features/behavior to choose
from
CENS Seminar
Bundles
Bundle specifications
–
•
•
Very few Bundle servers
Lots of Bundle features and behavior
to choose from
–
37
Experimental and evolving
In this area, Bundles have a richer
set of potential behavior than IP
Introducing this Approach to Sensor Networks
•
EmStar
–
–
–
–
–
–
Component based framework for sensor network applications (on Linux)
Lots of services and applications already existing
Simulation / Emulation / Deployed Modes
Heterogeneous applications (Stargates / Motes)
Debugging, monitoring, and visualization
Growing user-base
/dev/servicename
/dev/fusd
Transceiver
Mote
User
Transceiver
Mote
Server
Transceiver
Mote
Client
Transceiver
Mote
• UCLA, USC, MIT, Umass Amherst, Ohio State, etc.
• Intel Research, Xerox PARC, Microsoft Research
Emulation Array
HostMote Protocol
Kernel
MN
MN
MN
Node Node Node
001 002 003
kfusd.o
MN
Node
N
…
Emulation Mode
Robust multi-process,
microkernel architecture
11/12/04
CENS Seminar
Visualization Tools Simulation Framework
with real RF channels
38
Relationships Between EmStar Modules
Unclear
Example EmRun Configuration File
include link/link.run
process mdiff_test {
waitfor = mdiff;
type = once;
noclean;
cmd = "devel/microdiff/mdiff_test";
}
&link_udp(udp0);
&link_linkstats(udp0,ls0,show="leds:core");
&link_neighbors(ls0,show="hide");
process mdiff {
type = once;
noclean;
cmd = "devel/microdiff/mdiff --uses ls0";
waitfor = ls0;
}
process mdiff_filter {
waitfor = mdiff;
type = once;
noclean;
cmd = "devel/microdiff/mdiff_filter";
}
How to bring Click’s readability and ease-of-use to EmStar?
11/12/04
CENS Seminar
39
Bolt: Architecture Description Language for
EmStar
• Makes relationships between modules of an EmStar
application explicit
• Began as a class project with Eddie Kohler and Todd Millstein
last spring
• Implemented in CIL and Ocaml
11/12/04
CENS Seminar
40
Bolt Config.: White Board to Emacs Buffer
udp0 = udp() ;
ls0 = linkstatsd();
nd = neighbord();
mdiff = Mdiff();
Instantiating
Connecting
component
devices
Data Flow
components
mdiff_simple_app [packet_dev] <-> [app_packet_opts] mdiff;
mdiff_simple_filter [packet_dev] <-> [filter_packet_opts] mdiff;
mdiff [lu_opts] <----------------------> [lp_opts] ls0 [lu_opts] <-> [opts] udp0;
mdiff [sc_opts] <- [s_opts] nd [ls_opts] <-- [s_opts] ls0;
nd [lu_opts] <-> [lp_opts] ls0;
11/12/04
CENS Seminar
41
BUT WAIT, THERE’S MORE!…
11/12/04
CENS Seminar
42
Click Modules vs. EmStar Modules
•
•
•
•
Click modules are C++ classes
EmStar modules are separate processes
Click modules communicate via function calls
EmStar modules communicate by passing bits or text via device
files
• Since Click modules are compiled, they benefit from basic
compiler checks and optimizations. Click itself does higher level
checks and optimizations
• Wouldn’t it be nice if EmStar modules enjoyed similar
benefits?
11/12/04
CENS Seminar
43
Bolt: Compiler-like Analysis and Optimization for
EmStar
•
•
•
It Slices! It Dices!
It does static analysis and optimizations not otherwise possible
across EmStar modules
No user changes or annotation required of EmStar code
– Though it could help
•
Bolt statically infers type-safety violations between two functions in
different processes (EmStar modules)
– Despite type-obscuring function calls and heavy use of function pointers
•
•
11/12/04
Bolt uses subgraph isomorphism to identify and swap out
combinations of modules with more efficient combinations that
perform the same function
Analysis not completely sound nor complete, but hard given the
circumstances
CENS Seminar
44
Implementing DTN Using Bolt
•
•
DTN looks like any other set of EmStar modules
What exactly are the EmStar modules that make up the DTN suite?
– Click has benefited from the existence of numerous IP router
implementations
– DTN must build up experience from real deployments (Mexico and others)
– Identify and extract reusable DTN related components, for example:
•
•
•
•
Managing a large number of concurrent file transfers between a pair of nodes
Despite large delay and disconnections
Recover across restarts
LTP (S. Burleigh) & File Mover (A. Parker)
– Goal is not necessarily to be able to build a complete DTN solution for your
particular situation
– Rather, to reuse the ones that make sense, and build from there
11/12/04
CENS Seminar
45
Outline
•
•
•
•
•
•
11/12/04
Mexico Seismic Array Deployment: Set Up
Interplanetary Networking: Set Up & Relevance
Delay Tolerant Networking (DTN): Architecture
DTN vs. Email
Mexico Deployment: DTN Approach
Modular DTN Architecture for Sensor Networks
CENS Seminar
46
References
•
•
•
•
•
11/12/04
DTN: http://www.dtnrg.org
IPN: http://www.ipnsig.org
Click: http://www.pdos.lcs.mit.edu/click
EmStar: http://cvs.cens.ucla.edu/emstar
Bolt: http://lecs.cs.ucla.edu/~adparker/Bolt
CENS Seminar
47