Download spr09_sumans_candidacy_4

Ubiquitous Communication Model
for Different Network Scenarios
Suman Srinivasan
PhD Candidacy Talk
Feb 23, 2009
Advisor: Dr. Henning Schulzrinne
1
Introduction


Many different types of networks and connection
scenarios exist
I will cover the following











Traditional networking ←→ Sockets
Remote execution ←→ RPC, RMI, CORBA
Parallel computing ←→ MPI, PVM, Shared memory
Grid computing ←→ Javaspaces, Linda
Web services ←→ XML-RPC, SOAP, REST
Overlay networks ←→ DHTs, Bittorrent, JXTA
Opportunistic networks ←→ BonAHA, LightPeers, etc
Active Networks ←→ ANTS, Janos, etc.
… mainly from the programmer’s perspective
… and try to answer the following question
Is there a common communication model, or
programming abstraction, for all of the above
scenarios?
2
Comparison metrics

API for programmers


Distribution of messages




1-to-1, 1-to-N
Processing location


e.g., send/recv (or) find/get/set (or) remote
procedures
Where processed: node, network (router)
Delay
Destination known?
Message functionality

Active (execution) or passive (data)
3
Evaluation metrics
API
model
Dist
Location
Delay
Dest IP Message
known function
End-to-end
sockets
Message
(send/recv)
1-to-1 /
1-to-N
Endpoints /
Endpoints +
routers
Network
Yes
Passive
Remote
execution
RPC
1-to-1
Remote node
Network +
processing
Yes
Passive
(func+args)
Parallel
computing
Message;
Process
1-to-N
Endpoints
(clustered)
Network/bus
No
Passive
Grid
computing
Message
(active obj.)
1-to-N
Endpoints
(distributed)
Network +
processing
No
Active
Web
services
RPC;
get/set
1-to-1
Remote node
Network +
processing
Yes
Passive
(XML cmds)
Overlay
find/get/set
/rm
1-to-N
(DHT)
Endpoints
(overlay)
Setup time +
query process
+ churn
No
Passive
Opportunis
tic
networks
find/get/set
/rm
1-to-N
Endpoints
Delay on
wire/wireless
No
Passive
Active
Networks
Routing
1-to-N
(router)
Routers
Processing
time
No
Active
4
Stack diagram
Opportunistic
N/Ws
MPI
PVM
Shared
mem.
Parallel computing
Multicast
JXTA
/Jini
SOAP
Grid
computing
DHTs (and)
Overlay N/Ws
RPC
REST
XMLRPC
XML over
HTTP
Sockets
5
Type of API

Send/Recv (or) Read/Write


Remote Procedures


Remote execution (RPC, RMI, CORBA), Web
services (SOAP, XML-RPC)
Get/Set


Sockets, Parallel computing (MPI), Grid
computing (Javaspaces, Linda)
Web services (REST), Overlay networks
(DHTs), Opportunistic networks
Process execution

Parallel computing (PVM)
6
END-TO-END NETWORKING
 Sockets: Simple, end-to-end communication
 socket()
 connect() / listen()
 send() / recv()
 close()
 [RFC147] and [RFC2553]
7
Evaluation metrics
API
model
Dist
Location
Delay
Dest IP Message
known function
End-to-end
sockets
Message
(send/recv)
1-to-1 /
1-to-N
Endpoints /
Endpoints +
routers
Network
Yes
Passive
Remote
execution
RPC
1-to-1
Remote node
Network +
processing
Yes
Passive
(func+args)
Parallel
computing
Message;
Process
1-to-N
Endpoints
(clustered)
Network/bus
No
Passive
Grid
computing
Message
(active obj.)
1-to-N
Endpoints
(distributed)
Network +
processing
No
Active
Web
services
RPC;
get/set
1-to-1
Remote node
Network +
processing
Yes
Passive
(XML cmds)
Overlay
find/get/set
/rm
1-to-N
(DHT)
Endpoints
(overlay)
Setup time +
query process
+ churn
No
Passive
Opportunis
tic
networks
find/get/set
/rm
1-to-N
Endpoints
Delay on
wire/wireless
No
Passive
Active
Networks
Routing
1-to-N
(router)
Routers
Processing
time
No
Active
8
REMOTE EXECUTION
 Local function calls
 Executed on remote machine
 Results returned to caller over
network
 Communication using sockets
 Technologies
 Remote Procedure Calls
 Remote Method Invocation
 OMG CORBA
http://msdn.microsoft.com/en-us/library/aa373935.aspx
9
RPC

Remote Procedure Call

RFC 707 (1976): A High-Level
Framework for Network-Based
Resource Sharing
“… outlines an alternative to the approach
that ARPANET system builders have been
taking...”
 Argues that the "command/response
discipline" remains "crude“
 [rfc707]

10
RPC

Early versions


Xerox Courier (1981), Sun's ONC RPC
(1988, 1995)
Microsoft provides RPC APIs


DCOM (1996) built on top of Microsoft
RPC
Other implementations for Windows
11
RPC and RMI

Sun’s RPC


Data serialized using IETF XDR format,
via TCP or UDP
Port mapper



Maps RPC program numbers to port
numbers on server
[RFC1057] and [RFC1831]
Java RMI [javarmi]

Very similar to RPC, can work over
CORBA
12
OMG CORBA


Common Object Requesting Broker
Architecture
Interface Definition Language
(IDL) to specify interfaces


Program communicates with ORB
(Object Request Broker), which
interacts with other apps on network
Transport protocol: General Inter-ORB
protocol (GIOP): IIOP, SSLIOP, HTIOP
13
OMG CORBA
http://en.wikipedia.org/wiki/CORBA
14
Evaluation metrics
API
model
Dist
Location
Delay
Dest IP Message
known function
End-to-end
sockets
Message
(send/recv)
1-to-1 /
1-to-N
Endpoints /
Endpoints +
routers
Network
Yes
Passive
Remote
execution
RPC
1-to-1
Remote node
Network +
processing
Yes
Passive
(func+args)
Parallel
computing
Message;
Process
1-to-N
Endpoints
(clustered)
Network/bus
No
Passive
Grid
computing
Message
(active obj.)
1-to-N
Endpoints
(distributed)
Network +
processing
No
Active
Web
services
RPC;
get/set
1-to-1
Remote node
Network +
processing
Yes
Passive
(XML cmds)
Overlay
find/get/set
/rm
1-to-N
(DHT)
Endpoints
(overlay)
Setup time +
query process
+ churn
No
Passive
Opportunis
tic
networks
find/get/set
/rm
1-to-N
Endpoints
Delay on
wire/wireless
No
Passive
Active
Networks
Routing
1-to-N
(router)
Routers
Processing
time
No
Active
15
PARALLEL COMPUTING
 Deals with execution
or sharing of
data across multiple
processors
 Tightly coupled,
physically close nodes
https://computing.llnl.gov/tutorials/parallel_comp/
 Message Passing Interface (MPI)
 Parallel Virtual Machine (PVM)
 Shared memory
16
MPI and PVM

Message Passing Interface [mpi96]




Language independent communications
protocol
Communicators connect groups of
processes
mpi_send() and mpi_recv()
Parallel Virtual Machine [pvm90]


Software library: allows network of
computers to work as though it were one
distributed parallel processor
pvm_spawn() and pvm_notify()
17
[sharedmem07]
Shared Memory Model

Memory that can be accessed by
multiple programs


On same processor or multiple
processors
OpenMP: API using preprocessor
directives
1.
2.
3.
#pragma omp parallel for
for (i=0;i<N;i++)
a[i]= 2*I;
18
Evaluation metrics
API
model
Dist
Location
Delay
Dest IP Message
known function
End-to-end
sockets
Message
(send/recv)
1-to-1 /
1-to-N
Endpoints /
Endpoints +
routers
Network
Yes
Passive
Remote
execution
RPC
1-to-1
Remote node
Network +
processing
Yes
Passive
(func+args)
Parallel
computing
Message;
Process
1-to-N
Endpoints
(clustered)
Network/bus
No
Passive
Grid
computing
Message
(active obj.)
1-to-N
Endpoints
(distributed)
Network +
processing
No
Active
Web
services
RPC;
get/set
1-to-1
Remote node
Network +
processing
Yes
Passive
(XML cmds)
Overlay
find/get/set
/rm
1-to-N
(DHT)
Endpoints
(overlay)
Setup time +
query process
+ churn
No
Passive
Opportunis
tic
networks
find/get/set
/rm
1-to-N
Endpoints
Delay on
wire/wireless
No
Passive
Active
Networks
Routing
1-to-N
(router)
Routers
Processing
time
No
Active
19
GRID COMPUTING
 Several nodes working on the same
problem at the same time
 Program divided into portions
 Distributed for processing
 More loosely coupled, heterogeneous
and dispersed than parallel computing
 E.g.: Berkley’s BOINC, SETI@home,
Folding@home
 Javaspaces
20
[linda85] + [javaspaces]
Tuple Spaces


Stores distributed system state
Javaspaces: Use
Javaspace.write() and
Javaspace.read() to put and get
objects and results


Javaspaces modeled
after Linda (1985)
Master-worker
pattern

Tuple space
http://java.sun.com/developer/technicalArticles/tools/JavaSpaces/
21
Javaspaces code
1.
2.
3.
4.
5.
6.
7.
public static void main(String argv[]) {
try {
MessageEntry msg = new MessageEntry();
msg.content = "Hello there";
System.out.println("Searching for a JavaSpace...");
Lookup finder = new Lookup(JavaSpace.class);
JavaSpace space = (JavaSpace) finder.getService();
8.
9.
10.
space.write(msg, null, 60*60*1000);
MessageEntry template = new MessageEntry();
11.
12.
13.
14.
15.
16.
MessageEntry result = (MessageEntry) space.read(template, null,
Long.MAX_VALUE);
System.out.println("The message read is: "+result.content);
} catch(Exception e) {
}
}
22
Evaluation metrics
API
model
Dist
Location
Delay
Dest IP Message
known function
End-to-end
sockets
Message
(send/recv)
1-to-1 /
1-to-N
Endpoints /
Endpoints +
routers
Network
Yes
Passive
Remote
execution
RPC
1-to-1
Remote node
Network +
processing
Yes
Passive
(func+args)
Parallel
computing
Message;
Process
1-to-N
Endpoints
(clustered)
Network/bus
No
Passive
Grid
computing
Message
(active obj.)
1-to-N
Endpoints
(distributed)
Network +
processing
No
Active
Web
services
RPC;
get/set
1-to-1
Remote node
Network +
processing
Yes
Passive
(XML cmds)
Overlay
find/get/set
/rm
1-to-N
(DHT)
Endpoints
(overlay)
Setup time +
query process
+ churn
No
Passive
Opportunis
tic
networks
find/get/set
/rm
1-to-N
Endpoints
Delay on
wire/wireless
No
Passive
Active
Networks
Routing
1-to-N
(router)
Routers
Processing
time
No
Active
23
WEB SERVICES
 Remote procedure calls
 But using text, not binary objects
 Often using XML over HTTP
 To build distributed systems
over traditional web
protocols (XML/HTTP)
 Web-based consumer apps
 Enterprise apps
http://developer.garmin.com/wp-content/uploads/2007/05/picture-5.png
 XML-RPC
 SOAP
 REST
24
Web Services

XML-RPC




Example from: http://www.tutorialspoint.com/xmlrpc/xml_rpc_examples.htm
Function call







Simple protocol, created in 1998
Data types, commands
<?xml version="1.0" encoding="ISO-8859-1"?>
<methodCall>
<methodName>sample.sum</methodName>
<params> <param> <value><int>17</int></value> </param>
<param> <value><int>13</int></value> </param> </params>
</methodCall>
Response




<?xml version="1.0" encoding="ISO-8859-1"?>
<methodResponse>
<params><param> <value><int>30</int></value> </param></params>
</methodResponse>
25
SOAP

Simple Object Access Protocol


W3C standard
Uses XML for messages

HTTP and HTTPS used for message
negotiation and transmission
Can use SMTP as well
 But HTTP/HTTPS more preferred


SOAP slower than competing RPC
technologies because of XML
26
SOAP

<env:Envelope xmlns:env="http://www.w3.org/2003/05/soap-envelope">
<env:Header>
<n:alertcontrol xmlns:n="http://example.org/alertcontrol">
<n:priority>1</n:priority>
<n:expires>2001-06-22T14:00:00-05:00</n:expires>
</n:alertcontrol>
</env:Header>
<env:Body>
<m:alert xmlns:m="http://example.org/alert">
<m:msg>Pick up Mary at school at 2pm</m:msg> </m:alert>
</env:Body>
</env:Envelope>

From http://www.w3.org/TR/soap12-part1/











27
REST

Representational State Transfer


Resources with a identifier



2000 PhD Thesis by Roy Fielding
Exchange representations of resources
Connectors (clients, servers, caches,
tunnels)
Instead of calling getData(1), you
would get content from a URN:

http://somesite.com/data/1
28
REST


From http://developer.yahoo.com/maps/rest/V1/geocode.html
Call a “REST-like” service:


http://local.yahooapis.com/MapsService/V1/geocode?appid=YD9G7bey8_JXxQP6rxl.fBFGgCdNjoDMACQA-&street=701+First+Ave&city=Sunnyvale&state=CA
Response from Yahoo:














<?xml version="1.0" encoding="UTF-8"?>
<ResultSet xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="urn:yahoo:maps"
xsi:schemaLocation="urn:yahoo:maps
http://local.yahooapis.com/MapsService/V1/GeocodeResponse.xsd">
<Result precision="address">
<Latitude>37.416384</Latitude>
<Longitude>-122.024853</Longitude>
<Address>701 FIRST AVE</Address>
<City>SUNNYVALE</City>
<State>CA</State>
<Zip>94089-1019</Zip>
<Country>US</Country>
</Result>
</ResultSet>
29
Evaluation metrics
API
model
Dist
Location
Delay
Dest IP Message
known function
End-to-end
sockets
Message
(send/recv)
1-to-1 /
1-to-N
Endpoints /
Endpoints +
routers
Network
Yes
Passive
Remote
execution
RPC
1-to-1
Remote node
Network +
processing
Yes
Passive
(func+args)
Parallel
computing
Message;
Process
1-to-N
Endpoints
(clustered)
Network/bus
No
Passive
Grid
computing
Message
(active obj.)
1-to-N
Endpoints
(distributed)
Network +
processing
No
Active
Web
services
RPC;
get/set
1-to-1
Remote node
Network +
processing
Yes
Passive
(XML cmds)
Overlay
find/get/set
/rm
1-to-N
(DHT)
Endpoints
(overlay)
Setup time +
query process
+ churn
No
Passive
Opportunis
tic
networks
find/get/set
/rm
1-to-N
Endpoints
Delay on
wire/wireless
No
Passive
Active
Networks
Routing
1-to-N
(router)
Routers
Processing
time
No
Active
30
OVERLAY NETWORKS
 Built on top of always-connected
networks (and others)
 Form a sub-network with specific
functionality
 With routing protocol
 Distributed key-value pairs
 Structured overlays
 Tapestry, CAN, Chord
 Constrained, structured graph topology
 Exact-match queries
 Unstructured overlays
 Organized in random graph topology
 Keyword queries
 E.g.: Gnutella
http://www.dynamicobjects.com/papers/w4spot.pdf
31
DHT service model

API Example: [OpenDHT]




get(), put(), rm()
Behind the scenes – [BambooDHT]
Running on PlanetLab
Other DHTs use similar APIs

Can be extended to usage outside of
DHTs as well
32
DHTs

Well-known DHTs





Chord [Chord01]
CAN [Can01]
Kademilia [Kademilia02]
Pastry [Pastry01]
Partitioned keyspace


Distributed amongst nodes
Deals with churn
33
Evaluation metrics
API
model
Dist
Location
Delay
Dest IP Message
known function
End-to-end
sockets
Message
(send/recv)
1-to-1 /
1-to-N
Endpoints /
Endpoints +
routers
Network
Yes
Passive
Remote
execution
RPC
1-to-1
Remote node
Network +
processing
Yes
Passive
(func+args)
Parallel
computing
Message;
Process
1-to-N
Endpoints
(clustered)
Network/bus
No
Passive
Grid
computing
Message
(active obj.)
1-to-N
Endpoints
(distributed)
Network +
processing
No
Active
Web
services
RPC;
get/set
1-to-1
Remote node
Network +
processing
Yes
Passive
(XML cmds)
Overlay
find/get/set
/rm
1-to-N
(DHT)
Endpoints
(overlay)
Setup time +
query process
+ churn
No
Passive
Opportunis
tic
networks
find/get/set
/rm
1-to-N
Endpoints
Delay on
wire/wireless
No
Passive
Active
Networks
Routing
1-to-N
(router)
Routers
Processing
time
No
Active
34
OPPORTUNISTIC NETWORKS
 Mobile nodes; highly mobile networks
 No infrastructure
 OLPC; mesh networks
 Can be used for ad-hoc, community
applications
 Traditional network apps have to be adapted for
oppnets
 [Opp06]
35
BonAHA


Framework for
applications
running on
opportunistic
networks
Updates on
network



serviceUpdated()
serviceExited()
Node properties


key11 = value11
key12 = value12
key13 = value13
key14 = value14
[1] node1.register()
[2] node1.get(key13)
Node 1
[3] data =
node1.fileGet(
value13);
key21 = value21
key22 = value22
key23 = value23
key24 = value24
Node 2
[bonaha09]
node.get()
node.set()
36
Other OppNet Frameworks

For opportunistic and mobile
networks


Proem (2001)
Peer2Me (2004)


Market Contact Protocol (MCP) (2008)


File sharing on BlueTooth
Oppnet commerce
LightPeers (2007)
37
LightPeers

Similar model to BonAHA



Code
1.
2.
3.
4.

“Application”: Each application has its own GUID
that identifies it
“Session”: A group of nodes registered as running
the application
Application app = new Application(appid);
lpconn = new Connection(app);
ses = lpconn.CreateSession();
List<Session> sessions =
lpconn.GetSessionList();
[Lightpeers07]
38
Evaluation metrics
API
model
Dist
Location
Delay
Dest IP Message
known function
End-to-end
sockets
Message
(send/recv)
1-to-1 /
1-to-N
Endpoints /
Endpoints +
routers
Network
Yes
Passive
Remote
execution
RPC
1-to-1
Remote node
Network +
processing
Yes
Passive
(func+args)
Parallel
computing
Message;
Process
1-to-N
Endpoints
(clustered)
Network/bus
No
Passive
Grid
computing
Message
(active obj.)
1-to-N
Endpoints
(distributed)
Network +
processing
No
Active
Web
services
RPC;
get/set
1-to-1
Remote node
Network +
processing
Yes
Passive
(XML cmds)
Overlay
find/get/set
/rm
1-to-N
(DHT)
Endpoints
(overlay)
Setup time +
query process
+ churn
No
Passive
Opportunis
tic
networks
find/get/set
/rm
1-to-N
Endpoints
Delay on
wire/wireless
No
Passive
Active
Networks
Routing
1-to-N
(router)
Routers
Processing
time
No
Active
39
ACTIVE NETWORKS
 Allows “injection” of customized active code
into network core
 ANTS, JanOS, Open Multi-Service Router,
Netscript, Switchware
 All circa 1996-2001
 Why important?
 Example of frameworks for executable/active code that
change network operations
40
ANTS (Active Node Transfer Sys)

New network protocols
automatically deployed
using mobile code




Packets are replaced by
capsules
Capsules contain instructions with
executable code
Routers (active nodes) execute code
[Ants98]
41
Janos

Janos is a NodeOS (Moab) with:


Execution Environment (EE) : similar to
virtual machine
Active Applications (AA), which are
injected into the network
Written to run on ANTSR runtime
 Which runs on
top of Janos
virtual machine


[Janos01]
42
Switchware

Active packets, active extensions,
active router infrastructure




CAML-based
Packets contain PLAN
(Programming
Language for Active
Networks) code
SANE (Secure Active
Network Environment)
[Switchware98]
43
Evaluation metrics
API
model
Dist
Location
Delay
Dest IP Message
known function
End-to-end
sockets
Message
(send/recv)
1-to-1 /
1-to-N
Endpoints /
Endpoints +
routers
Network
Yes
Passive
Remote
execution
RPC
1-to-1
Remote node
Network +
processing
Yes
Passive
(func+args)
Parallel
computing
Message;
Process
1-to-N
Endpoints
(clustered)
Network/bus
No
Passive
Grid
computing
Message
(active obj.)
1-to-N
Endpoints
(distributed)
Network +
processing
No
Active
Web
services
RPC;
get/set
1-to-1
Remote node
Network +
processing
Yes
Passive
(XML cmds)
Overlay
find/get/set
/rm
1-to-N
(DHT)
Endpoints
(overlay)
Setup time +
query process
+ churn
No
Passive
Opportunis
tic
networks
find/get/set
/rm
1-to-N
Endpoints
Delay on
wire/wireless
No
Passive
Active
Networks
Routing
1-to-N
(router)
Routers
Processing
time
No
Active
44
Conclusion

Covered the following network topologies, and the
associated programming models:










Traditional networking ←→ Sockets
Remote execution ←→ RPC, RMI, CORBA
Parallel computing ←→ MPI, PVM, Shared memory
Grid computing ←→ Javaspaces, Linda
Web services ←→ XML-RPC, SOAP, REST
Overlay networks ←→ DHTs, Bittorrent, JXTA
Opportunistic networks ←→ BonAHA, LightPeers, etc
Active Networks ←→ ANTS, Janos, etc.
Question: Is there a common communication
model, or abstraction for all of these scenarios?
Answer: Hard to see common ground

Perhaps among some sub-groups, e.g., grid and
parallel computing, overlay and opportunistic networks
45
Backup Slides
46
RPC
http://msdn.microsoft.com/en-us/library/aa373937%28VS.85%29.aspx
47
Java RMI command

java -cp
/home/ann/src:/home/ann/public_h
tml/classes/compute.jar Djava.rmi.server.codebase=http://z
aphod/~ann/classes/compute.jar Djava.rmi.server.hostname=zaphod
.east.sun.com Djava.security.policy=server.policy
engine.ComputeEngine
48
MPI Sample code
1.
2.
3.
int main(int argc, char *argv[])
{
MPI_Status stat;
4.
5.
6.
7.
8.
// Init
MPI_Init(&argc,&argv); /* all MPI programs start with MPI_Init; all 'N' processes
exist thereafter */
MPI_Comm_size(MPI_COMM_WORLD,&numprocs); /* find out how big the SPMD
world is */
MPI_Comm_rank(MPI_COMM_WORLD,&myid); /* and this processes' rank is */
// Send any number of processses
MPI_Send(buff, BUFSIZE, MPI_CHAR, i, TAG, MPI_COMM_WORLD);
9.
10.
// Receive data
MPI_Recv(buff, BUFSIZE, MPI_CHAR, i, TAG, MPI_COMM_WORLD, &stat);
11.
12.
13.
// Weak synchronization
MPI_Finalize();
14.
15.
16.
}
49
PVM sample code
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
int main(int argc, char* argv[])
{
mytid = pvm_mytid(); /* find out my task id number */
/* find my parent's task id number */
myparent = pvm_parent();
/* if i don't have a parent then i am the parent */
if (myparent == PvmNoParent) {
info = pvm_spawn(argv[0], (char**)0, PvmTaskDebug, (char*)0,
ntask, child);
/* wait for the notification */
info = pvm_recv(-1, TASKDIED);
pvm_exit();
return 0;
}
/* i'm a child */
pvm_exit();
return 0;
}
50
ANTS example
51
Netscript

Agents dispatched to
remote systems


Virtual Network
Engines (VNE) interconnected by
Virtual Links (VL)


Uses SOS to manage
delegation agents
Packet arrives at VNE; headers used to
pass it on executing agent
[Netscript96]
52
JXTA


Framework for building P2P apps
Peers create a overlay network


Different protocols


Peer Resolver, Peer Information,
Rendezvous, Peer Membership, Pipe
Binding, Endpoint Routing
Peer types:


Nodes identified by 160-bit SHA-1 URN
Edge, Super, Rendezvous, Relay
[jxta]
53