Download Presentation

Recent Developments in
Logistical Networking
Micah Beck, Assoc. Prof. & Director
Logistical Computing &
Internetworking (LoCI) Lab
Computer Science Department
[email protected]
APAN Meeting, Honolulu, HI
Jan 29, 2004
What is Logistical Networking
» A scalable mechanism for deploying shared
storage resources throughout the network
» An general store-and-forward overlay networking
infrastructure
» A way to break long transfers into segments and
employ heterogeneous network technologies
» P2P storage and content delivery that doesn’t
using endpoint storage or bandwidth
The Network Storage Stack
• Our adaption of the network stack
architecture for storage
• Like the IP Stack
Applications
Logistical File System
Logistical Tools
L-Bone
• Each level encapsulates details from the
lower levels, while still exposing details
to higher levels
exNode
IBP
Local Access
Physical
IBP: The Internet Backplane Protocol
» Storage provisioned on community “depots”
» Very primitive service (similar to block service, but
more sharable)
• Goal is to be a common platform (exposed)
• Also part of end-to-end design
» Best effort service – no heroic measures
• Availability, reliability, security, performance
» Allocations are time-limited!
• Leases are respected, can be renewed
• Permanent storage is to strong to share!
Internet Backplane Protocol (IBP)
allocate!
Na
depot
capability
store!
Nw
data
depot
load!
Nr
depot
Software & Infrastructure
»
»
»
»
»
Tools open source, multiplatform
IBP Depot (server) and C client library
exNode and end-to-end services library
Logistical Backbone server (LDAP-based)
Linux/C is primary development platform
• Java client
» Command-line utilities, GUI
» Public L-Bone deployment
• 25 TB deployed globally
L-Bone: August 2003 (20TB)
Current Issues in IBP
1. Separate TCP connection for each IBP operation
• High latencies between dependent operations
2. Redundant DNS resolution for repetitive IBP ops
• High latency, load on DNS server
3. Client access control on IBP by IP address only
• Limiting, easily spoofed
4. All allocations on one depot managed uniformly
• Lack of control, inefficient
5. No computational capabilities at the depot
• Limiting, inefficient
New IBP Depot 1.4 Release (Feb 2004)
1. Persistent TCP connections optionally maintained
between client and depot to reduce overhead
2. Optional DNS caching eliminates redundant
lookups in some OS implementations
3. Secure IBP variant supported with X.509
certificate authentication
4. Multiple resources (File Sys, RAM) supported on
one depot with optimized “cut-through” transfers
Yong Zhen & Huadong Liu
The Network Functional Unit
5. Network Functional Unit computational operations
supported on data stored in depot allocations
• See “An End-to-End Approach to Globally
Scalable Programmable Networking” by Beck,
Moore & Plank in Future Directions in Network
Architecture, A SIGCOMM 2003 Workshop
Jeremy Millar, Alex Bassi & Yong Zheng
Current Issues with LoRS
1. Replication only means of increasing data
availability
• Inefficient in use of storage
2. Point-to-multipoint data transfer not supported
• Inefficient, cumbersome
3. Cumbersome client installation procedures
• Users prefer Web-like ease-of-use
Upcoming Tool Releases
1. Reed-Solomon encoding as an end-to-end service
in Logistical Runtime System (James S. Plank &
Stephen Soltesz)
• Reduces use of storage and controls F.T.
2. Asynchronous multicast engine with replaceable
policy module (Ying Ding)
• Generic framework for experimentation
3. Java Webstart implementation of LoRS Download
• Close to zero installation, speed matches C
Conclusions
» The LoCI Lab continues to improve and expand
the Logistical Networking toolset
» We are eager for more application groups and
more collaborative developers
» We are excited about the work you will hear about
today and hope to hear about your new projects in
the future
» Contact me at [email protected]
» http://loci.cs.utk.edu