Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Distributed firewall wikipedia , lookup
Piggybacking (Internet access) wikipedia , lookup
Cracking of wireless networks wikipedia , lookup
Computer network wikipedia , lookup
Recursive InterNetwork Architecture (RINA) wikipedia , lookup
Zero-configuration networking wikipedia , lookup
Network tap wikipedia , lookup
List of wireless community networks by region wikipedia , lookup
Introduction to Logistical Networking Micah Beck, Assoc. Prof. & Director Logistical Computing & Internetworking (LoCI) Lab [email protected] APAN Advanced Networking Conf Aug 28, 2003 Logistical Networking Research at UTK University of Tennessee • Micah Beck • James S. Plank • Jack Dongarra University of California, Santa Barbara • Rich Wolski US Govt. Funding • Dept. of Energy SciDAC • National Science Foundation ANIR Industry Collab. • Yotta Yotta Internet2 What is Logistical Networking? • A scalable mechanism for deploying shared storage resources throughout the network • A general store-and-forward overlay networking infrastructure • A way to break transfers into segments and employ heterogeneous network technologies on the pieces Why “Logistical Networking” • Analogy to logistics in distribution of industrial and military personnel & materiel • Fast highways alone are not enough Goods are also stored in warehouses for transfer or local distribution • Fast networks alone are not enough Data must be stored in buffers/files for transfer or local distribution The Network Storage Stack • Our adaption of the network stack architecture for storage • Like the IP Stack • Each level encapsulates details from the lower levels, while still exposing details to higher levels Applications Logistical File System Logistical Tools L-Bone exNode IBP Local Access Physical IBP: The Internet Backplane Protocol • Storage provisioned on community “depots” • Very primitive service (similar to block service, but more sharable) • Goal is to be a common platform (exposed) • Also part of end-to-end design • Best effort service – no heroic measures • Availability, reliability, security, performance • Allocations are time-limited! • Leases are respected, can be renewed • Permanent storage is to strong to share! Data Movers • Module implementing standard point-tomultipoint transfer between IBP allocations • Uniform API allows independence from the underlying data transfer protocol • Not every DM can apply to every transfer • Caller responsible for determining validity • Current options: Multi-TCP, Multi-SABUL (reliable), UDP Multicast (unreliable) The Network Storage Stack LoRS: The Logistical Runtime System: Aggregation tools and methodologies The L-bone: Resource Discovery & Proximity queries The exNode: A data structure for aggregation IBP: Allocating and managing network storage (like a network malloc) The Logistical Backbone (L-Bone) • LDAP-based storage resource discovery. • Query by capacity, network proximity, geographical proximity, stability, etc. • Periodic monitoring of depots. • 20 Terabytes of shared storage. (with plans to scale to a petabyte...) L-Bone: August 2003 Current Storage Capacity: 20 TB The Network Storage Stack LoRS: The Logistical Runtime System: Aggregation tools and methodologies The L-bone: Resource Discovery & Proximity queries The exNode: A data structure for aggregation IBP: Allocating and managing network storage (like a network malloc) The exNode • • • • • • The Network “File Descriptor XML-based data structure/serialization Map byte-extents to IBP buffers (or other allocations). Allows for replication, flexible decomposition of data. Also allows for error-correction/checksums Arbitrary metadata. ExNode vs inode IBP Allocations the network local system capabilities exNode inode user kernel block addresses disk blocks The Network Storage Stack LoRS: The Logistical Runtime System: Aggregation tools and methodologies The L-bone: Resource Discovery & Proximity queries The exNode: A data structure for aggregation IBP: Allocating and managing network storage (like a network malloc) Logistical Runtime System Basic Primitives: • Upload, Download, Augment, Refresh End-to-end Services • Checksums, Encryption, Compression Multithreaded Transfers Routed/Multipath Point-to-Multipoint Heterogeneous Multicast Caching/Staging Latency hiding through aggressive prestaging Remote database Wide Area Network LAN Depot Interactive Browser Further Advanced Capabilities • IBP over IPv6 • Specialized DataMovers • Aggressive UDP (SABUL) • Added features coming soon… • Pipelining, Authentication, RAM resources • Disk-to-disk transfer (Fiber Channel over IP) • Limited computation on the depot Architecture Publications An End-to-End Approach to Globally Scalable Network Storage Micah Beck, Terry Moore and James S. Plank ACM SIGCOMM 2002 Conference, Pittsburgh, PA, USA, August 19-23 An End-to-End Approach to Globally Scalable Programmable Networking Micah Beck, Terry Moore and James S. Plank Workshop on Future Directions in Network Architecture, ACM SIGCOMM 2003, Karlsruhe, Germany, August 27 Application Publications An Exposed Approach to Reliable Multicast in Heterogeneous Logistical Networks Micah Beck, Ying Ding, Erika Fuentes and Sharmila Kancherla Workshop on Grids and Advanced Networks, Tokyo, Japan, May 12-15, 2003 Remote Visualization by Browsing Image Based Databases with Logistical Networking Jin Ding, Jian Huang, Micah Beck, Shaotao Liu, Terry Moore, and Stephen Soltesz To appear in SC 2003, Phoenix, AZ, November, 2003 Conclusions • IBP supports a global 20 TB testbed for distributed applications • Transfer rates routinely exceed 100Mbps • New Data Movers under development • More advanced features coming soon • Server runs on Linux/Unix/OS X platforms • IBP Client & LoRS also on Win32, Java http://loci.cs.utk.edu [email protected]