Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Distributed firewall wikipedia , lookup
Internet protocol suite wikipedia , lookup
Piggybacking (Internet access) wikipedia , lookup
Cracking of wireless networks wikipedia , lookup
Computer network wikipedia , lookup
Recursive InterNetwork Architecture (RINA) wikipedia , lookup
Zero-configuration networking wikipedia , lookup
Network tap wikipedia , lookup
List of wireless community networks by region wikipedia , lookup
Logistical Networking as an Advanced Engineering Testbed Micah Beck, Assoc. Prof. & Director Logistical Computing & Internetworking (LoCI) Lab [email protected] Internet2 Spring Member Meeting April 10, 2003 Panel Participants • Gabriella Paolini, GARR (by phone) Topic: IPv6 • Geoff Hayward, Yotta Yotta Topic: Wide Area Storage Networking • Jim Ferguson, NCSA Topic: Web 100 / TCP Tuning Logistical Networking Research at UTK University of Tennessee • Micah Beck • James S. Plank • Jack Dongarra University of California, Santa Barbara • Rich Wolski Funding • Dept. of Energy SciDAC • National Science Foundation ANIR • UT Center for Info Technology Research What is Logistical Networking • A scalable mechanism for deploying shared storage resources throughout the network • A general store-and-forward overlay networking infrastructure • A way to break transfers into segments and employ heterogeneous network technologies on the pieces Why “Logistical Networking” • Analogy to logistics in distribution of industrial and military personnel & materiel • Fast highways alone are not enough Goods are also stored in warehouses for transfer or local distribution • Fast networks alone are not enough Data must be stored in buffers/files for transfer or local distribution The Network Storage Stack Applications • Our adaption of the network stack architecture for storage • Like the IP Stack • Each level encapsulates details from the lower levels, while still exposing details to higher levels Logistical File System Logistical Tools L-Bone exNode IBP Local Access Physical IBP: The Internet Backplane Protocol • Storage provisioned on community “depots” • Very primitive service (similar to block service, but more sharable) • Goal is to be a common platform (exposed) • Also part of end-to-end design • Best effort service – no heroic measures • Availability, reliability, security, performance • Allocations are time-limited! • Leases are respected, can be renewed • Permanent storage is to strong to share! Models of Sharing: Logistical Networking Moderately valuable resources • Storage, server cycles Sharing enabled by relative plenty Internet-like policies • Loose access control • No per-use accounting Primary design goal: scalability • Application autonomy • Resource transparency Burdens of scalability • The End-to-End Principles • Weak operation semantics • Vulnerability to Denial of Service Data Movers • Module implementing standard point-tomultipoint transfer between IBP allocations • Uniform API allows independence from the underlying data transfer protocol • Not every DM can apply to every transfer • Caller responsible for determining validity • Current options: Multi-TCP, Multi-UDP (reliable), UDP Multicast (unreliable) The Network Storage Stack LoRS: The Logistical Runtime System: Aggregation tools and methodologies The L-bone: Resource Discovery & Proximity queries The exNode: A data structure for aggregation IBP: Allocating and managing network storage (like a network malloc) The Logistical Backbone (L-Bone) • LDAP-based storage resource discovery. • Query by capacity, network proximity, geographical proximity, stability, etc. • Periodic monitoring of depots. • 10 Terabytes of shared storage. (with plans to scale to a petabyte...) L-Bone: January 2003 Current Storage Capacity: 10 TB The Network Storage Stack LoRS: The Logistical Runtime System: Aggregation tools and methodologies The L-bone: Resource Discovery & Proximity queries The exNode: A data structure for aggregation IBP: Allocating and managing network storage (like a network malloc) The exNode • The Network “File Descriptor • XML-based data structure/serialization • Map byte-extents to IBP buffers (or other allocations). • Allows for replication, flexible decomposition of data. • Also allows for error-correction/checksums • Arbitrary metadata. ExNode vs inode IBP Allocations the network local system capabilities exNode inode user kernel block addresses disk blocks The Network Storage Stack LoRS: The Logistical Runtime System: Aggregation tools and methodologies The L-bone: Resource Discovery & Proximity queries The exNode: A data structure for aggregation IBP: Allocating and managing network storage (like a network malloc) Logistical Runtime System Basic Primitives: • Upload, Download, Augment, Refresh End-to-end Services • Checksums, Encryption, Compression Download Movie Multithreaded Transfers Routed/Multipath Point-to-Multipoint Heterogeneous Multicast Caching/Staging Further Advanced Capabilities • IBP over IPv6 • Dual stack depot • Specialized DataMovers • UDP (SABUL, Tsunami) • Fiber Channel over IP • Non-standard TCP Stacks • Web 100 • Future: FAST? Panel Participants • Gabriella Paolini, GARR (by phone) Topic: IPv6 • Geoff Hayward, Yotta Yotta Topic: Wide Area Storage Networking • Jim Ferguson, NCSA Topic: Web 100 / TCP Tuning Conclusions • IBP is a global testbed for advanced network engineering • Transfer rates routinely exceed 100Mbps • New Data Movers under development can reach current applications • Dedicated depots can support global testbed for kernel modificaitons