* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Networking
Survey
Document related concepts
Net neutrality law wikipedia , lookup
Deep packet inspection wikipedia , lookup
Distributed firewall wikipedia , lookup
Cracking of wireless networks wikipedia , lookup
Internet protocol suite wikipedia , lookup
Computer network wikipedia , lookup
Network tap wikipedia , lookup
Zero-configuration networking wikipedia , lookup
Piggybacking (Internet access) wikipedia , lookup
Quality of service wikipedia , lookup
UniPro protocol stack wikipedia , lookup
Recursive InterNetwork Architecture (RINA) wikipedia , lookup
Transcript
Scalable Sharing of Network Storage Micah Beck, Research Assoc. Professor Director, Logistical Computing & Internetworking (LoCI) Lab Computer Science Department University of Tennessee CERN May 30, 2002 A Broad View of Networking • A network is a facility for sharing a pool of resources within a community. • Sharing is enabled by – – – – Common goals Standard protocols Low allocation overhead Mutual protection IP Routing as Scalable Sharing of Bandwidth • The Internet is a collection of links connected by routers • Internet Protocol (IP) datagrams flow from sender to reciever along some path of links • The choice of paths is made by routers according to topological considerations (tempered by policy) Wide Area Network Services Are Like the Network Itself • • • • • Intermittently inaccessible Vulnerable to partition Prone to corruption in transit Unpredictable latencies/jitter End-to-End: Never require a network service to be bigger, better or more complex than wide area access allows Why Is It Called “End-to-End”? • If the end-points requires a service that is stronger than the network can provide, they must implement it on top of the network. • The fundamental tools are – Aggregation (retransmission, fragmentation) – Timers – Protocol state maintained at the end-points How Can Storage Be Shared Scalably? • Don’t require it to be – – – – Unbounded in size Unbounded in duration Perfectly available Perfectly reliable • Best effort network storage • Implement these properties end-to-end Sharing Storage Enables Advanced Network Apps • • • • • • Caching Staging (Replication) Localization (Rendezvous) Distribution (Multicast) Spooling (E-Mail) Temporary State Management (Checkpointing) “Logistical” Networking • Analogy to military or industrial logistics • Distributing goods requires not only highways but also warehouses (depots) • Data transmission is the highway • Storage servers are the depots • This is not Storage Networking • Enabling new users and applications Dimensions in Communication Routing (spatial) Storage (temporal) Logistical Networking Logistical Networking • Principle Investigators – Micah Beck – James S. Plank • Graduate Students – Erica Fuentes – Xiang Li • Funding – DoE SciDAC – NSF Next Gen SW – NSF Internet Tech. • Research Staff – – – – – – – – – Scott Atchley Alex Bassi Ying Ding Hunter Hagewood Sharmila Kancherla Jeremy Millar Terry Moore Stephen Soltesz Yong Zheng Internet Backplane Protocol (IBP) • Servers that make allocation of primitive “byte arrays” available to clients • Byte arrays are not blocks (more abstract) – Network capabilities (primitive security) – Variable extents • Byte arrays are not files (weaker semantics) – Size & duration are limited – “Volatile” allocations – Best effort reliability and availability – No directory structure, accounting – No caching, replication • An end-to-end approach to network storage IBP Software Structure • IBP Depots (servers) are daemons that serve local storage to IBP clients. • IBP clients only link to a protocol library. • Clients talk to depots using TCP/IP. – Alternatives are being investigated • Design is for high-performance/scalability. Allocation Attributes • Permanent vs. Time-Limited • Volatile vs. Stable • Read/Write semantics: – – – – Byte Array Pipe Circular Queue Completely Destructive Write Impressions of Networking for High Energy Physics (?) • Reliable throughput is the limiting factor • Modes of sharing can be “modified” • The use of specialized data movement middleware is acceptable • The data transfer network has very few nodes and small user communities • Little patience for fancy architectural ideas IBP Enables Heterogeneity • The tyranny of “TCP-friendly” IP – Datagrams and usage policy must be homogeneous end-to-end – Non-standard IP policies in the middle can leak out onto public networks • Good fences make good neighbors • IBP can terminate a flow and regenerate it in different network, with localized control Example:Tsunami • Uses very aggressive UDP transfer with TCP back-link for control • Runs using Abilene less-than-best-effort “Scavenger Service” • Tsunami must not be routed onto a link where Scavenger Service is not supported! • Solution: do not route the packets, require explicit buffering and retransmission Scheduling Control Over Sharing Without IP QoS • Consider a high speed link that operates at 10Gb/s, connected to two types of links: – 100Mb/s and 1Gb/s • Jobs are 1TB transfers – ~20 hours at 100Mb/s, ~2 hours at 1Gb/s • Suspending a slow job to allow a fast job through delays it by only 10% Scheduled Sharing of Network Links Is In Your Future! • Ignoring buffer control won’t make the need for scheduled sharing go away • When was more control over time & space ever a bad thing? • Are we revisiting CERNET/UUCP/BITNET/DECnet? A Short History of Internet Protocols at CERN Ben Segal / CERN PDP-NS April, 1995 • IBP is designed according to end-to-end principles (see SIGCOMM ’02 paper) The Alternative to Interoperability In Buffering • High performance data transfer is an isolated service, with “job scheduling” and “data staging” like at computation centers • As more high performance links are built, concatenation is based on “gateways” rather than a scalable routing infrastructure • Should networking solutions be built without the benefit of network architecture? Application Areas • • • • • • Source routing Bandwidth adaption Reducing (BWdelay) Reliable multicast Content Distribution Remote access to structured data • Managing computation state (NetSolve caching) • Temporary storage • Very large data sets • Collaborative computing & visualization Building on IBP • Many applications assume file semantics – Unbounded size & duration – High reliability & availability – Caching & replication • In a layered architecture, these are implemented through aggregation and additional intelligence at the next level The Network Storage Stack Applications Logistical Runtime System Logistical Tools L-Bone exNode IBP (Internet Backplane Protocol) Access (OS) Physical ExNode vs inode IBP Allocations the network local system capabilities exNode inode user kernel block addresses disk blocks ExNode Mobility XML Serialization The exNode serialization is a portable soft link Sameple exNodes Services: Fragmentation, Replication 1 2 3 Building a Logistical Operating Environment • Resource Discovery – Logistical Backbone registry (LBone: Plank) – Can also use DNS, could use MDS • Programming Tools – exNode library, primitive commands – policy modules (replication, fragmentation) – cool applications (SC demo: IBPster MP3) Related Work • Peer-to-Peer Storage – Napster, Gnutella, etc – OceanStore (UC Berkeley) • Storage Area Networking – Fiber Channel, Ultraband, iSCSI – Network Attached Storage (CMU) • Overlay Networking – Caching – Multicast – Data Grids • Multimedia Protocols – Content Delivery – Streaming – Video on Demand Logistical Networking: E2E Storage and Bandwidth • • • • • • • Complex paths and buffers are exposed Assumptions are appropriately weak Resource management is passive Spatial and temporal control Applic. autonomy, network transparency Resources in the hands of end users Futures: more control, greater scope Logistical Computing and Internetworking Laboratory http://loci.cs.utk.edu • Documents and software downloads