Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
All-Optical Networks for Grids: Dream or Reality? Payam Torab Lambda Optical Systems Corporation September 28, 2005 www.lambdaopticalsystems.com Grids – Tflops vs. Tbps. Emergence of grids is the result of the synergism between communications and computing, just like cybernetic systems that came out of synergism between communications and control Role of the network in Grids: to provide throughput TeraGrid – Application-aware networks, or network-aware applications? – Network providing services, or network as a services? – Throughput is the theme unifying connectivity, delay and bandwidth NEESgrid Balanced growth of networking and computing results in Grids Clusters Computing power (Tflops) Grids North European Grid Internets Networking power (Tbps) Surfnet Enabling Data-Intensive Grid Applications with Advanced Optical Technologies - 9/28/2005 2 Need for High Throughput Throughput is a grid resource: Uniform grid growth requires growth in throughput Throughput growth requires improvement in bandwidth, delay and availability Examples of throughput requirements – GridFTP applications – Large Hadron Collider (LHC) at CERN Year Production Experimental 2001 2002 0.155 0.622 0.622-2.5 2.5 2003 2.5 10 DWDM; 1 + 10 GigE Integration 2005 10 2-4 X 10 Switch; Provisioning 2007 2-4 X 10 1st Gen. Grids 2009 ~10 X 10 or 1-2 X 40 ~5 X 40 or ~20 X 10 ~Terabit ~10 X 10; 40 Gbps ~5 X 40 or ~20-50 X 10 ~25 X 40 or ~100 X 10 2011 2013 ~MultiTbps Remarks SONET/SDH SONET/SDH DWDM; GigE Integ. PHENIX experiment – Used GridFTP to transfer 270 TB of data from Long Island, NY to Japan ESNET outage? Relativistic Heavy Ion Collider RHIC at Brookhaven: 600 Mbps peak 250 Mbps average Brookhaven National Lab Long Island, NY OC-48 link to ESNET 40 Gbps Switching 2nd Gen Grids Terabit Networks ~Fill One Fiber Source: Larry Smarr, “The Optiputer - Toward a Terabit LAN ,” The On*VECTOR Terabit LAN Workshop Hosted by Calit2, University of California, San Diego - January 2005 Transpacific 10 Gbps line to SINET in Japan Source: www.cerncourier.com/main/article/45/7/15 Enabling Data-Intensive Grid Applications with Advanced Optical Technologies - 9/28/2005 3 Photonic Switching: Key to End-to-End Transparency O-E-O Electrical Cross-Connect (EXC) ~O(102) wavelengths WDM and electrical switching O-E-O Photonic CrossConnect (PXC) Separate WDM and optical switching O-O-O ~O(102) wavelengths ~O(102) Gbps per wavelength WDM + Photonic switching Photonic CrossConnect (PXC) Integrated WDM and optical switching Full transparency – End-to-end transparency • Bitrate transparency (10 Gbps, 40 Gbps, …) • Payload transparency (SONET, SDH, Ethernet, …) – Transmission robustness • Simplification or even elimination of windowing • No packet loss due to congestion/buffer overrun • Simpler transport protocols, higher throughput From: “Development of a Large-scale 3D MEMS Optical Switch Module,” T. Yamamoto, J. Yamaguchi and R. Sawada, NTT Technical Review, Vol. 1, No. 7, Oct. 2003 Enabling Data-Intensive Grid Applications with Advanced Optical Technologies - 9/28/2005 4 Wavelength Switching Scalability Grid-scale applications will ultimately press even wavelength switching – Example: Year Require too many optical ports to provide nonblocking connectivity! Production Experimental 2001 2002 0.155 0.622 0.622-2.5 2.5 2003 2.5 10 DWDM; 1 + 10 GigE Integration 2005 10 2-4 X 10 Switch; Provisioning 2007 2-4 X 10 1st Gen. Grids 2009 ~10 X 10 or 1-2 X 40 ~5 X 40 or ~20 X 10 ~Terabit ~10 X 10; 40 Gbps ~5 X 40 or ~20-50 X 10 ~25 X 40 or ~100 X 10 2011 2013 ~MultiTbps PXC PXC PXC PXC Wavelength switching Remarks SONET/SDH 4 wavelengths over 4 hops 32 optical ports SONET/SDH DWDM; GigE Integ. 40 Gbps Switching PXC PXC PXC PXC Waveband switching 4 wavelengths over 4 hops 8 optical ports Waveband multiplexer Waveband demultiplexer 2nd Gen Grids Terabit Networks ~Fill One Fiber Source: Larry Smarr, “The Optiputer - Toward a Terabit LAN ,” The On*VECTOR Terabit LAN Workshop Hosted by Calit2, University of California, San Diego - January 2005 Similar to any other switching technology, aggregation is essential for scalability of wavelength switching – hence the emergence of transparent multigranular (wavelength and waveband) switching architectures From: “A Graph Model for Dynamic Waveband Switching in WDM Mesh Networks,” M. Li and B. Ramamurthy, IEEE ICC 2004, Vol. 3, June 2004, pp. 1821-1825. Enabling Data-Intensive Grid Applications with Advanced Optical Technologies - 9/28/2005 5 Waveband Switching Efficiency Switching efficiency (%) Waveband switching efficiency: Relative saving in the total number of optical ports in a network when waveband switching is used instead of wavelength switching n h 1 1 e 1 b nw h 1 bu 60 40 20 0 -20 -40 -60 -80 -100 4 nw = number of ports under wavelength switching Wavebandnb = number of ports under waveband switching Physical hops in switched waveband path (h) h = average number of physical hops in each waveband circuits (bu) 4 b = average number of wavelengths in a waveband Waveband-switching efficient region u = average waveband utilization (used wavelengths) 2 Waveband switching becomes only more efficient (more saving in optical ports) as more wavelength circuits are carried over longer paths Example: GridFTP using 4 parallel TCP streams over 4x40 Gbps circuits carried over 6 hops More than 0.1 Tbps throughput over 6 hops using only 30 ports Waveband-switched circuits (bu) 0 3 Increased waveband utilization 2 1 1 2 3 4 5 8 6 4 2 0 10 Waveband switching gets more efficient Increased waveband path length (hops) 6 7 8 9 10 Physical hops in waveband path (h) Enabling Data-Intensive Grid Applications with Advanced Optical Technologies - 9/28/2005 6 More on Waveband Switching Efficiency Example: WDM WAN 615 circuits@40Gbps ~ 2.5 Tbps ~6200 ports - wavelength switching ~5500 ports - waveband switching ~80 nodes, ~140 links This simple analysis does not consider the extra scalability from the increase in bitrate (160Gbps and beyond, OTDM). 30000 998 circuits@40Gbps ~ 40 Tbps ~11800 ports - wavelength switching ~9700 ports - waveband switching 2085 circuits@40Gbps ~ 85 Tbps ~28400 ports - wavelength switching ~21800 ports - waveband switching More to appear in: P. Torab and V. Hutcheon, “Waveband switching efficiency in all-optical networks: analysis and case study,” in preparation for OFC 2006. Waveband-switching 25000 20000 15000 Transmission breakthroughs Increase in throughput without increase in ports 10000 5000 0 20 40 60 80 Network throughput (Tbps) 100 Waveband-switched circuits (bu) Required optical ports Wavelength-switching 4 3 40 Tbps 2 1 1 Waveband switching gets more efficient Waveband-switching efficient region 80 Tbps 2.5 Tbps 2 3 4 5 6 7 8 9 10 Physical hops in waveband path (h) Enabling Data-Intensive Grid Applications with Advanced Optical Technologies - 9/28/2005 7 Hierarchical Transparent Switching Waveband switching adds another level of switching to the transparent switching hierarchy Multigranular switching Logical WDM topologies Node A Fiber Waveband Wavelength IP/TDM Node B Waveband XC Waveband XC 1 Waveband XC 2 h Waveband Multiplexer Waveband Demultiplexer Bandpath bp1 Wavelength XC Wavelength Interfaces Several physical hops are lumped into one logical WDM link, requiring switching only at the link endpoints Fast and still flexible dynamic wavelength service over reduced number of hops Wavelength XC Wavelength Interfaces h physical hops – one logical hop bp1 Node A Node B Two lightpaths with the same routes Node A Node B Waveband XC Waveband XC 1 Node C Waveband XC 2 h1 Waveband XC 1 Waveband XC 2 h2 Waveband Demultiplexer Waveband Multiplexer Bandpath bp1 Wavelength XC Wavelength Interfaces Bandpath bp2 Wavelength XC Wavelength Interfaces h1 physical hops – one logical hop bp1 Node A Payloadtransparent Switching lp1 Wavelength XC Wavelength Interfaces h2 physical hops – one logical hop bp2 Node B Node C lp2 Two lightpaths with partially overlapping routes Enabling Data-Intensive Grid Applications with Advanced Optical Technologies - 9/28/2005 8 Logical (Virtual) WDM Combined wavelength and waveband switching allows dynamic configuration of transparent optical topologies supporting dynamic lambdas (from connection ondemand to topology on-demand) Example: During the next 14 days, computing facility at site A, the storage center at site B, and the visualization room at site C will participate in an experiment that will require multiple dynamic lambdas (e.g., timescale in seconds) Logical WDM Topology Computing - A Computing - A Storage - B Waveband connections Visualization - C Storage - B Visualization - C Dynamic lambdas (fast setup and teardown) Enabling Data-Intensive Grid Applications with Advanced Optical Technologies - 9/28/2005 9 Lambda OpticalSystems Solutions Dedicated to transparent switching technology Addressing research community and carrier needs Deployed at U.S. Naval Research Lab (NRL) and Starlight LambdaNode 200 Transparent 64x64 full duplex ports GMPLS, CLI and web interface 5.25 inches tall LambdaNode 2000 Integrated WDM and photonic switching Multigranular switching for maximum scalability Provides waveband and wavelength switching GMPLS, CLI, TL1 and web interface Enabling Data-Intensive Grid Applications with Advanced Optical Technologies - 9/28/2005 10 NL 101, NL 103 Demos at iGrid 2005 AAA/DRAC AAA/DRAC AAA/DRAC VMT Controller 2 x GbE circuits HDXc Qwest / other wave service LambdaNode 200 CENIC VMT visualization host HDXc a **or other L2 switch 3 5 2 4 2 x GbE circuits 6 12/2 12/3 2/18 2/19 GbE OC192 STM64 y 2/13 VLAN 350 VLAN 350 iGRID B nud05 San Diego/UCSD (SAN) nud06 vangogh 5 Chicago/SL (CHI) vm vh x 4003(2) b 2/12 iGRID A OME E600 E1200 GbE OC192 STM64 **E600 1 HDXc vm / 2 vangogh 6 Amsterdam/NL (AMS) vm X / 2 X Enabling Data-Intensive Grid Applications with Advanced Optical Technologies - 9/28/2005 11 Control Plane: Enabler of Transient Services Grid’s balanced growth needs dynamic on-demand high network throughput What do we need to provide high throughput? 1. Dynamism: Make optimum use of all network resources for the tasks at hand • Example: If 1.0 Tbps throughput is needed between A and B for one hour, fill up the network with 25x40Gbps connections and kill them an hour later. 2. Availability: The ability to maintain high throughput through fast recovery • • Network failures do happen, therefore high bandwidth does not guarantee high throughput In a transient service environment protection is not as expensive – Telco thinking: 1+1 protection is expensive- I need to plan for twice the capacity, therefore I need to charge my customer twice as much (bronze service, silver service, platinum service, …) – Grid thinking: Provide as much protection that your schedule allows. The connections will not be there in an hour. The more network resources the more protected circuits. • (Dynamic) restoration can also add to reliability when (dedicated) protection is unavailable Key effort needed: Integrating traditional service levels (1+1 protection, 1:N protection, shared mesh restoration, …) into Grid services – – – Can a GridFTP application ask for transfer over 1+1 connection? Application Trade-off between replication/migration and network recovery intelligence (replication, Where does the optimal performance stand? migration) Network intelligence (protection, restoration) Enabling Data-Intensive Grid Applications with Advanced Optical Technologies - 9/28/2005 12 Generalized Multiprotocol Label Switching (GMPLS) IP-based control plane paradigm to control packet, time slot (TDM), wavelength, waveband and space (fiber) switching across multiple switching layers, and across multiple domains. Developed by IETF – CCAMP workgroup with liaison work with OIF and ITU-T Mature standard now (RFC 3945) with various extensions for different switching technologies (Layer 2, wavelength/waveband, SONET/SDH,…) Basic functionalities/protocols – Neighbor discovery/link management (Link Management Protocol - LMP) – Routing with traffic engineering extensions (OSPF-TE, ISIS-TE) Bidirectional – Signaling (RSVP-TE with GMPLS extensions) LSP PATH Applications/solutions – Recovery (protection, restoration) – Make-before-break – Layer 1 VPN (L1VPN working group) Ingress Node A PATH RESV RESV Transit Node B RESVCONF (optional) Cross-connect set upon receiving the PATH message PATH RFC 3473 bidirectional Bidirectional LSP setup data plane PATH More efficient bidirectional LSP setup Bidirectional data plane Egress Node C RESV Cross-connect set upon receiving the RESV message Both cross-connects set upon receiving the PATH message Enabling Data-Intensive Grid Applications with Advanced Optical Technologies - 9/28/2005 13 Generalized Multiprotocol Label Switching (GMPLS) New directions – Separation of path computation as a service – Attention to Ethernet as a Layer2 transport – Inter-domain traffic-engineering • Good work at NSF’s DRAGON project – Inter-domain circuit setup, path computation element (Network Aware Resource Broker –NARB) – The next step is interoperability with other networks Transport Layer Capability Set Exchange NARB NARB NARB End System End System AS 1 AS 3 AS 2 Source: Jerry Sobieski, Tom Lehman, Bijan Jabbari, “Dynamic Resource Allocation via GMPLS Optical Networks (DRAGON),” Presented to the NASA Optical Network Technologies Workshop, August 8, 2004 Enabling Data-Intensive Grid Applications with Advanced Optical Technologies - 9/28/2005 14 Conclusions: Dream or Reality? Key word for Grid networks is high throughput Lambda Grids are the only way to keep up with throughput demand – Reality When is access to dark fiber going to be cheap? Dream – Starting as islands of transparency • Regional Optical Networks (RONs) • Fiber sharing is critical, RONs have to have transparent access to each other • Wavebands as highways between RONs Photonic access to super – Islands growing as optical reach/transmission improves highway for RONs? • Digital wrapper, FEC High throughput needs end-to-end transparency – Data plane transparency • WDM and photonic switching – Control plane transparency • Inter-domain end-to-end circuit setup Availability and recovery are the new QoS for lambda grids Ethernet will be the dominant end-to-end payload HOPI Node – Transparent networks are ready for payload change Enabling Data-Intensive Grid Applications with Advanced Optical Technologies - 9/28/2005 15