Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Computer network wikipedia , lookup
Deep packet inspection wikipedia , lookup
Network tap wikipedia , lookup
Multiprotocol Label Switching wikipedia , lookup
Asynchronous Transfer Mode wikipedia , lookup
Airborne Networking wikipedia , lookup
Wireless security wikipedia , lookup
Policies promoting wireless broadband in the United States wikipedia , lookup
Piggybacking (Internet access) wikipedia , lookup
Center of Excellence Wireless and Information Technology TeraPaths: Managing Flow-Based End-to-End QoS Paths Experience and Lessons Learned Dimitrios Katramatos, Dantong Yu, Kunal Shroff Brookhaven National Laboratory Thomas Robertazzi Stony Brook University Shawn McKee University of Michigan CEWIT 2008 Center of Excellence Wireless and Information Technology Abstract • TeraPaths is a Department of Energy funded network research project to support efficient, predicable, and prioritized peta-scale data replication in modern highspeed networks • The TeraPaths network management framework establishes on-demand and manages true end-to-end, QoS-aware, virtual network paths across multiple administrative network domains • TeraPaths dedicates network resources to data flows specifically authorized to use such network paths, in a transparent and scalable manner. This ensures that only selected flows receive a pre-determined, guaranteed level of QoS in terms of bandwidth, jitter, delay, etc. CEWIT 2008 2 Center of Excellence Wireless and Information Technology Speaker’s Biography • Dantong Yu Brookhaven National Laboratory • Dantong Yu received the Ph.D. degree in Computer Science from State University of New York at Buffalo, USA, in 2001. His research interests include highspeed network performance, network Quality of Service, cluster/grid computing, information retrieval, data mining, databases, and data warehouses. He leads the large volume WAN data transfer between CERN, BNL, ATLAS and RHIC collaboration institutes over high-speed networks with Grid middleware CEWIT 2008 3 Center of Excellence Wireless and Information Technology Outline • • • • • • Background: the TeraPaths project Establishing flow-based end-to-end QoS paths Domain interoperation Encountered issues and proposed solutions Project status and future work Conclusions CEWIT 2008 4 Center of Excellence Wireless and Information Technology Background • Provide QoS guarantees at the individual data flow level, all the way to the end hosts, transparently – Data flows have varying priority/importance • Video streams • Critical data • Long duration transfers – Default “best effort” network behavior treats all data flows as equal – Capacity is not unlimited • Congestion causes bandwidth and latency variations • Performance and service disruption problems, unpredictability • Dynamic flow-based SLAs = schedule network utilization – Regulate and classify (prioritize) traffic CEWIT 2008 5 Center of Excellence Wireless and Information Technology End-to-End Setup Site B Site C host router site host / border router host b1 site border router ACLs: b1 a1 10.100.1.y2 10.100.1.x2 regional provider router regional provider router host c1 ACLs: c1 a2 WAN domains VLAN X VLAN Y 10.100.1.x1 site border router virtual border router 10.100.1.y1 ACLs: a1 b1 a2 c1 host router host a2 host a1 Site A CEWIT 2008 6 Center of Excellence Wireless and Information Technology Establishing End-to-End QoS Paths • Multiple administrative domains – Cooperation, trust, but each maintains full control – Heterogeneous environment – Domain controller coordination through web services • Coordination models … – Star • Requires extensive information for all domains … – Daisy chain • Requires common flexible protocol across all domains – Hybrid (end-sites first) … • Independent protocols • Direct end site negotiation CEWIT 2008 7 Center of Excellence Wireless and Information Technology Path Setup (2) • End site subnets are configured by TeraPaths software instances (TeraPaths Domain Controllers or TDCs) – TDCs configure end site LANs to prioritize and regulate authorized flows via the DiffServ framework at the network device level – Source site polices/marks authorized flow packets – Destination site admits/re-polices/re-marks packets – End site LANs tx/rx marked packets to/from the WAN • WAN provides MPLS tunnels or dynamic circuits – Initiating TDC requests MPLS tunnel or dynamic circuit with matching bandwidth and lifetime, or… – TDC groups flows with common src/dst into MPLS tunnel or dynamic circuit with aggregate bandwidth and lifetime – WAN preserves packet markings CEWIT 2008 8 Center of Excellence Wireless and Information Technology Path Setup (3) • WAN domains interoperate – Each end site’s TDC has a single point of contact for WAN services – TDCs have no knowledge of WAN internals other than what is exposed by the WAN services • End sites have no direct control over the WAN • Either tunnel or circuit through WAN – TeraPaths does not mix and match the layer 2 and layer 3 technology. • TeraPaths “proxy” servers – Implement interface required by TeraPaths core – Hide WAN service differences – Clients to WAN web services (currently OSCARS / DRAGON) • Close cooperation with ESnet and I2 development teams – Submit reservations for MPLS tunnels or dynamic circuits – Handle security requirements – Handle errors CEWIT 2008 9 Center of Excellence Wireless and Information Technology Addressing L2-Specific Issues • Limitations with VLANs – Tag range (tentatively selected 50 VLANs – 3550 to 3599) • Each site may have its own range – Tag conflicts • Rely on WAN service • Eliminate by synchronizing site databases • VLAN renaming (if/when possible) • Scalability issues – Limited number of VLAN tags/Circuits: • Flow grouping / circuit consolidation – Forward flows through same virtual WAN circuit » Create circuit with new parameters / switch current flows / cancel old circuit » Modify WAN reservations (if/when possible) – PBR overhead • Virtual border router • Sensitive/3rd party network segments – VLAN pass-thru CEWIT 2008 10 Center of Excellence Wireless and Information Technology •Flows between same src and dst sites can share circuit, policing maintains bandwidth guarantee •Multiple TeraPaths reservations associate with the same circuit reservation – Easy when requirements are known in advance bandwidth Flow Grouping/Circuit Consolidation Δbw 4 2 1 CEWIT 2008 5 current time – Modification of reservations required otherwise • Selection/optimization to minimize resource waste • Trade-off based on Δbw (bandwidth difference), Δtb, Δta (time period before and after a reservation) Δt 3 4 2 5 3 1 time 11 Center of Excellence Wireless and Information Technology Flow Grouping/Circuit Consolidation (2) • Similar approach to disk buffering (read ahead / write behind) – Bring up ahead / teardown behind – Reuse existing active circuits – Reserve circuits with more bandwidth and longer duration depending on differences in start time, duration, bandwidth of reservations Δtb bandwidth bandwidth – Delay teardown, modify circuit duration and/or bandwidth if possible Δta 2 2 4 5 3 1 Δta current time 2 1 2 4 3 5 1 time CEWIT 2008 5 3 1 current time 4 4 3 5 time 12 Center of Excellence Wireless and Information Technology Limitation of Dynamic Circuits • A recent incident in BNL’s LHCOPN subnet: – Cisco’s PBR implementation only uses the status of an interface to decide whether or not to forward packets – A network circuit breaks somewhere along the path, but the involved interfaces on both ends are still up – No probes and/or heartbeat exist to check the “health” of circuits – Fail-over to the backup link does not work since primary interfaces are up even when such a problem exists • End site monitoring is the most effective way to detect such a problem CEWIT 2008 13 Center of Excellence Wireless and Information Technology Active Circuit Probing Each TeraPaths site instance periodically verifies “well being” of reservations: – Selects active reservations initiated by site (site responsibility) – Finds circuit/VLAN associated with each reservation – Performs a circuit check with a quick pinging of other site’s router (private ip address space) – Less than 100% success triggers a recheck with longer duration pings in both directions (to and from other site) – Low success % triggers reservation cancellation reverting traffic to best effort network – Optionally, the system adapts reservation data and attempts to setup a new end-to-end path (for given time period/number of attempts) CEWIT 2008 14 Center of Excellence Wireless and Information Technology Prioritizing Traffic TeraPaths QoS test 1 (prioritize traffic) Bandwidth (Mbits/sec) 1200 1000 800 priority background total 600 QoS / circuit reservation active 400 200 0 0 CEWIT 2008 competing traffic causes dramatic drop in bandwidth 200 400 600 800 1000 time (sec) 15 Center of Excellence Wireless and Information Technology Recovering from Circuit Failure TeraPaths QoS test 2 (prioritize/fallback to best effort) Bandwidth (Mbits/sec) 1200 1000 800 priority background total 600 recovery to best effort 400 200 circuit interruption 0 0 200 600 400 800 1000 time (sec) CEWIT 2008 16 Center of Excellence Wireless and Information Technology Competing against BE traffic remote EF against remote and local BE remote EF against local BE 10000 10000 9000 9000 8000 8000 7000 7000 6000 6000 remote EF local BE remote BE 5000 4000 4000 3000 3000 2000 2000 1000 1000 0 remote EF local BE local BE 5000 0 0 100 CEWIT 2008 200 300 400 500 0 100 200 300 400 500 600 17 Center of Excellence Wireless and Information Technology Status • BNL, UMich, BU, all with 10Gbps connections, multiple pass-thru configurations (BNL, UMich, NoX, Merit, MiLR) • Utilization of L3 paths (MPLS tunnels, ESnet only), L2 paths (dynamic circuits, ESnet and Internet2) • Multiple QoS reservations through same circuit (support for circuit consolidation) • Multiple circuits per site subject to per-site VLAN availability (flow grouping/circuit consolidation) • Active circuit probing for failures with fallback to best effort network/attempt to reconfigure e2e path (in testing phase) • Dynamic bandwidth allocation within service classes (in testing phase) • New command line client CEWIT 2008 18 Center of Excellence Wireless and Information Technology Future Work • Continue working on automatic flow grouping / circuit consolidation. • Configurable reservation negotiation • Grid-style AAA (GUMS/VOMS) • Plug-ins: SRM (dCache), others • Compatibility with Lambda Station • Support for different hardware as needed • ATLAS Production: – Replicate ATLAS Physics data from BU and UMich with the existing ATLAS DDM stack, and with end-to-end QoS circuits – Tier 1 (BNL) and Tier 2 data replication • http://www.terapaths.org CEWIT 2008 19 Center of Excellence Wireless and Information Technology Conclusions • Demonstrated the effective prioritization and protection from interference of selected data transfers between three LHC experiment institutes – Brookhaven National Laboratory, the University of Michigan, and Boston University – through guaranteed bandwidth virtual paths, at the presence of intensive best-effort IP traffic sharing the same network resources • A practical and economical end-to-end network resource reservation system, extending new capabilities to users/applications of end sites without requiring additional, expensive network infrastructure components CEWIT 2008 20