Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download nsa08 - Princeton University
Document related concepts
Transcript
Traffic Engineering for ISP Networks Jennifer Rexford Computer Science Department Princeton University http://www.cs.princeton.edu/~jrex A Challenge in ISP Backbone Networks Finding a good way to route the data packets – Given the current network topology and offered traffic – For good performance and efficient use of resources Why the Problem is Hard? IP traffic varies, and the service is best effort – The offered traffic is not known in advance – The resources in the network are not reserved The routers do not adapt on their own – Load-sensitive routing is not widely deployed – Due to control overhead and stability challenges Routing protocols were not designed to be managed – At best indirect control over the flow of traffic Fine-grain traffic measurements often unavailable – E.g., only have coarse-grain link load statistics In This Talk… TE with traditional IP routing protocols – Shortest-path protocols with configurable link weights Two main research challenges – Optimization: tuning link weights to the offered traffic – Tomography: inferring the offered traffic from link load Deployed solutions in AT&T’s U.S. backbone – Our experiences working with the network operators – And how we improved the tools over time Ongoing research on traffic management Optimization: Tuning Link Weights Routing Inside an Internet Service Provider Routers flood information to learn the topology – Routers determine “next hop” to reach other routers… – By computing shortest paths based on the link weights Routers forward packets via the “next hop” link(s) 2 3 2 1 1 1 3 5 4 3 Link Weights Control the Flow of Traffic Routers compute paths – Shortest paths as sum of link weights Operators set the link weights – To control where the traffic goes 2 3 2 1 1 31 3 5 4 3 Heuristics for Setting the Link Weights Proportional to physical distance – Cross-country links have higher weights than local ones – Minimizes end-to-end propagation delay Inversely proportional to link capacity – Smaller weights for higher-bandwidth links – Attracts more traffic to links with more capacity Tuned based on the offered traffic – Network-wide optimization of weights based on traffic – Directly minimizes key metrics like max link utilization Why Are the Link Weights Static? Strawman alternative: load-sensitive routing – Link metrics based on traffic load – Flood dynamic metrics as they change – Adapt automatically to changes in offered load Reasons why this is typically not done – Delay-based routing unsuccessful in the early days – Oscillation as routers adapt to out-of-date information – Most Internet transfers are very short-lived Research and standards work continues… – … but operators have to work with what they have Big Picture: Measure, Model, and Control Network-wide “what if” model Offered Topology/ traffic Configuration measure Changes to the network control Operational network Traffic Engineering in an ISP Backbone Topology – Connectivity and capacity of routers and links Traffic matrix – Offered load between points in the network Link weights – Configurable weights for shortest-path routing Performance objective – Balanced load, low latency, service level agreements … Question: Given the topology and traffic matrix in an IP network, which link weights should be used? Key Ingredients of Our Approach Measurement – Topology: monitoring of the routing protocols – Traffic matrix: widely deployed traffic measurement Network-wide models – Representations of topology and traffic – “What-if” models of shortest-path routing Network optimization – Efficient algorithms to find good configurations – Operational experience to identify key constraints Formalizing the Optimization Problem Input: graph G(R,L) – R is the set of routers – L is the set of unidirectional links – cl is the capacity of link l Input: traffic matrix – Mi,j is traffic load from router i to j Output: setting of the link weights – wl is weight on unidirectional link l – Pi,j,l is fraction of traffic from i to j traversing link l Multiple Shortest Paths With Even Splitting 0.25 0.25 0.5 1.0 0.25 0.5 1.0 0.25 0.5 0.5 Values of Pi,j,l Defining the Objective Function Computing the link utilization – Link load: ul = Si,j Mi,j Pi,j,l – Utilization: ul/cl Objective functions – min(maxl(ul/cl)) f(x) – min(Sl f(ul/cl)) x 1 Complexity of the Optimization Problem NP-hard optimization problem – No efficient algorithm to find the link weights – Even for the simple convex objective functions Why can’t we just do multi-commodity flow? – E.g., solve the multi-commodity flow problem… – … and the link weights pop out as the dual – Because IP routers cannot split arbitrarily over ties What are the implications? – Have to resort to searching through weight settings Optimization Based on Local Search Start with an initial setting of the link weights – E.g., same integer weight on every link – E.g., weights inversely proportional to link capacity – E.g., existing weights in the operational network Compute the objective function – Compute the all-pairs shortest paths to get Pi,j,l – Apply the traffic matrix Mi,j to get link loads ul – Evaluate the objective function from the ul/cl Generate a new setting of the link weights repeat Making the Search Efficient Avoid repeating the same weight setting – Keep track of past values of the weight setting – … or keep a small signature (e.g., a hash) of past values – Do not evaluate a weight setting if signatures match Avoid computing the shortest paths from scratch – Explore weight settings that changes just one weight – Apply fast incremental shortest-path algorithms Limit the number of unique values of link weights – Do not explore all 216 possible values for each weight Stop early, before exploring the whole search space Incorporating Operational Realities Minimize number of changes to the network – Changing just 1 or 2 link weights is often enough Tolerate failure of network equipment – Weights settings usually remain good after failure – … or can be fixed by changing one or two weights Limit dependence on measurement accuracy – Good weights remain good, despite random noise Limit frequency of changes to the weights – Joint optimization for day and night traffic matrices Application to AT&T’s Backbone Network Performance of the optimized weights – Search finds a good solution within a few minutes – Much better than link capacity or physical distance – Competitive with multi-commodity flow solution How AT&T changes the link weights – Maintenance done every night from midnight to 6am – Predict effects of removing link(s) from the network – Reoptimize the link weights to avoid congestion – Configure new weights before disabling equipment Example from My Visit to AT&T’s Operations Center Amtrak repairing/moving part of the train track – Need to move some of the fiber optic cables – Or, heightened risk of the cables being cut – Amtrak notifies us of the time the work will be done AT&T engineers model the effects – Determine which IP links go over the affected fiber – Pretend the network no longer has these links – Evaluate the new shortest paths and traffic flow – Identify whether link loads will be too high Example Continued If load will be too high – Reoptimize the weights on the remaining links – Schedule the time for the new weights to be configured – Roll back to the old weight setting after Amtrak is done Same process applied to other cases – Assessing the network’s risk to possible failures – Planning for maintenance of existing equipment – Adapting the link weights to installation of new links – Adapting the link weights in response to traffic shifts Conclusions on Traffic Engineering IP networks do not adapt on their own – Routers compute shortest paths based on static weights Service providers need to adapt the weights – Due to failures, congestion, or planned maintenance Leads to an interesting optimization problems – Optimize link weights based on topology and traffic Optimization problem is computationally difficult – Forces the use of efficient local-search techniques Results of the local search are pretty good – Near-optimal solutions that minimize disruptions Extensions Robust link-weight assignments – Link/node failures – Range of traffic matrices More complex routing models – Destinations reachable via multiple “egress points” – Interdomain routing policies Interaction between ISPs – Inter-ISP negotiation for joint optimization – Grappling with scalability and trust issues Tomography: Inferring the Traffic Matrix Computing the Traffic Matrix Mi,j Hard to measure the traffic matrix – IP networks transmit data as individual packets – Routers do not keep traffic statistics, except link utilization on (say) a five-minute time scale Need to infer the traffic matrix Mi,j from – Current topology G(R,L) – Current routing Pi,j,l – Current link load ul – Link capacity cl Inference: Network Tomography From link counts to the traffic matrix Sources 5Mbps 3Mbps 4Mbps 4Mbps Destinations Tomography: Formalizing the Problem Ingress-egress pairs – p is a ingress-egress pair of nodes (i,j) – xp is the (unknown) traffic volume for this pair Mi,j Routing – Plp is proportion of p’s traffic that traverses l Links in the network – l is a unidirectional edge – ul is the observed traffic volume on this link Relationship: u = Px (work backwards to get x) Tomography: One Observation Not Enough Linear system of n nodes is underdetermined – Number of links e is around O(n) – Number of ingress-egress pairs c is O(n2) – Dimension of solution sub-space at least c - e Multiple observations are needed – k independent observations (over time) – Stochastic model with Poisson iid counts – Maximum likelihood estimation to infer matrix Doesn’t work all that well in practice… Approach Used at AT&T: Tomo-gravity Gravitational assumption – Ingress point a has traffic via – Egress point b has traffic veb – Pair (a,b) has traffic proportional to via * veb 9 6 20 3 14 21 7 10 Approach Used at AT&T: Tomo-gravity Problem with gravity model – Gravity model ignores the load on the inside links – Gravity assumption isn’t always 100% correct – Resulting traffic matrix might not satisfy the link loads Combining the two techniques – Gravity: find a traffic matrix using the gravity model – Tomography: find the family of traffic matrices consistent with all link load statistics – Tomo-gravity: find the tomography solution that is closest to the output of the gravity model Works extremely well (and fast) in practice Conclusions Managing IP networks is challenging – Routers don’t adapt on their own to congestion – Routers don’t reveal much information about traffic Measurement provides a network-wide view – Topology – Traffic matrix Optimization enables the network to adapt – Inferring the traffic matrix from the link loads – Optimizing the link weights based on the traffic matrix New Research Direction: Design for Manage-ability Two main parts of network management – Control: optimization – Measurement: tomography Two research approaches – Bottom up: do the best with what you have – Top down: design systems that are easier to manage Design for manage-ability – “If you are both the professor and the student, you create exam questions that are easy to answer.” Example: Changing the Path Computation Routers split traffic over multiple paths – More traffic on shorter paths, less on longer ones – In proportion to the exponential of path cost 2 3 2 Exciting result 1 1 1 4 3 5 3 – Can achieve optimal distribution of the traffic – With polynomial-time algorithm for setting the weights New Research Direction: Logically-Central Control Traditional division of labor – Routers: real-time, distributed protocols – Management system: offline, centralized algorithms Example: routing protocols and traffic engineering – Routing: routers react automatically to link failures – TE: management system sets the link weights The case for separating routing from routers – Better decisions with network-wide visibility – Routers only collect measurements and forward packets Example: Routing Control Platform (RCP) Logically-centralized server – Collects measurement data from the network – Pushes forwarding tables into the routers Benefits RCP – Network-wide policies – Flexible, easy to customize – Fewer nodes to upgrade ISP Feasibility – High-end PC can compute routes for large ISP – Simple replication to survive failures References Traffic engineering using traditional protocols – http://www.cs.princeton.edu/~jrex/papers/ieeecomm02.pdf – http://www.cs.princeton.edu/~jrex/papers/opthand04.pdf – http://www.cs.princeton.edu/~jrex/papers/ton-whatif.pdf Tomo-gravity to infer the traffic matrix – http://www.cs.utexas.edu/~yzhang/papers/mmi-ton05.pdf – http://www.cs.utexas.edu/~yzhang/papers/tomogravitysigm03.pdf – http://www.cs.princeton.edu/~jrex/papers/sfi.pdf References Design for manage-ability – http://www.cs.princeton.edu/~jrex/papers/pefti.pdf – http://www.cs.princeton.edu/~jrex/papers/optimizability.pdf – http://www.cs.princeton.edu/~jrex/papers/tie-long.pdf Routing Control Platform – http://www.cs.princeton.edu/~jrex/papers/rcp.pdf – http://www.cs.princeton.edu/~jrex/papers/ccr05-4d.pdf – http://www.cs.princeton.edu/~jrex/papers/rcp-nsdi.pdf – http://www.research.att.com/~kobus/docs/irscp.inm.pdf
