Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
MPLS-TE Doesn’t Scale Adrian Farrel Old Dog Consulting [email protected] www.mpls2007.com OLD DOG CONSULTING Is the Sky Falling? The only way to get your attention is to be alarmist MPLS-TE is perfectly functional in today’s networks But: MPLS-TE will not scale indefinitely The problem is the well-known “full mesh” or “n-squared” problem The number of LSPs scales as the square of the number of PEs 2 OLD DOG CONSULTING What Do We Want to Achieve? MPLS-TE is an important feature for many SPs Allow traffic to be groomed Optimize use of network resources Provide quality of service guaranties Carriers look to provide edge-to-edge tunnels across their core networks Differentiated Services VPNs VLANS and pseudowires Multimedia content distribution Normal IP traffic 3 OLD DOG CONSULTING What is the Scope of the Problem? Consider a service provider network with 1000 PEs This is not outrageously large Such a network may be broken into areas or ASes Consider a full mesh of PE-PE TE-LSPs Consider parallel tunnels for different services, QoS levels, and for protection May give rise to multiples of 999,000 LSPs in the core What is the capacity of a core LSR? What is the capacity of a management system? 4 OLD DOG CONSULTING What Are the Scaling Limits? Management NMS How many LSPs can the NMS process Management protocols Reporting on large numbers of LSPs may overload the management network LSR issues Memory capacity Per LSP data requirements CPU capacity – largely an RSVP-TE protocol issue Degradation of LSP setup times Soft state addressed by Refresh Reduction MPLS forwarding plane Number of labels (Only 1048559 per interface) 5 OLD DOG CONSULTING The Snowflake Topology Example network for analysis Meshed core of P nodes PE Called P1 nodes Each Pi+1 node connected to P2 just one Pi node PE nodes connected to just one P1 Pn node Well-defined connectivity and symmetry allows many important metrics to be computed Number of levels & number of nodes per level may be varied We We We We can can can can vary vary vary vary the the the the number of P1 nodes ratio of Pi+1 to Pi value n number of PE nodes per Pn node 6 OLD DOG CONSULTING Analysing the Snowflake Topology Define Discover Pn a node at the nth level (level 1 is core) Sn the number of nodes at the nth level Mn the multiplier at the nth level (how many Pn+1 nodes are connected to a Pn node) Ln number of LSPs seen by a Pn node LPE = 2*(SPE - 1) L2 = M2*(2*SPE - M2 - 1) L1 = M1*M2*(2*SPE – M2*(M1 + 1)) Practical numbers S1 = 10, M1 = 10, and M2 = 20 SPE = 2000 LPE = 3998 L2 = 79580 L1 = 756000 7 OLD DOG CONSULTING The Ladder Topology Example network for analysis Core of P1 nodes looks like a ladder Symmetrical trees subtended to core Similar to many national networks Each Pi+1 node connected to just one Pi node Each PE node connected to just one P node Again: Well-defined connectivity and symmetry allows many important metrics to be computed Number of levels & number of nodes per level may be varied 8 OLD DOG CONSULTING Analysing the Ladder Topology Same definitions as for snowflake network Discover E the number of subtended edge nodes (PEs) to each spar-node (E = M1*M2) LPE = 2*(SPE - 1) L2 = 2*M2*(SPE - 1) - M2*(M2 - 1) L1 ≈ E*E*S1*S1/2 + E*E*S1 + 3*E*E - E*M2 Practical numbers S 1 = 10, M1 = 10, and M2 = 20 E = 200 SPE = 2000 LPE = 3998 L2 = 79580 L1 = 2516000 9 OLD DOG CONSULTING Option 1 – Solve a Different Problem! If a full mesh of PE-PE LSPs is too big, don’t build it! The suggestion is to build a full mesh of Pn-to-Pn LSPs, and perform routing or routing-based MPLS between Pn and PE Scaling improves from O(10002) to O(1002) But we lose functionality This is the bottom line if we don’t fix the problem Why did we want a PE-PE mesh? How do we handle private address spaces? What if the traffic is not routable? This may simply not be good enough to provide the function 10 OLD DOG CONSULTING Option 2 – LSP Hierarchies Well-known, core MPLS function Label stacks Forwarding Adjacencies (RFC 4206) Configured or automatic grooming Possible to build a full or partial mesh of hierarchical tunnels For example connect all P2 nodes Each P2 node must encapsulate each PE-PE LSP in the correct tunnel Each P1 node only sees the P2-P2 tunnels 11 OLD DOG CONSULTING Scaling Properties of Hierarchies - Snowflake Note that PE-PE tunnels don’t help P1-P1 tunnels are also no benefit (core is fully meshed) P2 nodes see all PE-PE LSPs and new tunnels Situation at P1 nodes is much better L2 = M2*(2*SPE - M2 - 1) + 2*(S2 - 1) L1 = M1*(2*S2 - M1 - 1) Numbers (S1 = 10, M1 = 10, and M2 = 20) SPE LPE L2 L1 Flat 2-Level 2000 3998 79580 756000 Hierarchy 2000 3998 79778 1890 Maybe insert another layer (P3 ) to increase the scaling? L3 remains high 12 OLD DOG CONSULTING Scaling Properties of Hierarchies - Ladder Note that PE-PE tunnels don’t help But P1-P1 tunnels are good because core is not fully-meshed Another level of hierarchy is also possible L1 ≈ S1*S1/2 + 2*S1 + 2*E*E*(S1 - 1) - E*M2 - 2 Add a mesh of P2-P2 tunnels L1 = S1*S1/2 + 2*S1 + 2*M1*M1*S1 - M1(M1 + 1) – 2 L2 = 2*M2*(S(PE) - 1) - M2*(M2 - 1) + 2*(S(1)*M(1) - 1) Numbers (S 1 = 10, M1 = 10, and M2 = 20) Flat SPE LPE L2 L1 2000 3998 79580 2516000 2-Level Hierarchy 2000 3998 79580 716060 13 3-Level Hierarchy 2000 3998 79778 1958 OLD DOG CONSULTING Issues and Drawbacks for Hierarchies Scaling is not good enough! Management burden Plan and operate a secondary mesh Effectively the same burden as managing PEs or a layered network Possible to consider auto-mesh techniques Fast Reroute protection is a problem Impact on layer adjacent to PEs is negligible Actually impact is slightly negative FRR struggles to protect tunnel end-points Not obvious how to arrange the hierarchy when the network is not symmetrical E.g., some PEs closer to the core 14 OLD DOG CONSULTING Option 3 – Multipoint-to-Point LSPs LSPs merge automatically as they converge on the destination Reduces the number of LSPs toward the egress Other LSP properties (e.g., bandwidth) must be cumulative TE is still possible, but de-merge is not considered Should count “LSP state” not number of LSPs New definition Xn the amount of LSP state held at each Pn node For flat and hierarchical networks: Each LSP adds one state at ingress or egress Each LSP adds two states at each transit node 15 OLD DOG CONSULTING Scaling Properties of MP2P LSPs - Snowflake XPE = 2*(SPE - 1) X2 = SPE*(M2 + 1) X1 = M1*M2*(S1 - 2) + SPE*(M1 + 1) Numbers (S1 = 10, M1 = 10, and M2 = 20) SPE XPE X2 X1 Flat 2-Level Hierarchy 2000 2000 3998 3998 159160 159358 1512000 3780 16 P2MP 2000 3998 42000 23600 OLD DOG CONSULTING Scaling Properties of MP2P LSPs - Ladder XPE = 2*(SPE - 1) X2 = (M2 + 1)*S1*E X1 ≤ (4 + M1)*S1*E - M1*E Numbers (S1 = 10, M1 = 10, and M2 = 20) Flat SPE XPE X2 X1 2000 3998 159160 5032000 2-Level Hierarchy 2000 3998 159160 1433998 3-Level Hierarchy 2000 3998 159358 3898 17 P2MP 2000 3998 42000 26000 OLD DOG CONSULTING Issues and Drawbacks for MP2P LSPs Clear scaling benefits Better than flat networks Only thing that improves the situation adjacent to PEs But… Data plane support This will only ever be a packet/frame/cell technology Control plane support RSVP does have MP2P support RSVP-TE features not yet specified or implemented De-aggregation and disambiguation May be necessary to use label stack so that egress can detect sender of data OAM may be more complex and require source labels New management applications needed FRR still to be designed 18 OLD DOG CONSULTING Other Topics for Investigation Cost-effectiveness of the network Fast Reroute Revenue only generated by PEs K = S(PE)/(S(1)+S(2) + ... + S(n)) Many ways to improve scaling reduce cost-effectiveness What are the implications of FRR to scaling? Can scaling contributions be designed that can be protected by FRR? Point-to-multipoint What are the scaling properties of P2MP MPLS-TE? Domain boundaries (in particular AS boundaries) Boundaries such as at area and AS borders cause constrictions How can we reduce the number of LSPs seen by ABRs and ASBRs? 19 OLD DOG CONSULTING Conclusions, Next Steps, and References MPLS-TE is not a scaling issue today But it won’t scale arbitrarily We need to plan now for tomorrow’s scalability Hierarchical LSPs are not as good as expected MP2P LSPs may offer a better solution More research and implementation is needed draft-ietf-mpls-te-scaling-analysis-01.txt Seisho Yaukawa (NTT) Adrian Farrel (Old Dog Consulting) Olufemi Komolafe (Cisco Systems) 20 OLD DOG CONSULTING Questions? [email protected] 21 OLD DOG CONSULTING