Download Finance Evolution

Network Operator Perspective
MPLS: 12 Years After
Tom Bechly
IETF 74, San Francisco
IAB Plenary
March 18, 2009
1
MPLS: A Successful Protocol
• MPLS has been and is a successful protocol
– From perspective of RFC 5218 (What Makes for a Successful Protocol?),
MPLS was used for its intended purpose and at intended scale
• Goal was to switch packets to support rapidly expanding global
networks
– MPLS is “wildly successful” (RFC 5218) in that its use has exceeded its
original design goal thru development of numerous extensions
– From service provider perspective MPLS was successful in supporting
growth, reducing cost, and providing basis for new services
• Original goal of bringing Layer 2 switching speed to Layer 3 was
accomplished, but somewhat discounted over time due to hardware
evolution
– L2 was hardware switched and L3 was process switched
• MPLS was easily leveraged for traffic engineering, VPNs, and layer 2
transport.
• For the service provider, MPLS has become one the most reached
for and extended tools in the tool chest (150+ RFCs)
2
MPLS
CE Router
CE Router
MPLS CORE
PE Router
PE Router
P Router
PE Router
P Router
PE Router
P Router
CE Router
Customer Edge (CE) Router
CE Router
• Enables network edge
routers to apply simple
MPLS labels to packets or
frames
• Forwards packets by
swapping labels with
minimal lookup
• Integrates Layer 2
switching and Layer 3
routing
Provider Edge (PE) Router/Switch
Provider Core Router/Switch
3
MPLS/RSVP-TE Benefits
• MPLS with RSVP-TE provides overall path control in network
– Use with constraint based routing
– Control over latency and delay variation
– Bridges gap between ability to deploy capacity versus current demand in
existing network
• Use of MPLS allowed gathering measurement statistics on LSPs
– Probably more important than actual path control
– Provides ability to accurately measure traffic between router pairs
• Traffic volumes, latency, and delay variation
– Measure traffic between hubs, metros, and regions
– Measure asymmetry of flows, over time
– A time series depiction can be built to trend traffic for efficient investment
and to provide required service
• MPLS became an enabler for the development of additional services
– L2 VPNs and L3 VPNs
4
Verizon Public IP
• AS 701 was initially implemented as an overlay over a dedicated
frame relay network
– Path control was effected thru manipulating path of frame relay PVCs
• As capacity requirements increased, the network was migrated to an
overlay over ATM
– The cost of this became untenable, as capacity requirements continued
to increase
• MPLS with RSVP-TE deployed in EMEA (AS 702) in 1999
– First deployment of RSVP-TE in production network
– Deployed in US (AS 701) in 2000
• Deployed for traffic engineering to provide control over path selection
that was not available thru L3 protocols
– Shortest path algorithm did not always provide optimal route
• MPLS technology has enabled the Verizon Public IP network to grow
to be one of the largest in the world
5
Verizon IP Network
– 410 unique switch/router hubs (PoPs)
– Six continents, 150+ countries
6
Verizon Layer3 VPN Services: VBNS+ and Private IP
• vBNS (very-high-performance Backbone Network Service) was
established in 1995
– Cooperative research and development agreement between Verizon
(formerly MCI) and National Science Foundation (follow on to NSFnet)
– Evolved to a commercial product: vBNS+ for gov and edu market
• MPLS routing/switching implemented in network in 1999
– Initially MPLS was implemented for traffic engineering
• L3VPN (RFC 2547) was implemented in 2001
– There are approximately 40 nodes in 19 US cities, full mesh of TE LSPs
• Verizon PIP (Private IP) was established in 1999
– Layer 3 VPN (RFC 4364), wide area network for business customers
– Quality of Service, strong SLAs, etc.
• Large global network
– There are approximately 625 nodes across 162 cities in 59 countries
• Uses LDP for label distribution, with partial mesh of LSPs
7
Private IP Global Reach
MP10163v5.03
8
Verizon Layer 2 Services: MAE® Services and
Converged Packet Architecture (CPA)
• MAE® Services established 1992 as metro Internet Exchange point
• Evolved into MPLS based national service for extended peering and
L2 VPNs (VPWS), implemented in 2002
– Service interworking (ATM, Frame Relay, and Ethernet), based on draft
Martini pseudowires and draft Shah ARP Mediation
• Implemented across public internet within full mesh of GRE tunnels
– ISIS, RSVP-TE signaled LSPs, and LDP signaled pseudowires
• CPA supports Ethernet access and Ethernet services
– L2 VPNs: both EVPL (PWE3) and VPLS (RFC 4762)
– Quality of Service, strong SLAs, etc.
• Large global network
– There are approximately 115 nodes across 27 countries
• RSVP-TE used to signal LSPs
– Full mesh for EVPL and VPLS
– Currently 10,000+ LSPs
9
Lessons Learned
• Implementation defects significantly impact early perception of
technology
– For AS 701, there was internal resistance to moving from ATM underlay
network to MPLS
– When defects in the MPLS implementation on vendor equipment were
encountered these initially viewed by some as defects in the technology
10
Lessons not Learned (VPLS)
• RFC 4762: Virtual Private LAN Service (VPLS) Using Label
Distribution Protocol (LDP) Signaling
– Hierarchy is managed thru HVPLS, specified within RFC
• RFC 4761: Virtual Private LAN Service (VPLS) Using BGP for AutoDiscovery and Signaling
– Hierarchy is managed thru route reflectors and multi-segment
pseudowires
• Both approaches are currently in production in different service
provider networks
• Some vendors have implemented both standards
• This increases to overall cost and complexity of technology and
network development across the community
– Resolution and mitigation of differences is far more economic during
protocol development than once into implementation
– Gateway function has high development and operational cost
– The added costs and complexity are continuously accretive
11
Lessons not Learned (RFC5085 – PW VCCV)
• Pseudowire Virtual Circuit Connectivity Verification (VCCV) – RFC 5085
– Three modes of operation: (Type 1: PWE3 Control Word Bit, Type 2: MPLS Router
Alert Label, Type 3: MPLS PW Label with TTL == 1
– Mode is negotiated, so all three are optional
• Vendors, to this point, have not implemented all modes nor the same modes
• This leads to interoperability issues in mixed vendor networks
– Delays significantly availability of feature
– Adds to development and integration costs
VCCV Mode
Vendors Y
Vendors X
Control Word*
Yes
No
Router Alert Label
Yes
No
TTL Expiry*
No
Yes
12
Continuing Challenges
• Latency sensitive customers
– These are typically financial customers that are sensitive to a 2ms
increase or change in latency
• Require traffic to be on path with deterministic low latency
– Due to network event traffic may be rerouted, via Fast Reroute and the
re-signaled LSP
– Paths are recalculated periodically to ensure low latency path
– Once optimal path is available, traffic is re-routed (make before break) to
this path
– As this path could be significantly shorter (2 – 10ms), there will be out of
order packets that may impact some hosts
• Nodes in network within the core, may carry a high number of LSPs
– Latency sensitive customers are requesting notification on any
maintenance that will impact LSPs carrying their traffic
13
MPLS Going Forward
• MPLS has been an extremely successful protocol
– It has been widely deployed and extended
• MPLS based networks and facilities to continue to grow and expand
– This growth is continuing and will continue for some time
14