Download Deployment & Design issues of IP over Optical Networks

10 Gigabit Ethernet, WDM,
MPLS, Traffic Engineering
29th Speedup Workshop on
Distributed Computing and High-Speed Networks
Berne University Switzerland
March 22-23, 2001
3007
1367_06_2000_c2
© 2000, Cisco Systems, Inc.
1
Walter Dey
Consulting Engineer
Cisco Systems EMEA
[email protected]
3007
1367_06_2000_c2
© 2000, Cisco Systems, Inc.
2
10 Gigabit Ethernet
3007
1367_06_2000_c2
© 2000, Cisco Systems, Inc.
3
Ethernet History
• 1980’s
10 Mbps Ethernet IEEE 802.3
• 1992-95
100 Mbps Ethernet IEEE 802.3u
• 1995-1999
1000 Mbps Ethernet IEEE
802.3z, 802.3ab
• 1998-2000
10/100/1000 Mbps Ethernet Link
Aggregation IEEE 802.3ad
• 1999-2002
(March) 10 Gbps IEEE 802.3ae
© 2000, Cisco Systems, Inc.
4
Moving the Decimal Point: 10 GbE Performance
and Scalability
10 Gbps
OC-192
Gigabit EtherChannel
10 Gbps
Ethernet
• LAN applications
• Metro applications
1 Gbps
Gigabit Ethernet
• WAN applications
10 GbE IEEE
802.3ae
Standard
Fast EtherChannel
Fast Ethernet
100 Mbps
1996
1997
© 2000, Cisco Systems, Inc.
1998
1999
2000
2001
2002
5
Gigabit Ethernet Layer Diagram
Media Access Control (MAC)
Full Duplex and/or Half Duplex
802.3z
Gigabit Media Independent Interface (GMII)
(Optional)
802.3ab
1000BaseX PHY
8B/10B
1000BaseT
PCS
1000BaseLX
Fiber Optic
Xcvr
1000BaseSX
Fiber Optic
Xcvr
1000BaseCX
Copper
Xcvr
Single-Mode
or Multimode
Fiber
(10km or 550m)
Multimode
Fiber
(275 m)
Shielded
Copper
Cable
© 2000, Cisco Systems, Inc.
1000BaseT
PMA
Unshielded
Twisted Pair
6
GBIC Module Flexibility
SX
LX/LH
ZX
Multimode 275 meters
only
Multimode/ 550 meters/
singlemode 10 km
singlemode 70km-100km
only
• Modular transceiver—‘plug and play’
• Multiple suppliers
• Large volume—250K ports/month
• Low cost
© 2000, Cisco Systems, Inc.
7
Gigabit Ethernet Cost Analysis
Relative Cost per Megabit/sec
1.2
1
0.8
0.6
0.4
0.2
0
OC-48 SR
1000BaseLX
1000BaseSX
(2 km)
(10 km)
(300 m)
© 2000, Cisco Systems, Inc.
8
OC-192 Optics Size
First Generation
Third Generation
VSR
OC-192 Optics Circuitry
© 2000, Cisco Systems, Inc.
9
VSR (Very Short Reach) 12 x
1.25G
12x1.25
Mapping
OC-192
Framer
• 16/10
• Link Prot
• FEC
• Deskew
E/O
12 fiber
ribbon,
MM, 62.5
O/E
12x1.25
Features:
• 12 Tx + 12 Rx channels @ 1.25G (10 data + 2 control)
• 2 x 12 fiber ribbon (Tx and Rx)
• Protection Channel (1:10) (XOR of the 10 data channels)
• Error Detection Channel (CRC’s of the other 11 channels)
• Compatible with OC-192 framer interface (OIF99.102)
• Link length up to 400m
• Leverages mature GE technology and CMOS SERDES
• OIF (Optical Internetworking Forum) Contribution OIF-99.120
© 2000, Cisco Systems, Inc.
10
VSR Ribbon Cable Connector
© 2000, Cisco Systems, Inc.
11
The Growth of Ethernet
1,000,000
Ports (000s)
100,000
Sw
Sw
Sw
Sw
10
100
1000
10000
10,000
1,000
100
10
1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004
Sources: Sw 10, Sw 100, Sw 1000: Dell’Oro Group, Sw 10,000: Gartner Group Dataquest, Cisco Projections
© 2000, Cisco Systems, Inc.
12
Why 10 Gigabit Ethernet ?
• Aggregates Gigabit Ethernet segments
• Scales Enterprise and Service Provider LAN
backbones
• Leverages installed base of 250 million
Ethernet switch ports
• Supports all types of traffic and services (data,
packetized voice and video, IP)
• Supports metropolitan and wide area networks
• Faster and simpler than other alternatives
© 2000, Cisco Systems, Inc.
13
10 Gigabit Ethernet Standard
Status
• IEEE 802.3ae (Task Force)
• Project kicked off in March, 1999
• Project approval January 2000
• First draft September 2000
• First ballot March 2001
• Completion March, 2002
© 2000, Cisco Systems, Inc.
14
IEEE Goals for 10 Gigabit
Ethernet
• Preserve 802.3 Ethernet frame format
• Preserve minimum and maximum frame size of
current 802.3 Ethernet (No Jumbo Frames)
• Support only full duplex operation
• Support 10,000 Mbps at MAC interface
• Define two families of PHYs
LAN PHY operating at 10 Gbps
WAN PHY operating at a data rate compatible
with the payload rate of OC-192c/SDH VC-464c
Note: Partial list
© 2000, Cisco Systems, Inc.
15
Optical Transceivers for 10 Gigabit
Ethernet (802.3ae Task Force, late 2000)
© 2000, Cisco Systems, Inc.
16
Advantages of LAN PHY
• Ethernet style coding techniques,
simpler and lower cost than SONET
framing
• maximum compatibility 10 / 100 /
1000 / 10000 Mbps
• Full 10 Gb data rate
© 2000, Cisco Systems, Inc.
17
10 GbE LAN Applications
Building A
2 to 40 km
Building B
10 GbE
SM fiber
10GbE 100 to
300 m, MM fiber
• Cost-effective bandwidth for the LAN, Switch-to-Switch
• Aggregate multiple Gigabit Ethernet segments
• 10 GigaEtherChannel will enable 20 to 80 Gbps (future)
© 2000, Cisco Systems, Inc.
18
Advantages of WAN PHY
• Enables using SONET infrastructure for L1
• Requires some SONET features (OC-192,
Framing, min Path/Section/Line overhead
processing)
• Connects to SDH/SONET access devices
• Avoids costly function of SDH / SONET
(TDM, OAM&P, Stratum clocking)
© 2000, Cisco Systems, Inc.
19
10 GbE over WAN
•
•
•
•
Attachment to the optical cloud with WAN physical layer (WAN PHY)
Compatibility with the installed base of SONET OC-192
Interfaces and links between SP to IXC networks can be co-located
No need for protocol conversion, traffic remains IP/Ethernet
10 GbE
WAN PHY
(< 300 m)
DWD Mux
DWD Mux
DWD Mux
IXC WAN transport network
10 GbE (WAN
PHY)
(< 300 m)
OC-192 SONET and DWDM: 1000’s km
SONET framing
Service Provider POP
San Jose, CA
© 2000, Cisco Systems, Inc.
Service Provider POP
New York, NY
20
UNI PHY
• Consistent Encoding for seriel LAN
PHY and SONET/SDH payload for a
WAN PHY
• 64B/66B encode
• Low overhead (3%), seriel LAN PHY
runs at 10.3 Gbaud
• WAN PHY solution puts 64B/66B
encoded data stream into payload
portion of SDH/SONET data stream
© 2000, Cisco Systems, Inc.
21
WDM, UCP (Unified
Control Plane)
3007
1367_06_2000_c2
© 2000, Cisco Systems, Inc.
22
Metro DWDM Evolution
l128
10 GE
OC 192
OC 48
OC12
OC 3
100M
10M
SONET
ATM
POS
DPT
© 2000, Cisco Systems, Inc.
l64
l32
l1
Pt-Pt
ADM
Rings
Mesh
23
Metro DWDM Evolution
l128
10 GE
OC 192
OC 48
OC12
OC 3
100M
10M
SONET
ATM
POS
DPT
© 2000, Cisco Systems, Inc.
l64
Mux
l32
1
l1
Pt-Pt
ADM
Rings
Mesh
N
24
Metro DWDM Evolution
l128
Switching
10 GE
OC 192
OC 48
OC12
OC 3
100M
10M
SONET
ATM
POS
DPT
© 2000, Cisco Systems, Inc.
l64
Mux
l32
1
l1
Pt-Pt
ADM
Rings
Mesh
N
25
Migration to Mesh Architecture
• Putting the
“network” in
optical
networking
IP
Control plane
DWDM
transmission
Mesh topology
IP controlled
A-Z provisioning
Optical Core
(Sub-) Wavelength
switching
granularity
IP Terabit
Routing
Open protocols
Increasing
distance without
regeneration
© 2000, Cisco Systems, Inc.
26
Existing Control Planes
Network
Element
Standard
Body
Routing
Signaling
Available
Optical
Cross Connect
None
Proprietary
Proprietary
Future
ATM
Switch
ATM
Forum
PNNI
PNNI
Deployed
MPLS
IP-LSR
IETF
Constraint
Based
LDP/
RSVP
Deployed
Source: John Drake—MPLS Conference 1999
• Separate control planes exist for L1/2/3
• Limited communication creates isolation
• Results in an overlay network model
© 2000, Cisco Systems, Inc.
27
IP-Based Optical Control Plane
Control Plane
based on IP routing
Forwarding Plane
IP
ATM
Optical
© 2000, Cisco Systems, Inc.
28
Overlay Model (O-UNI, OIF)
Edge-LSR
LSR
ISP
LSR
Edge-LSR
Data Control Plane
(MPLS)
Optical Service
Provider
Wavelength
Router
Wavelength
Router
Optical Control Plane
(e.g. WaRP or MPLmS)
• Two Administrative Domains
Optical Transport Network
Internet Service Provider
• ISP requests circuits via a UNI interface
• OTN uses its own Control Plane for Provisioning
© 2000, Cisco Systems, Inc.
* ISP . . . Internet Service Provider
29
Peer Model (GMPLS, IETF)
Common Control Plane
(MPLmS)
Edge-LSR
LSR
OXC-LSR
OXC-LSR
LSR
Edge-LSR
Wavelength
Router
• One Administrative Domain
• LSRs and OXC-LSR are Peers
• Common Control Plane for both L3 and OTN
(full visibility of the topology at layer 3)
• reduced number of routing adjacencies
© 2000, Cisco Systems, Inc.
30
Conclusion
• MPLS TE with extensions has the potential
to be the control plane of choice for
optical transport networks
• Leverages great deal of existing software,
architecture, and operational experience
• Eases integration with IP devices
• Supported by many providers and vendors
at IETF
• Expect demonstrations 2001
© 2000, Cisco Systems, Inc.
31
MPLS, Traffic Engineering
3007
1367_06_2000_c2
© 2000, Cisco Systems, Inc.
32
MPLS Summary
• MPLS is IETF standards based
• MPLS is accepted as a technology and products of
production quality are available
• MPLS is evolving, e.g. Multicast, AToM (foo over
MPLS)
• MPLS is independent of any Datalink (ATM, FR, POS,
Ethernet,….)
• Not primarily implemented for reasons of
performance increase
• Mainly used by Service Providers, but recently also
by Enterprise customers
© 2000, Cisco Systems, Inc.
33
MPLS as Service Enabler
• VPN
• Traffic Engineering
• Sub-50ms Link/Node protection
• QoS
• foo over MPLS (Frame Relay, ATM,
Ethernet)
© 2000, Cisco Systems, Inc.
34
FR over MPLS
Only one direction
illustrated
Port Pi
FR Cust
777
FR 101
E1
y
Port Po
E2
z
FR 202
DLCI
101
In-EdgeLSR
E1
© 2000, Cisco Systems, Inc.
In-Port
Pi
Out-EdgeLSR
E2
OutPort
Po
Out-DLCI
202
35
AAL5 over MPLS
Only one direction
illustrated
Port Pi
ATM Cust
7
3/4
E1
y
Port Po
E2
z
5/6
vpi/vci
3/4
In-ELSR
E1
© 2000, Cisco Systems, Inc.
In-Port
Pi
Out-LSR
E2
OutPort
Po
vpi/vci
5/6
36
Traffic Engineering: Motivations
What is MPLS Traffic Engineering ?
MPLS TE is an optimization tool
• Reduce the overall cost of operations
by more efficient use of bandwidth
resources by preventing a situation
where some parts of a service provider
network are over-utilized (congested),
while other parts under-utilized
© 2000, Cisco Systems, Inc.
37
Traffic engineering with Layer 2
(e.g. ATM or Frame Relay)
R2
R3
R1
PVC for R2 to R3 traffic
PVC for R1 to R3 traffic
© 2000, Cisco Systems, Inc.
38
The “Overlay” Solution (e.g. ATM)
L3
L3
L2
L3
L2
L2
L2
L3
L2
L2
L3
L3
L3
L3
L3
L3
L3
L3
Physical
Logical
• Routing at layer 2 (ATM or FR) is used for traffic
engineering
• Layer 3 sees a complete mesh. Routing at layer 3 is trivial
(but poor scalability)
© 2000, Cisco Systems, Inc.
39
MPLS as the MultiService Infrastructure:
Layer Collapsing
Applications
IP
Hard Pt-2-Pt QoS
Soft Pt-2-Cloud QoS
ATM
SDH
IP
MPLS
Admission Control
Traffic Engineering
WDM
Fast Restoration
WDM
Transport
© 2000, Cisco Systems, Inc.
40
IP Traffic engineering with Layer 3
R2
R3
R1
IP routing: destination-based least-cost routing
Path for R2 to R3 traffic
Path for R1 to R3 traffic
under-utilized alternate path
© 2000, Cisco Systems, Inc.
41
MPLS Traffic Engineering
Example
R9
R3
R2
R1
150
150
50
150
R4
150
100
R6
100
R7
Trying to route a trunk from R1 to R9 with bandwidth 75 Mbps
© 2000, Cisco Systems, Inc.
42
MPLS Traffic Engineering
Example (cont)
R3
R2
R1
150
150
50
150
R4
R9
150
100
R6
100
R7
Path Computation
CSPF computes paths that obey the various constraints
and select the “best” one (shortest metric).
Trying to route a trunk from R1 to R9 with bandwidth 75 Mbps
R2-R3 link violates constraint (BW  75) so prune it
Pick shortest path on remaining topology
© 2000, Cisco Systems, Inc.
43
MPLS Traffic Engineering
Example (cont.)
Path Setup
establishes (explicit) routes for traffic
trunks. The signaling protocol to establish
the path is RSVP (with extension).
R9
R3
R2
R1
75
50
75
R4
75
25
R6
© 2000, Cisco Systems, Inc.
150
25
R7
44
Protection/restoration in IP/MPLS
networks
• Many various protection/restoration schemes exist
today:
Optical protection
Sonet/SDH (link): offering a 50ms convergence time,
largely deployed and stable
IP (link & node): Convergence = O(sec)
MPLS TE Fast Reroute (link & node) 50 msec
convergence.
© 2000, Cisco Systems, Inc.
45
Path Protection
• Controlled by the head-end of a trunk
• Fallback to either (pre)configured or dynamically
computed path
• Once the head-end has detected that the TE LSP
has suffered a failure (through the IGP and/or
RSVP), the TE LSP is being re signalled following
the new path (if any)
© 2000, Cisco Systems, Inc.
46
MPLS TE FRR Link/Node Protection
• Controlled by the routers at ends of a failed link
(know as MPLS TE Fast Reroute).
• FRR is a local protection mechanism
* link protection is configured on a per link basis
* the resilience attribute of a trunk allows to
control whether link protection could be applied
to the trunk which offers a fine granularity
• Uses nested LSPs (stack of labels)
* original LSP nested within link protection LSP
© 2000, Cisco Systems, Inc.
47
Link protection for R2-R4 link
R9
R8
R4
R2
R5
R1
Pop
17
R6
R7
22
Setup: Path (R2->R6->R7->R4)
Labels Established on Resv message
© 2000, Cisco Systems, Inc.
48
Routing prior R2-R4 link failure
R9
R8
R4
R2
Pop
R1
R5
14
37
R6
R7
Setup: Path (R1->R2->R4->R9)
Labels Established on Resv message
© 2000, Cisco Systems, Inc.
49
Link Protection Active
R9
R8
R4
R2
R5
R1
R6
R7
On failure of link from R2 -> R4, R2 simply changes outgoing
Label Stack from 14 to <17, 14>
© 2000, Cisco Systems, Inc.
50
Link Protection Active
R8
R9
Pop 14
Swap 37->14
Push 17
R4
R2
Push 37
R5
R1
R7
R6
Swap 17->22
Label Stack:
Pop 22
R1
R2
37
© 2000, Cisco Systems, Inc.
R6
17
14
R7
22
14
R4
14
R9
None
51
MPLS TE FRR
• FRR Link protection is available today.
• FRR Node protection is the next step.
• Tools to perform TE LSP back-up placement
are under study
© 2000, Cisco Systems, Inc.
52
Thank you!
3007
1367_06_2000_c2
© 2000, Cisco Systems, Inc.
53
3007
1367_06_2000_c2
© 2000, Cisco Systems, Inc.
54