Download Slide

Metropolitan IP Transport Networks
Eric A. Voit
Distinguished Member of Technical Staff
Technology
Verizon
Note
November 8th, 2000
View Slides in PowerPoint
‘Slide Show’ format
Disclaimer: The views expressed herein are those of the author, and do not necessarily reflect the position of Verizon.
 Copyright Verizon Communications, 2000. All Rights Reserved
eav
Page 1
Agenda
• Economic drivers and architectures for
Metropolitan IP Networks
• The importance of efficient facilities utilization
• More on IP QoS
eav
Page 2
Long Live IP
• IP divorces applications from transport
– Value added services will be IP based
– IP allows services to be delivered independently of the data link technology
– Customers want to exploit price-performance curves of access technologies while
having each of these technologies inter-operate
• IP QoS has arrived
– IP QoS mechanisms can be applied to specific applications of individual customers
– IP QoS is superior to ATM QoS for IP transport
• IP will be used for networks, ATM remains a viable link layer technology
– DLECs currently provide integrated IP services using ATM transport. As lower cost link
layer technologies are deployed (such as Gigabit Ethernet/Fiber), DLECs will abandon
ATM.
eav
Page 3
Review of Router Types and Business Segments
Existing
(Layer 2)
SP
AR
R
"Peer"
ISP
Portal
ISP
AR
IR
R
Enterprise
DLEC Metro
IP Transport Network
Interconnection
Routing
Wire
Center
AR
Access
Router
(R)
Access
Router
(AR)
Interconnection
Router (IR)
–––
–––
–––
––
eav
Provides
efficient
transport
between
of the network
Provides
(EndIPOffice)
access
to thenodes
network
Provideslocal
interconnection
with
external
IPIPnetworks
Ensures
proper
treatment
of
QoS
Customer
VPN
Supports
a variety
access technologies
(e.g.
(DSL)
, Fast Ethernet,
FR)
Ensures the
the
properof
treatment
of traffic
traffic to
to meet
meetATM
QoS and
and
Customer
VPN needs
needs
AR
IR must
also
perform
functions
the
Ensures
the
proper
treatment
ofthe
traffic
based
anRouter
individual
interface
Off and
the shelf
equipment
(potentially
uses
theofon
same
platform ascustomer
the Router)
Off the
the shelf
shelf equipment
equipment
Off
Page 4
A New Yorker’s View of the World
eav
Page 5
A DLEC View of the IP World
ISP Application Services
Chat
Web
Hosting
V/IP
GK
eMail
DLEC Metro IP Application Services
ISP Networks
Application
AOL
Transport
R
R
UUNET
R
R
DLEC Metro IP Network
R
R
Residence
eav
SBC
R
GTE-I
R
R
Sprint
R
R
R
R
R
Yahoo
R
R
R
R
R
Business
Page 6
Network Topology and Server Placement
•
•
•
•
• Control Systems
• High bandwidth • Very high bandwidth
Low volume services
• data
Low Latency
• Low capital per installation
Minimal QoS needs• Quickly changing
• base
LATA wide resources
• Local PSTN interconnection
• Limited local operations
World wide customer
• Expensive platforms
• Security / Network demarcation
Embedded applications
Off Net
Centralized
Distributed
Local
Efficient Design of
Layer 1 & 2 Transport Network is Critical
eav
Page 7
Scope of Metropolitan IP Network
DLEC
Hosted
Applications
DLEC Metro
IP Network
ISP Networks &
Applications
Application SLA
Service
User
SCOPE
Service Level Agreements (SLA) at IP
Network Interfaces are Negotiated based
on Application Specific Needs
eav
Latency
Jitter
Packet Loss
Packet Sequence
Bandwidth
Security
Availability
Page 8
End-to-End Service Level Agreements
Wire Center
Voice / IP
Gateway
50 MS
AR
Wire Center
IP Transport
Network
80 MilliSeconds (MS)
AR
Voice / IP
Gateway
50 MS
PSTN
PSTN
Application
Provider
•
•
eav
Transport
Provider
Application
Provider
An application layer budget needs to be partitioned between sub-systems
SLAs need to be measurable and actionable
Page 9
AR Integrates Diverse Transport Technologies
Residence
Central Office
DSLAM
Modem
Bank
WASL
Tower
DSL
PSTN Circuit
Broadband Wireless
ATU-R
Modem
WASL
Any Link Layer (2) Technology
AR
ATM
Laser
Rack
Frame Relay
Fast & Gig Ethernet / Fiber
Customer
R
SMDS
SONET
ADM
Packet over SONET
Business
Wire Center
•
Access Router acts as universal IP edge device for diverse customer access methods
–
•
•
eav
Inexpensive equipment from a highly competitive evolving marketplace
Access technologies only supported as new services demand them (incremental roll-out)
Sharing the IP WAN infrastructure allows AR to push further to the edge
Page 10
The Key to Access Scalability
Decoupling Logical and Physical Link Layer Termination
Central Office
One Fiber
Dozens of Twisted Pair
• Router Port 1
Layer 2
identity based
on VPI/VCI
– Multiple ATM VCs
from a DSLAM
• Router Port 2
Several Fibers
(redundancy)
Dozens of Feeder Fibers
Hundreds of Drop Fibers
•
AR
eav
– Ethernet access
‘Channelized’ for
Fiber to the home
Router Traffic engineering is
based on offered load and the
number of logical connections,
not physical port limitations
Page 11
Metropolitan IP Routing Topology
Engineering Decisions
•
Logical
– Redundancy and failover
supported by proven
routing protocols &
implementations
– Opportunity to efficiently
route local and
InterLATA IP traffic
•
ATM
AR
AR
Gigabit
Ethernet
over
Fiber
AR
AR
Packet
over
SONET
AR
AR
AR
AR
AR
R
R
Physical
– Router connections
engineered using the
most efficient / expedient
transport alternatives
– Can change without
impacting logical design
eav
Local Area
Wire Center 1
AR
AR
AR
Wire Center
IR
N
AR
ATM
IR
Page 12
Routing and Interconnection
Logical Topology
Metro Boundary
•
Metro is built from
multiple local areas
Local Area 4
AR
Local Area 1
AR
R
AR
AR
R
R
OSPF
AR
R
AR
PF
AR
PF
OS
OS
OSPF
AR
PF
R
R
F
O SP
OS
IR
PF
OS
O SP
F
OS
PF
O
R
SP
F
OSPF
R
IR
BGP4
O
SP
F
P4
BG
Existing
P4
Enterprise
Local
Area 2
R
(Layer 2)
SP
Video
"Peer"
ISP
Portal
ISP
eav
SR
AR
Content services can
be centralized, or
distributed to the edge
Interconnection with
ISPs is centralized to
minimize operations
complexity
Video
Voice
AR
AR
BG
•
AR
Local Area 3
F
OSP
•
Local areas are
connected to ensure
redundancy and
performance
BGP4
•
Vertical Services Domain
Voice
Other
Vertical Services Domain
Page 13
Network Topology and the Importance
of Efficient Facilities Utilization
eav
Page 14
1996-1999 Data Networks
Which of the following statements are true:
• IP, ATM, and SONET layers exist as

independent aggregations of signaling,
transport, and operations protocols and
equipment
Customer
IP LAN
Customer
IP LAN
Network Provider
ATM / FR
Network Provider
SONET
• This layering results from historical

accident
•
For networks carrying IP, this layering
is based on sound long term
engineering principles & economics
Network Provider
Fiber
eav
Page 15
Architectural Drawbacks for Local Data
Example: Two customer routers, both connected to a Layer 2 Cloud via PVCs
Layer 2 Switch
•
S
x2
•
Can require more than 6x the SONET ring bandwidth of
an IP/SONET connection
Why? SONET was designed assuming a local PSTN
switch that could choose a local trunk group. Star data
architectures are unable to leverage this ability.
–
–
•
At some point, local data volumes become large enough
for the economics to favor the deployment of data
switches and routers closer to the edge of the network
–
–
–
R
R
The current architecture was developed when data volumes
were low, switches were expensive, and operations
procedures and expertise was evolving
The current architecture is best when data traffic isn’t local
Any economic analysis would have to measure the amount
and characteristics of the local data to determine the
opportunity for savings
The analysis would then focus on facilities cost versus
switch placement, operations, and maintenance costs
An unlikely analysis conclusion would be to simply
distribute more switches and routers across our existing
SONET …
Customer owned Routers
eav
Page 16
Installing a Switch (or Router) on Every SONET
Ring Doesn’t Completely Solve the Problem
Network
Provider
Network Provider
S
ADM
ADM
ADM
ADM
SONET
PV SONET
Co
rS
VC
ADM
ADM
R
R
Customer
Customer
•
R
or
or
S
S
PV C or
SV C
ADM
ADM
x3
x2 Bandwidth
PVC or SVC
ADM
CC
PPVV
CC
SSVV
oorr
R
R
Customer
Customer
•
If a switch or router was placed on every SONET ring, you would still double the data bandwidth
required between two local offices (compared to a direct SONET connection)*
Adding a second switch on the ring in some cases will TRIPLE the SONET bandwidth required
•
Optimizing SONET utilization (for local data) requires switching capability in every C.O.
–
–
–
–
Optimizing SONET costs doesn’t mean you have optimized total service delivery cost
Today operational, service, and complete network topology roadblocks hinder such a network configuration
Architectures which address these problems are emerging
Price points for equipment and operations are changing, and are different than when Fast Packet services were
first deployed
* This example intentionally ignores the benefits of multiplexing traffic to multiple destinations, both local & remote.
eav
Page 17
SONET also ‘Wastes’ Bandwidth
C.O.
'Tandem'
C.O.
OC3
AR
ADM
OC -4 8
ADM
R
OC3
ADM
AR
C.O.
OC3
ADM
AR
OC3
Central Office
•
This fiber run is carrying an OC-12 worth of ‘wasted’ bandwidth.
With traditional SONET, you cannot use this available capacity
to transport low priority traffic.
(Note: this is not PoS interface protocol issue.)
eav
Page 18
Gigabit Ethernet & 10GBE over Fiber
C.O.
AR
Enabling Factors:
• Dark fiber
• New Routers
'Tandem'
C.O.
R
AR
C.O.
AR
– ASIC & FPGA
– Low latency
– Big backplanes
(50+ GB/s)
– Line Speed QoS
• Layer 3/4
– 802.3ad
– Interrupt driven
failure recovery
Central Office
When is the cost of leased fiber (over diverse paths) lower than SONET?
eav
Page 19
Ethernet / Fiber Organizational Interfaces
Access
& IOF
Laser
Rack
Fiber
Frame
Fiber
Frame
Access
& IOF
Laser
Rack
802.3ad
R
Central
Office
802.3ad
Central
Office
R
IP/Gigabit Ethernet/Fiber
•
eav
Different Organizations would manage the Routers and the Transport Layer
• Different skill sets for Operations
• Useful life of long haul Lasers is longer than life of Router
• Ethernet provides a simple interface within the C.O.
• Same network interface for Router to Router and Router to Customer
communications
• DWDM can be deployed to transport multiple Ethernet handoffs from the Routers
• Regulatory boundaries
Page 20
DLEC Local Topology & Existing Tariffs
DLEC Local Topology
C.O.
AR
'Tandem'
C.O.
Verizon Transport Alternatives
• ATM
to DLEC
Backbone
– DS3
– OC3
R
•
SONET
– DS3, OC3 on shared ring
– Dedicated OC48 ring
(OC12 drops to AR)
AR
C.O.
•
Dark Fiber
AR
Central Office
Airline perimeter around the four C.O.s
Urban
(10 miles)
eav
Rural
(50 miles)
Page 21
Verizon Recurring Monthly Tariffs
(FCC Tariff #1, 3 year commitment)
OC3
DS3
*
ATM
Distance to ATM hub
Price
Distance to ATM hub
Price
0-5 miles
5-25 miles
25-50 miles
$2,460
$3,360
$5,645
0-5 miles
5-25 miles
25-50 miles
$4,965
$6,610
$8,790
(UNI to Cell Relay Cloud)
FCC#1 16-6-1 (A)
SONET
(point to point)
ISAN (Shared Ring)
Per IEF connection $636
0-3 miles $2,500
4-20 miles $3,700
20+ miles $5,500
FCC#1 7-9-20 & 7-9-21 (shared)
Dark Fiber
OC12
OC192
Unavailable
Unavailable
FCC#1 16-6-1 (A)
ISAN (Shared Ring)
Per IEF connection $2,493
0-3 miles $5,600
4-20 miles $7,800
20+ miles $10,200
FCC#1 7-9-20 & 7-9-21 (shared)
not offered
technical issues
Dedicated OC48
(w/ OC12 Drops at Node)
Ring connectivity: $7,900
Per mile charge: $639
FCC#1 7-5-19 (dedicated)
Unavailable
not offered
4 fiber strands = $1200 per mile
(point to point)
FCC#1 7.5.10
* Per VC Charges excluded
eav
Page 22
More on IP QoS
eav
Page 23
Why IP QoS is Superior to IP/ATM QoS
• Applications talk IP
• IP routers can now identify IP application layer traffic flows, and prioritize
them across the LAN (i.e. QoS)
• Supporting IP flow QoS in the WAN is now becoming viable
• If IP QoS is deployable, having an intervening ATM QoS abstraction is
redundant, unnecessarily restrictive, and costly
–
–
–
–
QoS prioritization is done on the AR
Eliminates the need for the different types of ATM pipes
Eliminates traffic management and operational complexities of different pipes
Minimizes troubleshooting between the IP and ATM layers
• Managing QoS is now something customers can outsource to the network.
They don’t have to pre-sort their IP traffic into different types of ATM
QoS differentiated VC pipes
eav
Page 24
Moderate ATM Cell Loss Can Induce
Disproportionate IP Packet Loss
DLEC discards:
Subscriber experiences loss of:
eav
IP/ATM
IP
4% of Cells
4% of Packets
40% of Packets *
4% of Packets
Page 25