Download No Slide Title - University of Windsor

High Speed Optical Networks:
An Evolution of Dependency
November 2, 2001
Todd Sands, Ph.D
WEDnet Project
www.wednet.on.ca
University of Windsor
Latency
• The result of an event in time that slows the transport
or processing of information
• E.g. Machine (processing) latency in microsecs (n
=1.2)
• E.g. Network latency in millisecs (x < 130 ms)
• Optical transport max. = 300,000 km/sec
• Physical parameters of the transport media
• Convergence of voice, image and data in the path
• Switched cells and packet network behaviours
• Potential of WDM optically switched and SONET
architectures
OSI Reference Model – Networking 101
•
•
•
•
•
•
•
Application
Presentation
Session
Transport
Network
Data-link
Physical
When two computers communicate on a network, the software at each layer on one
computer assumes it is communicating with the same layer on the other computer.
e.g. For communication at the transport layers, that layer on the first computer has no
regard for how the communication actually passes through the lower layers of the
first computer, across the physical media, and then up through the lower layers of the
second computer.
Do we know the effects of latency!

Suspect that the answer is yes! We see it every day!

No. of processors, power requirements, processing capability, storage
capacity, and the needs of research that use most facilities can be intensive.

HPCS resources supplied and funded through a needs-based process, but
this can also be because of research

What about a GRID? Is it on the same path?

Are we mindful of details, such as latency…with respect to one of the most
fundamental parts of the GRID… THE NETWORK

Do we know how computing resources connect to the outside
world?…Maybe…

Do we have any control over the “extranet”?
Primary Network
Interface
To Machine Resources
These switches provide
Ethernet to ATM SONET
WAN interfaces
for TCP/IP traffic
PACKETS VS. CELLS VS. FRAMES
•
Frames – used for larger data amounts over high-speed, low error rate links
– 2,000 – 10,000 characters in size
– Data corrections not link by link
– Therefore link by link error checking impacts network latency greatly
•
Packets – used for smaller data amounts across lower speed, high error rate links
– 128 – 256 (bytes) characters in size
– Lower chances of error in each packet, small amounts re-transmitted
– Prioritization through tagging of packets leads to QoS
•
Cells – very small amounts of data with sometimes no error checking
– Highly reliable optical networks sometimes with no error checking
– Up to 48 - 53 (bytes) characters in size
– Small size allows for load balancing of traffic on network
– No payload in cells, no transmission - full payload, then transmission
– Uses ATM Adaptation Layers – AAL’s 1-5 for shaping the network
Optical Carrier Designations
•
•
•
•
•
•
OC-1/STS-1
OC-3
OC-12
OC-48
OC-192
OC-768
51.84 Mbps
155.52 Mbps
622.08 Mbps
2,488.32 Mbps
9,953.28 Mbps
39,813.12 Mbps
SONET
• digital hierarchy based on Optical Carriers
(OC’s)
• maximum t-speed of 39.81312 Gbps
• defines a base rate of 51.84 Mbps = STS-1s
• OC’s are multiples of the t-speed
• defines Synchronous Transport Signals –
STS’s and STS-3c = OC 3 = 155 Mbps
Overheads
•
•
SONET carries 8,000 frames per second, 810 characters in size (36 characters
of overhead and 774 characters of payload
Section Overhead includes:
– STS channel performance monitoring
– Data channels for management such as channel monitoring, channel administration,
maintenance functions and channel provisioning
– Performs functions necessary for repeaters, add drop multiplexers (ADMs),
termination gear, and digital access and cross connect systems (DACS)
•
Line Overhead includes:
– STS-1c performance monitoring
– Data channel management, payload pointers, protection switching information, line
alarm signals, and far-end failure to receive indicators
•
In addition to these overheads there are also Path overheads
Optical Wave Division
• WDM multiplies (up to 32 more times) the capacity of
existing fibre spans – cross (wide)-band, narrow band or
dense band transmission options
• DWDM Red waves 1550, 1552, 1555 & 1557 nm
• DWDM Blue waves 1529, 1530, 1532& 1533 nm
• Now can support 100 wavelengths with each wavelength
supporting a channel rate of up to 10 Gbps
Local Area Access Architectures
1-Meg or xDSL Modem Services in Communities
Alternate Carrier
MANs also Interface
Central CO for Access Nodes
Access Routers
1
M
M
1
M
M
G
b
E
PVCs – on carriers network
1000 Mb
GbE
Router
OC-12
ATM
Off Ramps -WDM
ATM Network – OC12-OC48
G
b
E
System Processors and
Interfaces 100 Mb- 1Gb
Grid Access
Node – GigaPoP?

All PVCs (SVCs or PVPs) usually terminate on 1 or more Centralized Access Routers

Most carrier PVCs are UBR with access at minimum OC48 speeds 2.4 Gb/sec

Backbone may be optically switched with P.O.S on wavelengths using TCP/IP as the
main transport protocol but getting direct access to it is the key!

Direct access will also minimize latency and the synergistic effects of latency
What does a 5 minute average measurement
show us with MRTG?
Network Protocol Stack Models (WAN with IP)
PC
1MM
DBIC
1MM
Ethernet
Switch Network
(Catalyst) (ATM)
LAC
(SMS-1000)
LNS
IP
IP
PPP
PPP
PPPOE
PPPOE
Ethernet
Ethernet
Ethernet
Ethernet
Ethernet
10BaseT
1MM
QAM
1MM
QAM
L2TP
UDP
UDP
IP
IP
Ethernet
LLC/SNAP
(1483)
LLC/SNAP
(1483)
LLC/SNAP
(1483)
LLC/SNAP
(1483)
AAL5
AAL5
AAL5
AAL5
SAR
SAR
SAR
SAR
ATM
ATM
ATM
ATM
10BaseT
L2TP
ATM
SONET/SDH SONET/SDH
SONET/SDH SONET/SDH
SONET/SDH
Making a Call
Video
Video
Television
V-Room
HDGH
Television
WEDnet uses WUC as
a carrier such as Bell
or METROnet with core gear
LS1010 and 7200 series for
ATM and IP routing
WRH Western
Campus
V-Room
LE25
LE25
LE25
OC3
S
D
LE 25 SMF
25 Mb ATM
FVC VGATE
IBM 8274 9 slot
FVC V-room
P-Tel Video
University
of
Windsor
Dial - up
Shared
H.261 ISDN
Making a Call
Video
Video
Television
V-Room
HDGH
Television
WEDnet uses WUC
as a carrier such
as a Bell or METROnet
with core gear LS1010 and
7200 series for ATM and
IP routing
WRH Western
Campus
V-Room
LE25
LE25
LE25
OC3
S
D
LE 25 SMF
25 Mb ATM
FVC VGATE
IBM 8274 9 slot
FVC V-room
P-Tel Video
University
of
Windsor
Dial - up
Shared
H.261 ISDN
Making a Call
Video
Video
Television
V-Room
HDGH
Television
WEDnet uses WUC as
a carrier such as Bell
or METROnet with
core gear LS1010 and 7200
series for ATM and
IP routing
WRH
Western
Campus
V-Room
LE25
LE25
LE25
OC3
S
D
LE 25 SMF
25 Mb ATM
FVC VGATE
IBM 8274 9 slot
FVC V-room
P-Tel Video
University
of
Windsor
Dial - up
Shared
H.261 ISDN
Codec Negotiation
Video
Video
Television
V-Room
HDGH
Television
WEDnet uses WUC as
a carrier such as a Bell
or METROnet with core gear
LS1010 and 7200 series for
ATM and IP routing
WRH
Western
Campus
V-Room
LE25
LE25
LE25
OC3
S
D
LE 25 SMF
25 Mb ATM
FVC VGATE
IBM 8274 9 slot
FVC V-room
P-Tel Video
University
of
Windsor
Dial - up
Shared
H.261 ISDN
Successful Call
Video
Video
Television
V-Room
HDGH
Television
WEDnet uses WUC as
a carrier such as a Bell
or METROnet with core gear
LS1010 and 7200 series for
ATM and IP routing
WRH
Western
Campus
V-Room
LE25
LE25
LE25
OC3
S
D
LE 25 SMF
25 Mb ATM
FVC VGATE
IBM 8274 9 slot
FVC V-room
P-Tel Video
University
of
Windsor
Dial - up
Shared
H.261 ISDN
Making an ISDN Call
Video
Video
Television
V-Room
HDGH
Television
WEDnet uses WUC
as a carrier such as a Bell
or METROnet with core gear
LS1010 and 7200 series for
ATM and IP routing
WRH
Western
Campus
V-Room
LE25
LE25
LE25
OC3
S
D
LE 25 SMF 25 Mb ATM
FVC VGATE
IBM 8274 9 slot
FVC V-room
P-Tel Video
University
of
Windsor
Dial - up
Shared
H.261 ISDN
Making an ISDN Call
Video
Video
Television
V-Room
HDGH
Television
WEDnet uses WUC as
a carrier such as a Bell
or METROnet with core
gear LS1010 and 7200
series for ATM and IP
routing
WRH
Western
Campus
V-Room
LE25
LE25
LE25
OC3
S
D
LE 25 SMF 25 Mb ATM
FVC VGATE
IBM 8274 9 slot
FVC V-room
P-Tel Video
University
of
Windsor
Dial - up
Shared
H.261 ISDN
Making an ISDN Call
Video
Video
Television
V-Room
HDGH
Television
WEDnet uses WUC as
a carrier such as a Bell
or METROnet with core
gear LS1010 and 7200
series for ATM and IP
routing
WRH
Western
Campus
V-Room
LE25
LE25
LE25
OC3
S
D
LE 25 SMF 25 Mb ATM
FVC VGATE
IBM 8274 9 slot
FVC V-room
P-Tel Video
University
of
Windsor
Dial - up
Shared
H.261 ISDN
Making an ISDN Call
Video
Video
Television
V-Room
HDGH
Television
WEDnet uses WUC
as a carrier such as a Bell
or METROnet with core gear
LS1010 and 7200 series for
ATM and IP routing
WRH
Western
Campus
V-Room
LE25
LE25
LE25
OC3
S
D
LE 25 SMF 25 Mb ATM
FVC VGATE
IBM 8274 9 slot
FVC V-room
P-Tel Video
University
of
Windsor
Dial - up
Shared
H.261 ISDN
Codec Negotiation
Video
Video
Television
V-Room
HDGH
Television
WEDnet uses WUC
as a carrier such as a
Bell or METROnet with core
gear LS1010 and 7200 series
for ATM and IP routing
WRH
Western
Campus
V-Room
LE25
LE25
LE25
OC3
S
D
LE 25 SMF
25 Mb ATM
FVC VGATE
IBM 8274 9 slot
FVC V-room
OC3
P-Tel Video
DEC Gigaswitch
18 gbps
Dial - up
Shared
H.261 ISDN
University
of
Windsor
Video
Television
Leamington District
Memorial Hospital
Centrex module
Successful Call
Video
Video
Television
V-Room
HDGH
Television
WEDnet uses WUC as
a carrier such as a Bell
or METROnet with core gear
LS1010 and 7200 series for
ATM and IP routing
WRH
Western
Campus
V-Room
LE25
LE25
LE25
OC3
S
D
LE 25 SMF
25 Mb ATM
FVC VGATE
IBM 8274 9 slot
FVC V-room
P-Tel Video
Dial - up
Shared
H.261 ISDN
University
of
Windsor
Video
Television
Leamington
District Memorial
Hospital
Centrex module
AT&T and Regional Gigapop
IP Architecture
CA*net3
AT&T Gigapop
BGP
AT&T Route
Server
ATM
/w SVC
Regional IGP
OCRINet /wOHI
AT&T
Network
Regional IGP
WEDNet
Regional IGP
SureNet
CA*Net AS iBGP
ATM interconnectivity
Router / RFC1577 Client
LAN interconnect
AT&T AS iBGP
Bhavani Krishnan
From LAN to WAN
This server and control
facility houses multiple
Digital Alpha, DEL
PowerEdge, IBM Netfinity
and RS/6000 servers.
Located at a single
campus the facility
supports 400 nodes
locally and 800 nodes 7.5
km away. SVCs are
provisioned on separate
PVPs for security and
LANE services provide
VLANs for ADT systems,
pharmacy, and document
imaging. The systems
use GUI interfaces to
assist visual references
for end-users
In the Ideal World!

Dark fibre between nodes

Homogenous switched architecture with minimal breakouts

Low latency at all layers

We will likely be dealing with something much different, unless there is about
$500 M available to support and sustain the network side of grids to help
minimize the synergistic effects of latency on applications

Latency studies are important and the synergy of latency effects are
important from the processor to the I/O architectures, to the network layers

If commercial carriers are to be used anywhere in the path, latency should
become a factor for selecting them as providers

Effective monitoring and support of the extranet is important to the success
of a GRID unless the GRID middleware can accommodate different types of
latency and the variation that exists

Internet routing is “best effort” with variable paths every time – not likely the
best GRID platform

Research networks like CA*net 3, Internet 2, ORION, etc. are the next best
bet! However, the last mile issue still has to be addressed.
The Future
•
“It is conceivable that future Internet networks may be a seamless composite of a variety of transport
protocols. An Optical Internet might be used for high volume, best efforts computer to computer
traffic, while IP over ATM might be used to support VPNs and mission critical IP networks, while IP
over SONET would be used to aggregate and deliver traditional IP network services that are
delivered via T1s, DS3s, and Gigabit uplinks”
•
From, Dr. Bill St. Arnaud, CANARIE