Download STARTAP_2001Mtg_StArnaud

CANARIE
CA*net 4 Planning
http://www.canarie.ca
http://www.canet3.net
[email protected]
Tel: +1.613.785.0426
Customer Empowered Network
City C
City A
Carrier Neutral IX
Condo Wavelengths
Carrier Neutral IX
City B
Condo Dark Fiber
Condo Wavelengths
What is eScience?
 The ultimate goal of e-science is to allow students and eventually members
of the general public to be full participants in basic research.
 Using advanced high speed networks like CA*net 4 and novel new concepts
in distributed peer to peer computing, called “Grids” many research
experiments that used to require high end super computers can now use the
computer capabilities of thousands of PCs located at our schools and in our
homes.
 High performance computers that are part of C3.ca can be seamlessly
integrated with eScience distributed computers using CANARIE Wavelength
Disk Drive over CA*net 4
 Allows researcher access to the significant computational capabilities of all
these distributed computers at our schools and homes
 With e-science it might be possible that the next big scientific discovery
could be by a student at your local school.
ALTA Cosmic Ray eScience
 The earth is constantly
bombarded by subatomic
particles from space, with an
energy spectrum that reaches far
higher than any terrestrial
accelerator could hope to probe.
 At the highest energies such
showers can be detected at the
Earth’s surface over areas on the
order of 100 square kilometers.
 It is believe some of these cosmic
rays were created at the creation
of the universe
 Will allow researchers to gainer a
deeper understanding of deepest
reaches of space and time
ALTA Cosmic Ray eScience
 The ALTA project is a collaborative scientific research project
involving the University of Alberta Center for Subatomic Research
and over 50 high schools across Canada in the area of cosmic ray
detection.
 Teachers and students actively contribute to the physics research
while learning about an exciting area of modern science.
 Distributed computing at schools will be required to analysize
data from sensors in near real time
 Program has now expanded into USA and soon countries around the
world




CHICOS (California HIgh school Cosmic ray ObServatory), Caltech, UC/Irvine and
Cal State/Northridge, California, USA.
CROP (the Cosmic Ray Observatory Project), University of Nebraska, Lincoln, NE,
USA.
WALTA (WAshington Large area Time coincidence Array), University of
Washington, Seattle, WA, USA.
SALTA Roaring Fork Valley area of Colorado
Neptune – Undersea Grid
Wavelength Disk Drives
 CA*net 4 will be “nation wide” virtual disk drive for grid
applications
 Big challenges with grids or distributed computers is
performance of sending data over the Internet
 TCP performance problems
 Congestion
 Rather than networks being used for “communications” they
will be a temporary storage device
 Ideal for “processor stealing” transaction intensive applications
where you don’t know where the next available processor is
located
 CFD
 Visualization
Wavelength Disk Drives
St. John’s
Regina
Calgary
CA*net 3/4
Winnipeg
Charlottetown
Montreal
Fredericton
Vancouver
WDD Node
Halifax
Toronto
Ottawa
Computer data continuously circulates around the WDD
WDD Architecture
WDD Partners:
CANARIE, Can-Sol, Viagenie
CRC, Carleton U, MACI
C3.Ca, Memorial, Dalhousie
UdeMontreal, UoToronto,
SFU, UoAlberta, BCnet
3. The SGI writes back the task onto the ring where it is
received by Forest Fire Raster Engine and results
displayed on X-Window terminal at CRC
UoAlberta
UdeMontreal Dalhousie
UoToronto
Memorial
SFU
CRC
WDD Node
Vancouver
WDD Node
WDD Node
Calgary
Halifax
WDD Ring on CA*net 3
Forest Fire
Modeling
Raster Engine
1. Forest Fire Modeling Raster Engine injects
64K x 64K raster computational tasks into
WDD ring
2. Tasks circulate in WDD ring and first available
SGI processor removes next task out of the ring and
completes computation
Forest Fire Modeling eScience
 Emergency officials and civic defense
officials need to model forest fires in
real time
 But each forest fire model may take
hours to compute
 By utilizing thousands of distributed
computers at our schools and
Wavelength Disk Drive on CA*net 4
network forest fire models in near
real time
 First prototype to be demonstrated on
CA*net 3 in May using 256 SGI
processors across the country on
WDD
CA*net 4 Possible Architecture
Layer 3 aggregation service
Optional Service Available to any GigaPOP
St. John’s
Calgary Regina
Winnipeg
Large channel
WDM system
Charlottetown
Vancouver
Europe
Montreal
Customer controlled
Seattle optical switches
Fredericton
Halifax
Ottawa
Chicago
Toronto
New York
STAR LIGHT Interconnection?
 We see STAR LIGHT, CA*net 4, DTF and Vancouver Transit exchange facing
same design issues
 How do we signal interconnect wavelengths (SDH/SONET subchannels) between
STAR LIGHT participants?
 Like STAR TAP we will probably need a mix of Layer 1-3 solutions
 Layer 1 cross connect ATM plus
 Layer 3 router and/or route server
 Current ATM approaches
 Full mesh ATM like current STAR TAP
 Not possible with wavelengths or SDH/SONET channels
 PVC created on demand
 E.g Peer maker at MAEs
STAR LIGHT Options
 Layer 0 - Patch panel or optical switch
 Needs common wavelength and protocol
 Not easily subject to change and will not allow multiple peers
 Layer 1 - SDH/SONET cross connect switch
 Issues related to how identify and address SDH/SONET channels
 Layer 2 - GMPLS using IP and SONET/optical switch
 Main thrust of industry –see Juniper/Nortel, Accelight, Cisco, NTT
 Requires significant centralized management
 Layer 2 -Map SDH/SONET channels to GbE channels & use GbE switch
 Layer 3 - Each network terminates on its own router & routers meshed
together
 N squared meshing
 Layer 3 - BIG ROUTER
 Will it scale and needs central management and AS
 Layer 4 – OBGP with CWDM with optical switch
 Each CWDM wavelength mapped to SDH/SONET channel
 Control of switch is by research networks
OBGP Status Report
 OBGP first draft submitted to IETF
 Prototype working at Carleton U
 We want input on next steps for OBGP and see if it will fit within
STAR LIGHT plans
 Key features:
 SDH/SONET & Optical cross connects controlled by attached networks
 SDH/SONET & Optical cross connects identified by IP addresses & AS
 RPSL with OON extensions is database used to query who is connected
at switch and at what port
 BGP OPEN message is used like Peer maker to request optical peering
across the switch
 BGP UPDATE message and community Tags ( and maybe GMPLS)
will be used to setup multihop wavelengths
OBGP
 Proposed new protocol to support control and management of wavelengths
and optical switch ports
 Control of optical routing and switches across an optical cloud is by the
customer – not the carrier – true peer to peer optical networking
 Use establishment of BGP neighbors or peers at network
configuration stage for process to establish light path cross connects
 Customers control of portions of OXC which becomes part of their
AS
 Optical cross connects look like BGP speaking peers – serves as a
proxy for link connection, loopback address, etc
 Traditional BGP gives no indication of route congestion or QoS, but with
DWDM wave lengths edge router will have a simple QoS path of
guaranteed bandwidth
 Wavelengths will become new instrument for settlement and exchange
eventually leading to futures market in wavelengths
 May allow smaller ISPs and R&E networks to route around large ISPs that
dominate the Internet by massive direct peerings with like minded
networks
Wavelength Scenarios
Workstation to Workstation Wavelength
CWDM
GigaPOP to GigaPOP Wavelength
Regina
BCnet
Vancouver
Campus
OBGP
switch
Winnipeg
St. John’s
RISQ
Halifax
Calgary
Montreal
Seattle
Toronto
Wavelength Setup
AS 2- AS 5 Peer
AS 3
12
10
University
Regional Network
3
13
AS 1
2
15
4
AS 5
14
AS 1- AS 6 Peer
AS 2
5
7
9
1
AS 4
Regional Network
Dark Fiber
8
AS 6
6
ISP router
Wavelength Object owned by primary customer
Wavelength Subcontracted by primary customer to a third party
University
Wavelength Logical Mapping
AS 2- AS 5 Peer
AS 3
12
10
University
Regional Network
3
13
AS 1
2
15
4
AS 5
14
AS 1- AS 6 Peer
AS 2
5
7
9
1
AS 4
Regional Network
Primary Route
Backup Route
8
AS 6
6
University
ISP router
Resultant Network Topologies
University
BGP Peering on switches
at the edge
Packet Forwarding in
the core
9
AS 1
15
Regional Network
13
12
8
AS 5
1
2
10
7
2
3
10
1
7
14
9
AS 2
8
AS 6
5
6
5
Regional Network
Potential OBGP Peering
ISP router
University
OBGP Variations
1.
OBGP Cut Thru

2.
OBGP Optical Peering

3.
External router controls one or more switch ports so that it can establish direct
light path connections with other devices in support peering etc
OBGP Optical Transit or QoS

4.
OBGP router controls the switch ports in order to establishes an optical cut
through path in response to an external request from another router or to carry
out local optimization in order to move high traffic flows to the OXC
To support end to end setup and tear down of optical wavelengths in support of
QoS applications or peer to peer network applications
OBGP Large Scale

To prototype the technology and management issues of scaling large Internet
networks where the network cloud is broken into customer empowered BGP
regions and treated as independent customers
OBGP Optical Peering
 Primary intent is to automate BGP peering process and patch panel process
 Operator initiates process by click and point to potential peer
 Original St. Arnaud concept
 Uses only option field in OPEN messages
 Requires initial BGP OPEN message for discovery of OBGP neighbors
 Virtual BGP routers are established for every OXC and new peering
relationships are established with new BGP OPEN message
 Full routing tables are not required for each virtual router
 No changes to UPDATE messages
 No optical transit as all wavelengths are owned by peer
 Uses ARP proxy for routers on different subnets
 Wade Hong Objects concept







Uses an external box (or process) to setup optical cross connects
SSH is used to query source router of AS path to destination router
Each optical cross connect is treated as an object with names given by AS path
Recursive queries are made to objects to discover optical path, reserve and setup
NEXT_HOP at source router is modified through SSH
End result is a direct peer and intermediate ASs disappear
Requires all devices to be on same subnet