Download cesnet2 - CzechLight

Customized Approaches to
Fibre-based E2E Services
www.ces.net
Jan Radil, Stanislav Šíma
[email protected]
Customized Approaches to Fibre-based E2E Services
Outline
First attempts and trials
Up-to-date situation in CESNET
Conclusions
2 Dec, 2008
TERENA 1st E2E Workshop, Amsterdam
2
Customized Approaches to Fibre-based E2E Services
First attempts and trials
For E2E services – dark fibre is really needed.


First mile issues with fibres in many places.
Wireless can be used too but limitations may be rather
significant (no WDM, 10G predicted, but higher speeds?,
reliability and availability...).
First DF in CESNET in 1999, 2.5 Gb/s PoS.

Ethernet-based E2E services, shared capacity, VLANs (not E2E
really).

More DFs have been acquired since then.
First international ‚lambda‘ link in the beginning 2003.

Connected to NetherLight, Amsterdam, the first international
lambda exchange point of GLIF (now GOLEs ), 2.5 Gb/s SONET.

Ethernet-based E2E services, dedicated EoSONET channels.
2 Dec, 2008
TERENA 1st E2E Workshop, Amsterdam
3
Customized Approaches to Fibre-based E2E Services
First attempts and trials
Both first (national and international) 2.5 Gb/s links
were single channels, no CWDM or DWDM,
Evident limitation for E2E circuits/channels/lambdas
when DFs are available.

‚Grey‘ router interfaces with PoS only, no GE.

SONET/SDH repeaters only, no optical amplifiers.
Problem with international links because no DFs were
available at that time (this situation has changed).
But the national infrastructure (MAN/WAN) started to
utilize DFs more and more and CWDM/DWDM
technology became available and economicaly feasible.
2 Dec, 2008
TERENA 1st E2E Workshop, Amsterdam
4
Customized Approaches to Fibre-based E2E Services
Progress with First attempts and trials (GLIF)
GLIF links for new experiments with ‚lambdas‘ (ie E2E
circuits).
The 2.5 Gb/s GLIF link to NetherLight upgraded to 10
Gb/s in September 2004.
Another direct 10 Gb/s GLIF link to StarLight, Chicago
from December 2006.
2 Dec, 2008
TERENA 1st E2E Workshop, Amsterdam
5
Customized Approaches to Fibre-based E2E Services
Progress in CESNET networks
Evolution of the CESNET network 1999 – 2007 (not with all details).
2000
2001
2005
2007
2002: first optical amplifiers (EDFA)
2 Dec, 2008
2005
2007
2004: first CLA
TERENA 1st E2E Workshop, Amsterdam
6
Customized Approaches to Fibre-based E2E Services
Up-to-date situation in CESNET: WAN
CESNET2 network 2008

4700 km of DFs

Including 980 km
single DF links
2 Dec, 2008
TERENA 1st E2E Workshop, Amsterdam
7
Customized Approaches to Fibre-based E2E Services
Up-to-date situation in CESNET: WAN
10 links Cisco 15454 MSTP n x 10 Gb/s DWDM
transmission system (with ROADM).
10 links (including 3 single fibre lines) deployed with
open photonic n x 10 Gb/s DWDM transmission systems.
5 other links will be lit in December 2008 (including 3
single fibre lines).
2 international CBF connections to Bratislava and Wien:
open photonic n x 10 Gb/s DWDM transmission system.

No bottleneck on international links anymore.
Customized Approaches in WANs.
2 Dec, 2008
TERENA 1st E2E Workshop, Amsterdam
8
Customized Approaches to Fibre-based E2E Services
Up-to-date situation in CESNET: WAN
DFs with DWDM, CWDM not recommended.

Problems with amplification and number of channels.

But 10G CWDM transceivers available for interesting prices.
 If distance is not an issue, use CWDM technology.
Equipment from different vendors in one network.

Open photonic DWDM systems are manufactured by photonic
companies/vendors.

Not a ‚home made‘ black box without support and guarantee.

ROADMs and OOO switches have been developed and tested
and can be deployed too, it is not restricted to ‚big vendors‘.

Booths both on OFC2008 and ECOC2008 – the global scope
company.
Customized Approaches in WANs.
2 Dec, 2008
TERENA 1st E2E Workshop, Amsterdam
9
Customized Approaches to Fibre-based E2E Services
Up-to-date situation in CESNET: WAN
OFC2008
2 Dec, 2008
ECOC2008
TERENA 1st E2E Workshop, Amsterdam
10
Customized Approaches to Fibre-based E2E Services
Up-to-date situation in CESNET: MAN
Open photonic DWDM/CWDM transmission systems are
used for end-user 1 Gb/s or 10 Gb/s wavelength
connections, if needed.

Open DWDM/CWDM systems are used because of economic
reasons, they are not that expensive but provide all required
demands.
Particle physics research premises in Praha, along with a
number of hospitals.
10 Gb/s wavelength connection to the Barrandov
Studios under construction.
Customized Approaches in MANs.
2 Dec, 2008
TERENA 1st E2E Workshop, Amsterdam
11
Customized Approaches to Fibre-based E2E Services
Up-to-date situation in CESNET: MAN
With FTTx deployment, usage of metropolitan fibres lit
by high speed open photonic DWDM is expected to be
common.

Economic reasons.
‚Openness‘ enables the solution of physical layer
interoperability issues and decreases expenses as well
as reducing equipment delivery time.

One example - equalizing power levels between two DWDM
systems during GLIF 2007 demos.
Demos07
2 Dec, 2008
TERENA 1st E2E Workshop, Amsterdam
12
Customized Approaches to Fibre-based E2E Services
Applications and E2E Connections I.
There are neither applications or E2E connections
without physical layer (dark fibre) and photonic
transmission technology.
Let‘s applications (end users/researchers) decide what
technologies really fit their needs.

Meetings/workshops to share information.
We see demand for E2E services but rather permanent
or semi-permanent, no on-demand.

Fibre in first miles is the most difficult issue.

With CWDM/DWDM it is easy to provide connectivity 24/7.
 So reservation systems not have to be necessary.

Bottleneck are international (transatlantic) links.
2 Dec, 2008
TERENA 1st E2E Workshop, Amsterdam
CESNET2
13
Customized Approaches to Fibre-based E2E Services
Applications and E2E Connections II.
GLIF demo on 8th Annual Global LambdaGrid Workshop 2008, Seattle,
USA.

HD video with real optical multicast.
GLIF demos on 7th Annual Global LambdaGrid Workshop 2007, Praha,
Czech Republic.

HD, 4k video transmissions.

Multiple 10 Gb/s links to a medieval building with open DWDM
transmission system.
HEP - data access and processing for ATLAS and ALICE on LHC, D0 on
TEVATRON, STAR on RHIC.
First VINI sites in Europe (Praha, Plzeň).

http://www.vini-veritas.net/
Intercontinental Remote Education on High Performance Computing
between Masaryk University Brno and Louisiana State University.
CESNET-TWAREN Hinchu lightpath: peering CESNET-TWAREN, Focused on
Multicasting and IPv6.
2 Dec, 2008
TERENA 1st E2E Workshop, Amsterdam
14
Customized Approaches to Fibre-based E2E Services
Applications and E2E Connections III.
LHC (Large Hadron Collider at CERN) distributed data simulation,
processing and physics analysis under the WLCG (Worldwide LHC
Computing Grid) project (cern.ch/lhc, cern.ch/lcg) for experiments ATLAS
(cern.ch/atlas) and ALICE (aliceinfo.cern.ch).
Experiment D0 at Fermilab data simulation and processing.

http://www-d0.fnal.gov/public/index.html
Experiment STAR at Brookhaven data simulation and processing.

http://www.star.bnl.gov/
Praha serves as Tier2 regional computing centre for HEP experiments.
Connection to Tier1 preferred centre Karlsruhe (via GÉANT2).
Connections to BNL, Fermilab, Taipei via GLIF are used as additional
connections (it is not part of LHCOPN network between LHC Tier1 centers).
Further references about Czech participation:

www.particle.cz/farm/, http://www-hep2.fzu.cz/d0/
2 Dec, 2008
TERENA 1st E2E Workshop, Amsterdam
15
Customized Approaches to Fibre-based E2E Services
GLIF 2007 demos solution
CESNET2
N x 10GE
2 Dec, 2008
DWDM
MUX
Chicago
DWDM
DEMUX
Amsterdam
CWDM
MUX
GOLE
StarLight
P
PASNET
CWDM
DEMUX
GOLE
NetherLight
CWDM
DEMUX
OC-192c
CWDM
MUX
OC-192c
B
CLA
PB01
DWDM
DEMUX
E300
λ1
λ2
λ3
λ4
λ1
λ2
λ3
λ4
DWDM
MUX
GOLE
CzechLight
Praha
Dark Fibre, G.652, 13 dB
(DWDM over CWDM)
λ1
λ2
λ3
Carolinum
λ4
λ1
Demos
λ2
λ3 PCs with XFP
λ4
Switches
Combination of

10G international lambdas, CWDM, DWDM

Open and ‚big‘ vendor transmission systems
TERENA 1st E2E Workshop, Amsterdam
16
Customized Approaches to Fibre-based E2E Services
GLIF 2007 demos solution with CLM
CL Multicast switch demo
Brno
LX 10GE optics
Comodity IP
HD camera
Plzeň
HD sender BRN
Iperf receiver PLZ
DWDM 10GE optics
HD receiver BRN
Iperf receiver BRN
LX XFP
DWDM XFP
T Ethernet
Comodity IP
DWDM XFP
DWDM XFP
Comodity IP
LX XFP
CESNET 2 DWDM
Celetna
HD receiver PRG - BRN
DWDM XFP
HD recv PRG - PLZ
DWDM XFP
CLA
HD sender PRG
CL MCAST SWITCH
DWDM XFP
DWDM over CWDM
Zikova
Iperf sender PRG
2 Dec, 2008
TERENA 1st E2E Workshop, Amsterdam
17
Customized Approaches to Fibre-based E2E Services
GLIF 2007 demos solution with CLM
2 Dec, 2008
TERENA 1st E2E Workshop, Amsterdam
18
Customized Approaches to Fibre-based E2E Services
CAVEwave
(2155)
7609
EVL
e600
CW
6506
PNWGP
IP: 10.200.200.200
MAC: 10:10:10:10:10:10
C-Wave
(2155)
CzechLight
Praha
E300
(2155)6/14
(441)6/10 (445)6/6
RX only RX only
‚TX only‘
Tx
RX only
(2155)
Rx
O2
O1
I1
CESNET
CLM
O3
C-Wave
(2155)
GLIF
Mog
HD RX
CW
6506
CESNET
(trunk 440,441)
6/2
Brno
C6506
O4
(441)
StarLight
HD RX
IP: 10.200.200.200
MAC: 10:10:10:10:10:10
(440)
HD TX
Combination of
CW
6506
6509
e1200
UCSD
Calit2
HD RX
IP: 10.200.200.200
MAC: 10:10:10:10:10:10
2 Dec, 2008

10G international lambdas, DWDM

Open and ‚big‘ vendor transmission systems
GLIF 2008 CLM Demonstration
in Cinegrid demos
TERENA 1st E2E Workshop, Amsterdam
19
Customized Approaches to Fibre-based E2E Services
GLIF Lightpaths (E2E) to HEP projects participants in the Czech Republic
GOLE
NetherLight
Amsterdam
AS – The Academy of Sciences of the
Czech Republic
n x 1GE
over SONET
CTU – Czech Technical University, Praha
OC-192c
CU – Charles University, Praha
ONS15454
Access 1GE
CWDM
DF
E300
GOLE
CzechLight
Praha
Combination of


2 Dec, 2008
Cisco6506
CWDM
DF
Nuclear Physics Institute
(AS) Bulovka, Praha
10GE
CWDM
Institute of Experimental
and Applied Physics (CTU)
Horská, Praha
Institute of Physics
(IoP AS), Na
Slovance, Praha
CWDM
Faculty of Mathematics
and Physics (CU), Trója, Praha
Trunk 10GE
FNAL
Faculty of Nuclear Sciences
and Physical Engineering (CTU)
Břehová, Praha
BNL
DF
TAIPEI
Nuclear Physics Institute (AS)
Řež
TERENA 1st E2E Workshop, Amsterdam
10G international lambdas,
CWDM
Open and ‚big‘ vendor
transmission systems
20
Customized Approaches to Fibre-based E2E Service
GOLE
CzechLight
Praha
1GE
GOLE
NetherLight
Amsterdam
1GE over
SONET
E300
1GE over
SONET
vinix.cesnet.cz
1GE
OC-192c
1GE
ONS15454
OC-192c
Lightpath Chicago/Amsterdam/Praha for VINI
GOLE
StarLight
vini1.cesnet.cz
1GE
simple extension
possibilities
CESNET2
PoP Praha
CESNET EF
n x 10Gb/s
Chicago
CESNET2
n x 10Gb/s
Plzeň
Brno
Combination of


vini network in the USA
10G international lambdas,
DWDM
Open and ‚big‘ vendor
transmission systems
CESNET
ONS15454
CESNET2
ONS15454
CESNET2
ONS15454
1GE
1GE
1GE
viniy.cesnet.cz
2 Dec, 2008
viniz.cesnet.cz
TERENA 1st E2E Workshop, Amsterdam
vini2.cesnet.cz
21
Customized Approaches to Fibre-based E2E Services
TWAREN-CESNET via
Taiwan/USA/Amsterdam/Praha
Applications
TWAREN
Combination of


10G international lambdas
Open and ‚big‘ vendor
transmission systems
TAIWANLight/T
WAREN
10GE
GOLE
CzechLight
Praha
Taiwan
OC-12c
Applications
CESNET2
New York City
GOLE
NetherLight
Chicago
Palo Alto
OC-12c
OC-192c
Amsterdam
Los Angeles
2 Dec, 2008
TERENA 1st E2E Workshop, Amsterdam
22
Customized Approaches to Fibre-based E2E Services
TWAREN-CESNET via Taiwan/USA/Amsterdam/Praha
Taiwan
U.S.
drawed by TWAREN/SURFnet/SARA/CESNET
Europe
Amsterdam
TAIWANLight
/TWAREN :
ChicagoPalo AltoLATaipeiHsinchu
15454
@Chicago CA*net4 :
NYC-Chicago
Back to back
fiber
15454
@Palo Alto
15454
@Taipei
I2 HDXc
@MANLAN
OME 6500
OME 6500
Asd001a_ome02
Asd001a_ome07
OME 6500
Applications
CESNET2
5002LE-TWHSZCZPRG_NL(NBDTWAREN-CESNET)
Back to back
fiber
7609P
@Hsinchu
2 Dec, 2008
GE2/9
BigIron
@Praha
TE1/3
IEEAF/VSNL
AMS/TXO2- OME 6500
I2 ONS NYC/TX01 Asd002a_ome04
@MANLANOC12cS00001
TE5/7
503/0/1
HDXc
E300
@Praha
GE1/17
GEANT2
622 Mb/s
DANTE
1 GE
10 GE
Applications
TWAREN
GE3/2
Asd002a_ome02
15454
@LA
15454
@Hsinchu
C7609
@Praha
BGP peer
GEANT2: amspra_LHC_CESNETSURFNET_06005
OC-192c
TERENA 1st E2E Workshop, Amsterdam
STS97-108
Back to back
fiber
s1/p1
s7/p1
15454
@Praha
23
Customized Approaches to Fibre-based E2E Services
POSN Private Optical Networks for Hospitals
Masaryk
Hospital
Ústí n.L.
Combination of

CWDM, DWDM
CESNET2
GigaPoP
Ústí n.L.
Open and ‚big‘ vendor
transmission systems
CWDM
MUX
MDS switch
IP router
MDS switch
8 x 10 Gb/s DWDM channels
Central
Military
Hospital
Praha
IP router
DWDM
8 channels

MDS FCIP - Multilayer Director
Switch Fibre Channel Over
TCP/IP
CESNET2
GigaPoP
Praha
1 Gb/s CWDM channel (for MDS switch,
IP router connected via PASNET)
CWDM
8 channels
PASNET
2 Dec, 2008
All IP routers and MDS FCIP
switches are connected with 1GE
TERENA 1st E2E Workshop, Amsterdam
University
Hospital
Motol
Praha
IP router
MDS switch
24
Customized Approaches to Fibre-based E2E Services
Conclusions I.
Open photonic systems offer freedom of design.

Device itself, following networks.

Important (but not limited) for first mile issues for E2E services.
Open photonic systems can offer beneficial cost
advantages.
Ability to make savings on energy consumption, housing
space usage, travel to remote huts.
The early adoption of the new photonic technology in
the production network.
Independent knowledge of possible solutions.
2 Dec, 2008
TERENA 1st E2E Workshop, Amsterdam
25
Customized Approaches to Fibre-based E2E Services
Conclusions II.
The use of experimental dark fibre facilities (after
laboratory testing) to verify feasibility of deployment in
the production network.
The use of some services of the experimental facility for
CESNET2 or end-user wavelength connections (support
of long term experiments with real usage).
Presented examples of E2E services are usable globally.

When DFs are available, of course.
Rather permanet or semi-permanent E2E fibre services
then SW reservation systems.

This is our experience with researchers – no additional SW.

But our CLM and CLSs devices include remote management and
a SW time scheduler.
2 Dec, 2008
TERENA 1st E2E Workshop, Amsterdam
26
Customized Approaches to Fibre-based E2E Services
Conclusions III.
CESNET is ready to support pilot deployments of E2E
fibre services.

Designs and concepts.

Theory or paper works.
CESNET can provide details of already deployed
projects.

Practical aspects and hands-on experience.
The vendor is a global company.

Not black boxes manufactured in a garage.

Support, guarantee.
2 Dec, 2008
TERENA 1st E2E Workshop, Amsterdam
27
Customized Approaches to Fibre-based E2E Services
Invitation to Future Internet Conference
11-13 May 2009, CLARION CONGRESS HOTEL Prague
Organized by the EC and CESNET during the Czech Republic presidency
http://www.future-internet.eu/events/eventview/article/eu-conference-the-future-of-the-internet.html
2 Dec, 2008
TERENA 1st E2E Workshop, Amsterdam
28
Customized Approaches to Fibre-based E2E Services
Invitation to 5th Customer Empowered Fiber Network Workshop
May 14 – 15, 2009 Praha, Organized by CESNET
Previous workshops http://www.ces.net/doc/seminars/cef2007/
Universitas Carolina, founded 1348.
2 Dec, 2008
TERENA 1st E2E Workshop, Amsterdam
29
Customized Approaches to Fibre-based E2E Services
References
[1] Josef Vojtěch, „CzechLight and CzechLight
Amplifiers “, In: 17th TF-NGN Zurich, 2005.
[2] Petr Holub, Josef Vojtech, Jan Radil, et. al., „Pure
Optical (Photonic) Multicast“, GLIF 2007 Demo, Praha,
2007.
[3] Josef Vojtěch, Jan Radil, „Transparent all optical
switching devices in CESNET“, 25th APAN meeting,
Honolulu, 2008.
[4] Stanislav Šíma, et. al., „ LTTx: Lightpaths to the
application, From GOLEs to dispersed end users “, GLIF
2008 Workshop, Seattle, 2008.
2 Dec, 2008
TERENA 1st E2E Workshop, Amsterdam
30
Customized Approaches to Fibre-based E2E Services
Acknowedgement
Jan Gruntorád, Jiří Navrátil, Josef Vojtěch, Miloslav Hůla,
Václav Novák, Lada Altmannová, Jan Nejman
2 Dec, 2008
TERENA 1st E2E Workshop, Amsterdam
31
Customized Approaches to Fibre-based E2E Services
Thank you for your attention.
2 Dec, 2008
TERENA 1st E2E Workshop, Amsterdam
32