Download International strategy - The University of Western Australia

International connectivity & “Big Science”
1
Copyright AARNet 2005
WA 20-21 September 2005
George McLaughlin
AARNet
About AARNet
• AARNet grew from an initiative of the
AVCC to build a TCP/IP (internet)
network in 1989
A not-for-profit company
that builds, manages
• In 1995 the AVCC sold the commercial
and operates the
customer base of AARNet to Telstra,
AARNet3 Network
spawning the commercial Internet in
Australia
Based on lighting fibre
across Australia, from
• In 2000 AARNet obtained a carrier
NextGen, power utilities
licence and IRU’s on international
and fibre suppliers
capacity
12 circuits from
• Shareholders are 38 Australian
Australia. 8 global
Universities and the CSIRO
2
points-of-presence
Copyright AARNet 2005
Australia
Japan Cable
Southern Cross
3
2 APCN2
SEAMEWE3
3
Copyright AARNet 2005
1
1. Going East & North – Southern Cross
• Currently 6 circuits
Where it gets us to:
Fiji
Hawaii
US West Coast
and US West Coast to:
North America
Central America
South America
Europe
Japan
(and other Asia Pacific)
4
Copyright AARNet 2005
– 2 x 10Gbps (SXTransPORT, R&E only)
– 2 X 622Mbps (commodity PAIX and LA)
– 2 X 155Mbps (mixed, Fiji, Hawaii to Japan)
• About to add 2 more 622Mbps circuits
– For commodity expansion
• This will also extend rights to use
SXTransPORT to end 2013
• Participation in the National Science
Foundation’s (NSF) International Research
Connections Program (IRNC)
• Strategy defined and largely implemented
TransLight Pacific Wave
An initiative of the US
National Science
Foundation’s
International Research
Network Connections
Program
5
Copyright AARNet 2005
• Partners: AARNet, CENIC, Pacific
Wave, University of Hawaii
• Distributed International Peering
Exchange along US West Coast
• Hybrid Optical Packet Infrastructure
• Seed Global Astronomy Initiative
based around the international
telescopes at Mauna Kea, Hawaii
• GLIF infrastructure between US,
Hawaii and Australia
AARNet, Pacific Wave, NLR, .……..
6
Copyright AARNet 2005
2. Going West and North (via Singapore)
Where it gets us to:|
Singapore
From Singapore to:
Indonesia, Malaysia
Thailand, Hong Kong
Japan, Korea
From Hong Kong to:
Vietnam, Beijing (then
via Russia to Europe)
7
Copyright AARNet 2005
• This strategy has been developed together
with the European Commission’s TEIN2
project
• It will provide a massive increase in
connectivity within the region and importantly
provides to geographically diverse routes
from the region going west to Europe
• Will significantly enhance the infrastructure
for engagement between AARNet’s members
and others in the region
• Future extensions to South Asia and Africa
likely
• Key element of AARNet global connectivity
strategy, soon to be implemented
Trans Eurasian Information Network (TEIN2)
Partners
An initiative of the
European Commission
with the objective of
improving connectivity
in certain developing
countries of the
Asia Pacific region
8
Copyright AARNet 2005
Beneficiaries:
China (CERNET)
Indonesia (ITB)
Malaysia (MDC)
Philippines (ASTI)
Thailand (ThaiREN)
Vietnam (MOST)
Non-beneficiaries:
Korea (KISDI)
Singapore (SingAREN)
Australia (AARNet)
France (RENATER)
Netherlands (SURFnet)
UK (UKERNA)
TEIN2 Topology
Singapore to
Perth (4 x 155Mbps, 2x2)
Frankfurt (west) (3 x 622Mbps)
Japan (at least 2 x 622Mbps)
Korea (622Mbps)
Hong Kong (622Mbps)
Thailand (155Mbps)
Malaysia (45Mbps)
Indonesia (45Mbps)
Hong Kong to:
Vietnam (45Mbps)
Beijing (622Mbps)
Beijing to Europe
via Russia (622Mbps)
9
Copyright AARNet 2005
2
How we currently get to these places
10
Copyright AARNet 2005
AARNet and TEIN2 circuits
11
Copyright AARNet 2005
TEIN2 – AARNet’s role (i)
12
Copyright AARNet 2005
• TEIN2, though funded by
EuropeAID, falls under the auspices
of ASEM (Asia Europe Meeting)
• Australia isn’t an ASEM member
• Australian Govt can’t participate in
ASEM meetings
• AARNet acting as proxy for Australia
in TEIN2 (firstly as project advisor,
then as project partner)
• Building on AARNet's leading role in
APAN
TEIN2 – AARNet’s role (ii)
13
Copyright AARNet 2005
• AARNet will establish PoPs in Singapore
and Frankfurt
• The 4 x 155Mbps (2 on SEAMEWE3 and 2
on APCN) from Perth to Singapore and
one of the 622Mbps circuits to Frankfurt
will be under AARNet’s control
• Up to 50% of the capacity may be used for
commodity traffic
• Application deployment already in planning
• AARNet will be intimately involved in the
engineering task force and NOC
implementation
• Current project runs to Dec 2007
TEIN2, Taiwan, AARNet and LHC
• Taiwan is not a direct partner in TEIN2
• Taiwan is the TEIR1 Large Hadron
Collider site in the region
• Taiwan will deploy a 2.5Gbps link
going west to Amsterdam via
Singapore
• AARNet will interconnect with this at
Singapore
• Taiwan plans to upgrade this circuit
going west to 10Gbps by end 2007
14
Copyright AARNet 2005
Combined Strategy elements 1 plus 2
AARNet owned and operated circuits
Circuit size path
15
2 AARNet International
Gateways:
Sydney
Perth
1x10Gbps
Sydney - Oahu-Seattle (R&E)
1x10Gbps
Sydney - Big Island - Los Angeles (R&E)
2x622Mbps
Sydney - Palo Alto (Commodity)
7 AARNet Global PoPs:
Seattle
Palo Alto
Los Angeles
Hawaii (Oahu)
Hawaii (Big Island)
Suva
Singapore
Frankfurt
2x622Mbps
Sydney - Los Angeles (Commodity)
1x155Mbps
Sydney - Suva - Oahu - Seattle (both)
1x155Mbps
Sydney - Seattle (both)
2x155Mbps
Perth - Singapore (both) SEAMEWE3 path
2x155Mbps
Perth - Singapore (both) APCN path
1x622Mbps
Singapore – Frankfurt (both) westerly path
Copyright AARNet 2005
Combined Strategy elements 1 plus 2
Other Circuits accessible from Singapore PoP
Circuit size City/country/economy
2 AARNet International
Gateways:
Sydney
Perth
7 AARNet Global PoPs:
Seattle
Palo Alto
Los Angeles
Hawaii (Oahu)
Hawaii (Big Island)
Suva
Singapore
Frankfurt
16
Copyright AARNet 2005
2x622Mbps
Frankfurt
nx622Mbps
Japan
1x622Mbps
Korea
1x622Mbps
Hong Kong
1 x 622Mbps Beijing (via Hong Kong)
1x2.5Gbps
Taiwan
1x155Mbps
Thailand
1x155Mbps
Philippines (via Japan)
1x45Mbps
Malaysia
1x45Mbps
Indonesia
1 x 45Mbps
Vietnam (via Hong Kong)
Facilitating recent S&T agreements
• Minister Nelson’s recent trip to Asia
and the signing or extension of
various agreements
–
–
–
–
17
Copyright AARNet 2005
China
Malaysia
Singapore
Indonesia
• All are both TEIN2 and APAN
partners
• New links will facilitate new
collaborations
3. Going North (Australia Japan Cable)
Where it gets us to:
Japan
(optionally Guam)
From Japan to:
North Asia
South East Asia
US
From Guam:
Cable interconnect
opportunities
18
Copyright AARNet 2005
• We have attractive pricing on this
cable system
• Need to avoid biting off more than
we can chew
• Logistics dictate that TEIN2 and
additional commodity are higher
priorities
• As soon as TEIN2 is bedded down,
should progress with AJC options
• Part of strategy yet to be
implemented
“Big Science” projects driving networks
•
Billion dollar globally
funded projects
Massive data transfer
needs
•
19
Copyright AARNet 2005
Large Hadron Collider
– Coming on-stream in 2007
– Particle collisions generating terabytes/second of
“raw” data at a single, central, well-connected site
– Need to transfer data to global “tier 1” sites. A tier
1 site must have a 10Gbps path to CERN
– Tier 1 sites need to ensure gigabit capacity to the
Tier2 sites they serve
Square Kilometre Array
– Coming on-stream in 2010?
– Greater data generator than LHC
– Up to 125 sites at remote locations, data need to
be brought together for correlation
– Can’t determine “noise” prior to correlation
– Many logistic issues to be addressed
From very small to very big
20
Copyright AARNet 2005
Scientists and Network Engineers coming together
• HEP community and R&E network
community have figured out
mechanisms for interaction – probably
because HEP is pushing network
boundaries
• eg the ICFA workshops on HEP, Grid
and the Global Digital Divide bring
together scientists, network engineers
and decision makers – and achieve
results
• http://agenda.cern.ch/List.php
21
Copyright AARNet 2005
What’s been achieved so far
22
Copyright AARNet 2005
 A new generation of real-time Grid systems is
emerging - support worldwide data analysis
by the physics community
 Leading role of HEP in developing new
systems and paradigms for data intensive
science
 Transformed view and theoretical
understanding of TCP as an efficient,
scalable protocol with a wide field of use
 Efficient standalone and shared use of 10
Gbps paths of virtually unlimited length;
progress towards 100 Gbps networking
 Emergence of a new generation of “hybrid”
packet- and circuit- switched networks
LHC data (simplified)
1 Megabyte (1MB)
A digital photo
Per experiment
40 million collisions per second
• After filtering, 100 collisions
of interest per second
• A Megabyte of digitised
information for each collision
= recording rate of 100
Megabytes/sec
• 1 billion collisions recorded =
1 Petabyte/year
CMS
23
Copyright AARNet 2005
LHCb
ATLAS
1 Gigabyte (1GB)
= 1000MB
A DVD movie
1 Terabyte (1TB)
= 1000GB
World annual
book production
1 Petabyte (1PB)
= 1000TB
10% of the annual production by
LHC experiments
1 Exabyte (1EB)
= 1000 PB
World annual information
production
ALICE
LHC Computing Hierarchy
CERN/Outside Resource Ratio ~1:2
Tier0/( Tier1)/( Tier2)
~1:1:1
~PByte/sec
~100-1500
MBytes/sec
Online System
Experiment
CERN Center
PBs of Disk;
Tape Robot
Tier 0 +1
Tier 1
~2.5-10 Gbps
IN2P3 Center
INFN Center
RAL Center
FNAL Center
2.5-10 Gbps
~2.5-10 Gbps
Tier 3
Tier 2
Institute Institute
Physics data cache
24
Workstations
Copyright AARNet 2005
Institute
Tier2 Center
Tier2 Center
Tier2 Center
Tier2 Center
Tier2 Center
Institute
0.1 to 10 Gbps
Tier 4
Tens of Petabytes by 2007-8.
An Exabyte ~5-7 Years later.
Lightpaths for Massive data transfers
• From CANARIE
A small number of users
with large data transfer
needs can use more
bandwidth than all other
users
30
25
20
15
10
5
0
25
Copyright AARNet 2005
n04
Ju
n03
Ju
n02
Ju
n01
Ju
n00
Ju
n99
Ju
Ju
n98
Lightpaths
IP Peak
IP Average
Why?
•
Type 3 users:
High Energy Physics
Astronomers, eVLBI,
High Definition
multimedia over IP
Massive data transfers
from experiments
running 24x7
26
Copyright AARNet 2005
Cees de Laat classifies network
users into 3 broad groups.
1. Lightweight users, browsing, mailing,
home use. Who need full Internet
routing, one to many;
2. Business applications, multicast,
streaming, VPN’s, mostly LAN. Who
need VPN services and full Internet
routing, several to several + uplink;
and
3. Scientific applications, distributed
data processing, all sorts of grids.
Need for very fat pipes, limited
multiple Virtual Organizations, few to
few, peer to peer.
What is the GLIF?
• Global Lambda Infrastructure Facility
- www.glif.is
• International virtual organization that
supports persistent data-intensive
scientific research and middleware
development
• Provides ability to create dedicated
international point to point Gigabit
Ethernet circuits for “fixed term”
experiments
27
Copyright AARNet 2005
Huygens Space Probe – a practical example
• Cassini spacecraft left Earth in October 1997
to travel to Saturn
• On Christmas Day 2004, the Huygens probe
separated from Cassini
Very Long Baseline
Interferometry (VLBI) is • Started it’s descent through the dense
a technique where
atmosphere of Titan on 14 Jan 2005
widely separated radio- • Using this technique 17 telescopes in
telescopes observe the
Australia, China, Japan and the US were
same region of the sky
able to accurately position the probe to
simultaneously to
within a kilometre (Titan is ~1.5 billion
generate images of
kilometres from Earth)
cosmic radio sources
• Need to transfer Terabytes of data between
Australia and the Netherlands
28
Copyright AARNet 2005
AARNet - CSIRO ATNF contribution
29
Copyright AARNet 2005
• Created “dedicated” circuit
• The data from two of the Australian
telescopes (Parkes [The Dish] &
Mopra) was transferred via light
plane to CSIRO Marsfield (Sydney)
• CeNTIE based fibre from CSIRO
Marsfield to AARNet3 GigaPOP
• SXTransPORT 10G to Seattle
• “Lightpath” to Joint Institute for VLBI
in Europe (JIVE) across CA*net4
and SURFnet optical infrastructure
But………..
• 9 organisations in 4 countries involved in
“making it happen”
• Required extensive human-human
Although time from
interaction (mainly emails…….lots of them)
concept to undertaking
the scientific experiment • Although a 1Gbps path was available,
maximum throughput was around 400Gbps
was only 3 weeks……..
• Issues with protocols, stack tuning, disk-todisk transfer, firewalls, different formats, etc
• Currently scientists and engineers need to
test thoroughly before important
experiments, not yet “turn up and use”
• Ultimate goal is for the control plane issues
to be transparent to the end-user who simply
presses the “make it happen” icon
30
Copyright AARNet 2005
International path for Huygens transfer
31
Copyright AARNet 2005
EXPReS and Square Kilometre Array
Australia one of
countries bidding for
SKA – significant
infrastructure challenges
Also, Eu Commision
funded EXPReS project
to link 16 radio
telescopes around the
world at gigabit speeds
32
Copyright AARNet 2005
• SKA bigger data generator than LHC
• But in a remote location
In Conclusion
• scientists and network engineers
working together can exploit the new
opportunities that high capacity
networking opens up for “big science”
• Need to solve issues associated with
scalability, control plane, ease of use
• QUESTIONS?
33
Copyright AARNet 2005