Download slice

Large-scale testbeds for network R&D
PlanetLab Everywhere
Rio de Janeiro, April 2009
Michael Stanton
Rede Nacional de Ensino e Pesquisa - RNP
[email protected]
© 2009 – RNP
Summary
• We discuss R&D testbeds for networking and distributed
systems, seen from the point of view of RNP
• In addition to activities carried out in Brazil, we also examine
some testbeds in use or planned in other countries, especially in
relation to future Internet development
• In conclusion suggestions are made about future steps to be
followed in Brazil.
Large-scale testbeds for R&D in Brazil
2
Introduction to Brazil
•
•
•
Tordesillas Line
In 1494 Spain and Portugal had
divided between themselves
undiscovered lands by the Treaty of
Tordesillas
– The Tordesillas Line was to be
the frontier between the
dominions of Spain (W) and
Portugal (E)
Brazil is the successor country to
the Portuguese dominions in South
America
– Rather over one half of present
Brazil lies to the WEST of the
Tordesillas Line
Brazil is a BIG place!
– diameter of about 4,200 km
to Spain
to Portugal
– 42 x 42 = 1764 ms2
•
Current population of about 180
millions, unevenly distributed
– most of the population and
communications infrastructure
concentrated to the EAST of
the Tordesillas Line
Large-scale testbeds for R&D in Brazil
3
RNP –
Rede Nacional de Ensino e Pesquisa
• RNP is the Brazilian NREN
– maintained by the Brazilian government (since 1989) to enable
network access to the national research and education community
– provides national (inter-state) and international R&E connectivity for
more than 300 public and private universities and research centers
through the provision of advanced networking infrastructure
• also provides commodity access – one-stop shopping
– promotes the development of advanced networking and
applications
• Since 2000, RNP is managed for the federal government by a
non-profit private company, RNP-OS, legally recognised as an
“Organização Social”, which allows the government to contract
its services without competitive tender.
Large-scale testbeds for R&D in Brazil
4
RNP’s service networks
RNP includes the following funded connectivity:
• National backbone network – Rede IPÊ
– 1 PoP (Point of Presence) in each state – usually a federal
university
– Link capacity depends on the available telco infrastructure
– Currently from 2 Mbps to 10 Gbps
• Direct intercity connections between state PoP and non-local
federal instituions (education, science and technology)
– Currently from 2 to 155 Mbps (depends on the institution)
• Community-based optical metro networks connected to PoPs
– Currently being built out – 9 out of 27 already in operation
No service charges are made to end user institutions
• Non-federal institutions are normally required to fund their own
access links
Large-scale testbeds for R&D in Brazil
5
National integration
RedClara, beyond 2008
• Extend RedClara to all
LA&C countries
• Promote applications in
education and health
• Start of a new project
(ALICE2), with the
support of the EC, in
2009
Source:
www.redclara.net
Large-scale testbeds for R&D in Brazil
7
A world-class network – RNP in GLIF
Source:
www.glif.is
Large-scale testbeds for R&D in Brazil
8
A world-class network – worldwide GLIF
Source: www.glif.is
Large-scale testbeds for R&D in Brazil
9
Rede IPÊ – national backbone network
Last big reform in 2005 (5th
phase)
Capacity reflects available telco
infrastructure
Currently composed of:
• Multigigabit core network
– 4 PoPs at 10 Gbps, and 6
PoPs at 2.5 Gbps
– IP over lambdas (12.000
km)
• Terrestrial SDH connections
to 15 PoPs
– Most links are 34 Mbps
– Some at 2 Mbps
– Some upgrades in 2007 to
102, 155 and 622 Mbps
• 2 PoPs connected by satellite
at 2 Mbps
Large-scale testbeds for R&D in Brazil
10
Evolution of
academic networks in Brazil
RNP
Phase
Year
Technology
Link capacities
Comment
1988
BITNET
up to 9.6 kbps
first national network
1
1992
Internet
9.6 and 64 kbps
first national IP network (RNP)
2
1995
up to 2 Mbps
also: commercial IP deployed
3
1999
IP/ATM,
IP/FR
VC up to 45 Mbps, RNP2 national backbone;
access up to 155
testbed metro networks in 14
Mbps
cities (using ATM/dark fiber)
4
2003
IP/SDH
34, 155, 622 Mbps
also: IP/WDM interstate
testbed network (Project GIGA)
5
2005
IP/WDM
2.5 and 10 Gbps
IPÊ national backbone;
metro networks in 27 capitals
Large-scale testbeds for R&D in Brazil
11
Evolution of
academic networks in Brazil
Capacidade dos enlaces
(Link capacity)
10.000.000
Phase 5
Ipê
1.000.000
Phase 4
RNP2+
kbps
100.000
Phase 3
RNP2
10.000
1.000
Phase 2
comercial
Internet
100
10
Phase 0
BITNET
Phase 1
Internet
1
1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005
Ano
Large-scale testbeds for R&D in Brazil
12
RNP and its innovating networks
• The first version of RNP’s network was deployed in 1992, and
pioneered in using TCP/IP technology nationwide
• Since then, the different generations of network deployed and
operated by RNP have innovated technically, at least within
Brazil
• With the development of the network industry here, since the
beginning of commercial IP networks starting in 1995, RNP is no
longer the only IP network operator in Brazil, but continues to
lead technologically, in the pursuit of new models of
infrastructure and applications
Large-scale testbeds for R&D in Brazil
13
Technology changes in the network
• The beginning of each new technology phase was a step in the
dark
– new circuits (or service, in the case of ATM/FR) were
ordered and delivered, and equipment configured
– after configuration by the network engineers, the new
network would begin to work, and would be put into
operation as soon as possible.
– users could then begin to use the resources of the new
network
• Problems:
– lack of familiarity with the new technologies before
operational deployment
– the technology transition became a singularity
Large-scale testbeds for R&D in Brazil
14
New application services
• Internet technology is very accommodating of new application
services
– any user can develop a new service, impelemnted as a
distributed application using the sockets API
– this permits and encourages experimentation with
applications that can be built initially in the laboratory (in a
LAN environment) and then migrated to the wide area
network
• Some problems which arise:
– some distributed services require componentes “within the
network” – security problems
– performance monitoring may be a problem
Large-scale testbeds for R&D in Brazil
15
Alternative solutions
• Both for
– preparing an upcoming network technology change
– developing a a new large-scale distributed application
we really need a testbed facility, isolated from the production
network, and which reproduces its characteristics of scale and
performance
• This isolation can be real or virtual
– Real – the testbed is based on separate physical
infrastructure, independent of the production network
• example: Project GIGA
– Virtual – the testbed shares the same infrastructure used by
the production network
• example: PlanetLab
Large-scale testbeds for R&D in Brazil
16
Project GIGA –
optical networking testbed
•
•
•
Partnership between
– RNP , CPqD (telco industry R&D centre in Campinas, SP), R&D
community in networks and distributed systems
– Financed by FUNTTEL between 2002 and 2007 – US$20M
– telcos – provide optical fiber at no cost
Objectives:
– build an advanced networking testbed for development and
demonstration purposes
– support R&D subprojects in optical and IP networking technology and
advanced applications and services
Network support (since May 2004)
– R&D subproject consortia provided with internal connectivity using
VLANs – 20 institutions connected
– however: the testbed (with some exceptions) did not provide exteral
connectivity, limiting its usefulness
FUNTTEL
Large-scale testbeds for R&D in Brazil
17
GIGA testbed network - location
•
dark fibre-based 700-km
inter-city backbone in
states of São Paulo and
Rio de Janeiro
(south-east Brazil)
•
Initially 20 universities
and R&D centers in 7
cities
•
2.5G DWDM in the
inter-city backbone
•
2.5G CWDM used in the
metropolitan area
Large-scale testbeds for R&D in Brazil
testbed
network
18
GIGA testbed network - location
Universities
IME
PUC-Rio
PUC-Campinas
UERJ
UFF
UFRJ
Mackenzie
UNICAMP
USP
R&D Centers
CBPF
CPqD
CPTEC
INCOR
CTA
FIOCRUZ
IMPA
INPE
LNCC
LNLS
Large-scale testbeds for R&D in Brazil
19
Testbed network design
•
2.5G DWDM inter-city network between Campinas
and Rio de Janeiro (some upgrades to 10G)
Campinas
– up to 6 waves per link (can use 8)
•
•
3λ
2.5G CWDM metro networks in São Paulo, Campinas
São
and Rio de Janeiro
Paulo
all links currently 1 Gigabit Ethernet
– optical equipment from the Brazilian firm, Padtec
1λ
2λ
2λ
1λ
Rio de
Janeiro
2λ
(www.padtec.com.br)
– IP equipment from Extreme Networks
São Paulo
S. José dos
Campos
MAN
SP
Campinas
3λ
S.J. dos
Campos
Cachoeira
Paulista
1λ
2λ
Rio de
Janeiro
Petrópolis
Niterói
MAN
CP
MAN
RJ
Large-scale testbeds for R&D in Brazil
20
What is PlanetLab?
(base d on slides by Marc Fiuczynski, Sept. 2007)
• Facility: Planetary-scale “overlay” & “underlay” network
– 900+ Linux-based servers at 400+ sites in 40+
countries
– Currently there exist a handful of PL sites in Brazil
operated by RNP and a few universities and
research institutions
PlanetLab Facility Today
1000+ servers at 450+ sites in 40+ countries
Co-located throughout the world @ Uni. & Companies
Co-located at network crossroads (Internet2, RNP, CERNET, …)
Large-scale testbeds for R&D in Brazil
22
Virtualization in PlanetLab?
• Research Community: Distributed Sys. & Networking
– Researchers can get a set of “virtual machines”
across these servers (SLICE)
– In a SLICE researchers can deploy & evaluate …
– … distributed systems services and applications
“The next Internet will be created as an overlay
in the current one”
– … network architectures and protocols
“The new Internet will be created in parallel next
to the current one”
(see later)
Example Network Services
• Scalable Large-File Transfer: CoBlitz—Princeton, LoCI—Tennessee
• Content Distribution: Coral—NYU, CoDeeN—Princeton, CobWeb—
Cornell
• Distributed Hash Tables: OpenDHT—UC Berkeley; Chord-MIT
• Routing Overlays: I3 Internet Indirection Infrastructure—UC Berkeley
• Multicast Delivery Nets: End System Multicast—CMU, Tmesh-U.
Michigan
•
•
•
•
•
•
Serverless Email: ePOST—Rice University
Publish-Subscribe News Access: CorONA—Cornell
Robust DNS Resolution: CoDNS—Princeton, CoDoNs—Cornell
Mobile Access: DHARMA—U. of Pennsylvania
Location/Anycast Services: OASIS—NYU, Meridian—Cornell
Internet Measurement: ScriptRoute—U. of Maryland
• Above services communicate with >1M real users
and transmit ~4TB of data per day
Large-scale testbeds for R&D in Brazil
24
Slice
Slice
Slice
Slice
Manager
(SM)
Slice
PlanetLab Node Software Architecture
Virtualization Software
x86 Server Hardware
Large-scale testbeds for R&D in Brazil
25
Slices
Large-scale testbeds for R&D in Brazil
26
Slices
Large-scale testbeds for R&D in Brazil
27
Slices
Large-scale testbeds for R&D in Brazil
28
Thoughts about the future Internet
• The success of the Internet has been so enormous
that it is tempting to imagine its future by
extrapolating from the present
• However, there are consequences of its design,
based on decisions taken in the 1970s, which
severely limit its security, availability, flexibility and
manageability
• These limitations can not be removed through small
incremental adjustments of the existing network, and,
if they are not removed, they will create huge
impediments to the ability to use and exploit the
Internet in the future
Large-scale testbeds for R&D in Brazil
29
Removing these limitations
• For many years, combatting Internet limitations has
been carried out through a series of “patches”,
introduced to solve specific problems.
• Unfortunately, these patches result in increased
complexity, resulting in a less robust system, whose
operation has become more difficult and costly
• A growing consensus exists in the network research
community that we have already reached the point
where patches are inadequate, and a fundamental
rethink of the Internet is required
(from the GENI Research Plan, 2007)
Large-scale testbeds for R&D in Brazil
30
GENI -
http://www.geni.net
(Global Environment for Network Innovations)
• An initiative of NSF (USA) to create a shared testbed
environment to allow the validation of new network architectures
– Initial phase: 2005 to 2007 – design of the GENI
environment
– Present phase: since 2008, deployment and use
• GENI will support research which can lead to a future Internet
with improved chacteristics
– more comprehensive security
– greater generality
– better integration of optical and wireless technologies
– integration of the world of sensors and embedded
processors
– improved options for the economic health of ISPs
Large-scale testbeds for R&D in Brazil
31
The GENI testbed environment
GENI will:
• allow experiments with alternative large-scale network
architectures, services and applications under realistic
conditions
• use vitualizations to permit carrying out multiple independent
and simultaneous experiments
• permit long-duration experiments, allowing mature prototypes to
serve “living” user communities
• facilitate experimental research through the use of extensive
tools for measurement and data collection
In summary, GENI will provide support for the taking ideas on
large-scale ideas from their conception to their deployment, by
means of experimental validation
Large-scale testbeds for R&D in Brazil
32
How the GENI environment will be built
• The GENI environment is inspired on PlanetLab, and especially
the “Meta-Testbed” VINI - http://www.vini-veritas.net
• VINI extends the scope of PlanetLab to allow
– “slicing” of links between the nodes (link virtualization)
– substitution of level 3 protocols (IP)
Large-scale testbeds for R&D in Brazil
33
Next Step: Meta Testbed
(base d on slides by Marc Fiuczynski, Sept. 2007)
• Goals
– support experimental validation of new architectures
• simultaneously support real users and clean slate designs
• allow a thousand flowers to bloom
– provide plausible deployment path
• Key ideas
– virtualization
• multiple architectures on a shared infrastructure
• shared management costs
– opt-in on a per-user / per-application basis
• attract real users
• demand drives deployment / adoption
Large-scale testbeds for R&D in Brazil
34
VINI: Our Meta Testbed approach
• Infrastructure
– PlanetLab provides “access network” with global reach
• user desktops run proxy that allows them to opt-in
• treat nearby PlanetLab node as ingress router
– NLR/I2 provides high-speed backbone in the United States
• populate with programmable routers
• extend slice abstraction to these routers
• Usage model
– each architecture (service) runs in its own slice
– two modes of use
• short-term experiments
• long-running stable architectures and services
Large-scale testbeds for R&D in Brazil
35
Extending Slices to a VINI testbed
Large-scale testbeds for R&D in Brazil
36
Extending Slices to a VINI testbed
Large-scale testbeds for R&D in Brazil
37
Extending Slices to a VINI testbed
Large-scale testbeds for R&D in Brazil
38
User Opt-in
Client
Server
NAT
wireless
Large-scale testbeds for R&D in Brazil
39
Internet in a Slice (IIAS)
XORP in Network Container
XORP
(routing protocols)
User
IPv4
Fwd table
– Adds routes to copy of kernel IPv4
forwarding table
– Kernel forwards packets between
virtual interfaces
Kernel Filters and shapers
– Add delay and loss, constrain
bandwidth
Virtual interfaces
Filters, shapers
vif0
vif1
– Appear as Ethernet devices in a
slice
– Reduce MTU for tunneling
vif2
E-GRE tunnels
E-GRE tunnels
– Hack standard GRE tunnels to
preserve MAC headers
PlanetLab VM
Large-scale testbeds for R&D in Brazil
40
GENI
• Extenssion of VINI – key ideas:
– virtualization
• multiple network architectures sharing a common infrastructure
– user opt-in: per user / per application
• intended to attract real users
• Infrastructure
– NLR/Internet2 provide high-capacity backbone in the US
• populate with programmable resources (processors, storage)
• populate with programmable routers
– more sophisticated than the PCs used in VINI
• extend the “slice” abstraction to these routers
• include “extensions” to wireless and sensor networks
Large-scale testbeds for R&D in Brazil
41
GENI: The Physical Network
(slides by C. Qiao, 2008)
• Large-Scale Facility of Networked Systems
–
–
–
–
Reasonable Representation of the Internet’s Complexity
A Nationwide Optical Network ~ 200 Universities
Clusters for Processing/Storage
Wireless Access Networks
• Mobility, Location Awareness
– Sensor Networks
– Connected to a large number of User Communities
• Partnerships to Extend GENI within the US
– Add Technologies and Users
• Federation to Extend GENI on a Global Scale
Large-scale testbeds for R&D in Brazil
42
Schematic GENI Network
Sensor Network
Core Nodes
Edge Site
Federated
International Facility
Mobile Wireless Network
Edge Nodes
Large-scale testbeds for R&D in Brazil
43
Programmability
All network elements programmable via open
interfaces and/or downloadable user code
GENI Control & Management Plane
Programmable
Sensor Node
Open API
Radio platform
Programmable
Core Node
Large-scale testbeds for R&D in Brazil
Programmable
Edge Node
44
Slicing and Virtualization
Sensor Network
Edge Site
Mobile Wireless Network
- share resources to support many simultaneous experiments
Large-scale testbeds for R&D in Brazil
45
GENI Design Principles
• Physical network ‘substrate’
– building block components
– elements / nodes / links / subnets
• Software control & management framework
– knits building blocks together
– allows many parallel experiments (slices)
– creates arbitrary logical topologies (virtualization)
• Programmable for ‘Clean Slate’ research
• Instrumented for accurate analysis
• Flexible and Phased Design
– Support Technology Introduction during GENI Lifetime
Large-scale testbeds for R&D in Brazil
46
GENI Phase 1(2008-9)
(based on slides by Chip Elliott – GENI program director)
• Provides the very first, national-scale prototype of an
interoperable infrastructure suite for Network Science and
Engineering experiments
• Creates an end-to-end GENI prototype in 6-12 months with
broad academic and industrial participation, while encouraging
strong competition in the design and implementation of GENI’s
control framework and clearinghouse
• Includes multiple national backbones and regional optical
networks, campuses, compute and storage clusters,
metropolitan wireless and sensor networks, instrumentation and
measurement, and user opt-in
• Because the GENI control framework software presents very
high technical and programmatic risk, the GPO has funded
multiple, competing teams to integrate and demonstrate
competing versions of the control software in Phase 1
Large-scale testbeds for R&D in Brazil
47
GENI phase 1: integration:
5 competing control schemes
Large-scale testbeds for R&D in Brazil
48
Research using GENI - FIND
• Prior to the building of the GENI testbed, NSF launched the
initiative FIND – Future Internet Design, with the aim of
identifying and financing research activities
• http://www.nets-find.net/
• One product of this initiative is the GENI Research Plan, which
details the research motivation for GENI, and some of the
research goals which will become possible:
• http://www.geni.net/GDD/GDD-06-28.pdf
Large-scale testbeds for R&D in Brazil
49
Experimental activities in Europe
FIRE - Future Internet Research & Experimentation
http://cordis.europa.eu/fp7/ict/fire/home_en.html
• European initiative directed towards the design of the future
Internet, similar to the FIND and GENI initiatives of the NSF
• promotes the concept of experimental research, combining
visionary academic research with validation and
experimentation typical of industry
• aims to create a multidisciplinary research environment to
investigate and validate experimentally innovative ideas for new
paradigms of networks and services
• plans to create a “European Experimental Facility” (EEF),
formed by the interconnection and federation of both existing
and future testbeds, for emerging and future Internet
technologies
• First projects selected in 2008
Large-scale testbeds for R&D in Brazil
50
FIRE/Panlab (FOKUS/DE)–
www.panlab.net
Large-scale testbeds for R&D in Brazil
51
FIRE/OneLab (UPMC, FR) –
www.onelab.eu
• History:
– Mar/04: based on ENEXT (Identification of critical testbeds
for networking research)
– Sep/06: OneLab1 project funded as an IST project under
the FP6 funding program:10 partners, 2 years
– Sep/08: proposal funded as an IST project under the FP7
funding program: 26 partners, 2 years
• OneLab1 – Goals
– extend PlanetLab into new environments, beyond the
traditional wired internet
– improve PlanetLab’s monitoring capabilities
– provide a European administration for PlanetLab nodes in
Europe
Large-scale testbeds for R&D in Brazil
52
FIRE/Federica –
www.fp7-federica.eu
Large-scale testbeds for R&D in Brazil
53
Comparison of FIRE prototypes
Large-scale testbeds for R&D in Brazil
54
Future Internet activities in Japan
• AKARI - http://akari-project.nict.go.jp/eng/overview.htm
– plans to deploy a new generation network by 2015, based on
a new architecture
– supposes the use of an experimental testbed incorporating
virtualization techniques, as in GENI
Large-scale testbeds for R&D in Brazil
55
Perspectives in Brazil
• Phase 2 of Project GIGA (RNP- CPqD)
– original Project GIGA funding ended in Dec/2007, although
the experimental network continues to operate
– since 2007 RNP and CPqD have been seeking to maintain
an experimental facility for their research communities
– RNP’s proposal is to place greater emphasis on research
into architectures for a future Internet
Large-scale testbeds for R&D in Brazil
56
Perspectives in Brazil
• INCT/Web Science (consortium led by UFRJ) – more than 100
researchers – approved in 2008 (3 to 5 years)
• A group of 8 researchers from (RNP, UFF, UFPA, UNIFACS,
USP) included a research proposal in “Future Internet
Architectures”
– main emphasis on experimental research, using an
environment based on PlanetLab / VINI, with extensions for
wireless access networks
– RNP network will be used for long-distance communications
(within Brazil and externally)
• Note:
– VINI requires access to a “lower than level 3” network – this
depends on the next phase of the RNP network
(FuturaRNP), expected in 2010
Large-scale testbeds for R&D in Brazil
57
FuturaRNP and 2010
• RNP is engaged in several initiatives, which are changing the
face of its infrastructure:
– adoption of a layer 2 national backbone, and introduction of
static and dynamic end-to-end circuits as well as routed IP
(Hybrid Network)
– extension to all 27 capitals and 10 other cities of optical
metro networks based on Ethernet technology (layer 2)
– huge increase in capacity, when possible through multiple
lambdas in DWDM systems
• A possible desirable consequence would be the permanent
reservation of capacity for experimental activities, segregated
from production traffic (à la GENI)
Large-scale testbeds for R&D in Brazil
58
Federation with other initiatives
• The deployment of an experimental facility in Brazil to support
research into new architectures and applications will simplify
international collaboration with similar initiatives abroad
• This would be brought about by the interconnection (federation)
of the national facility with similar facilities in other countries
• It should be noted that this style of federation is one of the
characteristics of the projects we have described from the USA
and Europe.
Large-scale testbeds for R&D in Brazil
59
The Future Internet will be polymorphic
Large-scale testbeds for R&D in Brazil
60
The Future Internet will be polymorphic
Large-scale testbeds for R&D in Brazil
61
Thank you!
Michael Stanton
([email protected])
www.rnp.br
Yellow ipê in blossom