Download Automatic Dynamic Run-time Optical Network Reservations

Automatic Dynamic Run-time
Optical Network Reservations
John R. Lange
Ananth I. Sundararaj and Peter A. Dinda
Prescience Lab
Department of Computer Science
Northwestern University
http://plab.cs.northwestern.edu
1
Introduction
• Recently the Grid community has begun turning
the network into a standard grid service
– Network and compute resources are still very different
– Requires in depth understanding of program behavior
or targeted application development
• VRESERVE
– Enable unmodified applications to efficiently utilize
reservation based networks without user intervention
2
Overview
• Reservations and Circuit Switching
• Optical Networking
– OMNInet and ODIN
• Virtual Machines
– Virtuoso
• Overlay Networking
– VNET and VTTIF
• Putting it all together
– VRESERVE
• Performance Evaluation
3
Reservable Networks
• Generic Reservation API
– CreatePath(<srcIP>, <dstIP>, <bw>, <lat>);
– TeardownPath(<srcIP>, <dstIP>);
• Somebody or something has to make the
API calls
• Circuit switching and network reservations
are inherently linked
– Establishing a circuit is similar to reserving the
network elements along a path
4
Reservation Issues
• Parameter Values
– How does someone determine the required
bandwidth and latency?
– Are the endpoints always evident?
• Highly parallel applications in a grid environment
• Migratable Virtual Machines
• Time
– What is the flow duration?
– What is the traffic behavior pattern?
• Applications often have cyclical behaviors (compute
and send)
5
Reservation Problems
• Complexity:
– Someone has to call the reservation API
• Developer must incorporate API calls into application
• User must reserve network on application’s behalf
• Efficiency:
– Efficient reservations require understanding
application’s behavior and topology
– Networks usually reserved in spatial and temporal
blocks
• Networks reserved to connect all hosts
• Networks reserved for entire execution time
6
Optical Networks
• Recent advances in switching technology have
brought renewed interest to the area
• Offer provisioning of dynamic lightpaths
– One of the latest incarnations of circuit switched
networking
– Emergence of network reservation grid services
– Generating a large deal of interest in the high
performance computing and grid communities
• NLR, Canarie, Netherlight
• OptIPuter
7
OMNInet and ODIN
• OMNInet
–
–
–
–
Experimental dynamically configurable optical network
Provisionable wavelengths (lambdas)
All-to-All topology
Connects research centers in Chicago and Northern Illinois
• ODIN
– Reservation system for OMNInet
• Developed by iCAIR
– International Center for Advanced Internet Research
– Mambretti, J., Weinberger, J., Chen, J., Bacon, E., Yeh, F.,
Lillethun, D., Grossman, B., Gu, Y., and Mazzuco, M. The
photonic terastream: Enabling next generation applications
through intelligent optical networking at iGRID2002.
8
ODIN
• Lightpath Reservation System
– Implements path discovery based on user supplied
endpoints
– Also beginning to look at scheduling services
– Command line client interfacing with a trusted server
• Interface
– oclient -c <srcIP> <dstIP> <lambda#> <flags>
• Path Creation
– oclient -t <pathID>
• Path Teardown
9
Fiber
W Taylor
Host
PP
VM
NWUEN
Link
1 Gbps
8600
10 GE
10 GE
Optera
5200
10Gb/s
TSPR
1
2
3
4
5
6
7
8
9
l 1 Photonic
l 2 Node
l3
l4
Optera 5200 OFA
NWUEN-2
NWUEN-3
Internet
5200 OFA
l
Photonic 1
l
Node 2
l3
l4
KM
35.3
10.3
12.4
7.2
24.1
24.1
24.9
6.7
5.3
MI
22.0
6.4
7.7
4.5
15.0
15.0
15.5
4.2
3.3
Lake Shore
10 GE
10 GE
Optera
5200
10Gb/s
TSPR
PP
8600
Internet
5200 OFA
NWUEN-8
Span Length
NWUEN-9
NWUEN-4
5200 OFA
Network Path
Taken
Potential Paths
http://www.dotresearch.org
Photonic l 1
Node l 2
l3
l4
S. Federal
10 GE
10 GE
Optera
5200
10Gb/s
TSPR
PP
8600
1 Gbps
Host
VM
10
Virtuoso
Virtual Machine marketplace
• Collection of remotely distributed VMs that appear to
reside on the same LAN
• Provides many opportunities for optimization
– Adaptive overlay networks, VM migration, resource scheduling,
etc…
– A. Sundararaj, A. Gupta, and P. Dinda, Increasing Application Performance In
Virtual Environments Through Run-time Inference and Adaptation
• HPDC 2005
– A. Sundararaj, M. Sanghi, J. Lange, P. Dinda, An Optimization Problem in
Adaptive Virtual Environments
• (MAMA 2005), To Appear
• http://virtuoso.cs.northwestern.edu
11
Application
Data
Control
VMM
VNET
VTTIF
VADAPT
VRESERVE
VNET Overlay
Reservation Service
(ODIN)
Internet
Reservable Network
(OMNInet)
12
VNET
Virtual Networking for Virtual Machines
• Allows VMs to behave as if they were on
the same LAN
– All remote VMs tunnel traffic to a central proxy
• Default star topology
• A. Sundararaj, P. Dinda, Towards Virtual Networks for
Virtual Machine Grid Computing
– USENIX VM 2004
13
Application
Data
Control
VMM
VNET
VTTIF
VADAPT
VRESERVE
VNET Overlay
Reservation Service
(ODIN)
Internet
Reservable Network
(OMNInet)
14
VTTIF
Application Topology Inference
• Extracts network topologies from application
behavior
• Generates a global traffic matrix defining the
global application network topology
• A. Gupta, P. Dinda, Inferring the Topology and Traffic Load of
Parallel Programs Running In a Virtual Machine Environment
– Workshop on Job Scheduling Policies for Parallel Processing, 2004
15
VADAPT
Overlay Network Adaptation
• Modify virtual network to match actual network
topology
– Creates overlay links between communicating VMs
• Based on global reduction of VTTIF matrices
• A. Sundararaj, A. Gupta, P. Dinda, Dynamic Topology
Adaptation In Virtual Networks of Virtual Machines
– LCR 2004
• A. Sundararaj, A. Gupta, and P. Dinda, Increasing
Application Performance In Virtual Environments
Through Run-time Inference and Adaptation
– HPDC 2005
16
Application
Data
Control
VMM
VNET
VTTIF
VADAPT
VRESERVE
VNET Overlay
Reservation Service
(ODIN)
Internet
Reservable Network
(OMNInet)
17
VRESERVE
Network Reservations for Overlay Networks
• Extension to VADAPT
– Allows true adaptation of the network
• Components
– Reservation API interface
• ODIN CLI
– Routing mechanism
• Address mapping service
– Path Discovery
• Depends on Reservation System functionality
• Allows for scheduled and delayed reservations
– Greatly increases usability for scheduler based
reservation systems
18
Optical Overlays
• VRESERVE routing is accomplished with VNET
overlay links
• Issues
– Reservation based resources usually aimed at high
performance
– Application unaware of changing network conditions
• TCP performance is typically poor in high performance
networks
• Benefits
– Application unaware of changing network conditions
– Routing is much easier at the overlay level
• Network is capable of reacting to global state changes
19
Fiber (MWUEN-4) length = 5 miles
W Taylor
Host
PP
VM
1 Gbps
8600
10 GE
10 GE
Optera
5200
10Gb/s
TSPR
VNET
l 1 Photonic
l 2 Node
l3
l4
Optera 5200 OFA
NWUEN-2
NWUEN-3
VTTIF +
VADAPT
VRESERVE
5200 OFA
l
Photonic 1
l
Node 2
l3
l4
ODIN
Lake Shore
10 GE
10 GE
Optera
5200
10Gb/s
TSPR
PP
8600
Internet
5200 OFA
NWUEN-8
NWUEN-9
NWUEN-4
5200 OFA
Internet
http://www.dotresearch.org
Photonic l 1
Node l 2
l3
l4
S. Federal
10 GE
10 GE
Optera
5200
10Gb/s
TSPR
PP
8600
1 Gbps
Host
VM
20
Evaluation
• This is an existence proof
– Possible to automatically reserve network resources
on unmodified application’s behalf
• Caveats and Disclaimers
– Scalability is unknown
– Final performance measurements are also unknown
• Infrastructure issues prevented complete
performance evaluations
– Only one functional optical link
– Network dismantled during experiments
21
Optical Network Performance
Latency (ms)
Optical
.321
Internet
.498
22
Initial VNET performance
(optical Network)
23
Performance Improvement
(Synthetic BSP benchmark)
Demonstrable improvement by using automatic reservations
24
Making VNET faster
• Overlay links switched to UDP
– Reliable transport implemented in VM TCP stack
– 2x improvement
• Lookup table caching
– 3x improvement
• Future:
– Memory mappings for packet operations
– In kernel forwarding
– Specialized VM device drivers
25
VNET Performance (Round 2)
• No results on Optical Network
– Infrastructure unavailable
• Experiments conducted between cluster
nodes w/ gigabit Ethernet
• VNET performance only
– Reservation and adaptation systems unused
• Overheads insignificant
26
VNET Performance
(Gigabit switch)
27
Future Work
• Where does this fit into the overall
optimization problem?
• How do we define the overall optimization
problem?
• Extension to other reservation systems,
e.g. GARA
28
Conclusion
• Networks are not generic grid services
• Middleware is required to make
reservation networks usable
• VRESERVE
– Enable unmodified applications to effectively
utilize reservation based networks without
user intervention
29
• Prescience Lab
– http://plab.cs.northwestern.edu
• Virtuoso
– http://virtuoso.cs.northwestern.edu
• DOT (Distributed Optical Testbed)
– http://www.dotresearch.org
30
Optical Network Performance
Throughput
(MB/sec)
Optical TTCP
79.48
Optical SCP
11.5
Internet TTCP
11.2
Internet SCP
10.4
Throughput
Latency (ms)
Optical Network
.321
Internet
.498
31