Download QoS Support in High-Speed, Wormhole Routing Networks

QoS Support in High-Speed,
Wormhole Routing Networks
Mario Gerla, B. Kannan, Bruce
Kwan, Prasasth Palanti,Simon
Walton
Overview
• Introduction
• QoS via separate subnets
• QoS via synchronous framework
• QoS via virtual channels
• Conclusions
Introduction
• Wormhole routing offers low latency, high speed
interconnection for supercomputers and
clusters.
• It’s a modification of virtual cut-through:
-A packet is forwarded to output port once its head is received at the switch
-If channel is busy, whole packet is buffered at input port
-Wormhole: packet composed of several flits is stored across several
switches
• Used in high speed LANs like Myrinet:
-Asynchronous LAN
-uses wormhole routing, source routing, backpressure flow control to
achieve low latency and high bandwidth
Supercomputer SuperNet
• Two level architecture:
– Optical backbone based on physical optical star topology
– High speed wormhole routing are Myrinet LANs
– Optical Channel Interface connects electronic LANs to optical backbone
• Provide support for distributed supercomputing.
– Scientific visualization, video display, parallel applications
– Different types of traffic (low latency datagram, high bandwidth
connection oriented)
– Different types of QoS
Objective
• Want to provide connection oriented traffic
with QoS parameters:
-reliable support: no worm loss
-scalable and deadlock free network
• Assumption:
-Traffic with QoS is connection oriented
-QoS parameters specified at connection setup
-Connection can be refused if no guarantee for Qos parameters
-QoS parameters: average bandwidth, end-to-end delay or jitter
QoS support via Separate Subnet
• Create two subnets
– One carries QoS traffic
– Another carries non QoS traffic
– Routing is independent for the subnets
(Myrinet has support)
• Issues:
– Call admission and control
– Source host behavior
– Number of interfaces at host
Call Admission & Control
• Admission agent maintains state of QoS subnet
• Request for QoS traffic connection comes in
• Upon receiving request, agent decides a suitable route
• If route not available, host can retry or use other subnet
• If route exists, connection is accepted, host can send
• Once completed, host informs admission agent
• Admission agent update its view or state of subnet
Host Behavior
• Host must be responsible for amount of
traffic injected in subnet according to QoS
parameters it required
• Solution: host uses pacing mechanism
– Allow only predetermined number of flits to be transmitted per
time period
Number of Interfaces
• Suppose host has only one interface
• Sender side:
– Host can schedule transmission into the network
• Receiver side:
– Possible non-QoS worm may block QoS worm
– QoS worm encounters delay if non-QoS worm is large
• Solutions:
– Two interfaces: this double cost of network
– Account for the worst case non-QoS traffic delay on single host
interface at call setup time
Alternative
• Subnets:
– difficult to provide delay bounds due to delay dynamics from blocking at
different cross points
• Alternative:
– Impose synchronous structure on top of the asynchronous network
– Enables control over the blocking
– Delay bounds and message priorities may be implemented
• Trade off:
– Network is no longer asynchronous
– Under low traffic load, messages suffer delay due to synchronous protocol
overhead
QoS support via Synchronous framework
• Similar to dedicated traffic channels
• Use timed-token to control traffic streams
• Provides tighter delay bounds and bandwidth guarantees
• Target Token Rotation Time TTRT limit the amount of
transmission
• Average delay = TTRT
• Worst case delay = 2 * TTRT
How to support timed token
• Dedicated unidirectional ring is
embedded in the network
• Attributes of Token scheme:
-fair
-deadlock free
Issues
• Number of host interfaces
– Caused by interaction of QoS & non-QoS traffic
– QoS traffic travel on core ring while non-QoS
travel on other links not on ring
– If host with one interface is busy receiving nonQoS message, a QoS message will suffer delay
– QoS message must have preemptive priority
• To increase non-QoS throughput, embedded
ring may be used if bandwidth is not
completely taken by QoS traffic
Continued
• Scalability:
– throughput performance maintained by
increasing TTRT parameter
– Allows nodes to transmit for longer time
when they have the token
– Causes less capability to provide tight
delay bounds
Virtual Channel Based QoS
• Each link is split into two different sets of virtual
channels used for datagram and QoS traffic
• Each input port buffer of switch is split into several
disjoint buffers
• Link between node and input port of switch is a
collection of virtual channels
• Allows worms to be interleaved
• Give QoS traffic priority in the network
Non-preemptive priority
• A worm arriving at QoS virtual channel does not get
transmitted right away
• Current worm (datagram or QoS) being transmitted
on outgoing link must either complete or get blocked
• Then, scan QoS virtual channels before datagram
channels to schedule the worm for transmission on
outgoing link
• Easy to apply preemptive priority by making arrived
worm preempt datagram worm at the QoS virtual
channel
Implementation
• Preemptive and non-preemptive implementation
require intelligence switch
• At a switch:
– monitor all traffic passing
– Schedule QoS & non-QoS traffic according to protocol
• Harder to implement preemtive:
– Switch must check arrival of QoS traffic at any input
port before transmitting non-QoS flit from output port
Advantage of virtual channels
• Network appears the same for both traffic
• Intelligent switches allocate bandwidth as
required to support QoS
• Can provide delay jitter bounds
• Bandwidth guarantee is provided by employing
call admission agent
conclusions
• Wormhole routing networks provide low latency, high
bandwidth support for datagram traffic
• To support QoS traffic is a challenge
• Dedicated QoS subnet with pacing and call
admission control can support QoS
• Synchronous framework on top of asynchronous
network provides guaranteed bandwidth and delay
• Virtual channels with priority mechanism also is
effective way to support QoS