Download Document

Prototyping, Implementation &
Large-Scale-Experimentation of
VIRO in GENI
Zhi-Li Zhang
Qwest Chair Professor
Department of Computer Science and
Engineering
University of Minnesota
Email: [email protected]
VIRO: Scalable, Robust Virtual-Id Routing
for Future Large, Dynamic Networks
Designed to meet the following challenges & requirements
 Scalability: capability to connect tens of thousands, millions or
more devices (& users)


routing table size, constrained by router memory, lookup speed
Availability & Reliability: must be resilient to failures
 need to be “proactive” instead of reactive
 need to localize effect of failures

Mobility: hosts (both clients & servers) are more mobile
 need to separate location (“addressing”) and identity (“naming”)

Manageability: ease of deployment, “plug-&-play”
need to minimize manual configuration
 self-configure, self-organize, while ensuring security and trust
 Agility: dynamically adapt to demand; net expands/shrinks, …


......
VIRO: A Scalable and Robust “Plug-&Play” Routing Architecture

Decoupling routing from naming/”addressing”
 “native” naming/address-independent
 “future-proof” & capable of supporting multiple namespaces

Introduce a “self-organizing” virtual id (vid) layer
 a layer 2 (LLC)/layer-3 convergence layer -- replacing IP
 subsume layer-2/layer-3 routing/forwarding functionality
 except for first/last hop: host to switch or switch to host
 layer-3 addresses (or higher layer names): global addressing or
naming for inter-networking and “persistent” identifiers

DHT-style routing using a topology-aware, structured vid space

highly scalable and robust
–

O(log N) routing table size, localize failures, enable fast rerouting
support multiple topologies or virtualized network services
Virtual ID layer and VID space
Topology-aware, structured virtual id (vid) space

 Kademlia-like “virtual” binary tree
 vid: embed topological “location” information structurally
 self-configurable and self-organizing
IPv4/IPv6 DNS Names
Other
Namespace
1
0
1
1
0
Virtual ID Layer
1
0
A
C
B
D
E
M
F
N
G
H
J
K
L
Layer 2 Physical Network
Topology
0
0
0
1
0
1
1 0
1
0
0
1
1
0
1
0
1
1
0
1
1 0 1 0 1 0 1 0 10 1 0 10
1 0 10 1
A - B - C - D - M- N - - - - - E - F - G - H - J - - - K - L -
VIRO: Three Core Components

Virtual id space construction and vid assignment
 performed most at the bootstrap process (i.e., network set up):
 a vid space “skeleton” is created
 once network is set up/vid space is constructed:



a new node (a “VIRO switch”) joins: assigned based on neighbors’ vid’s
end-host/device: inherits a vid (prefix) from “host switch” (to which it is
attached), plus a randomly assigned host id; host may be agnostic of its vid
VIRO routing algorithm/protocol:
 DHT-style, but needs to build end-to-end connectivity/routes



(Persistent) layer-2/3 address/name resolution and vid look-up
 DHT directory services built on top of the same vid space


a bottom-up, round-by-round process, no network-wide control flooding
O(log N) routing entries per node, N: # of VIRO switches
“persistent” identifier (e.g., MAC/IP address) hashed to a “vid” key, which
is then used for (pid, vid) mapping registration, look-up, etc.
Data forwarding among VIRO switches using vid only
VIRO Routing & Forwarding

Routing: Bottom-up, round-by-round process
 highly scalable: O(L) routing entries per node
 no single point of failure, and localize effect of failures


completely eliminate “network-wide” control flooding
Forwarding: a packet to a destination node, say, L
 compute the logical distance (“level”) to the dest. node
 look up gateway & next-hop pair at the distance level (bucket)
 if no routing entry, drop the packet
Bucket Gateway Nexthop
Distance
1
-
-
2
A
B
3
A
C,D
4
C, D
C,D
5
B,D,M,N
B,C,D
00100
01000
C
00000
M
01001
D
A
10000
B
00010
H
G
00110
E
N
F
10100
10010
10110
11000
J
K
11100
L
11110
Robustness: Localized Failures
01000
00100
M
C
00000
D
A
00110
N
E
00010
00100
01001
F
10010
H
G
10100
10000
B
01000
11000
J
10110
00000
L
D
A
11100
Initial Topology
Bucket Gateway Nexthop
Distance
1
-
-
2
A
B
3
A
C,D
4
C, D
C,D
5
B, D,N,M
B,C,D
00110
10000
11110
K
M
C
B
00010
E
10010
N
01001
H
G
10100
F
11000
J
10110
L
11110
K
11100
Link
F-K
fails
After link F-K
fails
Routing table for node A
does not change despite
the failure!
Built-in Multi-Path Routing, LoadBalancing and Resilient Fast Re-Routing

Learn multiple gateways at each level
 Default gateway is the one that is logically closest

Use additional gateways for multi-path routing,
load balancing & fast failure re-routing
 Requires consistent gateway selection strategies
otherwise forwarding loops may occur
 Use appropriate “forwarding directive” carried in
packets to select appropriate gateways to ensure
consistent gateway selection
 Forwarding directives may be modified to
dynamically adapt to network conditions

Key (Other) Features & Advantages

Load balancing & fast rerouting can be naturally incorporated


no additional complex mechanisms needed
Can support multiple namespaces, and inter-operability among
such namespaces (e.g., IPv4<->IPv6, IP<->Phone#, NDN, etc.)
 VIRO: “native” naming/address-independent
 simply incorporate more <namespace, vid> directory services

Support multiple topologies or virtualized network services
 e.g., support for multiple virtual networks
 multiple vid spaces may be constructed
 e.g., by defining different types of “physical” neighbors

Also facilitate security support
 host and access switches can perform access control
 “persistent” id is not used for data forwarding
 eliminate flooding attacks
GENI-VIRO Project: Goals
VIRO as a “non-IP” layer 3 protocol & service

Implement & prototype VIRO in GENI
 what capabilities does GENI, & how flexible GENI is to
support prototyping VIRO?

Large-scale Experimentation of VIRO in GENI
 test & evaluate VIRO functionality & performance
 what GENI capabilities/tools facilitate such large-scale
“shake-up” experiments, e.g., dynamic link/node failures?

Longer-Term Goal: VIRO as “long-lived” non-IP
layer 3 prototype service
 Potentially incorporated in GENI as a non-IP alternative
 to support research, experiments & educational
activities by other GENI researchers
GENI-VIRO:
Prototype & Implementation Challenges
Plan to implement using the GENI OVS platform,
as a GENI slice

OVS (open virtual switch) platform derived from the
openflow switch spec & SDN paradigm
 centralized control/management planes
 more flexible forwarding behavior, but still tied to existing
Ethernet/IP/TCP protocol suite

Can we implement VIRO using OVS platform?
 has its own “topology-aware” addressing (vid’s) scheme,
centrally managed and/or “self-configured”
 perform distributed routing, using novel “pub-sub” paradigm
 forwarding behavior: using both destination vid (via prefix
matching) and a forwarding directive to select gateway &
next-hop, with built-in multipath & fast failure (re)routing
GENI-VIRO:
Data Plane Prototype Challenges
•
•
Re-use MAC address (40 bits) as vid’s
Virtual id space construction and virtual id assignment
 Centralized vid construction & management plane
 via a centralized OVS/SDN controller
 distributed topology discover, but centralized topology
management: e.g., VIRO switch join or leave operations
 end host: dynamically assigned vid; report to controller

VIRO packet header: similar to Ethernet frame format,
but need an additional “forwarding directive” field
 as an extension to Ethernet, similar to VLAN extension

VIRO forwarding behavior: dest. vid + forwarding directive
 difficult to implement directly using openflow OVS
 need to define new flow table format & implement our own flow
classifier, as an extension to standard OVS
GENI-VIRO:
Control Plane Prototype Challenges

Can implement a centralized version using the
openflow OVS/SDN paradigm, but
 lose some of the key advantages of VIRO, e.g., its
ability to adapt to failures dynamically

Implement distributed VIRO routing via OVS:
 equip each OVS node with a local OVS controller
 configure local flow table to forward VIRO routing
messages to local OVS controller
 local OVS controller implements & emulates VIRO
routing protocols
 additional OVS controllers as rendezvous points (RPs)

Additional functions & modules for neighbor
discovery, monitoring, etc.
GENI-VIRO “Shakedown” Experiments

Test & evaluate functionality & performance of VIRO
 VIRO GENI experiment designs, topology configuration and
workload generation tools
 Large-scale experimental evaluation of VIRO scalability


as # of nodes and links, or richness of topology grows
metrics: routing entities, memory requirements, routing stretches,
routing overheads, etc.;
 Large-scale experimental evaluation of VIRO reliability


how well does it adapt to dynamic topology changes, failures
metrics: convergence speed, response time, packet loss, control load, …
 Experimental evaluation of VIRO multi-path routing at scale
-
What GENI facilities or tools can we leverage to
facilitate these experiments at scale?
 workload generation? dynamic (experiment) topology control?
failure generation? monitoring and measurement tools?
 what scale (# nodes, links, topology richness) can we attain?
Summary
VIRO as a non-IP (layer 3) service

Test, evaluate & shake-down GENI in terms of its
flexibility & limitations for realizing non-IP services
 in particular, the OVS platform used by GENI
 hope to report our results on these by end of Year 1

Test, evaluate & shake-down GENI in terms of its support
for large-scale evaluation of a non-IP layer 3 routing
protocol & service
 What control do experimenters have in experiment design?
 What scale can we attain? How realistic can we get?



hope to report our results on these by end of Year 2
If successful, incorporate into GENI as long-running
service? To support other research & experiments?
Will also enable our own research on SDN, CDN/CCN and
future network architecture designs
THANK YOU!
QUESTIONS?
Pros & Cons of Existing Technologies

(Layer-2) Ethernet/Wireless (Layer-3) IPv4/IPv6
LANs
 Pluses:
 Pluses:
• better data plane scalability,
 plug-&-play, minimal
more “optimal” routing, …
configuration, better
 Minuses:
mobility
• control plane flooding, global
 Minuses:
effect of network failures
 (occasional) data plane
• poor support for mobility
flooding, sub-optimal
• difficulty/complexity in
routing (using spanning
“network renaming”
tree), not robust to
• Esp., changing addressing
failures
schemes (IPv4 -> IPv6
 Not scalable to large
transition) requires
(& wide-area) networks
modifications in routing and
other network protocols
Vid Assignment : Key Properties
01000
M
N 01001
H
10110
D 00110
G
11000 L
10000 10100 J
E
11110
F
K
10010
11100
00100
C
00000
A
B
00010
1
1
1
0
0
0
0
1
0
1
1 0
close in the vid space, then
they are also close in the
physical topology esp., any two
logical neighbors must be
directly connected.
• connectivity: any logical
sub-trees must be physically
connected.
1
0
0
Key invariant properties
• closeness: if two nodes are
1
0
0
1
1
0
1
0
1
1
0
1
1 0 1 0 1 0 1 0 10 1 0 10
1 0 10 1
A - B - C - D - M- N - - - - - E - F - G - H - J - - - K - L -
Vid based distance: Logical distance

Logical distance defined on the vid space
d(vidx, vidy) = L – lcp (vidx,vidy)
L: max. tree height; lcp: longest common
prefix
e.g. d(00001, 00111) = 5 – lcp(00001, 00111)
=5–2=3
d(01001, 01011) = 5 – lcp(01001, 01011)
=5–3=2
Vid Assignment: Bootstrap Process
Initial vid assignment and vid space construction
performed during the network bootstrap process
Performed using either a centralized or distributed vid
assignment algorithm.


0
M
0
0
C
A
0
B
0100
0000
A
C
B
001
0
D 0
E
0
0
H
N 0
0 G
0
F
00
\
C
0
0
J
0
0
L
00 A
B
K
10
1000
M
N 1001
H 011
D 0110
0
G
100
L
010
000
J
0
E 0
0
111
F
K
0
001
110
0
0
00
M
D 10
000
A
010 B
00
F
10
110
D
E
G
J
E
00
100
C
10
N
K
000
M
000
F
010
N 010
G
100
10
H
00
L 10
00
110
H
J
K
000
100
L 110
VIRO Routing: Routing Table
Construction
Bottom-up, “round-by-round” process:

round 1: neighbor discovery


discover and find directly/locally connected neighbors
round k ( 2 ≤ k ≤L):
 build routing entry to reach level-k bucket Bk(x)
-- a list of one or more (gateway, next-hops)
 use “publish-query” (rendezvous) mechanisms
Algorithm for building Bk(x) routing entry at node x:
 if a node(x) is directly connected to a node in Bk(x), then it is a
gateway for Bk(x), and also publishes it within Sk-1(x).
 nexthop to reach Bk(x) = direct physical neighbor in Bk(x)
 else node x queries within Sk-1(x) to discover gateway(s) to reach
Bk(x), choose the logically closest if multiple gateways.
 nexthop to reach Bk(x) = nexthop(gateway)

Correctness of the algorithm can be formally established!
VIRO Routing: Key Features


Inspired by Kademlia DHT
 but need to build end-to-end connectivity/routes!
Bottom-up, round-by-round process
 first: neighbor discovery
 then: build routing entries to reach nodes within each level
of the vid space (virtual binary tree)


use “publish-query” mechanisms
Highly scalable: O(L) routing entries per node
 L = O(log N), N: number of nodes (VIRO switches)
 more importantly, path diversity (e.g., multi-path routing)
can be naturally exploited for load-balancing, etc.


routing is no longer “shortest” path based !
No single point of failure, and localize effect of failures
 unlike link state routing, node/link failure/recovery is not
flooded network-wide; impact scope is limited
 also enable localized fast rerouting

Completely eliminate “network-wide” control flooding
VIRO Routing: Some Definitions
For k =1, …, L, and any node x:
• (level-k) sub-tree, denoted by Sk(x):
• set of nodes within a logical distance of k from x
• (level-k) bucket, denoted by Bk(x):
• set of nodes exactly k logical distance from node x
• (level-k) gateway, denoted Gk(x):
• a node in Sk-1(x) which is connected to a node in Bk(x) is a gateway to
reach Bk(x) for node x; a direct neighbor of x that can reach this
gateway node is a next-hop for this node
Example:
1
0
1
1
0
1
0
0
0 1
0
0
1
10
1
0
0
1
1
0
1
0
1
1
0
1 0 11 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
A - B - C - D - M- N- - - - - E - F - G - H- J - - - K - L -
S1(A)= {A},
S2(A) ={A,B}, B2(A)={B}
G2(A)={A},
S3(A) = {A,B,C,D}
B3(A) = {C,d}
G3(A) = {A,B}
VIRO Routing: Routing Table
01000
00100
00000
Level
1
-
-
3
-
-
..
..
..
L
-
-
D
00110
E
B
N
01001
F
00010
H
G
10100
10000
-
2
C
A
Gateway Nexthop
-
M
11000
J
11110
11100
1
1
1
1
0
0
0 1
0
0
1
10
L
K
10010
0
0
10110
1
0
0
1
1
0
1
1
1
0
1 0 11 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
A - B - C - D - M- N- - - - - E - F - G - H- J - - - K - L -
VIRO Routing: Example

Round 1:
 each node x discovers and learns about its directly/locally
connected neighbors
 build the level-1 routing entry to reach nodes in B1(x)
E.g. Node A: discover three direct neighbors, B,C,D;
build the level-1 routing entry to reach B1(A)={}
01000
Bucket Gateway Nexthop
Distance
1
-
-
2
3
Routing Table for node A
00100
M
C
00000
D
A
00110
B
00010
01001
F
10010
H
G
10100
10000
E
N
11000
J
K
11100
10110
L
11110
VIRO Routing: Example …

Round 2:
 if directly connected to a node in B2(x), enter self as
gateway in level-2 routing entry, and publish in S1(x)
 otherwise, query “rendezvous point” in S1(x) and build
the level-2 routing entry to reach nodes in B2(x)
E.g. Node A: B2(A)={B};
node A directly connected to node B;
publish itself as gateway to B2(A)
Bucket Gateway Nexthop
Distance
1
2
-
A
-
00100
Routing Table for node A
M
C
00000
D
A
00110
B
00010
E
N
01001
F
10010
H
G
10100
10000
B
3
01000
11000
J
K
11100
10110
L
11110
VIRO Routing: Example …

Round 3:
 if directly connected to a node in B3(x), enter self as
gateway in level-3 routing entry, and publish in S2(x)
 otherwise, query “rendezvous point” in S2(x) and build
the level-2 routing entry to reach nodes in B3(x)
E.g. Node A: B3(A)={C,D};
A publishes edges A->C, A->D to “rendezvous point” in S2(A),
say, B;
01000
Bucket Gateway Nexthop
Distance
1
-
-
2
A
B
3
A
C,D
00100
M
C
00000
D
A
00110
B
00010
01001
F
10010
H
G
10100
10000
E
N
11000
J
K
11100
10110
L
11110
VIRO Routing: Example …

Round 4:
 if directly connected to a node in B4(x), enter self as
gateway in level-4 routing entry, and publish in S3(x)
 otherwise, query “rendezvous point” in S3(x) and build the
level-4 routing entry to reach nodes in B4(x)
E.g. Node A: B4(A)={M,N};
A queries “rendezvous point” in S3(A), say, C; learns C as
gateway
01000
00100
Bucket Gateway Nexthop
Distance
1
-
-
2
A
B
3
A
C,D
4
C
C
M
C
00000
D
A
00110
B
00010
01001
F
10010
H
G
10100
10000
E
N
11000
J
K
11100
10110
L
11110
VIRO Routing: Example …

Round 5:
 if directly connected to a node in B5(x), enter self as
gateway in level-5 routing entry, and publish in S4(x)
 otherwise, query “rendezvous point” in S4(x) and build the
level-4 routing entry to reach nodes in B5(x)
E.g. Node A: B5(A)={E,F,G,H,J,K,L};
A queries “rendezvous point” in S4(A), say, D; learns B as
gateway
Bucket Gateway Nexthop
Distance
1
-
-
2
A
B
3
A
C,D
4
C
C
5
B
B
00100
01000
C
00000
M
01001
D
A
10000
B
00010
H
G
00110
E
N
F
10100
10010
10110
11000
J
K
11100
L
11110
VIRO Routing: Packet Forwarding

To forward a packet to a destination node, say, L
 compute the logical distance to that node
 Use the nexthop corresponding to the logical distance for
forwarding the packet
 If no routing entry:

drop the packet
Bucket Gateway Nexthop
Distance
1
-
-
2
A
B
3
A
C,D
4
C
C
5
B
B
01000
00100
00000
C
A
M
D
00110
B
00010
F
10010
H
G
10100
10000
E
N
01001
11000
J
K
11100
10110
L
11110
<pid, vid> Mapping and vid Lookup

pid: persistent identifier of end host/device, or switch
e.g., MAC/IP address, or any other layer 2/3 address, “flat”
host id, or higher layer names
 can simultaneously support multiple namespaces


<pid, vid> mapping registration and look-up using Kademlia
DHT on top of the same vid space
 Hash(pid) -> vidkey: used for registration & look-up
 mapping stored at “access switch” whose vid is “closest” to vidkey

Look-up speed, scalability & mobility support trade-off



can use one-hop or multi-hop DHT
or use hierarchical (or “geographically scoped”) hash tables
vid look-up and data forwarding may be combined


use hierarchical (or “geographically scoped”) rendezvous points
provide better support for mobility
VEIL: a VIRO Realization over
Ethernet (and 802.11, etc)


Re-use 48-bit MAC addresses for vid
vid structure: divided into two fields
 switch vid (32 bits)


assigned to switches using the vid
assignment process
L: default 24 bits
 host id (16 bits)
 assigned by “host-switches”
 uniquely identify hosts directly
connected to a switch.



switch vid
L
End hosts agnostic of their vid’s
Host switch performs vid/MAC address translation
Backward compatible w/ Ethernet, 802.11, etc.
host id
VEIL: <IP/MAC, vid> Mapping

Host-switch:
 a switch directly connected to the host
 discover host MAC/IP through (gratuitous) ARP, and assign vid to
host
 host-switch publishes pid  vid mappings at an “access-switch”

Access-switch:
 a switch whose vid is closest to hash (pid of the host)
VIRO Access-switch for y
Switch
IPy
VIDy
Sz
register mapping
IPy  VIDy
x
VIRO
Switch
Sx
VIRO
Switch
Sy
y
Host-switch for y
An example using IP address as pid
IPy
MACy
VIDy
Address/vid Lookup & Data Forwarding


Use DHT look-up for address/vid
resolution
 with local cache
vid to MAC address translation at
last-hop
VIRO
Switch
Sz
3. ARP Reply
(IPy  VIDy)
1. ARP Query
(IPy  MAC?)
2. ARP query
forwarded as
unicast request
(IPy  MAC?)
VIRO
Switch
x
4. Ethernet packet
(MACx  VIDy)
Sx
Mapping Table at Sz
VID
IP Address
MAC Address
VIDy
IPy
MACy
…
…
…
6. Sy changes
destination
MAC address
Ethernet packet
(VIDx  MACy)
VIRO
Switch
5. Sx changes
source MAC address
Ethernet Packet
(VIDx  VIDy)
Sy
y
More Than One Gateway Exist
and Can be Used !
1
0
1
1
0
1
0
0
0
A - B -
0
1
0
1
1
1 0
C - D - M - N -
0
1
1
0
0
1
0 11
0
1
0
1
- - - - E - F -
0
1 0
1
0
1
1
0
1
1 0
0
1
G - H - J - - -
0
1 0
1
K - L -
VIRO:
Scalable, Robust & Name-Independent
Virtual Id Routing
for (future) Large-scale, Dynamic
Networks
main student contributors:
 Sourabh Jain
 Yingying Chen
 Saurabh Jain
 Guor-Huar Lu