Download ppt - CSE Labs User Home Pages

Router Design: Part I
• Overview of Generic Router Architecture
• Input-Queued Switches (Routers)
• IP Address Look-up Algorithms
• Packet Classification Algorithms
Readings: Do required readings (you can skip the
math in Section V and Appendix in [Mc+99]);
Also do some of the optional readings if
interested
Q: Any volunteers for scribes?
CSci5221:
Router Design
1
CSci5221:
Router Design
. . .
. . .
Routers in a Network
Sample Routers and Switches
Cisco 12416 Router
up to 160 Gb/s throughput
up to 10 Gb/s ports
Juniper Networks T640 Router
up to 160 Gb/s throughput
up to 10 Gb/s ports
CSci5221:
3Com 4950
24 port gigabit
Ethernet switch
Router Design
3
High Capacity Router
• Cisco CRS-1
– up to 46 Tb/s thruput
• two rack types
• line card rack
– 640 Gb/s thruput
– up to 16 line cards
• up to 40 Gb/s each
– up to 72 racks
• switch rack
– central switch stage
– up to 8 racks
• in-service scaling
CSci5221:
Router Design
4
Components of a Basic Router
• Input/Output Interfaces (II,
OI)
II
IPP
OPP
. . .
• Input Port Processor (IPP)
OI
output
queue
routing
table
. . .
– convert between optical signals
and electronic signals
– extract timing from received
signals
– encode (decode) data for
transmission
CP
– synchronize signals
– determine required OI or OIs
from routing table
• Output Port Processor (OPP)
– queue outgoing cells
• shared bus interconnects IPPs
and OPPs
CSci5221:
Router Design
 Control
Processor (CP)
» configures routing tables
» coordinates end-to-end channel setup
together with neighboring routers
Generic Router Architecture
Data Hdr
Header Processing
Lookup
IP Address
Update
Header
1
1
Buffer
Memory
Address
Table
Data Hdr
Header Processing
Lookup
IP Address
Update
Header
2
2
Header Processing
Lookup
IP Address
Update
Header
Address
Table
CSci5221:
Queue
Packet
Buffer
Memory
Address
Table
Data Hdr
Queue
Packet
Router Design
N
N
Queue
Packet
Buffer
Memory
6
Switch Fabric: Three Design Approaches
CSci5221:
Router Design
7
Switch Fabric: First Generation Routers
• Traditional computers with switching under direct
control of the CPU
• Packet copied to the system’s memory
• Speed limited by the memory bandwidth (two bus
crossings per packet)
Input
Port
Memory
Output
Port
System Bus
CSci5221:
Router Design
8
Shared Memory (1st Generation)
Shared Backplane
CPU
Route
Table
Buffer
Memory
Line
Interface
Line
Interface
Line
Interface
MAC
MAC
MAC
Typically < 0.5Gbps aggregate capacity
Limited by rate of shared memory
CSci5221:
Router Design
9
Switch Fabric: Switching Via a Bus
• Packet from input port
memory to output port
memory via a shared bus
• Bus contention: switching speed
limited by bus bandwidth
• 1 Gbps bus, Cisco 1900: sufficient speed
for access and enterprise
routers (not regional or backbone)
CSci5221:
Router Design
10
Shared Bus (2nd Generation)
CPU
Typically < 5Gb/s aggregate
capacity; Limited by shared bus
Router Design
Buffer
Memory
Line
Card
Line
Card
Line
Card
Buffer
Memory
Buffer
Memory
Buffer
Memory
Fwding
Cache
Fwding
Cache
Fwding
Cache
MAC
MAC
MAC
CSci5221:
Route
Table
11
Switch Fabric: Interconnection Network
• Banyan networks, other interconnection nets
initially created for multiprocessors
• Advanced design: fragmenting packet into fixed
length cells to send through the fabric
• Cisco 12000: switches Gbps through the
interconnection network
CSci5221:
Router Design
12
Point-to-Point Switch (3rd Generation)
Switched Backplane
Line
Card
CPU
Card
Line
Card
Local
Buffer
Memory
Routing
Table
Local
Buffer
Memory
Fwding
Table
Fwding
Table
MAC
MAC
Typically < 50Gbps
aggregate capacity
CSci5221:
Router Design
13
Buffer Placement: Output Port Queuing
• Buffering when the aggregate arrival rate exceeds the
output line speed
• Memory must operate at very high speed
CSci5221:
Router Design
14
Simple model of output queued
switch
Link 1, ingress
Link rate, R
Link 2, ingress
R
Link 3, ingress
R
Link 4, ingress
R
CSci5221:
Router Design
Link 1, egress
Link rate, R
Link 2, egress
R
Link 3, egress
R
Link 4, egress
R
15
Characteristics of an output
queued (OQ) switch
• arriving packets immediately written into output queue,
without intermediate buffering
• flow of packets to one output does not affect flow to
another output
• OQ switch is work conserving: output line always busy when
there is a packet in switch for it
• OQ switch has highest throughput, lowest average delay
CSci5221:
Router Design
16
Switching Speed-up Needed
Data Hdr
Header Processing
Lookup
IP Address
Update
Header
1
1
Buffer
Memory
Address
Table
Data Hdr
Header Processing
Lookup
IP Address
Queue
Packet
Update
Header
2
2
NQueue
times line
Packet
rate
Buffer
Memory
Address
Table
N times line rate
Data Hdr
Header Processing
Lookup
IP Address
Update
Header
Address
Table
CSci5221:
Router Design
N
N
Queue
Packet
Buffer
Memory
17
Buffer Placement: Input Port Queuing
• Fabric slower than input ports combined
– So, queuing may occur at input queues
• Head-of-the-Line (HOL) blocking
– Queued packet at the front of the queue prevents
others in queue from moving forward
CSci5221:
Router Design
18
Simple model of input queued
switch
Link 1, ingress
R
Link 2
Link 1
R1
Link 2, ingress
R
Link 3
Link 4
Link 3, ingress
R
Link 4, ingress
R
CSci5221:
Router Design
Link 1, egress
R
Link 2, egress
R
Link 3, egress
R
Link 4, egress
R
19
Head-of-line Blocking
• Packet at the head of an input queue cannot be
transferred, thus blocking the following
packets (or cells – packets of fixed size)
Cannot be transferred because
is blocked by red packet
Input 1
Output 1
Input 2
Output 2
Input 3
CSci5221:
Cannot be
transferred
because output
buffer full
Router Design
Output 3
20
Characteristics of an input
queued (IQ) switch
• arriving packets written into input queue
• only one packet can be sent to output link at a
time
• head-of-line blocking
• IQ switch cannot keep output links fully utilized
CSci5221:
Router Design
21
Buffer Placement: Design Trade-offs
• Output queues
– Pro: work-conserving, so maximizes throughput
– Con: memory must operate at speed N*R
• Input queues
– Pro: memory can operate at speed R
– Con: head-of-line blocking for access to output
• Work-conserving: output line is always busy when
there is a packet in the switch for it
• Head-of-line blocking: head packet in a FIFO
cannot be transmitted, forcing others to wait
CSci5221:
Router Design
22
What is capacity of IQ: Model
[optional: Karol et al Globecom’86]
•Large input-queued switch with
– single FIFO at each input
– packet destinations i.i.d. (independently, identically
distributed), uniform across outputs
– HoL blocked packets not flushed
•throughput analysis
–
–
–
–
–
–
saturated switch (i.e., always arrival at each input queue)
ball/urns model: N balls, N urns
focus on first urn
Xt - number of balls in urn at time t
Dt- number balls removed from all ums at end of time t
Dt is switch thruput
CSci5221:
Router Design
23
Model (cont’d)
• At+1 - no. balls dropped into urn 1 at t+1
• Xt+1 = (Xt-1)+ + At+1
• where
 Dt 
k
Dt  k
P At 1  k    1 / N  1  1 / N 
k 
• E(Dt) = ρN where ρ is output throughput
• for large N, binomial distribution can be
approximated by Poisson distribution,
P( At  k ) 
CSci5221:
Router Design

k
k!
e
24
Model (cont’d)
E ( A )  EA  2( EA)
EX 
2(1  EA)
2
2
where EA = ρ, E(A2) = ρ + ρ2
therefore
2   2
EX 
2(1   )
EX = 1, therefore
2   2
1
2(1   )
and ρ =2-√2 » 58.6%
CSci5221:
Router Design
25
A Router with Input Queues
Head of Line Blocking
Delay
The best that
any queueing
system can
achieve.
0%
20%
40%
60%
Load
CSci5221:
Router Design
80%
100%
2  2  58%
26
Solution to Avoid Head-of-line Blocking
• How to improve capacity without increasing switching fabric
speed ?
• Maintain at each input N virtual queues, i.e., one per output
– use non-FIFO scheduler, matching input/output
Input 1
Output 1
Output 2
Input 2
Output 3
Input 3
CSci5221:
Router Design
27
Virtual Output Queueing
• assume fixed length
packets
1
• each input manages
separate queue per output
• at each time, matching
scheduler finds best
possible packets from
inputs to said to outputs
• maximum-weight matching
N
CSci5221:
Router Design
1
.
.
.
matching
scheduler
N
.
.
.
28
Matching
• Lij(t): no. of packets at input i for output j at t
• bipartite graph (V1 x V2,E), E ÍV1xV2
– V1,V2 inputs, outputs; (i,j) ÎE iff Lij(t) > 0
• matching: subset of E such that no two edges are adjacent
• maximal matching: no more edges can be added
An aside:
stability (of a [queueing] system):
input
output
• Assuming the arrival rate is (i.e., # of
arrivals per unit of time) less or equal to
the system capacity
• The system is stable if and only if no
queue grows infinitely (under any arrival
patterns) as t ® ¥
CSci5221:
Router Design
29
Matching problems
• maximum size matching
– matching with largest number of edges
– when traffic uniform, provides 100% utilization
– network flow problem, O(N5/2)
•
maximum weight matching
– add weight wij to edge from i to j
–
–
–
–
• e.g., wij: # of packets from input i to output j in the queue
matching with highest weight
when wij = Lij(t) provides 100% utilization
equivalent to a network flow problem, O(N3)
MWM algorithms involve backtracking:
i.e. edges laid down in one iteration may be removed later
algorithm not amenable to pipelining
CSci5221:
Router Design
30
Scheduling Algorithms
19
3
4
1
21
18
7
19
19
1
7
Practical
Maximal Matchings
Max Size Matching
 Not stable
CSci5221:
18
 Not stable
Router Design
Max Wt Matching
 Stable
31
Switch Algorithms
19
19
1
18
7
Maximal matching
Max Size Matching
Not stable
Not stable
Max Wt Matching
Stable, low backlogs
Better performance
Easier to implement
CSci5221:
Router Design
32
Better Matching Algorithms
•
Need simple algorithms that perform well
•
Randomized algorithms with linear complexity available
–
–
–
efficient packet processing packets at line speeds
high throughput
low latencies/backlogs
– Tassiulas’ Randomized Algorithm
– LAURA
– SERENA
Use both randomization, history, problem structure and arrival
information
For more details, see optional reading [SGP02]: “Efficient Randomized
Algorithms for Input-Queued Switch Scheduling” by Shah, Giaccone
and Prabhakar, IEEE Micro Vol 22, Issue 1, Jan 2002
CSci5221:
Router Design
33
Combined Input-Output Queued
(CIOQ) Routers
• Both input and output
interfaces store packets
• Advantages
input interface
output interface
– Easy to built
• Utilization 1 can be achieved
with limited input/output
speedup (<= 2)
Backplane
• Disadvantages
– Harder to design algorithms
• Two congestion points
• Need to design flow control
CSci5221:
Router Design
RO
C
34
Output Queue Emulation
using CIOQ (with Speed-up)
Stable Marriage Problem
-- Gale Shapely Algorithm (GSA)
• As long as there is a free man m
– m proposes to highest ranked women w in his list he
hasn’t proposed yet
– If w is free, m an w are engaged
– If w is engaged to m’ and w prefers m to m’, w releases
m’
• Otherwise m remains free
• A stable matching exists for every set of
preference lists
• Complexity: worst-case O(N2)
CSci5221:
Router Design
35
Stable Marriage Problem
• Consider N women and N men
• Each woman/man ranks each man/woman in the
order of their preferences
• Stable matching, a matching with no blocking pairs
• Blocking pair; let p(i) denote the pair of i
– There are matched pairs (k, p(k)) and (j, p(j)) such that k
prefers p(j) to p(k), and p(j) prefers k to j
CSci5221:
Router Design
36
Example
men
1
2
3
4
pref. list
2 4 3 1
1 4 3 2
4 3 2 1
1 2 4 3
women
1
2
3
4
pref. list
1 4 3 2
3 1 4 2
1 2 3 4
2 1 4 3
• If men propose to women, the stable matching is
–
–
–
–
1st round: (1,2), (2,1), (3,4), (4,1) -> w1 releases m2
2nd round: (2,4) ->w4 releases m3;
3rd round: (3,3);
final match: (1,2), (2,4), (3,3), (4,1)
• What is the stable matching if women propose to
men?
CSci5221:
Router Design
37
OQ Emulation with a Speedup of 2
• Each input and output maintains a preference list
• Input preference list: list of cells at that input
ordered in the inverse order of their arrival
• Output preference list: list of all input cells to be
forwarded to that output ordered by the times
they would be served in an Output Queueing
schedule
• Use GSA to match inputs to outputs
– Outputs initiate the matching
• Can emulate all work-conserving schedulers
For more info, see the optional reading [C+99] “Matching Output
Queueing with a Combined Input Output Queued Switch.”
CSci5221:
Router Design
38
Line Cards
• Interfacing
to/from link
–
–
–
–
–
–
–
Packet forwarding (FIB)
Packet filtering (ACLs)
Buffer management
Link scheduling
Rate-limiting
Packet marking
Measurement
CSci5221:
Router Design
FIB
Transmit
• Packet handling
Receive
– Physical link
– Switching fabric
to/from switch
39
Line Card: Abstract view
Header Processing
Data
Hdr
Lookup
Update
IP Address Header
IP Address
Hdr
Next Hop
Address
Table
CSci5221:
Queue
Packet
Data
Router Design
Buffer
Memory
40
Line Cards: Longest-Prefix Match
Forwarding
• Forwarding Information Base in IP routers
– Maps each IP prefix to next-hop link(s)
• Destination-based forwarding
– Packet has a destination address
– Router identifies longest-matching prefix
– Pushing complexity into forwarding decisions
FIB
4.0.0.0/8
4.83.128.0/17
12.0.0.0/8
12.34.158.0/24
126.255.103.0/24
destination
12.34.158.5
CSci5221:
Router Design
outgoing link
Serial0/0.1
41
Line Cards: Packet Forwarding
Evolution
• Software on the router CPU
– Central processor makes forwarding decision
– Not scalable to large aggregate throughput
• Route cache on the line card
– Maintain a small FIB cache on each line card
– Store (destination, output link) mappings
– Cache misses handled by the router CPU
• Full FIB on each line card
– Store the entire FIB on each line card
– Apply dedicated hardware for longest-prefix match
CSci5221:
Router Design
42
Line Cards: Packet Filtering With
Access Control Lists
Should arriving
packet be allowed
in? Departing packet
let out?
• “Five tuple” for access control lists (ACLs)
– Source and destination IP addresses
– TCP/UDP source and destination ports
– Protocol (e.g., UDP vs. TCP)
CSci5221:
Router Design
43
ACL Examples
• Filter packets based on source address
– Customer access link to the service provider
– Source address should fall in customer prefix
• Filter packets based on port number
– Block traffic for unwanted applications
– Known security vulnerabilities, peer-to-peer, …
• Block pairs of hosts from communicating
– Protect access to special servers
– E.g., block the dorms from the grading server 
CSci5221:
Router Design
44
Line Cards: Mapping Traffic to
Classes
• Gold traffic
– All traffic to/from President’s IP address
– All traffic to/from the port number for DNS
• Silver traffic
– All traffic to/from academic and administrative buildings
• Bronze traffic
– All traffic on the public wireless network
• Then, schedule resources accordingly
– 50% for gold, 30% for silver, and 20% for bronze
CSci5221:
Router Design
45
A Case Study
[Partridge et al ’98]
• Goal: show that routers can keep pace with improvements
of transmission link bandwidths
• Architecture
– A CIOQ router
– 15 (input/output) line cards: C = 2.4 Gbps (3.3 Gpps including
packet headers)
• Each input card can handle up to 16 (input/output) interfaces
• Separate forward engines (FEs) to perform routing
– Backplane: Point-to-point (switched) bus, capacity B = 50 Gbps
(32 MPPS)
• B/C = 50/2.4 = 20
CSci5221:
Router Design
46
Router Architecture
packet
header
CSci5221:
Router Design
47
Router Architecture
input interface
output interfaces
1
Data in
15
Backplane
Update
routing
tables
forward engines
Control data
(e.g., routing)
CSci5221:
Router Design
Data out
Set scheduling
(QoS) state
Network
processor
48
Router Architecture: Data
Plane
• Line cards
– Input processing: can handle input links up to 2.4 Gbps
– Output processing: use a 52 MHz FPGA; implements QoS
• Forward engine:
– 415-MHz DEC Alpha 21164 processor, three level cache to
store recent routes
• Up to 12,000 routes in second level cache (96 kB); ~ 95% hit
rate
• Entire routing table in tertiary cache (16 MB divided in two
banks)
CSci5221:
Router Design
49
Router Architecture:
Control Plane
• Network processor: 233-MHz 21064 Alpha running
NetBSD 1.1
– Update routing
– Manage link status
– Implement reservation
• Backplane Allocator: implemented by an FPGA
– Schedule transfers between input/output interfaces
CSci5221:
Router Design
50